* [FFmpeg-devel] [PATCH] avfilter/alimiter: Add "flush_buffer" option to flush the remaining valid data to the output @ 2022-03-27 6:08 Wang Cao 2022-03-27 21:40 ` Marton Balint 0 siblings, 1 reply; 24+ messages in thread From: Wang Cao @ 2022-03-27 6:08 UTC (permalink / raw) To: ffmpeg-devel; +Cc: Wang Cao The change in the commit will add some samples to the end of the audio stream. The intention is to add a "zero_delay" option eventually to not have the delay in the begining the output from alimiter due to lookahead. Signed-off-by: Wang Cao <wangcao@google.com> --- doc/filters.texi | 5 ++++ libavfilter/af_alimiter.c | 57 ++++++++++++++++++++++++++++++++------- 2 files changed, 53 insertions(+), 9 deletions(-) If the intention is clear to you, do you prefer us add the "zero_delay" option to the same patch or it needs to go in a separate patch? Thanks! diff --git a/doc/filters.texi b/doc/filters.texi index d70ac3e237..bb8f7c1a0b 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -1978,6 +1978,11 @@ in release time while 1 produces higher release times. @item level Auto level output signal. Default is enabled. This normalizes audio back to 0dB if enabled. + +@item flush_buffer +Flushes the internal buffer so that all the input audio samples to the limiter +will appear to the output. Currently due to lookahead buffer, the total number +of output samples will be larger than the input. @end table Depending on picked setting it is recommended to upsample input 2x or 4x times diff --git a/libavfilter/af_alimiter.c b/libavfilter/af_alimiter.c index 133f98f165..ba0a1361ac 100644 --- a/libavfilter/af_alimiter.c +++ b/libavfilter/af_alimiter.c @@ -55,6 +55,9 @@ typedef struct AudioLimiterContext { int *nextpos; double *nextdelta; + int flush_buffer; + int total_samples_to_flush; + double delta; int nextiter; int nextlen; @@ -65,14 +68,15 @@ typedef struct AudioLimiterContext { #define AF AV_OPT_FLAG_AUDIO_PARAM | AV_OPT_FLAG_FILTERING_PARAM | AV_OPT_FLAG_RUNTIME_PARAM static const AVOption alimiter_options[] = { - { "level_in", "set input level", OFFSET(level_in), AV_OPT_TYPE_DOUBLE, {.dbl=1},.015625, 64, AF }, - { "level_out", "set output level", OFFSET(level_out), AV_OPT_TYPE_DOUBLE, {.dbl=1},.015625, 64, AF }, - { "limit", "set limit", OFFSET(limit), AV_OPT_TYPE_DOUBLE, {.dbl=1}, 0.0625, 1, AF }, - { "attack", "set attack", OFFSET(attack), AV_OPT_TYPE_DOUBLE, {.dbl=5}, 0.1, 80, AF }, - { "release", "set release", OFFSET(release), AV_OPT_TYPE_DOUBLE, {.dbl=50}, 1, 8000, AF }, - { "asc", "enable asc", OFFSET(auto_release), AV_OPT_TYPE_BOOL, {.i64=0}, 0, 1, AF }, - { "asc_level", "set asc level", OFFSET(asc_coeff), AV_OPT_TYPE_DOUBLE, {.dbl=0.5}, 0, 1, AF }, - { "level", "auto level", OFFSET(auto_level), AV_OPT_TYPE_BOOL, {.i64=1}, 0, 1, AF }, + { "level_in", "set input level", OFFSET(level_in), AV_OPT_TYPE_DOUBLE, {.dbl=1},.015625, 64, AF }, + { "level_out", "set output level", OFFSET(level_out), AV_OPT_TYPE_DOUBLE, {.dbl=1},.015625, 64, AF }, + { "limit", "set limit", OFFSET(limit), AV_OPT_TYPE_DOUBLE, {.dbl=1}, 0.0625, 1, AF }, + { "attack", "set attack", OFFSET(attack), AV_OPT_TYPE_DOUBLE, {.dbl=5}, 0.1, 80, AF }, + { "release", "set release", OFFSET(release), AV_OPT_TYPE_DOUBLE, {.dbl=50}, 1, 8000, AF }, + { "asc", "enable asc", OFFSET(auto_release), AV_OPT_TYPE_BOOL, {.i64=0}, 0, 1, AF }, + { "asc_level", "set asc level", OFFSET(asc_coeff), AV_OPT_TYPE_DOUBLE, {.dbl=0.5}, 0, 1, AF }, + { "level", "auto level", OFFSET(auto_level), AV_OPT_TYPE_BOOL, {.i64=1}, 0, 1, AF }, + { "flush_buffer","flush the samples in the lookahead buffer", OFFSET(flush_buffer), AV_OPT_TYPE_BOOL, {.i64=0}, 0, 1, AF }, { NULL } }; @@ -275,6 +279,39 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *in) return ff_filter_frame(outlink, out); } + +static int request_frame(AVFilterLink* outlink) { + AVFilterContext* ctx = outlink->src; + AudioLimiterContext* s = (AudioLimiterContext*)ctx->priv; + int ret; + AVFilterLink *inlink; + AVFrame *silence_frame; + + ret = ff_request_frame(ctx->inputs[0]); + + if (!s->flush_buffer) { + return 0; + } + + if (ret != AVERROR_EOF || s->total_samples_to_flush) { + // Not necessarily an error, just not EOF.s + return ret; + } + + // We reach here when input filters have finished producing data (i.e. EOF), + // but because of the attac param, s->buffer still has meaningful + // audio content that needs flushing. The amount of remaining audio to flush + // is the same as the amount of lookahead that was trimmed from the beginning. + inlink = ctx->inputs[0]; + // Pushes silence frame to flush valid audio in the s->buffer + silence_frame = ff_get_audio_buffer(inlink, s->total_samples_to_flush); + ret = filter_frame(inlink, silence_frame); + if (ret < 0) { + return ret; + } + return AVERROR_EOF; +} + static int config_input(AVFilterLink *inlink) { AVFilterContext *ctx = inlink->dst; @@ -292,7 +329,8 @@ static int config_input(AVFilterLink *inlink) return AVERROR(ENOMEM); memset(s->nextpos, -1, obuffer_size * sizeof(*s->nextpos)); - s->buffer_size = inlink->sample_rate * s->attack * inlink->ch_layout.nb_channels; + s->total_samples_to_flush = inlink->sample_rate * s->attack; + s->buffer_size = s->total_samples_to_flush * inlink->ch_layout.nb_channels; s->buffer_size -= s->buffer_size % inlink->ch_layout.nb_channels; if (s->buffer_size <= 0) { @@ -325,6 +363,7 @@ static const AVFilterPad alimiter_outputs[] = { { .name = "default", .type = AVMEDIA_TYPE_AUDIO, + .request_frame = request_frame, }, }; -- 2.35.1.1021.g381101b075-goog _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [FFmpeg-devel] [PATCH] avfilter/alimiter: Add "flush_buffer" option to flush the remaining valid data to the output 2022-03-27 6:08 [FFmpeg-devel] [PATCH] avfilter/alimiter: Add "flush_buffer" option to flush the remaining valid data to the output Wang Cao @ 2022-03-27 21:40 ` Marton Balint 2022-03-30 21:03 ` [FFmpeg-devel] [PATCH] avfilter/alimiter: Remove the delay introduced by lookahead buffer Wang Cao 2022-04-04 15:49 ` [FFmpeg-devel] [PATCH] avfilter/alimiter: Add "flush_buffer" option to flush the remaining valid data to the output Paul B Mahol 0 siblings, 2 replies; 24+ messages in thread From: Marton Balint @ 2022-03-27 21:40 UTC (permalink / raw) To: FFmpeg development discussions and patches On Sat, 26 Mar 2022, Wang Cao wrote: > The change in the commit will add some samples to the end of the audio > stream. The intention is to add a "zero_delay" option eventually to not > have the delay in the begining the output from alimiter due to > lookahead. I was very much suprised to see that the alimiter filter actually delays the audio - as in extra samples are inserted in the beginning and some samples are cut in the end. This trashes A-V sync, so it is a bug IMHO. So unless somebody has some valid usecase for the legacy way of operation I'd just simply change it to be "zero delay" without any additional user option, in a single patch. Regards, Marton _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 24+ messages in thread
* [FFmpeg-devel] [PATCH] avfilter/alimiter: Remove the delay introduced by lookahead buffer 2022-03-27 21:40 ` Marton Balint @ 2022-03-30 21:03 ` Wang Cao 2022-04-04 15:46 ` Paul B Mahol 2022-04-04 15:49 ` [FFmpeg-devel] [PATCH] avfilter/alimiter: Add "flush_buffer" option to flush the remaining valid data to the output Paul B Mahol 1 sibling, 1 reply; 24+ messages in thread From: Wang Cao @ 2022-03-30 21:03 UTC (permalink / raw) To: ffmpeg-devel; +Cc: Wang Cao The change essentially removes the delay introduces by lookahead buffer. The valid audio data in the internal buffer is also flushed to ensure the integrity of output. Signed-off-by: Wang Cao <wangcao@google.com> --- doc/filters.texi | 2 - libavfilter/af_alimiter.c | 97 +++++- tests/ref/fate/filter-alimiter | 518 ++++++++++++++++----------------- 3 files changed, 344 insertions(+), 273 deletions(-) Thanks a lot for your time to review my patch. I have made "zero_delay" option to default. Now the output samples will only be written after the lookahead buffer is full. We also uses "request_frame" to flush the internal buffers so that all valid input audio samples are written. diff --git a/doc/filters.texi b/doc/filters.texi index d70ac3e237..5d1adf88e1 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -1943,8 +1943,6 @@ aiir=z=1.3057 0 0 0:p=1.3057 2.3892 2.1860 1:f=sf:r=d The limiter prevents an input signal from rising over a desired threshold. This limiter uses lookahead technology to prevent your signal from distorting. -It means that there is a small delay after the signal is processed. Keep in mind -that the delay it produces is the attack time you set. The filter accepts the following options: diff --git a/libavfilter/af_alimiter.c b/libavfilter/af_alimiter.c index 133f98f165..e1fcf98574 100644 --- a/libavfilter/af_alimiter.c +++ b/libavfilter/af_alimiter.c @@ -30,6 +30,7 @@ #include "audio.h" #include "avfilter.h" +#include "bufferqueue.h" #include "formats.h" #include "internal.h" @@ -55,6 +56,13 @@ typedef struct AudioLimiterContext { int *nextpos; double *nextdelta; + int total_samples_to_flush; + int lookahead_buffer_full; + + struct FFBufQueue output_frame_queue; + int output_sample_pos; + int output_frame_pos; + double delta; int nextiter; int nextlen; @@ -129,6 +137,8 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *in) AVFrame *out; double *buf; int n, c, i; + int can_free_input_frame = 0; + double peak = 0; if (av_frame_is_writable(in)) { out = in; @@ -138,12 +148,23 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *in) av_frame_free(&in); return AVERROR(ENOMEM); } + can_free_input_frame = 1; av_frame_copy_props(out, in); } - dst = (double *)out->data[0]; + + if (ff_bufqueue_is_full(&s->output_frame_queue)) { + // In the runtime, the total number of frames in the queue is bounded by + // attack_time, sample rate and frame size. + return AVERROR_BUG; + } + + ff_bufqueue_add(ctx, &s->output_frame_queue, out); for (n = 0; n < in->nb_samples; n++) { - double peak = 0; + out = ff_bufqueue_peek(&s->output_frame_queue, s->output_frame_pos); + dst = (double *)out->data[0]; + dst += s->output_sample_pos * channels; + peak = 0; for (c = 0; c < channels; c++) { double sample = src[c] * level_in; @@ -213,8 +234,21 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *in) s->att += s->delta; - for (c = 0; c < channels; c++) - dst[c] = buf[c] * s->att; + // Checks lookahead buffer (s->buffer) has been filled + if (!s->lookahead_buffer_full && (s->pos + channels) % buffer_size == 0) { + s->lookahead_buffer_full = 1; + } + if (s->lookahead_buffer_full) { + for (c = 0; c < channels; c++) { + dst[c] = buf[c] * s->att; + dst[c] = av_clipd(dst[c], -limit, limit) * level * level_out; + } + s->output_sample_pos++; + if (s->output_sample_pos == out->nb_samples) { + s->output_frame_pos++; + s->output_sample_pos = 0; + } + } if ((s->pos + channels) % buffer_size == nextpos[s->nextiter]) { if (s->auto_release) { @@ -261,18 +295,55 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *in) if (s->delta != 0. && fabs(s->delta) < 0.00000000000001) s->delta = 0.; - for (c = 0; c < channels; c++) - dst[c] = av_clipd(dst[c], -limit, limit) * level * level_out; - s->pos = (s->pos + channels) % buffer_size; src += channels; - dst += channels; } - - if (in != out) + if (can_free_input_frame) { av_frame_free(&in); + } - return ff_filter_frame(outlink, out); + if (s->output_frame_pos > 0) { + s->output_frame_pos--; + out = ff_bufqueue_get(&s->output_frame_queue); + return ff_filter_frame(outlink, out); + } + return 0; +} + +static int request_frame(AVFilterLink* outlink) +{ + AVFilterContext *ctx = outlink->src; + AudioLimiterContext *s = (AudioLimiterContext*)ctx->priv; + int ret; + AVFilterLink *inlink; + AVFrame *out; + AVFrame *silence_frame; + + ret = ff_request_frame(ctx->inputs[0]); + + if (ret != AVERROR_EOF || s->total_samples_to_flush == 0) { + // Not necessarily an error, just not EOF. + return ret; + } + + // Flush the frames in the internal buffer + if (s->output_frame_pos > 0) { + out = ff_bufqueue_get(&s->output_frame_queue); + s->output_frame_pos--; + return ff_filter_frame(outlink, out); + } + + // We reach here when input filters have finished producing data (i.e. EOF), + // but because of the attack param, s->buffer still has meaningful + // audio content that needs flushing. + inlink = ctx->inputs[0]; + // Pushes silence frame to flush valid audio in the s->buffer + silence_frame = ff_get_audio_buffer(inlink, s->total_samples_to_flush); + ret = filter_frame(inlink, silence_frame); + if (ret < 0) { + return ret; + } + return AVERROR_EOF; } static int config_input(AVFilterLink *inlink) @@ -292,7 +363,8 @@ static int config_input(AVFilterLink *inlink) return AVERROR(ENOMEM); memset(s->nextpos, -1, obuffer_size * sizeof(*s->nextpos)); - s->buffer_size = inlink->sample_rate * s->attack * inlink->ch_layout.nb_channels; + s->total_samples_to_flush = inlink->sample_rate * s->attack; + s->buffer_size = s->total_samples_to_flush * inlink->ch_layout.nb_channels; s->buffer_size -= s->buffer_size % inlink->ch_layout.nb_channels; if (s->buffer_size <= 0) { @@ -325,6 +397,7 @@ static const AVFilterPad alimiter_outputs[] = { { .name = "default", .type = AVMEDIA_TYPE_AUDIO, + .request_frame = request_frame, }, }; diff --git a/tests/ref/fate/filter-alimiter b/tests/ref/fate/filter-alimiter index 56cfb614da..93daba92c6 100644 --- a/tests/ref/fate/filter-alimiter +++ b/tests/ref/fate/filter-alimiter @@ -3,262 +3,262 @@ #codec_id 0: pcm_s16le #sample_rate 0: 44100 #channel_layout_name 0: stereo -0, 0, 0, 1024, 4096, 0xd4194af4 -0, 1024, 1024, 1024, 4096, 0x686af4ab -0, 2048, 2048, 1024, 4096, 0xe80cee61 -0, 3072, 3072, 1024, 4096, 0xa686fbad -0, 4096, 4096, 1024, 4096, 0x191e062c -0, 5120, 5120, 1024, 4096, 0x1a13edb1 -0, 6144, 6144, 1024, 4096, 0xab21f8ef -0, 7168, 7168, 1024, 4096, 0xaa30e757 -0, 8192, 8192, 1024, 4096, 0x5afdf69f -0, 9216, 9216, 1024, 4096, 0x765e05ec -0, 10240, 10240, 1024, 4096, 0x6484f551 -0, 11264, 11264, 1024, 4096, 0x61b9e9f7 -0, 12288, 12288, 1024, 4096, 0xa735feb3 -0, 13312, 13312, 1024, 4096, 0xb47203aa -0, 14336, 14336, 1024, 4096, 0xb6b5fbf5 -0, 15360, 15360, 1024, 4096, 0xbc66f14f -0, 16384, 16384, 1024, 4096, 0x2003ff5b -0, 17408, 17408, 1024, 4096, 0x1160f17d -0, 18432, 18432, 1024, 4096, 0x308001a4 -0, 19456, 19456, 1024, 4096, 0x9df9f429 -0, 20480, 20480, 1024, 4096, 0x3e6eec0f -0, 21504, 21504, 1024, 4096, 0xca3301f2 -0, 22528, 22528, 1024, 4096, 0x9eb1f961 -0, 23552, 23552, 1024, 4096, 0xd0a50c8c -0, 24576, 24576, 1024, 4096, 0x071ee96b -0, 25600, 25600, 1024, 4096, 0x7a46f05b -0, 26624, 26624, 1024, 4096, 0x2cb2f475 -0, 27648, 27648, 1024, 4096, 0x5bda0a52 -0, 28672, 28672, 1024, 4096, 0x33c0f727 -0, 29696, 29696, 1024, 4096, 0x53cfee59 -0, 30720, 30720, 1024, 4096, 0x1588f221 -0, 31744, 31744, 1024, 4096, 0x95d400d6 -0, 32768, 32768, 1024, 4096, 0x0078009a -0, 33792, 33792, 1024, 4096, 0x686af4ab -0, 34816, 34816, 1024, 4096, 0xe80cee61 -0, 35840, 35840, 1024, 4096, 0xa686fbad -0, 36864, 36864, 1024, 4096, 0x191e062c -0, 37888, 37888, 1024, 4096, 0x1a13edb1 -0, 38912, 38912, 1024, 4096, 0xab21f8ef -0, 39936, 39936, 1024, 4096, 0xaa30e757 -0, 40960, 40960, 1024, 4096, 0x5afdf69f -0, 41984, 41984, 1024, 4096, 0x765e05ec -0, 43008, 43008, 1024, 4096, 0x6484f551 -0, 44032, 44032, 1024, 4096, 0x755e0600 -0, 45056, 45056, 1024, 4096, 0x5056ecb9 -0, 46080, 46080, 1024, 4096, 0xdcb609a8 -0, 47104, 47104, 1024, 4096, 0xc87bf4a1 -0, 48128, 48128, 1024, 4096, 0xecdfef95 -0, 49152, 49152, 1024, 4096, 0x905ff13f -0, 50176, 50176, 1024, 4096, 0x4e69eeb3 -0, 51200, 51200, 1024, 4096, 0x16e1082c -0, 52224, 52224, 1024, 4096, 0x606c0a22 -0, 53248, 53248, 1024, 4096, 0x9f94f351 -0, 54272, 54272, 1024, 4096, 0x2c47f63f -0, 55296, 55296, 1024, 4096, 0x9e14ebf9 -0, 56320, 56320, 1024, 4096, 0x7804fbcb -0, 57344, 57344, 1024, 4096, 0xc7c6ffb7 -0, 58368, 58368, 1024, 4096, 0xa547f68d -0, 59392, 59392, 1024, 4096, 0x0c87fed7 -0, 60416, 60416, 1024, 4096, 0x88b5fef1 -0, 61440, 61440, 1024, 4096, 0x325af00d -0, 62464, 62464, 1024, 4096, 0xd826edd9 -0, 63488, 63488, 1024, 4096, 0x1fbe1138 -0, 64512, 64512, 1024, 4096, 0xdf85da7b -0, 65536, 65536, 1024, 4096, 0xf987f9b3 -0, 66560, 66560, 1024, 4096, 0x5df5e12f -0, 67584, 67584, 1024, 4096, 0x259cf8ef -0, 68608, 68608, 1024, 4096, 0xa191eb4f -0, 69632, 69632, 1024, 4096, 0xfe5bf1fd -0, 70656, 70656, 1024, 4096, 0x7675e2cb -0, 71680, 71680, 1024, 4096, 0x796a2f90 -0, 72704, 72704, 1024, 4096, 0xaa52e7a5 -0, 73728, 73728, 1024, 4096, 0x2b56fd81 -0, 74752, 74752, 1024, 4096, 0xbf7c02ee -0, 75776, 75776, 1024, 4096, 0xfe4cebb1 -0, 76800, 76800, 1024, 4096, 0xe5b7fdf7 -0, 77824, 77824, 1024, 4096, 0xd271ece7 -0, 78848, 78848, 1024, 4096, 0x96e0f69f -0, 79872, 79872, 1024, 4096, 0x6e5eef33 -0, 80896, 80896, 1024, 4096, 0x78b70b6e -0, 81920, 81920, 1024, 4096, 0x61f2f075 -0, 82944, 82944, 1024, 4096, 0xf1dc47dc -0, 83968, 83968, 1024, 4096, 0xf4b406d2 -0, 84992, 84992, 1024, 4096, 0xe33806fe -0, 86016, 86016, 1024, 4096, 0x5b5ef087 -0, 87040, 87040, 1024, 4096, 0x3fa207da -0, 88064, 88064, 1024, 4096, 0x4719f201 -0, 89088, 89088, 1024, 4096, 0x641feaad -0, 90112, 90112, 1024, 4096, 0x905402ec -0, 91136, 91136, 1024, 4096, 0xf529e11f -0, 92160, 92160, 1024, 4096, 0x65a41838 -0, 93184, 93184, 1024, 4096, 0x651efb6f -0, 94208, 94208, 1024, 4096, 0x251b1390 -0, 95232, 95232, 1024, 4096, 0x45ea05a8 -0, 96256, 96256, 1024, 4096, 0xe65be141 -0, 97280, 97280, 1024, 4096, 0xd452d3dd -0, 98304, 98304, 1024, 4096, 0x3bdff821 -0, 99328, 99328, 1024, 4096, 0x74da00e6 -0, 100352, 100352, 1024, 4096, 0x4474fa19 -0, 101376, 101376, 1024, 4096, 0x11ddedef -0, 102400, 102400, 1024, 4096, 0xf027d72b -0, 103424, 103424, 1024, 4096, 0xaa0a1ce6 -0, 104448, 104448, 1024, 4096, 0x7168145e -0, 105472, 105472, 1024, 4096, 0x94a710a4 -0, 106496, 106496, 1024, 4096, 0x6f55ecf1 -0, 107520, 107520, 1024, 4096, 0xc12de6d7 -0, 108544, 108544, 1024, 4096, 0x8ff7db15 -0, 109568, 109568, 1024, 4096, 0x366d0cc6 -0, 110592, 110592, 1024, 4096, 0xb04afa4f -0, 111616, 111616, 1024, 4096, 0xf6abbee7 -0, 112640, 112640, 1024, 4096, 0xbe9910dc -0, 113664, 113664, 1024, 4096, 0x96c3047e -0, 114688, 114688, 1024, 4096, 0x9dc1e8e1 -0, 115712, 115712, 1024, 4096, 0x4b2c1092 -0, 116736, 116736, 1024, 4096, 0x592d212c -0, 117760, 117760, 1024, 4096, 0x5c3bda15 -0, 118784, 118784, 1024, 4096, 0x43c8e90f -0, 119808, 119808, 1024, 4096, 0x3bfa074e -0, 120832, 120832, 1024, 4096, 0xa2d7d443 -0, 121856, 121856, 1024, 4096, 0xd4b02844 -0, 122880, 122880, 1024, 4096, 0x9f5d2a92 -0, 123904, 123904, 1024, 4096, 0xc19bf69d -0, 124928, 124928, 1024, 4096, 0x47b8fe75 -0, 125952, 125952, 1024, 4096, 0x337334b0 -0, 126976, 126976, 1024, 4096, 0x96ed14f0 -0, 128000, 128000, 1024, 4096, 0x9af0f67b -0, 129024, 129024, 1024, 4096, 0xdf541a60 -0, 130048, 130048, 1024, 4096, 0x684f0c06 -0, 131072, 131072, 1024, 4096, 0xe3d0015e -0, 132096, 132096, 1024, 4096, 0xf648d73c -0, 133120, 133120, 1024, 4096, 0x4041f3ee -0, 134144, 134144, 1024, 4096, 0x1421025f -0, 135168, 135168, 1024, 4096, 0x4eb5fc97 -0, 136192, 136192, 1024, 4096, 0x510f02fe -0, 137216, 137216, 1024, 4096, 0x85e9e95e -0, 138240, 138240, 1024, 4096, 0xb1d0fe10 -0, 139264, 139264, 1024, 4096, 0xda66f5f8 -0, 140288, 140288, 1024, 4096, 0x3826eaa8 -0, 141312, 141312, 1024, 4096, 0xa7cc0176 -0, 142336, 142336, 1024, 4096, 0x71e8fe5a -0, 143360, 143360, 1024, 4096, 0x0149fcfa -0, 144384, 144384, 1024, 4096, 0xbd36fd8e -0, 145408, 145408, 1024, 4096, 0xec1afcc6 -0, 146432, 146432, 1024, 4096, 0xea80ec5e -0, 147456, 147456, 1024, 4096, 0x8da1f0ac -0, 148480, 148480, 1024, 4096, 0x63c3e61c -0, 149504, 149504, 1024, 4096, 0x188cf09b -0, 150528, 150528, 1024, 4096, 0x7eebea85 -0, 151552, 151552, 1024, 4096, 0x7ef6f718 -0, 152576, 152576, 1024, 4096, 0xcd9fecfb -0, 153600, 153600, 1024, 4096, 0x1aa7f624 -0, 154624, 154624, 1024, 4096, 0xcab5f1e3 -0, 155648, 155648, 1024, 4096, 0x36e9f795 -0, 156672, 156672, 1024, 4096, 0x952cf54d -0, 157696, 157696, 1024, 4096, 0x54a4ed3b -0, 158720, 158720, 1024, 4096, 0x1e24f992 -0, 159744, 159744, 1024, 4096, 0xab6bfa09 -0, 160768, 160768, 1024, 4096, 0xa021020b -0, 161792, 161792, 1024, 4096, 0xaa0600f5 -0, 162816, 162816, 1024, 4096, 0xadf7ec92 -0, 163840, 163840, 1024, 4096, 0x5dd5f63f -0, 164864, 164864, 1024, 4096, 0xc01bff89 -0, 165888, 165888, 1024, 4096, 0x05dbf1e7 -0, 166912, 166912, 1024, 4096, 0x44f00038 -0, 167936, 167936, 1024, 4096, 0x6dc5f886 -0, 168960, 168960, 1024, 4096, 0xeb88e729 -0, 169984, 169984, 1024, 4096, 0xfe5cf53e -0, 171008, 171008, 1024, 4096, 0x2692f7a0 -0, 172032, 172032, 1024, 4096, 0xe984f24b -0, 173056, 173056, 1024, 4096, 0x1a4ef732 -0, 174080, 174080, 1024, 4096, 0x3b60010b -0, 175104, 175104, 1024, 4096, 0x6dd0eaf2 -0, 176128, 176128, 1024, 4096, 0x313de851 -0, 177152, 177152, 1024, 4096, 0x239ef043 -0, 178176, 178176, 1024, 4096, 0x2366fe43 -0, 179200, 179200, 1024, 4096, 0xbfd6f3b9 -0, 180224, 180224, 1024, 4096, 0xb34bf67b -0, 181248, 181248, 1024, 4096, 0x00def7c1 -0, 182272, 182272, 1024, 4096, 0xa6d0f466 -0, 183296, 183296, 1024, 4096, 0xd1a1f1c2 -0, 184320, 184320, 1024, 4096, 0x32f8923c -0, 185344, 185344, 1024, 4096, 0x137001d2 -0, 186368, 186368, 1024, 4096, 0xb881f6cd -0, 187392, 187392, 1024, 4096, 0xf44e034a -0, 188416, 188416, 1024, 4096, 0xb43fecf7 -0, 189440, 189440, 1024, 4096, 0xe62ced50 -0, 190464, 190464, 1024, 4096, 0x221dfd0d -0, 191488, 191488, 1024, 4096, 0x85400147 -0, 192512, 192512, 1024, 4096, 0x8bf8d054 -0, 193536, 193536, 1024, 4096, 0x08370170 -0, 194560, 194560, 1024, 4096, 0xec62effa -0, 195584, 195584, 1024, 4096, 0xd58cf67c -0, 196608, 196608, 1024, 4096, 0xf597f9a1 -0, 197632, 197632, 1024, 4096, 0x63fcf0b0 -0, 198656, 198656, 1024, 4096, 0x8cc8fd32 -0, 199680, 199680, 1024, 4096, 0xf8c2072c -0, 200704, 200704, 1024, 4096, 0x0c7e93a5 -0, 201728, 201728, 1024, 4096, 0x1cc3f612 -0, 202752, 202752, 1024, 4096, 0x83adf8ee -0, 203776, 203776, 1024, 4096, 0x680e0195 -0, 204800, 204800, 1024, 4096, 0x9d34fd2e -0, 205824, 205824, 1024, 4096, 0x44cdfb34 -0, 206848, 206848, 1024, 4096, 0x193df790 -0, 207872, 207872, 1024, 4096, 0xb337ef64 -0, 208896, 208896, 1024, 4096, 0x0074ee38 -0, 209920, 209920, 1024, 4096, 0x239ef043 -0, 210944, 210944, 1024, 4096, 0x2366fe43 -0, 211968, 211968, 1024, 4096, 0xbfd6f3b9 -0, 212992, 212992, 1024, 4096, 0xb34bf67b -0, 214016, 214016, 1024, 4096, 0x00def7c1 -0, 215040, 215040, 1024, 4096, 0xa6d0f466 -0, 216064, 216064, 1024, 4096, 0xd1a1f1c2 -0, 217088, 217088, 1024, 4096, 0x32f8923c -0, 218112, 218112, 1024, 4096, 0x137001d2 -0, 219136, 219136, 1024, 4096, 0xb881f6cd -0, 220160, 220160, 1024, 4096, 0xf44e034a -0, 221184, 221184, 1024, 4096, 0xb43fecf7 -0, 222208, 222208, 1024, 4096, 0xe62ced50 -0, 223232, 223232, 1024, 4096, 0x221dfd0d -0, 224256, 224256, 1024, 4096, 0x85400147 -0, 225280, 225280, 1024, 4096, 0x8bf8d054 -0, 226304, 226304, 1024, 4096, 0x08370170 -0, 227328, 227328, 1024, 4096, 0xec62effa -0, 228352, 228352, 1024, 4096, 0xd58cf67c -0, 229376, 229376, 1024, 4096, 0xf597f9a1 -0, 230400, 230400, 1024, 4096, 0x63fcf0b0 -0, 231424, 231424, 1024, 4096, 0x8cc8fd32 -0, 232448, 232448, 1024, 4096, 0xf8c2072c -0, 233472, 233472, 1024, 4096, 0x0c7e93a5 -0, 234496, 234496, 1024, 4096, 0x1cc3f612 -0, 235520, 235520, 1024, 4096, 0x83adf8ee -0, 236544, 236544, 1024, 4096, 0x680e0195 -0, 237568, 237568, 1024, 4096, 0x9d34fd2e -0, 238592, 238592, 1024, 4096, 0x44cdfb34 -0, 239616, 239616, 1024, 4096, 0x193df790 -0, 240640, 240640, 1024, 4096, 0xb337ef64 -0, 241664, 241664, 1024, 4096, 0x0074ee38 -0, 242688, 242688, 1024, 4096, 0x239ef043 -0, 243712, 243712, 1024, 4096, 0x2366fe43 -0, 244736, 244736, 1024, 4096, 0xbfd6f3b9 -0, 245760, 245760, 1024, 4096, 0xb34bf67b -0, 246784, 246784, 1024, 4096, 0x00def7c1 -0, 247808, 247808, 1024, 4096, 0xa6d0f466 -0, 248832, 248832, 1024, 4096, 0xd1a1f1c2 -0, 249856, 249856, 1024, 4096, 0x32f8923c -0, 250880, 250880, 1024, 4096, 0x137001d2 -0, 251904, 251904, 1024, 4096, 0xb881f6cd -0, 252928, 252928, 1024, 4096, 0xf44e034a -0, 253952, 253952, 1024, 4096, 0xb43fecf7 -0, 254976, 254976, 1024, 4096, 0xe62ced50 -0, 256000, 256000, 1024, 4096, 0x221dfd0d -0, 257024, 257024, 1024, 4096, 0x85400147 -0, 258048, 258048, 1024, 4096, 0x8bf8d054 -0, 259072, 259072, 1024, 4096, 0x08370170 -0, 260096, 260096, 1024, 4096, 0xec62effa -0, 261120, 261120, 1024, 4096, 0xd58cf67c -0, 262144, 262144, 1024, 4096, 0xf597f9a1 -0, 263168, 263168, 1024, 4096, 0x63fcf0b0 -0, 264192, 264192, 408, 1632, 0xedfa314a +0, 0, 0, 1024, 4096, 0xd0b20048 +0, 1024, 1024, 1024, 4096, 0x8abaf0ab +0, 2048, 2048, 1024, 4096, 0x5a00f489 +0, 3072, 3072, 1024, 4096, 0x6d87f3d3 +0, 4096, 4096, 1024, 4096, 0x27fc04bc +0, 5120, 5120, 1024, 4096, 0x6d18f2e1 +0, 6144, 6144, 1024, 4096, 0x6a11f0c9 +0, 7168, 7168, 1024, 4096, 0xadf8ec27 +0, 8192, 8192, 1024, 4096, 0xf0e2fcf9 +0, 9216, 9216, 1024, 4096, 0x55280372 +0, 10240, 10240, 1024, 4096, 0x898ffad3 +0, 11264, 11264, 1024, 4096, 0x06a5eb6b +0, 12288, 12288, 1024, 4096, 0x9fadfe07 +0, 13312, 13312, 1024, 4096, 0x6f32fd99 +0, 14336, 14336, 1024, 4096, 0x7856f871 +0, 15360, 15360, 1024, 4096, 0xf742f893 +0, 16384, 16384, 1024, 4096, 0x7955f7e7 +0, 17408, 17408, 1024, 4096, 0xbb32f521 +0, 18432, 18432, 1024, 4096, 0x8302fda9 +0, 19456, 19456, 1024, 4096, 0x0d3cfdeb +0, 20480, 20480, 1024, 4096, 0xd034ed5f +0, 21504, 21504, 1024, 4096, 0xd3c1fb11 +0, 22528, 22528, 1024, 4096, 0x80bc0734 +0, 23552, 23552, 1024, 4096, 0x82a60000 +0, 24576, 24576, 1024, 4096, 0xaf6ceb09 +0, 25600, 25600, 1024, 4096, 0x0c0dec9f +0, 26624, 26624, 1024, 4096, 0x8ca8f335 +0, 27648, 27648, 1024, 4096, 0xc75604bc +0, 28672, 28672, 1024, 4096, 0x0a22f7b9 +0, 29696, 29696, 1024, 4096, 0x237cf4a5 +0, 30720, 30720, 1024, 4096, 0x5276f36f +0, 31744, 31744, 1024, 4096, 0x148a0194 +0, 32768, 32768, 1024, 4096, 0xd0b20048 +0, 33792, 33792, 1024, 4096, 0x8abaf0ab +0, 34816, 34816, 1024, 4096, 0x5a00f489 +0, 35840, 35840, 1024, 4096, 0x6d87f3d3 +0, 36864, 36864, 1024, 4096, 0x27fc04bc +0, 37888, 37888, 1024, 4096, 0x6d18f2e1 +0, 38912, 38912, 1024, 4096, 0x6a11f0c9 +0, 39936, 39936, 1024, 4096, 0xadf8ec27 +0, 40960, 40960, 1024, 4096, 0xf0e2fcf9 +0, 41984, 41984, 1024, 4096, 0x55280372 +0, 43008, 43008, 1024, 4096, 0x8ac5fad5 +0, 44032, 44032, 1024, 4096, 0x934007ba +0, 45056, 45056, 1024, 4096, 0x84acf4a3 +0, 46080, 46080, 1024, 4096, 0x7c8ff49b +0, 47104, 47104, 1024, 4096, 0x79030244 +0, 48128, 48128, 1024, 4096, 0x28d0e5f5 +0, 49152, 49152, 1024, 4096, 0xf12df059 +0, 50176, 50176, 1024, 4096, 0xd0eef7a7 +0, 51200, 51200, 1024, 4096, 0x2e2e02e4 +0, 52224, 52224, 1024, 4096, 0x7c421656 +0, 53248, 53248, 1024, 4096, 0x79fcea35 +0, 54272, 54272, 1024, 4096, 0xb566e88b +0, 55296, 55296, 1024, 4096, 0xad66f8e9 +0, 56320, 56320, 1024, 4096, 0xe0d001dc +0, 57344, 57344, 1024, 4096, 0xfc5df6cd +0, 58368, 58368, 1024, 4096, 0x9dcefad5 +0, 59392, 59392, 1024, 4096, 0xbde90702 +0, 60416, 60416, 1024, 4096, 0x4feef329 +0, 61440, 61440, 1024, 4096, 0xbb30ef39 +0, 62464, 62464, 1024, 4096, 0x131ff1e3 +0, 63488, 63488, 1024, 4096, 0x994c0184 +0, 64512, 64512, 1024, 4096, 0xc981e10b +0, 65536, 65536, 1024, 4096, 0xceff022a +0, 66560, 66560, 1024, 4096, 0x07d1e1d3 +0, 67584, 67584, 1024, 4096, 0x7ea5f31d +0, 68608, 68608, 1024, 4096, 0x16e1e885 +0, 69632, 69632, 1024, 4096, 0x2a92fb45 +0, 70656, 70656, 1024, 4096, 0x9f90ef6b +0, 71680, 71680, 1024, 4096, 0xed961010 +0, 72704, 72704, 1024, 4096, 0x6169f659 +0, 73728, 73728, 1024, 4096, 0xbe830c50 +0, 74752, 74752, 1024, 4096, 0x1744ea89 +0, 75776, 75776, 1024, 4096, 0x00bcf34f +0, 76800, 76800, 1024, 4096, 0x91f7f7f3 +0, 77824, 77824, 1024, 4096, 0xa4e1f1fb +0, 78848, 78848, 1024, 4096, 0xf3e5f73b +0, 79872, 79872, 1024, 4096, 0x64d0eb95 +0, 80896, 80896, 1024, 4096, 0xe0c90d40 +0, 81920, 81920, 1024, 4096, 0xe9701e32 +0, 82944, 82944, 1024, 4096, 0x760d26b0 +0, 83968, 83968, 1024, 4096, 0x3f4303e4 +0, 84992, 84992, 1024, 4096, 0x1cc502bc +0, 86016, 86016, 1024, 4096, 0x6d7bf089 +0, 87040, 87040, 1024, 4096, 0xe03807fc +0, 88064, 88064, 1024, 4096, 0xf12fe897 +0, 89088, 89088, 1024, 4096, 0x0123e87d +0, 90112, 90112, 1024, 4096, 0x869808d4 +0, 91136, 91136, 1024, 4096, 0x9538def3 +0, 92160, 92160, 1024, 4096, 0x92ed2ab0 +0, 93184, 93184, 1024, 4096, 0xc80bfe9d +0, 94208, 94208, 1024, 4096, 0x56130220 +0, 95232, 95232, 1024, 4096, 0xd3a7f0f3 +0, 96256, 96256, 1024, 4096, 0xfca0eeaf +0, 97280, 97280, 1024, 4096, 0xd409d82b +0, 98304, 98304, 1024, 4096, 0x29ba0424 +0, 99328, 99328, 1024, 4096, 0x4e9cdf19 +0, 100352, 100352, 1024, 4096, 0xb0511cc0 +0, 101376, 101376, 1024, 4096, 0xd499e541 +0, 102400, 102400, 1024, 4096, 0x0910f25f +0, 103424, 103424, 1024, 4096, 0x6e5d1c08 +0, 104448, 104448, 1024, 4096, 0xa07511a8 +0, 105472, 105472, 1024, 4096, 0xddd1ee73 +0, 106496, 106496, 1024, 4096, 0x5b46fb17 +0, 107520, 107520, 1024, 4096, 0xc558e301 +0, 108544, 108544, 1024, 4096, 0x9041e627 +0, 109568, 109568, 1024, 4096, 0xeadae195 +0, 110592, 110592, 1024, 4096, 0x663d0c5a +0, 111616, 111616, 1024, 4096, 0x1c57d55b +0, 112640, 112640, 1024, 4096, 0x7ce90702 +0, 113664, 113664, 1024, 4096, 0xf56c15d0 +0, 114688, 114688, 1024, 4096, 0xa2edd14d +0, 115712, 115712, 1024, 4096, 0x93f1171c +0, 116736, 116736, 1024, 4096, 0xe02e152a +0, 117760, 117760, 1024, 4096, 0x55aaeba7 +0, 118784, 118784, 1024, 4096, 0xacabd7a7 +0, 119808, 119808, 1024, 4096, 0xaa18150a +0, 120832, 120832, 1024, 4096, 0x065ed6a1 +0, 121856, 121856, 1024, 4096, 0x9df9268e +0, 122880, 122880, 1024, 4096, 0x8e0a23e6 +0, 123904, 123904, 1024, 4096, 0x872d019e +0, 124928, 124928, 1024, 4096, 0x3aff116c +0, 125952, 125952, 1024, 4096, 0x4a1717f8 +0, 126976, 126976, 1024, 4096, 0xa3812998 +0, 128000, 128000, 1024, 4096, 0x8f12ee4f +0, 129024, 129024, 1024, 4096, 0x1add13c4 +0, 130048, 130048, 1024, 4096, 0x4f3825da +0, 131072, 131072, 1024, 4096, 0xf732d42b +0, 132096, 132096, 1024, 4096, 0xa2b3e578 +0, 133120, 133120, 1024, 4096, 0xdc18ff83 +0, 134144, 134144, 1024, 4096, 0xd13df8c5 +0, 135168, 135168, 1024, 4096, 0x24c506ff +0, 136192, 136192, 1024, 4096, 0xb58cf9d4 +0, 137216, 137216, 1024, 4096, 0x734cecba +0, 138240, 138240, 1024, 4096, 0x9af5fb4e +0, 139264, 139264, 1024, 4096, 0x8510f05f +0, 140288, 140288, 1024, 4096, 0x81e0f2c3 +0, 141312, 141312, 1024, 4096, 0x5c060641 +0, 142336, 142336, 1024, 4096, 0xb15bfc84 +0, 143360, 143360, 1024, 4096, 0xe7dafcaa +0, 144384, 144384, 1024, 4096, 0x5419faaa +0, 145408, 145408, 1024, 4096, 0x6f97fa6f +0, 146432, 146432, 1024, 4096, 0x5e69e975 +0, 147456, 147456, 1024, 4096, 0xd614f89e +0, 148480, 148480, 1024, 4096, 0x2532e193 +0, 149504, 149504, 1024, 4096, 0x0ce9f04c +0, 150528, 150528, 1024, 4096, 0x0935ec67 +0, 151552, 151552, 1024, 4096, 0x92d6f5d9 +0, 152576, 152576, 1024, 4096, 0xccffea44 +0, 153600, 153600, 1024, 4096, 0x81e3eea0 +0, 154624, 154624, 1024, 4096, 0xc5c2fe66 +0, 155648, 155648, 1024, 4096, 0x75dbf93c +0, 156672, 156672, 1024, 4096, 0x6294f023 +0, 157696, 157696, 1024, 4096, 0xf39eef54 +0, 158720, 158720, 1024, 4096, 0x03b1fa13 +0, 159744, 159744, 1024, 4096, 0xacc0faf2 +0, 160768, 160768, 1024, 4096, 0xcd9ffc79 +0, 161792, 161792, 1024, 4096, 0x4ffc023f +0, 162816, 162816, 1024, 4096, 0x3126ed15 +0, 163840, 163840, 1024, 4096, 0x611ff8d6 +0, 164864, 164864, 1024, 4096, 0x6a9bfa1a +0, 165888, 165888, 1024, 4096, 0xb06cfb10 +0, 166912, 166912, 1024, 4096, 0x1c9f00bb +0, 167936, 167936, 1024, 4096, 0xc0bbeebf +0, 168960, 168960, 1024, 4096, 0x383eea91 +0, 169984, 169984, 1024, 4096, 0xf927f893 +0, 171008, 171008, 1024, 4096, 0xbfc3f9b4 +0, 172032, 172032, 1024, 4096, 0x9faaef83 +0, 173056, 173056, 1024, 4096, 0x4dc6f411 +0, 174080, 174080, 1024, 4096, 0x9432ff0f +0, 175104, 175104, 1024, 4096, 0x62d5e7d6 +0, 176128, 176128, 1024, 4096, 0x1af9ea59 +0, 177152, 177152, 1024, 4096, 0x3dcbf8d0 +0, 178176, 178176, 1024, 4096, 0x80fbf978 +0, 179200, 179200, 1024, 4096, 0xbe60f05b +0, 180224, 180224, 1024, 4096, 0xf12ffb83 +0, 181248, 181248, 1024, 4096, 0xfed0f81a +0, 182272, 182272, 1024, 4096, 0x892eede2 +0, 183296, 183296, 1024, 4096, 0x4101f200 +0, 184320, 184320, 1024, 4096, 0x08759c99 +0, 185344, 185344, 1024, 4096, 0xea80fab7 +0, 186368, 186368, 1024, 4096, 0x2b47fcf4 +0, 187392, 187392, 1024, 4096, 0x2933f949 +0, 188416, 188416, 1024, 4096, 0x10c3eaed +0, 189440, 189440, 1024, 4096, 0x38bb0089 +0, 190464, 190464, 1024, 4096, 0x4fa4f4b6 +0, 191488, 191488, 1024, 4096, 0x47c9fd4c +0, 192512, 192512, 1024, 4096, 0x6e95d5d1 +0, 193536, 193536, 1024, 4096, 0x6deaf758 +0, 194560, 194560, 1024, 4096, 0x96a2f230 +0, 195584, 195584, 1024, 4096, 0x6515fa5b +0, 196608, 196608, 1024, 4096, 0xbdf6f5a6 +0, 197632, 197632, 1024, 4096, 0x74d4f365 +0, 198656, 198656, 1024, 4096, 0x932903e3 +0, 199680, 199680, 1024, 4096, 0x7468085e +0, 200704, 200704, 1024, 4096, 0x0d908b65 +0, 201728, 201728, 1024, 4096, 0x86e7f740 +0, 202752, 202752, 1024, 4096, 0x09fff8d2 +0, 203776, 203776, 1024, 4096, 0xf0430838 +0, 204800, 204800, 1024, 4096, 0x525bf9ec +0, 205824, 205824, 1024, 4096, 0x17c7ee8d +0, 206848, 206848, 1024, 4096, 0x7632fc3b +0, 207872, 207872, 1024, 4096, 0xf12aefe9 +0, 208896, 208896, 1024, 4096, 0x0d72ed83 +0, 209920, 209920, 1024, 4096, 0x3dcbf8d0 +0, 210944, 210944, 1024, 4096, 0x80fbf978 +0, 211968, 211968, 1024, 4096, 0xbe60f05b +0, 212992, 212992, 1024, 4096, 0xf12ffb83 +0, 214016, 214016, 1024, 4096, 0xfed0f81a +0, 215040, 215040, 1024, 4096, 0x892eede2 +0, 216064, 216064, 1024, 4096, 0x4101f200 +0, 217088, 217088, 1024, 4096, 0x08759c99 +0, 218112, 218112, 1024, 4096, 0xea80fab7 +0, 219136, 219136, 1024, 4096, 0x2b47fcf4 +0, 220160, 220160, 1024, 4096, 0x2933f949 +0, 221184, 221184, 1024, 4096, 0x10c3eaed +0, 222208, 222208, 1024, 4096, 0x38bb0089 +0, 223232, 223232, 1024, 4096, 0x4fa4f4b6 +0, 224256, 224256, 1024, 4096, 0x47c9fd4c +0, 225280, 225280, 1024, 4096, 0x6e95d5d1 +0, 226304, 226304, 1024, 4096, 0x6deaf758 +0, 227328, 227328, 1024, 4096, 0x96a2f230 +0, 228352, 228352, 1024, 4096, 0x6515fa5b +0, 229376, 229376, 1024, 4096, 0xbdf6f5a6 +0, 230400, 230400, 1024, 4096, 0x74d4f365 +0, 231424, 231424, 1024, 4096, 0x932903e3 +0, 232448, 232448, 1024, 4096, 0x7468085e +0, 233472, 233472, 1024, 4096, 0x0d908b65 +0, 234496, 234496, 1024, 4096, 0x86e7f740 +0, 235520, 235520, 1024, 4096, 0x09fff8d2 +0, 236544, 236544, 1024, 4096, 0xf0430838 +0, 237568, 237568, 1024, 4096, 0x525bf9ec +0, 238592, 238592, 1024, 4096, 0x17c7ee8d +0, 239616, 239616, 1024, 4096, 0x7632fc3b +0, 240640, 240640, 1024, 4096, 0xf12aefe9 +0, 241664, 241664, 1024, 4096, 0x0d72ed83 +0, 242688, 242688, 1024, 4096, 0x3dcbf8d0 +0, 243712, 243712, 1024, 4096, 0x80fbf978 +0, 244736, 244736, 1024, 4096, 0xbe60f05b +0, 245760, 245760, 1024, 4096, 0xf12ffb83 +0, 246784, 246784, 1024, 4096, 0xfed0f81a +0, 247808, 247808, 1024, 4096, 0x892eede2 +0, 248832, 248832, 1024, 4096, 0x4101f200 +0, 249856, 249856, 1024, 4096, 0x08759c99 +0, 250880, 250880, 1024, 4096, 0xea80fab7 +0, 251904, 251904, 1024, 4096, 0x2b47fcf4 +0, 252928, 252928, 1024, 4096, 0x2933f949 +0, 253952, 253952, 1024, 4096, 0x10c3eaed +0, 254976, 254976, 1024, 4096, 0x38bb0089 +0, 256000, 256000, 1024, 4096, 0x4fa4f4b6 +0, 257024, 257024, 1024, 4096, 0x47c9fd4c +0, 258048, 258048, 1024, 4096, 0x6e95d5d1 +0, 259072, 259072, 1024, 4096, 0x6deaf758 +0, 260096, 260096, 1024, 4096, 0x96a2f230 +0, 261120, 261120, 1024, 4096, 0x6515fa5b +0, 262144, 262144, 1024, 4096, 0xbdf6f5a6 +0, 263168, 263168, 1024, 4096, 0x74d4f365 +0, 264192, 264192, 408, 1632, 0x637b3c21 -- 2.35.1.1094.g7c7d902a7c-goog _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [FFmpeg-devel] [PATCH] avfilter/alimiter: Remove the delay introduced by lookahead buffer 2022-03-30 21:03 ` [FFmpeg-devel] [PATCH] avfilter/alimiter: Remove the delay introduced by lookahead buffer Wang Cao @ 2022-04-04 15:46 ` Paul B Mahol 0 siblings, 0 replies; 24+ messages in thread From: Paul B Mahol @ 2022-04-04 15:46 UTC (permalink / raw) To: FFmpeg development discussions and patches; +Cc: Wang Cao On Wed, Mar 30, 2022 at 11:05 PM Wang Cao <wangcao-at-google.com@ffmpeg.org> wrote: > The change essentially removes the delay introduces by lookahead buffer. > The valid audio data in the internal buffer is also flushed to ensure > the integrity of output. > > This adds extra delay, and complicates current code a lot. > Signed-off-by: Wang Cao <wangcao@google.com> > --- > doc/filters.texi | 2 - > libavfilter/af_alimiter.c | 97 +++++- > tests/ref/fate/filter-alimiter | 518 ++++++++++++++++----------------- > 3 files changed, 344 insertions(+), 273 deletions(-) > > Thanks a lot for your time to review my patch. I have made "zero_delay" > option to default. Now the output samples will only be written after the > lookahead buffer is full. We also uses "request_frame" to flush the > internal buffers so that all valid input audio samples are written. > > diff --git a/doc/filters.texi b/doc/filters.texi > index d70ac3e237..5d1adf88e1 100644 > --- a/doc/filters.texi > +++ b/doc/filters.texi > @@ -1943,8 +1943,6 @@ aiir=z=1.3057 0 0 0:p=1.3057 2.3892 2.1860 1:f=sf:r=d > > The limiter prevents an input signal from rising over a desired threshold. > This limiter uses lookahead technology to prevent your signal from > distorting. > -It means that there is a small delay after the signal is processed. Keep > in mind > -that the delay it produces is the attack time you set. > > The filter accepts the following options: > > diff --git a/libavfilter/af_alimiter.c b/libavfilter/af_alimiter.c > index 133f98f165..e1fcf98574 100644 > --- a/libavfilter/af_alimiter.c > +++ b/libavfilter/af_alimiter.c > @@ -30,6 +30,7 @@ > > #include "audio.h" > #include "avfilter.h" > +#include "bufferqueue.h" > #include "formats.h" > #include "internal.h" > > @@ -55,6 +56,13 @@ typedef struct AudioLimiterContext { > int *nextpos; > double *nextdelta; > > + int total_samples_to_flush; > + int lookahead_buffer_full; > + > + struct FFBufQueue output_frame_queue; > + int output_sample_pos; > + int output_frame_pos; > + > double delta; > int nextiter; > int nextlen; > @@ -129,6 +137,8 @@ static int filter_frame(AVFilterLink *inlink, AVFrame > *in) > AVFrame *out; > double *buf; > int n, c, i; > + int can_free_input_frame = 0; > + double peak = 0; > > if (av_frame_is_writable(in)) { > out = in; > @@ -138,12 +148,23 @@ static int filter_frame(AVFilterLink *inlink, > AVFrame *in) > av_frame_free(&in); > return AVERROR(ENOMEM); > } > + can_free_input_frame = 1; > av_frame_copy_props(out, in); > } > - dst = (double *)out->data[0]; > + > + if (ff_bufqueue_is_full(&s->output_frame_queue)) { > + // In the runtime, the total number of frames in the queue is > bounded by > + // attack_time, sample rate and frame size. > + return AVERROR_BUG; > + } > + > + ff_bufqueue_add(ctx, &s->output_frame_queue, out); > > for (n = 0; n < in->nb_samples; n++) { > - double peak = 0; > + out = ff_bufqueue_peek(&s->output_frame_queue, > s->output_frame_pos); > + dst = (double *)out->data[0]; > + dst += s->output_sample_pos * channels; > + peak = 0; > > for (c = 0; c < channels; c++) { > double sample = src[c] * level_in; > @@ -213,8 +234,21 @@ static int filter_frame(AVFilterLink *inlink, AVFrame > *in) > > s->att += s->delta; > > - for (c = 0; c < channels; c++) > - dst[c] = buf[c] * s->att; > + // Checks lookahead buffer (s->buffer) has been filled > + if (!s->lookahead_buffer_full && (s->pos + channels) % > buffer_size == 0) { > + s->lookahead_buffer_full = 1; > + } > + if (s->lookahead_buffer_full) { > + for (c = 0; c < channels; c++) { > + dst[c] = buf[c] * s->att; > + dst[c] = av_clipd(dst[c], -limit, limit) * level * > level_out; > + } > + s->output_sample_pos++; > + if (s->output_sample_pos == out->nb_samples) { > + s->output_frame_pos++; > + s->output_sample_pos = 0; > + } > + } > > if ((s->pos + channels) % buffer_size == nextpos[s->nextiter]) { > if (s->auto_release) { > @@ -261,18 +295,55 @@ static int filter_frame(AVFilterLink *inlink, > AVFrame *in) > if (s->delta != 0. && fabs(s->delta) < 0.00000000000001) > s->delta = 0.; > > - for (c = 0; c < channels; c++) > - dst[c] = av_clipd(dst[c], -limit, limit) * level * level_out; > - > s->pos = (s->pos + channels) % buffer_size; > src += channels; > - dst += channels; > } > - > - if (in != out) > + if (can_free_input_frame) { > av_frame_free(&in); > + } > > - return ff_filter_frame(outlink, out); > + if (s->output_frame_pos > 0) { > + s->output_frame_pos--; > + out = ff_bufqueue_get(&s->output_frame_queue); > + return ff_filter_frame(outlink, out); > + } > + return 0; > +} > + > +static int request_frame(AVFilterLink* outlink) > +{ > + AVFilterContext *ctx = outlink->src; > + AudioLimiterContext *s = (AudioLimiterContext*)ctx->priv; > + int ret; > + AVFilterLink *inlink; > + AVFrame *out; > + AVFrame *silence_frame; > + > + ret = ff_request_frame(ctx->inputs[0]); > + > + if (ret != AVERROR_EOF || s->total_samples_to_flush == 0) { > + // Not necessarily an error, just not EOF. > + return ret; > + } > + > + // Flush the frames in the internal buffer > + if (s->output_frame_pos > 0) { > + out = ff_bufqueue_get(&s->output_frame_queue); > + s->output_frame_pos--; > + return ff_filter_frame(outlink, out); > + } > + > + // We reach here when input filters have finished producing data > (i.e. EOF), > + // but because of the attack param, s->buffer still has meaningful > + // audio content that needs flushing. > + inlink = ctx->inputs[0]; > + // Pushes silence frame to flush valid audio in the s->buffer > + silence_frame = ff_get_audio_buffer(inlink, > s->total_samples_to_flush); > + ret = filter_frame(inlink, silence_frame); > + if (ret < 0) { > + return ret; > + } > + return AVERROR_EOF; > } > > static int config_input(AVFilterLink *inlink) > @@ -292,7 +363,8 @@ static int config_input(AVFilterLink *inlink) > return AVERROR(ENOMEM); > > memset(s->nextpos, -1, obuffer_size * sizeof(*s->nextpos)); > - s->buffer_size = inlink->sample_rate * s->attack * > inlink->ch_layout.nb_channels; > + s->total_samples_to_flush = inlink->sample_rate * s->attack; > + s->buffer_size = s->total_samples_to_flush * > inlink->ch_layout.nb_channels; > s->buffer_size -= s->buffer_size % inlink->ch_layout.nb_channels; > > if (s->buffer_size <= 0) { > @@ -325,6 +397,7 @@ static const AVFilterPad alimiter_outputs[] = { > { > .name = "default", > .type = AVMEDIA_TYPE_AUDIO, > + .request_frame = request_frame, > }, > }; > > diff --git a/tests/ref/fate/filter-alimiter > b/tests/ref/fate/filter-alimiter > index 56cfb614da..93daba92c6 100644 > --- a/tests/ref/fate/filter-alimiter > +++ b/tests/ref/fate/filter-alimiter > @@ -3,262 +3,262 @@ > #codec_id 0: pcm_s16le > #sample_rate 0: 44100 > #channel_layout_name 0: stereo > -0, 0, 0, 1024, 4096, 0xd4194af4 > -0, 1024, 1024, 1024, 4096, 0x686af4ab > -0, 2048, 2048, 1024, 4096, 0xe80cee61 > -0, 3072, 3072, 1024, 4096, 0xa686fbad > -0, 4096, 4096, 1024, 4096, 0x191e062c > -0, 5120, 5120, 1024, 4096, 0x1a13edb1 > -0, 6144, 6144, 1024, 4096, 0xab21f8ef > -0, 7168, 7168, 1024, 4096, 0xaa30e757 > -0, 8192, 8192, 1024, 4096, 0x5afdf69f > -0, 9216, 9216, 1024, 4096, 0x765e05ec > -0, 10240, 10240, 1024, 4096, 0x6484f551 > -0, 11264, 11264, 1024, 4096, 0x61b9e9f7 > -0, 12288, 12288, 1024, 4096, 0xa735feb3 > -0, 13312, 13312, 1024, 4096, 0xb47203aa > -0, 14336, 14336, 1024, 4096, 0xb6b5fbf5 > -0, 15360, 15360, 1024, 4096, 0xbc66f14f > -0, 16384, 16384, 1024, 4096, 0x2003ff5b > -0, 17408, 17408, 1024, 4096, 0x1160f17d > -0, 18432, 18432, 1024, 4096, 0x308001a4 > -0, 19456, 19456, 1024, 4096, 0x9df9f429 > -0, 20480, 20480, 1024, 4096, 0x3e6eec0f > -0, 21504, 21504, 1024, 4096, 0xca3301f2 > -0, 22528, 22528, 1024, 4096, 0x9eb1f961 > -0, 23552, 23552, 1024, 4096, 0xd0a50c8c > -0, 24576, 24576, 1024, 4096, 0x071ee96b > -0, 25600, 25600, 1024, 4096, 0x7a46f05b > -0, 26624, 26624, 1024, 4096, 0x2cb2f475 > -0, 27648, 27648, 1024, 4096, 0x5bda0a52 > -0, 28672, 28672, 1024, 4096, 0x33c0f727 > -0, 29696, 29696, 1024, 4096, 0x53cfee59 > -0, 30720, 30720, 1024, 4096, 0x1588f221 > -0, 31744, 31744, 1024, 4096, 0x95d400d6 > -0, 32768, 32768, 1024, 4096, 0x0078009a > -0, 33792, 33792, 1024, 4096, 0x686af4ab > -0, 34816, 34816, 1024, 4096, 0xe80cee61 > -0, 35840, 35840, 1024, 4096, 0xa686fbad > -0, 36864, 36864, 1024, 4096, 0x191e062c > -0, 37888, 37888, 1024, 4096, 0x1a13edb1 > -0, 38912, 38912, 1024, 4096, 0xab21f8ef > -0, 39936, 39936, 1024, 4096, 0xaa30e757 > -0, 40960, 40960, 1024, 4096, 0x5afdf69f > -0, 41984, 41984, 1024, 4096, 0x765e05ec > -0, 43008, 43008, 1024, 4096, 0x6484f551 > -0, 44032, 44032, 1024, 4096, 0x755e0600 > -0, 45056, 45056, 1024, 4096, 0x5056ecb9 > -0, 46080, 46080, 1024, 4096, 0xdcb609a8 > -0, 47104, 47104, 1024, 4096, 0xc87bf4a1 > -0, 48128, 48128, 1024, 4096, 0xecdfef95 > -0, 49152, 49152, 1024, 4096, 0x905ff13f > -0, 50176, 50176, 1024, 4096, 0x4e69eeb3 > -0, 51200, 51200, 1024, 4096, 0x16e1082c > -0, 52224, 52224, 1024, 4096, 0x606c0a22 > -0, 53248, 53248, 1024, 4096, 0x9f94f351 > -0, 54272, 54272, 1024, 4096, 0x2c47f63f > -0, 55296, 55296, 1024, 4096, 0x9e14ebf9 > -0, 56320, 56320, 1024, 4096, 0x7804fbcb > -0, 57344, 57344, 1024, 4096, 0xc7c6ffb7 > -0, 58368, 58368, 1024, 4096, 0xa547f68d > -0, 59392, 59392, 1024, 4096, 0x0c87fed7 > -0, 60416, 60416, 1024, 4096, 0x88b5fef1 > -0, 61440, 61440, 1024, 4096, 0x325af00d > -0, 62464, 62464, 1024, 4096, 0xd826edd9 > -0, 63488, 63488, 1024, 4096, 0x1fbe1138 > -0, 64512, 64512, 1024, 4096, 0xdf85da7b > -0, 65536, 65536, 1024, 4096, 0xf987f9b3 > -0, 66560, 66560, 1024, 4096, 0x5df5e12f > -0, 67584, 67584, 1024, 4096, 0x259cf8ef > -0, 68608, 68608, 1024, 4096, 0xa191eb4f > -0, 69632, 69632, 1024, 4096, 0xfe5bf1fd > -0, 70656, 70656, 1024, 4096, 0x7675e2cb > -0, 71680, 71680, 1024, 4096, 0x796a2f90 > -0, 72704, 72704, 1024, 4096, 0xaa52e7a5 > -0, 73728, 73728, 1024, 4096, 0x2b56fd81 > -0, 74752, 74752, 1024, 4096, 0xbf7c02ee > -0, 75776, 75776, 1024, 4096, 0xfe4cebb1 > -0, 76800, 76800, 1024, 4096, 0xe5b7fdf7 > -0, 77824, 77824, 1024, 4096, 0xd271ece7 > -0, 78848, 78848, 1024, 4096, 0x96e0f69f > -0, 79872, 79872, 1024, 4096, 0x6e5eef33 > -0, 80896, 80896, 1024, 4096, 0x78b70b6e > -0, 81920, 81920, 1024, 4096, 0x61f2f075 > -0, 82944, 82944, 1024, 4096, 0xf1dc47dc > -0, 83968, 83968, 1024, 4096, 0xf4b406d2 > -0, 84992, 84992, 1024, 4096, 0xe33806fe > -0, 86016, 86016, 1024, 4096, 0x5b5ef087 > -0, 87040, 87040, 1024, 4096, 0x3fa207da > -0, 88064, 88064, 1024, 4096, 0x4719f201 > -0, 89088, 89088, 1024, 4096, 0x641feaad > -0, 90112, 90112, 1024, 4096, 0x905402ec > -0, 91136, 91136, 1024, 4096, 0xf529e11f > -0, 92160, 92160, 1024, 4096, 0x65a41838 > -0, 93184, 93184, 1024, 4096, 0x651efb6f > -0, 94208, 94208, 1024, 4096, 0x251b1390 > -0, 95232, 95232, 1024, 4096, 0x45ea05a8 > -0, 96256, 96256, 1024, 4096, 0xe65be141 > -0, 97280, 97280, 1024, 4096, 0xd452d3dd > -0, 98304, 98304, 1024, 4096, 0x3bdff821 > -0, 99328, 99328, 1024, 4096, 0x74da00e6 > -0, 100352, 100352, 1024, 4096, 0x4474fa19 > -0, 101376, 101376, 1024, 4096, 0x11ddedef > -0, 102400, 102400, 1024, 4096, 0xf027d72b > -0, 103424, 103424, 1024, 4096, 0xaa0a1ce6 > -0, 104448, 104448, 1024, 4096, 0x7168145e > -0, 105472, 105472, 1024, 4096, 0x94a710a4 > -0, 106496, 106496, 1024, 4096, 0x6f55ecf1 > -0, 107520, 107520, 1024, 4096, 0xc12de6d7 > -0, 108544, 108544, 1024, 4096, 0x8ff7db15 > -0, 109568, 109568, 1024, 4096, 0x366d0cc6 > -0, 110592, 110592, 1024, 4096, 0xb04afa4f > -0, 111616, 111616, 1024, 4096, 0xf6abbee7 > -0, 112640, 112640, 1024, 4096, 0xbe9910dc > -0, 113664, 113664, 1024, 4096, 0x96c3047e > -0, 114688, 114688, 1024, 4096, 0x9dc1e8e1 > -0, 115712, 115712, 1024, 4096, 0x4b2c1092 > -0, 116736, 116736, 1024, 4096, 0x592d212c > -0, 117760, 117760, 1024, 4096, 0x5c3bda15 > -0, 118784, 118784, 1024, 4096, 0x43c8e90f > -0, 119808, 119808, 1024, 4096, 0x3bfa074e > -0, 120832, 120832, 1024, 4096, 0xa2d7d443 > -0, 121856, 121856, 1024, 4096, 0xd4b02844 > -0, 122880, 122880, 1024, 4096, 0x9f5d2a92 > -0, 123904, 123904, 1024, 4096, 0xc19bf69d > -0, 124928, 124928, 1024, 4096, 0x47b8fe75 > -0, 125952, 125952, 1024, 4096, 0x337334b0 > -0, 126976, 126976, 1024, 4096, 0x96ed14f0 > -0, 128000, 128000, 1024, 4096, 0x9af0f67b > -0, 129024, 129024, 1024, 4096, 0xdf541a60 > -0, 130048, 130048, 1024, 4096, 0x684f0c06 > -0, 131072, 131072, 1024, 4096, 0xe3d0015e > -0, 132096, 132096, 1024, 4096, 0xf648d73c > -0, 133120, 133120, 1024, 4096, 0x4041f3ee > -0, 134144, 134144, 1024, 4096, 0x1421025f > -0, 135168, 135168, 1024, 4096, 0x4eb5fc97 > -0, 136192, 136192, 1024, 4096, 0x510f02fe > -0, 137216, 137216, 1024, 4096, 0x85e9e95e > -0, 138240, 138240, 1024, 4096, 0xb1d0fe10 > -0, 139264, 139264, 1024, 4096, 0xda66f5f8 > -0, 140288, 140288, 1024, 4096, 0x3826eaa8 > -0, 141312, 141312, 1024, 4096, 0xa7cc0176 > -0, 142336, 142336, 1024, 4096, 0x71e8fe5a > -0, 143360, 143360, 1024, 4096, 0x0149fcfa > -0, 144384, 144384, 1024, 4096, 0xbd36fd8e > -0, 145408, 145408, 1024, 4096, 0xec1afcc6 > -0, 146432, 146432, 1024, 4096, 0xea80ec5e > -0, 147456, 147456, 1024, 4096, 0x8da1f0ac > -0, 148480, 148480, 1024, 4096, 0x63c3e61c > -0, 149504, 149504, 1024, 4096, 0x188cf09b > -0, 150528, 150528, 1024, 4096, 0x7eebea85 > -0, 151552, 151552, 1024, 4096, 0x7ef6f718 > -0, 152576, 152576, 1024, 4096, 0xcd9fecfb > -0, 153600, 153600, 1024, 4096, 0x1aa7f624 > -0, 154624, 154624, 1024, 4096, 0xcab5f1e3 > -0, 155648, 155648, 1024, 4096, 0x36e9f795 > -0, 156672, 156672, 1024, 4096, 0x952cf54d > -0, 157696, 157696, 1024, 4096, 0x54a4ed3b > -0, 158720, 158720, 1024, 4096, 0x1e24f992 > -0, 159744, 159744, 1024, 4096, 0xab6bfa09 > -0, 160768, 160768, 1024, 4096, 0xa021020b > -0, 161792, 161792, 1024, 4096, 0xaa0600f5 > -0, 162816, 162816, 1024, 4096, 0xadf7ec92 > -0, 163840, 163840, 1024, 4096, 0x5dd5f63f > -0, 164864, 164864, 1024, 4096, 0xc01bff89 > -0, 165888, 165888, 1024, 4096, 0x05dbf1e7 > -0, 166912, 166912, 1024, 4096, 0x44f00038 > -0, 167936, 167936, 1024, 4096, 0x6dc5f886 > -0, 168960, 168960, 1024, 4096, 0xeb88e729 > -0, 169984, 169984, 1024, 4096, 0xfe5cf53e > -0, 171008, 171008, 1024, 4096, 0x2692f7a0 > -0, 172032, 172032, 1024, 4096, 0xe984f24b > -0, 173056, 173056, 1024, 4096, 0x1a4ef732 > -0, 174080, 174080, 1024, 4096, 0x3b60010b > -0, 175104, 175104, 1024, 4096, 0x6dd0eaf2 > -0, 176128, 176128, 1024, 4096, 0x313de851 > -0, 177152, 177152, 1024, 4096, 0x239ef043 > -0, 178176, 178176, 1024, 4096, 0x2366fe43 > -0, 179200, 179200, 1024, 4096, 0xbfd6f3b9 > -0, 180224, 180224, 1024, 4096, 0xb34bf67b > -0, 181248, 181248, 1024, 4096, 0x00def7c1 > -0, 182272, 182272, 1024, 4096, 0xa6d0f466 > -0, 183296, 183296, 1024, 4096, 0xd1a1f1c2 > -0, 184320, 184320, 1024, 4096, 0x32f8923c > -0, 185344, 185344, 1024, 4096, 0x137001d2 > -0, 186368, 186368, 1024, 4096, 0xb881f6cd > -0, 187392, 187392, 1024, 4096, 0xf44e034a > -0, 188416, 188416, 1024, 4096, 0xb43fecf7 > -0, 189440, 189440, 1024, 4096, 0xe62ced50 > -0, 190464, 190464, 1024, 4096, 0x221dfd0d > -0, 191488, 191488, 1024, 4096, 0x85400147 > -0, 192512, 192512, 1024, 4096, 0x8bf8d054 > -0, 193536, 193536, 1024, 4096, 0x08370170 > -0, 194560, 194560, 1024, 4096, 0xec62effa > -0, 195584, 195584, 1024, 4096, 0xd58cf67c > -0, 196608, 196608, 1024, 4096, 0xf597f9a1 > -0, 197632, 197632, 1024, 4096, 0x63fcf0b0 > -0, 198656, 198656, 1024, 4096, 0x8cc8fd32 > -0, 199680, 199680, 1024, 4096, 0xf8c2072c > -0, 200704, 200704, 1024, 4096, 0x0c7e93a5 > -0, 201728, 201728, 1024, 4096, 0x1cc3f612 > -0, 202752, 202752, 1024, 4096, 0x83adf8ee > -0, 203776, 203776, 1024, 4096, 0x680e0195 > -0, 204800, 204800, 1024, 4096, 0x9d34fd2e > -0, 205824, 205824, 1024, 4096, 0x44cdfb34 > -0, 206848, 206848, 1024, 4096, 0x193df790 > -0, 207872, 207872, 1024, 4096, 0xb337ef64 > -0, 208896, 208896, 1024, 4096, 0x0074ee38 > -0, 209920, 209920, 1024, 4096, 0x239ef043 > -0, 210944, 210944, 1024, 4096, 0x2366fe43 > -0, 211968, 211968, 1024, 4096, 0xbfd6f3b9 > -0, 212992, 212992, 1024, 4096, 0xb34bf67b > -0, 214016, 214016, 1024, 4096, 0x00def7c1 > -0, 215040, 215040, 1024, 4096, 0xa6d0f466 > -0, 216064, 216064, 1024, 4096, 0xd1a1f1c2 > -0, 217088, 217088, 1024, 4096, 0x32f8923c > -0, 218112, 218112, 1024, 4096, 0x137001d2 > -0, 219136, 219136, 1024, 4096, 0xb881f6cd > -0, 220160, 220160, 1024, 4096, 0xf44e034a > -0, 221184, 221184, 1024, 4096, 0xb43fecf7 > -0, 222208, 222208, 1024, 4096, 0xe62ced50 > -0, 223232, 223232, 1024, 4096, 0x221dfd0d > -0, 224256, 224256, 1024, 4096, 0x85400147 > -0, 225280, 225280, 1024, 4096, 0x8bf8d054 > -0, 226304, 226304, 1024, 4096, 0x08370170 > -0, 227328, 227328, 1024, 4096, 0xec62effa > -0, 228352, 228352, 1024, 4096, 0xd58cf67c > -0, 229376, 229376, 1024, 4096, 0xf597f9a1 > -0, 230400, 230400, 1024, 4096, 0x63fcf0b0 > -0, 231424, 231424, 1024, 4096, 0x8cc8fd32 > -0, 232448, 232448, 1024, 4096, 0xf8c2072c > -0, 233472, 233472, 1024, 4096, 0x0c7e93a5 > -0, 234496, 234496, 1024, 4096, 0x1cc3f612 > -0, 235520, 235520, 1024, 4096, 0x83adf8ee > -0, 236544, 236544, 1024, 4096, 0x680e0195 > -0, 237568, 237568, 1024, 4096, 0x9d34fd2e > -0, 238592, 238592, 1024, 4096, 0x44cdfb34 > -0, 239616, 239616, 1024, 4096, 0x193df790 > -0, 240640, 240640, 1024, 4096, 0xb337ef64 > -0, 241664, 241664, 1024, 4096, 0x0074ee38 > -0, 242688, 242688, 1024, 4096, 0x239ef043 > -0, 243712, 243712, 1024, 4096, 0x2366fe43 > -0, 244736, 244736, 1024, 4096, 0xbfd6f3b9 > -0, 245760, 245760, 1024, 4096, 0xb34bf67b > -0, 246784, 246784, 1024, 4096, 0x00def7c1 > -0, 247808, 247808, 1024, 4096, 0xa6d0f466 > -0, 248832, 248832, 1024, 4096, 0xd1a1f1c2 > -0, 249856, 249856, 1024, 4096, 0x32f8923c > -0, 250880, 250880, 1024, 4096, 0x137001d2 > -0, 251904, 251904, 1024, 4096, 0xb881f6cd > -0, 252928, 252928, 1024, 4096, 0xf44e034a > -0, 253952, 253952, 1024, 4096, 0xb43fecf7 > -0, 254976, 254976, 1024, 4096, 0xe62ced50 > -0, 256000, 256000, 1024, 4096, 0x221dfd0d > -0, 257024, 257024, 1024, 4096, 0x85400147 > -0, 258048, 258048, 1024, 4096, 0x8bf8d054 > -0, 259072, 259072, 1024, 4096, 0x08370170 > -0, 260096, 260096, 1024, 4096, 0xec62effa > -0, 261120, 261120, 1024, 4096, 0xd58cf67c > -0, 262144, 262144, 1024, 4096, 0xf597f9a1 > -0, 263168, 263168, 1024, 4096, 0x63fcf0b0 > -0, 264192, 264192, 408, 1632, 0xedfa314a > +0, 0, 0, 1024, 4096, 0xd0b20048 > +0, 1024, 1024, 1024, 4096, 0x8abaf0ab > +0, 2048, 2048, 1024, 4096, 0x5a00f489 > +0, 3072, 3072, 1024, 4096, 0x6d87f3d3 > +0, 4096, 4096, 1024, 4096, 0x27fc04bc > +0, 5120, 5120, 1024, 4096, 0x6d18f2e1 > +0, 6144, 6144, 1024, 4096, 0x6a11f0c9 > +0, 7168, 7168, 1024, 4096, 0xadf8ec27 > +0, 8192, 8192, 1024, 4096, 0xf0e2fcf9 > +0, 9216, 9216, 1024, 4096, 0x55280372 > +0, 10240, 10240, 1024, 4096, 0x898ffad3 > +0, 11264, 11264, 1024, 4096, 0x06a5eb6b > +0, 12288, 12288, 1024, 4096, 0x9fadfe07 > +0, 13312, 13312, 1024, 4096, 0x6f32fd99 > +0, 14336, 14336, 1024, 4096, 0x7856f871 > +0, 15360, 15360, 1024, 4096, 0xf742f893 > +0, 16384, 16384, 1024, 4096, 0x7955f7e7 > +0, 17408, 17408, 1024, 4096, 0xbb32f521 > +0, 18432, 18432, 1024, 4096, 0x8302fda9 > +0, 19456, 19456, 1024, 4096, 0x0d3cfdeb > +0, 20480, 20480, 1024, 4096, 0xd034ed5f > +0, 21504, 21504, 1024, 4096, 0xd3c1fb11 > +0, 22528, 22528, 1024, 4096, 0x80bc0734 > +0, 23552, 23552, 1024, 4096, 0x82a60000 > +0, 24576, 24576, 1024, 4096, 0xaf6ceb09 > +0, 25600, 25600, 1024, 4096, 0x0c0dec9f > +0, 26624, 26624, 1024, 4096, 0x8ca8f335 > +0, 27648, 27648, 1024, 4096, 0xc75604bc > +0, 28672, 28672, 1024, 4096, 0x0a22f7b9 > +0, 29696, 29696, 1024, 4096, 0x237cf4a5 > +0, 30720, 30720, 1024, 4096, 0x5276f36f > +0, 31744, 31744, 1024, 4096, 0x148a0194 > +0, 32768, 32768, 1024, 4096, 0xd0b20048 > +0, 33792, 33792, 1024, 4096, 0x8abaf0ab > +0, 34816, 34816, 1024, 4096, 0x5a00f489 > +0, 35840, 35840, 1024, 4096, 0x6d87f3d3 > +0, 36864, 36864, 1024, 4096, 0x27fc04bc > +0, 37888, 37888, 1024, 4096, 0x6d18f2e1 > +0, 38912, 38912, 1024, 4096, 0x6a11f0c9 > +0, 39936, 39936, 1024, 4096, 0xadf8ec27 > +0, 40960, 40960, 1024, 4096, 0xf0e2fcf9 > +0, 41984, 41984, 1024, 4096, 0x55280372 > +0, 43008, 43008, 1024, 4096, 0x8ac5fad5 > +0, 44032, 44032, 1024, 4096, 0x934007ba > +0, 45056, 45056, 1024, 4096, 0x84acf4a3 > +0, 46080, 46080, 1024, 4096, 0x7c8ff49b > +0, 47104, 47104, 1024, 4096, 0x79030244 > +0, 48128, 48128, 1024, 4096, 0x28d0e5f5 > +0, 49152, 49152, 1024, 4096, 0xf12df059 > +0, 50176, 50176, 1024, 4096, 0xd0eef7a7 > +0, 51200, 51200, 1024, 4096, 0x2e2e02e4 > +0, 52224, 52224, 1024, 4096, 0x7c421656 > +0, 53248, 53248, 1024, 4096, 0x79fcea35 > +0, 54272, 54272, 1024, 4096, 0xb566e88b > +0, 55296, 55296, 1024, 4096, 0xad66f8e9 > +0, 56320, 56320, 1024, 4096, 0xe0d001dc > +0, 57344, 57344, 1024, 4096, 0xfc5df6cd > +0, 58368, 58368, 1024, 4096, 0x9dcefad5 > +0, 59392, 59392, 1024, 4096, 0xbde90702 > +0, 60416, 60416, 1024, 4096, 0x4feef329 > +0, 61440, 61440, 1024, 4096, 0xbb30ef39 > +0, 62464, 62464, 1024, 4096, 0x131ff1e3 > +0, 63488, 63488, 1024, 4096, 0x994c0184 > +0, 64512, 64512, 1024, 4096, 0xc981e10b > +0, 65536, 65536, 1024, 4096, 0xceff022a > +0, 66560, 66560, 1024, 4096, 0x07d1e1d3 > +0, 67584, 67584, 1024, 4096, 0x7ea5f31d > +0, 68608, 68608, 1024, 4096, 0x16e1e885 > +0, 69632, 69632, 1024, 4096, 0x2a92fb45 > +0, 70656, 70656, 1024, 4096, 0x9f90ef6b > +0, 71680, 71680, 1024, 4096, 0xed961010 > +0, 72704, 72704, 1024, 4096, 0x6169f659 > +0, 73728, 73728, 1024, 4096, 0xbe830c50 > +0, 74752, 74752, 1024, 4096, 0x1744ea89 > +0, 75776, 75776, 1024, 4096, 0x00bcf34f > +0, 76800, 76800, 1024, 4096, 0x91f7f7f3 > +0, 77824, 77824, 1024, 4096, 0xa4e1f1fb > +0, 78848, 78848, 1024, 4096, 0xf3e5f73b > +0, 79872, 79872, 1024, 4096, 0x64d0eb95 > +0, 80896, 80896, 1024, 4096, 0xe0c90d40 > +0, 81920, 81920, 1024, 4096, 0xe9701e32 > +0, 82944, 82944, 1024, 4096, 0x760d26b0 > +0, 83968, 83968, 1024, 4096, 0x3f4303e4 > +0, 84992, 84992, 1024, 4096, 0x1cc502bc > +0, 86016, 86016, 1024, 4096, 0x6d7bf089 > +0, 87040, 87040, 1024, 4096, 0xe03807fc > +0, 88064, 88064, 1024, 4096, 0xf12fe897 > +0, 89088, 89088, 1024, 4096, 0x0123e87d > +0, 90112, 90112, 1024, 4096, 0x869808d4 > +0, 91136, 91136, 1024, 4096, 0x9538def3 > +0, 92160, 92160, 1024, 4096, 0x92ed2ab0 > +0, 93184, 93184, 1024, 4096, 0xc80bfe9d > +0, 94208, 94208, 1024, 4096, 0x56130220 > +0, 95232, 95232, 1024, 4096, 0xd3a7f0f3 > +0, 96256, 96256, 1024, 4096, 0xfca0eeaf > +0, 97280, 97280, 1024, 4096, 0xd409d82b > +0, 98304, 98304, 1024, 4096, 0x29ba0424 > +0, 99328, 99328, 1024, 4096, 0x4e9cdf19 > +0, 100352, 100352, 1024, 4096, 0xb0511cc0 > +0, 101376, 101376, 1024, 4096, 0xd499e541 > +0, 102400, 102400, 1024, 4096, 0x0910f25f > +0, 103424, 103424, 1024, 4096, 0x6e5d1c08 > +0, 104448, 104448, 1024, 4096, 0xa07511a8 > +0, 105472, 105472, 1024, 4096, 0xddd1ee73 > +0, 106496, 106496, 1024, 4096, 0x5b46fb17 > +0, 107520, 107520, 1024, 4096, 0xc558e301 > +0, 108544, 108544, 1024, 4096, 0x9041e627 > +0, 109568, 109568, 1024, 4096, 0xeadae195 > +0, 110592, 110592, 1024, 4096, 0x663d0c5a > +0, 111616, 111616, 1024, 4096, 0x1c57d55b > +0, 112640, 112640, 1024, 4096, 0x7ce90702 > +0, 113664, 113664, 1024, 4096, 0xf56c15d0 > +0, 114688, 114688, 1024, 4096, 0xa2edd14d > +0, 115712, 115712, 1024, 4096, 0x93f1171c > +0, 116736, 116736, 1024, 4096, 0xe02e152a > +0, 117760, 117760, 1024, 4096, 0x55aaeba7 > +0, 118784, 118784, 1024, 4096, 0xacabd7a7 > +0, 119808, 119808, 1024, 4096, 0xaa18150a > +0, 120832, 120832, 1024, 4096, 0x065ed6a1 > +0, 121856, 121856, 1024, 4096, 0x9df9268e > +0, 122880, 122880, 1024, 4096, 0x8e0a23e6 > +0, 123904, 123904, 1024, 4096, 0x872d019e > +0, 124928, 124928, 1024, 4096, 0x3aff116c > +0, 125952, 125952, 1024, 4096, 0x4a1717f8 > +0, 126976, 126976, 1024, 4096, 0xa3812998 > +0, 128000, 128000, 1024, 4096, 0x8f12ee4f > +0, 129024, 129024, 1024, 4096, 0x1add13c4 > +0, 130048, 130048, 1024, 4096, 0x4f3825da > +0, 131072, 131072, 1024, 4096, 0xf732d42b > +0, 132096, 132096, 1024, 4096, 0xa2b3e578 > +0, 133120, 133120, 1024, 4096, 0xdc18ff83 > +0, 134144, 134144, 1024, 4096, 0xd13df8c5 > +0, 135168, 135168, 1024, 4096, 0x24c506ff > +0, 136192, 136192, 1024, 4096, 0xb58cf9d4 > +0, 137216, 137216, 1024, 4096, 0x734cecba > +0, 138240, 138240, 1024, 4096, 0x9af5fb4e > +0, 139264, 139264, 1024, 4096, 0x8510f05f > +0, 140288, 140288, 1024, 4096, 0x81e0f2c3 > +0, 141312, 141312, 1024, 4096, 0x5c060641 > +0, 142336, 142336, 1024, 4096, 0xb15bfc84 > +0, 143360, 143360, 1024, 4096, 0xe7dafcaa > +0, 144384, 144384, 1024, 4096, 0x5419faaa > +0, 145408, 145408, 1024, 4096, 0x6f97fa6f > +0, 146432, 146432, 1024, 4096, 0x5e69e975 > +0, 147456, 147456, 1024, 4096, 0xd614f89e > +0, 148480, 148480, 1024, 4096, 0x2532e193 > +0, 149504, 149504, 1024, 4096, 0x0ce9f04c > +0, 150528, 150528, 1024, 4096, 0x0935ec67 > +0, 151552, 151552, 1024, 4096, 0x92d6f5d9 > +0, 152576, 152576, 1024, 4096, 0xccffea44 > +0, 153600, 153600, 1024, 4096, 0x81e3eea0 > +0, 154624, 154624, 1024, 4096, 0xc5c2fe66 > +0, 155648, 155648, 1024, 4096, 0x75dbf93c > +0, 156672, 156672, 1024, 4096, 0x6294f023 > +0, 157696, 157696, 1024, 4096, 0xf39eef54 > +0, 158720, 158720, 1024, 4096, 0x03b1fa13 > +0, 159744, 159744, 1024, 4096, 0xacc0faf2 > +0, 160768, 160768, 1024, 4096, 0xcd9ffc79 > +0, 161792, 161792, 1024, 4096, 0x4ffc023f > +0, 162816, 162816, 1024, 4096, 0x3126ed15 > +0, 163840, 163840, 1024, 4096, 0x611ff8d6 > +0, 164864, 164864, 1024, 4096, 0x6a9bfa1a > +0, 165888, 165888, 1024, 4096, 0xb06cfb10 > +0, 166912, 166912, 1024, 4096, 0x1c9f00bb > +0, 167936, 167936, 1024, 4096, 0xc0bbeebf > +0, 168960, 168960, 1024, 4096, 0x383eea91 > +0, 169984, 169984, 1024, 4096, 0xf927f893 > +0, 171008, 171008, 1024, 4096, 0xbfc3f9b4 > +0, 172032, 172032, 1024, 4096, 0x9faaef83 > +0, 173056, 173056, 1024, 4096, 0x4dc6f411 > +0, 174080, 174080, 1024, 4096, 0x9432ff0f > +0, 175104, 175104, 1024, 4096, 0x62d5e7d6 > +0, 176128, 176128, 1024, 4096, 0x1af9ea59 > +0, 177152, 177152, 1024, 4096, 0x3dcbf8d0 > +0, 178176, 178176, 1024, 4096, 0x80fbf978 > +0, 179200, 179200, 1024, 4096, 0xbe60f05b > +0, 180224, 180224, 1024, 4096, 0xf12ffb83 > +0, 181248, 181248, 1024, 4096, 0xfed0f81a > +0, 182272, 182272, 1024, 4096, 0x892eede2 > +0, 183296, 183296, 1024, 4096, 0x4101f200 > +0, 184320, 184320, 1024, 4096, 0x08759c99 > +0, 185344, 185344, 1024, 4096, 0xea80fab7 > +0, 186368, 186368, 1024, 4096, 0x2b47fcf4 > +0, 187392, 187392, 1024, 4096, 0x2933f949 > +0, 188416, 188416, 1024, 4096, 0x10c3eaed > +0, 189440, 189440, 1024, 4096, 0x38bb0089 > +0, 190464, 190464, 1024, 4096, 0x4fa4f4b6 > +0, 191488, 191488, 1024, 4096, 0x47c9fd4c > +0, 192512, 192512, 1024, 4096, 0x6e95d5d1 > +0, 193536, 193536, 1024, 4096, 0x6deaf758 > +0, 194560, 194560, 1024, 4096, 0x96a2f230 > +0, 195584, 195584, 1024, 4096, 0x6515fa5b > +0, 196608, 196608, 1024, 4096, 0xbdf6f5a6 > +0, 197632, 197632, 1024, 4096, 0x74d4f365 > +0, 198656, 198656, 1024, 4096, 0x932903e3 > +0, 199680, 199680, 1024, 4096, 0x7468085e > +0, 200704, 200704, 1024, 4096, 0x0d908b65 > +0, 201728, 201728, 1024, 4096, 0x86e7f740 > +0, 202752, 202752, 1024, 4096, 0x09fff8d2 > +0, 203776, 203776, 1024, 4096, 0xf0430838 > +0, 204800, 204800, 1024, 4096, 0x525bf9ec > +0, 205824, 205824, 1024, 4096, 0x17c7ee8d > +0, 206848, 206848, 1024, 4096, 0x7632fc3b > +0, 207872, 207872, 1024, 4096, 0xf12aefe9 > +0, 208896, 208896, 1024, 4096, 0x0d72ed83 > +0, 209920, 209920, 1024, 4096, 0x3dcbf8d0 > +0, 210944, 210944, 1024, 4096, 0x80fbf978 > +0, 211968, 211968, 1024, 4096, 0xbe60f05b > +0, 212992, 212992, 1024, 4096, 0xf12ffb83 > +0, 214016, 214016, 1024, 4096, 0xfed0f81a > +0, 215040, 215040, 1024, 4096, 0x892eede2 > +0, 216064, 216064, 1024, 4096, 0x4101f200 > +0, 217088, 217088, 1024, 4096, 0x08759c99 > +0, 218112, 218112, 1024, 4096, 0xea80fab7 > +0, 219136, 219136, 1024, 4096, 0x2b47fcf4 > +0, 220160, 220160, 1024, 4096, 0x2933f949 > +0, 221184, 221184, 1024, 4096, 0x10c3eaed > +0, 222208, 222208, 1024, 4096, 0x38bb0089 > +0, 223232, 223232, 1024, 4096, 0x4fa4f4b6 > +0, 224256, 224256, 1024, 4096, 0x47c9fd4c > +0, 225280, 225280, 1024, 4096, 0x6e95d5d1 > +0, 226304, 226304, 1024, 4096, 0x6deaf758 > +0, 227328, 227328, 1024, 4096, 0x96a2f230 > +0, 228352, 228352, 1024, 4096, 0x6515fa5b > +0, 229376, 229376, 1024, 4096, 0xbdf6f5a6 > +0, 230400, 230400, 1024, 4096, 0x74d4f365 > +0, 231424, 231424, 1024, 4096, 0x932903e3 > +0, 232448, 232448, 1024, 4096, 0x7468085e > +0, 233472, 233472, 1024, 4096, 0x0d908b65 > +0, 234496, 234496, 1024, 4096, 0x86e7f740 > +0, 235520, 235520, 1024, 4096, 0x09fff8d2 > +0, 236544, 236544, 1024, 4096, 0xf0430838 > +0, 237568, 237568, 1024, 4096, 0x525bf9ec > +0, 238592, 238592, 1024, 4096, 0x17c7ee8d > +0, 239616, 239616, 1024, 4096, 0x7632fc3b > +0, 240640, 240640, 1024, 4096, 0xf12aefe9 > +0, 241664, 241664, 1024, 4096, 0x0d72ed83 > +0, 242688, 242688, 1024, 4096, 0x3dcbf8d0 > +0, 243712, 243712, 1024, 4096, 0x80fbf978 > +0, 244736, 244736, 1024, 4096, 0xbe60f05b > +0, 245760, 245760, 1024, 4096, 0xf12ffb83 > +0, 246784, 246784, 1024, 4096, 0xfed0f81a > +0, 247808, 247808, 1024, 4096, 0x892eede2 > +0, 248832, 248832, 1024, 4096, 0x4101f200 > +0, 249856, 249856, 1024, 4096, 0x08759c99 > +0, 250880, 250880, 1024, 4096, 0xea80fab7 > +0, 251904, 251904, 1024, 4096, 0x2b47fcf4 > +0, 252928, 252928, 1024, 4096, 0x2933f949 > +0, 253952, 253952, 1024, 4096, 0x10c3eaed > +0, 254976, 254976, 1024, 4096, 0x38bb0089 > +0, 256000, 256000, 1024, 4096, 0x4fa4f4b6 > +0, 257024, 257024, 1024, 4096, 0x47c9fd4c > +0, 258048, 258048, 1024, 4096, 0x6e95d5d1 > +0, 259072, 259072, 1024, 4096, 0x6deaf758 > +0, 260096, 260096, 1024, 4096, 0x96a2f230 > +0, 261120, 261120, 1024, 4096, 0x6515fa5b > +0, 262144, 262144, 1024, 4096, 0xbdf6f5a6 > +0, 263168, 263168, 1024, 4096, 0x74d4f365 > +0, 264192, 264192, 408, 1632, 0x637b3c21 > -- > 2.35.1.1094.g7c7d902a7c-goog > > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". > _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [FFmpeg-devel] [PATCH] avfilter/alimiter: Add "flush_buffer" option to flush the remaining valid data to the output 2022-03-27 21:40 ` Marton Balint 2022-03-30 21:03 ` [FFmpeg-devel] [PATCH] avfilter/alimiter: Remove the delay introduced by lookahead buffer Wang Cao @ 2022-04-04 15:49 ` Paul B Mahol 2022-04-04 22:28 ` Marton Balint 1 sibling, 1 reply; 24+ messages in thread From: Paul B Mahol @ 2022-04-04 15:49 UTC (permalink / raw) To: FFmpeg development discussions and patches On Sun, Mar 27, 2022 at 11:41 PM Marton Balint <cus@passwd.hu> wrote: > > > On Sat, 26 Mar 2022, Wang Cao wrote: > > > The change in the commit will add some samples to the end of the audio > > stream. The intention is to add a "zero_delay" option eventually to not > > have the delay in the begining the output from alimiter due to > > lookahead. > > I was very much suprised to see that the alimiter filter actually delays > the audio - as in extra samples are inserted in the beginning and some > samples are cut in the end. This trashes A-V sync, so it is a bug IMHO. > > So unless somebody has some valid usecase for the legacy way of operation > I'd just simply change it to be "zero delay" without any additional user > option, in a single patch. > This is done by this patch in very complicated way and also it really should be optional. Something like in af_ladspa is more reasonable. > > Regards, > Marton > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". > _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [FFmpeg-devel] [PATCH] avfilter/alimiter: Add "flush_buffer" option to flush the remaining valid data to the output 2022-04-04 15:49 ` [FFmpeg-devel] [PATCH] avfilter/alimiter: Add "flush_buffer" option to flush the remaining valid data to the output Paul B Mahol @ 2022-04-04 22:28 ` Marton Balint 2022-04-05 18:56 ` Wang Cao 0 siblings, 1 reply; 24+ messages in thread From: Marton Balint @ 2022-04-04 22:28 UTC (permalink / raw) To: FFmpeg development discussions and patches On Mon, 4 Apr 2022, Paul B Mahol wrote: > On Sun, Mar 27, 2022 at 11:41 PM Marton Balint <cus@passwd.hu> wrote: > >> >> >> On Sat, 26 Mar 2022, Wang Cao wrote: >> >>> The change in the commit will add some samples to the end of the audio >>> stream. The intention is to add a "zero_delay" option eventually to not >>> have the delay in the begining the output from alimiter due to >>> lookahead. >> >> I was very much suprised to see that the alimiter filter actually delays >> the audio - as in extra samples are inserted in the beginning and some >> samples are cut in the end. This trashes A-V sync, so it is a bug IMHO. >> >> So unless somebody has some valid usecase for the legacy way of operation >> I'd just simply change it to be "zero delay" without any additional user >> option, in a single patch. >> > > > This is done by this patch in very complicated way and also it really > should be optional. But why does it make sense to keep the current (IMHO buggy) operational mode which adds silence in the beginning and trims the end? I understand that the original implementation worked like this, but libavfilter has packet timestamps and N:M filtering so there is absolutely no reason to use an 1:1 implementation and live with its limitations. Regards, Marton _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [FFmpeg-devel] [PATCH] avfilter/alimiter: Add "flush_buffer" option to flush the remaining valid data to the output 2022-04-04 22:28 ` Marton Balint @ 2022-04-05 18:56 ` Wang Cao 2022-04-06 11:49 ` Paul B Mahol 0 siblings, 1 reply; 24+ messages in thread From: Wang Cao @ 2022-04-05 18:56 UTC (permalink / raw) To: FFmpeg development discussions and patches On Mon, Apr 4, 2022 at 3:28 PM Marton Balint <cus@passwd.hu> wrote: > > > On Mon, 4 Apr 2022, Paul B Mahol wrote: > > > On Sun, Mar 27, 2022 at 11:41 PM Marton Balint <cus@passwd.hu> wrote: > > > >> > >> > >> On Sat, 26 Mar 2022, Wang Cao wrote: > >> > >>> The change in the commit will add some samples to the end of the audio > >>> stream. The intention is to add a "zero_delay" option eventually to not > >>> have the delay in the begining the output from alimiter due to > >>> lookahead. > >> > >> I was very much suprised to see that the alimiter filter actually delays > >> the audio - as in extra samples are inserted in the beginning and some > >> samples are cut in the end. This trashes A-V sync, so it is a bug IMHO. > >> > >> So unless somebody has some valid usecase for the legacy way of > operation > >> I'd just simply change it to be "zero delay" without any additional user > >> option, in a single patch. > >> > > > > > > This is done by this patch in very complicated way and also it really > > should be optional. > > But why does it make sense to keep the current (IMHO buggy) operational > mode which adds silence in the beginning and trims the end? I understand > that the original implementation worked like this, but libavfilter has > packet timestamps and N:M filtering so there is absolutely no reason to > use an 1:1 implementation and live with its limitations. > Hello Paul and Marton, thank you so much for taking time to review my patch. I totally understand that my patch may seem a little bit complicated but I can show with a FATE test that if we set the alimiter to behave as a passthrough filter, the output frames will be the same from "framecrc" with my patch. The existing behavior will not work for all gapless audio processing. The complete patch to fix this issue is at https://patchwork.ffmpeg.org/project/ffmpeg/patch/20220330210314.2055201-1-wangcao@google.com/ Regarding Paul's concern, I personally don't have any preference whether to put the patch as an extra option or not. With respect to the implementation, the patch is the best I can think of by preserving as much information as possible from input frames. I also understand it may break concept that "filter_frame" outputs one frame at a time. For alimiter with my patch, depending on the size of the lookahead buffer, it may take a few frames before one output frame can be generated. This is inevitable to compensate for the delay of the lookahead buffer. Thanks again for reviewing my patch and I'm looking forward to hearing from you :) -- Wang Cao | Software Engineer | wangcao@google.com | 650-203-7807 <(650)%20203-7807> _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [FFmpeg-devel] [PATCH] avfilter/alimiter: Add "flush_buffer" option to flush the remaining valid data to the output 2022-04-05 18:56 ` Wang Cao @ 2022-04-06 11:49 ` Paul B Mahol 2022-04-07 7:46 ` Paul B Mahol 0 siblings, 1 reply; 24+ messages in thread From: Paul B Mahol @ 2022-04-06 11:49 UTC (permalink / raw) To: FFmpeg development discussions and patches On Tue, Apr 5, 2022 at 8:57 PM Wang Cao <wangcao-at-google.com@ffmpeg.org> wrote: > On Mon, Apr 4, 2022 at 3:28 PM Marton Balint <cus@passwd.hu> wrote: > > > > > > > On Mon, 4 Apr 2022, Paul B Mahol wrote: > > > > > On Sun, Mar 27, 2022 at 11:41 PM Marton Balint <cus@passwd.hu> wrote: > > > > > >> > > >> > > >> On Sat, 26 Mar 2022, Wang Cao wrote: > > >> > > >>> The change in the commit will add some samples to the end of the > audio > > >>> stream. The intention is to add a "zero_delay" option eventually to > not > > >>> have the delay in the begining the output from alimiter due to > > >>> lookahead. > > >> > > >> I was very much suprised to see that the alimiter filter actually > delays > > >> the audio - as in extra samples are inserted in the beginning and some > > >> samples are cut in the end. This trashes A-V sync, so it is a bug > IMHO. > > >> > > >> So unless somebody has some valid usecase for the legacy way of > > operation > > >> I'd just simply change it to be "zero delay" without any additional > user > > >> option, in a single patch. > > >> > > > > > > > > > This is done by this patch in very complicated way and also it really > > > should be optional. > > > > But why does it make sense to keep the current (IMHO buggy) operational > > mode which adds silence in the beginning and trims the end? I understand > > that the original implementation worked like this, but libavfilter has > > packet timestamps and N:M filtering so there is absolutely no reason to > > use an 1:1 implementation and live with its limitations. > > > Hello Paul and Marton, thank you so much for taking time to review my > patch. > I totally understand that my patch may seem a little bit complicated but I > can > show with a FATE test that if we set the alimiter to behave as a > passthrough filter, > the output frames will be the same from "framecrc" with my patch. The > existing > behavior will not work for all gapless audio processing. > > The complete patch to fix this issue is at > > https://patchwork.ffmpeg.org/project/ffmpeg/patch/20220330210314.2055201-1-wangcao@google.com/ > > Regarding Paul's concern, I personally don't have any preference whether to > put > the patch as an extra option or not. With respect to the implementation, > the patch > is the best I can think of by preserving as much information as possible > from input > frames. I also understand it may break concept that "filter_frame" outputs > one frame > at a time. For alimiter with my patch, depending on the size of the > lookahead buffer, > it may take a few frames before one output frame can be generated. This is > inevitable > to compensate for the delay of the lookahead buffer. > > Thanks again for reviewing my patch and I'm looking forward to hearing from > you :) > Better than (because its no more 1 frame X nb_samples in, 1 frame X nb_samples out) to replace .filter_frame/.request_frame with .activate logic. And make this output delay compensation filtering optional. In this process make sure that output PTS frame timestamps are unchanged from input one, by keeping reference of needed frames in filter queue. Look how speechnorm/dynaudnorm does it. > -- > > Wang Cao | Software Engineer | wangcao@google.com | 650-203-7807 > <(650)%20203-7807> > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". > _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [FFmpeg-devel] [PATCH] avfilter/alimiter: Add "flush_buffer" option to flush the remaining valid data to the output 2022-04-06 11:49 ` Paul B Mahol @ 2022-04-07 7:46 ` Paul B Mahol 2022-04-07 21:56 ` Wang Cao 0 siblings, 1 reply; 24+ messages in thread From: Paul B Mahol @ 2022-04-07 7:46 UTC (permalink / raw) To: FFmpeg development discussions and patches On Wed, Apr 6, 2022 at 1:49 PM Paul B Mahol <onemda@gmail.com> wrote: > > > On Tue, Apr 5, 2022 at 8:57 PM Wang Cao <wangcao-at-google.com@ffmpeg.org> > wrote: > >> On Mon, Apr 4, 2022 at 3:28 PM Marton Balint <cus@passwd.hu> wrote: >> >> > >> > >> > On Mon, 4 Apr 2022, Paul B Mahol wrote: >> > >> > > On Sun, Mar 27, 2022 at 11:41 PM Marton Balint <cus@passwd.hu> wrote: >> > > >> > >> >> > >> >> > >> On Sat, 26 Mar 2022, Wang Cao wrote: >> > >> >> > >>> The change in the commit will add some samples to the end of the >> audio >> > >>> stream. The intention is to add a "zero_delay" option eventually to >> not >> > >>> have the delay in the begining the output from alimiter due to >> > >>> lookahead. >> > >> >> > >> I was very much suprised to see that the alimiter filter actually >> delays >> > >> the audio - as in extra samples are inserted in the beginning and >> some >> > >> samples are cut in the end. This trashes A-V sync, so it is a bug >> IMHO. >> > >> >> > >> So unless somebody has some valid usecase for the legacy way of >> > operation >> > >> I'd just simply change it to be "zero delay" without any additional >> user >> > >> option, in a single patch. >> > >> >> > > >> > > >> > > This is done by this patch in very complicated way and also it really >> > > should be optional. >> > >> > But why does it make sense to keep the current (IMHO buggy) operational >> > mode which adds silence in the beginning and trims the end? I understand >> > that the original implementation worked like this, but libavfilter has >> > packet timestamps and N:M filtering so there is absolutely no reason to >> > use an 1:1 implementation and live with its limitations. >> > >> Hello Paul and Marton, thank you so much for taking time to review my >> patch. >> I totally understand that my patch may seem a little bit complicated but I >> can >> show with a FATE test that if we set the alimiter to behave as a >> passthrough filter, >> the output frames will be the same from "framecrc" with my patch. The >> existing >> behavior will not work for all gapless audio processing. >> >> The complete patch to fix this issue is at >> >> https://patchwork.ffmpeg.org/project/ffmpeg/patch/20220330210314.2055201-1-wangcao@google.com/ >> >> Regarding Paul's concern, I personally don't have any preference whether >> to >> put >> the patch as an extra option or not. With respect to the implementation, >> the patch >> is the best I can think of by preserving as much information as possible >> from input >> frames. I also understand it may break concept that "filter_frame" outputs >> one frame >> at a time. For alimiter with my patch, depending on the size of the >> lookahead buffer, >> it may take a few frames before one output frame can be generated. This is >> inevitable >> to compensate for the delay of the lookahead buffer. >> >> Thanks again for reviewing my patch and I'm looking forward to hearing >> from >> you :) >> > > Better than (because its no more 1 frame X nb_samples in, 1 frame X > nb_samples out) to replace .filter_frame/.request_frame with .activate > logic. > > And make this output delay compensation filtering optional. > > In this process make sure that output PTS frame timestamps are unchanged > from input one, by keeping reference of needed frames in filter queue. > > Look how speechnorm/dynaudnorm does it. > Alternatively, use current logic in ladspa filter, (also add option with same name). This would need less code, and probably better approach, as there is no extra latency introduced. Than mapping 1:1 between same number of samples per frame is not hold any more, but I think that is not much important any more. > > >> -- >> >> Wang Cao | Software Engineer | wangcao@google.com | 650-203-7807 >> <(650)%20203-7807> >> _______________________________________________ >> ffmpeg-devel mailing list >> ffmpeg-devel@ffmpeg.org >> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel >> >> To unsubscribe, visit link above, or email >> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". >> > _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [FFmpeg-devel] [PATCH] avfilter/alimiter: Add "flush_buffer" option to flush the remaining valid data to the output 2022-04-07 7:46 ` Paul B Mahol @ 2022-04-07 21:56 ` Wang Cao 2022-04-08 18:42 ` Paul B Mahol 0 siblings, 1 reply; 24+ messages in thread From: Wang Cao @ 2022-04-07 21:56 UTC (permalink / raw) To: FFmpeg development discussions and patches On Thu, Apr 7, 2022 at 12:44 AM Paul B Mahol <onemda@gmail.com> wrote: > On Wed, Apr 6, 2022 at 1:49 PM Paul B Mahol <onemda@gmail.com> wrote: > > > > > > > On Tue, Apr 5, 2022 at 8:57 PM Wang Cao < > wangcao-at-google.com@ffmpeg.org> > > wrote: > > > >> On Mon, Apr 4, 2022 at 3:28 PM Marton Balint <cus@passwd.hu> wrote: > >> > >> > > >> > > >> > On Mon, 4 Apr 2022, Paul B Mahol wrote: > >> > > >> > > On Sun, Mar 27, 2022 at 11:41 PM Marton Balint <cus@passwd.hu> > wrote: > >> > > > >> > >> > >> > >> > >> > >> On Sat, 26 Mar 2022, Wang Cao wrote: > >> > >> > >> > >>> The change in the commit will add some samples to the end of the > >> audio > >> > >>> stream. The intention is to add a "zero_delay" option eventually > to > >> not > >> > >>> have the delay in the begining the output from alimiter due to > >> > >>> lookahead. > >> > >> > >> > >> I was very much suprised to see that the alimiter filter actually > >> delays > >> > >> the audio - as in extra samples are inserted in the beginning and > >> some > >> > >> samples are cut in the end. This trashes A-V sync, so it is a bug > >> IMHO. > >> > >> > >> > >> So unless somebody has some valid usecase for the legacy way of > >> > operation > >> > >> I'd just simply change it to be "zero delay" without any additional > >> user > >> > >> option, in a single patch. > >> > >> > >> > > > >> > > > >> > > This is done by this patch in very complicated way and also it > really > >> > > should be optional. > >> > > >> > But why does it make sense to keep the current (IMHO buggy) > operational > >> > mode which adds silence in the beginning and trims the end? I > understand > >> > that the original implementation worked like this, but libavfilter has > >> > packet timestamps and N:M filtering so there is absolutely no reason > to > >> > use an 1:1 implementation and live with its limitations. > >> > > >> Hello Paul and Marton, thank you so much for taking time to review my > >> patch. > >> I totally understand that my patch may seem a little bit complicated > but I > >> can > >> show with a FATE test that if we set the alimiter to behave as a > >> passthrough filter, > >> the output frames will be the same from "framecrc" with my patch. The > >> existing > >> behavior will not work for all gapless audio processing. > >> > >> The complete patch to fix this issue is at > >> > >> > https://patchwork.ffmpeg.org/project/ffmpeg/patch/20220330210314.2055201-1-wangcao@google.com/ > >> > >> Regarding Paul's concern, I personally don't have any preference whether > >> to > >> put > >> the patch as an extra option or not. With respect to the implementation, > >> the patch > >> is the best I can think of by preserving as much information as possible > >> from input > >> frames. I also understand it may break concept that "filter_frame" > outputs > >> one frame > >> at a time. For alimiter with my patch, depending on the size of the > >> lookahead buffer, > >> it may take a few frames before one output frame can be generated. This > is > >> inevitable > >> to compensate for the delay of the lookahead buffer. > >> > >> Thanks again for reviewing my patch and I'm looking forward to hearing > >> from > >> you :) > >> > > > > Better than (because its no more 1 frame X nb_samples in, 1 frame X > > nb_samples out) to replace .filter_frame/.request_frame with .activate > > logic. > > > > And make this output delay compensation filtering optional. > > > > In this process make sure that output PTS frame timestamps are unchanged > > from input one, by keeping reference of needed frames in filter queue. > > > > Look how speechnorm/dynaudnorm does it. > > > > > Alternatively, use current logic in ladspa filter, (also add option with > same name). > > This would need less code, and probably better approach, as there is no > extra latency introduced. > > Than mapping 1:1 between same number of samples per frame is not hold any > more, but I think that is not much important any more. > Thank you for replying to me with your valuable feedback! I have checked af_ladspa and the "request_frame" function in af_ladspa looks similar to what I'm doing. The difference comes from the fact that I need an internal frame buffer to keep track of output frames. Essentially I add a frame to the internal buffer when an input frame comes in. The frames in this buffer will be the future output frames. We start writing these output frame data buffers only when the internal lookahead buffer has been filled. When there is no more input frames, we flushing all the remaining data in the internal frame buffers and lookahead buffers. Can you advise on my approach here? Thank you! I can put my current implementation of "filter_frame" and "request_frame" into the "activate" approach as you suggested with speechnorm/dynaudnorm. -- Wang Cao | Software Engineer | wangcao@google.com | 650-203-7807 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [FFmpeg-devel] [PATCH] avfilter/alimiter: Add "flush_buffer" option to flush the remaining valid data to the output 2022-04-07 21:56 ` Wang Cao @ 2022-04-08 18:42 ` Paul B Mahol 2022-04-08 20:41 ` Wang Cao 0 siblings, 1 reply; 24+ messages in thread From: Paul B Mahol @ 2022-04-08 18:42 UTC (permalink / raw) To: FFmpeg development discussions and patches On Thu, Apr 7, 2022 at 11:56 PM Wang Cao <wangcao-at-google.com@ffmpeg.org> wrote: > On Thu, Apr 7, 2022 at 12:44 AM Paul B Mahol <onemda@gmail.com> wrote: > > > On Wed, Apr 6, 2022 at 1:49 PM Paul B Mahol <onemda@gmail.com> wrote: > > > > > > > > > > > On Tue, Apr 5, 2022 at 8:57 PM Wang Cao < > > wangcao-at-google.com@ffmpeg.org> > > > wrote: > > > > > >> On Mon, Apr 4, 2022 at 3:28 PM Marton Balint <cus@passwd.hu> wrote: > > >> > > >> > > > >> > > > >> > On Mon, 4 Apr 2022, Paul B Mahol wrote: > > >> > > > >> > > On Sun, Mar 27, 2022 at 11:41 PM Marton Balint <cus@passwd.hu> > > wrote: > > >> > > > > >> > >> > > >> > >> > > >> > >> On Sat, 26 Mar 2022, Wang Cao wrote: > > >> > >> > > >> > >>> The change in the commit will add some samples to the end of the > > >> audio > > >> > >>> stream. The intention is to add a "zero_delay" option eventually > > to > > >> not > > >> > >>> have the delay in the begining the output from alimiter due to > > >> > >>> lookahead. > > >> > >> > > >> > >> I was very much suprised to see that the alimiter filter actually > > >> delays > > >> > >> the audio - as in extra samples are inserted in the beginning and > > >> some > > >> > >> samples are cut in the end. This trashes A-V sync, so it is a bug > > >> IMHO. > > >> > >> > > >> > >> So unless somebody has some valid usecase for the legacy way of > > >> > operation > > >> > >> I'd just simply change it to be "zero delay" without any > additional > > >> user > > >> > >> option, in a single patch. > > >> > >> > > >> > > > > >> > > > > >> > > This is done by this patch in very complicated way and also it > > really > > >> > > should be optional. > > >> > > > >> > But why does it make sense to keep the current (IMHO buggy) > > operational > > >> > mode which adds silence in the beginning and trims the end? I > > understand > > >> > that the original implementation worked like this, but libavfilter > has > > >> > packet timestamps and N:M filtering so there is absolutely no reason > > to > > >> > use an 1:1 implementation and live with its limitations. > > >> > > > >> Hello Paul and Marton, thank you so much for taking time to review my > > >> patch. > > >> I totally understand that my patch may seem a little bit complicated > > but I > > >> can > > >> show with a FATE test that if we set the alimiter to behave as a > > >> passthrough filter, > > >> the output frames will be the same from "framecrc" with my patch. The > > >> existing > > >> behavior will not work for all gapless audio processing. > > >> > > >> The complete patch to fix this issue is at > > >> > > >> > > > https://patchwork.ffmpeg.org/project/ffmpeg/patch/20220330210314.2055201-1-wangcao@google.com/ > > >> > > >> Regarding Paul's concern, I personally don't have any preference > whether > > >> to > > >> put > > >> the patch as an extra option or not. With respect to the > implementation, > > >> the patch > > >> is the best I can think of by preserving as much information as > possible > > >> from input > > >> frames. I also understand it may break concept that "filter_frame" > > outputs > > >> one frame > > >> at a time. For alimiter with my patch, depending on the size of the > > >> lookahead buffer, > > >> it may take a few frames before one output frame can be generated. > This > > is > > >> inevitable > > >> to compensate for the delay of the lookahead buffer. > > >> > > >> Thanks again for reviewing my patch and I'm looking forward to hearing > > >> from > > >> you :) > > >> > > > > > > Better than (because its no more 1 frame X nb_samples in, 1 frame X > > > nb_samples out) to replace .filter_frame/.request_frame with .activate > > > logic. > > > > > > And make this output delay compensation filtering optional. > > > > > > In this process make sure that output PTS frame timestamps are > unchanged > > > from input one, by keeping reference of needed frames in filter queue. > > > > > > Look how speechnorm/dynaudnorm does it. > > > > > > > > > Alternatively, use current logic in ladspa filter, (also add option with > > same name). > > > > This would need less code, and probably better approach, as there is no > > extra latency introduced. > > > > Than mapping 1:1 between same number of samples per frame is not hold any > > more, but I think that is not much important any more. > > > Thank you for replying to me with your valuable feedback! I have checked > af_ladspa > and the "request_frame" function in af_ladspa looks similar to what I'm > doing. The > difference comes from the fact that I need an internal frame buffer to keep > track of > output frames. Essentially I add a frame to the internal buffer when an > input frame > comes in. The frames in this buffer will be the future output frames. We > start writing > these output frame data buffers only when the internal lookahead buffer has > been filled. > When there is no more input frames, we flushing all the remaining data in > the internal > frame buffers and lookahead buffers. Can you advise on my approach here? > Thank you! > > I can put my current implementation of "filter_frame" and "request_frame" > into the "activate" approach as you suggested with speechnorm/dynaudnorm. > No need to wait for all buffers to fill up, only lookahead buffer. Just trim initial samples that is size of lookahead, and than start outputing samples just once you get whatever modulo of current frame number of samples. So there should not be need for extra buffers to keep audio samples. Just buffers to keep input pts and number of samples of previous input frames, like in ladspa filter. > > -- > > Wang Cao | Software Engineer | wangcao@google.com | 650-203-7807 > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". > _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [FFmpeg-devel] [PATCH] avfilter/alimiter: Add "flush_buffer" option to flush the remaining valid data to the output 2022-04-08 18:42 ` Paul B Mahol @ 2022-04-08 20:41 ` Wang Cao 2022-04-09 12:37 ` Paul B Mahol 0 siblings, 1 reply; 24+ messages in thread From: Wang Cao @ 2022-04-08 20:41 UTC (permalink / raw) To: FFmpeg development discussions and patches On Fri, Apr 8, 2022 at 11:40 AM Paul B Mahol <onemda@gmail.com> wrote: > On Thu, Apr 7, 2022 at 11:56 PM Wang Cao <wangcao-at-google.com@ffmpeg.org > > > wrote: > > > On Thu, Apr 7, 2022 at 12:44 AM Paul B Mahol <onemda@gmail.com> wrote: > > > > > On Wed, Apr 6, 2022 at 1:49 PM Paul B Mahol <onemda@gmail.com> wrote: > > > > > > > > > > > > > > > On Tue, Apr 5, 2022 at 8:57 PM Wang Cao < > > > wangcao-at-google.com@ffmpeg.org> > > > > wrote: > > > > > > > >> On Mon, Apr 4, 2022 at 3:28 PM Marton Balint <cus@passwd.hu> wrote: > > > >> > > > >> > > > > >> > > > > >> > On Mon, 4 Apr 2022, Paul B Mahol wrote: > > > >> > > > > >> > > On Sun, Mar 27, 2022 at 11:41 PM Marton Balint <cus@passwd.hu> > > > wrote: > > > >> > > > > > >> > >> > > > >> > >> > > > >> > >> On Sat, 26 Mar 2022, Wang Cao wrote: > > > >> > >> > > > >> > >>> The change in the commit will add some samples to the end of > the > > > >> audio > > > >> > >>> stream. The intention is to add a "zero_delay" option > eventually > > > to > > > >> not > > > >> > >>> have the delay in the begining the output from alimiter due to > > > >> > >>> lookahead. > > > >> > >> > > > >> > >> I was very much suprised to see that the alimiter filter > actually > > > >> delays > > > >> > >> the audio - as in extra samples are inserted in the beginning > and > > > >> some > > > >> > >> samples are cut in the end. This trashes A-V sync, so it is a > bug > > > >> IMHO. > > > >> > >> > > > >> > >> So unless somebody has some valid usecase for the legacy way of > > > >> > operation > > > >> > >> I'd just simply change it to be "zero delay" without any > > additional > > > >> user > > > >> > >> option, in a single patch. > > > >> > >> > > > >> > > > > > >> > > > > > >> > > This is done by this patch in very complicated way and also it > > > really > > > >> > > should be optional. > > > >> > > > > >> > But why does it make sense to keep the current (IMHO buggy) > > > operational > > > >> > mode which adds silence in the beginning and trims the end? I > > > understand > > > >> > that the original implementation worked like this, but libavfilter > > has > > > >> > packet timestamps and N:M filtering so there is absolutely no > reason > > > to > > > >> > use an 1:1 implementation and live with its limitations. > > > >> > > > > >> Hello Paul and Marton, thank you so much for taking time to review > my > > > >> patch. > > > >> I totally understand that my patch may seem a little bit complicated > > > but I > > > >> can > > > >> show with a FATE test that if we set the alimiter to behave as a > > > >> passthrough filter, > > > >> the output frames will be the same from "framecrc" with my patch. > The > > > >> existing > > > >> behavior will not work for all gapless audio processing. > > > >> > > > >> The complete patch to fix this issue is at > > > >> > > > >> > > > > > > https://patchwork.ffmpeg.org/project/ffmpeg/patch/20220330210314.2055201-1-wangcao@google.com/ > > > >> > > > >> Regarding Paul's concern, I personally don't have any preference > > whether > > > >> to > > > >> put > > > >> the patch as an extra option or not. With respect to the > > implementation, > > > >> the patch > > > >> is the best I can think of by preserving as much information as > > possible > > > >> from input > > > >> frames. I also understand it may break concept that "filter_frame" > > > outputs > > > >> one frame > > > >> at a time. For alimiter with my patch, depending on the size of the > > > >> lookahead buffer, > > > >> it may take a few frames before one output frame can be generated. > > This > > > is > > > >> inevitable > > > >> to compensate for the delay of the lookahead buffer. > > > >> > > > >> Thanks again for reviewing my patch and I'm looking forward to > hearing > > > >> from > > > >> you :) > > > >> > > > > > > > > Better than (because its no more 1 frame X nb_samples in, 1 frame X > > > > nb_samples out) to replace .filter_frame/.request_frame with > .activate > > > > logic. > > > > > > > > And make this output delay compensation filtering optional. > > > > > > > > In this process make sure that output PTS frame timestamps are > > unchanged > > > > from input one, by keeping reference of needed frames in filter > queue. > > > > > > > > Look how speechnorm/dynaudnorm does it. > > > > > > > > > > > > > Alternatively, use current logic in ladspa filter, (also add option > with > > > same name). > > > > > > This would need less code, and probably better approach, as there is no > > > extra latency introduced. > > > > > > Than mapping 1:1 between same number of samples per frame is not hold > any > > > more, but I think that is not much important any more. > > > > > Thank you for replying to me with your valuable feedback! I have checked > > af_ladspa > > and the "request_frame" function in af_ladspa looks similar to what I'm > > doing. The > > difference comes from the fact that I need an internal frame buffer to > keep > > track of > > output frames. Essentially I add a frame to the internal buffer when an > > input frame > > comes in. The frames in this buffer will be the future output frames. We > > start writing > > these output frame data buffers only when the internal lookahead buffer > has > > been filled. > > When there is no more input frames, we flushing all the remaining data in > > the internal > > frame buffers and lookahead buffers. Can you advise on my approach here? > > Thank you! > > > > I can put my current implementation of "filter_frame" and "request_frame" > > into the "activate" approach as you suggested with speechnorm/dynaudnorm. > > > > No need to wait for all buffers to fill up, only lookahead buffer. > > Just trim initial samples that is size of lookahead, and than start > outputing samples > just once you get whatever modulo of current frame number of samples. > > So there should not be need for extra buffers to keep audio samples. > Just buffers to keep input pts and number of samples of previous input > frames, like in ladspa filter. > Thank you for the suggestion! From my understanding, we have two ways to achieve "zero_delay" functionality here. Option 1: as you mentioned, we can trim the initial samples of lookahead size. The size of the lookahead buffer can be multiple frames. For example when the attack is 0.08 second, sample rate is 44100 and frame size is 1024, the lookahead buffer size is about 3 frames so the filter needs to see at least 3 input audio frames before it can output one audio frame. We also need to make assumptions about the size of the audio frame (meaning the number of audio samples per frame) when flushing. The frame is probably 1024 conventionally but I think it's better to make less assumption as possible to allow the filter to be used as flexible as possible. Option 2: this is what I proposed before. We basically map the same number of input frames to the output and we also make sure everything about the frame the same as the input except for the audio signal data itself, which will be changed by whatever processing alimiter has to do with that. I think it is safer to make the filter only work on the signal itself. It can help other people who use this filter without worrying about any unexpected change on the frame. The downside is that the filter internally needs to store some empty frames, which will be written as the lookahead buffer is filled. I don't see any performance difference between these two options but option 2 looks better to me because it works solely on the signals without any changes on the frame metadata. -- Wang Cao | Software Engineer | wangcao@google.com | 650-203-7807 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [FFmpeg-devel] [PATCH] avfilter/alimiter: Add "flush_buffer" option to flush the remaining valid data to the output 2022-04-08 20:41 ` Wang Cao @ 2022-04-09 12:37 ` Paul B Mahol 2022-04-11 20:59 ` Wang Cao 0 siblings, 1 reply; 24+ messages in thread From: Paul B Mahol @ 2022-04-09 12:37 UTC (permalink / raw) To: FFmpeg development discussions and patches On Fri, Apr 8, 2022 at 10:41 PM Wang Cao <wangcao-at-google.com@ffmpeg.org> wrote: > On Fri, Apr 8, 2022 at 11:40 AM Paul B Mahol <onemda@gmail.com> wrote: > > > On Thu, Apr 7, 2022 at 11:56 PM Wang Cao < > wangcao-at-google.com@ffmpeg.org > > > > > wrote: > > > > > On Thu, Apr 7, 2022 at 12:44 AM Paul B Mahol <onemda@gmail.com> wrote: > > > > > > > On Wed, Apr 6, 2022 at 1:49 PM Paul B Mahol <onemda@gmail.com> > wrote: > > > > > > > > > > > > > > > > > > > On Tue, Apr 5, 2022 at 8:57 PM Wang Cao < > > > > wangcao-at-google.com@ffmpeg.org> > > > > > wrote: > > > > > > > > > >> On Mon, Apr 4, 2022 at 3:28 PM Marton Balint <cus@passwd.hu> > wrote: > > > > >> > > > > >> > > > > > >> > > > > > >> > On Mon, 4 Apr 2022, Paul B Mahol wrote: > > > > >> > > > > > >> > > On Sun, Mar 27, 2022 at 11:41 PM Marton Balint <cus@passwd.hu > > > > > > wrote: > > > > >> > > > > > > >> > >> > > > > >> > >> > > > > >> > >> On Sat, 26 Mar 2022, Wang Cao wrote: > > > > >> > >> > > > > >> > >>> The change in the commit will add some samples to the end of > > the > > > > >> audio > > > > >> > >>> stream. The intention is to add a "zero_delay" option > > eventually > > > > to > > > > >> not > > > > >> > >>> have the delay in the begining the output from alimiter due > to > > > > >> > >>> lookahead. > > > > >> > >> > > > > >> > >> I was very much suprised to see that the alimiter filter > > actually > > > > >> delays > > > > >> > >> the audio - as in extra samples are inserted in the beginning > > and > > > > >> some > > > > >> > >> samples are cut in the end. This trashes A-V sync, so it is a > > bug > > > > >> IMHO. > > > > >> > >> > > > > >> > >> So unless somebody has some valid usecase for the legacy way > of > > > > >> > operation > > > > >> > >> I'd just simply change it to be "zero delay" without any > > > additional > > > > >> user > > > > >> > >> option, in a single patch. > > > > >> > >> > > > > >> > > > > > > >> > > > > > > >> > > This is done by this patch in very complicated way and also it > > > > really > > > > >> > > should be optional. > > > > >> > > > > > >> > But why does it make sense to keep the current (IMHO buggy) > > > > operational > > > > >> > mode which adds silence in the beginning and trims the end? I > > > > understand > > > > >> > that the original implementation worked like this, but > libavfilter > > > has > > > > >> > packet timestamps and N:M filtering so there is absolutely no > > reason > > > > to > > > > >> > use an 1:1 implementation and live with its limitations. > > > > >> > > > > > >> Hello Paul and Marton, thank you so much for taking time to review > > my > > > > >> patch. > > > > >> I totally understand that my patch may seem a little bit > complicated > > > > but I > > > > >> can > > > > >> show with a FATE test that if we set the alimiter to behave as a > > > > >> passthrough filter, > > > > >> the output frames will be the same from "framecrc" with my patch. > > The > > > > >> existing > > > > >> behavior will not work for all gapless audio processing. > > > > >> > > > > >> The complete patch to fix this issue is at > > > > >> > > > > >> > > > > > > > > > > https://patchwork.ffmpeg.org/project/ffmpeg/patch/20220330210314.2055201-1-wangcao@google.com/ > > > > >> > > > > >> Regarding Paul's concern, I personally don't have any preference > > > whether > > > > >> to > > > > >> put > > > > >> the patch as an extra option or not. With respect to the > > > implementation, > > > > >> the patch > > > > >> is the best I can think of by preserving as much information as > > > possible > > > > >> from input > > > > >> frames. I also understand it may break concept that "filter_frame" > > > > outputs > > > > >> one frame > > > > >> at a time. For alimiter with my patch, depending on the size of > the > > > > >> lookahead buffer, > > > > >> it may take a few frames before one output frame can be generated. > > > This > > > > is > > > > >> inevitable > > > > >> to compensate for the delay of the lookahead buffer. > > > > >> > > > > >> Thanks again for reviewing my patch and I'm looking forward to > > hearing > > > > >> from > > > > >> you :) > > > > >> > > > > > > > > > > Better than (because its no more 1 frame X nb_samples in, 1 frame X > > > > > nb_samples out) to replace .filter_frame/.request_frame with > > .activate > > > > > logic. > > > > > > > > > > And make this output delay compensation filtering optional. > > > > > > > > > > In this process make sure that output PTS frame timestamps are > > > unchanged > > > > > from input one, by keeping reference of needed frames in filter > > queue. > > > > > > > > > > Look how speechnorm/dynaudnorm does it. > > > > > > > > > > > > > > > > > Alternatively, use current logic in ladspa filter, (also add option > > with > > > > same name). > > > > > > > > This would need less code, and probably better approach, as there is > no > > > > extra latency introduced. > > > > > > > > Than mapping 1:1 between same number of samples per frame is not hold > > any > > > > more, but I think that is not much important any more. > > > > > > > Thank you for replying to me with your valuable feedback! I have > checked > > > af_ladspa > > > and the "request_frame" function in af_ladspa looks similar to what I'm > > > doing. The > > > difference comes from the fact that I need an internal frame buffer to > > keep > > > track of > > > output frames. Essentially I add a frame to the internal buffer when an > > > input frame > > > comes in. The frames in this buffer will be the future output frames. > We > > > start writing > > > these output frame data buffers only when the internal lookahead buffer > > has > > > been filled. > > > When there is no more input frames, we flushing all the remaining data > in > > > the internal > > > frame buffers and lookahead buffers. Can you advise on my approach > here? > > > Thank you! > > > > > > I can put my current implementation of "filter_frame" and > "request_frame" > > > into the "activate" approach as you suggested with > speechnorm/dynaudnorm. > > > > > > > No need to wait for all buffers to fill up, only lookahead buffer. > > > > Just trim initial samples that is size of lookahead, and than start > > outputing samples > > just once you get whatever modulo of current frame number of samples. > > > > So there should not be need for extra buffers to keep audio samples. > > Just buffers to keep input pts and number of samples of previous input > > frames, like in ladspa filter. > > > Thank you for the suggestion! From my understanding, we have two ways to > achieve > "zero_delay" functionality here. > > Option 1: as you mentioned, we can trim the initial samples of lookahead > size. > The size of the lookahead buffer can be multiple frames. For example when > the > attack is 0.08 second, sample rate is 44100 and frame size is 1024, the > lookahead > buffer size is about 3 frames so the filter needs to see at least 3 input > audio frames > before it can output one audio frame. We also need to make assumptions > about the > size of the audio frame (meaning the number of audio samples per frame) > when flushing. > The frame is probably 1024 conventionally but I think it's better to make > less assumption > as possible to allow the filter to be used as flexible as possible. > > Option 2: this is what I proposed before. We basically map the same number > of input > frames to the output and we also make sure everything about the frame the > same as > the input except for the audio signal data itself, which will be changed by > whatever > processing alimiter has to do with that. I think it is safer to make the > filter only work on > the signal itself. It can help other people who use this filter without > worrying about > any unexpected change on the frame. The downside is that the filter > internally needs to > store some empty frames, which will be written as the lookahead buffer is > filled. > > I don't see any performance difference between these two options but option > 2 looks > better to me because it works solely on the signals without any changes on > the frame > option 1 does not add extra delay in processing chain at all, and option 2 adds extra delay. Just look at latest version of af_ladspa.c filter code. > metadata. > -- > > Wang Cao | Software Engineer | wangcao@google.com | 650-203-7807 > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". > _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [FFmpeg-devel] [PATCH] avfilter/alimiter: Add "flush_buffer" option to flush the remaining valid data to the output 2022-04-09 12:37 ` Paul B Mahol @ 2022-04-11 20:59 ` Wang Cao 2022-04-12 19:42 ` Paul B Mahol 0 siblings, 1 reply; 24+ messages in thread From: Wang Cao @ 2022-04-11 20:59 UTC (permalink / raw) To: FFmpeg development discussions and patches On Sat, Apr 9, 2022 at 5:35 AM Paul B Mahol <onemda@gmail.com> wrote: > On Fri, Apr 8, 2022 at 10:41 PM Wang Cao <wangcao-at-google.com@ffmpeg.org > > > wrote: > > > On Fri, Apr 8, 2022 at 11:40 AM Paul B Mahol <onemda@gmail.com> wrote: > > > > > On Thu, Apr 7, 2022 at 11:56 PM Wang Cao < > > wangcao-at-google.com@ffmpeg.org > > > > > > > wrote: > > > > > > > On Thu, Apr 7, 2022 at 12:44 AM Paul B Mahol <onemda@gmail.com> > wrote: > > > > > > > > > On Wed, Apr 6, 2022 at 1:49 PM Paul B Mahol <onemda@gmail.com> > > wrote: > > > > > > > > > > > > > > > > > > > > > > > On Tue, Apr 5, 2022 at 8:57 PM Wang Cao < > > > > > wangcao-at-google.com@ffmpeg.org> > > > > > > wrote: > > > > > > > > > > > >> On Mon, Apr 4, 2022 at 3:28 PM Marton Balint <cus@passwd.hu> > > wrote: > > > > > >> > > > > > >> > > > > > > >> > > > > > > >> > On Mon, 4 Apr 2022, Paul B Mahol wrote: > > > > > >> > > > > > > >> > > On Sun, Mar 27, 2022 at 11:41 PM Marton Balint < > cus@passwd.hu > > > > > > > > wrote: > > > > > >> > > > > > > > >> > >> > > > > > >> > >> > > > > > >> > >> On Sat, 26 Mar 2022, Wang Cao wrote: > > > > > >> > >> > > > > > >> > >>> The change in the commit will add some samples to the end > of > > > the > > > > > >> audio > > > > > >> > >>> stream. The intention is to add a "zero_delay" option > > > eventually > > > > > to > > > > > >> not > > > > > >> > >>> have the delay in the begining the output from alimiter > due > > to > > > > > >> > >>> lookahead. > > > > > >> > >> > > > > > >> > >> I was very much suprised to see that the alimiter filter > > > actually > > > > > >> delays > > > > > >> > >> the audio - as in extra samples are inserted in the > beginning > > > and > > > > > >> some > > > > > >> > >> samples are cut in the end. This trashes A-V sync, so it > is a > > > bug > > > > > >> IMHO. > > > > > >> > >> > > > > > >> > >> So unless somebody has some valid usecase for the legacy > way > > of > > > > > >> > operation > > > > > >> > >> I'd just simply change it to be "zero delay" without any > > > > additional > > > > > >> user > > > > > >> > >> option, in a single patch. > > > > > >> > >> > > > > > >> > > > > > > > >> > > > > > > > >> > > This is done by this patch in very complicated way and also > it > > > > > really > > > > > >> > > should be optional. > > > > > >> > > > > > > >> > But why does it make sense to keep the current (IMHO buggy) > > > > > operational > > > > > >> > mode which adds silence in the beginning and trims the end? I > > > > > understand > > > > > >> > that the original implementation worked like this, but > > libavfilter > > > > has > > > > > >> > packet timestamps and N:M filtering so there is absolutely no > > > reason > > > > > to > > > > > >> > use an 1:1 implementation and live with its limitations. > > > > > >> > > > > > > >> Hello Paul and Marton, thank you so much for taking time to > review > > > my > > > > > >> patch. > > > > > >> I totally understand that my patch may seem a little bit > > complicated > > > > > but I > > > > > >> can > > > > > >> show with a FATE test that if we set the alimiter to behave as a > > > > > >> passthrough filter, > > > > > >> the output frames will be the same from "framecrc" with my > patch. > > > The > > > > > >> existing > > > > > >> behavior will not work for all gapless audio processing. > > > > > >> > > > > > >> The complete patch to fix this issue is at > > > > > >> > > > > > >> > > > > > > > > > > > > > > > https://patchwork.ffmpeg.org/project/ffmpeg/patch/20220330210314.2055201-1-wangcao@google.com/ > > > > > >> > > > > > >> Regarding Paul's concern, I personally don't have any preference > > > > whether > > > > > >> to > > > > > >> put > > > > > >> the patch as an extra option or not. With respect to the > > > > implementation, > > > > > >> the patch > > > > > >> is the best I can think of by preserving as much information as > > > > possible > > > > > >> from input > > > > > >> frames. I also understand it may break concept that > "filter_frame" > > > > > outputs > > > > > >> one frame > > > > > >> at a time. For alimiter with my patch, depending on the size of > > the > > > > > >> lookahead buffer, > > > > > >> it may take a few frames before one output frame can be > generated. > > > > This > > > > > is > > > > > >> inevitable > > > > > >> to compensate for the delay of the lookahead buffer. > > > > > >> > > > > > >> Thanks again for reviewing my patch and I'm looking forward to > > > hearing > > > > > >> from > > > > > >> you :) > > > > > >> > > > > > > > > > > > > Better than (because its no more 1 frame X nb_samples in, 1 > frame X > > > > > > nb_samples out) to replace .filter_frame/.request_frame with > > > .activate > > > > > > logic. > > > > > > > > > > > > And make this output delay compensation filtering optional. > > > > > > > > > > > > In this process make sure that output PTS frame timestamps are > > > > unchanged > > > > > > from input one, by keeping reference of needed frames in filter > > > queue. > > > > > > > > > > > > Look how speechnorm/dynaudnorm does it. > > > > > > > > > > > > > > > > > > > > > Alternatively, use current logic in ladspa filter, (also add option > > > with > > > > > same name). > > > > > > > > > > This would need less code, and probably better approach, as there > is > > no > > > > > extra latency introduced. > > > > > > > > > > Than mapping 1:1 between same number of samples per frame is not > hold > > > any > > > > > more, but I think that is not much important any more. > > > > > > > > > Thank you for replying to me with your valuable feedback! I have > > checked > > > > af_ladspa > > > > and the "request_frame" function in af_ladspa looks similar to what > I'm > > > > doing. The > > > > difference comes from the fact that I need an internal frame buffer > to > > > keep > > > > track of > > > > output frames. Essentially I add a frame to the internal buffer when > an > > > > input frame > > > > comes in. The frames in this buffer will be the future output frames. > > We > > > > start writing > > > > these output frame data buffers only when the internal lookahead > buffer > > > has > > > > been filled. > > > > When there is no more input frames, we flushing all the remaining > data > > in > > > > the internal > > > > frame buffers and lookahead buffers. Can you advise on my approach > > here? > > > > Thank you! > > > > > > > > I can put my current implementation of "filter_frame" and > > "request_frame" > > > > into the "activate" approach as you suggested with > > speechnorm/dynaudnorm. > > > > > > > > > > No need to wait for all buffers to fill up, only lookahead buffer. > > > > > > Just trim initial samples that is size of lookahead, and than start > > > outputing samples > > > just once you get whatever modulo of current frame number of samples. > > > > > > So there should not be need for extra buffers to keep audio samples. > > > Just buffers to keep input pts and number of samples of previous input > > > frames, like in ladspa filter. > > > > > Thank you for the suggestion! From my understanding, we have two ways to > > achieve > > "zero_delay" functionality here. > > > > Option 1: as you mentioned, we can trim the initial samples of lookahead > > size. > > The size of the lookahead buffer can be multiple frames. For example when > > the > > attack is 0.08 second, sample rate is 44100 and frame size is 1024, the > > lookahead > > buffer size is about 3 frames so the filter needs to see at least 3 input > > audio frames > > before it can output one audio frame. We also need to make assumptions > > about the > > size of the audio frame (meaning the number of audio samples per frame) > > when flushing. > > The frame is probably 1024 conventionally but I think it's better to make > > less assumption > > as possible to allow the filter to be used as flexible as possible. > > > > Option 2: this is what I proposed before. We basically map the same > number > > of input > > frames to the output and we also make sure everything about the frame the > > same as > > the input except for the audio signal data itself, which will be changed > by > > whatever > > processing alimiter has to do with that. I think it is safer to make the > > filter only work on > > the signal itself. It can help other people who use this filter without > > worrying about > > any unexpected change on the frame. The downside is that the filter > > internally needs to > > store some empty frames, which will be written as the lookahead buffer is > > filled. > > > > I don't see any performance difference between these two options but > option > > 2 looks > > better to me because it works solely on the signals without any changes > on > > the frame > > > > option 1 does not add extra delay in processing chain at all, and option 2 > adds extra delay. > > Just look at latest version of af_ladspa.c filter code. > Hi Paul, I have checked af_ladspa filter. Option 1 certainly doesn't introduce any delay. Option 2 will at most introduce one audio frame of delay in the processing chain. Option 2 needs to fill the lookahead buffer and some frames of samples to push the data in the lookahead buffer out for one output audio frame. The processing delay is probably not a big concern. The reason I proposed option 2 is about the sync of metadata for the output frames: It appears to me that the only metadata we need to worry about is PTS and the number of samples in af_ladspa. However, AVFrame has other fields that also seem to specify some metadata for the frame: 1. AVDictionary *metadata 2. void *opaque (this seems to be client-specific) 3. AVFrameSideData** side_data (this is not handled by av_frame_copy_props). If we go for option 1, it is likely these fields in the input frame will not be mapped to the corresponding output frames. I believe this is also an unexpected behavior for other users who rely on these fields. I understand other filters may not consider this as important but conceptually I believe it is better to make the filter focus on changing what it is supposed to, which is the audio signal itself. Looking forward to hearing your opinion on this, thanks! -- Wang Cao | Software Engineer | wangcao@google.com | 650-203-7807 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [FFmpeg-devel] [PATCH] avfilter/alimiter: Add "flush_buffer" option to flush the remaining valid data to the output 2022-04-11 20:59 ` Wang Cao @ 2022-04-12 19:42 ` Paul B Mahol 2022-04-13 4:11 ` Wang Cao ` (2 more replies) 0 siblings, 3 replies; 24+ messages in thread From: Paul B Mahol @ 2022-04-12 19:42 UTC (permalink / raw) To: FFmpeg development discussions and patches On Mon, Apr 11, 2022 at 10:59 PM Wang Cao <wangcao-at-google.com@ffmpeg.org> wrote: > On Sat, Apr 9, 2022 at 5:35 AM Paul B Mahol <onemda@gmail.com> wrote: > > > On Fri, Apr 8, 2022 at 10:41 PM Wang Cao < > wangcao-at-google.com@ffmpeg.org > > > > > wrote: > > > > > On Fri, Apr 8, 2022 at 11:40 AM Paul B Mahol <onemda@gmail.com> wrote: > > > > > > > On Thu, Apr 7, 2022 at 11:56 PM Wang Cao < > > > wangcao-at-google.com@ffmpeg.org > > > > > > > > > wrote: > > > > > > > > > On Thu, Apr 7, 2022 at 12:44 AM Paul B Mahol <onemda@gmail.com> > > wrote: > > > > > > > > > > > On Wed, Apr 6, 2022 at 1:49 PM Paul B Mahol <onemda@gmail.com> > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > On Tue, Apr 5, 2022 at 8:57 PM Wang Cao < > > > > > > wangcao-at-google.com@ffmpeg.org> > > > > > > > wrote: > > > > > > > > > > > > > >> On Mon, Apr 4, 2022 at 3:28 PM Marton Balint <cus@passwd.hu> > > > wrote: > > > > > > >> > > > > > > >> > > > > > > > >> > > > > > > > >> > On Mon, 4 Apr 2022, Paul B Mahol wrote: > > > > > > >> > > > > > > > >> > > On Sun, Mar 27, 2022 at 11:41 PM Marton Balint < > > cus@passwd.hu > > > > > > > > > > wrote: > > > > > > >> > > > > > > > > >> > >> > > > > > > >> > >> > > > > > > >> > >> On Sat, 26 Mar 2022, Wang Cao wrote: > > > > > > >> > >> > > > > > > >> > >>> The change in the commit will add some samples to the > end > > of > > > > the > > > > > > >> audio > > > > > > >> > >>> stream. The intention is to add a "zero_delay" option > > > > eventually > > > > > > to > > > > > > >> not > > > > > > >> > >>> have the delay in the begining the output from alimiter > > due > > > to > > > > > > >> > >>> lookahead. > > > > > > >> > >> > > > > > > >> > >> I was very much suprised to see that the alimiter filter > > > > actually > > > > > > >> delays > > > > > > >> > >> the audio - as in extra samples are inserted in the > > beginning > > > > and > > > > > > >> some > > > > > > >> > >> samples are cut in the end. This trashes A-V sync, so it > > is a > > > > bug > > > > > > >> IMHO. > > > > > > >> > >> > > > > > > >> > >> So unless somebody has some valid usecase for the legacy > > way > > > of > > > > > > >> > operation > > > > > > >> > >> I'd just simply change it to be "zero delay" without any > > > > > additional > > > > > > >> user > > > > > > >> > >> option, in a single patch. > > > > > > >> > >> > > > > > > >> > > > > > > > > >> > > > > > > > > >> > > This is done by this patch in very complicated way and > also > > it > > > > > > really > > > > > > >> > > should be optional. > > > > > > >> > > > > > > > >> > But why does it make sense to keep the current (IMHO buggy) > > > > > > operational > > > > > > >> > mode which adds silence in the beginning and trims the end? > I > > > > > > understand > > > > > > >> > that the original implementation worked like this, but > > > libavfilter > > > > > has > > > > > > >> > packet timestamps and N:M filtering so there is absolutely > no > > > > reason > > > > > > to > > > > > > >> > use an 1:1 implementation and live with its limitations. > > > > > > >> > > > > > > > >> Hello Paul and Marton, thank you so much for taking time to > > review > > > > my > > > > > > >> patch. > > > > > > >> I totally understand that my patch may seem a little bit > > > complicated > > > > > > but I > > > > > > >> can > > > > > > >> show with a FATE test that if we set the alimiter to behave > as a > > > > > > >> passthrough filter, > > > > > > >> the output frames will be the same from "framecrc" with my > > patch. > > > > The > > > > > > >> existing > > > > > > >> behavior will not work for all gapless audio processing. > > > > > > >> > > > > > > >> The complete patch to fix this issue is at > > > > > > >> > > > > > > >> > > > > > > > > > > > > > > > > > > > > > https://patchwork.ffmpeg.org/project/ffmpeg/patch/20220330210314.2055201-1-wangcao@google.com/ > > > > > > >> > > > > > > >> Regarding Paul's concern, I personally don't have any > preference > > > > > whether > > > > > > >> to > > > > > > >> put > > > > > > >> the patch as an extra option or not. With respect to the > > > > > implementation, > > > > > > >> the patch > > > > > > >> is the best I can think of by preserving as much information > as > > > > > possible > > > > > > >> from input > > > > > > >> frames. I also understand it may break concept that > > "filter_frame" > > > > > > outputs > > > > > > >> one frame > > > > > > >> at a time. For alimiter with my patch, depending on the size > of > > > the > > > > > > >> lookahead buffer, > > > > > > >> it may take a few frames before one output frame can be > > generated. > > > > > This > > > > > > is > > > > > > >> inevitable > > > > > > >> to compensate for the delay of the lookahead buffer. > > > > > > >> > > > > > > >> Thanks again for reviewing my patch and I'm looking forward to > > > > hearing > > > > > > >> from > > > > > > >> you :) > > > > > > >> > > > > > > > > > > > > > > Better than (because its no more 1 frame X nb_samples in, 1 > > frame X > > > > > > > nb_samples out) to replace .filter_frame/.request_frame with > > > > .activate > > > > > > > logic. > > > > > > > > > > > > > > And make this output delay compensation filtering optional. > > > > > > > > > > > > > > In this process make sure that output PTS frame timestamps are > > > > > unchanged > > > > > > > from input one, by keeping reference of needed frames in filter > > > > queue. > > > > > > > > > > > > > > Look how speechnorm/dynaudnorm does it. > > > > > > > > > > > > > > > > > > > > > > > > > Alternatively, use current logic in ladspa filter, (also add > option > > > > with > > > > > > same name). > > > > > > > > > > > > This would need less code, and probably better approach, as there > > is > > > no > > > > > > extra latency introduced. > > > > > > > > > > > > Than mapping 1:1 between same number of samples per frame is not > > hold > > > > any > > > > > > more, but I think that is not much important any more. > > > > > > > > > > > Thank you for replying to me with your valuable feedback! I have > > > checked > > > > > af_ladspa > > > > > and the "request_frame" function in af_ladspa looks similar to what > > I'm > > > > > doing. The > > > > > difference comes from the fact that I need an internal frame buffer > > to > > > > keep > > > > > track of > > > > > output frames. Essentially I add a frame to the internal buffer > when > > an > > > > > input frame > > > > > comes in. The frames in this buffer will be the future output > frames. > > > We > > > > > start writing > > > > > these output frame data buffers only when the internal lookahead > > buffer > > > > has > > > > > been filled. > > > > > When there is no more input frames, we flushing all the remaining > > data > > > in > > > > > the internal > > > > > frame buffers and lookahead buffers. Can you advise on my approach > > > here? > > > > > Thank you! > > > > > > > > > > I can put my current implementation of "filter_frame" and > > > "request_frame" > > > > > into the "activate" approach as you suggested with > > > speechnorm/dynaudnorm. > > > > > > > > > > > > > No need to wait for all buffers to fill up, only lookahead buffer. > > > > > > > > Just trim initial samples that is size of lookahead, and than start > > > > outputing samples > > > > just once you get whatever modulo of current frame number of samples. > > > > > > > > So there should not be need for extra buffers to keep audio samples. > > > > Just buffers to keep input pts and number of samples of previous > input > > > > frames, like in ladspa filter. > > > > > > > Thank you for the suggestion! From my understanding, we have two ways > to > > > achieve > > > "zero_delay" functionality here. > > > > > > Option 1: as you mentioned, we can trim the initial samples of > lookahead > > > size. > > > The size of the lookahead buffer can be multiple frames. For example > when > > > the > > > attack is 0.08 second, sample rate is 44100 and frame size is 1024, the > > > lookahead > > > buffer size is about 3 frames so the filter needs to see at least 3 > input > > > audio frames > > > before it can output one audio frame. We also need to make assumptions > > > about the > > > size of the audio frame (meaning the number of audio samples per frame) > > > when flushing. > > > The frame is probably 1024 conventionally but I think it's better to > make > > > less assumption > > > as possible to allow the filter to be used as flexible as possible. > > > > > > Option 2: this is what I proposed before. We basically map the same > > number > > > of input > > > frames to the output and we also make sure everything about the frame > the > > > same as > > > the input except for the audio signal data itself, which will be > changed > > by > > > whatever > > > processing alimiter has to do with that. I think it is safer to make > the > > > filter only work on > > > the signal itself. It can help other people who use this filter without > > > worrying about > > > any unexpected change on the frame. The downside is that the filter > > > internally needs to > > > store some empty frames, which will be written as the lookahead buffer > is > > > filled. > > > > > > I don't see any performance difference between these two options but > > option > > > 2 looks > > > better to me because it works solely on the signals without any changes > > on > > > the frame > > > > > > > option 1 does not add extra delay in processing chain at all, and option > 2 > > adds extra delay. > > > > Just look at latest version of af_ladspa.c filter code. > > > Hi Paul, I have checked af_ladspa filter. Option 1 certainly doesn't > introduce any delay. > Option 2 will at most introduce one audio frame of delay in the processing > chain. Option > 2 needs to fill the lookahead buffer and some frames of samples to push the > data in the > lookahead buffer out for one output audio frame. The processing delay is > probably not a > big concern. > > The reason I proposed option 2 is about the sync of metadata for the output > frames: > > It appears to me that the only metadata we need to worry about is PTS and > the number of > samples in af_ladspa. However, AVFrame has other fields that also seem to > specify some metadata > for the frame: > 1. AVDictionary *metadata > 2. void *opaque (this seems to be client-specific) > 3. AVFrameSideData** side_data (this is not handled by > av_frame_copy_props). > If we go for option 1, it is likely these fields in the input frame will > not be mapped to the corresponding > output frames. I believe this is also an unexpected behavior for other > users who rely on these fields. I > understand other filters may not consider this as important but > conceptually I believe it is better to make > the filter focus on changing what it is supposed to, which is the audio > signal itself. > > Looking forward to hearing your opinion on this, thanks! > AVFrame Metadata is mostly useful for cases when input frame after filtering with filter that adds metadata no longer changes output frames later in filter graph. Thus using something like astats=metadata=1,alimiter is not very useful and logical. Why by, lightly insisting on no extra delay/latency introduction, and prefer 1, option for alimiter filter is mostly because in audio domain, latency is relevant and important subject for most audio processing specially for limiters. So it is very important to keep it at minimum if possible. > -- > > Wang Cao | Software Engineer | wangcao@google.com | 650-203-7807 > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". > _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [FFmpeg-devel] [PATCH] avfilter/alimiter: Add "flush_buffer" option to flush the remaining valid data to the output 2022-04-12 19:42 ` Paul B Mahol @ 2022-04-13 4:11 ` Wang Cao 2022-04-15 18:49 ` [FFmpeg-devel] [PATCH] avfilter/alimiter: Add an option "comp_delay" that removes the delay introduced by lookahead buffer Wang Cao 2022-04-15 18:51 ` [FFmpeg-devel] [PATCH] avfilter/alimiter: Add "flush_buffer" option to flush the remaining valid data to the output Wang Cao 2 siblings, 0 replies; 24+ messages in thread From: Wang Cao @ 2022-04-13 4:11 UTC (permalink / raw) To: FFmpeg development discussions and patches On Tue, Apr 12, 2022 at 12:40 PM Paul B Mahol <onemda@gmail.com> wrote: > On Mon, Apr 11, 2022 at 10:59 PM Wang Cao < > wangcao-at-google.com@ffmpeg.org> > wrote: > > > On Sat, Apr 9, 2022 at 5:35 AM Paul B Mahol <onemda@gmail.com> wrote: > > > > > On Fri, Apr 8, 2022 at 10:41 PM Wang Cao < > > wangcao-at-google.com@ffmpeg.org > > > > > > > wrote: > > > > > > > On Fri, Apr 8, 2022 at 11:40 AM Paul B Mahol <onemda@gmail.com> > wrote: > > > > > > > > > On Thu, Apr 7, 2022 at 11:56 PM Wang Cao < > > > > wangcao-at-google.com@ffmpeg.org > > > > > > > > > > > wrote: > > > > > > > > > > > On Thu, Apr 7, 2022 at 12:44 AM Paul B Mahol <onemda@gmail.com> > > > wrote: > > > > > > > > > > > > > On Wed, Apr 6, 2022 at 1:49 PM Paul B Mahol <onemda@gmail.com> > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Tue, Apr 5, 2022 at 8:57 PM Wang Cao < > > > > > > > wangcao-at-google.com@ffmpeg.org> > > > > > > > > wrote: > > > > > > > > > > > > > > > >> On Mon, Apr 4, 2022 at 3:28 PM Marton Balint <cus@passwd.hu > > > > > > wrote: > > > > > > > >> > > > > > > > >> > > > > > > > > >> > > > > > > > > >> > On Mon, 4 Apr 2022, Paul B Mahol wrote: > > > > > > > >> > > > > > > > > >> > > On Sun, Mar 27, 2022 at 11:41 PM Marton Balint < > > > cus@passwd.hu > > > > > > > > > > > > wrote: > > > > > > > >> > > > > > > > > > >> > >> > > > > > > > >> > >> > > > > > > > >> > >> On Sat, 26 Mar 2022, Wang Cao wrote: > > > > > > > >> > >> > > > > > > > >> > >>> The change in the commit will add some samples to the > > end > > > of > > > > > the > > > > > > > >> audio > > > > > > > >> > >>> stream. The intention is to add a "zero_delay" option > > > > > eventually > > > > > > > to > > > > > > > >> not > > > > > > > >> > >>> have the delay in the begining the output from > alimiter > > > due > > > > to > > > > > > > >> > >>> lookahead. > > > > > > > >> > >> > > > > > > > >> > >> I was very much suprised to see that the alimiter > filter > > > > > actually > > > > > > > >> delays > > > > > > > >> > >> the audio - as in extra samples are inserted in the > > > beginning > > > > > and > > > > > > > >> some > > > > > > > >> > >> samples are cut in the end. This trashes A-V sync, so > it > > > is a > > > > > bug > > > > > > > >> IMHO. > > > > > > > >> > >> > > > > > > > >> > >> So unless somebody has some valid usecase for the > legacy > > > way > > > > of > > > > > > > >> > operation > > > > > > > >> > >> I'd just simply change it to be "zero delay" without > any > > > > > > additional > > > > > > > >> user > > > > > > > >> > >> option, in a single patch. > > > > > > > >> > >> > > > > > > > >> > > > > > > > > > >> > > > > > > > > > >> > > This is done by this patch in very complicated way and > > also > > > it > > > > > > > really > > > > > > > >> > > should be optional. > > > > > > > >> > > > > > > > > >> > But why does it make sense to keep the current (IMHO > buggy) > > > > > > > operational > > > > > > > >> > mode which adds silence in the beginning and trims the > end? > > I > > > > > > > understand > > > > > > > >> > that the original implementation worked like this, but > > > > libavfilter > > > > > > has > > > > > > > >> > packet timestamps and N:M filtering so there is absolutely > > no > > > > > reason > > > > > > > to > > > > > > > >> > use an 1:1 implementation and live with its limitations. > > > > > > > >> > > > > > > > > >> Hello Paul and Marton, thank you so much for taking time to > > > review > > > > > my > > > > > > > >> patch. > > > > > > > >> I totally understand that my patch may seem a little bit > > > > complicated > > > > > > > but I > > > > > > > >> can > > > > > > > >> show with a FATE test that if we set the alimiter to behave > > as a > > > > > > > >> passthrough filter, > > > > > > > >> the output frames will be the same from "framecrc" with my > > > patch. > > > > > The > > > > > > > >> existing > > > > > > > >> behavior will not work for all gapless audio processing. > > > > > > > >> > > > > > > > >> The complete patch to fix this issue is at > > > > > > > >> > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://patchwork.ffmpeg.org/project/ffmpeg/patch/20220330210314.2055201-1-wangcao@google.com/ > > > > > > > >> > > > > > > > >> Regarding Paul's concern, I personally don't have any > > preference > > > > > > whether > > > > > > > >> to > > > > > > > >> put > > > > > > > >> the patch as an extra option or not. With respect to the > > > > > > implementation, > > > > > > > >> the patch > > > > > > > >> is the best I can think of by preserving as much information > > as > > > > > > possible > > > > > > > >> from input > > > > > > > >> frames. I also understand it may break concept that > > > "filter_frame" > > > > > > > outputs > > > > > > > >> one frame > > > > > > > >> at a time. For alimiter with my patch, depending on the size > > of > > > > the > > > > > > > >> lookahead buffer, > > > > > > > >> it may take a few frames before one output frame can be > > > generated. > > > > > > This > > > > > > > is > > > > > > > >> inevitable > > > > > > > >> to compensate for the delay of the lookahead buffer. > > > > > > > >> > > > > > > > >> Thanks again for reviewing my patch and I'm looking forward > to > > > > > hearing > > > > > > > >> from > > > > > > > >> you :) > > > > > > > >> > > > > > > > > > > > > > > > > Better than (because its no more 1 frame X nb_samples in, 1 > > > frame X > > > > > > > > nb_samples out) to replace .filter_frame/.request_frame with > > > > > .activate > > > > > > > > logic. > > > > > > > > > > > > > > > > And make this output delay compensation filtering optional. > > > > > > > > > > > > > > > > In this process make sure that output PTS frame timestamps > are > > > > > > unchanged > > > > > > > > from input one, by keeping reference of needed frames in > filter > > > > > queue. > > > > > > > > > > > > > > > > Look how speechnorm/dynaudnorm does it. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Alternatively, use current logic in ladspa filter, (also add > > option > > > > > with > > > > > > > same name). > > > > > > > > > > > > > > This would need less code, and probably better approach, as > there > > > is > > > > no > > > > > > > extra latency introduced. > > > > > > > > > > > > > > Than mapping 1:1 between same number of samples per frame is > not > > > hold > > > > > any > > > > > > > more, but I think that is not much important any more. > > > > > > > > > > > > > Thank you for replying to me with your valuable feedback! I have > > > > checked > > > > > > af_ladspa > > > > > > and the "request_frame" function in af_ladspa looks similar to > what > > > I'm > > > > > > doing. The > > > > > > difference comes from the fact that I need an internal frame > buffer > > > to > > > > > keep > > > > > > track of > > > > > > output frames. Essentially I add a frame to the internal buffer > > when > > > an > > > > > > input frame > > > > > > comes in. The frames in this buffer will be the future output > > frames. > > > > We > > > > > > start writing > > > > > > these output frame data buffers only when the internal lookahead > > > buffer > > > > > has > > > > > > been filled. > > > > > > When there is no more input frames, we flushing all the remaining > > > data > > > > in > > > > > > the internal > > > > > > frame buffers and lookahead buffers. Can you advise on my > approach > > > > here? > > > > > > Thank you! > > > > > > > > > > > > I can put my current implementation of "filter_frame" and > > > > "request_frame" > > > > > > into the "activate" approach as you suggested with > > > > speechnorm/dynaudnorm. > > > > > > > > > > > > > > > > No need to wait for all buffers to fill up, only lookahead buffer. > > > > > > > > > > Just trim initial samples that is size of lookahead, and than start > > > > > outputing samples > > > > > just once you get whatever modulo of current frame number of > samples. > > > > > > > > > > So there should not be need for extra buffers to keep audio > samples. > > > > > Just buffers to keep input pts and number of samples of previous > > input > > > > > frames, like in ladspa filter. > > > > > > > > > Thank you for the suggestion! From my understanding, we have two ways > > to > > > > achieve > > > > "zero_delay" functionality here. > > > > > > > > Option 1: as you mentioned, we can trim the initial samples of > > lookahead > > > > size. > > > > The size of the lookahead buffer can be multiple frames. For example > > when > > > > the > > > > attack is 0.08 second, sample rate is 44100 and frame size is 1024, > the > > > > lookahead > > > > buffer size is about 3 frames so the filter needs to see at least 3 > > input > > > > audio frames > > > > before it can output one audio frame. We also need to make > assumptions > > > > about the > > > > size of the audio frame (meaning the number of audio samples per > frame) > > > > when flushing. > > > > The frame is probably 1024 conventionally but I think it's better to > > make > > > > less assumption > > > > as possible to allow the filter to be used as flexible as possible. > > > > > > > > Option 2: this is what I proposed before. We basically map the same > > > number > > > > of input > > > > frames to the output and we also make sure everything about the frame > > the > > > > same as > > > > the input except for the audio signal data itself, which will be > > changed > > > by > > > > whatever > > > > processing alimiter has to do with that. I think it is safer to make > > the > > > > filter only work on > > > > the signal itself. It can help other people who use this filter > without > > > > worrying about > > > > any unexpected change on the frame. The downside is that the filter > > > > internally needs to > > > > store some empty frames, which will be written as the lookahead > buffer > > is > > > > filled. > > > > > > > > I don't see any performance difference between these two options but > > > option > > > > 2 looks > > > > better to me because it works solely on the signals without any > changes > > > on > > > > the frame > > > > > > > > > > option 1 does not add extra delay in processing chain at all, and > option > > 2 > > > adds extra delay. > > > > > > Just look at latest version of af_ladspa.c filter code. > > > > > Hi Paul, I have checked af_ladspa filter. Option 1 certainly doesn't > > introduce any delay. > > Option 2 will at most introduce one audio frame of delay in the > processing > > chain. Option > > 2 needs to fill the lookahead buffer and some frames of samples to push > the > > data in the > > lookahead buffer out for one output audio frame. The processing delay is > > probably not a > > big concern. > > > > The reason I proposed option 2 is about the sync of metadata for the > output > > frames: > > > > It appears to me that the only metadata we need to worry about is PTS and > > the number of > > samples in af_ladspa. However, AVFrame has other fields that also seem to > > specify some metadata > > for the frame: > > 1. AVDictionary *metadata > > 2. void *opaque (this seems to be client-specific) > > 3. AVFrameSideData** side_data (this is not handled by > > av_frame_copy_props). > > If we go for option 1, it is likely these fields in the input frame will > > not be mapped to the corresponding > > output frames. I believe this is also an unexpected behavior for other > > users who rely on these fields. I > > understand other filters may not consider this as important but > > conceptually I believe it is better to make > > the filter focus on changing what it is supposed to, which is the audio > > signal itself. > > > > Looking forward to hearing your opinion on this, thanks! > > > > > AVFrame Metadata is mostly useful for cases when input frame after > filtering > with filter that adds metadata no longer changes output frames later in > filter graph. > Thus using something like astats=metadata=1,alimiter is not very useful and > logical. > > Why by, lightly insisting on no extra delay/latency introduction, and > prefer 1, option for alimiter filter > is mostly because in audio domain, latency is relevant and important > subject for most audio processing specially for limiters. > So it is very important to keep it at minimum if possible. > Thank you so much for introducing me with the context. As I understand the possible metadata can be, I think I have fully understood what I can do. Thanks! -- Wang Cao | Software Engineer | wangcao@google.com | 650-203-7807 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 24+ messages in thread
* [FFmpeg-devel] [PATCH] avfilter/alimiter: Add an option "comp_delay" that removes the delay introduced by lookahead buffer 2022-04-12 19:42 ` Paul B Mahol 2022-04-13 4:11 ` Wang Cao @ 2022-04-15 18:49 ` Wang Cao 2022-04-29 20:48 ` Wang Cao 2022-04-15 18:51 ` [FFmpeg-devel] [PATCH] avfilter/alimiter: Add "flush_buffer" option to flush the remaining valid data to the output Wang Cao 2 siblings, 1 reply; 24+ messages in thread From: Wang Cao @ 2022-04-15 18:49 UTC (permalink / raw) To: ffmpeg-devel; +Cc: Wang Cao 1. The option also flushes all the valid audio samples in the lookahead buffer so the audio integrity is preserved. Previously the the output audio will lose the amount of audio samples equal to the size of lookahead buffer 2. Add a FATE test to verify that when the filter is working as passthrough filter, all audio samples are properly handled from the input to the output. Signed-off-by: Wang Cao <wangcao@google.com> --- doc/filters.texi | 5 +++ libavfilter/af_alimiter.c | 74 +++++++++++++++++++++++++++++++++++++ tests/fate/filter-audio.mak | 12 ++++++ 3 files changed, 91 insertions(+) diff --git a/doc/filters.texi b/doc/filters.texi index a161754233..2af0953c89 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -1978,6 +1978,11 @@ in release time while 1 produces higher release times. @item level Auto level output signal. Default is enabled. This normalizes audio back to 0dB if enabled. + +@item comp_delay +Compensate the delay introduced by using the lookahead buffer set with attack +parameter. Also flush the valid audio data in the lookahead buffer when the +stream hits EOF. @end table Depending on picked setting it is recommended to upsample input 2x or 4x times diff --git a/libavfilter/af_alimiter.c b/libavfilter/af_alimiter.c index 133f98f165..d10a90859b 100644 --- a/libavfilter/af_alimiter.c +++ b/libavfilter/af_alimiter.c @@ -55,6 +55,12 @@ typedef struct AudioLimiterContext { int *nextpos; double *nextdelta; + int lookahead_delay_samples; + int lookahead_flush_samples; + int64_t output_pts; + int64_t next_output_pts; + int comp_delay; + double delta; int nextiter; int nextlen; @@ -73,6 +79,7 @@ static const AVOption alimiter_options[] = { { "asc", "enable asc", OFFSET(auto_release), AV_OPT_TYPE_BOOL, {.i64=0}, 0, 1, AF }, { "asc_level", "set asc level", OFFSET(asc_coeff), AV_OPT_TYPE_DOUBLE, {.dbl=0.5}, 0, 1, AF }, { "level", "auto level", OFFSET(auto_level), AV_OPT_TYPE_BOOL, {.i64=1}, 0, 1, AF }, + { "comp_delay","compensate delay", OFFSET(comp_delay), AV_OPT_TYPE_BOOL, {.i64=0}, 0, 1, AF }, { NULL } }; @@ -129,6 +136,8 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *in) AVFrame *out; double *buf; int n, c, i; + int num_output_samples = in->nb_samples; + int trim_offset; if (av_frame_is_writable(in)) { out = in; @@ -271,10 +280,71 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *in) if (in != out) av_frame_free(&in); + + if (!s->comp_delay) { + return ff_filter_frame(outlink, out); + } + + if (s->output_pts == AV_NOPTS_VALUE) { + s->output_pts = in->pts; + } + + if (s->lookahead_delay_samples > 0) { + // The current output frame is completely silence + if (s->lookahead_delay_samples >= in->nb_samples) { + s->lookahead_delay_samples -= in->nb_samples; + return 0; + } + + // Trim the silence part + trim_offset = av_samples_get_buffer_size( + NULL, inlink->ch_layout.nb_channels, s->lookahead_delay_samples, + (enum AVSampleFormat)out->format, 1); + out->data[0] += trim_offset; + out->nb_samples = in->nb_samples - s->lookahead_delay_samples; + s->lookahead_delay_samples = 0; + num_output_samples = out->nb_samples; + } + + if (s->lookahead_delay_samples < 0) { + return AVERROR_BUG; + } + + out->pts = s->output_pts; + s->next_output_pts = s->output_pts + num_output_samples; + s->output_pts = s->next_output_pts; return ff_filter_frame(outlink, out); } +static int request_frame(AVFilterLink* outlink) +{ + AVFilterContext *ctx = outlink->src; + AudioLimiterContext *s = (AudioLimiterContext*)ctx->priv; + int ret; + AVFilterLink *inlink; + AVFrame *silence_frame; + + ret = ff_request_frame(ctx->inputs[0]); + + if (ret != AVERROR_EOF || s->lookahead_flush_samples == 0 || !s->comp_delay) { + // Not necessarily an error, just not EOF. + return ret; + } + + // We reach here when input filters have finished producing data (i.e. EOF), + // but because of the attack param, s->buffer still has meaningful + // audio content that needs flushing. + inlink = ctx->inputs[0]; + // Pushes silence frame to flush valid audio in the s->buffer + silence_frame = ff_get_audio_buffer(inlink, s->lookahead_flush_samples); + ret = filter_frame(inlink, silence_frame); + if (ret < 0) { + return ret; + } + return AVERROR_EOF; +} + static int config_input(AVFilterLink *inlink) { AVFilterContext *ctx = inlink->dst; @@ -294,6 +364,9 @@ static int config_input(AVFilterLink *inlink) memset(s->nextpos, -1, obuffer_size * sizeof(*s->nextpos)); s->buffer_size = inlink->sample_rate * s->attack * inlink->ch_layout.nb_channels; s->buffer_size -= s->buffer_size % inlink->ch_layout.nb_channels; + // the current logic outputs the next sample from the lookahead buffer from the beginning so the amount of delay + // compensation is less than the lookahead buffer size by 1 . + s->lookahead_delay_samples = s->lookahead_flush_samples = s->buffer_size / inlink->ch_layout.nb_channels - 1; if (s->buffer_size <= 0) { av_log(ctx, AV_LOG_ERROR, "Attack is too small.\n"); @@ -325,6 +398,7 @@ static const AVFilterPad alimiter_outputs[] = { { .name = "default", .type = AVMEDIA_TYPE_AUDIO, + .request_frame = request_frame, }, }; diff --git a/tests/fate/filter-audio.mak b/tests/fate/filter-audio.mak index eff32b9f81..3a51ca18a6 100644 --- a/tests/fate/filter-audio.mak +++ b/tests/fate/filter-audio.mak @@ -63,6 +63,18 @@ fate-filter-agate: tests/data/asynth-44100-2.wav fate-filter-agate: SRC = $(TARGET_PATH)/tests/data/asynth-44100-2.wav fate-filter-agate: CMD = framecrc -i $(SRC) -af aresample,agate=level_in=10:range=0:threshold=1:ratio=1:attack=1:knee=1:makeup=4,aresample +tests/data/filter-alimiter-passthrough: TAG = GEN +tests/data/filter-alimiter-passthrough: ffmpeg$(PROGSSUF)$(EXESUF) | tests/data + $(M)$(TARGET_EXEC) $(TARGET_PATH)/$< -nostdin \ + -i $(TARGET_PATH)/tests/data/asynth-44100-2.wav -af aresample -f crc $(TARGET_PATH)/$@ -y 2>/dev/null + +FATE_AFILTER-$(call FILTERDEMDECENCMUX, AFADE, WAV, PCM_S16LE, PCM_S16LE, WAV) += fate-filter-alimiter-passthrough +fate-filter-alimiter-passthrough: tests/data/asynth-44100-2.wav +fate-filter-alimiter-passthrough: tests/data/filter-alimiter-passthrough +fate-filter-alimiter-passthrough: SRC = $(TARGET_PATH)/tests/data/asynth-44100-2.wav +fate-filter-alimiter-passthrough: REF = $(TARGET_PATH)/tests/data/filter-alimiter-passthrough +fate-filter-alimiter-passthrough: CMD = crc -i $(SRC) -af aresample,alimiter=level_in=1:level_out=1:limit=1:level=0:comp_delay=1,aresample + FATE_AFILTER-$(call FILTERDEMDECENCMUX, AFADE, WAV, PCM_S16LE, PCM_S16LE, WAV) += fate-filter-alimiter fate-filter-alimiter: tests/data/asynth-44100-2.wav fate-filter-alimiter: SRC = $(TARGET_PATH)/tests/data/asynth-44100-2.wav -- 2.36.0.rc0.470.gd361397f0d-goog _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [FFmpeg-devel] [PATCH] avfilter/alimiter: Add an option "comp_delay" that removes the delay introduced by lookahead buffer 2022-04-15 18:49 ` [FFmpeg-devel] [PATCH] avfilter/alimiter: Add an option "comp_delay" that removes the delay introduced by lookahead buffer Wang Cao @ 2022-04-29 20:48 ` Wang Cao 2022-05-02 8:26 ` Paul B Mahol 0 siblings, 1 reply; 24+ messages in thread From: Wang Cao @ 2022-04-29 20:48 UTC (permalink / raw) To: ffmpeg-devel On Fri, Apr 15, 2022 at 11:50 AM Wang Cao <wangcao@google.com> wrote: > 1. The option also flushes all the valid audio samples in the lookahead > buffer so the audio integrity is preserved. Previously the the output > audio will lose the amount of audio samples equal to the size of > lookahead buffer > 2. Add a FATE test to verify that when the filter is working as > passthrough filter, all audio samples are properly handled from the > input to the output. > > Signed-off-by: Wang Cao <wangcao@google.com> > --- > doc/filters.texi | 5 +++ > libavfilter/af_alimiter.c | 74 +++++++++++++++++++++++++++++++++++++ > tests/fate/filter-audio.mak | 12 ++++++ > 3 files changed, 91 insertions(+) > > diff --git a/doc/filters.texi b/doc/filters.texi > index a161754233..2af0953c89 100644 > --- a/doc/filters.texi > +++ b/doc/filters.texi > @@ -1978,6 +1978,11 @@ in release time while 1 produces higher release > times. > @item level > Auto level output signal. Default is enabled. > This normalizes audio back to 0dB if enabled. > + > +@item comp_delay > +Compensate the delay introduced by using the lookahead buffer set with > attack > +parameter. Also flush the valid audio data in the lookahead buffer when > the > +stream hits EOF. > @end table > > Depending on picked setting it is recommended to upsample input 2x or 4x > times > diff --git a/libavfilter/af_alimiter.c b/libavfilter/af_alimiter.c > index 133f98f165..d10a90859b 100644 > --- a/libavfilter/af_alimiter.c > +++ b/libavfilter/af_alimiter.c > @@ -55,6 +55,12 @@ typedef struct AudioLimiterContext { > int *nextpos; > double *nextdelta; > > + int lookahead_delay_samples; > + int lookahead_flush_samples; > + int64_t output_pts; > + int64_t next_output_pts; > + int comp_delay; > + > double delta; > int nextiter; > int nextlen; > @@ -73,6 +79,7 @@ static const AVOption alimiter_options[] = { > { "asc", "enable asc", OFFSET(auto_release), > AV_OPT_TYPE_BOOL, {.i64=0}, 0, 1, AF }, > { "asc_level", "set asc level", OFFSET(asc_coeff), > AV_OPT_TYPE_DOUBLE, {.dbl=0.5}, 0, 1, AF }, > { "level", "auto level", OFFSET(auto_level), > AV_OPT_TYPE_BOOL, {.i64=1}, 0, 1, AF }, > + { "comp_delay","compensate delay", OFFSET(comp_delay), > AV_OPT_TYPE_BOOL, {.i64=0}, 0, 1, AF }, > { NULL } > }; > > @@ -129,6 +136,8 @@ static int filter_frame(AVFilterLink *inlink, AVFrame > *in) > AVFrame *out; > double *buf; > int n, c, i; > + int num_output_samples = in->nb_samples; > + int trim_offset; > > if (av_frame_is_writable(in)) { > out = in; > @@ -271,10 +280,71 @@ static int filter_frame(AVFilterLink *inlink, > AVFrame *in) > > if (in != out) > av_frame_free(&in); > + > + if (!s->comp_delay) { > + return ff_filter_frame(outlink, out); > + } > + > + if (s->output_pts == AV_NOPTS_VALUE) { > + s->output_pts = in->pts; > + } > + > + if (s->lookahead_delay_samples > 0) { > + // The current output frame is completely silence > + if (s->lookahead_delay_samples >= in->nb_samples) { > + s->lookahead_delay_samples -= in->nb_samples; > + return 0; > + } > + > + // Trim the silence part > + trim_offset = av_samples_get_buffer_size( > + NULL, inlink->ch_layout.nb_channels, > s->lookahead_delay_samples, > + (enum AVSampleFormat)out->format, 1); > + out->data[0] += trim_offset; > + out->nb_samples = in->nb_samples - s->lookahead_delay_samples; > + s->lookahead_delay_samples = 0; > + num_output_samples = out->nb_samples; > + } > + > + if (s->lookahead_delay_samples < 0) { > + return AVERROR_BUG; > + } > + > + out->pts = s->output_pts; > + s->next_output_pts = s->output_pts + num_output_samples; > + s->output_pts = s->next_output_pts; > > return ff_filter_frame(outlink, out); > } > > +static int request_frame(AVFilterLink* outlink) > +{ > + AVFilterContext *ctx = outlink->src; > + AudioLimiterContext *s = (AudioLimiterContext*)ctx->priv; > + int ret; > + AVFilterLink *inlink; > + AVFrame *silence_frame; > + > + ret = ff_request_frame(ctx->inputs[0]); > + > + if (ret != AVERROR_EOF || s->lookahead_flush_samples == 0 || > !s->comp_delay) { > + // Not necessarily an error, just not EOF. > + return ret; > + } > + > + // We reach here when input filters have finished producing data > (i.e. EOF), > + // but because of the attack param, s->buffer still has meaningful > + // audio content that needs flushing. > + inlink = ctx->inputs[0]; > + // Pushes silence frame to flush valid audio in the s->buffer > + silence_frame = ff_get_audio_buffer(inlink, > s->lookahead_flush_samples); > + ret = filter_frame(inlink, silence_frame); > + if (ret < 0) { > + return ret; > + } > + return AVERROR_EOF; > +} > + > static int config_input(AVFilterLink *inlink) > { > AVFilterContext *ctx = inlink->dst; > @@ -294,6 +364,9 @@ static int config_input(AVFilterLink *inlink) > memset(s->nextpos, -1, obuffer_size * sizeof(*s->nextpos)); > s->buffer_size = inlink->sample_rate * s->attack * > inlink->ch_layout.nb_channels; > s->buffer_size -= s->buffer_size % inlink->ch_layout.nb_channels; > + // the current logic outputs the next sample from the lookahead > buffer from the beginning so the amount of delay > + // compensation is less than the lookahead buffer size by 1 . > + s->lookahead_delay_samples = s->lookahead_flush_samples = > s->buffer_size / inlink->ch_layout.nb_channels - 1; > > if (s->buffer_size <= 0) { > av_log(ctx, AV_LOG_ERROR, "Attack is too small.\n"); > @@ -325,6 +398,7 @@ static const AVFilterPad alimiter_outputs[] = { > { > .name = "default", > .type = AVMEDIA_TYPE_AUDIO, > + .request_frame = request_frame, > }, > }; > > diff --git a/tests/fate/filter-audio.mak b/tests/fate/filter-audio.mak > index eff32b9f81..3a51ca18a6 100644 > --- a/tests/fate/filter-audio.mak > +++ b/tests/fate/filter-audio.mak > @@ -63,6 +63,18 @@ fate-filter-agate: tests/data/asynth-44100-2.wav > fate-filter-agate: SRC = $(TARGET_PATH)/tests/data/asynth-44100-2.wav > fate-filter-agate: CMD = framecrc -i $(SRC) -af > aresample,agate=level_in=10:range=0:threshold=1:ratio=1:attack=1:knee=1:makeup=4,aresample > > +tests/data/filter-alimiter-passthrough: TAG = GEN > +tests/data/filter-alimiter-passthrough: ffmpeg$(PROGSSUF)$(EXESUF) | > tests/data > + $(M)$(TARGET_EXEC) $(TARGET_PATH)/$< -nostdin \ > + -i $(TARGET_PATH)/tests/data/asynth-44100-2.wav -af aresample -f > crc $(TARGET_PATH)/$@ -y 2>/dev/null > + > +FATE_AFILTER-$(call FILTERDEMDECENCMUX, AFADE, WAV, PCM_S16LE, PCM_S16LE, > WAV) += fate-filter-alimiter-passthrough > +fate-filter-alimiter-passthrough: tests/data/asynth-44100-2.wav > +fate-filter-alimiter-passthrough: tests/data/filter-alimiter-passthrough > +fate-filter-alimiter-passthrough: SRC = > $(TARGET_PATH)/tests/data/asynth-44100-2.wav > +fate-filter-alimiter-passthrough: REF = > $(TARGET_PATH)/tests/data/filter-alimiter-passthrough > +fate-filter-alimiter-passthrough: CMD = crc -i $(SRC) -af > aresample,alimiter=level_in=1:level_out=1:limit=1:level=0:comp_delay=1,aresample > + > FATE_AFILTER-$(call FILTERDEMDECENCMUX, AFADE, WAV, PCM_S16LE, PCM_S16LE, > WAV) += fate-filter-alimiter > fate-filter-alimiter: tests/data/asynth-44100-2.wav > fate-filter-alimiter: SRC = $(TARGET_PATH)/tests/data/asynth-44100-2.wav > -- > 2.36.0.rc0.470.gd361397f0d-goog > > Hello folks, we would really appreciate any feedback on my patch. It looks confusing to me that "FATE" failed on the server while the test I added passed locally. I use "make fate-filter-alimiter-passthrough" to run the test FYI. Thank you! -- Wang Cao | Software Engineer | wangcao@google.com | 650-203-7807 <(650)%20203-7807> _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [FFmpeg-devel] [PATCH] avfilter/alimiter: Add an option "comp_delay" that removes the delay introduced by lookahead buffer 2022-04-29 20:48 ` Wang Cao @ 2022-05-02 8:26 ` Paul B Mahol 2022-05-05 21:14 ` [FFmpeg-devel] [PATCH] avfilter/alimiter:add latency compensation Wang Cao 2022-05-05 21:17 ` [FFmpeg-devel] [PATCH] avfilter/alimiter: Add an option "comp_delay" that removes the delay introduced by lookahead buffer Wang Cao 0 siblings, 2 replies; 24+ messages in thread From: Paul B Mahol @ 2022-05-02 8:26 UTC (permalink / raw) To: FFmpeg development discussions and patches On Fri, Apr 29, 2022 at 10:49 PM Wang Cao <wangcao-at-google.com@ffmpeg.org> wrote: > On Fri, Apr 15, 2022 at 11:50 AM Wang Cao <wangcao@google.com> wrote: > > > 1. The option also flushes all the valid audio samples in the lookahead > > buffer so the audio integrity is preserved. Previously the the output > > audio will lose the amount of audio samples equal to the size of > > lookahead buffer > > 2. Add a FATE test to verify that when the filter is working as > > passthrough filter, all audio samples are properly handled from the > > input to the output. > > > > Signed-off-by: Wang Cao <wangcao@google.com> > > --- > > doc/filters.texi | 5 +++ > > libavfilter/af_alimiter.c | 74 +++++++++++++++++++++++++++++++++++++ > > tests/fate/filter-audio.mak | 12 ++++++ > > 3 files changed, 91 insertions(+) > > > > diff --git a/doc/filters.texi b/doc/filters.texi > > index a161754233..2af0953c89 100644 > > --- a/doc/filters.texi > > +++ b/doc/filters.texi > > @@ -1978,6 +1978,11 @@ in release time while 1 produces higher release > > times. > > @item level > > Auto level output signal. Default is enabled. > > This normalizes audio back to 0dB if enabled. > > + > > +@item comp_delay > > +Compensate the delay introduced by using the lookahead buffer set with > > attack > > +parameter. Also flush the valid audio data in the lookahead buffer when > > the > > +stream hits EOF. > > @end table > > > > Depending on picked setting it is recommended to upsample input 2x or 4x > > times > > diff --git a/libavfilter/af_alimiter.c b/libavfilter/af_alimiter.c > > index 133f98f165..d10a90859b 100644 > > --- a/libavfilter/af_alimiter.c > > +++ b/libavfilter/af_alimiter.c > > @@ -55,6 +55,12 @@ typedef struct AudioLimiterContext { > > int *nextpos; > > double *nextdelta; > > > > + int lookahead_delay_samples; > > + int lookahead_flush_samples; > > + int64_t output_pts; > > + int64_t next_output_pts; > > + int comp_delay; > > + > > double delta; > > int nextiter; > > int nextlen; > > @@ -73,6 +79,7 @@ static const AVOption alimiter_options[] = { > > { "asc", "enable asc", OFFSET(auto_release), > > AV_OPT_TYPE_BOOL, {.i64=0}, 0, 1, AF }, > > { "asc_level", "set asc level", OFFSET(asc_coeff), > > AV_OPT_TYPE_DOUBLE, {.dbl=0.5}, 0, 1, AF }, > > { "level", "auto level", OFFSET(auto_level), > > AV_OPT_TYPE_BOOL, {.i64=1}, 0, 1, AF }, > > + { "comp_delay","compensate delay", OFFSET(comp_delay), > > AV_OPT_TYPE_BOOL, {.i64=0}, 0, 1, AF }, > > { NULL } > > }; > > > > @@ -129,6 +136,8 @@ static int filter_frame(AVFilterLink *inlink, AVFrame > > *in) > > AVFrame *out; > > double *buf; > > int n, c, i; > > + int num_output_samples = in->nb_samples; > > + int trim_offset; > > > > if (av_frame_is_writable(in)) { > > out = in; > > @@ -271,10 +280,71 @@ static int filter_frame(AVFilterLink *inlink, > > AVFrame *in) > > > > if (in != out) > > av_frame_free(&in); > > + > > + if (!s->comp_delay) { > > + return ff_filter_frame(outlink, out); > > + } > > + > > + if (s->output_pts == AV_NOPTS_VALUE) { > > + s->output_pts = in->pts; > > + } > > + > > + if (s->lookahead_delay_samples > 0) { > > + // The current output frame is completely silence > > + if (s->lookahead_delay_samples >= in->nb_samples) { > > + s->lookahead_delay_samples -= in->nb_samples; > > + return 0; > > + } > > + > > + // Trim the silence part > > + trim_offset = av_samples_get_buffer_size( > > + NULL, inlink->ch_layout.nb_channels, > > s->lookahead_delay_samples, > > + (enum AVSampleFormat)out->format, 1); > > + out->data[0] += trim_offset; > > + out->nb_samples = in->nb_samples - s->lookahead_delay_samples; > > + s->lookahead_delay_samples = 0; > > + num_output_samples = out->nb_samples; > > + } > > + > > + if (s->lookahead_delay_samples < 0) { > > + return AVERROR_BUG; > > + } > > + > > + out->pts = s->output_pts; > > + s->next_output_pts = s->output_pts + num_output_samples; > > + s->output_pts = s->next_output_pts; > > > > return ff_filter_frame(outlink, out); > > } > > > > +static int request_frame(AVFilterLink* outlink) > > +{ > > + AVFilterContext *ctx = outlink->src; > > + AudioLimiterContext *s = (AudioLimiterContext*)ctx->priv; > > + int ret; > > + AVFilterLink *inlink; > > + AVFrame *silence_frame; > > + > > + ret = ff_request_frame(ctx->inputs[0]); > > + > > + if (ret != AVERROR_EOF || s->lookahead_flush_samples == 0 || > > !s->comp_delay) { > > + // Not necessarily an error, just not EOF. > > + return ret; > > + } > > + > > + // We reach here when input filters have finished producing data > > (i.e. EOF), > > + // but because of the attack param, s->buffer still has meaningful > > + // audio content that needs flushing. > > + inlink = ctx->inputs[0]; > > + // Pushes silence frame to flush valid audio in the s->buffer > > + silence_frame = ff_get_audio_buffer(inlink, > > s->lookahead_flush_samples); > > + ret = filter_frame(inlink, silence_frame); > > + if (ret < 0) { > > + return ret; > > + } > > + return AVERROR_EOF; > > +} > > + > > static int config_input(AVFilterLink *inlink) > > { > > AVFilterContext *ctx = inlink->dst; > > @@ -294,6 +364,9 @@ static int config_input(AVFilterLink *inlink) > > memset(s->nextpos, -1, obuffer_size * sizeof(*s->nextpos)); > > s->buffer_size = inlink->sample_rate * s->attack * > > inlink->ch_layout.nb_channels; > > s->buffer_size -= s->buffer_size % inlink->ch_layout.nb_channels; > > + // the current logic outputs the next sample from the lookahead > > buffer from the beginning so the amount of delay > > + // compensation is less than the lookahead buffer size by 1 . > > + s->lookahead_delay_samples = s->lookahead_flush_samples = > > s->buffer_size / inlink->ch_layout.nb_channels - 1; > > > > if (s->buffer_size <= 0) { > > av_log(ctx, AV_LOG_ERROR, "Attack is too small.\n"); > > @@ -325,6 +398,7 @@ static const AVFilterPad alimiter_outputs[] = { > > { > > .name = "default", > > .type = AVMEDIA_TYPE_AUDIO, > > + .request_frame = request_frame, > > }, > > }; > > > > diff --git a/tests/fate/filter-audio.mak b/tests/fate/filter-audio.mak > > index eff32b9f81..3a51ca18a6 100644 > > --- a/tests/fate/filter-audio.mak > > +++ b/tests/fate/filter-audio.mak > > @@ -63,6 +63,18 @@ fate-filter-agate: tests/data/asynth-44100-2.wav > > fate-filter-agate: SRC = $(TARGET_PATH)/tests/data/asynth-44100-2.wav > > fate-filter-agate: CMD = framecrc -i $(SRC) -af > > > aresample,agate=level_in=10:range=0:threshold=1:ratio=1:attack=1:knee=1:makeup=4,aresample > > > > +tests/data/filter-alimiter-passthrough: TAG = GEN > > +tests/data/filter-alimiter-passthrough: ffmpeg$(PROGSSUF)$(EXESUF) | > > tests/data > > + $(M)$(TARGET_EXEC) $(TARGET_PATH)/$< -nostdin \ > > + -i $(TARGET_PATH)/tests/data/asynth-44100-2.wav -af aresample -f > > crc $(TARGET_PATH)/$@ -y 2>/dev/null > > + > > +FATE_AFILTER-$(call FILTERDEMDECENCMUX, AFADE, WAV, PCM_S16LE, > PCM_S16LE, > > WAV) += fate-filter-alimiter-passthrough > > +fate-filter-alimiter-passthrough: tests/data/asynth-44100-2.wav > > +fate-filter-alimiter-passthrough: tests/data/filter-alimiter-passthrough > > +fate-filter-alimiter-passthrough: SRC = > > $(TARGET_PATH)/tests/data/asynth-44100-2.wav > > +fate-filter-alimiter-passthrough: REF = > > $(TARGET_PATH)/tests/data/filter-alimiter-passthrough > > +fate-filter-alimiter-passthrough: CMD = crc -i $(SRC) -af > > > aresample,alimiter=level_in=1:level_out=1:limit=1:level=0:comp_delay=1,aresample > > + > > FATE_AFILTER-$(call FILTERDEMDECENCMUX, AFADE, WAV, PCM_S16LE, > PCM_S16LE, > > WAV) += fate-filter-alimiter > > fate-filter-alimiter: tests/data/asynth-44100-2.wav > > fate-filter-alimiter: SRC = $(TARGET_PATH)/tests/data/asynth-44100-2.wav > > -- > > 2.36.0.rc0.470.gd361397f0d-goog > > > > Hello folks, we would really appreciate any feedback on my patch. It > looks > confusing to > me that "FATE" failed on the server while the test I added passed locally. > > I use "make fate-filter-alimiter-passthrough" to run the test FYI. Thank > you! > Still not using logic as in af_ladspa.c Check latest version of FFmpeg source code. Please remove excessive self-explanatory comments in patch. I dunno how FATE test can work if non-trivial filter uses floats. Please follow dev guidelines. And make commit message follow other commits in repo. > -- > > Wang Cao | Software Engineer | wangcao@google.com | 650-203-7807 > <(650)%20203-7807> > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". > _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 24+ messages in thread
* [FFmpeg-devel] [PATCH] avfilter/alimiter:add latency compensation 2022-05-02 8:26 ` Paul B Mahol @ 2022-05-05 21:14 ` Wang Cao 2022-05-10 22:45 ` Wang Cao 2022-05-05 21:17 ` [FFmpeg-devel] [PATCH] avfilter/alimiter: Add an option "comp_delay" that removes the delay introduced by lookahead buffer Wang Cao 1 sibling, 1 reply; 24+ messages in thread From: Wang Cao @ 2022-05-05 21:14 UTC (permalink / raw) To: ffmpeg-devel; +Cc: Wang Cao Also added 2 FATE tests to verify delay is compenated correctly Signed-off-by: Wang Cao <wangcao@google.com> --- doc/filters.texi | 5 +++ libavfilter/af_alimiter.c | 90 +++++++++++++++++++++++++++++++++++++ tests/fate/filter-audio.mak | 24 ++++++++-- 3 files changed, 116 insertions(+), 3 deletions(-) diff --git a/doc/filters.texi b/doc/filters.texi index a161754233..75a43edd88 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -1978,6 +1978,11 @@ in release time while 1 produces higher release times. @item level Auto level output signal. Default is enabled. This normalizes audio back to 0dB if enabled. + +@item latency +Compensate the delay introduced by using the lookahead buffer set with attack +parameter. Also flush the valid audio data in the lookahead buffer when the +stream hits EOF @end table Depending on picked setting it is recommended to upsample input 2x or 4x times diff --git a/libavfilter/af_alimiter.c b/libavfilter/af_alimiter.c index 133f98f165..01265758d7 100644 --- a/libavfilter/af_alimiter.c +++ b/libavfilter/af_alimiter.c @@ -26,6 +26,7 @@ #include "libavutil/channel_layout.h" #include "libavutil/common.h" +#include "libavutil/fifo.h" #include "libavutil/opt.h" #include "audio.h" @@ -33,6 +34,11 @@ #include "formats.h" #include "internal.h" +typedef struct MetaItem { + int64_t pts; + int nb_samples; +} MetaItem; + typedef struct AudioLimiterContext { const AVClass *class; @@ -55,6 +61,14 @@ typedef struct AudioLimiterContext { int *nextpos; double *nextdelta; + int in_trim; + int out_pad; + int64_t next_in_pts; + int64_t next_out_pts; + int latency; + + AVFifo *fifo; + double delta; int nextiter; int nextlen; @@ -73,6 +87,7 @@ static const AVOption alimiter_options[] = { { "asc", "enable asc", OFFSET(auto_release), AV_OPT_TYPE_BOOL, {.i64=0}, 0, 1, AF }, { "asc_level", "set asc level", OFFSET(asc_coeff), AV_OPT_TYPE_DOUBLE, {.dbl=0.5}, 0, 1, AF }, { "level", "auto level", OFFSET(auto_level), AV_OPT_TYPE_BOOL, {.i64=1}, 0, 1, AF }, + { "latency", "compensate delay", OFFSET(latency), AV_OPT_TYPE_BOOL, {.i64=0}, 0, 1, AF }, { NULL } }; @@ -129,6 +144,11 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *in) AVFrame *out; double *buf; int n, c, i; + int new_out_samples; + int64_t out_duration; + int64_t in_duration; + int64_t in_pts; + MetaItem meta; if (av_frame_is_writable(in)) { out = in; @@ -269,12 +289,69 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *in) dst += channels; } + in_duration = av_rescale_q(in->nb_samples, inlink->time_base, av_make_q(1, in->sample_rate)); + in_pts = in->pts; + meta = (MetaItem){ in->pts, in->nb_samples }; + av_fifo_write(s->fifo, &meta, 1); if (in != out) av_frame_free(&in); + new_out_samples = out->nb_samples; + if (s->in_trim > 0) { + int trim = FFMIN(new_out_samples, s->in_trim); + new_out_samples -= trim; + s->in_trim -= trim; + } + + if (new_out_samples <= 0) { + av_frame_free(&out); + return 0; + } else if (new_out_samples < out->nb_samples) { + int offset = out->nb_samples - new_out_samples; + memmove(out->extended_data[0], out->extended_data[0] + sizeof(double) * offset * out->ch_layout.nb_channels, + sizeof(double) * new_out_samples * out->ch_layout.nb_channels); + out->nb_samples = new_out_samples; + s->in_trim = 0; + } + + av_fifo_read(s->fifo, &meta, 1); + + out_duration = av_rescale_q(out->nb_samples, inlink->time_base, av_make_q(1, out->sample_rate)); + in_duration = av_rescale_q(meta.nb_samples, inlink->time_base, av_make_q(1, out->sample_rate)); + in_pts = meta.pts; + + if (s->next_out_pts != AV_NOPTS_VALUE && out->pts != s->next_out_pts && + s->next_in_pts != AV_NOPTS_VALUE && in_pts == s->next_in_pts) { + out->pts = s->next_out_pts; + } else { + out->pts = in_pts; + } + s->next_in_pts = in_pts + in_duration; + s->next_out_pts = out->pts + out_duration; + return ff_filter_frame(outlink, out); } +static int request_frame(AVFilterLink* outlink) +{ + AVFilterContext *ctx = outlink->src; + AudioLimiterContext *s = (AudioLimiterContext*)ctx->priv; + int ret; + + ret = ff_request_frame(ctx->inputs[0]); + + if (ret == AVERROR_EOF && s->out_pad > 0) { + AVFrame *frame = ff_get_audio_buffer(outlink, FFMIN(1024, s->out_pad)); + if (!frame) + return AVERROR(ENOMEM); + + s->out_pad -= frame->nb_samples; + frame->pts = s->next_in_pts; + return filter_frame(ctx->inputs[0], frame); + } + return ret; +} + static int config_input(AVFilterLink *inlink) { AVFilterContext *ctx = inlink->dst; @@ -294,6 +371,16 @@ static int config_input(AVFilterLink *inlink) memset(s->nextpos, -1, obuffer_size * sizeof(*s->nextpos)); s->buffer_size = inlink->sample_rate * s->attack * inlink->ch_layout.nb_channels; s->buffer_size -= s->buffer_size % inlink->ch_layout.nb_channels; + if (s->latency) { + s->in_trim = s->out_pad = s->buffer_size / inlink->ch_layout.nb_channels - 1; + } + s->next_out_pts = AV_NOPTS_VALUE; + s->next_in_pts = AV_NOPTS_VALUE; + + s->fifo = av_fifo_alloc2(8, sizeof(MetaItem), AV_FIFO_FLAG_AUTO_GROW); + if (!s->fifo) { + return AVERROR(ENOMEM); + } if (s->buffer_size <= 0) { av_log(ctx, AV_LOG_ERROR, "Attack is too small.\n"); @@ -310,6 +397,8 @@ static av_cold void uninit(AVFilterContext *ctx) av_freep(&s->buffer); av_freep(&s->nextdelta); av_freep(&s->nextpos); + + av_fifo_freep2(&s->fifo); } static const AVFilterPad alimiter_inputs[] = { @@ -325,6 +414,7 @@ static const AVFilterPad alimiter_outputs[] = { { .name = "default", .type = AVMEDIA_TYPE_AUDIO, + .request_frame = request_frame, }, }; diff --git a/tests/fate/filter-audio.mak b/tests/fate/filter-audio.mak index eff32b9f81..e33ffdf37f 100644 --- a/tests/fate/filter-audio.mak +++ b/tests/fate/filter-audio.mak @@ -63,11 +63,29 @@ fate-filter-agate: tests/data/asynth-44100-2.wav fate-filter-agate: SRC = $(TARGET_PATH)/tests/data/asynth-44100-2.wav fate-filter-agate: CMD = framecrc -i $(SRC) -af aresample,agate=level_in=10:range=0:threshold=1:ratio=1:attack=1:knee=1:makeup=4,aresample -FATE_AFILTER-$(call FILTERDEMDECENCMUX, AFADE, WAV, PCM_S16LE, PCM_S16LE, WAV) += fate-filter-alimiter -fate-filter-alimiter: tests/data/asynth-44100-2.wav -fate-filter-alimiter: SRC = $(TARGET_PATH)/tests/data/asynth-44100-2.wav +tests/data/filter-alimiter-passthrough: TAG = GEN +tests/data/filter-alimiter-passthrough: ffmpeg$(PROGSSUF)$(EXESUF) | tests/data + $(M)$(TARGET_EXEC) $(TARGET_PATH)/$< -nostdin \ + -i $(TARGET_PATH)/tests/data/asynth-44100-2.wav -af aresample -f crc $(TARGET_PATH)/$@ -y 2>/dev/null + +FATE_ALIMITER += fate-filter-alimiter-passthrough-default-attack +fate-filter-alimiter-passthrough-default-attack: tests/data/filter-alimiter-passthrough +fate-filter-alimiter-passthrough-default-attack: REF = $(TARGET_PATH)/tests/data/filter-alimiter-passthrough +fate-filter-alimiter-passthrough-default-attack: CMD = crc -i $(SRC) -af aresample,alimiter=level_in=1:level_out=1:limit=1:level=0:latency=1,aresample + +FATE_ALIMITER += fate-filter-alimiter-passthrough-large-attack +fate-filter-alimiter-passthrough-large-attack: tests/data/filter-alimiter-passthrough +fate-filter-alimiter-passthrough-large-attack: REF = $(TARGET_PATH)/tests/data/filter-alimiter-passthrough +fate-filter-alimiter-passthrough-large-attack: CMD = crc -i $(SRC) -af aresample,alimiter=level_in=1:level_out=1:limit=1:level=0:latency=1:attack=80,aresample + +FATE_ALIMITER += fate-filter-alimiter fate-filter-alimiter: CMD = framecrc -i $(SRC) -af aresample,alimiter=level_in=1:level_out=2:limit=0.2,aresample +$(FATE_ALIMITER): tests/data/asynth-44100-2.wav +$(FATE_ALIMITER): SRC = $(TARGET_PATH)/tests/data/asynth-44100-2.wav + +FATE_AFILTER-$(call FILTERDEMDECENCMUX, ATRIM, WAV, PCM_S16LE, PCM_S16LE, WAV) += $(FATE_ALIMITER) + FATE_AFILTER-$(call FILTERDEMDECENCMUX, AMERGE, WAV, PCM_S16LE, PCM_S16LE, WAV) += fate-filter-amerge fate-filter-amerge: tests/data/asynth-44100-1.wav fate-filter-amerge: SRC = $(TARGET_PATH)/tests/data/asynth-44100-1.wav -- 2.36.0.512.ge40c2bad7a-goog _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [FFmpeg-devel] [PATCH] avfilter/alimiter:add latency compensation 2022-05-05 21:14 ` [FFmpeg-devel] [PATCH] avfilter/alimiter:add latency compensation Wang Cao @ 2022-05-10 22:45 ` Wang Cao 2022-05-12 12:05 ` Paul B Mahol 0 siblings, 1 reply; 24+ messages in thread From: Wang Cao @ 2022-05-10 22:45 UTC (permalink / raw) To: ffmpeg-devel On Thu, May 5, 2022 at 2:14 PM Wang Cao <wangcao@google.com> wrote: > Also added 2 FATE tests to verify delay is compenated correctly > > Signed-off-by: Wang Cao <wangcao@google.com> > --- > doc/filters.texi | 5 +++ > libavfilter/af_alimiter.c | 90 +++++++++++++++++++++++++++++++++++++ > tests/fate/filter-audio.mak | 24 ++++++++-- > 3 files changed, 116 insertions(+), 3 deletions(-) > > diff --git a/doc/filters.texi b/doc/filters.texi > index a161754233..75a43edd88 100644 > --- a/doc/filters.texi > +++ b/doc/filters.texi > @@ -1978,6 +1978,11 @@ in release time while 1 produces higher release > times. > @item level > Auto level output signal. Default is enabled. > This normalizes audio back to 0dB if enabled. > + > +@item latency > +Compensate the delay introduced by using the lookahead buffer set with > attack > +parameter. Also flush the valid audio data in the lookahead buffer when > the > +stream hits EOF > @end table > > Depending on picked setting it is recommended to upsample input 2x or 4x > times > diff --git a/libavfilter/af_alimiter.c b/libavfilter/af_alimiter.c > index 133f98f165..01265758d7 100644 > --- a/libavfilter/af_alimiter.c > +++ b/libavfilter/af_alimiter.c > @@ -26,6 +26,7 @@ > > #include "libavutil/channel_layout.h" > #include "libavutil/common.h" > +#include "libavutil/fifo.h" > #include "libavutil/opt.h" > > #include "audio.h" > @@ -33,6 +34,11 @@ > #include "formats.h" > #include "internal.h" > > +typedef struct MetaItem { > + int64_t pts; > + int nb_samples; > +} MetaItem; > + > typedef struct AudioLimiterContext { > const AVClass *class; > > @@ -55,6 +61,14 @@ typedef struct AudioLimiterContext { > int *nextpos; > double *nextdelta; > > + int in_trim; > + int out_pad; > + int64_t next_in_pts; > + int64_t next_out_pts; > + int latency; > + > + AVFifo *fifo; > + > double delta; > int nextiter; > int nextlen; > @@ -73,6 +87,7 @@ static const AVOption alimiter_options[] = { > { "asc", "enable asc", OFFSET(auto_release), > AV_OPT_TYPE_BOOL, {.i64=0}, 0, 1, AF }, > { "asc_level", "set asc level", OFFSET(asc_coeff), > AV_OPT_TYPE_DOUBLE, {.dbl=0.5}, 0, 1, AF }, > { "level", "auto level", OFFSET(auto_level), > AV_OPT_TYPE_BOOL, {.i64=1}, 0, 1, AF }, > + { "latency", "compensate delay", OFFSET(latency), > AV_OPT_TYPE_BOOL, {.i64=0}, 0, 1, AF }, > { NULL } > }; > > @@ -129,6 +144,11 @@ static int filter_frame(AVFilterLink *inlink, AVFrame > *in) > AVFrame *out; > double *buf; > int n, c, i; > + int new_out_samples; > + int64_t out_duration; > + int64_t in_duration; > + int64_t in_pts; > + MetaItem meta; > > if (av_frame_is_writable(in)) { > out = in; > @@ -269,12 +289,69 @@ static int filter_frame(AVFilterLink *inlink, > AVFrame *in) > dst += channels; > } > > + in_duration = av_rescale_q(in->nb_samples, inlink->time_base, > av_make_q(1, in->sample_rate)); > + in_pts = in->pts; > + meta = (MetaItem){ in->pts, in->nb_samples }; > + av_fifo_write(s->fifo, &meta, 1); > if (in != out) > av_frame_free(&in); > > + new_out_samples = out->nb_samples; > + if (s->in_trim > 0) { > + int trim = FFMIN(new_out_samples, s->in_trim); > + new_out_samples -= trim; > + s->in_trim -= trim; > + } > + > + if (new_out_samples <= 0) { > + av_frame_free(&out); > + return 0; > + } else if (new_out_samples < out->nb_samples) { > + int offset = out->nb_samples - new_out_samples; > + memmove(out->extended_data[0], out->extended_data[0] + > sizeof(double) * offset * out->ch_layout.nb_channels, > + sizeof(double) * new_out_samples * > out->ch_layout.nb_channels); > + out->nb_samples = new_out_samples; > + s->in_trim = 0; > + } > + > + av_fifo_read(s->fifo, &meta, 1); > + > + out_duration = av_rescale_q(out->nb_samples, inlink->time_base, > av_make_q(1, out->sample_rate)); > + in_duration = av_rescale_q(meta.nb_samples, inlink->time_base, > av_make_q(1, out->sample_rate)); > + in_pts = meta.pts; > + > + if (s->next_out_pts != AV_NOPTS_VALUE && out->pts != s->next_out_pts > && > + s->next_in_pts != AV_NOPTS_VALUE && in_pts == s->next_in_pts) { > + out->pts = s->next_out_pts; > + } else { > + out->pts = in_pts; > + } > + s->next_in_pts = in_pts + in_duration; > + s->next_out_pts = out->pts + out_duration; > + > return ff_filter_frame(outlink, out); > } > > +static int request_frame(AVFilterLink* outlink) > +{ > + AVFilterContext *ctx = outlink->src; > + AudioLimiterContext *s = (AudioLimiterContext*)ctx->priv; > + int ret; > + > + ret = ff_request_frame(ctx->inputs[0]); > + > + if (ret == AVERROR_EOF && s->out_pad > 0) { > + AVFrame *frame = ff_get_audio_buffer(outlink, FFMIN(1024, > s->out_pad)); > + if (!frame) > + return AVERROR(ENOMEM); > + > + s->out_pad -= frame->nb_samples; > + frame->pts = s->next_in_pts; > + return filter_frame(ctx->inputs[0], frame); > + } > + return ret; > +} > + > static int config_input(AVFilterLink *inlink) > { > AVFilterContext *ctx = inlink->dst; > @@ -294,6 +371,16 @@ static int config_input(AVFilterLink *inlink) > memset(s->nextpos, -1, obuffer_size * sizeof(*s->nextpos)); > s->buffer_size = inlink->sample_rate * s->attack * > inlink->ch_layout.nb_channels; > s->buffer_size -= s->buffer_size % inlink->ch_layout.nb_channels; > + if (s->latency) { > + s->in_trim = s->out_pad = s->buffer_size / > inlink->ch_layout.nb_channels - 1; > + } > + s->next_out_pts = AV_NOPTS_VALUE; > + s->next_in_pts = AV_NOPTS_VALUE; > + > + s->fifo = av_fifo_alloc2(8, sizeof(MetaItem), AV_FIFO_FLAG_AUTO_GROW); > + if (!s->fifo) { > + return AVERROR(ENOMEM); > + } > > if (s->buffer_size <= 0) { > av_log(ctx, AV_LOG_ERROR, "Attack is too small.\n"); > @@ -310,6 +397,8 @@ static av_cold void uninit(AVFilterContext *ctx) > av_freep(&s->buffer); > av_freep(&s->nextdelta); > av_freep(&s->nextpos); > + > + av_fifo_freep2(&s->fifo); > } > > static const AVFilterPad alimiter_inputs[] = { > @@ -325,6 +414,7 @@ static const AVFilterPad alimiter_outputs[] = { > { > .name = "default", > .type = AVMEDIA_TYPE_AUDIO, > + .request_frame = request_frame, > }, > }; > > diff --git a/tests/fate/filter-audio.mak b/tests/fate/filter-audio.mak > index eff32b9f81..e33ffdf37f 100644 > --- a/tests/fate/filter-audio.mak > +++ b/tests/fate/filter-audio.mak > @@ -63,11 +63,29 @@ fate-filter-agate: tests/data/asynth-44100-2.wav > fate-filter-agate: SRC = $(TARGET_PATH)/tests/data/asynth-44100-2.wav > fate-filter-agate: CMD = framecrc -i $(SRC) -af > aresample,agate=level_in=10:range=0:threshold=1:ratio=1:attack=1:knee=1:makeup=4,aresample > > -FATE_AFILTER-$(call FILTERDEMDECENCMUX, AFADE, WAV, PCM_S16LE, PCM_S16LE, > WAV) += fate-filter-alimiter > -fate-filter-alimiter: tests/data/asynth-44100-2.wav > -fate-filter-alimiter: SRC = $(TARGET_PATH)/tests/data/asynth-44100-2.wav > +tests/data/filter-alimiter-passthrough: TAG = GEN > +tests/data/filter-alimiter-passthrough: ffmpeg$(PROGSSUF)$(EXESUF) | > tests/data > + $(M)$(TARGET_EXEC) $(TARGET_PATH)/$< -nostdin \ > + -i $(TARGET_PATH)/tests/data/asynth-44100-2.wav -af aresample -f > crc $(TARGET_PATH)/$@ -y 2>/dev/null > + > +FATE_ALIMITER += fate-filter-alimiter-passthrough-default-attack > +fate-filter-alimiter-passthrough-default-attack: > tests/data/filter-alimiter-passthrough > +fate-filter-alimiter-passthrough-default-attack: REF = > $(TARGET_PATH)/tests/data/filter-alimiter-passthrough > +fate-filter-alimiter-passthrough-default-attack: CMD = crc -i $(SRC) -af > aresample,alimiter=level_in=1:level_out=1:limit=1:level=0:latency=1,aresample > + > +FATE_ALIMITER += fate-filter-alimiter-passthrough-large-attack > +fate-filter-alimiter-passthrough-large-attack: > tests/data/filter-alimiter-passthrough > +fate-filter-alimiter-passthrough-large-attack: REF = > $(TARGET_PATH)/tests/data/filter-alimiter-passthrough > +fate-filter-alimiter-passthrough-large-attack: CMD = crc -i $(SRC) -af > aresample,alimiter=level_in=1:level_out=1:limit=1:level=0:latency=1:attack=80,aresample > + > +FATE_ALIMITER += fate-filter-alimiter > fate-filter-alimiter: CMD = framecrc -i $(SRC) -af > aresample,alimiter=level_in=1:level_out=2:limit=0.2,aresample > > +$(FATE_ALIMITER): tests/data/asynth-44100-2.wav > +$(FATE_ALIMITER): SRC = $(TARGET_PATH)/tests/data/asynth-44100-2.wav > + > +FATE_AFILTER-$(call FILTERDEMDECENCMUX, ATRIM, WAV, PCM_S16LE, PCM_S16LE, > WAV) += $(FATE_ALIMITER) > + > FATE_AFILTER-$(call FILTERDEMDECENCMUX, AMERGE, WAV, PCM_S16LE, > PCM_S16LE, WAV) += fate-filter-amerge > fate-filter-amerge: tests/data/asynth-44100-1.wav > fate-filter-amerge: SRC = $(TARGET_PATH)/tests/data/asynth-44100-1.wav > -- > 2.36.0.512.ge40c2bad7a-goog > > Hi folks, I don't know how FATE failed on the server but this specific "make fate-filter-alimiter" and "make tests/data/filter-alimiter-passthrough" passed on my local machine. The code is basically referenced from af_ladspa. Can you advice on this implementation? Thanks! -- Wang Cao | Software Engineer | wangcao@google.com | 650-203-7807 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [FFmpeg-devel] [PATCH] avfilter/alimiter:add latency compensation 2022-05-10 22:45 ` Wang Cao @ 2022-05-12 12:05 ` Paul B Mahol 0 siblings, 0 replies; 24+ messages in thread From: Paul B Mahol @ 2022-05-12 12:05 UTC (permalink / raw) To: FFmpeg development discussions and patches On Wed, May 11, 2022 at 12:45 AM Wang Cao <wangcao-at-google.com@ffmpeg.org> wrote: > On Thu, May 5, 2022 at 2:14 PM Wang Cao <wangcao@google.com> wrote: > > > Also added 2 FATE tests to verify delay is compenated correctly > Applied with minor fixes and tests removed for now. > > > > Signed-off-by: Wang Cao <wangcao@google.com> > > --- > > doc/filters.texi | 5 +++ > > libavfilter/af_alimiter.c | 90 +++++++++++++++++++++++++++++++++++++ > > tests/fate/filter-audio.mak | 24 ++++++++-- > > 3 files changed, 116 insertions(+), 3 deletions(-) > > > > diff --git a/doc/filters.texi b/doc/filters.texi > > index a161754233..75a43edd88 100644 > > --- a/doc/filters.texi > > +++ b/doc/filters.texi > > @@ -1978,6 +1978,11 @@ in release time while 1 produces higher release > > times. > > @item level > > Auto level output signal. Default is enabled. > > This normalizes audio back to 0dB if enabled. > > + > > +@item latency > > +Compensate the delay introduced by using the lookahead buffer set with > > attack > > +parameter. Also flush the valid audio data in the lookahead buffer when > > the > > +stream hits EOF > > @end table > > > > Depending on picked setting it is recommended to upsample input 2x or 4x > > times > > diff --git a/libavfilter/af_alimiter.c b/libavfilter/af_alimiter.c > > index 133f98f165..01265758d7 100644 > > --- a/libavfilter/af_alimiter.c > > +++ b/libavfilter/af_alimiter.c > > @@ -26,6 +26,7 @@ > > > > #include "libavutil/channel_layout.h" > > #include "libavutil/common.h" > > +#include "libavutil/fifo.h" > > #include "libavutil/opt.h" > > > > #include "audio.h" > > @@ -33,6 +34,11 @@ > > #include "formats.h" > > #include "internal.h" > > > > +typedef struct MetaItem { > > + int64_t pts; > > + int nb_samples; > > +} MetaItem; > > + > > typedef struct AudioLimiterContext { > > const AVClass *class; > > > > @@ -55,6 +61,14 @@ typedef struct AudioLimiterContext { > > int *nextpos; > > double *nextdelta; > > > > + int in_trim; > > + int out_pad; > > + int64_t next_in_pts; > > + int64_t next_out_pts; > > + int latency; > > + > > + AVFifo *fifo; > > + > > double delta; > > int nextiter; > > int nextlen; > > @@ -73,6 +87,7 @@ static const AVOption alimiter_options[] = { > > { "asc", "enable asc", OFFSET(auto_release), > > AV_OPT_TYPE_BOOL, {.i64=0}, 0, 1, AF }, > > { "asc_level", "set asc level", OFFSET(asc_coeff), > > AV_OPT_TYPE_DOUBLE, {.dbl=0.5}, 0, 1, AF }, > > { "level", "auto level", OFFSET(auto_level), > > AV_OPT_TYPE_BOOL, {.i64=1}, 0, 1, AF }, > > + { "latency", "compensate delay", OFFSET(latency), > > AV_OPT_TYPE_BOOL, {.i64=0}, 0, 1, AF }, > > { NULL } > > }; > > > > @@ -129,6 +144,11 @@ static int filter_frame(AVFilterLink *inlink, > AVFrame > > *in) > > AVFrame *out; > > double *buf; > > int n, c, i; > > + int new_out_samples; > > + int64_t out_duration; > > + int64_t in_duration; > > + int64_t in_pts; > > + MetaItem meta; > > > > if (av_frame_is_writable(in)) { > > out = in; > > @@ -269,12 +289,69 @@ static int filter_frame(AVFilterLink *inlink, > > AVFrame *in) > > dst += channels; > > } > > > > + in_duration = av_rescale_q(in->nb_samples, inlink->time_base, > > av_make_q(1, in->sample_rate)); > > + in_pts = in->pts; > > + meta = (MetaItem){ in->pts, in->nb_samples }; > > + av_fifo_write(s->fifo, &meta, 1); > > if (in != out) > > av_frame_free(&in); > > > > + new_out_samples = out->nb_samples; > > + if (s->in_trim > 0) { > > + int trim = FFMIN(new_out_samples, s->in_trim); > > + new_out_samples -= trim; > > + s->in_trim -= trim; > > + } > > + > > + if (new_out_samples <= 0) { > > + av_frame_free(&out); > > + return 0; > > + } else if (new_out_samples < out->nb_samples) { > > + int offset = out->nb_samples - new_out_samples; > > + memmove(out->extended_data[0], out->extended_data[0] + > > sizeof(double) * offset * out->ch_layout.nb_channels, > > + sizeof(double) * new_out_samples * > > out->ch_layout.nb_channels); > > + out->nb_samples = new_out_samples; > > + s->in_trim = 0; > > + } > > + > > + av_fifo_read(s->fifo, &meta, 1); > > + > > + out_duration = av_rescale_q(out->nb_samples, inlink->time_base, > > av_make_q(1, out->sample_rate)); > > + in_duration = av_rescale_q(meta.nb_samples, inlink->time_base, > > av_make_q(1, out->sample_rate)); > > + in_pts = meta.pts; > > + > > + if (s->next_out_pts != AV_NOPTS_VALUE && out->pts != s->next_out_pts > > && > > + s->next_in_pts != AV_NOPTS_VALUE && in_pts == > s->next_in_pts) { > > + out->pts = s->next_out_pts; > > + } else { > > + out->pts = in_pts; > > + } > > + s->next_in_pts = in_pts + in_duration; > > + s->next_out_pts = out->pts + out_duration; > > + > > return ff_filter_frame(outlink, out); > > } > > > > +static int request_frame(AVFilterLink* outlink) > > +{ > > + AVFilterContext *ctx = outlink->src; > > + AudioLimiterContext *s = (AudioLimiterContext*)ctx->priv; > > + int ret; > > + > > + ret = ff_request_frame(ctx->inputs[0]); > > + > > + if (ret == AVERROR_EOF && s->out_pad > 0) { > > + AVFrame *frame = ff_get_audio_buffer(outlink, FFMIN(1024, > > s->out_pad)); > > + if (!frame) > > + return AVERROR(ENOMEM); > > + > > + s->out_pad -= frame->nb_samples; > > + frame->pts = s->next_in_pts; > > + return filter_frame(ctx->inputs[0], frame); > > + } > > + return ret; > > +} > > + > > static int config_input(AVFilterLink *inlink) > > { > > AVFilterContext *ctx = inlink->dst; > > @@ -294,6 +371,16 @@ static int config_input(AVFilterLink *inlink) > > memset(s->nextpos, -1, obuffer_size * sizeof(*s->nextpos)); > > s->buffer_size = inlink->sample_rate * s->attack * > > inlink->ch_layout.nb_channels; > > s->buffer_size -= s->buffer_size % inlink->ch_layout.nb_channels; > > + if (s->latency) { > > + s->in_trim = s->out_pad = s->buffer_size / > > inlink->ch_layout.nb_channels - 1; > > + } > > + s->next_out_pts = AV_NOPTS_VALUE; > > + s->next_in_pts = AV_NOPTS_VALUE; > > + > > + s->fifo = av_fifo_alloc2(8, sizeof(MetaItem), > AV_FIFO_FLAG_AUTO_GROW); > > + if (!s->fifo) { > > + return AVERROR(ENOMEM); > > + } > > > > if (s->buffer_size <= 0) { > > av_log(ctx, AV_LOG_ERROR, "Attack is too small.\n"); > > @@ -310,6 +397,8 @@ static av_cold void uninit(AVFilterContext *ctx) > > av_freep(&s->buffer); > > av_freep(&s->nextdelta); > > av_freep(&s->nextpos); > > + > > + av_fifo_freep2(&s->fifo); > > } > > > > static const AVFilterPad alimiter_inputs[] = { > > @@ -325,6 +414,7 @@ static const AVFilterPad alimiter_outputs[] = { > > { > > .name = "default", > > .type = AVMEDIA_TYPE_AUDIO, > > + .request_frame = request_frame, > > }, > > }; > > > > diff --git a/tests/fate/filter-audio.mak b/tests/fate/filter-audio.mak > > index eff32b9f81..e33ffdf37f 100644 > > --- a/tests/fate/filter-audio.mak > > +++ b/tests/fate/filter-audio.mak > > @@ -63,11 +63,29 @@ fate-filter-agate: tests/data/asynth-44100-2.wav > > fate-filter-agate: SRC = $(TARGET_PATH)/tests/data/asynth-44100-2.wav > > fate-filter-agate: CMD = framecrc -i $(SRC) -af > > > aresample,agate=level_in=10:range=0:threshold=1:ratio=1:attack=1:knee=1:makeup=4,aresample > > > > -FATE_AFILTER-$(call FILTERDEMDECENCMUX, AFADE, WAV, PCM_S16LE, > PCM_S16LE, > > WAV) += fate-filter-alimiter > > -fate-filter-alimiter: tests/data/asynth-44100-2.wav > > -fate-filter-alimiter: SRC = $(TARGET_PATH)/tests/data/asynth-44100-2.wav > > +tests/data/filter-alimiter-passthrough: TAG = GEN > > +tests/data/filter-alimiter-passthrough: ffmpeg$(PROGSSUF)$(EXESUF) | > > tests/data > > + $(M)$(TARGET_EXEC) $(TARGET_PATH)/$< -nostdin \ > > + -i $(TARGET_PATH)/tests/data/asynth-44100-2.wav -af aresample -f > > crc $(TARGET_PATH)/$@ -y 2>/dev/null > > + > > +FATE_ALIMITER += fate-filter-alimiter-passthrough-default-attack > > +fate-filter-alimiter-passthrough-default-attack: > > tests/data/filter-alimiter-passthrough > > +fate-filter-alimiter-passthrough-default-attack: REF = > > $(TARGET_PATH)/tests/data/filter-alimiter-passthrough > > +fate-filter-alimiter-passthrough-default-attack: CMD = crc -i $(SRC) -af > > > aresample,alimiter=level_in=1:level_out=1:limit=1:level=0:latency=1,aresample > > + > > +FATE_ALIMITER += fate-filter-alimiter-passthrough-large-attack > > +fate-filter-alimiter-passthrough-large-attack: > > tests/data/filter-alimiter-passthrough > > +fate-filter-alimiter-passthrough-large-attack: REF = > > $(TARGET_PATH)/tests/data/filter-alimiter-passthrough > > +fate-filter-alimiter-passthrough-large-attack: CMD = crc -i $(SRC) -af > > > aresample,alimiter=level_in=1:level_out=1:limit=1:level=0:latency=1:attack=80,aresample > > + > > +FATE_ALIMITER += fate-filter-alimiter > > fate-filter-alimiter: CMD = framecrc -i $(SRC) -af > > aresample,alimiter=level_in=1:level_out=2:limit=0.2,aresample > > > > +$(FATE_ALIMITER): tests/data/asynth-44100-2.wav > > +$(FATE_ALIMITER): SRC = $(TARGET_PATH)/tests/data/asynth-44100-2.wav > > + > > +FATE_AFILTER-$(call FILTERDEMDECENCMUX, ATRIM, WAV, PCM_S16LE, > PCM_S16LE, > > WAV) += $(FATE_ALIMITER) > > + > > FATE_AFILTER-$(call FILTERDEMDECENCMUX, AMERGE, WAV, PCM_S16LE, > > PCM_S16LE, WAV) += fate-filter-amerge > > fate-filter-amerge: tests/data/asynth-44100-1.wav > > fate-filter-amerge: SRC = $(TARGET_PATH)/tests/data/asynth-44100-1.wav > > -- > > 2.36.0.512.ge40c2bad7a-goog > > > > Hi folks, I don't know how FATE failed on the server but this specific > "make fate-filter-alimiter" and > "make tests/data/filter-alimiter-passthrough" passed on my local machine. > The code is basically referenced from af_ladspa. Can you advice on this > implementation? Thanks! > -- > > Wang Cao | Software Engineer | wangcao@google.com | 650-203-7807 > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". > _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [FFmpeg-devel] [PATCH] avfilter/alimiter: Add an option "comp_delay" that removes the delay introduced by lookahead buffer 2022-05-02 8:26 ` Paul B Mahol 2022-05-05 21:14 ` [FFmpeg-devel] [PATCH] avfilter/alimiter:add latency compensation Wang Cao @ 2022-05-05 21:17 ` Wang Cao 1 sibling, 0 replies; 24+ messages in thread From: Wang Cao @ 2022-05-05 21:17 UTC (permalink / raw) To: FFmpeg development discussions and patches On Mon, May 2, 2022 at 1:24 AM Paul B Mahol <onemda@gmail.com> wrote: > On Fri, Apr 29, 2022 at 10:49 PM Wang Cao < > wangcao-at-google.com@ffmpeg.org> > wrote: > > > On Fri, Apr 15, 2022 at 11:50 AM Wang Cao <wangcao@google.com> wrote: > > > > > 1. The option also flushes all the valid audio samples in the lookahead > > > buffer so the audio integrity is preserved. Previously the the > output > > > audio will lose the amount of audio samples equal to the size of > > > lookahead buffer > > > 2. Add a FATE test to verify that when the filter is working as > > > passthrough filter, all audio samples are properly handled from the > > > input to the output. > > > > > > Signed-off-by: Wang Cao <wangcao@google.com> > > > --- > > > doc/filters.texi | 5 +++ > > > libavfilter/af_alimiter.c | 74 +++++++++++++++++++++++++++++++++++++ > > > tests/fate/filter-audio.mak | 12 ++++++ > > > 3 files changed, 91 insertions(+) > > > > > > diff --git a/doc/filters.texi b/doc/filters.texi > > > index a161754233..2af0953c89 100644 > > > --- a/doc/filters.texi > > > +++ b/doc/filters.texi > > > @@ -1978,6 +1978,11 @@ in release time while 1 produces higher release > > > times. > > > @item level > > > Auto level output signal. Default is enabled. > > > This normalizes audio back to 0dB if enabled. > > > + > > > +@item comp_delay > > > +Compensate the delay introduced by using the lookahead buffer set with > > > attack > > > +parameter. Also flush the valid audio data in the lookahead buffer > when > > > the > > > +stream hits EOF. > > > @end table > > > > > > Depending on picked setting it is recommended to upsample input 2x or > 4x > > > times > > > diff --git a/libavfilter/af_alimiter.c b/libavfilter/af_alimiter.c > > > index 133f98f165..d10a90859b 100644 > > > --- a/libavfilter/af_alimiter.c > > > +++ b/libavfilter/af_alimiter.c > > > @@ -55,6 +55,12 @@ typedef struct AudioLimiterContext { > > > int *nextpos; > > > double *nextdelta; > > > > > > + int lookahead_delay_samples; > > > + int lookahead_flush_samples; > > > + int64_t output_pts; > > > + int64_t next_output_pts; > > > + int comp_delay; > > > + > > > double delta; > > > int nextiter; > > > int nextlen; > > > @@ -73,6 +79,7 @@ static const AVOption alimiter_options[] = { > > > { "asc", "enable asc", OFFSET(auto_release), > > > AV_OPT_TYPE_BOOL, {.i64=0}, 0, 1, AF }, > > > { "asc_level", "set asc level", OFFSET(asc_coeff), > > > AV_OPT_TYPE_DOUBLE, {.dbl=0.5}, 0, 1, AF }, > > > { "level", "auto level", OFFSET(auto_level), > > > AV_OPT_TYPE_BOOL, {.i64=1}, 0, 1, AF }, > > > + { "comp_delay","compensate delay", OFFSET(comp_delay), > > > AV_OPT_TYPE_BOOL, {.i64=0}, 0, 1, AF }, > > > { NULL } > > > }; > > > > > > @@ -129,6 +136,8 @@ static int filter_frame(AVFilterLink *inlink, > AVFrame > > > *in) > > > AVFrame *out; > > > double *buf; > > > int n, c, i; > > > + int num_output_samples = in->nb_samples; > > > + int trim_offset; > > > > > > if (av_frame_is_writable(in)) { > > > out = in; > > > @@ -271,10 +280,71 @@ static int filter_frame(AVFilterLink *inlink, > > > AVFrame *in) > > > > > > if (in != out) > > > av_frame_free(&in); > > > + > > > + if (!s->comp_delay) { > > > + return ff_filter_frame(outlink, out); > > > + } > > > + > > > + if (s->output_pts == AV_NOPTS_VALUE) { > > > + s->output_pts = in->pts; > > > + } > > > + > > > + if (s->lookahead_delay_samples > 0) { > > > + // The current output frame is completely silence > > > + if (s->lookahead_delay_samples >= in->nb_samples) { > > > + s->lookahead_delay_samples -= in->nb_samples; > > > + return 0; > > > + } > > > + > > > + // Trim the silence part > > > + trim_offset = av_samples_get_buffer_size( > > > + NULL, inlink->ch_layout.nb_channels, > > > s->lookahead_delay_samples, > > > + (enum AVSampleFormat)out->format, 1); > > > + out->data[0] += trim_offset; > > > + out->nb_samples = in->nb_samples - s->lookahead_delay_samples; > > > + s->lookahead_delay_samples = 0; > > > + num_output_samples = out->nb_samples; > > > + } > > > + > > > + if (s->lookahead_delay_samples < 0) { > > > + return AVERROR_BUG; > > > + } > > > + > > > + out->pts = s->output_pts; > > > + s->next_output_pts = s->output_pts + num_output_samples; > > > + s->output_pts = s->next_output_pts; > > > > > > return ff_filter_frame(outlink, out); > > > } > > > > > > +static int request_frame(AVFilterLink* outlink) > > > +{ > > > + AVFilterContext *ctx = outlink->src; > > > + AudioLimiterContext *s = (AudioLimiterContext*)ctx->priv; > > > + int ret; > > > + AVFilterLink *inlink; > > > + AVFrame *silence_frame; > > > + > > > + ret = ff_request_frame(ctx->inputs[0]); > > > + > > > + if (ret != AVERROR_EOF || s->lookahead_flush_samples == 0 || > > > !s->comp_delay) { > > > + // Not necessarily an error, just not EOF. > > > + return ret; > > > + } > > > + > > > + // We reach here when input filters have finished producing data > > > (i.e. EOF), > > > + // but because of the attack param, s->buffer still has meaningful > > > + // audio content that needs flushing. > > > + inlink = ctx->inputs[0]; > > > + // Pushes silence frame to flush valid audio in the s->buffer > > > + silence_frame = ff_get_audio_buffer(inlink, > > > s->lookahead_flush_samples); > > > + ret = filter_frame(inlink, silence_frame); > > > + if (ret < 0) { > > > + return ret; > > > + } > > > + return AVERROR_EOF; > > > +} > > > + > > > static int config_input(AVFilterLink *inlink) > > > { > > > AVFilterContext *ctx = inlink->dst; > > > @@ -294,6 +364,9 @@ static int config_input(AVFilterLink *inlink) > > > memset(s->nextpos, -1, obuffer_size * sizeof(*s->nextpos)); > > > s->buffer_size = inlink->sample_rate * s->attack * > > > inlink->ch_layout.nb_channels; > > > s->buffer_size -= s->buffer_size % inlink->ch_layout.nb_channels; > > > + // the current logic outputs the next sample from the lookahead > > > buffer from the beginning so the amount of delay > > > + // compensation is less than the lookahead buffer size by 1 . > > > + s->lookahead_delay_samples = s->lookahead_flush_samples = > > > s->buffer_size / inlink->ch_layout.nb_channels - 1; > > > > > > if (s->buffer_size <= 0) { > > > av_log(ctx, AV_LOG_ERROR, "Attack is too small.\n"); > > > @@ -325,6 +398,7 @@ static const AVFilterPad alimiter_outputs[] = { > > > { > > > .name = "default", > > > .type = AVMEDIA_TYPE_AUDIO, > > > + .request_frame = request_frame, > > > }, > > > }; > > > > > > diff --git a/tests/fate/filter-audio.mak b/tests/fate/filter-audio.mak > > > index eff32b9f81..3a51ca18a6 100644 > > > --- a/tests/fate/filter-audio.mak > > > +++ b/tests/fate/filter-audio.mak > > > @@ -63,6 +63,18 @@ fate-filter-agate: tests/data/asynth-44100-2.wav > > > fate-filter-agate: SRC = $(TARGET_PATH)/tests/data/asynth-44100-2.wav > > > fate-filter-agate: CMD = framecrc -i $(SRC) -af > > > > > > aresample,agate=level_in=10:range=0:threshold=1:ratio=1:attack=1:knee=1:makeup=4,aresample > > > > > > +tests/data/filter-alimiter-passthrough: TAG = GEN > > > +tests/data/filter-alimiter-passthrough: ffmpeg$(PROGSSUF)$(EXESUF) | > > > tests/data > > > + $(M)$(TARGET_EXEC) $(TARGET_PATH)/$< -nostdin \ > > > + -i $(TARGET_PATH)/tests/data/asynth-44100-2.wav -af aresample > -f > > > crc $(TARGET_PATH)/$@ -y 2>/dev/null > > > + > > > +FATE_AFILTER-$(call FILTERDEMDECENCMUX, AFADE, WAV, PCM_S16LE, > > PCM_S16LE, > > > WAV) += fate-filter-alimiter-passthrough > > > +fate-filter-alimiter-passthrough: tests/data/asynth-44100-2.wav > > > +fate-filter-alimiter-passthrough: > tests/data/filter-alimiter-passthrough > > > +fate-filter-alimiter-passthrough: SRC = > > > $(TARGET_PATH)/tests/data/asynth-44100-2.wav > > > +fate-filter-alimiter-passthrough: REF = > > > $(TARGET_PATH)/tests/data/filter-alimiter-passthrough > > > +fate-filter-alimiter-passthrough: CMD = crc -i $(SRC) -af > > > > > > aresample,alimiter=level_in=1:level_out=1:limit=1:level=0:comp_delay=1,aresample > > > + > > > FATE_AFILTER-$(call FILTERDEMDECENCMUX, AFADE, WAV, PCM_S16LE, > > PCM_S16LE, > > > WAV) += fate-filter-alimiter > > > fate-filter-alimiter: tests/data/asynth-44100-2.wav > > > fate-filter-alimiter: SRC = > $(TARGET_PATH)/tests/data/asynth-44100-2.wav > > > -- > > > 2.36.0.rc0.470.gd361397f0d-goog > > > > > > Hello folks, we would really appreciate any feedback on my patch. It > > looks > > confusing to > > me that "FATE" failed on the server while the test I added passed > locally. > > > > I use "make fate-filter-alimiter-passthrough" to run the test FYI. Thank > > you! > > > > Still not using logic as in af_ladspa.c > > Check latest version of FFmpeg source code. > > Please remove excessive self-explanatory comments in patch. > > I dunno how FATE test can work if non-trivial filter uses floats. > > Please follow dev guidelines. And make commit message follow other commits > in repo. > Hi Paul, I have prepared the patch according to your suggestion. During the development, I have also found the issue in af_ladspa, which caused the latency compensation not to work as intended. It's also fixed in this patch. FATE tests pass on my local development too. Please take a look, thank you! -- Wang Cao | Software Engineer | wangcao@google.com | 650-203-7807 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [FFmpeg-devel] [PATCH] avfilter/alimiter: Add "flush_buffer" option to flush the remaining valid data to the output 2022-04-12 19:42 ` Paul B Mahol 2022-04-13 4:11 ` Wang Cao 2022-04-15 18:49 ` [FFmpeg-devel] [PATCH] avfilter/alimiter: Add an option "comp_delay" that removes the delay introduced by lookahead buffer Wang Cao @ 2022-04-15 18:51 ` Wang Cao 2 siblings, 0 replies; 24+ messages in thread From: Wang Cao @ 2022-04-15 18:51 UTC (permalink / raw) To: FFmpeg development discussions and patches On Tue, Apr 12, 2022 at 12:40 PM Paul B Mahol <onemda@gmail.com> wrote: > On Mon, Apr 11, 2022 at 10:59 PM Wang Cao < > wangcao-at-google.com@ffmpeg.org> > wrote: > > > On Sat, Apr 9, 2022 at 5:35 AM Paul B Mahol <onemda@gmail.com> wrote: > > > > > On Fri, Apr 8, 2022 at 10:41 PM Wang Cao < > > wangcao-at-google.com@ffmpeg.org > > > > > > > wrote: > > > > > > > On Fri, Apr 8, 2022 at 11:40 AM Paul B Mahol <onemda@gmail.com> > wrote: > > > > > > > > > On Thu, Apr 7, 2022 at 11:56 PM Wang Cao < > > > > wangcao-at-google.com@ffmpeg.org > > > > > > > > > > > wrote: > > > > > > > > > > > On Thu, Apr 7, 2022 at 12:44 AM Paul B Mahol <onemda@gmail.com> > > > wrote: > > > > > > > > > > > > > On Wed, Apr 6, 2022 at 1:49 PM Paul B Mahol <onemda@gmail.com> > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Tue, Apr 5, 2022 at 8:57 PM Wang Cao < > > > > > > > wangcao-at-google.com@ffmpeg.org> > > > > > > > > wrote: > > > > > > > > > > > > > > > >> On Mon, Apr 4, 2022 at 3:28 PM Marton Balint <cus@passwd.hu > > > > > > wrote: > > > > > > > >> > > > > > > > >> > > > > > > > > >> > > > > > > > > >> > On Mon, 4 Apr 2022, Paul B Mahol wrote: > > > > > > > >> > > > > > > > > >> > > On Sun, Mar 27, 2022 at 11:41 PM Marton Balint < > > > cus@passwd.hu > > > > > > > > > > > > wrote: > > > > > > > >> > > > > > > > > > >> > >> > > > > > > > >> > >> > > > > > > > >> > >> On Sat, 26 Mar 2022, Wang Cao wrote: > > > > > > > >> > >> > > > > > > > >> > >>> The change in the commit will add some samples to the > > end > > > of > > > > > the > > > > > > > >> audio > > > > > > > >> > >>> stream. The intention is to add a "zero_delay" option > > > > > eventually > > > > > > > to > > > > > > > >> not > > > > > > > >> > >>> have the delay in the begining the output from > alimiter > > > due > > > > to > > > > > > > >> > >>> lookahead. > > > > > > > >> > >> > > > > > > > >> > >> I was very much suprised to see that the alimiter > filter > > > > > actually > > > > > > > >> delays > > > > > > > >> > >> the audio - as in extra samples are inserted in the > > > beginning > > > > > and > > > > > > > >> some > > > > > > > >> > >> samples are cut in the end. This trashes A-V sync, so > it > > > is a > > > > > bug > > > > > > > >> IMHO. > > > > > > > >> > >> > > > > > > > >> > >> So unless somebody has some valid usecase for the > legacy > > > way > > > > of > > > > > > > >> > operation > > > > > > > >> > >> I'd just simply change it to be "zero delay" without > any > > > > > > additional > > > > > > > >> user > > > > > > > >> > >> option, in a single patch. > > > > > > > >> > >> > > > > > > > >> > > > > > > > > > >> > > > > > > > > > >> > > This is done by this patch in very complicated way and > > also > > > it > > > > > > > really > > > > > > > >> > > should be optional. > > > > > > > >> > > > > > > > > >> > But why does it make sense to keep the current (IMHO > buggy) > > > > > > > operational > > > > > > > >> > mode which adds silence in the beginning and trims the > end? > > I > > > > > > > understand > > > > > > > >> > that the original implementation worked like this, but > > > > libavfilter > > > > > > has > > > > > > > >> > packet timestamps and N:M filtering so there is absolutely > > no > > > > > reason > > > > > > > to > > > > > > > >> > use an 1:1 implementation and live with its limitations. > > > > > > > >> > > > > > > > > >> Hello Paul and Marton, thank you so much for taking time to > > > review > > > > > my > > > > > > > >> patch. > > > > > > > >> I totally understand that my patch may seem a little bit > > > > complicated > > > > > > > but I > > > > > > > >> can > > > > > > > >> show with a FATE test that if we set the alimiter to behave > > as a > > > > > > > >> passthrough filter, > > > > > > > >> the output frames will be the same from "framecrc" with my > > > patch. > > > > > The > > > > > > > >> existing > > > > > > > >> behavior will not work for all gapless audio processing. > > > > > > > >> > > > > > > > >> The complete patch to fix this issue is at > > > > > > > >> > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://patchwork.ffmpeg.org/project/ffmpeg/patch/20220330210314.2055201-1-wangcao@google.com/ > > > > > > > >> > > > > > > > >> Regarding Paul's concern, I personally don't have any > > preference > > > > > > whether > > > > > > > >> to > > > > > > > >> put > > > > > > > >> the patch as an extra option or not. With respect to the > > > > > > implementation, > > > > > > > >> the patch > > > > > > > >> is the best I can think of by preserving as much information > > as > > > > > > possible > > > > > > > >> from input > > > > > > > >> frames. I also understand it may break concept that > > > "filter_frame" > > > > > > > outputs > > > > > > > >> one frame > > > > > > > >> at a time. For alimiter with my patch, depending on the size > > of > > > > the > > > > > > > >> lookahead buffer, > > > > > > > >> it may take a few frames before one output frame can be > > > generated. > > > > > > This > > > > > > > is > > > > > > > >> inevitable > > > > > > > >> to compensate for the delay of the lookahead buffer. > > > > > > > >> > > > > > > > >> Thanks again for reviewing my patch and I'm looking forward > to > > > > > hearing > > > > > > > >> from > > > > > > > >> you :) > > > > > > > >> > > > > > > > > > > > > > > > > Better than (because its no more 1 frame X nb_samples in, 1 > > > frame X > > > > > > > > nb_samples out) to replace .filter_frame/.request_frame with > > > > > .activate > > > > > > > > logic. > > > > > > > > > > > > > > > > And make this output delay compensation filtering optional. > > > > > > > > > > > > > > > > In this process make sure that output PTS frame timestamps > are > > > > > > unchanged > > > > > > > > from input one, by keeping reference of needed frames in > filter > > > > > queue. > > > > > > > > > > > > > > > > Look how speechnorm/dynaudnorm does it. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Alternatively, use current logic in ladspa filter, (also add > > option > > > > > with > > > > > > > same name). > > > > > > > > > > > > > > This would need less code, and probably better approach, as > there > > > is > > > > no > > > > > > > extra latency introduced. > > > > > > > > > > > > > > Than mapping 1:1 between same number of samples per frame is > not > > > hold > > > > > any > > > > > > > more, but I think that is not much important any more. > > > > > > > > > > > > > Thank you for replying to me with your valuable feedback! I have > > > > checked > > > > > > af_ladspa > > > > > > and the "request_frame" function in af_ladspa looks similar to > what > > > I'm > > > > > > doing. The > > > > > > difference comes from the fact that I need an internal frame > buffer > > > to > > > > > keep > > > > > > track of > > > > > > output frames. Essentially I add a frame to the internal buffer > > when > > > an > > > > > > input frame > > > > > > comes in. The frames in this buffer will be the future output > > frames. > > > > We > > > > > > start writing > > > > > > these output frame data buffers only when the internal lookahead > > > buffer > > > > > has > > > > > > been filled. > > > > > > When there is no more input frames, we flushing all the remaining > > > data > > > > in > > > > > > the internal > > > > > > frame buffers and lookahead buffers. Can you advise on my > approach > > > > here? > > > > > > Thank you! > > > > > > > > > > > > I can put my current implementation of "filter_frame" and > > > > "request_frame" > > > > > > into the "activate" approach as you suggested with > > > > speechnorm/dynaudnorm. > > > > > > > > > > > > > > > > No need to wait for all buffers to fill up, only lookahead buffer. > > > > > > > > > > Just trim initial samples that is size of lookahead, and than start > > > > > outputing samples > > > > > just once you get whatever modulo of current frame number of > samples. > > > > > > > > > > So there should not be need for extra buffers to keep audio > samples. > > > > > Just buffers to keep input pts and number of samples of previous > > input > > > > > frames, like in ladspa filter. > > > > > > > > > Thank you for the suggestion! From my understanding, we have two ways > > to > > > > achieve > > > > "zero_delay" functionality here. > > > > > > > > Option 1: as you mentioned, we can trim the initial samples of > > lookahead > > > > size. > > > > The size of the lookahead buffer can be multiple frames. For example > > when > > > > the > > > > attack is 0.08 second, sample rate is 44100 and frame size is 1024, > the > > > > lookahead > > > > buffer size is about 3 frames so the filter needs to see at least 3 > > input > > > > audio frames > > > > before it can output one audio frame. We also need to make > assumptions > > > > about the > > > > size of the audio frame (meaning the number of audio samples per > frame) > > > > when flushing. > > > > The frame is probably 1024 conventionally but I think it's better to > > make > > > > less assumption > > > > as possible to allow the filter to be used as flexible as possible. > > > > > > > > Option 2: this is what I proposed before. We basically map the same > > > number > > > > of input > > > > frames to the output and we also make sure everything about the frame > > the > > > > same as > > > > the input except for the audio signal data itself, which will be > > changed > > > by > > > > whatever > > > > processing alimiter has to do with that. I think it is safer to make > > the > > > > filter only work on > > > > the signal itself. It can help other people who use this filter > without > > > > worrying about > > > > any unexpected change on the frame. The downside is that the filter > > > > internally needs to > > > > store some empty frames, which will be written as the lookahead > buffer > > is > > > > filled. > > > > > > > > I don't see any performance difference between these two options but > > > option > > > > 2 looks > > > > better to me because it works solely on the signals without any > changes > > > on > > > > the frame > > > > > > > > > > option 1 does not add extra delay in processing chain at all, and > option > > 2 > > > adds extra delay. > > > > > > Just look at latest version of af_ladspa.c filter code. > > > > > Hi Paul, I have checked af_ladspa filter. Option 1 certainly doesn't > > introduce any delay. > > Option 2 will at most introduce one audio frame of delay in the > processing > > chain. Option > > 2 needs to fill the lookahead buffer and some frames of samples to push > the > > data in the > > lookahead buffer out for one output audio frame. The processing delay is > > probably not a > > big concern. > > > > The reason I proposed option 2 is about the sync of metadata for the > output > > frames: > > > > It appears to me that the only metadata we need to worry about is PTS and > > the number of > > samples in af_ladspa. However, AVFrame has other fields that also seem to > > specify some metadata > > for the frame: > > 1. AVDictionary *metadata > > 2. void *opaque (this seems to be client-specific) > > 3. AVFrameSideData** side_data (this is not handled by > > av_frame_copy_props). > > If we go for option 1, it is likely these fields in the input frame will > > not be mapped to the corresponding > > output frames. I believe this is also an unexpected behavior for other > > users who rely on these fields. I > > understand other filters may not consider this as important but > > conceptually I believe it is better to make > > the filter focus on changing what it is supposed to, which is the audio > > signal itself. > > > > Looking forward to hearing your opinion on this, thanks! > > > > > AVFrame Metadata is mostly useful for cases when input frame after > filtering > with filter that adds metadata no longer changes output frames later in > filter graph. > Thus using something like astats=metadata=1,alimiter is not very useful and > logical. > > Why by, lightly insisting on no extra delay/latency introduction, and > prefer 1, option for alimiter filter > is mostly because in audio domain, latency is relevant and important > subject for most audio processing specially for limiters. > So it is very important to keep it at minimum if possible. > Hi Paul, I have sent out the patch according to your insights. Thank you! -- Wang Cao | Software Engineer | wangcao@google.com | 650-203-7807 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 24+ messages in thread
end of thread, other threads:[~2022-05-12 12:03 UTC | newest] Thread overview: 24+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2022-03-27 6:08 [FFmpeg-devel] [PATCH] avfilter/alimiter: Add "flush_buffer" option to flush the remaining valid data to the output Wang Cao 2022-03-27 21:40 ` Marton Balint 2022-03-30 21:03 ` [FFmpeg-devel] [PATCH] avfilter/alimiter: Remove the delay introduced by lookahead buffer Wang Cao 2022-04-04 15:46 ` Paul B Mahol 2022-04-04 15:49 ` [FFmpeg-devel] [PATCH] avfilter/alimiter: Add "flush_buffer" option to flush the remaining valid data to the output Paul B Mahol 2022-04-04 22:28 ` Marton Balint 2022-04-05 18:56 ` Wang Cao 2022-04-06 11:49 ` Paul B Mahol 2022-04-07 7:46 ` Paul B Mahol 2022-04-07 21:56 ` Wang Cao 2022-04-08 18:42 ` Paul B Mahol 2022-04-08 20:41 ` Wang Cao 2022-04-09 12:37 ` Paul B Mahol 2022-04-11 20:59 ` Wang Cao 2022-04-12 19:42 ` Paul B Mahol 2022-04-13 4:11 ` Wang Cao 2022-04-15 18:49 ` [FFmpeg-devel] [PATCH] avfilter/alimiter: Add an option "comp_delay" that removes the delay introduced by lookahead buffer Wang Cao 2022-04-29 20:48 ` Wang Cao 2022-05-02 8:26 ` Paul B Mahol 2022-05-05 21:14 ` [FFmpeg-devel] [PATCH] avfilter/alimiter:add latency compensation Wang Cao 2022-05-10 22:45 ` Wang Cao 2022-05-12 12:05 ` Paul B Mahol 2022-05-05 21:17 ` [FFmpeg-devel] [PATCH] avfilter/alimiter: Add an option "comp_delay" that removes the delay introduced by lookahead buffer Wang Cao 2022-04-15 18:51 ` [FFmpeg-devel] [PATCH] avfilter/alimiter: Add "flush_buffer" option to flush the remaining valid data to the output Wang Cao
Git Inbox Mirror of the ffmpeg-devel mailing list - see https://ffmpeg.org/mailman/listinfo/ffmpeg-devel This inbox may be cloned and mirrored by anyone: git clone --mirror https://master.gitmailbox.com/ffmpegdev/0 ffmpegdev/git/0.git # If you have public-inbox 1.1+ installed, you may # initialize and index your mirror using the following commands: public-inbox-init -V2 ffmpegdev ffmpegdev/ https://master.gitmailbox.com/ffmpegdev \ ffmpegdev@gitmailbox.com public-inbox-index ffmpegdev Example config snippet for mirrors. AGPL code for this site: git clone https://public-inbox.org/public-inbox.git