From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTP id F104A42EB0 for ; Thu, 12 May 2022 12:03:29 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 8C83068B410; Thu, 12 May 2022 15:03:26 +0300 (EEST) Received: from mail-yb1-f169.google.com (mail-yb1-f169.google.com [209.85.219.169]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id D7E0668B33C for ; Thu, 12 May 2022 15:03:19 +0300 (EEST) Received: by mail-yb1-f169.google.com with SMTP id m190so9316852ybf.4 for ; Thu, 12 May 2022 05:03:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=Mbu9KtDhyyUC4x1kdugIs3tB5klJAAkMtXa9brpZBUc=; b=YFV+jkwOpbkzSWKHBgaKJaN1UZqcuP1l7OPXtuU9Gfcaa+Xq45xuehsWzBdW5ctD0j 5xzZVZv+Tnb1z3Fo89GEcMDIeHySFXfyfITrMBggYrv2fqlLKsrWcyhPSPDE75mS1ESW lqLV2PFyPe9QyExmHD67SWP4bL4G90tEkOz1vxq6E1ZUWyv/d7kfqLNsgcFrNSvl7WYO L4BbzeEDPzzKkXtYmuGsisCj4p0ABo/xVKXeTx83ybNgFz1zDNcr162c/1RxRlTact3T RAPx9sZbCCqi4g0rIQb8eFx3gWL9DJDin+SvPPQvPKNQm7e+5YuAw5xRF6BOnm0Y357c JfMg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=Mbu9KtDhyyUC4x1kdugIs3tB5klJAAkMtXa9brpZBUc=; b=w8r5f0OR2VfI3+KmNM9rzaK4n3A3kJrzYp9znJqPckmewSmNd+Fzy3GMpeyFM1wlKM Iw30lCf223y3CrAcxcfg5nu+Bj/uiMOjFdQt55KvngiByjZIXIykumOarzXkFNOX8hT8 6e7LmWumgcPqqMCpQz4pTqBXX2B3FJOx7U+x8QPo62stdzIpFGewCktL5DHbVz9k7Br+ GtdHIrjN3m9/7ZIO3ODfxQS61TVUJzzp2kIp64h5lq3ytwfvW82+SHhU96wQ0ChJvPc2 B43TIYdeGLRbelV7e5xW/U4NkrnjsC2krks0idImO17wnz2yTAvLctG/4bGwjokTGGsS P5oQ== X-Gm-Message-State: AOAM530JAFsz6jtGhbDpGdBfU6oz/zak1LAQ+3qul1c2s6XP1iP7uG7N U0yObpbc3Nz7CHN4SDKSyA7iZMGRghrDp6NlX+0v2bHS X-Google-Smtp-Source: ABdhPJytB4mPe1OQ/lMajeqgEKfBWKJfc743XQCbe3xonYUR13RJaQuuyqVipStR5kMyo3cuun8JdxOps7OpZzEKjYg= X-Received: by 2002:a25:44c1:0:b0:64a:94d7:6a5a with SMTP id r184-20020a2544c1000000b0064a94d76a5amr24765483yba.396.1652356998176; Thu, 12 May 2022 05:03:18 -0700 (PDT) MIME-Version: 1.0 References: <20220505211416.671275-1-wangcao@google.com> In-Reply-To: From: Paul B Mahol Date: Thu, 12 May 2022 14:05:55 +0200 Message-ID: To: FFmpeg development discussions and patches X-Content-Filtered-By: Mailman/MimeDel 2.1.29 Subject: Re: [FFmpeg-devel] [PATCH] avfilter/alimiter:add latency compensation X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: On Wed, May 11, 2022 at 12:45 AM Wang Cao wrote: > On Thu, May 5, 2022 at 2:14 PM Wang Cao wrote: > > > Also added 2 FATE tests to verify delay is compenated correctly > Applied with minor fixes and tests removed for now. > > > > Signed-off-by: Wang Cao > > --- > > doc/filters.texi | 5 +++ > > libavfilter/af_alimiter.c | 90 +++++++++++++++++++++++++++++++++++++ > > tests/fate/filter-audio.mak | 24 ++++++++-- > > 3 files changed, 116 insertions(+), 3 deletions(-) > > > > diff --git a/doc/filters.texi b/doc/filters.texi > > index a161754233..75a43edd88 100644 > > --- a/doc/filters.texi > > +++ b/doc/filters.texi > > @@ -1978,6 +1978,11 @@ in release time while 1 produces higher release > > times. > > @item level > > Auto level output signal. Default is enabled. > > This normalizes audio back to 0dB if enabled. > > + > > +@item latency > > +Compensate the delay introduced by using the lookahead buffer set with > > attack > > +parameter. Also flush the valid audio data in the lookahead buffer when > > the > > +stream hits EOF > > @end table > > > > Depending on picked setting it is recommended to upsample input 2x or 4x > > times > > diff --git a/libavfilter/af_alimiter.c b/libavfilter/af_alimiter.c > > index 133f98f165..01265758d7 100644 > > --- a/libavfilter/af_alimiter.c > > +++ b/libavfilter/af_alimiter.c > > @@ -26,6 +26,7 @@ > > > > #include "libavutil/channel_layout.h" > > #include "libavutil/common.h" > > +#include "libavutil/fifo.h" > > #include "libavutil/opt.h" > > > > #include "audio.h" > > @@ -33,6 +34,11 @@ > > #include "formats.h" > > #include "internal.h" > > > > +typedef struct MetaItem { > > + int64_t pts; > > + int nb_samples; > > +} MetaItem; > > + > > typedef struct AudioLimiterContext { > > const AVClass *class; > > > > @@ -55,6 +61,14 @@ typedef struct AudioLimiterContext { > > int *nextpos; > > double *nextdelta; > > > > + int in_trim; > > + int out_pad; > > + int64_t next_in_pts; > > + int64_t next_out_pts; > > + int latency; > > + > > + AVFifo *fifo; > > + > > double delta; > > int nextiter; > > int nextlen; > > @@ -73,6 +87,7 @@ static const AVOption alimiter_options[] = { > > { "asc", "enable asc", OFFSET(auto_release), > > AV_OPT_TYPE_BOOL, {.i64=0}, 0, 1, AF }, > > { "asc_level", "set asc level", OFFSET(asc_coeff), > > AV_OPT_TYPE_DOUBLE, {.dbl=0.5}, 0, 1, AF }, > > { "level", "auto level", OFFSET(auto_level), > > AV_OPT_TYPE_BOOL, {.i64=1}, 0, 1, AF }, > > + { "latency", "compensate delay", OFFSET(latency), > > AV_OPT_TYPE_BOOL, {.i64=0}, 0, 1, AF }, > > { NULL } > > }; > > > > @@ -129,6 +144,11 @@ static int filter_frame(AVFilterLink *inlink, > AVFrame > > *in) > > AVFrame *out; > > double *buf; > > int n, c, i; > > + int new_out_samples; > > + int64_t out_duration; > > + int64_t in_duration; > > + int64_t in_pts; > > + MetaItem meta; > > > > if (av_frame_is_writable(in)) { > > out = in; > > @@ -269,12 +289,69 @@ static int filter_frame(AVFilterLink *inlink, > > AVFrame *in) > > dst += channels; > > } > > > > + in_duration = av_rescale_q(in->nb_samples, inlink->time_base, > > av_make_q(1, in->sample_rate)); > > + in_pts = in->pts; > > + meta = (MetaItem){ in->pts, in->nb_samples }; > > + av_fifo_write(s->fifo, &meta, 1); > > if (in != out) > > av_frame_free(&in); > > > > + new_out_samples = out->nb_samples; > > + if (s->in_trim > 0) { > > + int trim = FFMIN(new_out_samples, s->in_trim); > > + new_out_samples -= trim; > > + s->in_trim -= trim; > > + } > > + > > + if (new_out_samples <= 0) { > > + av_frame_free(&out); > > + return 0; > > + } else if (new_out_samples < out->nb_samples) { > > + int offset = out->nb_samples - new_out_samples; > > + memmove(out->extended_data[0], out->extended_data[0] + > > sizeof(double) * offset * out->ch_layout.nb_channels, > > + sizeof(double) * new_out_samples * > > out->ch_layout.nb_channels); > > + out->nb_samples = new_out_samples; > > + s->in_trim = 0; > > + } > > + > > + av_fifo_read(s->fifo, &meta, 1); > > + > > + out_duration = av_rescale_q(out->nb_samples, inlink->time_base, > > av_make_q(1, out->sample_rate)); > > + in_duration = av_rescale_q(meta.nb_samples, inlink->time_base, > > av_make_q(1, out->sample_rate)); > > + in_pts = meta.pts; > > + > > + if (s->next_out_pts != AV_NOPTS_VALUE && out->pts != s->next_out_pts > > && > > + s->next_in_pts != AV_NOPTS_VALUE && in_pts == > s->next_in_pts) { > > + out->pts = s->next_out_pts; > > + } else { > > + out->pts = in_pts; > > + } > > + s->next_in_pts = in_pts + in_duration; > > + s->next_out_pts = out->pts + out_duration; > > + > > return ff_filter_frame(outlink, out); > > } > > > > +static int request_frame(AVFilterLink* outlink) > > +{ > > + AVFilterContext *ctx = outlink->src; > > + AudioLimiterContext *s = (AudioLimiterContext*)ctx->priv; > > + int ret; > > + > > + ret = ff_request_frame(ctx->inputs[0]); > > + > > + if (ret == AVERROR_EOF && s->out_pad > 0) { > > + AVFrame *frame = ff_get_audio_buffer(outlink, FFMIN(1024, > > s->out_pad)); > > + if (!frame) > > + return AVERROR(ENOMEM); > > + > > + s->out_pad -= frame->nb_samples; > > + frame->pts = s->next_in_pts; > > + return filter_frame(ctx->inputs[0], frame); > > + } > > + return ret; > > +} > > + > > static int config_input(AVFilterLink *inlink) > > { > > AVFilterContext *ctx = inlink->dst; > > @@ -294,6 +371,16 @@ static int config_input(AVFilterLink *inlink) > > memset(s->nextpos, -1, obuffer_size * sizeof(*s->nextpos)); > > s->buffer_size = inlink->sample_rate * s->attack * > > inlink->ch_layout.nb_channels; > > s->buffer_size -= s->buffer_size % inlink->ch_layout.nb_channels; > > + if (s->latency) { > > + s->in_trim = s->out_pad = s->buffer_size / > > inlink->ch_layout.nb_channels - 1; > > + } > > + s->next_out_pts = AV_NOPTS_VALUE; > > + s->next_in_pts = AV_NOPTS_VALUE; > > + > > + s->fifo = av_fifo_alloc2(8, sizeof(MetaItem), > AV_FIFO_FLAG_AUTO_GROW); > > + if (!s->fifo) { > > + return AVERROR(ENOMEM); > > + } > > > > if (s->buffer_size <= 0) { > > av_log(ctx, AV_LOG_ERROR, "Attack is too small.\n"); > > @@ -310,6 +397,8 @@ static av_cold void uninit(AVFilterContext *ctx) > > av_freep(&s->buffer); > > av_freep(&s->nextdelta); > > av_freep(&s->nextpos); > > + > > + av_fifo_freep2(&s->fifo); > > } > > > > static const AVFilterPad alimiter_inputs[] = { > > @@ -325,6 +414,7 @@ static const AVFilterPad alimiter_outputs[] = { > > { > > .name = "default", > > .type = AVMEDIA_TYPE_AUDIO, > > + .request_frame = request_frame, > > }, > > }; > > > > diff --git a/tests/fate/filter-audio.mak b/tests/fate/filter-audio.mak > > index eff32b9f81..e33ffdf37f 100644 > > --- a/tests/fate/filter-audio.mak > > +++ b/tests/fate/filter-audio.mak > > @@ -63,11 +63,29 @@ fate-filter-agate: tests/data/asynth-44100-2.wav > > fate-filter-agate: SRC = $(TARGET_PATH)/tests/data/asynth-44100-2.wav > > fate-filter-agate: CMD = framecrc -i $(SRC) -af > > > aresample,agate=level_in=10:range=0:threshold=1:ratio=1:attack=1:knee=1:makeup=4,aresample > > > > -FATE_AFILTER-$(call FILTERDEMDECENCMUX, AFADE, WAV, PCM_S16LE, > PCM_S16LE, > > WAV) += fate-filter-alimiter > > -fate-filter-alimiter: tests/data/asynth-44100-2.wav > > -fate-filter-alimiter: SRC = $(TARGET_PATH)/tests/data/asynth-44100-2.wav > > +tests/data/filter-alimiter-passthrough: TAG = GEN > > +tests/data/filter-alimiter-passthrough: ffmpeg$(PROGSSUF)$(EXESUF) | > > tests/data > > + $(M)$(TARGET_EXEC) $(TARGET_PATH)/$< -nostdin \ > > + -i $(TARGET_PATH)/tests/data/asynth-44100-2.wav -af aresample -f > > crc $(TARGET_PATH)/$@ -y 2>/dev/null > > + > > +FATE_ALIMITER += fate-filter-alimiter-passthrough-default-attack > > +fate-filter-alimiter-passthrough-default-attack: > > tests/data/filter-alimiter-passthrough > > +fate-filter-alimiter-passthrough-default-attack: REF = > > $(TARGET_PATH)/tests/data/filter-alimiter-passthrough > > +fate-filter-alimiter-passthrough-default-attack: CMD = crc -i $(SRC) -af > > > aresample,alimiter=level_in=1:level_out=1:limit=1:level=0:latency=1,aresample > > + > > +FATE_ALIMITER += fate-filter-alimiter-passthrough-large-attack > > +fate-filter-alimiter-passthrough-large-attack: > > tests/data/filter-alimiter-passthrough > > +fate-filter-alimiter-passthrough-large-attack: REF = > > $(TARGET_PATH)/tests/data/filter-alimiter-passthrough > > +fate-filter-alimiter-passthrough-large-attack: CMD = crc -i $(SRC) -af > > > aresample,alimiter=level_in=1:level_out=1:limit=1:level=0:latency=1:attack=80,aresample > > + > > +FATE_ALIMITER += fate-filter-alimiter > > fate-filter-alimiter: CMD = framecrc -i $(SRC) -af > > aresample,alimiter=level_in=1:level_out=2:limit=0.2,aresample > > > > +$(FATE_ALIMITER): tests/data/asynth-44100-2.wav > > +$(FATE_ALIMITER): SRC = $(TARGET_PATH)/tests/data/asynth-44100-2.wav > > + > > +FATE_AFILTER-$(call FILTERDEMDECENCMUX, ATRIM, WAV, PCM_S16LE, > PCM_S16LE, > > WAV) += $(FATE_ALIMITER) > > + > > FATE_AFILTER-$(call FILTERDEMDECENCMUX, AMERGE, WAV, PCM_S16LE, > > PCM_S16LE, WAV) += fate-filter-amerge > > fate-filter-amerge: tests/data/asynth-44100-1.wav > > fate-filter-amerge: SRC = $(TARGET_PATH)/tests/data/asynth-44100-1.wav > > -- > > 2.36.0.512.ge40c2bad7a-goog > > > > Hi folks, I don't know how FATE failed on the server but this specific > "make fate-filter-alimiter" and > "make tests/data/filter-alimiter-passthrough" passed on my local machine. > The code is basically referenced from af_ladspa. Can you advice on this > implementation? Thanks! > -- > > Wang Cao | Software Engineer | wangcao@google.com | 650-203-7807 > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". > _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".