From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTP id D661B4110C for ; Fri, 15 Apr 2022 18:50:51 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 8E7E268B4DA; Fri, 15 Apr 2022 21:50:48 +0300 (EEST) Received: from mail-yb1-f201.google.com (mail-yb1-f201.google.com [209.85.219.201]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 3657768B45D for ; Fri, 15 Apr 2022 21:50:42 +0300 (EEST) Received: by mail-yb1-f201.google.com with SMTP id q142-20020a25d994000000b00641d23dbeb9so7034306ybg.9 for ; Fri, 15 Apr 2022 11:50:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=wgTJ9HaZdr1a/u3IxQmUww1A4c9hoHbah9pE7e2Z4+0=; b=mR0diJI6w+ANg6rsGvIX3wY7sd6o/1j5Xx90vDY9WPhsKWzHJzry3864yWimJBvZ7x 57X5+ubTh5i+e77MksEO2DFx21c7vtW8GddCB7mHwLHU6FR7ZfUqHualTvO55Bm52/F7 r6jsn8jL/B3uxdu5Pw/qKbs62DH1lJeYXnsz0UWY79EcGtQgX/2QHf+mYtR6RK+ZVd7n NSg10f4/yEBh54M8sj+4lmIU4BUAG/rz9lNRRMUfRlPOSUFJKtqDAHJg/BA4DOKwVlbg nF8Yl5dW1l71xAIsdpUSdNIbN/Il97fxmXZun1XM6MZELFY1qLF92m/+w9THeOpXbjXv cL7g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=wgTJ9HaZdr1a/u3IxQmUww1A4c9hoHbah9pE7e2Z4+0=; b=FRMf0TwxfYfu+JzpLhCfZnlV9HdPI9wbPgDpanjcnK1ns21DHNROV2vdULxVnR7B3O nCEzgQfxaNaTmErFfo1qS9qY7c+7fNaCjHojBhS7V9nOfqU5zY2bj7JpTgexDxsC7H0N f/NZ/PFtlgMkC7mb//oQFJxrTvkcNlOsEy3nZACVN5D1E79SopTtlYyPaNcDNGKZLiLN rV4CV/3w92LH0KUJX84qNJfeSU9s7akJ8KWvpS2xk2FgXBDthDp0AhrWQOBh53CZGS8O dkP1qcAbbbKRqMi540djXK8YQg6mjN+ltEZ3uDBbLO7PSAFTMAw+ffeSFE8GEoTib0zZ Gg+Q== X-Gm-Message-State: AOAM530KBPbdde9Vh0++vzMDVHwT9YiGTG3JLnlNzb4Cb/QhJKCb4+Np Fq1FSUPqkVclLLStoRhxehgdaccFQ3Pvu2wBr2Ngte9Oz9qXG112dsg+Rudy50f5139040wKqP6 63ycLxnSHu90iyHGV9XKk84IDML/bXfi1uSuBRMtVsT0nj295wSOCBW6kVmKO0BKtXQ== X-Google-Smtp-Source: ABdhPJyJxKzjT5y+5CTZugGaPKuvaRvRiO8Ao9huku6Lp9+fu39NqIsVXnczifKLMkMwlcZs1A/GtcJf6Ao4 X-Received: from wangcao.mtv.corp.google.com ([2620:0:1000:4011:9c0:7189:8016:d159]) (user=wangcao job=sendgmr) by 2002:a25:312:0:b0:641:3cfa:372d with SMTP id 18-20020a250312000000b006413cfa372dmr524618ybd.580.1650048640560; Fri, 15 Apr 2022 11:50:40 -0700 (PDT) Date: Fri, 15 Apr 2022 11:49:35 -0700 In-Reply-To: Message-Id: <20220415184934.1891758-1-wangcao@google.com> Mime-Version: 1.0 References: X-Mailer: git-send-email 2.36.0.rc0.470.gd361397f0d-goog From: Wang Cao To: ffmpeg-devel@ffmpeg.org Subject: [FFmpeg-devel] [PATCH] avfilter/alimiter: Add an option "comp_delay" that removes the delay introduced by lookahead buffer X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Wang Cao Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: 1. The option also flushes all the valid audio samples in the lookahead buffer so the audio integrity is preserved. Previously the the output audio will lose the amount of audio samples equal to the size of lookahead buffer 2. Add a FATE test to verify that when the filter is working as passthrough filter, all audio samples are properly handled from the input to the output. Signed-off-by: Wang Cao --- doc/filters.texi | 5 +++ libavfilter/af_alimiter.c | 74 +++++++++++++++++++++++++++++++++++++ tests/fate/filter-audio.mak | 12 ++++++ 3 files changed, 91 insertions(+) diff --git a/doc/filters.texi b/doc/filters.texi index a161754233..2af0953c89 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -1978,6 +1978,11 @@ in release time while 1 produces higher release times. @item level Auto level output signal. Default is enabled. This normalizes audio back to 0dB if enabled. + +@item comp_delay +Compensate the delay introduced by using the lookahead buffer set with attack +parameter. Also flush the valid audio data in the lookahead buffer when the +stream hits EOF. @end table Depending on picked setting it is recommended to upsample input 2x or 4x times diff --git a/libavfilter/af_alimiter.c b/libavfilter/af_alimiter.c index 133f98f165..d10a90859b 100644 --- a/libavfilter/af_alimiter.c +++ b/libavfilter/af_alimiter.c @@ -55,6 +55,12 @@ typedef struct AudioLimiterContext { int *nextpos; double *nextdelta; + int lookahead_delay_samples; + int lookahead_flush_samples; + int64_t output_pts; + int64_t next_output_pts; + int comp_delay; + double delta; int nextiter; int nextlen; @@ -73,6 +79,7 @@ static const AVOption alimiter_options[] = { { "asc", "enable asc", OFFSET(auto_release), AV_OPT_TYPE_BOOL, {.i64=0}, 0, 1, AF }, { "asc_level", "set asc level", OFFSET(asc_coeff), AV_OPT_TYPE_DOUBLE, {.dbl=0.5}, 0, 1, AF }, { "level", "auto level", OFFSET(auto_level), AV_OPT_TYPE_BOOL, {.i64=1}, 0, 1, AF }, + { "comp_delay","compensate delay", OFFSET(comp_delay), AV_OPT_TYPE_BOOL, {.i64=0}, 0, 1, AF }, { NULL } }; @@ -129,6 +136,8 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *in) AVFrame *out; double *buf; int n, c, i; + int num_output_samples = in->nb_samples; + int trim_offset; if (av_frame_is_writable(in)) { out = in; @@ -271,10 +280,71 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *in) if (in != out) av_frame_free(&in); + + if (!s->comp_delay) { + return ff_filter_frame(outlink, out); + } + + if (s->output_pts == AV_NOPTS_VALUE) { + s->output_pts = in->pts; + } + + if (s->lookahead_delay_samples > 0) { + // The current output frame is completely silence + if (s->lookahead_delay_samples >= in->nb_samples) { + s->lookahead_delay_samples -= in->nb_samples; + return 0; + } + + // Trim the silence part + trim_offset = av_samples_get_buffer_size( + NULL, inlink->ch_layout.nb_channels, s->lookahead_delay_samples, + (enum AVSampleFormat)out->format, 1); + out->data[0] += trim_offset; + out->nb_samples = in->nb_samples - s->lookahead_delay_samples; + s->lookahead_delay_samples = 0; + num_output_samples = out->nb_samples; + } + + if (s->lookahead_delay_samples < 0) { + return AVERROR_BUG; + } + + out->pts = s->output_pts; + s->next_output_pts = s->output_pts + num_output_samples; + s->output_pts = s->next_output_pts; return ff_filter_frame(outlink, out); } +static int request_frame(AVFilterLink* outlink) +{ + AVFilterContext *ctx = outlink->src; + AudioLimiterContext *s = (AudioLimiterContext*)ctx->priv; + int ret; + AVFilterLink *inlink; + AVFrame *silence_frame; + + ret = ff_request_frame(ctx->inputs[0]); + + if (ret != AVERROR_EOF || s->lookahead_flush_samples == 0 || !s->comp_delay) { + // Not necessarily an error, just not EOF. + return ret; + } + + // We reach here when input filters have finished producing data (i.e. EOF), + // but because of the attack param, s->buffer still has meaningful + // audio content that needs flushing. + inlink = ctx->inputs[0]; + // Pushes silence frame to flush valid audio in the s->buffer + silence_frame = ff_get_audio_buffer(inlink, s->lookahead_flush_samples); + ret = filter_frame(inlink, silence_frame); + if (ret < 0) { + return ret; + } + return AVERROR_EOF; +} + static int config_input(AVFilterLink *inlink) { AVFilterContext *ctx = inlink->dst; @@ -294,6 +364,9 @@ static int config_input(AVFilterLink *inlink) memset(s->nextpos, -1, obuffer_size * sizeof(*s->nextpos)); s->buffer_size = inlink->sample_rate * s->attack * inlink->ch_layout.nb_channels; s->buffer_size -= s->buffer_size % inlink->ch_layout.nb_channels; + // the current logic outputs the next sample from the lookahead buffer from the beginning so the amount of delay + // compensation is less than the lookahead buffer size by 1 . + s->lookahead_delay_samples = s->lookahead_flush_samples = s->buffer_size / inlink->ch_layout.nb_channels - 1; if (s->buffer_size <= 0) { av_log(ctx, AV_LOG_ERROR, "Attack is too small.\n"); @@ -325,6 +398,7 @@ static const AVFilterPad alimiter_outputs[] = { { .name = "default", .type = AVMEDIA_TYPE_AUDIO, + .request_frame = request_frame, }, }; diff --git a/tests/fate/filter-audio.mak b/tests/fate/filter-audio.mak index eff32b9f81..3a51ca18a6 100644 --- a/tests/fate/filter-audio.mak +++ b/tests/fate/filter-audio.mak @@ -63,6 +63,18 @@ fate-filter-agate: tests/data/asynth-44100-2.wav fate-filter-agate: SRC = $(TARGET_PATH)/tests/data/asynth-44100-2.wav fate-filter-agate: CMD = framecrc -i $(SRC) -af aresample,agate=level_in=10:range=0:threshold=1:ratio=1:attack=1:knee=1:makeup=4,aresample +tests/data/filter-alimiter-passthrough: TAG = GEN +tests/data/filter-alimiter-passthrough: ffmpeg$(PROGSSUF)$(EXESUF) | tests/data + $(M)$(TARGET_EXEC) $(TARGET_PATH)/$< -nostdin \ + -i $(TARGET_PATH)/tests/data/asynth-44100-2.wav -af aresample -f crc $(TARGET_PATH)/$@ -y 2>/dev/null + +FATE_AFILTER-$(call FILTERDEMDECENCMUX, AFADE, WAV, PCM_S16LE, PCM_S16LE, WAV) += fate-filter-alimiter-passthrough +fate-filter-alimiter-passthrough: tests/data/asynth-44100-2.wav +fate-filter-alimiter-passthrough: tests/data/filter-alimiter-passthrough +fate-filter-alimiter-passthrough: SRC = $(TARGET_PATH)/tests/data/asynth-44100-2.wav +fate-filter-alimiter-passthrough: REF = $(TARGET_PATH)/tests/data/filter-alimiter-passthrough +fate-filter-alimiter-passthrough: CMD = crc -i $(SRC) -af aresample,alimiter=level_in=1:level_out=1:limit=1:level=0:comp_delay=1,aresample + FATE_AFILTER-$(call FILTERDEMDECENCMUX, AFADE, WAV, PCM_S16LE, PCM_S16LE, WAV) += fate-filter-alimiter fate-filter-alimiter: tests/data/asynth-44100-2.wav fate-filter-alimiter: SRC = $(TARGET_PATH)/tests/data/asynth-44100-2.wav -- 2.36.0.rc0.470.gd361397f0d-goog _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".