From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTP id 2F98D477F1 for ; Mon, 22 Jan 2024 01:20:20 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 95FD168CFA4; Mon, 22 Jan 2024 03:20:17 +0200 (EET) Received: from mail-wr1-f51.google.com (mail-wr1-f51.google.com [209.85.221.51]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 6627D68C248 for ; Mon, 22 Jan 2024 03:20:10 +0200 (EET) Received: by mail-wr1-f51.google.com with SMTP id ffacd0b85a97d-337cf4eabc9so2138412f8f.3 for ; Sun, 21 Jan 2024 17:20:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1705886409; x=1706491209; darn=ffmpeg.org; h=user-agent:in-reply-to:content-disposition:mime-version:references :mail-followup-to:message-id:subject:to:from:date:from:to:cc:subject :date:message-id:reply-to; bh=XJA672GpbfY9xCKHmke1+3dpeoGigNBkJsXj0wacllQ=; b=IQE99jN/ojrjvocNVdGijdmKg2ppNF2W6hlbDmZFkjbYRkgBTEQun5VgWJFOzuKg2d Pbsz3F+V0n95ObYKjHoyAvsGVLGN1WL5EsNiyXDMCRBgBuHosEjuVK4/+JSPouX+CcE1 iGr8FYq5euJOwLEvFcZY03QfsaZyR2MZhMcKVufYXkyvhGGMFGxvhC0v79UWEE75/i0p qQ4vOMP0sBJrk/CGFoQYMI36ZP7wzq85xDxFGzxdx7TMMTNmRdcISHkxOe6y25Bk9Jrk JjVsArkjGcbFjL8hqJ/y7G+OOd6cHPdjN/ciO2XZerrbRkt/bLHOKNB4rV0jeI747eB8 yLWA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1705886409; x=1706491209; h=user-agent:in-reply-to:content-disposition:mime-version:references :mail-followup-to:message-id:subject:to:from:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=XJA672GpbfY9xCKHmke1+3dpeoGigNBkJsXj0wacllQ=; b=ckNiPSBS4/rb1ezU9rDxmzISnVrPr/8fDgXaNPtXGy6EQgWDTV9oBZTvvXTUyZ6JNp uV32vaxrjyUDhGpaZE8tmGMia2iBIomZa90m0LkZaU552GVWITqMeYFDVsARp71HK8t9 p9OLrDBTvjpJxuqxbDwxqT+4P4NQpU1hrFIqrathSFJdi9q7vCRylruXfWBd5xHEgvNN uG950+Xr4+w4I8AOHDot5zQCH2+8ZiKpKxvmeNHCwOD1Zs3UorW4fSuI2QAwvWhPafdC nXYdRu4IUDoOPfr19srsI4MwpuWQQBfTOVnGqmN+CZSC7KnR45hU9AGLoK2bQirCTN6N GUuw== X-Gm-Message-State: AOJu0Yzgys2Dwl+ErbRLPy25B3MCmphQ534yUebQRh7o8VJnEsUugYxS DwWvmCVf44aM05OAdUNBlDzZceoXkfGmb5RDR40XdeEIyZ9CHIxmAIeh+R46 X-Google-Smtp-Source: AGHT+IE1Jkvqdw4ELWGd58gFB+S88CyY8ss2ksCRtA8BcSHdldteu8ZVXEQO9pu4wD+JjfCHwlbtYA== X-Received: by 2002:a5d:5191:0:b0:337:4221:4d24 with SMTP id k17-20020a5d5191000000b0033742214d24mr1656857wrv.2.1705886408508; Sun, 21 Jan 2024 17:20:08 -0800 (PST) Received: from mariano (dynamic-adsl-84-220-189-10.clienti.tiscali.it. [84.220.189.10]) by smtp.gmail.com with ESMTPSA id v1-20020a17090606c100b00a2a04754eb1sm12873656ejb.8.2024.01.21.17.20.08 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 21 Jan 2024 17:20:08 -0800 (PST) Received: by mariano (Postfix, from userid 1000) id E60D8BFCDC; Mon, 22 Jan 2024 02:20:06 +0100 (CET) Date: Mon, 22 Jan 2024 02:20:06 +0100 From: Stefano Sabatini To: FFmpeg development discussions and patches Message-ID: Mail-Followup-To: FFmpeg development discussions and patches References: <20240116121643.74916-1-karwalharshit@gmail.com> <20240116121643.74916-2-karwalharshit@gmail.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20240116121643.74916-2-karwalharshit@gmail.com> User-Agent: Mutt/2.1.4 (2021-12-11) Subject: Re: [FFmpeg-devel] [PATCH v4 1/2] avfilter: add audio overlay filter X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: On date Tuesday 2024-01-16 17:46:42 +0530, Harshit Karwal wrote: > Co-authored-by: Paul B Mahol > Signed-off-by: Harshit Karwal > --- > doc/filters.texi | 40 +++ > libavfilter/Makefile | 1 + > libavfilter/af_aoverlay.c | 538 ++++++++++++++++++++++++++++++++++++++ > libavfilter/allfilters.c | 1 + > 4 files changed, 580 insertions(+) > create mode 100644 libavfilter/af_aoverlay.c > > diff --git a/doc/filters.texi b/doc/filters.texi > index 20c91bab3a..79eb600ae3 100644 > --- a/doc/filters.texi > +++ b/doc/filters.texi > @@ -2779,6 +2779,46 @@ This filter supports the same commands as options, excluding option @code{order} > > Pass the audio source unchanged to the output. > > +@section aoverlay > + > +Replace a specified section of an audio stream with another input audio stream. > + > +In case no enable option for timeline editing is specified, the second audio stream will nit: @option{enable} > +be output at sections of the first stream which have a gap in PTS (Presentation TimeStamp) values > +such that the output stream's PTS values are monotonous. > + > +This filter also supports linear cross fading when transitioning from one > +input stream to another. > + > +The filter accepts the following option: nit: options in case we add more > + > +@table @option > +@item cf_duration > +Set duration (in seconds) for cross fade between the inputs. Default value is @code{100} milliseconds. > +@end table > + > +@subsection Examples > + > +@itemize > +@item > +Replace the first stream with the second stream from @code{t=10} seconds to @code{t=20} seconds: > +@example > +ffmpeg -i first.wav -i second.wav -filter_complex "aoverlay=enable='between(t,10,20)'" output.wav > +@end example > + > +@item > +Do the same as above, but with crossfading for @code{2} seconds between the streams: > +@example > +ffmpeg -i first.wav -i second.wav -filter_complex "aoverlay=cf_duration=2:enable='between(t,10,20)'" output.wav > +@end example > + > +@item > +Introduce a PTS gap from @code{t=4} seconds to @code{t=8} seconds in the first stream and output the second stream during this gap: > +@example > +ffmpeg -i first.wav -i second.wav -filter_complex "[0]aselect='not(between(t,4,8))'[temp];[temp][1]aoverlay[out]" -map "[out]" output.wav > +@end example > +@end itemize > + > @section apad > > Pad the end of an audio stream with silence. > diff --git a/libavfilter/Makefile b/libavfilter/Makefile > index bba0219876..0f2b403441 100644 > --- a/libavfilter/Makefile > +++ b/libavfilter/Makefile > @@ -81,6 +81,7 @@ OBJS-$(CONFIG_ANLMDN_FILTER) += af_anlmdn.o > OBJS-$(CONFIG_ANLMF_FILTER) += af_anlms.o > OBJS-$(CONFIG_ANLMS_FILTER) += af_anlms.o > OBJS-$(CONFIG_ANULL_FILTER) += af_anull.o > +OBJS-$(CONFIG_AOVERLAY_FILTER) += af_aoverlay.o > OBJS-$(CONFIG_APAD_FILTER) += af_apad.o > OBJS-$(CONFIG_APERMS_FILTER) += f_perms.o > OBJS-$(CONFIG_APHASER_FILTER) += af_aphaser.o generate_wave_table.o > diff --git a/libavfilter/af_aoverlay.c b/libavfilter/af_aoverlay.c > new file mode 100644 > index 0000000000..f7ac00dda1 > --- /dev/null > +++ b/libavfilter/af_aoverlay.c [...] > +static int crossfade_prepare(AOverlayContext *s, AVFilterLink *main_inlink, AVFilterLink *overlay_inlink, AVFilterLink *outlink, > + int nb_samples, AVFrame **main_buffer, AVFrame **overlay_buffer, int mode) > +{ > + int ret; > + > + *main_buffer = ff_get_audio_buffer(outlink, nb_samples); > + if (!(*main_buffer)) > + return AVERROR(ENOMEM); > + > + (*main_buffer)->pts = s->pts; > + s->pts += av_rescale_q(nb_samples, (AVRational){ 1, outlink->sample_rate }, outlink->time_base); > + > + if ((ret = av_audio_fifo_read(s->main_sample_buffers, (void **)(*main_buffer)->extended_data, nb_samples)) < 0) > + return ret; > + > + if (mode == 1) { > + s->previous_samples = (*main_buffer)->nb_samples; > + } else if (mode == -1 || (mode == 0 && s->is_disabled)) { it would help to use an enum to describe the mode value Also would help to introduce some debug log messages to aid troubleshooting/debugging. For instance, it would be very useful to show the exact time when the overlay stream is inserted. [...] > +static int activate(AVFilterContext *ctx) > +{ > + AOverlayContext *s = ctx->priv; > + int status, ret, nb_samples; > + int64_t pts; > + AVFrame *out = NULL, *main_buffer = NULL, *overlay_buffer = NULL; > + > + AVFilterLink *main_inlink = ctx->inputs[0]; > + AVFilterLink *overlay_inlink = ctx->inputs[1]; > + AVFilterLink *outlink = ctx->outputs[0]; > + > + FF_FILTER_FORWARD_STATUS_BACK_ALL(outlink, ctx); > + > + if (s->default_mode && (s->pts_gap_end - s->pts_gap_start <= 0 || s->overlay_eof)) { > + s->default_mode = 0; > + s->transition_pts2 = s->pts_gap_end; > + } > + > + if (av_audio_fifo_space(s->main_sample_buffers) != 0 && !s->main_eof && !s->default_mode) { > + nb_samples = FFMIN(SEGMENT_SIZE, av_audio_fifo_space(s->main_sample_buffers)); > + > + ret = ff_inlink_consume_samples(main_inlink, nb_samples, nb_samples, &s->main_input); > + if (ret > 0) { > + if (ctx->enable_str && s->is_disabled != ctx->is_disabled && !s->overlay_eof) { > + s->is_disabled = ctx->is_disabled; > + s->transition_pts = s->main_input->pts; > + > + if (s->main_input->nb_samples < av_audio_fifo_space(s->main_sample_buffers)) > + s->crossfade_ready = 1; > + if (av_audio_fifo_size(s->main_sample_buffers) == 0) { > + s->transition_pts = AV_NOPTS_VALUE; > + s->crossfade_ready = 0; > + } > + } > + if (!ctx->enable_str && !s->default_mode) { nit: else if to avoid this evaluation in case the first block is executed [...] > + > +static int config_output(AVFilterLink *outlink) > +{ > + AVFilterContext *ctx = outlink->src; > + AOverlayContext *s = ctx->priv; > + int size, fifo_size; > + > + switch (outlink->format) { > + case AV_SAMPLE_FMT_DBLP: s->crossfade_samples = crossfade_samples_dblp; > + size = sizeof(double); > + break; > + case AV_SAMPLE_FMT_FLTP: s->crossfade_samples = crossfade_samples_fltp; > + size = sizeof(float); > + break; > + case AV_SAMPLE_FMT_S16P: s->crossfade_samples = crossfade_samples_s16p; > + size = sizeof(int16_t); > + break; > + case AV_SAMPLE_FMT_S32P: s->crossfade_samples = crossfade_samples_s32p; > + size = sizeof(int32_t); > + break; > + } > + > + if (s->cf_duration) > + s->cf_samples = av_rescale(s->cf_duration, outlink->sample_rate, AV_TIME_BASE); > + else > + s->cf_samples = av_rescale(100000, outlink->sample_rate, AV_TIME_BASE); is this needed? shouldn't the duration be set also for the default case? [...] Thanks _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".