From: Niklas Haas via ffmpeg-devel <ffmpeg-devel@ffmpeg.org>
To: ffmpeg-devel@ffmpeg.org
Cc: Niklas Haas <code@ffmpeg.org>
Subject: [FFmpeg-devel] [PATCH] Generalize acrossfade filter to support multiple inputs (PR #20388)
Message-ID: <175675980335.25.15983611420732339356@463a07221176> (raw)
PR #20388 opened by Niklas Haas (haasn)
URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/20388
Patch URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/20388.patch
Effectively makes it a copy of the concat filter but with the ability to crossfade instead of hard cutting.
I did have to formally error out when one of the inputs is too short to cover the full crossfade duration; see the commit log for information why.
>From dd3a43a7e8b681bb700f0697617b094cb49223b4 Mon Sep 17 00:00:00 2001
From: Niklas Haas <git@haasn.dev>
Date: Mon, 1 Sep 2025 21:36:01 +0200
Subject: [PATCH 1/4] avfilter/af_afade: generalize pass_crossfade() signature
Prerequisite to an upcoming refactor.
---
libavfilter/af_afade.c | 12 ++++++------
1 file changed, 6 insertions(+), 6 deletions(-)
diff --git a/libavfilter/af_afade.c b/libavfilter/af_afade.c
index d4ea1a7bab..67ae3758a2 100644
--- a/libavfilter/af_afade.c
+++ b/libavfilter/af_afade.c
@@ -566,7 +566,7 @@ static int pass_samples(AVFilterLink *inlink, AVFilterLink *outlink, unsigned nb
return ff_filter_frame(outlink, in);
}
-static int pass_crossfade(AVFilterContext *ctx)
+static int pass_crossfade(AVFilterContext *ctx, AVFilterLink *in0, AVFilterLink *in1)
{
AudioFadeContext *s = ctx->priv;
AVFilterLink *outlink = ctx->outputs[0];
@@ -578,13 +578,13 @@ static int pass_crossfade(AVFilterContext *ctx)
if (!out)
return AVERROR(ENOMEM);
- ret = ff_inlink_consume_samples(ctx->inputs[0], s->nb_samples, s->nb_samples, &cf[0]);
+ ret = ff_inlink_consume_samples(in0, s->nb_samples, s->nb_samples, &cf[0]);
if (ret < 0) {
av_frame_free(&out);
return ret;
}
- ret = ff_inlink_consume_samples(ctx->inputs[1], s->nb_samples, s->nb_samples, &cf[1]);
+ ret = ff_inlink_consume_samples(in1, s->nb_samples, s->nb_samples, &cf[1]);
if (ret < 0) {
av_frame_free(&out);
return ret;
@@ -605,7 +605,7 @@ static int pass_crossfade(AVFilterContext *ctx)
if (!out)
return AVERROR(ENOMEM);
- ret = ff_inlink_consume_samples(ctx->inputs[0], s->nb_samples, s->nb_samples, &cf[0]);
+ ret = ff_inlink_consume_samples(in0, s->nb_samples, s->nb_samples, &cf[0]);
if (ret < 0) {
av_frame_free(&out);
return ret;
@@ -625,7 +625,7 @@ static int pass_crossfade(AVFilterContext *ctx)
if (!out)
return AVERROR(ENOMEM);
- ret = ff_inlink_consume_samples(ctx->inputs[1], s->nb_samples, s->nb_samples, &cf[1]);
+ ret = ff_inlink_consume_samples(in1, s->nb_samples, s->nb_samples, &cf[1]);
if (ret < 0) {
av_frame_free(&out);
return ret;
@@ -678,7 +678,7 @@ static int activate(AVFilterContext *ctx)
// TODO: Do some partial crossfade if not all inputs have enough duration?
if (ff_inlink_queued_samples(ctx->inputs[0]) >= s->nb_samples &&
ff_inlink_queued_samples(ctx->inputs[1]) >= s->nb_samples)
- return pass_crossfade(ctx);
+ return pass_crossfade(ctx, ctx->inputs[0], ctx->inputs[1]);
}
// Read second input until EOF
if (s->xfade_status == 3) {
--
2.49.1
>From 926876f88352d7511ed77bbd8ff69450bfe3191b Mon Sep 17 00:00:00 2001
From: Niklas Haas <git@haasn.dev>
Date: Mon, 1 Sep 2025 22:01:28 +0200
Subject: [PATCH 2/4] avfilter/af_afade: properly error out on too short inputs
This behavior is currently broken anyway, leading to an abrupt end of the
audio stream. I want to generalize this filter to multiple inputs, but having
too short input files will always represent a significant problem.
I considered a few approaches for how to handle this more gracefully, but
all of them come with their own problems; in particular when a short input
is sandwiched between two longer ones; or when there is a sequence of short
inputs. Ideally, we'd construct some sort of dynamically changing mixing ratio
between all inputs; which would require mixing together arbitrarily many
inputs at any given point in time.
Furthermore, it becomes difficult to define what "overlapping" even means
when the entire input is shorter than the overlap window. Alternatively, we
could just shorten the overlap window to the duration of the input, but this
is also nontrivial (if two short inputs are back to back, we need to shorten
it to maximum of half the number of samples) and complicated the activation
and state logic considerably.
After spending about an hour headdesking about this problem I decided to go
the easy way and just properly error out in this case, so I can continue
doing so when combining multiple filters.
---
libavfilter/af_afade.c | 11 +++++++++--
1 file changed, 9 insertions(+), 2 deletions(-)
diff --git a/libavfilter/af_afade.c b/libavfilter/af_afade.c
index 67ae3758a2..b21a0745fc 100644
--- a/libavfilter/af_afade.c
+++ b/libavfilter/af_afade.c
@@ -676,9 +676,16 @@ static int activate(AVFilterContext *ctx)
if (s->xfade_status == 2) {
s->xfade_status = 3;
// TODO: Do some partial crossfade if not all inputs have enough duration?
- if (ff_inlink_queued_samples(ctx->inputs[0]) >= s->nb_samples &&
- ff_inlink_queued_samples(ctx->inputs[1]) >= s->nb_samples)
+ int queued_samples0 = ff_inlink_queued_samples(ctx->inputs[0]);
+ int queued_samples1 = ff_inlink_queued_samples(ctx->inputs[1]);
+ if (queued_samples0 >= s->nb_samples && queued_samples1 >= s->nb_samples) {
return pass_crossfade(ctx, ctx->inputs[0], ctx->inputs[1]);
+ } else {
+ av_log(ctx, AV_LOG_ERROR, "Not enough samples to perform crossfade! "
+ "Expected %"PRId64", got %d\n", s->nb_samples,
+ FFMIN(queued_samples0, queued_samples1));
+ return AVERROR_PATCHWELCOME;
+ }
}
// Read second input until EOF
if (s->xfade_status == 3) {
--
2.49.1
>From 29ec62f45e923c8e6403275a96eb34b3ce45f444 Mon Sep 17 00:00:00 2001
From: Niklas Haas <git@haasn.dev>
Date: Mon, 1 Sep 2025 22:11:28 +0200
Subject: [PATCH 3/4] avfilter/af_afade: support multiple inputs
Instead of just 2 files, generalize this filter to support crossfading
arbitrarily many files. This makes the filter essentially operate similar
to the `concat` filter, chaining multiple files one after another.
Aside from just adding more input pads, this requires rewriting the
activate function. Instead of a finite state machine, we keep track of the
currently active input index; and advance it only once the current input is
fully exhausted.
This results in arguably simpler logic overall.
---
doc/filters.texi | 10 ++++
libavfilter/af_afade.c | 130 ++++++++++++++++++++++-------------------
2 files changed, 81 insertions(+), 59 deletions(-)
diff --git a/doc/filters.texi b/doc/filters.texi
index 4c774162e3..aee00424b6 100644
--- a/doc/filters.texi
+++ b/doc/filters.texi
@@ -577,6 +577,11 @@ The cross fade is applied for specified duration near the end of first stream.
The filter accepts the following options:
@table @option
+@item inputs, n
+Specify the number of inputs to crossfade. When crossfading multiple inputs,
+each input will be concatenated and crossfaded in sequence, similar to the
+@ref{concat} filter. Default is 2.
+
@item nb_samples, ns
Specify the number of samples for which the cross fade effect has to last.
At the end of the cross fade effect the first input audio will be completely
@@ -615,6 +620,11 @@ Cross fade from one input to another but without overlapping:
@example
ffmpeg -i first.flac -i second.flac -filter_complex acrossfade=d=10:o=0:c1=exp:c2=exp output.flac
@end example
+
+Concatenate multiple inputs with cross fade between each:
+@example
+ffmpeg -i first.flac -i second.flac -i third.flac -filter_complex acrossfade=n=3 output.flac
+@end example
@end itemize
@section acrossover
diff --git a/libavfilter/af_afade.c b/libavfilter/af_afade.c
index b21a0745fc..0cae0ed76c 100644
--- a/libavfilter/af_afade.c
+++ b/libavfilter/af_afade.c
@@ -26,6 +26,7 @@
#include "config_components.h"
#include "libavutil/avassert.h"
+#include "libavutil/avstring.h"
#include "libavutil/opt.h"
#include "audio.h"
#include "avfilter.h"
@@ -33,6 +34,7 @@
typedef struct AudioFadeContext {
const AVClass *class;
+ int nb_inputs;
int type;
int curve, curve2;
int64_t nb_samples;
@@ -43,7 +45,7 @@ typedef struct AudioFadeContext {
double unity;
int overlap;
int64_t pts;
- int xfade_status;
+ int xfade_idx;
void (*fade_samples)(uint8_t **dst, uint8_t * const *src,
int nb_samples, int channels, int direction,
@@ -451,6 +453,8 @@ const FFFilter ff_af_afade = {
#if CONFIG_ACROSSFADE_FILTER
static const AVOption acrossfade_options[] = {
+ { "inputs", "set number of input files to cross fade", OFFSET(nb_inputs), AV_OPT_TYPE_INT, {.i64 = 2}, 1, INT32_MAX, FLAGS },
+ { "n", "set number of input files to cross fade", OFFSET(nb_inputs), AV_OPT_TYPE_INT, {.i64 = 2}, 1, INT32_MAX, FLAGS },
{ "nb_samples", "set number of samples for cross fade duration", OFFSET(nb_samples), AV_OPT_TYPE_INT64, {.i64 = 44100}, 1, INT32_MAX/10, FLAGS },
{ "ns", "set number of samples for cross fade duration", OFFSET(nb_samples), AV_OPT_TYPE_INT64, {.i64 = 44100}, 1, INT32_MAX/10, FLAGS },
{ "duration", "set cross fade duration", OFFSET(duration), AV_OPT_TYPE_DURATION, {.i64 = 0 }, 0, 60000000, FLAGS },
@@ -645,57 +649,75 @@ static int activate(AVFilterContext *ctx)
{
AudioFadeContext *s = ctx->priv;
AVFilterLink *outlink = ctx->outputs[0];
+ AVFilterLink *in0 = ctx->inputs[s->xfade_idx];
FF_FILTER_FORWARD_STATUS_BACK_ALL(outlink, ctx);
- // Read first input until EOF
- if (s->xfade_status == 0) {
- int queued_samples = ff_inlink_queued_samples(ctx->inputs[0]);
- if (queued_samples > s->nb_samples) {
- AVFrame *frame = ff_inlink_peek_frame(ctx->inputs[0], 0);
- if (queued_samples - s->nb_samples >= frame->nb_samples)
- return pass_frame(ctx->inputs[0], outlink, &s->pts);
- }
- if (ff_outlink_get_status(ctx->inputs[0])) {
- if (queued_samples > s->nb_samples)
- return pass_samples(ctx->inputs[0], outlink, queued_samples - s->nb_samples, &s->pts);
- s->xfade_status = 1;
- } else {
- FF_FILTER_FORWARD_WANTED(outlink, ctx->inputs[0]);
- }
- }
- // Read second input until enough data is ready or EOF
- if (s->xfade_status == 1) {
- if (ff_inlink_queued_samples(ctx->inputs[1]) >= s->nb_samples || ff_outlink_get_status(ctx->inputs[1])) {
- s->xfade_status = 2;
- } else {
- FF_FILTER_FORWARD_WANTED(outlink, ctx->inputs[1]);
- }
- }
- // Do crossfade
- if (s->xfade_status == 2) {
- s->xfade_status = 3;
- // TODO: Do some partial crossfade if not all inputs have enough duration?
- int queued_samples0 = ff_inlink_queued_samples(ctx->inputs[0]);
- int queued_samples1 = ff_inlink_queued_samples(ctx->inputs[1]);
- if (queued_samples0 >= s->nb_samples && queued_samples1 >= s->nb_samples) {
- return pass_crossfade(ctx, ctx->inputs[0], ctx->inputs[1]);
- } else {
- av_log(ctx, AV_LOG_ERROR, "Not enough samples to perform crossfade! "
- "Expected %"PRId64", got %d\n", s->nb_samples,
- FFMIN(queued_samples0, queued_samples1));
- return AVERROR_PATCHWELCOME;
- }
- }
- // Read second input until EOF
- if (s->xfade_status == 3) {
- if (ff_inlink_queued_frames(ctx->inputs[1]))
- return pass_frame(ctx->inputs[1], outlink, &s->pts);
- FF_FILTER_FORWARD_STATUS(ctx->inputs[1], outlink);
- FF_FILTER_FORWARD_WANTED(outlink, ctx->inputs[1]);
+ if (s->xfade_idx == s->nb_inputs - 1) {
+ /* Last active input, read until EOF */
+ if (ff_inlink_queued_frames(in0))
+ return pass_frame(in0, outlink, &s->pts);
+ FF_FILTER_FORWARD_STATUS(in0, outlink);
+ FF_FILTER_FORWARD_WANTED(outlink, in0);
+ return FFERROR_NOT_READY;
}
- return FFERROR_NOT_READY;
+ AVFilterLink *in1 = ctx->inputs[s->xfade_idx + 1];
+ AVFrame *frame = ff_inlink_peek_frame(in0, 0);
+ int queued_samples0 = ff_inlink_queued_samples(in0);
+ if (frame && queued_samples0 - s->nb_samples >= frame->nb_samples)
+ return pass_frame(in0, outlink, &s->pts);
+
+ /* Continue reading until EOF */
+ if (ff_outlink_get_status(in0)) {
+ if (queued_samples0 > s->nb_samples) {
+ return pass_samples(in0, outlink, queued_samples0 - s->nb_samples, &s->pts);
+ } else if (queued_samples0 < s->nb_samples) {
+ av_log(ctx, AV_LOG_ERROR, "Not enough samples to perform crossfade! "
+ "Expected %"PRId64", got %d\n", s->nb_samples, queued_samples0);
+ return AVERROR_PATCHWELCOME;
+ }
+ } else {
+ FF_FILTER_FORWARD_WANTED(outlink, in0);
+ return FFERROR_NOT_READY;
+ }
+
+ /* At this point, in0 has reached EOF with no more samples remaining
+ * except those that we want to crossfade */
+ av_assert0(queued_samples0 == s->nb_samples);
+ int queued_samples1 = ff_inlink_queued_samples(in1);
+ if (queued_samples1 >= s->nb_samples) {
+ s->xfade_idx++;
+ return pass_crossfade(ctx, in0, in1);
+ } else if (ff_outlink_get_status(in1)) {
+ av_log(ctx, AV_LOG_ERROR, "Not enough samples to perform crossfade! "
+ "Expected %"PRId64", got %d\n", s->nb_samples, queued_samples1);
+ return AVERROR_PATCHWELCOME;
+ } else {
+ FF_FILTER_FORWARD_WANTED(outlink, in1);
+ return FFERROR_NOT_READY;
+ }
+}
+
+static av_cold int acrossfade_init(AVFilterContext *ctx)
+{
+ AudioFadeContext *s = ctx->priv;
+ int ret;
+
+ for (int i = 0; i < s->nb_inputs; i++) {
+ AVFilterPad pad = {
+ .name = av_asprintf("crossfade%d", i),
+ .type = AVMEDIA_TYPE_AUDIO,
+ };
+ if (!pad.name)
+ return AVERROR(ENOMEM);
+
+ ret = ff_append_inpad_free_name(ctx, &pad);
+ if (ret < 0)
+ return ret;
+ }
+
+ return 0;
}
static int acrossfade_config_output(AVFilterLink *outlink)
@@ -721,17 +743,6 @@ static int acrossfade_config_output(AVFilterLink *outlink)
return 0;
}
-static const AVFilterPad avfilter_af_acrossfade_inputs[] = {
- {
- .name = "crossfade0",
- .type = AVMEDIA_TYPE_AUDIO,
- },
- {
- .name = "crossfade1",
- .type = AVMEDIA_TYPE_AUDIO,
- },
-};
-
static const AVFilterPad avfilter_af_acrossfade_outputs[] = {
{
.name = "default",
@@ -744,9 +755,10 @@ const FFFilter ff_af_acrossfade = {
.p.name = "acrossfade",
.p.description = NULL_IF_CONFIG_SMALL("Cross fade two input audio streams."),
.p.priv_class = &acrossfade_class,
+ .p.flags = AVFILTER_FLAG_DYNAMIC_INPUTS,
.priv_size = sizeof(AudioFadeContext),
+ .init = acrossfade_init,
.activate = activate,
- FILTER_INPUTS(avfilter_af_acrossfade_inputs),
FILTER_OUTPUTS(avfilter_af_acrossfade_outputs),
FILTER_SAMPLEFMTS_ARRAY(sample_fmts),
};
--
2.49.1
>From 64abbdd78b309b0dcfe6d424dd4615d51ef78053 Mon Sep 17 00:00:00 2001
From: Niklas Haas <git@haasn.dev>
Date: Mon, 1 Sep 2025 22:29:22 +0200
Subject: [PATCH 4/4] fate/filter-audio: update acrossfade to test multiple
inputs
---
tests/fate/filter-audio.mak | 3 ++-
tests/ref/fate/filter-acrossfade | 27 ++++-----------------------
2 files changed, 6 insertions(+), 24 deletions(-)
diff --git a/tests/fate/filter-audio.mak b/tests/fate/filter-audio.mak
index ac7f353d59..eee0209c59 100644
--- a/tests/fate/filter-audio.mak
+++ b/tests/fate/filter-audio.mak
@@ -57,7 +57,8 @@ FATE_AFILTER_SAMPLES-$(call FILTERDEMDECENCMUX, ACROSSFADE, WAV, PCM_S16LE, PCM_
fate-filter-acrossfade: tests/data/asynth-44100-2.wav
fate-filter-acrossfade: SRC = $(TARGET_PATH)/tests/data/asynth-44100-2.wav
fate-filter-acrossfade: SRC2 = $(TARGET_SAMPLES)/audio-reference/luckynight_2ch_44kHz_s16.wav
-fate-filter-acrossfade: CMD = framecrc -i $(SRC) -i $(SRC2) -filter_complex acrossfade=d=2:c1=log:c2=exp
+fate-filter-acrossfade: SRC3 = $(TARGET_SAMPLES)/audio-reference/chorusnoise_2ch_44kHz_s16.wav
+fate-filter-acrossfade: CMD = framecrc -i $(SRC) -i $(SRC2) -i $(SRC3) -filter_complex acrossfade=n=3:d=2:c1=log:c2=exp
FATE_AFILTER-$(call FILTERDEMDECENCMUX, AGATE ARESAMPLE, WAV, PCM_S16LE, PCM_S16LE, WAV) += fate-filter-agate
fate-filter-agate: tests/data/asynth-44100-2.wav
diff --git a/tests/ref/fate/filter-acrossfade b/tests/ref/fate/filter-acrossfade
index 92231bec7d..fe45d4ec10 100644
--- a/tests/ref/fate/filter-acrossfade
+++ b/tests/ref/fate/filter-acrossfade
@@ -107,26 +107,7 @@
0, 491792, 491792, 4096, 16384, 0xad648c75
0, 495888, 495888, 4096, 16384, 0xe24fa60b
0, 499984, 499984, 4096, 16384, 0x96b1bf9e
-0, 504080, 504080, 4096, 16384, 0xf1e34827
-0, 508176, 508176, 4096, 16384, 0xc267c4d7
-0, 512272, 512272, 4096, 16384, 0x5d5a2115
-0, 516368, 516368, 4096, 16384, 0xfd6024aa
-0, 520464, 520464, 4096, 16384, 0x9cd58cc0
-0, 524560, 524560, 4096, 16384, 0xb458f309
-0, 528656, 528656, 4096, 16384, 0xaee359c9
-0, 532752, 532752, 4096, 16384, 0x9aa3c89b
-0, 536848, 536848, 4096, 16384, 0x2a4f1f66
-0, 540944, 540944, 4096, 16384, 0x0c817abf
-0, 545040, 545040, 4096, 16384, 0x1e97f79e
-0, 549136, 549136, 4096, 16384, 0x12a32ed0
-0, 553232, 553232, 4096, 16384, 0x722bfc87
-0, 557328, 557328, 4096, 16384, 0x65e0fab9
-0, 561424, 561424, 4096, 16384, 0x55cca852
-0, 565520, 565520, 4096, 16384, 0x19a10ac6
-0, 569616, 569616, 4096, 16384, 0x078180b2
-0, 573712, 573712, 4096, 16384, 0xf6f8fc53
-0, 577808, 577808, 4096, 16384, 0xc72ffe63
-0, 581904, 581904, 4096, 16384, 0xf9f837fd
-0, 586000, 586000, 4096, 16384, 0x78d70334
-0, 590096, 590096, 4096, 16384, 0x061d910c
-0, 594192, 594192, 1158, 4632, 0xdc098a7f
+0, 504080, 504080, 3070, 12280, 0x1368a641
+0, 507150, 507150, 88200, 352800, 0xfdb19b5d
+0, 595350, 595350, 1912, 7648, 0x5d71938c
+0, 597262, 597262, 3097, 12388, 0xd852f4c5
--
2.49.1
_______________________________________________
ffmpeg-devel mailing list -- ffmpeg-devel@ffmpeg.org
To unsubscribe send an email to ffmpeg-devel-leave@ffmpeg.org
reply other threads:[~2025-09-01 20:50 UTC|newest]
Thread overview: [no followups] expand[flat|nested] mbox.gz Atom feed
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=175675980335.25.15983611420732339356@463a07221176 \
--to=ffmpeg-devel@ffmpeg.org \
--cc=code@ffmpeg.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Git Inbox Mirror of the ffmpeg-devel mailing list - see https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
This inbox may be cloned and mirrored by anyone:
git clone --mirror https://master.gitmailbox.com/ffmpegdev/0 ffmpegdev/git/0.git
# If you have public-inbox 1.1+ installed, you may
# initialize and index your mirror using the following commands:
public-inbox-init -V2 ffmpegdev ffmpegdev/ https://master.gitmailbox.com/ffmpegdev \
ffmpegdev@gitmailbox.com
public-inbox-index ffmpegdev
Example config snippet for mirrors.
AGPL code for this site: git clone https://public-inbox.org/public-inbox.git