[FFmpeg-devel] [PATCH 1/2] lavc/internal: add skip_samples2 field

* [FFmpeg-devel] [PATCH 1/2] lavc/internal: add skip_samples2 field
@ 2023-10-19  2:37 Lynne
       [not found] ` <Nh4mqjI--3-9@lynne.ee-Nh4muoe----9>
  2023-10-19  8:39 ` [FFmpeg-devel] [PATCH 1/2] lavc/internal: add skip_samples2 field Anton Khirnov
  0 siblings, 2 replies; 10+ messages in thread
From: Lynne @ 2023-10-19  2:37 UTC (permalink / raw)
  To: Ffmpeg Devel

[-- Attachment #1: Type: text/plain, Size: 1285 bytes --]

The issue is that avci->skip_samples will be overridden by any side-data.

When operating on raw files (adts, for example), the decoder is free
to decide the amount of samples to skip. Usually, this is the algorithmic
delay of the decoder.

When operating on more complete containers, like ISOBMFF, the amount of
samples to be skipped is recorded and signalled by the encoder.

However, it turns out many encoders have an arbitrary choice of padding
to insert at the start. Normally, they would signal the amount into
the container. But with ISOBMFF, there isn't just a single option -
the format has been extended multiple times, and has multiple ways
to signal padding. In the case of fdkaac-encoded samples, the STTS
is used, rather than the CTTS, which ends up with us leaving the padding
in.

But it's not just containers, as it turns out, most AAC encoders use
an arbitrary amount of padding at the start that may, or may not be
trimmed (usually, it won't be).

Furthermore, AAC has specific amount of algorithmic delay for SBR
operation. This delay is not accounter for anywhere. While it's an
option to skip the samples in the decoder, doing this in decode.c,
along with the rest of the skip adjustments, is a neater way, and
can be extended to other codecs.

Patch attached.


[-- Attachment #2: 0001-lavc-internal-add-skip_samples2-field.patch --]
[-- Type: text/x-diff, Size: 2723 bytes --]

From 9986c7f0c71d944101f1c7fe7b1395ee21e34a8e Mon Sep 17 00:00:00 2001
From: Lynne <dev@lynne.ee>
Date: Thu, 19 Oct 2023 04:28:03 +0200
Subject: [PATCH 1/2] lavc/internal: add skip_samples2 field

The issue is that avci->skip_samples will be overridden by any side-data.

When operating on raw files (adts, for example), the decoder is free
to decide the amount of samples to skip. Usually, this is the algorithmic
delay of the decoder.

When operating on more complete containers, like ISOBMFF, the amount of
samples to be skipped is recorded and signalled by the encoder.

However, it turns out many encoders have an arbitrary choice of padding
to insert at the start. Normally, they would signal the amount into
the container. But with ISOBMFF, there isn't just a single option -
the format has been extended multiple times, and has multiple ways
to signal padding. In the case of fdkaac-encoded samples, the STTS
is used, rather than the CTTS, which ends up with us leaving the padding
in.

But it's not just containers, as it turns out, most AAC encoders use
an arbitrary amount of padding at the start that may, or may not be
trimmed (usually, it won't be).

Furthermore, AAC has specific amount of algorithmic delay for SBR
operation. This delay is not accounter for anywhere. While it's an
option to skip the samples in the decoder, doing this in decode.c,
along with the rest of the skip adjustments, is a neater way, and
can be extended to other codecs.
---
 libavcodec/decode.c   | 2 ++
 libavcodec/internal.h | 9 +++++++++
 2 files changed, 11 insertions(+)

diff --git a/libavcodec/decode.c b/libavcodec/decode.c
index ad39021354..32944a6b6a 100644
--- a/libavcodec/decode.c
+++ b/libavcodec/decode.c
@@ -338,6 +338,8 @@ static int discard_samples(AVCodecContext *avctx, AVFrame *frame, int64_t *disca
         return AVERROR(EAGAIN);
     }
 
+    avci->skip_samples += avci->skip_samples2;
+
     if (avci->skip_samples > 0) {
         if (frame->nb_samples <= avci->skip_samples){
             *discarded_samples += frame->nb_samples;
diff --git a/libavcodec/internal.h b/libavcodec/internal.h
index eb9e0d707c..3d8d4d9a4d 100644
--- a/libavcodec/internal.h
+++ b/libavcodec/internal.h
@@ -115,9 +115,18 @@ typedef struct AVCodecInternal {
 
     /**
      * Number of audio samples to skip at the start of the next decoded frame
+     *
+     * Note: This will be overridden by any side data.
      */
     int skip_samples;
 
+    /**
+     * Additional samples to skip ad the start of the next decoded frame.
+     *
+     * These will be added to any skip amount after taking side data into account.
+     */
+    int skip_samples2;
+
     /**
      * hwaccel-specific private data
      */
-- 
2.42.0


[-- Attachment #3: Type: text/plain, Size: 251 bytes --]

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

^ permalink raw reply	[flat|nested] 10+ messages in thread