[FFmpeg-devel] [PATCH] libavformat: fix rtpdec_av1 regarding OBU size (PR #20732)

[Chris Hodges via ffmpeg-devel <ffmpeg-devel@ffmpeg.org>]

PR #20732 opened by Chris Hodges (platon42)
URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/20732
Patch URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/20732.patch

Fix the AV1 RTP depacketizer for instances where the OBU elements
did not have their OBU size fields removed according to the
recommendation (SHOULD) of the AV1 RTP specification.

Roger Hardiman courteously reported that the depacketizer doesn't
work correctly for fragmented OBU elements with their OBU size
fields still present and will incorrectly state that the
continuation of such a fragment was unexpected, because the
frag_obu_size field was used as a state and only incremented
when the size field needed to be updated on OBU size field
restoration. This patch solves the slip.

Signed-off-by: Chris Hodges <chris.hodges@axis.com>


>From 6b0ccace6b3317d50804515bdcd999a0e47236bf Mon Sep 17 00:00:00 2001
From: Chris Hodges <chrishod@axis.com>
Date: Tue, 21 Oct 2025 11:41:06 +0200
Subject: [PATCH] libavformat: fix rtpdec_av1 regarding OBU size

Fix the AV1 RTP depacketizer for instances where the OBU elements
did not have their OBU size fields removed according to the
recommendation (SHOULD) of the AV1 RTP specification.

Roger Hardiman courteously reported that the depacketizer doesn't
work correctly for fragmented OBU elements with their OBU size
fields still present and will incorrectly state that the
continuation of such a fragment was unexpected, because the
frag_obu_size field was used as a state and only incremented
when the size field needed to be updated on OBU size field
restoration. This patch solves the slip.

Change-Id: I95af8cc89862e8ecf79aabcf029dd95a20dfd7ad
---
 libavformat/rtpdec_av1.c | 49 ++++++++++++++++++++--------------------
 1 file changed, 24 insertions(+), 25 deletions(-)

diff --git a/libavformat/rtpdec_av1.c b/libavformat/rtpdec_av1.c
index 7cfc83b03c..dda17be4a1 100644
--- a/libavformat/rtpdec_av1.c
+++ b/libavformat/rtpdec_av1.c
@@ -329,6 +329,8 @@ static int av1_handle_packet(AVFormatContext *ctx, PayloadContext *data,
             num_lebs = write_leb(pkt->data + pktpos, obu_payload_size);
             data->frag_lebs_res = num_lebs;
             pktpos += num_lebs;
+        } else if (!is_frag_cont) {
+            data->frag_lebs_res = 0;
         }
         // copy verbatim or without above header size patch
         memcpy(pkt->data + pktpos, buf_ptr, obu_payload_size);
@@ -339,35 +341,32 @@ static int av1_handle_packet(AVFormatContext *ctx, PayloadContext *data,
         // if we were handling a fragmented packet and this was the last
         // fragment, correct OBU size field
         if (data->frag_obu_size && (rem_pkt_size || !is_last_fragmented)) {
-            uint32_t final_obu_size = data->frag_obu_size + obu_size - data->frag_header_size;
-            uint8_t *lebptr = pkt->data + data->frag_pkt_leb_pos;
-            num_lebs = calc_leb_size(final_obu_size);
+            if (data->frag_lebs_res) {
+                uint32_t final_obu_size = data->frag_obu_size + obu_size - data->frag_header_size;
+                uint8_t *lebptr = pkt->data + data->frag_pkt_leb_pos;
+                num_lebs = calc_leb_size(final_obu_size);
 
-            // check if we had allocated enough LEB bytes in header,
-            // otherwise make some extra space
-            if (num_lebs > data->frag_lebs_res) {
-                int extra_bytes = num_lebs - data->frag_lebs_res;
-                if ((result = av_grow_packet(pkt, extra_bytes)) < 0)
-                    return result;
-                // update pointer in case buffer address changed
-                lebptr = pkt->data + data->frag_pkt_leb_pos;
-                // move existing data for OBU back a bit
-                memmove(lebptr + extra_bytes, lebptr,
-                        pkt->size - extra_bytes - data->frag_pkt_leb_pos);
-                // move pktpos further down for following OBUs in same packet.
-                pktpos += extra_bytes;
+                // check if we had allocated enough LEB bytes in header,
+                // otherwise make some extra space
+                if (num_lebs > data->frag_lebs_res) {
+                    int extra_bytes = num_lebs - data->frag_lebs_res;
+                    if ((result = av_grow_packet(pkt, extra_bytes)) < 0)
+                        return result;
+                    // update pointer in case buffer address changed
+                    lebptr = pkt->data + data->frag_pkt_leb_pos;
+                    // move existing data for OBU back a bit
+                    memmove(lebptr + extra_bytes, lebptr,
+                            pkt->size - extra_bytes - data->frag_pkt_leb_pos);
+                    // move pktpos further down for following OBUs in same packet.
+                    pktpos += extra_bytes;
+                }
+
+                // update OBU size field
+                write_leb(lebptr, final_obu_size);
             }
-
-            // update OBU size field
-            write_leb(lebptr, final_obu_size);
-
             data->frag_obu_size = 0; // signal end of fragment
         } else if (is_last_fragmented && !rem_pkt_size) {
-            // add to total OBU size, so we can fix that in OBU header
-            // (but only if the OBU size was missing!)
-            if (needs_size_field || data->frag_obu_size) {
-                data->frag_obu_size += obu_size;
-            }
+            data->frag_obu_size += obu_size;
             // fragment not yet finished!
             result = -1;
         }
-- 
2.49.1

_______________________________________________
ffmpeg-devel mailing list -- ffmpeg-devel@ffmpeg.org
To unsubscribe send an email to ffmpeg-devel-leave@ffmpeg.org