[FFmpeg-devel] [PATCH] avformat/iamf_parse: Fix heap-buffer-overflow (PR #21107)

[FFmpeg-devel] [PATCH] avformat/iamf_parse: Fix heap-buffer-overflow (PR #21107)

[oliverchang via ffmpeg-devel]

PR #21107 opened by oliverchang
URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/21107
Patch URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/21107.patch

The ASAN heap-buffer-overflow in `scalable_channel_layout_config` at
`libavformat/iamf_parse.c:435` was caused by an unchecked assumption
that the channel layout of a scalable audio layer is a superset of the
previous layer's channel layout.

`scalable_channel_layout_config` constructs a channel layout map by
copying channels from the previous layer and adding new ones. The memory
allocation is based on the target `loudspeaker_layout`. However, if the
target layout doesn't encompass all previous channels (e.g., Mono to
Stereo), copying previous channels followed by adding current ones could
exceed the allocated size, causing a heap buffer overflow.

This commit adds a check to ensure the previous layer's channel layout
is a subset of the current layer's layout by comparing their masks. If
the condition isn't met, `AVERROR_INVALIDDATA` is returned.

Fixes: https://issues.oss-fuzz.com/issues/464965414


>From 6b353995bea2f39dbb751ba868e156b4dd94e8a8 Mon Sep 17 00:00:00 2001
From: Oliver Chang <ochang@google.com>
Date: Fri, 5 Dec 2025 05:30:29 +0000
Subject: [PATCH] avformat/iamf_parse: Fix heap-buffer-overflow

The ASAN heap-buffer-overflow in `scalable_channel_layout_config` at
`libavformat/iamf_parse.c:435` was caused by an unchecked assumption
that the channel layout of a scalable audio layer is a superset of the
previous layer's channel layout.

`scalable_channel_layout_config` constructs a channel layout map by
copying channels from the previous layer and adding new ones. The memory
allocation is based on the target `loudspeaker_layout`. However, if the
target layout doesn't encompass all previous channels (e.g., Mono to
Stereo), copying previous channels followed by adding current ones could
exceed the allocated size, causing a heap buffer overflow.

This commit adds a check to ensure the previous layer's channel layout
is a subset of the current layer's layout by comparing their masks. If
the condition isn't met, `AVERROR_INVALIDDATA` is returned.

Fixes: https://issues.oss-fuzz.com/issues/464965414
---
 libavformat/iamf_parse.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/libavformat/iamf_parse.c b/libavformat/iamf_parse.c
index 597d800be0..3bb55eb0e9 100644
--- a/libavformat/iamf_parse.c
+++ b/libavformat/iamf_parse.c
@@ -399,8 +399,12 @@ static int scalable_channel_layout_config(void *s, AVIOContext *pb,
             av_channel_layout_copy(&ch_layout, &ff_iamf_expanded_scalable_ch_layouts[expanded_loudspeaker_layout]);
         } else if (loudspeaker_layout < 10) {
             av_channel_layout_copy(&ch_layout, &ff_iamf_scalable_ch_layouts[loudspeaker_layout]);
-            if (i)
-                ch_layout.u.mask &= ~av_channel_layout_subset(&audio_element->element->layers[i-1]->ch_layout, UINT64_MAX);
+            if (i) {
+                uint64_t mask = av_channel_layout_subset(&audio_element->element->layers[i-1]->ch_layout, UINT64_MAX);
+                if ((ch_layout.u.mask & mask) != mask)
+                    return AVERROR_INVALIDDATA;
+                ch_layout.u.mask &= ~mask;
+            }
         } else
             ch_layout = (AVChannelLayout){ .order = AV_CHANNEL_ORDER_UNSPEC,
                                                           .nb_channels = substream_count +
-- 
2.49.1

_______________________________________________
ffmpeg-devel mailing list -- ffmpeg-devel@ffmpeg.org
To unsubscribe send an email to ffmpeg-devel-leave@ffmpeg.org

[FFmpeg-devel] Re: [PATCH] avformat/iamf_parse: Fix heap-buffer-overflow (PR #21107)

[James Almer via ffmpeg-devel]

[-- Attachment #1.1.1: Type: text/plain, Size: 3968 bytes --]

On 12/5/2025 2:33 AM, oliverchang via ffmpeg-devel wrote:
> PR #21107 opened by oliverchang
> URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/21107
> Patch URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/21107.patch
> 
> The ASAN heap-buffer-overflow in `scalable_channel_layout_config` at
> `libavformat/iamf_parse.c:435` was caused by an unchecked assumption
> that the channel layout of a scalable audio layer is a superset of the
> previous layer's channel layout.
> 
> `scalable_channel_layout_config` constructs a channel layout map by
> copying channels from the previous layer and adding new ones. The memory
> allocation is based on the target `loudspeaker_layout`. However, if the
> target layout doesn't encompass all previous channels (e.g., Mono to
> Stereo), copying previous channels followed by adding current ones could
> exceed the allocated size, causing a heap buffer overflow.
> 
> This commit adds a check to ensure the previous layer's channel layout
> is a subset of the current layer's layout by comparing their masks. If
> the condition isn't met, `AVERROR_INVALIDDATA` is returned.
> 
> Fixes: https://issues.oss-fuzz.com/issues/464965414
> 
> 
>>From 6b353995bea2f39dbb751ba868e156b4dd94e8a8 Mon Sep 17 00:00:00 2001
> From: Oliver Chang <ochang@google.com>
> Date: Fri, 5 Dec 2025 05:30:29 +0000
> Subject: [PATCH] avformat/iamf_parse: Fix heap-buffer-overflow
> 
> The ASAN heap-buffer-overflow in `scalable_channel_layout_config` at
> `libavformat/iamf_parse.c:435` was caused by an unchecked assumption
> that the channel layout of a scalable audio layer is a superset of the
> previous layer's channel layout.
> 
> `scalable_channel_layout_config` constructs a channel layout map by
> copying channels from the previous layer and adding new ones. The memory
> allocation is based on the target `loudspeaker_layout`. However, if the
> target layout doesn't encompass all previous channels (e.g., Mono to
> Stereo), copying previous channels followed by adding current ones could
> exceed the allocated size, causing a heap buffer overflow.
> 
> This commit adds a check to ensure the previous layer's channel layout
> is a subset of the current layer's layout by comparing their masks. If
> the condition isn't met, `AVERROR_INVALIDDATA` is returned.
> 
> Fixes: https://issues.oss-fuzz.com/issues/464965414
> ---
>   libavformat/iamf_parse.c | 8 ++++++--
>   1 file changed, 6 insertions(+), 2 deletions(-)
> 
> diff --git a/libavformat/iamf_parse.c b/libavformat/iamf_parse.c
> index 597d800be0..3bb55eb0e9 100644
> --- a/libavformat/iamf_parse.c
> +++ b/libavformat/iamf_parse.c
> @@ -399,8 +399,12 @@ static int scalable_channel_layout_config(void *s, AVIOContext *pb,
>               av_channel_layout_copy(&ch_layout, &ff_iamf_expanded_scalable_ch_layouts[expanded_loudspeaker_layout]);
>           } else if (loudspeaker_layout < 10) {
>               av_channel_layout_copy(&ch_layout, &ff_iamf_scalable_ch_layouts[loudspeaker_layout]);
> -            if (i)
> -                ch_layout.u.mask &= ~av_channel_layout_subset(&audio_element->element->layers[i-1]->ch_layout, UINT64_MAX);
> +            if (i) {
> +                uint64_t mask = av_channel_layout_subset(&audio_element->element->layers[i-1]->ch_layout, UINT64_MAX);
> +                if ((ch_layout.u.mask & mask) != mask)
> +                    return AVERROR_INVALIDDATA;
> +                ch_layout.u.mask &= ~mask;
> +            }
>           } else
>               ch_layout = (AVChannelLayout){ .order = AV_CHANNEL_ORDER_UNSPEC,
>                                                             .nb_channels = substream_count +

This breaks demuxing 
https://github.com/AOMediaCodec/libiamf/blob/main/tests/test_000230.iamf 
because the C channel is not in the second scalable layout (FL+FR).

The logic in the function is evidently flawed, so I'll have give it 
another look.


[-- Attachment #1.2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 495 bytes --]

[-- Attachment #2: Type: text/plain, Size: 163 bytes --]

_______________________________________________
ffmpeg-devel mailing list -- ffmpeg-devel@ffmpeg.org
To unsubscribe send an email to ffmpeg-devel-leave@ffmpeg.org