[FFmpeg-devel] [PATCH] avfilter/x86/vf_noise: Use unaligned access (PR #21163)

[FFmpeg-devel] [PATCH] avfilter/x86/vf_noise: Use unaligned access (PR #21163)

[michaelni via ffmpeg-devel]

PR #21163 opened by michaelni
URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/21163
Patch URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/21163.patch

Regression since: 3ba570de8b666f5ae274b99d33cf4d4628cc385b (port from MMX to SSE2).

The SSE2 inline asm in libavfilter/x86/vf_noise.c (line_noise_sse2 and
line_noise_avg_sse2) uses aligned loads/stores (movdqa, movntdq) but never
checks pointer alignment. When the filter reuses an input frame (common
path when av_frame_is_writable() is true), it may receive misaligned data
from upstream filters that adjust frame->data[i] in place, notably vf_crop:

- vf_crop adjusts plane pointers by arbitrary byte offsets
(frame->data[plane] += ...), so an x offset of 1 on 8-bit formats produces
a 1‑byte misalignment.
- The noise filter then calls the SSE2 path directly on those pointers
without realigning or falling back.

Repro on x86_64/SSE2 (current HEAD at that commit):

./ffmpeg -v error -f lavfi -i testsrc=s=320x240:rate=1 \
-vf "format=yuv420p,crop=w=319:x=1:h=240:exact=1,noise=alls=50" \
-frames:v 1 -f null -

This crashes with SIGSEGV at the aligned load in line_noise_sse2 (movdqa
(%r9,%rax),%xmm0; effective address misaligned by 1 byte).

Impact: denial of service via crafted filtergraphs (e.g., crop + noise).
Applies to planar 8-bit formats where upstream filters can shift data
pointers without reallocating.

Found-by: Pwno OSS Team


From f9244505a596c74269238ab938daf7a7b6c6ee1f Mon Sep 17 00:00:00 2001
From: Ruikai Peng <ruikai@pwno.io>
Date: Thu, 11 Dec 2025 02:53:02 +0100
Subject: [PATCH] avfilter/x86/vf_noise: Use unaligned access
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Regression since: 3ba570de8b666f5ae274b99d33cf4d4628cc385b (port from MMX to SSE2).

The SSE2 inline asm in libavfilter/x86/vf_noise.c (line_noise_sse2 and
line_noise_avg_sse2) uses aligned loads/stores (movdqa, movntdq) but never
checks pointer alignment. When the filter reuses an input frame (common
path when av_frame_is_writable() is true), it may receive misaligned data
from upstream filters that adjust frame->data[i] in place, notably vf_crop:

- vf_crop adjusts plane pointers by arbitrary byte offsets
(frame->data[plane] += ...), so an x offset of 1 on 8-bit formats produces
a 1‑byte misalignment.
- The noise filter then calls the SSE2 path directly on those pointers
without realigning or falling back.

Repro on x86_64/SSE2 (current HEAD at that commit):

./ffmpeg -v error -f lavfi -i testsrc=s=320x240:rate=1 \
-vf "format=yuv420p,crop=w=319:x=1:h=240:exact=1,noise=alls=50" \
-frames:v 1 -f null -

This crashes with SIGSEGV at the aligned load in line_noise_sse2 (movdqa
(%r9,%rax),%xmm0; effective address misaligned by 1 byte).

Impact: denial of service via crafted filtergraphs (e.g., crop + noise).
Applies to planar 8-bit formats where upstream filters can shift data
pointers without reallocating.

Found-by: Pwno OSS Team
---
 libavfilter/x86/vf_noise.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/libavfilter/x86/vf_noise.c b/libavfilter/x86/vf_noise.c
index 95f97b3b06..3edcbd64e5 100644
--- a/libavfilter/x86/vf_noise.c
+++ b/libavfilter/x86/vf_noise.c
@@ -39,7 +39,7 @@ static void line_noise_avg_sse2(uint8_t *dst, const uint8_t *src,
             "movdqu (%1, %%"FF_REG_a"), %%xmm1             \n\t"
             "movdqu (%2, %%"FF_REG_a"), %%xmm2             \n\t"
             "movdqu (%3, %%"FF_REG_a"), %%xmm3             \n\t"
-            "movdqa (%0, %%"FF_REG_a"), %%xmm0             \n\t"
+            "movdqu (%0, %%"FF_REG_a"), %%xmm0             \n\t"
             "paddb              %%xmm2, %%xmm1             \n\t"
             "paddb              %%xmm3, %%xmm1             \n\t"
             "movdqa             %%xmm4, %%xmm5             \n\t"
@@ -59,7 +59,7 @@ static void line_noise_avg_sse2(uint8_t *dst, const uint8_t *src,
             "psraw                  $7, %%xmm3             \n\t"
             "packsswb           %%xmm3, %%xmm1             \n\t"
             "paddb              %%xmm6, %%xmm1             \n\t"
-            "movdqa             %%xmm1, (%4, %%"FF_REG_a") \n\t"
+            "movdqu             %%xmm1, (%4, %%"FF_REG_a") \n\t"
             "add                   $16, %%"FF_REG_a"       \n\t"
             " js 1b                         \n\t"
             :: "r" (src+xmm_len), "r" (shift[0]+xmm_len), "r" (shift[1]+xmm_len), "r" (shift[2]+xmm_len),
@@ -88,12 +88,12 @@ static void line_noise_sse2(uint8_t *dst, const uint8_t *src,
             "packsswb           %%xmm2, %%xmm2             \n\t"
             ".p2align 4                                    \n\t"
             "1:                                            \n\t"
-            "movdqa (%0, %%"FF_REG_a"), %%xmm0             \n\t"
+            "movdqu (%0, %%"FF_REG_a"), %%xmm0             \n\t"
             "movdqu (%1, %%"FF_REG_a"), %%xmm1             \n\t"
             "pxor               %%xmm2, %%xmm0             \n\t"
             "paddsb             %%xmm1, %%xmm0             \n\t"
             "pxor               %%xmm2, %%xmm0             \n\t"
-            "movntdq            %%xmm0, (%2, %%"FF_REG_a") \n\t"
+            "movdqu             %%xmm0, (%2, %%"FF_REG_a") \n\t"
             "add                   $16, %%"FF_REG_a"       \n\t"
             " js                    1b                     \n\t"
             :: "r" (src+xmm_len), "r" (noise+xmm_len), "r" (dst+xmm_len), "g" (-xmm_len)
-- 
2.49.1

_______________________________________________
ffmpeg-devel mailing list -- ffmpeg-devel@ffmpeg.org
To unsubscribe send an email to ffmpeg-devel-leave@ffmpeg.org