[FFmpeg-devel] [PATCH v1] lavc/vvc: Avoid UB in DB strength derivation for PLT CUs

[FFmpeg-devel] [PATCH v1] lavc/vvc: Avoid UB in DB strength derivation for PLT CUs

[Frank Plowman]

When called for palette-predicted CUs, boundary_strength could cause
undefined behaviour due to accessing uninitialised motion information.
The spec doesn't include this, but in the reference software it seems
the deblock strength is always set to 0 for palette CUs due to some
implementation details: perhaps this is a spec issue?

Signed-off-by: Frank Plowman <post@frankplowman.com>
---
 libavcodec/vvc/filter.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/libavcodec/vvc/filter.c b/libavcodec/vvc/filter.c
index e3886d008e..3815668bcf 100644
--- a/libavcodec/vvc/filter.c
+++ b/libavcodec/vvc/filter.c
@@ -385,6 +385,9 @@ static int boundary_strength(const VVCLocalContext *lc, const MvField *curr, con
 {
     RefPicList *rpl = lc->sc->rpl;
 
+    if (curr->pred_flag == PF_PLT)
+        return 0;
+
     if (curr->pred_flag == PF_IBC)
         return FFABS(neigh->mv[0].x - curr->mv[0].x) >= 8 || FFABS(neigh->mv[0].y - curr->mv[0].y) >= 8;
 
-- 
2.47.0

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".