From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.ffmpeg.org (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTPS id D1C8848E03 for ; Wed, 16 Jul 2025 10:08:59 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.ffmpeg.org (Postfix) with ESMTP id 2D0F368E294; Wed, 16 Jul 2025 13:08:53 +0300 (EEST) Received: from sender-op-o11.zoho.eu (sender-op-o11.zoho.eu [136.143.169.11]) by ffbox0-bg.ffmpeg.org (Postfix) with ESMTPS id 2D7F668E262 for ; Wed, 16 Jul 2025 13:08:46 +0300 (EEST) ARC-Seal: i=1; a=rsa-sha256; t=1752660524; cv=none; d=zohomail.eu; s=zohoarc; b=buHyTvRCzjFd59zXdvGDxawKHD5wvJYYV/zXzxLL0mpL2qwQS+xfz2MpJuTs75Paw4zzRVaPGXWwbiqd9GzWjhXlxfxRpnKec346OncEBczZGRpw7WCCnTe2L6+ecmz7LMzrCfG1wm78sAY1VM5NjKcLQgjue4gbvH3HKqkz93k= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.eu; s=zohoarc; t=1752660524; h=Content-Type:Date:Date:From:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:Subject:To:To:Message-Id:Reply-To:Cc; bh=q4rLO4QMQQR5MVLH4QjGGEbMrJbrTi1QRAPDIJQajww=; b=OQiuDZzSlRu1eTH34DEKWY1mwJtu53y/bi6aVpUmnH/eIISyUsG814UoWLPd9Rhfqp9PiaA3nKCsxVmwQ5so7QqU5BM/tHzFDu8UZdXVOOAcgm4O387nUg89TEnMxzB17uqekj+DOoqPSv3IePKs7AlPacKfRCRnHBOrtfurlTc= ARC-Authentication-Results: i=1; mx.zohomail.eu; dkim=pass header.i=frankplowman.com; spf=pass smtp.mailfrom=post@frankplowman.com; dmarc=pass header.from= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1752660524; s=zmail; d=frankplowman.com; i=post@frankplowman.com; h=Message-ID:Date:Date:MIME-Version:Subject:Subject:To:To:References:From:From:In-Reply-To:Content-Type:Message-Id:Reply-To:Cc; bh=q4rLO4QMQQR5MVLH4QjGGEbMrJbrTi1QRAPDIJQajww=; b=BBa4otupy7U5zU67ttD0rWNjQMhYw8oV46x25HtyD8Vduhoy4d9AZwXz+kDWLX4u EPAGf/zTwsG7iaJyQ3k2VzNfLqGxqxrMJJoVPyT9tIhMVPAJMGLkqzx3VQOlnHgFNOs Rbk8Rxp/Q6TIyOVr32k4mTZErmhKmpGsaLK7bPq0= Received: by mx.zoho.eu with SMTPS id 1752660523008436.37664048586726; Wed, 16 Jul 2025 12:08:43 +0200 (CEST) Message-ID: Date: Wed, 16 Jul 2025 11:08:42 +0100 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Content-Language: en-GB To: ffmpeg-devel@ffmpeg.org References: <20250715184905.25488-1-post@frankplowman.com> <20250715184905.25488-3-post@frankplowman.com> From: Frank Plowman In-Reply-To: <20250715184905.25488-3-post@frankplowman.com> X-Zoho-Virus-Status: 1 X-Zoho-Virus-Status: 1 X-Zoho-AV-Stamp: zmail-av-1.4.2/252.641.34 X-ZohoMailClient: External Subject: Re: [FFmpeg-devel] [PATCH v2 2/2] lavc/vvc: Add max parameter to kth_order_egk_decode X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Content-Type: multipart/mixed; boundary="===============6743079334266062672==" Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --===============6743079334266062672== Content-Language: en-GB Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="------------LIhm0Ug6v1CqeuH3ywA3tCYu" This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --------------LIhm0Ug6v1CqeuH3ywA3tCYu Content-Type: multipart/mixed; boundary="------------Icfi8kojGRribW7xGpmm47E0"; protected-headers="v1" From: Frank Plowman To: ffmpeg-devel@ffmpeg.org Message-ID: Subject: Re: [FFmpeg-devel] [PATCH v2 2/2] lavc/vvc: Add max parameter to kth_order_egk_decode References: <20250715184905.25488-1-post@frankplowman.com> <20250715184905.25488-3-post@frankplowman.com> In-Reply-To: <20250715184905.25488-3-post@frankplowman.com> --------------Icfi8kojGRribW7xGpmm47E0 Content-Type: multipart/mixed; boundary="------------FkHFVIFBNIBtydWMsKGk0S9G" --------------FkHFVIFBNIBtydWMsKGk0S9G Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On 15/07/2025 03:48, Frank Plowman wrote: > Prior to this patch, kth_order_egk_decode could read arbitrarily > large values which then overflowed and caused various issues. > Patch fixes this by making kth_order_egk_decode falliable, > requiring the caller to specify an upper bound and returning an > error if the read value would exceed that bound. >=20 > This patch resolves the same issue as > eb52251c0ab025b6b40b28994bc9dc616813b190, but I think this is the prope= r > fix as it also addresses issues with syntax elements besides > ff_vvc_num_signalled_palette_entries. >=20 > Patch also includes a minor fix in hls_palette_coding, where the > error code returned by palette_subblock_data was previously unchecked. Spotted I forgot to remove the paragraph above from the commit message when splitting this aspect into a separate commit. I (or whoever else pushes) can remove this when commiting. >=20 > Signed-off-by: Frank Plowman > --- > Changes since v1: > * Split change to hls_palette_coding to its own commit. > * Return values from syntax functions in return val, rather than by > modifying pointer parameter. > --- > libavcodec/vvc/cabac.c | 19 ++++++++++++------- > libavcodec/vvc/cabac.h | 6 +++--- > libavcodec/vvc/ctu.c | 30 ++++++++++++++++++------------ > 3 files changed, 33 insertions(+), 22 deletions(-) >=20 > diff --git a/libavcodec/vvc/cabac.c b/libavcodec/vvc/cabac.c > index 6847ce59af..c2dbd46709 100644 > --- a/libavcodec/vvc/cabac.c > +++ b/libavcodec/vvc/cabac.c > @@ -929,7 +929,7 @@ static int truncated_binary_decode(VVCLocalContext = *lc, const int c_max) > } > =20 > // 9.3.3.5 k-th order Exp - Golomb binarization process > -static int kth_order_egk_decode(CABACContext *c, int k) > +static int kth_order_egk_decode(CABACContext *c, int k, const int max)= > { > int bit =3D 1; > int value =3D 0; > @@ -937,6 +937,8 @@ static int kth_order_egk_decode(CABACContext *c, in= t k) > =20 > while (bit) { > bit =3D get_cabac_bypass(c); > + if (max - value < (bit << k)) > + return AVERROR_INVALIDDATA; > value +=3D bit << k++; > } > =20 > @@ -946,6 +948,9 @@ static int kth_order_egk_decode(CABACContext *c, in= t k) > value +=3D symbol; > } > =20 > + if (value > max) > + return AVERROR_INVALIDDATA; > + > return value; > } > =20 > @@ -1377,14 +1382,14 @@ int ff_vvc_intra_chroma_pred_mode(VVCLocalConte= xt *lc) > return (get_cabac_bypass(&lc->ep->cc) << 1) | get_cabac_bypass(&lc= ->ep->cc); > } > =20 > -int ff_vvc_palette_predictor_run(VVCLocalContext *lc) > +int ff_vvc_palette_predictor_run(VVCLocalContext *lc, const int max) > { > - return kth_order_egk_decode(&lc->ep->cc, 0); > + return kth_order_egk_decode(&lc->ep->cc, 0, max); > } > =20 > -int ff_vvc_num_signalled_palette_entries(VVCLocalContext *lc) > +int ff_vvc_num_signalled_palette_entries(VVCLocalContext *lc, const in= t max) > { > - return kth_order_egk_decode(&lc->ep->cc, 0); > + return kth_order_egk_decode(&lc->ep->cc, 0, max); > } > =20 > int ff_vvc_new_palette_entries(VVCLocalContext *lc, const int bit_dept= h) > @@ -1424,9 +1429,9 @@ int ff_vvc_palette_idx_idc(VVCLocalContext *lc, c= onst int max_palette_index, con > return truncated_binary_decode(lc, max_palette_index - adjust); > } > =20 > -int ff_vvc_palette_escape_val(VVCLocalContext *lc) > +int ff_vvc_palette_escape_val(VVCLocalContext *lc, const int max) > { > - return kth_order_egk_decode(&lc->ep->cc, 5); > + return kth_order_egk_decode(&lc->ep->cc, 5, max); > } > =20 > int ff_vvc_general_merge_flag(VVCLocalContext *lc) > diff --git a/libavcodec/vvc/cabac.h b/libavcodec/vvc/cabac.h > index 972890317e..6a0e713d19 100644 > --- a/libavcodec/vvc/cabac.h > +++ b/libavcodec/vvc/cabac.h > @@ -81,15 +81,15 @@ int ff_vvc_intra_luma_mpm_remainder(VVCLocalContext= *lc); > int ff_vvc_cclm_mode_flag(VVCLocalContext *lc); > int ff_vvc_cclm_mode_idx(VVCLocalContext *lc); > int ff_vvc_intra_chroma_pred_mode(VVCLocalContext *lc); > -int ff_vvc_palette_predictor_run(VVCLocalContext *lc); > -int ff_vvc_num_signalled_palette_entries(VVCLocalContext *lc); > +int ff_vvc_palette_predictor_run(VVCLocalContext *lc, const int max); > +int ff_vvc_num_signalled_palette_entries(VVCLocalContext *lc, const in= t max); > int ff_vvc_new_palette_entries(VVCLocalContext *lc, int bit_dpeth); > bool ff_vvc_palette_escape_val_present_flag(VVCLocalContext *lc); > bool ff_vvc_palette_transpose_flag(VVCLocalContext *lc); > bool ff_vvc_run_copy_flag(VVCLocalContext *lc, int prev_run_type, int = prev_run_position, int cur_pos); > bool ff_vvc_copy_above_palette_indices_flag(VVCLocalContext *lc); > int ff_vvc_palette_idx_idc(VVCLocalContext *lc, int max_palette_index,= bool adjust); > -int ff_vvc_palette_escape_val(VVCLocalContext *lc); > +int ff_vvc_palette_escape_val(VVCLocalContext *lc, const int max); > =20 > //inter > int ff_vvc_general_merge_flag(VVCLocalContext *lc); > diff --git a/libavcodec/vvc/ctu.c b/libavcodec/vvc/ctu.c > index 35c18e78f6..9f875d0a20 100644 > --- a/libavcodec/vvc/ctu.c > +++ b/libavcodec/vvc/ctu.c > @@ -1857,16 +1857,16 @@ static int palette_predicted(VVCLocalContext *l= c, const bool local_dual_tree, in > } > =20 > for (int i =3D 0; i < predictor_size && nb_predicted < max_entries= ; i++) { > - const int run =3D ff_vvc_palette_predictor_run(lc); > + const int run =3D ff_vvc_palette_predictor_run(lc, predictor_s= ize - i); > + if (run < 0) > + return run; > + > if (run =3D=3D 1) > break; > =20 > if (run > 1) > i +=3D run - 1; > =20 > - if (i >=3D predictor_size) > - return AVERROR_INVALIDDATA; > - > predictor_reused[i] =3D true; > for (int c =3D start; c < end; c++) > cu->plt[c].entries[nb_predicted] =3D lc->ep->pp[c].entries= [i]; > @@ -1885,12 +1885,17 @@ static int palette_signaled(VVCLocalContext *lc= , const bool local_dual_tree, > const VVCSPS *sps =3D lc->fc->ps.sps; > CodingUnit *cu =3D lc->cu; > const int nb_predicted =3D cu->plt[start].size; > - const int nb_signaled =3D nb_predicted < max_entries ? ff_vvc_= num_signalled_palette_entries(lc) : 0; > - const int size =3D nb_predicted + nb_signaled; > const bool dual_tree_luma =3D local_dual_tree && cu->tree_type =3D= =3D DUAL_TREE_LUMA; > + int nb_signaled, size; > =20 > - if (size > max_entries || nb_signaled < 0) > - return AVERROR_INVALIDDATA; > + if (nb_predicted < max_entries) { > + nb_signaled =3D ff_vvc_num_signalled_palette_entries(lc, max_e= ntries - nb_predicted); > + if (nb_signaled < 0) > + return nb_signaled; > + } else > + nb_signaled =3D 0; > + > + size =3D nb_predicted + nb_signaled; > =20 > for (int c =3D start; c < end; c++) { > Palette *plt =3D cu->plt + c; > @@ -2052,10 +2057,11 @@ static int palette_subblock_data(VVCLocalContex= t *lc, > if (!(xc & hs) && !(yc & vs)) { > const int v =3D PALETTE_INDEX(xc, yc); > if (v =3D=3D esc) { > - const int coeff =3D ff_vvc_palette_escape_val(lc);= > - if (coeff >=3D (1U << sps->bit_depth)) > - return AVERROR_INVALIDDATA; > - const int pixel =3D av_clip_intp2(RSHIFT(coeff * s= cale, 6), sps->bit_depth); > + int pixel; > + const int coeff =3D ff_vvc_palette_escape_val(lc, = (1 << sps->bit_depth) - 1); > + if (coeff < 0) > + return coeff; > + pixel =3D av_clip_intp2(RSHIFT(coeff * scale, 6), = sps->bit_depth); > PALETTE_SET_PIXEL(xc, yc, pixel); > } else { > PALETTE_SET_PIXEL(xc, yc, plt->entries[v]); --------------FkHFVIFBNIBtydWMsKGk0S9G Content-Type: application/pgp-keys; name="OpenPGP_0x03A84C6A098F2C6B.asc" Content-Disposition: attachment; filename="OpenPGP_0x03A84C6A098F2C6B.asc" Content-Description: OpenPGP public key Content-Transfer-Encoding: quoted-printable -----BEGIN PGP PUBLIC KEY BLOCK----- xjMEaDsehBYJKwYBBAHaRw8BAQdAtPY6tq5662mjGsrl9wixeTHtvaO/qPBio/aX SmF+pjnNJUZyYW5rIFBsb3dtYW4gPHBvc3RAZnJhbmtwbG93bWFuLmNvbT7ClgQT FgoAPgIbAQIXgAUJAeEzrBYhBDTiSNa330dpcMczBAOoTGoJjyxrBQJoOyIVBQsJ CAcCBhUKCQgLAgQWAgMBAh4BAAoJEAOoTGoJjyxrMhUBAKT3f80IXiufxCSU1rve Xp5BKwvNbY6szRdR/m21m9GAAQCGhR6t64PlMspXLtwWq9o1vjKzjWH+Bgw+UybQ CG2zDM4zBGg7HsQWCSsGAQQB2kcPAQEHQPmGNV50IFIJBy9F60ZIxe5my91pL/Sl LUWq9Z5DHBsYwsA1BBgWCgAmFiEENOJI1rffR2lwxzMEA6hMagmPLGsFAmg7HsQC GwIFCQHhM4AAgQkQA6hMagmPLGt2IAQZFgoAHRYhBEh0l9eKWUIIcnmv1LvizNAX Sd6KBQJoOx7EAAoJELvizNAXSd6KtkEBAP0jS/nBlqaVJq0iNeNDk+PnKkNBmC9p i4SuwFD71nQfAP9eWO32RXbRoIwvbrdsalWxWYmqSOdIdQE51qGMsJUuAjtIAQDA 6o6O2o2pHnWWdSqKs7Zkp4BImBD+kztmzA0Du3vJeQD/TR+qrBCXaG1FCDKn9c+Q XQNtHTxI9c6dCywjkQljOAzOOARoOyM7EgorBgEEAZdVAQUBAQdAKS1xy5312Agk deedJEXplopXco5pyANBz/edWeb7zEkDAQgHwn4EGBYKACYWIQQ04kjWt99HaXDH MwQDqExqCY8sawUCaDsjOwIbDAUJAeEzgAAKCRADqExqCY8saxvgAP9zskNI1Sti DXM4RFMoJb1aQ5TPToQZSSkGhw7JyMbLpgD/VCG7BOFjKgJGtXYEb5zpmRzNzdEB QkLsgE/vAD9GXQs=3D =3DReZ+ -----END PGP PUBLIC KEY BLOCK----- --------------FkHFVIFBNIBtydWMsKGk0S9G-- --------------Icfi8kojGRribW7xGpmm47E0-- --------------LIhm0Ug6v1CqeuH3ywA3tCYu Content-Type: application/pgp-signature; name="OpenPGP_signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="OpenPGP_signature.asc" -----BEGIN PGP SIGNATURE----- wnsEABYIACMWIQRIdJfXillCCHJ5r9S74szQF0neigUCaHd6KgUDAAAAAAAKCRC74szQF0neindg AQCn7IMzkYiSPGDX8z3QQiFrTLLukdTnLXo4if00K9RVhAEAghZ1PPHeaJAouiPM1zMGhnt10MJx 2pKYmHEbcgD9lAk= =Km8w -----END PGP SIGNATURE----- --------------LIhm0Ug6v1CqeuH3ywA3tCYu-- --===============6743079334266062672== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". --===============6743079334266062672==--