[FFmpeg-devel] [PATCH] avcodec/nvdec_mpeg2: fix order of quant matrix coefficients

[FFmpeg-devel] [PATCH] avcodec/nvdec_mpeg2: fix order of quant matrix coefficients

[Hendrik Leppkes]

mpeg2dec stores them permutated for the IDCT, nvdec expects them in
plain raster order.
---
 libavcodec/nvdec_mpeg12.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/libavcodec/nvdec_mpeg12.c b/libavcodec/nvdec_mpeg12.c
index e10735587d..3b9ff60734 100644
--- a/libavcodec/nvdec_mpeg12.c
+++ b/libavcodec/nvdec_mpeg12.c
@@ -83,8 +83,9 @@ static int nvdec_mpeg12_start_frame(AVCodecContext *avctx, const uint8_t *buffer
     };
 
     for (i = 0; i < 64; ++i) {
-        ppc->QuantMatrixIntra[i] = s->intra_matrix[i];
-        ppc->QuantMatrixInter[i] = s->inter_matrix[i];
+        int n = s->idsp.idct_permutation[i];
+        ppc->QuantMatrixIntra[i] = s->intra_matrix[n];
+        ppc->QuantMatrixInter[i] = s->inter_matrix[n];
     }
 
     return 0;
-- 
2.40.1.windows.1

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] avcodec/nvdec_mpeg2: fix order of quant matrix coefficients

[Hendrik Leppkes]

On Mon, May 15, 2023 at 12:51 PM Hendrik Leppkes <h.leppkes@gmail.com> wrote:
>
> mpeg2dec stores them permutated for the IDCT, nvdec expects them in
> plain raster order.
> ---
>  libavcodec/nvdec_mpeg12.c | 5 +++--
>  1 file changed, 3 insertions(+), 2 deletions(-)
>
> diff --git a/libavcodec/nvdec_mpeg12.c b/libavcodec/nvdec_mpeg12.c
> index e10735587d..3b9ff60734 100644
> --- a/libavcodec/nvdec_mpeg12.c
> +++ b/libavcodec/nvdec_mpeg12.c
> @@ -83,8 +83,9 @@ static int nvdec_mpeg12_start_frame(AVCodecContext *avctx, const uint8_t *buffer
>      };
>
>      for (i = 0; i < 64; ++i) {
> -        ppc->QuantMatrixIntra[i] = s->intra_matrix[i];
> -        ppc->QuantMatrixInter[i] = s->inter_matrix[i];
> +        int n = s->idsp.idct_permutation[i];
> +        ppc->QuantMatrixIntra[i] = s->intra_matrix[n];
> +        ppc->QuantMatrixInter[i] = s->inter_matrix[n];
>      }
>
>      return 0;
> --
> 2.40.1.windows.1
>

Apparently mpeg4 and VDPAU have the same issue. I can test and fix
mpeg4, but do not have VDPAU setup, so .. untested commits incoming?

- Hendrik
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH 1/4] avcodec/nvdec_mpeg2: fix order of quant matrix coefficients

[Hendrik Leppkes]

The matrix coefficients are stored permutated for the IDCT,
rather then in plain raster order, and need to be un-permutated
for the hardware.
---
 libavcodec/nvdec_mpeg12.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/libavcodec/nvdec_mpeg12.c b/libavcodec/nvdec_mpeg12.c
index e10735587d..3b9ff60734 100644
--- a/libavcodec/nvdec_mpeg12.c
+++ b/libavcodec/nvdec_mpeg12.c
@@ -83,8 +83,9 @@ static int nvdec_mpeg12_start_frame(AVCodecContext *avctx, const uint8_t *buffer
     };
 
     for (i = 0; i < 64; ++i) {
-        ppc->QuantMatrixIntra[i] = s->intra_matrix[i];
-        ppc->QuantMatrixInter[i] = s->inter_matrix[i];
+        int n = s->idsp.idct_permutation[i];
+        ppc->QuantMatrixIntra[i] = s->intra_matrix[n];
+        ppc->QuantMatrixInter[i] = s->inter_matrix[n];
     }
 
     return 0;
-- 
2.40.1.windows.1

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH 2/4] avcodec/nvdec_mpeg4: fix order of quant matrix coefficients

[Hendrik Leppkes]

The matrix coefficients are stored permutated for the IDCT,
rather then in plain raster order, and need to be un-permutated
for the hardware.
---
 libavcodec/nvdec_mpeg4.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/libavcodec/nvdec_mpeg4.c b/libavcodec/nvdec_mpeg4.c
index eac138cc38..c193f6b6e4 100644
--- a/libavcodec/nvdec_mpeg4.c
+++ b/libavcodec/nvdec_mpeg4.c
@@ -88,8 +88,9 @@ static int nvdec_mpeg4_start_frame(AVCodecContext *avctx, const uint8_t *buffer,
     };
 
     for (i = 0; i < 64; ++i) {
-        ppc->QuantMatrixIntra[i] = s->intra_matrix[i];
-        ppc->QuantMatrixInter[i] = s->inter_matrix[i];
+        int n = s->idsp.idct_permutation[i];
+        ppc->QuantMatrixIntra[i] = s->intra_matrix[n];
+        ppc->QuantMatrixInter[i] = s->inter_matrix[n];
     }
 
     // We need to pass the full frame buffer and not just the slice
-- 
2.40.1.windows.1

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH 3/4] avcodec/vdpau_mpeg12: fix order of quant matrix coefficients

[Hendrik Leppkes]

The matrix coefficients are stored permutated for the IDCT,
rather then in plain raster order, and need to be un-permutated
for the hardware.
---
 libavcodec/vdpau_mpeg12.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/libavcodec/vdpau_mpeg12.c b/libavcodec/vdpau_mpeg12.c
index 354239cad5..79007aa1a8 100644
--- a/libavcodec/vdpau_mpeg12.c
+++ b/libavcodec/vdpau_mpeg12.c
@@ -75,8 +75,9 @@ static int vdpau_mpeg_start_frame(AVCodecContext *avctx,
     info->f_code[1][0]               = s->mpeg_f_code[1][0];
     info->f_code[1][1]               = s->mpeg_f_code[1][1];
     for (i = 0; i < 64; ++i) {
-        info->intra_quantizer_matrix[i]     = s->intra_matrix[i];
-        info->non_intra_quantizer_matrix[i] = s->inter_matrix[i];
+        int n = s->idsp.idct_permutation[i];
+        info->intra_quantizer_matrix[i]     = s->intra_matrix[n];
+        info->non_intra_quantizer_matrix[i] = s->inter_matrix[n];
     }
 
     return ff_vdpau_common_start_frame(pic_ctx, buffer, size);
-- 
2.40.1.windows.1

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH 4/4] avcodec/vdpau_mpeg4: fix order of quant matrix coefficients

[Hendrik Leppkes]

The matrix coefficients are stored permutated for the IDCT,
rather then in plain raster order, and need to be un-permutated
for the hardware.
---
 libavcodec/vdpau_mpeg4.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/libavcodec/vdpau_mpeg4.c b/libavcodec/vdpau_mpeg4.c
index 6e082eefc6..1211b1df2c 100644
--- a/libavcodec/vdpau_mpeg4.c
+++ b/libavcodec/vdpau_mpeg4.c
@@ -74,8 +74,9 @@ static int vdpau_mpeg4_start_frame(AVCodecContext *avctx,
     info->alternate_vertical_scan_flag      = s->alternate_scan;
     info->top_field_first                   = s->top_field_first;
     for (i = 0; i < 64; ++i) {
-        info->intra_quantizer_matrix[i]     = s->intra_matrix[i];
-        info->non_intra_quantizer_matrix[i] = s->inter_matrix[i];
+        int n = s->idsp.idct_permutation[i];
+        info->intra_quantizer_matrix[i]     = s->intra_matrix[n];
+        info->non_intra_quantizer_matrix[i] = s->inter_matrix[n];
     }
 
     ff_vdpau_common_start_frame(pic_ctx, buffer, size);
-- 
2.40.1.windows.1

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] avcodec/nvdec_mpeg2: fix order of quant matrix coefficients

[Hendrik Leppkes]

On Mon, May 15, 2023 at 1:30 PM Hendrik Leppkes <h.leppkes@gmail.com> wrote:
>
> Apparently mpeg4 and VDPAU have the same issue. I can test and fix
> mpeg4, but do not have VDPAU setup, so .. untested commits incoming?
>

Tested and confirmed mpeg2 and mpeg4 are fixed for NVDEC, and elenril
tested VDPAU with the same result.
OK'ed by BtbN on IRC.

Going to push later today.

- Hendrik
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".