Re: [FFmpeg-devel] libavc/libx264: add support to propagate SSE values through encoder stats

["Carotti, Elias via ffmpeg-devel" <ffmpeg-devel@ffmpeg.org>]

[-- Attachment #1: Type: text/plain, Size: 2954 bytes --]

Hi Anton, 

On Tue, 2023-10-10 at 12:54 +0200, Anton Khirnov wrote:
> 
> Quoting Carotti, Elias via ffmpeg-devel (2023-10-02 19:35:09)
> > diff --git a/libavcodec/libx264.c b/libavcodec/libx264.c
> > index 77a9f173b4..4c643c9066 100644
> > --- a/libavcodec/libx264.c
> > +++ b/libavcodec/libx264.c
> > @@ -129,6 +129,8 @@ typedef struct X264Context {
> >      int roi_warned;
> > 
> >      int mb_info;
> > +
> > +    int64_t sse[3];
> 
> The values don't need to be preserved across frames, so might as well
> put this on stack in the block calling
> ff_side_data_set_encoder_stats().

Agreed.

> 
> >  } X264Context;
> > 
> >  static void X264_log(void *p, int level, const char *fmt, va_list
> > args)
> > @@ -726,7 +728,40 @@ FF_ENABLE_DEPRECATION_WARNINGS
> > 
> >      pkt->flags |= AV_PKT_FLAG_KEY*pic_out.b_keyframe;
> >      if (ret) {
> > -        ff_side_data_set_encoder_stats(pkt, (pic_out.i_qpplus1 -
> > 1) * FF_QP2LAMBDA, NULL, 0, pict_type);
> > +        const AVPixFmtDescriptor *pix_desc =
> > av_pix_fmt_desc_get(csp_to_pixfmt(pic_out.img.i_csp));
> > +        int error_count = 0;
> > +        int64_t *errors = NULL;
> > +
> > +        if (ctx->flags & AV_CODEC_FLAG_PSNR) {
> > +            double scale[3] = { 1,
> > +                (1 << pix_desc->log2_chroma_h) * (double)(1 <<
> > pix_desc->log2_chroma_w),
> > +                (1 << pix_desc->log2_chroma_h) * (double)(1 <<
> > pix_desc->log2_chroma_w),
> 
> Any particular reason the cast is on the second value? It looks
> strange.
> 

Just my habit. Fixed.

> > +            };
> > +            double mse;
> > +            int i;
> > +
> > +            error_count = pix_desc->nb_components;
> > +
> > +            av_log(ctx, AV_LOG_DEBUG, "PSNR values from libx264:
> > %.3f %.3f %.3f.\n",
> > +                   pic_out.prop.f_psnr[0], pic_out.prop.f_psnr[1],
> > pic_out.prop.f_psnr[2]);
> > +
> > +            for (i = 0; i < pix_desc->nb_components; ++i) {
> 
> for (int i ....

Agreed.

I also found the - (minus) sign in the mse formula was wrong and I
removed it.
Numbers seem to be coherent with those from libx264.
Please find attached a new patch rebased against the latest master with
the above fixes.

There is an increasing error (over increasing PSNRs and resolutions)
when reconstructing the PSNR from the SSE as computed above due to the
approximations and the roundings back and forth, however it seems to
yield similar values as those computed by libx264.

Best





NICE SRL, viale Monte Grappa 3/5, 20124 Milano, Italia, Registro delle Imprese di Milano Monza Brianza Lodi REA n. 2096882, Capitale Sociale: 10.329,14 EUR i.v., Cod. Fisc. e P.IVA 01133050052, Societa con Socio Unico



[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-avcodec-libx264-Add-the-SSE-computation-for-libx264.patch --]
[-- Type: text/x-patch; name="0001-avcodec-libx264-Add-the-SSE-computation-for-libx264.patch", Size: 2413 bytes --]

From 8c9456042e0cd333702b8e77d3e80767a4c0b7cf Mon Sep 17 00:00:00 2001
From: Elias Carotti <eliascrt _at_ amazon _dot_ it>
Date: Fri, 15 Sep 2023 20:05:43 +0200
Subject: [PATCH] avcodec/libx264: Add the SSE computation for libx264.

Since libx264 only provides a per-frame per-channel PSNR, this is inverted to get back the SSE.
---
 libavcodec/libx264.c | 34 +++++++++++++++++++++++++++++++++-
 1 file changed, 33 insertions(+), 1 deletion(-)

diff --git a/libavcodec/libx264.c b/libavcodec/libx264.c
index 77a9f173b4..85bd870f5d 100644
--- a/libavcodec/libx264.c
+++ b/libavcodec/libx264.c
@@ -726,7 +726,39 @@ FF_ENABLE_DEPRECATION_WARNINGS

     pkt->flags |= AV_PKT_FLAG_KEY*pic_out.b_keyframe;
     if (ret) {
-        ff_side_data_set_encoder_stats(pkt, (pic_out.i_qpplus1 - 1) * FF_QP2LAMBDA, NULL, 0, pict_type);
+        const AVPixFmtDescriptor *pix_desc = av_pix_fmt_desc_get(csp_to_pixfmt(pic_out.img.i_csp));
+        int error_count = 0;
+        int64_t *errors = NULL;
+        int64_t sse[3] = {0};
+
+        if (ctx->flags & AV_CODEC_FLAG_PSNR) {
+            double scale[3] = { 1,
+                (double)(1 << pix_desc->log2_chroma_h) * (1 << pix_desc->log2_chroma_w),
+                (double)(1 << pix_desc->log2_chroma_h) * (1 << pix_desc->log2_chroma_w),
+            };
+
+            error_count = pix_desc->nb_components;
+
+            av_log(ctx, AV_LOG_DEBUG, "PSNR values from libx264: %.3f %.3f %.3f.\n",
+                   pic_out.prop.f_psnr[0], pic_out.prop.f_psnr[1], pic_out.prop.f_psnr[2]);
+
+            for (int i = 0; i < pix_desc->nb_components; ++i) {
+                double max_value = (double)(1 << pix_desc->comp[i].depth) - 1.0;
+                double plane_size = ctx->width * (double)ctx->height / scale[i];
+
+                /* psnr = 10 * log10(max_value * max_value / mse) */
+                double mse = (max_value * max_value) / pow(10, pic_out.prop.f_psnr[i] / 10.0);
+
+                /* SSE = MSE * width * height / scale -> because of possible chroma downsampling */
+                sse[i] = (int64_t)floor(mse * plane_size + .5);
+            };
+
+            errors = sse;
+        }
+
+        ff_side_data_set_encoder_stats(pkt, (pic_out.i_qpplus1 - 1) * FF_QP2LAMBDA,
+                                       errors, error_count, pict_type);
+
         if (wallclock)
             ff_side_data_set_prft(pkt, wallclock);
     }
--
2.34.1


[-- Attachment #3: Type: text/plain, Size: 251 bytes --]

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".