From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <ffmpeg-devel-bounces@ffmpeg.org>
Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100])
by master.gitmailbox.com (Postfix) with ESMTPS id 508F14E81A
for <ffmpegdev@gitmailbox.com>; Mon, 17 Mar 2025 10:46:02 +0000 (UTC)
Received: from [127.0.1.1] (localhost [127.0.0.1])
by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id CB4CE68BDF9;
Mon, 17 Mar 2025 12:44:18 +0200 (EET)
Received: from haasn.dev (haasn.dev [78.46.187.166])
by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 9D26F68B792
for <ffmpeg-devel@ffmpeg.org>; Mon, 17 Mar 2025 12:44:06 +0200 (EET)
Received: from haasn.dev (unknown [10.30.1.1])
by haasn.dev (Postfix) with UTF8SMTP id D8A7243C75;
Mon, 17 Mar 2025 11:44:03 +0100 (CET)
From: Niklas Haas <ffmpeg@haasn.xyz>
To: ffmpeg-devel@ffmpeg.org
Date: Mon, 17 Mar 2025 11:43:55 +0100
Message-ID: <20250317104357.307832-9-ffmpeg@haasn.xyz>
X-Mailer: git-send-email 2.48.1
In-Reply-To: <20250317104357.307832-1-ffmpeg@haasn.xyz>
References: <20250317104357.307832-1-ffmpeg@haasn.xyz>
MIME-Version: 1.0
Subject: [FFmpeg-devel] [PATCH 09/11] tests/swscale: calculate theoretical
expected SSIM
X-BeenThere: ffmpeg-devel@ffmpeg.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: FFmpeg development discussions and patches <ffmpeg-devel.ffmpeg.org>
List-Unsubscribe: <https://ffmpeg.org/mailman/options/ffmpeg-devel>,
<mailto:ffmpeg-devel-request@ffmpeg.org?subject=unsubscribe>
List-Archive: <https://ffmpeg.org/pipermail/ffmpeg-devel>
List-Post: <mailto:ffmpeg-devel@ffmpeg.org>
List-Help: <mailto:ffmpeg-devel-request@ffmpeg.org?subject=help>
List-Subscribe: <https://ffmpeg.org/mailman/listinfo/ffmpeg-devel>,
<mailto:ffmpeg-devel-request@ffmpeg.org?subject=subscribe>
Reply-To: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org>
Cc: Niklas Haas <git@haasn.dev>
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Errors-To: ffmpeg-devel-bounces@ffmpeg.org
Sender: "ffmpeg-devel" <ffmpeg-devel-bounces@ffmpeg.org>
Archived-At: <https://master.gitmailbox.com/ffmpegdev/20250317104357.307832-9-ffmpeg@haasn.xyz/>
List-Archive: <https://master.gitmailbox.com/ffmpegdev/>
List-Post: <mailto:ffmpegdev@gitmailbox.com>
From: Niklas Haas <git@haasn.dev>
We can calculate with some confidence the theoretical expected SSIM
from an "ideal" conversion, by computing the reference SSIM level
for an image dithered with uniformly distributed quatization noise.
This gives us an additional safety net to check for regressions even in
the absence of a reference to compare against.
---
libswscale/tests/swscale.c | 74 +++++++++++++++++++++++++++++++-------
1 file changed, 62 insertions(+), 12 deletions(-)
diff --git a/libswscale/tests/swscale.c b/libswscale/tests/swscale.c
index 47c58524f6..bce495db90 100644
--- a/libswscale/tests/swscale.c
+++ b/libswscale/tests/swscale.c
@@ -99,6 +99,29 @@ static void exit_handler(int sig)
exit(sig);
}
+/* Estimate luma variance assuming uniform dither noise distribution */
+static float estimate_quantization_noise(enum AVPixelFormat fmt)
+{
+ const AVPixFmtDescriptor *desc = av_pix_fmt_desc_get(fmt);
+ float variance = 1.0 / 12;
+ if (desc->comp[0].depth < 8) {
+ /* Extra headroom for very low bit depth output */
+ variance *= (8 - desc->comp[0].depth);
+ }
+
+ if (desc->flags & AV_PIX_FMT_FLAG_FLOAT) {
+ return 0.0;
+ } else if (desc->flags & AV_PIX_FMT_FLAG_RGB) {
+ const float r = 0.299 / (1 << desc->comp[0].depth);
+ const float g = 0.587 / (1 << desc->comp[1].depth);
+ const float b = 0.114 / (1 << desc->comp[2].depth);
+ return (r * r + g * g + b * b) * variance;
+ } else {
+ const float y = 1.0 / (1 << desc->comp[0].depth);
+ return y * y * variance;
+ }
+}
+
static int fmt_comps(enum AVPixelFormat fmt)
{
const AVPixFmtDescriptor *desc = av_pix_fmt_desc_get(fmt);
@@ -156,6 +179,18 @@ static void get_ssim(float ssim[4], const AVFrame *out, const AVFrame *ref, int
}
}
+static float get_loss(const float ssim[4])
+{
+ const float weights[3] = { 0.8, 0.1, 0.1 }; /* tuned for Y'CrCr */
+
+ float sum = 0;
+ for (int i = 0; i < 3; i++)
+ sum += weights[i] * ssim[i];
+ sum *= ssim[3]; /* ensure alpha errors get caught */
+
+ return 1.0 - sum;
+}
+
static int scale_legacy(AVFrame *dst, const AVFrame *src, struct mode mode,
struct options opts)
{
@@ -198,6 +233,18 @@ static int run_test(enum AVPixelFormat src_fmt, enum AVPixelFormat dst_fmt,
int64_t time, time_ref = 0;
int ret = -1;
+ /* Estimate the expected amount of loss from bit depth reduction */
+ const float c1 = 0.01 * 0.01; /* stabilization constant */
+ const float ref_var = 1.0 / 12.0; /* uniformly distributed signal */
+ const float src_var = estimate_quantization_noise(src_fmt);
+ const float dst_var = estimate_quantization_noise(dst_fmt);
+ const float out_var = estimate_quantization_noise(ref->format);
+ const float total_var = src_var + dst_var + out_var;
+ const float ssim_luma = (2 * ref_var + c1) / (2 * ref_var + total_var + c1);
+ const float ssim_expected[4] = { ssim_luma, 1, 1, 1 }; /* for simplicity */
+ const float expected_loss = get_loss(ssim_expected);
+ float loss;
+
src = av_frame_alloc();
dst = av_frame_alloc();
out = av_frame_alloc();
@@ -251,6 +298,15 @@ static int run_test(enum AVPixelFormat src_fmt, enum AVPixelFormat dst_fmt,
mode.flags, mode.dither,
ssim[0], ssim[1], ssim[2], ssim[3]);
+ loss = get_loss(ssim);
+ if (loss - expected_loss > 1e-4 && dst_w >= ref->width && dst_h >= ref->height) {
+ int bad = loss - expected_loss > 1e-2;
+ printf("\033[1;31m loss %g is %s by %g, expected loss %g\033[0m\n",
+ loss, bad ? "WORSE" : "worse", loss - expected_loss, expected_loss);
+ if (bad)
+ goto error;
+ }
+
if (!ssim_ref && sws_isSupportedInput(src->format) && sws_isSupportedOutput(dst->format)) {
/* Compare against the legacy swscale API as a reference */
time_ref = av_gettime_relative();
@@ -269,18 +325,12 @@ static int run_test(enum AVPixelFormat src_fmt, enum AVPixelFormat dst_fmt,
}
if (ssim_ref) {
- const float weights[4] = { 0.8, 0.1, 0.1, 1.0 }; /* tuned for Y'CrCr */
- float err, sum = 0, sum_ref = 0;
- for (int i = 0; i < 4; i++) {
- sum += weights[i] * ssim[i];
- sum_ref += weights[i] * ssim_ref[i];
- }
-
- err = sum_ref / sum - 1.0; /* relative error */
- if (err > 1e-4 /* 0.01% headroom for dither noise etc */) {
- int bad = err > 1e-2; /* 1% */
- printf("\033[1;31m %s by %f%%, ref SSIM {Y=%f U=%f V=%f A=%f}\033[0m\n",
- bad ? "WORSE" : "worse", 100.0 * err,
+ const float loss_ref = get_loss(ssim_ref);
+ if (loss - loss_ref > 1e-4) {
+ int bad = loss - loss_ref > 1e-2;
+ printf("\033[1;31m loss %g is %s by %g, ref loss %g, "
+ "SSIM {Y=%f U=%f V=%f A=%f}\033[0m\n",
+ loss, bad ? "WORSE" : "worse", loss - loss_ref, loss_ref,
ssim_ref[0], ssim_ref[1], ssim_ref[2], ssim_ref[3]);
if (bad)
goto error;
--
2.48.1
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".