[FFmpeg-devel] [PATCH] lavfi/bwdif: remove interpolated sample clipping

[FFmpeg-devel] [PATCH] lavfi/bwdif: remove interpolated sample clipping

[Lynne]

[-- Attachment #1: Type: text/plain, Size: 1262 bytes --]

The issue is that clipping the interpolated temporal sample against
the spatially predicted sample causes artifacts to appear.

Discovered while writing the Vulkan version (where I omitted the
same check).

The clipping in the code is carried over from yadif. Removing the
same code in yadif does not make any difference to the output.
I think that the check was simply ill-adapted to the new prediction
code and does more harm.

I tested replacing the range clip with only an FFMAX, and only an
FFMIN, but in both cases, artifacts still appeared.

Test sample 1: https://files.lynne.ee/testsamples/mbaff_1080i60_idx.mkvTest sample 2: https://files.lynne.ee/testsamples/mbaff_bdmv_1080i60_8slice.mkv

Command line:
./ffmpeg_g -cpuflags 0 -i <INPUT> -vf bwdif=mode=send_field -c:v rawvideo -y <OUTPUT>.nut
Make sure to disable the assembly.

Comparisons:
https://files.lynne.ee/bwdif_01_before.png
https://files.lynne.ee/bwdif_01_after.png
Generated from sample 1 via:
ffmpeg -ss 00:00:00.184 -i <INPUT>.nut -vf crop=w=420:h=240:x=700:y=300,scale=iw*2:ih*2 -y <OUTPUT>.png

https://files.lynne.ee/bwdif_02_before.png
https://files.lynne.ee/bwdif_02_after.pngffmpeg -ss 00:00:00.417 -i <INPUT>.nut -vf crop=w=420:h=240:x=1100:y=200,scale=iw*2:ih*2 -y <OUTPUT>.png

[-- Attachment #2: 0001-lavfi-bwdif-remove-interpolated-sample-clipping.patch --]
[-- Type: text/x-diff, Size: 904 bytes --]

From abda922dc040ce7237ad3048f55ab20b03f96a27 Mon Sep 17 00:00:00 2001
From: Lynne <dev@lynne.ee>
Date: Sun, 2 Jul 2023 18:10:47 +0200
Subject: [PATCH] lavfi/bwdif: remove interpolated sample clipping

The issue is that clipping the interpolated temporal sample against
the spatially predicted sample causes artifacts.
---
 libavfilter/vf_bwdif.c | 4 ----
 1 file changed, 4 deletions(-)

diff --git a/libavfilter/vf_bwdif.c b/libavfilter/vf_bwdif.c
index e278cf1217..55dec526bf 100644
--- a/libavfilter/vf_bwdif.c
+++ b/libavfilter/vf_bwdif.c
@@ -106,10 +106,6 @@ typedef struct ThreadData {
             interpol = (c + e) >> 1;
 
 #define FILTER2() \
-            if (interpol > d + diff) \
-                interpol = d + diff; \
-            else if (interpol < d - diff) \
-                interpol = d - diff; \
  \
             dst[0] = av_clip(interpol, 0, clip_max); \
         } \
-- 
2.40.1


[-- Attachment #3: Type: text/plain, Size: 251 bytes --]

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] lavfi/bwdif: remove interpolated sample clipping

[Lynne]

Jul 2, 2023, 18:54 by dev@lynne.ee:

> The issue is that clipping the interpolated temporal sample against
> the spatially predicted sample causes artifacts to appear.
>
> Discovered while writing the Vulkan version (where I omitted the
> same check).
>
> The clipping in the code is carried over from yadif. Removing the
> same code in yadif does not make any difference to the output.
> I think that the check was simply ill-adapted to the new prediction
> code and does more harm.
>
> I tested replacing the range clip with only an FFMAX, and only an
> FFMIN, but in both cases, artifacts still appeared.
>
> Test sample 1: https://files.lynne.ee/testsamples/mbaff_1080i60_idx.mkvTest sample 2: https://files.lynne.ee/testsamples/mbaff_bdmv_1080i60_8slice.mkv
>
> Command line:
> ./ffmpeg_g -cpuflags 0 -i <INPUT> -vf bwdif=mode=send_field -c:v rawvideo -y <OUTPUT>.nut
> Make sure to disable the assembly.
>
> Comparisons:
> https://files.lynne.ee/bwdif_01_before.png
> https://files.lynne.ee/bwdif_01_after.png
> Generated from sample 1 via:
> ffmpeg -ss 00:00:00.184 -i <INPUT>.nut -vf crop=w=420:h=240:x=700:y=300,scale=iw*2:ih*2 -y <OUTPUT>.png
>
> https://files.lynne.ee/bwdif_02_before.png
> https://files.lynne.ee/bwdif_02_after.pngffmpeg -ss 00:00:00.417 -i <INPUT>.nut -vf crop=w=420:h=240:x=1100:y=200,scale=iw*2:ih*2 -y <OUTPUT>.png
>

Corrected links for the second sample:

https://files.lynne.ee/bwdif_02_before.png
https://files.lynne.ee/bwdif_02_after.png
ffmpeg -ss 00:00:00.417 -i <INPUT>.nut -vf crop=w=420:h=240:x=1100:y=200,scale=iw*2:ih*2 -y <OUTPUT>.png

I'm sure I hit a newline. The artifacts are a lot more noticeable in the second sample.

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] lavfi/bwdif: remove interpolated sample clipping

[Thomas Mundt]

Am So., 2. Juli 2023 um 18:57 Uhr schrieb Lynne <dev@lynne.ee>:

> Jul 2, 2023, 18:54 by dev@lynne.ee:
>
> > The issue is that clipping the interpolated temporal sample against
> > the spatially predicted sample causes artifacts to appear.
> >
> > Discovered while writing the Vulkan version (where I omitted the
> > same check).
> >
> > The clipping in the code is carried over from yadif. Removing the
> > same code in yadif does not make any difference to the output.
> > I think that the check was simply ill-adapted to the new prediction
> > code and does more harm.
> >
> > I tested replacing the range clip with only an FFMAX, and only an
> > FFMIN, but in both cases, artifacts still appeared.
> >
> > Test sample 1:
> https://files.lynne.ee/testsamples/mbaff_1080i60_idx.mkvTest sample 2:
> https://files.lynne.ee/testsamples/mbaff_bdmv_1080i60_8slice.mkv
> >
> > Command line:
> > ./ffmpeg_g -cpuflags 0 -i <INPUT> -vf bwdif=mode=send_field -c:v
> rawvideo -y <OUTPUT>.nut
> > Make sure to disable the assembly.
> >
> > Comparisons:
> > https://files.lynne.ee/bwdif_01_before.png
> > https://files.lynne.ee/bwdif_01_after.png
> > Generated from sample 1 via:
> > ffmpeg -ss 00:00:00.184 -i <INPUT>.nut -vf
> crop=w=420:h=240:x=700:y=300,scale=iw*2:ih*2 -y <OUTPUT>.png
> >
> > https://files.lynne.ee/bwdif_02_before.png
> > https://files.lynne.ee/bwdif_02_after.pngffmpeg -ss 00:00:00.417 -i
> <INPUT>.nut -vf crop=w=420:h=240:x=1100:y=200,scale=iw*2:ih*2 -y
> <OUTPUT>.png
> >
>
> Corrected links for the second sample:
>
> https://files.lynne.ee/bwdif_02_before.png
> https://files.lynne.ee/bwdif_02_after.png
> ffmpeg -ss 00:00:00.417 -i <INPUT>.nut -vf
> crop=w=420:h=240:x=1100:y=200,scale=iw*2:ih*2 -y <OUTPUT>.png
>
> I'm sure I hit a newline. The artifacts are a lot more noticeable in the
> second sample.
>

I developed the bwdif to achieve the best possible balance between speed
and quality of all different image contents from the broadcast point of
view. This includes moving video as well as moving and static graphic
elements. Unfortunately, the improvement of one image content often leads
to the degradation of another. The code you removed fundamentally
stabilizes the static graphic elements. This outweighs the slightly more
frequent artifacts in moving video considering the general purpose of the
filter.
For noisy pure motion video content, for example, the w3fdif is better
suited, since it does not make if/else decisions and thus does not produce
artifacts.

Regards,
Thomas
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] lavfi/bwdif: remove interpolated sample clipping

[Paul B Mahol]

On Sun, Jul 2, 2023 at 6:57 PM Lynne <dev@lynne.ee> wrote:

> Jul 2, 2023, 18:54 by dev@lynne.ee:
>
> > The issue is that clipping the interpolated temporal sample against
> > the spatially predicted sample causes artifacts to appear.
> >
> > Discovered while writing the Vulkan version (where I omitted the
> > same check).
> >
> > The clipping in the code is carried over from yadif. Removing the
> > same code in yadif does not make any difference to the output.
> > I think that the check was simply ill-adapted to the new prediction
> > code and does more harm.
> >
> > I tested replacing the range clip with only an FFMAX, and only an
> > FFMIN, but in both cases, artifacts still appeared.
> >
> > Test sample 1:
> https://files.lynne.ee/testsamples/mbaff_1080i60_idx.mkvTest sample 2:
> https://files.lynne.ee/testsamples/mbaff_bdmv_1080i60_8slice.mkv
> >
> > Command line:
> > ./ffmpeg_g -cpuflags 0 -i <INPUT> -vf bwdif=mode=send_field -c:v
> rawvideo -y <OUTPUT>.nut
> > Make sure to disable the assembly.
> >
> > Comparisons:
> > https://files.lynne.ee/bwdif_01_before.png
> > https://files.lynne.ee/bwdif_01_after.png
> > Generated from sample 1 via:
> > ffmpeg -ss 00:00:00.184 -i <INPUT>.nut -vf
> crop=w=420:h=240:x=700:y=300,scale=iw*2:ih*2 -y <OUTPUT>.png
> >
> > https://files.lynne.ee/bwdif_02_before.png
> > https://files.lynne.ee/bwdif_02_after.pngffmpeg -ss 00:00:00.417 -i
> <INPUT>.nut -vf crop=w=420:h=240:x=1100:y=200,scale=iw*2:ih*2 -y
> <OUTPUT>.png
> >
>
> Corrected links for the second sample:
>
> https://files.lynne.ee/bwdif_02_before.png
> https://files.lynne.ee/bwdif_02_after.png
> ffmpeg -ss 00:00:00.417 -i <INPUT>.nut -vf
> crop=w=420:h=240:x=1100:y=200,scale=iw*2:ih*2 -y <OUTPUT>.png
>
> I'm sure I hit a newline. The artifacts are a lot more noticeable in the
> second sample.
>

Single png images are not way to prove something.

Please provide videos and not just single file that exhibit this issue.
(Keep showing same file over and over is not going to help show that it
helps)

Also how PSNR/SSIM/VMAF changes before after not just in single sample but
in more samples.



>
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
>
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] lavfi/bwdif: remove interpolated sample clipping

[Lynne]

Jul 2, 2023, 20:48 by onemda@gmail.com:

> On Sun, Jul 2, 2023 at 6:57 PM Lynne <dev@lynne.ee> wrote:
>
>> Jul 2, 2023, 18:54 by dev@lynne.ee:
>>
>> > The issue is that clipping the interpolated temporal sample against
>> > the spatially predicted sample causes artifacts to appear.
>> >
>> > Discovered while writing the Vulkan version (where I omitted the
>> > same check).
>> >
>> > The clipping in the code is carried over from yadif. Removing the
>> > same code in yadif does not make any difference to the output.
>> > I think that the check was simply ill-adapted to the new prediction
>> > code and does more harm.
>> >
>> > I tested replacing the range clip with only an FFMAX, and only an
>> > FFMIN, but in both cases, artifacts still appeared.
>> >
>> > Test sample 1:
>> https://files.lynne.ee/testsamples/mbaff_1080i60_idx.mkvTest sample 2:
>> https://files.lynne.ee/testsamples/mbaff_bdmv_1080i60_8slice.mkv
>> >
>> > Command line:
>> > ./ffmpeg_g -cpuflags 0 -i <INPUT> -vf bwdif=mode=send_field -c:v
>> rawvideo -y <OUTPUT>.nut
>> > Make sure to disable the assembly.
>> >
>> > Comparisons:
>> > https://files.lynne.ee/bwdif_01_before.png
>> > https://files.lynne.ee/bwdif_01_after.png
>> > Generated from sample 1 via:
>> > ffmpeg -ss 00:00:00.184 -i <INPUT>.nut -vf
>> crop=w=420:h=240:x=700:y=300,scale=iw*2:ih*2 -y <OUTPUT>.png
>> >
>> > https://files.lynne.ee/bwdif_02_before.png
>> > https://files.lynne.ee/bwdif_02_after.pngffmpeg -ss 00:00:00.417 -i
>> <INPUT>.nut -vf crop=w=420:h=240:x=1100:y=200,scale=iw*2:ih*2 -y
>> <OUTPUT>.png
>> >
>>
>> Corrected links for the second sample:
>>
>> https://files.lynne.ee/bwdif_02_before.png
>> https://files.lynne.ee/bwdif_02_after.png
>> ffmpeg -ss 00:00:00.417 -i <INPUT>.nut -vf
>> crop=w=420:h=240:x=1100:y=200,scale=iw*2:ih*2 -y <OUTPUT>.png
>>
>> I'm sure I hit a newline. The artifacts are a lot more noticeable in the
>> second sample.
>>
>
> Single png images are not way to prove something.
>
> Please provide videos and not just single file that exhibit this issue.
> (Keep showing same file over and over is not going to help show that it
> helps)
>
> Also how PSNR/SSIM/VMAF changes before after not just in single sample but
> in more samples.
>

I posted samples and instructions. Enough for a discussion.
I'm not posting gigabytes of uncompressed samples.
PSNR is irrelevant if there are visible artifacts.
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] lavfi/bwdif: remove interpolated sample clipping

[Paul B Mahol]

On Sun, Jul 2, 2023 at 9:13 PM Lynne <dev@lynne.ee> wrote:

> Jul 2, 2023, 20:48 by onemda@gmail.com:
>
> > On Sun, Jul 2, 2023 at 6:57 PM Lynne <dev@lynne.ee> wrote:
> >
> >> Jul 2, 2023, 18:54 by dev@lynne.ee:
> >>
> >> > The issue is that clipping the interpolated temporal sample against
> >> > the spatially predicted sample causes artifacts to appear.
> >> >
> >> > Discovered while writing the Vulkan version (where I omitted the
> >> > same check).
> >> >
> >> > The clipping in the code is carried over from yadif. Removing the
> >> > same code in yadif does not make any difference to the output.
> >> > I think that the check was simply ill-adapted to the new prediction
> >> > code and does more harm.
> >> >
> >> > I tested replacing the range clip with only an FFMAX, and only an
> >> > FFMIN, but in both cases, artifacts still appeared.
> >> >
> >> > Test sample 1:
> >> https://files.lynne.ee/testsamples/mbaff_1080i60_idx.mkvTest sample 2:
> >> https://files.lynne.ee/testsamples/mbaff_bdmv_1080i60_8slice.mkv
> >> >
> >> > Command line:
> >> > ./ffmpeg_g -cpuflags 0 -i <INPUT> -vf bwdif=mode=send_field -c:v
> >> rawvideo -y <OUTPUT>.nut
> >> > Make sure to disable the assembly.
> >> >
> >> > Comparisons:
> >> > https://files.lynne.ee/bwdif_01_before.png
> >> > https://files.lynne.ee/bwdif_01_after.png
> >> > Generated from sample 1 via:
> >> > ffmpeg -ss 00:00:00.184 -i <INPUT>.nut -vf
> >> crop=w=420:h=240:x=700:y=300,scale=iw*2:ih*2 -y <OUTPUT>.png
> >> >
> >> > https://files.lynne.ee/bwdif_02_before.png
> >> > https://files.lynne.ee/bwdif_02_after.pngffmpeg -ss 00:00:00.417 -i
> >> <INPUT>.nut -vf crop=w=420:h=240:x=1100:y=200,scale=iw*2:ih*2 -y
> >> <OUTPUT>.png
> >> >
> >>
> >> Corrected links for the second sample:
> >>
> >> https://files.lynne.ee/bwdif_02_before.png
> >> https://files.lynne.ee/bwdif_02_after.png
> >> ffmpeg -ss 00:00:00.417 -i <INPUT>.nut -vf
> >> crop=w=420:h=240:x=1100:y=200,scale=iw*2:ih*2 -y <OUTPUT>.png
> >>
> >> I'm sure I hit a newline. The artifacts are a lot more noticeable in the
> >> second sample.
> >>
> >
> > Single png images are not way to prove something.
> >
> > Please provide videos and not just single file that exhibit this issue.
> > (Keep showing same file over and over is not going to help show that it
> > helps)
> >
> > Also how PSNR/SSIM/VMAF changes before after not just in single sample
> but
> > in more samples.
> >
>
> I posted samples and instructions. Enough for a discussion.
> I'm not posting gigabytes of uncompressed samples.
> PSNR is irrelevant if there are visible artifacts.
>

Nope, you havent.

Some strange samples.
PSNR is relevant as also SSIM and VMAF.

Ignoring this does not help project, but just force tyrannic behavior.


_______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
>
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] lavfi/bwdif: remove interpolated sample clipping

[Lynne]

Jul 2, 2023, 21:38 by onemda@gmail.com:

> On Sun, Jul 2, 2023 at 9:13 PM Lynne <dev@lynne.ee> wrote:
>
>> Jul 2, 2023, 20:48 by onemda@gmail.com:
>>
>> > On Sun, Jul 2, 2023 at 6:57 PM Lynne <dev@lynne.ee> wrote:
>> >
>> >> Jul 2, 2023, 18:54 by dev@lynne.ee:
>> >>
>> >> > The issue is that clipping the interpolated temporal sample against
>> >> > the spatially predicted sample causes artifacts to appear.
>> >> >
>> >> > Discovered while writing the Vulkan version (where I omitted the
>> >> > same check).
>> >> >
>> >> > The clipping in the code is carried over from yadif. Removing the
>> >> > same code in yadif does not make any difference to the output.
>> >> > I think that the check was simply ill-adapted to the new prediction
>> >> > code and does more harm.
>> >> >
>> >> > I tested replacing the range clip with only an FFMAX, and only an
>> >> > FFMIN, but in both cases, artifacts still appeared.
>> >> >
>> >> > Test sample 1:
>> >> https://files.lynne.ee/testsamples/mbaff_1080i60_idx.mkvTest sample 2:
>> >> https://files.lynne.ee/testsamples/mbaff_bdmv_1080i60_8slice.mkv
>> >> >
>> >> > Command line:
>> >> > ./ffmpeg_g -cpuflags 0 -i <INPUT> -vf bwdif=mode=send_field -c:v
>> >> rawvideo -y <OUTPUT>.nut
>> >> > Make sure to disable the assembly.
>> >> >
>> >> > Comparisons:
>> >> > https://files.lynne.ee/bwdif_01_before.png
>> >> > https://files.lynne.ee/bwdif_01_after.png
>> >> > Generated from sample 1 via:
>> >> > ffmpeg -ss 00:00:00.184 -i <INPUT>.nut -vf
>> >> crop=w=420:h=240:x=700:y=300,scale=iw*2:ih*2 -y <OUTPUT>.png
>> >> >
>> >> > https://files.lynne.ee/bwdif_02_before.png
>> >> > https://files.lynne.ee/bwdif_02_after.pngffmpeg -ss 00:00:00.417 -i
>> >> <INPUT>.nut -vf crop=w=420:h=240:x=1100:y=200,scale=iw*2:ih*2 -y
>> >> <OUTPUT>.png
>> >> >
>> >>
>> >> Corrected links for the second sample:
>> >>
>> >> https://files.lynne.ee/bwdif_02_before.png
>> >> https://files.lynne.ee/bwdif_02_after.png
>> >> ffmpeg -ss 00:00:00.417 -i <INPUT>.nut -vf
>> >> crop=w=420:h=240:x=1100:y=200,scale=iw*2:ih*2 -y <OUTPUT>.png
>> >>
>> >> I'm sure I hit a newline. The artifacts are a lot more noticeable in the
>> >> second sample.
>> >>
>> >
>> > Single png images are not way to prove something.
>> >
>> > Please provide videos and not just single file that exhibit this issue.
>> > (Keep showing same file over and over is not going to help show that it
>> > helps)
>> >
>> > Also how PSNR/SSIM/VMAF changes before after not just in single sample
>> but
>> > in more samples.
>> >
>>
>> I posted samples and instructions. Enough for a discussion.
>> I'm not posting gigabytes of uncompressed samples.
>> PSNR is irrelevant if there are visible artifacts.
>>
>
> Nope, you havent.
>
> Some strange samples.
> PSNR is relevant as also SSIM and VMAF.
>

Fair enough, on a 4k60 game recording, with lots of text and movement,
ffmpeg -i game_4k60test.mkv -filter_complex "[0:0] split [t1] [t2] ; [t2] interlace [t2] ; [t2] bwdif=mode=send_field [t2] ; [t1] [t2] ssim [t3]" -map "[t3]" -f null -

Before:
0.990397 (20.175775)
After:
0.990417 (20.184970)
Slightly higher, but not really significant.

PSNR (average):
32.711758
vs
32.704465
Slightly lower, but not really significant.


> Ignoring this does not help project, but just force tyrannic behavior.
>

Relax, I'm not going to make you write a stablediffusion filter source :)
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] lavfi/bwdif: remove interpolated sample clipping

[Paul B Mahol]

On Sun, Jul 2, 2023 at 10:25 PM Lynne <dev@lynne.ee> wrote:

> Jul 2, 2023, 21:38 by onemda@gmail.com:
>
> > On Sun, Jul 2, 2023 at 9:13 PM Lynne <dev@lynne.ee> wrote:
> >
> >> Jul 2, 2023, 20:48 by onemda@gmail.com:
> >>
> >> > On Sun, Jul 2, 2023 at 6:57 PM Lynne <dev@lynne.ee> wrote:
> >> >
> >> >> Jul 2, 2023, 18:54 by dev@lynne.ee:
> >> >>
> >> >> > The issue is that clipping the interpolated temporal sample against
> >> >> > the spatially predicted sample causes artifacts to appear.
> >> >> >
> >> >> > Discovered while writing the Vulkan version (where I omitted the
> >> >> > same check).
> >> >> >
> >> >> > The clipping in the code is carried over from yadif. Removing the
> >> >> > same code in yadif does not make any difference to the output.
> >> >> > I think that the check was simply ill-adapted to the new prediction
> >> >> > code and does more harm.
> >> >> >
> >> >> > I tested replacing the range clip with only an FFMAX, and only an
> >> >> > FFMIN, but in both cases, artifacts still appeared.
> >> >> >
> >> >> > Test sample 1:
> >> >> https://files.lynne.ee/testsamples/mbaff_1080i60_idx.mkvTest sample
> 2:
> >> >> https://files.lynne.ee/testsamples/mbaff_bdmv_1080i60_8slice.mkv
> >> >> >
> >> >> > Command line:
> >> >> > ./ffmpeg_g -cpuflags 0 -i <INPUT> -vf bwdif=mode=send_field -c:v
> >> >> rawvideo -y <OUTPUT>.nut
> >> >> > Make sure to disable the assembly.
> >> >> >
> >> >> > Comparisons:
> >> >> > https://files.lynne.ee/bwdif_01_before.png
> >> >> > https://files.lynne.ee/bwdif_01_after.png
> >> >> > Generated from sample 1 via:
> >> >> > ffmpeg -ss 00:00:00.184 -i <INPUT>.nut -vf
> >> >> crop=w=420:h=240:x=700:y=300,scale=iw*2:ih*2 -y <OUTPUT>.png
> >> >> >
> >> >> > https://files.lynne.ee/bwdif_02_before.png
> >> >> > https://files.lynne.ee/bwdif_02_after.pngffmpeg -ss 00:00:00.417
> -i
> >> >> <INPUT>.nut -vf crop=w=420:h=240:x=1100:y=200,scale=iw*2:ih*2 -y
> >> >> <OUTPUT>.png
> >> >> >
> >> >>
> >> >> Corrected links for the second sample:
> >> >>
> >> >> https://files.lynne.ee/bwdif_02_before.png
> >> >> https://files.lynne.ee/bwdif_02_after.png
> >> >> ffmpeg -ss 00:00:00.417 -i <INPUT>.nut -vf
> >> >> crop=w=420:h=240:x=1100:y=200,scale=iw*2:ih*2 -y <OUTPUT>.png
> >> >>
> >> >> I'm sure I hit a newline. The artifacts are a lot more noticeable in
> the
> >> >> second sample.
> >> >>
> >> >
> >> > Single png images are not way to prove something.
> >> >
> >> > Please provide videos and not just single file that exhibit this
> issue.
> >> > (Keep showing same file over and over is not going to help show that
> it
> >> > helps)
> >> >
> >> > Also how PSNR/SSIM/VMAF changes before after not just in single sample
> >> but
> >> > in more samples.
> >> >
> >>
> >> I posted samples and instructions. Enough for a discussion.
> >> I'm not posting gigabytes of uncompressed samples.
> >> PSNR is irrelevant if there are visible artifacts.
> >>
> >
> > Nope, you havent.
> >
> > Some strange samples.
> > PSNR is relevant as also SSIM and VMAF.
> >
>
> Fair enough, on a 4k60 game recording, with lots of text and movement,
> ffmpeg -i game_4k60test.mkv -filter_complex "[0:0] split [t1] [t2] ; [t2]
> interlace [t2] ; [t2] bwdif=mode=send_field [t2] ; [t1] [t2] ssim [t3]"
> -map "[t3]" -f null -
>
> Before:
> 0.990397 (20.175775)
> After:
> 0.990417 (20.184970)
> Slightly higher, but not really significant.
>
> PSNR (average):
> 32.711758
> vs
> 32.704465
> Slightly lower, but not really significant.
>
>
Not very conclusive, I guess I need to test it myself on this sample
visually:

https://media.xiph.org/video/derf/y4m/west_wind_easy_1080p.y4m


>
> > Ignoring this does not help project, but just force tyrannic behavior.
> >
>
> Relax, I'm not going to make you write a stablediffusion filter source :)
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
>
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] lavfi/bwdif: remove interpolated sample clipping

[Lynne]

Jul 2, 2023, 20:41 by tmundt75@gmail.com:

> Am So., 2. Juli 2023 um 18:57 Uhr schrieb Lynne <dev@lynne.ee>:
>
>> Jul 2, 2023, 18:54 by dev@lynne.ee:
>>
>> > The issue is that clipping the interpolated temporal sample against
>> > the spatially predicted sample causes artifacts to appear.
>> >
>> > Discovered while writing the Vulkan version (where I omitted the
>> > same check).
>> >
>> > The clipping in the code is carried over from yadif. Removing the
>> > same code in yadif does not make any difference to the output.
>> > I think that the check was simply ill-adapted to the new prediction
>> > code and does more harm.
>> >
>> > I tested replacing the range clip with only an FFMAX, and only an
>> > FFMIN, but in both cases, artifacts still appeared.
>> >
>> > Test sample 1:
>> https://files.lynne.ee/testsamples/mbaff_1080i60_idx.mkvTest sample 2:
>> https://files.lynne.ee/testsamples/mbaff_bdmv_1080i60_8slice.mkv
>> >
>> > Command line:
>> > ./ffmpeg_g -cpuflags 0 -i <INPUT> -vf bwdif=mode=send_field -c:v
>> rawvideo -y <OUTPUT>.nut
>> > Make sure to disable the assembly.
>> >
>> > Comparisons:
>> > https://files.lynne.ee/bwdif_01_before.png
>> > https://files.lynne.ee/bwdif_01_after.png
>> > Generated from sample 1 via:
>> > ffmpeg -ss 00:00:00.184 -i <INPUT>.nut -vf
>> crop=w=420:h=240:x=700:y=300,scale=iw*2:ih*2 -y <OUTPUT>.png
>> >
>> > https://files.lynne.ee/bwdif_02_before.png
>> > https://files.lynne.ee/bwdif_02_after.pngffmpeg -ss 00:00:00.417 -i
>> <INPUT>.nut -vf crop=w=420:h=240:x=1100:y=200,scale=iw*2:ih*2 -y
>> <OUTPUT>.png
>> >
>>
>> Corrected links for the second sample:
>>
>> https://files.lynne.ee/bwdif_02_before.png
>> https://files.lynne.ee/bwdif_02_after.png
>> ffmpeg -ss 00:00:00.417 -i <INPUT>.nut -vf
>> crop=w=420:h=240:x=1100:y=200,scale=iw*2:ih*2 -y <OUTPUT>.png
>>
>> I'm sure I hit a newline. The artifacts are a lot more noticeable in the
>> second sample.
>>
>
> I developed the bwdif to achieve the best possible balance between speed
> and quality of all different image contents from the broadcast point of
> view. This includes moving video as well as moving and static graphic
> elements. Unfortunately, the improvement of one image content often leads
> to the degradation of another. The code you removed fundamentally
> stabilizes the static graphic elements. This outweighs the slightly more
> frequent artifacts in moving video considering the general purpose of the
> filter.
>

Could you post examples? I've been unable to find any that look worse
with the patch.
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] lavfi/bwdif: remove interpolated sample clipping

[Thomas Mundt]

Am So., 2. Juli 2023 um 20:58 Uhr schrieb Lynne <dev@lynne.ee>:

> Jul 2, 2023, 20:41 by tmundt75@gmail.com:
>
> > Am So., 2. Juli 2023 um 18:57 Uhr schrieb Lynne <dev@lynne.ee>:
> >
> >> Jul 2, 2023, 18:54 by dev@lynne.ee:
> >>
> >> > The issue is that clipping the interpolated temporal sample against
> >> > the spatially predicted sample causes artifacts to appear.
> >> >
> >> > Discovered while writing the Vulkan version (where I omitted the
> >> > same check).
> >> >
> >> > The clipping in the code is carried over from yadif. Removing the
> >> > same code in yadif does not make any difference to the output.
> >> > I think that the check was simply ill-adapted to the new prediction
> >> > code and does more harm.
> >> >
> >> > I tested replacing the range clip with only an FFMAX, and only an
> >> > FFMIN, but in both cases, artifacts still appeared.
> >> >
> >> > Test sample 1:
> >> https://files.lynne.ee/testsamples/mbaff_1080i60_idx.mkvTest sample 2:
> >> https://files.lynne.ee/testsamples/mbaff_bdmv_1080i60_8slice.mkv
> >> >
> >> > Command line:
> >> > ./ffmpeg_g -cpuflags 0 -i <INPUT> -vf bwdif=mode=send_field -c:v
> >> rawvideo -y <OUTPUT>.nut
> >> > Make sure to disable the assembly.
> >> >
> >> > Comparisons:
> >> > https://files.lynne.ee/bwdif_01_before.png
> >> > https://files.lynne.ee/bwdif_01_after.png
> >> > Generated from sample 1 via:
> >> > ffmpeg -ss 00:00:00.184 -i <INPUT>.nut -vf
> >> crop=w=420:h=240:x=700:y=300,scale=iw*2:ih*2 -y <OUTPUT>.png
> >> >
> >> > https://files.lynne.ee/bwdif_02_before.png
> >> > https://files.lynne.ee/bwdif_02_after.pngffmpeg -ss 00:00:00.417 -i
> >> <INPUT>.nut -vf crop=w=420:h=240:x=1100:y=200,scale=iw*2:ih*2 -y
> >> <OUTPUT>.png
> >> >
> >>
> >> Corrected links for the second sample:
> >>
> >> https://files.lynne.ee/bwdif_02_before.png
> >> https://files.lynne.ee/bwdif_02_after.png
> >> ffmpeg -ss 00:00:00.417 -i <INPUT>.nut -vf
> >> crop=w=420:h=240:x=1100:y=200,scale=iw*2:ih*2 -y <OUTPUT>.png
> >>
> >> I'm sure I hit a newline. The artifacts are a lot more noticeable in the
> >> second sample.
> >>
> >
> > I developed the bwdif to achieve the best possible balance between speed
> > and quality of all different image contents from the broadcast point of
> > view. This includes moving video as well as moving and static graphic
> > elements. Unfortunately, the improvement of one image content often leads
> > to the degradation of another. The code you removed fundamentally
> > stabilizes the static graphic elements. This outweighs the slightly more
> > frequent artifacts in moving video considering the general purpose of the
> > filter.
> >
>
> Could you post examples? I've been unable to find any that look worse
> with the patch.
>

Unfortunately, I no longer have most of the test material that I used years
ago at the development of the bwdif.
But on the quick I have this clip with an "Archiv" insert. With your patch
the letters are jumping. Without your patch they stay static.
https://www.dropbox.com/s/jzoezjbi3ho9nja/bwdif-test.mov?dl=1
ffmpeg.exe -cpuflags 0 -i "bwdif-test.mov" -vf "bwdif=1:-1:1,
scale=1920:1080" -sws_flags lanczos -aspect 16:9 -c:v libx264 -crf 21
"bwdif_original.mp4"
ffmpeg_lynne_patch.exe -cpuflags 0 -i "bwdif-test.mov" -vf "bwdif=1:-1:1,
scale=1920:1080" -sws_flags lanczos -aspect 16:9 -c:v libx264 -crf 21
"bwdif_lynne_patch.mp4"
https://www.dropbox.com/s/tonsomtkhyaha91/bwdif_original.mp4?dl=1
https://www.dropbox.com/s/aaj8o5yzlocu55z/bwdif_lynne_patch.mp4?dl=1

Regards,
Thomas
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] lavfi/bwdif: remove interpolated sample clipping

[Lynne]

Jul 4, 2023, 00:08 by tmundt75@gmail.com:

> Am So., 2. Juli 2023 um 20:58 Uhr schrieb Lynne <dev@lynne.ee>:
>
>> Jul 2, 2023, 20:41 by tmundt75@gmail.com:
>>
>> > Am So., 2. Juli 2023 um 18:57 Uhr schrieb Lynne <dev@lynne.ee>:
>> >
>> >> Jul 2, 2023, 18:54 by dev@lynne.ee:
>> >>
>> >> > The issue is that clipping the interpolated temporal sample against
>> >> > the spatially predicted sample causes artifacts to appear.
>> >> >
>> >> > Discovered while writing the Vulkan version (where I omitted the
>> >> > same check).
>> >> >
>> >> > The clipping in the code is carried over from yadif. Removing the
>> >> > same code in yadif does not make any difference to the output.
>> >> > I think that the check was simply ill-adapted to the new prediction
>> >> > code and does more harm.
>> >> >
>> >> > I tested replacing the range clip with only an FFMAX, and only an
>> >> > FFMIN, but in both cases, artifacts still appeared.
>> >> >
>> >> > Test sample 1:
>> >> https://files.lynne.ee/testsamples/mbaff_1080i60_idx.mkvTest sample 2:
>> >> https://files.lynne.ee/testsamples/mbaff_bdmv_1080i60_8slice.mkv
>> >> >
>> >> > Command line:
>> >> > ./ffmpeg_g -cpuflags 0 -i <INPUT> -vf bwdif=mode=send_field -c:v
>> >> rawvideo -y <OUTPUT>.nut
>> >> > Make sure to disable the assembly.
>> >> >
>> >> > Comparisons:
>> >> > https://files.lynne.ee/bwdif_01_before.png
>> >> > https://files.lynne.ee/bwdif_01_after.png
>> >> > Generated from sample 1 via:
>> >> > ffmpeg -ss 00:00:00.184 -i <INPUT>.nut -vf
>> >> crop=w=420:h=240:x=700:y=300,scale=iw*2:ih*2 -y <OUTPUT>.png
>> >> >
>> >> > https://files.lynne.ee/bwdif_02_before.png
>> >> > https://files.lynne.ee/bwdif_02_after.pngffmpeg -ss 00:00:00.417 -i
>> >> <INPUT>.nut -vf crop=w=420:h=240:x=1100:y=200,scale=iw*2:ih*2 -y
>> >> <OUTPUT>.png
>> >> >
>> >>
>> >> Corrected links for the second sample:
>> >>
>> >> https://files.lynne.ee/bwdif_02_before.png
>> >> https://files.lynne.ee/bwdif_02_after.png
>> >> ffmpeg -ss 00:00:00.417 -i <INPUT>.nut -vf
>> >> crop=w=420:h=240:x=1100:y=200,scale=iw*2:ih*2 -y <OUTPUT>.png
>> >>
>> >> I'm sure I hit a newline. The artifacts are a lot more noticeable in the
>> >> second sample.
>> >>
>> >
>> > I developed the bwdif to achieve the best possible balance between speed
>> > and quality of all different image contents from the broadcast point of
>> > view. This includes moving video as well as moving and static graphic
>> > elements. Unfortunately, the improvement of one image content often leads
>> > to the degradation of another. The code you removed fundamentally
>> > stabilizes the static graphic elements. This outweighs the slightly more
>> > frequent artifacts in moving video considering the general purpose of the
>> > filter.
>> >
>>
>> Could you post examples? I've been unable to find any that look worse
>> with the patch.
>>
>
> Unfortunately, I no longer have most of the test material that I used years
> ago at the development of the bwdif.
> But on the quick I have this clip with an "Archiv" insert. With your patch
> the letters are jumping. Without your patch they stay static.
> https://www.dropbox.com/s/jzoezjbi3ho9nja/bwdif-test.mov?dl=1
> ffmpeg.exe -cpuflags 0 -i "bwdif-test.mov" -vf "bwdif=1:-1:1,
> scale=1920:1080" -sws_flags lanczos -aspect 16:9 -c:v libx264 -crf 21
> "bwdif_original.mp4"
> ffmpeg_lynne_patch.exe -cpuflags 0 -i "bwdif-test.mov" -vf "bwdif=1:-1:1,
> scale=1920:1080" -sws_flags lanczos -aspect 16:9 -c:v libx264 -crf 21
> "bwdif_lynne_patch.mp4"
> https://www.dropbox.com/s/tonsomtkhyaha91/bwdif_original.mp4?dl=1
> https://www.dropbox.com/s/aaj8o5yzlocu55z/bwdif_lynne_patch.mp4?dl=1
>

Can confirm the letters are jumping with my patch.
Fair enough, consider this patch dropped. I've added the check
in Vulkan to make that version exact to C.
Thanks for testing and writing the filter!
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH] lavfi/bwdif: remove interpolated sample clipping

[Thomas Mundt]

Lynne <dev@lynne.ee> schrieb am Di., 4. Juli 2023, 00:54:

> Jul 4, 2023, 00:08 by tmundt75@gmail.com:
>
> > Am So., 2. Juli 2023 um 20:58 Uhr schrieb Lynne <dev@lynne.ee>:
> >
> >> Jul 2, 2023, 20:41 by tmundt75@gmail.com:
> >>
> >> > Am So., 2. Juli 2023 um 18:57 Uhr schrieb Lynne <dev@lynne.ee>:
> >> >
> >> >> Jul 2, 2023, 18:54 by dev@lynne.ee:
> >> >>
> >> >> > The issue is that clipping the interpolated temporal sample against
> >> >> > the spatially predicted sample causes artifacts to appear.
> >> >> >
> >> >> > Discovered while writing the Vulkan version (where I omitted the
> >> >> > same check).
> >> >> >
> >> >> > The clipping in the code is carried over from yadif. Removing the
> >> >> > same code in yadif does not make any difference to the output.
> >> >> > I think that the check was simply ill-adapted to the new prediction
> >> >> > code and does more harm.
> >> >> >
> >> >> > I tested replacing the range clip with only an FFMAX, and only an
> >> >> > FFMIN, but in both cases, artifacts still appeared.
> >> >> >
> >> >> > Test sample 1:
> >> >> https://files.lynne.ee/testsamples/mbaff_1080i60_idx.mkvTest sample
> 2:
> >> >> https://files.lynne.ee/testsamples/mbaff_bdmv_1080i60_8slice.mkv
> >> >> >
> >> >> > Command line:
> >> >> > ./ffmpeg_g -cpuflags 0 -i <INPUT> -vf bwdif=mode=send_field -c:v
> >> >> rawvideo -y <OUTPUT>.nut
> >> >> > Make sure to disable the assembly.
> >> >> >
> >> >> > Comparisons:
> >> >> > https://files.lynne.ee/bwdif_01_before.png
> >> >> > https://files.lynne.ee/bwdif_01_after.png
> >> >> > Generated from sample 1 via:
> >> >> > ffmpeg -ss 00:00:00.184 -i <INPUT>.nut -vf
> >> >> crop=w=420:h=240:x=700:y=300,scale=iw*2:ih*2 -y <OUTPUT>.png
> >> >> >
> >> >> > https://files.lynne.ee/bwdif_02_before.png
> >> >> > https://files.lynne.ee/bwdif_02_after.pngffmpeg -ss 00:00:00.417
> -i
> >> >> <INPUT>.nut -vf crop=w=420:h=240:x=1100:y=200,scale=iw*2:ih*2 -y
> >> >> <OUTPUT>.png
> >> >> >
> >> >>
> >> >> Corrected links for the second sample:
> >> >>
> >> >> https://files.lynne.ee/bwdif_02_before.png
> >> >> https://files.lynne.ee/bwdif_02_after.png
> >> >> ffmpeg -ss 00:00:00.417 -i <INPUT>.nut -vf
> >> >> crop=w=420:h=240:x=1100:y=200,scale=iw*2:ih*2 -y <OUTPUT>.png
> >> >>
> >> >> I'm sure I hit a newline. The artifacts are a lot more noticeable in
> the
> >> >> second sample.
> >> >>
> >> >
> >> > I developed the bwdif to achieve the best possible balance between
> speed
> >> > and quality of all different image contents from the broadcast point
> of
> >> > view. This includes moving video as well as moving and static graphic
> >> > elements. Unfortunately, the improvement of one image content often
> leads
> >> > to the degradation of another. The code you removed fundamentally
> >> > stabilizes the static graphic elements. This outweighs the slightly
> more
> >> > frequent artifacts in moving video considering the general purpose of
> the
> >> > filter.
> >> >
> >>
> >> Could you post examples? I've been unable to find any that look worse
> >> with the patch.
> >>
> >
> > Unfortunately, I no longer have most of the test material that I used
> years
> > ago at the development of the bwdif.
> > But on the quick I have this clip with an "Archiv" insert. With your
> patch
> > the letters are jumping. Without your patch they stay static.
> > https://www.dropbox.com/s/jzoezjbi3ho9nja/bwdif-test.mov?dl=1
> > ffmpeg.exe -cpuflags 0 -i "bwdif-test.mov" -vf "bwdif=1:-1:1,
> > scale=1920:1080" -sws_flags lanczos -aspect 16:9 -c:v libx264 -crf 21
> > "bwdif_original.mp4"
> > ffmpeg_lynne_patch.exe -cpuflags 0 -i "bwdif-test.mov" -vf "bwdif=1:-1:1,
> > scale=1920:1080" -sws_flags lanczos -aspect 16:9 -c:v libx264 -crf 21
> > "bwdif_lynne_patch.mp4"
> > https://www.dropbox.com/s/tonsomtkhyaha91/bwdif_original.mp4?dl=1
> > https://www.dropbox.com/s/aaj8o5yzlocu55z/bwdif_lynne_patch.mp4?dl=1
> >
>
> Can confirm the letters are jumping with my patch.
> Fair enough, consider this patch dropped. I've added the check
> in Vulkan to make that version exact to C.
> Thanks for testing and writing the filter!
>

Thanks

>
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".