From: Chema Gonzalez <chemag@gmail.com>
To: ffmpeg-devel@ffmpeg.org
Cc: Chema Gonzalez <chemag@gmail.com>, anton@khirnov.net
Subject: [FFmpeg-devel] [PATCH 1/1] libswscale: force a minimum size of the slide for bayer sources
Date: Mon, 26 Sep 2022 09:11:22 -0700
Message-ID: <20220926161122.1352372-1-chemag@gmail.com> (raw)
Bayer sources are read in groups of 2 lines (e.g. for a
BGGR flavor, the first row contains only B and G samples,
while the second row contains only G and R samples). They
need to be read as a whole.
Tested:
``
$ echo -ne '\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff'
> image.raw
$ xxd image.raw
00000000: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000010: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000020: ffff ffff ffff ffff ffff ffff ffff ffff ................
00000030: ffff ffff ffff ffff ffff ffff ffff ffff ................
```
And then:
```
$ ./ffmpeg -y -f rawvideo -pixel_format bayer_bggr8 -s 8x8 \
-i image.raw -f rawvideo -pix_fmt rgb24 \
-video_size 8x8 image.raw.rgb
...
Assertion srcSliceH > 1 failed at libswscale/swscale_unscaled.c:1310
Aborted (core dumped)ated 2 times
```
We can see that the issue relates to the ffmpeg parallelization.
```
$ ffmpeg -y -filter_threads 1 -f rawvideo -pixel_format bayer_bggr8 \
-s 8x8 -i image.raw -f rawvideo -pix_fmt rgb24 \
-video_size 8x8 image.raw.rgb
...
frame= 1 fps=0.0 q=-0.0 Lsize= 0kB time=00:00:00.00
bitrate=N/A speed= 0x eed=N/A
video:0kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB
muxing overhead: 0.000000%
$ xxd image.raw.rgb
00000000: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000010: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000020: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000030: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000040: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000050: 7f00 3f7f 0000 7f00 3f7f 0000 0000 0000 ..?.....?.......
00000060: ffff ffff ffff 7fbf ff7f ffff 7fbf ff7f ................
00000070: ffff ffff ffff ffff ffff ffff ffff ffff ................
00000080: ffff ffff ffff ffff ffff ffff ffff ffff ................
00000090: ffff ffff ffff ffff ffff ffff ffff ffff ................
000000a0: ffff ffff ffff ffff ffff ffff ffff ffff ................
000000b0: ffff ffff ffff ffff ffff ffff ffff ffff ................
```
Problem seems to be that `ff_sws_slice_worker()`
[libswscale/swscale.c:1222] tries to slice the input to parallelize
the scaling task, in my case in 16 different jobs (gdb'ing the process
shows `nb_threads == nb_jobs == 16`). The 8x8 input is therefore
divided in eight 8x1 slices (1-pixel height), which eventually breaks
in `bayer_to_rgb24_wrapper()` as it asserts `srcSliceH > 1`. The
problem is the same in the 3 Bayer conversion functions
(`bayer_to_rgb24_wrapper()`, `bayer_to_rgb48_wrapper()`, and
`bayer_to_yv12_wrapper()`.
The solution was suggested by Anton Khirnov. We set the `dst_slice_align`
value to 2 for Bayer conversions.
```
$ ./ffmpeg -y -f rawvideo -pixel_format bayer_bggr8 -s 8x8 \
-i image.raw -f rawvideo -pix_fmt rgb24 \
-video_size 8x8 image.raw.rgb
...
frame= 1 fps=0.0 q=-0.0 Lsize= 0kB time=00:00:00.00 bitrate=N/A speed= 0x eed=N/A
video:0kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.000000%
$ xxd image.raw.rgb
00000000: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000010: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000020: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000030: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000040: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000050: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000060: ffff ffff ffff ffff ffff ffff ffff ffff ................
00000070: ffff ffff ffff ffff ffff ffff ffff ffff ................
00000080: ffff ffff ffff ffff ffff ffff ffff ffff ................
00000090: ffff ffff ffff ffff ffff ffff ffff ffff ................
000000a0: ffff ffff ffff ffff ffff ffff ffff ffff ................
000000b0: ffff ffff ffff ffff ffff ffff ffff ffff ................
```
We can see the slicing at work, though: The demosaicing does not
carry through different slices (workers). Compare to forcing a
single worker:
```
$ ./ffmpeg -y -filter_threads 1 -f rawvideo -pixel_format bayer_bggr8 \
-s 8x8 -i image.raw -f rawvideo -pix_fmt rgb24 \
-video_size 8x8 image.raw.alt.rgb
...
frame= 1 fps=0.0 q=-0.0 Lsize= 0kB time=00:00:00.00
bitrate=N/A speed= 0x eed=N/A
video:0kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB
muxing overhead: 0.000000%
$ xxd image.raw.rgb
00000000: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000010: 0000 0000 0000 0000 0000 0000 0000 0000 ................
$ xxd /tmp/image.raw.alt.rgb
00000000: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000010: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000020: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000030: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000040: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000050: 7f00 3f7f 0000 7f00 3f7f 0000 0000 0000 ..?.....?.......
00000060: ffff ffff ffff 7fbf ff7f ffff 7fbf ff7f ................
00000070: ffff ffff ffff ffff ffff ffff ffff ffff ................
00000080: ffff ffff ffff ffff ffff ffff ffff ffff ................
00000090: ffff ffff ffff ffff ffff ffff ffff ffff ................
000000a0: ffff ffff ffff ffff ffff ffff ffff ffff ................
000000b0: ffff ffff ffff ffff ffff ffff ffff ffff ................
```
---
libswscale/swscale_unscaled.c | 1 +
1 file changed, 1 insertion(+)
diff --git a/libswscale/swscale_unscaled.c b/libswscale/swscale_unscaled.c
index 8838cc8b53..9af2e7ecc3 100644
--- a/libswscale/swscale_unscaled.c
+++ b/libswscale/swscale_unscaled.c
@@ -2095,6 +2095,7 @@ void ff_get_unscaled_swscale(SwsContext *c)
c->convert_unscaled = rgbToPlanarRgbWrapper;
if (isBayer(srcFormat)) {
+ c->dst_slice_align = 2;
if (dstFormat == AV_PIX_FMT_RGB24)
c->convert_unscaled = bayer_to_rgb24_wrapper;
else if (dstFormat == AV_PIX_FMT_RGB48)
--
2.37.3
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
next reply other threads:[~2022-09-26 16:11 UTC|newest]
Thread overview: 5+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-09-26 16:11 Chema Gonzalez [this message]
2022-09-28 15:09 ` Anton Khirnov
2022-09-28 16:20 ` Chema Gonzalez
2022-09-30 14:15 ` Anton Khirnov
2022-10-13 15:07 ` Anton Khirnov
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20220926161122.1352372-1-chemag@gmail.com \
--to=chemag@gmail.com \
--cc=anton@khirnov.net \
--cc=ffmpeg-devel@ffmpeg.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Git Inbox Mirror of the ffmpeg-devel mailing list - see https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
This inbox may be cloned and mirrored by anyone:
git clone --mirror https://master.gitmailbox.com/ffmpegdev/0 ffmpegdev/git/0.git
# If you have public-inbox 1.1+ installed, you may
# initialize and index your mirror using the following commands:
public-inbox-init -V2 ffmpegdev ffmpegdev/ https://master.gitmailbox.com/ffmpegdev \
ffmpegdev@gitmailbox.com
public-inbox-index ffmpegdev
Example config snippet for mirrors.
AGPL code for this site: git clone https://public-inbox.org/public-inbox.git