* [FFmpeg-devel] Bug on Bayer conversions
@ 2022-09-25 15:54 Chema Gonzalez
2022-09-26 7:58 ` Anton Khirnov
0 siblings, 1 reply; 3+ messages in thread
From: Chema Gonzalez @ 2022-09-25 15:54 UTC (permalink / raw)
To: FFmpeg development discussions and patches
Hi,
I found an issue while playing with Bayer pixel format conversions.
```
$ echo -ne '\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff'
> image.raw
$ xxd image.raw
00000000: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000010: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000020: ffff ffff ffff ffff ffff ffff ffff ffff ................
00000030: ffff ffff ffff ffff ffff ffff ffff ffff ................
```
And then:
```
$ ffmpeg -y -f rawvideo -pixel_format bayer_bggr8 -s 8x8 -i image.raw
-f rawvideo -pix_fmt rgb24 -video_size 8x8 image.raw.rgb
...
Assertion srcSliceH > 1 failed at libswscale/swscale_unscaled.c:1310
Aborted (core dumped)ated 2 times
```
The issue relates to the ffmpeg parallelization.
```
$ ffmpeg -y -filter_threads 1 -f rawvideo -pixel_format bayer_bggr8 -s
8x8 -i image.raw -f rawvideo -pix_fmt rgb24 -video_size 8x8
image.raw.rgb
...
frame= 1 fps=0.0 q=-0.0 Lsize= 0kB time=00:00:00.00
bitrate=N/A speed= 0x eed=N/A
video:0kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB
muxing overhead: 0.000000%
$ xxd image.raw.rgb
00000000: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000010: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000020: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000030: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000040: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000050: 7f00 3f7f 0000 7f00 3f7f 0000 0000 0000 ..?.....?.......
00000060: ffff ffff ffff 7fbf ff7f ffff 7fbf ff7f ................
00000070: ffff ffff ffff ffff ffff ffff ffff ffff ................
00000080: ffff ffff ffff ffff ffff ffff ffff ffff ................
00000090: ffff ffff ffff ffff ffff ffff ffff ffff ................
000000a0: ffff ffff ffff ffff ffff ffff ffff ffff ................
000000b0: ffff ffff ffff ffff ffff ffff ffff ffff ................
```
FYI:
```
$ grep processor /proc/cpuinfo |wc -l
64
```
Problem seems to be that `ff_sws_slice_worker()`
[libswscale/swscale.c:1222] tries to slice the input to parallelize
the scaling task, in my case in 16 different jobs (gdb'ing the process
shows `nb_threads == nb_jobs == 16`). The 8x8 input is therefore
divided in eight 8x1 slices (1-pixel height), which eventually breaks
in `bayer_to_rgb24_wrapper()` as it asserts `srcSliceH > 1`. The
problem is the same in the 3 Bayer conversion functions
(`bayer_to_rgb24_wrapper()`, `bayer_to_rgb48_wrapper()`, and
`bayer_to_yv12_wrapper()`.
Wondering about the right solution: We could just enforce `nb_threads
= nb_jobs = 1` for all Bayer inputs. That may be the simplest
solution. Or we could make sure that `nb_threads` (and `nb_jobs`) are
capped at `input_height / 2` (to ensure at least 2 pixels per thread).
Any suggestions?
Thanks,
-Chema
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: [FFmpeg-devel] Bug on Bayer conversions
2022-09-25 15:54 [FFmpeg-devel] Bug on Bayer conversions Chema Gonzalez
@ 2022-09-26 7:58 ` Anton Khirnov
2022-09-26 16:11 ` Chema Gonzalez
0 siblings, 1 reply; 3+ messages in thread
From: Anton Khirnov @ 2022-09-26 7:58 UTC (permalink / raw)
To: FFmpeg development discussions and patches
Quoting Chema Gonzalez (2022-09-25 17:54:16)
> Hi,
>
> I found an issue while playing with Bayer pixel format conversions.
>
> ```
> $ echo -ne '\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff'
> > image.raw
> $ xxd image.raw
> 00000000: 0000 0000 0000 0000 0000 0000 0000 0000 ................
> 00000010: 0000 0000 0000 0000 0000 0000 0000 0000 ................
> 00000020: ffff ffff ffff ffff ffff ffff ffff ffff ................
> 00000030: ffff ffff ffff ffff ffff ffff ffff ffff ................
> ```
>
> And then:
> ```
> $ ffmpeg -y -f rawvideo -pixel_format bayer_bggr8 -s 8x8 -i image.raw
> -f rawvideo -pix_fmt rgb24 -video_size 8x8 image.raw.rgb
> ...
> Assertion srcSliceH > 1 failed at libswscale/swscale_unscaled.c:1310
> Aborted (core dumped)ated 2 times
> ```
>
> The issue relates to the ffmpeg parallelization.
> ```
> $ ffmpeg -y -filter_threads 1 -f rawvideo -pixel_format bayer_bggr8 -s
> 8x8 -i image.raw -f rawvideo -pix_fmt rgb24 -video_size 8x8
> image.raw.rgb
> ...
> frame= 1 fps=0.0 q=-0.0 Lsize= 0kB time=00:00:00.00
> bitrate=N/A speed= 0x eed=N/A
> video:0kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB
> muxing overhead: 0.000000%
> $ xxd image.raw.rgb
> 00000000: 0000 0000 0000 0000 0000 0000 0000 0000 ................
> 00000010: 0000 0000 0000 0000 0000 0000 0000 0000 ................
> 00000020: 0000 0000 0000 0000 0000 0000 0000 0000 ................
> 00000030: 0000 0000 0000 0000 0000 0000 0000 0000 ................
> 00000040: 0000 0000 0000 0000 0000 0000 0000 0000 ................
> 00000050: 7f00 3f7f 0000 7f00 3f7f 0000 0000 0000 ..?.....?.......
> 00000060: ffff ffff ffff 7fbf ff7f ffff 7fbf ff7f ................
> 00000070: ffff ffff ffff ffff ffff ffff ffff ffff ................
> 00000080: ffff ffff ffff ffff ffff ffff ffff ffff ................
> 00000090: ffff ffff ffff ffff ffff ffff ffff ffff ................
> 000000a0: ffff ffff ffff ffff ffff ffff ffff ffff ................
> 000000b0: ffff ffff ffff ffff ffff ffff ffff ffff ................
> ```
>
> FYI:
> ```
> $ grep processor /proc/cpuinfo |wc -l
> 64
> ```
>
> Problem seems to be that `ff_sws_slice_worker()`
> [libswscale/swscale.c:1222] tries to slice the input to parallelize
> the scaling task, in my case in 16 different jobs (gdb'ing the process
> shows `nb_threads == nb_jobs == 16`). The 8x8 input is therefore
> divided in eight 8x1 slices (1-pixel height), which eventually breaks
> in `bayer_to_rgb24_wrapper()` as it asserts `srcSliceH > 1`. The
> problem is the same in the 3 Bayer conversion functions
> (`bayer_to_rgb24_wrapper()`, `bayer_to_rgb48_wrapper()`, and
> `bayer_to_yv12_wrapper()`.
>
> Wondering about the right solution: We could just enforce `nb_threads
> = nb_jobs = 1` for all Bayer inputs. That may be the simplest
> solution. Or we could make sure that `nb_threads` (and `nb_jobs`) are
> capped at `input_height / 2` (to ensure at least 2 pixels per thread).
>
> Any suggestions?
There already is code that constrains minimum slice size, see
dst_slice_align in utils.c. Would extending that code to handle bayer
input fix the problem?
--
Anton Khirnov
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: [FFmpeg-devel] Bug on Bayer conversions
2022-09-26 7:58 ` Anton Khirnov
@ 2022-09-26 16:11 ` Chema Gonzalez
0 siblings, 0 replies; 3+ messages in thread
From: Chema Gonzalez @ 2022-09-26 16:11 UTC (permalink / raw)
To: FFmpeg development discussions and patches
Yeah, it does.
Will send a patch.
Thanks!
-Chema
On Mon, Sep 26, 2022 at 12:58 AM Anton Khirnov <anton@khirnov.net> wrote:
>
> Quoting Chema Gonzalez (2022-09-25 17:54:16)
> > Hi,
> >
> > I found an issue while playing with Bayer pixel format conversions.
> >
> > ```
> > $ echo -ne '\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff'
> > > image.raw
> > $ xxd image.raw
> > 00000000: 0000 0000 0000 0000 0000 0000 0000 0000 ................
> > 00000010: 0000 0000 0000 0000 0000 0000 0000 0000 ................
> > 00000020: ffff ffff ffff ffff ffff ffff ffff ffff ................
> > 00000030: ffff ffff ffff ffff ffff ffff ffff ffff ................
> > ```
> >
> > And then:
> > ```
> > $ ffmpeg -y -f rawvideo -pixel_format bayer_bggr8 -s 8x8 -i image.raw
> > -f rawvideo -pix_fmt rgb24 -video_size 8x8 image.raw.rgb
> > ...
> > Assertion srcSliceH > 1 failed at libswscale/swscale_unscaled.c:1310
> > Aborted (core dumped)ated 2 times
> > ```
> >
> > The issue relates to the ffmpeg parallelization.
> > ```
> > $ ffmpeg -y -filter_threads 1 -f rawvideo -pixel_format bayer_bggr8 -s
> > 8x8 -i image.raw -f rawvideo -pix_fmt rgb24 -video_size 8x8
> > image.raw.rgb
> > ...
> > frame= 1 fps=0.0 q=-0.0 Lsize= 0kB time=00:00:00.00
> > bitrate=N/A speed= 0x eed=N/A
> > video:0kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB
> > muxing overhead: 0.000000%
> > $ xxd image.raw.rgb
> > 00000000: 0000 0000 0000 0000 0000 0000 0000 0000 ................
> > 00000010: 0000 0000 0000 0000 0000 0000 0000 0000 ................
> > 00000020: 0000 0000 0000 0000 0000 0000 0000 0000 ................
> > 00000030: 0000 0000 0000 0000 0000 0000 0000 0000 ................
> > 00000040: 0000 0000 0000 0000 0000 0000 0000 0000 ................
> > 00000050: 7f00 3f7f 0000 7f00 3f7f 0000 0000 0000 ..?.....?.......
> > 00000060: ffff ffff ffff 7fbf ff7f ffff 7fbf ff7f ................
> > 00000070: ffff ffff ffff ffff ffff ffff ffff ffff ................
> > 00000080: ffff ffff ffff ffff ffff ffff ffff ffff ................
> > 00000090: ffff ffff ffff ffff ffff ffff ffff ffff ................
> > 000000a0: ffff ffff ffff ffff ffff ffff ffff ffff ................
> > 000000b0: ffff ffff ffff ffff ffff ffff ffff ffff ................
> > ```
> >
> > FYI:
> > ```
> > $ grep processor /proc/cpuinfo |wc -l
> > 64
> > ```
> >
> > Problem seems to be that `ff_sws_slice_worker()`
> > [libswscale/swscale.c:1222] tries to slice the input to parallelize
> > the scaling task, in my case in 16 different jobs (gdb'ing the process
> > shows `nb_threads == nb_jobs == 16`). The 8x8 input is therefore
> > divided in eight 8x1 slices (1-pixel height), which eventually breaks
> > in `bayer_to_rgb24_wrapper()` as it asserts `srcSliceH > 1`. The
> > problem is the same in the 3 Bayer conversion functions
> > (`bayer_to_rgb24_wrapper()`, `bayer_to_rgb48_wrapper()`, and
> > `bayer_to_yv12_wrapper()`.
> >
> > Wondering about the right solution: We could just enforce `nb_threads
> > = nb_jobs = 1` for all Bayer inputs. That may be the simplest
> > solution. Or we could make sure that `nb_threads` (and `nb_jobs`) are
> > capped at `input_height / 2` (to ensure at least 2 pixels per thread).
> >
> > Any suggestions?
>
> There already is code that constrains minimum slice size, see
> dst_slice_align in utils.c. Would extending that code to handle bayer
> input fix the problem?
>
> --
> Anton Khirnov
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2022-09-26 16:11 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-09-25 15:54 [FFmpeg-devel] Bug on Bayer conversions Chema Gonzalez
2022-09-26 7:58 ` Anton Khirnov
2022-09-26 16:11 ` Chema Gonzalez
Git Inbox Mirror of the ffmpeg-devel mailing list - see https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
This inbox may be cloned and mirrored by anyone:
git clone --mirror https://master.gitmailbox.com/ffmpegdev/0 ffmpegdev/git/0.git
# If you have public-inbox 1.1+ installed, you may
# initialize and index your mirror using the following commands:
public-inbox-init -V2 ffmpegdev ffmpegdev/ https://master.gitmailbox.com/ffmpegdev \
ffmpegdev@gitmailbox.com
public-inbox-index ffmpegdev
Example config snippet for mirrors.
AGPL code for this site: git clone https://public-inbox.org/public-inbox.git