From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTP id 2FAEC448C5 for ; Mon, 26 Sep 2022 16:11:38 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 34A9468BB0B; Mon, 26 Sep 2022 19:11:35 +0300 (EEST) Received: from mail-pl1-f173.google.com (mail-pl1-f173.google.com [209.85.214.173]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id B87D268B7DE for ; Mon, 26 Sep 2022 19:11:28 +0300 (EEST) Received: by mail-pl1-f173.google.com with SMTP id b21so6667613plz.7 for ; Mon, 26 Sep 2022 09:11:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date; bh=1hzSaECtdxj2cgEK37rNHRQI1W/EAqZbeV/g9vjgTFk=; b=fyNNMdu7sCS99iEfs1/fUeHMN9Eqj7HHoFSLCCmjv6/857BXGD5Tzmnwurv0YKGG+a qpN60K6fis+RX+IFYg3BUbmi67Xqzlk1Nis7ANcHLRE2vT8bVRUA58EbotcqlKG8JNQo VDPT/jh04gHItu3LAdbmIuAkcrM0mnYmf2xplhfjtz7bZXpoeOdxOU0C3RJSvHMhSDb8 +VjrPQqWUA8+5t+vWEUpdg9KxM49IwGiifyUpPUE+gC5Iixb9wgWDembHIdFNUv7pcnX 2KE9i3wZJ4g7JrFuqEBpB0ocvGjRDrDmPintFVzLm9RqIrUzAxiiYWznxwGsDWjVn0TG GV1w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date; bh=1hzSaECtdxj2cgEK37rNHRQI1W/EAqZbeV/g9vjgTFk=; b=YsWKyGW6RXbDFql4VJ8fNSZx/PZ6HIV+oLg95BpnG4nb1+yB/5/iSVzUFzBqecP/EX ucF2PnkBZwyJeKDsFxDNAc6noqEfbAKjSQj4qq0bupVUP97m/mPo3vU2cN5/4xXRCSxu lf8TgrctUJFl36ZLSMUi/e2L5oa3xw3UXSTXIoU+AJHsI2ANDJ3d4wtEoEZqdqAE+jPm 3stwqgIyMUkf8nCB9y19sxs1OaOC3jk53DX/JoK0QbgC+FIlsiwM2fjIBqN+XFJbkdWC 4zf/e3/M6xVj1LuZOoghnTiz9l5Psrq0bPRx3J8WpEmhau93lH8h6+d+JHZD/tijOYaw lT7g== X-Gm-Message-State: ACrzQf3fpa64aa940U8gS5K0yJugoTjA6PxPp0Ok03tGv0bdssvNrA4J cA/nq1ZpEqQ1g11RERgOuqjaR0tq661gjQ== X-Google-Smtp-Source: AMsMyM79bPWs9MMvCW0ECUGED9+bkizO7JbyF4W6CO4BkEY+MPcoBKSy2ud/ukCdyH/nReBNRkDQMA== X-Received: by 2002:a17:902:904c:b0:178:afa2:43f5 with SMTP id w12-20020a170902904c00b00178afa243f5mr23375750plz.78.1664208686374; Mon, 26 Sep 2022 09:11:26 -0700 (PDT) Received: from wilferico.thefacebook.com ([2620:10d:c090:400::5:80bc]) by smtp.gmail.com with ESMTPSA id s13-20020a170902ea0d00b0016d72804664sm11391541plg.205.2022.09.26.09.11.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 26 Sep 2022 09:11:26 -0700 (PDT) From: Chema Gonzalez To: ffmpeg-devel@ffmpeg.org Date: Mon, 26 Sep 2022 09:11:22 -0700 Message-Id: <20220926161122.1352372-1-chemag@gmail.com> X-Mailer: git-send-email 2.37.3 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 1/1] libswscale: force a minimum size of the slide for bayer sources X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Chema Gonzalez , anton@khirnov.net Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: Bayer sources are read in groups of 2 lines (e.g. for a BGGR flavor, the first row contains only B and G samples, while the second row contains only G and R samples). They need to be read as a whole. Tested: `` $ echo -ne '\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff' > image.raw $ xxd image.raw 00000000: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000010: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000020: ffff ffff ffff ffff ffff ffff ffff ffff ................ 00000030: ffff ffff ffff ffff ffff ffff ffff ffff ................ ``` And then: ``` $ ./ffmpeg -y -f rawvideo -pixel_format bayer_bggr8 -s 8x8 \ -i image.raw -f rawvideo -pix_fmt rgb24 \ -video_size 8x8 image.raw.rgb ... Assertion srcSliceH > 1 failed at libswscale/swscale_unscaled.c:1310 Aborted (core dumped)ated 2 times ``` We can see that the issue relates to the ffmpeg parallelization. ``` $ ffmpeg -y -filter_threads 1 -f rawvideo -pixel_format bayer_bggr8 \ -s 8x8 -i image.raw -f rawvideo -pix_fmt rgb24 \ -video_size 8x8 image.raw.rgb ... frame= 1 fps=0.0 q=-0.0 Lsize= 0kB time=00:00:00.00 bitrate=N/A speed= 0x eed=N/A video:0kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.000000% $ xxd image.raw.rgb 00000000: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000010: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000020: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000030: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000040: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000050: 7f00 3f7f 0000 7f00 3f7f 0000 0000 0000 ..?.....?....... 00000060: ffff ffff ffff 7fbf ff7f ffff 7fbf ff7f ................ 00000070: ffff ffff ffff ffff ffff ffff ffff ffff ................ 00000080: ffff ffff ffff ffff ffff ffff ffff ffff ................ 00000090: ffff ffff ffff ffff ffff ffff ffff ffff ................ 000000a0: ffff ffff ffff ffff ffff ffff ffff ffff ................ 000000b0: ffff ffff ffff ffff ffff ffff ffff ffff ................ ``` Problem seems to be that `ff_sws_slice_worker()` [libswscale/swscale.c:1222] tries to slice the input to parallelize the scaling task, in my case in 16 different jobs (gdb'ing the process shows `nb_threads == nb_jobs == 16`). The 8x8 input is therefore divided in eight 8x1 slices (1-pixel height), which eventually breaks in `bayer_to_rgb24_wrapper()` as it asserts `srcSliceH > 1`. The problem is the same in the 3 Bayer conversion functions (`bayer_to_rgb24_wrapper()`, `bayer_to_rgb48_wrapper()`, and `bayer_to_yv12_wrapper()`. The solution was suggested by Anton Khirnov. We set the `dst_slice_align` value to 2 for Bayer conversions. ``` $ ./ffmpeg -y -f rawvideo -pixel_format bayer_bggr8 -s 8x8 \ -i image.raw -f rawvideo -pix_fmt rgb24 \ -video_size 8x8 image.raw.rgb ... frame= 1 fps=0.0 q=-0.0 Lsize= 0kB time=00:00:00.00 bitrate=N/A speed= 0x eed=N/A video:0kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.000000% $ xxd image.raw.rgb 00000000: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000010: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000020: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000030: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000040: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000050: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000060: ffff ffff ffff ffff ffff ffff ffff ffff ................ 00000070: ffff ffff ffff ffff ffff ffff ffff ffff ................ 00000080: ffff ffff ffff ffff ffff ffff ffff ffff ................ 00000090: ffff ffff ffff ffff ffff ffff ffff ffff ................ 000000a0: ffff ffff ffff ffff ffff ffff ffff ffff ................ 000000b0: ffff ffff ffff ffff ffff ffff ffff ffff ................ ``` We can see the slicing at work, though: The demosaicing does not carry through different slices (workers). Compare to forcing a single worker: ``` $ ./ffmpeg -y -filter_threads 1 -f rawvideo -pixel_format bayer_bggr8 \ -s 8x8 -i image.raw -f rawvideo -pix_fmt rgb24 \ -video_size 8x8 image.raw.alt.rgb ... frame= 1 fps=0.0 q=-0.0 Lsize= 0kB time=00:00:00.00 bitrate=N/A speed= 0x eed=N/A video:0kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.000000% $ xxd image.raw.rgb 00000000: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000010: 0000 0000 0000 0000 0000 0000 0000 0000 ................ $ xxd /tmp/image.raw.alt.rgb 00000000: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000010: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000020: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000030: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000040: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000050: 7f00 3f7f 0000 7f00 3f7f 0000 0000 0000 ..?.....?....... 00000060: ffff ffff ffff 7fbf ff7f ffff 7fbf ff7f ................ 00000070: ffff ffff ffff ffff ffff ffff ffff ffff ................ 00000080: ffff ffff ffff ffff ffff ffff ffff ffff ................ 00000090: ffff ffff ffff ffff ffff ffff ffff ffff ................ 000000a0: ffff ffff ffff ffff ffff ffff ffff ffff ................ 000000b0: ffff ffff ffff ffff ffff ffff ffff ffff ................ ``` --- libswscale/swscale_unscaled.c | 1 + 1 file changed, 1 insertion(+) diff --git a/libswscale/swscale_unscaled.c b/libswscale/swscale_unscaled.c index 8838cc8b53..9af2e7ecc3 100644 --- a/libswscale/swscale_unscaled.c +++ b/libswscale/swscale_unscaled.c @@ -2095,6 +2095,7 @@ void ff_get_unscaled_swscale(SwsContext *c) c->convert_unscaled = rgbToPlanarRgbWrapper; if (isBayer(srcFormat)) { + c->dst_slice_align = 2; if (dstFormat == AV_PIX_FMT_RGB24) c->convert_unscaled = bayer_to_rgb24_wrapper; else if (dstFormat == AV_PIX_FMT_RGB48) -- 2.37.3 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".