* [FFmpeg-devel] Defect: MediaCodec Encoder - Pixel ColorSpace of many Adreno GPUs not supported @ 2022-12-06 22:52 Ben Temple 2022-12-07 17:16 ` Zhao Zhili 0 siblings, 1 reply; 4+ messages in thread From: Ben Temple @ 2022-12-06 22:52 UTC (permalink / raw) To: ffmpeg-devel, Zhao Zhili [-- Attachment #1: Type: text/plain, Size: 2647 bytes --] MediaCodec Encoder: Pixel ColorSpace of many Adreno GPUs is not supported (<= Adreno 650 in our testing)What you were trying to accomplish? I am trying to transcode an MP4 file using the MediaCodec encoder support that was recently added by Zhao Zhili <zhilizhao@tencent.com> here <https://git.ffmpeg.org/gitweb/ffmpeg.git/commit/0ff18a7d6d496f89d9e007ddd1cad7116baf5c7c> ( https://git.ffmpeg.org/gitweb/ffmpeg.git/commit/0ff18a7d6d496f89d9e007ddd1cad7116baf5c7c ) The problem you encountered: On multiple phones containing an Adreno GPU, the encoding process immediately fails because the color space expected by the GPU is unsupported. The exact error output: “does not support color format 19” Phone information: Model Name: Samsung Galaxy S10 Model Number: SM-G793U1 GPU Information: Adreno 640 Other phones that were tested and not working with this same error: Brand Model name Model number OS GPU Huawei Honor 10 COL-L29 Android 8.1 Mali-G72 Xiaomi Redmi Note 9S Redmi Note 9S Android 11 Adreno (TM) 618 Samsung Galaxy Z Flip SM-F700U1 Android 11 Adreno (TM) 640 Razer Razer phone Phone Android 7.1.1 Adreno (TM) 540 Essential PH-1 PH-1 Android 8.1 Adreno (TM) 541 Huawei Nexus 6P Nexus 6P Android 8.1 Adreno (TM) 430 Samsung Galaxy S6 SM-G920A Android 7 Mali-T760 Samsung Galaxy Tab S4 (Tablet) SM-T830 Android 10 Adreno (TM) 540 Samsung Galaxy S20 FE SM-G780G/DS Android 12 Adreno (TM) 650 Samsung Galaxy S10 SM-G973U1 Android 11 Adreno (TM) 640 Samsung Galaxy Note10 SM-N970U1 Android 12 Adreno (TM) 640 The following phones did not exhibit this behavior and were able to encode without issues: Brand Model name Model number OS GPU Samsung Galaxy S21 Ultra 5G SM-G998U1 Android 12 Adreno (TM) 660 Huawei P10 plus VKY-L29 Android 8.1 Mali-G71 Huawei P40 Lite JNY-LX1 Android 10 Mali-G52 Samsung Galaxy S8 SM-G950F Android 9 Mali-G71 Samsung Galaxy S10+ Sm-G975F Android 12 Mali-G76 Huawei P30 ELE-L29 Android 10 Mali-G76 Google Pixel 4XL Pixel 4XL Android 13 Adreno (TM) 640 Samsung Galaxy S7 SM-G930F Android 8 Mali-T880 Samsung Galaxy S10 SM-G973F Android 11 Mali-G76 Samsung Galaxy S20 FE SM-G780F Android 11 Mali G77 Google Pixel 6 Pixel 6 Android 13 Mali-G78 The exact command line you were using: -hwaccel mediacodec -hwaccel_output_format mediacodec -i "/data/user/0/com.test.ffmpeg_kit_android_demo/cache/test_input.mp4" -c:v h264_mediacodec -ndk_codec 1 -y "/data/user/0/com.test.ffmpeg_kit_android_demo/cache/test_output.mp4" -loglevel trace Console output: See attached console_output.log Input Files: See attached test_input.mp4 [-- Attachment #2: console_output.log --] [-- Type: application/octet-stream, Size: 12509 bytes --] [-- Attachment #3: test_input.mp4 --] [-- Type: video/mp4, Size: 13771 bytes --] [-- Attachment #4: Type: text/plain, Size: 251 bytes --] _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [FFmpeg-devel] Defect: MediaCodec Encoder - Pixel ColorSpace of many Adreno GPUs not supported 2022-12-06 22:52 [FFmpeg-devel] Defect: MediaCodec Encoder - Pixel ColorSpace of many Adreno GPUs not supported Ben Temple @ 2022-12-07 17:16 ` Zhao Zhili 2022-12-07 18:03 ` Ben Temple 2022-12-12 15:30 ` Tomas Härdin 0 siblings, 2 replies; 4+ messages in thread From: Zhao Zhili @ 2022-12-07 17:16 UTC (permalink / raw) To: 'FFmpeg development discussions and patches' > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Ben Temple > Sent: 2022年12月7日 6:53 > To: ffmpeg-devel@ffmpeg.org; Zhao Zhili <zhilizhao@tencent.com> > Subject: [FFmpeg-devel] Defect: MediaCodec Encoder - Pixel ColorSpace of many Adreno GPUs not supported > > MediaCodec Encoder: Pixel ColorSpace of many Adreno GPUs is not supported > (<= Adreno 650 in our testing)What you were trying to accomplish? > > I am trying to transcode an MP4 file using the MediaCodec encoder support > that was recently added by Zhao Zhili <zhilizhao@tencent.com> here > <https://git.ffmpeg.org/gitweb/ffmpeg.git/commit/0ff18a7d6d496f89d9e007ddd1cad7116baf5c7c> > ( > https://git.ffmpeg.org/gitweb/ffmpeg.git/commit/0ff18a7d6d496f89d9e007ddd1cad7116baf5c7c > ) > The problem you encountered: > > On multiple phones containing an Adreno GPU, the encoding process > immediately fails because the color space expected by the GPU is > unsupported. > > The exact error output: “does not support color format 19” > ... > > The exact command line you were using: > > -hwaccel mediacodec -hwaccel_output_format mediacodec -i > "/data/user/0/com.test.ffmpeg_kit_android_demo/cache/test_input.mp4" -c:v > h264_mediacodec -ndk_codec 1 -y > "/data/user/0/com.test.ffmpeg_kit_android_demo/cache/test_output.mp4" > -loglevel trace Thank you for your valuable test! There is an unexpected behavior in the test. The test was conducted with ffmpeg-kit, not ffmpeg cmdline. With FFmpeg cmdline and -hwcontext mediacodec, the encoder should have "-pix_fmt mediacodec" by default, and have an AVMediaCodecDeviceContext (with Android 8.0 or higher). The encoder doesn't setup properly with ffmpeg-kit. pix_fmt mediacodec should be supported by most of Android devices. It do have its limitation, e.g., it needs a Surface object or require Android 8.0, and it doesn't work with avfilter. The above issue can be fixed on the ffmpeg-kit's side. Now comes the dark corner of Mediacodec. It should work without pix_fmt mediacodec. Can you try specify the encoder pix_fmt as nv12, that's -hwaccel mediacodec -hwaccel_output_format mediacodec -i test_input.mp4 -c:v h264_mediacodec -ndk_codec 1 -pix_fmt nv12 -y test_output.mp4 I guess these devices work with nv12. There is an old story "MediaCodec and device compatibility" https://groups.google.com/g/android-platform/c/p_MoSk0JPNM Ten years on, the situation doesn't improved much: 1. There is an ugly API to get color format supported by a codec. 2. It's only available in Java. 3. Different codec component have different color format. Yes, we can use a bunch of JNI to get the supported color format. It must be done before create AVCodecContext. But different mediacodec component have different color format, user can select mediacodec by name (see the following patch), we have only one AVCodec.pix_fmts. avcodec/mediacodecenc: add option to select codec by name https://patchwork.ffmpeg.org/project/ffmpeg/patch/tencent_729F52ABE34E4770972DB89AF840549A8208@qq.com/ We can add query the supported color format and add some hints when mediacodec configure failed. Or we can add a helper function for user to query the color format. It will be a bunch of JNI code, limited by the ugly MediaCodecInfo.CodecCapabilities API. It's easy to be done with Java. TBH, it's much easier with trial and error, try nv12 then yuv420. We can do something with FFCodec.init_static_data, but we need a strategy first. I don't want to go with a long blacklist/whitelist. ** We only need one pixel format which supported by all devices, COLOR_FormatSurface doesn't count, and COLOR_FormatYUV420Flexible doesn't count. ** Maybe Google can fix these issues with another 10 year. > Console output: > > See attached console_output.log > Input Files: > > See attached test_input.mp4 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [FFmpeg-devel] Defect: MediaCodec Encoder - Pixel ColorSpace of many Adreno GPUs not supported 2022-12-07 17:16 ` Zhao Zhili @ 2022-12-07 18:03 ` Ben Temple 2022-12-12 15:30 ` Tomas Härdin 1 sibling, 0 replies; 4+ messages in thread From: Ben Temple @ 2022-12-07 18:03 UTC (permalink / raw) To: FFmpeg development discussions and patches My apologies for using the incorrect channel. I mis-read the new ticket description that states that *patches* must come through the mailing list, not new tickets. I've created a ticket here: https://trac.ffmpeg.org/ticket/10096 where we can continue the discussion. As a quick side-note, using -px_fmt nv12 does resolve the issue on a quick test that I ran on a phone that was failing. I'll respond further on the ticket so as to not add noise to this mailing list. Thank you for your response! On Wed, Dec 7, 2022 at 9:16 AM Zhao Zhili <quinkblack@foxmail.com> wrote: > > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Ben > Temple > > Sent: 2022年12月7日 6:53 > > To: ffmpeg-devel@ffmpeg.org; Zhao Zhili <zhilizhao@tencent.com> > > Subject: [FFmpeg-devel] Defect: MediaCodec Encoder - Pixel ColorSpace of > many Adreno GPUs not supported > > > > MediaCodec Encoder: Pixel ColorSpace of many Adreno GPUs is not supported > > (<= Adreno 650 in our testing)What you were trying to accomplish? > > > > I am trying to transcode an MP4 file using the MediaCodec encoder support > > that was recently added by Zhao Zhili <zhilizhao@tencent.com> here > > < > https://urldefense.com/v3/__https://git.ffmpeg.org/gitweb/ffmpeg.git/commit/0ff18a7d6d496f89d9e007ddd1cad7116baf5c7c__;!!N1d2nkBXpQ!K7sceiwL4SR1PIFjy4TY99mSRBvCQZi78fR4-CH4X5H9Q_KXbtmpW22WH9y-eagaazoZSyHLeITHcSseZdyLWTgl$ > > > > ( > > > https://urldefense.com/v3/__https://git.ffmpeg.org/gitweb/ffmpeg.git/commit/0ff18a7d6d496f89d9e007ddd1cad7116baf5c7c__;!!N1d2nkBXpQ!K7sceiwL4SR1PIFjy4TY99mSRBvCQZi78fR4-CH4X5H9Q_KXbtmpW22WH9y-eagaazoZSyHLeITHcSseZdyLWTgl$ > > ) > > The problem you encountered: > > > > On multiple phones containing an Adreno GPU, the encoding process > > immediately fails because the color space expected by the GPU is > > unsupported. > > > > The exact error output: “does not support color format 19” > > > ... > > > > The exact command line you were using: > > > > -hwaccel mediacodec -hwaccel_output_format mediacodec -i > > "/data/user/0/com.test.ffmpeg_kit_android_demo/cache/test_input.mp4" > -c:v > > h264_mediacodec -ndk_codec 1 -y > > "/data/user/0/com.test.ffmpeg_kit_android_demo/cache/test_output.mp4" > > -loglevel trace > > Thank you for your valuable test! > > There is an unexpected behavior in the test. The test was conducted with > ffmpeg-kit, not ffmpeg cmdline. > With FFmpeg cmdline and -hwcontext mediacodec, the encoder should have > "-pix_fmt mediacodec" by > default, and have an AVMediaCodecDeviceContext (with Android 8.0 or > higher). The encoder doesn't > setup properly with ffmpeg-kit. > > pix_fmt mediacodec should be supported by most of Android devices. It do > have its limitation, e.g., it needs > a Surface object or require Android 8.0, and it doesn't work with avfilter. > > The above issue can be fixed on the ffmpeg-kit's side. Now comes the dark > corner of Mediacodec. It should > work without pix_fmt mediacodec. Can you try specify the encoder pix_fmt > as nv12, that's > > -hwaccel mediacodec -hwaccel_output_format mediacodec -i test_input.mp4 > -c:v h264_mediacodec -ndk_codec 1 -pix_fmt nv12 -y test_output.mp4 > > I guess these devices work with nv12. > > There is an old story "MediaCodec and device compatibility" > > > https://urldefense.com/v3/__https://groups.google.com/g/android-platform/c/p_MoSk0JPNM__;!!N1d2nkBXpQ!K7sceiwL4SR1PIFjy4TY99mSRBvCQZi78fR4-CH4X5H9Q_KXbtmpW22WH9y-eagaazoZSyHLeITHcSseZZeyTDJ9$ > > Ten years on, the situation doesn't improved much: > 1. There is an ugly API to get color format supported by a codec. > 2. It's only available in Java. > 3. Different codec component have different color format. > > Yes, we can use a bunch of JNI to get the supported color format. It must > be done before create > AVCodecContext. But different mediacodec component have different color > format, user can select > mediacodec by name (see the following patch), we have only one > AVCodec.pix_fmts. > > avcodec/mediacodecenc: add option to select codec by name > > https://urldefense.com/v3/__https://patchwork.ffmpeg.org/project/ffmpeg/patch/tencent_729F52ABE34E4770972DB89AF840549A8208@qq.com/__;!!N1d2nkBXpQ!K7sceiwL4SR1PIFjy4TY99mSRBvCQZi78fR4-CH4X5H9Q_KXbtmpW22WH9y-eagaazoZSyHLeITHcSseZTsxLMxB$ > > We can add query the supported color format and add some hints when > mediacodec configure failed. > Or we can add a helper function for user to query the color format. It > will be a bunch of JNI code, limited > by the ugly MediaCodecInfo.CodecCapabilities API. It's easy to be done > with Java. TBH, it's much easier with > trial and error, try nv12 then yuv420. > > We can do something with FFCodec.init_static_data, but we need a strategy > first. I don't want to go with a long > blacklist/whitelist. > > ** We only need one pixel format which supported by all devices, > COLOR_FormatSurface doesn't count, and > COLOR_FormatYUV420Flexible doesn't count. ** > > Maybe Google can fix these issues with another 10 year. > > > Console output: > > > > See attached console_output.log > > Input Files: > > > > See attached test_input.mp4 > > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > > https://urldefense.com/v3/__https://ffmpeg.org/mailman/listinfo/ffmpeg-devel__;!!N1d2nkBXpQ!K7sceiwL4SR1PIFjy4TY99mSRBvCQZi78fR4-CH4X5H9Q_KXbtmpW22WH9y-eagaazoZSyHLeITHcSseZR7beCWm$ > > To unsubscribe, visit link above, or email > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". > _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [FFmpeg-devel] Defect: MediaCodec Encoder - Pixel ColorSpace of many Adreno GPUs not supported 2022-12-07 17:16 ` Zhao Zhili 2022-12-07 18:03 ` Ben Temple @ 2022-12-12 15:30 ` Tomas Härdin 1 sibling, 0 replies; 4+ messages in thread From: Tomas Härdin @ 2022-12-12 15:30 UTC (permalink / raw) To: FFmpeg development discussions and patches tor 2022-12-08 klockan 01:16 +0800 skrev Zhao Zhili: > > We can add query the supported color format and add some hints when > mediacodec configure failed. > Or we can add a helper function for user to query the color format. > It will be a bunch of JNI code, limited > by the ugly MediaCodecInfo.CodecCapabilities API. It's easy to be > done with Java. TBH, it's much easier with > trial and error, try nv12 then yuv420. > > We can do something with FFCodec.init_static_data, but we need a > strategy first. I don't want to go with a long > blacklist/whitelist. I plan on hacking on something for this this week /Tomas _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2022-12-12 15:30 UTC | newest] Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2022-12-06 22:52 [FFmpeg-devel] Defect: MediaCodec Encoder - Pixel ColorSpace of many Adreno GPUs not supported Ben Temple 2022-12-07 17:16 ` Zhao Zhili 2022-12-07 18:03 ` Ben Temple 2022-12-12 15:30 ` Tomas Härdin
Git Inbox Mirror of the ffmpeg-devel mailing list - see https://ffmpeg.org/mailman/listinfo/ffmpeg-devel This inbox may be cloned and mirrored by anyone: git clone --mirror https://master.gitmailbox.com/ffmpegdev/0 ffmpegdev/git/0.git # If you have public-inbox 1.1+ installed, you may # initialize and index your mirror using the following commands: public-inbox-init -V2 ffmpegdev ffmpegdev/ https://master.gitmailbox.com/ffmpegdev \ ffmpegdev@gitmailbox.com public-inbox-index ffmpegdev Example config snippet for mirrors. AGPL code for this site: git clone https://public-inbox.org/public-inbox.git