Git Inbox Mirror of the ffmpeg-devel mailing list - see https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
 help / color / mirror / Atom feed
* [FFmpeg-devel] [PATCH 1/2] fftools/ffmpeg_filter: fix null pointer dereference
@ 2023-12-17 11:01 Zhao Zhili
  2023-12-18 11:19 ` Anton Khirnov
  0 siblings, 1 reply; 5+ messages in thread
From: Zhao Zhili @ 2023-12-17 11:01 UTC (permalink / raw)
  To: ffmpeg-devel; +Cc: Zhao Zhili

From: Zhao Zhili <zhilizhao@tencent.com>

A dummy frame is created with format NONE passed to enc_open(),
which doesn't prepare for it. The null pointer dereference happened
at av_pix_fmt_desc_get(enc_ctx->pix_fmt)->comp[0].depth.

frame #0: 0x0000005555bc34a4 ffmpeg_g`enc_open(opaque=0xb400007efe2db690, frame=0xb400007efe2d9f70) at ffmpeg_enc.c:235:44
frame #1: 0x0000005555bef250 ffmpeg_g`enc_open(sch=0xb400007dde2d4090, enc=0xb400007e4e2daad0, frame=0xb400007efe2d9f70) at ffmpeg_sched.c:1462:11
frame #2: 0x0000005555bee094 ffmpeg_g`send_to_enc(sch=0xb400007dde2d4090, enc=0xb400007e4e2daad0, frame=0xb400007efe2d9f70) at ffmpeg_sched.c:1571:19
frame #3: 0x0000005555bee01c ffmpeg_g`sch_filter_send(sch=0xb400007dde2d4090, fg_idx=0, out_idx=0, frame=0xb400007efe2d9f70) at ffmpeg_sched.c:2154:12
frame #4: 0x0000005555bcf124 ffmpeg_g`close_output(ofp=0xb400007e4e2d85b0, fgt=0x0000007d1790eb08) at ffmpeg_filter.c:2225:15
frame #5: 0x0000005555bcb000 ffmpeg_g`fg_output_frame(ofp=0xb400007e4e2d85b0, fgt=0x0000007d1790eb08, frame=0x0000000000000000) at ffmpeg_filter.c:2317:16
frame #6: 0x0000005555bc7e48 ffmpeg_g`filter_thread(arg=0xb400007eae2ce7a0) at ffmpeg_filter.c:2836:15
frame #7: 0x0000005555bee568 ffmpeg_g`task_wrapper(arg=0xb400007d8e2db478) at ffmpeg_sched.c:2200:21

Signed-off-by: Zhao Zhili <zhilizhao@tencent.com>
---
This bug is trigged by 10bit H.264 which doesn't supported by mediacodec.
There is some misleading error messages from ffmpeg cli:

[h264_mediacodec @ 0xb400007513788010] Failed to dequeue output buffer (status=-10000)
[vist#0:0/h264 @ 0xb4000074a3786b30] Error submitting packet to decoder: Generic error in an external library
[h264_mediacodec @ 0xb400007513788010] Failed to dequeue output buffer (status=-10000)
[vist#0:0/h264 @ 0xb4000074a3786b30] Error submitting packet to decoder: Generic error in an external library
[vist#0:0/h264 @ 0xb4000074a3786b30] A decoder returned an unexpected error code. This is a bug, please report it.
[vist#0:0/h264 @ 0xb4000074a3786b30] Error processing packet in decoder: Internal bug, should not have happened

 fftools/ffmpeg_filter.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/fftools/ffmpeg_filter.c b/fftools/ffmpeg_filter.c
index bb755d7bb4..9dc47f9d90 100644
--- a/fftools/ffmpeg_filter.c
+++ b/fftools/ffmpeg_filter.c
@@ -2192,7 +2192,7 @@ static int close_output(OutputFilterPriv *ofp, FilterGraphThread *fgt)
 
     // we are finished and no frames were ever seen at this output,
     // at least initialize the encoder with a dummy frame
-    if (!fgt->got_frame) {
+    if (!fgt->got_frame && ofp->format != -1) {
         AVFrame *frame = fgt->frame;
         FrameData *fd;
 
-- 
2.34.1

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

^ permalink raw reply	[flat|nested] 5+ messages in thread

end of thread, other threads:[~2023-12-18 13:03 UTC | newest]

Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-12-17 11:01 [FFmpeg-devel] [PATCH 1/2] fftools/ffmpeg_filter: fix null pointer dereference Zhao Zhili
2023-12-18 11:19 ` Anton Khirnov
2023-12-18 12:04   ` Zhao Zhili
2023-12-18 12:57     ` Zhao Zhili
2023-12-18 13:03     ` Anton Khirnov

Git Inbox Mirror of the ffmpeg-devel mailing list - see https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

This inbox may be cloned and mirrored by anyone:

	git clone --mirror https://master.gitmailbox.com/ffmpegdev/0 ffmpegdev/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 ffmpegdev ffmpegdev/ https://master.gitmailbox.com/ffmpegdev \
		ffmpegdev@gitmailbox.com
	public-inbox-index ffmpegdev

Example config snippet for mirrors.


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git