* [FFmpeg-devel] [PATCH] fftools/ffmpeg_filter: at EOF, unref frame used for FPS conversion logic @ 2025-06-24 3:09 Steven Zhou 2025-06-24 3:33 ` Zhao Zhili 0 siblings, 1 reply; 4+ messages in thread From: Steven Zhou @ 2025-06-24 3:09 UTC (permalink / raw) To: ffmpeg-devel fftools/ffmpeg_filter: at EOF, unref frame used for FPS conversion logic Ever since commit 64f3feb added `avcodec_free_context(&enc->enc_ctx);` to `enc_free()` the encoder context is closed before filtergraph output context. The filtergraph output context holds a reference to last AVFrame to pass through it for FPS conversion purposes. This is problematic for HWFrameContexts which may rely on encoder context to free HW AVFrames. This change unrefs the AVFrame ref held by filtergraph output context during processing of EOF frame. Signed-off-by: Steven Zhou <steven.zhou@netint.ca> --- fftools/ffmpeg_filter.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/fftools/ffmpeg_filter.c b/fftools/ffmpeg_filter.c index a0dc4c745e..cf2d016b2d 100644 --- a/fftools/ffmpeg_filter.c +++ b/fftools/ffmpeg_filter.c @@ -2560,8 +2560,10 @@ static int fg_output_frame(OutputFilterPriv *ofp, FilterGraphThread *fgt, av_frame_move_ref(frame_prev, frame); } - if (!frame) + if (!frame) { + av_frame_unref(frame_prev); return close_output(ofp, fgt); + } return 0; } -- 2.25.1 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [FFmpeg-devel] [PATCH] fftools/ffmpeg_filter: at EOF, unref frame used for FPS conversion logic 2025-06-24 3:09 [FFmpeg-devel] [PATCH] fftools/ffmpeg_filter: at EOF, unref frame used for FPS conversion logic Steven Zhou @ 2025-06-24 3:33 ` Zhao Zhili 2025-06-24 4:47 ` Steven Zhou 0 siblings, 1 reply; 4+ messages in thread From: Zhao Zhili @ 2025-06-24 3:33 UTC (permalink / raw) To: FFmpeg development discussions and patches > On Jun 24, 2025, at 11:09, Steven Zhou <steven.zhou@netint.ca> wrote: > > fftools/ffmpeg_filter: at EOF, unref frame used for FPS conversion logic > > Ever since commit 64f3feb added `avcodec_free_context(&enc->enc_ctx);` to > `enc_free()` the encoder context is closed before filtergraph output context. > The filtergraph output context holds a reference to last AVFrame to pass through > it for FPS conversion purposes. This is problematic for HWFrameContexts which > may rely on encoder context to free HW AVFrames. What’s the encoder? This sounds a defect in encoder implementation. > This change unrefs the AVFrame > ref held by filtergraph output context during processing of EOF frame. > > Signed-off-by: Steven Zhou <steven.zhou@netint.ca> > --- > fftools/ffmpeg_filter.c | 4 +++- > 1 file changed, 3 insertions(+), 1 deletion(-) > > diff --git a/fftools/ffmpeg_filter.c b/fftools/ffmpeg_filter.c > index a0dc4c745e..cf2d016b2d 100644 > --- a/fftools/ffmpeg_filter.c > +++ b/fftools/ffmpeg_filter.c > @@ -2560,8 +2560,10 @@ static int fg_output_frame(OutputFilterPriv *ofp, FilterGraphThread *fgt, > av_frame_move_ref(frame_prev, frame); > } > > - if (!frame) > + if (!frame) { > + av_frame_unref(frame_prev); > return close_output(ofp, fgt); > + } > > return 0; > } > -- > 2.25.1 > > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [FFmpeg-devel] [PATCH] fftools/ffmpeg_filter: at EOF, unref frame used for FPS conversion logic 2025-06-24 3:33 ` Zhao Zhili @ 2025-06-24 4:47 ` Steven Zhou 2025-06-24 8:38 ` Zhao Zhili 0 siblings, 1 reply; 4+ messages in thread From: Steven Zhou @ 2025-06-24 4:47 UTC (permalink / raw) To: FFmpeg development discussions and patches > -----Original Message----- > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Zhao > Zhili > Sent: Monday, June 23, 2025 8:34 PM > To: FFmpeg development discussions and patches <ffmpeg- > devel@ffmpeg.org> > Subject: Re: [FFmpeg-devel] [PATCH] fftools/ffmpeg_filter: at EOF, unref frame > used for FPS conversion logic > > > > > On Jun 24, 2025, at 11:09, Steven Zhou <steven.zhou@netint.ca> wrote: > > > > fftools/ffmpeg_filter: at EOF, unref frame used for FPS conversion > > logic > > > > Ever since commit 64f3feb added `avcodec_free_context(&enc->enc_ctx);` > > to `enc_free()` the encoder context is closed before filtergraph output > context. > > The filtergraph output context holds a reference to last AVFrame to > > pass through it for FPS conversion purposes. This is problematic for > > HWFrameContexts which may rely on encoder context to free HW > AVFrames. > > What’s the encoder? This sounds a defect in encoder implementation. The encoder is the yet to be upstreamed Netint encoders. I will put them on Forgejo for review later this week. Perhaps the encoder implementation does not conform to FFmpeg convention. If so, I can try to address this issue another way. The Netint HW frame data held in `AVFrame->buf[0]` is tied 'sessions' of Netint HW dec/filt/enc in the Netint driver based on what Netint HW session most recently accessed it. The AVBufferRef destructor for `AVFrame->buf[0]` in Netint HW frames will use the most recent session the HW frame passed through to communicate its destruction to the HW's driver. This allows for the decoder session which generated a Netint HW AVframe to have its context and session closed whilst the AVFrame continues onto a Netint encoder. Typically this occurs when decoder is closed for EOF whilst frames are still in flight to the encoder. I noticed commit 64f3feb closes encoder before filtergraph; but filtergraph holds a ref to last AVFrame for FPS conversion usage. This ref in filtergraph could be discarded earlier when filtergraph processes the EOF frame, hence this patch. Alternatively, the Netint codecs can do without this patch. The frame's resources on HW would be free'd by other garbage-collection mechanisms. > > > This change unrefs the AVFrame > > ref held by filtergraph output context during processing of EOF frame. > > > > Signed-off-by: Steven Zhou <steven.zhou@netint.ca> > > --- > > fftools/ffmpeg_filter.c | 4 +++- > > 1 file changed, 3 insertions(+), 1 deletion(-) > > > > diff --git a/fftools/ffmpeg_filter.c b/fftools/ffmpeg_filter.c index > > a0dc4c745e..cf2d016b2d 100644 > > --- a/fftools/ffmpeg_filter.c > > +++ b/fftools/ffmpeg_filter.c > > @@ -2560,8 +2560,10 @@ static int fg_output_frame(OutputFilterPriv > *ofp, FilterGraphThread *fgt, > > av_frame_move_ref(frame_prev, frame); > > } > > > > - if (!frame) > > + if (!frame) { > > + av_frame_unref(frame_prev); > > return close_output(ofp, fgt); > > + } > > > > return 0; > > } > > -- > > 2.25.1 > > > > _______________________________________________ > > ffmpeg-devel mailing list > > ffmpeg-devel@ffmpeg.org > > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > > > To unsubscribe, visit link above, or email > > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". > > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org > with subject "unsubscribe". _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [FFmpeg-devel] [PATCH] fftools/ffmpeg_filter: at EOF, unref frame used for FPS conversion logic 2025-06-24 4:47 ` Steven Zhou @ 2025-06-24 8:38 ` Zhao Zhili 0 siblings, 0 replies; 4+ messages in thread From: Zhao Zhili @ 2025-06-24 8:38 UTC (permalink / raw) To: FFmpeg development discussions and patches > On Jun 24, 2025, at 12:47, Steven Zhou <steven.zhou@netint.ca> wrote: > > > >> -----Original Message----- >> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Zhao >> Zhili >> Sent: Monday, June 23, 2025 8:34 PM >> To: FFmpeg development discussions and patches <ffmpeg- >> devel@ffmpeg.org> >> Subject: Re: [FFmpeg-devel] [PATCH] fftools/ffmpeg_filter: at EOF, unref frame >> used for FPS conversion logic >> >> >> >>> On Jun 24, 2025, at 11:09, Steven Zhou <steven.zhou@netint.ca> wrote: >>> >>> fftools/ffmpeg_filter: at EOF, unref frame used for FPS conversion >>> logic >>> >>> Ever since commit 64f3feb added `avcodec_free_context(&enc->enc_ctx);` >>> to `enc_free()` the encoder context is closed before filtergraph output >> context. >>> The filtergraph output context holds a reference to last AVFrame to >>> pass through it for FPS conversion purposes. This is problematic for >>> HWFrameContexts which may rely on encoder context to free HW >> AVFrames. >> >> What’s the encoder? This sounds a defect in encoder implementation. > > The encoder is the yet to be upstreamed Netint encoders. I will put them on Forgejo for review later this week. > Perhaps the encoder implementation does not conform to FFmpeg convention. If so, I can try to address this issue another way. > > The Netint HW frame data held in `AVFrame->buf[0]` is tied 'sessions' of Netint HW dec/filt/enc in the Netint driver based on what Netint HW session most recently accessed it. > The AVBufferRef destructor for `AVFrame->buf[0]` in Netint HW frames will use the most recent session the HW frame passed through to communicate its destruction to the HW's driver. > This allows for the decoder session which generated a Netint HW AVframe to have its context and session closed whilst the AVFrame continues onto a Netint encoder. > Typically this occurs when decoder is closed for EOF whilst frames are still in flight to the encoder. Both v4l2_m2m and mediacodec decoders have similar issue, and they use reference count to delay destroy decoder context when there are frames on the fly, e.g., mediacodec_decode_close. It’s a little complex, but more reliable than fix on the user side of libavcodec. > > I noticed commit 64f3feb closes encoder before filtergraph; but filtergraph holds a ref to last AVFrame for FPS conversion usage. > This ref in filtergraph could be discarded earlier when filtergraph processes the EOF frame, hence this patch. > > Alternatively, the Netint codecs can do without this patch. The frame's resources on HW would be free'd by other garbage-collection mechanisms. > >> >>> This change unrefs the AVFrame >>> ref held by filtergraph output context during processing of EOF frame. >>> >>> Signed-off-by: Steven Zhou <steven.zhou@netint.ca> >>> --- >>> fftools/ffmpeg_filter.c | 4 +++- >>> 1 file changed, 3 insertions(+), 1 deletion(-) >>> >>> diff --git a/fftools/ffmpeg_filter.c b/fftools/ffmpeg_filter.c index >>> a0dc4c745e..cf2d016b2d 100644 >>> --- a/fftools/ffmpeg_filter.c >>> +++ b/fftools/ffmpeg_filter.c >>> @@ -2560,8 +2560,10 @@ static int fg_output_frame(OutputFilterPriv >> *ofp, FilterGraphThread *fgt, >>> av_frame_move_ref(frame_prev, frame); >>> } >>> >>> - if (!frame) >>> + if (!frame) { >>> + av_frame_unref(frame_prev); >>> return close_output(ofp, fgt); >>> + } >>> >>> return 0; >>> } >>> -- >>> 2.25.1 >>> >>> _______________________________________________ >>> ffmpeg-devel mailing list >>> ffmpeg-devel@ffmpeg.org >>> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel >>> >>> To unsubscribe, visit link above, or email >>> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". >> >> _______________________________________________ >> ffmpeg-devel mailing list >> ffmpeg-devel@ffmpeg.org >> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel >> >> To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org >> with subject "unsubscribe". > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2025-06-24 8:39 UTC | newest] Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2025-06-24 3:09 [FFmpeg-devel] [PATCH] fftools/ffmpeg_filter: at EOF, unref frame used for FPS conversion logic Steven Zhou 2025-06-24 3:33 ` Zhao Zhili 2025-06-24 4:47 ` Steven Zhou 2025-06-24 8:38 ` Zhao Zhili
Git Inbox Mirror of the ffmpeg-devel mailing list - see https://ffmpeg.org/mailman/listinfo/ffmpeg-devel This inbox may be cloned and mirrored by anyone: git clone --mirror https://master.gitmailbox.com/ffmpegdev/0 ffmpegdev/git/0.git # If you have public-inbox 1.1+ installed, you may # initialize and index your mirror using the following commands: public-inbox-init -V2 ffmpegdev ffmpegdev/ https://master.gitmailbox.com/ffmpegdev \ ffmpegdev@gitmailbox.com public-inbox-index ffmpegdev Example config snippet for mirrors. AGPL code for this site: git clone https://public-inbox.org/public-inbox.git