Re: [FFmpeg-devel] [PATCH] fftools/ffmpeg_filter: at EOF, unref frame used for FPS conversion logic

[Steven Zhou <steven.zhou@netint.ca>]

> -----Original Message-----
> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Zhao
> Zhili
> Sent: Monday, June 23, 2025 8:34 PM
> To: FFmpeg development discussions and patches <ffmpeg-
> devel@ffmpeg.org>
> Subject: Re: [FFmpeg-devel] [PATCH] fftools/ffmpeg_filter: at EOF, unref frame
> used for FPS conversion logic
> 
> 
> 
> > On Jun 24, 2025, at 11:09, Steven Zhou <steven.zhou@netint.ca> wrote:
> >
> > fftools/ffmpeg_filter: at EOF, unref frame used for FPS conversion
> > logic
> >
> > Ever since commit 64f3feb added `avcodec_free_context(&enc->enc_ctx);`
> > to `enc_free()` the encoder context is closed before filtergraph output
> context.
> > The filtergraph output context holds a reference to last AVFrame to
> > pass through it for FPS conversion purposes. This is problematic for
> > HWFrameContexts which may rely on encoder context to free HW
> AVFrames.
> 
> What’s the encoder? This sounds a defect in encoder implementation.

The encoder is the yet to be upstreamed Netint encoders. I will put them on Forgejo for review later this week.
Perhaps the encoder implementation does not conform to FFmpeg convention. If so, I can try to address this issue another way.

The Netint HW frame data held in `AVFrame->buf[0]` is tied 'sessions' of Netint HW dec/filt/enc in the Netint driver based on what Netint HW session most recently accessed it.
The AVBufferRef destructor for `AVFrame->buf[0]` in Netint HW frames will use the most recent session the HW frame passed through to communicate its destruction to the HW's driver.
This allows for the decoder session which generated a Netint HW AVframe to have its context and session closed whilst the AVFrame continues onto a Netint encoder.
Typically this occurs when decoder is closed for EOF whilst frames are still in flight to the encoder.

I noticed commit 64f3feb closes encoder before filtergraph; but filtergraph holds a ref to last AVFrame for FPS conversion usage.
This ref in filtergraph could be discarded earlier when filtergraph processes the EOF frame, hence this patch.

Alternatively, the Netint codecs can do without this patch. The frame's resources on HW would be free'd by other garbage-collection mechanisms.

> 
> > This change unrefs the AVFrame
> > ref held by filtergraph output context during processing of EOF frame.
> >
> > Signed-off-by: Steven Zhou <steven.zhou@netint.ca>
> > ---
> > fftools/ffmpeg_filter.c | 4 +++-
> > 1 file changed, 3 insertions(+), 1 deletion(-)
> >
> > diff --git a/fftools/ffmpeg_filter.c b/fftools/ffmpeg_filter.c index
> > a0dc4c745e..cf2d016b2d 100644
> > --- a/fftools/ffmpeg_filter.c
> > +++ b/fftools/ffmpeg_filter.c
> > @@ -2560,8 +2560,10 @@ static int fg_output_frame(OutputFilterPriv
> *ofp, FilterGraphThread *fgt,
> >         av_frame_move_ref(frame_prev, frame);
> >     }
> >
> > -    if (!frame)
> > +    if (!frame) {
> > +        av_frame_unref(frame_prev);
> >         return close_output(ofp, fgt);
> > +    }
> >
> >     return 0;
> > }
> > --
> > 2.25.1
> >
> > _______________________________________________
> > ffmpeg-devel mailing list
> > ffmpeg-devel@ffmpeg.org
> > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> >
> > To unsubscribe, visit link above, or email
> > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
> 
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> 
> To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org
> with subject "unsubscribe".
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".