From: Lynne <dev@lynne.ee>
To: ffmpeg-devel@ffmpeg.org
Subject: Re: [FFmpeg-devel] [PATCH 11/13] ffv1dec: reference the current packet into the main context
Date: Thu, 13 Mar 2025 19:45:25 +0100
Message-ID: <964e0504-efda-4f29-a567-225bf250b075@lynne.ee> (raw)
In-Reply-To: <AS8P250MB0744E15CBF40D1BADF2BF7C98FD32@AS8P250MB0744.EURP250.PROD.OUTLOOK.COM>
On 13/03/2025 18:48, Andreas Rheinhardt wrote:
> Lynne:
>> On 13/03/2025 05:57, Andreas Rheinhardt wrote:
>>> Lynne:
>>>>
>>>>
>>>> On 13/03/2025 02:24, Andreas Rheinhardt wrote:
>>>>> Lynne:
>>>>>> On 10/03/2025 18:42, Lynne wrote:
>>>>>>> On 10/03/2025 04:14, Andreas Rheinhardt wrote:
>>>>>>>> Lynne:
>>>>>>>>> ---
>>>>>>>>> libavcodec/ffv1.h | 3 +++
>>>>>>>>> libavcodec/ffv1dec.c | 19 +++++++++++++++++--
>>>>>>>>> 2 files changed, 20 insertions(+), 2 deletions(-)
>>>>>>>>>
>>>>>>>>> diff --git a/libavcodec/ffv1.h b/libavcodec/ffv1.h
>>>>>>>>> index 8c0e71284d..860a5c14b1 100644
>>>>>>>>> --- a/libavcodec/ffv1.h
>>>>>>>>> +++ b/libavcodec/ffv1.h
>>>>>>>>> @@ -174,6 +174,9 @@ typedef struct FFV1Context {
>>>>>>>>> * NOT shared between frame threads.
>>>>>>>>> */
>>>>>>>>> uint8_t frame_damaged;
>>>>>>>>> +
>>>>>>>>> + /* Reference to the current packet */
>>>>>>>>> + AVPacket *pkt_ref;
>>>>>>>>> } FFV1Context;
>>>>>>>>> int ff_ffv1_common_init(AVCodecContext *avctx, FFV1Context *s);
>>>>>>>>> diff --git a/libavcodec/ffv1dec.c b/libavcodec/ffv1dec.c
>>>>>>>>> index eaa21eebdf..6396f22f79 100644
>>>>>>>>> --- a/libavcodec/ffv1dec.c
>>>>>>>>> +++ b/libavcodec/ffv1dec.c
>>>>>>>>> @@ -469,6 +469,10 @@ static av_cold int decode_init(AVCodecContext
>>>>>>>>> *avctx)
>>>>>>>>> f->pix_fmt = AV_PIX_FMT_NONE;
>>>>>>>>> f->configured_pix_fmt = AV_PIX_FMT_NONE;
>>>>>>>>> + f->pkt_ref = av_packet_alloc();
>>>>>>>>> + if (!f->pkt_ref)
>>>>>>>>> + return AVERROR(ENOMEM);
>>>>>>>>> +
>>>>>>>>> if ((ret = ff_ffv1_common_init(avctx, f)) < 0)
>>>>>>>>> return ret;
>>>>>>>>> @@ -701,6 +705,10 @@ static int decode_frame(AVCodecContext *avctx,
>>>>>>>>> AVFrame *rframe,
>>>>>>>>> /* Start */
>>>>>>>>> if (hwaccel) {
>>>>>>>>> + ret = av_packet_ref(f->pkt_ref, avpkt);
>>>>>>>>> + if (ret < 0)
>>>>>>>>> + return ret;
>>>>>>>>> +
>>>>>>>>> ret = hwaccel->start_frame(avctx, avpkt->data, avpkt-
>>>>>>>>>> size);
>>>>>>>>> if (ret < 0)
>>>>>>>>> return ret;
>>>>>>>>> @@ -720,15 +728,21 @@ static int decode_frame(AVCodecContext
>>>>>>>>> *avctx,
>>>>>>>>> AVFrame *rframe,
>>>>>>>>> uint32_t len;
>>>>>>>>> ret = find_next_slice(avctx, avpkt->data,
>>>>>>>>> buf_end, i,
>>>>>>>>> &pos, &len);
>>>>>>>>> - if (ret < 0)
>>>>>>>>> + if (ret < 0) {
>>>>>>>>> + av_packet_unref(f->pkt_ref);
>>>>>>>>> return ret;
>>>>>>>>> + }
>>>>>>>>> buf_end -= len;
>>>>>>>>> ret = hwaccel->decode_slice(avctx, pos, len);
>>>>>>>>> - if (ret < 0)
>>>>>>>>> + if (ret < 0) {
>>>>>>>>> + av_packet_unref(f->pkt_ref);
>>>>>>>>> return ret;
>>>>>>>>> + }
>>>>>>>>> }
>>>>>>>>> +
>>>>>>>>> + av_packet_unref(f->pkt_ref);
>>>>>>>>> } else {
>>>>>>>>> ret = decode_slices(avctx, c, avpkt);
>>>>>>>>> if (ret < 0)
>>>>>>>>> @@ -827,6 +841,7 @@ static av_cold int
>>>>>>>>> ffv1_decode_close(AVCodecContext *avctx)
>>>>>>>>> ff_progress_frame_unref(&s->last_picture);
>>>>>>>>> av_refstruct_unref(&s->hwaccel_last_picture_private);
>>>>>>>>> + av_packet_free(&s->pkt_ref);
>>>>>>>>> ff_ffv1_close(s);
>>>>>>>>> return 0;
>>>>>>>>
>>>>>>>> Why not simply use a const AVPacket*?
>>>>>>>
>>>>>>> No reason. Fixed locally.
>>>>>>> Thanks.
>>>>>>
>>>>>> *reverted this change.
>>>>>> We need to ref the packet, since we map its memory and let the GPU use
>>>>>> it directly without copying the contents. 6k16bit content at 24fps is
>>>>>> typically around 2Gbps when compressed, so avoiding copies is
>>>>>> important.
>>>>>
>>>>> How long does the hwaccel need this data?
>>>>
>>>> Until the frame has been asynchronously decoded. We give an output frame
>>>> with a semaphore that receivers need to wait on to determine when
>>>> that is.
>
> Does this mean that the frames the caller of avcodec_receive_frame()
> receives are not completely decoded yet?
>
>>>>
>>>> On the decoder-side, the hardware has a fixed number of queues where
>>>> submissions can be sent to asynchronously. We treat it as a ring buffer
>>>> and keep a reference to all resources our side for each submission,
>>>> until we need to reuse the slot, at which point we wait on the frame
>>>> decoding to complete (which it usually has), and we release all
>>>> resources used.
>>>>
>>>> Output frames also have a bit of state that has to be freed once the
>>>> frame is marked (unreferenced) by the decoder as no longer being needed
>>>> as a reference, this is done in the FFHWAccel.free_frame_priv callback.
>>>> There, we have to wait for the last internal use of the frame to be
>>>> finished (done via the vp->wait_semaphores() call in vulkan_decode.c).
>>>>
>>>> This is valid for both ASIC hardware decoders and a compute shader based
>>>> implementation, since the two share the same code, except for decode
>>>> submissions.
>>>
>>> 1. If you need a reference to the packet's data, then reference
>>> AVPacket.buf, not the whole AVPacket. This avoids allocating a spare
>>> AVPacket as well as copying side data.
>>> 2. It sounds very wrong and fragile that the decoder has to keep a
>>> reference because the hwaccel might need it. There may be future
>>> hwaccels that don't need such a reference etc. It seems better to extend
>>> e.g. the start_frame callback and pass a reference to the input data (no
>>> need to change this for all other start_frame calls; they can pass NULL
>>> until needed).
>>> 3. If the user closes the decoder (which is allowed at any time, even
>>> without draining the decoder), ff_codec_close() uninitializes the
>>> hwaccel after calling the decoder's close function; the latter
>>> unreferences the reference to the packet. Is this really safe?
>>
>> I wanted to avoid having to change the FFHWAccel API for a single
>> hwaccel, so I opted to simply add a field to the codec private context.
>> I'll add an argument to start_frame and resubmit.
>>
>> I don't see why it wouldn't be safe, upon uninit, all submissions are
>> waited on to complete before unreferencing all resources they held.
>> The output frames themselves have no references to any resources used
>> during decoding, so all resources are guaranteed to be freed at uninit.
>
> The problem is that codec close might free the packet's data and the
> decoder could still process it (or what guarantees that it doesn't?).
The reference to the packet that the hwaccel keeps.
If a packet is not refcounted, we fallback on copying automatically.
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
prev parent reply other threads:[~2025-03-13 18:45 UTC|newest]
Thread overview: 22+ messages / expand[flat|nested] mbox.gz Atom feed top
2025-03-10 3:08 [FFmpeg-devel] [PATCH 01/13] vulkan: rename ff_vk_set_descriptor_image to ff_vk_shader_update_img Lynne
2025-03-10 3:08 ` [FFmpeg-devel] [PATCH 02/13] vulkan: add ff_vk_create_imageview Lynne
2025-03-10 3:08 ` [FFmpeg-devel] [PATCH 03/13] vulkan: copy host-mapping buffer code from hwcontext Lynne
2025-03-10 3:08 ` [FFmpeg-devel] [PATCH 04/13] vulkan: workaround BGR storage image undefined behaviour Lynne
2025-03-10 3:08 ` [FFmpeg-devel] [PATCH 05/13] vulkan_decode: support software-defined decoders Lynne
2025-03-10 3:08 ` [FFmpeg-devel] [PATCH 06/13] vulkan_decode: support multiple image views Lynne
2025-03-10 3:08 ` [FFmpeg-devel] [PATCH 07/13] vulkan_decode: adjust number of async contexts created Lynne
2025-03-10 3:08 ` [FFmpeg-devel] [PATCH 08/13] ffv1enc_vulkan: refactor shaders slightly to support sharing Lynne
2025-03-10 3:08 ` [FFmpeg-devel] [PATCH 09/13] vulkan: unify handling of BGR and simplify ffv1_rct Lynne
2025-03-10 3:08 ` [FFmpeg-devel] [PATCH 10/13] ffv1dec: add support for hwaccels Lynne
2025-03-10 3:08 ` [FFmpeg-devel] [PATCH 11/13] ffv1dec: reference the current packet into the main context Lynne
2025-03-10 3:08 ` [FFmpeg-devel] [PATCH 12/13] vulkan: add ff_vk_exec_add_dep_wait_sem() Lynne
2025-03-10 3:08 ` [FFmpeg-devel] [PATCH 13/13] ffv1: add a Vulkan-based decoder Lynne
2025-03-10 3:14 ` [FFmpeg-devel] [PATCH 11/13] ffv1dec: reference the current packet into the main context Andreas Rheinhardt
2025-03-10 17:42 ` Lynne
2025-03-13 0:30 ` Lynne
2025-03-13 1:24 ` Andreas Rheinhardt
2025-03-13 1:56 ` Lynne
2025-03-13 4:57 ` Andreas Rheinhardt
2025-03-13 12:51 ` Lynne
2025-03-13 17:48 ` Andreas Rheinhardt
2025-03-13 18:45 ` Lynne [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=964e0504-efda-4f29-a567-225bf250b075@lynne.ee \
--to=dev@lynne.ee \
--cc=ffmpeg-devel@ffmpeg.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Git Inbox Mirror of the ffmpeg-devel mailing list - see https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
This inbox may be cloned and mirrored by anyone:
git clone --mirror https://master.gitmailbox.com/ffmpegdev/0 ffmpegdev/git/0.git
# If you have public-inbox 1.1+ installed, you may
# initialize and index your mirror using the following commands:
public-inbox-init -V2 ffmpegdev ffmpegdev/ https://master.gitmailbox.com/ffmpegdev \
ffmpegdev@gitmailbox.com
public-inbox-index ffmpegdev
Example config snippet for mirrors.
AGPL code for this site: git clone https://public-inbox.org/public-inbox.git