[FFmpeg-devel] [PATCH] lavc/vulkan_video: fix double-free and leak (PR #21173)

[cgutman via ffmpeg-devel <ffmpeg-devel@ffmpeg.org>]

PR #21173 opened by cgutman
URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/21173
Patch URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/21173.patch

I ran into this double-free when `ff_vk_alloc_mem()` failed in `ff_vk_video_common_init()` because the GPU was out of memory. In this case, `ff_vk_video_common_init()` calls `ff_vk_video_common_uninit()` in the `fail:` path which leaves dangling object handles in `FFVkVideoCommon`. Those get freed again when the destructor of `FFVulkanDecodeShared` calls `ff_vk_video_common_uninit()` a second time. I also included a simple leak fix that I found while investigating.

This should be cherry-picked into `release/8.0` and `release/7.1`, but I can also create new PRs for that if necessary.


>From d72a59dbd3df829e06b5229cab93e3989cdba909 Mon Sep 17 00:00:00 2001
From: Cameron Gutman <aicommander@gmail.com>
Date: Thu, 11 Dec 2025 17:39:16 -0600
Subject: [PATCH 1/2] lavc/vulkan_video: fix double-free if ff_vk_decode_init()
 fails

ff_vk_video_common_init() calls ff_vk_video_common_uninit() on failure
which leaves dangling object handles. Those get freed again when the
destructor of FFVulkanDecodeShared calls ff_vk_video_common_uninit()
again.

Signed-off-by: Cameron Gutman <aicommander@gmail.com>
---
 libavcodec/vulkan_video.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/libavcodec/vulkan_video.c b/libavcodec/vulkan_video.c
index 819940460f..66fc493ff1 100644
--- a/libavcodec/vulkan_video.c
+++ b/libavcodec/vulkan_video.c
@@ -349,17 +349,21 @@ av_cold void ff_vk_video_common_uninit(FFVulkanContext *s,
 
     av_freep(&common->mem);
 
-    if (common->layered_view)
+    if (common->layered_view) {
         vk->DestroyImageView(s->hwctx->act_dev, common->layered_view,
                              s->hwctx->alloc);
+        common->layered_view = VK_NULL_HANDLE;
+    }
 
     av_frame_free(&common->layered_frame);
 
     av_buffer_unref(&common->dpb_hwfc_ref);
 
-    if (common->yuv_sampler)
+    if (common->yuv_sampler) {
         vk->DestroySamplerYcbcrConversion(s->hwctx->act_dev, common->yuv_sampler,
                                           s->hwctx->alloc);
+        common->yuv_sampler = VK_NULL_HANDLE;
+    }
 }
 
 av_cold int ff_vk_video_common_init(AVCodecContext *avctx, FFVulkanContext *s,
-- 
2.49.1


>From cb01782b90bef5af569e3ff9940b9803f7033a8c Mon Sep 17 00:00:00 2001
From: Cameron Gutman <aicommander@gmail.com>
Date: Thu, 11 Dec 2025 21:52:37 -0600
Subject: [PATCH 2/2] lavc/vulkan_video: fix leak on CreateVideoSessionKHR
 failure

Signed-off-by: Cameron Gutman <aicommander@gmail.com>
---
 libavcodec/vulkan_video.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/libavcodec/vulkan_video.c b/libavcodec/vulkan_video.c
index 66fc493ff1..d73479d14d 100644
--- a/libavcodec/vulkan_video.c
+++ b/libavcodec/vulkan_video.c
@@ -398,8 +398,10 @@ av_cold int ff_vk_video_common_init(AVCodecContext *avctx, FFVulkanContext *s,
     /* Create session */
     ret = vk->CreateVideoSessionKHR(s->hwctx->act_dev, session_create,
                                     s->hwctx->alloc, &common->session);
-    if (ret != VK_SUCCESS)
-        return AVERROR_EXTERNAL;
+    if (ret != VK_SUCCESS) {
+        err = AVERROR_EXTERNAL;
+        goto fail;
+    }
 
     /* Get memory requirements */
     ret = vk->GetVideoSessionMemoryRequirementsKHR(s->hwctx->act_dev,
-- 
2.49.1

_______________________________________________
ffmpeg-devel mailing list -- ffmpeg-devel@ffmpeg.org
To unsubscribe send an email to ffmpeg-devel-leave@ffmpeg.org