From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTP id 1748E41A43 for ; Tue, 19 Jul 2022 16:56:15 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 8D12668B607; Tue, 19 Jul 2022 19:56:12 +0300 (EEST) Received: from mail-wm1-f50.google.com (mail-wm1-f50.google.com [209.85.128.50]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 7976E68A467 for ; Tue, 19 Jul 2022 19:56:06 +0300 (EEST) Received: by mail-wm1-f50.google.com with SMTP id p26-20020a1c545a000000b003a2fb7c1274so7788851wmi.1 for ; Tue, 19 Jul 2022 09:56:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=YITKel03Snormo2Fa/J70IbQzMTC/5cw4tARiiJyb8s=; b=MxFBMVo9O6np8214hCs66N0ah5CO9LpxDlHv8nv7zesWUa2bcLLmuuRLW+m+sNnsKL c2FqZK3bArD0y5mnoQ3iu3dlnpHGy1b1Nm0WO1MQF0wfmu0yR3T87piIZ49LfhC+I2It +SYBY0jxxTeZbEuXg7r6O6BE5AFku4HXMWiw8lbn2pc2uGWgWyt+JB9ERsAt7FCvy64w iUaZdyQH6fWqRyKjwFRfnHjxKXGUBAd0ZvtIWqoraGSYDAiSeCJs3C8ZtnDS3RXVR0SY MJIvV4GgceeTB8XASy/QNmg16CRVuyhPOXAwXMWNyWHYALx19vdOjeRFnDlK9/yI4VJ7 +MoQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=YITKel03Snormo2Fa/J70IbQzMTC/5cw4tARiiJyb8s=; b=LL7aLmvX51/p0aPgk1e4G4b3bDGsgIqPF3Uegm/Ci7HtUoHymZbblAvL5YiCW/Mgcs Zb2MUJ0Cm5SC+OJfY7Oo31Gq5yFumUcFkg2vQ7CTSX9VbXxKb0s2oIJ6NNwYmttCEnAb KUj9C+QCJ3ocCO/8bqZEqwhQlYIxKECOh5uDG7Rj+SkD+cCmJgivpPgqj57+QgGlNkxR fiigs8FbAqoTAB7iGa3p9qZhKBysSUz45PjO+nKkkNlk9y+uLs4TNcUAf/98H6NvbVYu AggrM7BwXzvjYYnXOy5p7o35VHkJPl+3TpXPbdK8htfiiy2GOJXVc3M30r+AY301RUHk jfvQ== X-Gm-Message-State: AJIora8v7xhZWqSGH/VtbLlc+dOmIFWDnY5nb2mIB2A17BZrk80MOwmp WFeMVjV6FiKEN457g2cLExwioB5R7NyL6Q== X-Google-Smtp-Source: AGRyM1s8oL/fkA1ok4YEi82j/gFeUaj29SLc5gAPD60y/q5Oef5gdD2ethsBKuEs2UWTVFHUlvg/Mw== X-Received: by 2002:a05:600c:3845:b0:3a3:227d:c1c6 with SMTP id s5-20020a05600c384500b003a3227dc1c6mr226619wmr.9.1658249765597; Tue, 19 Jul 2022 09:56:05 -0700 (PDT) Received: from localhost ([2a00:5f00:102:0:ed02:7072:7dbf:2431]) by smtp.gmail.com with UTF8SMTPSA id n11-20020a05600c294b00b003a325bd8517sm2389553wmd.5.2022.07.19.09.56.05 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 19 Jul 2022 09:56:05 -0700 (PDT) From: Emil Velikov To: ffmpeg-devel@ffmpeg.org Date: Tue, 19 Jul 2022 17:56:02 +0100 Message-Id: <20220719165604.640770-1-emil.l.velikov@gmail.com> X-Mailer: git-send-email 2.37.0 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 1/3] hwcontext_vaapi: do not link against libva-x11.so X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Mark Thompson , emil.l.velikov@gmail.com Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: From: Emil Velikov There is an internal ABI between libva.so the libva-XXX.so libraries. With a recent change, the internal va_fool API was removed breaking the ABI. So if libva.so and libva-x11.so are from different version, the whole stack will crash. Instead we can dlopen() the libva-x11 library and gracefully error out. Cc: Mark Thompson Signed-off-by: Emil Velikov --- Greetings team, please consider backporting this for the stable releases. I've noticed that we've got plenty of pre-existing memory and state leaks in vaapi_device_create(). Would it make sense to fix those up? If so I can follow-up with a patch. --- configure | 2 +- libavutil/hwcontext_vaapi.c | 34 +++++++++++++++++++++++++++++++++- 2 files changed, 34 insertions(+), 2 deletions(-) diff --git a/configure b/configure index 18d9b61a99..ab2a29544c 100755 --- a/configure +++ b/configure @@ -3816,7 +3816,7 @@ swscale_suggest="libm stdatomic" avcodec_extralibs="pthreads_extralibs iconv_extralibs dxva2_extralibs" avfilter_extralibs="pthreads_extralibs" -avutil_extralibs="d3d11va_extralibs nanosleep_extralibs pthreads_extralibs vaapi_drm_extralibs vaapi_x11_extralibs vdpau_x11_extralibs" +avutil_extralibs="d3d11va_extralibs nanosleep_extralibs pthreads_extralibs vaapi_drm_extralibs vdpau_x11_extralibs" # programs ffmpeg_deps="avcodec avfilter avformat" diff --git a/libavutil/hwcontext_vaapi.c b/libavutil/hwcontext_vaapi.c index c3a98bc4b1..e44d324928 100644 --- a/libavutil/hwcontext_vaapi.c +++ b/libavutil/hwcontext_vaapi.c @@ -18,8 +18,16 @@ #include "config.h" +#if CONFIG_VAAPI_1 +# define VA_ABI ".2" +#else +# define VA_ABI ".1" +#endif + #if HAVE_VAAPI_X11 # include +# include +# define VA_X11_LIB "libva-x11.so" VA_ABI #endif #if HAVE_VAAPI_DRM # include @@ -54,6 +62,7 @@ typedef struct VAAPIDevicePriv { #if HAVE_VAAPI_X11 + void *libva_x11; Display *x11_display; #endif @@ -1565,6 +1574,8 @@ static void vaapi_device_free(AVHWDeviceContext *ctx) vaTerminate(hwctx->display); #if HAVE_VAAPI_X11 + if (priv->libva_x11) + dlclose(priv->libva_x11); if (priv->x11_display) XCloseDisplay(priv->x11_display); #endif @@ -1723,14 +1734,35 @@ static int vaapi_device_create(AVHWDeviceContext *ctx, const char *device, #if HAVE_VAAPI_X11 if (!display && try_x11) { + VADisplay (*GetDisplay)(Display *dpy); + + priv->libva_x11 = dlopen(VA_X11_LIB, RTLD_NOW | RTLD_LOCAL); + if (!priv->libva_x11) { + av_log(ctx, AV_LOG_ERROR, "Cannot open %s library %s.\n", + VA_X11_LIB, dlerror()); + return AVERROR_UNKNOWN; + } + + GetDisplay = dlsym(priv->libva_x11, "vaGetDisplay"); + if (!GetDisplay) { + av_log(ctx, AV_LOG_ERROR, "Cannot retrieve %s entrypoint %s.\n", + "vaGetDisplay", dlerror()); + // Always dlclose after the dlerror(). The former can alter the + // error string returned by the latter. + dlclose(priv->libva_x11); + return AVERROR_UNKNOWN; + } + // Try to open the device as an X11 display. priv->x11_display = XOpenDisplay(device); if (!priv->x11_display) { + dlclose(priv->libva_x11); av_log(ctx, AV_LOG_VERBOSE, "Cannot open X11 display " "%s.\n", XDisplayName(device)); } else { - display = vaGetDisplay(priv->x11_display); + display = GetDisplay(priv->x11_display); if (!display) { + dlclose(priv->libva_x11); av_log(ctx, AV_LOG_ERROR, "Cannot open a VA display " "from X11 display %s.\n", XDisplayName(device)); return AVERROR_UNKNOWN; -- 2.37.0 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".