* [FFmpeg-devel] [PATCH v7 00/10] make QSV works with the Intel's oneVPL
@ 2022-03-11 8:16 Xiang, Haihao
2022-03-11 8:16 ` [FFmpeg-devel] [PATCH v7 01/10] configure: ensure --enable-libmfx uses libmfx 1.x Xiang, Haihao
` (9 more replies)
0 siblings, 10 replies; 26+ messages in thread
From: Xiang, Haihao @ 2022-03-11 8:16 UTC (permalink / raw)
To: ffmpeg-devel; +Cc: Haihao Xiang
From: Haihao Xiang <haihao.xiang@intel.com>
The oneAPI Video Processing Library (oneVPL) is a single interface for
encode, decode and video processing[1][2]. oneVPL is a successor to Intel(R) Media
SDK, but removed obsolete features. Intel(R) Media SDK lifetime comes to an
end now, new features for new Intel Gen platforms will be supported in oneVPL
only[3].
It is recommended to use oneVPL for new work, even for currently available
hardwares[4]. Hence, this patchset added a new option --enable-onevpl to bring
the support for oneVPL in QSV, new features for oneVPL will be implemented in
other patchset, for example, we are implementing av1_qsv encoder
(https://github.com/intel-media-ci/ffmpeg/pull/515).
option --enble-libmfx still works with Intel(R) Media SDK.
Note user can't enable onevpl and libmfx together.
oneVPL dispatcher:
https://github.com/oneapi-src/oneVPL
oneVPL GPU runtime for new Intel Gen platforms:
https://github.com/oneapi-src/oneVPL-intel-gpu
v7:
- Changed the required minimal oneVPL version from 2.2 to 2.6
- Added support for multiple GPUs on Linux
- Rebased this patchset against the latest master and fixed bugs
[1] https://spec.oneapi.io/versions/latest/elements/oneVPL/source/index.html
[2] https://www.intel.com/content/www/us/en/developer/tools/oneapi/onevpl.html
[3] https://github.com/Intel-Media-SDK/MediaSDK/#media-sdk-support-matrix
[4] https://www.intel.com/content/www/us/en/develop/documentation/upgrading-from-msdk-to-onevpl/top.html
Haihao Xiang (10):
configure: ensure --enable-libmfx uses libmfx 1.x
configure: fix the check for MFX_CODEC_VP9
qsv: remove mfx/ prefix from mfx headers
qsv: load user plugin for MFX_VERSION < 2.0
qsv: build audio related code when MFX_VERSION < 2.0
qsvenc: support multi-frame encode when MFX_VERSION < 2.0
qsvenc: support MFX_RATECONTROL_LA_EXT when MFX_VERSION < 2.0
qsv: support OPAQUE memory when MFX_VERSION < 2.0
qsv: use a new method to create mfx session when using oneVPL
configure: add --enable-libvpl option
configure | 29 +-
libavcodec/qsv.c | 220 +++++++++++++--
libavcodec/qsv.h | 4 +-
libavcodec/qsv_internal.h | 6 +-
libavcodec/qsvdec.c | 21 +-
libavcodec/qsvenc.c | 25 +-
libavcodec/qsvenc.h | 9 +-
libavcodec/qsvenc_h264.c | 3 +-
libavcodec/qsvenc_hevc.c | 3 +-
libavcodec/qsvenc_jpeg.c | 3 +-
libavcodec/qsvenc_mpeg2.c | 3 +-
libavcodec/qsvenc_vp9.c | 3 +-
libavfilter/qsvvpp.c | 145 +++++++++-
libavfilter/qsvvpp.h | 12 +-
libavfilter/vf_deinterlace_qsv.c | 73 ++---
libavfilter/vf_scale_qsv.c | 88 +++---
libavutil/hwcontext_d3d11va.c | 13 +
libavutil/hwcontext_d3d11va.h | 5 +
libavutil/hwcontext_dxva2.c | 8 +
libavutil/hwcontext_dxva2.h | 4 +
libavutil/hwcontext_opencl.c | 2 +-
libavutil/hwcontext_qsv.c | 444 +++++++++++++++++++++++++++----
libavutil/hwcontext_qsv.h | 3 +-
libavutil/hwcontext_vaapi.c | 13 +
libavutil/hwcontext_vaapi.h | 4 +
25 files changed, 956 insertions(+), 187 deletions(-)
--
2.17.1
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
^ permalink raw reply [flat|nested] 26+ messages in thread* [FFmpeg-devel] [PATCH v7 01/10] configure: ensure --enable-libmfx uses libmfx 1.x 2022-03-11 8:16 [FFmpeg-devel] [PATCH v7 00/10] make QSV works with the Intel's oneVPL Xiang, Haihao @ 2022-03-11 8:16 ` Xiang, Haihao 2022-03-11 8:16 ` [FFmpeg-devel] [PATCH v7 02/10] configure: fix the check for MFX_CODEC_VP9 Xiang, Haihao ` (8 subsequent siblings) 9 siblings, 0 replies; 26+ messages in thread From: Xiang, Haihao @ 2022-03-11 8:16 UTC (permalink / raw) To: ffmpeg-devel; +Cc: Haihao Xiang From: Haihao Xiang <haihao.xiang@intel.com> Intel's oneVPL is a successor to MediaSDK, but removed some obsolete features of MediaSDK[1]. Some early versions of oneVPL still uses libmfx as library name[2], however some of obsolete features, including OPAQUE memory, multi-frame encode, user plugins and LA_EXT rate control mode etc, have been enabled in QSV, so user can not use --enable-libmfx to enable QSV if using an early version of oneVPL SDK. In order to make sure user builds FFmpeg against a right version of libmfx, this patch added a check for the version of libmfx and warning message about the used obsolete features. [1] https://spec.oneapi.io/versions/latest/elements/oneVPL/source/VPL_intel_media_sdk.html [2] https://github.com/oneapi-src/oneVPL --- configure | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/configure b/configure index 7d22c2a345..f3c60d9e40 100755 --- a/configure +++ b/configure @@ -6548,8 +6548,11 @@ enabled liblensfun && require_pkg_config liblensfun lensfun lensfun.h lf_ # Media SDK or Intel Media Server Studio, these don't come with # pkg-config support. Instead, users should make sure that the build # can find the libraries and headers through other means. -enabled libmfx && { check_pkg_config libmfx libmfx "mfx/mfxvideo.h" MFXInit || - { require libmfx "mfx/mfxvideo.h" MFXInit "-llibmfx $advapi32_extralibs" && warn "using libmfx without pkg-config"; } } +enabled libmfx && { { check_pkg_config libmfx "libmfx < 2.0" "mfx/mfxvideo.h" MFXInit || + { require "libmfx < 2.0" "mfx/mfxvideo.h" MFXInit "-llibmfx $advapi32_extralibs" && warn "using libmfx without pkg-config"; } } && + warn "build FFmpeg against libmfx 1.x, obsolete features of libmfx such as OPAQUE memory,\n"\ + "multi-frame encode, user plugins and LA_EXT rate control mode are enabled"; } + if enabled libmfx; then check_cc MFX_CODEC_VP9 "mfx/mfxvp9.h mfx/mfxstructures.h" "MFX_CODEC_VP9" fi -- 2.17.1 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 26+ messages in thread
* [FFmpeg-devel] [PATCH v7 02/10] configure: fix the check for MFX_CODEC_VP9 2022-03-11 8:16 [FFmpeg-devel] [PATCH v7 00/10] make QSV works with the Intel's oneVPL Xiang, Haihao 2022-03-11 8:16 ` [FFmpeg-devel] [PATCH v7 01/10] configure: ensure --enable-libmfx uses libmfx 1.x Xiang, Haihao @ 2022-03-11 8:16 ` Xiang, Haihao 2022-03-11 8:16 ` [FFmpeg-devel] [PATCH v7 03/10] qsv: remove mfx/ prefix from mfx headers Xiang, Haihao ` (7 subsequent siblings) 9 siblings, 0 replies; 26+ messages in thread From: Xiang, Haihao @ 2022-03-11 8:16 UTC (permalink / raw) To: ffmpeg-devel; +Cc: Haihao Xiang From: Haihao Xiang <haihao.xiang@intel.com> The data structures for VP9 in mfxvp9.h is wrapped by MFX_VERSION_NEXT, which means those data structures have never been used in a public release. Actually MFX_CODEC_VP9 and other VP9 stuffs is added in mfxstructures.h. In addition, mfxdefs.h is included in mfxvp9.h, so we may use the check in this patch for MFX_CODEC_VP9 This is in preparation for oneVPL support because mfxvp9.h is removed from oneVPL [1] [1]: https://github.com/oneapi-src/oneVPL --- configure | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/configure b/configure index f3c60d9e40..31d4f4928b 100755 --- a/configure +++ b/configure @@ -6554,7 +6554,7 @@ enabled libmfx && { { check_pkg_config libmfx "libmfx < 2.0" "mfx/mfx "multi-frame encode, user plugins and LA_EXT rate control mode are enabled"; } if enabled libmfx; then - check_cc MFX_CODEC_VP9 "mfx/mfxvp9.h mfx/mfxstructures.h" "MFX_CODEC_VP9" + check_cc MFX_CODEC_VP9 "mfx/mfxdefs.h mfx/mfxstructures.h" "MFX_CODEC_VP9" fi enabled libmodplug && require_pkg_config libmodplug libmodplug libmodplug/modplug.h ModPlug_Load -- 2.17.1 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 26+ messages in thread
* [FFmpeg-devel] [PATCH v7 03/10] qsv: remove mfx/ prefix from mfx headers 2022-03-11 8:16 [FFmpeg-devel] [PATCH v7 00/10] make QSV works with the Intel's oneVPL Xiang, Haihao 2022-03-11 8:16 ` [FFmpeg-devel] [PATCH v7 01/10] configure: ensure --enable-libmfx uses libmfx 1.x Xiang, Haihao 2022-03-11 8:16 ` [FFmpeg-devel] [PATCH v7 02/10] configure: fix the check for MFX_CODEC_VP9 Xiang, Haihao @ 2022-03-11 8:16 ` Xiang, Haihao 2022-03-11 8:16 ` [FFmpeg-devel] [PATCH v7 04/10] qsv: load user plugin for MFX_VERSION < 2.0 Xiang, Haihao ` (6 subsequent siblings) 9 siblings, 0 replies; 26+ messages in thread From: Xiang, Haihao @ 2022-03-11 8:16 UTC (permalink / raw) To: ffmpeg-devel; +Cc: Haihao Xiang From: Haihao Xiang <haihao.xiang@intel.com> The following Cflags has been added to libmfx.pc, so mfx/ prefix is no longer needed when including mfx headers in FFmpeg. Cflags: -I${includedir} -I${includedir}/mfx Some old versions of libmfx have the following Cflags in libmfx.pc Cflags: -I${includedir} We may add -I${includedir}/mfx to CFLAGS when running 'configure --enable-libmfx' for old versions of libmfx, if so, mfx headers without mfx/ prefix can be included too. If libmfx comes without pkg-config support, we may do a small change to the settings of the environment(e.g. set -I/opt/intel/mediasdk/include/mfx instead of -I/opt/intel/mediasdk/include to CFLAGS), then the build can find the mfx headers without mfx/ prefix After applying this change, we won't need to change #include for mfx headers when mfx headers are installed under a new directory. This is in preparation for oneVPL support (mfx headers in oneVPL are installed under vpl directory) --- configure | 13 +++++++++---- libavcodec/qsv.c | 8 ++++---- libavcodec/qsv.h | 2 +- libavcodec/qsv_internal.h | 2 +- libavcodec/qsvdec.c | 2 +- libavcodec/qsvenc.c | 2 +- libavcodec/qsvenc.h | 2 +- libavcodec/qsvenc_h264.c | 2 +- libavcodec/qsvenc_hevc.c | 2 +- libavcodec/qsvenc_jpeg.c | 2 +- libavcodec/qsvenc_mpeg2.c | 2 +- libavcodec/qsvenc_vp9.c | 2 +- libavfilter/qsvvpp.h | 2 +- libavfilter/vf_deinterlace_qsv.c | 2 +- libavfilter/vf_scale_qsv.c | 2 +- libavutil/hwcontext_opencl.c | 2 +- libavutil/hwcontext_qsv.c | 2 +- libavutil/hwcontext_qsv.h | 2 +- 18 files changed, 29 insertions(+), 24 deletions(-) diff --git a/configure b/configure index 31d4f4928b..eacdf03c18 100755 --- a/configure +++ b/configure @@ -6548,13 +6548,18 @@ enabled liblensfun && require_pkg_config liblensfun lensfun lensfun.h lf_ # Media SDK or Intel Media Server Studio, these don't come with # pkg-config support. Instead, users should make sure that the build # can find the libraries and headers through other means. -enabled libmfx && { { check_pkg_config libmfx "libmfx < 2.0" "mfx/mfxvideo.h" MFXInit || - { require "libmfx < 2.0" "mfx/mfxvideo.h" MFXInit "-llibmfx $advapi32_extralibs" && warn "using libmfx without pkg-config"; } } && + +enabled libmfx && { { check_pkg_config libmfx "libmfx < 2.0" "mfxvideo.h" MFXInit || +# Some old versions of libmfx have the following settings in libmfx.pc: +# includedir=/usr/include +# Cflags: -I${includedir} +# So add -I${includedir}/mfx to CFLAGS + { check_pkg_config libmfx "libmfx < 2.0" "mfx/mfxvideo.h" MFXInit && add_cflags -I${libmfx_incdir}/mfx; } || + { require "libmfx < 2.0" "mfxvideo.h" MFXInit "-llibmfx $advapi32_extralibs" && warn "using libmfx without pkg-config"; } } && warn "build FFmpeg against libmfx 1.x, obsolete features of libmfx such as OPAQUE memory,\n"\ "multi-frame encode, user plugins and LA_EXT rate control mode are enabled"; } - if enabled libmfx; then - check_cc MFX_CODEC_VP9 "mfx/mfxdefs.h mfx/mfxstructures.h" "MFX_CODEC_VP9" + check_cc MFX_CODEC_VP9 "mfxdefs.h mfxstructures.h" "MFX_CODEC_VP9" fi enabled libmodplug && require_pkg_config libmodplug libmodplug libmodplug/modplug.h ModPlug_Load diff --git a/libavcodec/qsv.c b/libavcodec/qsv.c index 67d0e3934a..f749e77d3a 100644 --- a/libavcodec/qsv.c +++ b/libavcodec/qsv.c @@ -18,9 +18,9 @@ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA */ -#include <mfx/mfxvideo.h> -#include <mfx/mfxplugin.h> -#include <mfx/mfxjpeg.h> +#include <mfxvideo.h> +#include <mfxplugin.h> +#include <mfxjpeg.h> #include <stdio.h> #include <string.h> @@ -39,7 +39,7 @@ #define MFX_IMPL_VIA_MASK(impl) (0x0f00 & (impl)) #if QSV_VERSION_ATLEAST(1, 12) -#include "mfx/mfxvp8.h" +#include "mfxvp8.h" #endif int ff_qsv_codec_id_to_mfx(enum AVCodecID codec_id) diff --git a/libavcodec/qsv.h b/libavcodec/qsv.h index b77158ec26..04ae0d6f34 100644 --- a/libavcodec/qsv.h +++ b/libavcodec/qsv.h @@ -21,7 +21,7 @@ #ifndef AVCODEC_QSV_H #define AVCODEC_QSV_H -#include <mfx/mfxvideo.h> +#include <mfxvideo.h> #include "libavutil/buffer.h" diff --git a/libavcodec/qsv_internal.h b/libavcodec/qsv_internal.h index 58186ea7ca..09fc76f6c4 100644 --- a/libavcodec/qsv_internal.h +++ b/libavcodec/qsv_internal.h @@ -39,7 +39,7 @@ #include "libavutil/hwcontext_vaapi.h" #endif -#include <mfx/mfxvideo.h> +#include <mfxvideo.h> #include "libavutil/frame.h" diff --git a/libavcodec/qsvdec.c b/libavcodec/qsvdec.c index 6236391357..1a492da74b 100644 --- a/libavcodec/qsvdec.c +++ b/libavcodec/qsvdec.c @@ -25,7 +25,7 @@ #include <string.h> #include <sys/types.h> -#include <mfx/mfxvideo.h> +#include <mfxvideo.h> #include "libavutil/common.h" #include "libavutil/fifo.h" diff --git a/libavcodec/qsvenc.c b/libavcodec/qsvenc.c index 9b71487666..0170ae14e4 100644 --- a/libavcodec/qsvenc.c +++ b/libavcodec/qsvenc.c @@ -23,7 +23,7 @@ #include <string.h> #include <sys/types.h> -#include <mfx/mfxvideo.h> +#include <mfxvideo.h> #include "libavutil/common.h" #include "libavutil/hwcontext.h" diff --git a/libavcodec/qsvenc.h b/libavcodec/qsvenc.h index 2bda858427..559d7cbe3e 100644 --- a/libavcodec/qsvenc.h +++ b/libavcodec/qsvenc.h @@ -26,7 +26,7 @@ #include <stdint.h> #include <sys/types.h> -#include <mfx/mfxvideo.h> +#include <mfxvideo.h> #include "libavutil/avutil.h" #include "libavutil/fifo.h" diff --git a/libavcodec/qsvenc_h264.c b/libavcodec/qsvenc_h264.c index bec3633268..7c6f3a530c 100644 --- a/libavcodec/qsvenc_h264.c +++ b/libavcodec/qsvenc_h264.c @@ -24,7 +24,7 @@ #include <stdint.h> #include <sys/types.h> -#include <mfx/mfxvideo.h> +#include <mfxvideo.h> #include "libavutil/common.h" #include "libavutil/opt.h" diff --git a/libavcodec/qsvenc_hevc.c b/libavcodec/qsvenc_hevc.c index ade546d4ca..e383caf238 100644 --- a/libavcodec/qsvenc_hevc.c +++ b/libavcodec/qsvenc_hevc.c @@ -22,7 +22,7 @@ #include <stdint.h> #include <sys/types.h> -#include <mfx/mfxvideo.h> +#include <mfxvideo.h> #include "libavutil/common.h" #include "libavutil/opt.h" diff --git a/libavcodec/qsvenc_jpeg.c b/libavcodec/qsvenc_jpeg.c index dd082692be..ad8f09befe 100644 --- a/libavcodec/qsvenc_jpeg.c +++ b/libavcodec/qsvenc_jpeg.c @@ -22,7 +22,7 @@ #include <stdint.h> #include <sys/types.h> -#include <mfx/mfxvideo.h> +#include <mfxvideo.h> #include "libavutil/common.h" #include "libavutil/opt.h" diff --git a/libavcodec/qsvenc_mpeg2.c b/libavcodec/qsvenc_mpeg2.c index 525df99e50..610bbf79c1 100644 --- a/libavcodec/qsvenc_mpeg2.c +++ b/libavcodec/qsvenc_mpeg2.c @@ -22,7 +22,7 @@ #include <stdint.h> #include <sys/types.h> -#include <mfx/mfxvideo.h> +#include <mfxvideo.h> #include "libavutil/common.h" #include "libavutil/opt.h" diff --git a/libavcodec/qsvenc_vp9.c b/libavcodec/qsvenc_vp9.c index 1168ddda0e..5a0c04481d 100644 --- a/libavcodec/qsvenc_vp9.c +++ b/libavcodec/qsvenc_vp9.c @@ -22,7 +22,7 @@ #include <stdint.h> #include <sys/types.h> -#include <mfx/mfxvideo.h> +#include <mfxvideo.h> #include "libavutil/common.h" #include "libavutil/opt.h" diff --git a/libavfilter/qsvvpp.h b/libavfilter/qsvvpp.h index 4fe07ab1f7..543c58a967 100644 --- a/libavfilter/qsvvpp.h +++ b/libavfilter/qsvvpp.h @@ -24,7 +24,7 @@ #ifndef AVFILTER_QSVVPP_H #define AVFILTER_QSVVPP_H -#include <mfx/mfxvideo.h> +#include <mfxvideo.h> #include "avfilter.h" #include "libavutil/fifo.h" diff --git a/libavfilter/vf_deinterlace_qsv.c b/libavfilter/vf_deinterlace_qsv.c index fb54d179ed..b8ff3e8339 100644 --- a/libavfilter/vf_deinterlace_qsv.c +++ b/libavfilter/vf_deinterlace_qsv.c @@ -21,7 +21,7 @@ * deinterlace video filter - QSV */ -#include <mfx/mfxvideo.h> +#include <mfxvideo.h> #include <stdio.h> #include <string.h> diff --git a/libavfilter/vf_scale_qsv.c b/libavfilter/vf_scale_qsv.c index 371f629457..2ba7d086b7 100644 --- a/libavfilter/vf_scale_qsv.c +++ b/libavfilter/vf_scale_qsv.c @@ -21,7 +21,7 @@ * scale video filter - QSV */ -#include <mfx/mfxvideo.h> +#include <mfxvideo.h> #include <stdio.h> #include <string.h> diff --git a/libavutil/hwcontext_opencl.c b/libavutil/hwcontext_opencl.c index 4f4bd13405..55e4e566fa 100644 --- a/libavutil/hwcontext_opencl.c +++ b/libavutil/hwcontext_opencl.c @@ -47,7 +47,7 @@ #if HAVE_OPENCL_VAAPI_INTEL_MEDIA #if CONFIG_LIBMFX -#include <mfx/mfxstructures.h> +#include <mfxstructures.h> #endif #include <va/va.h> #include <CL/cl_va_api_media_sharing_intel.h> diff --git a/libavutil/hwcontext_qsv.c b/libavutil/hwcontext_qsv.c index 95f8071abe..dbe7bafb8e 100644 --- a/libavutil/hwcontext_qsv.c +++ b/libavutil/hwcontext_qsv.c @@ -20,7 +20,7 @@ #include <stdint.h> #include <string.h> -#include <mfx/mfxvideo.h> +#include <mfxvideo.h> #include "config.h" diff --git a/libavutil/hwcontext_qsv.h b/libavutil/hwcontext_qsv.h index b98d611cfc..42e34d0dda 100644 --- a/libavutil/hwcontext_qsv.h +++ b/libavutil/hwcontext_qsv.h @@ -19,7 +19,7 @@ #ifndef AVUTIL_HWCONTEXT_QSV_H #define AVUTIL_HWCONTEXT_QSV_H -#include <mfx/mfxvideo.h> +#include <mfxvideo.h> /** * @file -- 2.17.1 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 26+ messages in thread
* [FFmpeg-devel] [PATCH v7 04/10] qsv: load user plugin for MFX_VERSION < 2.0 2022-03-11 8:16 [FFmpeg-devel] [PATCH v7 00/10] make QSV works with the Intel's oneVPL Xiang, Haihao ` (2 preceding siblings ...) 2022-03-11 8:16 ` [FFmpeg-devel] [PATCH v7 03/10] qsv: remove mfx/ prefix from mfx headers Xiang, Haihao @ 2022-03-11 8:16 ` Xiang, Haihao 2022-03-11 8:16 ` [FFmpeg-devel] [PATCH v7 05/10] qsv: build audio related code when " Xiang, Haihao ` (5 subsequent siblings) 9 siblings, 0 replies; 26+ messages in thread From: Xiang, Haihao @ 2022-03-11 8:16 UTC (permalink / raw) To: ffmpeg-devel; +Cc: Haihao Xiang From: Haihao Xiang <haihao.xiang@intel.com> User plugin isn't supported for MFX_VERSION >= 2.0[1][2]. This is in preparation for oneVPL Support [1]: https://spec.oneapi.io/versions/latest/elements/oneVPL/source/VPL_intel_media_sdk.html#msdk-full-name-feature-removals [2]: https://github.com/oneapi-src/oneVPL --- libavcodec/qsv.c | 8 +++++++- libavcodec/qsv_internal.h | 2 ++ 2 files changed, 9 insertions(+), 1 deletion(-) diff --git a/libavcodec/qsv.c b/libavcodec/qsv.c index f749e77d3a..0eb0d83ac0 100644 --- a/libavcodec/qsv.c +++ b/libavcodec/qsv.c @@ -19,7 +19,6 @@ */ #include <mfxvideo.h> -#include <mfxplugin.h> #include <mfxjpeg.h> #include <stdio.h> @@ -37,11 +36,16 @@ #include "qsv_internal.h" #define MFX_IMPL_VIA_MASK(impl) (0x0f00 & (impl)) +#define QSV_HAVE_USER_PLUGIN !QSV_ONEVPL #if QSV_VERSION_ATLEAST(1, 12) #include "mfxvp8.h" #endif +#if QSV_HAVE_USER_PLUGIN +#include <mfxplugin.h> +#endif + int ff_qsv_codec_id_to_mfx(enum AVCodecID codec_id) { switch (codec_id) { @@ -307,6 +311,7 @@ enum AVPictureType ff_qsv_map_pictype(int mfx_pic_type) static int qsv_load_plugins(mfxSession session, const char *load_plugins, void *logctx) { +#if QSV_HAVE_USER_PLUGIN if (!load_plugins || !*load_plugins) return 0; @@ -350,6 +355,7 @@ load_plugin_fail: if (err < 0) return err; } +#endif return 0; diff --git a/libavcodec/qsv_internal.h b/libavcodec/qsv_internal.h index 09fc76f6c4..6d6a6eb239 100644 --- a/libavcodec/qsv_internal.h +++ b/libavcodec/qsv_internal.h @@ -62,6 +62,8 @@ ((MFX_VERSION.Major > (MAJOR)) || \ (MFX_VERSION.Major == (MAJOR) && MFX_VERSION.Minor >= (MINOR))) +#define QSV_ONEVPL QSV_VERSION_ATLEAST(2, 0) + typedef struct QSVMid { AVBufferRef *hw_frames_ref; mfxHDLPair *handle_pair; -- 2.17.1 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 26+ messages in thread
* [FFmpeg-devel] [PATCH v7 05/10] qsv: build audio related code when MFX_VERSION < 2.0 2022-03-11 8:16 [FFmpeg-devel] [PATCH v7 00/10] make QSV works with the Intel's oneVPL Xiang, Haihao ` (3 preceding siblings ...) 2022-03-11 8:16 ` [FFmpeg-devel] [PATCH v7 04/10] qsv: load user plugin for MFX_VERSION < 2.0 Xiang, Haihao @ 2022-03-11 8:16 ` Xiang, Haihao 2022-04-05 11:50 ` Anton Khirnov 2022-03-11 8:16 ` [FFmpeg-devel] [PATCH v7 06/10] qsvenc: support multi-frame encode " Xiang, Haihao ` (4 subsequent siblings) 9 siblings, 1 reply; 26+ messages in thread From: Xiang, Haihao @ 2022-03-11 8:16 UTC (permalink / raw) To: ffmpeg-devel; +Cc: Haihao Xiang From: Haihao Xiang <haihao.xiang@intel.com> Audio isn't supported for MFX_VERSION >= 2.0[1][2]. This is in preparation for oneVPL support [1]: https://spec.oneapi.io/versions/latest/elements/oneVPL/source/VPL_intel_media_sdk.html#msdk-full-name-feature-removals [2]: https://github.com/oneapi-src/oneVPL --- libavcodec/qsv.c | 5 +++++ libavfilter/qsvvpp.c | 6 ++++++ libavfilter/qsvvpp.h | 2 ++ 3 files changed, 13 insertions(+) diff --git a/libavcodec/qsv.c b/libavcodec/qsv.c index 0eb0d83ac0..063d6dfb0a 100644 --- a/libavcodec/qsv.c +++ b/libavcodec/qsv.c @@ -37,6 +37,7 @@ #define MFX_IMPL_VIA_MASK(impl) (0x0f00 & (impl)) #define QSV_HAVE_USER_PLUGIN !QSV_ONEVPL +#define QSV_HAVE_AUDIO !QSV_ONEVPL #if QSV_VERSION_ATLEAST(1, 12) #include "mfxvp8.h" @@ -137,8 +138,10 @@ static const struct { { MFX_ERR_INVALID_VIDEO_PARAM, AVERROR(EINVAL), "invalid video parameters" }, { MFX_ERR_UNDEFINED_BEHAVIOR, AVERROR_BUG, "undefined behavior" }, { MFX_ERR_DEVICE_FAILED, AVERROR(EIO), "device failed" }, +#if QSV_HAVE_AUDIO { MFX_ERR_INCOMPATIBLE_AUDIO_PARAM, AVERROR(EINVAL), "incompatible audio parameters" }, { MFX_ERR_INVALID_AUDIO_PARAM, AVERROR(EINVAL), "invalid audio parameters" }, +#endif { MFX_WRN_IN_EXECUTION, 0, "operation in execution" }, { MFX_WRN_DEVICE_BUSY, 0, "device busy" }, @@ -148,7 +151,9 @@ static const struct { { MFX_WRN_VALUE_NOT_CHANGED, 0, "value is saturated" }, { MFX_WRN_OUT_OF_RANGE, 0, "value out of range" }, { MFX_WRN_FILTER_SKIPPED, 0, "filter skipped" }, +#if QSV_HAVE_AUDIO { MFX_WRN_INCOMPATIBLE_AUDIO_PARAM, 0, "incompatible audio parameters" }, +#endif }; /** diff --git a/libavfilter/qsvvpp.c b/libavfilter/qsvvpp.c index 954f882637..3647891d13 100644 --- a/libavfilter/qsvvpp.c +++ b/libavfilter/qsvvpp.c @@ -38,6 +38,8 @@ #define IS_SYSTEM_MEMORY(mode) (mode & MFX_MEMTYPE_SYSTEM_MEMORY) #define MFX_IMPL_VIA_MASK(impl) (0x0f00 & (impl)) +#define QSV_HAVE_AUDIO !QSV_ONEVPL + static const AVRational default_tb = { 1, 90000 }; typedef struct QSVAsyncFrame { @@ -100,8 +102,10 @@ static const struct { { MFX_ERR_INVALID_VIDEO_PARAM, AVERROR(EINVAL), "invalid video parameters" }, { MFX_ERR_UNDEFINED_BEHAVIOR, AVERROR_BUG, "undefined behavior" }, { MFX_ERR_DEVICE_FAILED, AVERROR(EIO), "device failed" }, +#if QSV_HAVE_AUDIO { MFX_ERR_INCOMPATIBLE_AUDIO_PARAM, AVERROR(EINVAL), "incompatible audio parameters" }, { MFX_ERR_INVALID_AUDIO_PARAM, AVERROR(EINVAL), "invalid audio parameters" }, +#endif { MFX_WRN_IN_EXECUTION, 0, "operation in execution" }, { MFX_WRN_DEVICE_BUSY, 0, "device busy" }, @@ -111,7 +115,9 @@ static const struct { { MFX_WRN_VALUE_NOT_CHANGED, 0, "value is saturated" }, { MFX_WRN_OUT_OF_RANGE, 0, "value out of range" }, { MFX_WRN_FILTER_SKIPPED, 0, "filter skipped" }, +#if QSV_HAVE_AUDIO { MFX_WRN_INCOMPATIBLE_AUDIO_PARAM, 0, "incompatible audio parameters" }, +#endif }; static int qsv_map_error(mfxStatus mfx_err, const char **desc) diff --git a/libavfilter/qsvvpp.h b/libavfilter/qsvvpp.h index 543c58a967..802abd987d 100644 --- a/libavfilter/qsvvpp.h +++ b/libavfilter/qsvvpp.h @@ -40,6 +40,8 @@ ((MFX_VERSION.Major > (MAJOR)) || \ (MFX_VERSION.Major == (MAJOR) && MFX_VERSION.Minor >= (MINOR))) +#define QSV_ONEVPL QSV_VERSION_ATLEAST(2, 0) + typedef struct QSVFrame { AVFrame *frame; mfxFrameSurface1 surface; -- 2.17.1 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [FFmpeg-devel] [PATCH v7 05/10] qsv: build audio related code when MFX_VERSION < 2.0 2022-03-11 8:16 ` [FFmpeg-devel] [PATCH v7 05/10] qsv: build audio related code when " Xiang, Haihao @ 2022-04-05 11:50 ` Anton Khirnov 2022-04-06 3:38 ` Xiang, Haihao 0 siblings, 1 reply; 26+ messages in thread From: Anton Khirnov @ 2022-04-05 11:50 UTC (permalink / raw) To: FFmpeg development discussions and patches; +Cc: Haihao Xiang Quoting Xiang, Haihao (2022-03-11 09:16:25) > From: Haihao Xiang <haihao.xiang@intel.com> > > Audio isn't supported for MFX_VERSION >= 2.0[1][2]. This is in > preparation for oneVPL support > > [1]: https://spec.oneapi.io/versions/latest/elements/oneVPL/source/VPL_intel_media_sdk.html#msdk-full-name-feature-removals > [2]: https://github.com/oneapi-src/oneVPL > --- > libavcodec/qsv.c | 5 +++++ > libavfilter/qsvvpp.c | 6 ++++++ > libavfilter/qsvvpp.h | 2 ++ > 3 files changed, 13 insertions(+) Why not just remove this completely? None of our QSV code does anything with audio. -- Anton Khirnov _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [FFmpeg-devel] [PATCH v7 05/10] qsv: build audio related code when MFX_VERSION < 2.0 2022-04-05 11:50 ` Anton Khirnov @ 2022-04-06 3:38 ` Xiang, Haihao 2022-04-30 16:51 ` Soft Works 0 siblings, 1 reply; 26+ messages in thread From: Xiang, Haihao @ 2022-04-06 3:38 UTC (permalink / raw) To: anton, ffmpeg-devel On Tue, 2022-04-05 at 13:50 +0200, Anton Khirnov wrote: > Quoting Xiang, Haihao (2022-03-11 09:16:25) > > From: Haihao Xiang <haihao.xiang@intel.com> > > > > Audio isn't supported for MFX_VERSION >= 2.0[1][2]. This is in > > preparation for oneVPL support > > > > [1]: > > https://spec.oneapi.io/versions/latest/elements/oneVPL/source/VPL_intel_media_sdk.html#msdk-full-name-feature-removals > > [2]: https://github.com/oneapi-src/oneVPL > > --- > > libavcodec/qsv.c | 5 +++++ > > libavfilter/qsvvpp.c | 6 ++++++ > > libavfilter/qsvvpp.h | 2 ++ > > 3 files changed, 13 insertions(+) > > Why not just remove this completely? > None of our QSV code does anything with audio. It was removed in an older version, however someone objected the removal of this. See https://patchwork.ffmpeg.org/project/ffmpeg/patch/20200819061023.443304-2-haihao.xiang@intel.com/ Thanks Haihao _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [FFmpeg-devel] [PATCH v7 05/10] qsv: build audio related code when MFX_VERSION < 2.0 2022-04-06 3:38 ` Xiang, Haihao @ 2022-04-30 16:51 ` Soft Works 2022-05-01 2:16 ` Xiang, Haihao 0 siblings, 1 reply; 26+ messages in thread From: Soft Works @ 2022-04-30 16:51 UTC (permalink / raw) To: FFmpeg development discussions and patches, anton > -----Original Message----- > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of > Xiang, Haihao > Sent: Wednesday, April 6, 2022 5:38 AM > To: anton@khirnov.net; ffmpeg-devel@ffmpeg.org > Subject: Re: [FFmpeg-devel] [PATCH v7 05/10] qsv: build audio related > code when MFX_VERSION < 2.0 > > On Tue, 2022-04-05 at 13:50 +0200, Anton Khirnov wrote: > > Quoting Xiang, Haihao (2022-03-11 09:16:25) > > > From: Haihao Xiang <haihao.xiang@intel.com> > > > > > > Audio isn't supported for MFX_VERSION >= 2.0[1][2]. This is in > > > preparation for oneVPL support > > > > > > [1]: > > > > https://spec.oneapi.io/versions/latest/elements/oneVPL/source/VPL_inte > l_media_sdk.html#msdk-full-name-feature-removals > > > [2]: https://github.com/oneapi-src/oneVPL > > > --- > > > libavcodec/qsv.c | 5 +++++ > > > libavfilter/qsvvpp.c | 6 ++++++ > > > libavfilter/qsvvpp.h | 2 ++ > > > 3 files changed, 13 insertions(+) > > > > Why not just remove this completely? > > None of our QSV code does anything with audio. > > It was removed in an older version, however someone objected the > removal of > this. See > https://patchwork.ffmpeg.org/project/ffmpeg/patch/20200819061023.44330 > 4-2-haihao.xiang@intel.com/ I think this was a misunderstanding. I see not objection. One was just asking "why" and the other one had missed the point that audio has never been functional. I agree with Anton that the audio stuff should be removed completely - ideally even beforehand in a separate patch. softworkz _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [FFmpeg-devel] [PATCH v7 05/10] qsv: build audio related code when MFX_VERSION < 2.0 2022-04-30 16:51 ` Soft Works @ 2022-05-01 2:16 ` Xiang, Haihao 2022-05-01 3:10 ` Soft Works 0 siblings, 1 reply; 26+ messages in thread From: Xiang, Haihao @ 2022-05-01 2:16 UTC (permalink / raw) To: anton, ffmpeg-devel On Sat, 2022-04-30 at 16:51 +0000, Soft Works wrote: > > -----Original Message----- > > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of > > Xiang, Haihao > > Sent: Wednesday, April 6, 2022 5:38 AM > > To: anton@khirnov.net; ffmpeg-devel@ffmpeg.org > > Subject: Re: [FFmpeg-devel] [PATCH v7 05/10] qsv: build audio related > > code when MFX_VERSION < 2.0 > > > > On Tue, 2022-04-05 at 13:50 +0200, Anton Khirnov wrote: > > > Quoting Xiang, Haihao (2022-03-11 09:16:25) > > > > From: Haihao Xiang <haihao.xiang@intel.com> > > > > > > > > Audio isn't supported for MFX_VERSION >= 2.0[1][2]. This is in > > > > preparation for oneVPL support > > > > > > > > [1]: > > > > > > > > https://spec.oneapi.io/versions/latest/elements/oneVPL/source/VPL_inte > > l_media_sdk.html#msdk-full-name-feature-removals > > > > [2]: https://github.com/oneapi-src/oneVPL > > > > --- > > > > libavcodec/qsv.c | 5 +++++ > > > > libavfilter/qsvvpp.c | 6 ++++++ > > > > libavfilter/qsvvpp.h | 2 ++ > > > > 3 files changed, 13 insertions(+) > > > > > > Why not just remove this completely? > > > None of our QSV code does anything with audio. > > > > It was removed in an older version, however someone objected the > > removal of > > this. See > > https://patchwork.ffmpeg.org/project/ffmpeg/patch/20200819061023.44330 > > 4-2-haihao.xiang@intel.com/ > > I think this was a misunderstanding. I see not objection. One was > just asking "why" and the other one had missed the point that audio > has never been functional. Please find the comment below in the original thread "This seems like a generic translation from the library errors to FF error codes. So even if we'll never touch the audio functionality of it, I'd prefer to have that struct complete already" So my understanding was that the reviewer preferred to keep the audio stuff unchanged for libmfx. > > I agree with Anton that the audio stuff should be removed > completely - ideally even beforehand in a separate patch. Actually I prefer to remove the audio stuff too. I may submit a separate patch if no more comments. Thanks Haihao _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [FFmpeg-devel] [PATCH v7 05/10] qsv: build audio related code when MFX_VERSION < 2.0 2022-05-01 2:16 ` Xiang, Haihao @ 2022-05-01 3:10 ` Soft Works 2022-05-01 4:51 ` Xiang, Haihao 0 siblings, 1 reply; 26+ messages in thread From: Soft Works @ 2022-05-01 3:10 UTC (permalink / raw) To: FFmpeg development discussions and patches, anton > -----Original Message----- > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of > Xiang, Haihao > Sent: Sunday, May 1, 2022 4:16 AM > To: anton@khirnov.net; ffmpeg-devel@ffmpeg.org > Subject: Re: [FFmpeg-devel] [PATCH v7 05/10] qsv: build audio related > code when MFX_VERSION < 2.0 > > On Sat, 2022-04-30 at 16:51 +0000, Soft Works wrote: > > > -----Original Message----- > > > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of > > > Xiang, Haihao > > > Sent: Wednesday, April 6, 2022 5:38 AM > > > To: anton@khirnov.net; ffmpeg-devel@ffmpeg.org > > > Subject: Re: [FFmpeg-devel] [PATCH v7 05/10] qsv: build audio > related > > > code when MFX_VERSION < 2.0 > > > > > > On Tue, 2022-04-05 at 13:50 +0200, Anton Khirnov wrote: > > > > Quoting Xiang, Haihao (2022-03-11 09:16:25) > > > > > From: Haihao Xiang <haihao.xiang@intel.com> > > > > > > > > > > Audio isn't supported for MFX_VERSION >= 2.0[1][2]. This is in > > > > > preparation for oneVPL support > > > > > > > > > > [1]: > > > > > > > > > > > > https://spec.oneapi.io/versions/latest/elements/oneVPL/source/VPL_inte > > > l_media_sdk.html#msdk-full-name-feature-removals > > > > > [2]: https://github.com/oneapi-src/oneVPL > > > > > --- > > > > > libavcodec/qsv.c | 5 +++++ > > > > > libavfilter/qsvvpp.c | 6 ++++++ > > > > > libavfilter/qsvvpp.h | 2 ++ > > > > > 3 files changed, 13 insertions(+) > > > > > > > > Why not just remove this completely? > > > > None of our QSV code does anything with audio. > > > > > > It was removed in an older version, however someone objected the > > > removal of > > > this. See > > > > https://patchwork.ffmpeg.org/project/ffmpeg/patch/20200819061023.44330 > > > 4-2-haihao.xiang@intel.com/ > > > > I think this was a misunderstanding. I see not objection. One was > > just asking "why" and the other one had missed the point that audio > > has never been functional. > > Please find the comment below in the original thread > > "This seems like a generic translation from the library errors to FF > error > codes. So even if we'll never touch the audio functionality of it, I'd > prefer > to have that struct complete already" > > So my understanding was that the reviewer preferred to keep the audio > stuff > unchanged for libmfx. Hm, I hadn't see that here: https://patchwork.ffmpeg.org/project/ffmpeg/patch/20200819061023.443304-2-haihao.xiang@intel.com/ Considering the text again: > "This seems like a generic translation from the library errors to FF > error > codes. So even if we'll never touch the audio functionality of it, I'd > prefer > to have that struct complete already" I understand the idea. Normally those lines wouldn't hurt. But now, that we're facing some kind of "#ifdef hell" anyway, I think it would be much better to minimize this as much as possible, and there's really no point in translating audio error codes. Also, the struct has never been really complete. Instead of retaining unused audio error codes, we should better add those that are missing (like -21, -22 and others) and relevant. @Thilo - can we get you warm with that? As an alternative, we could simply replace the two audio definitions with plain integer values (MFX_ERR_INVALID_AUDIO_PARAM >> -19) This is a small bit only, but before adding the oneVPL stuff, I think we should consolidate the conditional stuff as much as possible. As discussed before, we also need to settle for a minimum libmfx SDK version (compile-time, not runtime!). This will allow to drop quite an amount of conditional code, and this cleanup should be done before getting to oneVPL. Another thing that is a bit unfortunate is that we are duplicating this error mapping struct in qsv.c and qsvvpp.c. I don't mean that it should be linked as an external between avfilter and avcodec, but it should come (be included) from a single file. Kind regards, softworkz _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [FFmpeg-devel] [PATCH v7 05/10] qsv: build audio related code when MFX_VERSION < 2.0 2022-05-01 3:10 ` Soft Works @ 2022-05-01 4:51 ` Xiang, Haihao 2022-05-01 5:13 ` Soft Works 0 siblings, 1 reply; 26+ messages in thread From: Xiang, Haihao @ 2022-05-01 4:51 UTC (permalink / raw) To: anton, ffmpeg-devel On Sun, 2022-05-01 at 03:10 +0000, Soft Works wrote: > > -----Original Message----- > > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of > > Xiang, Haihao > > Sent: Sunday, May 1, 2022 4:16 AM > > To: anton@khirnov.net; ffmpeg-devel@ffmpeg.org > > Subject: Re: [FFmpeg-devel] [PATCH v7 05/10] qsv: build audio related > > code when MFX_VERSION < 2.0 > > > > On Sat, 2022-04-30 at 16:51 +0000, Soft Works wrote: > > > > -----Original Message----- > > > > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of > > > > Xiang, Haihao > > > > Sent: Wednesday, April 6, 2022 5:38 AM > > > > To: anton@khirnov.net; ffmpeg-devel@ffmpeg.org > > > > Subject: Re: [FFmpeg-devel] [PATCH v7 05/10] qsv: build audio > > > > related > > > > code when MFX_VERSION < 2.0 > > > > > > > > On Tue, 2022-04-05 at 13:50 +0200, Anton Khirnov wrote: > > > > > Quoting Xiang, Haihao (2022-03-11 09:16:25) > > > > > > From: Haihao Xiang <haihao.xiang@intel.com> > > > > > > > > > > > > Audio isn't supported for MFX_VERSION >= 2.0[1][2]. This is in > > > > > > preparation for oneVPL support > > > > > > > > > > > > [1]: > > > > > > > > > > > > > > > > > > https://spec.oneapi.io/versions/latest/elements/oneVPL/source/VPL_inte > > > > l_media_sdk.html#msdk-full-name-feature-removals > > > > > > [2]: https://github.com/oneapi-src/oneVPL > > > > > > --- > > > > > > libavcodec/qsv.c | 5 +++++ > > > > > > libavfilter/qsvvpp.c | 6 ++++++ > > > > > > libavfilter/qsvvpp.h | 2 ++ > > > > > > 3 files changed, 13 insertions(+) > > > > > > > > > > Why not just remove this completely? > > > > > None of our QSV code does anything with audio. > > > > > > > > It was removed in an older version, however someone objected the > > > > removal of > > > > this. See > > > > > > > > https://patchwork.ffmpeg.org/project/ffmpeg/patch/20200819061023.44330 > > > > 4-2-haihao.xiang@intel.com/ > > > > > > I think this was a misunderstanding. I see not objection. One was > > > just asking "why" and the other one had missed the point that audio > > > has never been functional. > > > > Please find the comment below in the original thread > > > > "This seems like a generic translation from the library errors to FF > > error > > codes. So even if we'll never touch the audio functionality of it, I'd > > prefer > > to have that struct complete already" > > > > So my understanding was that the reviewer preferred to keep the audio > > stuff > > unchanged for libmfx. > > Hm, I hadn't see that here: > > https://patchwork.ffmpeg.org/project/ffmpeg/patch/20200819061023.443304-2-haihao.xiang@intel.com/ > > Considering the text again: > > > "This seems like a generic translation from the library errors to FF > > error > > codes. So even if we'll never touch the audio functionality of it, I'd > > prefer > > to have that struct complete already" > > I understand the idea. Normally those lines wouldn't hurt. But now, that > we're facing some kind of "#ifdef hell" anyway, I think it would be much > better to minimize this as much as possible, and there's really no point > in translating audio error codes. > Also, the struct has never been really complete. Instead of retaining > unused audio error codes, we should better add those that are missing > (like -21, -22 and others) and relevant. > > @Thilo - can we get you warm with that? > > As an alternative, we could simply replace the two audio definitions with > plain integer values (MFX_ERR_INVALID_AUDIO_PARAM >> -19) > > > This is a small bit only, but before adding the oneVPL stuff, I think > we should consolidate the conditional stuff as much as possible. > > As discussed before, we also need to settle for a minimum libmfx SDK > version (compile-time, not runtime!). This will allow to drop quite > an amount of conditional code, and this cleanup should be done before > getting to oneVPL. Sure, I'll submit a patch for it. > > Another thing that is a bit unfortunate is that we are duplicating this > error mapping struct in qsv.c and qsvvpp.c. > I don't mean that it should be linked as an external between avfilter > and avcodec, but it should come (be included) from a single file. We moved the static error table to a .h in the past however it resulted in link error when building FFmpeg with static libraries. Thanks Haihao _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [FFmpeg-devel] [PATCH v7 05/10] qsv: build audio related code when MFX_VERSION < 2.0 2022-05-01 4:51 ` Xiang, Haihao @ 2022-05-01 5:13 ` Soft Works 2022-05-31 8:58 ` Xiang, Haihao 0 siblings, 1 reply; 26+ messages in thread From: Soft Works @ 2022-05-01 5:13 UTC (permalink / raw) To: FFmpeg development discussions and patches, anton > -----Original Message----- > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of > Xiang, Haihao > Sent: Sunday, May 1, 2022 6:52 AM > To: anton@khirnov.net; ffmpeg-devel@ffmpeg.org > Subject: Re: [FFmpeg-devel] [PATCH v7 05/10] qsv: build audio related > code when MFX_VERSION < 2.0 > > On Sun, 2022-05-01 at 03:10 +0000, Soft Works wrote: > > > -----Original Message----- > > > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of > > > Xiang, Haihao > > > Sent: Sunday, May 1, 2022 4:16 AM > > > To: anton@khirnov.net; ffmpeg-devel@ffmpeg.org > > > Subject: Re: [FFmpeg-devel] [PATCH v7 05/10] qsv: build audio > related > > > code when MFX_VERSION < 2.0 > > > > > > On Sat, 2022-04-30 at 16:51 +0000, Soft Works wrote: > > > > > -----Original Message----- > > > > > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf > Of > > > > > Xiang, Haihao > > > > > Sent: Wednesday, April 6, 2022 5:38 AM > > > > > To: anton@khirnov.net; ffmpeg-devel@ffmpeg.org > > > > > Subject: Re: [FFmpeg-devel] [PATCH v7 05/10] qsv: build audio > > > > > > related > > > > > code when MFX_VERSION < 2.0 > > > > > > > > > > On Tue, 2022-04-05 at 13:50 +0200, Anton Khirnov wrote: > > > > > > Quoting Xiang, Haihao (2022-03-11 09:16:25) > > > > > > > From: Haihao Xiang <haihao.xiang@intel.com> > > > > > > > > > > > > > > Audio isn't supported for MFX_VERSION >= 2.0[1][2]. This > is in > > > > > > > preparation for oneVPL support > > > > > > > > > > > > > > [1]: > > > > > > > > > > > > > > > > > > > > > > > > https://spec.oneapi.io/versions/latest/elements/oneVPL/source/VPL_inte > > > > > l_media_sdk.html#msdk-full-name-feature-removals > > > > > > > [2]: https://github.com/oneapi-src/oneVPL > > > > > > > --- > > > > > > > libavcodec/qsv.c | 5 +++++ > > > > > > > libavfilter/qsvvpp.c | 6 ++++++ > > > > > > > libavfilter/qsvvpp.h | 2 ++ > > > > > > > 3 files changed, 13 insertions(+) > > > > > > > > > > > > Why not just remove this completely? > > > > > > None of our QSV code does anything with audio. > > > > > > > > > > It was removed in an older version, however someone objected > the > > > > > removal of > > > > > this. See > > > > > > > > > > > > https://patchwork.ffmpeg.org/project/ffmpeg/patch/20200819061023.44330 > > > > > 4-2-haihao.xiang@intel.com/ > > > > > > > > I think this was a misunderstanding. I see not objection. One > was > > > > just asking "why" and the other one had missed the point that > audio > > > > has never been functional. > > > > > > Please find the comment below in the original thread > > > > > > "This seems like a generic translation from the library errors to > FF > > > error > > > codes. So even if we'll never touch the audio functionality of it, > I'd > > > prefer > > > to have that struct complete already" > > > > > > So my understanding was that the reviewer preferred to keep the > audio > > > stuff > > > unchanged for libmfx. > > > > Hm, I hadn't see that here: > > > > > https://patchwork.ffmpeg.org/project/ffmpeg/patch/20200819061023.44330 > 4-2-haihao.xiang@intel.com/ > > > > Considering the text again: > > > > > "This seems like a generic translation from the library errors to > FF > > > error > > > codes. So even if we'll never touch the audio functionality of it, > I'd > > > prefer > > > to have that struct complete already" > > > > I understand the idea. Normally those lines wouldn't hurt. But now, > that > > we're facing some kind of "#ifdef hell" anyway, I think it would be > much > > better to minimize this as much as possible, and there's really no > point > > in translating audio error codes. > > Also, the struct has never been really complete. Instead of > retaining > > unused audio error codes, we should better add those that are > missing > > (like -21, -22 and others) and relevant. > > > > @Thilo - can we get you warm with that? > > > > As an alternative, we could simply replace the two audio definitions > with > > plain integer values (MFX_ERR_INVALID_AUDIO_PARAM >> -19) > > > > > > This is a small bit only, but before adding the oneVPL stuff, I > think > > we should consolidate the conditional stuff as much as possible. > > > > As discussed before, we also need to settle for a minimum libmfx SDK > > version (compile-time, not runtime!). This will allow to drop quite > > an amount of conditional code, and this cleanup should be done > before > > getting to oneVPL. > > Sure, I'll submit a patch for it. > > > > > Another thing that is a bit unfortunate is that we are duplicating > this > > error mapping struct in qsv.c and qsvvpp.c. > > I don't mean that it should be linked as an external between > avfilter > > and avcodec, but it should come (be included) from a single file. > > We moved the static error table to a .h in the past however it > resulted in link > error when building FFmpeg with static libraries. That's why I said that it doesn't need to be exported - just a single file that can be included/inlined in both cases where it is used (lavcodec, lavfilter). Best, softworkz _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [FFmpeg-devel] [PATCH v7 05/10] qsv: build audio related code when MFX_VERSION < 2.0 2022-05-01 5:13 ` Soft Works @ 2022-05-31 8:58 ` Xiang, Haihao 0 siblings, 0 replies; 26+ messages in thread From: Xiang, Haihao @ 2022-05-31 8:58 UTC (permalink / raw) To: anton, ffmpeg-devel On Sun, 2022-05-01 at 05:13 +0000, Soft Works wrote: > > -----Original Message----- > > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of > > Xiang, Haihao > > Sent: Sunday, May 1, 2022 6:52 AM > > To: anton@khirnov.net; ffmpeg-devel@ffmpeg.org > > Subject: Re: [FFmpeg-devel] [PATCH v7 05/10] qsv: build audio related > > code when MFX_VERSION < 2.0 > > > > On Sun, 2022-05-01 at 03:10 +0000, Soft Works wrote: > > > > -----Original Message----- > > > > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of > > > > Xiang, Haihao > > > > Sent: Sunday, May 1, 2022 4:16 AM > > > > To: anton@khirnov.net; ffmpeg-devel@ffmpeg.org > > > > Subject: Re: [FFmpeg-devel] [PATCH v7 05/10] qsv: build audio > > > > related > > > > code when MFX_VERSION < 2.0 > > > > > > > > On Sat, 2022-04-30 at 16:51 +0000, Soft Works wrote: > > > > > > -----Original Message----- > > > > > > From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf > > > > Of > > > > > > Xiang, Haihao > > > > > > Sent: Wednesday, April 6, 2022 5:38 AM > > > > > > To: anton@khirnov.net; ffmpeg-devel@ffmpeg.org > > > > > > Subject: Re: [FFmpeg-devel] [PATCH v7 05/10] qsv: build audio > > > > > > > > related > > > > > > code when MFX_VERSION < 2.0 > > > > > > > > > > > > On Tue, 2022-04-05 at 13:50 +0200, Anton Khirnov wrote: > > > > > > > Quoting Xiang, Haihao (2022-03-11 09:16:25) > > > > > > > > From: Haihao Xiang <haihao.xiang@intel.com> > > > > > > > > > > > > > > > > Audio isn't supported for MFX_VERSION >= 2.0[1][2]. This > > > > is in > > > > > > > > preparation for oneVPL support > > > > > > > > > > > > > > > > [1]: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://spec.oneapi.io/versions/latest/elements/oneVPL/source/VPL_inte > > > > > > l_media_sdk.html#msdk-full-name-feature-removals > > > > > > > > [2]: https://github.com/oneapi-src/oneVPL > > > > > > > > --- > > > > > > > > libavcodec/qsv.c | 5 +++++ > > > > > > > > libavfilter/qsvvpp.c | 6 ++++++ > > > > > > > > libavfilter/qsvvpp.h | 2 ++ > > > > > > > > 3 files changed, 13 insertions(+) > > > > > > > > > > > > > > Why not just remove this completely? > > > > > > > None of our QSV code does anything with audio. > > > > > > > > > > > > It was removed in an older version, however someone objected > > > > the > > > > > > removal of > > > > > > this. See > > > > > > > > > > > > > > > > > > https://patchwork.ffmpeg.org/project/ffmpeg/patch/20200819061023.44330 > > > > > > 4-2-haihao.xiang@intel.com/ > > > > > > > > > > I think this was a misunderstanding. I see not objection. One > > > > was > > > > > just asking "why" and the other one had missed the point that > > > > audio > > > > > has never been functional. > > > > > > > > Please find the comment below in the original thread > > > > > > > > "This seems like a generic translation from the library errors to > > > > FF > > > > error > > > > codes. So even if we'll never touch the audio functionality of it, > > > > I'd > > > > prefer > > > > to have that struct complete already" > > > > > > > > So my understanding was that the reviewer preferred to keep the > > > > audio > > > > stuff > > > > unchanged for libmfx. > > > > > > Hm, I hadn't see that here: > > > > > > > > > > https://patchwork.ffmpeg.org/project/ffmpeg/patch/20200819061023.44330 > > 4-2-haihao.xiang@intel.com/ > > > > > > Considering the text again: > > > > > > > "This seems like a generic translation from the library errors to > > > > FF > > > > error > > > > codes. So even if we'll never touch the audio functionality of it, > > > > I'd > > > > prefer > > > > to have that struct complete already" > > > > > > I understand the idea. Normally those lines wouldn't hurt. But now, > > > > that > > > we're facing some kind of "#ifdef hell" anyway, I think it would be > > > > much > > > better to minimize this as much as possible, and there's really no > > > > point > > > in translating audio error codes. > > > Also, the struct has never been really complete. Instead of > > > > retaining > > > unused audio error codes, we should better add those that are > > > > missing > > > (like -21, -22 and others) and relevant. > > > > > > @Thilo - can we get you warm with that? > > > > > > As an alternative, we could simply replace the two audio definitions > > > > with > > > plain integer values (MFX_ERR_INVALID_AUDIO_PARAM >> -19) > > > > > > > > > This is a small bit only, but before adding the oneVPL stuff, I > > > > think > > > we should consolidate the conditional stuff as much as possible. > > > > > > As discussed before, we also need to settle for a minimum libmfx SDK > > > version (compile-time, not runtime!). This will allow to drop quite > > > an amount of conditional code, and this cleanup should be done > > > > before > > > getting to oneVPL. > > > > Sure, I'll submit a patch for it. > > > > > > > > Another thing that is a bit unfortunate is that we are duplicating > > > > this > > > error mapping struct in qsv.c and qsvvpp.c. > > > I don't mean that it should be linked as an external between > > > > avfilter > > > and avcodec, but it should come (be included) from a single file. > > > > We moved the static error table to a .h in the past however it > > resulted in link > > error when building FFmpeg with static libraries. > > That's why I said that it doesn't need to be exported - just a > single file that can be included/inlined in both cases where it > is used (lavcodec, lavfilter). Thanks for your comment, I'll consider to change it later. BTW we don't get response from Thilo, I'll keep this patch unchanged in the new version. I think we may update it in the future if there are more comments. Thanks Haihao _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 26+ messages in thread
* [FFmpeg-devel] [PATCH v7 06/10] qsvenc: support multi-frame encode when MFX_VERSION < 2.0 2022-03-11 8:16 [FFmpeg-devel] [PATCH v7 00/10] make QSV works with the Intel's oneVPL Xiang, Haihao ` (4 preceding siblings ...) 2022-03-11 8:16 ` [FFmpeg-devel] [PATCH v7 05/10] qsv: build audio related code when " Xiang, Haihao @ 2022-03-11 8:16 ` Xiang, Haihao 2022-03-11 8:16 ` [FFmpeg-devel] [PATCH v7 07/10] qsvenc: support MFX_RATECONTROL_LA_EXT " Xiang, Haihao ` (3 subsequent siblings) 9 siblings, 0 replies; 26+ messages in thread From: Xiang, Haihao @ 2022-03-11 8:16 UTC (permalink / raw) To: ffmpeg-devel; +Cc: Haihao Xiang From: Haihao Xiang <haihao.xiang@intel.com> Multi-frame encode isn't supported for MFX_VERSION >= 2.0[1][2]. This is in preparation for oneVPL support [1]: https://spec.oneapi.io/versions/latest/elements/oneVPL/source/VPL_intel_media_sdk.html#msdk-full-name-feature-removals [2]: https://github.com/oneapi-src/oneVPL --- libavcodec/qsvenc.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/libavcodec/qsvenc.h b/libavcodec/qsvenc.h index 559d7cbe3e..43437c6992 100644 --- a/libavcodec/qsvenc.h +++ b/libavcodec/qsvenc.h @@ -67,7 +67,7 @@ #define QSV_HAVE_ICQ QSV_VERSION_ATLEAST(1, 28) #define QSV_HAVE_VCM 0 #define QSV_HAVE_QVBR QSV_VERSION_ATLEAST(1, 28) -#define QSV_HAVE_MF QSV_VERSION_ATLEAST(1, 25) +#define QSV_HAVE_MF QSV_VERSION_ATLEAST(1, 25) && !QSV_ONEVPL #endif #if !QSV_HAVE_LA_DS -- 2.17.1 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 26+ messages in thread
* [FFmpeg-devel] [PATCH v7 07/10] qsvenc: support MFX_RATECONTROL_LA_EXT when MFX_VERSION < 2.0 2022-03-11 8:16 [FFmpeg-devel] [PATCH v7 00/10] make QSV works with the Intel's oneVPL Xiang, Haihao ` (5 preceding siblings ...) 2022-03-11 8:16 ` [FFmpeg-devel] [PATCH v7 06/10] qsvenc: support multi-frame encode " Xiang, Haihao @ 2022-03-11 8:16 ` Xiang, Haihao 2022-03-11 8:16 ` [FFmpeg-devel] [PATCH v7 08/10] qsv: support OPAQUE memory " Xiang, Haihao ` (2 subsequent siblings) 9 siblings, 0 replies; 26+ messages in thread From: Xiang, Haihao @ 2022-03-11 8:16 UTC (permalink / raw) To: ffmpeg-devel; +Cc: Haihao Xiang From: Haihao Xiang <haihao.xiang@intel.com> MFX_RATECONTROL_LA_EXT isn't supported for MFX_VERSION >= 2.0[1][2]. This is in preparation for oneVPL support [1]: https://spec.oneapi.io/versions/latest/elements/oneVPL/source/VPL_intel_media_sdk.html#msdk-full-name-feature-removals [2]: https://github.com/oneapi-src/oneVPL --- libavcodec/qsvenc.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/libavcodec/qsvenc.c b/libavcodec/qsvenc.c index 0170ae14e4..47009c3bf2 100644 --- a/libavcodec/qsvenc.c +++ b/libavcodec/qsvenc.c @@ -151,7 +151,7 @@ static const struct { #if QSV_HAVE_VCM { MFX_RATECONTROL_VCM, "VCM" }, #endif -#if QSV_VERSION_ATLEAST(1, 10) +#if QSV_VERSION_ATLEAST(1, 10) && !QSV_ONEVPL { MFX_RATECONTROL_LA_EXT, "LA_EXT" }, #endif #if QSV_HAVE_LA_HRD -- 2.17.1 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 26+ messages in thread
* [FFmpeg-devel] [PATCH v7 08/10] qsv: support OPAQUE memory when MFX_VERSION < 2.0 2022-03-11 8:16 [FFmpeg-devel] [PATCH v7 00/10] make QSV works with the Intel's oneVPL Xiang, Haihao ` (6 preceding siblings ...) 2022-03-11 8:16 ` [FFmpeg-devel] [PATCH v7 07/10] qsvenc: support MFX_RATECONTROL_LA_EXT " Xiang, Haihao @ 2022-03-11 8:16 ` Xiang, Haihao 2022-03-11 8:16 ` [FFmpeg-devel] [PATCH v7 09/10] qsv: use a new method to create mfx session when using oneVPL Xiang, Haihao 2022-03-11 8:16 ` [FFmpeg-devel] [PATCH v7 10/10] configure: add --enable-libvpl option Xiang, Haihao 9 siblings, 0 replies; 26+ messages in thread From: Xiang, Haihao @ 2022-03-11 8:16 UTC (permalink / raw) To: ffmpeg-devel; +Cc: Haihao Xiang From: Haihao Xiang <haihao.xiang@intel.com> OPAQUE memory isn't supported for MFX_VERSION >= 2.0[1][2]. This is in preparation for oneVPL support [1]: https://spec.oneapi.io/versions/latest/elements/oneVPL/source/VPL_intel_media_sdk.html#msdk-full-name-feature-removals [2]: https://github.com/oneapi-src/oneVPL --- libavcodec/qsv.c | 4 ++ libavcodec/qsv.h | 2 + libavcodec/qsv_internal.h | 1 + libavcodec/qsvdec.c | 9 ++++ libavcodec/qsvenc.c | 21 +++++++++ libavcodec/qsvenc.h | 2 + libavfilter/qsvvpp.c | 26 ++++++++++- libavfilter/qsvvpp.h | 3 ++ libavfilter/vf_deinterlace_qsv.c | 57 +++++++++++++----------- libavfilter/vf_scale_qsv.c | 74 ++++++++++++++++++-------------- libavutil/hwcontext_qsv.c | 56 +++++++++++++++++------- 11 files changed, 181 insertions(+), 74 deletions(-) diff --git a/libavcodec/qsv.c b/libavcodec/qsv.c index 063d6dfb0a..c708964c80 100644 --- a/libavcodec/qsv.c +++ b/libavcodec/qsv.c @@ -89,10 +89,14 @@ static const struct { } qsv_iopatterns[] = { {MFX_IOPATTERN_IN_VIDEO_MEMORY, "input is video memory surface" }, {MFX_IOPATTERN_IN_SYSTEM_MEMORY, "input is system memory surface" }, +#if QSV_HAVE_OPAQUE {MFX_IOPATTERN_IN_OPAQUE_MEMORY, "input is opaque memory surface" }, +#endif {MFX_IOPATTERN_OUT_VIDEO_MEMORY, "output is video memory surface" }, {MFX_IOPATTERN_OUT_SYSTEM_MEMORY, "output is system memory surface" }, +#if QSV_HAVE_OPAQUE {MFX_IOPATTERN_OUT_OPAQUE_MEMORY, "output is opaque memory surface" }, +#endif }; int ff_qsv_print_iopattern(void *log_ctx, int mfx_iopattern, diff --git a/libavcodec/qsv.h b/libavcodec/qsv.h index 04ae0d6f34..c156b08d07 100644 --- a/libavcodec/qsv.h +++ b/libavcodec/qsv.h @@ -61,6 +61,8 @@ typedef struct AVQSVContext { * required by the encoder and the user-provided value nb_opaque_surfaces. * The array of the opaque surfaces will be exported to the caller through * the opaque_surfaces field. + * + * The caller must set this field to zero for oneVPL (MFX_VERSION >= 2.0) */ int opaque_alloc; diff --git a/libavcodec/qsv_internal.h b/libavcodec/qsv_internal.h index 6d6a6eb239..c788293286 100644 --- a/libavcodec/qsv_internal.h +++ b/libavcodec/qsv_internal.h @@ -63,6 +63,7 @@ (MFX_VERSION.Major == (MAJOR) && MFX_VERSION.Minor >= (MINOR))) #define QSV_ONEVPL QSV_VERSION_ATLEAST(2, 0) +#define QSV_HAVE_OPAQUE !QSV_ONEVPL typedef struct QSVMid { AVBufferRef *hw_frames_ref; diff --git a/libavcodec/qsvdec.c b/libavcodec/qsvdec.c index 1a492da74b..391e5d1c72 100644 --- a/libavcodec/qsvdec.c +++ b/libavcodec/qsvdec.c @@ -178,7 +178,11 @@ static int qsv_init_session(AVCodecContext *avctx, QSVContext *q, mfxSession ses ret = ff_qsv_init_session_frames(avctx, &q->internal_qs.session, &q->frames_ctx, q->load_plugins, +#if QSV_HAVE_OPAQUE q->iopattern == MFX_IOPATTERN_OUT_OPAQUE_MEMORY, +#else + 0, +#endif q->gpu_copy); if (ret < 0) { av_buffer_unref(&q->frames_ctx.hw_frames_ctx); @@ -291,10 +295,15 @@ static int qsv_decode_preinit(AVCodecContext *avctx, QSVContext *q, enum AVPixel AVQSVFramesContext *frames_hwctx = frames_ctx->hwctx; if (!iopattern) { +#if QSV_HAVE_OPAQUE if (frames_hwctx->frame_type & MFX_MEMTYPE_OPAQUE_FRAME) iopattern = MFX_IOPATTERN_OUT_OPAQUE_MEMORY; else if (frames_hwctx->frame_type & MFX_MEMTYPE_VIDEO_MEMORY_DECODER_TARGET) iopattern = MFX_IOPATTERN_OUT_VIDEO_MEMORY; +#else + if (frames_hwctx->frame_type & MFX_MEMTYPE_VIDEO_MEMORY_DECODER_TARGET) + iopattern = MFX_IOPATTERN_OUT_VIDEO_MEMORY; +#endif } } diff --git a/libavcodec/qsvenc.c b/libavcodec/qsvenc.c index 47009c3bf2..adfaf5a280 100644 --- a/libavcodec/qsvenc.c +++ b/libavcodec/qsvenc.c @@ -1256,6 +1256,7 @@ static int qsv_retrieve_enc_params(AVCodecContext *avctx, QSVEncContext *q) return 0; } +#if QSV_HAVE_OPAQUE static int qsv_init_opaque_alloc(AVCodecContext *avctx, QSVEncContext *q) { AVQSVContext *qsv = avctx->hwaccel_context; @@ -1292,6 +1293,7 @@ static int qsv_init_opaque_alloc(AVCodecContext *avctx, QSVEncContext *q) return 0; } +#endif static int qsvenc_init_session(AVCodecContext *avctx, QSVEncContext *q) { @@ -1307,7 +1309,11 @@ static int qsvenc_init_session(AVCodecContext *avctx, QSVEncContext *q) ret = ff_qsv_init_session_frames(avctx, &q->internal_qs.session, &q->frames_ctx, q->load_plugins, +#if QSV_HAVE_OPAQUE q->param.IOPattern == MFX_IOPATTERN_IN_OPAQUE_MEMORY, +#else + 0, +#endif MFX_GPUCOPY_OFF); if (ret < 0) { av_buffer_unref(&q->frames_ctx.hw_frames_ctx); @@ -1359,11 +1365,17 @@ int ff_qsv_enc_init(AVCodecContext *avctx, QSVEncContext *q) AVQSVFramesContext *frames_hwctx = frames_ctx->hwctx; if (!iopattern) { +#if QSV_HAVE_OPAQUE if (frames_hwctx->frame_type & MFX_MEMTYPE_OPAQUE_FRAME) iopattern = MFX_IOPATTERN_IN_OPAQUE_MEMORY; else if (frames_hwctx->frame_type & (MFX_MEMTYPE_VIDEO_MEMORY_DECODER_TARGET | MFX_MEMTYPE_VIDEO_MEMORY_PROCESSOR_TARGET)) iopattern = MFX_IOPATTERN_IN_VIDEO_MEMORY; +#else + if (frames_hwctx->frame_type & + (MFX_MEMTYPE_VIDEO_MEMORY_DECODER_TARGET | MFX_MEMTYPE_VIDEO_MEMORY_PROCESSOR_TARGET)) + iopattern = MFX_IOPATTERN_IN_VIDEO_MEMORY; +#endif } } @@ -1437,9 +1449,16 @@ int ff_qsv_enc_init(AVCodecContext *avctx, QSVEncContext *q) "Error querying (IOSurf) the encoding parameters"); if (opaque_alloc) { +#if QSV_HAVE_OPAQUE ret = qsv_init_opaque_alloc(avctx, q); if (ret < 0) return ret; +#else + av_log(avctx, AV_LOG_ERROR, "User is requesting to allocate OPAQUE surface, " + "however libmfx %d.%d doesn't support OPAQUE memory.\n", + q->ver.Major, q->ver.Minor); + return AVERROR_UNKNOWN; +#endif } ret = MFXVideoENCODE_Init(q->session, &q->param); @@ -1868,8 +1887,10 @@ int ff_qsv_enc_close(AVCodecContext *avctx, QSVEncContext *q) av_fifo_freep2(&q->async_fifo); } +#if QSV_HAVE_OPAQUE av_freep(&q->opaque_surfaces); av_buffer_unref(&q->opaque_alloc_buf); +#endif av_freep(&q->extparam); diff --git a/libavcodec/qsvenc.h b/libavcodec/qsvenc.h index 43437c6992..8be00d43f7 100644 --- a/libavcodec/qsvenc.h +++ b/libavcodec/qsvenc.h @@ -140,9 +140,11 @@ typedef struct QSVEncContext { mfxExtVP9Param extvp9param; #endif +#if QSV_HAVE_OPAQUE mfxExtOpaqueSurfaceAlloc opaque_alloc; mfxFrameSurface1 **opaque_surfaces; AVBufferRef *opaque_alloc_buf; +#endif mfxExtVideoSignalInfo extvsi; diff --git a/libavfilter/qsvvpp.c b/libavfilter/qsvvpp.c index 3647891d13..3f984fd5f9 100644 --- a/libavfilter/qsvvpp.c +++ b/libavfilter/qsvvpp.c @@ -34,7 +34,9 @@ #define IS_VIDEO_MEMORY(mode) (mode & (MFX_MEMTYPE_VIDEO_MEMORY_DECODER_TARGET | \ MFX_MEMTYPE_VIDEO_MEMORY_PROCESSOR_TARGET)) +#if QSV_HAVE_OPAQUE #define IS_OPAQUE_MEMORY(mode) (mode & MFX_MEMTYPE_OPAQUE_FRAME) +#endif #define IS_SYSTEM_MEMORY(mode) (mode & MFX_MEMTYPE_SYSTEM_MEMORY) #define MFX_IMPL_VIA_MASK(impl) (0x0f00 & (impl)) @@ -53,10 +55,14 @@ static const struct { } qsv_iopatterns[] = { {MFX_IOPATTERN_IN_VIDEO_MEMORY, "input is video memory surface" }, {MFX_IOPATTERN_IN_SYSTEM_MEMORY, "input is system memory surface" }, +#if QSV_HAVE_OPAQUE {MFX_IOPATTERN_IN_OPAQUE_MEMORY, "input is opaque memory surface" }, +#endif {MFX_IOPATTERN_OUT_VIDEO_MEMORY, "output is video memory surface" }, {MFX_IOPATTERN_OUT_SYSTEM_MEMORY, "output is system memory surface" }, +#if QSV_HAVE_OPAQUE {MFX_IOPATTERN_OUT_OPAQUE_MEMORY, "output is opaque memory surface" }, +#endif }; int ff_qsvvpp_print_iopattern(void *log_ctx, int mfx_iopattern, @@ -536,9 +542,13 @@ static int init_vpp_session(AVFilterContext *avctx, QSVVPPContext *s) if (!out_frames_ref) return AVERROR(ENOMEM); +#if QSV_HAVE_OPAQUE s->out_mem_mode = IS_OPAQUE_MEMORY(s->in_mem_mode) ? MFX_MEMTYPE_OPAQUE_FRAME : MFX_MEMTYPE_VIDEO_MEMORY_DECODER_TARGET | MFX_MEMTYPE_FROM_VPPOUT; +#else + s->out_mem_mode = MFX_MEMTYPE_VIDEO_MEMORY_DECODER_TARGET | MFX_MEMTYPE_FROM_VPPOUT; +#endif out_frames_ctx = (AVHWFramesContext *)out_frames_ref->data; out_frames_hwctx = out_frames_ctx->hwctx; @@ -624,6 +634,7 @@ static int init_vpp_session(AVFilterContext *avctx, QSVVPPContext *s) return AVERROR_UNKNOWN; } +#if QSV_HAVE_OPAQUE if (IS_OPAQUE_MEMORY(s->in_mem_mode) || IS_OPAQUE_MEMORY(s->out_mem_mode)) { s->opaque_alloc.In.Surfaces = s->surface_ptrs_in; s->opaque_alloc.In.NumSurface = s->nb_surface_ptrs_in; @@ -635,7 +646,9 @@ static int init_vpp_session(AVFilterContext *avctx, QSVVPPContext *s) s->opaque_alloc.Header.BufferId = MFX_EXTBUFF_OPAQUE_SURFACE_ALLOCATION; s->opaque_alloc.Header.BufferSz = sizeof(s->opaque_alloc); - } else if (IS_VIDEO_MEMORY(s->in_mem_mode) || IS_VIDEO_MEMORY(s->out_mem_mode)) { + } else +#endif + if (IS_VIDEO_MEMORY(s->in_mem_mode) || IS_VIDEO_MEMORY(s->out_mem_mode)) { mfxFrameAllocator frame_allocator = { .pthis = s, .Alloc = frame_alloc, @@ -707,6 +720,7 @@ int ff_qsvvpp_create(AVFilterContext *avctx, QSVVPPContext **vpp, QSVVPPParam *p goto failed; } +#if QSV_HAVE_OPAQUE if (IS_OPAQUE_MEMORY(s->in_mem_mode) || IS_OPAQUE_MEMORY(s->out_mem_mode)) { s->nb_ext_buffers = param->num_ext_buf + 1; s->ext_buffers = av_calloc(s->nb_ext_buffers, sizeof(*s->ext_buffers)); @@ -724,6 +738,10 @@ int ff_qsvvpp_create(AVFilterContext *avctx, QSVVPPContext **vpp, QSVVPPParam *p s->vpp_param.NumExtParam = param->num_ext_buf; s->vpp_param.ExtParam = param->ext_buf; } +#else + s->vpp_param.NumExtParam = param->num_ext_buf; + s->vpp_param.ExtParam = param->ext_buf; +#endif s->got_frame = 0; @@ -741,15 +759,19 @@ int ff_qsvvpp_create(AVFilterContext *avctx, QSVVPPContext **vpp, QSVVPPParam *p s->vpp_param.IOPattern |= MFX_IOPATTERN_IN_SYSTEM_MEMORY; else if (IS_VIDEO_MEMORY(s->in_mem_mode)) s->vpp_param.IOPattern |= MFX_IOPATTERN_IN_VIDEO_MEMORY; +#if QSV_HAVE_OPAQUE else if (IS_OPAQUE_MEMORY(s->in_mem_mode)) s->vpp_param.IOPattern |= MFX_IOPATTERN_IN_OPAQUE_MEMORY; +#endif if (IS_SYSTEM_MEMORY(s->out_mem_mode)) s->vpp_param.IOPattern |= MFX_IOPATTERN_OUT_SYSTEM_MEMORY; else if (IS_VIDEO_MEMORY(s->out_mem_mode)) s->vpp_param.IOPattern |= MFX_IOPATTERN_OUT_VIDEO_MEMORY; +#if QSV_HAVE_OPAQUE else if (IS_OPAQUE_MEMORY(s->out_mem_mode)) s->vpp_param.IOPattern |= MFX_IOPATTERN_OUT_OPAQUE_MEMORY; +#endif /* Print input memory mode */ ff_qsvvpp_print_iopattern(avctx, s->vpp_param.IOPattern & 0x0F, "VPP"); @@ -788,7 +810,9 @@ int ff_qsvvpp_free(QSVVPPContext **vpp) clear_frame_list(&s->out_frame_list); av_freep(&s->surface_ptrs_in); av_freep(&s->surface_ptrs_out); +#if QSV_HAVE_OPAQUE av_freep(&s->ext_buffers); +#endif av_freep(&s->frame_infos); av_fifo_freep2(&s->async_fifo); av_freep(vpp); diff --git a/libavfilter/qsvvpp.h b/libavfilter/qsvvpp.h index 802abd987d..3e7d56021b 100644 --- a/libavfilter/qsvvpp.h +++ b/libavfilter/qsvvpp.h @@ -41,6 +41,7 @@ (MFX_VERSION.Major == (MAJOR) && MFX_VERSION.Minor >= (MINOR))) #define QSV_ONEVPL QSV_VERSION_ATLEAST(2, 0) +#define QSV_HAVE_OPAQUE !QSV_ONEVPL typedef struct QSVFrame { AVFrame *frame; @@ -66,10 +67,12 @@ typedef struct QSVVPPContext { mfxFrameSurface1 **surface_ptrs_in; mfxFrameSurface1 **surface_ptrs_out; +#if QSV_HAVE_OPAQUE /** MFXVPP extern parameters */ mfxExtOpaqueSurfaceAlloc opaque_alloc; mfxExtBuffer **ext_buffers; int nb_ext_buffers; +#endif int got_frame; int async_depth; diff --git a/libavfilter/vf_deinterlace_qsv.c b/libavfilter/vf_deinterlace_qsv.c index b8ff3e8339..50f9156d14 100644 --- a/libavfilter/vf_deinterlace_qsv.c +++ b/libavfilter/vf_deinterlace_qsv.c @@ -62,7 +62,9 @@ typedef struct QSVDeintContext { mfxFrameSurface1 **surface_ptrs; int nb_surface_ptrs; +#if QSV_HAVE_OPAQUE mfxExtOpaqueSurfaceAlloc opaque_alloc; +#endif mfxExtVPPDeinterlacing deint_conf; mfxExtBuffer *ext_buffers[2]; int num_ext_buffers; @@ -154,9 +156,7 @@ static int init_out_session(AVFilterContext *ctx) AVHWFramesContext *hw_frames_ctx = (AVHWFramesContext*)s->hw_frames_ctx->data; AVQSVFramesContext *hw_frames_hwctx = hw_frames_ctx->hwctx; AVQSVDeviceContext *device_hwctx = hw_frames_ctx->device_ctx->hwctx; - - int opaque = !!(hw_frames_hwctx->frame_type & MFX_MEMTYPE_OPAQUE_FRAME); - + int opaque = 0; mfxHDL handle = NULL; mfxHandleType handle_type; mfxVersion ver; @@ -165,6 +165,9 @@ static int init_out_session(AVFilterContext *ctx) mfxStatus err; int i; +#if QSV_HAVE_OPAQUE + opaque = !!(hw_frames_hwctx->frame_type & MFX_MEMTYPE_OPAQUE_FRAME); +#endif /* extract the properties of the "master" session given to us */ err = MFXQueryIMPL(device_hwctx->session, &impl); if (err == MFX_ERR_NONE) @@ -223,28 +226,7 @@ static int init_out_session(AVFilterContext *ctx) s->ext_buffers[s->num_ext_buffers++] = (mfxExtBuffer *)&s->deint_conf; - if (opaque) { - s->surface_ptrs = av_calloc(hw_frames_hwctx->nb_surfaces, - sizeof(*s->surface_ptrs)); - if (!s->surface_ptrs) - return AVERROR(ENOMEM); - for (i = 0; i < hw_frames_hwctx->nb_surfaces; i++) - s->surface_ptrs[i] = hw_frames_hwctx->surfaces + i; - s->nb_surface_ptrs = hw_frames_hwctx->nb_surfaces; - - s->opaque_alloc.In.Surfaces = s->surface_ptrs; - s->opaque_alloc.In.NumSurface = s->nb_surface_ptrs; - s->opaque_alloc.In.Type = hw_frames_hwctx->frame_type; - - s->opaque_alloc.Out = s->opaque_alloc.In; - - s->opaque_alloc.Header.BufferId = MFX_EXTBUFF_OPAQUE_SURFACE_ALLOCATION; - s->opaque_alloc.Header.BufferSz = sizeof(s->opaque_alloc); - - s->ext_buffers[s->num_ext_buffers++] = (mfxExtBuffer *)&s->opaque_alloc; - - par.IOPattern = MFX_IOPATTERN_IN_OPAQUE_MEMORY | MFX_IOPATTERN_OUT_OPAQUE_MEMORY; - } else { + if (!opaque) { mfxFrameAllocator frame_allocator = { .pthis = ctx, .Alloc = frame_alloc, @@ -268,6 +250,31 @@ static int init_out_session(AVFilterContext *ctx) par.IOPattern = MFX_IOPATTERN_IN_VIDEO_MEMORY | MFX_IOPATTERN_OUT_VIDEO_MEMORY; } +#if QSV_HAVE_OPAQUE + else { + s->surface_ptrs = av_calloc(hw_frames_hwctx->nb_surfaces, + sizeof(*s->surface_ptrs)); + + if (!s->surface_ptrs) + return AVERROR(ENOMEM); + for (i = 0; i < hw_frames_hwctx->nb_surfaces; i++) + s->surface_ptrs[i] = hw_frames_hwctx->surfaces + i; + s->nb_surface_ptrs = hw_frames_hwctx->nb_surfaces; + + s->opaque_alloc.In.Surfaces = s->surface_ptrs; + s->opaque_alloc.In.NumSurface = s->nb_surface_ptrs; + s->opaque_alloc.In.Type = hw_frames_hwctx->frame_type; + + s->opaque_alloc.Out = s->opaque_alloc.In; + + s->opaque_alloc.Header.BufferId = MFX_EXTBUFF_OPAQUE_SURFACE_ALLOCATION; + s->opaque_alloc.Header.BufferSz = sizeof(s->opaque_alloc); + + s->ext_buffers[s->num_ext_buffers++] = (mfxExtBuffer *)&s->opaque_alloc; + + par.IOPattern = MFX_IOPATTERN_IN_OPAQUE_MEMORY | MFX_IOPATTERN_OUT_OPAQUE_MEMORY; + } +#endif par.ExtParam = s->ext_buffers; par.NumExtParam = s->num_ext_buffers; diff --git a/libavfilter/vf_scale_qsv.c b/libavfilter/vf_scale_qsv.c index 2ba7d086b7..30434033d1 100644 --- a/libavfilter/vf_scale_qsv.c +++ b/libavfilter/vf_scale_qsv.c @@ -90,7 +90,9 @@ typedef struct QSVScaleContext { mfxFrameSurface1 **surface_ptrs_out; int nb_surface_ptrs_out; +#if QSV_HAVE_OPAQUE mfxExtOpaqueSurfaceAlloc opaque_alloc; +#endif #if QSV_HAVE_SCALING_CONFIG mfxExtVPPScaling scale_conf; @@ -271,7 +273,7 @@ static int init_out_session(AVFilterContext *ctx) AVQSVFramesContext *out_frames_hwctx = out_frames_ctx->hwctx; AVQSVDeviceContext *device_hwctx = in_frames_ctx->device_ctx->hwctx; - int opaque = !!(in_frames_hwctx->frame_type & MFX_MEMTYPE_OPAQUE_FRAME); + int opaque = 0; mfxHDL handle = NULL; mfxHandleType handle_type; @@ -281,6 +283,9 @@ static int init_out_session(AVFilterContext *ctx) mfxStatus err; int i; +#if QSV_HAVE_OPAQUE + opaque = !!(in_frames_hwctx->frame_type & MFX_MEMTYPE_OPAQUE_FRAME); +#endif s->num_ext_buf = 0; /* extract the properties of the "master" session given to us */ @@ -333,38 +338,7 @@ static int init_out_session(AVFilterContext *ctx) memset(&par, 0, sizeof(par)); - if (opaque) { - s->surface_ptrs_in = av_calloc(in_frames_hwctx->nb_surfaces, - sizeof(*s->surface_ptrs_in)); - if (!s->surface_ptrs_in) - return AVERROR(ENOMEM); - for (i = 0; i < in_frames_hwctx->nb_surfaces; i++) - s->surface_ptrs_in[i] = in_frames_hwctx->surfaces + i; - s->nb_surface_ptrs_in = in_frames_hwctx->nb_surfaces; - - s->surface_ptrs_out = av_calloc(out_frames_hwctx->nb_surfaces, - sizeof(*s->surface_ptrs_out)); - if (!s->surface_ptrs_out) - return AVERROR(ENOMEM); - for (i = 0; i < out_frames_hwctx->nb_surfaces; i++) - s->surface_ptrs_out[i] = out_frames_hwctx->surfaces + i; - s->nb_surface_ptrs_out = out_frames_hwctx->nb_surfaces; - - s->opaque_alloc.In.Surfaces = s->surface_ptrs_in; - s->opaque_alloc.In.NumSurface = s->nb_surface_ptrs_in; - s->opaque_alloc.In.Type = in_frames_hwctx->frame_type; - - s->opaque_alloc.Out.Surfaces = s->surface_ptrs_out; - s->opaque_alloc.Out.NumSurface = s->nb_surface_ptrs_out; - s->opaque_alloc.Out.Type = out_frames_hwctx->frame_type; - - s->opaque_alloc.Header.BufferId = MFX_EXTBUFF_OPAQUE_SURFACE_ALLOCATION; - s->opaque_alloc.Header.BufferSz = sizeof(s->opaque_alloc); - - s->ext_buffers[s->num_ext_buf++] = (mfxExtBuffer*)&s->opaque_alloc; - - par.IOPattern = MFX_IOPATTERN_IN_OPAQUE_MEMORY | MFX_IOPATTERN_OUT_OPAQUE_MEMORY; - } else { + if (!opaque) { mfxFrameAllocator frame_allocator = { .pthis = ctx, .Alloc = frame_alloc, @@ -396,6 +370,40 @@ static int init_out_session(AVFilterContext *ctx) par.IOPattern = MFX_IOPATTERN_IN_VIDEO_MEMORY | MFX_IOPATTERN_OUT_VIDEO_MEMORY; } +#if QSV_HAVE_OPAQUE + else { + s->surface_ptrs_in = av_calloc(in_frames_hwctx->nb_surfaces, + sizeof(*s->surface_ptrs_in)); + if (!s->surface_ptrs_in) + return AVERROR(ENOMEM); + for (i = 0; i < in_frames_hwctx->nb_surfaces; i++) + s->surface_ptrs_in[i] = in_frames_hwctx->surfaces + i; + s->nb_surface_ptrs_in = in_frames_hwctx->nb_surfaces; + + s->surface_ptrs_out = av_calloc(out_frames_hwctx->nb_surfaces, + sizeof(*s->surface_ptrs_out)); + if (!s->surface_ptrs_out) + return AVERROR(ENOMEM); + for (i = 0; i < out_frames_hwctx->nb_surfaces; i++) + s->surface_ptrs_out[i] = out_frames_hwctx->surfaces + i; + s->nb_surface_ptrs_out = out_frames_hwctx->nb_surfaces; + + s->opaque_alloc.In.Surfaces = s->surface_ptrs_in; + s->opaque_alloc.In.NumSurface = s->nb_surface_ptrs_in; + s->opaque_alloc.In.Type = in_frames_hwctx->frame_type; + + s->opaque_alloc.Out.Surfaces = s->surface_ptrs_out; + s->opaque_alloc.Out.NumSurface = s->nb_surface_ptrs_out; + s->opaque_alloc.Out.Type = out_frames_hwctx->frame_type; + + s->opaque_alloc.Header.BufferId = MFX_EXTBUFF_OPAQUE_SURFACE_ALLOCATION; + s->opaque_alloc.Header.BufferSz = sizeof(s->opaque_alloc); + + s->ext_buffers[s->num_ext_buf++] = (mfxExtBuffer*)&s->opaque_alloc; + + par.IOPattern = MFX_IOPATTERN_IN_OPAQUE_MEMORY | MFX_IOPATTERN_OUT_OPAQUE_MEMORY; + } +#endif #if QSV_HAVE_SCALING_CONFIG memset(&s->scale_conf, 0, sizeof(mfxExtVPPScaling)); diff --git a/libavutil/hwcontext_qsv.c b/libavutil/hwcontext_qsv.c index dbe7bafb8e..f5bdb84f52 100644 --- a/libavutil/hwcontext_qsv.c +++ b/libavutil/hwcontext_qsv.c @@ -55,6 +55,8 @@ MFX_VERSION_MAJOR == (MAJOR) && MFX_VERSION_MINOR >= (MINOR)) #define MFX_IMPL_VIA_MASK(impl) (0x0f00 & (impl)) +#define QSV_ONEVPL QSV_VERSION_ATLEAST(2, 0) +#define QSV_HAVE_OPAQUE !QSV_ONEVPL typedef struct QSVDevicePriv { AVBufferRef *child_device_ctx; @@ -86,11 +88,13 @@ typedef struct QSVFramesContext { // used in the frame allocator for non-opaque surfaces mfxMemId *mem_ids; +#if QSV_HAVE_OPAQUE // used in the opaque alloc request for opaque surfaces mfxFrameSurface1 **surface_ptrs; mfxExtOpaqueSurfaceAlloc opaque_alloc; mfxExtBuffer *ext_buffers[1]; +#endif AVFrame realigned_tmp_frame; } QSVFramesContext; @@ -300,7 +304,9 @@ static void qsv_frames_uninit(AVHWFramesContext *ctx) #endif av_freep(&s->mem_ids); +#if QSV_HAVE_OPAQUE av_freep(&s->surface_ptrs); +#endif av_freep(&s->surfaces_internal); av_freep(&s->handle_pairs_internal); av_frame_unref(&s->realigned_tmp_frame); @@ -535,11 +541,17 @@ static int qsv_init_pool(AVHWFramesContext *ctx, uint32_t fourcc) return ret; } +#if QSV_HAVE_OPAQUE if (!(frames_hwctx->frame_type & MFX_MEMTYPE_OPAQUE_FRAME)) { ret = qsv_init_child_ctx(ctx); if (ret < 0) return ret; } +#else + ret = qsv_init_child_ctx(ctx); + if (ret < 0) + return ret; +#endif ctx->internal->pool_internal = av_buffer_pool_init2(sizeof(mfxFrameSurface1), ctx, qsv_pool_alloc, NULL); @@ -610,10 +622,9 @@ static mfxStatus frame_get_hdl(mfxHDL pthis, mfxMemId mid, mfxHDL *hdl) static int qsv_init_internal_session(AVHWFramesContext *ctx, mfxSession *session, int upload) { - QSVFramesContext *s = ctx->internal->priv; AVQSVFramesContext *frames_hwctx = ctx->hwctx; QSVDeviceContext *device_priv = ctx->device_ctx->internal->priv; - int opaque = !!(frames_hwctx->frame_type & MFX_MEMTYPE_OPAQUE_FRAME); + int opaque = 0; mfxFrameAllocator frame_allocator = { .pthis = ctx, @@ -627,6 +638,11 @@ static int qsv_init_internal_session(AVHWFramesContext *ctx, mfxVideoParam par; mfxStatus err; +#if QSV_HAVE_OPAQUE + QSVFramesContext *s = ctx->internal->priv; + opaque = !!(frames_hwctx->frame_type & MFX_MEMTYPE_OPAQUE_FRAME); +#endif + err = MFXInit(device_priv->impl, &device_priv->ver, session); if (err != MFX_ERR_NONE) { av_log(ctx, AV_LOG_ERROR, "Error initializing an internal session\n"); @@ -648,15 +664,18 @@ static int qsv_init_internal_session(AVHWFramesContext *ctx, memset(&par, 0, sizeof(par)); - if (opaque) { + if (!opaque) { + par.IOPattern = upload ? MFX_IOPATTERN_OUT_VIDEO_MEMORY : + MFX_IOPATTERN_IN_VIDEO_MEMORY; + } +#if QSV_HAVE_OPAQUE + else { par.ExtParam = s->ext_buffers; par.NumExtParam = FF_ARRAY_ELEMS(s->ext_buffers); par.IOPattern = upload ? MFX_IOPATTERN_OUT_OPAQUE_MEMORY : MFX_IOPATTERN_IN_OPAQUE_MEMORY; - } else { - par.IOPattern = upload ? MFX_IOPATTERN_OUT_VIDEO_MEMORY : - MFX_IOPATTERN_IN_VIDEO_MEMORY; } +#endif par.IOPattern |= upload ? MFX_IOPATTERN_IN_SYSTEM_MEMORY : MFX_IOPATTERN_OUT_SYSTEM_MEMORY; @@ -688,11 +707,15 @@ static int qsv_frames_init(AVHWFramesContext *ctx) QSVFramesContext *s = ctx->internal->priv; AVQSVFramesContext *frames_hwctx = ctx->hwctx; - int opaque = !!(frames_hwctx->frame_type & MFX_MEMTYPE_OPAQUE_FRAME); + int opaque = 0; uint32_t fourcc; int i, ret; +#if QSV_HAVE_OPAQUE + opaque = !!(frames_hwctx->frame_type & MFX_MEMTYPE_OPAQUE_FRAME); +#endif + fourcc = qsv_fourcc_from_pix_fmt(ctx->sw_format); if (!fourcc) { av_log(ctx, AV_LOG_ERROR, "Unsupported pixel format\n"); @@ -707,7 +730,16 @@ static int qsv_frames_init(AVHWFramesContext *ctx) } } - if (opaque) { + if (!opaque) { + s->mem_ids = av_calloc(frames_hwctx->nb_surfaces, sizeof(*s->mem_ids)); + if (!s->mem_ids) + return AVERROR(ENOMEM); + + for (i = 0; i < frames_hwctx->nb_surfaces; i++) + s->mem_ids[i] = frames_hwctx->surfaces[i].Data.MemId; + } +#if QSV_HAVE_OPAQUE + else { s->surface_ptrs = av_calloc(frames_hwctx->nb_surfaces, sizeof(*s->surface_ptrs)); if (!s->surface_ptrs) @@ -726,14 +758,8 @@ static int qsv_frames_init(AVHWFramesContext *ctx) s->opaque_alloc.Header.BufferSz = sizeof(s->opaque_alloc); s->ext_buffers[0] = (mfxExtBuffer*)&s->opaque_alloc; - } else { - s->mem_ids = av_calloc(frames_hwctx->nb_surfaces, sizeof(*s->mem_ids)); - if (!s->mem_ids) - return AVERROR(ENOMEM); - - for (i = 0; i < frames_hwctx->nb_surfaces; i++) - s->mem_ids[i] = frames_hwctx->surfaces[i].Data.MemId; } +#endif s->session_download = NULL; s->session_upload = NULL; -- 2.17.1 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 26+ messages in thread
* [FFmpeg-devel] [PATCH v7 09/10] qsv: use a new method to create mfx session when using oneVPL 2022-03-11 8:16 [FFmpeg-devel] [PATCH v7 00/10] make QSV works with the Intel's oneVPL Xiang, Haihao ` (7 preceding siblings ...) 2022-03-11 8:16 ` [FFmpeg-devel] [PATCH v7 08/10] qsv: support OPAQUE memory " Xiang, Haihao @ 2022-03-11 8:16 ` Xiang, Haihao 2022-03-11 8:35 ` Hendrik Leppkes 2022-03-11 8:16 ` [FFmpeg-devel] [PATCH v7 10/10] configure: add --enable-libvpl option Xiang, Haihao 9 siblings, 1 reply; 26+ messages in thread From: Xiang, Haihao @ 2022-03-11 8:16 UTC (permalink / raw) To: ffmpeg-devel; +Cc: Haihao Xiang, galinart From: Haihao Xiang <haihao.xiang@intel.com> In oneVPL, MFXLoad() and MFXCreateSession() are required to create a workable mfx session[1] Add AccelerationMode config filter for D3D9/D3D11 session (galinart) The default device is changed to d3d11va for oneVPL when both d3d11va and dxva2 are enabled on Microsoft Windows This is in preparation for oneVPL support [1] https://spec.oneapi.io/versions/latest/elements/oneVPL/source/programming_guide/VPL_prg_session.html#onevpl-dispatcher Signed-off-by: galinart <artem.galin@intel.com> --- libavcodec/qsv.c | 197 ++++++++++++++-- libavcodec/qsv_internal.h | 1 + libavcodec/qsvdec.c | 10 + libavcodec/qsvenc.h | 3 + libavcodec/qsvenc_h264.c | 1 - libavcodec/qsvenc_hevc.c | 1 - libavcodec/qsvenc_jpeg.c | 1 - libavcodec/qsvenc_mpeg2.c | 1 - libavcodec/qsvenc_vp9.c | 1 - libavfilter/qsvvpp.c | 113 ++++++++- libavfilter/qsvvpp.h | 5 + libavfilter/vf_deinterlace_qsv.c | 14 +- libavfilter/vf_scale_qsv.c | 12 +- libavutil/hwcontext_d3d11va.c | 13 ++ libavutil/hwcontext_d3d11va.h | 5 + libavutil/hwcontext_dxva2.c | 8 + libavutil/hwcontext_dxva2.h | 4 + libavutil/hwcontext_qsv.c | 386 +++++++++++++++++++++++++++---- libavutil/hwcontext_qsv.h | 1 + libavutil/hwcontext_vaapi.c | 13 ++ libavutil/hwcontext_vaapi.h | 4 + 21 files changed, 706 insertions(+), 88 deletions(-) diff --git a/libavcodec/qsv.c b/libavcodec/qsv.c index c708964c80..0a0a42932f 100644 --- a/libavcodec/qsv.c +++ b/libavcodec/qsv.c @@ -47,6 +47,12 @@ #include <mfxplugin.h> #endif +#if QSV_ONEVPL +#include <mfxdispatcher.h> +#else +#define MFXUnload(a) do { } while(0) +#endif + int ff_qsv_codec_id_to_mfx(enum AVCodecID codec_id) { switch (codec_id) { @@ -403,6 +409,164 @@ static int ff_qsv_set_display_handle(AVCodecContext *avctx, QSVSession *qs) } #endif //AVCODEC_QSV_LINUX_SESSION_HANDLE +#if QSV_ONEVPL + +static int qsv_create_mfx_session(AVCodecContext *avctx, + mfxIMPL implementation, + mfxVersion *pver, + int gpu_copy, + mfxSession *psession, + void **ploader) +{ + mfxStatus sts; + mfxLoader loader = NULL; + mfxSession session = NULL; + mfxConfig cfg; + mfxVariant impl_value; + uint32_t impl_idx = 0; + + *psession = NULL; + + /* Don't create a new MFX loader if the input loader is valid */ + if (*ploader == NULL) { + av_log(avctx, AV_LOG_VERBOSE, + "Use Intel(R) oneVPL to create MFX session, the required " + "implementation version is %d.%d\n", + pver->Major, pver->Minor); + + loader = MFXLoad(); + + if (!loader) { + av_log(avctx, AV_LOG_ERROR, "Error creating a MFX loader\n"); + goto fail; + } + + /* Create configurations for implementation */ + cfg = MFXCreateConfig(loader); + + if (!cfg) { + av_log(avctx, AV_LOG_ERROR, "Error creating a MFX configurations\n"); + goto fail; + } + + impl_value.Type = MFX_VARIANT_TYPE_U32; + impl_value.Data.U32 = (implementation == MFX_IMPL_SOFTWARE) ? + MFX_IMPL_TYPE_SOFTWARE : MFX_IMPL_TYPE_HARDWARE; + sts = MFXSetConfigFilterProperty(cfg, + (const mfxU8 *)"mfxImplDescription.Impl", impl_value); + + if (sts != MFX_ERR_NONE) { + av_log(avctx, AV_LOG_ERROR, "Error adding a MFX configuration " + "property: %d\n", sts); + goto fail; + } + + impl_value.Type = MFX_VARIANT_TYPE_U32; + impl_value.Data.U32 = pver->Version; + sts = MFXSetConfigFilterProperty(cfg, + (const mfxU8 *)"mfxImplDescription.ApiVersion.Version", + impl_value); + + if (sts != MFX_ERR_NONE) { + av_log(avctx, AV_LOG_ERROR, "Error adding a MFX configuration " + "property: %d\n", sts); + goto fail; + } + } else { + av_log(avctx, AV_LOG_VERBOSE, + "Use Intel(R) oneVPL to create MFX session with the specified MFX loader\n"); + + loader = *ploader; + } + + while (1) { + /* Enumerate all implementations */ + mfxImplDescription *impl_desc; + + sts = MFXEnumImplementations(loader, impl_idx, + MFX_IMPLCAPS_IMPLDESCSTRUCTURE, + (mfxHDL *)&impl_desc); + + /* Failed to find an available implementation */ + if (sts == MFX_ERR_NOT_FOUND) + break; + else if (sts != MFX_ERR_NONE) { + impl_idx++; + continue; + } + + sts = MFXCreateSession(loader, impl_idx, &session); + MFXDispReleaseImplDescription(loader, impl_desc); + + if (sts == MFX_ERR_NONE) + break; + + impl_idx++; + } + + if (sts != MFX_ERR_NONE) { + av_log(avctx, AV_LOG_ERROR, "Error creating a MFX session: %d.\n", sts); + goto fail; + } + + *psession = session; + + if (!*ploader) + *ploader = loader; + + return 0; + +fail: + if (!*ploader && loader) + MFXUnload(loader); + + return AVERROR_UNKNOWN; +} + +#else + +static int qsv_create_mfx_session(AVCodecContext *avctx, + mfxIMPL implementation, + mfxVersion *pver, + int gpu_copy, + mfxSession *psession, + void **ploader) +{ + mfxInitParam init_par = { MFX_IMPL_AUTO_ANY }; + mfxSession session = NULL; + mfxStatus sts; + + av_log(avctx, AV_LOG_VERBOSE, + "Use Intel(R) Media SDK to create MFX session, the required " + "implementation version is %d.%d\n", + pver->Major, pver->Minor); + + *psession = NULL; + *ploader = NULL; + +#if QSV_VERSION_ATLEAST(1, 16) + init_par.GPUCopy = gpu_copy; +#endif + init_par.Implementation = implementation; + init_par.Version = *pver; + sts = MFXInitEx(init_par, &session); + + if (sts < 0) + return ff_qsv_print_error(avctx, sts, + "Error initializing a MFX session"); + else if (sts > 0) { + ff_qsv_print_warning(avctx, sts, + "Warning in MFX initialization"); + return AVERROR_UNKNOWN; + } + + *psession = session; + + return 0; +} + +#endif + int ff_qsv_init_internal_session(AVCodecContext *avctx, QSVSession *qs, const char *load_plugins, int gpu_copy) { @@ -412,20 +576,13 @@ int ff_qsv_init_internal_session(AVCodecContext *avctx, QSVSession *qs, mfxIMPL impl = MFX_IMPL_AUTO_ANY; #endif mfxVersion ver = { { QSV_VERSION_MINOR, QSV_VERSION_MAJOR } }; - mfxInitParam init_par = { MFX_IMPL_AUTO_ANY }; const char *desc; - int ret; + int ret = qsv_create_mfx_session(avctx, impl, &ver, gpu_copy, &qs->session, + &qs->loader); -#if QSV_VERSION_ATLEAST(1, 16) - init_par.GPUCopy = gpu_copy; -#endif - init_par.Implementation = impl; - init_par.Version = ver; - ret = MFXInitEx(init_par, &qs->session); - if (ret < 0) - return ff_qsv_print_error(avctx, ret, - "Error initializing an internal MFX session"); + if (ret) + return ret; #ifdef AVCODEC_QSV_LINUX_SESSION_HANDLE ret = ff_qsv_set_display_handle(avctx, qs); @@ -729,7 +886,7 @@ int ff_qsv_init_session_device(AVCodecContext *avctx, mfxSession *psession, AVHWDeviceContext *device_ctx = (AVHWDeviceContext*)device_ref->data; AVQSVDeviceContext *device_hwctx = device_ctx->hwctx; mfxSession parent_session = device_hwctx->session; - mfxInitParam init_par = { MFX_IMPL_AUTO_ANY }; + void *loader = device_hwctx->loader; mfxHDL handle = NULL; int hw_handle_supported = 0; @@ -770,15 +927,11 @@ int ff_qsv_init_session_device(AVCodecContext *avctx, mfxSession *psession, "from the session\n"); } -#if QSV_VERSION_ATLEAST(1, 16) - init_par.GPUCopy = gpu_copy; -#endif - init_par.Implementation = impl; - init_par.Version = ver; - err = MFXInitEx(init_par, &session); - if (err != MFX_ERR_NONE) - return ff_qsv_print_error(avctx, err, - "Error initializing a child MFX session"); + ret = qsv_create_mfx_session(avctx, impl, &ver, gpu_copy, &session, + &loader); + + if (ret) + return ret; if (handle) { err = MFXVideoCORE_SetHandle(session, handle_type, handle); @@ -855,7 +1008,9 @@ int ff_qsv_close_internal_session(QSVSession *qs) { if (qs->session) { MFXClose(qs->session); + MFXUnload(qs->loader); qs->session = NULL; + qs->loader = NULL; } #ifdef AVCODEC_QSV_LINUX_SESSION_HANDLE av_buffer_unref(&qs->va_device_ref); diff --git a/libavcodec/qsv_internal.h b/libavcodec/qsv_internal.h index c788293286..09c915aeee 100644 --- a/libavcodec/qsv_internal.h +++ b/libavcodec/qsv_internal.h @@ -99,6 +99,7 @@ typedef struct QSVSession { AVBufferRef *va_device_ref; AVHWDeviceContext *va_device_ctx; #endif + void *loader; } QSVSession; typedef struct QSVFramesContext { diff --git a/libavcodec/qsvdec.c b/libavcodec/qsvdec.c index 391e5d1c72..da0d691c4c 100644 --- a/libavcodec/qsvdec.c +++ b/libavcodec/qsvdec.c @@ -47,6 +47,12 @@ #include "qsv.h" #include "qsv_internal.h" +#if QSV_ONEVPL +#include <mfxdispatcher.h> +#else +#define MFXUnload(a) do { } while(0) +#endif + static const AVRational mfx_tb = { 1, 90000 }; #define PTS_TO_MFX_PTS(pts, pts_tb) ((pts) == AV_NOPTS_VALUE ? \ @@ -168,7 +174,9 @@ static int qsv_init_session(AVCodecContext *avctx, QSVContext *q, mfxSession ses } else if (hw_frames_ref) { if (q->internal_qs.session) { MFXClose(q->internal_qs.session); + MFXUnload(q->internal_qs.loader); q->internal_qs.session = NULL; + q->internal_qs.loader = NULL; } av_buffer_unref(&q->frames_ctx.hw_frames_ctx); @@ -193,7 +201,9 @@ static int qsv_init_session(AVCodecContext *avctx, QSVContext *q, mfxSession ses } else if (hw_device_ref) { if (q->internal_qs.session) { MFXClose(q->internal_qs.session); + MFXUnload(q->internal_qs.loader); q->internal_qs.session = NULL; + q->internal_qs.loader = NULL; } ret = ff_qsv_init_session_device(avctx, &q->internal_qs.session, diff --git a/libavcodec/qsvenc.h b/libavcodec/qsvenc.h index 8be00d43f7..7fec3ca322 100644 --- a/libavcodec/qsvenc.h +++ b/libavcodec/qsvenc.h @@ -28,6 +28,9 @@ #include <mfxvideo.h> +#include "libavutil/common.h" +#include "libavutil/hwcontext.h" +#include "libavutil/hwcontext_qsv.h" #include "libavutil/avutil.h" #include "libavutil/fifo.h" diff --git a/libavcodec/qsvenc_h264.c b/libavcodec/qsvenc_h264.c index 7c6f3a530c..fed6cf420e 100644 --- a/libavcodec/qsvenc_h264.c +++ b/libavcodec/qsvenc_h264.c @@ -32,7 +32,6 @@ #include "avcodec.h" #include "internal.h" #include "qsv.h" -#include "qsv_internal.h" #include "qsvenc.h" #include "atsc_a53.h" diff --git a/libavcodec/qsvenc_hevc.c b/libavcodec/qsvenc_hevc.c index e383caf238..7a37db52d6 100644 --- a/libavcodec/qsvenc_hevc.c +++ b/libavcodec/qsvenc_hevc.c @@ -35,7 +35,6 @@ #include "h2645_parse.h" #include "internal.h" #include "qsv.h" -#include "qsv_internal.h" #include "qsvenc.h" enum LoadPlugin { diff --git a/libavcodec/qsvenc_jpeg.c b/libavcodec/qsvenc_jpeg.c index ad8f09befe..f473b1ddbc 100644 --- a/libavcodec/qsvenc_jpeg.c +++ b/libavcodec/qsvenc_jpeg.c @@ -30,7 +30,6 @@ #include "avcodec.h" #include "internal.h" #include "qsv.h" -#include "qsv_internal.h" #include "qsvenc.h" typedef struct QSVMJPEGEncContext { diff --git a/libavcodec/qsvenc_mpeg2.c b/libavcodec/qsvenc_mpeg2.c index 610bbf79c1..0e2a51811c 100644 --- a/libavcodec/qsvenc_mpeg2.c +++ b/libavcodec/qsvenc_mpeg2.c @@ -30,7 +30,6 @@ #include "avcodec.h" #include "internal.h" #include "qsv.h" -#include "qsv_internal.h" #include "qsvenc.h" typedef struct QSVMpeg2EncContext { diff --git a/libavcodec/qsvenc_vp9.c b/libavcodec/qsvenc_vp9.c index 5a0c04481d..aa58ea9540 100644 --- a/libavcodec/qsvenc_vp9.c +++ b/libavcodec/qsvenc_vp9.c @@ -30,7 +30,6 @@ #include "avcodec.h" #include "internal.h" #include "qsv.h" -#include "qsv_internal.h" #include "qsvenc.h" typedef struct QSVVP9EncContext { diff --git a/libavfilter/qsvvpp.c b/libavfilter/qsvvpp.c index 3f984fd5f9..bc87f316aa 100644 --- a/libavfilter/qsvvpp.c +++ b/libavfilter/qsvvpp.c @@ -23,8 +23,6 @@ #include "libavutil/common.h" #include "libavutil/mathematics.h" -#include "libavutil/hwcontext.h" -#include "libavutil/hwcontext_qsv.h" #include "libavutil/time.h" #include "libavutil/pixdesc.h" @@ -32,6 +30,12 @@ #include "qsvvpp.h" #include "video.h" +#if QSV_ONEVPL +#include <mfxdispatcher.h> +#else +#define MFXUnload(a) do { } while(0) +#endif + #define IS_VIDEO_MEMORY(mode) (mode & (MFX_MEMTYPE_VIDEO_MEMORY_DECODER_TARGET | \ MFX_MEMTYPE_VIDEO_MEMORY_PROCESSOR_TARGET)) #if QSV_HAVE_OPAQUE @@ -614,13 +618,11 @@ static int init_vpp_session(AVFilterContext *avctx, QSVVPPContext *s) } /* create a "slave" session with those same properties, to be used for vpp */ - ret = MFXInit(impl, &ver, &s->session); - if (ret < 0) - return ff_qsvvpp_print_error(avctx, ret, "Error initializing a session"); - else if (ret > 0) { - ff_qsvvpp_print_warning(avctx, ret, "Warning in session initialization"); - return AVERROR_UNKNOWN; - } + ret = ff_qsvvpp_create_mfx_session(avctx, device_hwctx->loader, impl, &ver, + &s->session); + + if (ret) + return ret; if (handle) { ret = MFXVideoCORE_SetHandle(s->session, handle_type, handle); @@ -906,3 +908,96 @@ int ff_qsvvpp_filter_frame(QSVVPPContext *s, AVFilterLink *inlink, AVFrame *picr return 0; } + +#if QSV_ONEVPL + +int ff_qsvvpp_create_mfx_session(void *ctx, + void *loader, + mfxIMPL implementation, + mfxVersion *pver, + mfxSession *psession) +{ + mfxStatus sts; + mfxSession session = NULL; + uint32_t impl_idx = 0; + + av_log(ctx, AV_LOG_VERBOSE, + "Use Intel(R) oneVPL to create MFX session with the specified MFX loader\n"); + + if (!loader) { + av_log(ctx, AV_LOG_ERROR, "Invalid MFX Loader handle\n"); + return AVERROR(EINVAL); + } + + while (1) { + /* Enumerate all implementations */ + mfxImplDescription *impl_desc; + + sts = MFXEnumImplementations(loader, impl_idx, + MFX_IMPLCAPS_IMPLDESCSTRUCTURE, + (mfxHDL *)&impl_desc); + + /* Failed to find an available implementation */ + if (sts == MFX_ERR_NOT_FOUND) + break; + else if (sts != MFX_ERR_NONE) { + impl_idx++; + continue; + } + + sts = MFXCreateSession(loader, impl_idx, &session); + MFXDispReleaseImplDescription(loader, impl_desc); + + if (sts == MFX_ERR_NONE) + break; + + impl_idx++; + } + + if (sts < 0) + return ff_qsvvpp_print_error(ctx, sts, + "Error creating a MFX session"); + else if (sts > 0) { + ff_qsvvpp_print_warning(ctx, sts, + "Warning in MFX session creation"); + return AVERROR_UNKNOWN; + } + + *psession = session; + + return 0; +} + +#else + +int ff_qsvvpp_create_mfx_session(void *ctx, + void *loader, + mfxIMPL implementation, + mfxVersion *pver, + mfxSession *psession) +{ + mfxSession session = NULL; + mfxStatus sts; + + av_log(ctx, AV_LOG_VERBOSE, + "Use Intel(R) Media SDK to create MFX session, API version is " + "%d.%d, the required implementation version is %d.%d\n", + MFX_VERSION_MAJOR, MFX_VERSION_MINOR, pver->Major, pver->Minor); + + *psession = NULL; + sts = MFXInit(implementation, pver, &session); + + if (sts < 0) + return ff_qsvvpp_print_error(ctx, sts, + "Error initializing an MFX session"); + else if (sts > 0) { + ff_qsvvpp_print_warning(ctx, sts, "Warning in MFX session initialization"); + return AVERROR_UNKNOWN; + } + + *psession = session; + + return 0; +} + +#endif diff --git a/libavfilter/qsvvpp.h b/libavfilter/qsvvpp.h index 3e7d56021b..a8cfcc565a 100644 --- a/libavfilter/qsvvpp.h +++ b/libavfilter/qsvvpp.h @@ -28,6 +28,8 @@ #include "avfilter.h" #include "libavutil/fifo.h" +#include "libavutil/hwcontext.h" +#include "libavutil/hwcontext_qsv.h" #define FF_INLINK_IDX(link) ((int)((link)->dstpad - (link)->dst->input_pads)) #define FF_OUTLINK_IDX(link) ((int)((link)->srcpad - (link)->src->output_pads)) @@ -122,4 +124,7 @@ int ff_qsvvpp_print_error(void *log_ctx, mfxStatus err, int ff_qsvvpp_print_warning(void *log_ctx, mfxStatus err, const char *warning_string); +int ff_qsvvpp_create_mfx_session(void *ctx, void *loader, mfxIMPL implementation, + mfxVersion *pver, mfxSession *psession); + #endif /* AVFILTER_QSVVPP_H */ diff --git a/libavfilter/vf_deinterlace_qsv.c b/libavfilter/vf_deinterlace_qsv.c index 50f9156d14..4986873cbb 100644 --- a/libavfilter/vf_deinterlace_qsv.c +++ b/libavfilter/vf_deinterlace_qsv.c @@ -163,7 +163,7 @@ static int init_out_session(AVFilterContext *ctx) mfxIMPL impl; mfxVideoParam par; mfxStatus err; - int i; + int i, ret; #if QSV_HAVE_OPAQUE opaque = !!(hw_frames_hwctx->frame_type & MFX_MEMTYPE_OPAQUE_FRAME); @@ -198,13 +198,11 @@ static int init_out_session(AVFilterContext *ctx) /* create a "slave" session with those same properties, to be used for * actual deinterlacing */ - err = MFXInit(impl, &ver, &s->session); - if (err < 0) - return ff_qsvvpp_print_error(ctx, err, "Error initializing a session for deinterlacing"); - else if (err > 0) { - ff_qsvvpp_print_warning(ctx, err, "Warning in session initialization"); - return AVERROR_UNKNOWN; - } + ret = ff_qsvvpp_create_mfx_session(ctx, device_hwctx->loader, impl, &ver, + &s->session); + + if (ret) + return ret; if (handle) { err = MFXVideoCORE_SetHandle(s->session, handle_type, handle); diff --git a/libavfilter/vf_scale_qsv.c b/libavfilter/vf_scale_qsv.c index 30434033d1..82a51ee837 100644 --- a/libavfilter/vf_scale_qsv.c +++ b/libavfilter/vf_scale_qsv.c @@ -281,7 +281,7 @@ static int init_out_session(AVFilterContext *ctx) mfxIMPL impl; mfxVideoParam par; mfxStatus err; - int i; + int i, ret; #if QSV_HAVE_OPAQUE opaque = !!(in_frames_hwctx->frame_type & MFX_MEMTYPE_OPAQUE_FRAME); @@ -318,11 +318,11 @@ static int init_out_session(AVFilterContext *ctx) /* create a "slave" session with those same properties, to be used for * actual scaling */ - err = MFXInit(impl, &ver, &s->session); - if (err != MFX_ERR_NONE) { - av_log(ctx, AV_LOG_ERROR, "Error initializing a session for scaling\n"); - return AVERROR_UNKNOWN; - } + ret = ff_qsvvpp_create_mfx_session(ctx, device_hwctx->loader, impl, &ver, + &s->session); + + if (ret) + return ret; if (handle) { err = MFXVideoCORE_SetHandle(s->session, handle_type, handle); diff --git a/libavutil/hwcontext_d3d11va.c b/libavutil/hwcontext_d3d11va.c index 8ab96bad25..e0e820f164 100644 --- a/libavutil/hwcontext_d3d11va.c +++ b/libavutil/hwcontext_d3d11va.c @@ -525,6 +525,13 @@ static void d3d11va_device_uninit(AVHWDeviceContext *hwdev) } } +static void d3d11va_device_free(AVHWDeviceContext *ctx) +{ + AVD3D11VADeviceContext *hwctx = ctx->hwctx; + + av_free(hwctx->device_name); +} + static int d3d11va_device_create(AVHWDeviceContext *ctx, const char *device, AVDictionary *opts, int flags) { @@ -537,6 +544,8 @@ static int d3d11va_device_create(AVHWDeviceContext *ctx, const char *device, int is_debug = !!av_dict_get(opts, "debug", NULL, 0); int ret; + ctx->free = d3d11va_device_free; + // (On UWP we can't check this.) #if !HAVE_UWP if (!LoadLibrary("d3d11_1sdklayers.dll")) @@ -561,6 +570,10 @@ static int d3d11va_device_create(AVHWDeviceContext *ctx, const char *device, if (FAILED(IDXGIFactory2_EnumAdapters(pDXGIFactory, adapter, &pAdapter))) pAdapter = NULL; IDXGIFactory2_Release(pDXGIFactory); + + device_hwctx->device_name = av_strdup(device); + if (!device_hwctx->device_name) + return AVERROR(ENOMEM); } } diff --git a/libavutil/hwcontext_d3d11va.h b/libavutil/hwcontext_d3d11va.h index 77d2d72f1b..41a315b9e6 100644 --- a/libavutil/hwcontext_d3d11va.h +++ b/libavutil/hwcontext_d3d11va.h @@ -94,6 +94,11 @@ typedef struct AVD3D11VADeviceContext { void (*lock)(void *lock_ctx); void (*unlock)(void *lock_ctx); void *lock_ctx; + + /** + * The string for the used adapter + */ + char *device_name; } AVD3D11VADeviceContext; /** diff --git a/libavutil/hwcontext_dxva2.c b/libavutil/hwcontext_dxva2.c index 53d00fa815..6967357093 100644 --- a/libavutil/hwcontext_dxva2.c +++ b/libavutil/hwcontext_dxva2.c @@ -431,6 +431,7 @@ static void dxva2_device_free(AVHWDeviceContext *ctx) dlclose(priv->dxva2lib); av_freep(&ctx->user_opaque); + av_free(hwctx->device_name); } static int dxva2_device_create9(AVHWDeviceContext *ctx, UINT adapter) @@ -571,6 +572,13 @@ static int dxva2_device_create(AVHWDeviceContext *ctx, const char *device, return AVERROR_UNKNOWN; } + if (device) { + hwctx->device_name = av_strdup(device); + + if (!hwctx->device_name) + return AVERROR(ENOMEM); + } + return 0; } diff --git a/libavutil/hwcontext_dxva2.h b/libavutil/hwcontext_dxva2.h index e1b79bc0de..253ddbed51 100644 --- a/libavutil/hwcontext_dxva2.h +++ b/libavutil/hwcontext_dxva2.h @@ -38,6 +38,10 @@ */ typedef struct AVDXVA2DeviceContext { IDirect3DDeviceManager9 *devmgr; + /** + * The string for the used adapter + */ + char *device_name; } AVDXVA2DeviceContext; /** diff --git a/libavutil/hwcontext_qsv.c b/libavutil/hwcontext_qsv.c index f5bdb84f52..b6cbdf264d 100644 --- a/libavutil/hwcontext_qsv.c +++ b/libavutil/hwcontext_qsv.c @@ -58,6 +58,12 @@ #define QSV_ONEVPL QSV_VERSION_ATLEAST(2, 0) #define QSV_HAVE_OPAQUE !QSV_ONEVPL +#if QSV_ONEVPL +#include <mfxdispatcher.h> +#else +#define MFXUnload(a) do { } while(0) +#endif + typedef struct QSVDevicePriv { AVBufferRef *child_device_ctx; } QSVDevicePriv; @@ -70,12 +76,15 @@ typedef struct QSVDeviceContext { enum AVHWDeviceType child_device_type; enum AVPixelFormat child_pix_fmt; + char *child_device; } QSVDeviceContext; typedef struct QSVFramesContext { mfxSession session_download; + void *loader_download; atomic_int session_download_init; mfxSession session_upload; + void *loader_upload; atomic_int session_upload_init; #if HAVE_PTHREADS pthread_mutex_t session_lock; @@ -288,15 +297,19 @@ static void qsv_frames_uninit(AVHWFramesContext *ctx) if (s->session_download) { MFXVideoVPP_Close(s->session_download); MFXClose(s->session_download); + MFXUnload(s->loader_download); } s->session_download = NULL; + s->loader_download = NULL; s->session_download_init = 0; if (s->session_upload) { MFXVideoVPP_Close(s->session_upload); MFXClose(s->session_upload); + MFXUnload(s->loader_upload); } s->session_upload = NULL; + s->loader_upload = NULL; s->session_upload_init = 0; #if HAVE_PTHREADS @@ -619,8 +632,265 @@ static mfxStatus frame_get_hdl(mfxHDL pthis, mfxMemId mid, mfxHDL *hdl) return MFX_ERR_NONE; } +#if QSV_ONEVPL + +static int qsv_create_mfx_session(void *ctx, + mfxHandleType handle_type, + mfxIMPL implementation, + char *child_device, + mfxVersion *pver, + mfxSession *psession, + void **ploader) +{ + mfxStatus sts; + mfxLoader loader = NULL; + mfxSession session = NULL; + mfxConfig cfg; + mfxVersion ver; + mfxVariant impl_value; + uint32_t impl_idx = 0; + + av_log(ctx, AV_LOG_VERBOSE, + "Use Intel(R) oneVPL to create MFX session, API version is " + "%d.%d, the required implementation version is %d.%d\n", + MFX_VERSION_MAJOR, MFX_VERSION_MINOR, pver->Major, pver->Minor); + + if (handle_type != MFX_HANDLE_VA_DISPLAY && + handle_type != MFX_HANDLE_D3D9_DEVICE_MANAGER && + handle_type != MFX_HANDLE_D3D11_DEVICE) { + av_log(ctx, AV_LOG_ERROR, + "Invalid MFX device handle\n"); + return AVERROR(EXDEV); + } + + *psession = NULL; + *ploader = NULL; + loader = MFXLoad(); + + if (!loader) { + av_log(ctx, AV_LOG_ERROR, "Error creating a MFX loader\n"); + goto fail; + } + + /* Create configurations for implementation */ + cfg = MFXCreateConfig(loader); + + if (!cfg) { + av_log(ctx, AV_LOG_ERROR, "Error creating a MFX configuration\n"); + goto fail; + } + + impl_value.Type = MFX_VARIANT_TYPE_U32; + impl_value.Data.U32 = (implementation == MFX_IMPL_SOFTWARE) ? + MFX_IMPL_TYPE_SOFTWARE : MFX_IMPL_TYPE_HARDWARE; + sts = MFXSetConfigFilterProperty(cfg, + (const mfxU8 *)"mfxImplDescription.Impl", impl_value); + + if (sts != MFX_ERR_NONE) { + av_log(ctx, AV_LOG_ERROR, "Error adding a MFX configuration " + "property: %d.\n", sts); + goto fail; + } + + impl_value.Type = MFX_VARIANT_TYPE_U32; + + if (MFX_HANDLE_VA_DISPLAY == handle_type) + impl_value.Data.U32 = MFX_ACCEL_MODE_VIA_VAAPI; + else if (MFX_HANDLE_D3D9_DEVICE_MANAGER == handle_type) + impl_value.Data.U32 = MFX_ACCEL_MODE_VIA_D3D9; + else + impl_value.Data.U32 = MFX_ACCEL_MODE_VIA_D3D11; + + sts = MFXSetConfigFilterProperty(cfg, + (const mfxU8 *)"mfxImplDescription.AccelerationMode", impl_value); + + if (sts != MFX_ERR_NONE) { + av_log(ctx, AV_LOG_ERROR, "Error adding a MFX configuration" + "MFX_ACCEL_MODE_VIA_D3D9 property: %d.\n", sts); + goto fail; + } + + if (child_device) { + if ((MFX_HANDLE_D3D9_DEVICE_MANAGER == handle_type || + MFX_HANDLE_D3D11_DEVICE == handle_type)) { + uint32_t idx = atoi(child_device); + + impl_value.Type = MFX_VARIANT_TYPE_U32; + impl_value.Data.U32 = idx; + sts = MFXSetConfigFilterProperty(cfg, + (const mfxU8 *)"mfxImplDescription.VendorImplID", impl_value); + + if (sts != MFX_ERR_NONE) { + av_log(ctx, AV_LOG_ERROR, "Error adding a MFX configuration" + "VendorImplID property: %d.\n", sts); + goto fail; + } + } else { + uint32_t node; + + if ((sscanf(child_device, "/dev/dri/renderD%d", &node) != 1) && + (sscanf(child_device, "/dev/dri/card%d", &node) != 1)) { + av_log(ctx, AV_LOG_ERROR, "Invalid DRI device\n"); + goto fail; + } + + /* Use the corresponding render node to find the implementation for card0, card1, ... */ + if (node < 128) + node += 128; + + impl_value.Type = MFX_VARIANT_TYPE_U32; + impl_value.Data.U32 = node; + MFXSetConfigFilterProperty(cfg, + (const mfxU8 *)"mfxExtendedDeviceId.DRMRenderNodeNum", impl_value); + + if (sts != MFX_ERR_NONE) { + av_log(ctx, AV_LOG_ERROR, "Error adding a MFX configuration" + "DRMRenderNodeNum property: %d.\n", sts); + goto fail; + } + } + } + + impl_value.Type = MFX_VARIANT_TYPE_U32; + impl_value.Data.U32 = pver->Version; + sts = MFXSetConfigFilterProperty(cfg, + (const mfxU8 *)"mfxImplDescription.ApiVersion.Version", + impl_value); + + if (sts != MFX_ERR_NONE) { + av_log(ctx, AV_LOG_ERROR, "Error adding a MFX configuration " + "property: %d.\n", sts); + goto fail; + } + + while (1) { + /* Enumerate all implementations */ + mfxImplDescription *impl_desc; + + sts = MFXEnumImplementations(loader, impl_idx, + MFX_IMPLCAPS_IMPLDESCSTRUCTURE, + (mfxHDL *)&impl_desc); + + /* Failed to find an available implementation */ + if (sts == MFX_ERR_NOT_FOUND) + break; + else if (sts != MFX_ERR_NONE) { + impl_idx++; + continue; + } + + sts = MFXCreateSession(loader, impl_idx, &session); + MFXDispReleaseImplDescription(loader, impl_desc); + + if (sts == MFX_ERR_NONE) + break; + + impl_idx++; + } + + if (sts != MFX_ERR_NONE) { + av_log(ctx, AV_LOG_ERROR, "Error creating a MFX session: %d.\n", sts); + goto fail; + } + + sts = MFXQueryVersion(session, &ver); + + if (sts != MFX_ERR_NONE) { + av_log(ctx, AV_LOG_ERROR, "Error querying a MFX session: %d.\n", sts); + goto fail; + } + + av_log(ctx, AV_LOG_VERBOSE, "Initialize MFX session: implementation " + "version is %d.%d\n", ver.Major, ver.Minor); + + *psession = session; + *ploader = loader; + + return 0; + +fail: + if (session) + MFXClose(session); + + MFXUnload(loader); + + return AVERROR_UNKNOWN; +} + +#else + +static int qsv_create_mfx_session(void *ctx, + mfxHandleType handle_type, + mfxIMPL implementation, + char *child_device, + mfxVersion *pver, + mfxSession *psession, + void **ploader) +{ + mfxVersion ver; + mfxStatus sts; + mfxSession session = NULL; + + av_log(ctx, AV_LOG_VERBOSE, + "Use Intel(R) Media SDK to create MFX session, API version is " + "%d.%d, the required implementation version is %d.%d\n", + MFX_VERSION_MAJOR, MFX_VERSION_MINOR, pver->Major, pver->Minor); + + if (handle_type != MFX_HANDLE_VA_DISPLAY && + handle_type != MFX_HANDLE_D3D9_DEVICE_MANAGER && + handle_type != MFX_HANDLE_D3D11_DEVICE) { + av_log(ctx, AV_LOG_ERROR, + "Invalid MFX device handle\n"); + return AVERROR(EXDEV); + } + + *ploader = NULL; + *psession = NULL; + ver = *pver; + sts = MFXInit(implementation, &ver, &session); + + if (sts != MFX_ERR_NONE) { + av_log(ctx, AV_LOG_ERROR, "Error initializing an MFX session: " + "%d.\n", sts); + goto fail; + } + + sts = MFXQueryVersion(session, &ver); + + if (sts != MFX_ERR_NONE) { + av_log(ctx, AV_LOG_ERROR, "Error querying an MFX session: " + "%d.\n", sts); + goto fail; + } + + av_log(ctx, AV_LOG_VERBOSE, "Initialize MFX session: implementation " + "version is %d.%d\n", ver.Major, ver.Minor); + + MFXClose(session); + sts = MFXInit(implementation, &ver, &session); + + if (sts != MFX_ERR_NONE) { + av_log(ctx, AV_LOG_ERROR, "Error initializing an MFX session: " + "%d.\n", sts); + goto fail; + } + + *psession = session; + + return 0; + +fail: + if (session) + MFXClose(session); + + return AVERROR_UNKNOWN; +} + +#endif + static int qsv_init_internal_session(AVHWFramesContext *ctx, - mfxSession *session, int upload) + mfxSession *session, void **loader, + int upload) { AVQSVFramesContext *frames_hwctx = ctx->hwctx; QSVDeviceContext *device_priv = ctx->device_ctx->internal->priv; @@ -637,29 +907,36 @@ static int qsv_init_internal_session(AVHWFramesContext *ctx, mfxVideoParam par; mfxStatus err; + int ret = AVERROR_UNKNOWN; #if QSV_HAVE_OPAQUE QSVFramesContext *s = ctx->internal->priv; opaque = !!(frames_hwctx->frame_type & MFX_MEMTYPE_OPAQUE_FRAME); #endif - err = MFXInit(device_priv->impl, &device_priv->ver, session); - if (err != MFX_ERR_NONE) { - av_log(ctx, AV_LOG_ERROR, "Error initializing an internal session\n"); - return AVERROR_UNKNOWN; - } + ret = qsv_create_mfx_session(ctx, device_priv->handle_type, + device_priv->impl, device_priv->child_device, + &device_priv->ver, session, + loader); + + if (ret) + goto fail; if (device_priv->handle) { err = MFXVideoCORE_SetHandle(*session, device_priv->handle_type, device_priv->handle); - if (err != MFX_ERR_NONE) - return AVERROR_UNKNOWN; + if (err != MFX_ERR_NONE) { + ret = AVERROR_UNKNOWN; + goto fail; + } } if (!opaque) { err = MFXVideoCORE_SetFrameAllocator(*session, &frame_allocator); - if (err != MFX_ERR_NONE) - return AVERROR_UNKNOWN; + if (err != MFX_ERR_NONE) { + ret = AVERROR_UNKNOWN; + goto fail; + } } memset(&par, 0, sizeof(par)); @@ -695,11 +972,22 @@ static int qsv_init_internal_session(AVHWFramesContext *ctx, if (err != MFX_ERR_NONE) { av_log(ctx, AV_LOG_VERBOSE, "Error opening the internal VPP session." "Surface upload/download will not be possible\n"); - MFXClose(*session); - *session = NULL; + + ret = AVERROR_UNKNOWN; + goto fail; } return 0; + +fail: + if (*session) + MFXClose(*session); + + MFXUnload(*loader); + *session = NULL; + *loader = NULL; + + return ret; } static int qsv_frames_init(AVHWFramesContext *ctx) @@ -764,6 +1052,9 @@ static int qsv_frames_init(AVHWFramesContext *ctx) s->session_download = NULL; s->session_upload = NULL; + s->loader_download = NULL; + s->loader_upload = NULL; + s->session_download_init = 0; s->session_upload_init = 0; @@ -1053,6 +1344,7 @@ static int qsv_internal_session_check_init(AVHWFramesContext *ctx, int upload) QSVFramesContext *s = ctx->internal->priv; atomic_int *inited = upload ? &s->session_upload_init : &s->session_download_init; mfxSession *session = upload ? &s->session_upload : &s->session_download; + void **loader = upload ? &s->loader_upload : &s->loader_download; int ret = 0; if (atomic_load(inited)) @@ -1063,7 +1355,7 @@ static int qsv_internal_session_check_init(AVHWFramesContext *ctx, int upload) #endif if (!atomic_load(inited)) { - ret = qsv_init_internal_session(ctx, session, upload); + ret = qsv_init_internal_session(ctx, session, loader, upload); atomic_store(inited, 1); } @@ -1406,10 +1698,14 @@ static void qsv_device_free(AVHWDeviceContext *ctx) { AVQSVDeviceContext *hwctx = ctx->hwctx; QSVDevicePriv *priv = ctx->user_opaque; + QSVDeviceContext *device_priv = ctx->internal->priv; + + av_free(device_priv->child_device); if (hwctx->session) MFXClose(hwctx->session); + MFXUnload(hwctx->loader); av_buffer_unref(&priv->child_device_ctx); av_freep(&priv); } @@ -1459,6 +1755,7 @@ static int qsv_device_derive_from_child(AVHWDeviceContext *ctx, int flags) { AVQSVDeviceContext *hwctx = ctx->hwctx; + char *child_device = NULL; mfxVersion ver = { { 3, 1 } }; mfxHDL handle; @@ -1473,6 +1770,7 @@ static int qsv_device_derive_from_child(AVHWDeviceContext *ctx, AVVAAPIDeviceContext *child_device_hwctx = child_device_ctx->hwctx; handle_type = MFX_HANDLE_VA_DISPLAY; handle = (mfxHDL)child_device_hwctx->display; + child_device = child_device_hwctx->device_name; } break; #endif @@ -1482,6 +1780,7 @@ static int qsv_device_derive_from_child(AVHWDeviceContext *ctx, AVD3D11VADeviceContext *child_device_hwctx = child_device_ctx->hwctx; handle_type = MFX_HANDLE_D3D11_DEVICE; handle = (mfxHDL)child_device_hwctx->device; + child_device = child_device_hwctx->device_name; } break; #endif @@ -1491,6 +1790,7 @@ static int qsv_device_derive_from_child(AVHWDeviceContext *ctx, AVDXVA2DeviceContext *child_device_hwctx = child_device_ctx->hwctx; handle_type = MFX_HANDLE_D3D9_DEVICE_MANAGER; handle = (mfxHDL)child_device_hwctx->devmgr; + child_device = child_device_hwctx->device_name; } break; #endif @@ -1499,34 +1799,11 @@ static int qsv_device_derive_from_child(AVHWDeviceContext *ctx, goto fail; } - err = MFXInit(implementation, &ver, &hwctx->session); - if (err != MFX_ERR_NONE) { - av_log(ctx, AV_LOG_ERROR, "Error initializing an MFX session: " - "%d.\n", err); - ret = AVERROR_UNKNOWN; - goto fail; - } + ret = qsv_create_mfx_session(ctx, handle_type, implementation, child_device, + &ver, &hwctx->session, &hwctx->loader); - err = MFXQueryVersion(hwctx->session, &ver); - if (err != MFX_ERR_NONE) { - av_log(ctx, AV_LOG_ERROR, "Error querying an MFX session: %d.\n", err); - ret = AVERROR_UNKNOWN; + if (ret) goto fail; - } - - av_log(ctx, AV_LOG_VERBOSE, - "Initialize MFX session: API version is %d.%d, implementation version is %d.%d\n", - MFX_VERSION_MAJOR, MFX_VERSION_MINOR, ver.Major, ver.Minor); - - MFXClose(hwctx->session); - - err = MFXInit(implementation, &ver, &hwctx->session); - if (err != MFX_ERR_NONE) { - av_log(ctx, AV_LOG_ERROR, - "Error initializing an MFX session: %d.\n", err); - ret = AVERROR_UNKNOWN; - goto fail; - } err = MFXVideoCORE_SetHandle(hwctx->session, handle_type, handle); if (err != MFX_ERR_NONE) { @@ -1541,6 +1818,8 @@ static int qsv_device_derive_from_child(AVHWDeviceContext *ctx, fail: if (hwctx->session) MFXClose(hwctx->session); + + MFXUnload(hwctx->loader); return ret; } @@ -1562,6 +1841,7 @@ static int qsv_device_create(AVHWDeviceContext *ctx, const char *device, AVHWDeviceContext *child_device; AVDictionary *child_device_opts; AVDictionaryEntry *e; + QSVDeviceContext *device_priv; mfxIMPL impl; int ret; @@ -1583,6 +1863,16 @@ static int qsv_device_create(AVHWDeviceContext *ctx, const char *device, } } else if (CONFIG_VAAPI) { child_device_type = AV_HWDEVICE_TYPE_VAAPI; +#if QSV_ONEVPL + } else if (CONFIG_D3D11VA) { // Use D3D11 by default if d3d11va is enabled + av_log(NULL, AV_LOG_WARNING, + "WARNING: defaulting child_device_type to AV_HWDEVICE_TYPE_D3D11VA for " + "oneVPL. Please explicitly set child device type via \"-init_hw_device\" " + "option if needed.\n"); + child_device_type = AV_HWDEVICE_TYPE_D3D11VA; + } else if (CONFIG_DXVA2) { + child_device_type = AV_HWDEVICE_TYPE_DXVA2; +#else } else if (CONFIG_DXVA2) { av_log(NULL, AV_LOG_WARNING, "WARNING: defaulting child_device_type to AV_HWDEVICE_TYPE_DXVA2 for compatibility " @@ -1591,6 +1881,7 @@ static int qsv_device_create(AVHWDeviceContext *ctx, const char *device, child_device_type = AV_HWDEVICE_TYPE_DXVA2; } else if (CONFIG_D3D11VA) { child_device_type = AV_HWDEVICE_TYPE_D3D11VA; +#endif } else { av_log(ctx, AV_LOG_ERROR, "No supported child device type is enabled\n"); return AVERROR(ENOSYS); @@ -1617,6 +1908,13 @@ static int qsv_device_create(AVHWDeviceContext *ctx, const char *device, #endif #if CONFIG_DXVA2 case AV_HWDEVICE_TYPE_DXVA2: +#if QSV_ONEVPL + { + av_log(NULL, AV_LOG_WARNING, + "WARNING: d3d11va is not available or child device type is " + "set to dxva2 explicitly for oneVPL.\n"); + } +#endif break; #endif default: @@ -1627,7 +1925,17 @@ static int qsv_device_create(AVHWDeviceContext *ctx, const char *device, break; } + device_priv = ctx->internal->priv; + device_priv->child_device = NULL; e = av_dict_get(opts, "child_device", NULL, 0); + + if (e) { + device_priv->child_device = av_strdup(e->value); + + if (!device_priv->child_device) + return AVERROR(ENOMEM); + } + ret = av_hwdevice_ctx_create(&priv->child_device_ctx, child_device_type, e ? e->value : NULL, child_device_opts, 0); diff --git a/libavutil/hwcontext_qsv.h b/libavutil/hwcontext_qsv.h index 42e34d0dda..2485daa899 100644 --- a/libavutil/hwcontext_qsv.h +++ b/libavutil/hwcontext_qsv.h @@ -34,6 +34,7 @@ */ typedef struct AVQSVDeviceContext { mfxSession session; + void *loader; } AVQSVDeviceContext; /** diff --git a/libavutil/hwcontext_vaapi.c b/libavutil/hwcontext_vaapi.c index 994b744e4d..89fdefee68 100644 --- a/libavutil/hwcontext_vaapi.c +++ b/libavutil/hwcontext_vaapi.c @@ -1517,6 +1517,7 @@ static void vaapi_device_free(AVHWDeviceContext *ctx) if (priv->drm_fd >= 0) close(priv->drm_fd); + av_free(hwctx->device_name); av_freep(&priv); } @@ -1565,6 +1566,7 @@ static int vaapi_device_connect(AVHWDeviceContext *ctx, static int vaapi_device_create(AVHWDeviceContext *ctx, const char *device, AVDictionary *opts, int flags) { + AVVAAPIDeviceContext *hwctx = ctx->hwctx; VAAPIDevicePriv *priv; VADisplay display = NULL; const AVDictionaryEntry *ent; @@ -1610,6 +1612,11 @@ static int vaapi_device_create(AVHWDeviceContext *ctx, const char *device, "DRM device node.\n", device); break; } + + hwctx->device_name = av_strdup(device); + + if (!hwctx->device_name) + return AVERROR(ENOMEM); } else { char path[64]; int n, max_devices = 8; @@ -1650,6 +1657,12 @@ static int vaapi_device_create(AVHWDeviceContext *ctx, const char *device, av_log(ctx, AV_LOG_VERBOSE, "Trying to use " "DRM render node for device %d.\n", n); } + + hwctx->device_name = av_strdup(path); + + if (!hwctx->device_name) + return AVERROR(ENOMEM); + break; } if (n >= max_devices) diff --git a/libavutil/hwcontext_vaapi.h b/libavutil/hwcontext_vaapi.h index 0b2e071cb3..3e0b54f5e9 100644 --- a/libavutil/hwcontext_vaapi.h +++ b/libavutil/hwcontext_vaapi.h @@ -78,6 +78,10 @@ typedef struct AVVAAPIDeviceContext { * operations using VAAPI with the same VADisplay. */ unsigned int driver_quirks; + /** + * The string for the used device + */ + char *device_name; } AVVAAPIDeviceContext; /** -- 2.17.1 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [FFmpeg-devel] [PATCH v7 09/10] qsv: use a new method to create mfx session when using oneVPL 2022-03-11 8:16 ` [FFmpeg-devel] [PATCH v7 09/10] qsv: use a new method to create mfx session when using oneVPL Xiang, Haihao @ 2022-03-11 8:35 ` Hendrik Leppkes 2022-03-11 13:43 ` Xiang, Haihao 0 siblings, 1 reply; 26+ messages in thread From: Hendrik Leppkes @ 2022-03-11 8:35 UTC (permalink / raw) To: FFmpeg development discussions and patches On Fri, Mar 11, 2022 at 9:18 AM Xiang, Haihao <haihao.xiang-at-intel.com@ffmpeg.org> wrote: > diff --git a/libavutil/hwcontext_d3d11va.c b/libavutil/hwcontext_d3d11va.c > index 8ab96bad25..e0e820f164 100644 > --- a/libavutil/hwcontext_d3d11va.c > +++ b/libavutil/hwcontext_d3d11va.c > @@ -525,6 +525,13 @@ static void d3d11va_device_uninit(AVHWDeviceContext *hwdev) > } > } > > +static void d3d11va_device_free(AVHWDeviceContext *ctx) > +{ > + AVD3D11VADeviceContext *hwctx = ctx->hwctx; > + > + av_free(hwctx->device_name); > +} > + > static int d3d11va_device_create(AVHWDeviceContext *ctx, const char *device, > AVDictionary *opts, int flags) > { > @@ -537,6 +544,8 @@ static int d3d11va_device_create(AVHWDeviceContext *ctx, const char *device, > int is_debug = !!av_dict_get(opts, "debug", NULL, 0); > int ret; > > + ctx->free = d3d11va_device_free; > + > // (On UWP we can't check this.) > #if !HAVE_UWP > if (!LoadLibrary("d3d11_1sdklayers.dll")) > @@ -561,6 +570,10 @@ static int d3d11va_device_create(AVHWDeviceContext *ctx, const char *device, > if (FAILED(IDXGIFactory2_EnumAdapters(pDXGIFactory, adapter, &pAdapter))) > pAdapter = NULL; > IDXGIFactory2_Release(pDXGIFactory); > + > + device_hwctx->device_name = av_strdup(device); > + if (!device_hwctx->device_name) > + return AVERROR(ENOMEM); > } > } > > diff --git a/libavutil/hwcontext_d3d11va.h b/libavutil/hwcontext_d3d11va.h > index 77d2d72f1b..41a315b9e6 100644 > --- a/libavutil/hwcontext_d3d11va.h > +++ b/libavutil/hwcontext_d3d11va.h > @@ -94,6 +94,11 @@ typedef struct AVD3D11VADeviceContext { > void (*lock)(void *lock_ctx); > void (*unlock)(void *lock_ctx); > void *lock_ctx; > + > + /** > + * The string for the used adapter > + */ > + char *device_name; > } AVD3D11VADeviceContext; > > /** > diff --git a/libavutil/hwcontext_dxva2.c b/libavutil/hwcontext_dxva2.c > index 53d00fa815..6967357093 100644 > --- a/libavutil/hwcontext_dxva2.c > +++ b/libavutil/hwcontext_dxva2.c > @@ -431,6 +431,7 @@ static void dxva2_device_free(AVHWDeviceContext *ctx) > dlclose(priv->dxva2lib); > > av_freep(&ctx->user_opaque); > + av_free(hwctx->device_name); > } > > static int dxva2_device_create9(AVHWDeviceContext *ctx, UINT adapter) > @@ -571,6 +572,13 @@ static int dxva2_device_create(AVHWDeviceContext *ctx, const char *device, > return AVERROR_UNKNOWN; > } > > + if (device) { > + hwctx->device_name = av_strdup(device); > + > + if (!hwctx->device_name) > + return AVERROR(ENOMEM); > + } > + > return 0; > } > > diff --git a/libavutil/hwcontext_dxva2.h b/libavutil/hwcontext_dxva2.h > index e1b79bc0de..253ddbed51 100644 > --- a/libavutil/hwcontext_dxva2.h > +++ b/libavutil/hwcontext_dxva2.h > @@ -38,6 +38,10 @@ > */ > typedef struct AVDXVA2DeviceContext { > IDirect3DDeviceManager9 *devmgr; > + /** > + * The string for the used adapter > + */ > + char *device_name; > } AVDXVA2DeviceContext; > > /** Why are these device names required? I would think deriving a child device would use the actual device, eg. ID3D11Device or IDirect3DDeviceManager9 (and whatever for VAAPI), and not some string (that may or may not even be set). It feels quite a bit icky to store these in the context just for qsv to do... what with? - Hendrik _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [FFmpeg-devel] [PATCH v7 09/10] qsv: use a new method to create mfx session when using oneVPL 2022-03-11 8:35 ` Hendrik Leppkes @ 2022-03-11 13:43 ` Xiang, Haihao 2022-03-11 14:00 ` Hendrik Leppkes 0 siblings, 1 reply; 26+ messages in thread From: Xiang, Haihao @ 2022-03-11 13:43 UTC (permalink / raw) To: ffmpeg-devel On Fri, 2022-03-11 at 09:35 +0100, Hendrik Leppkes wrote: > On Fri, Mar 11, 2022 at 9:18 AM Xiang, Haihao > <haihao.xiang-at-intel.com@ffmpeg.org> wrote: > > diff --git a/libavutil/hwcontext_d3d11va.c b/libavutil/hwcontext_d3d11va.c > > index 8ab96bad25..e0e820f164 100644 > > --- a/libavutil/hwcontext_d3d11va.c > > +++ b/libavutil/hwcontext_d3d11va.c > > @@ -525,6 +525,13 @@ static void d3d11va_device_uninit(AVHWDeviceContext > > *hwdev) > > } > > } > > > > +static void d3d11va_device_free(AVHWDeviceContext *ctx) > > +{ > > + AVD3D11VADeviceContext *hwctx = ctx->hwctx; > > + > > + av_free(hwctx->device_name); > > +} > > + > > static int d3d11va_device_create(AVHWDeviceContext *ctx, const char > > *device, > > AVDictionary *opts, int flags) > > { > > @@ -537,6 +544,8 @@ static int d3d11va_device_create(AVHWDeviceContext *ctx, > > const char *device, > > int is_debug = !!av_dict_get(opts, "debug", NULL, 0); > > int ret; > > > > + ctx->free = d3d11va_device_free; > > + > > // (On UWP we can't check this.) > > #if !HAVE_UWP > > if (!LoadLibrary("d3d11_1sdklayers.dll")) > > @@ -561,6 +570,10 @@ static int d3d11va_device_create(AVHWDeviceContext > > *ctx, const char *device, > > if (FAILED(IDXGIFactory2_EnumAdapters(pDXGIFactory, adapter, > > &pAdapter))) > > pAdapter = NULL; > > IDXGIFactory2_Release(pDXGIFactory); > > + > > + device_hwctx->device_name = av_strdup(device); > > + if (!device_hwctx->device_name) > > + return AVERROR(ENOMEM); > > } > > } > > > > diff --git a/libavutil/hwcontext_d3d11va.h b/libavutil/hwcontext_d3d11va.h > > index 77d2d72f1b..41a315b9e6 100644 > > --- a/libavutil/hwcontext_d3d11va.h > > +++ b/libavutil/hwcontext_d3d11va.h > > @@ -94,6 +94,11 @@ typedef struct AVD3D11VADeviceContext { > > void (*lock)(void *lock_ctx); > > void (*unlock)(void *lock_ctx); > > void *lock_ctx; > > + > > + /** > > + * The string for the used adapter > > + */ > > + char *device_name; > > } AVD3D11VADeviceContext; > > > > /** > > diff --git a/libavutil/hwcontext_dxva2.c b/libavutil/hwcontext_dxva2.c > > index 53d00fa815..6967357093 100644 > > --- a/libavutil/hwcontext_dxva2.c > > +++ b/libavutil/hwcontext_dxva2.c > > @@ -431,6 +431,7 @@ static void dxva2_device_free(AVHWDeviceContext *ctx) > > dlclose(priv->dxva2lib); > > > > av_freep(&ctx->user_opaque); > > + av_free(hwctx->device_name); > > } > > > > static int dxva2_device_create9(AVHWDeviceContext *ctx, UINT adapter) > > @@ -571,6 +572,13 @@ static int dxva2_device_create(AVHWDeviceContext *ctx, > > const char *device, > > return AVERROR_UNKNOWN; > > } > > > > + if (device) { > > + hwctx->device_name = av_strdup(device); > > + > > + if (!hwctx->device_name) > > + return AVERROR(ENOMEM); > > + } > > + > > return 0; > > } > > > > diff --git a/libavutil/hwcontext_dxva2.h b/libavutil/hwcontext_dxva2.h > > index e1b79bc0de..253ddbed51 100644 > > --- a/libavutil/hwcontext_dxva2.h > > +++ b/libavutil/hwcontext_dxva2.h > > @@ -38,6 +38,10 @@ > > */ > > typedef struct AVDXVA2DeviceContext { > > IDirect3DDeviceManager9 *devmgr; > > + /** > > + * The string for the used adapter > > + */ > > + char *device_name; > > } AVDXVA2DeviceContext; > > > > /** > > Why are these device names required? I would think deriving a child > device would use the actual device, eg. ID3D11Device or > IDirect3DDeviceManager9 (and whatever for VAAPI), and not some string > (that may or may not even be set). > It feels quite a bit icky to store these in the context just for qsv > to do... what with? Yes, it is a little ugly here. MediaSDK or oneVPL application creates mfx session and the device (dxva2, d3d11va or vaapi), then pass this device to the SDK through MFXVideoCORE_SetHandle(). implementation is introduced in oneVPL ( https://spec.oneapi.io/versions/latest/elements/oneVPL/source/API_ref/VPL_disp_api_struct.html#structmfx_impl_description ) and user must select an available implementation before the creation of mfx session, however the device handle is unknown in the SDK when selecting an available implementation, the SDK provides a method to select implementation via the given adapter (on Windows) or DRI device node (on Linux). The default implementation will be selected if child device name is unknown. Thanks Haihao _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [FFmpeg-devel] [PATCH v7 09/10] qsv: use a new method to create mfx session when using oneVPL 2022-03-11 13:43 ` Xiang, Haihao @ 2022-03-11 14:00 ` Hendrik Leppkes 2022-03-15 7:24 ` Xiang, Haihao 0 siblings, 1 reply; 26+ messages in thread From: Hendrik Leppkes @ 2022-03-11 14:00 UTC (permalink / raw) To: FFmpeg development discussions and patches On Fri, Mar 11, 2022 at 2:43 PM Xiang, Haihao <haihao.xiang-at-intel.com@ffmpeg.org> wrote: > > On Fri, 2022-03-11 at 09:35 +0100, Hendrik Leppkes wrote: > > On Fri, Mar 11, 2022 at 9:18 AM Xiang, Haihao > > <haihao.xiang-at-intel.com@ffmpeg.org> wrote: > > > diff --git a/libavutil/hwcontext_d3d11va.c b/libavutil/hwcontext_d3d11va.c > > > index 8ab96bad25..e0e820f164 100644 > > > --- a/libavutil/hwcontext_d3d11va.c > > > +++ b/libavutil/hwcontext_d3d11va.c > > > @@ -525,6 +525,13 @@ static void d3d11va_device_uninit(AVHWDeviceContext > > > *hwdev) > > > } > > > } > > > > > > +static void d3d11va_device_free(AVHWDeviceContext *ctx) > > > +{ > > > + AVD3D11VADeviceContext *hwctx = ctx->hwctx; > > > + > > > + av_free(hwctx->device_name); > > > +} > > > + > > > static int d3d11va_device_create(AVHWDeviceContext *ctx, const char > > > *device, > > > AVDictionary *opts, int flags) > > > { > > > @@ -537,6 +544,8 @@ static int d3d11va_device_create(AVHWDeviceContext *ctx, > > > const char *device, > > > int is_debug = !!av_dict_get(opts, "debug", NULL, 0); > > > int ret; > > > > > > + ctx->free = d3d11va_device_free; > > > + > > > // (On UWP we can't check this.) > > > #if !HAVE_UWP > > > if (!LoadLibrary("d3d11_1sdklayers.dll")) > > > @@ -561,6 +570,10 @@ static int d3d11va_device_create(AVHWDeviceContext > > > *ctx, const char *device, > > > if (FAILED(IDXGIFactory2_EnumAdapters(pDXGIFactory, adapter, > > > &pAdapter))) > > > pAdapter = NULL; > > > IDXGIFactory2_Release(pDXGIFactory); > > > + > > > + device_hwctx->device_name = av_strdup(device); > > > + if (!device_hwctx->device_name) > > > + return AVERROR(ENOMEM); > > > } > > > } > > > > > > diff --git a/libavutil/hwcontext_d3d11va.h b/libavutil/hwcontext_d3d11va.h > > > index 77d2d72f1b..41a315b9e6 100644 > > > --- a/libavutil/hwcontext_d3d11va.h > > > +++ b/libavutil/hwcontext_d3d11va.h > > > @@ -94,6 +94,11 @@ typedef struct AVD3D11VADeviceContext { > > > void (*lock)(void *lock_ctx); > > > void (*unlock)(void *lock_ctx); > > > void *lock_ctx; > > > + > > > + /** > > > + * The string for the used adapter > > > + */ > > > + char *device_name; > > > } AVD3D11VADeviceContext; > > > > > > /** > > > diff --git a/libavutil/hwcontext_dxva2.c b/libavutil/hwcontext_dxva2.c > > > index 53d00fa815..6967357093 100644 > > > --- a/libavutil/hwcontext_dxva2.c > > > +++ b/libavutil/hwcontext_dxva2.c > > > @@ -431,6 +431,7 @@ static void dxva2_device_free(AVHWDeviceContext *ctx) > > > dlclose(priv->dxva2lib); > > > > > > av_freep(&ctx->user_opaque); > > > + av_free(hwctx->device_name); > > > } > > > > > > static int dxva2_device_create9(AVHWDeviceContext *ctx, UINT adapter) > > > @@ -571,6 +572,13 @@ static int dxva2_device_create(AVHWDeviceContext *ctx, > > > const char *device, > > > return AVERROR_UNKNOWN; > > > } > > > > > > + if (device) { > > > + hwctx->device_name = av_strdup(device); > > > + > > > + if (!hwctx->device_name) > > > + return AVERROR(ENOMEM); > > > + } > > > + > > > return 0; > > > } > > > > > > diff --git a/libavutil/hwcontext_dxva2.h b/libavutil/hwcontext_dxva2.h > > > index e1b79bc0de..253ddbed51 100644 > > > --- a/libavutil/hwcontext_dxva2.h > > > +++ b/libavutil/hwcontext_dxva2.h > > > @@ -38,6 +38,10 @@ > > > */ > > > typedef struct AVDXVA2DeviceContext { > > > IDirect3DDeviceManager9 *devmgr; > > > + /** > > > + * The string for the used adapter > > > + */ > > > + char *device_name; > > > } AVDXVA2DeviceContext; > > > > > > /** > > > > Why are these device names required? I would think deriving a child > > device would use the actual device, eg. ID3D11Device or > > IDirect3DDeviceManager9 (and whatever for VAAPI), and not some string > > (that may or may not even be set). > > It feels quite a bit icky to store these in the context just for qsv > > to do... what with? > > Yes, it is a little ugly here. MediaSDK or oneVPL application creates mfx > session and the device (dxva2, d3d11va or vaapi), then pass this device to the > SDK through MFXVideoCORE_SetHandle(). implementation is introduced in oneVPL ( > https://spec.oneapi.io/versions/latest/elements/oneVPL/source/API_ref/VPL_disp_api_struct.html#structmfx_impl_description > ) and user must select an available implementation before the creation of mfx > session, however the device handle is unknown in the SDK when selecting an > available implementation, the SDK provides a method to select implementation via > the given adapter (on Windows) or DRI device node (on Linux). The default > implementation will be selected if child device name is unknown. > First of all, whoever made that API should get a stern message. Expecting to properly interoperate with the dominant platform APIs should be a primary goal, and it sounds like it was somehow shoehorned in after the fact. For D3D11 for example, you can get the IDXGIAdapter a device was created from, isn't there enough information in there to pass-on without storing a string field? IDXGIAdapter::GetDesc has tons of identification information to identify the device in use. D3D9 probably has something similar, haven't checked right now. - Hendrik _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [FFmpeg-devel] [PATCH v7 09/10] qsv: use a new method to create mfx session when using oneVPL 2022-03-11 14:00 ` Hendrik Leppkes @ 2022-03-15 7:24 ` Xiang, Haihao 2022-04-04 9:46 ` Hendrik Leppkes 0 siblings, 1 reply; 26+ messages in thread From: Xiang, Haihao @ 2022-03-15 7:24 UTC (permalink / raw) To: ffmpeg-devel On Fri, 2022-03-11 at 15:00 +0100, Hendrik Leppkes wrote: > On Fri, Mar 11, 2022 at 2:43 PM Xiang, Haihao > <haihao.xiang-at-intel.com@ffmpeg.org> wrote: > > > > On Fri, 2022-03-11 at 09:35 +0100, Hendrik Leppkes wrote: > > > On Fri, Mar 11, 2022 at 9:18 AM Xiang, Haihao > > > <haihao.xiang-at-intel.com@ffmpeg.org> wrote: > > > > diff --git a/libavutil/hwcontext_d3d11va.c > > > > b/libavutil/hwcontext_d3d11va.c > > > > index 8ab96bad25..e0e820f164 100644 > > > > --- a/libavutil/hwcontext_d3d11va.c > > > > +++ b/libavutil/hwcontext_d3d11va.c > > > > @@ -525,6 +525,13 @@ static void d3d11va_device_uninit(AVHWDeviceContext > > > > *hwdev) > > > > } > > > > } > > > > > > > > +static void d3d11va_device_free(AVHWDeviceContext *ctx) > > > > +{ > > > > + AVD3D11VADeviceContext *hwctx = ctx->hwctx; > > > > + > > > > + av_free(hwctx->device_name); > > > > +} > > > > + > > > > static int d3d11va_device_create(AVHWDeviceContext *ctx, const char > > > > *device, > > > > AVDictionary *opts, int flags) > > > > { > > > > @@ -537,6 +544,8 @@ static int d3d11va_device_create(AVHWDeviceContext > > > > *ctx, > > > > const char *device, > > > > int is_debug = !!av_dict_get(opts, "debug", NULL, 0); > > > > int ret; > > > > > > > > + ctx->free = d3d11va_device_free; > > > > + > > > > // (On UWP we can't check this.) > > > > #if !HAVE_UWP > > > > if (!LoadLibrary("d3d11_1sdklayers.dll")) > > > > @@ -561,6 +570,10 @@ static int d3d11va_device_create(AVHWDeviceContext > > > > *ctx, const char *device, > > > > if (FAILED(IDXGIFactory2_EnumAdapters(pDXGIFactory, > > > > adapter, > > > > &pAdapter))) > > > > pAdapter = NULL; > > > > IDXGIFactory2_Release(pDXGIFactory); > > > > + > > > > + device_hwctx->device_name = av_strdup(device); > > > > + if (!device_hwctx->device_name) > > > > + return AVERROR(ENOMEM); > > > > } > > > > } > > > > > > > > diff --git a/libavutil/hwcontext_d3d11va.h > > > > b/libavutil/hwcontext_d3d11va.h > > > > index 77d2d72f1b..41a315b9e6 100644 > > > > --- a/libavutil/hwcontext_d3d11va.h > > > > +++ b/libavutil/hwcontext_d3d11va.h > > > > @@ -94,6 +94,11 @@ typedef struct AVD3D11VADeviceContext { > > > > void (*lock)(void *lock_ctx); > > > > void (*unlock)(void *lock_ctx); > > > > void *lock_ctx; > > > > + > > > > + /** > > > > + * The string for the used adapter > > > > + */ > > > > + char *device_name; > > > > } AVD3D11VADeviceContext; > > > > > > > > /** > > > > diff --git a/libavutil/hwcontext_dxva2.c b/libavutil/hwcontext_dxva2.c > > > > index 53d00fa815..6967357093 100644 > > > > --- a/libavutil/hwcontext_dxva2.c > > > > +++ b/libavutil/hwcontext_dxva2.c > > > > @@ -431,6 +431,7 @@ static void dxva2_device_free(AVHWDeviceContext > > > > *ctx) > > > > dlclose(priv->dxva2lib); > > > > > > > > av_freep(&ctx->user_opaque); > > > > + av_free(hwctx->device_name); > > > > } > > > > > > > > static int dxva2_device_create9(AVHWDeviceContext *ctx, UINT adapter) > > > > @@ -571,6 +572,13 @@ static int dxva2_device_create(AVHWDeviceContext > > > > *ctx, > > > > const char *device, > > > > return AVERROR_UNKNOWN; > > > > } > > > > > > > > + if (device) { > > > > + hwctx->device_name = av_strdup(device); > > > > + > > > > + if (!hwctx->device_name) > > > > + return AVERROR(ENOMEM); > > > > + } > > > > + > > > > return 0; > > > > } > > > > > > > > diff --git a/libavutil/hwcontext_dxva2.h b/libavutil/hwcontext_dxva2.h > > > > index e1b79bc0de..253ddbed51 100644 > > > > --- a/libavutil/hwcontext_dxva2.h > > > > +++ b/libavutil/hwcontext_dxva2.h > > > > @@ -38,6 +38,10 @@ > > > > */ > > > > typedef struct AVDXVA2DeviceContext { > > > > IDirect3DDeviceManager9 *devmgr; > > > > + /** > > > > + * The string for the used adapter > > > > + */ > > > > + char *device_name; > > > > } AVDXVA2DeviceContext; > > > > > > > > /** > > > > > > Why are these device names required? I would think deriving a child > > > device would use the actual device, eg. ID3D11Device or > > > IDirect3DDeviceManager9 (and whatever for VAAPI), and not some string > > > (that may or may not even be set). > > > It feels quite a bit icky to store these in the context just for qsv > > > to do... what with? > > > > Yes, it is a little ugly here. MediaSDK or oneVPL application creates mfx > > session and the device (dxva2, d3d11va or vaapi), then pass this device to > > the > > SDK through MFXVideoCORE_SetHandle(). implementation is introduced in oneVPL > > ( > > https://spec.oneapi.io/versions/latest/elements/oneVPL/source/API_ref/VPL_disp_api_struct.html#structmfx_impl_description > > ) and user must select an available implementation before the creation of > > mfx > > session, however the device handle is unknown in the SDK when selecting an > > available implementation, the SDK provides a method to select implementation > > via > > the given adapter (on Windows) or DRI device node (on Linux). The default > > implementation will be selected if child device name is unknown. > > > > First of all, whoever made that API should get a stern message. > Expecting to properly interoperate with the dominant platform APIs > should be a primary goal, and it sounds like it was somehow shoehorned > in after the fact. > > For D3D11 for example, you can get the IDXGIAdapter a device was > created from, isn't there enough information in there to pass-on > without storing a string field? > IDXGIAdapter::GetDesc has tons of identification information to > identify the device in use. > > D3D9 probably has something similar, haven't checked right now. Thanks for the info, I may get AdapterLuid from the adapter description, however the required parameter in oneVPL is the index of the adapter, is there a way to map AdapterLuid to adapter index ? (Sorry for this dumb question) There is `IDirect3DDeviceManager9 *devmgr` only in AVDXVA2DeviceContext for D3D9, it seems we have to add other members to get adapter description. As for vaapi, there is no API to get the used DRI device from VADisplay handle, we have to store this info in AVVAAPIDeviceContext, and I prefer using the same way for d3d9 & d3d11va too. Note We may get the specified child device (or default device) in hwcontext_qsv.c if initializing qsv device directly, e.g. -init_hw_device qsv:hw_any,child_device=/dev/dri/renderD129, and needn't change AVD3D11VADeviceContext / AVDXVA2DeviceContext / AVVAAPIDeviceContext. This change is required if deriving QSV device from a child device, e.g. -init_hw_device vaapi=va:/dev/dri/renderD129 -init_hw_device qsv=hw@va Thanks Haihao _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [FFmpeg-devel] [PATCH v7 09/10] qsv: use a new method to create mfx session when using oneVPL 2022-03-15 7:24 ` Xiang, Haihao @ 2022-04-04 9:46 ` Hendrik Leppkes 2022-04-06 3:58 ` Xiang, Haihao 0 siblings, 1 reply; 26+ messages in thread From: Hendrik Leppkes @ 2022-04-04 9:46 UTC (permalink / raw) To: FFmpeg development discussions and patches On Tue, Mar 15, 2022 at 8:24 AM Xiang, Haihao <haihao.xiang-at-intel.com@ffmpeg.org> wrote: > > On Fri, 2022-03-11 at 15:00 +0100, Hendrik Leppkes wrote: > > On Fri, Mar 11, 2022 at 2:43 PM Xiang, Haihao > > <haihao.xiang-at-intel.com@ffmpeg.org> wrote: > > > > > > On Fri, 2022-03-11 at 09:35 +0100, Hendrik Leppkes wrote: > > > > On Fri, Mar 11, 2022 at 9:18 AM Xiang, Haihao > > > > <haihao.xiang-at-intel.com@ffmpeg.org> wrote: > > > > > diff --git a/libavutil/hwcontext_d3d11va.c > > > > > b/libavutil/hwcontext_d3d11va.c > > > > > index 8ab96bad25..e0e820f164 100644 > > > > > --- a/libavutil/hwcontext_d3d11va.c > > > > > +++ b/libavutil/hwcontext_d3d11va.c > > > > > @@ -525,6 +525,13 @@ static void d3d11va_device_uninit(AVHWDeviceContext > > > > > *hwdev) > > > > > } > > > > > } > > > > > > > > > > +static void d3d11va_device_free(AVHWDeviceContext *ctx) > > > > > +{ > > > > > + AVD3D11VADeviceContext *hwctx = ctx->hwctx; > > > > > + > > > > > + av_free(hwctx->device_name); > > > > > +} > > > > > + > > > > > static int d3d11va_device_create(AVHWDeviceContext *ctx, const char > > > > > *device, > > > > > AVDictionary *opts, int flags) > > > > > { > > > > > @@ -537,6 +544,8 @@ static int d3d11va_device_create(AVHWDeviceContext > > > > > *ctx, > > > > > const char *device, > > > > > int is_debug = !!av_dict_get(opts, "debug", NULL, 0); > > > > > int ret; > > > > > > > > > > + ctx->free = d3d11va_device_free; > > > > > + > > > > > // (On UWP we can't check this.) > > > > > #if !HAVE_UWP > > > > > if (!LoadLibrary("d3d11_1sdklayers.dll")) > > > > > @@ -561,6 +570,10 @@ static int d3d11va_device_create(AVHWDeviceContext > > > > > *ctx, const char *device, > > > > > if (FAILED(IDXGIFactory2_EnumAdapters(pDXGIFactory, > > > > > adapter, > > > > > &pAdapter))) > > > > > pAdapter = NULL; > > > > > IDXGIFactory2_Release(pDXGIFactory); > > > > > + > > > > > + device_hwctx->device_name = av_strdup(device); > > > > > + if (!device_hwctx->device_name) > > > > > + return AVERROR(ENOMEM); > > > > > } > > > > > } > > > > > > > > > > diff --git a/libavutil/hwcontext_d3d11va.h > > > > > b/libavutil/hwcontext_d3d11va.h > > > > > index 77d2d72f1b..41a315b9e6 100644 > > > > > --- a/libavutil/hwcontext_d3d11va.h > > > > > +++ b/libavutil/hwcontext_d3d11va.h > > > > > @@ -94,6 +94,11 @@ typedef struct AVD3D11VADeviceContext { > > > > > void (*lock)(void *lock_ctx); > > > > > void (*unlock)(void *lock_ctx); > > > > > void *lock_ctx; > > > > > + > > > > > + /** > > > > > + * The string for the used adapter > > > > > + */ > > > > > + char *device_name; > > > > > } AVD3D11VADeviceContext; > > > > > > > > > > /** > > > > > diff --git a/libavutil/hwcontext_dxva2.c b/libavutil/hwcontext_dxva2.c > > > > > index 53d00fa815..6967357093 100644 > > > > > --- a/libavutil/hwcontext_dxva2.c > > > > > +++ b/libavutil/hwcontext_dxva2.c > > > > > @@ -431,6 +431,7 @@ static void dxva2_device_free(AVHWDeviceContext > > > > > *ctx) > > > > > dlclose(priv->dxva2lib); > > > > > > > > > > av_freep(&ctx->user_opaque); > > > > > + av_free(hwctx->device_name); > > > > > } > > > > > > > > > > static int dxva2_device_create9(AVHWDeviceContext *ctx, UINT adapter) > > > > > @@ -571,6 +572,13 @@ static int dxva2_device_create(AVHWDeviceContext > > > > > *ctx, > > > > > const char *device, > > > > > return AVERROR_UNKNOWN; > > > > > } > > > > > > > > > > + if (device) { > > > > > + hwctx->device_name = av_strdup(device); > > > > > + > > > > > + if (!hwctx->device_name) > > > > > + return AVERROR(ENOMEM); > > > > > + } > > > > > + > > > > > return 0; > > > > > } > > > > > > > > > > diff --git a/libavutil/hwcontext_dxva2.h b/libavutil/hwcontext_dxva2.h > > > > > index e1b79bc0de..253ddbed51 100644 > > > > > --- a/libavutil/hwcontext_dxva2.h > > > > > +++ b/libavutil/hwcontext_dxva2.h > > > > > @@ -38,6 +38,10 @@ > > > > > */ > > > > > typedef struct AVDXVA2DeviceContext { > > > > > IDirect3DDeviceManager9 *devmgr; > > > > > + /** > > > > > + * The string for the used adapter > > > > > + */ > > > > > + char *device_name; > > > > > } AVDXVA2DeviceContext; > > > > > > > > > > /** > > > > > > > > Why are these device names required? I would think deriving a child > > > > device would use the actual device, eg. ID3D11Device or > > > > IDirect3DDeviceManager9 (and whatever for VAAPI), and not some string > > > > (that may or may not even be set). > > > > It feels quite a bit icky to store these in the context just for qsv > > > > to do... what with? > > > > > > Yes, it is a little ugly here. MediaSDK or oneVPL application creates mfx > > > session and the device (dxva2, d3d11va or vaapi), then pass this device to > > > the > > > SDK through MFXVideoCORE_SetHandle(). implementation is introduced in oneVPL > > > ( > > > > https://spec.oneapi.io/versions/latest/elements/oneVPL/source/API_ref/VPL_disp_api_struct.html#structmfx_impl_description > > > ) and user must select an available implementation before the creation of > > > mfx > > > session, however the device handle is unknown in the SDK when selecting an > > > available implementation, the SDK provides a method to select implementation > > > via > > > the given adapter (on Windows) or DRI device node (on Linux). The default > > > implementation will be selected if child device name is unknown. > > > > > > > First of all, whoever made that API should get a stern message. > > Expecting to properly interoperate with the dominant platform APIs > > should be a primary goal, and it sounds like it was somehow shoehorned > > in after the fact. > > > > For D3D11 for example, you can get the IDXGIAdapter a device was > > created from, isn't there enough information in there to pass-on > > without storing a string field? > > IDXGIAdapter::GetDesc has tons of identification information to > > identify the device in use. > > > > D3D9 probably has something similar, haven't checked right now. > > > Thanks for the info, I may get AdapterLuid from the adapter description, however > the required parameter in oneVPL is the index of the adapter, is there a way to > map AdapterLuid to adapter index ? (Sorry for this dumb question) > > There is `IDirect3DDeviceManager9 *devmgr` only in AVDXVA2DeviceContext for > D3D9, it seems we have to add other members to get adapter description. > > As for vaapi, there is no API to get the used DRI device from VADisplay handle, > we have to store this info in AVVAAPIDeviceContext, and I prefer using the same > way for d3d9 & d3d11va too. > I'm not sure about VAAPI, but not storing a string just for this one purpose seems like a win to me. What values would mfxImplDescription.mfxDeviceDescription.device.DeviceID contain, and would that perhaps match something in the D3D11 or DXVA2 device description? You already have big alternate pathes inside those functions for windows or vaapi, so that doesn't seem like a big step to handle separately. - Hendrik _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [FFmpeg-devel] [PATCH v7 09/10] qsv: use a new method to create mfx session when using oneVPL 2022-04-04 9:46 ` Hendrik Leppkes @ 2022-04-06 3:58 ` Xiang, Haihao 2022-04-28 9:31 ` Xiang, Haihao 0 siblings, 1 reply; 26+ messages in thread From: Xiang, Haihao @ 2022-04-06 3:58 UTC (permalink / raw) To: ffmpeg-devel On Mon, 2022-04-04 at 11:46 +0200, Hendrik Leppkes wrote: > On Tue, Mar 15, 2022 at 8:24 AM Xiang, Haihao > <haihao.xiang-at-intel.com@ffmpeg.org> wrote: > > > > On Fri, 2022-03-11 at 15:00 +0100, Hendrik Leppkes wrote: > > > On Fri, Mar 11, 2022 at 2:43 PM Xiang, Haihao > > > <haihao.xiang-at-intel.com@ffmpeg.org> wrote: > > > > > > > > On Fri, 2022-03-11 at 09:35 +0100, Hendrik Leppkes wrote: > > > > > On Fri, Mar 11, 2022 at 9:18 AM Xiang, Haihao > > > > > <haihao.xiang-at-intel.com@ffmpeg.org> wrote: > > > > > > diff --git a/libavutil/hwcontext_d3d11va.c > > > > > > b/libavutil/hwcontext_d3d11va.c > > > > > > index 8ab96bad25..e0e820f164 100644 > > > > > > --- a/libavutil/hwcontext_d3d11va.c > > > > > > +++ b/libavutil/hwcontext_d3d11va.c > > > > > > @@ -525,6 +525,13 @@ static void > > > > > > d3d11va_device_uninit(AVHWDeviceContext > > > > > > *hwdev) > > > > > > } > > > > > > } > > > > > > > > > > > > +static void d3d11va_device_free(AVHWDeviceContext *ctx) > > > > > > +{ > > > > > > + AVD3D11VADeviceContext *hwctx = ctx->hwctx; > > > > > > + > > > > > > + av_free(hwctx->device_name); > > > > > > +} > > > > > > + > > > > > > static int d3d11va_device_create(AVHWDeviceContext *ctx, const char > > > > > > *device, > > > > > > AVDictionary *opts, int flags) > > > > > > { > > > > > > @@ -537,6 +544,8 @@ static int > > > > > > d3d11va_device_create(AVHWDeviceContext > > > > > > *ctx, > > > > > > const char *device, > > > > > > int is_debug = !!av_dict_get(opts, "debug", NULL, 0); > > > > > > int ret; > > > > > > > > > > > > + ctx->free = d3d11va_device_free; > > > > > > + > > > > > > // (On UWP we can't check this.) > > > > > > #if !HAVE_UWP > > > > > > if (!LoadLibrary("d3d11_1sdklayers.dll")) > > > > > > @@ -561,6 +570,10 @@ static int > > > > > > d3d11va_device_create(AVHWDeviceContext > > > > > > *ctx, const char *device, > > > > > > if (FAILED(IDXGIFactory2_EnumAdapters(pDXGIFactory, > > > > > > adapter, > > > > > > &pAdapter))) > > > > > > pAdapter = NULL; > > > > > > IDXGIFactory2_Release(pDXGIFactory); > > > > > > + > > > > > > + device_hwctx->device_name = av_strdup(device); > > > > > > + if (!device_hwctx->device_name) > > > > > > + return AVERROR(ENOMEM); > > > > > > } > > > > > > } > > > > > > > > > > > > diff --git a/libavutil/hwcontext_d3d11va.h > > > > > > b/libavutil/hwcontext_d3d11va.h > > > > > > index 77d2d72f1b..41a315b9e6 100644 > > > > > > --- a/libavutil/hwcontext_d3d11va.h > > > > > > +++ b/libavutil/hwcontext_d3d11va.h > > > > > > @@ -94,6 +94,11 @@ typedef struct AVD3D11VADeviceContext { > > > > > > void (*lock)(void *lock_ctx); > > > > > > void (*unlock)(void *lock_ctx); > > > > > > void *lock_ctx; > > > > > > + > > > > > > + /** > > > > > > + * The string for the used adapter > > > > > > + */ > > > > > > + char *device_name; > > > > > > } AVD3D11VADeviceContext; > > > > > > > > > > > > /** > > > > > > diff --git a/libavutil/hwcontext_dxva2.c > > > > > > b/libavutil/hwcontext_dxva2.c > > > > > > index 53d00fa815..6967357093 100644 > > > > > > --- a/libavutil/hwcontext_dxva2.c > > > > > > +++ b/libavutil/hwcontext_dxva2.c > > > > > > @@ -431,6 +431,7 @@ static void dxva2_device_free(AVHWDeviceContext > > > > > > *ctx) > > > > > > dlclose(priv->dxva2lib); > > > > > > > > > > > > av_freep(&ctx->user_opaque); > > > > > > + av_free(hwctx->device_name); > > > > > > } > > > > > > > > > > > > static int dxva2_device_create9(AVHWDeviceContext *ctx, UINT > > > > > > adapter) > > > > > > @@ -571,6 +572,13 @@ static int > > > > > > dxva2_device_create(AVHWDeviceContext > > > > > > *ctx, > > > > > > const char *device, > > > > > > return AVERROR_UNKNOWN; > > > > > > } > > > > > > > > > > > > + if (device) { > > > > > > + hwctx->device_name = av_strdup(device); > > > > > > + > > > > > > + if (!hwctx->device_name) > > > > > > + return AVERROR(ENOMEM); > > > > > > + } > > > > > > + > > > > > > return 0; > > > > > > } > > > > > > > > > > > > diff --git a/libavutil/hwcontext_dxva2.h > > > > > > b/libavutil/hwcontext_dxva2.h > > > > > > index e1b79bc0de..253ddbed51 100644 > > > > > > --- a/libavutil/hwcontext_dxva2.h > > > > > > +++ b/libavutil/hwcontext_dxva2.h > > > > > > @@ -38,6 +38,10 @@ > > > > > > */ > > > > > > typedef struct AVDXVA2DeviceContext { > > > > > > IDirect3DDeviceManager9 *devmgr; > > > > > > + /** > > > > > > + * The string for the used adapter > > > > > > + */ > > > > > > + char *device_name; > > > > > > } AVDXVA2DeviceContext; > > > > > > > > > > > > /** > > > > > > > > > > Why are these device names required? I would think deriving a child > > > > > device would use the actual device, eg. ID3D11Device or > > > > > IDirect3DDeviceManager9 (and whatever for VAAPI), and not some string > > > > > (that may or may not even be set). > > > > > It feels quite a bit icky to store these in the context just for qsv > > > > > to do... what with? > > > > > > > > Yes, it is a little ugly here. MediaSDK or oneVPL application creates > > > > mfx > > > > session and the device (dxva2, d3d11va or vaapi), then pass this device > > > > to > > > > the > > > > SDK through MFXVideoCORE_SetHandle(). implementation is introduced in > > > > oneVPL > > > > ( > > > > > > > > https://spec.oneapi.io/versions/latest/elements/oneVPL/source/API_ref/VPL_disp_api_struct.html#structmfx_impl_description > > > > ) and user must select an available implementation before the creation > > > > of > > > > mfx > > > > session, however the device handle is unknown in the SDK when selecting > > > > an > > > > available implementation, the SDK provides a method to select > > > > implementation > > > > via > > > > the given adapter (on Windows) or DRI device node (on Linux). The > > > > default > > > > implementation will be selected if child device name is unknown. > > > > > > > > > > First of all, whoever made that API should get a stern message. > > > Expecting to properly interoperate with the dominant platform APIs > > > should be a primary goal, and it sounds like it was somehow shoehorned > > > in after the fact. > > > > > > For D3D11 for example, you can get the IDXGIAdapter a device was > > > created from, isn't there enough information in there to pass-on > > > without storing a string field? > > > IDXGIAdapter::GetDesc has tons of identification information to > > > identify the device in use. > > > > > > D3D9 probably has something similar, haven't checked right now. > > > > > > Thanks for the info, I may get AdapterLuid from the adapter description, > > however > > the required parameter in oneVPL is the index of the adapter, is there a way > > to > > map AdapterLuid to adapter index ? (Sorry for this dumb question) > > > > There is `IDirect3DDeviceManager9 *devmgr` only in AVDXVA2DeviceContext for > > D3D9, it seems we have to add other members to get adapter description. > > > > As for vaapi, there is no API to get the used DRI device from VADisplay > > handle, > > we have to store this info in AVVAAPIDeviceContext, and I prefer using the > > same > > way for d3d9 & d3d11va too. > > > > I'm not sure about VAAPI, but not storing a string just for this one > purpose seems like a win to me. > What values would > mfxImplDescription.mfxDeviceDescription.device.DeviceID contain, and > would that perhaps match something in the D3D11 or DXVA2 device > description? > > You already have big alternate pathes inside those functions for > windows or vaapi, so that doesn't seem like a big step to handle > separately. Thanks, I will try other members in mfxImplDescription and update you whether it is doable. BRs Haihao _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [FFmpeg-devel] [PATCH v7 09/10] qsv: use a new method to create mfx session when using oneVPL 2022-04-06 3:58 ` Xiang, Haihao @ 2022-04-28 9:31 ` Xiang, Haihao 0 siblings, 0 replies; 26+ messages in thread From: Xiang, Haihao @ 2022-04-28 9:31 UTC (permalink / raw) To: ffmpeg-devel On Wed, 2022-04-06 at 03:58 +0000, Xiang, Haihao wrote: [...] > > > > > > > > https://spec.oneapi.io/versions/latest/elements/oneVPL/source/API_ref/VPL_disp_api_struct.html#structmfx_impl_description > > > > > ) and user must select an available implementation before the creation > > > > > of > > > > > mfx > > > > > session, however the device handle is unknown in the SDK when > > > > > selecting > > > > > an > > > > > available implementation, the SDK provides a method to select > > > > > implementation > > > > > via > > > > > the given adapter (on Windows) or DRI device node (on Linux). The > > > > > default > > > > > implementation will be selected if child device name is unknown. > > > > > > > > > > > > > First of all, whoever made that API should get a stern message. > > > > Expecting to properly interoperate with the dominant platform APIs > > > > should be a primary goal, and it sounds like it was somehow shoehorned > > > > in after the fact. > > > > > > > > For D3D11 for example, you can get the IDXGIAdapter a device was > > > > created from, isn't there enough information in there to pass-on > > > > without storing a string field? > > > > IDXGIAdapter::GetDesc has tons of identification information to > > > > identify the device in use. > > > > > > > > D3D9 probably has something similar, haven't checked right now. > > > > > > > > > Thanks for the info, I may get AdapterLuid from the adapter description, > > > however > > > the required parameter in oneVPL is the index of the adapter, is there a > > > way > > > to > > > map AdapterLuid to adapter index ? (Sorry for this dumb question) > > > > > > There is `IDirect3DDeviceManager9 *devmgr` only in AVDXVA2DeviceContext > > > for > > > D3D9, it seems we have to add other members to get adapter description. > > > > > > As for vaapi, there is no API to get the used DRI device from VADisplay > > > handle, > > > we have to store this info in AVVAAPIDeviceContext, and I prefer using the > > > same > > > way for d3d9 & d3d11va too. > > > > > > > I'm not sure about VAAPI, but not storing a string just for this one > > purpose seems like a win to me. > > What values would > > mfxImplDescription.mfxDeviceDescription.device.DeviceID contain, and > > would that perhaps match something in the D3D11 or DXVA2 device > > description? > > > > You already have big alternate pathes inside those functions for > > windows or vaapi, so that doesn't seem like a big step to handle > > separately. > > Thanks, I will try other members in mfxImplDescription and update you whether > it > is doable. We updated the patch not to change AVD3D11VADeviceContext and AVDXVA2DeviceContext. Could you take a look when you have time ? https://ffmpeg.org/pipermail/ffmpeg-devel/2022-April/295760.html Thanks Haihao _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 26+ messages in thread
* [FFmpeg-devel] [PATCH v7 10/10] configure: add --enable-libvpl option 2022-03-11 8:16 [FFmpeg-devel] [PATCH v7 00/10] make QSV works with the Intel's oneVPL Xiang, Haihao ` (8 preceding siblings ...) 2022-03-11 8:16 ` [FFmpeg-devel] [PATCH v7 09/10] qsv: use a new method to create mfx session when using oneVPL Xiang, Haihao @ 2022-03-11 8:16 ` Xiang, Haihao 9 siblings, 0 replies; 26+ messages in thread From: Xiang, Haihao @ 2022-03-11 8:16 UTC (permalink / raw) To: ffmpeg-devel; +Cc: Haihao Xiang From: Haihao Xiang <haihao.xiang@intel.com> This allows user to build FFmpeg against Intel oneVPL. oneVPL 2.6 is the required minimum version when building Intel oneVPL code. It will fail to run configure script if both libmfx and libvpl are enabled. It is recommended to use oneVPL for new work, even for currently available hardwares [1] Note the preferred child device type is d3d11va for libvpl on Windows. The commands below will use d3d11va if d3d11va is available on Windows. $ ffmpeg -hwaccel qsv -c:v h264_qsv ... $ ffmpeg -qsv_device 0 -hwaccel qsv -c:v h264_qsv ... $ ffmpeg -init_hw_device qsv=qsv:hw_any -hwaccel qsv -c:v h264_qsv ... $ ffmpeg -init_hw_device qsv=qsv:hw_any,child_device=0 -hwaccel qsv -c:v h264_qsv ... User may use child_device_type option to specify child device type to dxva2 or derive a qsv device from a dxva2 device $ ffmpeg -init_hw_device qsv=qsv:hw_any,child_device=0,child_device_type=dxva2 -hwaccel qsv -c:v h264_qsv ... $ ffmpeg -init_hw_device dxva2=d3d9:0 -init_hw_device qsv=qsv@d3d9 -hwaccel qsv -c:v h264_qsv ... [1] https://www.intel.com/content/www/us/en/develop/documentation/upgrading-from-msdk-to-onevpl/top.html --- configure | 27 +++++++++++++++++++++------ 1 file changed, 21 insertions(+), 6 deletions(-) diff --git a/configure b/configure index eacdf03c18..aea3ddda05 100755 --- a/configure +++ b/configure @@ -339,6 +339,7 @@ External library support: --disable-ffnvcodec disable dynamically linked Nvidia code [autodetect] --enable-libdrm enable DRM code (Linux) [no] --enable-libmfx enable Intel MediaSDK (AKA Quick Sync Video) code via libmfx [no] + --enable-libvpl enable Intel oneVPL code via libvpl if libmfx is not used [no] --enable-libnpp enable Nvidia Performance Primitives-based code [no] --enable-mmal enable Broadcom Multi-Media Abstraction Layer (Raspberry Pi) via MMAL [no] --disable-nvdec disable Nvidia video decoding acceleration (via hwaccel) [autodetect] @@ -1917,6 +1918,7 @@ HWACCEL_LIBRARY_NONFREE_LIST=" HWACCEL_LIBRARY_LIST=" $HWACCEL_LIBRARY_NONFREE_LIST libmfx + libvpl mmal omx opencl @@ -6542,22 +6544,35 @@ enabled libilbc && require libilbc ilbc.h WebRtcIlbcfix_InitDecode -li enabled libklvanc && require libklvanc libklvanc/vanc.h klvanc_context_create -lklvanc enabled libkvazaar && require_pkg_config libkvazaar "kvazaar >= 0.8.1" kvazaar.h kvz_api_get enabled liblensfun && require_pkg_config liblensfun lensfun lensfun.h lf_db_new + +if enabled libmfx && enabled libvpl; then + die "ERROR: can not use libmfx and libvpl together" # While it may appear that require is being used as a pkg-config # fallback for libmfx, it is actually being used to detect a different # installation route altogether. If libmfx is installed via the Intel # Media SDK or Intel Media Server Studio, these don't come with # pkg-config support. Instead, users should make sure that the build # can find the libraries and headers through other means. - -enabled libmfx && { { check_pkg_config libmfx "libmfx < 2.0" "mfxvideo.h" MFXInit || +elif enabled libmfx; then + { check_pkg_config libmfx "libmfx < 2.0" "mfxvideo.h" MFXInit || \ # Some old versions of libmfx have the following settings in libmfx.pc: # includedir=/usr/include # Cflags: -I${includedir} # So add -I${includedir}/mfx to CFLAGS - { check_pkg_config libmfx "libmfx < 2.0" "mfx/mfxvideo.h" MFXInit && add_cflags -I${libmfx_incdir}/mfx; } || - { require "libmfx < 2.0" "mfxvideo.h" MFXInit "-llibmfx $advapi32_extralibs" && warn "using libmfx without pkg-config"; } } && - warn "build FFmpeg against libmfx 1.x, obsolete features of libmfx such as OPAQUE memory,\n"\ - "multi-frame encode, user plugins and LA_EXT rate control mode are enabled"; } + { check_pkg_config libmfx "libmfx < 2.0" "mfx/mfxvideo.h" MFXInit && add_cflags -I${libmfx_incdir}/mfx; } || + { require "libmfx < 2.0" "mfxvideo.h" MFXInit "-llibmfx $advapi32_extralibs" && warn "using libmfx without pkg-config"; } } && + warn "build FFmpeg against libmfx 1.x, obsolete features of libmfx such as OPAQUE memory,\n"\ + "multi-frame encode, user plugins and LA_EXT rate control mode are enabled" +elif enabled libvpl; then +# Consider pkg-config only. The name of libmfx is still passed to check_pkg_config function for --enable-libvpl option +# because QSV has dependency on libmfx, we can use the same dependency if using libmfx in this check. The package name +# is extracted from "vpl >= 2.6" + check_pkg_config libmfx "vpl >= 2.6" "mfxvideo.h mfxdispatcher.h" MFXLoad && \ + warn "build FFmpeg against oneVPL 2.6+, OPAQUE memory, multi-frame encode, user plugins\n"\ + "and LA_EXT rate control mode in FFmpeg QSV won't be supported." || + die "ERROR: libvpl >= 2.6 not found" +fi + if enabled libmfx; then check_cc MFX_CODEC_VP9 "mfxdefs.h mfxstructures.h" "MFX_CODEC_VP9" fi -- 2.17.1 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 26+ messages in thread
end of thread, other threads:[~2022-05-31 8:59 UTC | newest] Thread overview: 26+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2022-03-11 8:16 [FFmpeg-devel] [PATCH v7 00/10] make QSV works with the Intel's oneVPL Xiang, Haihao 2022-03-11 8:16 ` [FFmpeg-devel] [PATCH v7 01/10] configure: ensure --enable-libmfx uses libmfx 1.x Xiang, Haihao 2022-03-11 8:16 ` [FFmpeg-devel] [PATCH v7 02/10] configure: fix the check for MFX_CODEC_VP9 Xiang, Haihao 2022-03-11 8:16 ` [FFmpeg-devel] [PATCH v7 03/10] qsv: remove mfx/ prefix from mfx headers Xiang, Haihao 2022-03-11 8:16 ` [FFmpeg-devel] [PATCH v7 04/10] qsv: load user plugin for MFX_VERSION < 2.0 Xiang, Haihao 2022-03-11 8:16 ` [FFmpeg-devel] [PATCH v7 05/10] qsv: build audio related code when " Xiang, Haihao 2022-04-05 11:50 ` Anton Khirnov 2022-04-06 3:38 ` Xiang, Haihao 2022-04-30 16:51 ` Soft Works 2022-05-01 2:16 ` Xiang, Haihao 2022-05-01 3:10 ` Soft Works 2022-05-01 4:51 ` Xiang, Haihao 2022-05-01 5:13 ` Soft Works 2022-05-31 8:58 ` Xiang, Haihao 2022-03-11 8:16 ` [FFmpeg-devel] [PATCH v7 06/10] qsvenc: support multi-frame encode " Xiang, Haihao 2022-03-11 8:16 ` [FFmpeg-devel] [PATCH v7 07/10] qsvenc: support MFX_RATECONTROL_LA_EXT " Xiang, Haihao 2022-03-11 8:16 ` [FFmpeg-devel] [PATCH v7 08/10] qsv: support OPAQUE memory " Xiang, Haihao 2022-03-11 8:16 ` [FFmpeg-devel] [PATCH v7 09/10] qsv: use a new method to create mfx session when using oneVPL Xiang, Haihao 2022-03-11 8:35 ` Hendrik Leppkes 2022-03-11 13:43 ` Xiang, Haihao 2022-03-11 14:00 ` Hendrik Leppkes 2022-03-15 7:24 ` Xiang, Haihao 2022-04-04 9:46 ` Hendrik Leppkes 2022-04-06 3:58 ` Xiang, Haihao 2022-04-28 9:31 ` Xiang, Haihao 2022-03-11 8:16 ` [FFmpeg-devel] [PATCH v7 10/10] configure: add --enable-libvpl option Xiang, Haihao
Git Inbox Mirror of the ffmpeg-devel mailing list - see https://ffmpeg.org/mailman/listinfo/ffmpeg-devel This inbox may be cloned and mirrored by anyone: git clone --mirror https://master.gitmailbox.com/ffmpegdev/0 ffmpegdev/git/0.git # If you have public-inbox 1.1+ installed, you may # initialize and index your mirror using the following commands: public-inbox-init -V2 ffmpegdev ffmpegdev/ https://master.gitmailbox.com/ffmpegdev \ ffmpegdev@gitmailbox.com public-inbox-index ffmpegdev Example config snippet for mirrors. AGPL code for this site: git clone https://public-inbox.org/public-inbox.git