From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.ffmpeg.org (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTPS id 1C2944C4D0 for ; Sat, 6 Sep 2025 22:27:59 +0000 (UTC) Authentication-Results: ffbox; dkim=fail (body hash mismatch (got b'vnHZNSfh+mQTMImb9pDnwgKP+mtAgK+kj8lcqlwlN9s=', expected b'rJSULS2GtyNjKZmLF362b9Yamhb4FDPVvT/Gy5VXlzQ=')) header.d=ffmpeg.org header.i=@ffmpeg.org header.a=rsa-sha256 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ffmpeg.org; i=@ffmpeg.org; q=dns/txt; s=mail; t=1757197654; h=mime-version : to : date : message-id : reply-to : subject : list-id : list-archive : list-archive : list-help : list-owner : list-post : list-subscribe : list-unsubscribe : from : cc : content-type : content-transfer-encoding : from; bh=vnHZNSfh+mQTMImb9pDnwgKP+mtAgK+kj8lcqlwlN9s=; b=qMC2b1uCZ0GPpHffhaPNFsylHo3yDp5OR4ywMptelsxpDe85I7WHKaHMlM6CH6zyNGyz/ KRktg3Z3Bf775heGUZn0biYTE9u/2nY7RxLh3Y8Cah2iS3AZH3lnpFrKLhegwUKEsowo6AH swxEaWEG1WMJBQ6PoPDnreGRZbxWZ8xc63rt3hzrDASdh3MjI6/aiTo6se7QZcid4MuW/gx mR3vQYe4V2f1/0r3X7DP4hvhxIskaWIAOIAz/gQEg2uEXjzrVCvB+rodKFlVDlLcfwCSNnc IgbikOT679YaIurPCrzaiGRj4w0FQKaDyBVZtXUlH6PVDhwfX/ihQ9/TQAiQ== Received: from [172.19.0.4] (unknown [172.19.0.4]) by ffbox0-bg.ffmpeg.org (Postfix) with ESMTP id 559C068DA91; Sun, 7 Sep 2025 01:27:34 +0300 (EEST) ARC-Seal: i=1; cv=none; a=rsa-sha256; d=ffmpeg.org; s=arc; t=1757197650; b=RDCRk8ARwaQSMS404/v3mSV61yo2h4eisJKO6SoBzmfMe5pRAo9t7uuV6EvJLk9H/bRQY Tu0fj2qdpWEezbyxLlyZ777NLPmSJgCec4LGZ0+VkgXV3la6IiOJ6AXSyDOB5hYHQ8/20oN YM7wy6gyFL6WSggAuqEWD1a5+Z14zwtPKoFPZnQp02/iXcjJoCGq22gUQRE+nsk/VduP1AF 1CR5HziDxB8ugIaxDf2mBnVGHLSdGRrV9K5s8n7UplR4xGC0mZm3udkvfQ+vlojQmDc2Kei l55MG0JmnhiJpvvrMGFMpHCzDFl93wcbibwsxs49BkZbhpUMYIHqcuhrFSBQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=ffmpeg.org; s=arc; t=1757197650; h=from : sender : reply-to : subject : date : message-id : to : cc : mime-version : content-type : content-transfer-encoding : content-id : content-description : resent-date : resent-from : resent-sender : resent-to : resent-cc : resent-message-id : in-reply-to : references : list-id : list-help : list-unsubscribe : list-subscribe : list-post : list-owner : list-archive; bh=sF2R+S5D4rJuPI0G/SmrLk8YxYhvnhCMHElnYgmYweQ=; b=qjlZTV+bm8Xy9V90pYij64apPZeY7eCKI1Ml4FVdatZX+26HJ79QQAM6lbznkxKu7YWar lDpInz9/3270mtAZbdDlFgYb5lh8lfY83UVaxAG5tsUrS+klYHJmAMy2L0TaOGUNgUsP5VU 1HCYdCo2M7HGTkoqFwQTUZLEI5MSTtTu/lb6PoPqnKz4F4Gd82JwUeM6pbcxobbbbcp2DDY sxJPLnZZHz6Vn9fDvOeg5F1Flis1ehpFErmEYrLSOkaE0aL3NybomjA7hirJ2POLScDKTNc 8DxxM5GHi8xxPOjkMZa9ZQOFjP/2Y3Kq6SiqEygdoz/tsTzqQjSLerjIhCyA== ARC-Authentication-Results: i=1; ffmpeg.org; dkim=pass header.d=ffmpeg.org header.i=@ffmpeg.org; arc=none; dmarc=none Authentication-Results: ffmpeg.org; dkim=pass header.d=ffmpeg.org header.i=@ffmpeg.org; arc=none (Message is not ARC signed); dmarc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ffmpeg.org; i=@ffmpeg.org; q=dns/txt; s=mail; t=1757197633; h=content-type : mime-version : content-transfer-encoding : from : to : reply-to : subject : date : from; bh=rJSULS2GtyNjKZmLF362b9Yamhb4FDPVvT/Gy5VXlzQ=; b=0IeW9JSWcfg7m/nX8LeurGcS9najMCv7mMe05HXvR5455RFiDGplfBFQsGaLb3cu+XGlK MBGRn0R6rReTntVF/o5oGjoTfzK6lkA+ZdqCL5RS7t/4YWt1YUax9KBm1f9GJtxSrAMz22h 9x0FJZol7cITCkq/pRHoZ1+QnxT/NIElyWDU5/YyI/e3g6E9DSRgJazR2/S0gflxrSXGRDD DrJ4+adwcmgsdaquT6dKr55sUOIHKImYOLqnJmzYMKmyE7/A2rDGZN8MSY75RIj1NLiA6+r vvbqK+rrjKm6p3FLlpoC8dWlSkRLE4AKWEdD1pSvxg5TC2LnOrzJ+Z7SWGiw== Received: from 3f9d35a0eedc (code.ffmpeg.org [188.245.149.3]) by ffbox0-bg.ffmpeg.org (Postfix) with ESMTPS id AC9E968B606 for ; Sun, 7 Sep 2025 01:27:13 +0300 (EEST) MIME-Version: 1.0 To: ffmpeg-devel@ffmpeg.org Date: Sat, 06 Sep 2025 22:27:13 -0000 Message-ID: <175719763446.25.1948704460278222116@463a07221176> Message-ID-Hash: QUDRHGIZHGBB2AGWTPN24COGLSRXNCX4 X-Message-ID-Hash: QUDRHGIZHGBB2AGWTPN24COGLSRXNCX4 X-MailFrom: code@ffmpeg.org X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; header-match-ffmpeg-devel.ffmpeg.org-0; header-match-ffmpeg-devel.ffmpeg.org-1; header-match-ffmpeg-devel.ffmpeg.org-2; header-match-ffmpeg-devel.ffmpeg.org-3; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.10 Precedence: list Reply-To: FFmpeg development discussions and patches Subject: [FFmpeg-devel] [PATCH] Add Windows.Graphics.Capture based video source filter (PR #20455) List-Id: FFmpeg development discussions and patches Archived-At: Archived-At: List-Archive: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: From: Timo Rothenpieler via ffmpeg-devel Cc: Timo Rothenpieler Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Archived-At: List-Archive: List-Post: PR #20455 opened by Timo Rothenpieler (BtbN) URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/20455 Patch URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/20455.patch Captures windows and monitors with the respective WinRt API. The filter itself is written in C++, since interacting with WinRt without it is incredibly painful and would inflate the source by a lot. To achieve that, the C++ infrastructure in configure/Makefiles needed a bit of work, since it so far was only used for decklinks, which seemingly barely held together. >>From 058c05d8588ce29226750624eefb9bfeaac62ed4 Mon Sep 17 00:00:00 2001 From: Timo Rothenpieler Date: Wed, 3 Sep 2025 01:34:57 +0200 Subject: [PATCH 1/5] build: link with CXX when -lstdc++ on linker commandline --- Makefile | 20 ++++++++++---------- ffbuild/common.mak | 11 ++++++++++- ffbuild/library.mak | 10 +++++++--- tests/api/Makefile | 2 +- tests/checkasm/Makefile | 2 +- 5 files changed, 29 insertions(+), 16 deletions(-) diff --git a/Makefile b/Makefile index 877b0071f6..17fadd58c2 100644 --- a/Makefile +++ b/Makefile @@ -53,31 +53,31 @@ FF_DEP_LIBS := $(DEP_LIBS) FF_STATIC_DEP_LIBS := $(STATIC_DEP_LIBS) $(TOOLS): %$(EXESUF): %.o - $(LD) $(LDFLAGS) $(LDEXEFLAGS) $(LD_O) $(filter-out $(FF_DEP_LIBS), $^) $(EXTRALIBS-$(*F)) $(EXTRALIBS) $(ELIBS) + $(call LINK,$(LDFLAGS) $(LDEXEFLAGS) $(LD_O) $(filter-out $(FF_DEP_LIBS), $^) $(EXTRALIBS-$(*F)) $(EXTRALIBS) $(ELIBS)) target_dec_%_fuzzer$(EXESUF): target_dec_%_fuzzer.o $(FF_DEP_LIBS) - $(LD) $(LDFLAGS) $(LDEXEFLAGS) $(LD_O) $^ $(ELIBS) $(FF_EXTRALIBS) $(LIBFUZZER_PATH) + $(call LINK,$(LDFLAGS) $(LDEXEFLAGS) $(LD_O) $^ $(ELIBS) $(FF_EXTRALIBS) $(LIBFUZZER_PATH)) target_enc_%_fuzzer$(EXESUF): target_enc_%_fuzzer.o $(FF_DEP_LIBS) - $(LD) $(LDFLAGS) $(LDEXEFLAGS) $(LD_O) $^ $(ELIBS) $(FF_EXTRALIBS) $(LIBFUZZER_PATH) + $(call LINK,$(LDFLAGS) $(LDEXEFLAGS) $(LD_O) $^ $(ELIBS) $(FF_EXTRALIBS) $(LIBFUZZER_PATH)) tools/target_bsf_%_fuzzer$(EXESUF): tools/target_bsf_%_fuzzer.o $(FF_DEP_LIBS) - $(LD) $(LDFLAGS) $(LDEXEFLAGS) $(LD_O) $^ $(ELIBS) $(FF_EXTRALIBS) $(LIBFUZZER_PATH) + $(call LINK,$(LDFLAGS) $(LDEXEFLAGS) $(LD_O) $^ $(ELIBS) $(FF_EXTRALIBS) $(LIBFUZZER_PATH)) target_dem_%_fuzzer$(EXESUF): target_dem_%_fuzzer.o $(FF_DEP_LIBS) - $(LD) $(LDFLAGS) $(LDEXEFLAGS) $(LD_O) $^ $(ELIBS) $(FF_EXTRALIBS) $(LIBFUZZER_PATH) + $(call LINK,$(LDFLAGS) $(LDEXEFLAGS) $(LD_O) $^ $(ELIBS) $(FF_EXTRALIBS) $(LIBFUZZER_PATH)) tools/target_dem_fuzzer$(EXESUF): tools/target_dem_fuzzer.o $(FF_DEP_LIBS) - $(LD) $(LDFLAGS) $(LDEXEFLAGS) $(LD_O) $^ $(ELIBS) $(FF_EXTRALIBS) $(LIBFUZZER_PATH) + $(call LINK,$(LDFLAGS) $(LDEXEFLAGS) $(LD_O) $^ $(ELIBS) $(FF_EXTRALIBS) $(LIBFUZZER_PATH)) tools/target_io_dem_fuzzer$(EXESUF): tools/target_io_dem_fuzzer.o $(FF_DEP_LIBS) - $(LD) $(LDFLAGS) $(LDEXEFLAGS) $(LD_O) $^ $(ELIBS) $(FF_EXTRALIBS) $(LIBFUZZER_PATH) + $(call LINK,$(LDFLAGS) $(LDEXEFLAGS) $(LD_O) $^ $(ELIBS) $(FF_EXTRALIBS) $(LIBFUZZER_PATH)) tools/target_sws_fuzzer$(EXESUF): tools/target_sws_fuzzer.o $(FF_DEP_LIBS) - $(LD) $(LDFLAGS) $(LDEXEFLAGS) $(LD_O) $^ $(ELIBS) $(FF_EXTRALIBS) $(LIBFUZZER_PATH) + $(call LINK,$(LDFLAGS) $(LDEXEFLAGS) $(LD_O) $^ $(ELIBS) $(FF_EXTRALIBS) $(LIBFUZZER_PATH)) tools/target_swr_fuzzer$(EXESUF): tools/target_swr_fuzzer.o $(FF_DEP_LIBS) - $(LD) $(LDFLAGS) $(LDEXEFLAGS) $(LD_O) $^ $(ELIBS) $(FF_EXTRALIBS) $(LIBFUZZER_PATH) + $(call LINK,$(LDFLAGS) $(LDEXEFLAGS) $(LD_O) $^ $(ELIBS) $(FF_EXTRALIBS) $(LIBFUZZER_PATH)) tools/enum_options$(EXESUF): ELIBS = $(FF_EXTRALIBS) tools/enum_options$(EXESUF): $(FF_DEP_LIBS) @@ -144,7 +144,7 @@ else endif %$(PROGSSUF)_g$(EXESUF): $(FF_DEP_LIBS) - $(LD) $(LDFLAGS) $(LDEXEFLAGS) $(LD_O) $(OBJS-$*) $(FF_EXTRALIBS) + $(call LINK,$(LDFLAGS) $(LDEXEFLAGS) $(LD_O) $(OBJS-$*) $(FF_EXTRALIBS)) VERSION_SH = $(SRC_PATH)/ffbuild/version.sh ifeq ($(VERSION_TRACKING),yes) diff --git a/ffbuild/common.mak b/ffbuild/common.mak index 0a60d01623..0dcf38bcfc 100644 --- a/ffbuild/common.mak +++ b/ffbuild/common.mak @@ -12,13 +12,22 @@ endif ifndef SUBDIR +LINK = $(LD) $(1) + +ifeq ($(LD),$(CC)) +ifneq ($(CXX),) +LDXX := $(CXX) +LINK = $(if $(filter -lstdc++,$(1)),$(LDXX) $(filter-out -lstdc++,$(1)),$(LD) $(1)) +endif +endif + BIN2CEXE = ffbuild/bin2c$(HOSTEXESUF) BIN2C = $(BIN2CEXE) ifndef V Q = @ ECHO = printf "$(1)\t%s\n" $(2) -BRIEF = CC CXX OBJCC HOSTCC HOSTLD AS X86ASM AR LD STRIP CP WINDRES NVCC BIN2C METALCC METALLIB +BRIEF = CC CXX OBJCC HOSTCC HOSTLD AS X86ASM AR LD LDXX STRIP CP WINDRES NVCC BIN2C METALCC METALLIB SILENT = DEPCC DEPHOSTCC DEPAS DEPX86ASM RANLIB RM MSG = $@ diff --git a/ffbuild/library.mak b/ffbuild/library.mak index dee05c5acd..91daa9c25f 100644 --- a/ffbuild/library.mak +++ b/ffbuild/library.mak @@ -55,8 +55,12 @@ $(TESTPROGS): THISLIB = $(SUBDIR)$(LIBNAME) $(LIBOBJS): CPPFLAGS += -DBUILDING_$(NAME) +$(NAME)LINK_EXE_ARGS = $(LDFLAGS) $(LDEXEFLAGS) +$(NAME)LINK_SO_ARGS = $(SHFLAGS) $(LDFLAGS) $(LDSOFLAGS) +$(NAME)LINK_EXTRA = $(FFEXTRALIBS) + $(TESTPROGS) $(TOOLS): %$(EXESUF): %.o - $$(LD) $(LDFLAGS) $(LDEXEFLAGS) $$(LD_O) $$(filter %.o,$$^) $$(THISLIB) $(FFEXTRALIBS) $$(EXTRALIBS-$$(*F)) $$(ELIBS) + $$(call LINK,$$(call $(NAME)LINK_EXE_ARGS) $$(LD_O) $$(filter %.o,$$^) $$(THISLIB) $$(call $(NAME)LINK_EXTRA) $$(EXTRALIBS-$$(*F)) $$(ELIBS)) $(SUBDIR)lib$(NAME).version: $(SUBDIR)version.h $(SUBDIR)version_major.h | $(SUBDIR) $$(M) $$(SRC_PATH)/ffbuild/libversion.sh $(NAME) $$^ > $$@ @@ -74,9 +78,9 @@ $(SUBDIR)$(SLIBNAME_WITH_MAJOR): $(OBJS) $(SHLIBOBJS) $(SUBDIR)lib$(NAME).ver $(SLIB_CREATE_DEF_CMD) ifeq ($(RESPONSE_FILES),yes) $(Q)echo $$(filter %.o,$$^) > $$@.objs - $$(LD) $(SHFLAGS) $(LDFLAGS) $(LDSOFLAGS) $$(LD_O) @$$@.objs $(FFEXTRALIBS) + $$(call LINK,$$(call $(NAME)LINK_SO_ARGS) $$(LD_O) @$$@.objs $$(call $(NAME)LINK_EXTRA)) else - $$(LD) $(SHFLAGS) $(LDFLAGS) $(LDSOFLAGS) $$(LD_O) $$(filter %.o,$$^) $(FFEXTRALIBS) + $$(call LINK,$$(call $(NAME)LINK_SO_ARGS) $$(LD_O) $$(filter %.o,$$^) $$(call $(NAME)LINK_EXTRA)) endif $(SLIB_EXTRA_CMD) -$(RM) $$@.objs diff --git a/tests/api/Makefile b/tests/api/Makefile index a2cb06a729..899aeb1f54 100644 --- a/tests/api/Makefile +++ b/tests/api/Makefile @@ -15,7 +15,7 @@ $(APITESTOBJS) $(APITESTOBJS:.o=.i): CPPFLAGS += -DTEST $(APITESTOBJS) $(APITESTOBJS:.o=.i): CFLAGS += -Umain $(APITESTPROGS): %$(EXESUF): %.o $(FF_DEP_LIBS) - $(LD) $(LDFLAGS) $(LDEXEFLAGS) $(LD_O) $(filter %.o,$^) $(FF_EXTRALIBS) $(ELIBS) + $(call LINK,$(LDFLAGS) $(LDEXEFLAGS) $(LD_O) $(filter %.o,$^) $(FF_EXTRALIBS) $(ELIBS)) testclean:: $(RM) $(addprefix $(APITESTSDIR)/,$(CLEANSUFFIXES) *-test$(EXESUF)) diff --git a/tests/checkasm/Makefile b/tests/checkasm/Makefile index 5ce4725543..9f1dd57fa6 100644 --- a/tests/checkasm/Makefile +++ b/tests/checkasm/Makefile @@ -107,7 +107,7 @@ tests/checkasm/checkasm.o: CFLAGS += -Umain CHECKASM := tests/checkasm/checkasm$(EXESUF) $(CHECKASM): $(CHECKASMOBJS) $(FF_STATIC_DEP_LIBS) - $(LD) $(LDFLAGS) $(LDEXEFLAGS) $(LD_O) $(CHECKASMOBJS) $(FF_STATIC_DEP_LIBS) $(EXTRALIBS-avcodec) $(EXTRALIBS-avfilter) $(EXTRALIBS-avformat) $(EXTRALIBS-avutil) $(EXTRALIBS-swresample) $(EXTRALIBS) + $(call LINK,$(LDFLAGS) $(LDEXEFLAGS) $(LD_O) $(CHECKASMOBJS) $(FF_STATIC_DEP_LIBS) $(EXTRALIBS-avcodec) $(EXTRALIBS-avfilter) $(EXTRALIBS-avformat) $(EXTRALIBS-avutil) $(EXTRALIBS-swresample) $(EXTRALIBS)) run-checkasm: $(CHECKASM) run-checkasm: -- 2.49.1 >>From b0d23e5e628975ef5b9a85d95329e2d59fe9db41 Mon Sep 17 00:00:00 2001 From: Timo Rothenpieler Date: Tue, 2 Sep 2025 00:53:55 +0200 Subject: [PATCH 2/5] configure: properly split C/CXX and CPP flags Right now, CFLAGS like -std=c17 leak onto the C++ compilers commandline, leading to tons of warnings and even some errors. Undefining __STRICT_ANSI__ causes strong warnings from the stdlib headers. So only set it for C. --- configure | 160 ++++++++++++++++++++++++++------------------- ffbuild/common.mak | 2 +- 2 files changed, 94 insertions(+), 68 deletions(-) diff --git a/configure b/configure index a38cbdb89f..2f466dbd48 100755 --- a/configure +++ b/configure @@ -976,6 +976,12 @@ add_objcflags(){ append OBJCFLAGS $($objcflags_filter "$@") } +add_allcflags(){ + add_cflags "$@" + add_cxxflags "$@" + add_objcflags "$@" +} + add_asflags(){ append ASFLAGS $($asflags_filter "$@") } @@ -1071,14 +1077,14 @@ test_cxx(){ log test_cxx "$@" cat > $TMPCPP log_file $TMPCPP - test_cmd $cxx $CPPFLAGS $CFLAGS $CXXFLAGS "$@" $CXX_C -o $TMPO $TMPCPP + test_cmd $cxx $CPPFLAGS $CXXFLAGS "$@" $CXX_C -o $TMPO $TMPCPP } test_objcc(){ log test_objcc "$@" cat > $TMPM log_file $TMPM - test_cmd $objcc -Werror=missing-prototypes $CPPFLAGS $CFLAGS $OBJCFLAGS "$@" $OBJCC_C $(cc_o $TMPO) $TMPM + test_cmd $objcc -Werror=missing-prototypes $CPPFLAGS $OBJCFLAGS "$@" $OBJCC_C $(cc_o $TMPO) $TMPM } test_nvcc(){ @@ -1288,14 +1294,19 @@ check_cflags(){ test_cflags "$@" && add_cflags "$@" } -check_cxxflags(){ - log check_cxxflags "$@" +test_cxxflags(){ + log test_cxxflags "$@" set -- $($cflags_filter "$@") - test_cxx "$@" <= 201103L" || { check_cxxflags -std=c++11 && stdcxx="c++11" || { check_cxxflags -std=c++0x && stdcxx="c++0x"; }; } +test_cxxflags_cc -std=$stdcxx ctype.h "__cplusplus >= 201703L" && enable cxx17 + # some compilers silently accept -std=c11, so we also need to check that the # version macro is defined properly check_cflags_cc -std=$stdc ctype.h "__STDC_VERSION__ >= 201112L" || @@ -5851,7 +5872,7 @@ case $target_os in android) disable symver enable section_data_rel_ro - add_cflags -fPIE + add_allcflags -fPIE add_ldexeflags -fPIE -pie SLIB_INSTALL_NAME='$(SLIBNAME)' SLIB_INSTALL_LINKS= @@ -5932,7 +5953,7 @@ case $target_os in # Workaround for Xcode 11 -fstack-check bug if enabled clang; then clang_version=$($cc -dumpversion) - test ${clang_version%%.*} -eq 11 && add_cflags -fno-stack-check + test ${clang_version%%.*} -eq 11 && add_allcflags -fno-stack-check fi # Xcode Clang doesn't default to -fno-common while upstream llvm.org @@ -6151,15 +6172,17 @@ probe_libc(){ # MinGW headers can be installed on Cygwin, so check for newlib first. elif test_${pfx}cpp_condition newlib.h "defined _NEWLIB_VERSION"; then eval ${pfx}libc_type=newlib - add_${pfx}cppflags -U__STRICT_ANSI__ -D_XOPEN_SOURCE=600 + add_${pfx}cflags -U__STRICT_ANSI__ + add_${pfx}cppflags -D_XOPEN_SOURCE=600 # MinGW64 is backwards compatible with MinGW32, so check for it first. elif test_${pfx}cpp_condition _mingw.h "defined __MINGW64_VERSION_MAJOR"; then eval ${pfx}libc_type=mingw64 if test_${pfx}cpp_condition _mingw.h "__MINGW64_VERSION_MAJOR < 3"; then add_compat msvcrt/snprintf.o - add_cflags "-include $source_path/compat/msvcrt/snprintf.h" + add_allcflags "-include $source_path/compat/msvcrt/snprintf.h" fi - add_${pfx}cppflags -U__STRICT_ANSI__ -D__USE_MINGW_ANSI_STDIO=1 + add_${pfx}cflags -U__STRICT_ANSI__ + add_${pfx}cppflags -D__USE_MINGW_ANSI_STDIO=1 eval test \$${pfx_no_}cc_type = "gcc" && add_${pfx}cppflags -D__printf__=__gnu_printf__ test_${pfx}cpp_condition windows.h "!defined(_WIN32_WINNT) || _WIN32_WINNT < 0x0600" && @@ -6171,7 +6194,8 @@ probe_libc(){ test_${pfx}cpp_condition _mingw.h "__MINGW32_MAJOR_VERSION > 3 || \ (__MINGW32_MAJOR_VERSION == 3 && __MINGW32_MINOR_VERSION >= 15)" || die "ERROR: MinGW32 runtime version must be >= 3.15." - add_${pfx}cppflags -U__STRICT_ANSI__ -D__USE_MINGW_ANSI_STDIO=1 + add_${pfx}cflags -U__STRICT_ANSI__ + add_${pfx}cppflags -D__USE_MINGW_ANSI_STDIO=1 test_${pfx}cpp_condition _mingw.h "__MSVCRT_VERSION__ < 0x0700" && add_${pfx}cppflags -D__MSVCRT_VERSION__=0x0700 test_${pfx}cpp_condition windows.h "!defined(_WIN32_WINNT) || _WIN32_WINNT < 0x0600" && @@ -6212,8 +6236,8 @@ probe_libc(){ #endif EOF if [ "$pfx" = "" ]; then - check_func strtoll || add_cflags -Dstrtoll=_strtoi64 - check_func strtoull || add_cflags -Dstrtoull=_strtoui64 + check_func strtoll || add_allcflags -Dstrtoll=_strtoi64 + check_func strtoull || add_allcflags -Dstrtoull=_strtoui64 fi elif test_${pfx}cpp_condition stddef.h "defined __KLIBC__"; then eval ${pfx}libc_type=klibc @@ -6225,7 +6249,7 @@ EOF elif test_${pfx}cpp_condition sys/version.h "defined __DJGPP__"; then eval ${pfx}libc_type=djgpp add_cppflags -U__STRICT_ANSI__ - add_cflags "-include $source_path/compat/djgpp/math.h" + add_allcflags "-include $source_path/compat/djgpp/math.h" add_compat djgpp/math.o fi test_${pfx}cc <>From 6043c152b3196bcaf5c6303cb9001add3ad0c172 Mon Sep 17 00:00:00 2001 From: Timo Rothenpieler Date: Thu, 4 Sep 2025 20:51:58 +0200 Subject: [PATCH 3/5] configure: fix naming of some C++/cxx functions being mislabeled as cpp --- configure | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/configure b/configure index 2f466dbd48..a9257a3557 100755 --- a/configure +++ b/configure @@ -1437,8 +1437,8 @@ check_func_headers(){ } | test_ld "cc" "$@" && enable $funcs && enable_sanitized $headers } -check_class_headers_cpp(){ - log check_class_headers_cpp "$@" +check_class_headers_cxx(){ + log check_class_headers_cxx "$@" headers=$1 classes=$2 shift 2 @@ -1530,14 +1530,14 @@ check_lib(){ enable $name && eval ${name}_extralibs="\$@" } -check_lib_cpp(){ - log check_lib_cpp "$@" +check_lib_cxx(){ + log check_lib_cxx "$@" name="$1" headers="$2" classes="$3" shift 3 disable $name - check_class_headers_cpp "$headers" "$classes" "$@" && + check_class_headers_cxx "$headers" "$classes" "$@" && enable $name && eval ${name}_extralibs="\$@" } @@ -1708,12 +1708,12 @@ require_cc(){ check_cc "$@" || die "ERROR: $name failed" } -require_cpp(){ - log require_cpp "$@" +require_cxx(){ + log require_cxx "$@" name_version="$1" name="${1%% *}" shift - check_lib_cpp "$name" "$@" || die "ERROR: $name_version not found" + check_lib_cxx "$name" "$@" || die "ERROR: $name_version not found" } require_headers(){ @@ -7189,7 +7189,7 @@ enabled libtensorflow && require libtensorflow tensorflow/c/c_api.h TF_Versi enabled libtesseract && require_pkg_config libtesseract tesseract tesseract/capi.h TessBaseAPICreate enabled libtheora && require libtheora theora/theoraenc.h th_info_init -ltheoraenc -ltheoradec -logg enabled libtls && require_pkg_config libtls libtls tls.h tls_configure -enabled libtorch && check_cxxflags -std=c++17 && require_cpp libtorch torch/torch.h "torch::Tensor" -ltorch -lc10 -ltorch_cpu -lstdc++ -lpthread +enabled libtorch && check_cxxflags -std=c++17 && require_cxx libtorch torch/torch.h "torch::Tensor" -ltorch -lc10 -ltorch_cpu -lstdc++ -lpthread enabled libtwolame && require libtwolame twolame.h twolame_init -ltwolame && { check_lib libtwolame twolame.h twolame_encode_buffer_float32_interleaved -ltwolame || die "ERROR: libtwolame must be installed and version must be >= 0.3.10"; } -- 2.49.1 >>From 82db3076104580a43728b3f6338b6c1e96a7795b Mon Sep 17 00:00:00 2001 From: Timo Rothenpieler Date: Tue, 2 Sep 2025 01:00:24 +0200 Subject: [PATCH 4/5] configure: don't use MinGW ANSI stdio when using UCRT --- configure | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/configure b/configure index a9257a3557..6e55a55e40 100755 --- a/configure +++ b/configure @@ -6182,7 +6182,9 @@ probe_libc(){ add_allcflags "-include $source_path/compat/msvcrt/snprintf.h" fi add_${pfx}cflags -U__STRICT_ANSI__ - add_${pfx}cppflags -D__USE_MINGW_ANSI_STDIO=1 + if ! test_${pfx}cpp_condition crtdefs.h "defined(_UCRT)"; then + add_${pfx}cppflags -D__USE_MINGW_ANSI_STDIO=1 + fi eval test \$${pfx_no_}cc_type = "gcc" && add_${pfx}cppflags -D__printf__=__gnu_printf__ test_${pfx}cpp_condition windows.h "!defined(_WIN32_WINNT) || _WIN32_WINNT < 0x0600" && @@ -6195,7 +6197,9 @@ probe_libc(){ (__MINGW32_MAJOR_VERSION == 3 && __MINGW32_MINOR_VERSION >= 15)" || die "ERROR: MinGW32 runtime version must be >= 3.15." add_${pfx}cflags -U__STRICT_ANSI__ - add_${pfx}cppflags -D__USE_MINGW_ANSI_STDIO=1 + if ! test_${pfx}cpp_condition crtdefs.h "defined(_UCRT)"; then + add_${pfx}cppflags -D__USE_MINGW_ANSI_STDIO=1 + fi test_${pfx}cpp_condition _mingw.h "__MSVCRT_VERSION__ < 0x0700" && add_${pfx}cppflags -D__MSVCRT_VERSION__=0x0700 test_${pfx}cpp_condition windows.h "!defined(_WIN32_WINNT) || _WIN32_WINNT < 0x0600" && -- 2.49.1 >>From 40ac47cb72008b95fdcf5e8879020a8e17068abb Mon Sep 17 00:00:00 2001 From: Timo Rothenpieler Date: Sat, 30 Aug 2025 00:45:22 +0200 Subject: [PATCH 5/5] avfilter: add gfxcapture, Windows.Graphics.Capture based window/monitor capture --- configure | 8 + doc/filters.texi | 133 +++ libavfilter/Makefile | 1 + libavfilter/allfilters.c | 1 + libavfilter/version.h | 2 +- libavfilter/vsrc_gfxcapture.c | 91 ++ libavfilter/vsrc_gfxcapture.h | 66 ++ libavfilter/vsrc_gfxcapture_winrt.cpp | 1387 +++++++++++++++++++++++++ libavfilter/vsrc_gfxcapture_winrt.h | 180 ++++ 9 files changed, 1868 insertions(+), 1 deletion(-) create mode 100644 libavfilter/vsrc_gfxcapture.c create mode 100644 libavfilter/vsrc_gfxcapture.h create mode 100644 libavfilter/vsrc_gfxcapture_winrt.cpp create mode 100644 libavfilter/vsrc_gfxcapture_winrt.h diff --git a/configure b/configure index 6e55a55e40..570a8fafb7 100755 --- a/configure +++ b/configure @@ -2508,6 +2508,8 @@ TOOLCHAIN_FEATURES=" TYPES_LIST=" DPI_AWARENESS_CONTEXT IDXGIOutput5 + __x_ABI_CWindows_CGraphics_CCapture_CIGraphicsCaptureSession5 + IDirect3DDxgiInterfaceAccess kCMVideoCodecType_HEVC kCMVideoCodecType_HEVCWithAlpha kCMVideoCodecType_VP9 @@ -3404,6 +3406,8 @@ pad_cuda_filter_deps_any="cuda_nvcc cuda_llvm" sharpen_npp_filter_deps="ffnvcodec libnpp" ddagrab_filter_deps="d3d11va IDXGIOutput1 DXGI_OUTDUPL_FRAME_INFO" +gfxcapture_filter_deps="cxx17 d3d11va IGraphicsCaptureItemInterop __x_ABI_CWindows_CGraphics_CCapture_CIGraphicsCaptureSession3" +gfxcapture_filter_extralibs="-lstdc++" scale_d3d11_filter_deps="d3d11va" amf_deps_any="libdl LoadLibrary" @@ -6922,6 +6926,10 @@ check_type "windows.h" "DPI_AWARENESS_CONTEXT" -D_WIN32_WINNT=0x0A00 check_type "windows.h security.h schnlsp.h" SecPkgContext_KeyingMaterialInfo -DSECURITY_WIN32 check_type "d3d9.h dxva2api.h" DXVA2_ConfigPictureDecode -D_WIN32_WINNT=0x0602 check_func_headers mfapi.h MFCreateAlignedMemoryBuffer -lmfplat +check_type "windows.h windows.graphics.capture.h" __x_ABI_CWindows_CGraphics_CCapture_CIGraphicsCaptureSession3 -D_WIN32_WINNT=0x0A00 -DWINDOWS_FOUNDATION_UNIVERSALAPICONTRACT_VERSION=0x130000 -DCOBJMACROS +check_type "windows.h windows.graphics.capture.h" __x_ABI_CWindows_CGraphics_CCapture_CIGraphicsCaptureSession5 -D_WIN32_WINNT=0x0A00 -DWINDOWS_FOUNDATION_UNIVERSALAPICONTRACT_VERSION=0x130000 -DCOBJMACROS +check_type "windows.h windows.graphics.capture.interop.h" IGraphicsCaptureItemInterop -D_WIN32_WINNT=0x0A00 -DWINDOWS_FOUNDATION_UNIVERSALAPICONTRACT_VERSION=0x130000 -DCOBJMACROS +check_type "windows.h windows.graphics.directx.direct3d11.interop.h" IDirect3DDxgiInterfaceAccess -D_WIN32_WINNT=0x0A00 -DWINDOWS_FOUNDATION_UNIVERSALAPICONTRACT_VERSION=0x130000 -DCOBJMACROS check_type "vdpau/vdpau.h" "VdpPictureInfoHEVC" check_type "vdpau/vdpau.h" "VdpPictureInfoVP9" diff --git a/doc/filters.texi b/doc/filters.texi index 5b52fc5521..df4f4deabf 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -29614,6 +29614,139 @@ ddagrab=video_size=800x600:offset_x=100:offset_y=100 @end example +@section gfxcapture + +Capture windows or monitors using Windows.Graphics.Capture API. + +This source provides low overhead capture of application windows or entire monitors. +The filter outputs hardware frames in @code{d3d11} format; use @code{hwdownload,format=} +if system memory frames are required. + +The window to be captured can be selected via regular expressions on its title, +class name or backing executable name, by explicit native handles, or by monitor +index or explicit native handle. A window must match all provided expressions to be +selected. The first matching window will be picked, in whatever order Windows +returns them. + +Explicit handles (@option{hwnd}, @option{hmonitor}) override pattern or index +based selection. If neither handles nor a monitor index are given, the first +window matching the provided regular expressions is captured. + +This source does NOT hold a stable FPS. It returns frames at whatever rate the compositor +provides them, only capped by the @option{max_framerate}. +If you need a stable rate, you need to add an fps filter to drop/duplicate frames as needed. + +If the capture source disappears mid-capture (window closed, monitor disconnected), the filter will return EOF. + +This source accepts the following options: +@table @option +@item window_text +ECMAScript regular expression matched against the window title. Supports a +PCRE style @code{(?i)} prefix for case-insensitive matching. + +@item window_class +As @option{window_text}, but matched against the window class name. + +@item window_exe +As @option{window_text}, but matched against the executable file name of the +window's process. + +@item monitor_idx +Zero-based index of the monitor to capture. + +@item hwnd +Explicit native window handle (HWND). + +@item hmonitor +Explicit native monitor handle (HMONITOR). + +@item capture_cursor +Capture the mouse cursor. Enabled by default. + +@item display_border +Draw a yellow highlight border around the captured window. Disabled by default. + +@item max_framerate +Maximum capture frame rate. Accepts a video rate (e.g. @code{30}, @code{60/1}, +@code{24000/1001}). The default is @code{1000}, effectively uncapped. +The actual rate is the rate at which the compositor renders the window/monitor. + +@item width +Force the output canvas width. If zero (default) the initial captured source +width is used. See @option{resize_mode}. + +@item height +Force the output canvas height. If zero (default) the initial captured source +height is used. See @option{resize_mode}. + +@item premultiplied +If set to 1, return frames with premultiplied alpha. Default is 0 (straight +alpha). + +@item resize_mode +Defines how the captured content is fitted into the output canvas size. +Possible values: +@table @samp +@item crop +Crop (or pad with black) to the canvas size. (default) +@item scale +Scale the source to fill the canvas, potentially altering aspect ratio. +@item scale_aspect +Scale the source to fit inside the canvas while preserving aspect ratio. +Remaining area is filled with black. +@end table + +@item scale_mode +Scaling algorithm used when resizing is required. + +Possible values: +@table @samp +@item point +Nearest neighbour (pixelated) scaling. +@item bilinear +Bilinear filtering. (default) +@item bicubic +Bicubic filtering. Potentially more blurry, but fewer scaling artifacts depending on contents. +@end table + +@item output_fmt +Desired output pixel format inside the D3D11 hardware frames. + +Possible values: +@table @samp +@item bgra +@item 8bit +8 bit BGRA output (default) +@item x2bgr10 +@item 10bit +10 bit BGR output +@item rgbaf16 +@item 16bit +16bit float RGBA output +@end table + +@end table + +@subsection Examples +@itemize +@item Capture a window by title (case-insensitive) at a maximum of 60 fps: +@example +ffmpeg -filter_complex gfxcapture=window_text='(?i)My Application':max_framerate=60,hwdownload,format=bgra,format=yuv420p -c:v libx264 -crf 15 capture.mp4 +@end example + +@item Capture monitor 1 at native refresh, 10bit color depth, scale to 1920x1080 preserving aspect: +@example +ffmpeg -filter_complex gfxcapture=monitor_idx=1:width=1920:height=1080:resize_mode=scale_aspect:output_fmt=10bit -c:v hevc_nvenc -cq 15 capture.mp4 +@end example + +@item Capture a window by executable name, draw border, force point scaling, fixed 60 fps: +@example +ffmpeg -filter_complex gfxcapture=window_exe='^firefox.exe$':display_border=1:scale_mode=point,fps=60 -rc qvbr -qvbr_quality_level 15 -c:v h264_amf capture.mp4 +@end example + +@end itemize + + @section gradients Generate several gradients. diff --git a/libavfilter/Makefile b/libavfilter/Makefile index bd3f6da27d..70b100aff1 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -609,6 +609,7 @@ OBJS-$(CONFIG_COLORSPECTRUM_FILTER) += vsrc_testsrc.o OBJS-$(CONFIG_COREIMAGESRC_FILTER) += vf_coreimage.o OBJS-$(CONFIG_DDAGRAB_FILTER) += vsrc_ddagrab.o OBJS-$(CONFIG_FREI0R_SRC_FILTER) += vf_frei0r.o +OBJS-$(CONFIG_GFXCAPTURE_FILTER) += vsrc_gfxcapture.o vsrc_gfxcapture_winrt.o OBJS-$(CONFIG_GRADIENTS_FILTER) += vsrc_gradients.o OBJS-$(CONFIG_HALDCLUTSRC_FILTER) += vsrc_testsrc.o OBJS-$(CONFIG_LIFE_FILTER) += vsrc_life.o diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index 3ac1502254..84f15f85c5 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -571,6 +571,7 @@ extern const FFFilter ff_vsrc_colorspectrum; extern const FFFilter ff_vsrc_coreimagesrc; extern const FFFilter ff_vsrc_ddagrab; extern const FFFilter ff_vsrc_frei0r_src; +extern const FFFilter ff_vsrc_gfxcapture; extern const FFFilter ff_vsrc_gradients; extern const FFFilter ff_vsrc_haldclutsrc; extern const FFFilter ff_vsrc_life; diff --git a/libavfilter/version.h b/libavfilter/version.h index ba8a6fdab2..77f38cb9b4 100644 --- a/libavfilter/version.h +++ b/libavfilter/version.h @@ -31,7 +31,7 @@ #include "version_major.h" -#define LIBAVFILTER_VERSION_MINOR 8 +#define LIBAVFILTER_VERSION_MINOR 9 #define LIBAVFILTER_VERSION_MICRO 100 diff --git a/libavfilter/vsrc_gfxcapture.c b/libavfilter/vsrc_gfxcapture.c new file mode 100644 index 0000000000..0cade90fc4 --- /dev/null +++ b/libavfilter/vsrc_gfxcapture.c @@ -0,0 +1,91 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "config.h" + +#include "libavutil/internal.h" +#include "libavutil/opt.h" +#include "avfilter.h" +#include "filters.h" + +#include "vsrc_gfxcapture.h" + +#define OFFSET(x) offsetof(GfxCaptureContext, x) +#define FLAGS AV_OPT_FLAG_VIDEO_PARAM|AV_OPT_FLAG_FILTERING_PARAM +static const AVOption gfxcapture_options[] = { + { "window_text", "ECMAScript regular expression to match against the window text. " + "Supports PCRE style (?i) prefix for case-insensitivity.", + OFFSET(window_text), AV_OPT_TYPE_STRING, { .str = NULL }, 0, INT_MAX, FLAGS }, + { "window_class", "as window_text, but against the window class", + OFFSET(window_class), AV_OPT_TYPE_STRING, { .str = NULL }, 0, INT_MAX, FLAGS }, + { "window_exe", "as window_text, but against the windows executable name", + OFFSET(window_exe), AV_OPT_TYPE_STRING, { .str = NULL }, 0, INT_MAX, FLAGS }, + { "monitor_idx", "index of the monitor to capture", OFFSET(monitor_idx), AV_OPT_TYPE_INT, { .i64 = -1 }, -1, INT_MAX, FLAGS }, + + { "capture_cursor", "capture mouse cursor", OFFSET(capture_cursor), AV_OPT_TYPE_BOOL, { .i64 = 1 }, 0, 1, FLAGS }, + { "display_border", "display yellow border around captured window", + OFFSET(display_border), AV_OPT_TYPE_BOOL, { .i64 = 0 }, 0, 1, FLAGS }, + { "max_framerate", "set maximum capture frame rate", OFFSET(frame_rate), AV_OPT_TYPE_VIDEO_RATE, { .str = "1000" }, 0.001, 1000, FLAGS }, + { "hwnd", "pre-existing HWND handle", OFFSET(user_hwnd), AV_OPT_TYPE_UINT64, { .i64 = 0 }, 0, UINT64_MAX, FLAGS }, + { "hmonitor", "pre-existing HMONITOR handle", OFFSET(user_hmonitor), AV_OPT_TYPE_UINT64, { .i64 = 0 }, 0, UINT64_MAX, FLAGS }, + { "width", "force width of the output frames", OFFSET(canvas_width), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, INT_MAX, FLAGS }, + { "height", "force height of the output frames", OFFSET(canvas_height), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, INT_MAX, FLAGS }, + { "premultiplied", "return premultiplied alpha frames", OFFSET(premult_alpha), AV_OPT_TYPE_BOOL, { .i64 = 0 }, 0, 1, FLAGS }, + { "resize_mode", "capture source resize behavior", OFFSET(resize_mode), AV_OPT_TYPE_INT, { .i64 = GFX_RESIZE_CROP }, 0, GFX_RESIZE_NB - 1, FLAGS, .unit = "resize_mode" }, + { "crop", "crop or add black bars into frame", 0, AV_OPT_TYPE_CONST, { .i64 = GFX_RESIZE_CROP }, 0, 0, FLAGS, .unit = "resize_mode" }, + { "scale", "scale source to fit initial size", 0, AV_OPT_TYPE_CONST, { .i64 = GFX_RESIZE_SCALE }, 0, 0, FLAGS, .unit = "resize_mode" }, + { "scale_aspect", "scale source to fit initial size while preserving aspect ratio", + 0, AV_OPT_TYPE_CONST, { .i64 = GFX_RESIZE_SCALE_ASPECT }, 0, 0, FLAGS, .unit = "resize_mode" }, + { "scale_mode", "scaling algorithm", OFFSET(scale_mode), AV_OPT_TYPE_INT, { .i64 = GFX_SCALE_BILINEAR }, 0, GFX_SCALE_NB - 1, FLAGS, .unit = "scale_mode" }, + { "point", "use point scaling", 0, AV_OPT_TYPE_CONST, { .i64 = GFX_SCALE_POINT }, 0, 0, FLAGS, .unit = "scale_mode" }, + { "bilinear", "use bilinear scaling", 0, AV_OPT_TYPE_CONST, { .i64 = GFX_SCALE_BILINEAR }, 0, 0, FLAGS, .unit = "scale_mode" }, + { "bicubic", "use bicubic scaling", 0, AV_OPT_TYPE_CONST, { .i64 = GFX_SCALE_BICUBIC }, 0, 0, FLAGS, .unit = "scale_mode" }, + { "output_fmt", "desired output format", OFFSET(out_fmt), AV_OPT_TYPE_INT, { .i64 = AV_PIX_FMT_BGRA }, 0, INT_MAX, FLAGS, .unit = "output_fmt" }, + { "8bit", "output 8 Bit BGRA", 0, AV_OPT_TYPE_CONST, { .i64 = AV_PIX_FMT_BGRA }, 0, INT_MAX, FLAGS, .unit = "output_fmt" }, + { "bgra", "output 8 Bit BGRA", 0, AV_OPT_TYPE_CONST, { .i64 = AV_PIX_FMT_BGRA }, 0, INT_MAX, FLAGS, .unit = "output_fmt" }, + { "10bit", "output 10 Bit X2BGR10", 0, AV_OPT_TYPE_CONST, { .i64 = AV_PIX_FMT_X2BGR10 }, 0, INT_MAX, FLAGS, .unit = "output_fmt" }, + { "x2bgr10", "output 10 Bit X2BGR10", 0, AV_OPT_TYPE_CONST, { .i64 = AV_PIX_FMT_X2BGR10 }, 0, INT_MAX, FLAGS, .unit = "output_fmt" }, + { "16bit", "output 16 Bit RGBAF16", 0, AV_OPT_TYPE_CONST, { .i64 = AV_PIX_FMT_RGBAF16 }, 0, INT_MAX, FLAGS, .unit = "output_fmt" }, + { "rgbaf16", "output 16 Bit RGBAF16", 0, AV_OPT_TYPE_CONST, { .i64 = AV_PIX_FMT_RGBAF16 }, 0, INT_MAX, FLAGS, .unit = "output_fmt" }, + { NULL } +}; + +AVFILTER_DEFINE_CLASS(gfxcapture); + +static const AVFilterPad gfxcapture_outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + .config_props = ff_gfxcapture_config_props, + }, +}; + +const FFFilter ff_vsrc_gfxcapture = { + .p.name = "gfxcapture", + .p.description = NULL_IF_CONFIG_SMALL("Capture graphics/screen content as a video source"), + .p.priv_class = &gfxcapture_class, + .p.inputs = NULL, + .p.flags = AVFILTER_FLAG_HWDEVICE, + .priv_size = sizeof(GfxCaptureContext), + .init = ff_gfxcapture_init, + .uninit = ff_gfxcapture_uninit, + FILTER_OUTPUTS(gfxcapture_outputs), + FILTER_SINGLE_PIXFMT(AV_PIX_FMT_D3D11), + .flags_internal = FF_FILTER_FLAG_HWFRAME_AWARE, + .activate = ff_gfxcapture_activate, +}; diff --git a/libavfilter/vsrc_gfxcapture.h b/libavfilter/vsrc_gfxcapture.h new file mode 100644 index 0000000000..f2310b7dba --- /dev/null +++ b/libavfilter/vsrc_gfxcapture.h @@ -0,0 +1,66 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#ifndef AVFILTER_VSRC_GFXCAPTURE_H +#define AVFILTER_VSRC_GFXCAPTURE_H + +typedef struct GfxCaptureContextCpp GfxCaptureContextCpp; + +enum GfxResizeMode { + GFX_RESIZE_CROP = 0, + GFX_RESIZE_SCALE, + GFX_RESIZE_SCALE_ASPECT, + GFX_RESIZE_NB +}; + +enum GfxScaleMode { + GFX_SCALE_POINT = 0, + GFX_SCALE_BILINEAR, + GFX_SCALE_BICUBIC, + GFX_SCALE_NB +}; + +typedef struct GfxCaptureContext { + const AVClass *avclass; + + GfxCaptureContextCpp *ctx; + + const char *window_text; + const char *window_class; + const char *window_exe; + int monitor_idx; + + uint64_t user_hwnd; + uint64_t user_hmonitor; + + int capture_cursor; + int display_border; + AVRational frame_rate; + int canvas_width, canvas_height; + int out_fmt; + int resize_mode; + int scale_mode; + int premult_alpha; +} GfxCaptureContext; + +av_cold int ff_gfxcapture_init(AVFilterContext *avctx); +av_cold void ff_gfxcapture_uninit(AVFilterContext *avctx); +int ff_gfxcapture_activate(AVFilterContext *avctx); +int ff_gfxcapture_config_props(AVFilterLink *outlink); + +#endif /* AVFILTER_VSRC_GFXCAPTURE_H */ diff --git a/libavfilter/vsrc_gfxcapture_winrt.cpp b/libavfilter/vsrc_gfxcapture_winrt.cpp new file mode 100644 index 0000000000..1572d004ad --- /dev/null +++ b/libavfilter/vsrc_gfxcapture_winrt.cpp @@ -0,0 +1,1387 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +extern "C" { +#include "config.h" +} + +#if !defined(_WIN32_WINNT) || _WIN32_WINNT < 0x0A00 +#undef _WIN32_WINNT +#define _WIN32_WINNT 0x0A00 +#endif + +#define WINDOWS_FOUNDATION_UNIVERSALAPICONTRACT_VERSION 0x130000 + +// work around bug in mingw double-defining IReference (BYTE == boolean) +#define ____FIReference_1_boolean_INTERFACE_DEFINED__ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#if HAVE_IDIRECT3DDXGIINTERFACEACCESS +#include +#endif + +extern "C" { +#include "libavutil/avassert.h" +#include "libavutil/internal.h" +#include "libavutil/mem.h" +#include "libavutil/opt.h" +#include "libavutil/time.h" +#include "libavutil/pixdesc.h" +#include "libavutil/hwcontext.h" +#include "libavutil/hwcontext_d3d11va.h" +#include "avfilter.h" +#include "filters.h" +#include "video.h" + +#include "vsrc_gfxcapture.h" +} + +#include +#include +#include +#include +#include +#include +#include +#include + +#include "vsrc_gfxcapture_winrt.h" + +using namespace ABI::Windows::System; +using namespace ABI::Windows::Foundation; +using namespace ABI::Windows::Graphics::Capture; +using namespace ABI::Windows::Graphics::DirectX::Direct3D11; +using namespace Windows::Graphics::DirectX::Direct3D11; +using Microsoft::WRL::ComPtr; +using ABI::Windows::Graphics::SizeInt32; +using ABI::Windows::Foundation::TimeSpan; +using ABI::Windows::Graphics::DirectX::DirectXPixelFormat; + +#define TIMESPAN_RES 10000000 +#define TIMESPAN_RES64 INT64_C(10000000) + +#define CAPTURE_POOL_SIZE 2 + +enum { + WM_RT_THREAD_SHUTDOWN = WM_APP + 1, + WM_RT_SHUTDOWN_DONE = WM_APP + 2 +}; + +#define CCTX(ctx) static_cast(ctx) + +typedef struct GfxCaptureFunctions { + hmodule_ptr_t graphicscapture_handle; + + hmodule_ptr_t combase_handle; + HRESULT (WINAPI *RoInitialize)(RO_INIT_TYPE initType); + void (WINAPI *RoUninitialize)(void); + HRESULT (WINAPI *RoGetActivationFactory)(HSTRING activatableClassId, REFIID iid, void **factory); + HRESULT (WINAPI *WindowsCreateStringReference)(PCWSTR sourceString, UINT32 length, HSTRING_HEADER *hstringHeader, HSTRING *string); + + hmodule_ptr_t dwmapi_handle; + HRESULT (WINAPI *DwmGetWindowAttribute)(HWND hwnd, DWORD dwAttribute, PVOID pvAttribute, DWORD cbAttribute); + + hmodule_ptr_t d3d11_handle; + HRESULT (WINAPI *CreateDirect3D11DeviceFromDXGIDevice)(IDXGIDevice *dxgiDevice, IInspectable **graphicsDevice); + + hmodule_ptr_t coremsg_handle; + HRESULT (WINAPI *CreateDispatcherQueueController)(DispatcherQueueOptions options, PDISPATCHERQUEUECONTROLLER *dispatcherQueueController); + + hmodule_ptr_t user32_handle; + DPI_AWARENESS_CONTEXT (WINAPI *SetThreadDpiAwarenessContext)(DPI_AWARENESS_CONTEXT dpiContext); + + hmodule_ptr_t d3dcompiler_handle; + HRESULT (WINAPI *D3DCompile)(LPCVOID pSrcData, SIZE_T SrcDataSize, LPCSTR pSourceName, const D3D10_SHADER_MACRO *pDefines, ID3DInclude *pInclude, + LPCSTR pEntrypoint, LPCSTR pTarget, UINT Flags1, UINT Flags2, ID3DBlob **ppCode, ID3DBlob **ppErrorMsgs); +} GfxCaptureFunctions; + +// This struct contains all data handled by the capture thread +struct GfxCaptureContextRt { + ComPtr dispatcher_queue_controller; + ComPtr dispatcher_queue; + + ComPtr capture_item; + ComPtr d3d_device; + ComPtr frame_pool; + ComPtr capture_session; + + EventRegistrationToken frame_arrived_token { 0 }; + EventRegistrationToken closed_token { 0 }; + + std::mutex frame_arrived_lock; + std::condition_variable frame_arrived_cond; + std::atomic window_closed { false }; + std::atomic frame_seq { 0 }; + + SizeInt32 cap_size { 0, 0 }; +}; + +struct GfxCaptureContextCpp { + GfxCaptureFunctions fn; + std::unique_ptr rt; + + std::thread rt_thread; + DWORD rt_thread_id { 0 }; + std::mutex rt_thread_init_lock; + std::condition_variable rt_thread_init_cond; + volatile int rt_thread_init_res { INT_MAX }; + std::mutex rt_thread_lock; + volatile int rt_thread_res { INT_MAX }; + + HWND capture_hwnd { nullptr }; + HMONITOR capture_hmonitor { nullptr }; + + AVBufferRef *device_ref { nullptr }; + AVHWDeviceContext *device_ctx { nullptr }; + AVD3D11VADeviceContext *device_hwctx { nullptr }; + + AVBufferRef *frames_ref { nullptr }; + AVHWFramesContext *frames_ctx { nullptr }; + AVD3D11VAFramesContext *frames_hwctx { nullptr }; + + int64_t first_pts { 0 }; + int64_t last_pts { 0 }; + + ComPtr vertex_shader; + ComPtr pixel_shader; + ComPtr sampler_state; + ComPtr shader_cb; + ComPtr deferred_ctx; +}; + +template +static HRESULT get_activation_factory(GfxCaptureContextCpp *ctx, PCWSTR clsid, T** factory) { + HSTRING_HEADER hsheader = { 0 }; + HSTRING hs = NULL; + + HRESULT hr = ctx->fn.WindowsCreateStringReference(clsid, (UINT32)wcslen(clsid), &hsheader, &hs); + if (FAILED(hr)) + return hr; + + return ctx->fn.RoGetActivationFactory(hs, IID_PPV_ARGS(factory)); +} + +#define CHECK_HR(fcall, action) \ + do { \ + HRESULT fhr = fcall; \ + if (FAILED(fhr)) { \ + av_log(avctx, AV_LOG_ERROR, #fcall " failed: 0x%08lX\n", fhr); \ + action; \ + } \ + } while (0) +#define CHECK_HR_RET(...) CHECK_HR((__VA_ARGS__), return AVERROR_EXTERNAL) +#define CHECK_HR_FAIL(...) CHECK_HR((__VA_ARGS__), ret = AVERROR_EXTERNAL; goto fail) +#define CHECK_HR_LOG(...) CHECK_HR((__VA_ARGS__), (void)0) + +/************************************************** + * WinRT Worker Thread * + * All rt_* functions must run only on RT thread! * + **************************************************/ + +static void rt_frame_arrived_handler(const std::unique_ptr &rtctx) { + rtctx->frame_seq.fetch_add(1, std::memory_order_release); + rtctx->frame_arrived_cond.notify_one(); +} + +static void rt_closed_handler(const std::unique_ptr &rtctx) { + rtctx->window_closed.store(true, std::memory_order_release); + rtctx->frame_arrived_cond.notify_one(); +} + +static void rt_stop_capture_session(AVFilterContext *avctx) +{ + GfxCaptureContext *cctx = CCTX(avctx->priv); + GfxCaptureContextCpp *ctx = cctx->ctx; + std::unique_ptr &rtctx = ctx->rt; + + if (rtctx->closed_token.value && rtctx->capture_item) { + CHECK_HR_LOG(rtctx->capture_item->remove_Closed(rtctx->closed_token)); + rtctx->closed_token.value = 0; + } + + if (rtctx->frame_arrived_token.value && rtctx->frame_pool) { + CHECK_HR_LOG(rtctx->frame_pool->remove_FrameArrived(rtctx->frame_arrived_token)); + rtctx->frame_arrived_token.value = 0; + } + + if (rtctx->capture_session) { + ComPtr closable; + if (SUCCEEDED(rtctx->capture_session.As(&closable))) { + closable->Close(); + } else { + av_log(avctx, AV_LOG_ERROR, "Failed to get capture session IClosable interface\n"); + } + } + + if (rtctx->frame_pool) { + ComPtr capture_frame; + while(SUCCEEDED(rtctx->frame_pool->TryGetNextFrame(&capture_frame)) && capture_frame) { + capture_frame = nullptr; + } + + ComPtr closable; + if (SUCCEEDED(rtctx->frame_pool.As(&closable))) { + CHECK_HR_LOG(closable->Close()); + } else { + av_log(avctx, AV_LOG_ERROR, "Failed to get frame pool IClosable interface\n"); + } + } +} + +static int rt_setup_gfxcapture_session(AVFilterContext *avctx) +{ + GfxCaptureContext *cctx = CCTX(avctx->priv); + GfxCaptureContextCpp *ctx = cctx->ctx; + std::unique_ptr &rtctx = ctx->rt; + + ComPtr frame_pool_statics; + ComPtr d3d11_device = ctx->device_hwctx->device; + ComPtr d3d10_multithread; + ComPtr dxgi_device; + ComPtr session2; + ComPtr session3; + ComPtr session5; + + DirectXPixelFormat fmt = DirectXPixelFormat::DirectXPixelFormat_B8G8R8A8UIntNormalized; + if (cctx->out_fmt != AV_PIX_FMT_BGRA) + fmt = DirectXPixelFormat::DirectXPixelFormat_R16G16B16A16Float; + + CHECK_HR_RET(d3d11_device.As(&d3d10_multithread)); + d3d10_multithread->SetMultithreadProtected(TRUE); + + CHECK_HR_RET(get_activation_factory(ctx, RuntimeClass_Windows_Graphics_Capture_Direct3D11CaptureFramePool, &frame_pool_statics)); + + CHECK_HR_RET(d3d11_device.As(&dxgi_device)); + CHECK_HR_RET(ctx->fn.CreateDirect3D11DeviceFromDXGIDevice(dxgi_device.Get(), &rtctx->d3d_device)); + CHECK_HR_RET(rtctx->capture_item->get_Size(&rtctx->cap_size)); + CHECK_HR_RET(frame_pool_statics->Create(rtctx->d3d_device.Get(), fmt, CAPTURE_POOL_SIZE, rtctx->cap_size, &rtctx->frame_pool)); + CHECK_HR_RET(rtctx->frame_pool->CreateCaptureSession(rtctx->capture_item.Get(), &rtctx->capture_session)); + + if (SUCCEEDED(rtctx->capture_session.As(&session2))) { + if (FAILED(session2->put_IsCursorCaptureEnabled(cctx->capture_cursor))) { + av_log(avctx, AV_LOG_WARNING, "Failed setting cursor capture mode\n"); + } + } else { + av_log(avctx, AV_LOG_WARNING, "Cursor capture unavailable\n"); + } + + if (SUCCEEDED(rtctx->capture_session.As(&session3))) { + // this one is weird, it can return failure but still work + if (FAILED(session3->put_IsBorderRequired(cctx->display_border))) { + av_log(avctx, AV_LOG_WARNING, "Failed setting border drawing mode\n"); + } + } else { + av_log(avctx, AV_LOG_WARNING, "Disabling border drawing unavailable\n"); + } + + if (SUCCEEDED(rtctx->capture_session.As(&session5))) { + TimeSpan ivl = { av_rescale_q(1, av_inv_q(cctx->frame_rate), (AVRational){1, TIMESPAN_RES}) }; + if (FAILED(session5->put_MinUpdateInterval(ivl))) { + av_log(avctx, AV_LOG_WARNING, "Failed setting minimum update interval, framerate may be limited\n"); + } + } else { + av_log(avctx, AV_LOG_WARNING, "Setting minimum update interval unavailable, framerate may be limited\n"); + } + + rtctx->window_closed = 0; + + CHECK_HR_RET(rtctx->capture_item->add_Closed( + create_cb_handler, IGraphicsCaptureItem*, IInspectable*>( + [avctx, ctx](auto, auto) { + av_log(avctx, AV_LOG_INFO, "Capture item closed\n"); + rt_closed_handler(ctx->rt); + return S_OK; + }).Get(), &rtctx->closed_token)); + + CHECK_HR_RET(rtctx->frame_pool->add_FrameArrived( + create_cb_handler, IDirect3D11CaptureFramePool*, IInspectable*>( + [avctx, ctx](auto, auto) { + av_log(avctx, AV_LOG_TRACE, "Frame arrived\n"); + rt_frame_arrived_handler(ctx->rt); + return S_OK; + }).Get(), &rtctx->frame_arrived_token)); + + return 0; +} + +static int rt_setup_gfxcapture_capture(AVFilterContext *avctx) +{ + GfxCaptureContext *cctx = CCTX(avctx->priv); + GfxCaptureContextCpp *ctx = cctx->ctx; + std::unique_ptr &rtctx = ctx->rt; + int ret = 0; + + ComPtr capture_item_interop; + CHECK_HR_RET(get_activation_factory(ctx, RuntimeClass_Windows_Graphics_Capture_GraphicsCaptureItem, &capture_item_interop)); + + if (ctx->capture_hmonitor) { + HRESULT hr = capture_item_interop->CreateForMonitor(ctx->capture_hmonitor, IID_PPV_ARGS(&rtctx->capture_item)); + if (FAILED(hr)) { + av_log(avctx, AV_LOG_ERROR, "Failed to setup graphics capture for monitor\n"); + return AVERROR_EXTERNAL; + } + } else if (ctx->capture_hwnd) { + HRESULT hr = capture_item_interop->CreateForWindow(ctx->capture_hwnd, IID_PPV_ARGS(&rtctx->capture_item)); + if (FAILED(hr)) { + av_log(avctx, AV_LOG_ERROR, "Failed to setup graphics capture for window\n"); + return AVERROR_EXTERNAL; + } + } + + ret = rt_setup_gfxcapture_session(avctx); + if (ret < 0) { + av_log(avctx, AV_LOG_ERROR, "Failed to setup graphics capture pool\n"); + return ret; + } + + if (FAILED(ctx->rt->capture_session->StartCapture())) { + av_log(avctx, AV_LOG_ERROR, "Failed to start graphics capture session\n"); + ret = AVERROR_EXTERNAL; + return ret; + } + + return 0; +} + +static int rt_try_get_next_frame(AVFilterContext *avctx, ComPtr *capture_frame) +{ + GfxCaptureContext *cctx = CCTX(avctx->priv); + GfxCaptureContextCpp *ctx = cctx->ctx; + std::unique_ptr &rtctx = ctx->rt; + + ComPtr capture_surface; + ComPtr dxgi_interface_access; + ComPtr frame_texture; + SizeInt32 frame_size = { 0, 0 }; + + CHECK_HR_RET(rtctx->frame_pool->TryGetNextFrame(capture_frame->ReleaseAndGetAddressOf())); + if (!capture_frame->Get()) + return AVERROR(EAGAIN); + + CHECK_HR_RET(capture_frame->Get()->get_ContentSize(&frame_size)); + if (frame_size.Width != rtctx->cap_size.Width || frame_size.Height != rtctx->cap_size.Height) { + av_log(avctx, AV_LOG_VERBOSE, "Capture size changed to %dx%d\n", frame_size.Width, frame_size.Height); + + DirectXPixelFormat fmt = DirectXPixelFormat::DirectXPixelFormat_B8G8R8A8UIntNormalized; + if (cctx->out_fmt != AV_PIX_FMT_BGRA) + fmt = DirectXPixelFormat::DirectXPixelFormat_R16G16B16A16Float; + + CHECK_HR_RET(rtctx->frame_pool->Recreate(rtctx->d3d_device.Get(), fmt, CAPTURE_POOL_SIZE, frame_size)); + rtctx->cap_size = frame_size; + } + + return 0; +} + +static int rt_setup_winrt(AVFilterContext *avctx) +{ + GfxCaptureContext *cctx = CCTX(avctx->priv); + GfxCaptureContextCpp *ctx = cctx->ctx; + std::unique_ptr &rtctx = ctx->rt; + MSG msg; + + // pre-create the message-queue + PeekMessage(&msg, nullptr, 0, 0, PM_NOREMOVE); + + DispatcherQueueOptions options { + .dwSize = sizeof(DispatcherQueueOptions), + .threadType = DISPATCHERQUEUE_THREAD_TYPE::DQTYPE_THREAD_CURRENT, + .apartmentType = DISPATCHERQUEUE_THREAD_APARTMENTTYPE::DQTAT_COM_NONE + }; + CHECK_HR_RET(ctx->fn.CreateDispatcherQueueController(options, &rtctx->dispatcher_queue_controller)); + CHECK_HR_RET(rtctx->dispatcher_queue_controller->get_DispatcherQueue(&rtctx->dispatcher_queue)); + + return 0; +} + +static void rt_thread_uninit(AVFilterContext *avctx) +{ + GfxCaptureContext *cctx = CCTX(avctx->priv); + GfxCaptureContextCpp *ctx = cctx->ctx; + + rt_stop_capture_session(avctx); + + ctx->rt.reset(); + ctx->fn.RoUninitialize(); +} + +static int rt_thread_init(AVFilterContext *avctx) +{ + GfxCaptureContext *cctx = CCTX(avctx->priv); + GfxCaptureContextCpp *ctx = cctx->ctx; + HRESULT hr; + int ret; + + ctx->rt = std::make_unique(); + if (!ctx->rt) + return AVERROR(ENOMEM); + + ctx->fn.SetThreadDpiAwarenessContext(DPI_AWARENESS_CONTEXT_PER_MONITOR_AWARE_V2); + + hr = ctx->fn.RoInitialize(RO_INIT_MULTITHREADED); + if (FAILED(hr)) { + av_log(avctx, AV_LOG_ERROR, "Failed to initialize WinRT\n"); + ctx->rt.reset(); + return AVERROR_EXTERNAL; + } + + ret = rt_setup_winrt(avctx); + if (ret < 0) { + av_log(avctx, AV_LOG_ERROR, "Failed to setup WinRT\n"); + goto fail; + } + + ret = rt_setup_gfxcapture_capture(avctx); + if (ret < 0) { + av_log(avctx, AV_LOG_ERROR, "Failed to setup graphics capture\n"); + goto fail; + } + + return 0; + +fail: + rt_thread_uninit(avctx); + return ret; +} + +static int rt_thread_worker(AVFilterContext *avctx) +{ + GfxCaptureContext *cctx = CCTX(avctx->priv); + GfxCaptureContextCpp *ctx = cctx->ctx; + std::unique_ptr &rtctx = ctx->rt; + ComPtr async; + MSG msg; + + av_log(avctx, AV_LOG_DEBUG, "Starting message loop\n"); + + while (BOOL res = GetMessage(&msg, NULL, 0, 0)) { + if (res == -1) { + av_log(avctx, AV_LOG_ERROR, "Failed to get message\n"); + return AVERROR(EIO); + } + + if (!msg.hwnd && msg.message == WM_RT_THREAD_SHUTDOWN) { + av_log(avctx, AV_LOG_DEBUG, "Initializing RT thread shutdown\n"); + if (FAILED(rtctx->dispatcher_queue_controller->ShutdownQueueAsync(&async))) { + av_log(avctx, AV_LOG_ERROR, "Failed to shutdown dispatcher queue\n"); + return AVERROR_EXTERNAL; + } + async->put_Completed(create_cb_handler( + [avctx, ctx](auto, auto status) { + PostThreadMessage(ctx->rt_thread_id, WM_QUIT, 0, 0); + av_log(avctx, AV_LOG_DEBUG, "RT thread async shutdown completed: %d\n", (int)status); + return S_OK; + }).Get()); + continue; + } + + av_log(avctx, AV_LOG_TRACE, "Got message: %u\n", msg.message); + + TranslateMessage(&msg); + DispatchMessage(&msg); + } + + if (!async) { + av_log(avctx, AV_LOG_ERROR, "RT Thread message loop ended without proper shutdown\n"); + return AVERROR_EXTERNAL; + } + + av_log(avctx, AV_LOG_DEBUG, "Message loop ended\n"); + + return msg.wParam; +} + +static void rt_thread_entry(AVFilterContext *avctx) +{ + GfxCaptureContext *cctx = CCTX(avctx->priv); + GfxCaptureContextCpp *ctx = cctx->ctx; + + std::lock_guard rt_lock(ctx->rt_thread_lock); + + { + std::lock_guard init_lock(ctx->rt_thread_init_lock); + ctx->rt_thread_id = GetCurrentThreadId(); + ctx->rt_thread_init_res = rt_thread_init(avctx); + ctx->rt_thread_init_cond.notify_all(); + if (ctx->rt_thread_init_res < 0) + return; + } + + ctx->rt_thread_res = rt_thread_worker(avctx); + + std::lock_guard uninit_lock(ctx->rt_thread_init_lock); + rt_thread_uninit(avctx); +} + +/********************************** + * RT Thread Management Functions * + **********************************/ + +static void stop_rt_thread(AVFilterContext *avctx) +{ + GfxCaptureContext *cctx = CCTX(avctx->priv); + GfxCaptureContextCpp *ctx = cctx->ctx; + + if (ctx->rt_thread.joinable()) { + if (!PostThreadMessage(ctx->rt_thread_id, WM_RT_THREAD_SHUTDOWN, 0, 0)) { + av_log(avctx, AV_LOG_ERROR, "Failed to post shutdown message to RT thread\n"); + return; + } + ctx->rt_thread.join(); + } +} + +static int start_rt_thread(AVFilterContext *avctx) +{ + GfxCaptureContext *cctx = CCTX(avctx->priv); + GfxCaptureContextCpp *ctx = cctx->ctx; + + std::unique_lock rt_lock(ctx->rt_thread_init_lock); + ctx->rt_thread_init_res = INT_MAX; + + ctx->rt_thread = std::thread(rt_thread_entry, avctx); + if (!ctx->rt_thread.joinable()) + return AVERROR(ENOMEM); + + if (!ctx->rt_thread_init_cond.wait_for(rt_lock, std::chrono::seconds(1), [&]() { + return ctx->rt_thread_init_res != INT_MAX; + })) { + av_log(avctx, AV_LOG_ERROR, "RT thread init timed out\n"); + return AVERROR(ETIMEDOUT); + } + + return ctx->rt_thread_init_res; +} + +template +static int run_on_rt_thread(AVFilterContext *avctx, F &&cb) +{ + GfxCaptureContext *cctx = CCTX(avctx->priv); + GfxCaptureContextCpp *ctx = cctx->ctx; + std::unique_ptr &rtctx = ctx->rt; + + std::lock_guard rt_lock(ctx->rt_thread_init_lock); + + if (!rtctx) { + av_log(avctx, AV_LOG_ERROR, "RT thread not initialized\n"); + return AVERROR(ENOSYS); + } + + struct CBData { + std::mutex mutex; + std::condition_variable cond; + std::atomic done { false }; + std::atomic cancel { false }; + int ret = AVERROR_BUG; + }; + auto cbdata = std::make_shared(); + + std::unique_lock cblock(cbdata->mutex); + + boolean res = 0; + CHECK_HR_RET(rtctx->dispatcher_queue->TryEnqueue( + create_cb_handler( + [cb = std::forward(cb), cbdata]() { + { + std::lock_guard lock(cbdata->mutex); + if (cbdata->cancel.load(std::memory_order_acquire)) + return S_OK; + cbdata->ret = cb(); + cbdata->done.store(true, std::memory_order_release); + } + + cbdata->cond.notify_one(); + return S_OK; + }).Get(), &res)); + if (!res) { + av_log(avctx, AV_LOG_ERROR, "Failed to enqueue RT thread callback\n"); + return AVERROR_EXTERNAL; + } + + if (!cbdata->cond.wait_for(cblock, std::chrono::seconds(1), [&]() { return cbdata->done.load(std::memory_order_acquire); })) { + cbdata->cancel.store(true, std::memory_order_release); + av_log(avctx, AV_LOG_ERROR, "RT thread callback timed out\n"); + return AVERROR(ETIMEDOUT); + } + + return cbdata->ret; +} + +/******************************* + * Standard AVFilter functions * + *******************************/ + +static int build_regex(AVFilterContext *avctx, const char *pattern, std::regex *out) +{ + if (!pattern) + return 0; + + std::string pat(pattern); + + auto flags = std::regex::ECMAScript | std::regex::optimize; + if (pat.rfind("(?i)", 0) == 0 || pat.rfind("(?I)", 0) == 0) { + pat.erase(0, 4); + flags |= std::regex::icase; + } else if(pat.rfind("(?c)", 0) == 0 || pat.rfind("(?C)", 0) == 0) { + pat.erase(0, 4); + } + + try { + *out = std::regex(pat, flags); + } catch (const std::regex_error &e) { + av_log(avctx, AV_LOG_ERROR, "Failed to compile regex '%s': %s\n", pat.c_str(), e.what()); + return AVERROR(EINVAL); + } + + av_log(avctx, AV_LOG_DEBUG, "Built regex: %s\n", pattern); + + return 0; +} + +static int get_window_exe_name(HWND hwnd, std::string *out) +{ + out->clear(); + + DWORD pid = 0; + if (!GetWindowThreadProcessId(hwnd, &pid)) + return AVERROR(ENOENT); + + HANDLE proc = OpenProcess(PROCESS_QUERY_LIMITED_INFORMATION, FALSE, pid); + if (!proc) + return AVERROR(EACCES); + + std::wstring image_name; + DWORD image_name_size = 512; + int ret = 0; + + for (;;) { + DWORD len = image_name_size; + image_name.resize(len); + if (QueryFullProcessImageNameW(proc, 0, image_name.data(), &len)) { + image_name.resize(len); + break; + } + if (GetLastError() == ERROR_INSUFFICIENT_BUFFER) { + image_name_size *= 2; + continue; + } + ret = AVERROR_EXTERNAL; + break; + } + CloseHandle(proc); + if (ret < 0) + return ret; + if (image_name.empty()) + return AVERROR_EXTERNAL; + + const wchar_t *base = image_name.c_str(); + size_t pos = image_name.find_last_of(L"\\/"); + if (pos != std::string::npos) + base += pos + 1; + + int utf8size = WideCharToMultiByte(CP_UTF8, 0, base, -1, nullptr, 0, nullptr, nullptr); + if (utf8size <= 0) + return AVERROR(EINVAL); + + out->resize(utf8size - 1); + WideCharToMultiByte(CP_UTF8, 0, base, -1, out->data(), utf8size, nullptr, nullptr); + + return 0; +} + +static int find_capture_source(AVFilterContext *avctx) +{ + GfxCaptureContext *cctx = CCTX(avctx->priv); + GfxCaptureContextCpp *ctx = cctx->ctx; + int cur_idx = 0; + + ctx->capture_hwnd = NULL; + ctx->capture_hmonitor = NULL; + + if (cctx->user_hmonitor) { + ctx->capture_hmonitor = (HMONITOR)(uintptr_t)cctx->user_hmonitor; + return 0; + } else if (cctx->user_hwnd) { + ctx->capture_hwnd = (HWND)(uintptr_t)cctx->user_hwnd; + return 0; + } else if (cctx->monitor_idx >= 0) { + auto cb = make_win32_callback([&](HMONITOR hmonitor, HDC, LPRECT) { + if (cur_idx++ == cctx->monitor_idx) { + av_log(avctx, AV_LOG_DEBUG, "Found capture monitor: %d\n", cctx->monitor_idx); + ctx->capture_hmonitor = hmonitor; + return FALSE; + } + return TRUE; + }); + if (EnumDisplayMonitors(NULL, NULL, cb->proc, cb->lparam) || !ctx->capture_hmonitor) + return AVERROR(ENOENT); + return 0; + } else if (cctx->window_text || cctx->window_class || cctx->window_exe) { + std::regex text_regex; + if (build_regex(avctx, cctx->window_text, &text_regex) < 0) + return AVERROR(EINVAL); + + std::regex class_regex; + if (build_regex(avctx, cctx->window_class, &class_regex) < 0) + return AVERROR(EINVAL); + + std::regex exe_regex; + if (build_regex(avctx, cctx->window_exe, &exe_regex) < 0) + return AVERROR(EINVAL); + + std::string window_text; + std::string window_class; + std::string window_exe; + auto cb = make_win32_callback([&](HWND hwnd) { + RECT r = { 0 }; + if (!GetWindowRect(hwnd, &r) || r.right <= r.left || r.bottom <= r.top || !IsWindowVisible(hwnd)) + return TRUE; + + window_text.resize(GetWindowTextLengthA(hwnd) + 1); + int len = GetWindowTextA(hwnd, window_text.data(), window_text.size()); + if (len >= 0) + window_text.resize(len); + else + window_text.clear(); + + window_class.resize(512); + len = GetClassNameA(hwnd, window_class.data(), window_class.size()); + if (len >= 0) + window_class.resize(len); + else + window_class.clear(); + + get_window_exe_name(hwnd, &window_exe); + + av_log(avctx, AV_LOG_TRACE, "Checking window: hwnd=%p text=%s class=%s exe=%s\n", + hwnd, window_text.c_str(), window_class.c_str(), window_exe.c_str()); + + if (cctx->window_text) { + if (window_text.empty() || !std::regex_search(window_text, text_regex)) + return TRUE; + } + + if (cctx->window_class) { + if (window_class.empty() || !std::regex_search(window_class, class_regex)) + return TRUE; + } + + if (cctx->window_exe) { + if (window_exe.empty() || !std::regex_search(window_exe, exe_regex)) + return TRUE; + } + + av_log(avctx, AV_LOG_VERBOSE, "Found capture window: %s (Class: %s, Exe: %s)\n", + window_text.c_str(), window_class.c_str(), window_exe.c_str()); + ctx->capture_hwnd = hwnd; + return FALSE; + }); + if (EnumWindows(cb->proc, cb->lparam) || !ctx->capture_hwnd) + return AVERROR(ENOENT); + return 0; + } + + av_log(avctx, AV_LOG_ERROR, "No capture source specified\n"); + return AVERROR(EINVAL); +} + +av_cold void ff_gfxcapture_uninit(AVFilterContext *avctx) +{ + GfxCaptureContext *cctx = CCTX(avctx->priv); + GfxCaptureContextCpp *ctx = cctx->ctx; + + if (!ctx) + return; + + stop_rt_thread(avctx); + + ctx->vertex_shader = nullptr; + ctx->pixel_shader = nullptr; + ctx->sampler_state = nullptr; + ctx->deferred_ctx = nullptr; + ctx->shader_cb = nullptr; + + av_buffer_unref(&ctx->frames_ref); + av_buffer_unref(&ctx->device_ref); + + delete ctx; +} + +template +static av_cold void GetProcAddressTyped(const hmodule_ptr_t &hModule, LPCSTR lpProcName, T *out) { + *out = reinterpret_cast(GetProcAddress(hModule.get(), lpProcName)); +} + +static av_cold int load_functions(AVFilterContext *avctx) +{ + GfxCaptureContext *cctx = CCTX(avctx->priv); + GfxCaptureContextCpp *ctx = cctx->ctx; + +#define LOAD_DLL(handle, name) \ + handle = std::move(hmodule_ptr_t(LoadLibraryExA(name, NULL, LOAD_LIBRARY_SEARCH_SYSTEM32))); \ + if (!handle) { \ + av_log(avctx, AV_LOG_ERROR, "Failed opening " #name "\n"); \ + return AVERROR(ENOSYS); \ + } + +#define LOAD_FUNC(handle, name) \ + GetProcAddressTyped(handle, #name, &ctx->fn.name); \ + if (!ctx->fn.name) { \ + av_log(avctx, AV_LOG_ERROR, "Failed loading " #name "\n"); \ + return AVERROR(ENOSYS); \ + } + + // this handle is not used anywhere, but letting it get auto-freed during RoUninit causes crashes + LOAD_DLL(ctx->fn.graphicscapture_handle, "graphicscapture.dll"); + + LOAD_DLL(ctx->fn.combase_handle, "combase.dll"); + LOAD_DLL(ctx->fn.dwmapi_handle, "dwmapi.dll"); + LOAD_DLL(ctx->fn.d3d11_handle, "d3d11.dll"); + LOAD_DLL(ctx->fn.coremsg_handle, "coremessaging.dll"); + LOAD_DLL(ctx->fn.user32_handle, "user32.dll"); + LOAD_DLL(ctx->fn.d3dcompiler_handle, "d3dcompiler_47.dll"); + + LOAD_FUNC(ctx->fn.combase_handle, RoInitialize); + LOAD_FUNC(ctx->fn.combase_handle, RoUninitialize); + LOAD_FUNC(ctx->fn.combase_handle, RoGetActivationFactory); + LOAD_FUNC(ctx->fn.combase_handle, WindowsCreateStringReference); + + LOAD_FUNC(ctx->fn.dwmapi_handle, DwmGetWindowAttribute); + + LOAD_FUNC(ctx->fn.d3d11_handle, CreateDirect3D11DeviceFromDXGIDevice); + + LOAD_FUNC(ctx->fn.coremsg_handle, CreateDispatcherQueueController); + + LOAD_FUNC(ctx->fn.user32_handle, SetThreadDpiAwarenessContext); + + LOAD_FUNC(ctx->fn.d3dcompiler_handle, D3DCompile); + +#undef LOAD_FUNC +#undef LOAD_DLL + return 0; +} + +av_cold int ff_gfxcapture_init(AVFilterContext *avctx) +{ + GfxCaptureContext *cctx = CCTX(avctx->priv); + int ret = 0; + + GfxCaptureContextCpp *ctx = cctx->ctx = new GfxCaptureContextCpp(); + if (!ctx) + return AVERROR(ENOMEM); + + ret = load_functions(avctx); + if (ret < 0) { + ctx->fn.RoUninitialize = nullptr; + goto fail; + } + + return 0; + +fail: + ff_gfxcapture_uninit(avctx); + return ret; +} + +static int init_hwframes_ctx(AVFilterContext *avctx) +{ + GfxCaptureContext *cctx = CCTX(avctx->priv); + GfxCaptureContextCpp *ctx = cctx->ctx; + int ret = 0; + + ctx->frames_ref = av_hwframe_ctx_alloc(ctx->device_ref); + if (!ctx->frames_ref) + return AVERROR(ENOMEM); + ctx->frames_ctx = (AVHWFramesContext*)ctx->frames_ref->data; + ctx->frames_hwctx = (AVD3D11VAFramesContext*)ctx->frames_ctx->hwctx; + + ctx->frames_ctx->format = AV_PIX_FMT_D3D11; + ctx->frames_ctx->width = cctx->canvas_width; + ctx->frames_ctx->height = cctx->canvas_height; + ctx->frames_ctx->sw_format = (AVPixelFormat)cctx->out_fmt; + + ctx->frames_hwctx->BindFlags = D3D11_BIND_RENDER_TARGET; + + ret = av_hwframe_ctx_init(ctx->frames_ref); + if (ret < 0) { + av_log(avctx, AV_LOG_ERROR, "Failed to initialise hardware frames context: %d.\n", ret); + goto fail; + } + + return 0; +fail: + av_buffer_unref(&ctx->frames_ref); + return ret; +} + +static int setup_gfxcapture_capture(AVFilterContext *avctx) +{ + GfxCaptureContext *cctx = CCTX(avctx->priv); + GfxCaptureContextCpp *ctx = cctx->ctx; + std::unique_ptr &rtctx = ctx->rt; + int ret = 0; + + stop_rt_thread(avctx); + + ret = find_capture_source(avctx); + if (ret < 0) { + av_log(avctx, AV_LOG_ERROR, "Failed to find capture source\n"); + return ret; + } + + ret = start_rt_thread(avctx); + if (ret < 0) { + av_log(avctx, AV_LOG_ERROR, "Failed to start graphics capture thread\n"); + return ret; + } + + if (cctx->canvas_width == 0) + cctx->canvas_width = rtctx->cap_size.Width; + if (cctx->canvas_height == 0) + cctx->canvas_height = rtctx->cap_size.Height; + + return 0; +} + +static const char render_shader_src[] = HLSL( + struct VSOut { + float4 pos : SV_Position; + float2 uv : TEXCOORD0; + }; + + VSOut main_vs(uint id : SV_VertexID) { + float2 p = float2(id == 2 ? 3.0 : -1.0, id == 1 ? 3.0 : -1.0); + VSOut o; + o.pos = float4(p, 0, 1); + o.uv = float2((p.x + 1) * 0.5, 1 - (p.y + 1) * 0.5); + return o; + } + + Texture2D t0 : register(t0); + SamplerState s0 : register(s0); + + cbuffer cb : register(b0) { + float2 tS; + float2 dS; + uint to_unpremult; + uint to_srgb; + uint2 pad; + }; + + float4 cubic(float v) { + float4 n = float4(1.0, 2.0, 3.0, 4.0) - v; + float4 s = n * n * n; + float x = s.x; + float y = s.y - 4.0 * s.x; + float z = s.z - 4.0 * s.y + 6.0 * s.x; + float w = 6.0 - x - y - z; + return float4(x, y, z, w) * (1.0 / 6.0); + } + + float4 texBicubic(Texture2D t, SamplerState ss, float2 uv) { + float2 itS = 1.0 / tS; + + float2 tc = uv * tS - 0.5; + float2 fxy = frac(tc); + tc -= fxy; + + float4 xc = cubic(fxy.x); + float4 yc = cubic(fxy.y); + + float4 s = float4(xc.xz + xc.yw, yc.xz + yc.yw); + float4 o = tc.xxyy + (float2(-0.5, 1.5)).xyxy + float4(xc.yw, yc.yw) / s; + o *= itS.xxyy; + + float4 s0 = t.Sample(ss, o.xz); + float4 s1 = t.Sample(ss, o.yz); + float4 s2 = t.Sample(ss, o.xw); + float4 s3 = t.Sample(ss, o.yw); + + float sx = s.x / (s.x + s.y); + float sy = s.z / (s.z + s.w); + + return lerp(lerp(s3, s2, sx), lerp(s1, s0, sx), sy); + } + + float4 unpremultiply(float4 c) { + if (c.a < 1e-6) + return float4(0.0, 0.0, 0.0, 0.0); + return float4(c.rgb / c.a, c.a); + } + + float4 premultiply(float4 c) { + return float4(c.rgb * c.a, c.a); + } + + float3 linear_to_srgb(float3 c) { + c = max(c, 0.0); + float3 lo = 12.92 * c; + float3 hi = 1.055 * pow(c, 1.0 / 2.4) - 0.055; + return saturate(lerp(hi, lo, step(c, 0.0031308))); + } + + float4 fix_color(float4 c) { + if (to_unpremult || to_srgb) + c = unpremultiply(c); + if (to_srgb) { + c.rgb = linear_to_srgb(c.rgb); + if (!to_unpremult) + c = premultiply(c); + } + return c; + } + + float4 main_ps(VSOut i) : SV_Target { + return fix_color(t0.Sample(s0, i.uv)); + } + + float4 main_ps_bicubic(VSOut i) : SV_Target { + if (all(tS == dS)) + return main_ps(i); + return fix_color(texBicubic(t0, s0, i.uv)); + } +); + +static int prepare_render_resources(AVFilterContext *avctx) +{ + GfxCaptureContext *cctx = CCTX(avctx->priv); + GfxCaptureContextCpp *ctx = cctx->ctx; + HRESULT hr; + + ComPtr vs_blob, ps_blob, err_blob; + CD3D11_SAMPLER_DESC sampler_desc(CD3D11_DEFAULT{}); + UINT flags = D3DCOMPILE_OPTIMIZATION_LEVEL3; + + hr = ctx->fn.D3DCompile(render_shader_src, sizeof(render_shader_src) - 1, NULL, NULL, NULL, "main_vs", "vs_4_0", flags, 0, &vs_blob, &err_blob); + if (FAILED(hr)) { + if (err_blob) { + av_log(avctx, AV_LOG_ERROR, "Failed compiling vertex shader: %.*s\n", (int)err_blob->GetBufferSize(), (char*)err_blob->GetBufferPointer()); + } else { + av_log(avctx, AV_LOG_ERROR, "Failed compiling vertex shader: 0x%08lX\n", hr); + } + return AVERROR_EXTERNAL; + } + + const char *ps_entry = "main_ps_bicubic"; + if (cctx->resize_mode == GFX_RESIZE_CROP || cctx->scale_mode == GFX_SCALE_POINT) { + ps_entry = "main_ps"; + sampler_desc.Filter = D3D11_FILTER_MIN_MAG_MIP_POINT; + } + + hr = ctx->fn.D3DCompile(render_shader_src, sizeof(render_shader_src) - 1, NULL, NULL, NULL, ps_entry, "ps_4_0", flags, 0, &ps_blob, &err_blob); + if (FAILED(hr)) { + if (err_blob) { + av_log(avctx, AV_LOG_ERROR, "Failed compiling pixel shader: %.*s\n", (int)err_blob->GetBufferSize(), (char*)err_blob->GetBufferPointer()); + } else { + av_log(avctx, AV_LOG_ERROR, "Failed compiling pixel shader: 0x%08lX\n", hr); + } + return AVERROR_EXTERNAL; + } + + CHECK_HR_RET(ctx->device_hwctx->device->CreateVertexShader(vs_blob->GetBufferPointer(), vs_blob->GetBufferSize(), NULL, &ctx->vertex_shader)); + CHECK_HR_RET(ctx->device_hwctx->device->CreatePixelShader(ps_blob->GetBufferPointer(), ps_blob->GetBufferSize(), NULL, &ctx->pixel_shader)); + + CHECK_HR_RET(ctx->device_hwctx->device->CreateSamplerState(&sampler_desc, &ctx->sampler_state)); + + D3D11_BUFFER_DESC cb_desc = { + .ByteWidth = 32, + .Usage = D3D11_USAGE_DYNAMIC, + .BindFlags = D3D11_BIND_CONSTANT_BUFFER, + .CPUAccessFlags = D3D11_CPU_ACCESS_WRITE + }; + CHECK_HR_RET(ctx->device_hwctx->device->CreateBuffer(&cb_desc, NULL, &ctx->shader_cb)); + + CHECK_HR_RET(ctx->device_hwctx->device->CreateDeferredContext(0, &ctx->deferred_ctx)); + + return 0; +} + +int ff_gfxcapture_config_props(AVFilterLink *outlink) +{ + AVFilterContext *avctx = outlink->src; + GfxCaptureContext *cctx = CCTX(avctx->priv); + GfxCaptureContextCpp *ctx = cctx->ctx; + + FilterLink *link = ff_filter_link(outlink); + int ret; + + if (avctx->hw_device_ctx) { + ctx->device_ctx = (AVHWDeviceContext*)avctx->hw_device_ctx->data; + + if (ctx->device_ctx->type != AV_HWDEVICE_TYPE_D3D11VA) { + av_log(avctx, AV_LOG_ERROR, "Non-D3D11VA input hw_device_ctx\n"); + return AVERROR(EINVAL); + } + + ctx->device_ref = av_buffer_ref(avctx->hw_device_ctx); + if (!ctx->device_ref) + return AVERROR(ENOMEM); + + av_log(avctx, AV_LOG_VERBOSE, "Using provided hw_device_ctx\n"); + } else { + ret = av_hwdevice_ctx_create(&ctx->device_ref, AV_HWDEVICE_TYPE_D3D11VA, NULL, NULL, 0); + if (ret < 0) { + av_log(avctx, AV_LOG_ERROR, "Failed to create D3D11VA device.\n"); + return ret; + } + + ctx->device_ctx = (AVHWDeviceContext*)ctx->device_ref->data; + + av_log(avctx, AV_LOG_VERBOSE, "Created internal hw_device_ctx\n"); + } + + ctx->device_hwctx = (AVD3D11VADeviceContext*)ctx->device_ctx->hwctx; + + ret = prepare_render_resources(avctx); + if (ret < 0) { + av_log(avctx, AV_LOG_ERROR, "Failed to prepare render resources\n"); + return ret; + } + + ret = setup_gfxcapture_capture(avctx); + if (ret < 0) { + av_log(avctx, AV_LOG_ERROR, "Failed to setup graphics capture\n"); + return ret; + } + + ret = init_hwframes_ctx(avctx); + if (ret < 0) + return ret; + + link->hw_frames_ctx = av_buffer_ref(ctx->frames_ref); + if (!link->hw_frames_ctx) + return AVERROR(ENOMEM); + + outlink->w = ctx->rt->cap_size.Width; + outlink->h = ctx->rt->cap_size.Height; + outlink->time_base = (AVRational){1, TIMESPAN_RES}; + outlink->alpha_mode = cctx->premult_alpha ? AVALPHA_MODE_PREMULTIPLIED : AVALPHA_MODE_STRAIGHT; + link->frame_rate = cctx->frame_rate; + + av_log(avctx, AV_LOG_DEBUG, "Capture setup with res %dx%d\n", outlink->w, outlink->h); + + return 0; +} + +static int render_capture_to_frame(AVFilterContext *avctx, AVFrame *frame, const ComPtr &src_tex) +{ + GfxCaptureContext *cctx = CCTX(avctx->priv); + GfxCaptureContextCpp *ctx = cctx->ctx; + ID3D11Device *dev = ctx->device_hwctx->device; + ID3D11DeviceContext *dev_ctx = ctx->device_hwctx->device_context; + ID3D11DeviceContext *def_ctx = ctx->deferred_ctx.Get(); + + D3D11_TEXTURE2D_DESC dst_tex_desc; + reinterpret_cast(frame->data[0])->GetDesc(&dst_tex_desc); + if (dst_tex_desc.ArraySize != 1) { + av_log(avctx, AV_LOG_ERROR, "Rendering to array texture not implemented\n"); + return AVERROR(EINVAL); + } + + D3D11_TEXTURE2D_DESC src_tex_desc; + src_tex->GetDesc(&src_tex_desc); + + D3D11_RENDER_TARGET_VIEW_DESC target_desc = { + .Format = dst_tex_desc.Format, + .ViewDimension = D3D11_RTV_DIMENSION_TEXTURE2D, + .Texture2D = { .MipSlice = 0 } + }; + + ComPtr rtv; + CHECK_HR_RET(dev->CreateRenderTargetView( + reinterpret_cast(frame->data[0]), &target_desc, &rtv)); + + ComPtr srv; + CHECK_HR_RET(dev->CreateShaderResourceView(src_tex.Get(), nullptr, &srv)); + + D3D11_MAPPED_SUBRESOURCE map; + CHECK_HR_RET(def_ctx->Map(ctx->shader_cb.Get(), 0, D3D11_MAP_WRITE_DISCARD, 0, &map)); + float *cb_data_f = static_cast(map.pData); + uint32_t *cb_data_u = static_cast(map.pData); + cb_data_f[0] = (float)src_tex_desc.Width; + cb_data_f[1] = (float)src_tex_desc.Height; + cb_data_f[2] = (float)dst_tex_desc.Width; + cb_data_f[3] = (float)dst_tex_desc.Height; + cb_data_u[4] = !cctx->premult_alpha; // to_unpremult + cb_data_u[5] = src_tex_desc.Format == DXGI_FORMAT_R16G16B16A16_FLOAT && dst_tex_desc.Format != DXGI_FORMAT_R16G16B16A16_FLOAT; // to_srgb + cb_data_u[6] = 0; + cb_data_u[7] = 0; + def_ctx->Unmap(ctx->shader_cb.Get(), 0); + + def_ctx->OMSetRenderTargets(1, rtv.GetAddressOf(), nullptr); + def_ctx->ClearRenderTargetView(rtv.Get(), (const float[4]){0.f, 0.f, 0.f, 1.f}); + + D3D11_VIEWPORT viewport = { + .MinDepth = 0.f, + .MaxDepth = 1.f + }; + + switch (cctx->resize_mode) { + case GFX_RESIZE_CROP: + viewport.Width = src_tex_desc.Width; + viewport.Height = src_tex_desc.Height; + break; + case GFX_RESIZE_SCALE: + viewport.Width = dst_tex_desc.Width; + viewport.Height = dst_tex_desc.Height; + break; + case GFX_RESIZE_SCALE_ASPECT: { + float scale = FFMIN(dst_tex_desc.Width / (float)src_tex_desc.Width, + dst_tex_desc.Height / (float)src_tex_desc.Height); + viewport.Width = src_tex_desc.Width * scale; + viewport.Height = src_tex_desc.Height * scale; + break; + } + default: + av_log(avctx, AV_LOG_ERROR, "Invalid scaling mode\n"); + return AVERROR_BUG; + }; + + def_ctx->RSSetViewports(1, &viewport); + + def_ctx->IASetPrimitiveTopology(D3D11_PRIMITIVE_TOPOLOGY_TRIANGLELIST); + def_ctx->VSSetShader(ctx->vertex_shader.Get(), nullptr, 0); + def_ctx->PSSetShader(ctx->pixel_shader.Get(), nullptr, 0); + def_ctx->PSSetSamplers(0, 1, ctx->sampler_state.GetAddressOf()); + def_ctx->PSSetShaderResources(0, 1, srv.GetAddressOf()); + def_ctx->PSSetConstantBuffers(0, 1, ctx->shader_cb.GetAddressOf()); + + def_ctx->Draw(3, 0); + + ComPtr cmd_list; + CHECK_HR_RET(def_ctx->FinishCommandList(FALSE, &cmd_list)); + dev_ctx->ExecuteCommandList(cmd_list.Get(), FALSE); + + return 0; +} + +static int process_frame_if_exists(AVFilterLink *outlink) +{ + AVFilterContext *avctx = outlink->src; + GfxCaptureContext *cctx = CCTX(avctx->priv); + GfxCaptureContextCpp *ctx = cctx->ctx; + int ret; + + AVFrame *frame = nullptr; + + ret = run_on_rt_thread(avctx, [&]() { + ComPtr capture_frame; + ComPtr capture_surface; + ComPtr dxgi_interface_access; + ComPtr frame_texture; + TimeSpan frame_time = { 0 }; + + ret = rt_try_get_next_frame(avctx, &capture_frame); + if (ret < 0) + return ret; + + CHECK_HR_RET(capture_frame->get_SystemRelativeTime(&frame_time)); + + CHECK_HR_RET(capture_frame->get_Surface(&capture_surface)); + CHECK_HR_RET(capture_surface.As(&dxgi_interface_access)); + CHECK_HR_RET(dxgi_interface_access->GetInterface(IID_PPV_ARGS(&frame_texture))); + + if (!frame_texture) + return AVERROR(EAGAIN); + + frame = ff_get_video_buffer(outlink, cctx->canvas_width, cctx->canvas_height); + if (!frame) + return AVERROR(ENOMEM); + + frame->pts = frame_time.Duration; + + return render_capture_to_frame(avctx, frame, frame_texture); + }); + if (ret < 0) + return ret; + + frame->sample_aspect_ratio = (AVRational){1, 1}; + + if (ctx->frames_ctx->sw_format == AV_PIX_FMT_RGBAF16) { + // According to MSDN, all floating point formats contain sRGB image data with linear 1.0 gamma. + frame->color_range = AVCOL_RANGE_JPEG; + frame->color_primaries = AVCOL_PRI_BT709; + frame->color_trc = AVCOL_TRC_LINEAR; + frame->colorspace = AVCOL_SPC_RGB; + } else { + // According to MSDN, all integer formats contain sRGB image data + frame->color_range = AVCOL_RANGE_JPEG; + frame->color_primaries = AVCOL_PRI_BT709; + frame->color_trc = AVCOL_TRC_IEC61966_2_1; + frame->colorspace = AVCOL_SPC_RGB; + } + + ctx->last_pts = frame->pts; + + if (!ctx->first_pts) + ctx->first_pts = frame->pts; + frame->pts -= ctx->first_pts; + + return ff_filter_frame(outlink, frame); +} + +int ff_gfxcapture_activate(AVFilterContext *avctx) +{ + AVFilterLink *outlink = avctx->outputs[0]; + GfxCaptureContext *cctx = CCTX(avctx->priv); + GfxCaptureContextCpp *ctx = cctx->ctx; + std::unique_ptr &rtctx = ctx->rt; + + std::lock_guard rtlock(ctx->rt_thread_init_lock); + if (!rtctx) { + av_log(avctx, AV_LOG_ERROR, "RT thread not initialized\n"); + return AVERROR(ENOSYS); + } + + if (!ff_outlink_frame_wanted(outlink)) + return FFERROR_NOT_READY; + + std::unique_lock frame_lock(rtctx->frame_arrived_lock); + + for (;;) { + uint64_t last_seq = rtctx->frame_seq.load(std::memory_order_acquire); + + int ret = process_frame_if_exists(outlink); + if (ret != AVERROR(EAGAIN)) + return ret; + + if (rtctx->window_closed.load(std::memory_order_acquire)) { + ff_outlink_set_status(outlink, AVERROR_EOF, ctx->last_pts - ctx->first_pts + 1); + break; + } + + if (!rtctx->frame_arrived_cond.wait_for(frame_lock, std::chrono::seconds(1), [&]() { + return rtctx->frame_seq.load(std::memory_order_acquire) != last_seq || + rtctx->window_closed.load(std::memory_order_acquire); + })) + break; + } + + return 0; +} diff --git a/libavfilter/vsrc_gfxcapture_winrt.h b/libavfilter/vsrc_gfxcapture_winrt.h new file mode 100644 index 0000000000..9a665da7df --- /dev/null +++ b/libavfilter/vsrc_gfxcapture_winrt.h @@ -0,0 +1,180 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#ifndef AVFILTER_VSRC_GFXCAPTURE_WINRT_H +#define AVFILTER_VSRC_GFXCAPTURE_WINRT_H + +// Forward-declare IDirect3DDxgiInterfaceAccess if headers too old +#if !HAVE_IDIRECT3DDXGIINTERFACEACCESS +namespace Windows::Graphics::DirectX::Direct3D11 { + MIDL_INTERFACE("A9B3D012-3DF2-4EE3-B8D1-8695F457D3C1") + IDirect3DDxgiInterfaceAccess : public IUnknown + { + public: + IFACEMETHOD(GetInterface)(REFIID iid, _COM_Outptr_ void** p) = 0; + }; +} +#ifdef __MINGW32__ +__CRT_UUID_DECL(Windows::Graphics::DirectX::Direct3D11::IDirect3DDxgiInterfaceAccess, + 0xa9b3d012, 0x3df2, 0x4ee3, 0xb8, 0xd1, 0x86, 0x95, 0xf4, 0x57, 0xd3, 0xc1) +#endif +#endif /* !HAVE_IDIRECT3DDXGIINTERFACEACCESS */ + +// Forward-declare IGraphicsCaptureSession5 if headers too old +#if !HAVE___X_ABI_CWINDOWS_CGRAPHICS_CCAPTURE_CIGRAPHICSCAPTURESESSION5 +namespace ABI::Windows ::Graphics::Capture { + MIDL_INTERFACE("67C0EA62-1F85-5061-925A-239BE0AC09CB") + IGraphicsCaptureSession5 : public IInspectable + { + public: + IFACEMETHOD(get_MinUpdateInterval)(ABI::Windows::Foundation::TimeSpan* value) = 0; + IFACEMETHOD(put_MinUpdateInterval)(ABI::Windows::Foundation::TimeSpan value) = 0; + }; +} +#ifdef __MINGW32__ +__CRT_UUID_DECL(ABI::Windows ::Graphics::Capture::IGraphicsCaptureSession5, + 0x67c0ea62, 0x1f85, 0x5061, 0x92, 0x5a, 0x23, 0x9b, 0xe0, 0xac, 0x09, 0xcb) +#endif +#endif /* !HAVE___X_ABI_CWINDOWS_CGRAPHICS_CCAPTURE_CIGRAPHICSCAPTURESESSION5 */ + +/**************************************************** + * Helper class to implement refcounted COM objects * + ****************************************************/ +template +struct FFComObject : Interfaces... +{ + virtual ~FFComObject() = default; + + HRESULT STDMETHODCALLTYPE QueryInterface(REFIID riid, void** ppvObject) override + { + if (!ppvObject) + return E_POINTER; + + if (query_all(riid, ppvObject)) + { + AddRef(); + return S_OK; + } + + *ppvObject = nullptr; + return E_NOINTERFACE; + } + + ULONG STDMETHODCALLTYPE AddRef() override + { + return ++ref_count; + } + + ULONG STDMETHODCALLTYPE Release() override + { + ULONG rc = --ref_count; + if (rc == 0) + delete this; + return rc; + } + +private: + template + bool query_all(REFIID riid, void** ppvObject) + { + if (riid == __uuidof(Iface)) { + *ppvObject = static_cast(this); + return true; + } + if constexpr (sizeof...(IFaces)) { + return query_all(riid, ppvObject); + } else if (riid == __uuidof(IUnknown)) { + *ppvObject = static_cast(static_cast(this)); + return true; + } + return false; + } + + std::atomic ref_count { 1 }; +}; + +/******************************************* + * Helper to implement COM/WinRT callbacks * + *******************************************/ +template +struct FFTypedCBHandler : FFComObject +{ + using Func = std::decay_t; + + explicit FFTypedCBHandler(F&& f) : cb_func(std::forward(f)) {} + + std::invoke_result_t STDMETHODCALLTYPE Invoke(Args... args) override + { + if constexpr (std::is_same_v, HRESULT>) { + return cb_func(std::forward(args)...); + } else { + cb_func(std::forward(args)...); + return S_OK; + } + } + +private: + Func cb_func; +}; + +template +static Microsoft::WRL::ComPtr create_cb_handler(F&& cb_func) +{ + return Microsoft::WRL::ComPtr( + new FFTypedCBHandler(std::forward(cb_func)) + ); +} + +/****************************************** + * Helpers to implement C style callbacks * + ******************************************/ +template +struct Win32Callback { + std::function fn; + static Ret CALLBACK thunk(Args... args, LPARAM lparam) { + auto self = reinterpret_cast(lparam); + return self->fn(std::forward(args)...); + } + decltype(&Win32Callback::thunk) proc = &Win32Callback::thunk; + LPARAM lparam = 0; +}; + +template +auto make_win32_callback(const std::function &&fn) { + using T = Win32Callback; + auto res = std::make_unique(T{ std::forward(fn) }); + res->lparam = reinterpret_cast(res.get()); + return res; +} +#define make_win32_callback(...) make_win32_callback(std::function(__VA_ARGS__)) + +/***************************** + * Small convenience helpers * + *****************************/ +struct HMODULEDeleter { + typedef HMODULE pointer; + void operator()(HMODULE handle) const { + if (handle) + FreeLibrary(handle); + } +}; +typedef std::unique_ptr hmodule_ptr_t; + +#define HLSL(shader) #shader + +#endif /* AVFILTER_VSRC_GFXCAPTURE_WINRT_H */ -- 2.49.1 _______________________________________________ ffmpeg-devel mailing list -- ffmpeg-devel@ffmpeg.org To unsubscribe send an email to ffmpeg-devel-leave@ffmpeg.org