From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.ffmpeg.org (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTPS id 3006E474A2 for ; Tue, 15 Jul 2025 07:09:47 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.ffmpeg.org (Postfix) with ESMTP id 14D9C68DBDE; Tue, 15 Jul 2025 10:09:44 +0300 (EEST) Received: from MA0PR01CU009.outbound.protection.outlook.com (mail-southindiaazon11020119.outbound.protection.outlook.com [52.101.227.119]) by ffbox0-bg.ffmpeg.org (Postfix) with ESMTPS id A44BE68A112 for ; Tue, 15 Jul 2025 10:09:36 +0300 (EEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=eNWvPeLasIMxEYxFJolvJiOwa3fdMET6xl9qg6OFyNy+UkprJgMys/r4/GgMgymklQ7vvLOBf9CQw/+wvN+Z9X/pCuu0Al4e41A9VsZnATpNMl43xSG5V/z8MzGgDSqapSWG6AzRYZWZKareY1jNpSYup38MwtOn9h6t19CWRlMc3zYfySSdKV8LYeVFg1nnx8uGMow5AVGLnk8ZMeYgeo1He868v9ib9q4K+vOzEYhk49B3EOR4eV5vB4GY1NWBXjZpMquxpVYZu4vVwAk7eAPEZIZoNCYpz1I7plikIRInt5SViFE20s2HuzD8wjQWak6OTvup/4KWCR1TwMl1uw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=UrWlJdkIgQOwr/99tNQf4EtZlGIDzbE8lMg7K/Iu0Ok=; b=YhASUQg0yVrmrCV63Lzul9xUNIIcYlCtldKbqZ32mD9HUu4dPSgibZhGM8c2MBnc051ZckDf/bLtKT2iY/JKdYjShNcLmj+5dtdmbAtJuUqe/oNIdYeeNymBCYZDxfkyAiAEvJitDz8SJgXH9/HOjOlyFiCWyMSfycAOyPYbHODELrzcsu9doNIGepO4rtS/1WvIFu1lxp4/szFLqgFJPc3ULXjVlNIFxFY9i3UXdNMPzXkOIn5eyoJa/LowgWM1IuqgXj9/xvZQYReZpOksRWqcZSLpZ18L0zqFGTfnqybsisyoQ8qbwHf5xLa7b6xAHRncmBkzpSe3WKrhq6UIjQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=multicorewareinc.com; dmarc=pass action=none header.from=multicorewareinc.com; dkim=pass header.d=multicorewareinc.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=multicorewareinc.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=UrWlJdkIgQOwr/99tNQf4EtZlGIDzbE8lMg7K/Iu0Ok=; b=d3ptwi8k+TwyOeYZCPB58M/BXAeOXSJ7wmgZYuVsZMP5/nUzy8R4bt/1/eXh0O2pqPG6gJmp25SfrPa15UEdaNN4z2tB7OnzTK9tgR4EvBAAY87qFqH3ouD6th8jk4uWqkP1bfJhOJ7aKHN3Al23Ac+arslusI2x5FXF7Re5HNo0z39hI9UsbmViq2VHjqm1ABmkrRYM/qwuq2HUGR9C/ojqRIqJ09LuKYqN10z2azBwU9MJqQ9yrw7c+S078Zr1aNgEQzKD2v7+DgUAFA/ULMD/pp1AuFWb+DlH24k9W/nhk7AObjjdBoGkZ7wVn4g4OQM9Cjch/Nac99sClHIboA== Received: from PN0P287MB1494.INDP287.PROD.OUTLOOK.COM (2603:1096:c01:186::9) by PN4P287MB4332.INDP287.PROD.OUTLOOK.COM (2603:1096:c01:2a2::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8922.32; Tue, 15 Jul 2025 06:55:13 +0000 Received: from PN0P287MB1494.INDP287.PROD.OUTLOOK.COM ([fe80::1a78:65d:b254:359]) by PN0P287MB1494.INDP287.PROD.OUTLOOK.COM ([fe80::1a78:65d:b254:359%5]) with mapi id 15.20.8922.028; Tue, 15 Jul 2025 06:55:13 +0000 From: Dash Santosh Sathyanarayanan To: FFmpeg development discussions and patches Thread-Topic: [PATCH v4 1/2] avfilter: add scale_d3d11 filter Thread-Index: AQHb9VTYyaSRTTCY5kyXzycZ4maLIg== Date: Tue, 15 Jul 2025 06:55:13 +0000 Message-ID: Accept-Language: en-IN, en-US Content-Language: en-IN X-MS-Has-Attach: X-MS-TNEF-Correlator: msip_labels: authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=multicorewareinc.com; x-ms-publictraffictype: Email x-ms-traffictypediagnostic: PN0P287MB1494:EE_|PN4P287MB4332:EE_ x-ms-office365-filtering-correlation-id: 5163f8d1-41bf-4ff5-5f6b-08ddc36c8c8a x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; ARA:13230040|366016|376014|1800799024|38070700018; x-microsoft-antispam-message-info: =?Windows-1252?Q?6zIogsMUHX2fAeIf5bFYzup7Lgm9FMiwwu+X4L0Tgcxgg+IPJ1LMwq7G?= =?Windows-1252?Q?4O2sLWOpNvyL7rt99t7uaN1APbDo7Kj8ngSkVsfgvP1r6sFpISgaKpAs?= =?Windows-1252?Q?10X4QAYuZrDGeNZO0S7wiUQE49p1qYPXP3wEnkUyRs8ouXH9kX0g3ysB?= =?Windows-1252?Q?71sEsIOmoPhEVJkVCRx5wGVrIbidaBcXyxbiVoxOSyo8p+WdioLiC/lE?= =?Windows-1252?Q?GTSudX4QKUF6PxT8TjNjpdNYK5yN72pqGjE4Ff8N6bj4KPJVUgX5P1oZ?= =?Windows-1252?Q?Dy1icO0JKUF9mcygcCb0V77v9kSv3V8mlEUFBD6XLADX3lWum4XDc3rx?= =?Windows-1252?Q?98ye47hwaDE1slc0f/9/4Yz33Dq4X6yFIsuiKkDMD18G+An3KhNVRozT?= =?Windows-1252?Q?5omW3PtPncld1G5MXVLYcW+Xz7jmz3rATT1/vaYt3v63vZQOZz+LxNNZ?= =?Windows-1252?Q?qmm+f2SRTr6wf+HRFoVD+uWaboc3W2PdAvLqs4yrh24D0D6lHAQvY4pG?= =?Windows-1252?Q?6O11h+dRSPPu7zDXpLnI3mg33NadJabPaQCgZvBFiTGbleKZ4duYa4fD?= =?Windows-1252?Q?P0pNrhHyJhodXJqOwPPVWFSeZkQ9JWL0kMLyRHVbhJBSdlK90VZL7THn?= =?Windows-1252?Q?t9FHSrgnicjaW5fIPcGWmb4qbNFupZKLqb2Ed+gfRyBNbPC8JNhQPHf/?= =?Windows-1252?Q?EFCTLJubRP83sDfLWHnxLnKoNyKqsqeg2XeU2M/fZLaw8LXzw3ftXxGw?= =?Windows-1252?Q?0Xm9pZBu6QYMmHkNHpPBtbZqq0+jdR/DL+RmexqIq6E+FsodVcaAXvB8?= =?Windows-1252?Q?fjRsMzxFaJt4a434pThoKSLOMRU42xGa7jy1hLjsPzcf9Acs/k5+WEl0?= =?Windows-1252?Q?fA+qtH7N682SdyBLAW15Xv/KWgXOD5KDGkXid/rgHuC0JCkvDSTiXBx2?= =?Windows-1252?Q?ITTH9t8vAo7SH/WuiPo4DHpJSfad1aeZ5YC+UY2KyErOnO01fk1vQNYd?= =?Windows-1252?Q?xQtvUK5MFnfAP18wdUJOFPHI7KN1AflDqISi9MVzKCm6zVQmjcOjC72F?= =?Windows-1252?Q?Ij2e2WFkKtawNsRA1T6LxxpdHVk0H4AmGkmP3l8GgML+RXLOGu9B4bUW?= =?Windows-1252?Q?n7iNF8G5lmpvAx5IhMeLDmpwR035zl/JRNWKvtKNdgzV0utjILyXK+Ff?= =?Windows-1252?Q?t1INXZA2H+m+SxkHRZiPWSF1alwfhpXqi6SGTKZy0/yD0eEq6Mlz3ImY?= =?Windows-1252?Q?q5qV4AoHqCMpWkShzTeH8oDgnn+1w9kEEriaw+fQaB5nbCAeLurJhXNf?= =?Windows-1252?Q?UktFdSogIVrz7Cb65R7fBDpzO2ChMRIMXDXAfOnaCzETRtachGev8loO?= =?Windows-1252?Q?WxHuni3v2OSI7rJbc/VV86llquISUmYBl0rvTnv8Zr/MuZVJ8IdMDT7R?= =?Windows-1252?Q?r3zxvitreT+RZTeYq5AiQFmusKvUcQmkf/rlYMD7OPGLnPM1QEDAlkUB?= =?Windows-1252?Q?/Nm0HfW3n8FzYzH1MJcWhacpEQPUCL91JEWgli/mPWksms1/ZNXowf+I?= =?Windows-1252?Q?oNZzNWIklaILb53UU388CkHEY7RomNDmr3rGIA=3D=3D?= x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:PN0P287MB1494.INDP287.PROD.OUTLOOK.COM; PTR:; CAT:NONE; SFS:(13230040)(366016)(376014)(1800799024)(38070700018); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?Windows-1252?Q?i45ZVfzdwEV/UmX+ee1tEcg5k7Rlx96EQiluJaKJSHcm5+zB0cr7abOz?= =?Windows-1252?Q?lZQEpYpW+9HCXAhTalIdwuF1tpsE0pp6Jk4eq+1h1s2NOZDGSu638/6K?= =?Windows-1252?Q?sH8vdwko8h8xI7IsZQo6a0oY1N29DATJuBHZoVfwmz1c8AgQt7IFayll?= =?Windows-1252?Q?6q/J7kkfrCyYtbpLOLE91UxMw2oOPk88Q73meDCgPuY5iClZe82noG7x?= =?Windows-1252?Q?6b9b1tZufUQ9R+rFHARGQe9LOc//hQA80yzxPFwNCDpYViuvibZ9d+Qf?= =?Windows-1252?Q?VOh5dmMIkRR1//jjbcW9O8/DebT0MLlNjPtfXbUzzGwSDsqyUNJcqK6b?= =?Windows-1252?Q?xiSGid4X4pUdMjEbYVYuHM2vH6wOHVLy6k7ppDXqP4Cesd3P1G+MYuPR?= =?Windows-1252?Q?aMTwTQZtqZpkJ7Lo2y2CHQSmY3PrX90/g2vT89om8HB4Uztow6qIFBCN?= =?Windows-1252?Q?bjSEPhtZlWLYKjj0UGdCWR4puIXiDacRd3TtFdrGxFRj+YyY5DUyGxEJ?= =?Windows-1252?Q?B0bLh8N+YUE8KY8GW9l4BIR7xXR/KAztctRYzr9e6Pa4srQ6SBlkExt9?= =?Windows-1252?Q?lduvVfXJ6vby8Q604IjYjAo8COhRuua2DDuMhCQ8ccp3hFQLgkXHoOpJ?= =?Windows-1252?Q?mAIm/B5zicOnouSJoMGsC5ubsCJk9bv0BdSCTEouUPnKL3Y4LgRLA3gK?= =?Windows-1252?Q?YLXRYUWC2uzSpFo49vemP/oJz2R1Vpw+53zyG7udZtdzSjEUSj9NlVbT?= =?Windows-1252?Q?NLoMh9Bj69+BA33IRKYgm36Z3L9WAPLdqbDBsFZxXx5r4xQl1eaUoe0l?= =?Windows-1252?Q?yDK7WMv3S2ZwD12FVIQl/KUDJv9Wjm4T/4X7mEPboLN6ctHAGi9bYYkG?= =?Windows-1252?Q?ecmIw97nLxzSU224Jfy+LQpcT7Bp7vDq/OpmFlPn12oXbWpNH6U/pyyV?= =?Windows-1252?Q?nAGwuQO1VcVEGpuDBoupv50/1XrmcaQe0/zWEtfaq8+/IjAEOdomlNTP?= =?Windows-1252?Q?Zs/q5cbf6keUten9yWEtkpB1lnf3ch1S1JhxFm1W7MmerDZ83yUQNh4O?= =?Windows-1252?Q?C0YobVAOYL7iiGzDKxC7l9VfWmweeRkId6/RBxsxGjXqK9/8VZsYieZd?= =?Windows-1252?Q?EHCVPuLzzyTXFZWQTY8zSfuLoUrgSPJvTLEw3QiEJsLkXhUIxuwvzHu/?= =?Windows-1252?Q?HbRvSjE089d+5go+wFJBng3yWlvxra+0SyBScyWhAJEC4X7bc1TY5NKR?= =?Windows-1252?Q?SXhiGtEfllzfyk8gnmAvmJMUny1tjPWdzZmVOVeVKwSmhbiljValKajr?= =?Windows-1252?Q?qxbJGmwZuVcRI83DTgWad/SiqPvT27UGr6Vys6rmXI0hPRhO5KgXMD2W?= =?Windows-1252?Q?Yv4+KZ5QmMc+FurNcBaN7HzYIdKP/4kTouw2QL/k8OMBQHImjIKR6zT8?= =?Windows-1252?Q?S/MRSBW6j+PaA78USjauflCBtGjvOyNlfL3DQKYdD8UUItn753bC71wX?= =?Windows-1252?Q?+rBjWGqotM1xZrOcNz9cUnnTCWociMYCdU3FUiiBolkHxYqnbOCjXlAo?= =?Windows-1252?Q?H+afyiwNj4iI+pltJos6WYy3J0mpm7i/midhHK4Rl+9qfP7TBHeyZ0lG?= =?Windows-1252?Q?udgrUC8JSx11gYwBIMx/5xO1YQH1WGp3Y1UZyrX3auk7aiPsyUerdPUp?= =?Windows-1252?Q?bmuCvJzASJzm9+jRvrngaEJXTjlmPuOv+/g0EeNdcFtxtRBpByHCt19N?= =?Windows-1252?Q?+t3qKNzOs/o9WBlLGuM=3D?= MIME-Version: 1.0 X-OriginatorOrg: multicorewareinc.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: PN0P287MB1494.INDP287.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-Network-Message-Id: 5163f8d1-41bf-4ff5-5f6b-08ddc36c8c8a X-MS-Exchange-CrossTenant-originalarrivaltime: 15 Jul 2025 06:55:13.7924 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: ffc5e88b-3fa2-4d69-a468-344b6b766e7d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: 3o7bT8D/pZvXGugf991XlgW106oGz4xPLKIDiLO2Hahoh6lg4dW5JR1XDl9bizJS2H2jJ5QuPYfOEQP1A4cT+K3H6mulsm61YGg1ZDp1zisvf4U4ua0y3/OyphZt/E9rOBWdzpTCFElutMVkY8utEQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PN4P287MB4332 Subject: [FFmpeg-devel] [PATCH v4 1/2] avfilter: add scale_d3d11 filter X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Content-Type: text/plain; charset="windows-1252" Content-Transfer-Encoding: quoted-printable Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: > > This part is still not acceptable. You blindly set a flag based on > > what you need, if special flags on the context are needed they should > > be negotiated between the different components, guided by the users > > use-case and input, and not hardcoded based on an arbitrary condition > > of a pixel format. > = > Couldn't find a way to negotiate between decoder and encoder, but it shou= ld > be possible to use scale_d3d11 filter as passthrough from which the output > can be sent directly to the encoder. So removed the change from dxva2.c > = > > A potentially infinite loop in get_buffer is an absolute no-go. There > > is no way to control this behavior, and the surface may never become > > available if the caller is blocked in a loop here. You assume a > > certain threading model which may not exist. > > This sort of behavior should be handled by the caller, not here (and > > no, a maximum wait time or anything like that does not resolve this > > concern, move it to another layer where the caller can control this). > = > Thanks for the feedback. I did try moving the wait logic out of get_buffe= r, > but couldn't find a clean way to handle it at a different layer. Without = the loop, > it ends up breaking the pipeline with a =93static surface pool size excee= ded=94 > error, as all surfaces remain busy. > = > I also attempted returning EAGAIN instead of ENOMEM, but that leads to an > assertion failure in the decoder path - apparently EAGAIN isn=92t expecte= d at that > stage and causes a hard abort. > = > At this point I=92m honestly not sure how to proceed. I'd really apprecia= te any > guidance or suggestions on alternative approaches that might work here. > Sorry if I=92m missing something obvious. Fixed FATE test failures. Please let me know if get_buffer loop can be impl= emented in a better way. Thanks - Dash This commit introduces a new hardware-accelerated video filter, scale_d3d11, which performs scaling and format conversion using Direct3D 11. The filter = enables efficient GPU-based scaling and pixel format conversion (p010 to nv12), red= ucing CPU overhead and latency in video pipelines. --- Changelog | 1 + configure | 1 + libavfilter/Makefile | 1 + libavfilter/allfilters.c | 1 + libavfilter/vf_scale_d3d11.c | 464 ++++++++++++++++++++++++++++++++++ libavutil/hwcontext_d3d11va.c | 37 ++- 6 files changed, 494 insertions(+), 11 deletions(-) create mode 100644 libavfilter/vf_scale_d3d11.c diff --git a/Changelog b/Changelog index 4217449438..56122ef555 100644 --- a/Changelog +++ b/Changelog @@ -18,6 +18,7 @@ version : - APV encoding support through a libopenapv wrapper - VVC decoder supports all content of SCC (Screen Content Coding): IBC (Inter Block Copy), Palette Mode and ACT (Adaptive Color Transform +- Add vf_scale_d3d11 filter = = version 7.1: diff --git a/configure b/configure index a1f483f02c..10c7c9e7b5 100755 --- a/configure +++ b/configure @@ -3359,6 +3359,7 @@ overlay_cuda_filter_deps_any=3D"cuda_nvcc cuda_llvm" sharpen_npp_filter_deps=3D"ffnvcodec libnpp" = ddagrab_filter_deps=3D"d3d11va IDXGIOutput1 DXGI_OUTDUPL_FRAME_INFO" +scale_d3d11_filter_deps=3D"d3d11va" = amf_deps_any=3D"libdl LoadLibrary" nvenc_deps=3D"ffnvcodec" diff --git a/libavfilter/Makefile b/libavfilter/Makefile index 97f8f17272..95612d753d 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -462,6 +462,7 @@ OBJS-$(CONFIG_ROBERTS_OPENCL_FILTER) +=3D vf_co= nvolution_opencl.o opencl.o OBJS-$(CONFIG_ROTATE_FILTER) +=3D vf_rotate.o OBJS-$(CONFIG_SAB_FILTER) +=3D vf_sab.o OBJS-$(CONFIG_SCALE_FILTER) +=3D vf_scale.o scale_eval.o = framesync.o +OBJS-$(CONFIG_SCALE_D3D11_FILTER) +=3D vf_scale_d3d11.o scale_e= val.o OBJS-$(CONFIG_SCALE_CUDA_FILTER) +=3D vf_scale_cuda.o scale_ev= al.o \ vf_scale_cuda.ptx.o cuda/l= oad_helper.o OBJS-$(CONFIG_SCALE_NPP_FILTER) +=3D vf_scale_npp.o scale_eva= l.o diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index 3bc045b28f..ec5e09494a 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -436,6 +436,7 @@ extern const FFFilter ff_vf_scale; extern const FFFilter ff_vf_vpp_amf; extern const FFFilter ff_vf_sr_amf; extern const FFFilter ff_vf_scale_cuda; +extern const FFFilter ff_vf_scale_d3d11; extern const FFFilter ff_vf_scale_npp; extern const FFFilter ff_vf_scale_qsv; extern const FFFilter ff_vf_scale_vaapi; diff --git a/libavfilter/vf_scale_d3d11.c b/libavfilter/vf_scale_d3d11.c new file mode 100644 index 0000000000..0b7476dd15 --- /dev/null +++ b/libavfilter/vf_scale_d3d11.c @@ -0,0 +1,464 @@ +/* + * Copyright (C) 2025 MulticorewWare, Inc. + * + * Authors: Dash Santosh + * Sachin + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-130= 1 USA + */ + +#include "libavutil/opt.h" +#include "libavutil/pixdesc.h" +#include "compat/w32dlfcn.h" + +#include "libavutil/hwcontext.h" +#include "libavutil/hwcontext_d3d11va.h" + +#include "filters.h" +#include "scale_eval.h" +#include "video.h" + +typedef struct ScaleD3D11Context { + const AVClass *classCtx; + char *w_expr; + char *h_expr; + enum AVPixelFormat format; + + ///< D3D11 objects + ID3D11Device *device; + ID3D11DeviceContext *context; + ID3D11VideoDevice *videoDevice; + ID3D11VideoProcessor *processor; + ID3D11VideoProcessorEnumerator *enumerator; + ID3D11VideoProcessorOutputView *outputView; + ID3D11VideoProcessorInputView *inputView; + + ///< Buffer references + AVBufferRef *hw_device_ctx; + AVBufferRef *hw_frames_ctx_out; + + ///< Dimensions and formats + int width, height; + int inputWidth, inputHeight; + DXGI_FORMAT input_format; + DXGI_FORMAT output_format; +} ScaleD3D11Context; + +static av_cold int scale_d3d11_init(AVFilterContext *ctx) { + ///< all real work is done in config_props and filter_frame + return 0; +} + +static void release_d3d11_resources(ScaleD3D11Context *s) { + if (s->outputView) { + s->outputView->lpVtbl->Release(s->outputView); + s->outputView =3D NULL; + } + + if (s->processor) { + s->processor->lpVtbl->Release(s->processor); + s->processor =3D NULL; + } + + if (s->enumerator) { + s->enumerator->lpVtbl->Release(s->enumerator); + s->enumerator =3D NULL; + } + + if (s->videoDevice) { + s->videoDevice->lpVtbl->Release(s->videoDevice); + s->videoDevice =3D NULL; + } +} + +static int scale_d3d11_configure_processor(ScaleD3D11Context *s, AVFilterC= ontext *ctx) { + HRESULT hr; + + switch (s->format) { + case AV_PIX_FMT_NV12: + s->output_format =3D DXGI_FORMAT_NV12; + break; + case AV_PIX_FMT_P010: + s->output_format =3D DXGI_FORMAT_P010; + break; + default: + av_log(ctx, AV_LOG_ERROR, "Invalid output format specified\n"); + return AVERROR(EINVAL); + } + + ///< Get D3D11 device and context from hardware device context + AVHWDeviceContext *hwctx =3D (AVHWDeviceContext *)s->hw_device_ctx->da= ta; + AVD3D11VADeviceContext *d3d11_hwctx =3D (AVD3D11VADeviceContext *)hwct= x->hwctx; + s->device =3D d3d11_hwctx->device; + s->context =3D d3d11_hwctx->device_context; + + av_log(ctx, AV_LOG_VERBOSE, "Configuring D3D11 video processor: %dx%d = -> %dx%d\n", + s->inputWidth, s->inputHeight, s->width, s->height); + + ///< Define the video processor content description + D3D11_VIDEO_PROCESSOR_CONTENT_DESC contentDesc =3D { + .InputFrameFormat =3D D3D11_VIDEO_FRAME_FORMAT_PROGRESSIVE, + .InputWidth =3D s->inputWidth, + .InputHeight =3D s->inputHeight, + .OutputWidth =3D s->width, + .OutputHeight =3D s->height, + .Usage =3D D3D11_VIDEO_USAGE_PLAYBACK_NORMAL, + }; + + ///< Query video device interface + hr =3D s->device->lpVtbl->QueryInterface(s->device, &IID_ID3D11VideoDe= vice, (void **)&s->videoDevice); + if (FAILED(hr)) { + av_log(ctx, AV_LOG_ERROR, "Failed to get D3D11 video device interf= ace: HRESULT 0x%lX\n", hr); + return AVERROR_EXTERNAL; + } + + ///< Create video processor enumerator + hr =3D s->videoDevice->lpVtbl->CreateVideoProcessorEnumerator(s->video= Device, &contentDesc, &s->enumerator); + if (FAILED(hr)) { + av_log(ctx, AV_LOG_ERROR, "Failed to create video processor enumer= ator: HRESULT 0x%lX\n", hr); + return AVERROR_EXTERNAL; + } + + ///< Create the video processor + hr =3D s->videoDevice->lpVtbl->CreateVideoProcessor(s->videoDevice, s-= >enumerator, 0, &s->processor); + if (FAILED(hr)) { + av_log(ctx, AV_LOG_ERROR, "Failed to create video processor: HRESU= LT 0x%lX\n", hr); + return AVERROR_EXTERNAL; + } + + av_log(ctx, AV_LOG_VERBOSE, "D3D11 video processor successfully config= ured\n"); + return 0; +} + +static int scale_d3d11_filter_frame(AVFilterLink *inlink, AVFrame *in) +{ + AVFilterContext *ctx =3D inlink->dst; + ScaleD3D11Context *s =3D ctx->priv; + AVFilterLink *outlink =3D ctx->outputs[0]; + ID3D11VideoProcessorInputView *inputView =3D NULL; + ID3D11VideoContext *videoContext =3D NULL; + AVFrame *out =3D NULL; + int ret =3D 0; + HRESULT hr; + + ///< Validate input frame + if (!in) { + av_log(ctx, AV_LOG_ERROR, "Null input frame\n"); + return AVERROR(EINVAL); + } + + if (!in->hw_frames_ctx) { + av_log(ctx, AV_LOG_ERROR, "No hardware frames context in input fra= me\n"); + av_frame_free(&in); + return AVERROR(EINVAL); + } + + ///< Verify hardware device contexts + AVHWFramesContext *frames_ctx =3D (AVHWFramesContext *)in->hw_frames_c= tx->data; + + if (!s->hw_device_ctx) { + av_log(ctx, AV_LOG_ERROR, "Filter hardware device context is unini= tialized\n"); + av_frame_free(&in); + return AVERROR(EINVAL); + } + + AVHWDeviceContext *input_device_ctx =3D (AVHWDeviceContext *)frames_ct= x->device_ref->data; + AVHWDeviceContext *filter_device_ctx =3D (AVHWDeviceContext *)s->hw_de= vice_ctx->data; + + if (input_device_ctx->type !=3D filter_device_ctx->type) { + av_log(ctx, AV_LOG_ERROR, "Mismatch between input and filter hardw= are device types\n"); + av_frame_free(&in); + return AVERROR(EINVAL); + } + + ///< Allocate output frame + out =3D av_frame_alloc(); + if (!out) { + av_log(ctx, AV_LOG_ERROR, "Failed to allocate output frame\n"); + av_frame_free(&in); + return AVERROR(ENOMEM); + } + + ret =3D av_hwframe_get_buffer(s->hw_frames_ctx_out, out, 0); + if (ret < 0) { + av_log(ctx, AV_LOG_ERROR, "Failed to get output frame from pool\n"= ); + av_frame_free(&in); + av_frame_free(&out); + return ret; + } + + ///< Configure the D3D11 video processor if not already configured + if (!s->processor) { + ///< Get info from input texture + D3D11_TEXTURE2D_DESC textureDesc; + ID3D11Texture2D *input_texture =3D (ID3D11Texture2D *)in->data[0]; + input_texture->lpVtbl->GetDesc(input_texture, &textureDesc); + + s->inputWidth =3D textureDesc.Width; + s->inputHeight =3D textureDesc.Height; + s->input_format =3D textureDesc.Format; + + ret =3D scale_d3d11_configure_processor(s, ctx); + if (ret < 0) { + av_log(ctx, AV_LOG_ERROR, "Failed to configure processor\n"); + av_frame_free(&in); + av_frame_free(&out); + return ret; + } + } + + ///< Get input texture and prepare input view + ID3D11Texture2D *d3d11_texture =3D (ID3D11Texture2D *)in->data[0]; + int subIdx =3D (int)(intptr_t)in->data[1]; + + D3D11_VIDEO_PROCESSOR_INPUT_VIEW_DESC inputViewDesc =3D { + .FourCC =3D s->input_format, + .ViewDimension =3D D3D11_VPIV_DIMENSION_TEXTURE2D, + .Texture2D.ArraySlice =3D subIdx + }; + + hr =3D s->videoDevice->lpVtbl->CreateVideoProcessorInputView( + s->videoDevice, (ID3D11Resource *)d3d11_texture, s->enumerator, &i= nputViewDesc, &inputView); + if (FAILED(hr)) { + av_log(ctx, AV_LOG_ERROR, "Failed to create input view: HRESULT 0x= %lX\n", hr); + av_frame_free(&in); + av_frame_free(&out); + return AVERROR_EXTERNAL; + } + + ///< Create output view for current texture + ID3D11Texture2D *output_texture =3D (ID3D11Texture2D *)out->data[0]; + D3D11_VIDEO_PROCESSOR_OUTPUT_VIEW_DESC outputViewDesc =3D { + .ViewDimension =3D D3D11_VPOV_DIMENSION_TEXTURE2D, + .Texture2D =3D { .MipSlice =3D 0 }, + }; + + hr =3D s->videoDevice->lpVtbl->CreateVideoProcessorOutputView( + s->videoDevice, (ID3D11Resource *)output_texture, s->enumerator, &= outputViewDesc, &s->outputView); + if (FAILED(hr)) { + av_log(ctx, AV_LOG_ERROR, "Failed to create output view: HRESULT 0= x%lX\n", hr); + av_frame_free(&in); + av_frame_free(&out); + return AVERROR_EXTERNAL; + } + + ///< Set up processing stream + D3D11_VIDEO_PROCESSOR_STREAM stream =3D { + .Enable =3D TRUE, + .pInputSurface =3D inputView, + .OutputIndex =3D 0 + }; + + ///< Get video context + hr =3D s->context->lpVtbl->QueryInterface(s->context, &IID_ID3D11Video= Context, (void **)&videoContext); + if (FAILED(hr)) { + av_log(ctx, AV_LOG_ERROR, "Failed to get video context: HRESULT 0x= %lX\n", hr); + inputView->lpVtbl->Release(inputView); + av_frame_free(&in); + av_frame_free(&out); + return AVERROR_EXTERNAL; + } + + ///< Process the frame + hr =3D videoContext->lpVtbl->VideoProcessorBlt(videoContext, s->proces= sor, s->outputView, 0, 1, &stream); + if (FAILED(hr)) { + av_log(ctx, AV_LOG_ERROR, "VideoProcessorBlt failed: HRESULT 0x%lX= \n", hr); + videoContext->lpVtbl->Release(videoContext); + inputView->lpVtbl->Release(inputView); + av_frame_free(&in); + av_frame_free(&out); + return AVERROR_EXTERNAL; + } + + ///< Set up output frame + ret =3D av_frame_copy_props(out, in); + if (ret < 0) { + av_log(ctx, AV_LOG_ERROR, "Failed to copy frame properties\n"); + videoContext->lpVtbl->Release(videoContext); + inputView->lpVtbl->Release(inputView); + av_frame_free(&in); + av_frame_free(&out); + return ret; + } + + out->data[0] =3D (uint8_t *)output_texture; + out->data[1] =3D (uint8_t *)(intptr_t)0; + out->width =3D s->width; + out->height =3D s->height; + out->format =3D AV_PIX_FMT_D3D11; + + ///< Clean up resources + inputView->lpVtbl->Release(inputView); + videoContext->lpVtbl->Release(videoContext); + if (s->outputView) { + s->outputView->lpVtbl->Release(s->outputView); + s->outputView =3D NULL; + } + av_frame_free(&in); + + ///< Forward the frame + return ff_filter_frame(outlink, out); +} + +static int scale_d3d11_config_props(AVFilterLink *outlink) +{ + AVFilterContext *ctx =3D outlink->src; + ScaleD3D11Context *s =3D ctx->priv; + AVFilterLink *inlink =3D ctx->inputs[0]; + FilterLink *inl =3D ff_filter_link(inlink); + FilterLink *outl =3D ff_filter_link(outlink); + int ret; + + ///< Clean up any previous resources + release_d3d11_resources(s); + + ///< Evaluate output dimensions + ret =3D ff_scale_eval_dimensions(s, s->w_expr, s->h_expr, inlink, outl= ink, &s->width, &s->height); + if (ret < 0) { + av_log(ctx, AV_LOG_ERROR, "Failed to evaluate dimensions\n"); + return ret; + } + + outlink->w =3D s->width; + outlink->h =3D s->height; + + ///< Validate input hw_frames_ctx + if (!inl->hw_frames_ctx) { + av_log(ctx, AV_LOG_ERROR, "No hw_frames_ctx available on input lin= k\n"); + return AVERROR(EINVAL); + } + + ///< Propagate hw_frames_ctx to output + outl->hw_frames_ctx =3D av_buffer_ref(inl->hw_frames_ctx); + if (!outl->hw_frames_ctx) { + av_log(ctx, AV_LOG_ERROR, "Failed to propagate hw_frames_ctx to ou= tput\n"); + return AVERROR(ENOMEM); + } + + ///< Initialize filter's hardware device context + if (!s->hw_device_ctx) { + AVHWFramesContext *in_frames_ctx =3D (AVHWFramesContext *)inl->hw_= frames_ctx->data; + s->hw_device_ctx =3D av_buffer_ref(in_frames_ctx->device_ref); + if (!s->hw_device_ctx) { + av_log(ctx, AV_LOG_ERROR, "Failed to initialize filter hardwar= e device context\n"); + return AVERROR(ENOMEM); + } + } + + ///< Get D3D11 device and context (but don't initialize processor yet = - done in filter_frame) + AVHWDeviceContext *hwctx =3D (AVHWDeviceContext *)s->hw_device_ctx->da= ta; + AVD3D11VADeviceContext *d3d11_hwctx =3D (AVD3D11VADeviceContext *)hwct= x->hwctx; + + s->device =3D d3d11_hwctx->device; + s->context =3D d3d11_hwctx->device_context; + + if (!s->device || !s->context) { + av_log(ctx, AV_LOG_ERROR, "Failed to get valid D3D11 device or con= text\n"); + return AVERROR(EINVAL); + } + + ///< Create new hardware frames context for output + AVHWFramesContext *in_frames_ctx =3D (AVHWFramesContext *)inl->hw_fram= es_ctx->data; + s->hw_frames_ctx_out =3D av_hwframe_ctx_alloc(s->hw_device_ctx); + if (!s->hw_frames_ctx_out) + return AVERROR(ENOMEM); + + AVHWFramesContext *frames_ctx =3D (AVHWFramesContext *)s->hw_frames_ct= x_out->data; + frames_ctx->format =3D AV_PIX_FMT_D3D11; + frames_ctx->sw_format =3D s->format; + frames_ctx->width =3D s->width; + frames_ctx->height =3D s->height; + frames_ctx->initial_pool_size =3D 10; + + if (ctx->extra_hw_frames > 0) + frames_ctx->initial_pool_size +=3D ctx->extra_hw_frames; + + AVD3D11VAFramesContext *frames_hwctx =3D frames_ctx->hwctx; + frames_hwctx->MiscFlags =3D 0; + frames_hwctx->BindFlags =3D D3D11_BIND_RENDER_TARGET | D3D11_BIND_VIDE= O_ENCODER; + + ret =3D av_hwframe_ctx_init(s->hw_frames_ctx_out); + if (ret < 0) { + av_buffer_unref(&s->hw_frames_ctx_out); + return ret; + } + + outl->hw_frames_ctx =3D av_buffer_ref(s->hw_frames_ctx_out); + if (!outl->hw_frames_ctx) + return AVERROR(ENOMEM); + + av_log(ctx, AV_LOG_VERBOSE, "D3D11 scale config: %dx%d -> %dx%d\n", + inlink->w, inlink->h, outlink->w, outlink->h); + return 0; +} + +static av_cold void scale_d3d11_uninit(AVFilterContext *ctx) { + ScaleD3D11Context *s =3D ctx->priv; + + ///< Release D3D11 resources + release_d3d11_resources(s); + + ///< Free the hardware device context reference + av_buffer_unref(&s->hw_frames_ctx_out); + av_buffer_unref(&s->hw_device_ctx); + + ///< Free option strings + av_freep(&s->w_expr); + av_freep(&s->h_expr); +} + +static const AVFilterPad scale_d3d11_inputs[] =3D { + { + .name =3D "default", + .type =3D AVMEDIA_TYPE_VIDEO, + .filter_frame =3D scale_d3d11_filter_frame, + }, +}; + +static const AVFilterPad scale_d3d11_outputs[] =3D { + { + .name =3D "default", + .type =3D AVMEDIA_TYPE_VIDEO, + .config_props =3D scale_d3d11_config_props, + }, +}; + +#define OFFSET(x) offsetof(ScaleD3D11Context, x) +#define FLAGS (AV_OPT_FLAG_FILTERING_PARAM | AV_OPT_FLAG_VIDEO_PARAM) + +static const AVOption scale_d3d11_options[] =3D { + { "width", "Output video width", OFFSET(w_expr), AV_OPT_TYPE_STRING,= {.str =3D "iw"}, .flags =3D FLAGS }, + { "height", "Output video height", OFFSET(h_expr), AV_OPT_TYPE_STRING,= {.str =3D "ih"}, .flags =3D FLAGS }, + { "format", "Output video pixel format", OFFSET(format), AV_OPT_TYPE_P= IXEL_FMT, { .i64 =3D AV_PIX_FMT_NONE }, INT_MIN, INT_MAX, .flags=3DFLAGS }, + { NULL } +}; + +AVFILTER_DEFINE_CLASS(scale_d3d11); + +const FFFilter ff_vf_scale_d3d11 =3D { + .p.name =3D "scale_d3d11", + .p.description =3D NULL_IF_CONFIG_SMALL("Scale video using Direct3D= 11"), + .priv_size =3D sizeof(ScaleD3D11Context), + .p.priv_class =3D &scale_d3d11_class, + .init =3D scale_d3d11_init, + .uninit =3D scale_d3d11_uninit, + FILTER_INPUTS(scale_d3d11_inputs), + FILTER_OUTPUTS(scale_d3d11_outputs), + FILTER_SINGLE_PIXFMT(AV_PIX_FMT_D3D11), + .p.flags =3D AVFILTER_FLAG_HWDEVICE, + .flags_internal =3D FF_FILTER_FLAG_HWFRAME_AWARE, +}; \ No newline at end of file diff --git a/libavutil/hwcontext_d3d11va.c b/libavutil/hwcontext_d3d11va.c index 1a047ce57b..a5947e1618 100644 --- a/libavutil/hwcontext_d3d11va.c +++ b/libavutil/hwcontext_d3d11va.c @@ -42,6 +42,8 @@ #include "thread.h" #include "compat/w32dlfcn.h" = +#define MAX_ARRAY_SIZE 64 // Driver specification limits ArraySize to 64 f= or decoder-bound resources + typedef HRESULT(WINAPI *PFN_CREATE_DXGI_FACTORY)(REFIID riid, void **ppFac= tory); = static AVOnce functions_loaded =3D AV_ONCE_INIT; @@ -288,6 +290,8 @@ static int d3d11va_frames_init(AVHWFramesContext *ctx) return AVERROR(EINVAL); } = + ctx->initial_pool_size =3D FFMIN(ctx->initial_pool_size, MAX_ARRAY_SIZ= E); + texDesc =3D (D3D11_TEXTURE2D_DESC){ .Width =3D ctx->width, .Height =3D ctx->height, @@ -340,19 +344,30 @@ static int d3d11va_get_buffer(AVHWFramesContext *ctx,= AVFrame *frame) { AVD3D11FrameDescriptor *desc; = - frame->buf[0] =3D av_buffer_pool_get(ctx->pool); - if (!frame->buf[0]) - return AVERROR(ENOMEM); - - desc =3D (AVD3D11FrameDescriptor *)frame->buf[0]->data; + /** + * Loop until a buffer becomes available from the pool. + * In a full hardware pipeline, all buffers may be temporarily in use = by + * other modules (encoder/filter/decoder). Rather than immediately fai= ling + * with ENOMEM, we wait for a buffer to be released back to the pool, = which + * maintains pipeline flow and prevents unnecessary allocation failures + * during normal operation. + */ + do { + frame->buf[0] =3D av_buffer_pool_get(ctx->pool); + if (frame->buf[0]) { + desc =3D (AVD3D11FrameDescriptor *)frame->buf[0]->data; + frame->data[0] =3D (uint8_t *)desc->texture; + frame->data[1] =3D (uint8_t *)desc->index; + frame->format =3D AV_PIX_FMT_D3D11; + frame->width =3D ctx->width; + frame->height =3D ctx->height; + return 0; + } = - frame->data[0] =3D (uint8_t *)desc->texture; - frame->data[1] =3D (uint8_t *)desc->index; - frame->format =3D AV_PIX_FMT_D3D11; - frame->width =3D ctx->width; - frame->height =3D ctx->height; + av_usleep(500); + } while (1); = - return 0; + return AVERROR(ENOMEM); } = static int d3d11va_transfer_get_formats(AVHWFramesContext *ctx, -- = 2.34.1 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".