From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTP id C56B04B982 for ; Tue, 2 Jul 2024 13:27:11 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 1F7D168D8B4; Tue, 2 Jul 2024 16:27:08 +0300 (EEST) Received: from haasn.dev (haasn.dev [78.46.187.166]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 9228868D82C for ; Tue, 2 Jul 2024 16:27:01 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=haasn.xyz; s=mail; t=1719926821; bh=78DEnPZRLfvF4ml2As1lfnAJvUZCvrZho07Br1YGx9o=; h=Date:From:To:Subject:In-Reply-To:References:From; b=HXsH7Dc8ZeL0CTvPNbZ1heM2NjEfzQj+3kbNnRb34/0JECykgDQx2vI2JaCuuD33n UEXCjYsD6Ep0Cm6LALkkpcVjkql6ADtNIuL3+/R/2im0+6sMr5KF0pQn56O9BM0AVN 0VxmMtiX7cIYid/Lfl2R/YXqonKoUkKa3WtjwGMw= Received: from haasn.dev (unknown [10.30.0.2]) by haasn.dev (Postfix) with ESMTP id D7D8444623 for ; Tue, 2 Jul 2024 15:27:00 +0200 (CEST) Date: Tue, 2 Jul 2024 15:27:00 +0200 Message-ID: <20240702152700.GC15496@haasn.xyz> From: Niklas Haas To: ffmpeg-devel@ffmpeg.org In-Reply-To: <20240622151334.GD14140@haasn.xyz> References: <20240622151334.GD14140@haasn.xyz> MIME-Version: 1.0 Content-Disposition: inline Subject: Re: [FFmpeg-devel] [RFC]] swscale modernization proposal X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: On Sat, 22 Jun 2024 15:13:34 +0200 Niklas Haas wrote: > Finally, avscale_* should ultimately also support hardware frames > directly, in which case it will dispatch to some equivalent of > scale_vulkan/vaapi/cuda or possibly even libplacebo. (But I will defer > this to a future milestone) How do people feel about being able to control the "backend" directly from the interface? For example, struct AVScaleContext { int backend; ... } enum AVScaleBackend { AV_SCALE_AUTO = 0, // select first backend that passes avscale_test_* AV_SCALE_NATIVE, // use new scaling pipeline (when implemented) AV_SCALE_SWSCALE, // use existing SwsContext code 1:1, to-be-deprecated AV_SCALE_ZIMG, AV_SCALE_LIBPLACEBO, AV_SCALE_CUDA, AV_SCALE_VAAPI, ... }; int avscale_test_format(enum AVScaleBackend backend, enum AVPixelFormat format, int output); int avscale_test_colorspace(...); ... Or alternatively, a perhaps more extensible design: struct AVScaleBackend { int (*test_format)(enum AVPixelFormat format, int output); int (*test_colorspace)(...); ... /* somehow expose per-backend options here? */ }; My over-arching goal here is to be able to unify vf_scale_*, vf_libplacebo, vf_zscale etc. into a single common API, and in doing so, effectively also have a suitable vf_scale_* variant auto-inserted into filter chains. I'm also thinking about supporting per-backend options in this case, since some backends (notably libplacebo) expose about five billion tunable parameters that go beyond the basic "quality preset, scaling mode, dither mode" that I plan on exposing at the top level. Open questions: 1. Is this a good idea, or too confusing / complex to be worth the gain? Specifically, I am worried about confusion arising due to differences in behavior, and implemented options, between all of the above. That said, I think there is a big win to be had from unifying all of the different scaling and/or conversion filters we have in e.g. libavfilter, as well as making it trivial for users of this API to try using e.g. GPU scaling instead of CPU scaling. 2. How should we expose per-backend options? Probably the safest bet is to wrap the relevant options inside AVOptions, as vf_zscale / vf_libplacebo / vf_scale etc. currently do. Alternatively, it could be nice if we could give users direct access to the underlying API configuration structs (pl_render_params, zimg_graph_builder_params, SwsContext, etc.). _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".