From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTP id 6592249558 for ; Wed, 14 Feb 2024 01:56:44 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 8418568D1F4; Wed, 14 Feb 2024 03:55:48 +0200 (EET) Received: from mail-lf1-f49.google.com (mail-lf1-f49.google.com [209.85.167.49]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id D061068D1CB for ; Wed, 14 Feb 2024 03:55:39 +0200 (EET) Received: by mail-lf1-f49.google.com with SMTP id 2adb3069b0e04-511972043c3so1933103e87.1 for ; Tue, 13 Feb 2024 17:55:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1707875738; x=1708480538; darn=ffmpeg.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=KBRdw+LwhVL/bT7XYd7Yhjl5w/jM5TqC9x5HFPTZB94=; b=iu4ZX0wigxd39hZFvdrnmU2L7YO5A4X4p4dlUibfazrXgKxS3QQzy3Vj4tF/tm6kYO 0AhJfATZTuN1PjKbp4UsVAI588/d8M5Hw00dHQvUY8IGVjgVEz2TpJOWZEuGnrem7rB7 tLoSZAtxb6wPMcDYNmlWmONEOeTYNzNXViUqMxgYWJzuLCS7UHGeUvIOYT8vlP4lJJEX Pe6NhfYTgoJVxxOuysRZcgB7Xxgo2EAt2JNHOs+u9QniT0s61wkKo9+sWVrynLYJ8ERA BRJxLIjn0LR5ezCsKLUDfJOZsW5wEjW85i7Tc5MLFCXDQCwuaGU2E0yAUJkolXNy+iJL ZArw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1707875738; x=1708480538; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=KBRdw+LwhVL/bT7XYd7Yhjl5w/jM5TqC9x5HFPTZB94=; b=vg7tqdp+m3x3mSu6DS9hTUHyrRJ+o6IL0kvTQnViK5I94TXuxM8ccbUtEfHZ9DEHSx qLidkWMiGFHkEEOC67lWEwBhiIgbXZLZ8LNRvTxVS7bUwtwjbn/0CzZnotYM8MrGPjym Ri/qDwVr88KyowqIOnMG0ywN5EVe2KgfSbwdTLHiipIfVxChXbBo0LmSSqhDefrXcu/x oynxNGE0WqScgqCvJUI2QLoNLfJcO2iVo0Ssdx8YlnToUNvNj5iW2J6CR1RkBwDCDffl oKtGk9L9wy7BkghnDebwQagd+DK7Z4KOymbwO0x/mOUjc5G+mVwQmoGSV/YJcd5NIBM5 F/Zw== X-Gm-Message-State: AOJu0YyLFSKmGS+Xn3wTWLsGWFBs0wGjz7pw788KoGkwOSs8EfHhlWJR zsMzZrA6P9j8mWClusEnuDMVMhaH9TDjEtJcKUKXAKfMEbU79lAc0GHSuG8NZX0= X-Google-Smtp-Source: AGHT+IH2XXDv+9JS3dMFp/FAmye0mPSU4ZFApRn4OpJFZhVfqJwPQGRJF//hyXMMnuzFHlDYL9qnKg== X-Received: by 2002:ac2:44b9:0:b0:511:87c0:a327 with SMTP id c25-20020ac244b9000000b0051187c0a327mr843519lfm.68.1707875738218; Tue, 13 Feb 2024 17:55:38 -0800 (PST) Received: from dovchinn.amd.com (cable-178-148-16-149.dynamic.sbb.rs. [178.148.16.149]) by smtp.gmail.com with ESMTPSA id vi16-20020a170907d41000b00a3ce31d3ffdsm1663230ejc.93.2024.02.13.17.55.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Feb 2024 17:55:37 -0800 (PST) From: Dmitrii Ovchinnikov To: ffmpeg-devel@ffmpeg.org Date: Wed, 14 Feb 2024 02:55:14 +0100 Message-Id: <20240214015515.1027-8-ovchinnikov.dmitrii@gmail.com> X-Mailer: git-send-email 2.38.1.windows.1 In-Reply-To: <20240214015515.1027-1-ovchinnikov.dmitrii@gmail.com> References: <20240214015515.1027-1-ovchinnikov.dmitrii@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 8/9] avfilter/scale_amf: Add AMF HW scaler & color converter X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Evgeny Pavlov Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: From: Evgeny Pavlov This commit adds two AMF filters: scale_amf & scale_amf_hq. Both filters are using AMF hardware acceleration. scale_amf supports simple scaling algorithms & color conversion. scale_amf_hq supports advanced scaling algorithms & might be used for upscaling only. Signed-off-by: Evgeny Pavlov --- configure | 1 + libavfilter/Makefile | 2 + libavfilter/allfilters.c | 2 + libavfilter/vf_scale_amf.c | 266 +++++++++++++++ libavfilter/vf_scale_amf_common.c | 515 ++++++++++++++++++++++++++++++ libavfilter/vf_scale_amf_common.h | 71 ++++ libavfilter/vf_scale_amf_hq.c | 191 +++++++++++ 7 files changed, 1048 insertions(+) create mode 100644 libavfilter/vf_scale_amf.c create mode 100644 libavfilter/vf_scale_amf_common.c create mode 100644 libavfilter/vf_scale_amf_common.h create mode 100644 libavfilter/vf_scale_amf_hq.c diff --git a/configure b/configure index f72533b7d2..3d1c44d7ae 100755 --- a/configure +++ b/configure @@ -3826,6 +3826,7 @@ rubberband_filter_deps="librubberband" sab_filter_deps="gpl swscale" scale2ref_filter_deps="swscale" scale_filter_deps="swscale" +scale_amf_filter_deps="amf" scale_qsv_filter_deps="libmfx" scale_qsv_filter_select="qsvvpp" scdet_filter_select="scene_sad" diff --git a/libavfilter/Makefile b/libavfilter/Makefile index f6c1d641d6..a87e519b85 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -456,6 +456,8 @@ OBJS-$(CONFIG_ROBERTS_OPENCL_FILTER) += vf_convolution_opencl.o opencl.o OBJS-$(CONFIG_ROTATE_FILTER) += vf_rotate.o OBJS-$(CONFIG_SAB_FILTER) += vf_sab.o OBJS-$(CONFIG_SCALE_FILTER) += vf_scale.o scale_eval.o +OBJS-$(CONFIG_SCALE_AMF_FILTER) += vf_scale_amf.o scale_eval.o vf_scale_amf_common.o +OBJS-$(CONFIG_SCALE_AMF_FILTER) += vf_scale_amf_hq.o scale_eval.o vf_scale_amf_common.o OBJS-$(CONFIG_SCALE_CUDA_FILTER) += vf_scale_cuda.o scale_eval.o \ vf_scale_cuda.ptx.o cuda/load_helper.o OBJS-$(CONFIG_SCALE_NPP_FILTER) += vf_scale_npp.o scale_eval.o diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index 149bf50997..299cfb148b 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -430,6 +430,8 @@ extern const AVFilter ff_vf_roberts_opencl; extern const AVFilter ff_vf_rotate; extern const AVFilter ff_vf_sab; extern const AVFilter ff_vf_scale; +extern const AVFilter ff_vf_scale_amf; +extern const AVFilter ff_vf_scale_amf_hq; extern const AVFilter ff_vf_scale_cuda; extern const AVFilter ff_vf_scale_npp; extern const AVFilter ff_vf_scale_qsv; diff --git a/libavfilter/vf_scale_amf.c b/libavfilter/vf_scale_amf.c new file mode 100644 index 0000000000..6d2c211d83 --- /dev/null +++ b/libavfilter/vf_scale_amf.c @@ -0,0 +1,266 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * scale video filter - AMF + */ + +#include +#include + +#include "libavutil/avassert.h" +#include "libavutil/imgutils.h" +#include "libavutil/opt.h" +#include "libavutil/pixdesc.h" +#include "libavutil/time.h" + +#include "libavutil/hwcontext.h" +#include "libavutil/hwcontext_amf.h" + +#include "AMF/components/VideoConverter.h" +#include "vf_scale_amf_common.h" + +#include "avfilter.h" +#include "formats.h" +#include "video.h" +#include "scale_eval.h" +#include "internal.h" + +#if CONFIG_DXVA2 +#include +#endif + +#if CONFIG_D3D11VA +#include +#endif + +static int amf_scale_query_formats(AVFilterContext *avctx) +{ + const enum AVPixelFormat *output_pix_fmts; + static const enum AVPixelFormat input_pix_fmts[] = { + AV_PIX_FMT_NV12, + AV_PIX_FMT_P010, + AV_PIX_FMT_0RGB, + AV_PIX_FMT_BGR0, + AV_PIX_FMT_BGRA, + AV_PIX_FMT_RGB0, + AV_PIX_FMT_RGBA, + AV_PIX_FMT_GRAY8, + AV_PIX_FMT_YUV420P, + AV_PIX_FMT_YUV420P10, + AV_PIX_FMT_YUYV422, + AV_PIX_FMT_AMF, + AV_PIX_FMT_NONE, + }; + static const enum AVPixelFormat output_pix_fmts_default[] = { + AV_PIX_FMT_AMF, + AV_PIX_FMT_D3D11, + AV_PIX_FMT_DXVA2_VLD, + AV_PIX_FMT_NV12, + AV_PIX_FMT_BGRA, + AV_PIX_FMT_YUV420P, + AV_PIX_FMT_NONE, + }; + output_pix_fmts = output_pix_fmts_default; + + return amf_setup_input_output_formats(avctx, input_pix_fmts, output_pix_fmts); +} + +static int amf_scale_config_output(AVFilterLink *outlink) +{ + AVFilterContext *avctx = outlink->src; + AVFilterLink *inlink = avctx->inputs[0]; + AMFScaleContext *ctx = avctx->priv; + AVAMFDeviceContextInternal * internal = NULL; + AVHWFramesContext *hwframes_out = NULL; + AMFSize out_size; + int err; + AMF_RESULT res; + enum AMF_VIDEO_CONVERTER_COLOR_PROFILE_ENUM amf_color_profile; + enum AVPixelFormat in_format; + + err = amf_init_scale_config(outlink, &in_format); + if (err < 0) + return err; + // FIXME: add checks whether we have HW context + hwframes_out = (AVHWFramesContext*)ctx->hwframes_out_ref->data; + + internal = (AVAMFDeviceContextInternal * )ctx->amf_device_ctx_internal->data; + res = internal->factory->pVtbl->CreateComponent(internal->factory, internal->context, AMFVideoConverter, &ctx->scaler); + AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR_FILTER_NOT_FOUND, "CreateComponent(%ls) failed with error %d\n", AMFVideoConverter, res); + // FIXME: add checks whether we have HW context + AMF_ASSIGN_PROPERTY_INT64(res, ctx->scaler, AMF_VIDEO_CONVERTER_OUTPUT_FORMAT, (amf_int32)av_amf_av_to_amf_format(hwframes_out->sw_format)); + AMF_RETURN_IF_FALSE(avctx, res == AMF_OK, AVERROR_UNKNOWN, "AMFConverter-SetProperty() failed with error %d\n", res); + + out_size.width = outlink->w; + out_size.height = outlink->h; + AMF_ASSIGN_PROPERTY_SIZE(res, ctx->scaler, AMF_VIDEO_CONVERTER_OUTPUT_SIZE, out_size); + AMF_RETURN_IF_FALSE(avctx, res == AMF_OK, AVERROR_UNKNOWN, "AMFConverter-SetProperty() failed with error %d\n", res); + + AMF_ASSIGN_PROPERTY_INT64(res, ctx->scaler, AMF_VIDEO_CONVERTER_SCALE, (amf_int32)ctx->scale_type); + AMF_RETURN_IF_FALSE(avctx, res == AMF_OK, AVERROR_UNKNOWN, "AMFConverter-SetProperty() failed with error %d\n", res); + + amf_color_profile = AMF_VIDEO_CONVERTER_COLOR_PROFILE_UNKNOWN; + + switch(ctx->color_profile) { + case AMF_VIDEO_CONVERTER_COLOR_PROFILE_601: + if (ctx->color_range == AMF_COLOR_RANGE_FULL) { + amf_color_profile = AMF_VIDEO_CONVERTER_COLOR_PROFILE_FULL_601; + } else { + amf_color_profile = AMF_VIDEO_CONVERTER_COLOR_PROFILE_601; + } + break; + case AMF_VIDEO_CONVERTER_COLOR_PROFILE_709: + if (ctx->color_range == AMF_COLOR_RANGE_FULL) { + amf_color_profile = AMF_VIDEO_CONVERTER_COLOR_PROFILE_FULL_709; + } else { + amf_color_profile = AMF_VIDEO_CONVERTER_COLOR_PROFILE_709; + } + break; + case AMF_VIDEO_CONVERTER_COLOR_PROFILE_2020: + if (ctx->color_range == AMF_COLOR_RANGE_FULL) { + amf_color_profile = AMF_VIDEO_CONVERTER_COLOR_PROFILE_FULL_2020; + } else { + amf_color_profile = AMF_VIDEO_CONVERTER_COLOR_PROFILE_2020; + } + break; + default: + amf_color_profile = AMF_VIDEO_CONVERTER_COLOR_PROFILE_UNKNOWN; + break; + } + + if (amf_color_profile != AMF_VIDEO_CONVERTER_COLOR_PROFILE_UNKNOWN) { + AMF_ASSIGN_PROPERTY_INT64(res, ctx->scaler, AMF_VIDEO_CONVERTER_COLOR_PROFILE, amf_color_profile); + } + + if (ctx->color_range != AMF_COLOR_RANGE_UNDEFINED) { + AMF_ASSIGN_PROPERTY_INT64(res, ctx->scaler, AMF_VIDEO_CONVERTER_OUTPUT_COLOR_RANGE, ctx->color_range); + } + + if (ctx->primaries != AMF_COLOR_PRIMARIES_UNDEFINED) { + AMF_ASSIGN_PROPERTY_INT64(res, ctx->scaler, AMF_VIDEO_CONVERTER_OUTPUT_COLOR_PRIMARIES, ctx->primaries); + } + + if (ctx->trc != AMF_COLOR_TRANSFER_CHARACTERISTIC_UNDEFINED) { + AMF_ASSIGN_PROPERTY_INT64(res, ctx->scaler, AMF_VIDEO_CONVERTER_OUTPUT_TRANSFER_CHARACTERISTIC, ctx->trc); + } + + res = ctx->scaler->pVtbl->Init(ctx->scaler, av_amf_av_to_amf_format(in_format), inlink->w, inlink->h); + AMF_RETURN_IF_FALSE(avctx, res == AMF_OK, AVERROR_UNKNOWN, "AMFConverter-Init() failed with error %d\n", res); + + return 0; +} + +#define OFFSET(x) offsetof(AMFScaleContext, x) +#define FLAGS AV_OPT_FLAG_VIDEO_PARAM|AV_OPT_FLAG_FILTERING_PARAM +static const AVOption scale_amf_options[] = { + { "w", "Output video width", OFFSET(w_expr), AV_OPT_TYPE_STRING, { .str = "iw" }, .flags = FLAGS }, + { "h", "Output video height", OFFSET(h_expr), AV_OPT_TYPE_STRING, { .str = "ih" }, .flags = FLAGS }, + { "format", "Output pixel format", OFFSET(format_str), AV_OPT_TYPE_STRING, { .str = "same" }, .flags = FLAGS }, + + { "scale_type", "Scale type", OFFSET(scale_type), AV_OPT_TYPE_INT, { .i64 = AMF_VIDEO_CONVERTER_SCALE_BILINEAR }, AMF_VIDEO_CONVERTER_SCALE_BILINEAR, AMF_VIDEO_CONVERTER_SCALE_BICUBIC, FLAGS, "scale_type" }, + { "bilinear", "Bilinear", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_VIDEO_CONVERTER_SCALE_BILINEAR }, 0, 0, FLAGS, "scale_type" }, + { "bicubic", "Bicubic", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_VIDEO_CONVERTER_SCALE_BICUBIC }, 0, 0, FLAGS, "scale_type" }, + + { "color_profile", "Color profile", OFFSET(color_profile), AV_OPT_TYPE_INT, { .i64 = AMF_VIDEO_CONVERTER_COLOR_PROFILE_UNKNOWN }, AMF_VIDEO_CONVERTER_COLOR_PROFILE_UNKNOWN, AMF_VIDEO_CONVERTER_COLOR_PROFILE_FULL_2020, FLAGS, "color_profile" }, + { "bt601", "BT.601", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_VIDEO_CONVERTER_COLOR_PROFILE_601 }, 0, 0, FLAGS, "color_profile" }, + { "bt709", "BT.709", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_VIDEO_CONVERTER_COLOR_PROFILE_709 }, 0, 0, FLAGS, "color_profile" }, + { "bt2020", "BT.2020", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_VIDEO_CONVERTER_COLOR_PROFILE_2020 }, 0, 0, FLAGS, "color_profile" }, + + { "color_range", "Color range", OFFSET(color_range), AV_OPT_TYPE_INT, { .i64 = AMF_COLOR_RANGE_UNDEFINED }, AMF_COLOR_RANGE_UNDEFINED, AMF_COLOR_RANGE_FULL, FLAGS, "color_range" }, + { "studio", "Studio", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_COLOR_RANGE_STUDIO }, 0, 0, FLAGS, "color_range" }, + { "full", "Full", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_COLOR_RANGE_FULL }, 0, 0, FLAGS, "color_range" }, + + { "primaries", "Output color primaries", OFFSET(primaries), AV_OPT_TYPE_INT, { .i64 = AMF_COLOR_PRIMARIES_UNDEFINED }, AMF_COLOR_PRIMARIES_UNDEFINED, AMF_COLOR_PRIMARIES_JEDEC_P22, FLAGS, "primaries" }, + { "bt709", "BT.709", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_COLOR_PRIMARIES_BT709 }, 0, 0, FLAGS, "primaries" }, + { "bt470m", "BT.470M", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_COLOR_PRIMARIES_BT470M }, 0, 0, FLAGS, "primaries" }, + { "bt470bg", "BT.470BG", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_COLOR_PRIMARIES_BT470BG }, 0, 0, FLAGS, "primaries" }, + { "smpte170m", "SMPTE170M", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_COLOR_PRIMARIES_SMPTE170M }, 0, 0, FLAGS, "primaries" }, + { "smpte240m", "SMPTE240M", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_COLOR_PRIMARIES_SMPTE240M }, 0, 0, FLAGS, "primaries" }, + { "film", "FILM", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_COLOR_PRIMARIES_FILM }, 0, 0, FLAGS, "primaries" }, + { "bt2020", "BT2020", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_COLOR_PRIMARIES_BT2020 }, 0, 0, FLAGS, "primaries" }, + { "smpte428", "SMPTE428", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_COLOR_PRIMARIES_SMPTE428 }, 0, 0, FLAGS, "primaries" }, + { "smpte431", "SMPTE431", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_COLOR_PRIMARIES_SMPTE431 }, 0, 0, FLAGS, "primaries" }, + { "smpte432", "SMPTE432", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_COLOR_PRIMARIES_SMPTE432 }, 0, 0, FLAGS, "primaries" }, + { "jedec-p22", "JEDEC_P22", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_COLOR_PRIMARIES_JEDEC_P22 }, 0, 0, FLAGS, "primaries" }, + + { "trc", "Output transfer characteristics", OFFSET(trc), AV_OPT_TYPE_INT, { .i64 = AMF_COLOR_TRANSFER_CHARACTERISTIC_UNDEFINED }, AMF_COLOR_TRANSFER_CHARACTERISTIC_UNDEFINED, AMF_COLOR_TRANSFER_CHARACTERISTIC_ARIB_STD_B67, FLAGS, "trc" }, + { "bt709", "BT.709", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_COLOR_TRANSFER_CHARACTERISTIC_BT709 }, 0, 0, FLAGS, "trc" }, + { "gamma22", "GAMMA22", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_COLOR_TRANSFER_CHARACTERISTIC_GAMMA22 }, 0, 0, FLAGS, "trc" }, + { "gamma28", "GAMMA28", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_COLOR_TRANSFER_CHARACTERISTIC_GAMMA28 }, 0, 0, FLAGS, "trc" }, + { "smpte170m", "SMPTE170M", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_COLOR_TRANSFER_CHARACTERISTIC_SMPTE170M }, 0, 0, FLAGS, "trc" }, + { "smpte240m", "SMPTE240M", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_COLOR_TRANSFER_CHARACTERISTIC_SMPTE240M }, 0, 0, FLAGS, "trc" }, + { "linear", "Linear", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_COLOR_TRANSFER_CHARACTERISTIC_LINEAR }, 0, 0, FLAGS, "trc" }, + { "log", "LOG", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_COLOR_TRANSFER_CHARACTERISTIC_LOG }, 0, 0, FLAGS, "trc" }, + { "log-sqrt", "LOG_SQRT", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_COLOR_TRANSFER_CHARACTERISTIC_LOG_SQRT }, 0, 0, FLAGS, "trc" }, + { "iec61966-2-4", "IEC61966_2_4", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_COLOR_TRANSFER_CHARACTERISTIC_IEC61966_2_4 }, 0, 0, FLAGS, "trc" }, + { "bt1361-ecg", "BT1361_ECG", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_COLOR_TRANSFER_CHARACTERISTIC_BT1361_ECG }, 0, 0, FLAGS, "trc" }, + { "iec61966-2-1", "IEC61966_2_1", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_COLOR_TRANSFER_CHARACTERISTIC_IEC61966_2_1 }, 0, 0, FLAGS, "trc" }, + { "bt2020-10", "BT.2020_10", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_COLOR_TRANSFER_CHARACTERISTIC_BT2020_10 }, 0, 0, FLAGS, "trc" }, + { "bt2020-12", "BT.2020-12", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_COLOR_TRANSFER_CHARACTERISTIC_BT2020_12 }, 0, 0, FLAGS, "trc" }, + { "smpte2084", "SMPTE2084", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_COLOR_TRANSFER_CHARACTERISTIC_SMPTE2084 }, 0, 0, FLAGS, "trc" }, + { "smpte428", "SMPTE428", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_COLOR_TRANSFER_CHARACTERISTIC_SMPTE428 }, 0, 0, FLAGS, "trc" }, + { "arib-std-b67", "ARIB_STD_B67", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_COLOR_TRANSFER_CHARACTERISTIC_ARIB_STD_B67 }, 0, 0, FLAGS, "trc" }, + + { "force_original_aspect_ratio", "decrease or increase w/h if necessary to keep the original AR", OFFSET(force_original_aspect_ratio), AV_OPT_TYPE_INT, { .i64 = 0}, 0, 2, FLAGS, "force_oar" }, + { "disable", NULL, 0, AV_OPT_TYPE_CONST, {.i64 = 0 }, 0, 0, FLAGS, "force_oar" }, + { "decrease", NULL, 0, AV_OPT_TYPE_CONST, {.i64 = 1 }, 0, 0, FLAGS, "force_oar" }, + { "increase", NULL, 0, AV_OPT_TYPE_CONST, {.i64 = 2 }, 0, 0, FLAGS, "force_oar" }, + { "force_divisible_by", "enforce that the output resolution is divisible by a defined integer when force_original_aspect_ratio is used", OFFSET(force_divisible_by), AV_OPT_TYPE_INT, { .i64 = 1}, 1, 256, FLAGS }, + + { NULL }, +}; + + +AVFILTER_DEFINE_CLASS(scale_amf); + +static const AVFilterPad amf_scale_inputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + .filter_frame = amf_scale_filter_frame, + } +}; + +static const AVFilterPad amf_scale_outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + .config_props = amf_scale_config_output, + } +}; + +AVFilter ff_vf_scale_amf = { + .name = "scale_amf", + .description = NULL_IF_CONFIG_SMALL("AMF video scaling and format conversion"), + + .init = amf_scale_init, + .uninit = amf_scale_uninit, + FILTER_QUERY_FUNC(&amf_scale_query_formats), + + .priv_size = sizeof(AMFScaleContext), + .priv_class = &scale_amf_class, + + FILTER_INPUTS(amf_scale_inputs), + FILTER_OUTPUTS(amf_scale_outputs), + FILTER_SINGLE_PIXFMT(AV_PIX_FMT_AMF), + + .flags_internal = FF_FILTER_FLAG_HWFRAME_AWARE, + .flags = AVFILTER_FLAG_HWDEVICE, +}; diff --git a/libavfilter/vf_scale_amf_common.c b/libavfilter/vf_scale_amf_common.c new file mode 100644 index 0000000000..5c26ebd2c9 --- /dev/null +++ b/libavfilter/vf_scale_amf_common.c @@ -0,0 +1,515 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "vf_scale_amf_common.h" + +#include "libavutil/avassert.h" +#include "avfilter.h" +#include "internal.h" +#include "formats.h" +#include "libavutil/imgutils.h" + +#include "libavutil/hwcontext_amf.h" +#include "AMF/components/ColorSpace.h" +#include "scale_eval.h" + +#if CONFIG_DXVA2 +#include +#endif + +#if CONFIG_D3D11VA +#include +#endif + +int amf_scale_init(AVFilterContext *avctx) +{ + AMFScaleContext *ctx = avctx->priv; + + if (!strcmp(ctx->format_str, "same")) { + ctx->format = AV_PIX_FMT_NONE; + } else { + ctx->format = av_get_pix_fmt(ctx->format_str); + if (ctx->format == AV_PIX_FMT_NONE) { + av_log(avctx, AV_LOG_ERROR, "Unrecognized pixel format: %s\n", ctx->format_str); + return AVERROR(EINVAL); + } + } + + return 0; +} + +void amf_scale_uninit(AVFilterContext *avctx) +{ + AMFScaleContext *ctx = avctx->priv; + + if (ctx->scaler) { + ctx->scaler->pVtbl->Terminate(ctx->scaler); + ctx->scaler->pVtbl->Release(ctx->scaler); + ctx->scaler = NULL; + } + + av_buffer_unref(&ctx->amf_device_ref); + av_buffer_unref(&ctx->hwdevice_ref); + av_buffer_unref(&ctx->hwframes_in_ref); + av_buffer_unref(&ctx->hwframes_out_ref); +} + +int amf_scale_filter_frame(AVFilterLink *inlink, AVFrame *in) +{ + AVFilterContext *avctx = inlink->dst; + AMFScaleContext *ctx = avctx->priv; + AVFilterLink *outlink = avctx->outputs[0]; + AMF_RESULT res; + AMFSurface *surface_in; + AMFSurface *surface_out; + AMFData *data_out = NULL; + enum AVColorSpace out_colorspace; + enum AVColorRange out_color_range; + + AVFrame *out = NULL; + int ret = 0; + + if (!ctx->scaler) + return AVERROR(EINVAL); + + ret = amf_avframe_to_amfsurface(avctx, in, &surface_in); + if (ret < 0) + goto fail; + + res = ctx->scaler->pVtbl->SubmitInput(ctx->scaler, (AMFData*)surface_in); + AMF_GOTO_FAIL_IF_FALSE(avctx, res == AMF_OK, AVERROR_UNKNOWN, "SubmitInput() failed with error %d\n", res); + res = ctx->scaler->pVtbl->QueryOutput(ctx->scaler, &data_out); + AMF_GOTO_FAIL_IF_FALSE(avctx, res == AMF_OK, AVERROR_UNKNOWN, "QueryOutput() failed with error %d\n", res); + + if (data_out) { + AMFGuid guid = IID_AMFSurface(); + data_out->pVtbl->QueryInterface(data_out, &guid, (void**)&surface_out); // query for buffer interface + data_out->pVtbl->Release(data_out); + } + + out = amf_amfsurface_to_avframe(avctx, surface_out); + + ret = av_frame_copy_props(out, in); + av_frame_unref(in); + + out_colorspace = AVCOL_SPC_UNSPECIFIED; + + if (ctx->color_profile != AMF_VIDEO_CONVERTER_COLOR_PROFILE_UNKNOWN) { + switch(ctx->color_profile) { + case AMF_VIDEO_CONVERTER_COLOR_PROFILE_601: + out_colorspace = AVCOL_SPC_SMPTE170M; + break; + case AMF_VIDEO_CONVERTER_COLOR_PROFILE_709: + out_colorspace = AVCOL_SPC_BT709; + break; + case AMF_VIDEO_CONVERTER_COLOR_PROFILE_2020: + out_colorspace = AVCOL_SPC_BT2020_NCL; + break; + case AMF_VIDEO_CONVERTER_COLOR_PROFILE_JPEG: + out_colorspace = AVCOL_SPC_RGB; + break; + default: + out_colorspace = AVCOL_SPC_UNSPECIFIED; + break; + } + out->colorspace = out_colorspace; + } + + out_color_range = AVCOL_RANGE_UNSPECIFIED; + if (ctx->color_range == AMF_COLOR_RANGE_FULL) + out_color_range = AVCOL_RANGE_JPEG; + else if (ctx->color_range == AMF_COLOR_RANGE_STUDIO) + out_color_range = AVCOL_RANGE_MPEG; + + if (ctx->color_range != AMF_COLOR_RANGE_UNDEFINED) + out->color_range = out_color_range; + + if (ctx->primaries != AMF_COLOR_PRIMARIES_UNDEFINED) + out->color_primaries = ctx->primaries; + + if (ctx->trc != AMF_COLOR_TRANSFER_CHARACTERISTIC_UNDEFINED) + out->color_trc = ctx->trc; + + + if (ret < 0) + goto fail; + + out->format = outlink->format; + out->width = outlink->w; + out->height = outlink->h; + + out->hw_frames_ctx = av_buffer_ref(ctx->hwframes_out_ref); + if (!out->hw_frames_ctx) { + ret = AVERROR(ENOMEM); + goto fail; + } + + if (inlink->sample_aspect_ratio.num) { + outlink->sample_aspect_ratio = av_mul_q((AVRational){outlink->h * inlink->w, outlink->w * inlink->h}, inlink->sample_aspect_ratio); + } else + outlink->sample_aspect_ratio = inlink->sample_aspect_ratio; + + av_frame_free(&in); + return ff_filter_frame(outlink, out); +fail: + av_frame_free(&in); + av_frame_free(&out); + return ret; +} + + + +int amf_setup_input_output_formats(AVFilterContext *avctx, + const enum AVPixelFormat *input_pix_fmts, + const enum AVPixelFormat *output_pix_fmts) +{ + int err; + int i; + AVFilterFormats *input_formats; + + //in case if hw_device_ctx is set to DXVA2 we change order of pixel formats to set DXVA2 be choosen by default + //The order is ignored if hw_frames_ctx is not NULL on the config_output stage + if (avctx->hw_device_ctx) { + AVHWDeviceContext *device_ctx = (AVHWDeviceContext*)avctx->hw_device_ctx->data; + + switch (device_ctx->type) { + #if CONFIG_D3D11VA + case AV_HWDEVICE_TYPE_D3D11VA: + { + static const enum AVPixelFormat output_pix_fmts_d3d11[] = { + AV_PIX_FMT_D3D11, + AV_PIX_FMT_NONE, + }; + output_pix_fmts = output_pix_fmts_d3d11; + } + break; + #endif + #if CONFIG_DXVA2 + case AV_HWDEVICE_TYPE_DXVA2: + { + static const enum AVPixelFormat output_pix_fmts_dxva2[] = { + AV_PIX_FMT_DXVA2_VLD, + AV_PIX_FMT_NONE, + }; + output_pix_fmts = output_pix_fmts_dxva2; + } + break; + #endif + default: + { + av_log(avctx, AV_LOG_ERROR, "Unsupported device : %s\n", av_hwdevice_get_type_name(device_ctx->type)); + return AVERROR(EINVAL); + } + break; + } + } + + input_formats = ff_make_format_list(output_pix_fmts); + if (!input_formats) { + return AVERROR(ENOMEM); + } + + for (i = 0; input_pix_fmts[i] != AV_PIX_FMT_NONE; i++) { + err = ff_add_format(&input_formats, input_pix_fmts[i]); + if (err < 0) + return err; + } + + if ((err = ff_formats_ref(input_formats, &avctx->inputs[0]->outcfg.formats)) < 0 || + (err = ff_formats_ref(ff_make_format_list(output_pix_fmts), + &avctx->outputs[0]->incfg.formats)) < 0) + return err; + return 0; +} + +int amf_copy_surface(AVFilterContext *avctx, const AVFrame *frame, + AMFSurface* surface) +{ + AMFPlane *plane; + uint8_t *dst_data[4]; + int dst_linesize[4]; + int planes; + int i; + + planes = surface->pVtbl->GetPlanesCount(surface); + av_assert0(planes < FF_ARRAY_ELEMS(dst_data)); + + for (i = 0; i < planes; i++) { + plane = surface->pVtbl->GetPlaneAt(surface, i); + dst_data[i] = plane->pVtbl->GetNative(plane); + dst_linesize[i] = plane->pVtbl->GetHPitch(plane); + } + av_image_copy(dst_data, dst_linesize, + (const uint8_t**)frame->data, frame->linesize, frame->format, + frame->width, frame->height); + + return 0; +} + +int amf_init_scale_config(AVFilterLink *outlink, enum AVPixelFormat *in_format) +{ + AVFilterContext *avctx = outlink->src; + AVFilterLink *inlink = avctx->inputs[0]; + AMFScaleContext *ctx = avctx->priv; + AVHWFramesContext *hwframes_out; + int err; + AMF_RESULT res; + + if ((err = ff_scale_eval_dimensions(avctx, + ctx->w_expr, ctx->h_expr, + inlink, outlink, + &ctx->width, &ctx->height)) < 0) + return err; + + ff_scale_adjust_dimensions(inlink, &ctx->width, &ctx->height, + ctx->force_original_aspect_ratio, ctx->force_divisible_by); + + av_buffer_unref(&ctx->amf_device_ref); + av_buffer_unref(&ctx->hwframes_in_ref); + av_buffer_unref(&ctx->hwframes_out_ref); + + if (inlink->hw_frames_ctx) { + AVHWFramesContext *frames_ctx = (AVHWFramesContext*)inlink->hw_frames_ctx->data; + if (frames_ctx->device_ctx->type == AV_HWDEVICE_TYPE_AMF) { + AVAMFDeviceContext * amf_ctx = frames_ctx->device_ctx->hwctx; + ctx->amf_device_ctx_internal = av_buffer_ref(amf_ctx->internal); + } + if (av_amf_av_to_amf_format(frames_ctx->sw_format) == AMF_SURFACE_UNKNOWN) { + av_log(avctx, AV_LOG_ERROR, "Format of input frames context (%s) is not supported by AMF.\n", + av_get_pix_fmt_name(frames_ctx->sw_format)); + return AVERROR(EINVAL); + } + + err = av_hwdevice_ctx_create_derived(&ctx->amf_device_ref, AV_HWDEVICE_TYPE_AMF, frames_ctx->device_ref, 0); + if (err < 0) + return err; + + ctx->hwframes_in_ref = av_buffer_ref(inlink->hw_frames_ctx); + if (!ctx->hwframes_in_ref) + return AVERROR(ENOMEM); + + ctx->hwframes_out_ref = av_hwframe_ctx_alloc(frames_ctx->device_ref); + if (!ctx->hwframes_out_ref) + return AVERROR(ENOMEM); + + hwframes_out = (AVHWFramesContext*)ctx->hwframes_out_ref->data; + hwframes_out->format = outlink->format; + hwframes_out->sw_format = frames_ctx->sw_format; + } else if (avctx->hw_device_ctx) { + AVHWDeviceContext *hwdev_ctx; + err = av_hwdevice_ctx_create_derived(&ctx->amf_device_ref, AV_HWDEVICE_TYPE_AMF, avctx->hw_device_ctx, 0); + if (err < 0) + return err; + hwdev_ctx = (AVHWDeviceContext*)avctx->hw_device_ctx->data; + if (hwdev_ctx->type == AV_HWDEVICE_TYPE_AMF) + { + AVAMFDeviceContext * amf_ctx = hwdev_ctx->hwctx; + ctx->amf_device_ctx_internal = av_buffer_ref(amf_ctx->internal); + } + ctx->hwdevice_ref = av_buffer_ref(avctx->hw_device_ctx); + if (!ctx->hwdevice_ref) + return AVERROR(ENOMEM); + + ctx->hwframes_out_ref = av_hwframe_ctx_alloc(ctx->hwdevice_ref); + if (!ctx->hwframes_out_ref) + return AVERROR(ENOMEM); + + hwframes_out = (AVHWFramesContext*)ctx->hwframes_out_ref->data; + hwframes_out->format = AV_PIX_FMT_AMF; + hwframes_out->sw_format = outlink->format; + } else { + AVAMFDeviceContextInternal *wrapped = av_mallocz(sizeof(*wrapped)); + ctx->amf_device_ctx_internal = av_buffer_create((uint8_t *)wrapped, sizeof(*wrapped), + av_amf_context_internal_free, NULL, 0); + if ((res == av_amf_context_internal_create((AVAMFDeviceContextInternal *)ctx->amf_device_ctx_internal->data, avctx, "", NULL, 0)) != 0) { + return res; + } + ctx->hwframes_out_ref = av_hwframe_ctx_alloc(ctx->amf_device_ref); + if (!ctx->hwframes_out_ref) + return AVERROR(ENOMEM); + + hwframes_out = (AVHWFramesContext*)ctx->hwframes_out_ref->data; + hwframes_out->format = outlink->format; + hwframes_out->sw_format = inlink->format; + } + + if (ctx->format != AV_PIX_FMT_NONE) { + hwframes_out->sw_format = ctx->format; + } + + if (inlink->format == AV_PIX_FMT_AMF) { + if (!inlink->hw_frames_ctx || !inlink->hw_frames_ctx->data) + return AVERROR(EINVAL); + else + *in_format = ((AVHWFramesContext*)inlink->hw_frames_ctx->data)->sw_format; + } else + *in_format = inlink->format; + + outlink->w = ctx->width; + outlink->h = ctx->height; + + hwframes_out->width = outlink->w; + hwframes_out->height = outlink->h; + + err = av_hwframe_ctx_init(ctx->hwframes_out_ref); + if (err < 0) + return err; + + outlink->hw_frames_ctx = av_buffer_ref(ctx->hwframes_out_ref); + if (!outlink->hw_frames_ctx) { + return AVERROR(ENOMEM); + } + return 0; +} + +void amf_free_amfsurface(void *opaque, uint8_t *data) +{ + AMFSurface *surface = (AMFSurface*)data; + surface->pVtbl->Release(surface); +} + +AVFrame *amf_amfsurface_to_avframe(AVFilterContext *avctx, AMFSurface* pSurface) +{ + AVFrame *frame = av_frame_alloc(); + AMFScaleContext *ctx = avctx->priv; + + if (!frame) + return NULL; + + if (ctx->hwframes_out_ref) { + AVHWFramesContext *hwframes_out = (AVHWFramesContext *)ctx->hwframes_out_ref->data; + if (hwframes_out->format == AV_PIX_FMT_AMF) { + int ret = av_hwframe_get_buffer(ctx->hwframes_out_ref, frame, 0); + if (ret < 0) { + av_log(avctx, AV_LOG_ERROR, "Get hw frame failed.\n"); + av_frame_free(&frame); + return NULL; + } + frame->data[3] = (uint8_t *)pSurface; + frame->buf[1] = av_buffer_create((uint8_t *)pSurface, sizeof(AMFSurface), + amf_free_amfsurface, + (void*)avctx, + AV_BUFFER_FLAG_READONLY); + } else { // FIXME: add processing of other hw formats + av_log(ctx, AV_LOG_ERROR, "Unknown pixel format\n"); + return NULL; + } + } else { + + switch (pSurface->pVtbl->GetMemoryType(pSurface)) + { + #if CONFIG_D3D11VA + case AMF_MEMORY_DX11: + { + AMFPlane *plane0 = pSurface->pVtbl->GetPlaneAt(pSurface, 0); + frame->data[0] = plane0->pVtbl->GetNative(plane0); + frame->data[1] = (uint8_t*)(intptr_t)0; + + frame->buf[0] = av_buffer_create(NULL, + 0, + amf_free_amfsurface, + pSurface, + AV_BUFFER_FLAG_READONLY); + } + break; + #endif + #if CONFIG_DXVA2 + case AMF_MEMORY_DX9: + { + AMFPlane *plane0 = pSurface->pVtbl->GetPlaneAt(pSurface, 0); + frame->data[3] = plane0->pVtbl->GetNative(plane0); + + frame->buf[0] = av_buffer_create(NULL, + 0, + amf_free_amfsurface, + pSurface, + AV_BUFFER_FLAG_READONLY); + } + break; + #endif + default: + { + av_log(avctx, AV_LOG_ERROR, "Unsupported memory type : %d\n", pSurface->pVtbl->GetMemoryType(pSurface)); + return NULL; + } + } + } + + return frame; +} + +int amf_avframe_to_amfsurface(AVFilterContext *avctx, const AVFrame *frame, AMFSurface** ppSurface) +{ + AMFScaleContext *ctx = avctx->priv; + AVAMFDeviceContextInternal* internal = (AVAMFDeviceContextInternal *)ctx->amf_device_ctx_internal->data; + AMFSurface *surface; + AMF_RESULT res; + int hw_surface = 0; + + switch (frame->format) { +#if CONFIG_D3D11VA + case AV_PIX_FMT_D3D11: + { + static const GUID AMFTextureArrayIndexGUID = { 0x28115527, 0xe7c3, 0x4b66, { 0x99, 0xd3, 0x4f, 0x2a, 0xe6, 0xb4, 0x7f, 0xaf } }; + ID3D11Texture2D *texture = (ID3D11Texture2D*)frame->data[0]; // actual texture + int index = (intptr_t)frame->data[1]; // index is a slice in texture array is - set to tell AMF which slice to use + texture->lpVtbl->SetPrivateData(texture, &AMFTextureArrayIndexGUID, sizeof(index), &index); + + res = internal->context->pVtbl->CreateSurfaceFromDX11Native(internal->context, texture, &surface, NULL); // wrap to AMF surface + AMF_RETURN_IF_FALSE(avctx, res == AMF_OK, AVERROR(ENOMEM), "CreateSurfaceFromDX11Native() failed with error %d\n", res); + hw_surface = 1; + } + break; +#endif + case AV_PIX_FMT_AMF: + { + surface = (AMFSurface*)frame->data[3]; // actual surface + hw_surface = 1; + } + break; + +#if CONFIG_DXVA2 + case AV_PIX_FMT_DXVA2_VLD: + { + IDirect3DSurface9 *texture = (IDirect3DSurface9 *)frame->data[3]; // actual texture + + res = internal->context->pVtbl->CreateSurfaceFromDX9Native(internal->context, texture, &surface, NULL); // wrap to AMF surface + AMF_RETURN_IF_FALSE(avctx, res == AMF_OK, AVERROR(ENOMEM), "CreateSurfaceFromDX9Native() failed with error %d\n", res); + hw_surface = 1; + } + break; +#endif + default: + { + AMF_SURFACE_FORMAT amf_fmt = av_amf_av_to_amf_format(frame->format); + res = internal->context->pVtbl->AllocSurface(internal->context, AMF_MEMORY_HOST, amf_fmt, frame->width, frame->height, &surface); + AMF_RETURN_IF_FALSE(avctx, res == AMF_OK, AVERROR(ENOMEM), "AllocSurface() failed with error %d\n", res); + amf_copy_surface(avctx, frame, surface); + } + break; + } + + if (hw_surface) { + // input HW surfaces can be vertically aligned by 16; tell AMF the real size + surface->pVtbl->SetCrop(surface, 0, 0, frame->width, frame->height); + } + + surface->pVtbl->SetPts(surface, frame->pts); + *ppSurface = surface; + return 0; +} diff --git a/libavfilter/vf_scale_amf_common.h b/libavfilter/vf_scale_amf_common.h new file mode 100644 index 0000000000..a43c7602cb --- /dev/null +++ b/libavfilter/vf_scale_amf_common.h @@ -0,0 +1,71 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#ifndef AVFILTER_AMF_COMMON_H +#define AVFILTER_AMF_COMMON_H + +#include "avfilter.h" + +#include "AMF/core/Surface.h" +#include "AMF/components/Component.h" + +typedef struct AMFScaleContext { + const AVClass *class; + + int width, height; + enum AVPixelFormat format; + int scale_type; + int color_profile; + int color_range; + int primaries; + int trc; + int fill; + int fill_color; + int keep_ratio; + + // HQScaler properties + int algorithm; + float sharpness; + + char *w_expr; + char *h_expr; + char *format_str; + int force_original_aspect_ratio; + int force_divisible_by; + + AMFComponent *scaler; + AVBufferRef *amf_device_ref; + + AVBufferRef *hwframes_in_ref; + AVBufferRef *hwframes_out_ref; + AVBufferRef *hwdevice_ref; + + AVBufferRef *amf_device_ctx_internal; +} AMFScaleContext; + +int amf_scale_init(AVFilterContext *avctx); +void amf_scale_uninit(AVFilterContext *avctx); +int amf_init_scale_config(AVFilterLink *outlink, enum AVPixelFormat *in_format); +int amf_copy_surface(AVFilterContext *avctx, const AVFrame *frame, AMFSurface* surface); +void amf_free_amfsurface(void *opaque, uint8_t *data); +AVFrame *amf_amfsurface_to_avframe(AVFilterContext *avctx, AMFSurface* pSurface); +int amf_avframe_to_amfsurface(AVFilterContext *avctx, const AVFrame *frame, AMFSurface** ppSurface); +int amf_setup_input_output_formats(AVFilterContext *avctx, const enum AVPixelFormat *input_pix_fmts, const enum AVPixelFormat *output_pix_fmts); +int amf_scale_filter_frame(AVFilterLink *inlink, AVFrame *in); + +#endif /* AVFILTER_AMF_COMMON_H */ diff --git a/libavfilter/vf_scale_amf_hq.c b/libavfilter/vf_scale_amf_hq.c new file mode 100644 index 0000000000..63c9bd8301 --- /dev/null +++ b/libavfilter/vf_scale_amf_hq.c @@ -0,0 +1,191 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * scale video filter - AMF + */ + +#include +#include + +#include "libavutil/avassert.h" +#include "libavutil/imgutils.h" +#include "libavutil/opt.h" +#include "libavutil/pixdesc.h" +#include "libavutil/time.h" + +#include "libavutil/hwcontext.h" +#include "libavutil/hwcontext_amf.h" + +#include "AMF/components/HQScaler.h" +#include "AMF/components/ColorSpace.h" +#include "vf_scale_amf_common.h" + +#include "avfilter.h" +#include "internal.h" +#include "formats.h" +#include "video.h" + +#if CONFIG_DXVA2 +#include +#endif + +#if CONFIG_D3D11VA +#include +#endif + + +static int amf_scale_query_formats(AVFilterContext *avctx) +{ + const enum AVPixelFormat *output_pix_fmts; + static const enum AVPixelFormat input_pix_fmts[] = { + AV_PIX_FMT_NV12, + AV_PIX_FMT_P010, + AV_PIX_FMT_BGRA, + AV_PIX_FMT_RGBA, + AV_PIX_FMT_AMF, + AV_PIX_FMT_RGBAF16, + AV_PIX_FMT_NONE, + }; + static const enum AVPixelFormat output_pix_fmts_default[] = { + AV_PIX_FMT_NV12, + AV_PIX_FMT_P010, + AV_PIX_FMT_BGRA, + AV_PIX_FMT_RGBA, + AV_PIX_FMT_AMF, + AV_PIX_FMT_D3D11, + AV_PIX_FMT_DXVA2_VLD, + AV_PIX_FMT_RGBAF16, + AV_PIX_FMT_NONE, + }; + output_pix_fmts = output_pix_fmts_default; + + return amf_setup_input_output_formats(avctx, input_pix_fmts, output_pix_fmts); +} + +static int amf_scale_config_output(AVFilterLink *outlink) +{ + AVFilterContext *avctx = outlink->src; + AVFilterLink *inlink = avctx->inputs[0]; + AMFScaleContext *ctx = avctx->priv; + AVAMFDeviceContextInternal * internal = NULL; + AMFSize out_size; + int err; + AMF_RESULT res; + enum AVPixelFormat in_format; + + err = amf_init_scale_config(outlink, &in_format); + if (err < 0) + return err; + + // HQ scaler should be used for upscaling only + if (inlink->w > outlink->w || inlink->h > outlink->h) { + av_log(avctx, AV_LOG_ERROR, "AMF HQ scaler should be used for upscaling only.\n"); + return AVERROR_UNKNOWN; + } + // FIXME: add checks whether we have HW context + + internal = (AVAMFDeviceContextInternal * )ctx->amf_device_ctx_internal->data; + res = internal->factory->pVtbl->CreateComponent(internal->factory, internal->context, AMFHQScaler, &ctx->scaler); + AMF_RETURN_IF_FALSE(ctx, res == AMF_OK, AVERROR_FILTER_NOT_FOUND, "CreateComponent(%ls) failed with error %d\n", AMFHQScaler, res); + + out_size.width = outlink->w; + out_size.height = outlink->h; + AMF_ASSIGN_PROPERTY_SIZE(res, ctx->scaler, AMF_HQ_SCALER_OUTPUT_SIZE, out_size); + AMF_RETURN_IF_FALSE(avctx, res == AMF_OK, AVERROR_UNKNOWN, "AMFHQScaler-SetProperty() failed with error %d\n", res); + + if (ctx->algorithm != -1) { + AMF_ASSIGN_PROPERTY_INT64(res, ctx->scaler, AMF_HQ_SCALER_ALGORITHM, ctx->algorithm); + } + if (ctx->sharpness != -1) { + AMF_ASSIGN_PROPERTY_DOUBLE(res, ctx->scaler, AMF_HQ_SCALER_SHARPNESS, ctx->sharpness); + } + AMF_ASSIGN_PROPERTY_BOOL(res, ctx->scaler, AMF_HQ_SCALER_FILL, ctx->fill); + AMF_ASSIGN_PROPERTY_BOOL(res, ctx->scaler, AMF_HQ_SCALER_KEEP_ASPECT_RATIO, ctx->keep_ratio); + // Setup default options to skip color conversion + ctx->color_profile = AMF_VIDEO_CONVERTER_COLOR_PROFILE_UNKNOWN; + ctx->color_range = AMF_COLOR_RANGE_UNDEFINED; + ctx->primaries = AMF_COLOR_PRIMARIES_UNDEFINED; + ctx->trc = AMF_COLOR_TRANSFER_CHARACTERISTIC_UNDEFINED; + + res = ctx->scaler->pVtbl->Init(ctx->scaler, av_amf_av_to_amf_format(in_format), inlink->w, inlink->h); + AMF_RETURN_IF_FALSE(avctx, res == AMF_OK, AVERROR_UNKNOWN, "AMFHQScaler-Init() failed with error %d\n", res); + + return 0; +} + +#define OFFSET(x) offsetof(AMFScaleContext, x) +#define FLAGS AV_OPT_FLAG_VIDEO_PARAM|AV_OPT_FLAG_FILTERING_PARAM +static const AVOption scale_amf_hq_options[] = { + { "w", "Output video width", OFFSET(w_expr), AV_OPT_TYPE_STRING, { .str = "iw" }, .flags = FLAGS }, + { "h", "Output video height", OFFSET(h_expr), AV_OPT_TYPE_STRING, { .str = "ih" }, .flags = FLAGS }, + + { "format", "Output pixel format", OFFSET(format_str), AV_OPT_TYPE_STRING, { .str = "same" }, .flags = FLAGS }, + { "sharpness", "Sharpness", OFFSET(sharpness), AV_OPT_TYPE_FLOAT, { .dbl = -1 }, -1, 2., FLAGS, "sharpness" }, + { "keep-ratio", "Keep aspect ratio", OFFSET(keep_ratio), AV_OPT_TYPE_BOOL, { .i64 = 0 }, 0, 1, FLAGS, "keep_ration" }, + { "fill", "Fill", OFFSET(fill), AV_OPT_TYPE_BOOL, { .i64 = 0 }, 0, 1, FLAGS, "fill" }, + + { "algorithm", "Scaling algorithm", OFFSET(algorithm), AV_OPT_TYPE_INT, { .i64 = -1 }, -1, AMF_HQ_SCALER_ALGORITHM_VIDEOSR1_1, FLAGS, "algorithm" }, + { "bilinear", "Bilinear", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_HQ_SCALER_ALGORITHM_BILINEAR }, 0, 0, FLAGS, "algorithm" }, + { "bicubic", "Bicubic", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_HQ_SCALER_ALGORITHM_BICUBIC }, 0, 0, FLAGS, "algorithm" }, + { "sr1-0", "Video SR1.0", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_HQ_SCALER_ALGORITHM_VIDEOSR1_0 }, 0, 0, FLAGS, "algorithm" }, + { "point", "Point", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_HQ_SCALER_ALGORITHM_POINT }, 0, 0, FLAGS, "algorithm" }, + { "sr1-1", "Video SR1.1", 0, AV_OPT_TYPE_CONST, { .i64 = AMF_HQ_SCALER_ALGORITHM_VIDEOSR1_1 }, 0, 0, FLAGS, "algorithm" }, + + { NULL }, +}; + + +AVFILTER_DEFINE_CLASS(scale_amf_hq); + +static const AVFilterPad amf_scale_inputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + .filter_frame = amf_scale_filter_frame, + } +}; + +static const AVFilterPad amf_scale_outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + .config_props = amf_scale_config_output, + } +}; + +AVFilter ff_vf_scale_amf_hq = { + .name = "scale_amf_hq", + .description = NULL_IF_CONFIG_SMALL("AMF HQ video upscaling"), + + .init = amf_scale_init, + .uninit = amf_scale_uninit, + FILTER_QUERY_FUNC(&amf_scale_query_formats), + + .priv_size = sizeof(AMFScaleContext), + .priv_class = &scale_amf_hq_class, + + FILTER_INPUTS(amf_scale_inputs), + FILTER_OUTPUTS(amf_scale_outputs), + + FILTER_SINGLE_PIXFMT(AV_PIX_FMT_AMF), + + .flags_internal = FF_FILTER_FLAG_HWFRAME_AWARE, + .flags = AVFILTER_FLAG_HWDEVICE, +}; \ No newline at end of file -- 2.38.1.windows.1 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".