From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTP id 3373B4014F for ; Thu, 3 Nov 2022 12:52:01 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id E613A68BF75; Thu, 3 Nov 2022 14:51:59 +0200 (EET) Received: from mail-pg1-f176.google.com (mail-pg1-f176.google.com [209.85.215.176]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 3D73E68BF45 for ; Thu, 3 Nov 2022 14:51:53 +0200 (EET) Received: by mail-pg1-f176.google.com with SMTP id r18so1553975pgr.12 for ; Thu, 03 Nov 2022 05:51:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=spin-digital-com.20210112.gappssmtp.com; s=20210112; h=to:subject:message-id:date:from:in-reply-to:references:mime-version :from:to:cc:subject:date:message-id:reply-to; bh=+RIhgt0l8O3ss7+vUyBOqmoi6B+zuDMTQj3NR/a/BDo=; b=c6zj9SQ5ad3uWJJm6BTfCq1c3mUziePxKh6TP1zRdbrq6tnWzJeEg5RYuXUGgfCviB KOSMQF4/xejp140r8885Eis1OVlfdRnv6aafYLZHb4H0liw2OETR/dRnRKjVTZ5nMq1c HjcZV/LbZ7VaSm2qqUdkduSeufsNFCzI7GYoklFlstuT6+lVn5v3dTxoUTjS8QQVnJ8V PHnxxRt+eV3QCaRFQbmrGFVn9VLElu0+gPonVVjZtD7jrMAL7nzOSmDV7hU2PkT5bY/7 4Zfs9I4ovEK6vqzpmIi5Z2R3JdvEC6r9YiIu6n8XnUZ97dTvqhtrMmIDLN5p5djfF1BC pr/w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=to:subject:message-id:date:from:in-reply-to:references:mime-version :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=+RIhgt0l8O3ss7+vUyBOqmoi6B+zuDMTQj3NR/a/BDo=; b=DcsY9fjUTvtj83XBwgLb6x7oGkRs5YsNLxMokZMPrmm4mhCxQGdRWfsvcl6HyEcn6T SawvZABWV4NKHo0rm3z/6WqoA8mGrlE+wJ227YJU7Ye0nB8JfcISX1yqJB4593UdPmWf JCR3LD2iWA5KxALgjTyQM55xKmyJoDZn3ee0fdOjaiM0zMYng9kKwl8h9aMnsjNXVtnu G16WgA0LBBY9TUXzj/grfjSKjkfNu0dkkMQ7QUTGUbs+RbZs2et0zBKiA22czRdRUa3s 92soDeX1kRoXlPgKKuBkNuVPHpGw2OYDLaNj7Sqk2rdEsIkWYKciCUya+aG5dHrdkeHd PvZg== X-Gm-Message-State: ACrzQf2NGcgqVZq0BjnQAmwnb45BhfD10vSa0/uxv+ywy+AHd4do4YPW OcqHDeIebXQBYLRVit5Q8zyqMieHRo2Me1xMXBtGFh/7EZ79hw== X-Google-Smtp-Source: AMsMyM4xmB7ETUSEZK01yOAjo+OgOg53kFYWH6JRQmlbBft9GkSCcpVQSn+LqRoqNDXvgDcP+/xCWd7/BZ5WN7BZvWU= X-Received: by 2002:a65:5386:0:b0:46e:dbd3:413 with SMTP id x6-20020a655386000000b0046edbd30413mr25309265pgq.240.1667479911220; Thu, 03 Nov 2022 05:51:51 -0700 (PDT) MIME-Version: 1.0 References: <20221024140645.4945-1-thomas.ff@spin-digital.com> <20221024140645.4945-9-thomas.ff@spin-digital.com> <6329329f-75ac-acfe-c517-9c9be10920dc@gmail.com> In-Reply-To: <6329329f-75ac-acfe-c517-9c9be10920dc@gmail.com> From: Thomas Siedel Date: Thu, 3 Nov 2022 13:51:38 +0100 Message-ID: To: FFmpeg development discussions and patches X-Content-Filtered-By: Mailman/MimeDel 2.1.29 Subject: Re: [FFmpeg-devel] [PATCH v2 08/10] avcodec: add external encoder libvvenc for H266/VVC X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: On Mon, 24 Oct 2022 at 16:38, quietvoid wrote: > On 24/10/2022 10.06, Thomas Siedel wrote: > > > Add external encoder VVenC for H266/VVC encoding. > > Register new encoder libvvencc > > Add libvvenc to wrap the vvenc interface into ffmpeg > > libvvenc implements encoder option: > > preset,qp,period,subjopt,vvenc-params,levelidc,tier > > Enable encoder by adding --enable-libvvenc in configure step > > > > Signed-off-by: Thomas Siedel > > --- > > configure | 5 + > > libavcodec/Makefile | 1 + > > libavcodec/allcodecs.c | 1 + > > libavcodec/libvvenc.c | 442 +++++++++++++++++++++++++++++++++++++++++ > > 4 files changed, 449 insertions(+) > > create mode 100644 libavcodec/libvvenc.c > > > > diff --git a/configure b/configure > > index 978f15f772..f8f7965371 100755 > > --- a/configure > > +++ b/configure > > @@ -289,6 +289,7 @@ External library support: > > native implementation exists [no] > > --enable-libvpx enable VP8 and VP9 de/encoding via libvpx [no] > > --enable-libvvdec enable VVC decoding via vvdec [no] > > + --enable-libvvenc enable VVC encoding via vvenc [no] > > --enable-libwebp enable WebP encoding via libwebp [no] > > --enable-libx264 enable H.264 encoding via x264 [no] > > --enable-libx265 enable HEVC encoding via x265 [no] > > @@ -1877,6 +1878,7 @@ EXTERNAL_LIBRARY_LIST=" > > libvorbis > > libvpx > > libvvdec > > + libvvenc > > libwebp > > libxml2 > > libzimg > > @@ -3409,6 +3411,8 @@ libvpx_vp9_decoder_deps="libvpx" > > libvpx_vp9_encoder_deps="libvpx" > > libvvdec_decoder_deps="libvvdec" > > libvvdec_decoder_select="vvc_mp4toannexb_bsf" > > +libvvenc_encoder_deps="libvvenc" > > +libvvenc_encoder_select="atsc_a53" > > libwebp_encoder_deps="libwebp" > > libwebp_anim_encoder_deps="libwebp" > > libx262_encoder_deps="libx262" > > @@ -6740,6 +6744,7 @@ enabled libvpx && { > > fi > > } > > enabled libvvdec && require_pkg_config libvvdec "libvvdec >= 1.6.0" > > "vvdec/vvdec.h" vvdec_get_version > > +enabled libvvenc && require_pkg_config libvvenc "libvvenc >= 1.6.1" > > "vvenc/vvenc.h" vvenc_get_version > > enabled libwebp && { > > enabled libwebp_encoder && require_pkg_config libwebp "libwebp >= > > 0.2.0" webp/encode.h WebPGetEncoderVersion > > diff --git a/libavcodec/Makefile b/libavcodec/Makefile > > index f4cdbc9be1..badd266e66 100644 > > --- a/libavcodec/Makefile > > +++ b/libavcodec/Makefile > > @@ -1105,6 +1105,7 @@ OBJS-$(CONFIG_LIBVPX_VP8_ENCODER) += libvpxenc.o > > OBJS-$(CONFIG_LIBVPX_VP9_DECODER) += libvpxdec.o libvpx.o > > OBJS-$(CONFIG_LIBVPX_VP9_ENCODER) += libvpxenc.o libvpx.o > > OBJS-$(CONFIG_LIBVVDEC_DECODER) += libvvdec.o vvc_parse_extradata.o > > vvc_paramset.o > > +OBJS-$(CONFIG_LIBVVENC_ENCODER) += libvvenc.o > > OBJS-$(CONFIG_LIBWEBP_ENCODER) += libwebpenc_common.o libwebpenc.o > > OBJS-$(CONFIG_LIBWEBP_ANIM_ENCODER) += libwebpenc_common.o > > libwebpenc_animencoder.o > > OBJS-$(CONFIG_LIBX262_ENCODER) += libx264.o > > diff --git a/libavcodec/allcodecs.c b/libavcodec/allcodecs.c > > index 9813b291f4..73c36f3134 100644 > > --- a/libavcodec/allcodecs.c > > +++ b/libavcodec/allcodecs.c > > @@ -794,6 +794,7 @@ extern const FFCodec ff_libvpx_vp8_decoder; > > extern FFCodec ff_libvpx_vp9_encoder; > > extern FFCodec ff_libvpx_vp9_decoder; > > extern const FFCodec ff_libvvdec_decoder; > > +extern const FFCodec ff_libvvenc_encoder; > > /* preferred over libwebp */ > > extern const FFCodec ff_libwebp_anim_encoder; > > extern const FFCodec ff_libwebp_encoder; > > diff --git a/libavcodec/libvvenc.c b/libavcodec/libvvenc.c > > new file mode 100644 > > index 0000000000..508fe6ceff > > --- /dev/null > > +++ b/libavcodec/libvvenc.c > > @@ -0,0 +1,442 @@ > > +/* > > + * H.266 encoding using the VVenC library > > + * > > + * Copyright (C) 2022, Thomas Siedel > > + * > > + * This file is part of FFmpeg. > > + * > > + * FFmpeg is free software; you can redistribute it and/or > > + * modify it under the terms of the GNU Lesser General Public > > + * License as published by the Free Software Foundation; either > > + * version 2.1 of the License, or (at your option) any later version. > > + * > > + * FFmpeg is distributed in the hope that it will be useful, > > + * but WITHOUT ANY WARRANTY; without even the implied warranty of > > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > > + * Lesser General Public License for more details. > > + * > > + * You should have received a copy of the GNU Lesser General Public > > + * License along with FFmpeg; if not, write to the Free Software > > + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA > > 02110-1301 USA > > + */ > > + > > +#include "config_components.h" > > + > > +#include > > +#include > > + > > +#include "avcodec.h" > > +#include "codec_internal.h" > > +#include "encode.h" > > +#include "internal.h" > > +#include "packet_internal.h" > > +#include "profiles.h" > > + > > +#include "libavutil/avutil.h" > > +#include "libavutil/pixdesc.h" > > +#include "libavutil/opt.h" > > +#include "libavutil/common.h" > > +#include "libavutil/imgutils.h" > > +#include "libavutil/frame.h" > > +#include "libavutil/log.h" > > + > > +typedef struct VVenCOptions { > > + int preset; // preset 0: faster 4: slower > > + int qp; // quantization parameter 0-63 > > + int subjectiveOptimization; // perceptually motivated QP adaptation, > > XPSNR based > > + int intraRefreshSec; // intra period/refresh in seconds > > + int levelIdc; // vvc level_idc > > + int tier; // vvc tier > > + AVDictionary *vvenc_opts; > > +} VVenCOptions; > > + > > +typedef struct VVenCContext { > > + AVClass *av_class; > > + VVenCOptions options; // encoder options > > + vvencEncoder *vvencEnc; > > + vvencAccessUnit *pAU; > > + bool encodeDone; > > +}VVenCContext; > > + > > + > > +static av_cold void ff_vvenc_log_callback(void *avctx, int level, > > const char *fmt, va_list args ) > > +{ > > + vfprintf( level == 1 ? stderr : stdout, fmt, args ); > > +} > > + > > +static void ff_vvenc_internalLog( void* ctx, int level, const char* > > fmt, ... ) > > +{ > > + va_list args; > > + va_start( args, fmt ); > > + ff_vvenc_log_callback( ctx, level, fmt, args ); > > + va_end( args ); > > +} > > + > > +static av_cold int ff_vvenc_encode_init(AVCodecContext *avctx) > > +{ > > + int ret; > > + int framerate, qp, parse_ret; > > + VVenCContext *s; > > + vvenc_config params; > > + vvencPresetMode preset; > > + AVDictionaryEntry *en; > > + char statsfile[1024]="vvenc-rcstats.json"; > > + > > + s = (VVenCContext*)avctx->priv_data; > > + qp = (vvencPresetMode)s->options.qp; > > + preset = (vvencPresetMode)s->options.preset; > > + > > + if( avctx->flags & AV_CODEC_FLAG_INTERLACED_DCT ) { > > + av_log(avctx, AV_LOG_ERROR, "ff_vvenc_encode_init::init() interlaced > > encoding not supported yet\n"); > > + return AVERROR_INVALIDDATA; > > + } > > + > > + vvenc_config_default( ¶ms ); > > + > > + // set desired encoding options > > + framerate = avctx->time_base.den / avctx->time_base.num; > > + vvenc_init_default ( ¶ms, avctx->width, avctx->height, > > framerate, avctx->bit_rate, qp, preset ); > > + params.m_FrameRate = avctx->time_base.den; > > + params.m_FrameScale = avctx->time_base.num; > > + > > + params.m_verbosity = VVENC_VERBOSE; > > + if ( av_log_get_level() >= AV_LOG_DEBUG ) > > + params.m_verbosity = VVENC_DETAILS; > > + else if( av_log_get_level() >= AV_LOG_VERBOSE ) > > + params.m_verbosity = VVENC_NOTICE; // output per picture info > > + else if( av_log_get_level() >= AV_LOG_INFO ) > > + params.m_verbosity = VVENC_WARNING; // ffmpeg default ffmpeg loglevel > > + else > > + params.m_verbosity = VVENC_SILENT; > > + > > + if( avctx->ticks_per_frame == 1 ) { > > + params.m_TicksPerSecond = -1; // auto mode for ticks per frame = 1 > > + } > > + else{ > > + params.m_TicksPerSecond = > > > ceil((avctx->time_base.den/(double)avctx->time_base.num)*(double)avctx->ticks_per_frame); > > + } > > + > > + if( avctx->thread_count > 0 ) > > + params.m_numThreads = avctx->thread_count; > > + > > + // GOP settings (IDR/CRA) > > + if( avctx->flags & AV_CODEC_FLAG_CLOSED_GOP ) > > + params.m_DecodingRefreshType = VVENC_DRT_IDR; > > + > > + if( avctx->gop_size == 1 ) { > > + params.m_GOPSize = 1; > > + params.m_IntraPeriod = 1; > > + } > > + else { > > + params.m_IntraPeriodSec = s->options.intraRefreshSec; > > + } > > + > > + params.m_usePerceptQPA = s->options.subjectiveOptimization; > > + params.m_level = (vvencLevel)s->options.levelIdc; > > + params.m_levelTier = (vvencTier)s->options.tier; > > + > > + params.m_AccessUnitDelimiter = true; > > + > > + params.m_internChromaFormat = VVENC_CHROMA_420; > > + switch( avctx->pix_fmt ) { > > + case AV_PIX_FMT_YUV420P : > > + params.m_inputBitDepth[0] = 8; > > + break; > > + case AV_PIX_FMT_YUV420P10LE: > > + params.m_inputBitDepth[0] = 10; > > + break; > > + default: { > > + av_log(avctx, AV_LOG_ERROR, "unsupported pixel format %s, choose > > yuv420p or yuv420p10le\n", av_get_pix_fmt_name(avctx->pix_fmt)); > > + return AVERROR(EINVAL); > > + break; > > + } > > + }; > > + > > + if( avctx->color_primaries != AVCOL_PRI_UNSPECIFIED ) > > + params.m_colourPrimaries = (int)avctx->color_primaries; > > + if( avctx->colorspace != AVCOL_SPC_UNSPECIFIED ) > > + params.m_matrixCoefficients = (int)avctx->colorspace; > > + if( avctx->color_trc != AVCOL_TRC_UNSPECIFIED ) { > > + params.m_transferCharacteristics = (int)avctx->color_trc; > > + > > + if ( params.m_transferCharacteristics == 16 ) > > + params.m_HdrMode = (params.m_colourPrimaries == 9) ? > VVENC_HDR_PQ_BT2020 > > + : VVENC_HDR_PQ; > > + else if ( params.m_transferCharacteristics == 14 && > > params.m_colourPrimaries == 9 ) > > + params.m_HdrMode = VVENC_HDR_HLG_BT2020; > > > Should probably still be using the AVColor enum in the conditions > instead of the values. > Thanks for pointing this out, I now changed it to use the AVColor enums in the new patch set version 3. > > > + else if ( params.m_transferCharacteristics == 1 && > > params.m_colourPrimaries == 1 && > > + params.m_matrixCoefficients == 1 ) > > + params.m_HdrMode = VVENC_HDR_HLG; > > > Is this not supposed to be VVENC_HDR_OFF? > HLG should be matching TRC 14, not 1. > HDR HLG mode was set for color formats/colorspaces that are used for SDR signaling. That code was taken from the vvenc implementation, where bt709 formats are used for HLG 8bit. Now I changed it. In the new version, HLG is only set if the transfer function is HLG (AVCOL_TRC_BT2020_10) or AVCOL_TRC_ARIB_STD_B67. HLG 2020 is used if the input color primaries or color space is also bt2020. SDR signalization has been added. So if any color primaries/colorspace/transfer characteristics are set and it is not an HDR mode, libbvvenc enables VUI signalization in vvenc. This is needed as vvenc currently only enables VUI signalization if any HDR mode is set. This is an issue in vvenc. _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".