* [FFmpeg-devel] [PATCH 2/3] Provided support for MPEG-5 EVC (Essential Video Coding) codec [not found] <CGME20220622064904eucas1p19bbbc922c4f1b734730c48169bada35a@eucas1p1.samsung.com> @ 2022-06-22 6:49 ` Dawid Kozinski 2022-07-01 9:46 ` Anton Khirnov 0 siblings, 1 reply; 6+ messages in thread From: Dawid Kozinski @ 2022-06-22 6:49 UTC (permalink / raw) To: ffmpeg-devel - Added xeve encoder wrapper - Added xevd dencoder wrapper - Added documentation for xeve and xevd wrappers - Added parser for EVC format - Changes in project configuration file and libavcodec Makefile Signed-off-by: Dawid Kozinski <d.kozinski@samsung.com> --- configure | 8 + doc/decoders.texi | 24 ++ doc/encoders.texi | 112 ++++++ doc/general_contents.texi | 19 + libavcodec/Makefile | 3 + libavcodec/allcodecs.c | 2 + libavcodec/avcodec.h | 3 + libavcodec/evc_parser.c | 527 ++++++++++++++++++++++++++ libavcodec/libxevd.c | 440 ++++++++++++++++++++++ libavcodec/libxeve.c | 753 ++++++++++++++++++++++++++++++++++++++ libavcodec/parsers.c | 1 + 11 files changed, 1892 insertions(+) create mode 100644 libavcodec/evc_parser.c create mode 100644 libavcodec/libxevd.c create mode 100644 libavcodec/libxeve.c diff --git a/configure b/configure index 3a97610209..8071e22859 100755 --- a/configure +++ b/configure @@ -291,6 +291,8 @@ External library support: --enable-libwebp enable WebP encoding via libwebp [no] --enable-libx264 enable H.264 encoding via x264 [no] --enable-libx265 enable HEVC encoding via x265 [no] + --enable-libxeve enable MPEG-5 Essential Video Coding (EVC) encoding via libxeve [no] + --enable-libxevd enable MPEG-5 Essential Video Coding (EVC) decoding via libxevd [no] --enable-libxavs enable AVS encoding via xavs [no] --enable-libxavs2 enable AVS2 encoding via xavs2 [no] --enable-libxcb enable X11 grabbing using XCB [autodetect] @@ -1874,6 +1876,8 @@ EXTERNAL_LIBRARY_LIST=" libvorbis libvpx libwebp + libxevd + libxeve libxml2 libzimg libzmq @@ -3384,6 +3388,8 @@ libx264rgb_encoder_select="libx264_encoder" libx265_encoder_deps="libx265" libxavs_encoder_deps="libxavs" libxavs2_encoder_deps="libxavs2" +libxevd_decoder_deps="libxevd" +libxeve_encoder_deps="libxeve" libxvid_encoder_deps="libxvid" libzvbi_teletext_decoder_deps="libzvbi" vapoursynth_demuxer_deps="vapoursynth" @@ -6668,6 +6674,8 @@ enabled libx265 && require_pkg_config libx265 x265 x265.h x265_api_get require_cpp_condition libx265 x265.h "X265_BUILD >= 70" enabled libxavs && require libxavs "stdint.h xavs.h" xavs_encoder_encode "-lxavs $pthreads_extralibs $libm_extralibs" enabled libxavs2 && require_pkg_config libxavs2 "xavs2 >= 1.3.0" "stdint.h xavs2.h" xavs2_api_get +enabled libxevd && require_pkg_config libxevd "xevd >= 0.3.3" "xevd.h" xevd_decode +enabled libxeve && require_pkg_config libxeve "xeve >= 0.3.5" "xeve.h" xeve_encode enabled libxvid && require libxvid xvid.h xvid_global -lxvidcore enabled libzimg && require_pkg_config libzimg "zimg >= 2.7.0" zimg.h zimg_get_api_version enabled libzmq && require_pkg_config libzmq "libzmq >= 4.2.1" zmq.h zmq_ctx_new diff --git a/doc/decoders.texi b/doc/decoders.texi index e2fcbf5dc9..0bd9096114 100644 --- a/doc/decoders.texi +++ b/doc/decoders.texi @@ -126,6 +126,30 @@ Set amount of frame threads to use during decoding. The default value is 0 (auto @end table +@section libxevd + +eXtra-fast Essential Video Decoder (XEVD) MPEG-5 EVC decoder wrapper. + +This decoder requires the presence of the libxevd headers and library +during configuration. You need to explicitly configure the build with +@option{--enable-libxevd}. + +The xevd project website is at @url{https://github.com/mpeg5/xevd}. + +@subsection Options + +The following options are supported by the libxevd wrapper. +The xevd-equivalent options or values are listed in parentheses for easy migration. + +To get a more accurate and extensive documentation of the libxevd options, +invoke the command @code{xevd_app --help} or consult the libxevd documentation. + +@table @option +@item threads (@emph{threads}) +Force to use a specific number of threads + +@end table + @section QSV Decoders The family of Intel QuickSync Video decoders (VC1, MPEG-2, H.264, HEVC, diff --git a/doc/encoders.texi b/doc/encoders.texi index 1850c99fe9..245df680d2 100644 --- a/doc/encoders.texi +++ b/doc/encoders.texi @@ -2892,6 +2892,118 @@ ffmpeg -i input -c:v libxavs2 -xavs2-params RdoqLevel=0 output.avs2 @end example @end table +@section libxeve + +eXtra-fast Essential Video Encoder (XEVE) MPEG-5 EVC encoder wrapper. + +This encoder requires the presence of the libxeve headers and library +during configuration. You need to explicitly configure the build with +@option{--enable-libxeve}. + +@float NOTE +Many libxeve encoder options are mapped to FFmpeg global codec options, +while unique encoder options are provided through private options. +Additionally the xeve-params private options allows one to pass a list +of key=value tuples as accepted by the libxeve @code{parse_xeve_params} function. +@end float + +The xeve project website is at @url{https://github.com/mpeg5/xeve}. + +@subsection Options + +The following options are supported by the libxeve wrapper. +The xeve-equivalent options or values are listed in parentheses for easy migration. + +@float NOTE +To reduce the duplication of documentation, only the private options +and some others requiring special attention are documented here. For +the documentation of the undocumented generic options, see +@ref{codec-options,,the Codec Options chapter}. +@end float + +@float NOTE +To get a more accurate and extensive documentation of the libxeve options, +invoke the command @code{xeve_app --help} or consult the libxeve documentation. +@end float + +@table @option +@item b (@emph{bitrate}) +Set target video bitrate in bits/s. +Note that FFmpeg’s b option is expressed in bits/s, while xeve’s bitrate is in kilobits/s. + +@item bf (@emph{bframes}) +Set the maximum number of B frames (1,3,7,15). + +@item g (@emph{keyint}) +Set the GOP size (I-picture period). + +@item preset (@emph{preset}) +Set the xeve preset. +Set the encoder preset value to determine encoding speed [fast, medium, slow, placebo] + +@item tune (@emph{tune}) +Set the encoder tune parameter [psnr, zerolatency] + +@item profile (@emph{profile}) +Set the encoder profile [0: baselie; 1: main] + +@item crf (@emph{crf}) +Set the quality for constant quality mode. +Constant rate factor <-1..51> [default: 32] + +@item qp (@emph{qp}) +Set constant quantization rate control method parameter. +Quantization parameter qp <0..51> [default: 32] + +@item threads (@emph{threads}) +Force to use a specific number of threads + +@item xeve-params +Set xeve options using a list of @var{key}=@var{value} couples separated +by ":". + +For example to specify libxeve encoding options with @option{-xeve-params}: + +@example +ffmpeg -i input -c:v libxeve -xeve-params profile=main:tune=zerolatency:preset=fast:q=17:bframes=15:bitrate=100k output.mp4 +@end example +@end table + +@subsubsection Supported key values for xeve-params + +Set xeve options using a list of @var{key}=@var{value} couples separated + +@table @option + +@item hash +Embed the picture signature (HASH) for conformance checking in decoding, (0: Off, 1: On) + +@item vbv-bufsize +Set the VBV buffer size: Kbits(none,K,k), Mbits(M,m) + +@item rc-type +Set the rate control type, (0: OFF, 1: ABR, 2: CRF) + +@item bframes +Set the maximum number of B frames (1,3,7,15) + +@item profile +Set the PROFILE (main, baseline) + +@item preset +Set the encoder PRESET (fast, medium, slow, placebo) + +@item tune +Set the encoder TUNE (psnr, zerolatency) + +@item bitrate +Set the Bitrate in terms of kilo-bits per second: Kbps(none,K,k), Mbps(M,m) + +@item qp +Set the QP value (0~51) + +@end table + @section libxvid Xvid MPEG-4 Part 2 encoder wrapper. diff --git a/doc/general_contents.texi b/doc/general_contents.texi index 987a2f82fb..6c4b7506ba 100644 --- a/doc/general_contents.texi +++ b/doc/general_contents.texi @@ -343,6 +343,22 @@ libxavs2 is under the GNU Public License Version 2 or later details), you must upgrade FFmpeg's license to GPL in order to use it. @end float +@section eXtra-fast Essential Video Encoder (XEVE) + +FFmpeg can make use of the XEVE library for EVC video encoding. + +Go to @url{https://github.com/mpeg5/xeve} and follow the instructions for +installing the XEVE library. Then pass @code{--enable-libxeve} to configure to +enable it. + +@section eXtra-fast Essential Video Decoder (XEVD) + +FFmpeg can make use of the XEVD library for EVC video decoding. + +Go to @url{https://github.com/mpeg5/xevd} and follow the instructions for +installing the XEVD library. Then pass @code{--enable-libxevd} to configure to +enable it. + @section ZVBI ZVBI is a VBI decoding library which can be used by FFmpeg to decode DVB @@ -591,6 +607,7 @@ library: @item raw DTS @tab X @tab X @item raw DTS-HD @tab @tab X @item raw E-AC-3 @tab X @tab X +@item raw EVC @tab X @tab X @item raw FLAC @tab X @tab X @item raw GSM @tab @tab X @item raw H.261 @tab X @tab X @@ -924,6 +941,8 @@ following image formats are supported: @item Electronic Arts TQI video @tab @tab X @item Escape 124 @tab @tab X @item Escape 130 @tab @tab X +@item EVC / MPEG-5 Part 1 @tab X @tab X + @tab encoding and decoding supported through external libraries libxeve and libxevd @item FFmpeg video codec #1 @tab X @tab X @tab lossless codec (fourcc: FFV1) @item Flash Screen Video v1 @tab X @tab X diff --git a/libavcodec/Makefile b/libavcodec/Makefile index 3b8f7b5e01..cb2d438a14 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -1103,6 +1103,8 @@ OBJS-$(CONFIG_LIBX264_ENCODER) += libx264.o OBJS-$(CONFIG_LIBX265_ENCODER) += libx265.o OBJS-$(CONFIG_LIBXAVS_ENCODER) += libxavs.o OBJS-$(CONFIG_LIBXAVS2_ENCODER) += libxavs2.o +OBJS-$(CONFIG_LIBXEVD_DECODER) += libxevd.o +OBJS-$(CONFIG_LIBXEVE_ENCODER) += libxeve.o OBJS-$(CONFIG_LIBXVID_ENCODER) += libxvid.o OBJS-$(CONFIG_LIBZVBI_TELETEXT_DECODER) += libzvbi-teletextdec.o ass.o @@ -1130,6 +1132,7 @@ OBJS-$(CONFIG_DVAUDIO_PARSER) += dvaudio_parser.o OBJS-$(CONFIG_DVBSUB_PARSER) += dvbsub_parser.o OBJS-$(CONFIG_DVD_NAV_PARSER) += dvd_nav_parser.o OBJS-$(CONFIG_DVDSUB_PARSER) += dvdsub_parser.o +OBJS-$(CONFIG_EVC_PARSER) += evc_parser.o OBJS-$(CONFIG_FLAC_PARSER) += flac_parser.o flacdata.o flac.o OBJS-$(CONFIG_G723_1_PARSER) += g723_1_parser.o OBJS-$(CONFIG_G729_PARSER) += g729_parser.o diff --git a/libavcodec/allcodecs.c b/libavcodec/allcodecs.c index f0b01051b0..2e0acc6f9a 100644 --- a/libavcodec/allcodecs.c +++ b/libavcodec/allcodecs.c @@ -794,6 +794,8 @@ extern LIBX264_CONST FFCodec ff_libx264_encoder; #endif extern const FFCodec ff_libx264rgb_encoder; extern FFCodec ff_libx265_encoder; +extern const FFCodec ff_libxeve_encoder; +extern const FFCodec ff_libxevd_decoder; extern const FFCodec ff_libxavs_encoder; extern const FFCodec ff_libxavs2_encoder; extern const FFCodec ff_libxvid_encoder; diff --git a/libavcodec/avcodec.h b/libavcodec/avcodec.h index 4dae23d06e..b83870c174 100644 --- a/libavcodec/avcodec.h +++ b/libavcodec/avcodec.h @@ -1666,6 +1666,9 @@ typedef struct AVCodecContext { #define FF_PROFILE_KLVA_SYNC 0 #define FF_PROFILE_KLVA_ASYNC 1 +#define FF_PROFILE_EVC_BASELINE 0 +#define FF_PROFILE_EVC_MAIN 1 + /** * level * - encoding: Set by user. diff --git a/libavcodec/evc_parser.c b/libavcodec/evc_parser.c new file mode 100644 index 0000000000..03faf12d62 --- /dev/null +++ b/libavcodec/evc_parser.c @@ -0,0 +1,527 @@ +/* + * EVC AVC format parser + * + * Copyright (C) 2021 Dawid Kozinski <d.kozinski@samsung.com> + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include <stdint.h> + +#include "libavutil/common.h" + +#include "parser.h" +#include "golomb.h" + +// The length field that indicates the length in bytes of the following NAL unit is configured to be of 4 bytes +#define EVC_NAL_UNIT_LENGTH_BYTE (4) /* byte */ + +#define EVC_NAL_HEADER_SIZE (2) /* byte */ +#define MAX_SPS_CNT (16) /* defined value in EVC standard */ + +// NALU types +// @see ISO_IEC_23094-1_2020 7.4.2.2 NAL unit header semantics +// +#define EVC_NUT_NONIDR (0) /* Coded slice of a non-IDR picture */ +#define EVC_NUT_IDR (1) /* Coded slice of an IDR picture */ +#define EVC_NUT_SPS (24) /* Sequence parameter set */ +#define EVC_NUT_PPS (25) /* Picture paremeter set */ +#define EVC_NUT_APS (26) /* Adaptation parameter set */ +#define EVC_NUT_FD (27) /* Filler data */ +#define EVC_NUT_SEI (28) /* Supplemental enhancement information */ + +// The sturcture reflects SPS RBSP(raw byte sequence payload) layout +// @see ISO_IEC_23094-1 section 7.3.2.1 +// +// The following descriptors specify the parsing process of each element +// u(n) - unsigned integer using n bits +// ue(v) - unsigned integer 0-th order Exp_Golomb-coded syntax element with the left bit first +typedef struct EVCParserSPS { + int sps_seq_parameter_set_id; // ue(v) + int profile_idc; // u(8) + int level_idc; // u(8) + int toolset_idc_h; // u(32) + int toolset_idc_l; // u(32) + int chroma_format_idc; // ue(v) + int pic_width_in_luma_samples; // ue(v) + int pic_height_in_luma_samples; // ue(v) + int bit_depth_luma_minus8; // ue(v) + int bit_depth_chroma_minus8; // ue(v) + + int sps_btt_flag; // u(1) + int log2_ctu_size_minus5; // ue(v) + int log2_min_cb_size_minus2; // ue(v) + int log2_diff_ctu_max_14_cb_size; // ue(v) + int log2_diff_ctu_max_tt_cb_size; // ue(v) + int log2_diff_min_cb_min_tt_cb_size_minus2; // ue(v) + + int sps_suco_flag; // u(1) + int log2_diff_ctu_size_max_suco_cb_size; // ue(v) + int log2_diff_max_suco_min_suco_cb_size; // ue(v) + + int sps_admvp_flag; // u(1) + int sps_affine_flag; // u(1) + int sps_amvr_flag; // u(1) + int sps_dmvr_flag; // u(1) + int sps_mmvd_flag; // u(1) + int sps_hmvp_flag; // u(1) + + int sps_eipd_flag; // u(1) + int sps_ibc_flag; // u(1) + int log2_max_ibc_cand_size_minus2; // ue(v) + + int sps_cm_init_flag; // u(1) + int sps_adcc_flag; // u(1) + + int sps_iqt_flag; // u(1) + int sps_ats_flag; // u(1) + + int sps_addb_flag; // u(1) + int sps_alf_flag; // u(1) + int sps_htdf_flag; // u(1) + int sps_rpl_flag; // u(1) + int sps_pocs_flag; // u(1) + int sps_dquant_flag; // u(1) + int sps_dra_flag; // u(1) + + int log2_max_pic_order_cnt_lsb_minus4; // ue(v) + int log2_sub_gop_length; // ue(v) + int log2_ref_pic_gap_length; // ue(v) + + int max_num_tid0_ref_pics; // ue(v) + + int sps_max_dec_pic_buffering_minus1; // ue(v) + int long_term_ref_pic_flag; // u(1) + int rpl1_same_as_rpl0_flag; // u(1) + int num_ref_pic_list_in_sps[2]; // ue(v) + + int picture_cropping_flag; // u(1) + int picture_crop_left_offset; // ue(v) + int picture_crop_right_offset; // ue(v) + int picture_crop_top_offset; // ue(v) + int picture_crop_bottom_offset; // ue(v) + +} EVCParserSPS; + +typedef struct EVCParserContext { + ParseContext pc; + EVCParserSPS sps[MAX_SPS_CNT]; + int is_avc; + int nal_length_size; + int to_read; + int incomplete_nalu_prefix_read; // The flag is set to 1 when an incomplete NAL unit prefix has been read + + int got_sps; + int got_pps; + int got_sei; + int got_slice; +} EVCParserContext; + +static int get_nalu_type(const uint8_t *bits, int bits_size, AVCodecContext *avctx) +{ + int unit_type_plus1 = 0; + + if(bits_size >= EVC_NAL_HEADER_SIZE) { + unsigned char *p = (unsigned char *)bits; + // forbidden_zero_bit + if ((p[0] & 0x80) != 0) { + av_log(avctx, AV_LOG_ERROR, "Cannot get bitstream information. Malformed bitstream.\n"); + return -1; + } + + // nal_unit_type + unit_type_plus1 = (p[0] >> 1) & 0x3F; + } + + return unit_type_plus1 - 1; +} + +static uint32_t read_nal_unit_length(const uint8_t *bits, int bits_size, AVCodecContext *avctx) +{ + uint32_t nalu_len = 0; + + if(bits_size >= EVC_NAL_UNIT_LENGTH_BYTE) { + + int t = 0; + unsigned char *p = (unsigned char *)bits; + + for(int i=0; i<EVC_NAL_UNIT_LENGTH_BYTE; i++) { + t = (t << 8) | p[i]; + } + + nalu_len = t; + if(nalu_len == 0) { + av_log(avctx, AV_LOG_ERROR, "Invalid bitstream size!\n"); + return 0; + } + } + + return nalu_len; +} + +// @see ISO_IEC_23094-1 (7.3.2.1 SPS RBSP syntax) +static EVCParserSPS *parse_sps(const uint8_t *bs, int bs_size, EVCParserContext *ev) +{ + GetBitContext gb; + EVCParserSPS *sps; + int sps_seq_parameter_set_id; + + if(init_get_bits8(&gb, bs, bs_size) < 0) + return NULL; + + sps_seq_parameter_set_id = get_ue_golomb(&gb); + + if(sps_seq_parameter_set_id >= MAX_SPS_CNT) + return NULL; + + sps = &ev->sps[sps_seq_parameter_set_id]; + sps->sps_seq_parameter_set_id = sps_seq_parameter_set_id; + + // the Baseline profile is indicated by profile_idc eqal to 0 + // the Main profile is indicated by profile_idc eqal to 1 + sps->profile_idc = get_bits(&gb, 8); + + sps->level_idc = get_bits(&gb, 8); + + skip_bits_long(&gb, 32); /* skip toolset_idc_h */ + skip_bits_long(&gb, 32); /* skip toolset_idc_l */ + + // 0 - monochrome + // 1 - 4:2:0 + // 2 - 4:2:2 + // 3 - 4:4:4 + sps->chroma_format_idc = get_ue_golomb(&gb); + + sps->pic_width_in_luma_samples = get_ue_golomb(&gb); + sps->pic_height_in_luma_samples = get_ue_golomb(&gb); + + sps->bit_depth_luma_minus8 = get_ue_golomb(&gb); + sps->bit_depth_chroma_minus8 = get_ue_golomb(&gb); + + sps->sps_btt_flag = get_bits(&gb, 1); + if(sps->sps_btt_flag) { + sps->log2_ctu_size_minus5 = get_ue_golomb(&gb); + sps->log2_min_cb_size_minus2 = get_ue_golomb(&gb); + sps->log2_diff_ctu_max_14_cb_size = get_ue_golomb(&gb); + sps->log2_diff_ctu_max_tt_cb_size = get_ue_golomb(&gb); + sps->log2_diff_min_cb_min_tt_cb_size_minus2 = get_ue_golomb(&gb); + } + + sps->sps_suco_flag = get_bits(&gb, 1); + if(sps->sps_suco_flag) { + sps->log2_diff_ctu_size_max_suco_cb_size = get_ue_golomb(&gb); + sps->log2_diff_max_suco_min_suco_cb_size = get_ue_golomb(&gb); + } + + sps->sps_admvp_flag = get_bits(&gb, 1); + if(sps->sps_admvp_flag) { + sps->sps_affine_flag = get_bits(&gb, 1); + sps->sps_amvr_flag = get_bits(&gb, 1); + sps->sps_dmvr_flag = get_bits(&gb, 1); + sps->sps_mmvd_flag = get_bits(&gb, 1); + sps->sps_hmvp_flag = get_bits(&gb, 1); + } + + sps->sps_eipd_flag = get_bits(&gb, 1); + if(sps->sps_eipd_flag) { + sps->sps_ibc_flag = get_bits(&gb, 1); + if(sps->sps_ibc_flag) + sps->log2_max_ibc_cand_size_minus2 = get_ue_golomb(&gb); + } + + sps->sps_cm_init_flag = get_bits(&gb, 1); + if(sps->sps_cm_init_flag) + sps->sps_adcc_flag = get_bits(&gb, 1); + + sps->sps_iqt_flag = get_bits(&gb, 1); + if(sps->sps_iqt_flag) + sps->sps_ats_flag = get_bits(&gb, 1); + + sps->sps_addb_flag = get_bits(&gb, 1); + sps->sps_alf_flag = get_bits(&gb, 1); + sps->sps_htdf_flag = get_bits(&gb, 1); + sps->sps_rpl_flag = get_bits(&gb, 1); + sps->sps_pocs_flag = get_bits(&gb, 1); + sps->sps_dquant_flag = get_bits(&gb, 1); + sps->sps_dra_flag = get_bits(&gb, 1); + + if(sps->sps_pocs_flag) + sps->log2_max_pic_order_cnt_lsb_minus4 = get_ue_golomb(&gb); + + if(!sps->sps_pocs_flag || !sps->sps_rpl_flag) { + sps->log2_sub_gop_length = get_ue_golomb(&gb); + if(sps->log2_sub_gop_length == 0) + sps->log2_ref_pic_gap_length = get_ue_golomb(&gb); + } + + return sps; +} + +static int parse_nal_units(AVCodecParserContext *s, const uint8_t *bs, + int bs_size, AVCodecContext *avctx) +{ + EVCParserContext *ev = s->priv_data; + int nalu_type, nalu_size; + unsigned char *bits = (unsigned char *)bs; + int bits_size = bs_size; + + avctx->codec_id = AV_CODEC_ID_EVC; + + nalu_size = read_nal_unit_length(bits, bits_size, avctx); + if(nalu_size == 0) { + av_log(avctx, AV_LOG_ERROR, "Invalid NAL unit size: (%d)\n", nalu_size); + return -1; + } + + bits += EVC_NAL_UNIT_LENGTH_BYTE; + bits_size -= EVC_NAL_UNIT_LENGTH_BYTE; + + nalu_type = get_nalu_type(bits, bits_size, avctx); + + bits += EVC_NAL_HEADER_SIZE; + bits_size -= EVC_NAL_HEADER_SIZE; + + if (nalu_type == EVC_NUT_SPS) { // NAL Unit type: SPS (Sequence Parameter Set) + EVCParserSPS *sps; + + sps = parse_sps(bits, bits_size, ev); + + avctx->coded_width = sps->pic_width_in_luma_samples; + avctx->coded_height = sps->pic_height_in_luma_samples; + avctx->width = sps->pic_width_in_luma_samples; + avctx->height = sps->pic_height_in_luma_samples; + + if(sps->profile_idc == 0) avctx->profile = FF_PROFILE_EVC_BASELINE; + else if (sps->profile_idc == 1) avctx->profile = FF_PROFILE_EVC_MAIN; + else { + av_log(avctx, AV_LOG_ERROR, "Not supported profile (%d)\n", sps->profile_idc); + return -1; + } + + // Currently XEVD decoder supports ony YCBCR420_10LE chroma format for EVC stream + switch(sps->chroma_format_idc) { + case 0: /* YCBCR400_10LE */ + av_log(avctx, AV_LOG_ERROR, "YCBCR400_10LE: Not supported chroma format\n"); + avctx->pix_fmt = AV_PIX_FMT_GRAY10LE; + return -1; + case 1: /* YCBCR420_10LE */ + avctx->pix_fmt = AV_PIX_FMT_YUV420P10LE; + break; + case 2: /* YCBCR422_10LE */ + av_log(avctx, AV_LOG_ERROR, "YCBCR422_10LE: Not supported chroma format\n"); + avctx->pix_fmt = AV_PIX_FMT_YUV422P10LE; + return -1; + case 3: /* YCBCR444_10LE */ + av_log(avctx, AV_LOG_ERROR, "YCBCR444_10LE: Not supported chroma format\n"); + avctx->pix_fmt = AV_PIX_FMT_YUV444P10LE; + return -1; + default: + avctx->pix_fmt = AV_PIX_FMT_NONE; + av_log(avctx, AV_LOG_ERROR, "Unknown supported chroma format\n"); + return -1; + } + + // The current implementation of parse_sps function doesn't handle VUI parameters parsing, + // so at the moment it's impossible to initialize has_b_frames and max_b_frames AVCodecContex fields here. + // Currently, initialization of has_b_frames and max_b_frames AVCodecContex fields have been moved to + // libxevd_decode function where we can use xevd_config function being a part of xevd library API + // to get the needed information. + // However, if it will be needed, parse_sps function should be extended to handle VUI parameters parsing + // and the following lines should be used to initialize has_b_frames and max_b_frames fields of the AVCodecContex. + // + // sps->vui_parameters.num_reorder_pics + // if (sps->bitstream_restriction_flag && sps->vui_parameters.num_reorder_pics) { + // avctx->has_b_frames = sps->vui_parameters.num_reorder_pics; + // } + + ev->got_sps = 1; + + } else if (nalu_type == EVC_NUT_PPS) // NAL Unit type: PPS (Video Parameter Set) + ev->got_pps = 1; + else if(nalu_type == EVC_NUT_SEI) // NAL unit type: SEI (Supplemental Enhancement Information) + ev->got_sei = 1; + else if (nalu_type == EVC_NUT_IDR || nalu_type == EVC_NUT_NONIDR) // NAL Unit type: Coded slice of a IDR or non-IDR picture + ev->got_slice++; + else { + av_log(avctx, AV_LOG_ERROR, "Invalid NAL unit type: %d\n", nalu_type); + return -1; + } + + return 0; +} + +/** + * Find the end of the current frame in the bitstream. + * @return the position of the first byte of the next frame, or END_NOT_FOUND + */ +static int evc_find_frame_end(AVCodecParserContext *s, const uint8_t *buf, + int buf_size, AVCodecContext *avctx) +{ + EVCParserContext *ev = s->priv_data; + + if(!ev->to_read) { + int nal_unit_size = 0; + int next = END_NOT_FOUND; + + // This is the case when buffer size is not enough for buffer to store NAL unit length + if(buf_size < EVC_NAL_UNIT_LENGTH_BYTE) { + ev->to_read = EVC_NAL_UNIT_LENGTH_BYTE; + ev->nal_length_size = buf_size; + ev->incomplete_nalu_prefix_read = 1; + + return END_NOT_FOUND; + } + + nal_unit_size = read_nal_unit_length(buf, buf_size, avctx); + ev->nal_length_size = EVC_NAL_UNIT_LENGTH_BYTE; + + next = nal_unit_size + EVC_NAL_UNIT_LENGTH_BYTE; + ev->to_read = next; + if(next < buf_size) + return next; + else + return END_NOT_FOUND; + } else if(ev->to_read > buf_size) { + return END_NOT_FOUND; + } else { + if(ev->incomplete_nalu_prefix_read == 1) { + EVCParserContext *ev = s->priv_data; + ParseContext *pc = &ev->pc; + uint8_t nalu_len[EVC_NAL_UNIT_LENGTH_BYTE] = {0}; + int nal_unit_size = 0; + + // 1. pc->buffer contains previously read bytes of NALU prefix + // 2. buf contains the rest of NAL unit prefix bytes + // + // ~~~~~~~ + // EXAMPLE + // ~~~~~~~ + // + // In the following example we assumed that the number of already read NAL Unit prefix bytes is equal 1 + // + // ---------- + // pc->buffer -> conatins already read bytes + // ---------- + // __ pc->index == 1 + // | + // V + // ------------------------------------------------------- + // | 0 | 1 | 2 | 3 | 4 | ... | N | + // ------------------------------------------------------- + // | 0x00 | 0xXX | 0xXX | 0xXX | 0xXX | ... | 0xXX | + // ------------------------------------------------------- + // + // ---------- + // buf -> contains newly read bytes + // ---------- + // ------------------------------------------------------- + // | 0 | 1 | 2 | 3 | 4 | ... | N | + // ------------------------------------------------------- + // | 0x00 | 0x00 | 0x3C | 0xXX | 0xXX | ... | 0xXX | + // ------------------------------------------------------- + // + for(int i = 0; i < EVC_NAL_UNIT_LENGTH_BYTE; i++) { + if(i < pc->index) + nalu_len[i] = pc->buffer[i]; + else + nalu_len[i] = buf[i - pc->index]; + } + + // ---------- + // nalu_len + // ---------- + // --------------------------------- + // | 0 | 1 | 2 | 3 | + // --------------------------------- + // | 0x00 | 0x00 | 0x00 | 0x3C | + // --------------------------------- + // | NALU LENGTH | + // --------------------------------- + // NAL Unit lenght = 60 (0x0000003C) + + nal_unit_size = read_nal_unit_length(nalu_len, EVC_NAL_UNIT_LENGTH_BYTE, avctx); + + ev->to_read = nal_unit_size + EVC_NAL_UNIT_LENGTH_BYTE - pc->index; + + ev->incomplete_nalu_prefix_read = 0; + + if(ev->to_read > buf_size) + return END_NOT_FOUND; + else + return ev->to_read; + } + return ev->to_read; + } + + return END_NOT_FOUND; +} + +static int evc_parser_init(AVCodecParserContext *s) +{ + EVCParserContext *ev = s->priv_data; + + ev->got_sps = 0; + ev->got_pps = 0; + ev->got_sei = 0; + ev->got_slice = 0; + ev->nal_length_size = EVC_NAL_UNIT_LENGTH_BYTE; + ev->incomplete_nalu_prefix_read = 0; + + return 0; +} + +static int evc_parse(AVCodecParserContext *s, AVCodecContext *avctx, + const uint8_t **poutbuf, int *poutbuf_size, + const uint8_t *buf, int buf_size) +{ + int next; + EVCParserContext *ev = s->priv_data; + ParseContext *pc = &ev->pc; + int is_dummy_buf = !buf_size; + const uint8_t *dummy_buf = buf; + + if (s->flags & PARSER_FLAG_COMPLETE_FRAMES) + next = buf_size; + else { + next = evc_find_frame_end(s, buf, buf_size, avctx); + if (ff_combine_frame(pc, next, &buf, &buf_size) < 0) { + *poutbuf = NULL; + *poutbuf_size = 0; + ev->to_read -= buf_size; + return buf_size; + } + } + + is_dummy_buf &= (dummy_buf == buf); + + if (!is_dummy_buf) + parse_nal_units(s, buf, buf_size, avctx); + + *poutbuf = buf; + *poutbuf_size = buf_size; + ev->to_read -= next; + + return next; +} + +const AVCodecParser ff_evc_parser = { + .codec_ids = { AV_CODEC_ID_EVC }, + .priv_data_size = sizeof(EVCParserContext), + .parser_init = evc_parser_init, + .parser_parse = evc_parse, + .parser_close = ff_parse_close, +}; diff --git a/libavcodec/libxevd.c b/libavcodec/libxevd.c new file mode 100644 index 0000000000..ce6862d72f --- /dev/null +++ b/libavcodec/libxevd.c @@ -0,0 +1,440 @@ +/* + * libxevd decoder + * EVC (MPEG-5 Essential Video Coding) decoding using XEVD MPEG-5 EVC decoder library + * + * Copyright (C) 2021 Dawid Kozinski <d.kozinski@samsung.com> + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include <float.h> +#include <stdlib.h> + +#include <xevd.h> + +#include "libavutil/internal.h" +#include "libavutil/common.h" +#include "libavutil/opt.h" +#include "libavutil/pixdesc.h" +#include "libavutil/pixfmt.h" +#include "libavutil/imgutils.h" +#include "libavutil/cpu.h" + +#include "avcodec.h" +#include "internal.h" +#include "packet_internal.h" +#include "codec_internal.h" + +#define XEVD_PARAM_BAD_NAME -1 +#define XEVD_PARAM_BAD_VALUE -2 + +#define EVC_NAL_HEADER_SIZE 2 /* byte */ + +/** + * The structure stores all the states associated with the instance of Xeve MPEG-5 EVC decoder + */ +typedef struct XevdContext { + const AVClass *class; + + XEVD id; // XEVD instance identifier @see xevd.h + XEVD_CDSC cdsc; // decoding parameters @see xevd.h + + int decod_frames; // number of decoded frames + int packet_count; // number of packets created by decoder + + // configuration parameters + AVDictionary *xevd_opts; // xevd configuration read from a : separated list of key=value parameters + +} XevdContext; + +/** + * The function returns a pointer to a variable of type XEVD_CDSC. + * XEVD_CDSC contains all decoder parameters that should be initialized before its use. + * + * The field values of the XEVD_CDSC structure are populated based on: + * - the corresponding field values of the AvCodecConetxt structure, + * - the xevd decoder specific option values, + * (the full list of options available for the xevd encoder is displayed after executing the command ./ffmpeg --help decoder = libxevd) + * - and the xevd encoder options specified as a list of key-value pairs following the xevd-params option + * + * Order of input processing and populating the XEVD_CDSC structure + * 1. first, the corresponding fields of the AVCodecContext structure are processed, (i.e -threads 4) + * 2. then xevd-specific options are added as AVOption to the xevd AVCodec implementation (i.e -threads 3) + * 3. finally, the options specified after the xevd-params option as the parameter list of type key value are processed (i.e -xevd-params "m=2") + * + * Some options can be set in different ways. In this case, please follow the above-mentioned order of processing. + * The most recent assignments overwrite the previous values. + * + * @param[in] avctx codec context + * @param[out] cdsc contains all encoder parameters that should be initialized before its use + * + * @return 0 on success, negative error code on failure + */ +static int get_conf(AVCodecContext *avctx, XEVD_CDSC *cdsc) +{ + int cpu_count = av_cpu_count(); + + /* clear XEVS_CDSC structure */ + memset(cdsc, 0, sizeof(XEVD_CDSC)); + + /* init XEVD_CDSC */ + if(avctx->thread_count <= 0) + cdsc->threads = (cpu_count < XEVD_MAX_TASK_CNT) ? cpu_count : XEVD_MAX_TASK_CNT; + else if(avctx->thread_count > XEVD_MAX_TASK_CNT) + cdsc->threads = XEVD_MAX_TASK_CNT; + else + cdsc->threads = avctx->thread_count; + + return 0; +} + +/** + * Read NAL unit length + * @param bs input data (bitstream) + * @return the length of NAL unit on success, 0 value on failure + */ +static uint32_t read_nal_unit_length(const uint8_t *bs, int bs_size, AVCodecContext *avctx) +{ + uint32_t len = 0; + XEVD_INFO info; + int ret; + + if(bs_size == XEVD_NAL_UNIT_LENGTH_BYTE) { + ret = xevd_info((void *)bs, XEVD_NAL_UNIT_LENGTH_BYTE, 1, &info); + if (XEVD_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Cannot get bitstream information\n"); + return 0; + } + len = info.nalu_len; + if(len == 0) { + av_log(avctx, AV_LOG_ERROR, "Invalid bitstream size! [%d]\n", bs_size); + return 0; + } + } + + return len; +} + +/** + * @param[in] xectx the structure that stores all the state associated with the instance of Xeve MPEG-5 EVC decoder + * @param[out] avctx codec context + * @return 0 on success, negative value on failure + */ +static int export_stream_params(const XevdContext *xectx, AVCodecContext *avctx) +{ + int ret; + int size; + int color_space; + + avctx->pix_fmt = AV_PIX_FMT_YUV420P10; + + size = 4; + ret = xevd_config(xectx->id, XEVD_CFG_GET_CODED_WIDTH, &avctx->coded_width, &size); + if (XEVD_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to get coded_width\n"); + return AVERROR_EXTERNAL; + } + + ret = xevd_config(xectx->id, XEVD_CFG_GET_CODED_HEIGHT, &avctx->coded_height, &size); + if (XEVD_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to get coded_height\n"); + return AVERROR_EXTERNAL; + } + + ret = xevd_config(xectx->id, XEVD_CFG_GET_WIDTH, &avctx->width, &size); + if (XEVD_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to get width\n"); + return AVERROR_EXTERNAL; + } + + ret = xevd_config(xectx->id, XEVD_CFG_GET_HEIGHT, &avctx->height, &size); + if (XEVD_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to get height\n"); + return AVERROR_EXTERNAL; + } + + ret = xevd_config(xectx->id, XEVD_CFG_GET_COLOR_SPACE, &color_space, &size); + if (XEVD_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to get color_space\n"); + return AVERROR_EXTERNAL; + } + switch(color_space) { + case XEVD_CS_YCBCR400_10LE: + avctx->pix_fmt = AV_PIX_FMT_GRAY10LE; + break; + case XEVD_CS_YCBCR420_10LE: + avctx->pix_fmt = AV_PIX_FMT_YUV420P10LE; + break; + case XEVD_CS_YCBCR422_10LE: + avctx->pix_fmt = AV_PIX_FMT_YUV422P10LE; + break; + case XEVD_CS_YCBCR444_10LE: + avctx->pix_fmt = AV_PIX_FMT_YUV444P10LE; + break; + default: + av_log(avctx, AV_LOG_ERROR, "Unknown color space\n"); + avctx->pix_fmt = AV_PIX_FMT_NONE; + return AVERROR_INVALIDDATA; + } + + // the function returns sps->num_reorder_pics + ret = xevd_config(xectx->id, XEVD_CFG_GET_MAX_CODING_DELAY, &avctx->max_b_frames, &size); + if (XEVD_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to get max_coding_delay\n"); + return AVERROR_EXTERNAL; + } + + avctx->has_b_frames = (avctx->max_b_frames) ? 1 : 0; + + avctx->color_primaries = AVCOL_PRI_UNSPECIFIED; + avctx->color_trc = AVCOL_TRC_UNSPECIFIED; + avctx->colorspace = AVCOL_SPC_UNSPECIFIED; + + return 0; +} + +/** + * Initialize decoder + * Create a decoder instance and allocate all the needed resources + * + * @param avctx codec context + * @return 0 on success, negative error code on failure + */ +static av_cold int libxevd_init(AVCodecContext *avctx) +{ + XevdContext *xectx = avctx->priv_data; + int ret = 0; + XEVD_CDSC *cdsc = &(xectx->cdsc); + + /* read configurations and set values for created descriptor (XEVD_CDSC) */ + ret = get_conf(avctx, cdsc); + if (ret != 0) { + av_log(avctx, AV_LOG_ERROR, "Cannot get configuration\n"); + return AVERROR_INVALIDDATA; + } + + /* create decoder */ + xectx->id = xevd_create(&(xectx->cdsc), NULL); + if(xectx->id == NULL) { + av_log(avctx, AV_LOG_ERROR, "Cannot create XEVD encoder\n"); + return AVERROR_EXTERNAL; + } + + xectx->packet_count = 0; + xectx->decod_frames = 0; + + return 0; +} + +/** + * Decode picture + * + * @param avctx codec context + * @param[out] frame decoded frame + * @param[out] got_frame_ptr decoder sets to 0 or 1 to indicate that a + * non-empty frame or subtitle was returned in + * outdata. + * @param[in] avpkt AVPacket containing encoded data to be decoded + * + * @return amount of bytes read from the packet on success, negative error + * code on failure + */ + static int libxevd_decode(struct AVCodecContext *avctx, struct AVFrame *frame, int *got_frame_ptr, AVPacket *avpkt) +{ + XevdContext *xectx = NULL; + XEVD_IMGB *imgb = NULL; + XEVD_STAT stat; + XEVD_BITB bitb; + int xevd_ret, nalu_size, bs_read_pos; + int ret = 0; + + if(avctx == NULL) { + av_log(avctx, AV_LOG_ERROR, "Invalid input parameter: AVCodecContext\n"); + return AVERROR(EINVAL); + } + xectx = avctx->priv_data; + if(xectx == NULL) { + av_log(avctx, AV_LOG_ERROR, "Invalid XEVD context\n"); + return AVERROR(EINVAL); + } + + if(avpkt->size > 0) { + bs_read_pos = 0; + imgb = NULL; + while(avpkt->size > (bs_read_pos + XEVD_NAL_UNIT_LENGTH_BYTE)) { + + memset(&stat, 0, sizeof(XEVD_STAT)); + memset(&bitb, 0, sizeof(XEVD_BITB)); + + nalu_size = read_nal_unit_length(avpkt->data + bs_read_pos, XEVD_NAL_UNIT_LENGTH_BYTE, avctx); + if(nalu_size == 0) { + av_log(avctx, AV_LOG_ERROR, "Invalid bitstream\n"); + ret = AVERROR_INVALIDDATA; + goto ERR; + } + bs_read_pos += XEVD_NAL_UNIT_LENGTH_BYTE; + + bitb.addr = avpkt->data + bs_read_pos; + bitb.ssize = nalu_size; + + /* main decoding block */ + xevd_ret = xevd_decode(xectx->id, &bitb, &stat); + if(XEVD_FAILED(xevd_ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to decode bitstream\n"); + ret = AVERROR_EXTERNAL; + goto ERR; + } + + bs_read_pos += nalu_size; + + if(stat.nalu_type == XEVD_NUT_SPS) { // EVC stream parameters changed + if((ret = export_stream_params(xectx, avctx)) != 0) + goto ERR; + } + + if(stat.read != nalu_size) + av_log(avctx, AV_LOG_INFO, "Different reading of bitstream (in:%d, read:%d)\n,", nalu_size, stat.read); + if(stat.fnum >= 0) { + // already has a decoded image + if (imgb) { + // xevd_pull uses pool of objects of type XEVD_IMGB. + // The pool size is equal MAX_PB_SIZE (26), so release object when it is no more needed + imgb->release(imgb); + imgb = NULL; + } + xevd_ret = xevd_pull(xectx->id, &imgb); + if(XEVD_FAILED(xevd_ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to pull the decoded image (xevd error code: %d, frame#=%d)\n", xevd_ret, stat.fnum); + ret = AVERROR_EXTERNAL; + goto ERR; + } else if (xevd_ret == XEVD_OK_FRM_DELAYED) { + if(imgb) { + // xevd_pull uses pool of objects of type XEVD_IMGB. + // The pool size is equal MAX_PB_SIZE (26), so release object when it is no more needed + imgb->release(imgb); + imgb = NULL; + } + } + } + } + } else { // bumping + xevd_ret = xevd_pull(xectx->id, &(imgb)); + if(xevd_ret == XEVD_ERR_UNEXPECTED) { // bumping process completed + *got_frame_ptr = 0; + return 0; + } else if(XEVD_FAILED(xevd_ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to pull the decoded image (xevd error code: %d)\n", xevd_ret); + ret = AVERROR_EXTERNAL; + goto ERR; + } + } + + if(imgb) { + // @todo supports other color space and bit depth + if(imgb->cs != XEVD_CS_YCBCR420_10LE) { + av_log(avctx, AV_LOG_ERROR, "Not supported pixel format: %s\n", av_get_pix_fmt_name(avctx->pix_fmt)); + ret = AVERROR_INVALIDDATA; + goto ERR; + } + + if (imgb->w[0] != avctx->width || imgb->h[0] != avctx->height) { // stream resolution changed + if(ff_set_dimensions(avctx, imgb->w[0], imgb->h[0]) < 0) { + av_log(avctx, AV_LOG_ERROR, "Cannot set new dimension\n"); + ret = AVERROR_INVALIDDATA; + goto ERR; + } + } + + frame->coded_picture_number++; + frame->display_picture_number++; + frame->format = AV_PIX_FMT_YUV420P10LE; + + if (ff_get_buffer(avctx, frame, 0) < 0) { + av_log(avctx, AV_LOG_ERROR, "Cannot get AV buffer\n"); + ret = AVERROR(ENOMEM); + goto ERR; + } + + frame->pts = avpkt->pts; + + av_image_copy(frame->data, frame->linesize, (const uint8_t **)imgb->a, + imgb->s, avctx->pix_fmt, + imgb->w[0], imgb->h[0]); + + xectx->decod_frames++; + *got_frame_ptr = 1; + + // xevd_pull uses pool of objects of type XEVD_IMGB. + // The pool size is equal MAX_PB_SIZE (26), so release object when it is no more needed + imgb->release(imgb); + imgb = NULL; + } else + *got_frame_ptr = 0; + + xectx->packet_count++; + + return avpkt->size; + +ERR: + if(imgb) { + imgb->release(imgb); + imgb = NULL; + } + *got_frame_ptr = 0; + + return ret; +} + +/** + * Destroy decoder + * + * @param avctx codec context + * @return 0 on success + */ +static av_cold int libxevd_close(AVCodecContext *avctx) +{ + XevdContext *xectx = avctx->priv_data; + if(xectx->id) { + xevd_delete(xectx->id); + xectx->id = NULL; + } + + return 0; +} + +#define OFFSET(x) offsetof(XevdContext, x) +#define VD AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_DECODING_PARAM + +static const AVClass xevd_class = { + .class_name = "libxevd", + .item_name = av_default_item_name, + .version = LIBAVUTIL_VERSION_INT, +}; + +const FFCodec ff_libxevd_decoder = { + .p.name = "evc", + .p.long_name = NULL_IF_CONFIG_SMALL("EVC / MPEG-5 Essential Video Coding (EVC)"), + .p.type = AVMEDIA_TYPE_VIDEO, + .p.id = AV_CODEC_ID_EVC, + .init = libxevd_init, + FF_CODEC_DECODE_CB(libxevd_decode), + .close = libxevd_close, + .priv_data_size = sizeof(XevdContext), + .p.priv_class = &xevd_class, + .p.capabilities = AV_CODEC_CAP_DELAY | AV_CODEC_CAP_AUTO_THREADS | AV_CODEC_CAP_AVOID_PROBING | AV_CODEC_CAP_DR1, + .p.wrapper_name = "libxevd", +}; diff --git a/libavcodec/libxeve.c b/libavcodec/libxeve.c new file mode 100644 index 0000000000..e115ce63d2 --- /dev/null +++ b/libavcodec/libxeve.c @@ -0,0 +1,753 @@ +/* + * libxeve encoder + * EVC (MPEG-5 Essential Video Coding) encoding using XEVE MPEG-5 EVC encoder library + * + * Copyright (C) 2021 Dawid Kozinski <d.kozinski@samsung.com> + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include <float.h> +#include <stdlib.h> + +#include <xeve.h> + +#include "libavutil/internal.h" +#include "libavutil/common.h" +#include "libavutil/opt.h" +#include "libavutil/pixdesc.h" +#include "libavutil/pixfmt.h" +#include "libavutil/time.h" +#include "libavutil/cpu.h" +#include "libavutil/avstring.h" + +#include "avcodec.h" +#include "internal.h" +#include "packet_internal.h" +#include "codec_internal.h" +#include "encode.h" + +#define MAX_BS_BUF (16*1024*1024) + +/** + * Error codes + */ +#define XEVE_PARAM_BAD_NAME -100 +#define XEVE_PARAM_BAD_VALUE -200 + +/** + * Encoder states + * + * STATE_ENCODING - the encoder receives and processes input frames + * STATE_BUMPING - there are no more input frames, however the encoder still processes previously received data + * STATE_SKIPPING - skipping input frames + */ +typedef enum State { + STATE_ENCODING, + STATE_BUMPING, + STATE_SKIPPING +} State; + +/** + * The structure stores all the states associated with the instance of Xeve MPEG-5 EVC encoder + */ +typedef struct XeveContext { + const AVClass *class; + + XEVE id; // XEVE instance identifier + XEVE_CDSC cdsc; // coding parameters i.e profile, width & height of input frame, num of therads, frame rate ... + XEVE_BITB bitb; // bitstream buffer (output) + XEVE_STAT stat; // encoding status (output) + XEVE_IMGB imgb; // image buffer (input) + + State state; // encoder state (skipping, encoding, bumping) + + // Chroma subsampling + int width_luma; + int height_luma; + int width_chroma; + int height_chroma; + + int profile_id; // encoder profile (main, baseline) + int preset_id; // preset of xeve ( fast, medium, slow, placebo) + int tune_id; // tune of xeve (psnr, zerolatency) + int input_depth; // input bit-depth: 8bit, 10bit + int color_format; // input data color format: currently only XEVE_CF_YCBCR420 is supported + int hash; // embed picture signature (HASH) for conformance checking in decoding + + /* variables for input parameter */ + char *op_preset; + char *op_tune; + int op_qp; + int op_crf; + + // configuration parameters + // xeve configuration read from a : separated list of key=value parameters + AVDictionary *xeve_params; +} XeveContext; + +/** + * Gets Xeve pre-defined preset + * + * @param preset string describing Xeve encoder preset (fast, medium, slow, placebo) + * @return XEVE pre-defined profile on success, negative value on failure + */ +static int get_preset_id(const char *preset) +{ + if((!strcmp(preset, "fast"))) + return XEVE_PRESET_FAST; + else if (!strcmp(preset, "medium")) + return XEVE_PRESET_MEDIUM; + else if (!strcmp(preset, "slow")) + return XEVE_PRESET_SLOW; + else if (!strcmp(preset, "placebo")) + return XEVE_PRESET_PLACEBO; + else + return AVERROR(EINVAL); +} + +/** + * Gets Xeve pre-defined tune id + * + * @param[in] tune string describing Xeve encoder tune (psnr, zerolatency) + * @return XEVE pre-defined profile on success, negative value on failure + */ +static int get_tune_id(const char *tune) +{ + if((!strcmp(tune, "psnr"))) + return XEVE_TUNE_PSNR; + else if (!strcmp(tune, "zerolatency")) + return XEVE_TUNE_ZEROLATENCY; + else + return AVERROR(EINVAL); +} + +/** + * Convert FFmpeg pixel format (AVPixelFormat) to XEVE pre-defined color format + * + * @param[in] av_pix_fmt pixel format (@see https://ffmpeg.org/doxygen/trunk/pixfmt_8h.html#a9a8e335cf3be472042bc9f0cf80cd4c5) + * @param[out] xeve_col_fmt XEVE pre-defined color format (@see xeve.h) + * + * @return 0 on success, negative value on failure + */ +static int xeve_color_fmt(enum AVPixelFormat av_pix_fmt, int *xeve_col_fmt) +{ + switch (av_pix_fmt) { + case AV_PIX_FMT_YUV420P: + *xeve_col_fmt = XEVE_CF_YCBCR420; + break; + case AV_PIX_FMT_YUV420P10: + *xeve_col_fmt = XEVE_CF_YCBCR420; + break; + default: + *xeve_col_fmt = XEVE_CF_UNKNOWN; + return AVERROR_INVALIDDATA; + } + + return 0; +} + +/** + * Convert FFmpeg pixel format (AVPixelFormat) into XEVE pre-defined color space + * + * @param[in] px_fmt pixel format (@see https://ffmpeg.org/doxygen/trunk/pixfmt_8h.html#a9a8e335cf3be472042bc9f0cf80cd4c5) + * + * @return XEVE pre-defined color space (@see xeve.h) on success, XEVE_CF_UNKNOWN on failure + */ +static int xeve_color_space(enum AVPixelFormat av_pix_fmt) +{ + /* color space of input image */ + int cs = XEVE_CF_UNKNOWN; + + switch (av_pix_fmt) { + case AV_PIX_FMT_YUV420P: + cs = XEVE_CS_YCBCR420; + break; + case AV_PIX_FMT_YUV420P10: +#if AV_HAVE_BIGENDIAN + cs = XEVE_CS_SET(XEVE_CF_YCBCR420, 10, 1); +#else + cs = XEVE_CS_YCBCR420_10LE; +#endif + + break; + default: + cs = XEVE_CF_UNKNOWN; + break; + } + + return cs; +} + +static int kbps_str_to_int(char *str) +{ + int kbps = 0; + char *saveptr = NULL; + if (strchr(str, 'K') || strchr(str, 'k')) { + char *tmp = av_strtok(str, "Kk ", &saveptr); + kbps = (int)(strtof(tmp, NULL)); + } else if (strchr(str, 'M') || strchr(str, 'm')) { + char *tmp = av_strtok(str, "Mm ", &saveptr); + kbps = (int)(strtof(tmp, NULL) * 1000); + } else { + kbps = strtol(str, NULL, 10); + } + + return kbps; +} + +/** + * Parse : separated list of key=value parameters + * + * @param[in] avctx context for logger + * @param[in] key + * @param[in] value + * @param[out] cdsc contains all Xeve MPEG-5 EVC encoder encoder parameters that + * should be initialized before the encoder is use + * + * @return 0 on success, negative value on failure + */ +static int parse_xeve_params(AVCodecContext *avctx, const char *key, const char *value, XEVE_CDSC* cdsc) +{ + XeveContext *xectx = NULL; + xectx = avctx->priv_data; + + if(!key) { + av_log(avctx, AV_LOG_ERROR, "Ivalid argument: key string is NULL\n"); + return XEVE_PARAM_BAD_VALUE; + } + if(!value) { + if (strcmp(key, "hash") == 0) { + xectx->hash = 1; + av_log(avctx, AV_LOG_INFO, "embedding signature is enabled\n"); + } else { + av_log(avctx, AV_LOG_ERROR, "Ivalid argument: value string is NULL\n"); + return XEVE_PARAM_BAD_VALUE; + } + } else if (strcmp(key, "vbv-bufsize") == 0 ) { + cdsc->param.vbv_bufsize = kbps_str_to_int((char *)value); + av_log(avctx, AV_LOG_INFO, "VBV buffer size: %dkbits\n", cdsc->param.vbv_bufsize); + } else { + av_log(avctx, AV_LOG_ERROR, "Unknown xeve codec option: %s\n", key); + return XEVE_PARAM_BAD_NAME; + } + + return 0; +} + +/** + * The function returns a pointer to the object of the XEVE_CDSC type. + * XEVE_CDSC contains all encoder parameters that should be initialized before the encoder is used. + * + * The field values of the XEVE_CDSC structure are populated based on: + * - the corresponding field values of the AvCodecConetxt structure, + * - the xeve encoder specific option values, + * (the full list of options available for xeve encoder is displayed after executing the command ./ffmpeg --help encoder = libxeve) + * - and the xeve encoder options specified as a list of key-value pairs following the xeve-params option + * + * The order of processing input data and populating the XEVE_CDSC structure + * 1) first, the fields of the AVCodecContext structure corresponding to the provided input options are processed, + * (i.e -pix_fmt yuv420p -s:v 1920x1080 -r 30 -profile:v 0) + * 2) then xeve-specific options added as AVOption to the xeve AVCodec implementation + * (i.e -threads_cnt 3 -preset 0) + * 3) finally, the options specified after the xeve-params option as the parameter list of type key value are processed + * (i.e -xeve-params "m=2:q=17") + * + * Keep in mind that, there are options that can be set in different ways. + * In this case, please follow the above-mentioned order of processing. + * The most recent assignments overwrite the previous values. + * + * @param[in] avctx codec context (AVCodecContext) + * @param[out] cdsc contains all Xeve MPEG-5 EVC encoder encoder parameters that should be initialized before the encoder is use + * + * @return 0 on success, negative error code on failure + */ +static int get_conf(AVCodecContext *avctx, XEVE_CDSC *cdsc) +{ + XeveContext *xectx = NULL; + int ret; + + xectx = avctx->priv_data; + xectx->hash = 0; + + /* initialize xeve_param struct with default values */ + ret = xeve_param_default(&cdsc->param); + if (XEVE_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Cannot set_default parameter\n"); + return AVERROR_EXTERNAL; + } + + /* read options from AVCodecContext */ + if(avctx->width > 0) + cdsc->param.w = xectx->width_luma = avctx->width; + + if(avctx->height > 0) + cdsc->param.h = xectx->height_luma = avctx->height; + + if(avctx->framerate.num > 0) { + // fps can be float number, but xeve API doesn't support it + cdsc->param.fps = (int)(((float)avctx->framerate.num / avctx->framerate.den) + 0.5); + } + + if(avctx->gop_size >= 0) { // GOP size (key-frame interval) + cdsc->param.keyint = avctx->gop_size; // 0: only one I-frame at the first time; 1: every frame is coded in I-frame + } + + if (avctx->max_b_frames == 0 || avctx->max_b_frames == 1 || avctx->max_b_frames == 3 || + avctx->max_b_frames == 7 || avctx->max_b_frames == 15) { // number of b-frames + cdsc->param.bframes = avctx->max_b_frames; + } else { + av_log(avctx, AV_LOG_ERROR, "Incorrect value for maximum number of B frames: (%d) \n" + "Acceptable values for bf option (maximum number of B frames) are 0,1,3,7 or 15\n", avctx->max_b_frames); + return AVERROR_INVALIDDATA; + } + + if (avctx->level >= 0) + cdsc->param.level_idc = avctx->level; + + if (avctx->rc_buffer_size > 0) { // VBV buf size + cdsc->param.vbv_bufsize = (int)(avctx->rc_buffer_size / 1000); + } + + // rc_type: Rate control type [ 0(CQP) / 1(ABR) / 2(CRF) ] + if (avctx->bit_rate > 0) { + if (avctx->bit_rate / 1000 > INT_MAX || avctx->rc_max_rate / 1000 > INT_MAX) { + av_log(avctx, AV_LOG_ERROR, "Not supported bitrate bit_rate and rc_max_rate > %d000\n", INT_MAX); + return AVERROR_INVALIDDATA; + } + cdsc->param.bitrate = (int)(avctx->bit_rate / 1000); + cdsc->param.rc_type = XEVE_RC_ABR; + } + + if (xectx->op_crf >= 0) { + cdsc->param.crf = xectx->op_crf; + cdsc->param.rc_type = XEVE_RC_CRF; + } + + if(avctx->thread_count <= 0) { + int cpu_count = av_cpu_count(); + cdsc->param.threads = (cpu_count < XEVE_MAX_THREADS) ? cpu_count : XEVE_MAX_THREADS; + } else if(avctx->thread_count > XEVE_MAX_THREADS) + cdsc->param.threads = XEVE_MAX_THREADS; + else + cdsc->param.threads = avctx->thread_count; + + + xeve_color_fmt(avctx->pix_fmt, &xectx->color_format); + +#if AV_HAVE_BIGENDIAN + cdsc->param.cs = XEVE_CS_SET(xectx->color_format, cdsc->param.codec_bit_depth, 1); +#else + cdsc->param.cs = XEVE_CS_SET(xectx->color_format, cdsc->param.codec_bit_depth, 0); +#endif + + cdsc->max_bs_buf_size = MAX_BS_BUF; + + if(avctx->profile == FF_PROFILE_EVC_BASELINE) + xectx->profile_id = XEVE_PROFILE_BASELINE; + else if(avctx->profile == FF_PROFILE_EVC_MAIN) + xectx->profile_id = XEVE_PROFILE_MAIN; + else { + av_log(avctx, AV_LOG_ERROR, "Unknown encoder profile (%d)\n" + "Acceptable values for profile option are 0 and 1 (0: baseline profile; 1: main profile)\n", avctx->profile); + return AVERROR_INVALIDDATA; + } + + if (xectx->op_preset) { // preset + xectx->preset_id = get_preset_id(xectx->op_preset); + } + + if (xectx->op_tune) { // tune + xectx->tune_id = get_tune_id(xectx->op_tune); + } + + ret = xeve_param_ppt(&cdsc->param, xectx->profile_id, xectx->preset_id, xectx->tune_id); + if (XEVE_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Cannot set profile(%d), preset(%d), tune(%d)\n", xectx->profile_id, xectx->preset_id, xectx->tune_id); + return AVERROR_EXTERNAL; + } + + /* parse : separated list of key=value parameters and set values for created descriptor (XEVE_CDSC) */ + { + const AVDictionaryEntry *en = NULL; + + // Start to parse xeve_params + while ((en = av_dict_get(xectx->xeve_params, "", en, AV_DICT_IGNORE_SUFFIX))) { + int parse_ret = parse_xeve_params(avctx, en->key, en->value, cdsc); + + switch (parse_ret) { + case XEVE_PARAM_BAD_NAME: + av_log(avctx, AV_LOG_WARNING, "Unknown option: %s.\n", en->key); + break; + case XEVE_PARAM_BAD_VALUE: + av_log(avctx, AV_LOG_WARNING, "Invalid value for %s: %s.\n", en->key, en->value); + break; + default: + break; + } + } + } + + return 0; +} + +/** + * Set XEVE_CFG_SET_USE_PIC_SIGNATURE for encoder + * + * @param[in] logger context + * @param[in] id XEVE encodec instance identifier + * @param[in] ctx the structure stores all the states associated with the instance of Xeve MPEG-5 EVC encoder + * + * @return 0 on success, negative error code on failure + */ +static int set_extra_config(AVCodecContext* avctx, XEVE id, XeveContext *ctx) +{ + int ret, size, value; + size = 4; + + // embed SEI messages identifying encoder parameters and command line arguments + // - 0: off\n" + // - 1: emit sei info" + // + // SEI - Supplemental enhancement information contains information + // that is not necessary to decode the samples of coded pictures from VCL NAL units. + // Some SEI message information is required to check bitstream conformance + // and for output timing decoder conformance. + // @see ISO_IEC_23094-1_2020 7.4.3.5 + // @see ISO_IEC_23094-1_2020 Annex D + ret = xeve_config(id, XEVE_CFG_SET_SEI_CMD, &value, &size); // sei_cmd_info + if (XEVE_FAILED(ret)) + { + av_log(avctx, AV_LOG_ERROR, "Failed to set config for sei command info messages\n"); + return AVERROR_EXTERNAL; + } + + if(ctx->hash) { + value = 1; + ret = xeve_config(id, XEVE_CFG_SET_USE_PIC_SIGNATURE, &value, &size); + if(XEVE_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to set config for picture signature\n"); + return AVERROR_EXTERNAL; + } + } + + return 0; +} + +/** + * @brief Switch encoder to bumping mode + * + * @param id XEVE encodec instance identifier + * @return 0 on success, negative error code on failure + */ +static int setup_bumping(XEVE id) +{ + int val = 1; + int size = sizeof(int); + if(XEVE_FAILED(xeve_config(id, XEVE_CFG_SET_FORCE_OUT, (void *)(&val), &size))) + return AVERROR_EXTERNAL; + + return 0; +} + +/** + * @brief Initialize eXtra-fast Essential Video Encoder codec + * Create an encoder instance and allocate all the needed resources + * + * @param avctx codec context + * @return 0 on success, negative error code on failure + */ +static av_cold int libxeve_init(AVCodecContext *avctx) +{ + XeveContext *xectx = avctx->priv_data; + unsigned char *bs_buf = NULL; + int i; + int shift_h = 0; + int shift_v = 0; + XEVE_IMGB *imgb = NULL; + int ret = 0; + + XEVE_CDSC *cdsc = &(xectx->cdsc); + + /* allocate bitstream buffer */ + bs_buf = av_malloc(MAX_BS_BUF); + if(bs_buf == NULL) { + av_log(avctx, AV_LOG_ERROR, "Cannot allocate bitstream buffer\n"); + return AVERROR(ENOMEM); + } + xectx->bitb.addr = bs_buf; + xectx->bitb.bsize = MAX_BS_BUF; + + /* read configurations and set values for created descriptor (XEVE_CDSC) */ + if ((ret = get_conf(avctx, cdsc)) != 0) { + av_log(avctx, AV_LOG_ERROR, "Cannot get configuration\n"); + return AVERROR(EINVAL); + } + + if ((ret = xeve_param_check(&cdsc->param)) != 0) { + av_log(avctx, AV_LOG_ERROR, "Invalid configuration\n"); + return AVERROR(EINVAL); + } + + /* create encoder */ + xectx->id = xeve_create(cdsc, NULL); + if(xectx->id == NULL) { + av_log(avctx, AV_LOG_ERROR, "Cannot create XEVE encoder\n"); + return AVERROR_EXTERNAL; + } + + if((ret = set_extra_config(avctx, xectx->id, xectx))!=0) { + av_log(avctx, AV_LOG_ERROR, "Cannot set extra configuration\n"); + return AVERROR(EINVAL); + } + + if((ret = av_pix_fmt_get_chroma_sub_sample(avctx->pix_fmt, &shift_h, &shift_v)) != 0) { + av_log(avctx, AV_LOG_ERROR, "Failed to get chroma shift\n"); + return AVERROR(EINVAL); + } + + // YUV format explanation + // shift_h == 1 && shift_v == 1 : YUV420 + // shift_h == 1 && shift_v == 0 : YUV422 + // shift_h == 0 && shift_v == 0 : YUV444 + // + xectx->width_chroma = AV_CEIL_RSHIFT(xectx->width_luma, shift_h); + xectx->height_chroma = AV_CEIL_RSHIFT(xectx->height_luma, shift_v); + + /* set default values for input image buffer */ + imgb = &xectx->imgb; + imgb->cs = xeve_color_space(avctx->pix_fmt); + imgb->np = 3; /* only for yuv420p, yuv420ple */ + + for (i = 0; i < imgb->np; i++) + imgb->x[i] = imgb->y[i] = 0; + + imgb->w[0] = imgb->aw[0] = xectx->width_luma; + imgb->w[1] = imgb->w[2] = imgb->aw[1] = imgb->aw[2] = xectx->width_chroma; + imgb->h[0] = imgb->ah[0] = xectx->height_luma; + imgb->h[1] = imgb->h[2] = imgb->ah[1] = imgb->ah[2] = xectx->height_chroma; + + xectx->state = STATE_ENCODING; + + return 0; +} + +/** + * Encode raw data frame into EVC packet + * + * @param[in] avctx codec context + * @param[out] pkt output AVPacket containing encoded data + * @param[in] frame AVFrame containing the raw data to be encoded + * @param[out] got_packet encoder sets to 0 or 1 to indicate that a + * non-empty packet was returned in pkt + * + * @return 0 on success, negative error code on failure + */ +static int libxeve_encode(AVCodecContext *avctx, AVPacket *avpkt, + const AVFrame *frame, int *got_packet) +{ + XeveContext *xectx = avctx->priv_data; + int ret = -1; + int xeve_cs; + + if(xectx->state == STATE_SKIPPING && frame ) { + xectx->state = STATE_ENCODING; // Entering encoding process + } else if(xectx->state == STATE_ENCODING && frame == NULL) { + if (setup_bumping(xectx->id) == 0) + xectx->state = STATE_BUMPING; // Entering bumping process + else { + av_log(avctx, AV_LOG_ERROR, "Failed to setup bumping\n"); + xectx->state = STATE_SKIPPING; + } + } + + if(xectx->state == STATE_ENCODING) { + const AVPixFmtDescriptor *pixel_fmt_desc = av_pix_fmt_desc_get (frame->format); + if(!pixel_fmt_desc) { + av_log(avctx, AV_LOG_ERROR, "Invalid pixel format descriptor for pixel format: %s\n", av_get_pix_fmt_name(avctx->pix_fmt)); + return AVERROR_INVALIDDATA; + } + + xeve_cs = xeve_color_space(avctx->pix_fmt); + if(xeve_cs != XEVE_CS_YCBCR420 && xeve_cs != XEVE_CS_YCBCR420_10LE) { + av_log(avctx, AV_LOG_ERROR, "Invalid pixel format: %s\n", av_get_pix_fmt_name(avctx->pix_fmt)); + return AVERROR_INVALIDDATA; + } + + { + int i; + XEVE_IMGB *imgb = NULL; + + imgb = &xectx->imgb; + + for (i = 0; i < imgb->np; i++) { + imgb->a[i] = frame->data[i]; + imgb->s[i] = frame->linesize[i]; + } + + if(xectx->id == NULL) { + av_log(avctx, AV_LOG_ERROR, "Invalid XEVE encoder\n"); + return AVERROR_INVALIDDATA; + } + + imgb->ts[0] = frame->pts; + imgb->ts[1] = 0; + + /* push image to encoder */ + ret = xeve_push(xectx->id, imgb); + if(XEVE_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "xeve_push() failed\n"); + return AVERROR_EXTERNAL; + } + } + } + if(xectx->state == STATE_ENCODING || xectx->state == STATE_BUMPING) { + + /* encoding */ + ret = xeve_encode(xectx->id, &(xectx->bitb), &(xectx->stat)); + if(XEVE_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "xeve_encode() failed\n"); + return AVERROR_EXTERNAL; + } + + /* store bitstream */ + if (ret == XEVE_OK_OUT_NOT_AVAILABLE) { // Return OK but picture is not available yet + *got_packet = 0; + return 0; + } else if(ret == XEVE_OK) { + int av_pic_type; + + if(xectx->stat.write > 0) { + + ret = ff_get_encode_buffer(avctx, avpkt, xectx->stat.write, 0); + if (ret < 0) { + return ret; + } + + memcpy(avpkt->data, xectx->bitb.addr, xectx->stat.write); + + avpkt->pts = xectx->bitb.ts[0]; + avpkt->dts = xectx->bitb.ts[1]; + + switch(xectx->stat.stype) { + case XEVE_ST_I: + av_pic_type = AV_PICTURE_TYPE_I; + avpkt->flags |= AV_PKT_FLAG_KEY; + break; + case XEVE_ST_P: + av_pic_type = AV_PICTURE_TYPE_P; + break; + case XEVE_ST_B: + av_pic_type = AV_PICTURE_TYPE_B; + break; + case XEVE_ST_UNKNOWN: + av_log(avctx, AV_LOG_ERROR, "Unknown slice type\n"); + return AVERROR_INVALIDDATA; + } + + ff_side_data_set_encoder_stats(avpkt, xectx->stat.qp * FF_QP2LAMBDA, NULL, 0, av_pic_type); + + *got_packet = 1; + } + } else if (ret == XEVE_OK_NO_MORE_FRM) { + // Return OK but no more frames + return 0; + } else { + av_log(avctx, AV_LOG_ERROR, "Invalid return value: %d\n", ret); + return AVERROR_EXTERNAL; + } + } else { + av_log(avctx, AV_LOG_ERROR, "Udefined encoder state\n"); + return AVERROR_INVALIDDATA; + } + + return 0; +} + +/** + * Destroy the encoder and release all the allocated resources + * + * @param avctx codec context + * @return 0 on success, negative error code on failure + */ +static av_cold int libxeve_close(AVCodecContext *avctx) +{ + XeveContext *xectx = avctx->priv_data; + + xeve_delete(xectx->id); + + av_free(xectx->bitb.addr); /* release bitstream buffer */ + + return 0; +} + +#define OFFSET(x) offsetof(XeveContext, x) +#define VE AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_ENCODING_PARAM + +static const enum AVPixelFormat supported_pixel_formats[] = { + AV_PIX_FMT_YUV420P, + AV_PIX_FMT_YUV420P10, + AV_PIX_FMT_NONE +}; + +// Example of using: ./ffmpeg -xeve-params "m=2:q=17" +// Consider using following options (./ffmpeg --help encoder=libxeve) +// +static const AVOption xeve_options[] = { + { "preset", "Encoding preset for setting encoding speed [fast, medium, slow, placebo]", OFFSET(op_preset), AV_OPT_TYPE_STRING, { .str = "medium" }, 0, 0, VE }, + { "tune", "Tuneing parameter for special purpose operation [psnr, zerolatency]", OFFSET(op_tune), AV_OPT_TYPE_STRING, { 0 }, 0, 0, VE}, + { "qp", "quantization parameter qp <0..51> [default: 32]", OFFSET(op_qp), AV_OPT_TYPE_INT, { .i64 = 32 }, 0, 51, VE }, + { "crf", "constant rate factor <-1..51> [default: 32]", OFFSET(op_crf), AV_OPT_TYPE_INT, { .i64 = -1 }, -1, 51, VE }, + { "xeve-params", "override the xeve configuration using a : separated list of key=value parameters", OFFSET(xeve_params), AV_OPT_TYPE_DICT, { 0 }, 0, 0, VE }, + { NULL } +}; + +static const AVClass xeve_class = { + .class_name = "libxeve", + .item_name = av_default_item_name, + .option = xeve_options, + .version = LIBAVUTIL_VERSION_INT, +}; + +/** + * libavcodec generic global options, which can be set on all the encoders and decoders + * @see https://www.ffmpeg.org/ffmpeg-codecs.html#Codec-Options + */ +static const FFCodecDefault xeve_defaults[] = { + { "b", "0" }, // bitrate + { "g", "0" }, // gop_size (key-frame interval 0: only one I-frame at the first time; 1: every frame is coded in I-frame) + { "bf", "15"}, // bframes (0: no B-frames) + { "profile", "0"}, // encoder codec profile (0: baselie; 1: main) + { "threads", "0"}, // number of threads to be used (0: automatically select the number of threads to set) + { NULL }, +}; + +const FFCodec ff_libxeve_encoder = { + .p.name = "libxeve", + .p.long_name = NULL_IF_CONFIG_SMALL("libxeve MPEG-5 EVC"), + .p.type = AVMEDIA_TYPE_VIDEO, + .p.id = AV_CODEC_ID_EVC, + .init = libxeve_init, + FF_CODEC_ENCODE_CB(libxeve_encode), + .close = libxeve_close, + .priv_data_size = sizeof(XeveContext), + .p.priv_class = &xeve_class, + .defaults = xeve_defaults, + .p.capabilities = FF_CODEC_CAP_INIT_CLEANUP | AV_CODEC_CAP_DELAY | AV_CODEC_CAP_AUTO_THREADS | AV_CODEC_CAP_DR1, + .p.wrapper_name = "libxeve", + .p.pix_fmts = supported_pixel_formats, +}; diff --git a/libavcodec/parsers.c b/libavcodec/parsers.c index b59388835d..553f7ee23e 100644 --- a/libavcodec/parsers.c +++ b/libavcodec/parsers.c @@ -41,6 +41,7 @@ extern const AVCodecParser ff_dvaudio_parser; extern const AVCodecParser ff_dvbsub_parser; extern const AVCodecParser ff_dvdsub_parser; extern const AVCodecParser ff_dvd_nav_parser; +extern const AVCodecParser ff_evc_parser; extern const AVCodecParser ff_flac_parser; extern const AVCodecParser ff_g723_1_parser; extern const AVCodecParser ff_g729_parser; -- 2.17.1 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [FFmpeg-devel] [PATCH 2/3] Provided support for MPEG-5 EVC (Essential Video Coding) codec 2022-06-22 6:49 ` [FFmpeg-devel] [PATCH 2/3] Provided support for MPEG-5 EVC (Essential Video Coding) codec Dawid Kozinski @ 2022-07-01 9:46 ` Anton Khirnov 0 siblings, 0 replies; 6+ messages in thread From: Anton Khirnov @ 2022-07-01 9:46 UTC (permalink / raw) To: FFmpeg development discussions and patches Quoting Dawid Kozinski (2022-06-22 08:49:04) > - Added xeve encoder wrapper > - Added xevd dencoder wrapper > - Added documentation for xeve and xevd wrappers > - Added parser for EVC format > - Changes in project configuration file and libavcodec Makefile > > Signed-off-by: Dawid Kozinski <d.kozinski@samsung.com> > --- > configure | 8 + > doc/decoders.texi | 24 ++ > doc/encoders.texi | 112 ++++++ > doc/general_contents.texi | 19 + > libavcodec/Makefile | 3 + > libavcodec/allcodecs.c | 2 + > libavcodec/avcodec.h | 3 + > libavcodec/evc_parser.c | 527 ++++++++++++++++++++++++++ > libavcodec/libxevd.c | 440 ++++++++++++++++++++++ > libavcodec/libxeve.c | 753 ++++++++++++++++++++++++++++++++++++++ > libavcodec/parsers.c | 1 + > 11 files changed, 1892 insertions(+) > create mode 100644 libavcodec/evc_parser.c > create mode 100644 libavcodec/libxevd.c > create mode 100644 libavcodec/libxeve.c Numerous violations of our coding style, such as - keywords (if, for, etc.) should be followed by a space before parentheses - opening brace for blocks inside functions should be on the same line as their keyword (if, for, etc.) - useless extra parentheses in if()s I will not point out each instance, please find and fix those. > diff --git a/doc/encoders.texi b/doc/encoders.texi > index 1850c99fe9..245df680d2 100644 > --- a/doc/encoders.texi > +++ b/doc/encoders.texi > @@ -2892,6 +2892,118 @@ ffmpeg -i input -c:v libxavs2 -xavs2-params RdoqLevel=0 output.avs2 > @end example > @end table > > +@section libxeve > + > +eXtra-fast Essential Video Encoder (XEVE) MPEG-5 EVC encoder wrapper. > + > +This encoder requires the presence of the libxeve headers and library > +during configuration. You need to explicitly configure the build with > +@option{--enable-libxeve}. > + > +@float NOTE > +Many libxeve encoder options are mapped to FFmpeg global codec options, > +while unique encoder options are provided through private options. > +Additionally the xeve-params private options allows one to pass a list > +of key=value tuples as accepted by the libxeve @code{parse_xeve_params} function. > +@end float > + > +The xeve project website is at @url{https://github.com/mpeg5/xeve}. > + > +@subsection Options > + > +The following options are supported by the libxeve wrapper. > +The xeve-equivalent options or values are listed in parentheses for easy migration. > + > +@float NOTE > +To reduce the duplication of documentation, only the private options > +and some others requiring special attention are documented here. For > +the documentation of the undocumented generic options, see > +@ref{codec-options,,the Codec Options chapter}. > +@end float > + > +@float NOTE > +To get a more accurate and extensive documentation of the libxeve options, > +invoke the command @code{xeve_app --help} or consult the libxeve documentation. > +@end float > + > +@table @option > +@item b (@emph{bitrate}) > +Set target video bitrate in bits/s. > +Note that FFmpeg’s b option is expressed in bits/s, while xeve’s bitrate is in kilobits/s. Looks broken. > diff --git a/libavcodec/libxeve.c b/libavcodec/libxeve.c > new file mode 100644 > index 0000000000..e115ce63d2 > --- /dev/null > +++ b/libavcodec/libxeve.c > +/** > + * The structure stores all the states associated with the instance of Xeve MPEG-5 EVC encoder > + */ > +typedef struct XeveContext { > + const AVClass *class; > + > + XEVE id; // XEVE instance identifier > + XEVE_CDSC cdsc; // coding parameters i.e profile, width & height of input frame, num of therads, frame rate ... > + XEVE_BITB bitb; // bitstream buffer (output) > + XEVE_STAT stat; // encoding status (output) > + XEVE_IMGB imgb; // image buffer (input) > + > + State state; // encoder state (skipping, encoding, bumping) > + > + // Chroma subsampling > + int width_luma; > + int height_luma; > + int width_chroma; > + int height_chroma; All these are only used in init, no need to store them in the context. > +/** > + * Gets Xeve pre-defined preset > + * > + * @param preset string describing Xeve encoder preset (fast, medium, slow, placebo) > + * @return XEVE pre-defined profile on success, negative value on failure > + */ > +static int get_preset_id(const char *preset) > +{ > + if((!strcmp(preset, "fast"))) > + return XEVE_PRESET_FAST; > + else if (!strcmp(preset, "medium")) > + return XEVE_PRESET_MEDIUM; > + else if (!strcmp(preset, "slow")) > + return XEVE_PRESET_SLOW; > + else if (!strcmp(preset, "placebo")) > + return XEVE_PRESET_PLACEBO; > + else > + return AVERROR(EINVAL); > +} This parsing is unnecessary, change the relevant option into an int and add named constants for it (see e.g. "coder" in libx264.c and many others). Same for tune. > +/** > + * Convert FFmpeg pixel format (AVPixelFormat) into XEVE pre-defined color space > + * > + * @param[in] px_fmt pixel format (@see https://ffmpeg.org/doxygen/trunk/pixfmt_8h.html#a9a8e335cf3be472042bc9f0cf80cd4c5) > + * > + * @return XEVE pre-defined color space (@see xeve.h) on success, XEVE_CF_UNKNOWN on failure > + */ > +static int xeve_color_space(enum AVPixelFormat av_pix_fmt) IIUC the "xeve_" namespace is used by the library, so we should not declare any such names ourselves. Use "libxeve_" or something. > +/** > + * Parse : separated list of key=value parameters > + * > + * @param[in] avctx context for logger > + * @param[in] key > + * @param[in] value > + * @param[out] cdsc contains all Xeve MPEG-5 EVC encoder encoder parameters that > + * should be initialized before the encoder is use > + * > + * @return 0 on success, negative value on failure > + */ > +static int parse_xeve_params(AVCodecContext *avctx, const char *key, const char *value, XEVE_CDSC* cdsc) What is the point of parsing options here instead of declaring them as AVOptions? Especially when vbv-bufsize can already be set using the global "bufsize" option. > + xeve_color_fmt(avctx->pix_fmt, &xectx->color_format); > + > +#if AV_HAVE_BIGENDIAN > + cdsc->param.cs = XEVE_CS_SET(xectx->color_format, cdsc->param.codec_bit_depth, 1); > +#else > + cdsc->param.cs = XEVE_CS_SET(xectx->color_format, cdsc->param.codec_bit_depth, 0); > +#endif just pass AV_HAVE_BIGENDIAN as the last macro parameter, no need for #ifs > +static int set_extra_config(AVCodecContext* avctx, XEVE id, XeveContext *ctx) > +{ > + int ret, size, value; > + size = 4; > + > + // embed SEI messages identifying encoder parameters and command line arguments > + // - 0: off\n" > + // - 1: emit sei info" > + // > + // SEI - Supplemental enhancement information contains information > + // that is not necessary to decode the samples of coded pictures from VCL NAL units. > + // Some SEI message information is required to check bitstream conformance > + // and for output timing decoder conformance. > + // @see ISO_IEC_23094-1_2020 7.4.3.5 > + // @see ISO_IEC_23094-1_2020 Annex D > + ret = xeve_config(id, XEVE_CFG_SET_SEI_CMD, &value, &size); // sei_cmd_info Is this not reading value, which is uninitialized? > +/** > + * Encode raw data frame into EVC packet > + * > + * @param[in] avctx codec context > + * @param[out] pkt output AVPacket containing encoded data > + * @param[in] frame AVFrame containing the raw data to be encoded > + * @param[out] got_packet encoder sets to 0 or 1 to indicate that a > + * non-empty packet was returned in pkt > + * > + * @return 0 on success, negative error code on failure > + */ > +static int libxeve_encode(AVCodecContext *avctx, AVPacket *avpkt, > + const AVFrame *frame, int *got_packet) > +{ > + XeveContext *xectx = avctx->priv_data; > + int ret = -1; > + int xeve_cs; > + > + if(xectx->state == STATE_SKIPPING && frame ) { > + xectx->state = STATE_ENCODING; // Entering encoding process > + } else if(xectx->state == STATE_ENCODING && frame == NULL) { This state switching looks wrong. frame will be NULL only at the end of encoding. It should never happen that libxeve_encode() is called with frame==NULL and then later with frame!=NULL. Also, the logic might be simpler if you used the receive_packet callback rather than encode. > + if (setup_bumping(xectx->id) == 0) > + xectx->state = STATE_BUMPING; // Entering bumping process > + else { > + av_log(avctx, AV_LOG_ERROR, "Failed to setup bumping\n"); > + xectx->state = STATE_SKIPPING; > + } > + } > + > + if(xectx->state == STATE_ENCODING) { > + const AVPixFmtDescriptor *pixel_fmt_desc = av_pix_fmt_desc_get (frame->format); > + if(!pixel_fmt_desc) { > + av_log(avctx, AV_LOG_ERROR, "Invalid pixel format descriptor for pixel format: %s\n", av_get_pix_fmt_name(avctx->pix_fmt)); > + return AVERROR_INVALIDDATA; > + } This seems unused. > + > + xeve_cs = xeve_color_space(avctx->pix_fmt); > + if(xeve_cs != XEVE_CS_YCBCR420 && xeve_cs != XEVE_CS_YCBCR420_10LE) { > + av_log(avctx, AV_LOG_ERROR, "Invalid pixel format: %s\n", av_get_pix_fmt_name(avctx->pix_fmt)); > + return AVERROR_INVALIDDATA; > + } This should be done once during encoder init. Also declaring a list of supported pixel formats in the FFCodec struct guarantees that you will only get one of those formats, so you don't need to check for it yourself. > + > + { > + int i; > + XEVE_IMGB *imgb = NULL; > + > + imgb = &xectx->imgb; > + > + for (i = 0; i < imgb->np; i++) { > + imgb->a[i] = frame->data[i]; > + imgb->s[i] = frame->linesize[i]; > + } How does the ownership semantics work here? Does libxeve make a copy? > + if(xectx->id == NULL) { > + av_log(avctx, AV_LOG_ERROR, "Invalid XEVE encoder\n"); > + return AVERROR_INVALIDDATA; > + } Can this happen? > + > + imgb->ts[0] = frame->pts; > + imgb->ts[1] = 0; Will a proper dts value be generated by the library? -- Anton Khirnov _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 6+ messages in thread
[parent not found: <CGME20220512060536eucas1p262e353ab046a9c672e44aec493ef7d4f@eucas1p2.samsung.com>]
* [FFmpeg-devel] [PATCH 2/3] Provided support for MPEG-5 EVC (Essential Video Coding) codec [not found] <CGME20220512060536eucas1p262e353ab046a9c672e44aec493ef7d4f@eucas1p2.samsung.com> @ 2022-05-12 6:05 ` Dawid Kozinski 2022-06-01 11:41 ` Andreas Rheinhardt 0 siblings, 1 reply; 6+ messages in thread From: Dawid Kozinski @ 2022-05-12 6:05 UTC (permalink / raw) To: ffmpeg-devel - Added xeve encoder wrapper - Added xevd dencoder wrapper - Added documentation for xeve and xevd wrappers - Added parser for EVC format - Changes in project configuration file and libavcodec Makefile Signed-off-by: Dawid Kozinski <d.kozinski@samsung.com> --- configure | 8 + doc/decoders.texi | 24 + doc/encoders.texi | 112 ++++ doc/general_contents.texi | 19 + libavcodec/Makefile | 3 + libavcodec/allcodecs.c | 3 + libavcodec/avcodec.h | 3 + libavcodec/evc_parser.c | 527 +++++++++++++++++ libavcodec/libxevd.c | 444 ++++++++++++++ libavcodec/libxeve.c | 1152 +++++++++++++++++++++++++++++++++++++ libavcodec/parsers.c | 1 + 11 files changed, 2296 insertions(+) create mode 100644 libavcodec/evc_parser.c create mode 100644 libavcodec/libxevd.c create mode 100644 libavcodec/libxeve.c diff --git a/configure b/configure index 4d2f4d9112..85765de2bb 100755 --- a/configure +++ b/configure @@ -291,6 +291,8 @@ External library support: --enable-libwebp enable WebP encoding via libwebp [no] --enable-libx264 enable H.264 encoding via x264 [no] --enable-libx265 enable HEVC encoding via x265 [no] + --enable-libxeve enable MPEG-5 Essential Video Coding (EVC) encoding via libxeve [no] + --enable-libxevd enable MPEG-5 Essential Video Coding (EVC) decoding via libxevd [no] --enable-libxavs enable AVS encoding via xavs [no] --enable-libxavs2 enable AVS2 encoding via xavs2 [no] --enable-libxcb enable X11 grabbing using XCB [autodetect] @@ -1874,6 +1876,8 @@ EXTERNAL_LIBRARY_LIST=" libvorbis libvpx libwebp + libxevd + libxeve libxml2 libzimg libzmq @@ -3384,6 +3388,8 @@ libx264rgb_encoder_select="libx264_encoder" libx265_encoder_deps="libx265" libxavs_encoder_deps="libxavs" libxavs2_encoder_deps="libxavs2" +libxevd_decoder_deps="libxevd" +libxeve_encoder_deps="libxeve" libxvid_encoder_deps="libxvid" libzvbi_teletext_decoder_deps="libzvbi" vapoursynth_demuxer_deps="vapoursynth" @@ -6664,6 +6670,8 @@ enabled libx265 && require_pkg_config libx265 x265 x265.h x265_api_get require_cpp_condition libx265 x265.h "X265_BUILD >= 70" enabled libxavs && require libxavs "stdint.h xavs.h" xavs_encoder_encode "-lxavs $pthreads_extralibs $libm_extralibs" enabled libxavs2 && require_pkg_config libxavs2 "xavs2 >= 1.3.0" "stdint.h xavs2.h" xavs2_api_get +enabled libxevd && require_pkg_config libxevd "xevd >= 1.0.0" "xevd.h" xevd_decode +enabled libxeve && require_pkg_config libxeve "xeve >= 1.0.0" "xeve.h" xeve_encode enabled libxvid && require libxvid xvid.h xvid_global -lxvidcore enabled libzimg && require_pkg_config libzimg "zimg >= 2.7.0" zimg.h zimg_get_api_version enabled libzmq && require_pkg_config libzmq "libzmq >= 4.2.1" zmq.h zmq_ctx_new diff --git a/doc/decoders.texi b/doc/decoders.texi index de2429abba..0222967b2d 100644 --- a/doc/decoders.texi +++ b/doc/decoders.texi @@ -126,6 +126,30 @@ Set amount of frame threads to use during decoding. The default value is 0 (auto @end table +@section libxevd + +eXtra-fast Essential Video Decoder (XEVD) MPEG-5 EVC decoder wrapper. + +This decoder requires the presence of the libxevd headers and library +during configuration. You need to explicitly configure the build with +@option{--enable-libxevd}. + +The xevd project website is at @url{https://github.com/mpeg5/xevd}. + +@subsection Options + +The following options are supported by the libxevd wrapper. +The xevd-equivalent options or values are listed in parentheses for easy migration. + +To get a more accurate and extensive documentation of the libxevd options, +invoke the command @code{xevd_app --help} or consult the libxevd documentation. + +@table @option +@item threads (@emph{threads}) +Force to use a specific number of threads + +@end table + @section QSV Decoders The family of Intel QuickSync Video decoders (VC1, MPEG-2, H.264, HEVC, diff --git a/doc/encoders.texi b/doc/encoders.texi index 966032a720..58e8ceae79 100644 --- a/doc/encoders.texi +++ b/doc/encoders.texi @@ -2843,6 +2843,118 @@ ffmpeg -i input -c:v libxavs2 -xavs2-params RdoqLevel=0 output.avs2 @end example @end table +@section libxeve + +eXtra-fast Essential Video Encoder (XEVE) MPEG-5 EVC encoder wrapper. + +This encoder requires the presence of the libxeve headers and library +during configuration. You need to explicitly configure the build with +@option{--enable-libxeve}. + +@float NOTE +Many libxeve encoder options are mapped to FFmpeg global codec options, +while unique encoder options are provided through private options. +Additionally the xeve-params private options allows one to pass a list +of key=value tuples as accepted by the libxeve @code{parse_xeve_params} function. +@end float + +The xeve project website is at @url{https://github.com/mpeg5/xeve}. + +@subsection Options + +The following options are supported by the libxeve wrapper. +The xeve-equivalent options or values are listed in parentheses for easy migration. + +@float NOTE +To reduce the duplication of documentation, only the private options +and some others requiring special attention are documented here. For +the documentation of the undocumented generic options, see +@ref{codec-options,,the Codec Options chapter}. +@end float + +@float NOTE +To get a more accurate and extensive documentation of the libxeve options, +invoke the command @code{xeve_app --help} or consult the libxeve documentation. +@end float + +@table @option +@item b (@emph{bitrate}) +Set target video bitrate in bits/s. +Note that FFmpeg’s b option is expressed in bits/s, while xeve’s bitrate is in kilobits/s. + +@item bf (@emph{bframes}) +Set the maximum number of B frames (1,3,7,15). + +@item g (@emph{keyint}) +Set the GOP size (I-picture period). + +@item preset (@emph{preset}) +Set the xeve preset. +Set the encoder preset value to determine encoding speed [fast, medium, slow, placebo] + +@item tune (@emph{tune}) +Set the encoder tune parameter [psnr, zerolatency] + +@item profile (@emph{profile}) +Set the encoder profile [0: baselie; 1: main] + +@item crf (@emph{crf}) +Set the quality for constant quality mode. +Constant rate factor <-1..51> [default: 32] + +@item qp (@emph{qp}) +Set constant quantization rate control method parameter. +Quantization parameter qp <0..51> [default: 32] + +@item threads (@emph{threads}) +Force to use a specific number of threads + +@item xeve-params +Set xeve options using a list of @var{key}=@var{value} couples separated +by ":". + +For example to specify libxeve encoding options with @option{-xeve-params}: + +@example +ffmpeg -i input -c:v libxeve -xeve-params profile=main:tune=zerolatency:preset=fast:q=17:bframes=15:bitrate=100k output.mp4 +@end example +@end table + +@subsubsection Supported key values for xeve-params + +Set xeve options using a list of @var{key}=@var{value} couples separated + +@table @option + +@item hash +Embed the picture signature (HASH) for conformance checking in decoding, (0: Off, 1: On) + +@item vbv-bufsize +Set the VBV buffer size: Kbits(none,K,k), Mbits(M,m) + +@item rc-type +Set the rate control type, (0: OFF, 1: ABR, 2: CRF) + +@item bframes +Set the maximum number of B frames (1,3,7,15) + +@item profile +Set the PROFILE (main, baseline) + +@item preset +Set the encoder PRESET (fast, medium, slow, placebo) + +@item tune +Set the encoder TUNE (psnr, zerolatency) + +@item bitrate +Set the Bitrate in terms of kilo-bits per second: Kbps(none,K,k), Mbps(M,m) + +@item qp +Set the QP value (0~51) + +@end table + @section libxvid Xvid MPEG-4 Part 2 encoder wrapper. diff --git a/doc/general_contents.texi b/doc/general_contents.texi index 93a90a5e52..e9e78676ed 100644 --- a/doc/general_contents.texi +++ b/doc/general_contents.texi @@ -343,6 +343,22 @@ libxavs2 is under the GNU Public License Version 2 or later details), you must upgrade FFmpeg's license to GPL in order to use it. @end float +@section eXtra-fast Essential Video Encoder (XEVE) + +FFmpeg can make use of the XEVE library for EVC video encoding. + +Go to @url{https://github.com/mpeg5/xeve} and follow the instructions for +installing the XEVE library. Then pass @code{--enable-libxeve} to configure to +enable it. + +@section eXtra-fast Essential Video Decoder (XEVD) + +FFmpeg can make use of the XEVD library for EVC video decoding. + +Go to @url{https://github.com/mpeg5/xevd} and follow the instructions for +installing the XEVD library. Then pass @code{--enable-libxevd} to configure to +enable it. + @section ZVBI ZVBI is a VBI decoding library which can be used by FFmpeg to decode DVB @@ -591,6 +607,7 @@ library: @item raw DTS @tab X @tab X @item raw DTS-HD @tab @tab X @item raw E-AC-3 @tab X @tab X +@item raw EVC @tab X @tab X @item raw FLAC @tab X @tab X @item raw GSM @tab @tab X @item raw H.261 @tab X @tab X @@ -922,6 +939,8 @@ following image formats are supported: @item Electronic Arts TQI video @tab @tab X @item Escape 124 @tab @tab X @item Escape 130 @tab @tab X +@item EVC / MPEG-5 Part 1 @tab X @tab X + @tab encoding and decoding supported through external libraries libxeve and libxevd @item FFmpeg video codec #1 @tab X @tab X @tab lossless codec (fourcc: FFV1) @item Flash Screen Video v1 @tab X @tab X diff --git a/libavcodec/Makefile b/libavcodec/Makefile index 38425d2f22..8338d9796d 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -1101,6 +1101,8 @@ OBJS-$(CONFIG_LIBX264_ENCODER) += libx264.o OBJS-$(CONFIG_LIBX265_ENCODER) += libx265.o OBJS-$(CONFIG_LIBXAVS_ENCODER) += libxavs.o OBJS-$(CONFIG_LIBXAVS2_ENCODER) += libxavs2.o +OBJS-$(CONFIG_LIBXEVD_DECODER) += libxevd.o +OBJS-$(CONFIG_LIBXEVE_ENCODER) += libxeve.o OBJS-$(CONFIG_LIBXVID_ENCODER) += libxvid.o OBJS-$(CONFIG_LIBZVBI_TELETEXT_DECODER) += libzvbi-teletextdec.o ass.o @@ -1128,6 +1130,7 @@ OBJS-$(CONFIG_DVAUDIO_PARSER) += dvaudio_parser.o OBJS-$(CONFIG_DVBSUB_PARSER) += dvbsub_parser.o OBJS-$(CONFIG_DVD_NAV_PARSER) += dvd_nav_parser.o OBJS-$(CONFIG_DVDSUB_PARSER) += dvdsub_parser.o +OBJS-$(CONFIG_EVC_PARSER) += evc_parser.o OBJS-$(CONFIG_FLAC_PARSER) += flac_parser.o flacdata.o flac.o OBJS-$(CONFIG_G723_1_PARSER) += g723_1_parser.o OBJS-$(CONFIG_G729_PARSER) += g729_parser.o diff --git a/libavcodec/allcodecs.c b/libavcodec/allcodecs.c index c47133aa18..34371af3f4 100644 --- a/libavcodec/allcodecs.c +++ b/libavcodec/allcodecs.c @@ -790,8 +790,11 @@ extern const FFCodec ff_libx262_encoder; #endif extern LIBX264_CONST FFCodec ff_libx264_encoder; #endif + extern const FFCodec ff_libx264rgb_encoder; extern FFCodec ff_libx265_encoder; +extern const FFCodec ff_libxeve_encoder; +extern const FFCodec ff_libxevd_decoder; extern const FFCodec ff_libxavs_encoder; extern const FFCodec ff_libxavs2_encoder; extern const FFCodec ff_libxvid_encoder; diff --git a/libavcodec/avcodec.h b/libavcodec/avcodec.h index 4dae23d06e..b83870c174 100644 --- a/libavcodec/avcodec.h +++ b/libavcodec/avcodec.h @@ -1666,6 +1666,9 @@ typedef struct AVCodecContext { #define FF_PROFILE_KLVA_SYNC 0 #define FF_PROFILE_KLVA_ASYNC 1 +#define FF_PROFILE_EVC_BASELINE 0 +#define FF_PROFILE_EVC_MAIN 1 + /** * level * - encoding: Set by user. diff --git a/libavcodec/evc_parser.c b/libavcodec/evc_parser.c new file mode 100644 index 0000000000..03faf12d62 --- /dev/null +++ b/libavcodec/evc_parser.c @@ -0,0 +1,527 @@ +/* + * EVC AVC format parser + * + * Copyright (C) 2021 Dawid Kozinski <d.kozinski@samsung.com> + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include <stdint.h> + +#include "libavutil/common.h" + +#include "parser.h" +#include "golomb.h" + +// The length field that indicates the length in bytes of the following NAL unit is configured to be of 4 bytes +#define EVC_NAL_UNIT_LENGTH_BYTE (4) /* byte */ + +#define EVC_NAL_HEADER_SIZE (2) /* byte */ +#define MAX_SPS_CNT (16) /* defined value in EVC standard */ + +// NALU types +// @see ISO_IEC_23094-1_2020 7.4.2.2 NAL unit header semantics +// +#define EVC_NUT_NONIDR (0) /* Coded slice of a non-IDR picture */ +#define EVC_NUT_IDR (1) /* Coded slice of an IDR picture */ +#define EVC_NUT_SPS (24) /* Sequence parameter set */ +#define EVC_NUT_PPS (25) /* Picture paremeter set */ +#define EVC_NUT_APS (26) /* Adaptation parameter set */ +#define EVC_NUT_FD (27) /* Filler data */ +#define EVC_NUT_SEI (28) /* Supplemental enhancement information */ + +// The sturcture reflects SPS RBSP(raw byte sequence payload) layout +// @see ISO_IEC_23094-1 section 7.3.2.1 +// +// The following descriptors specify the parsing process of each element +// u(n) - unsigned integer using n bits +// ue(v) - unsigned integer 0-th order Exp_Golomb-coded syntax element with the left bit first +typedef struct EVCParserSPS { + int sps_seq_parameter_set_id; // ue(v) + int profile_idc; // u(8) + int level_idc; // u(8) + int toolset_idc_h; // u(32) + int toolset_idc_l; // u(32) + int chroma_format_idc; // ue(v) + int pic_width_in_luma_samples; // ue(v) + int pic_height_in_luma_samples; // ue(v) + int bit_depth_luma_minus8; // ue(v) + int bit_depth_chroma_minus8; // ue(v) + + int sps_btt_flag; // u(1) + int log2_ctu_size_minus5; // ue(v) + int log2_min_cb_size_minus2; // ue(v) + int log2_diff_ctu_max_14_cb_size; // ue(v) + int log2_diff_ctu_max_tt_cb_size; // ue(v) + int log2_diff_min_cb_min_tt_cb_size_minus2; // ue(v) + + int sps_suco_flag; // u(1) + int log2_diff_ctu_size_max_suco_cb_size; // ue(v) + int log2_diff_max_suco_min_suco_cb_size; // ue(v) + + int sps_admvp_flag; // u(1) + int sps_affine_flag; // u(1) + int sps_amvr_flag; // u(1) + int sps_dmvr_flag; // u(1) + int sps_mmvd_flag; // u(1) + int sps_hmvp_flag; // u(1) + + int sps_eipd_flag; // u(1) + int sps_ibc_flag; // u(1) + int log2_max_ibc_cand_size_minus2; // ue(v) + + int sps_cm_init_flag; // u(1) + int sps_adcc_flag; // u(1) + + int sps_iqt_flag; // u(1) + int sps_ats_flag; // u(1) + + int sps_addb_flag; // u(1) + int sps_alf_flag; // u(1) + int sps_htdf_flag; // u(1) + int sps_rpl_flag; // u(1) + int sps_pocs_flag; // u(1) + int sps_dquant_flag; // u(1) + int sps_dra_flag; // u(1) + + int log2_max_pic_order_cnt_lsb_minus4; // ue(v) + int log2_sub_gop_length; // ue(v) + int log2_ref_pic_gap_length; // ue(v) + + int max_num_tid0_ref_pics; // ue(v) + + int sps_max_dec_pic_buffering_minus1; // ue(v) + int long_term_ref_pic_flag; // u(1) + int rpl1_same_as_rpl0_flag; // u(1) + int num_ref_pic_list_in_sps[2]; // ue(v) + + int picture_cropping_flag; // u(1) + int picture_crop_left_offset; // ue(v) + int picture_crop_right_offset; // ue(v) + int picture_crop_top_offset; // ue(v) + int picture_crop_bottom_offset; // ue(v) + +} EVCParserSPS; + +typedef struct EVCParserContext { + ParseContext pc; + EVCParserSPS sps[MAX_SPS_CNT]; + int is_avc; + int nal_length_size; + int to_read; + int incomplete_nalu_prefix_read; // The flag is set to 1 when an incomplete NAL unit prefix has been read + + int got_sps; + int got_pps; + int got_sei; + int got_slice; +} EVCParserContext; + +static int get_nalu_type(const uint8_t *bits, int bits_size, AVCodecContext *avctx) +{ + int unit_type_plus1 = 0; + + if(bits_size >= EVC_NAL_HEADER_SIZE) { + unsigned char *p = (unsigned char *)bits; + // forbidden_zero_bit + if ((p[0] & 0x80) != 0) { + av_log(avctx, AV_LOG_ERROR, "Cannot get bitstream information. Malformed bitstream.\n"); + return -1; + } + + // nal_unit_type + unit_type_plus1 = (p[0] >> 1) & 0x3F; + } + + return unit_type_plus1 - 1; +} + +static uint32_t read_nal_unit_length(const uint8_t *bits, int bits_size, AVCodecContext *avctx) +{ + uint32_t nalu_len = 0; + + if(bits_size >= EVC_NAL_UNIT_LENGTH_BYTE) { + + int t = 0; + unsigned char *p = (unsigned char *)bits; + + for(int i=0; i<EVC_NAL_UNIT_LENGTH_BYTE; i++) { + t = (t << 8) | p[i]; + } + + nalu_len = t; + if(nalu_len == 0) { + av_log(avctx, AV_LOG_ERROR, "Invalid bitstream size!\n"); + return 0; + } + } + + return nalu_len; +} + +// @see ISO_IEC_23094-1 (7.3.2.1 SPS RBSP syntax) +static EVCParserSPS *parse_sps(const uint8_t *bs, int bs_size, EVCParserContext *ev) +{ + GetBitContext gb; + EVCParserSPS *sps; + int sps_seq_parameter_set_id; + + if(init_get_bits8(&gb, bs, bs_size) < 0) + return NULL; + + sps_seq_parameter_set_id = get_ue_golomb(&gb); + + if(sps_seq_parameter_set_id >= MAX_SPS_CNT) + return NULL; + + sps = &ev->sps[sps_seq_parameter_set_id]; + sps->sps_seq_parameter_set_id = sps_seq_parameter_set_id; + + // the Baseline profile is indicated by profile_idc eqal to 0 + // the Main profile is indicated by profile_idc eqal to 1 + sps->profile_idc = get_bits(&gb, 8); + + sps->level_idc = get_bits(&gb, 8); + + skip_bits_long(&gb, 32); /* skip toolset_idc_h */ + skip_bits_long(&gb, 32); /* skip toolset_idc_l */ + + // 0 - monochrome + // 1 - 4:2:0 + // 2 - 4:2:2 + // 3 - 4:4:4 + sps->chroma_format_idc = get_ue_golomb(&gb); + + sps->pic_width_in_luma_samples = get_ue_golomb(&gb); + sps->pic_height_in_luma_samples = get_ue_golomb(&gb); + + sps->bit_depth_luma_minus8 = get_ue_golomb(&gb); + sps->bit_depth_chroma_minus8 = get_ue_golomb(&gb); + + sps->sps_btt_flag = get_bits(&gb, 1); + if(sps->sps_btt_flag) { + sps->log2_ctu_size_minus5 = get_ue_golomb(&gb); + sps->log2_min_cb_size_minus2 = get_ue_golomb(&gb); + sps->log2_diff_ctu_max_14_cb_size = get_ue_golomb(&gb); + sps->log2_diff_ctu_max_tt_cb_size = get_ue_golomb(&gb); + sps->log2_diff_min_cb_min_tt_cb_size_minus2 = get_ue_golomb(&gb); + } + + sps->sps_suco_flag = get_bits(&gb, 1); + if(sps->sps_suco_flag) { + sps->log2_diff_ctu_size_max_suco_cb_size = get_ue_golomb(&gb); + sps->log2_diff_max_suco_min_suco_cb_size = get_ue_golomb(&gb); + } + + sps->sps_admvp_flag = get_bits(&gb, 1); + if(sps->sps_admvp_flag) { + sps->sps_affine_flag = get_bits(&gb, 1); + sps->sps_amvr_flag = get_bits(&gb, 1); + sps->sps_dmvr_flag = get_bits(&gb, 1); + sps->sps_mmvd_flag = get_bits(&gb, 1); + sps->sps_hmvp_flag = get_bits(&gb, 1); + } + + sps->sps_eipd_flag = get_bits(&gb, 1); + if(sps->sps_eipd_flag) { + sps->sps_ibc_flag = get_bits(&gb, 1); + if(sps->sps_ibc_flag) + sps->log2_max_ibc_cand_size_minus2 = get_ue_golomb(&gb); + } + + sps->sps_cm_init_flag = get_bits(&gb, 1); + if(sps->sps_cm_init_flag) + sps->sps_adcc_flag = get_bits(&gb, 1); + + sps->sps_iqt_flag = get_bits(&gb, 1); + if(sps->sps_iqt_flag) + sps->sps_ats_flag = get_bits(&gb, 1); + + sps->sps_addb_flag = get_bits(&gb, 1); + sps->sps_alf_flag = get_bits(&gb, 1); + sps->sps_htdf_flag = get_bits(&gb, 1); + sps->sps_rpl_flag = get_bits(&gb, 1); + sps->sps_pocs_flag = get_bits(&gb, 1); + sps->sps_dquant_flag = get_bits(&gb, 1); + sps->sps_dra_flag = get_bits(&gb, 1); + + if(sps->sps_pocs_flag) + sps->log2_max_pic_order_cnt_lsb_minus4 = get_ue_golomb(&gb); + + if(!sps->sps_pocs_flag || !sps->sps_rpl_flag) { + sps->log2_sub_gop_length = get_ue_golomb(&gb); + if(sps->log2_sub_gop_length == 0) + sps->log2_ref_pic_gap_length = get_ue_golomb(&gb); + } + + return sps; +} + +static int parse_nal_units(AVCodecParserContext *s, const uint8_t *bs, + int bs_size, AVCodecContext *avctx) +{ + EVCParserContext *ev = s->priv_data; + int nalu_type, nalu_size; + unsigned char *bits = (unsigned char *)bs; + int bits_size = bs_size; + + avctx->codec_id = AV_CODEC_ID_EVC; + + nalu_size = read_nal_unit_length(bits, bits_size, avctx); + if(nalu_size == 0) { + av_log(avctx, AV_LOG_ERROR, "Invalid NAL unit size: (%d)\n", nalu_size); + return -1; + } + + bits += EVC_NAL_UNIT_LENGTH_BYTE; + bits_size -= EVC_NAL_UNIT_LENGTH_BYTE; + + nalu_type = get_nalu_type(bits, bits_size, avctx); + + bits += EVC_NAL_HEADER_SIZE; + bits_size -= EVC_NAL_HEADER_SIZE; + + if (nalu_type == EVC_NUT_SPS) { // NAL Unit type: SPS (Sequence Parameter Set) + EVCParserSPS *sps; + + sps = parse_sps(bits, bits_size, ev); + + avctx->coded_width = sps->pic_width_in_luma_samples; + avctx->coded_height = sps->pic_height_in_luma_samples; + avctx->width = sps->pic_width_in_luma_samples; + avctx->height = sps->pic_height_in_luma_samples; + + if(sps->profile_idc == 0) avctx->profile = FF_PROFILE_EVC_BASELINE; + else if (sps->profile_idc == 1) avctx->profile = FF_PROFILE_EVC_MAIN; + else { + av_log(avctx, AV_LOG_ERROR, "Not supported profile (%d)\n", sps->profile_idc); + return -1; + } + + // Currently XEVD decoder supports ony YCBCR420_10LE chroma format for EVC stream + switch(sps->chroma_format_idc) { + case 0: /* YCBCR400_10LE */ + av_log(avctx, AV_LOG_ERROR, "YCBCR400_10LE: Not supported chroma format\n"); + avctx->pix_fmt = AV_PIX_FMT_GRAY10LE; + return -1; + case 1: /* YCBCR420_10LE */ + avctx->pix_fmt = AV_PIX_FMT_YUV420P10LE; + break; + case 2: /* YCBCR422_10LE */ + av_log(avctx, AV_LOG_ERROR, "YCBCR422_10LE: Not supported chroma format\n"); + avctx->pix_fmt = AV_PIX_FMT_YUV422P10LE; + return -1; + case 3: /* YCBCR444_10LE */ + av_log(avctx, AV_LOG_ERROR, "YCBCR444_10LE: Not supported chroma format\n"); + avctx->pix_fmt = AV_PIX_FMT_YUV444P10LE; + return -1; + default: + avctx->pix_fmt = AV_PIX_FMT_NONE; + av_log(avctx, AV_LOG_ERROR, "Unknown supported chroma format\n"); + return -1; + } + + // The current implementation of parse_sps function doesn't handle VUI parameters parsing, + // so at the moment it's impossible to initialize has_b_frames and max_b_frames AVCodecContex fields here. + // Currently, initialization of has_b_frames and max_b_frames AVCodecContex fields have been moved to + // libxevd_decode function where we can use xevd_config function being a part of xevd library API + // to get the needed information. + // However, if it will be needed, parse_sps function should be extended to handle VUI parameters parsing + // and the following lines should be used to initialize has_b_frames and max_b_frames fields of the AVCodecContex. + // + // sps->vui_parameters.num_reorder_pics + // if (sps->bitstream_restriction_flag && sps->vui_parameters.num_reorder_pics) { + // avctx->has_b_frames = sps->vui_parameters.num_reorder_pics; + // } + + ev->got_sps = 1; + + } else if (nalu_type == EVC_NUT_PPS) // NAL Unit type: PPS (Video Parameter Set) + ev->got_pps = 1; + else if(nalu_type == EVC_NUT_SEI) // NAL unit type: SEI (Supplemental Enhancement Information) + ev->got_sei = 1; + else if (nalu_type == EVC_NUT_IDR || nalu_type == EVC_NUT_NONIDR) // NAL Unit type: Coded slice of a IDR or non-IDR picture + ev->got_slice++; + else { + av_log(avctx, AV_LOG_ERROR, "Invalid NAL unit type: %d\n", nalu_type); + return -1; + } + + return 0; +} + +/** + * Find the end of the current frame in the bitstream. + * @return the position of the first byte of the next frame, or END_NOT_FOUND + */ +static int evc_find_frame_end(AVCodecParserContext *s, const uint8_t *buf, + int buf_size, AVCodecContext *avctx) +{ + EVCParserContext *ev = s->priv_data; + + if(!ev->to_read) { + int nal_unit_size = 0; + int next = END_NOT_FOUND; + + // This is the case when buffer size is not enough for buffer to store NAL unit length + if(buf_size < EVC_NAL_UNIT_LENGTH_BYTE) { + ev->to_read = EVC_NAL_UNIT_LENGTH_BYTE; + ev->nal_length_size = buf_size; + ev->incomplete_nalu_prefix_read = 1; + + return END_NOT_FOUND; + } + + nal_unit_size = read_nal_unit_length(buf, buf_size, avctx); + ev->nal_length_size = EVC_NAL_UNIT_LENGTH_BYTE; + + next = nal_unit_size + EVC_NAL_UNIT_LENGTH_BYTE; + ev->to_read = next; + if(next < buf_size) + return next; + else + return END_NOT_FOUND; + } else if(ev->to_read > buf_size) { + return END_NOT_FOUND; + } else { + if(ev->incomplete_nalu_prefix_read == 1) { + EVCParserContext *ev = s->priv_data; + ParseContext *pc = &ev->pc; + uint8_t nalu_len[EVC_NAL_UNIT_LENGTH_BYTE] = {0}; + int nal_unit_size = 0; + + // 1. pc->buffer contains previously read bytes of NALU prefix + // 2. buf contains the rest of NAL unit prefix bytes + // + // ~~~~~~~ + // EXAMPLE + // ~~~~~~~ + // + // In the following example we assumed that the number of already read NAL Unit prefix bytes is equal 1 + // + // ---------- + // pc->buffer -> conatins already read bytes + // ---------- + // __ pc->index == 1 + // | + // V + // ------------------------------------------------------- + // | 0 | 1 | 2 | 3 | 4 | ... | N | + // ------------------------------------------------------- + // | 0x00 | 0xXX | 0xXX | 0xXX | 0xXX | ... | 0xXX | + // ------------------------------------------------------- + // + // ---------- + // buf -> contains newly read bytes + // ---------- + // ------------------------------------------------------- + // | 0 | 1 | 2 | 3 | 4 | ... | N | + // ------------------------------------------------------- + // | 0x00 | 0x00 | 0x3C | 0xXX | 0xXX | ... | 0xXX | + // ------------------------------------------------------- + // + for(int i = 0; i < EVC_NAL_UNIT_LENGTH_BYTE; i++) { + if(i < pc->index) + nalu_len[i] = pc->buffer[i]; + else + nalu_len[i] = buf[i - pc->index]; + } + + // ---------- + // nalu_len + // ---------- + // --------------------------------- + // | 0 | 1 | 2 | 3 | + // --------------------------------- + // | 0x00 | 0x00 | 0x00 | 0x3C | + // --------------------------------- + // | NALU LENGTH | + // --------------------------------- + // NAL Unit lenght = 60 (0x0000003C) + + nal_unit_size = read_nal_unit_length(nalu_len, EVC_NAL_UNIT_LENGTH_BYTE, avctx); + + ev->to_read = nal_unit_size + EVC_NAL_UNIT_LENGTH_BYTE - pc->index; + + ev->incomplete_nalu_prefix_read = 0; + + if(ev->to_read > buf_size) + return END_NOT_FOUND; + else + return ev->to_read; + } + return ev->to_read; + } + + return END_NOT_FOUND; +} + +static int evc_parser_init(AVCodecParserContext *s) +{ + EVCParserContext *ev = s->priv_data; + + ev->got_sps = 0; + ev->got_pps = 0; + ev->got_sei = 0; + ev->got_slice = 0; + ev->nal_length_size = EVC_NAL_UNIT_LENGTH_BYTE; + ev->incomplete_nalu_prefix_read = 0; + + return 0; +} + +static int evc_parse(AVCodecParserContext *s, AVCodecContext *avctx, + const uint8_t **poutbuf, int *poutbuf_size, + const uint8_t *buf, int buf_size) +{ + int next; + EVCParserContext *ev = s->priv_data; + ParseContext *pc = &ev->pc; + int is_dummy_buf = !buf_size; + const uint8_t *dummy_buf = buf; + + if (s->flags & PARSER_FLAG_COMPLETE_FRAMES) + next = buf_size; + else { + next = evc_find_frame_end(s, buf, buf_size, avctx); + if (ff_combine_frame(pc, next, &buf, &buf_size) < 0) { + *poutbuf = NULL; + *poutbuf_size = 0; + ev->to_read -= buf_size; + return buf_size; + } + } + + is_dummy_buf &= (dummy_buf == buf); + + if (!is_dummy_buf) + parse_nal_units(s, buf, buf_size, avctx); + + *poutbuf = buf; + *poutbuf_size = buf_size; + ev->to_read -= next; + + return next; +} + +const AVCodecParser ff_evc_parser = { + .codec_ids = { AV_CODEC_ID_EVC }, + .priv_data_size = sizeof(EVCParserContext), + .parser_init = evc_parser_init, + .parser_parse = evc_parse, + .parser_close = ff_parse_close, +}; diff --git a/libavcodec/libxevd.c b/libavcodec/libxevd.c new file mode 100644 index 0000000000..512d1263d5 --- /dev/null +++ b/libavcodec/libxevd.c @@ -0,0 +1,444 @@ +/* + * libxevd decoder + * EVC (MPEG-5 Essential Video Coding) decoding using XEVD MPEG-5 EVC decoder library + * + * Copyright (C) 2021 Dawid Kozinski <d.kozinski@samsung.com> + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#if defined(_MSC_VER) +#define XEVD_API_IMPORTS 1 +#endif + +#include <float.h> +#include <stdlib.h> + +#include <xevd.h> + +#include "libavutil/internal.h" +#include "libavutil/common.h" +#include "libavutil/opt.h" +#include "libavutil/pixdesc.h" +#include "libavutil/pixfmt.h" +#include "libavutil/imgutils.h" +#include "libavutil/cpu.h" + +#include "avcodec.h" +#include "internal.h" +#include "packet_internal.h" +#include "codec_internal.h" + +#define XEVD_PARAM_BAD_NAME -1 +#define XEVD_PARAM_BAD_VALUE -2 + +#define EVC_NAL_HEADER_SIZE 2 /* byte */ + +/** + * The structure stores all the states associated with the instance of Xeve MPEG-5 EVC decoder + */ +typedef struct XevdContext { + const AVClass *class; + + XEVD id; // XEVD instance identifier @see xevd.h + XEVD_CDSC cdsc; // decoding parameters @see xevd.h + + int decod_frames; // number of decoded frames + int packet_count; // number of packets created by decoder + + // configuration parameters + AVDictionary *xevd_opts; // xevd configuration read from a : separated list of key=value parameters + +} XevdContext; + +/** + * The function returns a pointer to a variable of type XEVD_CDSC. + * XEVD_CDSC contains all decoder parameters that should be initialized before its use. + * + * The field values of the XEVD_CDSC structure are populated based on: + * - the corresponding field values of the AvCodecConetxt structure, + * - the xevd decoder specific option values, + * (the full list of options available for the xevd encoder is displayed after executing the command ./ffmpeg --help decoder = libxevd) + * - and the xevd encoder options specified as a list of key-value pairs following the xevd-params option + * + * Order of input processing and populating the XEVD_CDSC structure + * 1. first, the corresponding fields of the AVCodecContext structure are processed, (i.e -threads 4) + * 2. then xevd-specific options are added as AVOption to the xevd AVCodec implementation (i.e -threads 3) + * 3. finally, the options specified after the xevd-params option as the parameter list of type key value are processed (i.e -xevd-params "m=2") + * + * Some options can be set in different ways. In this case, please follow the above-mentioned order of processing. + * The most recent assignments overwrite the previous values. + * + * @param[in] avctx codec context + * @param[out] cdsc contains all encoder parameters that should be initialized before its use + * + * @return 0 on success, negative error code on failure + */ +static int get_conf(AVCodecContext *avctx, XEVD_CDSC *cdsc) +{ + int cpu_count = av_cpu_count(); + + /* clear XEVS_CDSC structure */ + memset(cdsc, 0, sizeof(XEVD_CDSC)); + + /* init XEVD_CDSC */ + if(avctx->thread_count <= 0) + cdsc->threads = (cpu_count < XEVD_MAX_TASK_CNT) ? cpu_count : XEVD_MAX_TASK_CNT; + else if(avctx->thread_count > XEVD_MAX_TASK_CNT) + cdsc->threads = XEVD_MAX_TASK_CNT; + else + cdsc->threads = avctx->thread_count; + + return 0; +} + +/** + * Read NAL unit length + * @param bs input data (bitstream) + * @return the length of NAL unit on success, 0 value on failure + */ +static uint32_t read_nal_unit_length(const uint8_t *bs, int bs_size, AVCodecContext *avctx) +{ + uint32_t len = 0; + XEVD_INFO info; + int ret; + + if(bs_size == XEVD_NAL_UNIT_LENGTH_BYTE) { + ret = xevd_info((void *)bs, XEVD_NAL_UNIT_LENGTH_BYTE, 1, &info); + if (XEVD_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Cannot get bitstream information\n"); + return 0; + } + len = info.nalu_len; + if(len == 0) { + av_log(avctx, AV_LOG_ERROR, "Invalid bitstream size! [%d]\n", bs_size); + return 0; + } + } + + return len; +} + +/** + * @param[in] xectx the structure that stores all the state associated with the instance of Xeve MPEG-5 EVC decoder + * @param[out] avctx codec context + * @return 0 on success, negative value on failure + */ +static int export_stream_params(const XevdContext *xectx, AVCodecContext *avctx) +{ + int ret; + int size; + int color_space; + + avctx->pix_fmt = AV_PIX_FMT_YUV420P10; + + size = 4; + ret = xevd_config(xectx->id, XEVD_CFG_GET_CODED_WIDTH, &avctx->coded_width, &size); + if (XEVD_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to get coded_width\n"); + return AVERROR_EXTERNAL; + } + + ret = xevd_config(xectx->id, XEVD_CFG_GET_CODED_HEIGHT, &avctx->coded_height, &size); + if (XEVD_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to get coded_height\n"); + return AVERROR_EXTERNAL; + } + + ret = xevd_config(xectx->id, XEVD_CFG_GET_WIDTH, &avctx->width, &size); + if (XEVD_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to get width\n"); + return AVERROR_EXTERNAL; + } + + ret = xevd_config(xectx->id, XEVD_CFG_GET_HEIGHT, &avctx->height, &size); + if (XEVD_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to get height\n"); + return AVERROR_EXTERNAL; + } + + ret = xevd_config(xectx->id, XEVD_CFG_GET_COLOR_SPACE, &color_space, &size); + if (XEVD_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to get color_space\n"); + return AVERROR_EXTERNAL; + } + switch(color_space) { + case XEVD_CS_YCBCR400_10LE: + avctx->pix_fmt = AV_PIX_FMT_GRAY10LE; + break; + case XEVD_CS_YCBCR420_10LE: + avctx->pix_fmt = AV_PIX_FMT_YUV420P10LE; + break; + case XEVD_CS_YCBCR422_10LE: + avctx->pix_fmt = AV_PIX_FMT_YUV422P10LE; + break; + case XEVD_CS_YCBCR444_10LE: + avctx->pix_fmt = AV_PIX_FMT_YUV444P10LE; + break; + default: + av_log(avctx, AV_LOG_ERROR, "Unknown color space\n"); + avctx->pix_fmt = AV_PIX_FMT_NONE; + return AVERROR_INVALIDDATA; + } + + // the function returns sps->num_reorder_pics + ret = xevd_config(xectx->id, XEVD_CFG_GET_MAX_CODING_DELAY, &avctx->max_b_frames, &size); + if (XEVD_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to get max_coding_delay\n"); + return AVERROR_EXTERNAL; + } + + avctx->has_b_frames = (avctx->max_b_frames) ? 1 : 0; + + avctx->color_primaries = AVCOL_PRI_UNSPECIFIED; + avctx->color_trc = AVCOL_TRC_UNSPECIFIED; + avctx->colorspace = AVCOL_SPC_UNSPECIFIED; + + return 0; +} + +/** + * Initialize decoder + * Create a decoder instance and allocate all the needed resources + * + * @param avctx codec context + * @return 0 on success, negative error code on failure + */ +static av_cold int libxevd_init(AVCodecContext *avctx) +{ + XevdContext *xectx = avctx->priv_data; + int ret = 0; + XEVD_CDSC *cdsc = &(xectx->cdsc); + + /* read configurations and set values for created descriptor (XEVD_CDSC) */ + ret = get_conf(avctx, cdsc); + if (ret != 0) { + av_log(avctx, AV_LOG_ERROR, "Cannot get configuration\n"); + return AVERROR_INVALIDDATA; + } + + /* create decoder */ + xectx->id = xevd_create(&(xectx->cdsc), NULL); + if(xectx->id == NULL) { + av_log(avctx, AV_LOG_ERROR, "Cannot create XEVD encoder\n"); + return AVERROR_EXTERNAL; + } + + xectx->packet_count = 0; + xectx->decod_frames = 0; + + return 0; +} + +/** + * Decode picture + * + * @param avctx codec context + * @param[out] frame decoded frame + * @param[out] got_frame_ptr decoder sets to 0 or 1 to indicate that a + * non-empty frame or subtitle was returned in + * outdata. + * @param[in] avpkt AVPacket containing encoded data to be decoded + * + * @return amount of bytes read from the packet on success, negative error + * code on failure + */ + static int libxevd_decode(struct AVCodecContext *avctx, struct AVFrame *frame, int *got_frame_ptr, AVPacket *avpkt) +{ + XevdContext *xectx = NULL; + XEVD_IMGB *imgb = NULL; + XEVD_STAT stat; + XEVD_BITB bitb; + int xevd_ret, nalu_size, bs_read_pos; + int ret = 0; + + if(avctx == NULL) { + av_log(avctx, AV_LOG_ERROR, "Invalid input parameter: AVCodecContext\n"); + return AVERROR(EINVAL); + } + xectx = avctx->priv_data; + if(xectx == NULL) { + av_log(avctx, AV_LOG_ERROR, "Invalid XEVD context\n"); + return AVERROR(EINVAL); + } + + if(avpkt->size > 0) { + bs_read_pos = 0; + imgb = NULL; + while(avpkt->size > (bs_read_pos + XEVD_NAL_UNIT_LENGTH_BYTE)) { + + memset(&stat, 0, sizeof(XEVD_STAT)); + memset(&bitb, 0, sizeof(XEVD_BITB)); + + nalu_size = read_nal_unit_length(avpkt->data + bs_read_pos, XEVD_NAL_UNIT_LENGTH_BYTE, avctx); + if(nalu_size == 0) { + av_log(avctx, AV_LOG_ERROR, "Invalid bitstream\n"); + ret = AVERROR_INVALIDDATA; + goto ERR; + } + bs_read_pos += XEVD_NAL_UNIT_LENGTH_BYTE; + + bitb.addr = avpkt->data + bs_read_pos; + bitb.ssize = nalu_size; + + /* main decoding block */ + xevd_ret = xevd_decode(xectx->id, &bitb, &stat); + if(XEVD_FAILED(xevd_ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to decode bitstream\n"); + ret = AVERROR_EXTERNAL; + goto ERR; + } + + bs_read_pos += nalu_size; + + if(stat.nalu_type == XEVD_NUT_SPS) { // EVC stream parameters changed + if((ret = export_stream_params(xectx, avctx)) != 0) + goto ERR; + } + + if(stat.read != nalu_size) + av_log(avctx, AV_LOG_INFO, "Different reading of bitstream (in:%d, read:%d)\n,", nalu_size, stat.read); + if(stat.fnum >= 0) { + // already has a decoded image + if (imgb) { + // xevd_pull uses pool of objects of type XEVD_IMGB. + // The pool size is equal MAX_PB_SIZE (26), so release object when it is no more needed + imgb->release(imgb); + imgb = NULL; + } + xevd_ret = xevd_pull(xectx->id, &imgb); + if(XEVD_FAILED(xevd_ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to pull the decoded image (xevd error code: %d, frame#=%d)\n", xevd_ret, stat.fnum); + ret = AVERROR_EXTERNAL; + goto ERR; + } else if (xevd_ret == XEVD_OK_FRM_DELAYED) { + if(imgb) { + // xevd_pull uses pool of objects of type XEVD_IMGB. + // The pool size is equal MAX_PB_SIZE (26), so release object when it is no more needed + imgb->release(imgb); + imgb = NULL; + } + } + } + } + } else { // bumping + xevd_ret = xevd_pull(xectx->id, &(imgb)); + if(xevd_ret == XEVD_ERR_UNEXPECTED) { // bumping process completed + *got_frame_ptr = 0; + return 0; + } else if(XEVD_FAILED(xevd_ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to pull the decoded image (xevd error code: %d)\n", xevd_ret); + ret = AVERROR_EXTERNAL; + goto ERR; + } + } + + if(imgb) { + // @todo supports other color space and bit depth + if(imgb->cs != XEVD_CS_YCBCR420_10LE) { + av_log(avctx, AV_LOG_ERROR, "Not supported pixel format: %s\n", av_get_pix_fmt_name(avctx->pix_fmt)); + ret = AVERROR_INVALIDDATA; + goto ERR; + } + + if (imgb->w[0] != avctx->width || imgb->h[0] != avctx->height) { // stream resolution changed + if(ff_set_dimensions(avctx, imgb->w[0], imgb->h[0]) < 0) { + av_log(avctx, AV_LOG_ERROR, "Cannot set new dimension\n"); + ret = AVERROR_INVALIDDATA; + goto ERR; + } + } + + frame->coded_picture_number++; + frame->display_picture_number++; + frame->format = AV_PIX_FMT_YUV420P10LE; + + if (ff_get_buffer(avctx, frame, 0) < 0) { + av_log(avctx, AV_LOG_ERROR, "Cannot get AV buffer\n"); + ret = AVERROR(ENOMEM); + goto ERR; + } + + frame->pts = avpkt->pts; + + av_image_copy(frame->data, frame->linesize, (const uint8_t **)imgb->a, + imgb->s, avctx->pix_fmt, + imgb->w[0], imgb->h[0]); + + xectx->decod_frames++; + *got_frame_ptr = 1; + + // xevd_pull uses pool of objects of type XEVD_IMGB. + // The pool size is equal MAX_PB_SIZE (26), so release object when it is no more needed + imgb->release(imgb); + imgb = NULL; + } else + *got_frame_ptr = 0; + + xectx->packet_count++; + + return avpkt->size; + +ERR: + if(imgb) { + imgb->release(imgb); + imgb = NULL; + } + *got_frame_ptr = 0; + + return ret; +} + +/** + * Destroy decoder + * + * @param avctx codec context + * @return 0 on success + */ +static av_cold int libxevd_close(AVCodecContext *avctx) +{ + XevdContext *xectx = avctx->priv_data; + if(xectx->id) { + xevd_delete(xectx->id); + xectx->id = NULL; + } + + return 0; +} + +#define OFFSET(x) offsetof(XevdContext, x) +#define VD AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_DECODING_PARAM + +static const AVClass xevd_class = { + .class_name = "libxevd", + .item_name = av_default_item_name, + .version = LIBAVUTIL_VERSION_INT, +}; + +const FFCodec ff_libxevd_decoder = { + .p.name = "evc", + .p.long_name = NULL_IF_CONFIG_SMALL("EVC / MPEG-5 Essential Video Coding (EVC)"), + .p.type = AVMEDIA_TYPE_VIDEO, + .p.id = AV_CODEC_ID_EVC, + .init = libxevd_init, + FF_CODEC_DECODE_CB(libxevd_decode), + .close = libxevd_close, + .priv_data_size = sizeof(XevdContext), + .p.priv_class = &xevd_class, + .p.capabilities = AV_CODEC_CAP_DELAY | AV_CODEC_CAP_AUTO_THREADS | AV_CODEC_CAP_AVOID_PROBING | AV_CODEC_CAP_DR1, + .p.wrapper_name = "libxevd", +}; diff --git a/libavcodec/libxeve.c b/libavcodec/libxeve.c new file mode 100644 index 0000000000..937b7fc5ce --- /dev/null +++ b/libavcodec/libxeve.c @@ -0,0 +1,1152 @@ +/* + * libxeve encoder + * EVC (MPEG-5 Essential Video Coding) encoding using XEVE MPEG-5 EVC encoder library + * + * Copyright (C) 2021 Dawid Kozinski <d.kozinski@samsung.com> + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#if defined(_MSC_VER) +#define XEVE_API_IMPORTS 1 +#endif + +#include <float.h> +#include <stdlib.h> + +#include <xeve.h> + +#include "libavutil/internal.h" +#include "libavutil/common.h" +#include "libavutil/opt.h" +#include "libavutil/pixdesc.h" +#include "libavutil/pixfmt.h" +#include "libavutil/time.h" +#include "libavutil/cpu.h" + +#include "avcodec.h" +#include "internal.h" +#include "packet_internal.h" +#include "codec_internal.h" +#include "encode.h" + +#define MAX_BS_BUF (16*1024*1024) + +/** + * Error codes + */ +#define XEVE_PARAM_BAD_NAME -100 +#define XEVE_PARAM_BAD_VALUE -200 + +/** + * Encoder states + * + * STATE_ENCODING - the encoder receives and processes input frames + * STATE_BUMPING - there are no more input frames, however the encoder still processes previously received data + * STATE_SKIPPING - skipping input frames + */ +typedef enum State { + STATE_ENCODING, + STATE_BUMPING, + STATE_SKIPPING +} State; + +/** + * The structure stores all the states associated with the instance of Xeve MPEG-5 EVC encoder + */ +typedef struct XeveContext { + const AVClass *class; + + XEVE id; // XEVE instance identifier + XEVE_CDSC cdsc; // coding parameters i.e profile, width & height of input frame, num of therads, frame rate ... + XEVE_BITB bitb; // bitstream buffer (output) + XEVE_STAT stat; // encoding status (output) + XEVE_IMGB imgb; // image buffer (input) + + State state; // encoder state (skipping, encoding, bumping) + + int encod_frames; // num of encoded frames + double bytes_total; // encoded bitstream byte size + double bitrate; // bits per second + int packet_count; // num of packets created by encoder + + // Chroma subsampling + int width_luma; + int height_luma; + int width_chroma; + int height_chroma; + + int profile_id; // encoder profile (main, baseline) + int preset_id; // preset of xeve ( fast, medium, slow, placebo) + int tune_id; // tune of xeve (psnr, zerolatency) + int input_depth; // input bit-depth: 8bit, 10bit + int hash; // embed picture signature (HASH) for conformance checking in decoding + + /* variables for input parameter */ + char *op_preset; + char *op_tune; + int op_qp; + int op_crf; + + // configuration parameters + // xeve configuration read from a : separated list of key=value parameters + AVDictionary *xeve_params; +} XeveContext; + +/** + * Gets Xeve encoder pre-defined profile + * + * @param profile string describing Xeve encoder profile (baseline, main) + * @return XEVE pre-defined profile on success, negative value on failure + */ +static int get_profile_id(const char *profile) +{ + if (!strcmp(profile, "baseline")) + return XEVE_PROFILE_BASELINE; + else if (!strcmp(profile, "main")) + return XEVE_PROFILE_MAIN; + else + return AVERROR(EINVAL); +} + +/** + * Gets Xeve pre-defined preset + * + * @param preset string describing Xeve encoder preset (fast, medium, slow, placebo) + * @return XEVE pre-defined profile on success, negative value on failure + */ +static int get_preset_id(const char *preset) +{ + if((!strcmp(preset, "fast"))) + return XEVE_PRESET_FAST; + else if (!strcmp(preset, "medium")) + return XEVE_PRESET_MEDIUM; + else if (!strcmp(preset, "slow")) + return XEVE_PRESET_SLOW; + else if (!strcmp(preset, "placebo")) + return XEVE_PRESET_PLACEBO; + else + return AVERROR(EINVAL); +} + +/** + * Gets Xeve pre-defined tune id + * + * @param[in] tune string describing Xeve encoder tune (psnr, zerolatency) + * @return XEVE pre-defined profile on success, negative value on failure + */ +static int get_tune_id(const char *tune) +{ + if((!strcmp(tune, "psnr"))) + return XEVE_TUNE_PSNR; + else if (!strcmp(tune, "zerolatency")) + return XEVE_TUNE_ZEROLATENCY; + else + return AVERROR(EINVAL); +} + +/** + * Convert FFmpeg pixel format (AVPixelFormat) to XEVE pre-defined color format + * + * @param[in] px_fmt pixel format (@see https://ffmpeg.org/doxygen/trunk/pixfmt_8h.html#a9a8e335cf3be472042bc9f0cf80cd4c5) + * @param[out] color_format XEVE pre-defined color format (@see xeve.h) + * @param[out] bit_depth bit depth + * + * @return 0 on success, negative value on failure + */ +static int get_pix_fmt(enum AVPixelFormat pix_fmt, int *color_format, int *bit_depth) +{ + switch (pix_fmt) { + case AV_PIX_FMT_YUV420P: + *color_format = XEVE_CF_YCBCR420; + *bit_depth = 8; + break; + case AV_PIX_FMT_YUV422P: + *color_format = XEVE_CF_YCBCR422; + *bit_depth = 8; + break; + case AV_PIX_FMT_YUV444P: + *color_format = XEVE_CF_YCBCR444; + *bit_depth = 8; + break; + case AV_PIX_FMT_YUV420P10: + *color_format = XEVE_CF_YCBCR420; + *bit_depth = 10; + break; + case AV_PIX_FMT_YUV422P10: + *color_format = XEVE_CF_YCBCR422; + *bit_depth = 10; + break; + case AV_PIX_FMT_YUV444P10: + *color_format = XEVE_CF_YCBCR444; + *bit_depth = 10; + break; + default: + *color_format = XEVE_CF_UNKNOWN; + return AVERROR_INVALIDDATA; + } + + return 0; +} + +static int kbps_str_to_int(char *str) +{ + int kbps = 0; + if (strchr(str, 'K') || strchr(str, 'k')) { + char *tmp = strtok(str, "Kk "); + kbps = (int)(atof(tmp)); + } else if (strchr(str, 'M') || strchr(str, 'm')) { + char *tmp = strtok(str, "Mm "); + kbps = (int)(atof(tmp) * 1000); + } else + kbps = atoi(str); + + return kbps; +} + +/** + * Parse : separated list of key=value parameters + * + * @param[in] avctx context for logger + * @param[in] key + * @param[in] value + * @param[out] cdsc contains all Xeve MPEG-5 EVC encoder encoder parameters that + * should be initialized before the encoder is use + * + * @return 0 on success, negative value on failure + */ +static int parse_xeve_params(AVCodecContext *avctx, const char *key, const char *value, XEVE_CDSC* cdsc) +{ + XeveContext *xectx = NULL; + xectx = avctx->priv_data; + + if(!key) { + av_log(avctx, AV_LOG_ERROR, "Ivalid argument: key string is NULL\n"); + return XEVE_PARAM_BAD_VALUE; + } + if(!value) { + if (strcmp(key, "hash") == 0) { + xectx->hash = 1; + av_log(avctx, AV_LOG_INFO, "embedding signature is enabled\n"); + } else { + av_log(avctx, AV_LOG_ERROR, "Ivalid argument: value string is NULL\n"); + return XEVE_PARAM_BAD_VALUE; + } + } else if (strcmp(key, "vbv-bufsize") == 0 ) { + cdsc->param.vbv_bufsize = kbps_str_to_int((char *)value); + av_log(avctx, AV_LOG_INFO, "VBV buffer size: %dkbits\n", cdsc->param.vbv_bufsize); + } else if (strcmp(key, "rc-type") == 0 ) { + int rc_type = atoi(value); + if(rc_type < 0 || rc_type > 2) { + av_log(avctx, AV_LOG_ERROR, "Rate control type [ 0(rc_off) / 1(CBR) ] bad value: %d\n", rc_type); + return XEVE_PARAM_BAD_VALUE; + } + cdsc->param.rc_type = rc_type; + av_log(avctx, AV_LOG_INFO, "Rate control type [ 0(rc_off) / 1(CBR) ] : %d\n", rc_type); + } else if (strcmp(key, "bframes") == 0 ) { + int bframes = atoi(value); + if(bframes < 0) { + av_log(avctx, AV_LOG_ERROR, "bframes: bad value: %d\n", bframes); + return XEVE_PARAM_BAD_VALUE; + } + cdsc->param.bframes = bframes; + av_log(avctx, AV_LOG_INFO, "bframes : %d\n", bframes); + } else if (strcmp(key, "profile") == 0 ) { + const char *profile = value; + int profile_id; + av_log(avctx, AV_LOG_INFO, "profile (baseline, main): %s\n", profile); + profile_id = get_profile_id(profile); + if (profile_id < 0) { + av_log(avctx, AV_LOG_ERROR, "Invalid xeve param: profile(%s)\n", profile); + return XEVE_PARAM_BAD_VALUE; + } + xectx->profile_id = profile_id; + } else if (strcmp(key, "preset") == 0 ) { + const char *preset = value; + int preset_id; + av_log(avctx, AV_LOG_INFO, "Preset of xeve (fast, medium, slow, placebo): %s\n", preset); + preset_id = get_preset_id(preset); + if( preset_id < 0) { + av_log(avctx, AV_LOG_ERROR, "Invalid xeve param: preset(%s)\n", preset); + return XEVE_PARAM_BAD_VALUE; + } + xectx->preset_id = preset_id; + } else if (strcmp(key, "tune") == 0 ) { + const char *tune = value; + int tune_id; + av_log(avctx, AV_LOG_INFO, "Tune of xeve (psnr, zerolatency): %s\n", tune); + tune_id = get_tune_id(tune); + if( tune_id < 0) { + av_log(avctx, AV_LOG_ERROR, "Invalid xeve param: tune(%s)\n", tune); + return XEVE_PARAM_BAD_VALUE; + } + xectx->tune_id = tune_id; + } else if (strcmp(key, "bitrate") == 0 ) { + cdsc->param.bitrate = kbps_str_to_int((char *)value); + av_log(avctx, AV_LOG_INFO, "Bitrate = %dkbps\n", cdsc->param.bitrate); + } else if (strcmp(key, "q") == 0 || strcmp(key, "qp") == 0) { + int qp = atoi(value); + if(qp < 0 || qp > 51) { + av_log(avctx, AV_LOG_ERROR, "Invalid QP value (0~51) :%d\n", qp); + return XEVE_PARAM_BAD_VALUE; + } + cdsc->param.qp = qp; + av_log(avctx, AV_LOG_INFO, "QP value (0~51): %d\n", cdsc->param.qp); + } else { + av_log(avctx, AV_LOG_ERROR, "Unknown xeve codec option: %s\n", key); + return XEVE_PARAM_BAD_NAME; + } + + return 0; +} + +/** + * The function returns a pointer to the object of the XEVE_CDSC type. + * XEVE_CDSC contains all encoder parameters that should be initialized before the encoder is used. + * + * The field values of the XEVE_CDSC structure are populated based on: + * - the corresponding field values of the AvCodecConetxt structure, + * - the xeve encoder specific option values, + * (the full list of options available for xeve encoder is displayed after executing the command ./ffmpeg --help encoder = libxeve) + * - and the xeve encoder options specified as a list of key-value pairs following the xeve-params option + * + * The order of processing input data and populating the XEVE_CDSC structure + * 1) first, the fields of the AVCodecContext structure corresponding to the provided input options are processed, + * (i.e -pix_fmt yuv420p -s:v 1920x1080 -r 30 -profile:v 0) + * 2) then xeve-specific options added as AVOption to the xeve AVCodec implementation + * (i.e -threads_cnt 3 -preset 0) + * 3) finally, the options specified after the xeve-params option as the parameter list of type key value are processed + * (i.e -xeve-params "m=2:q=17") + * + * Keep in mind that, there are options that can be set in different ways. + * In this case, please follow the above-mentioned order of processing. + * The most recent assignments overwrite the previous values. + * + * @param[in] avctx codec context (AVCodecContext) + * @param[out] cdsc contains all Xeve MPEG-5 EVC encoder encoder parameters that should be initialized before the encoder is use + * + * @return 0 on success, negative error code on failure + */ +static int get_conf(AVCodecContext *avctx, XEVE_CDSC *cdsc) +{ + XeveContext *xectx = NULL; + int color_format; + int cpu_count = av_cpu_count(); + int ret; + + xectx = avctx->priv_data; + xectx->hash = 0; + + /* initialize xeve_param struct with default values */ + ret = xeve_param_default(&cdsc->param); + if (XEVE_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Cannot set_default parameter\n"); + return AVERROR_EXTERNAL; + } + + /* read options from AVCodecContext */ + if(avctx->width > 0) + cdsc->param.w = xectx->width_luma = avctx->width; + + if(avctx->height > 0) + cdsc->param.h = xectx->height_luma = avctx->height; + + if(avctx->framerate.num > 0) { + // fps can be float number, but xeve API doesn't support it + cdsc->param.fps = (int)(((float)avctx->framerate.num / avctx->framerate.den) + 0.5); + } + + if(avctx->gop_size >= 0) { // GOP size (key-frame interval) + cdsc->param.keyint = avctx->gop_size; // 0: only one I-frame at the first time; 1: every frame is coded in I-frame + } + + if (avctx->max_b_frames == 0 || avctx->max_b_frames == 1 || avctx->max_b_frames == 3 || + avctx->max_b_frames == 7 || avctx->max_b_frames == 15) { // number of b-frames + cdsc->param.bframes = avctx->max_b_frames; + } else { + av_log(avctx, AV_LOG_ERROR, "Incorrect value for maximum number of B frames: (%d) \n" + "Acceptable values for bf option (maximum number of B frames) are 0,1,3,7 or 15\n", avctx->max_b_frames); + return AVERROR_INVALIDDATA; + } + + if (avctx->level >= 0) + cdsc->param.level_idc = avctx->level; + + ret = get_pix_fmt(avctx->pix_fmt, &color_format, &xectx->input_depth); + if (ret != 0) { + av_log(avctx, AV_LOG_ERROR, "Unsupported pixel format.\n"); + return AVERROR_INVALIDDATA; + } + cdsc->param.cs = XEVE_CS_SET(color_format, xectx->input_depth, 0); + + if (avctx->rc_buffer_size > 0) { // VBV buf size + cdsc->param.vbv_bufsize = (int)(avctx->rc_buffer_size / 1000); + } + + // rc_type: Rate control type [ 0(CQP) / 1(ABR) / 2(CRF) ] + if (avctx->bit_rate > 0) { + if (avctx->bit_rate / 1000 > INT_MAX || avctx->rc_max_rate / 1000 > INT_MAX) { + av_log(avctx, AV_LOG_ERROR, "Not supported bitrate bit_rate and rc_max_rate > %d000\n", INT_MAX); + return AVERROR_INVALIDDATA; + } + cdsc->param.bitrate = (int)(avctx->bit_rate / 1000); + cdsc->param.rc_type = XEVE_RC_ABR; + } + + if (xectx->op_crf >= 0) { + cdsc->param.crf = xectx->op_crf; + cdsc->param.rc_type = XEVE_RC_CRF; + } + + if(avctx->thread_count <= 0) { + cdsc->param.threads = (cpu_count < XEVE_MAX_THREADS) ? cpu_count : XEVE_MAX_THREADS; + } else if(avctx->thread_count > XEVE_MAX_THREADS) + cdsc->param.threads = XEVE_MAX_THREADS; + else + cdsc->param.threads = avctx->thread_count; + + cdsc->param.cs = XEVE_CS_SET(color_format, cdsc->param.codec_bit_depth, 0); + cdsc->max_bs_buf_size = MAX_BS_BUF; + + if(avctx->profile == FF_PROFILE_EVC_BASELINE) + xectx->profile_id = XEVE_PROFILE_BASELINE; + else if(avctx->profile == FF_PROFILE_EVC_MAIN) + xectx->profile_id = XEVE_PROFILE_MAIN; + else { + av_log(avctx, AV_LOG_ERROR, "Unknown encoder profile (%d)\n" + "Acceptable values for profile option are 0 and 1 (0: baseline profile; 1: main profile)\n", avctx->profile); + return AVERROR_INVALIDDATA; + } + + if (xectx->op_preset) { // preset + xectx->preset_id = get_preset_id(xectx->op_preset); + } + + if (xectx->op_tune) { // tune + xectx->tune_id = get_tune_id(xectx->op_tune); + } + + ret = xeve_param_ppt(&cdsc->param, xectx->profile_id, xectx->preset_id, xectx->tune_id); + if (XEVE_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Cannot set profile(%d), preset(%d), tune(%d)\n", xectx->profile_id, xectx->preset_id, xectx->tune_id); + return AVERROR_EXTERNAL; + } + + /* parse : separated list of key=value parameters and set values for created descriptor (XEVE_CDSC) */ + { + AVDictionaryEntry *en = NULL; + + // Start to parse xeve_params + while ((en = av_dict_get(xectx->xeve_params, "", en, AV_DICT_IGNORE_SUFFIX))) { + int parse_ret = parse_xeve_params(avctx, en->key, en->value, cdsc); + + switch (parse_ret) { + case XEVE_PARAM_BAD_NAME: + av_log(avctx, AV_LOG_WARNING, "Unknown option: %s.\n", en->key); + break; + case XEVE_PARAM_BAD_VALUE: + av_log(avctx, AV_LOG_WARNING, "Invalid value for %s: %s.\n", en->key, en->value); + break; + default: + break; + } + } + } + + return 0; +} + +/** + * Check encoder configuration + * + * @param[in] avctx context for logger + * @param[in] cdsc contains all encoder parameters that should be initialized before its use + * + * @return 0 on success, negative error code on failure + */ +static int check_conf(AVCodecContext *avctx, XEVE_CDSC *cdsc) +{ + int ret = 0; + int min_block_size = 4; + int pic_m; + + if(cdsc->param.profile == XEVE_PROFILE_BASELINE) { + if (cdsc->param.tool_amvr == 1) { + av_log(avctx, AV_LOG_ERROR, "AMVR cannot be on in base profile\n"); + ret = -1; + } + if (cdsc->param.tool_mmvd == 1) { + av_log(avctx, AV_LOG_ERROR, "MMVD cannot be on in base profile\n"); + ret = -1; + } + if (cdsc->param.tool_affine == 1) { + av_log(avctx, AV_LOG_ERROR, "Affine cannot be on in base profile\n"); + ret = -1; + } + if (cdsc->param.tool_dmvr == 1) { + av_log(avctx, AV_LOG_ERROR, "DMVR cannot be on in base profile\n"); + ret = -1; + } + if (cdsc->param.tool_admvp == 1) { + av_log(avctx, AV_LOG_ERROR, "ADMVP cannot be on in base profile\n"); + ret = -1; + } + if (cdsc->param.tool_hmvp == 1) { + av_log(avctx, AV_LOG_ERROR, "HMVP cannot be on in base profile\n"); + ret = -1; + } + if (cdsc->param.tool_addb == 1) { + av_log(avctx, AV_LOG_ERROR, "ADDB cannot be on in base profile\n"); + ret = -1; + } + if (cdsc->param.tool_alf == 1) { + av_log(avctx, AV_LOG_ERROR, "ALF cannot be on in base profile\n"); + ret = -1; + } + if (cdsc->param.tool_htdf == 1) { + av_log(avctx, AV_LOG_ERROR, "HTDF cannot be on in base profile\n"); + ret = -1; + } + if (cdsc->param.btt == 1) { + av_log(avctx, AV_LOG_ERROR, "BTT cannot be on in base profile\n"); + ret = -1; + } + if (cdsc->param.suco == 1) { + av_log(avctx, AV_LOG_ERROR, "SUCO cannot be on in base profile\n"); + ret = -1; + } + if (cdsc->param.tool_eipd == 1) { + av_log(avctx, AV_LOG_ERROR, "EIPD cannot be on in base profile\n"); + ret = -1; + } + if (cdsc->param.tool_iqt == 1) { + av_log(avctx, AV_LOG_ERROR, "IQT cannot be on in base profile\n"); + ret = -1; + } + if (cdsc->param.tool_cm_init == 1) { + av_log(avctx, AV_LOG_ERROR, "CM_INIT cannot be on in base profile\n"); + ret = -1; + } + if (cdsc->param.tool_adcc == 1) { + av_log(avctx, AV_LOG_ERROR, "ADCC cannot be on in base profile\n"); + ret = -1; + } + if (cdsc->param.tool_ats == 1) { + av_log(avctx, AV_LOG_ERROR, "ATS_INTRA cannot be on in base profile\n"); + ret = -1; + } + if (cdsc->param.ibc_flag == 1) { + av_log(avctx, AV_LOG_ERROR, "IBC cannot be on in base profile\n"); + ret = -1; + } + if (cdsc->param.tool_rpl == 1) { + av_log(avctx, AV_LOG_ERROR, "RPL cannot be on in base profile\n"); + ret = -1; + } + if (cdsc->param.tool_pocs == 1) { + av_log(avctx, AV_LOG_ERROR, "POCS cannot be on in base profile\n"); + ret = -1; + } + } else { + if (cdsc->param.tool_admvp == 0 && cdsc->param.tool_affine == 1) { + av_log(avctx, AV_LOG_ERROR, "AFFINE cannot be on when ADMVP is off\n"); + ret = -1; + } + if (cdsc->param.tool_admvp == 0 && cdsc->param.tool_amvr == 1) { + av_log(avctx, AV_LOG_ERROR, "AMVR cannot be on when ADMVP is off\n"); + ret = -1; + } + if (cdsc->param.tool_admvp == 0 && cdsc->param.tool_dmvr == 1) { + av_log(avctx, AV_LOG_ERROR, "DMVR cannot be on when ADMVP is off\n"); + ret = -1; + } + if (cdsc->param.tool_admvp == 0 && cdsc->param.tool_mmvd == 1) { + av_log(avctx, AV_LOG_ERROR, "MMVD cannot be on when ADMVP is off\n"); + ret = -1; + } + if (cdsc->param.tool_eipd == 0 && cdsc->param.ibc_flag == 1) { + av_log(avctx, AV_LOG_ERROR, "IBC cannot be on when EIPD is off\n"); + ret = -1; + } + if (cdsc->param.tool_iqt == 0 && cdsc->param.tool_ats == 1) { + av_log(avctx, AV_LOG_ERROR, "ATS cannot be on when IQT is off\n"); + ret = -1; + } + if (cdsc->param.tool_cm_init == 0 && cdsc->param.tool_adcc == 1) { + av_log(avctx, AV_LOG_ERROR, "ADCC cannot be on when CM_INIT is off\n"); + ret = -1; + } + } + + if (cdsc->param.btt == 1) { + if (cdsc->param.framework_cb_max && cdsc->param.framework_cb_max < 5) { + av_log(avctx, AV_LOG_ERROR, "Maximun Coding Block size cannot be smaller than 5\n"); + ret = -1; + } + if (cdsc->param.framework_cb_max > 7) { + av_log(avctx, AV_LOG_ERROR, "Maximun Coding Block size cannot be greater than 7\n"); + ret = -1; + } + if (cdsc->param.framework_cb_min && cdsc->param.framework_cb_min < 2) { + av_log(avctx, AV_LOG_ERROR, "Minimum Coding Block size cannot be smaller than 2\n"); + ret = -1; + } + if ((cdsc->param.framework_cb_max || cdsc->param.framework_cb_min) && + cdsc->param.framework_cb_min > cdsc->param.framework_cb_max) { + av_log(avctx, AV_LOG_ERROR, "Minimum Coding Block size cannot be greater than Maximum coding Block size\n"); + ret = -1; + } + if (cdsc->param.framework_cu14_max > 6) { + av_log(avctx, AV_LOG_ERROR, "Maximun 1:4 Coding Block size cannot be greater than 6\n"); + ret = -1; + } + if ((cdsc->param.framework_cb_max || cdsc->param.framework_cu14_max) && + cdsc->param.framework_cu14_max > cdsc->param.framework_cb_max) { + av_log(avctx, AV_LOG_ERROR, "Maximun 1:4 Coding Block size cannot be greater than Maximum coding Block size\n"); + ret = -1; + } + if (cdsc->param.framework_tris_max > 6) { + av_log(avctx, AV_LOG_ERROR, "Maximun Tri-split Block size be greater than 6\n"); + ret = -1; + } + if ((cdsc->param.framework_tris_max || cdsc->param.framework_cb_max) && + cdsc->param.framework_tris_max > cdsc->param.framework_cb_max) { + av_log(avctx, AV_LOG_ERROR, "Maximun Tri-split Block size cannot be greater than Maximum coding Block size\n"); + ret = -1; + } + if ((cdsc->param.framework_tris_min || cdsc->param.framework_cb_min) && + cdsc->param.framework_tris_min < cdsc->param.framework_cb_min + 2) { + av_log(avctx, AV_LOG_ERROR, "Maximun Tri-split Block size cannot be smaller than Minimum Coding Block size plus two\n"); + ret = -1; + } + if(cdsc->param.framework_cb_min) min_block_size = 1 << cdsc->param.framework_cb_min; + else min_block_size = 8; + } + + if (cdsc->param.suco == 1) { + if (cdsc->param.framework_suco_max > 6) { + av_log(avctx, AV_LOG_ERROR, "Maximun SUCO size cannot be greater than 6\n"); + ret = -1; + } + if (cdsc->param.framework_cb_max && cdsc->param.framework_suco_max > cdsc->param.framework_cb_max) { + av_log(avctx, AV_LOG_ERROR, "Maximun SUCO size cannot be greater than Maximum coding Block size\n"); + ret = -1; + } + if (cdsc->param.framework_suco_min < 4) { + av_log(avctx, AV_LOG_ERROR, "Minimun SUCO size cannot be smaller than 4\n"); + ret = -1; + } + if (cdsc->param.framework_cb_min && cdsc->param.framework_suco_min < cdsc->param.framework_cb_min) { + av_log(avctx, AV_LOG_ERROR, "Minimun SUCO size cannot be smaller than Minimum coding Block size\n"); + ret = -1; + } + if (cdsc->param.framework_suco_min > cdsc->param.framework_suco_max) { + av_log(avctx, AV_LOG_ERROR, "Minimum SUCO size cannot be greater than Maximum SUCO size\n"); + ret = -1; + } + } + + pic_m = (8 > min_block_size) ? min_block_size : 8; + if ((cdsc->param.w & (pic_m - 1)) != 0) { + av_log(avctx, AV_LOG_ERROR, "Current encoder does not support picture width, not multiple of max(8, minimum CU size)\n"); + ret = -1; + } + if ((cdsc->param.h & (pic_m - 1)) != 0) { + av_log(avctx, AV_LOG_ERROR, "Current encoder does not support picture height, not multiple of max(8, minimum CU size)\n"); + ret = -1; + } + + if(ret == -1) { + return AVERROR_INVALIDDATA; + } + + return 0; +} + +/** + * Set XEVE_CFG_SET_USE_PIC_SIGNATURE for encoder + * + * @param[in] logger context + * @param[in] id XEVE encodec instance identifier + * @param[in] ctx the structure stores all the states associated with the instance of Xeve MPEG-5 EVC encoder + * + * @return 0 on success, negative error code on failure + */ +static int set_extra_config(AVCodecContext* avctx, XEVE id, XeveContext *ctx) +{ + int ret, size, value; + size = 4; + + // embed SEI messages identifying encoder parameters and command line arguments + // - 0: off\n" + // - 1: emit sei info" + // + // SEI - Supplemental enhancement information contains information + // that is not necessary to decode the samples of coded pictures from VCL NAL units. + // Some SEI message information is required to check bitstream conformance + // and for output timing decoder conformance. + // @see ISO_IEC_23094-1_2020 7.4.3.5 + // @see ISO_IEC_23094-1_2020 Annex D + ret = xeve_config(id, XEVE_CFG_SET_SEI_CMD, &value, &size); // sei_cmd_info + if (XEVE_FAILED(ret)) + { + av_log(avctx, AV_LOG_ERROR, "Failed to set config for sei command info messages\n"); + return AVERROR_EXTERNAL; + } + + if(ctx->hash) { + value = 1; + ret = xeve_config(id, XEVE_CFG_SET_USE_PIC_SIGNATURE, &value, &size); + if(XEVE_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to set config for picture signature\n"); + return AVERROR_EXTERNAL; + } + } + + return 0; +} + +/** + * Convert FFmpeg pixel format (AVPixelFormat) into XEVE pre-defined color space + * + * @param[in] px_fmt pixel format (@see https://ffmpeg.org/doxygen/trunk/pixfmt_8h.html#a9a8e335cf3be472042bc9f0cf80cd4c5) + * + * @return XEVE pre-defined color space (@see xeve.h) on success, XEVE_CF_UNKNOWN on failure + */ +static int xeve_color_space(enum AVPixelFormat pix_fmt) +{ + /* color space of input image */ + int cs = XEVE_CF_UNKNOWN; + + switch (pix_fmt) { + case AV_PIX_FMT_YUV420P: + cs = XEVE_CS_YCBCR420; + break; + case AV_PIX_FMT_YUV420P10: +#if AV_HAVE_BIGENDIAN + cs = XEVE_CS_SET(XEVE_CF_YCBCR420, 10, 1); +#else + cs = XEVE_CS_YCBCR420_10LE; +#endif + + break; + case AV_PIX_FMT_YUV420P12: +#if AV_HAVE_BIGENDIAN + cs = XEVE_CS_SET(XEVE_CF_YCBCR420, 12, 1); +#else + cs = XEVE_CS_YCBCR420_12LE; +#endif + + break; + case AV_PIX_FMT_YUV422P: + cs = XEVE_CS_YCBCR422; + break; + case AV_PIX_FMT_YUV422P10: +#if AV_HAVE_BIGENDIAN + cs = XEVE_CS_SET(XEVE_CF_YCBCR422, 10, 1); +#else + cs = XEVE_CS_YCBCR422_10LE; +#endif + + break; + case AV_PIX_FMT_YUV422P12: +#if AV_HAVE_BIGENDIAN + cs = XEVE_CS_SET(XEVE_CF_YCBCR422, 12, 1); +#else + cs = XEVE_CS_SET(XEVE_CF_YCBCR422, 12, 0); +#endif + + break; + case AV_PIX_FMT_GBRP: + case AV_PIX_FMT_GBRP10: + case AV_PIX_FMT_GBRP12: + cs = XEVE_CF_UNKNOWN; + break; + case AV_PIX_FMT_YUV444P: + cs = XEVE_CF_YCBCR444; + break; + case AV_PIX_FMT_YUV444P10: +#if AV_HAVE_BIGENDIAN + cs = XEVE_CS_SET(XEVE_CF_YCBCR444, 10, 1); +#else + cs = XEVE_CS_YCBCR444_10LE; +#endif + + break; + case AV_PIX_FMT_YUV444P12: +#if AV_HAVE_BIGENDIAN + cs = XEVE_CS_SET(XEVE_CF_YCBCR444, 12, 1); +#else + cs = XEVE_CS_SET(XEVE_CF_YCBCR444, 12, 0); +#endif + + break; + case AV_PIX_FMT_GRAY8: + cs = XEVE_CF_YCBCR400; + break; + case AV_PIX_FMT_GRAY10: +#if AV_HAVE_BIGENDIAN + cs = XEVE_CS_SET(XEVE_CF_YCBCR400, 10, 1); +#else + cs = XEVE_CS_YCBCR400_10LE; +#endif + + break; + case AV_PIX_FMT_GRAY12: +#if AV_HAVE_BIGENDIAN + cs = XEVE_CS_SET(XEVE_CF_YCBCR400, 12, 1); +#else + cs = XEVE_CS_YCBCR400_12LE; +#endif + + break; + default: + cs = XEVE_CF_UNKNOWN; + break; + } + + return cs; +} + +/** + * @brief Switch encoder to bumping mode + * + * @param id XEVE encodec instance identifier + * @return 0 on success, negative error code on failure + */ +static int setup_bumping(XEVE id) +{ + int val, size; + val = 1; + size = sizeof(int); + if(XEVE_FAILED(xeve_config(id, XEVE_CFG_SET_FORCE_OUT, (void *)(&val), &size))) + return AVERROR_EXTERNAL; + + return 0; +} + +/** + * @brief Initialize eXtra-fast Essential Video Encoder codec + * Create an encoder instance and allocate all the needed resources + * + * @param avctx codec context + * @return 0 on success, negative error code on failure + */ +static av_cold int libxeve_init(AVCodecContext *avctx) +{ + XeveContext *xectx = avctx->priv_data; + unsigned char *bs_buf = NULL; + int i; + int shift_h = 0; + int shift_v = 0; + XEVE_IMGB *imgb = NULL; + int ret = 0; + + XEVE_CDSC *cdsc = &(xectx->cdsc); + + if(avctx->pix_fmt != AV_PIX_FMT_YUV420P && avctx->pix_fmt != AV_PIX_FMT_YUV420P10) { + av_log(avctx, AV_LOG_ERROR, "Invalid pixel format: %s\n", av_get_pix_fmt_name(avctx->pix_fmt)); + ret = AVERROR_INVALIDDATA; + goto ERR; + } + + /* allocate bitstream buffer */ + bs_buf = (unsigned char *)av_malloc(MAX_BS_BUF); + if(bs_buf == NULL) { + av_log(avctx, AV_LOG_ERROR, "Cannot allocate bitstream buffer\n"); + ret = AVERROR(ENOMEM); + goto ERR; + } + + /* read configurations and set values for created descriptor (XEVE_CDSC) */ + if ((ret = get_conf(avctx, cdsc)) != 0) { + av_log(avctx, AV_LOG_ERROR, "Cannot get configuration\n"); + goto ERR; + } + + if ((ret = check_conf(avctx, cdsc)) != 0) { + av_log(avctx, AV_LOG_ERROR, "Invalid configuration\n"); + goto ERR; + } + + /* create encoder */ + xectx->id = xeve_create(cdsc, NULL); + if(xectx->id == NULL) { + av_log(avctx, AV_LOG_ERROR, "Cannot create XEVE encoder\n"); + ret = AVERROR_EXTERNAL; + goto ERR; + } + + if((ret = set_extra_config(avctx, xectx->id, xectx))!=0) { + av_log(avctx, AV_LOG_ERROR, "Cannot set extra configuration\n"); + goto ERR; + } + + xectx->bitb.addr = bs_buf; + xectx->bitb.bsize = MAX_BS_BUF; + + if((ret = av_pix_fmt_get_chroma_sub_sample(avctx->pix_fmt, &shift_h, &shift_v)) != 0) { + av_log(avctx, AV_LOG_ERROR, "Failed to get chroma shift\n"); + goto ERR; + } + + // YUV format explanation + // shift_h == 1 && shift_v == 1 : YUV420 + // shift_h == 1 && shift_v == 0 : YUV422 + // shift_h == 0 && shift_v == 0 : YUV444 + // + xectx->width_chroma = AV_CEIL_RSHIFT(xectx->width_luma, shift_h); + xectx->height_chroma = AV_CEIL_RSHIFT(xectx->height_luma, shift_v); + + /* set default values for input image buffer */ + imgb = &xectx->imgb; + imgb->cs = xeve_color_space(avctx->pix_fmt); + imgb->np = 3; /* only for yuv420p, yuv420ple */ + + for (i = 0; i < imgb->np; i++) + imgb->x[i] = imgb->y[i] = 0; + + imgb->w[0] = imgb->aw[0] = xectx->width_luma; + imgb->w[1] = imgb->w[2] = imgb->aw[1] = imgb->aw[2] = xectx->width_chroma; + imgb->h[0] = imgb->ah[0] = xectx->height_luma; + imgb->h[1] = imgb->h[2] = imgb->ah[1] = imgb->ah[2] = xectx->height_chroma; + + xectx->encod_frames = 0; + xectx->bytes_total = 0; + xectx->state = STATE_ENCODING; + xectx->packet_count = 0; + xectx->bitrate = 0; + + return 0; + +ERR: + if(bs_buf) + av_free(bs_buf); + + return ret; +} + +/** + * Encode raw data frame into EVC packet + * + * @param[in] avctx codec context + * @param[out] pkt output AVPacket containing encoded data + * @param[in] frame AVFrame containing the raw data to be encoded + * @param[out] got_packet encoder sets to 0 or 1 to indicate that a + * non-empty packet was returned in pkt + * + * @return 0 on success, negative error code on failure + */ +static int libxeve_encode(AVCodecContext *avctx, AVPacket *avpkt, + const AVFrame *frame, int *got_packet) +{ + XeveContext *xectx = NULL; + int ret = -1; + int xeve_cs; + + xectx = avctx->priv_data; + + if(xectx->state == STATE_SKIPPING && frame ) { + xectx->state = STATE_ENCODING; // Entering encoding process + } else if(xectx->state == STATE_ENCODING && frame == NULL) { + if (setup_bumping(xectx->id) == 0) + xectx->state = STATE_BUMPING; // Entering bumping process + else { + av_log(avctx, AV_LOG_ERROR, "Failed to setup bumping\n"); + xectx->state = STATE_SKIPPING; + } + } + + if(xectx->state == STATE_ENCODING) { + const AVPixFmtDescriptor *pixel_fmt_desc = av_pix_fmt_desc_get (frame->format); + if(!pixel_fmt_desc) { + av_log(avctx, AV_LOG_ERROR, "Invalid pixel format descriptor for pixel format: %s\n", av_get_pix_fmt_name(avctx->pix_fmt)); + return AVERROR_INVALIDDATA; + } + + xeve_cs = xeve_color_space(avctx->pix_fmt); + if(xeve_cs != XEVE_CS_YCBCR420 && xeve_cs != XEVE_CS_YCBCR420_10LE) { + av_log(avctx, AV_LOG_ERROR, "Invalid pixel format: %s\n", av_get_pix_fmt_name(avctx->pix_fmt)); + return AVERROR_INVALIDDATA; + } + + { + int i; + XEVE_IMGB *imgb = NULL; + + imgb = &xectx->imgb; + + for (i = 0; i < imgb->np; i++) { + imgb->a[i] = frame->data[i]; + imgb->s[i] = frame->linesize[i]; + } + + if(xectx->id == NULL) { + av_log(avctx, AV_LOG_ERROR, "Invalid XEVE encoder\n"); + return AVERROR_INVALIDDATA; + } + + imgb->ts[0] = frame->pts; + imgb->ts[1] = 0; + + /* push image to encoder */ + ret = xeve_push(xectx->id, imgb); + if(XEVE_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "xeve_push() failed\n"); + return AVERROR_EXTERNAL; + } + } + } + if(xectx->state == STATE_ENCODING || xectx->state == STATE_BUMPING) { + + /* encoding */ + ret = xeve_encode(xectx->id, &(xectx->bitb), &(xectx->stat)); + if(XEVE_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "xeve_encode() failed\n"); + return AVERROR_EXTERNAL; + } + + xectx->encod_frames++; + + /* store bitstream */ + if (ret == XEVE_OK_OUT_NOT_AVAILABLE) { // Return OK but picture is not available yet + *got_packet = 0; + return 0; + } else if(ret == XEVE_OK) { + int av_pic_type; + + if(xectx->stat.write > 0) { + xectx->bytes_total += xectx->stat.write; + + ret = ff_get_encode_buffer(avctx, avpkt, xectx->stat.write, AV_GET_ENCODE_BUFFER_FLAG_REF); + if (ret < 0) { + av_log(avctx, AV_LOG_ERROR, "Can't allocate memory for AVPacket data\n"); + return ret; + } + + memcpy(avpkt->data, xectx->bitb.addr, xectx->stat.write); + + avpkt->pts = xectx->bitb.ts[0]; + avpkt->dts = xectx->bitb.ts[1]; + + xectx->bitrate += (xectx->stat.write - xectx->stat.sei_size); + + switch(xectx->stat.stype) { + case XEVE_ST_I: + av_pic_type = AV_PICTURE_TYPE_I; + avpkt->flags |= AV_PKT_FLAG_KEY; + break; + case XEVE_ST_P: + av_pic_type = AV_PICTURE_TYPE_P; + break; + case XEVE_ST_B: + av_pic_type = AV_PICTURE_TYPE_B; + break; + case XEVE_ST_UNKNOWN: + av_log(avctx, AV_LOG_ERROR, "Unknown slice type\n"); + return AVERROR_INVALIDDATA; + } + + ff_side_data_set_encoder_stats(avpkt, xectx->stat.qp * FF_QP2LAMBDA, NULL, 0, av_pic_type); + + xectx->bitrate += (xectx->stat.write - xectx->stat.sei_size); + + *got_packet = 1; + xectx->packet_count++; + } + } else if (ret == XEVE_OK_NO_MORE_FRM) { + // Return OK but no more frames + return 0; + } else { + av_log(avctx, AV_LOG_ERROR, "Invalid return value: %d\n", ret); + return AVERROR_EXTERNAL; + } + } else { + av_log(avctx, AV_LOG_ERROR, "Udefined encoder state\n"); + return AVERROR_INVALIDDATA; + } + + return 0; +} + +/** + * Destroy the encoder and release all the allocated resources + * + * @param avctx codec context + * @return 0 on success, negative error code on failure + */ +static av_cold int libxeve_close(AVCodecContext *avctx) +{ + XeveContext *xectx = avctx->priv_data; + + xeve_delete(xectx->id); + + if(xectx->bitb.addr) + av_free(xectx->bitb.addr); /* release bitstream buffer */ + + return 0; +} + +#define OFFSET(x) offsetof(XeveContext, x) +#define VE AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_ENCODING_PARAM + +// Example of using: ./ffmpeg -xeve-params "m=2:q=17" +// Consider using following options (./ffmpeg --help encoder=libxeve) +// +static const AVOption xeve_options[] = { + { "preset", "Encoding preset for setting encoding speed [fast, medium, slow, placebo]", OFFSET(op_preset), AV_OPT_TYPE_STRING, { .str = "medium" }, 0, 0, VE }, + { "tune", "Tuneing parameter for special purpose operation [psnr, zerolatency]", OFFSET(op_tune), AV_OPT_TYPE_STRING, { 0 }, 0, 0, VE}, + { "qp", "quantization parameter qp <0..51> [default: 32]", OFFSET(op_qp), AV_OPT_TYPE_INT, { .i64 = 32 }, 0, 51, VE }, + { "crf", "constant rate factor <-1..51> [default: 32]", OFFSET(op_crf), AV_OPT_TYPE_INT, { .i64 = -1 }, -1, 51, VE }, + { "xeve-params", "override the xeve configuration using a : separated list of key=value parameters", OFFSET(xeve_params), AV_OPT_TYPE_DICT, { 0 }, 0, 0, VE }, + { NULL } +}; + +static const AVClass xeve_class = { + .class_name = "libxeve", + .item_name = av_default_item_name, + .option = xeve_options, + .version = LIBAVUTIL_VERSION_INT, +}; + +/** + * libavcodec generic global options, which can be set on all the encoders and decoders + * @see https://www.ffmpeg.org/ffmpeg-codecs.html#Codec-Options + */ +static const FFCodecDefault xeve_defaults[] = { + { "b", "0" }, // bitrate + { "g", "0" }, // gop_size (key-frame interval 0: only one I-frame at the first time; 1: every frame is coded in I-frame) + { "bf", "15"}, // bframes (0: no B-frames) + { "profile", "0"}, // encoder codec profile (0: baselie; 1: main) + { "threads", "0"}, // number of threads to be used (0: automatically select the number of threads to set) + { NULL }, +}; + +const FFCodec ff_libxeve_encoder = { + .p.name = "libxeve", + .p.long_name = NULL_IF_CONFIG_SMALL("libxeve MPEG-5 EVC"), + .p.type = AVMEDIA_TYPE_VIDEO, + .p.id = AV_CODEC_ID_EVC, + .init = libxeve_init, + FF_CODEC_ENCODE_CB(libxeve_encode), + .close = libxeve_close, + .priv_data_size = sizeof(XeveContext), + .p.priv_class = &xeve_class, + .defaults = xeve_defaults, + .p.capabilities = AV_CODEC_CAP_DELAY | AV_CODEC_CAP_AUTO_THREADS | AV_CODEC_CAP_DR1, + .p.wrapper_name = "libxeve", +}; diff --git a/libavcodec/parsers.c b/libavcodec/parsers.c index 6b40c18d80..aa49ce1c9a 100644 --- a/libavcodec/parsers.c +++ b/libavcodec/parsers.c @@ -41,6 +41,7 @@ extern const AVCodecParser ff_dvaudio_parser; extern const AVCodecParser ff_dvbsub_parser; extern const AVCodecParser ff_dvdsub_parser; extern const AVCodecParser ff_dvd_nav_parser; +extern const AVCodecParser ff_evc_parser; extern const AVCodecParser ff_flac_parser; extern const AVCodecParser ff_g723_1_parser; extern const AVCodecParser ff_g729_parser; -- 2.17.1 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [FFmpeg-devel] [PATCH 2/3] Provided support for MPEG-5 EVC (Essential Video Coding) codec 2022-05-12 6:05 ` Dawid Kozinski @ 2022-06-01 11:41 ` Andreas Rheinhardt 0 siblings, 0 replies; 6+ messages in thread From: Andreas Rheinhardt @ 2022-06-01 11:41 UTC (permalink / raw) To: ffmpeg-devel Dawid Kozinski: > - Added xeve encoder wrapper > - Added xevd dencoder wrapper > - Added documentation for xeve and xevd wrappers > - Added parser for EVC format > - Changes in project configuration file and libavcodec Makefile > > Signed-off-by: Dawid Kozinski <d.kozinski@samsung.com> > --- > configure | 8 + > doc/decoders.texi | 24 + > doc/encoders.texi | 112 ++++ > doc/general_contents.texi | 19 + > libavcodec/Makefile | 3 + > libavcodec/allcodecs.c | 3 + > libavcodec/avcodec.h | 3 + > libavcodec/evc_parser.c | 527 +++++++++++++++++ > libavcodec/libxevd.c | 444 ++++++++++++++ > libavcodec/libxeve.c | 1152 +++++++++++++++++++++++++++++++++++++ > libavcodec/parsers.c | 1 + > 11 files changed, 2296 insertions(+) > create mode 100644 libavcodec/evc_parser.c > create mode 100644 libavcodec/libxevd.c > create mode 100644 libavcodec/libxeve.c > > diff --git a/libavcodec/allcodecs.c b/libavcodec/allcodecs.c > index c47133aa18..34371af3f4 100644 > --- a/libavcodec/allcodecs.c > +++ b/libavcodec/allcodecs.c > @@ -790,8 +790,11 @@ extern const FFCodec ff_libx262_encoder; > #endif > extern LIBX264_CONST FFCodec ff_libx264_encoder; > #endif > + Spurious change > extern const FFCodec ff_libx264rgb_encoder; > extern FFCodec ff_libx265_encoder; > +extern const FFCodec ff_libxeve_encoder; > +extern const FFCodec ff_libxevd_decoder; > extern const FFCodec ff_libxavs_encoder; > extern const FFCodec ff_libxavs2_encoder; > extern const FFCodec ff_libxvid_encoder; > diff --git a/libavcodec/libxeve.c b/libavcodec/libxeve.c > new file mode 100644 > index 0000000000..937b7fc5ce > --- /dev/null > +++ b/libavcodec/libxeve.c > @@ -0,0 +1,1152 @@ > +/* > + * libxeve encoder > + * EVC (MPEG-5 Essential Video Coding) encoding using XEVE MPEG-5 EVC encoder library > + * > + * Copyright (C) 2021 Dawid Kozinski <d.kozinski@samsung.com> > + * > + * This file is part of FFmpeg. > + * > + * FFmpeg is free software; you can redistribute it and/or > + * modify it under the terms of the GNU Lesser General Public > + * License as published by the Free Software Foundation; either > + * version 2.1 of the License, or (at your option) any later version. > + * > + * FFmpeg is distributed in the hope that it will be useful, > + * but WITHOUT ANY WARRANTY; without even the implied warranty of > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > + * Lesser General Public License for more details. > + * > + * You should have received a copy of the GNU Lesser General Public > + * License along with FFmpeg; if not, write to the Free Software > + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA > + */ > + > +#if defined(_MSC_VER) > +#define XEVE_API_IMPORTS 1 > +#endif This value should come from pkg-config and not be set solely based upon the compiler. See the discussion about removing this from libx264.c for more info. > + > +#include <float.h> > +#include <stdlib.h> > + > +#include <xeve.h> > + > +#include "libavutil/internal.h" > +#include "libavutil/common.h" > +#include "libavutil/opt.h" > +#include "libavutil/pixdesc.h" > +#include "libavutil/pixfmt.h" > +#include "libavutil/time.h" > +#include "libavutil/cpu.h" > + > +#include "avcodec.h" > +#include "internal.h" > +#include "packet_internal.h" > +#include "codec_internal.h" > +#include "encode.h" > + > +#define MAX_BS_BUF (16*1024*1024) > + > +/** > + * Error codes > + */ > +#define XEVE_PARAM_BAD_NAME -100 > +#define XEVE_PARAM_BAD_VALUE -200 > + > +/** > + * Encoder states > + * > + * STATE_ENCODING - the encoder receives and processes input frames > + * STATE_BUMPING - there are no more input frames, however the encoder still processes previously received data > + * STATE_SKIPPING - skipping input frames > + */ > +typedef enum State { > + STATE_ENCODING, > + STATE_BUMPING, > + STATE_SKIPPING > +} State; > + > +/** > + * The structure stores all the states associated with the instance of Xeve MPEG-5 EVC encoder > + */ > +typedef struct XeveContext { > + const AVClass *class; > + > + XEVE id; // XEVE instance identifier > + XEVE_CDSC cdsc; // coding parameters i.e profile, width & height of input frame, num of therads, frame rate ... > + XEVE_BITB bitb; // bitstream buffer (output) > + XEVE_STAT stat; // encoding status (output) > + XEVE_IMGB imgb; // image buffer (input) > + > + State state; // encoder state (skipping, encoding, bumping) > + > + int encod_frames; // num of encoded frames This field is write-only; some of the other fields are write-only, too. > + double bytes_total; // encoded bitstream byte size > + double bitrate; // bits per second > + int packet_count; // num of packets created by encoder > + > + // Chroma subsampling > + int width_luma; > + int height_luma; > + int width_chroma; > + int height_chroma; > + > + int profile_id; // encoder profile (main, baseline) > + int preset_id; // preset of xeve ( fast, medium, slow, placebo) > + int tune_id; // tune of xeve (psnr, zerolatency) > + int input_depth; // input bit-depth: 8bit, 10bit > + int hash; // embed picture signature (HASH) for conformance checking in decoding > + > + /* variables for input parameter */ > + char *op_preset; > + char *op_tune; > + int op_qp; > + int op_crf; > + > + // configuration parameters > + // xeve configuration read from a : separated list of key=value parameters > + AVDictionary *xeve_params; > +} XeveContext; > + > +/** > + * Gets Xeve encoder pre-defined profile > + * > + * @param profile string describing Xeve encoder profile (baseline, main) > + * @return XEVE pre-defined profile on success, negative value on failure > + */ > +static int get_profile_id(const char *profile) > +{ > + if (!strcmp(profile, "baseline")) > + return XEVE_PROFILE_BASELINE; > + else if (!strcmp(profile, "main")) > + return XEVE_PROFILE_MAIN; > + else > + return AVERROR(EINVAL); > +} > + > +/** > + * Gets Xeve pre-defined preset > + * > + * @param preset string describing Xeve encoder preset (fast, medium, slow, placebo) > + * @return XEVE pre-defined profile on success, negative value on failure > + */ > +static int get_preset_id(const char *preset) > +{ > + if((!strcmp(preset, "fast"))) > + return XEVE_PRESET_FAST; > + else if (!strcmp(preset, "medium")) > + return XEVE_PRESET_MEDIUM; > + else if (!strcmp(preset, "slow")) > + return XEVE_PRESET_SLOW; > + else if (!strcmp(preset, "placebo")) > + return XEVE_PRESET_PLACEBO; > + else > + return AVERROR(EINVAL); > +} > + > +/** > + * Gets Xeve pre-defined tune id > + * > + * @param[in] tune string describing Xeve encoder tune (psnr, zerolatency) > + * @return XEVE pre-defined profile on success, negative value on failure > + */ > +static int get_tune_id(const char *tune) > +{ > + if((!strcmp(tune, "psnr"))) > + return XEVE_TUNE_PSNR; > + else if (!strcmp(tune, "zerolatency")) > + return XEVE_TUNE_ZEROLATENCY; > + else > + return AVERROR(EINVAL); > +} > + > +/** > + * Convert FFmpeg pixel format (AVPixelFormat) to XEVE pre-defined color format > + * > + * @param[in] px_fmt pixel format (@see https://ffmpeg.org/doxygen/trunk/pixfmt_8h.html#a9a8e335cf3be472042bc9f0cf80cd4c5) > + * @param[out] color_format XEVE pre-defined color format (@see xeve.h) > + * @param[out] bit_depth bit depth > + * > + * @return 0 on success, negative value on failure > + */ > +static int get_pix_fmt(enum AVPixelFormat pix_fmt, int *color_format, int *bit_depth) > +{ > + switch (pix_fmt) { > + case AV_PIX_FMT_YUV420P: > + *color_format = XEVE_CF_YCBCR420; > + *bit_depth = 8; > + break; > + case AV_PIX_FMT_YUV422P: > + *color_format = XEVE_CF_YCBCR422; > + *bit_depth = 8; > + break; > + case AV_PIX_FMT_YUV444P: > + *color_format = XEVE_CF_YCBCR444; > + *bit_depth = 8; > + break; > + case AV_PIX_FMT_YUV420P10: > + *color_format = XEVE_CF_YCBCR420; > + *bit_depth = 10; > + break; > + case AV_PIX_FMT_YUV422P10: > + *color_format = XEVE_CF_YCBCR422; > + *bit_depth = 10; > + break; > + case AV_PIX_FMT_YUV444P10: > + *color_format = XEVE_CF_YCBCR444; > + *bit_depth = 10; > + break; > + default: > + *color_format = XEVE_CF_UNKNOWN; > + return AVERROR_INVALIDDATA; > + } > + > + return 0; > +} > + > +static int kbps_str_to_int(char *str) > +{ > + int kbps = 0; > + if (strchr(str, 'K') || strchr(str, 'k')) { > + char *tmp = strtok(str, "Kk "); > + kbps = (int)(atof(tmp)); > + } else if (strchr(str, 'M') || strchr(str, 'm')) { > + char *tmp = strtok(str, "Mm "); strtok is not threadsafe, so it must not be used at all. We have an av_strtok() replacement for this, yet it (like strtok) trashes the string given to it and so this should be avoided, too. > + kbps = (int)(atof(tmp) * 1000); > + } else > + kbps = atoi(str); > + > + return kbps; > +} > + > +/** > + * Parse : separated list of key=value parameters > + * > + * @param[in] avctx context for logger > + * @param[in] key > + * @param[in] value > + * @param[out] cdsc contains all Xeve MPEG-5 EVC encoder encoder parameters that > + * should be initialized before the encoder is use > + * > + * @return 0 on success, negative value on failure > + */ > +static int parse_xeve_params(AVCodecContext *avctx, const char *key, const char *value, XEVE_CDSC* cdsc) > +{ > + XeveContext *xectx = NULL; > + xectx = avctx->priv_data; > + > + if(!key) { > + av_log(avctx, AV_LOG_ERROR, "Ivalid argument: key string is NULL\n"); ^ n > + return XEVE_PARAM_BAD_VALUE; > + } > + if(!value) { > + if (strcmp(key, "hash") == 0) { > + xectx->hash = 1; > + av_log(avctx, AV_LOG_INFO, "embedding signature is enabled\n"); > + } else { > + av_log(avctx, AV_LOG_ERROR, "Ivalid argument: value string is NULL\n"); > + return XEVE_PARAM_BAD_VALUE; > + } > + } else if (strcmp(key, "vbv-bufsize") == 0 ) { > + cdsc->param.vbv_bufsize = kbps_str_to_int((char *)value); > + av_log(avctx, AV_LOG_INFO, "VBV buffer size: %dkbits\n", cdsc->param.vbv_bufsize); > + } else if (strcmp(key, "rc-type") == 0 ) { > + int rc_type = atoi(value); > + if(rc_type < 0 || rc_type > 2) { > + av_log(avctx, AV_LOG_ERROR, "Rate control type [ 0(rc_off) / 1(CBR) ] bad value: %d\n", rc_type); > + return XEVE_PARAM_BAD_VALUE; > + } > + cdsc->param.rc_type = rc_type; > + av_log(avctx, AV_LOG_INFO, "Rate control type [ 0(rc_off) / 1(CBR) ] : %d\n", rc_type); > + } else if (strcmp(key, "bframes") == 0 ) { > + int bframes = atoi(value); atoi has undefined behaviour if the value does not fit into an integer. > + if(bframes < 0) { > + av_log(avctx, AV_LOG_ERROR, "bframes: bad value: %d\n", bframes); > + return XEVE_PARAM_BAD_VALUE; > + } > + cdsc->param.bframes = bframes; > + av_log(avctx, AV_LOG_INFO, "bframes : %d\n", bframes); What is the point of having yet another way to specify bframes, profile_id etc.? If you look at libx264.c (or libx265.c), you'll see that we don't duplicate parsing for them for all the "x264-params" options; instead, we offload it to the library, so that we automatically support everything that the library supports. With this approach here meanwhile we would have to add support for every new addition to libxeve's API. > + } else if (strcmp(key, "profile") == 0 ) { > + const char *profile = value; > + int profile_id; > + av_log(avctx, AV_LOG_INFO, "profile (baseline, main): %s\n", profile); > + profile_id = get_profile_id(profile); > + if (profile_id < 0) { > + av_log(avctx, AV_LOG_ERROR, "Invalid xeve param: profile(%s)\n", profile); > + return XEVE_PARAM_BAD_VALUE; > + } > + xectx->profile_id = profile_id; > + } else if (strcmp(key, "preset") == 0 ) { > + const char *preset = value; > + int preset_id; > + av_log(avctx, AV_LOG_INFO, "Preset of xeve (fast, medium, slow, placebo): %s\n", preset); > + preset_id = get_preset_id(preset); > + if( preset_id < 0) { > + av_log(avctx, AV_LOG_ERROR, "Invalid xeve param: preset(%s)\n", preset); > + return XEVE_PARAM_BAD_VALUE; > + } > + xectx->preset_id = preset_id; > + } else if (strcmp(key, "tune") == 0 ) { > + const char *tune = value; > + int tune_id; > + av_log(avctx, AV_LOG_INFO, "Tune of xeve (psnr, zerolatency): %s\n", tune); > + tune_id = get_tune_id(tune); > + if( tune_id < 0) { > + av_log(avctx, AV_LOG_ERROR, "Invalid xeve param: tune(%s)\n", tune); > + return XEVE_PARAM_BAD_VALUE; > + } > + xectx->tune_id = tune_id; > + } else if (strcmp(key, "bitrate") == 0 ) { > + cdsc->param.bitrate = kbps_str_to_int((char *)value); > + av_log(avctx, AV_LOG_INFO, "Bitrate = %dkbps\n", cdsc->param.bitrate); > + } else if (strcmp(key, "q") == 0 || strcmp(key, "qp") == 0) { > + int qp = atoi(value); > + if(qp < 0 || qp > 51) { > + av_log(avctx, AV_LOG_ERROR, "Invalid QP value (0~51) :%d\n", qp); > + return XEVE_PARAM_BAD_VALUE; > + } > + cdsc->param.qp = qp; > + av_log(avctx, AV_LOG_INFO, "QP value (0~51): %d\n", cdsc->param.qp); > + } else { > + av_log(avctx, AV_LOG_ERROR, "Unknown xeve codec option: %s\n", key); > + return XEVE_PARAM_BAD_NAME; > + } > + > + return 0; > +} > + > +/** > + * The function returns a pointer to the object of the XEVE_CDSC type. > + * XEVE_CDSC contains all encoder parameters that should be initialized before the encoder is used. > + * > + * The field values of the XEVE_CDSC structure are populated based on: > + * - the corresponding field values of the AvCodecConetxt structure, > + * - the xeve encoder specific option values, > + * (the full list of options available for xeve encoder is displayed after executing the command ./ffmpeg --help encoder = libxeve) > + * - and the xeve encoder options specified as a list of key-value pairs following the xeve-params option > + * > + * The order of processing input data and populating the XEVE_CDSC structure > + * 1) first, the fields of the AVCodecContext structure corresponding to the provided input options are processed, > + * (i.e -pix_fmt yuv420p -s:v 1920x1080 -r 30 -profile:v 0) > + * 2) then xeve-specific options added as AVOption to the xeve AVCodec implementation > + * (i.e -threads_cnt 3 -preset 0) > + * 3) finally, the options specified after the xeve-params option as the parameter list of type key value are processed > + * (i.e -xeve-params "m=2:q=17") > + * > + * Keep in mind that, there are options that can be set in different ways. > + * In this case, please follow the above-mentioned order of processing. > + * The most recent assignments overwrite the previous values. > + * > + * @param[in] avctx codec context (AVCodecContext) > + * @param[out] cdsc contains all Xeve MPEG-5 EVC encoder encoder parameters that should be initialized before the encoder is use > + * > + * @return 0 on success, negative error code on failure > + */ > +static int get_conf(AVCodecContext *avctx, XEVE_CDSC *cdsc) > +{ > + XeveContext *xectx = NULL; > + int color_format; > + int cpu_count = av_cpu_count(); > + int ret; > + > + xectx = avctx->priv_data; > + xectx->hash = 0; > + > + /* initialize xeve_param struct with default values */ > + ret = xeve_param_default(&cdsc->param); > + if (XEVE_FAILED(ret)) { > + av_log(avctx, AV_LOG_ERROR, "Cannot set_default parameter\n"); > + return AVERROR_EXTERNAL; > + } > + > + /* read options from AVCodecContext */ > + if(avctx->width > 0) > + cdsc->param.w = xectx->width_luma = avctx->width; > + > + if(avctx->height > 0) > + cdsc->param.h = xectx->height_luma = avctx->height; > + > + if(avctx->framerate.num > 0) { > + // fps can be float number, but xeve API doesn't support it > + cdsc->param.fps = (int)(((float)avctx->framerate.num / avctx->framerate.den) + 0.5); > + } > + > + if(avctx->gop_size >= 0) { // GOP size (key-frame interval) > + cdsc->param.keyint = avctx->gop_size; // 0: only one I-frame at the first time; 1: every frame is coded in I-frame > + } > + > + if (avctx->max_b_frames == 0 || avctx->max_b_frames == 1 || avctx->max_b_frames == 3 || > + avctx->max_b_frames == 7 || avctx->max_b_frames == 15) { // number of b-frames > + cdsc->param.bframes = avctx->max_b_frames; > + } else { > + av_log(avctx, AV_LOG_ERROR, "Incorrect value for maximum number of B frames: (%d) \n" > + "Acceptable values for bf option (maximum number of B frames) are 0,1,3,7 or 15\n", avctx->max_b_frames); > + return AVERROR_INVALIDDATA; > + } > + > + if (avctx->level >= 0) > + cdsc->param.level_idc = avctx->level; > + > + ret = get_pix_fmt(avctx->pix_fmt, &color_format, &xectx->input_depth); > + if (ret != 0) { > + av_log(avctx, AV_LOG_ERROR, "Unsupported pixel format.\n"); > + return AVERROR_INVALIDDATA; > + } The pixel format has already been checked before, so this should not error out. Furthermore, get_pix_fmt() uses a different list of allowed pixel format than the earlier check. > + cdsc->param.cs = XEVE_CS_SET(color_format, xectx->input_depth, 0); > + > + if (avctx->rc_buffer_size > 0) { // VBV buf size > + cdsc->param.vbv_bufsize = (int)(avctx->rc_buffer_size / 1000); > + } > + > + // rc_type: Rate control type [ 0(CQP) / 1(ABR) / 2(CRF) ] > + if (avctx->bit_rate > 0) { > + if (avctx->bit_rate / 1000 > INT_MAX || avctx->rc_max_rate / 1000 > INT_MAX) { > + av_log(avctx, AV_LOG_ERROR, "Not supported bitrate bit_rate and rc_max_rate > %d000\n", INT_MAX); > + return AVERROR_INVALIDDATA; > + } > + cdsc->param.bitrate = (int)(avctx->bit_rate / 1000); > + cdsc->param.rc_type = XEVE_RC_ABR; > + } > + > + if (xectx->op_crf >= 0) { > + cdsc->param.crf = xectx->op_crf; > + cdsc->param.rc_type = XEVE_RC_CRF; > + } > + > + if(avctx->thread_count <= 0) { Move cpu_count to this block; this increases locality and avoids calling av_cpu_count() unnecessarily. > + cdsc->param.threads = (cpu_count < XEVE_MAX_THREADS) ? cpu_count : XEVE_MAX_THREADS; > + } else if(avctx->thread_count > XEVE_MAX_THREADS) > + cdsc->param.threads = XEVE_MAX_THREADS; > + else > + cdsc->param.threads = avctx->thread_count; > + > + cdsc->param.cs = XEVE_CS_SET(color_format, cdsc->param.codec_bit_depth, 0); > + cdsc->max_bs_buf_size = MAX_BS_BUF; > + > + if(avctx->profile == FF_PROFILE_EVC_BASELINE) > + xectx->profile_id = XEVE_PROFILE_BASELINE; > + else if(avctx->profile == FF_PROFILE_EVC_MAIN) > + xectx->profile_id = XEVE_PROFILE_MAIN; > + else { > + av_log(avctx, AV_LOG_ERROR, "Unknown encoder profile (%d)\n" > + "Acceptable values for profile option are 0 and 1 (0: baseline profile; 1: main profile)\n", avctx->profile); > + return AVERROR_INVALIDDATA; > + } > + > + if (xectx->op_preset) { // preset > + xectx->preset_id = get_preset_id(xectx->op_preset); > + } > + > + if (xectx->op_tune) { // tune > + xectx->tune_id = get_tune_id(xectx->op_tune); > + } > + > + ret = xeve_param_ppt(&cdsc->param, xectx->profile_id, xectx->preset_id, xectx->tune_id); > + if (XEVE_FAILED(ret)) { > + av_log(avctx, AV_LOG_ERROR, "Cannot set profile(%d), preset(%d), tune(%d)\n", xectx->profile_id, xectx->preset_id, xectx->tune_id); > + return AVERROR_EXTERNAL; > + } > + > + /* parse : separated list of key=value parameters and set values for created descriptor (XEVE_CDSC) */ > + { > + AVDictionaryEntry *en = NULL; const AVDictionaryEntry *en > + > + // Start to parse xeve_params > + while ((en = av_dict_get(xectx->xeve_params, "", en, AV_DICT_IGNORE_SUFFIX))) { > + int parse_ret = parse_xeve_params(avctx, en->key, en->value, cdsc); > + > + switch (parse_ret) { > + case XEVE_PARAM_BAD_NAME: > + av_log(avctx, AV_LOG_WARNING, "Unknown option: %s.\n", en->key); > + break; > + case XEVE_PARAM_BAD_VALUE: > + av_log(avctx, AV_LOG_WARNING, "Invalid value for %s: %s.\n", en->key, en->value); > + break; > + default: > + break; > + } > + } > + } > + > + return 0; > +} > + > +/** > + * Check encoder configuration > + * > + * @param[in] avctx context for logger > + * @param[in] cdsc contains all encoder parameters that should be initialized before its use > + * > + * @return 0 on success, negative error code on failure > + */ > +static int check_conf(AVCodecContext *avctx, XEVE_CDSC *cdsc) > +{ > + int ret = 0; > + int min_block_size = 4; > + int pic_m; > + > + if(cdsc->param.profile == XEVE_PROFILE_BASELINE) { > + if (cdsc->param.tool_amvr == 1) { > + av_log(avctx, AV_LOG_ERROR, "AMVR cannot be on in base profile\n"); > + ret = -1; > + } > + if (cdsc->param.tool_mmvd == 1) { > + av_log(avctx, AV_LOG_ERROR, "MMVD cannot be on in base profile\n"); > + ret = -1; > + } > + if (cdsc->param.tool_affine == 1) { > + av_log(avctx, AV_LOG_ERROR, "Affine cannot be on in base profile\n"); > + ret = -1; > + } > + if (cdsc->param.tool_dmvr == 1) { > + av_log(avctx, AV_LOG_ERROR, "DMVR cannot be on in base profile\n"); > + ret = -1; > + } > + if (cdsc->param.tool_admvp == 1) { > + av_log(avctx, AV_LOG_ERROR, "ADMVP cannot be on in base profile\n"); > + ret = -1; > + } > + if (cdsc->param.tool_hmvp == 1) { > + av_log(avctx, AV_LOG_ERROR, "HMVP cannot be on in base profile\n"); > + ret = -1; > + } > + if (cdsc->param.tool_addb == 1) { > + av_log(avctx, AV_LOG_ERROR, "ADDB cannot be on in base profile\n"); > + ret = -1; > + } > + if (cdsc->param.tool_alf == 1) { > + av_log(avctx, AV_LOG_ERROR, "ALF cannot be on in base profile\n"); > + ret = -1; > + } > + if (cdsc->param.tool_htdf == 1) { > + av_log(avctx, AV_LOG_ERROR, "HTDF cannot be on in base profile\n"); > + ret = -1; > + } > + if (cdsc->param.btt == 1) { > + av_log(avctx, AV_LOG_ERROR, "BTT cannot be on in base profile\n"); > + ret = -1; > + } > + if (cdsc->param.suco == 1) { > + av_log(avctx, AV_LOG_ERROR, "SUCO cannot be on in base profile\n"); > + ret = -1; > + } > + if (cdsc->param.tool_eipd == 1) { > + av_log(avctx, AV_LOG_ERROR, "EIPD cannot be on in base profile\n"); > + ret = -1; > + } > + if (cdsc->param.tool_iqt == 1) { > + av_log(avctx, AV_LOG_ERROR, "IQT cannot be on in base profile\n"); > + ret = -1; > + } > + if (cdsc->param.tool_cm_init == 1) { > + av_log(avctx, AV_LOG_ERROR, "CM_INIT cannot be on in base profile\n"); > + ret = -1; > + } > + if (cdsc->param.tool_adcc == 1) { > + av_log(avctx, AV_LOG_ERROR, "ADCC cannot be on in base profile\n"); > + ret = -1; > + } > + if (cdsc->param.tool_ats == 1) { > + av_log(avctx, AV_LOG_ERROR, "ATS_INTRA cannot be on in base profile\n"); > + ret = -1; > + } > + if (cdsc->param.ibc_flag == 1) { > + av_log(avctx, AV_LOG_ERROR, "IBC cannot be on in base profile\n"); > + ret = -1; > + } > + if (cdsc->param.tool_rpl == 1) { > + av_log(avctx, AV_LOG_ERROR, "RPL cannot be on in base profile\n"); > + ret = -1; > + } > + if (cdsc->param.tool_pocs == 1) { > + av_log(avctx, AV_LOG_ERROR, "POCS cannot be on in base profile\n"); > + ret = -1; > + } > + } else { > + if (cdsc->param.tool_admvp == 0 && cdsc->param.tool_affine == 1) { > + av_log(avctx, AV_LOG_ERROR, "AFFINE cannot be on when ADMVP is off\n"); > + ret = -1; > + } > + if (cdsc->param.tool_admvp == 0 && cdsc->param.tool_amvr == 1) { > + av_log(avctx, AV_LOG_ERROR, "AMVR cannot be on when ADMVP is off\n"); > + ret = -1; > + } > + if (cdsc->param.tool_admvp == 0 && cdsc->param.tool_dmvr == 1) { > + av_log(avctx, AV_LOG_ERROR, "DMVR cannot be on when ADMVP is off\n"); > + ret = -1; > + } > + if (cdsc->param.tool_admvp == 0 && cdsc->param.tool_mmvd == 1) { > + av_log(avctx, AV_LOG_ERROR, "MMVD cannot be on when ADMVP is off\n"); > + ret = -1; > + } > + if (cdsc->param.tool_eipd == 0 && cdsc->param.ibc_flag == 1) { > + av_log(avctx, AV_LOG_ERROR, "IBC cannot be on when EIPD is off\n"); > + ret = -1; > + } > + if (cdsc->param.tool_iqt == 0 && cdsc->param.tool_ats == 1) { > + av_log(avctx, AV_LOG_ERROR, "ATS cannot be on when IQT is off\n"); > + ret = -1; > + } > + if (cdsc->param.tool_cm_init == 0 && cdsc->param.tool_adcc == 1) { > + av_log(avctx, AV_LOG_ERROR, "ADCC cannot be on when CM_INIT is off\n"); > + ret = -1; > + } > + } > + > + if (cdsc->param.btt == 1) { > + if (cdsc->param.framework_cb_max && cdsc->param.framework_cb_max < 5) { > + av_log(avctx, AV_LOG_ERROR, "Maximun Coding Block size cannot be smaller than 5\n"); > + ret = -1; > + } > + if (cdsc->param.framework_cb_max > 7) { > + av_log(avctx, AV_LOG_ERROR, "Maximun Coding Block size cannot be greater than 7\n"); > + ret = -1; > + } > + if (cdsc->param.framework_cb_min && cdsc->param.framework_cb_min < 2) { > + av_log(avctx, AV_LOG_ERROR, "Minimum Coding Block size cannot be smaller than 2\n"); > + ret = -1; > + } > + if ((cdsc->param.framework_cb_max || cdsc->param.framework_cb_min) && > + cdsc->param.framework_cb_min > cdsc->param.framework_cb_max) { > + av_log(avctx, AV_LOG_ERROR, "Minimum Coding Block size cannot be greater than Maximum coding Block size\n"); > + ret = -1; > + } > + if (cdsc->param.framework_cu14_max > 6) { > + av_log(avctx, AV_LOG_ERROR, "Maximun 1:4 Coding Block size cannot be greater than 6\n"); > + ret = -1; > + } > + if ((cdsc->param.framework_cb_max || cdsc->param.framework_cu14_max) && > + cdsc->param.framework_cu14_max > cdsc->param.framework_cb_max) { > + av_log(avctx, AV_LOG_ERROR, "Maximun 1:4 Coding Block size cannot be greater than Maximum coding Block size\n"); > + ret = -1; > + } > + if (cdsc->param.framework_tris_max > 6) { > + av_log(avctx, AV_LOG_ERROR, "Maximun Tri-split Block size be greater than 6\n"); > + ret = -1; > + } > + if ((cdsc->param.framework_tris_max || cdsc->param.framework_cb_max) && > + cdsc->param.framework_tris_max > cdsc->param.framework_cb_max) { > + av_log(avctx, AV_LOG_ERROR, "Maximun Tri-split Block size cannot be greater than Maximum coding Block size\n"); > + ret = -1; > + } > + if ((cdsc->param.framework_tris_min || cdsc->param.framework_cb_min) && > + cdsc->param.framework_tris_min < cdsc->param.framework_cb_min + 2) { > + av_log(avctx, AV_LOG_ERROR, "Maximun Tri-split Block size cannot be smaller than Minimum Coding Block size plus two\n"); > + ret = -1; > + } > + if(cdsc->param.framework_cb_min) min_block_size = 1 << cdsc->param.framework_cb_min; > + else min_block_size = 8; > + } > + > + if (cdsc->param.suco == 1) { > + if (cdsc->param.framework_suco_max > 6) { > + av_log(avctx, AV_LOG_ERROR, "Maximun SUCO size cannot be greater than 6\n"); > + ret = -1; > + } > + if (cdsc->param.framework_cb_max && cdsc->param.framework_suco_max > cdsc->param.framework_cb_max) { > + av_log(avctx, AV_LOG_ERROR, "Maximun SUCO size cannot be greater than Maximum coding Block size\n"); > + ret = -1; > + } > + if (cdsc->param.framework_suco_min < 4) { > + av_log(avctx, AV_LOG_ERROR, "Minimun SUCO size cannot be smaller than 4\n"); > + ret = -1; > + } > + if (cdsc->param.framework_cb_min && cdsc->param.framework_suco_min < cdsc->param.framework_cb_min) { > + av_log(avctx, AV_LOG_ERROR, "Minimun SUCO size cannot be smaller than Minimum coding Block size\n"); > + ret = -1; > + } > + if (cdsc->param.framework_suco_min > cdsc->param.framework_suco_max) { > + av_log(avctx, AV_LOG_ERROR, "Minimum SUCO size cannot be greater than Maximum SUCO size\n"); > + ret = -1; > + } > + } > + > + pic_m = (8 > min_block_size) ? min_block_size : 8; > + if ((cdsc->param.w & (pic_m - 1)) != 0) { > + av_log(avctx, AV_LOG_ERROR, "Current encoder does not support picture width, not multiple of max(8, minimum CU size)\n"); > + ret = -1; > + } > + if ((cdsc->param.h & (pic_m - 1)) != 0) { > + av_log(avctx, AV_LOG_ERROR, "Current encoder does not support picture height, not multiple of max(8, minimum CU size)\n"); > + ret = -1; > + } > + > + if(ret == -1) { > + return AVERROR_INVALIDDATA; > + } > + > + return 0; > +} As has already been said last time: Why does this function exist? Why can't we just pass the parameters to the library and let it error out and return an error message in case the parameters are inconsistent or out-of-spec? > + > +/** > + * Set XEVE_CFG_SET_USE_PIC_SIGNATURE for encoder > + * > + * @param[in] logger context > + * @param[in] id XEVE encodec instance identifier > + * @param[in] ctx the structure stores all the states associated with the instance of Xeve MPEG-5 EVC encoder > + * > + * @return 0 on success, negative error code on failure > + */ > +static int set_extra_config(AVCodecContext* avctx, XEVE id, XeveContext *ctx) > +{ > + int ret, size, value; > + size = 4; > + > + // embed SEI messages identifying encoder parameters and command line arguments > + // - 0: off\n" > + // - 1: emit sei info" > + // > + // SEI - Supplemental enhancement information contains information > + // that is not necessary to decode the samples of coded pictures from VCL NAL units. > + // Some SEI message information is required to check bitstream conformance > + // and for output timing decoder conformance. > + // @see ISO_IEC_23094-1_2020 7.4.3.5 > + // @see ISO_IEC_23094-1_2020 Annex D > + ret = xeve_config(id, XEVE_CFG_SET_SEI_CMD, &value, &size); // sei_cmd_info > + if (XEVE_FAILED(ret)) > + { > + av_log(avctx, AV_LOG_ERROR, "Failed to set config for sei command info messages\n"); > + return AVERROR_EXTERNAL; > + } > + > + if(ctx->hash) { > + value = 1; > + ret = xeve_config(id, XEVE_CFG_SET_USE_PIC_SIGNATURE, &value, &size); > + if(XEVE_FAILED(ret)) { > + av_log(avctx, AV_LOG_ERROR, "Failed to set config for picture signature\n"); > + return AVERROR_EXTERNAL; > + } > + } > + > + return 0; > +} > + > +/** > + * Convert FFmpeg pixel format (AVPixelFormat) into XEVE pre-defined color space > + * > + * @param[in] px_fmt pixel format (@see https://ffmpeg.org/doxygen/trunk/pixfmt_8h.html#a9a8e335cf3be472042bc9f0cf80cd4c5) > + * > + * @return XEVE pre-defined color space (@see xeve.h) on success, XEVE_CF_UNKNOWN on failure > + */ > +static int xeve_color_space(enum AVPixelFormat pix_fmt) > +{ > + /* color space of input image */ > + int cs = XEVE_CF_UNKNOWN; > + > + switch (pix_fmt) { > + case AV_PIX_FMT_YUV420P: > + cs = XEVE_CS_YCBCR420; > + break; > + case AV_PIX_FMT_YUV420P10: > +#if AV_HAVE_BIGENDIAN > + cs = XEVE_CS_SET(XEVE_CF_YCBCR420, 10, 1); > +#else > + cs = XEVE_CS_YCBCR420_10LE; > +#endif > + > + break; > + case AV_PIX_FMT_YUV420P12: > +#if AV_HAVE_BIGENDIAN > + cs = XEVE_CS_SET(XEVE_CF_YCBCR420, 12, 1); > +#else > + cs = XEVE_CS_YCBCR420_12LE; > +#endif > + > + break; > + case AV_PIX_FMT_YUV422P: > + cs = XEVE_CS_YCBCR422; > + break; > + case AV_PIX_FMT_YUV422P10: > +#if AV_HAVE_BIGENDIAN > + cs = XEVE_CS_SET(XEVE_CF_YCBCR422, 10, 1); > +#else > + cs = XEVE_CS_YCBCR422_10LE; > +#endif > + > + break; > + case AV_PIX_FMT_YUV422P12: > +#if AV_HAVE_BIGENDIAN > + cs = XEVE_CS_SET(XEVE_CF_YCBCR422, 12, 1); > +#else > + cs = XEVE_CS_SET(XEVE_CF_YCBCR422, 12, 0); > +#endif > + > + break; > + case AV_PIX_FMT_GBRP: > + case AV_PIX_FMT_GBRP10: > + case AV_PIX_FMT_GBRP12: > + cs = XEVE_CF_UNKNOWN; > + break; > + case AV_PIX_FMT_YUV444P: > + cs = XEVE_CF_YCBCR444; > + break; > + case AV_PIX_FMT_YUV444P10: > +#if AV_HAVE_BIGENDIAN > + cs = XEVE_CS_SET(XEVE_CF_YCBCR444, 10, 1); > +#else > + cs = XEVE_CS_YCBCR444_10LE; > +#endif > + > + break; > + case AV_PIX_FMT_YUV444P12: > +#if AV_HAVE_BIGENDIAN > + cs = XEVE_CS_SET(XEVE_CF_YCBCR444, 12, 1); > +#else > + cs = XEVE_CS_SET(XEVE_CF_YCBCR444, 12, 0); > +#endif > + > + break; > + case AV_PIX_FMT_GRAY8: > + cs = XEVE_CF_YCBCR400; > + break; > + case AV_PIX_FMT_GRAY10: > +#if AV_HAVE_BIGENDIAN > + cs = XEVE_CS_SET(XEVE_CF_YCBCR400, 10, 1); > +#else > + cs = XEVE_CS_YCBCR400_10LE; > +#endif > + > + break; > + case AV_PIX_FMT_GRAY12: > +#if AV_HAVE_BIGENDIAN > + cs = XEVE_CS_SET(XEVE_CF_YCBCR400, 12, 1); > +#else > + cs = XEVE_CS_YCBCR400_12LE; > +#endif > + > + break; > + default: > + cs = XEVE_CF_UNKNOWN; > + break; > + } > + > + return cs; > +} > + > +/** > + * @brief Switch encoder to bumping mode > + * > + * @param id XEVE encodec instance identifier > + * @return 0 on success, negative error code on failure > + */ > +static int setup_bumping(XEVE id) > +{ > + int val, size; > + val = 1; > + size = sizeof(int); Can be initialized directly. > + if(XEVE_FAILED(xeve_config(id, XEVE_CFG_SET_FORCE_OUT, (void *)(&val), &size))) > + return AVERROR_EXTERNAL; > + > + return 0; > +} > + > +/** > + * @brief Initialize eXtra-fast Essential Video Encoder codec > + * Create an encoder instance and allocate all the needed resources > + * > + * @param avctx codec context > + * @return 0 on success, negative error code on failure > + */ > +static av_cold int libxeve_init(AVCodecContext *avctx) > +{ > + XeveContext *xectx = avctx->priv_data; > + unsigned char *bs_buf = NULL; > + int i; > + int shift_h = 0; > + int shift_v = 0; > + XEVE_IMGB *imgb = NULL; > + int ret = 0; > + > + XEVE_CDSC *cdsc = &(xectx->cdsc); > + > + if(avctx->pix_fmt != AV_PIX_FMT_YUV420P && avctx->pix_fmt != AV_PIX_FMT_YUV420P10) { > + av_log(avctx, AV_LOG_ERROR, "Invalid pixel format: %s\n", av_get_pix_fmt_name(avctx->pix_fmt)); > + ret = AVERROR_INVALIDDATA; > + goto ERR; 1. This would be AVERROR(EINVAL), not AVERROR_INVALIDDATA. 2. Use AVCodec.pix_fmts instead. 3. get_pix_fmt() and xeve_color_space() have a different set of supported pixel formats. > + } > + > + /* allocate bitstream buffer */ > + bs_buf = (unsigned char *)av_malloc(MAX_BS_BUF); The cast is unnecessary -- this is not C++. > + if(bs_buf == NULL) { > + av_log(avctx, AV_LOG_ERROR, "Cannot allocate bitstream buffer\n"); return AVERROR(ENOMEM) is enough. > + ret = AVERROR(ENOMEM); > + goto ERR; > + } > + > + /* read configurations and set values for created descriptor (XEVE_CDSC) */ > + if ((ret = get_conf(avctx, cdsc)) != 0) { > + av_log(avctx, AV_LOG_ERROR, "Cannot get configuration\n"); > + goto ERR; > + } > + > + if ((ret = check_conf(avctx, cdsc)) != 0) { > + av_log(avctx, AV_LOG_ERROR, "Invalid configuration\n"); > + goto ERR; > + } > + > + /* create encoder */ > + xectx->id = xeve_create(cdsc, NULL); > + if(xectx->id == NULL) { > + av_log(avctx, AV_LOG_ERROR, "Cannot create XEVE encoder\n"); > + ret = AVERROR_EXTERNAL; > + goto ERR; > + } > + > + if((ret = set_extra_config(avctx, xectx->id, xectx))!=0) { > + av_log(avctx, AV_LOG_ERROR, "Cannot set extra configuration\n"); > + goto ERR; > + } > + > + xectx->bitb.addr = bs_buf; > + xectx->bitb.bsize = MAX_BS_BUF; > + > + if((ret = av_pix_fmt_get_chroma_sub_sample(avctx->pix_fmt, &shift_h, &shift_v)) != 0) { > + av_log(avctx, AV_LOG_ERROR, "Failed to get chroma shift\n"); > + goto ERR; > + } > + > + // YUV format explanation > + // shift_h == 1 && shift_v == 1 : YUV420 > + // shift_h == 1 && shift_v == 0 : YUV422 > + // shift_h == 0 && shift_v == 0 : YUV444 > + // > + xectx->width_chroma = AV_CEIL_RSHIFT(xectx->width_luma, shift_h); > + xectx->height_chroma = AV_CEIL_RSHIFT(xectx->height_luma, shift_v); > + > + /* set default values for input image buffer */ > + imgb = &xectx->imgb; > + imgb->cs = xeve_color_space(avctx->pix_fmt); > + imgb->np = 3; /* only for yuv420p, yuv420ple */ > + > + for (i = 0; i < imgb->np; i++) > + imgb->x[i] = imgb->y[i] = 0; > + > + imgb->w[0] = imgb->aw[0] = xectx->width_luma; > + imgb->w[1] = imgb->w[2] = imgb->aw[1] = imgb->aw[2] = xectx->width_chroma; > + imgb->h[0] = imgb->ah[0] = xectx->height_luma; > + imgb->h[1] = imgb->h[2] = imgb->ah[1] = imgb->ah[2] = xectx->height_chroma; > + > + xectx->encod_frames = 0; > + xectx->bytes_total = 0; > + xectx->state = STATE_ENCODING; > + xectx->packet_count = 0; > + xectx->bitrate = 0; > + > + return 0; > + > +ERR: > + if(bs_buf) > + av_free(bs_buf); It seems to me that you are missing a lot of cleanup here; it would be better to make libxeve_close() handle the case of a partially initialized context and let it take over cleanup automatically by setting the caps_internal FF_CODEC_CAP_INIT_CLEANUP. > + > + return ret; > +} > + > +/** > + * Encode raw data frame into EVC packet > + * > + * @param[in] avctx codec context > + * @param[out] pkt output AVPacket containing encoded data > + * @param[in] frame AVFrame containing the raw data to be encoded > + * @param[out] got_packet encoder sets to 0 or 1 to indicate that a > + * non-empty packet was returned in pkt > + * > + * @return 0 on success, negative error code on failure > + */ > +static int libxeve_encode(AVCodecContext *avctx, AVPacket *avpkt, > + const AVFrame *frame, int *got_packet) > +{ > + XeveContext *xectx = NULL; > + int ret = -1; > + int xeve_cs; > + > + xectx = avctx->priv_data; This can be directly initialized above. > + > + if(xectx->state == STATE_SKIPPING && frame ) { > + xectx->state = STATE_ENCODING; // Entering encoding process > + } else if(xectx->state == STATE_ENCODING && frame == NULL) { > + if (setup_bumping(xectx->id) == 0) > + xectx->state = STATE_BUMPING; // Entering bumping process > + else { > + av_log(avctx, AV_LOG_ERROR, "Failed to setup bumping\n"); > + xectx->state = STATE_SKIPPING; > + } > + } > + > + if(xectx->state == STATE_ENCODING) { > + const AVPixFmtDescriptor *pixel_fmt_desc = av_pix_fmt_desc_get (frame->format); > + if(!pixel_fmt_desc) { > + av_log(avctx, AV_LOG_ERROR, "Invalid pixel format descriptor for pixel format: %s\n", av_get_pix_fmt_name(avctx->pix_fmt)); > + return AVERROR_INVALIDDATA; > + } > + > + xeve_cs = xeve_color_space(avctx->pix_fmt); > + if(xeve_cs != XEVE_CS_YCBCR420 && xeve_cs != XEVE_CS_YCBCR420_10LE) { > + av_log(avctx, AV_LOG_ERROR, "Invalid pixel format: %s\n", av_get_pix_fmt_name(avctx->pix_fmt)); > + return AVERROR_INVALIDDATA; > + } > + > + { > + int i; > + XEVE_IMGB *imgb = NULL; > + > + imgb = &xectx->imgb; > + > + for (i = 0; i < imgb->np; i++) { > + imgb->a[i] = frame->data[i]; > + imgb->s[i] = frame->linesize[i]; > + } > + > + if(xectx->id == NULL) { > + av_log(avctx, AV_LOG_ERROR, "Invalid XEVE encoder\n"); > + return AVERROR_INVALIDDATA; > + } > + > + imgb->ts[0] = frame->pts; > + imgb->ts[1] = 0; > + > + /* push image to encoder */ > + ret = xeve_push(xectx->id, imgb); > + if(XEVE_FAILED(ret)) { > + av_log(avctx, AV_LOG_ERROR, "xeve_push() failed\n"); > + return AVERROR_EXTERNAL; > + } > + } > + } > + if(xectx->state == STATE_ENCODING || xectx->state == STATE_BUMPING) { > + > + /* encoding */ > + ret = xeve_encode(xectx->id, &(xectx->bitb), &(xectx->stat)); > + if(XEVE_FAILED(ret)) { > + av_log(avctx, AV_LOG_ERROR, "xeve_encode() failed\n"); > + return AVERROR_EXTERNAL; > + } > + > + xectx->encod_frames++; > + > + /* store bitstream */ > + if (ret == XEVE_OK_OUT_NOT_AVAILABLE) { // Return OK but picture is not available yet > + *got_packet = 0; > + return 0; > + } else if(ret == XEVE_OK) { > + int av_pic_type; > + > + if(xectx->stat.write > 0) { > + xectx->bytes_total += xectx->stat.write; > + > + ret = ff_get_encode_buffer(avctx, avpkt, xectx->stat.write, AV_GET_ENCODE_BUFFER_FLAG_REF); This encoder does not keep a reference to avpkt's data, so AV_GET_ENCODE_BUFFER_FLAG_REF should be removed. > + if (ret < 0) { > + av_log(avctx, AV_LOG_ERROR, "Can't allocate memory for AVPacket data\n"); ff_get_encode_buffer already emits an error message of its own. > + return ret; > + } > + > + memcpy(avpkt->data, xectx->bitb.addr, xectx->stat.write); > + > + avpkt->pts = xectx->bitb.ts[0]; > + avpkt->dts = xectx->bitb.ts[1]; > + > + xectx->bitrate += (xectx->stat.write - xectx->stat.sei_size); > + > + switch(xectx->stat.stype) { > + case XEVE_ST_I: > + av_pic_type = AV_PICTURE_TYPE_I; > + avpkt->flags |= AV_PKT_FLAG_KEY; > + break; > + case XEVE_ST_P: > + av_pic_type = AV_PICTURE_TYPE_P; > + break; > + case XEVE_ST_B: > + av_pic_type = AV_PICTURE_TYPE_B; > + break; > + case XEVE_ST_UNKNOWN: > + av_log(avctx, AV_LOG_ERROR, "Unknown slice type\n"); > + return AVERROR_INVALIDDATA; > + } > + > + ff_side_data_set_encoder_stats(avpkt, xectx->stat.qp * FF_QP2LAMBDA, NULL, 0, av_pic_type); > + > + xectx->bitrate += (xectx->stat.write - xectx->stat.sei_size); > + > + *got_packet = 1; > + xectx->packet_count++; > + } > + } else if (ret == XEVE_OK_NO_MORE_FRM) { > + // Return OK but no more frames > + return 0; > + } else { > + av_log(avctx, AV_LOG_ERROR, "Invalid return value: %d\n", ret); > + return AVERROR_EXTERNAL; > + } > + } else { > + av_log(avctx, AV_LOG_ERROR, "Udefined encoder state\n"); > + return AVERROR_INVALIDDATA; > + } > + > + return 0; > +} > + > +/** > + * Destroy the encoder and release all the allocated resources > + * > + * @param avctx codec context > + * @return 0 on success, negative error code on failure > + */ > +static av_cold int libxeve_close(AVCodecContext *avctx) > +{ > + XeveContext *xectx = avctx->priv_data; > + > + xeve_delete(xectx->id); > + > + if(xectx->bitb.addr) > + av_free(xectx->bitb.addr); /* release bitstream buffer */ av_freep(). And remove the check for whether the pointer is != NULL as our freeing functions can handle this case just fine. > + > + return 0; > +} > + > +#define OFFSET(x) offsetof(XeveContext, x) > +#define VE AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_ENCODING_PARAM > + > +// Example of using: ./ffmpeg -xeve-params "m=2:q=17" > +// Consider using following options (./ffmpeg --help encoder=libxeve) > +// > +static const AVOption xeve_options[] = { > + { "preset", "Encoding preset for setting encoding speed [fast, medium, slow, placebo]", OFFSET(op_preset), AV_OPT_TYPE_STRING, { .str = "medium" }, 0, 0, VE }, > + { "tune", "Tuneing parameter for special purpose operation [psnr, zerolatency]", OFFSET(op_tune), AV_OPT_TYPE_STRING, { 0 }, 0, 0, VE}, > + { "qp", "quantization parameter qp <0..51> [default: 32]", OFFSET(op_qp), AV_OPT_TYPE_INT, { .i64 = 32 }, 0, 51, VE }, > + { "crf", "constant rate factor <-1..51> [default: 32]", OFFSET(op_crf), AV_OPT_TYPE_INT, { .i64 = -1 }, -1, 51, VE }, > + { "xeve-params", "override the xeve configuration using a : separated list of key=value parameters", OFFSET(xeve_params), AV_OPT_TYPE_DICT, { 0 }, 0, 0, VE }, > + { NULL } > +}; > + > +static const AVClass xeve_class = { > + .class_name = "libxeve", > + .item_name = av_default_item_name, > + .option = xeve_options, > + .version = LIBAVUTIL_VERSION_INT, > +}; > + > +/** > + * libavcodec generic global options, which can be set on all the encoders and decoders > + * @see https://www.ffmpeg.org/ffmpeg-codecs.html#Codec-Options > + */ > +static const FFCodecDefault xeve_defaults[] = { > + { "b", "0" }, // bitrate > + { "g", "0" }, // gop_size (key-frame interval 0: only one I-frame at the first time; 1: every frame is coded in I-frame) > + { "bf", "15"}, // bframes (0: no B-frames) > + { "profile", "0"}, // encoder codec profile (0: baselie; 1: main) > + { "threads", "0"}, // number of threads to be used (0: automatically select the number of threads to set) > + { NULL }, > +}; > + > +const FFCodec ff_libxeve_encoder = { > + .p.name = "libxeve", > + .p.long_name = NULL_IF_CONFIG_SMALL("libxeve MPEG-5 EVC"), > + .p.type = AVMEDIA_TYPE_VIDEO, > + .p.id = AV_CODEC_ID_EVC, > + .init = libxeve_init, > + FF_CODEC_ENCODE_CB(libxeve_encode), > + .close = libxeve_close, > + .priv_data_size = sizeof(XeveContext), > + .p.priv_class = &xeve_class, > + .defaults = xeve_defaults, > + .p.capabilities = AV_CODEC_CAP_DELAY | AV_CODEC_CAP_AUTO_THREADS | AV_CODEC_CAP_DR1, > + .p.wrapper_name = "libxeve", > +}; > diff --git a/libavcodec/parsers.c b/libavcodec/parsers.c > index 6b40c18d80..aa49ce1c9a 100644 > --- a/libavcodec/parsers.c > +++ b/libavcodec/parsers.c > @@ -41,6 +41,7 @@ extern const AVCodecParser ff_dvaudio_parser; > extern const AVCodecParser ff_dvbsub_parser; > extern const AVCodecParser ff_dvdsub_parser; > extern const AVCodecParser ff_dvd_nav_parser; > +extern const AVCodecParser ff_evc_parser; > extern const AVCodecParser ff_flac_parser; > extern const AVCodecParser ff_g723_1_parser; > extern const AVCodecParser ff_g729_parser; _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 6+ messages in thread
[parent not found: <CGME20220420123208eucas1p1359df4487ab7f4a7edde29b4f80cdfdc@eucas1p1.samsung.com>]
* [FFmpeg-devel] [PATCH 2/3] Provided support for MPEG-5 EVC (Essential Video Coding) codec [not found] <CGME20220420123208eucas1p1359df4487ab7f4a7edde29b4f80cdfdc@eucas1p1.samsung.com> @ 2022-04-20 12:32 ` Dawid Kozinski 0 siblings, 0 replies; 6+ messages in thread From: Dawid Kozinski @ 2022-04-20 12:32 UTC (permalink / raw) To: ffmpeg-devel - Added xeve encoder wrapper - Added xevd dencoder wrapper - Added documentation for xeve and xevd wrappers - Added parser for EVC format - Changes in project configuration file and libavcodec Makefile Signed-off-by: Dawid Kozinski <d.kozinski@samsung.com> --- configure | 10 + doc/decoders.texi | 23 + doc/encoders.texi | 112 ++++ doc/general_contents.texi | 19 + libavcodec/Makefile | 3 + libavcodec/allcodecs.c | 3 + libavcodec/avcodec.h | 3 + libavcodec/evc_parser.c | 514 +++++++++++++++++ libavcodec/libxevd.c | 445 +++++++++++++++ libavcodec/libxeve.c | 1141 +++++++++++++++++++++++++++++++++++++ libavcodec/parsers.c | 1 + 11 files changed, 2274 insertions(+) create mode 100644 libavcodec/evc_parser.c create mode 100644 libavcodec/libxevd.c create mode 100644 libavcodec/libxeve.c diff --git a/configure b/configure index 358a614854..48e0bf79ef 100755 --- a/configure +++ b/configure @@ -289,6 +289,8 @@ External library support: --enable-libwebp enable WebP encoding via libwebp [no] --enable-libx264 enable H.264 encoding via x264 [no] --enable-libx265 enable HEVC encoding via x265 [no] + --enable-libxeve enable MPEG-5 Essential Video Coding (EVC) encoding via libxeve [no] + --enable-libxevd enable MPEG-5 Essential Video Coding (EVC) decoding via libxevd [no] --enable-libxavs enable AVS encoding via xavs [no] --enable-libxavs2 enable AVS2 encoding via xavs2 [no] --enable-libxcb enable X11 grabbing using XCB [autodetect] @@ -1881,6 +1883,8 @@ EXTERNAL_LIBRARY_LIST=" openssl pocketsphinx vapoursynth + libxeve + libxevd " HWACCEL_AUTODETECT_LIBRARY_LIST=" @@ -2455,6 +2459,7 @@ CONFIG_EXTRA=" h264pred h264qpel hevcparse + evcparse hpeldsp huffman huffyuvdsp @@ -3252,6 +3257,7 @@ mpegaudio_parser_select="mpegaudioheader" mpegvideo_parser_select="mpegvideo" mpeg4video_parser_select="h263dsp mpegvideodec qpeldsp" vc1_parser_select="vc1dsp" +evc_parser_select="evcparse" # bitstream_filters aac_adtstoasc_bsf_select="adts_header mpeg4audio" @@ -3376,6 +3382,8 @@ libx264_encoder_select="atsc_a53" libx264rgb_encoder_deps="libx264" libx264rgb_encoder_select="libx264_encoder" libx265_encoder_deps="libx265" +libxeve_encoder_deps="libxeve" +libxevd_decoder_deps="libxevd" libxavs_encoder_deps="libxavs" libxavs2_encoder_deps="libxavs2" libxvid_encoder_deps="libxvid" @@ -6651,6 +6659,8 @@ enabled libx264 && { check_pkg_config libx264 x264 "stdint.h x264.h" x check_cpp_condition libx262 x264.h "X264_MPEG2" enabled libx265 && require_pkg_config libx265 x265 x265.h x265_api_get && require_cpp_condition libx265 x265.h "X265_BUILD >= 70" +enabled libxeve && require_pkg_config libxeve "xeve >= 1.0.0" "xeve.h" xeve_encode +enabled libxevd && require_pkg_config libxevd "xevd >= 1.0.0" "xevd.h" xevd_decode enabled libxavs && require libxavs "stdint.h xavs.h" xavs_encoder_encode "-lxavs $pthreads_extralibs $libm_extralibs" enabled libxavs2 && require_pkg_config libxavs2 "xavs2 >= 1.3.0" "stdint.h xavs2.h" xavs2_api_get enabled libxvid && require libxvid xvid.h xvid_global -lxvidcore diff --git a/doc/decoders.texi b/doc/decoders.texi index de2429abba..1754c517b8 100644 --- a/doc/decoders.texi +++ b/doc/decoders.texi @@ -126,6 +126,29 @@ Set amount of frame threads to use during decoding. The default value is 0 (auto @end table +@section libxevd + +eXtra-fast Essential Video Decoder (XEVD) MPEG-5 EVC decoder wrapper. + +This decoder requires the presence of the libxevd headers and library +during configuration. You need to explicitly configure the build with +@option{--enable-libxevd}. + +The xevd project website is at @url{https://github.com/mpeg5/xevd}. + +@subsection Options + +The following options are supported by the libxevd wrapper. +The xevd-equivalent options or values are listed in parentheses for easy migration. + +To get a more accurate and extensive documentation of the libxevd options, +invoke the command @code{xevd_app --help} or consult the libxevd documentation. + +@table @option +@item threads (@emph{threads}) +Force to use a specific number of threads +@end table + @section QSV Decoders The family of Intel QuickSync Video decoders (VC1, MPEG-2, H.264, HEVC, diff --git a/doc/encoders.texi b/doc/encoders.texi index aac9f25e55..c4e4bdaa5e 100644 --- a/doc/encoders.texi +++ b/doc/encoders.texi @@ -2808,6 +2808,118 @@ ffmpeg -i input -c:v libxavs2 -xavs2-params RdoqLevel=0 output.avs2 @end example @end table +@section libxeve + +eXtra-fast Essential Video Encoder (XEVE) MPEG-5 EVC encoder wrapper. + +This encoder requires the presence of the libxeve headers and library +during configuration. You need to explicitly configure the build with +@option{--enable-libxeve}. + +@float NOTE +Many libxeve encoder options are mapped to FFmpeg global codec options, +while unique encoder options are provided through private options. +Additionally the xeve-params private options allows one to pass a list +of key=value tuples as accepted by the libxeve @code{parse_xeve_params} function. +@end float + +The xeve project website is at @url{https://github.com/mpeg5/xeve}. + +@subsection Options + +The following options are supported by the libxeve wrapper. +The xeve-equivalent options or values are listed in parentheses for easy migration. + +@float NOTE +To reduce the duplication of documentation, only the private options +and some others requiring special attention are documented here. For +the documentation of the undocumented generic options, see +@ref{codec-options,,the Codec Options chapter}. +@end float + +@float NOTE +To get a more accurate and extensive documentation of the libxeve options, +invoke the command @code{xeve_app --help} or consult the libxeve documentation. +@end float + +@table @option +@item b (@emph{bitrate}) +Set target video bitrate in bits/s. +Note that FFmpeg’s b option is expressed in bits/s, while xeve’s bitrate is in kilobits/s. + +@item bf (@emph{bframes}) +Set the maximum number of B frames (1,3,7,15). + +@item g (@emph{keyint}) +Set the GOP size (I-picture period). + +@item preset (@emph{preset}) +Set the xeve preset. +Set the encoder preset value to determine encoding speed [fast, medium, slow, placebo] + +@item tune (@emph{tune}) +Set the encoder tune parameter [psnr, zerolatency] + +@item profile (@emph{profile}) +Set the encoder profile [0: baselie; 1: main] + +@item crf (@emph{crf}) +Set the quality for constant quality mode. +Constant rate factor <-1..51> [default: 32] + +@item qp (@emph{qp}) +Set constant quantization rate control method parameter. +Quantization parameter qp <0..51> [default: 32] + +@item threads (@emph{threads}) +Force to use a specific number of threads + +@item xeve-params +Set xeve options using a list of @var{key}=@var{value} couples separated +by ":". + +For example to specify libxeve encoding options with @option{-xeve-params}: + +@example +ffmpeg -i input -c:v libxeve -xeve-params profile=main:tune=zerolatency:preset=fast:q=17:bframes=15:bitrate=100k output.mp4 +@end example +@end table + +@subsubsection Supported key values for xeve-params + +Set xeve options using a list of @var{key}=@var{value} couples separated + +@table @option + +@item hash +Embed the picture signature (HASH) for conformance checking in decoding, (0: Off, 1: On) + +@item vbv-bufsize +Set the VBV buffer size: Kbits(none,K,k), Mbits(M,m) + +@item rc-type +Set the rate control type, (0: OFF, 1: ABR, 2: CRF) + +@item bframes +Set the maximum number of B frames (1,3,7,15) + +@item profile +Set the PROFILE (main, baseline) + +@item preset +Set the encoder PRESET (fast, medium, slow, placebo) + +@item tune +Set the encoder TUNE (psnr, zerolatency) + +@item bitrate +Set the Bitrate in terms of kilo-bits per second: Kbps(none,K,k), Mbps(M,m) + +@item qp +Set the QP value (0~51) + +@end table + @section libxvid Xvid MPEG-4 Part 2 encoder wrapper. diff --git a/doc/general_contents.texi b/doc/general_contents.texi index 238568f2bb..5b663a43ab 100644 --- a/doc/general_contents.texi +++ b/doc/general_contents.texi @@ -336,6 +336,22 @@ libxavs2 is under the GNU Public License Version 2 or later details), you must upgrade FFmpeg's license to GPL in order to use it. @end float +@section eXtra-fast Essential Video Encoder (XEVE) + +FFmpeg can make use of the XEVE library for EVC video encoding. + +Go to @url{https://github.com/mpeg5/xeve} and follow the instructions for +installing the XEVE library. Then pass @code{--enable-libxeve} to configure to +enable it. + +@section eXtra-fast Essential Video Decoder (XEVD) + +FFmpeg can make use of the XEVD library for EVC video decoding. + +Go to @url{https://github.com/mpeg5/xevd} and follow the instructions for +installing the XEVD library. Then pass @code{--enable-libxevd} to configure to +enable it. + @section ZVBI ZVBI is a VBI decoding library which can be used by FFmpeg to decode DVB @@ -584,6 +600,7 @@ library: @item raw DTS @tab X @tab X @item raw DTS-HD @tab @tab X @item raw E-AC-3 @tab X @tab X +@item raw EVC @tab X @tab X @item raw FLAC @tab X @tab X @item raw GSM @tab @tab X @item raw H.261 @tab X @tab X @@ -915,6 +932,8 @@ following image formats are supported: @item Electronic Arts TQI video @tab @tab X @item Escape 124 @tab @tab X @item Escape 130 @tab @tab X +@item EVC / MPEG-5 Part 1 @tab X @tab X + @tab encoding and decoding supported through external libraries libxeve and libxevd @item FFmpeg video codec #1 @tab X @tab X @tab lossless codec (fourcc: FFV1) @item Flash Screen Video v1 @tab X @tab X diff --git a/libavcodec/Makefile b/libavcodec/Makefile index 4b12228070..6f65e5902a 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -1096,6 +1096,8 @@ OBJS-$(CONFIG_LIBWEBP_ANIM_ENCODER) += libwebpenc_common.o libwebpenc_anim OBJS-$(CONFIG_LIBX262_ENCODER) += libx264.o OBJS-$(CONFIG_LIBX264_ENCODER) += libx264.o OBJS-$(CONFIG_LIBX265_ENCODER) += libx265.o +OBJS-$(CONFIG_LIBXEVE_ENCODER) += libxeve.o +OBJS-$(CONFIG_LIBXEVD_DECODER) += libxevd.o OBJS-$(CONFIG_LIBXAVS_ENCODER) += libxavs.o OBJS-$(CONFIG_LIBXAVS2_ENCODER) += libxavs2.o OBJS-$(CONFIG_LIBXVID_ENCODER) += libxvid.o @@ -1161,6 +1163,7 @@ OBJS-$(CONFIG_VP9_PARSER) += vp9_parser.o OBJS-$(CONFIG_WEBP_PARSER) += webp_parser.o OBJS-$(CONFIG_XBM_PARSER) += xbm_parser.o OBJS-$(CONFIG_XMA_PARSER) += xma_parser.o +OBJS-$(CONFIG_EVC_PARSER) += evc_parser.o # bitstream filters OBJS-$(CONFIG_AAC_ADTSTOASC_BSF) += aac_adtstoasc_bsf.o diff --git a/libavcodec/allcodecs.c b/libavcodec/allcodecs.c index 585918da93..0b7d5a5b6a 100644 --- a/libavcodec/allcodecs.c +++ b/libavcodec/allcodecs.c @@ -788,8 +788,11 @@ extern const FFCodec ff_libx262_encoder; #endif extern LIBX264_CONST FFCodec ff_libx264_encoder; #endif + extern const FFCodec ff_libx264rgb_encoder; extern FFCodec ff_libx265_encoder; +extern const FFCodec ff_libxeve_encoder; +extern const FFCodec ff_libxevd_decoder; extern const FFCodec ff_libxavs_encoder; extern const FFCodec ff_libxavs2_encoder; extern const FFCodec ff_libxvid_encoder; diff --git a/libavcodec/avcodec.h b/libavcodec/avcodec.h index 4dae23d06e..b83870c174 100644 --- a/libavcodec/avcodec.h +++ b/libavcodec/avcodec.h @@ -1666,6 +1666,9 @@ typedef struct AVCodecContext { #define FF_PROFILE_KLVA_SYNC 0 #define FF_PROFILE_KLVA_ASYNC 1 +#define FF_PROFILE_EVC_BASELINE 0 +#define FF_PROFILE_EVC_MAIN 1 + /** * level * - encoding: Set by user. diff --git a/libavcodec/evc_parser.c b/libavcodec/evc_parser.c new file mode 100644 index 0000000000..7c41eb9af4 --- /dev/null +++ b/libavcodec/evc_parser.c @@ -0,0 +1,514 @@ +/* + * EVC AVC format parser + * + * Copyright (C) 2021 Dawid Kozinski <d.kozinski@samsung.com> + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include <stdint.h> + +#include <xevd.h> + +#include "libavutil/common.h" + +#include "parser.h" +#include "golomb.h" + +#define EVC_NAL_HEADER_SIZE 2 /* byte */ +#define MAX_SPS_CNT 16 /* defined value in EVC standard */ + +// The sturcture reflects SPS RBSP(raw byte sequence payload) layout +// @see ISO_IEC_23094-1 section 7.3.2.1 +// +// The following descriptors specify the parsing process of each element +// u(n) - unsigned integer using n bits +// ue(v) - unsigned integer 0-th order Exp_Golomb-coded syntax element with the left bit first +typedef struct EVCParserSPS { + int sps_seq_parameter_set_id; // ue(v) + int profile_idc; // u(8) + int level_idc; // u(8) + int toolset_idc_h; // u(32) + int toolset_idc_l; // u(32) + int chroma_format_idc; // ue(v) + int pic_width_in_luma_samples; // ue(v) + int pic_height_in_luma_samples; // ue(v) + int bit_depth_luma_minus8; // ue(v) + int bit_depth_chroma_minus8; // ue(v) + + int sps_btt_flag; // u(1) + int log2_ctu_size_minus5; // ue(v) + int log2_min_cb_size_minus2; // ue(v) + int log2_diff_ctu_max_14_cb_size; // ue(v) + int log2_diff_ctu_max_tt_cb_size; // ue(v) + int log2_diff_min_cb_min_tt_cb_size_minus2; // ue(v) + + int sps_suco_flag; // u(1) + int log2_diff_ctu_size_max_suco_cb_size; // ue(v) + int log2_diff_max_suco_min_suco_cb_size; // ue(v) + + int sps_admvp_flag; // u(1) + int sps_affine_flag; // u(1) + int sps_amvr_flag; // u(1) + int sps_dmvr_flag; // u(1) + int sps_mmvd_flag; // u(1) + int sps_hmvp_flag; // u(1) + + int sps_eipd_flag; // u(1) + int sps_ibc_flag; // u(1) + int log2_max_ibc_cand_size_minus2; // ue(v) + + int sps_cm_init_flag; // u(1) + int sps_adcc_flag; // u(1) + + int sps_iqt_flag; // u(1) + int sps_ats_flag; // u(1) + + int sps_addb_flag; // u(1) + int sps_alf_flag; // u(1) + int sps_htdf_flag; // u(1) + int sps_rpl_flag; // u(1) + int sps_pocs_flag; // u(1) + int sps_dquant_flag; // u(1) + int sps_dra_flag; // u(1) + + int log2_max_pic_order_cnt_lsb_minus4; // ue(v) + int log2_sub_gop_length; // ue(v) + int log2_ref_pic_gap_length; // ue(v) + + int max_num_tid0_ref_pics; // ue(v) + + int sps_max_dec_pic_buffering_minus1; // ue(v) + int long_term_ref_pic_flag; // u(1) + int rpl1_same_as_rpl0_flag; // u(1) + int num_ref_pic_list_in_sps[2]; // ue(v) + + int picture_cropping_flag; // u(1) + int picture_crop_left_offset; // ue(v) + int picture_crop_right_offset; // ue(v) + int picture_crop_top_offset; // ue(v) + int picture_crop_bottom_offset; // ue(v) + +} EVCParserSPS; + +typedef struct EVCParserContext { + ParseContext pc; + EVCParserSPS sps[MAX_SPS_CNT]; + int is_avc; + int nal_length_size; + int to_read; + int incomplete_nalu_prefix_read; // The flag is set to 1 when an incomplete NAL unit prefix has been read + + int got_sps; + int got_pps; + int got_sei; + int got_slice; +} EVCParserContext; + +static int get_nalu_type(const uint8_t *bs, int bs_size, AVCodecContext *avctx) +{ + int unit_type_plus1 = 0; + XEVD_INFO info; + int ret; + + if(bs_size >= EVC_NAL_HEADER_SIZE) { + ret = xevd_info((void *)bs, EVC_NAL_HEADER_SIZE, 1, &info); + if (XEVD_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Cannot get bitstream information\n"); + return -1; + } + unit_type_plus1 = info.nalu_type; + + } + + return unit_type_plus1 - 1; +} + +static uint32_t read_nal_unit_length(const uint8_t *bs, int bs_size, AVCodecContext *avctx) +{ + uint32_t len = 0; + XEVD_INFO info; + int ret; + + if(bs_size >= XEVD_NAL_UNIT_LENGTH_BYTE) { + ret = xevd_info((void *)bs, XEVD_NAL_UNIT_LENGTH_BYTE, 1, &info); + if (XEVD_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Cannot get bitstream information\n"); + return 0; + } + len = info.nalu_len; + if(len == 0) { + av_log(avctx, AV_LOG_ERROR, "Invalid bitstream size!\n"); + return 0; + } + } + + return len; +} + +// @see ISO_IEC_23094-1 (7.3.2.1 SPS RBSP syntax) +static EVCParserSPS *parse_sps(const uint8_t *bs, int bs_size, EVCParserContext *ev) +{ + GetBitContext gb; + EVCParserSPS *sps; + int sps_seq_parameter_set_id; + + if(init_get_bits8(&gb, bs, bs_size) < 0) + return NULL; + + sps_seq_parameter_set_id = get_ue_golomb(&gb); + + if(sps_seq_parameter_set_id >= MAX_SPS_CNT) + return NULL; + + sps = &ev->sps[sps_seq_parameter_set_id]; + sps->sps_seq_parameter_set_id = sps_seq_parameter_set_id; + + // the Baseline profile is indicated by profile_idc eqal to 0 + // the Main profile is indicated by profile_idc eqal to 1 + sps->profile_idc = get_bits(&gb, 8); + + sps->level_idc = get_bits(&gb, 8); + + skip_bits_long(&gb, 32); /* skip toolset_idc_h */ + skip_bits_long(&gb, 32); /* skip toolset_idc_l */ + + // 0 - monochrome + // 1 - 4:2:0 + // 2 - 4:2:2 + // 3 - 4:4:4 + sps->chroma_format_idc = get_ue_golomb(&gb); + + sps->pic_width_in_luma_samples = get_ue_golomb(&gb); + sps->pic_height_in_luma_samples = get_ue_golomb(&gb); + + sps->bit_depth_luma_minus8 = get_ue_golomb(&gb); + sps->bit_depth_chroma_minus8 = get_ue_golomb(&gb); + + sps->sps_btt_flag = get_bits(&gb, 1); + if(sps->sps_btt_flag) { + sps->log2_ctu_size_minus5 = get_ue_golomb(&gb); + sps->log2_min_cb_size_minus2 = get_ue_golomb(&gb); + sps->log2_diff_ctu_max_14_cb_size = get_ue_golomb(&gb); + sps->log2_diff_ctu_max_tt_cb_size = get_ue_golomb(&gb); + sps->log2_diff_min_cb_min_tt_cb_size_minus2 = get_ue_golomb(&gb); + } + + sps->sps_suco_flag = get_bits(&gb, 1); + if(sps->sps_suco_flag) { + sps->log2_diff_ctu_size_max_suco_cb_size = get_ue_golomb(&gb); + sps->log2_diff_max_suco_min_suco_cb_size = get_ue_golomb(&gb); + } + + sps->sps_admvp_flag = get_bits(&gb, 1); + if(sps->sps_admvp_flag) { + sps->sps_affine_flag = get_bits(&gb, 1); + sps->sps_amvr_flag = get_bits(&gb, 1); + sps->sps_dmvr_flag = get_bits(&gb, 1); + sps->sps_mmvd_flag = get_bits(&gb, 1); + sps->sps_hmvp_flag = get_bits(&gb, 1); + } + + sps->sps_eipd_flag = get_bits(&gb, 1); + if(sps->sps_eipd_flag) { + sps->sps_ibc_flag = get_bits(&gb, 1); + if(sps->sps_ibc_flag) + sps->log2_max_ibc_cand_size_minus2 = get_ue_golomb(&gb); + } + + sps->sps_cm_init_flag = get_bits(&gb, 1); + if(sps->sps_cm_init_flag) + sps->sps_adcc_flag = get_bits(&gb, 1); + + sps->sps_iqt_flag = get_bits(&gb, 1); + if(sps->sps_iqt_flag) + sps->sps_ats_flag = get_bits(&gb, 1); + + sps->sps_addb_flag = get_bits(&gb, 1); + sps->sps_alf_flag = get_bits(&gb, 1); + sps->sps_htdf_flag = get_bits(&gb, 1); + sps->sps_rpl_flag = get_bits(&gb, 1); + sps->sps_pocs_flag = get_bits(&gb, 1); + sps->sps_dquant_flag = get_bits(&gb, 1); + sps->sps_dra_flag = get_bits(&gb, 1); + + if(sps->sps_pocs_flag) + sps->log2_max_pic_order_cnt_lsb_minus4 = get_ue_golomb(&gb); + + if(!sps->sps_pocs_flag || !sps->sps_rpl_flag) { + sps->log2_sub_gop_length = get_ue_golomb(&gb); + if(sps->log2_sub_gop_length == 0) + sps->log2_ref_pic_gap_length = get_ue_golomb(&gb); + } + + return sps; +} + +static int parse_nal_units(AVCodecParserContext *s, const uint8_t *bs, + int bs_size, AVCodecContext *avctx) +{ + EVCParserContext *ev = s->priv_data; + int nalu_type, nalu_size; + unsigned char *bits = (unsigned char *)bs; + int bits_size = bs_size; + + avctx->codec_id = AV_CODEC_ID_EVC; + + nalu_size = read_nal_unit_length(bits, bits_size, avctx); + if(nalu_size == 0) { + av_log(avctx, AV_LOG_ERROR, "Invalid NAL unit size: (%d)\n", nalu_size); + return -1; + } + + bits += XEVD_NAL_UNIT_LENGTH_BYTE; + bits_size -= XEVD_NAL_UNIT_LENGTH_BYTE; + + nalu_type = get_nalu_type(bits, bits_size, avctx); + + bits += EVC_NAL_HEADER_SIZE; + bits_size -= EVC_NAL_HEADER_SIZE; + + if (nalu_type == XEVD_NUT_SPS) { // NAL Unit type: SPS (Sequence Parameter Set) + EVCParserSPS *sps; + + sps = parse_sps(bits, bits_size, ev); + + avctx->coded_width = sps->pic_width_in_luma_samples; + avctx->coded_height = sps->pic_height_in_luma_samples; + avctx->width = sps->pic_width_in_luma_samples; + avctx->height = sps->pic_height_in_luma_samples; + + if(sps->profile_idc == 0) avctx->profile = FF_PROFILE_EVC_BASELINE; + else if (sps->profile_idc == 1) avctx->profile = FF_PROFILE_EVC_MAIN; + else { + av_log(avctx, AV_LOG_ERROR, "Not supported profile (%d)\n", sps->profile_idc); + return -1; + } + + // Currently XEVD decoder supports ony YCBCR420_10LE chroma format for EVC stream + switch(sps->chroma_format_idc) { + case 0: /* YCBCR400_10LE */ + av_log(avctx, AV_LOG_ERROR, "YCBCR400_10LE: Not supported chroma format\n"); + avctx->pix_fmt = AV_PIX_FMT_GRAY10LE; + return -1; + case 1: /* YCBCR420_10LE */ + avctx->pix_fmt = AV_PIX_FMT_YUV420P10LE; + break; + case 2: /* YCBCR422_10LE */ + av_log(avctx, AV_LOG_ERROR, "YCBCR422_10LE: Not supported chroma format\n"); + avctx->pix_fmt = AV_PIX_FMT_YUV422P10LE; + return -1; + case 3: /* YCBCR444_10LE */ + av_log(avctx, AV_LOG_ERROR, "YCBCR444_10LE: Not supported chroma format\n"); + avctx->pix_fmt = AV_PIX_FMT_YUV444P10LE; + return -1; + default: + avctx->pix_fmt = AV_PIX_FMT_NONE; + av_log(avctx, AV_LOG_ERROR, "Unknown supported chroma format\n"); + return -1; + } + + // The current implementation of parse_sps function doesn't handle VUI parameters parsing, + // so at the moment it's impossible to initialize has_b_frames and max_b_frames AVCodecContex fields here. + // Currently, initialization of has_b_frames and max_b_frames AVCodecContex fields have been moved to + // libxevd_decode function where we can use xevd_config function being a part of xevd library API + // to get the needed information. + // However, if it will be needed, parse_sps function should be extended to handle VUI parameters parsing + // and the following lines should be used to initialize has_b_frames and max_b_frames fields of the AVCodecContex. + // + // sps->vui_parameters.num_reorder_pics + // if (sps->bitstream_restriction_flag && sps->vui_parameters.num_reorder_pics) { + // avctx->has_b_frames = sps->vui_parameters.num_reorder_pics; + // } + + ev->got_sps = 1; + + } else if (nalu_type == XEVD_NUT_PPS) // NAL Unit type: PPS (Video Parameter Set) + ev->got_pps = 1; + else if(nalu_type == XEVD_NUT_SEI) // NAL unit type: SEI (Supplemental Enhancement Information) + ev->got_sei = 1; + else if (nalu_type == XEVD_NUT_IDR || nalu_type == XEVD_NUT_NONIDR) // NAL Unit type: Coded slice of a IDR or non-IDR picture + ev->got_slice++; + else { + av_log(avctx, AV_LOG_ERROR, "Invalid NAL unit type: %d\n", nalu_type); + return -1; + } + + return 0; +} + +/** + * Find the end of the current frame in the bitstream. + * @return the position of the first byte of the next frame, or END_NOT_FOUND + */ +static int evc_find_frame_end(AVCodecParserContext *s, const uint8_t *buf, + int buf_size, AVCodecContext *avctx) +{ + EVCParserContext *ev = s->priv_data; + + if(!ev->to_read) { + int nal_unit_size = 0; + int next = END_NOT_FOUND; + + // This is the case when buffer size is not enough for buffer to store NAL unit length + if(buf_size < XEVD_NAL_UNIT_LENGTH_BYTE) { + ev->to_read = XEVD_NAL_UNIT_LENGTH_BYTE; + ev->nal_length_size = buf_size; + ev->incomplete_nalu_prefix_read = 1; + + return END_NOT_FOUND; + } + + nal_unit_size = read_nal_unit_length(buf, buf_size, avctx); + ev->nal_length_size = XEVD_NAL_UNIT_LENGTH_BYTE; + + next = nal_unit_size + XEVD_NAL_UNIT_LENGTH_BYTE; + ev->to_read = next; + if(next < buf_size) + return next; + else + return END_NOT_FOUND; + } else if(ev->to_read > buf_size) { + return END_NOT_FOUND; + } else { + if(ev->incomplete_nalu_prefix_read == 1) { + EVCParserContext *ev = s->priv_data; + ParseContext *pc = &ev->pc; + uint8_t nalu_len[XEVD_NAL_UNIT_LENGTH_BYTE] = {0}; + int nal_unit_size = 0; + + // 1. pc->buffer contains previously read bytes of NALU prefix + // 2. buf contains the rest of NAL unit prefix bytes + // + // ~~~~~~~ + // EXAMPLE + // ~~~~~~~ + // + // In the following example we assumed that the number of already read NAL Unit prefix bytes is equal 1 + // + // ---------- + // pc->buffer -> conatins already read bytes + // ---------- + // __ pc->index == 1 + // | + // V + // ------------------------------------------------------- + // | 0 | 1 | 2 | 3 | 4 | ... | N | + // ------------------------------------------------------- + // | 0x00 | 0xXX | 0xXX | 0xXX | 0xXX | ... | 0xXX | + // ------------------------------------------------------- + // + // ---------- + // buf -> contains newly read bytes + // ---------- + // ------------------------------------------------------- + // | 0 | 1 | 2 | 3 | 4 | ... | N | + // ------------------------------------------------------- + // | 0x00 | 0x00 | 0x3C | 0xXX | 0xXX | ... | 0xXX | + // ------------------------------------------------------- + // + for(int i = 0; i < XEVD_NAL_UNIT_LENGTH_BYTE; i++) { + if(i < pc->index) + nalu_len[i] = pc->buffer[i]; + else + nalu_len[i] = buf[i - pc->index]; + } + + // ---------- + // nalu_len + // ---------- + // --------------------------------- + // | 0 | 1 | 2 | 3 | + // --------------------------------- + // | 0x00 | 0x00 | 0x00 | 0x3C | + // --------------------------------- + // | NALU LENGTH | + // --------------------------------- + // NAL Unit lenght = 60 (0x0000003C) + + nal_unit_size = read_nal_unit_length(nalu_len, XEVD_NAL_UNIT_LENGTH_BYTE, avctx); + + ev->to_read = nal_unit_size + XEVD_NAL_UNIT_LENGTH_BYTE - pc->index; + + ev->incomplete_nalu_prefix_read = 0; + + if(ev->to_read > buf_size) + return END_NOT_FOUND; + else + return ev->to_read; + } + return ev->to_read; + } + + return END_NOT_FOUND; +} + +static int evc_parser_init(AVCodecParserContext *s) +{ + EVCParserContext *ev = s->priv_data; + + ev->got_sps = 0; + ev->got_pps = 0; + ev->got_sei = 0; + ev->got_slice = 0; + ev->nal_length_size = XEVD_NAL_UNIT_LENGTH_BYTE; + ev->incomplete_nalu_prefix_read = 0; + + return 0; +} + +static int evc_parse(AVCodecParserContext *s, AVCodecContext *avctx, + const uint8_t **poutbuf, int *poutbuf_size, + const uint8_t *buf, int buf_size) +{ + int next; + EVCParserContext *ev = s->priv_data; + ParseContext *pc = &ev->pc; + int is_dummy_buf = !buf_size; + const uint8_t *dummy_buf = buf; + + if (s->flags & PARSER_FLAG_COMPLETE_FRAMES) + next = buf_size; + else { + next = evc_find_frame_end(s, buf, buf_size, avctx); + if (ff_combine_frame(pc, next, &buf, &buf_size) < 0) { + *poutbuf = NULL; + *poutbuf_size = 0; + ev->to_read -= buf_size; + return buf_size; + } + } + + is_dummy_buf &= (dummy_buf == buf); + + if (!is_dummy_buf) + parse_nal_units(s, buf, buf_size, avctx); + + *poutbuf = buf; + *poutbuf_size = buf_size; + ev->to_read -= next; + + return next; +} + +AVCodecParser ff_evc_parser = { + .codec_ids = { AV_CODEC_ID_EVC }, + .priv_data_size = sizeof(EVCParserContext), + .parser_init = evc_parser_init, + .parser_parse = evc_parse, + .parser_close = ff_parse_close, +}; diff --git a/libavcodec/libxevd.c b/libavcodec/libxevd.c new file mode 100644 index 0000000000..ed19aaabaf --- /dev/null +++ b/libavcodec/libxevd.c @@ -0,0 +1,445 @@ +/* + * libxevd decoder + * EVC (MPEG-5 Essential Video Coding) decoding using XEVD MPEG-5 EVC decoder library + * + * Copyright (C) 2021 Dawid Kozinski <d.kozinski@samsung.com> + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#if defined(_MSC_VER) +#define XEVD_API_IMPORTS 1 +#endif + +#include <float.h> +#include <stdlib.h> + +#include <xevd.h> + +#include "libavutil/internal.h" +#include "libavutil/common.h" +#include "libavutil/opt.h" +#include "libavutil/pixdesc.h" +#include "libavutil/pixfmt.h" +#include "libavutil/imgutils.h" +#include "libavutil/cpu.h" + +#include "avcodec.h" +#include "internal.h" +#include "packet_internal.h" +#include "codec_internal.h" + +#define XEVD_PARAM_BAD_NAME -1 +#define XEVD_PARAM_BAD_VALUE -2 + +#define EVC_NAL_HEADER_SIZE 2 /* byte */ + +/** + * The structure stores all the states associated with the instance of Xeve MPEG-5 EVC decoder + * The first field is a pointer to an AVClass struct (@see https://ffmpeg.org/doxygen/trunk/structAVClass.html#details). + */ +typedef struct XevdContext { + const AVClass *class; + + XEVD id; // XEVD instance identifier @see xevd.h + XEVD_CDSC cdsc; // decoding parameters @see xevd.h + + int decod_frames; // number of decoded frames + int packet_count; // number of packets created by decoder + + // configuration parameters + AVDictionary *xevd_opts; // xevd configuration read from a : separated list of key=value parameters + +} XevdContext; + +/** + * The function returns a pointer to a variable of type XEVD_CDSC. + * XEVD_CDSC contains all decoder parameters that should be initialized before its use. + * + * The field values of the XEVD_CDSC structure are populated based on: + * - the corresponding field values of the AvCodecConetxt structure, + * - the xevd decoder specific option values, + * (the full list of options available for the xevd encoder is displayed after executing the command ./ffmpeg --help decoder = libxevd) + * - and the xevd encoder options specified as a list of key-value pairs following the xevd-params option + * + * Order of input processing and populating the XEVD_CDSC structure + * 1. first, the corresponding fields of the AVCodecContext structure are processed, (i.e -threads 4) + * 2. then xevd-specific options are added as AVOption to the xevd AVCodec implementation (i.e -threads 3) + * 3. finally, the options specified after the xevd-params option as the parameter list of type key value are processed (i.e -xevd-params "m=2") + * + * Some options can be set in different ways. In this case, please follow the above-mentioned order of processing. + * The most recent assignments overwrite the previous values. + * + * @param[in] avctx codec context + * @param[out] cdsc contains all encoder parameters that should be initialized before its use + * + * @return 0 on success, negative error code on failure + */ +static int get_conf(AVCodecContext *avctx, XEVD_CDSC *cdsc) +{ + int cpu_count = av_cpu_count(); + + /* clear XEVS_CDSC structure */ + memset(cdsc, 0, sizeof(XEVD_CDSC)); + + /* init XEVD_CDSC */ + if(avctx->thread_count <= 0) + cdsc->threads = (cpu_count < XEVD_MAX_TASK_CNT) ? cpu_count : XEVD_MAX_TASK_CNT; + else if(avctx->thread_count > XEVD_MAX_TASK_CNT) + cdsc->threads = XEVD_MAX_TASK_CNT; + else + cdsc->threads = avctx->thread_count; + + return XEVD_OK; +} + +/** + * Read NAL unit length + * @param bs input data (bitstream) + * @return the length of NAL unit on success, 0 value on failure + */ +static uint32_t read_nal_unit_length(const uint8_t *bs, int bs_size, AVCodecContext *avctx) +{ + uint32_t len = 0; + XEVD_INFO info; + int ret; + + if(bs_size == XEVD_NAL_UNIT_LENGTH_BYTE) { + ret = xevd_info((void *)bs, XEVD_NAL_UNIT_LENGTH_BYTE, 1, &info); + if (XEVD_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Cannot get bitstream information\n"); + return 0; + } + len = info.nalu_len; + if(len == 0) { + av_log(avctx, AV_LOG_ERROR, "Invalid bitstream size! [%d]\n", bs_size); + return 0; + } + } + + return len; +} + +/** + * @param[in] xectx the structure that stores all the state associated with the instance of Xeve MPEG-5 EVC decoder + * @param[out] avctx codec context + * @return 0 on success, negative value on failure + */ +static int export_stream_params(const XevdContext *xectx, AVCodecContext *avctx) +{ + int ret; + int size; + int color_space; + + avctx->pix_fmt = AV_PIX_FMT_YUV420P10; + + size = 4; + ret = xevd_config(xectx->id, XEVD_CFG_GET_CODED_WIDTH, &avctx->coded_width, &size); + if (XEVD_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to get coded_width\n"); + return -1; + } + + ret = xevd_config(xectx->id, XEVD_CFG_GET_CODED_HEIGHT, &avctx->coded_height, &size); + if (XEVD_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to get coded_height\n"); + return -1; + } + + ret = xevd_config(xectx->id, XEVD_CFG_GET_WIDTH, &avctx->width, &size); + if (XEVD_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to get width\n"); + return -1; + } + + ret = xevd_config(xectx->id, XEVD_CFG_GET_HEIGHT, &avctx->height, &size); + if (XEVD_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to get height\n"); + return -1; + } + + ret = xevd_config(xectx->id, XEVD_CFG_GET_COLOR_SPACE, &color_space, &size); + if (XEVD_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to get color_space\n"); + return -1; + } + switch(color_space) { + case XEVD_CS_YCBCR400_10LE: + avctx->pix_fmt = AV_PIX_FMT_GRAY10LE; + break; + case XEVD_CS_YCBCR420_10LE: + avctx->pix_fmt = AV_PIX_FMT_YUV420P10LE; + break; + case XEVD_CS_YCBCR422_10LE: + avctx->pix_fmt = AV_PIX_FMT_YUV422P10LE; + break; + case XEVD_CS_YCBCR444_10LE: + avctx->pix_fmt = AV_PIX_FMT_YUV444P10LE; + break; + default: + av_log(avctx, AV_LOG_ERROR, "Unknown color space\n"); + avctx->pix_fmt = AV_PIX_FMT_NONE; + return -1; + } + + // the function returns sps->num_reorder_pics + ret = xevd_config(xectx->id, XEVD_CFG_GET_MAX_CODING_DELAY, &avctx->max_b_frames, &size); + if (XEVD_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to get max_coding_delay\n"); + return -1; + } + + avctx->has_b_frames = (avctx->max_b_frames) ? 1 : 0; + + avctx->color_primaries = AVCOL_PRI_UNSPECIFIED; + avctx->color_trc = AVCOL_TRC_UNSPECIFIED; + avctx->colorspace = AVCOL_SPC_UNSPECIFIED; + + return 0; +} + +/** + * Initialize decoder + * Create a decoder instance and allocate all the needed resources + * + * @param avctx codec context + * @return 0 on success, negative error code on failure + */ +static av_cold int libxevd_init(AVCodecContext *avctx) +{ + XevdContext *xectx = avctx->priv_data; + int val = 0; + XEVD_CDSC *cdsc = &(xectx->cdsc); + + /* read configurations and set values for created descriptor (XEVD_CDSC) */ + val = get_conf(avctx, cdsc); + if (val != XEVD_OK) { + av_log(avctx, AV_LOG_ERROR, "Cannot get configuration\n"); + return -1; + } + + /* create decoder */ + xectx->id = xevd_create(&(xectx->cdsc), NULL); + if(xectx->id == NULL) { + av_log(avctx, AV_LOG_ERROR, "Cannot create XEVD encoder\n"); + return -1; + } + + xectx->packet_count = 0; + xectx->decod_frames = 0; + + return 0; +} + +/** + * Decode picture + * + * @param avctx codec context + * @param[out] data decoded frame + * @param[out] got_frame decoder sets to 0 or 1 to indicate that a + * non-empty frame or subtitle was returned in + * outdata. + * @param[in] pkt AVPacket containing encoded data to be decoded + * + * @return amount of bytes read from the packet on success, negative error + * code on failure + */ +static int libxevd_decode(AVCodecContext *avctx, void *data, int *got_frame, AVPacket *pkt) +{ + AVFrame *frame = data; + XevdContext *xectx = NULL; + XEVD_IMGB *imgb = NULL; + XEVD_STAT stat; + XEVD_BITB bitb; + int ret, nalu_size, bs_read_pos; + + if(avctx == NULL) { + av_log(avctx, AV_LOG_ERROR, "Invalid input parameter: AVCodecContext\n"); + return -1; + } + xectx = avctx->priv_data; + if(xectx == NULL) { + av_log(avctx, AV_LOG_ERROR, "Invalid XEVD context\n"); + return -1; + } + + if(pkt->size > 0) { + bs_read_pos = 0; + imgb = NULL; + while(pkt->size > (bs_read_pos + XEVD_NAL_UNIT_LENGTH_BYTE)) { + + memset(&stat, 0, sizeof(XEVD_STAT)); + memset(&bitb, 0, sizeof(XEVD_BITB)); + + nalu_size = read_nal_unit_length(pkt->data + bs_read_pos, XEVD_NAL_UNIT_LENGTH_BYTE, avctx); + if(nalu_size == 0) { + av_log(avctx, AV_LOG_ERROR, "Invalid bitstream\n"); + goto ERR; + } + bs_read_pos += XEVD_NAL_UNIT_LENGTH_BYTE; + + bitb.addr = pkt->data + bs_read_pos; + bitb.ssize = nalu_size; + + /* main decoding block */ + ret = xevd_decode(xectx->id, &bitb, &stat); + if(XEVD_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to decode bitstream\n"); + goto ERR; + } + + bs_read_pos += nalu_size; + + if(stat.nalu_type == XEVD_NUT_SPS) { // EVC stream parameters changed + if(export_stream_params(xectx, avctx) != 0) + goto ERR; + } + + if(stat.read != nalu_size) + av_log(avctx, AV_LOG_INFO, "Different reading of bitstream (in:%d, read:%d)\n,", nalu_size, stat.read); + if(stat.fnum >= 0) { + if (imgb) { /* already has a decoded image */ + imgb->release(imgb); + imgb = NULL; + } + ret = xevd_pull(xectx->id, &imgb); + if(XEVD_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to pull the decoded image (xevd error code: %d, frame#=%d)\n", ret, stat.fnum); + goto ERR; + } else if (ret == XEVD_OK_FRM_DELAYED) { + if(imgb) { + imgb->release(imgb); + imgb = NULL; + } + } + } + } + } else { // bumping + ret = xevd_pull(xectx->id, &(imgb)); + if(ret == XEVD_ERR_UNEXPECTED) { // bumping process completed + *got_frame = 0; + return 0; + } else if(XEVD_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to pull the decoded image (xevd error code: %d)\n", ret); + goto ERR; + } + } + + if(imgb) { + // @todo supports other color space and bit depth + if(imgb->cs != XEVD_CS_YCBCR420_10LE) { + av_log(avctx, AV_LOG_ERROR, "Not supported pixel format: %s\n", av_get_pix_fmt_name(avctx->pix_fmt)); + goto ERR; + } + + if (imgb->w[0] != avctx->width || imgb->h[0] != avctx->height) { // stream resolution changed + if(ff_set_dimensions(avctx, imgb->w[0], imgb->h[0]) < 0) { + av_log(avctx, AV_LOG_ERROR, "Cannot set new dimension\n"); + goto ERR; + } + } + + frame->coded_picture_number++; + frame->display_picture_number++; + frame->format = AV_PIX_FMT_YUV420P10LE; + + if (ff_get_buffer(avctx, frame, 0) < 0) { + av_log(avctx, AV_LOG_ERROR, "Cannot get AV buffer\n"); + goto ERR; + } + + frame->pts = pkt->pts; + + av_image_copy(frame->data, frame->linesize, (const uint8_t **)imgb->a, + imgb->s, avctx->pix_fmt, + imgb->w[0], imgb->h[0]); + + xectx->decod_frames++; + *got_frame = 1; + + imgb->release(imgb); + imgb = NULL; + } else + *got_frame = 0; + + xectx->packet_count++; + + return pkt->size; + +ERR: + if(imgb) { + imgb->release(imgb); + imgb = NULL; + } + *got_frame = 0; + + return -1; +} + +/** + * Destroy decoder + * + * @param avctx codec context + * @return 0 on success + */ +static av_cold int libxevd_close(AVCodecContext *avctx) +{ + XevdContext *xectx = avctx->priv_data; + if(xectx->id) { + xevd_delete(xectx->id); + xectx->id = NULL; + } + + return 0; +} + +#define OFFSET(x) offsetof(XevdContext, x) +#define VD AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_DECODING_PARAM + +// @todo consider using following options (./ffmpeg --help decoder=libxevd) +// +static const AVOption options[] = { + { "xevd-params", "override the xevd configuration using a : separated list of key=value parameters", OFFSET(xevd_opts), AV_OPT_TYPE_DICT, { 0 }, 0, 0, VD }, + { NULL } +}; + +static const AVClass xevd_class = { + .class_name = "libxevd", + .item_name = av_default_item_name, + .option = options, + .version = LIBAVUTIL_VERSION_INT, +}; + +static const FFCodecDefault xevd_defaults[] = { + { "b", "0" }, + { NULL }, +}; + +FFCodec ff_libxevd_decoder = { + .p.name = "evc", + .p.long_name = NULL_IF_CONFIG_SMALL("EVC / MPEG-5 Essential Video Coding (EVC)"), + .p.type = AVMEDIA_TYPE_VIDEO, + .p.id = AV_CODEC_ID_EVC, + .init = libxevd_init, + FF_CODEC_DECODE_CB(libxevd_decode), + .close = libxevd_close, + .priv_data_size = sizeof(XevdContext), + .p.priv_class = &xevd_class, + .defaults = xevd_defaults, + .caps_internal = AV_CODEC_CAP_DELAY | AV_CODEC_CAP_AUTO_THREADS | AV_CODEC_CAP_AVOID_PROBING, + .p.wrapper_name = "libxevd", +}; diff --git a/libavcodec/libxeve.c b/libavcodec/libxeve.c new file mode 100644 index 0000000000..e93007431b --- /dev/null +++ b/libavcodec/libxeve.c @@ -0,0 +1,1141 @@ +/* + * libxeve encoder + * EVC (MPEG-5 Essential Video Coding) encoding using XEVE MPEG-5 EVC encoder library + * + * Copyright (C) 2021 Dawid Kozinski <d.kozinski@samsung.com> + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#if defined(_MSC_VER) +#define XEVE_API_IMPORTS 1 +#endif + +#include <float.h> +#include <stdlib.h> + +#include <xeve.h> + +#include "libavutil/internal.h" +#include "libavutil/common.h" +#include "libavutil/opt.h" +#include "libavutil/pixdesc.h" +#include "libavutil/pixfmt.h" +#include "libavutil/time.h" +#include "libavutil/cpu.h" + +#include "avcodec.h" +#include "internal.h" +#include "packet_internal.h" +#include "codec_internal.h" + +#define MAX_BS_BUF (16*1024*1024) + +/** + * Error codes + */ +#define XEVE_PARAM_BAD_NAME -100 +#define XEVE_PARAM_BAD_VALUE -200 + +/** + * Encoder states + * + * STATE_ENCODING - the encoder receives and processes input frames + * STATE_BUMPING - there are no more input frames, however the encoder still processes previously received data + * STATE_SKIPPING - skipping input frames + */ +typedef enum State { + STATE_ENCODING, + STATE_BUMPING, + STATE_SKIPPING +} State; + +/** + * The structure stores all the states associated with the instance of Xeve MPEG-5 EVC encoder + * The first field is a pointer to an AVClass struct (@see https://ffmpeg.org/doxygen/trunk/structAVClass.html#details). + */ +typedef struct XeveContext { + const AVClass *class; + + XEVE id; // XEVE instance identifier + XEVE_CDSC cdsc; // coding parameters i.e profile, width & height of input frame, num of therads, frame rate ... + XEVE_BITB bitb; // bitstream buffer (output) + XEVE_STAT stat; // encoding status (output) + XEVE_IMGB imgb; // image buffer (input) + + State state; // encoder state (skipping, encoding, bumping) + + int encod_frames; // num of encoded frames + double bytes_total; // encoded bitstream byte size + double bitrate; // bits per second + int packet_count; // num of packets created by encoder + + // Chroma subsampling + int width_luma; + int height_luma; + int width_chroma; + int height_chroma; + + int profile_id; // encoder profile (main, baseline) + int preset_id; // preset of xeve ( fast, medium, slow, placebo) + int tune_id; // tune of xeve (psnr, zerolatency) + int input_depth; // input bit-depth: 8bit, 10bit + int hash; // embed picture signature (HASH) for conformance checking in decoding + + /* variables for input parameter */ + char *op_preset; + char *op_tune; + int op_qp; + int op_crf; + + // configuration parameters + // xeve configuration read from a : separated list of key=value parameters + AVDictionary *xeve_params; +} XeveContext; + +/** + * Gets Xeve encoder pre-defined profile + * + * @param profile string describing Xeve encoder profile (baseline, main) + * @return XEVE pre-defined profile on success, negative value on failure + */ +static int get_profile_id(const char *profile) +{ + if (!strcmp(profile, "baseline")) + return XEVE_PROFILE_BASELINE; + else if (!strcmp(profile, "main")) + return XEVE_PROFILE_MAIN; + else + return -1; +} + +/** + * Gets Xeve pre-defined preset + * + * @param preset string describing Xeve encoder preset (fast, medium, slow, placebo) + * @return XEVE pre-defined profile on success, negative value on failure + */ +static int get_preset_id(const char *preset) +{ + if((!strcmp(preset, "fast"))) + return XEVE_PRESET_FAST; + else if (!strcmp(preset, "medium")) + return XEVE_PRESET_MEDIUM; + else if (!strcmp(preset, "slow")) + return XEVE_PRESET_SLOW; + else if (!strcmp(preset, "placebo")) + return XEVE_PRESET_PLACEBO; + else + return -1; +} + +/** + * Gets Xeve pre-defined tune id + * + * @param[in] tune string describing Xeve encoder tune (psnr, zerolatency) + * @return XEVE pre-defined profile on success, negative value on failure + */ +static int get_tune_id(const char *tune) +{ + if((!strcmp(tune, "psnr"))) + return XEVE_TUNE_PSNR; + else if (!strcmp(tune, "zerolatency")) + return XEVE_TUNE_ZEROLATENCY; + else + return -1; +} + +/** + * Convert FFmpeg pixel format (AVPixelFormat) to XEVE pre-defined color format + * + * @param[in] px_fmt pixel format (@see https://ffmpeg.org/doxygen/trunk/pixfmt_8h.html#a9a8e335cf3be472042bc9f0cf80cd4c5) + * @param[out] color_format XEVE pre-defined color format (@see xeve.h) + * @param[out] bit_depth bit depth + * + * @return 0 on success, negative value on failure + */ +static int get_pix_fmt(enum AVPixelFormat pix_fmt, int *color_format, int *bit_depth) +{ + switch (pix_fmt) { + case AV_PIX_FMT_YUV420P: + *color_format = XEVE_CF_YCBCR420; + *bit_depth = 8; + break; + case AV_PIX_FMT_YUV422P: + *color_format = XEVE_CF_YCBCR422; + *bit_depth = 8; + break; + case AV_PIX_FMT_YUV444P: + *color_format = XEVE_CF_YCBCR444; + *bit_depth = 8; + break; + case AV_PIX_FMT_YUV420P10: + *color_format = XEVE_CF_YCBCR420; + *bit_depth = 10; + break; + case AV_PIX_FMT_YUV422P10: + *color_format = XEVE_CF_YCBCR422; + *bit_depth = 10; + break; + case AV_PIX_FMT_YUV444P10: + *color_format = XEVE_CF_YCBCR444; + *bit_depth = 10; + break; + default: + *color_format = XEVE_CF_UNKNOWN; + return -1; + } + + return 0; +} + +static int kbps_str_to_int(char *str) +{ + int kbps = 0; + if (strchr(str, 'K') || strchr(str, 'k')) { + char *tmp = strtok(str, "Kk "); + kbps = (int)(atof(tmp)); + } else if (strchr(str, 'M') || strchr(str, 'm')) { + char *tmp = strtok(str, "Mm "); + kbps = (int)(atof(tmp) * 1000); + } else + kbps = atoi(str); + + return kbps; +} + +/** + * Parse : separated list of key=value parameters + * + * @param[in] avctx context for logger + * @param[in] key + * @param[in] value + * @param[out] cdsc contains all Xeve MPEG-5 EVC encoder encoder parameters that + * should be initialized before the encoder is use + * + * @return 0 on success, negative value on failure + */ +static int parse_xeve_params(AVCodecContext *avctx, const char *key, const char *value, XEVE_CDSC* cdsc) +{ + XeveContext *xectx = NULL; + xectx = avctx->priv_data; + + if(!key) { + av_log(avctx, AV_LOG_ERROR, "Ivalid argument: key string is NULL\n"); + return XEVE_PARAM_BAD_VALUE; + } + if(!value) { + if (strcmp(key, "hash") == 0) { + xectx->hash = 1; + av_log(avctx, AV_LOG_INFO, "embedding signature is enabled\n"); + } else { + av_log(avctx, AV_LOG_ERROR, "Ivalid argument: value string is NULL\n"); + return XEVE_PARAM_BAD_VALUE; + } + } else if (strcmp(key, "vbv-bufsize") == 0 ) { + cdsc->param.vbv_bufsize = kbps_str_to_int((char *)value); + av_log(avctx, AV_LOG_INFO, "VBV buffer size: %dkbits\n", cdsc->param.vbv_bufsize); + } else if (strcmp(key, "rc-type") == 0 ) { + int rc_type = atoi(value); + if(rc_type < 0 || rc_type > 2) { + av_log(avctx, AV_LOG_ERROR, "Rate control type [ 0(rc_off) / 1(CBR) ] bad value: %d\n", rc_type); + return XEVE_PARAM_BAD_VALUE; + } + cdsc->param.rc_type = rc_type; + av_log(avctx, AV_LOG_INFO, "Rate control type [ 0(rc_off) / 1(CBR) ] : %d\n", rc_type); + } else if (strcmp(key, "bframes") == 0 ) { + int bframes = atoi(value); + if(bframes < 0) { + av_log(avctx, AV_LOG_ERROR, "bframes: bad value: %d\n", bframes); + return XEVE_PARAM_BAD_VALUE; + } + cdsc->param.bframes = bframes; + av_log(avctx, AV_LOG_INFO, "bframes : %d\n", bframes); + } else if (strcmp(key, "profile") == 0 ) { + const char *profile = value; + int profile_id; + av_log(avctx, AV_LOG_INFO, "profile (baseline, main): %s\n", profile); + profile_id = get_profile_id(profile); + if (profile_id < 0) { + av_log(avctx, AV_LOG_ERROR, "Invalid xeve param: profile(%s)\n", profile); + return XEVE_PARAM_BAD_VALUE; + } + xectx->profile_id = profile_id; + } else if (strcmp(key, "preset") == 0 ) { + const char *preset = value; + int preset_id; + av_log(avctx, AV_LOG_INFO, "Preset of xeve (fast, medium, slow, placebo): %s\n", preset); + preset_id = get_preset_id(preset); + if( preset_id < 0) { + av_log(avctx, AV_LOG_ERROR, "Invalid xeve param: preset(%s)\n", preset); + return XEVE_PARAM_BAD_VALUE; + } + xectx->preset_id = preset_id; + } else if (strcmp(key, "tune") == 0 ) { + const char *tune = value; + int tune_id; + av_log(avctx, AV_LOG_INFO, "Tune of xeve (psnr, zerolatency): %s\n", tune); + tune_id = get_tune_id(tune); + if( tune_id < 0) { + av_log(avctx, AV_LOG_ERROR, "Invalid xeve param: tune(%s)\n", tune); + return XEVE_PARAM_BAD_VALUE; + } + xectx->tune_id = tune_id; + } else if (strcmp(key, "bitrate") == 0 ) { + cdsc->param.bitrate = kbps_str_to_int((char *)value); + av_log(avctx, AV_LOG_INFO, "Bitrate = %dkbps\n", cdsc->param.bitrate); + } else if (strcmp(key, "q") == 0 || strcmp(key, "qp") == 0) { + int qp = atoi(value); + if(qp < 0 || qp > 51) { + av_log(avctx, AV_LOG_ERROR, "Invalid QP value (0~51) :%d\n", qp); + return XEVE_PARAM_BAD_VALUE; + } + cdsc->param.qp = qp; + av_log(avctx, AV_LOG_INFO, "QP value (0~51): %d\n", cdsc->param.qp); + } else { + av_log(avctx, AV_LOG_ERROR, "Unknown xeve codec option: %s\n", key); + return XEVE_PARAM_BAD_NAME; + } + + return 0; +} + +/** + * The function returns a pointer to the object of the XEVE_CDSC type. + * XEVE_CDSC contains all encoder parameters that should be initialized before the encoder is used. + * + * The field values of the XEVE_CDSC structure are populated based on: + * - the corresponding field values of the AvCodecConetxt structure, + * - the xeve encoder specific option values, + * (the full list of options available for xeve encoder is displayed after executing the command ./ffmpeg --help encoder = libxeve) + * - and the xeve encoder options specified as a list of key-value pairs following the xeve-params option + * + * The order of processing input data and populating the XEVE_CDSC structure + * 1) first, the fields of the AVCodecContext structure corresponding to the provided input options are processed, + * (i.e -pix_fmt yuv420p -s:v 1920x1080 -r 30 -profile:v 0) + * 2) then xeve-specific options added as AVOption to the xeve AVCodec implementation + * (i.e -threads_cnt 3 -preset 0) + * 3) finally, the options specified after the xeve-params option as the parameter list of type key value are processed + * (i.e -xeve-params "m=2:q=17") + * + * Keep in mind that, there are options that can be set in different ways. + * In this case, please follow the above-mentioned order of processing. + * The most recent assignments overwrite the previous values. + * + * @param[in] avctx codec context (AVCodecContext) + * @param[out] cdsc contains all Xeve MPEG-5 EVC encoder encoder parameters that should be initialized before the encoder is use + * + * @return 0 on success, negative error code on failure + */ +static int get_conf(AVCodecContext *avctx, XEVE_CDSC *cdsc) +{ + XeveContext *xectx = NULL; + int color_format; + int cpu_count = av_cpu_count(); + int ret; + + xectx = avctx->priv_data; + xectx->hash = 0; + + /* initialize xeve_param struct with default values */ + ret = xeve_param_default(&cdsc->param); + if (XEVE_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Cannot set_default parameter\n"); + goto ERR; + } + + /* read options from AVCodecContext */ + if(avctx->width > 0) + cdsc->param.w = xectx->width_luma = avctx->width; + + if(avctx->height > 0) + cdsc->param.h = xectx->height_luma = avctx->height; + + if(avctx->framerate.num > 0) { + // fps can be float number, but xeve API doesn't support it + cdsc->param.fps = (int)(((float)avctx->framerate.num / avctx->framerate.den) + 0.5); + } + + if(avctx->gop_size >= 0) { // GOP size (key-frame interval) + cdsc->param.keyint = avctx->gop_size; // 0: only one I-frame at the first time; 1: every frame is coded in I-frame + } + + if (avctx->max_b_frames == 0 || avctx->max_b_frames == 1 || avctx->max_b_frames == 3 || + avctx->max_b_frames == 7 || avctx->max_b_frames == 15) { // number of b-frames + cdsc->param.bframes = avctx->max_b_frames; + } else { + av_log(avctx, AV_LOG_ERROR, "Incorrect value for maximum number of B frames: (%d) \n" + "Acceptable values for bf option (maximum number of B frames) are 0,1,3,7 or 15\n", avctx->max_b_frames); + goto ERR; + } + + if (avctx->level >= 0) + cdsc->param.level_idc = avctx->level; + + ret = get_pix_fmt(avctx->pix_fmt, &color_format, &xectx->input_depth); + + if (ret != 0) { + av_log(avctx, AV_LOG_ERROR, "Unsupported pixel format.\n"); + goto ERR; + } + cdsc->param.cs = XEVE_CS_SET(color_format, xectx->input_depth, 0); + + if (avctx->rc_buffer_size > 0) { // VBV buf size + cdsc->param.vbv_bufsize = (int)(avctx->rc_buffer_size / 1000); + } + + // rc_type: Rate control type [ 0(CQP) / 1(ABR) / 2(CRF) ] + if (avctx->bit_rate > 0) { + if (avctx->bit_rate / 1000 > INT_MAX || avctx->rc_max_rate / 1000 > INT_MAX) { + av_log(avctx, AV_LOG_ERROR, "Not supported bitrate bit_rate and rc_max_rate > %d000\n", INT_MAX); + goto ERR; + } + cdsc->param.bitrate = (int)(avctx->bit_rate / 1000); + cdsc->param.rc_type = XEVE_RC_ABR; + } + + if (xectx->op_crf >= 0) { + cdsc->param.crf = xectx->op_crf; + cdsc->param.rc_type = XEVE_RC_CRF; + } + + if(avctx->thread_count <= 0) { + cdsc->param.threads = (cpu_count < XEVE_MAX_THREADS) ? cpu_count : XEVE_MAX_THREADS; + } else if(avctx->thread_count > XEVE_MAX_THREADS) + cdsc->param.threads = XEVE_MAX_THREADS; + else + cdsc->param.threads = avctx->thread_count; + + cdsc->param.cs = XEVE_CS_SET(color_format, cdsc->param.codec_bit_depth, 0); + cdsc->max_bs_buf_size = MAX_BS_BUF; + + if(avctx->profile == FF_PROFILE_EVC_BASELINE) + xectx->profile_id = XEVE_PROFILE_BASELINE; + else if(avctx->profile == FF_PROFILE_EVC_MAIN) + xectx->profile_id = XEVE_PROFILE_MAIN; + else { + av_log(avctx, AV_LOG_ERROR, "Unknown encoder profile (%d)\n" + "Acceptable values for profile option are 0 and 1 (0: baseline profile; 1: main profile)\n", avctx->profile); + goto ERR; + } + + if (xectx->op_preset) { // preset + xectx->preset_id = get_preset_id(xectx->op_preset); + } + + if (xectx->op_tune) { // tune + xectx->tune_id = get_tune_id(xectx->op_tune); + } + + ret = xeve_param_ppt(&cdsc->param, xectx->profile_id, xectx->preset_id, xectx->tune_id); + if (XEVE_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Cannot set profile(%d), preset(%d), tune(%d)\n", xectx->profile_id, xectx->preset_id, xectx->tune_id); + goto ERR; + } + + /* parse : separated list of key=value parameters and set values for created descriptor (XEVE_CDSC) */ + { + AVDictionaryEntry *en = NULL; + + // Start to parse xeve_params + while ((en = av_dict_get(xectx->xeve_params, "", en, AV_DICT_IGNORE_SUFFIX))) { + int parse_ret = parse_xeve_params(avctx, en->key, en->value, cdsc); + + switch (parse_ret) { + case XEVE_PARAM_BAD_NAME: + av_log(avctx, AV_LOG_WARNING, "Unknown option: %s.\n", en->key); + break; + case XEVE_PARAM_BAD_VALUE: + av_log(avctx, AV_LOG_WARNING, "Invalid value for %s: %s.\n", en->key, en->value); + break; + default: + break; + } + } + } + + return 0; + +ERR: + return AVERROR(EINVAL); +} + +/** + * Check encoder configuration + * + * @param[in] avctx context for logger + * @param[in] cdsc contains all encoder parameters that should be initialized before its use + * + * @return 0 on success, negative error code on failure + */ +static int check_conf(AVCodecContext *avctx, XEVE_CDSC *cdsc) +{ + int ret = 0; + int min_block_size = 4; + int pic_m; + + if(cdsc->param.profile == XEVE_PROFILE_BASELINE) { + if (cdsc->param.tool_amvr == 1) { + av_log(avctx, AV_LOG_ERROR, "AMVR cannot be on in base profile\n"); + ret = -1; + } + if (cdsc->param.tool_mmvd == 1) { + av_log(avctx, AV_LOG_ERROR, "MMVD cannot be on in base profile\n"); + ret = -1; + } + if (cdsc->param.tool_affine == 1) { + av_log(avctx, AV_LOG_ERROR, "Affine cannot be on in base profile\n"); + ret = -1; + } + if (cdsc->param.tool_dmvr == 1) { + av_log(avctx, AV_LOG_ERROR, "DMVR cannot be on in base profile\n"); + ret = -1; + } + if (cdsc->param.tool_admvp == 1) { + av_log(avctx, AV_LOG_ERROR, "ADMVP cannot be on in base profile\n"); + ret = -1; + } + if (cdsc->param.tool_hmvp == 1) { + av_log(avctx, AV_LOG_ERROR, "HMVP cannot be on in base profile\n"); + ret = -1; + } + if (cdsc->param.tool_addb == 1) { + av_log(avctx, AV_LOG_ERROR, "ADDB cannot be on in base profile\n"); + ret = -1; + } + if (cdsc->param.tool_alf == 1) { + av_log(avctx, AV_LOG_ERROR, "ALF cannot be on in base profile\n"); + ret = -1; + } + if (cdsc->param.tool_htdf == 1) { + av_log(avctx, AV_LOG_ERROR, "HTDF cannot be on in base profile\n"); + ret = -1; + } + if (cdsc->param.btt == 1) { + av_log(avctx, AV_LOG_ERROR, "BTT cannot be on in base profile\n"); + ret = -1; + } + if (cdsc->param.suco == 1) { + av_log(avctx, AV_LOG_ERROR, "SUCO cannot be on in base profile\n"); + ret = -1; + } + if (cdsc->param.tool_eipd == 1) { + av_log(avctx, AV_LOG_ERROR, "EIPD cannot be on in base profile\n"); + ret = -1; + } + if (cdsc->param.tool_iqt == 1) { + av_log(avctx, AV_LOG_ERROR, "IQT cannot be on in base profile\n"); + ret = -1; + } + if (cdsc->param.tool_cm_init == 1) { + av_log(avctx, AV_LOG_ERROR, "CM_INIT cannot be on in base profile\n"); + ret = -1; + } + if (cdsc->param.tool_adcc == 1) { + av_log(avctx, AV_LOG_ERROR, "ADCC cannot be on in base profile\n"); + ret = -1; + } + if (cdsc->param.tool_ats == 1) { + av_log(avctx, AV_LOG_ERROR, "ATS_INTRA cannot be on in base profile\n"); + ret = -1; + } + if (cdsc->param.ibc_flag == 1) { + av_log(avctx, AV_LOG_ERROR, "IBC cannot be on in base profile\n"); + ret = -1; + } + if (cdsc->param.tool_rpl == 1) { + av_log(avctx, AV_LOG_ERROR, "RPL cannot be on in base profile\n"); + ret = -1; + } + if (cdsc->param.tool_pocs == 1) { + av_log(avctx, AV_LOG_ERROR, "POCS cannot be on in base profile\n"); + ret = -1; + } + } else { + if (cdsc->param.tool_admvp == 0 && cdsc->param.tool_affine == 1) { + av_log(avctx, AV_LOG_ERROR, "AFFINE cannot be on when ADMVP is off\n"); + ret = -1; + } + if (cdsc->param.tool_admvp == 0 && cdsc->param.tool_amvr == 1) { + av_log(avctx, AV_LOG_ERROR, "AMVR cannot be on when ADMVP is off\n"); + ret = -1; + } + if (cdsc->param.tool_admvp == 0 && cdsc->param.tool_dmvr == 1) { + av_log(avctx, AV_LOG_ERROR, "DMVR cannot be on when ADMVP is off\n"); + ret = -1; + } + if (cdsc->param.tool_admvp == 0 && cdsc->param.tool_mmvd == 1) { + av_log(avctx, AV_LOG_ERROR, "MMVD cannot be on when ADMVP is off\n"); + ret = -1; + } + if (cdsc->param.tool_eipd == 0 && cdsc->param.ibc_flag == 1) { + av_log(avctx, AV_LOG_ERROR, "IBC cannot be on when EIPD is off\n"); + ret = -1; + } + if (cdsc->param.tool_iqt == 0 && cdsc->param.tool_ats == 1) { + av_log(avctx, AV_LOG_ERROR, "ATS cannot be on when IQT is off\n"); + ret = -1; + } + if (cdsc->param.tool_cm_init == 0 && cdsc->param.tool_adcc == 1) { + av_log(avctx, AV_LOG_ERROR, "ADCC cannot be on when CM_INIT is off\n"); + ret = -1; + } + } + + if (cdsc->param.btt == 1) { + if (cdsc->param.framework_cb_max && cdsc->param.framework_cb_max < 5) { + av_log(avctx, AV_LOG_ERROR, "Maximun Coding Block size cannot be smaller than 5\n"); + ret = -1; + } + if (cdsc->param.framework_cb_max > 7) { + av_log(avctx, AV_LOG_ERROR, "Maximun Coding Block size cannot be greater than 7\n"); + ret = -1; + } + if (cdsc->param.framework_cb_min && cdsc->param.framework_cb_min < 2) { + av_log(avctx, AV_LOG_ERROR, "Minimum Coding Block size cannot be smaller than 2\n"); + ret = -1; + } + if ((cdsc->param.framework_cb_max || cdsc->param.framework_cb_min) && + cdsc->param.framework_cb_min > cdsc->param.framework_cb_max) { + av_log(avctx, AV_LOG_ERROR, "Minimum Coding Block size cannot be greater than Maximum coding Block size\n"); + ret = -1; + } + if (cdsc->param.framework_cu14_max > 6) { + av_log(avctx, AV_LOG_ERROR, "Maximun 1:4 Coding Block size cannot be greater than 6\n"); + ret = -1; + } + if ((cdsc->param.framework_cb_max || cdsc->param.framework_cu14_max) && + cdsc->param.framework_cu14_max > cdsc->param.framework_cb_max) { + av_log(avctx, AV_LOG_ERROR, "Maximun 1:4 Coding Block size cannot be greater than Maximum coding Block size\n"); + ret = -1; + } + if (cdsc->param.framework_tris_max > 6) { + av_log(avctx, AV_LOG_ERROR, "Maximun Tri-split Block size be greater than 6\n"); + ret = -1; + } + if ((cdsc->param.framework_tris_max || cdsc->param.framework_cb_max) && + cdsc->param.framework_tris_max > cdsc->param.framework_cb_max) { + av_log(avctx, AV_LOG_ERROR, "Maximun Tri-split Block size cannot be greater than Maximum coding Block size\n"); + ret = -1; + } + if ((cdsc->param.framework_tris_min || cdsc->param.framework_cb_min) && + cdsc->param.framework_tris_min < cdsc->param.framework_cb_min + 2) { + av_log(avctx, AV_LOG_ERROR, "Maximun Tri-split Block size cannot be smaller than Minimum Coding Block size plus two\n"); + ret = -1; + } + if(cdsc->param.framework_cb_min) min_block_size = 1 << cdsc->param.framework_cb_min; + else min_block_size = 8; + } + + if (cdsc->param.suco == 1) { + if (cdsc->param.framework_suco_max > 6) { + av_log(avctx, AV_LOG_ERROR, "Maximun SUCO size cannot be greater than 6\n"); + ret = -1; + } + if (cdsc->param.framework_cb_max && cdsc->param.framework_suco_max > cdsc->param.framework_cb_max) { + av_log(avctx, AV_LOG_ERROR, "Maximun SUCO size cannot be greater than Maximum coding Block size\n"); + ret = -1; + } + if (cdsc->param.framework_suco_min < 4) { + av_log(avctx, AV_LOG_ERROR, "Minimun SUCO size cannot be smaller than 4\n"); + ret = -1; + } + if (cdsc->param.framework_cb_min && cdsc->param.framework_suco_min < cdsc->param.framework_cb_min) { + av_log(avctx, AV_LOG_ERROR, "Minimun SUCO size cannot be smaller than Minimum coding Block size\n"); + ret = -1; + } + if (cdsc->param.framework_suco_min > cdsc->param.framework_suco_max) { + av_log(avctx, AV_LOG_ERROR, "Minimum SUCO size cannot be greater than Maximum SUCO size\n"); + ret = -1; + } + } + + pic_m = (8 > min_block_size) ? min_block_size : 8; + if ((cdsc->param.w & (pic_m - 1)) != 0) { + av_log(avctx, AV_LOG_ERROR, "Current encoder does not support picture width, not multiple of max(8, minimum CU size)\n"); + ret = -1; + } + if ((cdsc->param.h & (pic_m - 1)) != 0) { + av_log(avctx, AV_LOG_ERROR, "Current encoder does not support picture height, not multiple of max(8, minimum CU size)\n"); + ret = -1; + } + + return ret; +} + +/** + * Set XEVE_CFG_SET_USE_PIC_SIGNATURE for encoder + * + * @param[in] logger context + * @param[in] id XEVE encodec instance identifier + * @param[in] ctx the structure stores all the states associated with the instance of Xeve MPEG-5 EVC encoder + * + * @return 0 on success, negative error code on failure + */ +static int set_extra_config(AVCodecContext* avctx, XEVE id, XeveContext *ctx) +{ + int ret, size, value; + + if(ctx->hash) { + value = 1; + size = 4; + ret = xeve_config(id, XEVE_CFG_SET_USE_PIC_SIGNATURE, &value, &size); + if(XEVE_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to set config for picture signature\n"); + return -1; + } + } + + return 0; +} + +/** + * Convert FFmpeg pixel format (AVPixelFormat) into XEVE pre-defined color space + * + * @param[in] px_fmt pixel format (@see https://ffmpeg.org/doxygen/trunk/pixfmt_8h.html#a9a8e335cf3be472042bc9f0cf80cd4c5) + * + * @return XEVE pre-defined color space (@see xeve.h) on success, XEVE_CF_UNKNOWN on failure + */ +static int xeve_color_space(enum AVPixelFormat pix_fmt) +{ + /* color space of input image */ + int cs = XEVE_CF_UNKNOWN; + + switch (pix_fmt) { + case AV_PIX_FMT_YUV420P: + cs = XEVE_CS_YCBCR420; + break; + case AV_PIX_FMT_YUV420P10: +#if AV_HAVE_BIGENDIAN + cs = XEVE_CS_SET(XEVE_CF_YCBCR420, 10, 1); +#else + cs = XEVE_CS_YCBCR420_10LE; +#endif + + break; + case AV_PIX_FMT_YUV420P12: +#if AV_HAVE_BIGENDIAN + cs = XEVE_CS_SET(XEVE_CF_YCBCR420, 12, 1); +#else + cs = XEVE_CS_YCBCR420_12LE; +#endif + + break; + case AV_PIX_FMT_YUV422P: + cs = XEVE_CS_YCBCR422; + break; + case AV_PIX_FMT_YUV422P10: +#if AV_HAVE_BIGENDIAN + cs = XEVE_CS_SET(XEVE_CF_YCBCR422, 10, 1); +#else + cs = XEVE_CS_YCBCR422_10LE; +#endif + + break; + case AV_PIX_FMT_YUV422P12: +#if AV_HAVE_BIGENDIAN + cs = XEVE_CS_SET(XEVE_CF_YCBCR422, 12, 1); +#else + cs = XEVE_CS_SET(XEVE_CF_YCBCR422, 12, 0); +#endif + + break; + case AV_PIX_FMT_GBRP: + case AV_PIX_FMT_GBRP10: + case AV_PIX_FMT_GBRP12: + cs = XEVE_CF_UNKNOWN; + break; + case AV_PIX_FMT_YUV444P: + cs = XEVE_CF_YCBCR444; + break; + case AV_PIX_FMT_YUV444P10: +#if AV_HAVE_BIGENDIAN + cs = XEVE_CS_SET(XEVE_CF_YCBCR444, 10, 1); +#else + cs = XEVE_CS_YCBCR444_10LE; +#endif + + break; + case AV_PIX_FMT_YUV444P12: +#if AV_HAVE_BIGENDIAN + cs = XEVE_CS_SET(XEVE_CF_YCBCR444, 12, 1); +#else + cs = XEVE_CS_SET(XEVE_CF_YCBCR444, 12, 0); +#endif + + break; + case AV_PIX_FMT_GRAY8: + cs = XEVE_CF_YCBCR400; + break; + case AV_PIX_FMT_GRAY10: +#if AV_HAVE_BIGENDIAN + cs = XEVE_CS_SET(XEVE_CF_YCBCR400, 10, 1); +#else + cs = XEVE_CS_YCBCR400_10LE; +#endif + + break; + case AV_PIX_FMT_GRAY12: +#if AV_HAVE_BIGENDIAN + cs = XEVE_CS_SET(XEVE_CF_YCBCR400, 12, 1); +#else + cs = XEVE_CS_YCBCR400_12LE; +#endif + + break; + default: + cs = XEVE_CF_UNKNOWN; + break; + } + + return cs; +} + +/** + * @brief Switch encoder to bumping mode + * + * @param id XEVE encodec instance identifier + * @return 0 on success, negative error code on failure + */ +static int setup_bumping(XEVE id) +{ + int val, size; + val = 1; + size = sizeof(int); + if(XEVE_FAILED(xeve_config(id, XEVE_CFG_SET_FORCE_OUT, (void *)(&val), &size))) + return -1; + + return 0; +} + +/** + * @brief Initialize eXtra-fast Essential Video Encoder codec + * Create an encoder instance and allocate all the needed resources + * + * @param avctx codec context + * @return 0 on success, negative error code on failure + */ +static av_cold int libxeve_init(AVCodecContext *avctx) +{ + XeveContext *xectx = avctx->priv_data; + unsigned char *bs_buf = NULL; + int i, val = 0; + int shift_h = 0; + int shift_v = 0; + XEVE_IMGB *imgb = NULL; + + XEVE_CDSC *cdsc = &(xectx->cdsc); + + if(avctx->pix_fmt != AV_PIX_FMT_YUV420P && avctx->pix_fmt != AV_PIX_FMT_YUV420P10) { + av_log(avctx, AV_LOG_ERROR, "Invalid pixel format: %s\n", av_get_pix_fmt_name(avctx->pix_fmt)); + goto ERR; + } + + /* allocate bitstream buffer */ + bs_buf = (unsigned char *)malloc(MAX_BS_BUF); + if(bs_buf == NULL) { + av_log(avctx, AV_LOG_ERROR, "Cannot allocate bitstream buffer\n"); + goto ERR; + } + + /* read configurations and set values for created descriptor (XEVE_CDSC) */ + val = get_conf(avctx, cdsc); + if (val != XEVE_OK) { + av_log(avctx, AV_LOG_ERROR, "Cannot get configuration\n"); + goto ERR; + } + + if (check_conf(avctx, cdsc) != 0) { + av_log(avctx, AV_LOG_ERROR, "Invalid configuration\n"); + goto ERR; + } + + /* create encoder */ + xectx->id = xeve_create(cdsc, NULL); + if(xectx->id == NULL) { + av_log(avctx, AV_LOG_ERROR, "Cannot create XEVE encoder\n"); + goto ERR; + } + + if(set_extra_config(avctx, xectx->id, xectx)) { + av_log(avctx, AV_LOG_ERROR, "Cannot set extra configuration\n"); + goto ERR; + } + + xectx->bitb.addr = bs_buf; + xectx->bitb.bsize = MAX_BS_BUF; + + if(av_pix_fmt_get_chroma_sub_sample(avctx->pix_fmt, &shift_h, &shift_v)) { + av_log(avctx, AV_LOG_ERROR, "Failed to get chroma shift\n"); + goto ERR; + } + + // YUV format explanation + // shift_h == 1 && shift_v == 1 : YUV420 + // shift_h == 1 && shift_v == 0 : YUV422 + // shift_h == 0 && shift_v == 0 : YUV444 + // + xectx->width_chroma = AV_CEIL_RSHIFT(xectx->width_luma, shift_h); + xectx->height_chroma = AV_CEIL_RSHIFT(xectx->height_luma, shift_v); + + /* set default values for input image buffer */ + imgb = &xectx->imgb; + imgb->cs = xeve_color_space(avctx->pix_fmt); + imgb->np = 3; /* only for yuv420p, yuv420ple */ + + for (i = 0; i < imgb->np; i++) + imgb->x[i] = imgb->y[i] = 0; + + imgb->w[0] = imgb->aw[0] = xectx->width_luma; + imgb->w[1] = imgb->w[2] = imgb->aw[1] = imgb->aw[2] = xectx->width_chroma; + imgb->h[0] = imgb->ah[0] = xectx->height_luma; + imgb->h[1] = imgb->h[2] = imgb->ah[1] = imgb->ah[2] = xectx->height_chroma; + + xectx->encod_frames = 0; + xectx->bytes_total = 0; + xectx->state = STATE_ENCODING; + xectx->packet_count = 0; + xectx->bitrate = 0; + + return 0; + +ERR: + if(bs_buf) + free(bs_buf); + + return -1; +} + +/** + * Encode raw data frame into EVC packet + * + * @param[in] avctx codec context + * @param[out] pkt output AVPacket containing encoded data + * @param[in] frame AVFrame containing the raw data to be encoded + * @param[out] got_packet encoder sets to 0 or 1 to indicate that a + * non-empty packet was returned in pkt + * + * @return 0 on success, negative error code on failure + */ +static int libxeve_encode(AVCodecContext *avctx, AVPacket *pkt, + const AVFrame *frame, int *got_packet) +{ + XeveContext *xectx = NULL; + int ret = -1; + int xeve_cs; + if(avctx == NULL || pkt == NULL || got_packet == NULL) { + av_log(avctx, AV_LOG_ERROR, "Invalid arguments\n"); + return -1; + } + xectx = avctx->priv_data; + if(xectx == NULL) { + av_log(avctx, AV_LOG_ERROR, "Invalid XEVE context\n"); + return -1; + } + if(xectx->state == STATE_SKIPPING && frame ) { + xectx->state = STATE_ENCODING; // Entering encoding process + } else if(xectx->state == STATE_ENCODING && frame == NULL) { + if (setup_bumping(xectx->id) == 0) + xectx->state = STATE_BUMPING; // Entering bumping process + else { + av_log(avctx, AV_LOG_ERROR, "Failed to setup bumping\n"); + xectx->state = STATE_SKIPPING; + } + } + + if(xectx->state == STATE_ENCODING) { + const AVPixFmtDescriptor *pixel_fmt_desc = av_pix_fmt_desc_get (frame->format); + if(!pixel_fmt_desc) { + av_log(avctx, AV_LOG_ERROR, "Invalid pixel format descriptor for pixel format: %s\n", av_get_pix_fmt_name(avctx->pix_fmt)); + return -1; + } + + xeve_cs = xeve_color_space(avctx->pix_fmt); + if(xeve_cs != XEVE_CS_YCBCR420 && xeve_cs != XEVE_CS_YCBCR420_10LE) { + av_log(avctx, AV_LOG_ERROR, "Invalid pixel format: %s\n", av_get_pix_fmt_name(avctx->pix_fmt)); + return -1; + } + + { + int i; + XEVE_IMGB *imgb = NULL; + + imgb = &xectx->imgb; + + for (i = 0; i < imgb->np; i++) { + imgb->a[i] = frame->data[i]; + imgb->s[i] = frame->linesize[i]; + } + + if(xectx->id == NULL) { + av_log(avctx, AV_LOG_ERROR, "Invalid XEVE encoder\n"); + return -1; + } + + imgb->ts[0] = frame->pts; + imgb->ts[1] = 0; + imgb->ts[2] = 0; + imgb->ts[3] = 0; + + /* push image to encoder */ + ret = xeve_push(xectx->id, imgb); + if(XEVE_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "xeve_push() failed\n"); + return -1; + } + } + } + if(xectx->state == STATE_ENCODING || xectx->state == STATE_BUMPING) { + + /* encoding */ + ret = xeve_encode(xectx->id, &(xectx->bitb), &(xectx->stat)); + if(XEVE_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "xeve_encode() failed\n"); + return -1; + } + + xectx->encod_frames++; + + /* store bitstream */ + if (ret == XEVE_OK_OUT_NOT_AVAILABLE) { // Return OK but picture is not available yet + *got_packet = 0; + return 0; + } else if(ret == XEVE_OK) { + int av_pic_type; + + if(xectx->stat.write > 0) { + xectx->bytes_total += xectx->stat.write; + + ret = av_grow_packet(pkt, xectx->stat.write); + if (ret < 0) { + av_log(avctx, AV_LOG_ERROR, "Can't allocate memory for AVPacket data\n"); + return ret; + } + + memcpy(pkt->data, xectx->bitb.addr, xectx->stat.write); + + pkt->pts = xectx->bitb.ts[0]; + pkt->dts = xectx->bitb.ts[1]; + + xectx->bitrate += (xectx->stat.write - xectx->stat.sei_size); + + switch(xectx->stat.stype) { + case XEVE_ST_I: + av_pic_type = AV_PICTURE_TYPE_I; + pkt->flags |= AV_PKT_FLAG_KEY; + break; + case XEVE_ST_P: + av_pic_type = AV_PICTURE_TYPE_P; + break; + case XEVE_ST_B: + av_pic_type = AV_PICTURE_TYPE_B; + break; + case XEVE_ST_UNKNOWN: + av_log(avctx, AV_LOG_ERROR, "Unknown slice type\n"); + return -1; + } + + ff_side_data_set_encoder_stats(pkt, xectx->stat.qp * FF_QP2LAMBDA, NULL, 0, av_pic_type); + + xectx->bitrate += (xectx->stat.write - xectx->stat.sei_size); + + *got_packet = 1; + xectx->packet_count++; + } + } else if (ret == XEVE_OK_NO_MORE_FRM) { + // Return OK but no more frames + return 0; + } else { + av_log(avctx, AV_LOG_ERROR, "Invalid return value: %d\n", ret); + return -1; + } + } else { + av_log(avctx, AV_LOG_ERROR, "Udefined encoder state\n", xectx->state); + return -1; + } + + return 0; +} + +/** + * Destroy the encoder and release all the allocated resources + * + * @param avctx codec context + * @return 0 on success, negative error code on failure + */ +static av_cold int libxeve_close(AVCodecContext *avctx) +{ + XeveContext *xectx = avctx->priv_data; + + xeve_delete(xectx->id); + + if(xectx->bitb.addr) + free(xectx->bitb.addr); /* release bitstream buffer */ + + return 0; +} + +#define OFFSET(x) offsetof(XeveContext, x) +#define VE AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_ENCODING_PARAM + +// Example of using: ./ffmpeg -xeve-params "m=2:q=17" +// Consider using following options (./ffmpeg --help encoder=libxeve) +// +static const AVOption xeve_options[] = { + { "preset", "Encoding preset for setting encoding speed [fast, medium, slow, placebo]", OFFSET(op_preset), AV_OPT_TYPE_STRING, { .str = "medium" }, 0, 0, VE }, + { "tune", "Tuneing parameter for special purpose operation [psnr, zerolatency]", OFFSET(op_tune), AV_OPT_TYPE_STRING, { 0 }, 0, 0, VE}, + { "qp", "quantization parameter qp <0..51> [default: 32]", OFFSET(op_qp), AV_OPT_TYPE_INT, { .i64 = 32 }, 0, 51, VE }, + { "crf", "constant rate factor <-1..51> [default: 32]", OFFSET(op_crf), AV_OPT_TYPE_INT, { .i64 = -1 }, -1, 51, VE }, + { "xeve-params", "override the xeve configuration using a : separated list of key=value parameters", OFFSET(xeve_params), AV_OPT_TYPE_DICT, { 0 }, 0, 0, VE }, + { NULL } +}; + +static const AVClass xeve_class = { + .class_name = "libxeve", + .item_name = av_default_item_name, + .option = xeve_options, + .version = LIBAVUTIL_VERSION_INT, +}; + +/** + * libavcodec generic global options, which can be set on all the encoders and decoders + * @see https://www.ffmpeg.org/ffmpeg-codecs.html#Codec-Options + */ +static const FFCodecDefault xeve_defaults[] = { + { "b", "0" }, // bitrate + { "g", "0" }, // gop_size (key-frame interval 0: only one I-frame at the first time; 1: every frame is coded in I-frame) + { "bf", "15"}, // bframes (0: no B-frames) + { "profile", "0"}, // encoder codec profile (0: baselie; 1: main) + { "threads", "0"}, // number of threads to be used (0: automatically select the number of threads to set) + { NULL }, +}; + +FFCodec ff_libxeve_encoder = { + .p.name = "libxeve", + .p.long_name = NULL_IF_CONFIG_SMALL("libxeve MPEG-5 EVC"), + .p.type = AVMEDIA_TYPE_VIDEO, + .p.id = AV_CODEC_ID_EVC, + .init = libxeve_init, + FF_CODEC_ENCODE_CB(libxeve_encode), + .close = libxeve_close, + .priv_data_size = sizeof(XeveContext), + .p.priv_class = &xeve_class, + .defaults = xeve_defaults, + .caps_internal = AV_CODEC_CAP_DELAY | AV_CODEC_CAP_AUTO_THREADS | + AV_CODEC_CAP_ENCODER_REORDERED_OPAQUE, + .p.wrapper_name = "libxeve", +}; diff --git a/libavcodec/parsers.c b/libavcodec/parsers.c index 6b40c18d80..295fa33f52 100644 --- a/libavcodec/parsers.c +++ b/libavcodec/parsers.c @@ -73,6 +73,7 @@ extern const AVCodecParser ff_vp9_parser; extern const AVCodecParser ff_webp_parser; extern const AVCodecParser ff_xbm_parser; extern const AVCodecParser ff_xma_parser; +extern const AVCodecParser ff_evc_parser; #include "libavcodec/parser_list.c" -- 2.17.1 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 6+ messages in thread
[parent not found: <CGME20220420095424eucas1p13b9dc5cacc7b763a05ac32ca25b14e69@eucas1p1.samsung.com>]
* [FFmpeg-devel] [PATCH 2/3] Provided support for MPEG-5 EVC (Essential Video Coding) codec [not found] <CGME20220420095424eucas1p13b9dc5cacc7b763a05ac32ca25b14e69@eucas1p1.samsung.com> @ 2022-04-20 9:54 ` Dawid Kozinski 0 siblings, 0 replies; 6+ messages in thread From: Dawid Kozinski @ 2022-04-20 9:54 UTC (permalink / raw) To: ffmpeg-devel - Added xeve encoder wrapper - Added xevd dencoder wrapper - Added documentation for xeve and xevd wrappers - Added parser for EVC format - Changes in project configuration file and libavcodec Makefile Signed-off-by: Dawid Kozinski <d.kozinski@samsung.com> --- configure | 10 + doc/decoders.texi | 23 + doc/encoders.texi | 112 ++++ doc/general_contents.texi | 19 + libavcodec/Makefile | 3 + libavcodec/allcodecs.c | 2 + libavcodec/avcodec.h | 3 + libavcodec/evc_parser.c | 514 +++++++++++++++++ libavcodec/libxevd.c | 443 +++++++++++++++ libavcodec/libxeve.c | 1139 +++++++++++++++++++++++++++++++++++++ libavcodec/parsers.c | 1 + 11 files changed, 2269 insertions(+) create mode 100644 libavcodec/evc_parser.c create mode 100644 libavcodec/libxevd.c create mode 100644 libavcodec/libxeve.c diff --git a/configure b/configure index 7a62f0c248..ef3b3cfc1d 100755 --- a/configure +++ b/configure @@ -289,6 +289,8 @@ External library support: --enable-libwebp enable WebP encoding via libwebp [no] --enable-libx264 enable H.264 encoding via x264 [no] --enable-libx265 enable HEVC encoding via x265 [no] + --enable-libxeve enable MPEG-5 Essential Video Coding (EVC) encoding via libxeve [no] + --enable-libxevd enable MPEG-5 Essential Video Coding (EVC) decoding via libxevd [no] --enable-libxavs enable AVS encoding via xavs [no] --enable-libxavs2 enable AVS2 encoding via xavs2 [no] --enable-libxcb enable X11 grabbing using XCB [autodetect] @@ -1880,6 +1882,8 @@ EXTERNAL_LIBRARY_LIST=" openssl pocketsphinx vapoursynth + libxeve + libxevd " HWACCEL_AUTODETECT_LIBRARY_LIST=" @@ -2453,6 +2457,7 @@ CONFIG_EXTRA=" h264pred h264qpel hevcparse + evcparse hpeldsp huffman huffyuvdsp @@ -3252,6 +3257,7 @@ mpegaudio_parser_select="mpegaudioheader" mpegvideo_parser_select="mpegvideo" mpeg4video_parser_select="h263dsp mpegvideo qpeldsp" vc1_parser_select="vc1dsp" +evc_parser_select="evcparse" # bitstream_filters aac_adtstoasc_bsf_select="adts_header mpeg4audio" @@ -3377,6 +3383,8 @@ libx264_encoder_select="atsc_a53" libx264rgb_encoder_deps="libx264" libx264rgb_encoder_select="libx264_encoder" libx265_encoder_deps="libx265" +libxeve_encoder_deps="libxeve" +libxevd_decoder_deps="libxevd" libxavs_encoder_deps="libxavs" libxavs2_encoder_deps="libxavs2" libxvid_encoder_deps="libxvid" @@ -6659,6 +6667,8 @@ enabled libx264 && { check_pkg_config libx264 x264 "stdint.h x264.h" x check_cpp_condition libx262 x264.h "X264_MPEG2" enabled libx265 && require_pkg_config libx265 x265 x265.h x265_api_get && require_cpp_condition libx265 x265.h "X265_BUILD >= 70" +enabled libxeve && require_pkg_config libxeve "xeve >= 1.0.0" "xeve.h" xeve_encode +enabled libxevd && require_pkg_config libxevd "xevd >= 1.0.0" "xevd.h" xevd_decode enabled libxavs && require libxavs "stdint.h xavs.h" xavs_encoder_encode "-lxavs $pthreads_extralibs $libm_extralibs" enabled libxavs2 && require_pkg_config libxavs2 "xavs2 >= 1.3.0" "stdint.h xavs2.h" xavs2_api_get enabled libxvid && require libxvid xvid.h xvid_global -lxvidcore diff --git a/doc/decoders.texi b/doc/decoders.texi index 49ab735dce..9277e20be9 100644 --- a/doc/decoders.texi +++ b/doc/decoders.texi @@ -126,6 +126,29 @@ Set amount of frame threads to use during decoding. The default value is 0 (auto @end table +@section libxevd + +eXtra-fast Essential Video Decoder (XEVD) MPEG-5 EVC decoder wrapper. + +This decoder requires the presence of the libxevd headers and library +during configuration. You need to explicitly configure the build with +@option{--enable-libxevd}. + +The xevd project website is at @url{https://github.com/mpeg5/xevd}. + +@subsection Options + +The following options are supported by the libxevd wrapper. +The xevd-equivalent options or values are listed in parentheses for easy migration. + +To get a more accurate and extensive documentation of the libxevd options, +invoke the command @code{xevd_app --help} or consult the libxevd documentation. + +@table @option +@item threads (@emph{threads}) +Force to use a specific number of threads +@end table + @c man end VIDEO DECODERS @chapter Audio Decoders diff --git a/doc/encoders.texi b/doc/encoders.texi index e3b61de5a1..87aaaa0dbc 100644 --- a/doc/encoders.texi +++ b/doc/encoders.texi @@ -2815,6 +2815,118 @@ ffmpeg -i input -c:v libxavs2 -xavs2-params RdoqLevel=0 output.avs2 @end example @end table +@section libxeve + +eXtra-fast Essential Video Encoder (XEVE) MPEG-5 EVC encoder wrapper. + +This encoder requires the presence of the libxeve headers and library +during configuration. You need to explicitly configure the build with +@option{--enable-libxeve}. + +@float NOTE +Many libxeve encoder options are mapped to FFmpeg global codec options, +while unique encoder options are provided through private options. +Additionally the xeve-params private options allows one to pass a list +of key=value tuples as accepted by the libxeve @code{parse_xeve_params} function. +@end float + +The xeve project website is at @url{https://github.com/mpeg5/xeve}. + +@subsection Options + +The following options are supported by the libxeve wrapper. +The xeve-equivalent options or values are listed in parentheses for easy migration. + +@float NOTE +To reduce the duplication of documentation, only the private options +and some others requiring special attention are documented here. For +the documentation of the undocumented generic options, see +@ref{codec-options,,the Codec Options chapter}. +@end float + +@float NOTE +To get a more accurate and extensive documentation of the libxeve options, +invoke the command @code{xeve_app --help} or consult the libxeve documentation. +@end float + +@table @option +@item b (@emph{bitrate}) +Set target video bitrate in bits/s. +Note that FFmpeg’s b option is expressed in bits/s, while xeve’s bitrate is in kilobits/s. + +@item bf (@emph{bframes}) +Set the maximum number of B frames (1,3,7,15). + +@item g (@emph{keyint}) +Set the GOP size (I-picture period). + +@item preset (@emph{preset}) +Set the xeve preset. +Set the encoder preset value to determine encoding speed [fast, medium, slow, placebo] + +@item tune (@emph{tune}) +Set the encoder tune parameter [psnr, zerolatency] + +@item profile (@emph{profile}) +Set the encoder profile [0: baselie; 1: main] + +@item crf (@emph{crf}) +Set the quality for constant quality mode. +Constant rate factor <-1..51> [default: 32] + +@item qp (@emph{qp}) +Set constant quantization rate control method parameter. +Quantization parameter qp <0..51> [default: 32] + +@item threads (@emph{threads}) +Force to use a specific number of threads + +@item xeve-params +Set xeve options using a list of @var{key}=@var{value} couples separated +by ":". + +For example to specify libxeve encoding options with @option{-xeve-params}: + +@example +ffmpeg -i input -c:v libxeve -xeve-params profile=main:tune=zerolatency:preset=fast:q=17:bframes=15:bitrate=100k output.mp4 +@end example +@end table + +@subsubsection Supported key values for xeve-params + +Set xeve options using a list of @var{key}=@var{value} couples separated + +@table @option + +@item hash +Embed the picture signature (HASH) for conformance checking in decoding, (0: Off, 1: On) + +@item vbv-bufsize +Set the VBV buffer size: Kbits(none,K,k), Mbits(M,m) + +@item rc-type +Set the rate control type, (0: OFF, 1: ABR, 2: CRF) + +@item bframes +Set the maximum number of B frames (1,3,7,15) + +@item profile +Set the PROFILE (main, baseline) + +@item preset +Set the encoder PRESET (fast, medium, slow, placebo) + +@item tune +Set the encoder TUNE (psnr, zerolatency) + +@item bitrate +Set the Bitrate in terms of kilo-bits per second: Kbps(none,K,k), Mbps(M,m) + +@item qp +Set the QP value (0~51) + +@end table + @section libxvid Xvid MPEG-4 Part 2 encoder wrapper. diff --git a/doc/general_contents.texi b/doc/general_contents.texi index df1692c8df..544b496f0c 100644 --- a/doc/general_contents.texi +++ b/doc/general_contents.texi @@ -336,6 +336,22 @@ libxavs2 is under the GNU Public License Version 2 or later details), you must upgrade FFmpeg's license to GPL in order to use it. @end float +@section eXtra-fast Essential Video Encoder (XEVE) + +FFmpeg can make use of the XEVE library for EVC video encoding. + +Go to @url{https://github.com/mpeg5/xeve} and follow the instructions for +installing the XEVE library. Then pass @code{--enable-libxeve} to configure to +enable it. + +@section eXtra-fast Essential Video Decoder (XEVD) + +FFmpeg can make use of the XEVD library for EVC video decoding. + +Go to @url{https://github.com/mpeg5/xevd} and follow the instructions for +installing the XEVD library. Then pass @code{--enable-libxevd} to configure to +enable it. + @section ZVBI ZVBI is a VBI decoding library which can be used by FFmpeg to decode DVB @@ -583,6 +599,7 @@ library: @item raw DTS @tab X @tab X @item raw DTS-HD @tab @tab X @item raw E-AC-3 @tab X @tab X +@item raw EVC @tab X @tab X @item raw FLAC @tab X @tab X @item raw GSM @tab @tab X @item raw H.261 @tab X @tab X @@ -912,6 +929,8 @@ following image formats are supported: @item Electronic Arts TQI video @tab @tab X @item Escape 124 @tab @tab X @item Escape 130 @tab @tab X +@item EVC / MPEG-5 Part 1 @tab X @tab X + @tab encoding and decoding supported through external libraries libxeve and libxevd @item FFmpeg video codec #1 @tab X @tab X @tab lossless codec (fourcc: FFV1) @item Flash Screen Video v1 @tab X @tab X diff --git a/libavcodec/Makefile b/libavcodec/Makefile index 3adf1536d8..8134f40e3e 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -1087,6 +1087,8 @@ OBJS-$(CONFIG_LIBWEBP_ANIM_ENCODER) += libwebpenc_common.o libwebpenc_anim OBJS-$(CONFIG_LIBX262_ENCODER) += libx264.o OBJS-$(CONFIG_LIBX264_ENCODER) += libx264.o OBJS-$(CONFIG_LIBX265_ENCODER) += libx265.o +OBJS-$(CONFIG_LIBXEVE_ENCODER) += libxeve.o +OBJS-$(CONFIG_LIBXEVD_DECODER) += libxevd.o OBJS-$(CONFIG_LIBXAVS_ENCODER) += libxavs.o OBJS-$(CONFIG_LIBXAVS2_ENCODER) += libxavs2.o OBJS-$(CONFIG_LIBXVID_ENCODER) += libxvid.o @@ -1152,6 +1154,7 @@ OBJS-$(CONFIG_VP9_PARSER) += vp9_parser.o OBJS-$(CONFIG_WEBP_PARSER) += webp_parser.o OBJS-$(CONFIG_XBM_PARSER) += xbm_parser.o OBJS-$(CONFIG_XMA_PARSER) += xma_parser.o +OBJS-$(CONFIG_EVC_PARSER) += evc_parser.o # bitstream filters OBJS-$(CONFIG_AAC_ADTSTOASC_BSF) += aac_adtstoasc_bsf.o diff --git a/libavcodec/allcodecs.c b/libavcodec/allcodecs.c index d1e10197de..12e6c92f65 100644 --- a/libavcodec/allcodecs.c +++ b/libavcodec/allcodecs.c @@ -783,6 +783,8 @@ extern LIBX264_CONST AVCodec ff_libx264_encoder; #endif extern const AVCodec ff_libx264rgb_encoder; extern AVCodec ff_libx265_encoder; +extern const AVCodec ff_libxeve_encoder; +extern const AVCodec ff_libxevd_decoder; extern const AVCodec ff_libxavs_encoder; extern const AVCodec ff_libxavs2_encoder; extern const AVCodec ff_libxvid_encoder; diff --git a/libavcodec/avcodec.h b/libavcodec/avcodec.h index 7ee8bc2b7c..a070e0f1d7 100644 --- a/libavcodec/avcodec.h +++ b/libavcodec/avcodec.h @@ -1643,6 +1643,9 @@ typedef struct AVCodecContext { #define FF_PROFILE_KLVA_SYNC 0 #define FF_PROFILE_KLVA_ASYNC 1 +#define FF_PROFILE_EVC_BASELINE 0 +#define FF_PROFILE_EVC_MAIN 1 + /** * level * - encoding: Set by user. diff --git a/libavcodec/evc_parser.c b/libavcodec/evc_parser.c new file mode 100644 index 0000000000..7c41eb9af4 --- /dev/null +++ b/libavcodec/evc_parser.c @@ -0,0 +1,514 @@ +/* + * EVC AVC format parser + * + * Copyright (C) 2021 Dawid Kozinski <d.kozinski@samsung.com> + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include <stdint.h> + +#include <xevd.h> + +#include "libavutil/common.h" + +#include "parser.h" +#include "golomb.h" + +#define EVC_NAL_HEADER_SIZE 2 /* byte */ +#define MAX_SPS_CNT 16 /* defined value in EVC standard */ + +// The sturcture reflects SPS RBSP(raw byte sequence payload) layout +// @see ISO_IEC_23094-1 section 7.3.2.1 +// +// The following descriptors specify the parsing process of each element +// u(n) - unsigned integer using n bits +// ue(v) - unsigned integer 0-th order Exp_Golomb-coded syntax element with the left bit first +typedef struct EVCParserSPS { + int sps_seq_parameter_set_id; // ue(v) + int profile_idc; // u(8) + int level_idc; // u(8) + int toolset_idc_h; // u(32) + int toolset_idc_l; // u(32) + int chroma_format_idc; // ue(v) + int pic_width_in_luma_samples; // ue(v) + int pic_height_in_luma_samples; // ue(v) + int bit_depth_luma_minus8; // ue(v) + int bit_depth_chroma_minus8; // ue(v) + + int sps_btt_flag; // u(1) + int log2_ctu_size_minus5; // ue(v) + int log2_min_cb_size_minus2; // ue(v) + int log2_diff_ctu_max_14_cb_size; // ue(v) + int log2_diff_ctu_max_tt_cb_size; // ue(v) + int log2_diff_min_cb_min_tt_cb_size_minus2; // ue(v) + + int sps_suco_flag; // u(1) + int log2_diff_ctu_size_max_suco_cb_size; // ue(v) + int log2_diff_max_suco_min_suco_cb_size; // ue(v) + + int sps_admvp_flag; // u(1) + int sps_affine_flag; // u(1) + int sps_amvr_flag; // u(1) + int sps_dmvr_flag; // u(1) + int sps_mmvd_flag; // u(1) + int sps_hmvp_flag; // u(1) + + int sps_eipd_flag; // u(1) + int sps_ibc_flag; // u(1) + int log2_max_ibc_cand_size_minus2; // ue(v) + + int sps_cm_init_flag; // u(1) + int sps_adcc_flag; // u(1) + + int sps_iqt_flag; // u(1) + int sps_ats_flag; // u(1) + + int sps_addb_flag; // u(1) + int sps_alf_flag; // u(1) + int sps_htdf_flag; // u(1) + int sps_rpl_flag; // u(1) + int sps_pocs_flag; // u(1) + int sps_dquant_flag; // u(1) + int sps_dra_flag; // u(1) + + int log2_max_pic_order_cnt_lsb_minus4; // ue(v) + int log2_sub_gop_length; // ue(v) + int log2_ref_pic_gap_length; // ue(v) + + int max_num_tid0_ref_pics; // ue(v) + + int sps_max_dec_pic_buffering_minus1; // ue(v) + int long_term_ref_pic_flag; // u(1) + int rpl1_same_as_rpl0_flag; // u(1) + int num_ref_pic_list_in_sps[2]; // ue(v) + + int picture_cropping_flag; // u(1) + int picture_crop_left_offset; // ue(v) + int picture_crop_right_offset; // ue(v) + int picture_crop_top_offset; // ue(v) + int picture_crop_bottom_offset; // ue(v) + +} EVCParserSPS; + +typedef struct EVCParserContext { + ParseContext pc; + EVCParserSPS sps[MAX_SPS_CNT]; + int is_avc; + int nal_length_size; + int to_read; + int incomplete_nalu_prefix_read; // The flag is set to 1 when an incomplete NAL unit prefix has been read + + int got_sps; + int got_pps; + int got_sei; + int got_slice; +} EVCParserContext; + +static int get_nalu_type(const uint8_t *bs, int bs_size, AVCodecContext *avctx) +{ + int unit_type_plus1 = 0; + XEVD_INFO info; + int ret; + + if(bs_size >= EVC_NAL_HEADER_SIZE) { + ret = xevd_info((void *)bs, EVC_NAL_HEADER_SIZE, 1, &info); + if (XEVD_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Cannot get bitstream information\n"); + return -1; + } + unit_type_plus1 = info.nalu_type; + + } + + return unit_type_plus1 - 1; +} + +static uint32_t read_nal_unit_length(const uint8_t *bs, int bs_size, AVCodecContext *avctx) +{ + uint32_t len = 0; + XEVD_INFO info; + int ret; + + if(bs_size >= XEVD_NAL_UNIT_LENGTH_BYTE) { + ret = xevd_info((void *)bs, XEVD_NAL_UNIT_LENGTH_BYTE, 1, &info); + if (XEVD_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Cannot get bitstream information\n"); + return 0; + } + len = info.nalu_len; + if(len == 0) { + av_log(avctx, AV_LOG_ERROR, "Invalid bitstream size!\n"); + return 0; + } + } + + return len; +} + +// @see ISO_IEC_23094-1 (7.3.2.1 SPS RBSP syntax) +static EVCParserSPS *parse_sps(const uint8_t *bs, int bs_size, EVCParserContext *ev) +{ + GetBitContext gb; + EVCParserSPS *sps; + int sps_seq_parameter_set_id; + + if(init_get_bits8(&gb, bs, bs_size) < 0) + return NULL; + + sps_seq_parameter_set_id = get_ue_golomb(&gb); + + if(sps_seq_parameter_set_id >= MAX_SPS_CNT) + return NULL; + + sps = &ev->sps[sps_seq_parameter_set_id]; + sps->sps_seq_parameter_set_id = sps_seq_parameter_set_id; + + // the Baseline profile is indicated by profile_idc eqal to 0 + // the Main profile is indicated by profile_idc eqal to 1 + sps->profile_idc = get_bits(&gb, 8); + + sps->level_idc = get_bits(&gb, 8); + + skip_bits_long(&gb, 32); /* skip toolset_idc_h */ + skip_bits_long(&gb, 32); /* skip toolset_idc_l */ + + // 0 - monochrome + // 1 - 4:2:0 + // 2 - 4:2:2 + // 3 - 4:4:4 + sps->chroma_format_idc = get_ue_golomb(&gb); + + sps->pic_width_in_luma_samples = get_ue_golomb(&gb); + sps->pic_height_in_luma_samples = get_ue_golomb(&gb); + + sps->bit_depth_luma_minus8 = get_ue_golomb(&gb); + sps->bit_depth_chroma_minus8 = get_ue_golomb(&gb); + + sps->sps_btt_flag = get_bits(&gb, 1); + if(sps->sps_btt_flag) { + sps->log2_ctu_size_minus5 = get_ue_golomb(&gb); + sps->log2_min_cb_size_minus2 = get_ue_golomb(&gb); + sps->log2_diff_ctu_max_14_cb_size = get_ue_golomb(&gb); + sps->log2_diff_ctu_max_tt_cb_size = get_ue_golomb(&gb); + sps->log2_diff_min_cb_min_tt_cb_size_minus2 = get_ue_golomb(&gb); + } + + sps->sps_suco_flag = get_bits(&gb, 1); + if(sps->sps_suco_flag) { + sps->log2_diff_ctu_size_max_suco_cb_size = get_ue_golomb(&gb); + sps->log2_diff_max_suco_min_suco_cb_size = get_ue_golomb(&gb); + } + + sps->sps_admvp_flag = get_bits(&gb, 1); + if(sps->sps_admvp_flag) { + sps->sps_affine_flag = get_bits(&gb, 1); + sps->sps_amvr_flag = get_bits(&gb, 1); + sps->sps_dmvr_flag = get_bits(&gb, 1); + sps->sps_mmvd_flag = get_bits(&gb, 1); + sps->sps_hmvp_flag = get_bits(&gb, 1); + } + + sps->sps_eipd_flag = get_bits(&gb, 1); + if(sps->sps_eipd_flag) { + sps->sps_ibc_flag = get_bits(&gb, 1); + if(sps->sps_ibc_flag) + sps->log2_max_ibc_cand_size_minus2 = get_ue_golomb(&gb); + } + + sps->sps_cm_init_flag = get_bits(&gb, 1); + if(sps->sps_cm_init_flag) + sps->sps_adcc_flag = get_bits(&gb, 1); + + sps->sps_iqt_flag = get_bits(&gb, 1); + if(sps->sps_iqt_flag) + sps->sps_ats_flag = get_bits(&gb, 1); + + sps->sps_addb_flag = get_bits(&gb, 1); + sps->sps_alf_flag = get_bits(&gb, 1); + sps->sps_htdf_flag = get_bits(&gb, 1); + sps->sps_rpl_flag = get_bits(&gb, 1); + sps->sps_pocs_flag = get_bits(&gb, 1); + sps->sps_dquant_flag = get_bits(&gb, 1); + sps->sps_dra_flag = get_bits(&gb, 1); + + if(sps->sps_pocs_flag) + sps->log2_max_pic_order_cnt_lsb_minus4 = get_ue_golomb(&gb); + + if(!sps->sps_pocs_flag || !sps->sps_rpl_flag) { + sps->log2_sub_gop_length = get_ue_golomb(&gb); + if(sps->log2_sub_gop_length == 0) + sps->log2_ref_pic_gap_length = get_ue_golomb(&gb); + } + + return sps; +} + +static int parse_nal_units(AVCodecParserContext *s, const uint8_t *bs, + int bs_size, AVCodecContext *avctx) +{ + EVCParserContext *ev = s->priv_data; + int nalu_type, nalu_size; + unsigned char *bits = (unsigned char *)bs; + int bits_size = bs_size; + + avctx->codec_id = AV_CODEC_ID_EVC; + + nalu_size = read_nal_unit_length(bits, bits_size, avctx); + if(nalu_size == 0) { + av_log(avctx, AV_LOG_ERROR, "Invalid NAL unit size: (%d)\n", nalu_size); + return -1; + } + + bits += XEVD_NAL_UNIT_LENGTH_BYTE; + bits_size -= XEVD_NAL_UNIT_LENGTH_BYTE; + + nalu_type = get_nalu_type(bits, bits_size, avctx); + + bits += EVC_NAL_HEADER_SIZE; + bits_size -= EVC_NAL_HEADER_SIZE; + + if (nalu_type == XEVD_NUT_SPS) { // NAL Unit type: SPS (Sequence Parameter Set) + EVCParserSPS *sps; + + sps = parse_sps(bits, bits_size, ev); + + avctx->coded_width = sps->pic_width_in_luma_samples; + avctx->coded_height = sps->pic_height_in_luma_samples; + avctx->width = sps->pic_width_in_luma_samples; + avctx->height = sps->pic_height_in_luma_samples; + + if(sps->profile_idc == 0) avctx->profile = FF_PROFILE_EVC_BASELINE; + else if (sps->profile_idc == 1) avctx->profile = FF_PROFILE_EVC_MAIN; + else { + av_log(avctx, AV_LOG_ERROR, "Not supported profile (%d)\n", sps->profile_idc); + return -1; + } + + // Currently XEVD decoder supports ony YCBCR420_10LE chroma format for EVC stream + switch(sps->chroma_format_idc) { + case 0: /* YCBCR400_10LE */ + av_log(avctx, AV_LOG_ERROR, "YCBCR400_10LE: Not supported chroma format\n"); + avctx->pix_fmt = AV_PIX_FMT_GRAY10LE; + return -1; + case 1: /* YCBCR420_10LE */ + avctx->pix_fmt = AV_PIX_FMT_YUV420P10LE; + break; + case 2: /* YCBCR422_10LE */ + av_log(avctx, AV_LOG_ERROR, "YCBCR422_10LE: Not supported chroma format\n"); + avctx->pix_fmt = AV_PIX_FMT_YUV422P10LE; + return -1; + case 3: /* YCBCR444_10LE */ + av_log(avctx, AV_LOG_ERROR, "YCBCR444_10LE: Not supported chroma format\n"); + avctx->pix_fmt = AV_PIX_FMT_YUV444P10LE; + return -1; + default: + avctx->pix_fmt = AV_PIX_FMT_NONE; + av_log(avctx, AV_LOG_ERROR, "Unknown supported chroma format\n"); + return -1; + } + + // The current implementation of parse_sps function doesn't handle VUI parameters parsing, + // so at the moment it's impossible to initialize has_b_frames and max_b_frames AVCodecContex fields here. + // Currently, initialization of has_b_frames and max_b_frames AVCodecContex fields have been moved to + // libxevd_decode function where we can use xevd_config function being a part of xevd library API + // to get the needed information. + // However, if it will be needed, parse_sps function should be extended to handle VUI parameters parsing + // and the following lines should be used to initialize has_b_frames and max_b_frames fields of the AVCodecContex. + // + // sps->vui_parameters.num_reorder_pics + // if (sps->bitstream_restriction_flag && sps->vui_parameters.num_reorder_pics) { + // avctx->has_b_frames = sps->vui_parameters.num_reorder_pics; + // } + + ev->got_sps = 1; + + } else if (nalu_type == XEVD_NUT_PPS) // NAL Unit type: PPS (Video Parameter Set) + ev->got_pps = 1; + else if(nalu_type == XEVD_NUT_SEI) // NAL unit type: SEI (Supplemental Enhancement Information) + ev->got_sei = 1; + else if (nalu_type == XEVD_NUT_IDR || nalu_type == XEVD_NUT_NONIDR) // NAL Unit type: Coded slice of a IDR or non-IDR picture + ev->got_slice++; + else { + av_log(avctx, AV_LOG_ERROR, "Invalid NAL unit type: %d\n", nalu_type); + return -1; + } + + return 0; +} + +/** + * Find the end of the current frame in the bitstream. + * @return the position of the first byte of the next frame, or END_NOT_FOUND + */ +static int evc_find_frame_end(AVCodecParserContext *s, const uint8_t *buf, + int buf_size, AVCodecContext *avctx) +{ + EVCParserContext *ev = s->priv_data; + + if(!ev->to_read) { + int nal_unit_size = 0; + int next = END_NOT_FOUND; + + // This is the case when buffer size is not enough for buffer to store NAL unit length + if(buf_size < XEVD_NAL_UNIT_LENGTH_BYTE) { + ev->to_read = XEVD_NAL_UNIT_LENGTH_BYTE; + ev->nal_length_size = buf_size; + ev->incomplete_nalu_prefix_read = 1; + + return END_NOT_FOUND; + } + + nal_unit_size = read_nal_unit_length(buf, buf_size, avctx); + ev->nal_length_size = XEVD_NAL_UNIT_LENGTH_BYTE; + + next = nal_unit_size + XEVD_NAL_UNIT_LENGTH_BYTE; + ev->to_read = next; + if(next < buf_size) + return next; + else + return END_NOT_FOUND; + } else if(ev->to_read > buf_size) { + return END_NOT_FOUND; + } else { + if(ev->incomplete_nalu_prefix_read == 1) { + EVCParserContext *ev = s->priv_data; + ParseContext *pc = &ev->pc; + uint8_t nalu_len[XEVD_NAL_UNIT_LENGTH_BYTE] = {0}; + int nal_unit_size = 0; + + // 1. pc->buffer contains previously read bytes of NALU prefix + // 2. buf contains the rest of NAL unit prefix bytes + // + // ~~~~~~~ + // EXAMPLE + // ~~~~~~~ + // + // In the following example we assumed that the number of already read NAL Unit prefix bytes is equal 1 + // + // ---------- + // pc->buffer -> conatins already read bytes + // ---------- + // __ pc->index == 1 + // | + // V + // ------------------------------------------------------- + // | 0 | 1 | 2 | 3 | 4 | ... | N | + // ------------------------------------------------------- + // | 0x00 | 0xXX | 0xXX | 0xXX | 0xXX | ... | 0xXX | + // ------------------------------------------------------- + // + // ---------- + // buf -> contains newly read bytes + // ---------- + // ------------------------------------------------------- + // | 0 | 1 | 2 | 3 | 4 | ... | N | + // ------------------------------------------------------- + // | 0x00 | 0x00 | 0x3C | 0xXX | 0xXX | ... | 0xXX | + // ------------------------------------------------------- + // + for(int i = 0; i < XEVD_NAL_UNIT_LENGTH_BYTE; i++) { + if(i < pc->index) + nalu_len[i] = pc->buffer[i]; + else + nalu_len[i] = buf[i - pc->index]; + } + + // ---------- + // nalu_len + // ---------- + // --------------------------------- + // | 0 | 1 | 2 | 3 | + // --------------------------------- + // | 0x00 | 0x00 | 0x00 | 0x3C | + // --------------------------------- + // | NALU LENGTH | + // --------------------------------- + // NAL Unit lenght = 60 (0x0000003C) + + nal_unit_size = read_nal_unit_length(nalu_len, XEVD_NAL_UNIT_LENGTH_BYTE, avctx); + + ev->to_read = nal_unit_size + XEVD_NAL_UNIT_LENGTH_BYTE - pc->index; + + ev->incomplete_nalu_prefix_read = 0; + + if(ev->to_read > buf_size) + return END_NOT_FOUND; + else + return ev->to_read; + } + return ev->to_read; + } + + return END_NOT_FOUND; +} + +static int evc_parser_init(AVCodecParserContext *s) +{ + EVCParserContext *ev = s->priv_data; + + ev->got_sps = 0; + ev->got_pps = 0; + ev->got_sei = 0; + ev->got_slice = 0; + ev->nal_length_size = XEVD_NAL_UNIT_LENGTH_BYTE; + ev->incomplete_nalu_prefix_read = 0; + + return 0; +} + +static int evc_parse(AVCodecParserContext *s, AVCodecContext *avctx, + const uint8_t **poutbuf, int *poutbuf_size, + const uint8_t *buf, int buf_size) +{ + int next; + EVCParserContext *ev = s->priv_data; + ParseContext *pc = &ev->pc; + int is_dummy_buf = !buf_size; + const uint8_t *dummy_buf = buf; + + if (s->flags & PARSER_FLAG_COMPLETE_FRAMES) + next = buf_size; + else { + next = evc_find_frame_end(s, buf, buf_size, avctx); + if (ff_combine_frame(pc, next, &buf, &buf_size) < 0) { + *poutbuf = NULL; + *poutbuf_size = 0; + ev->to_read -= buf_size; + return buf_size; + } + } + + is_dummy_buf &= (dummy_buf == buf); + + if (!is_dummy_buf) + parse_nal_units(s, buf, buf_size, avctx); + + *poutbuf = buf; + *poutbuf_size = buf_size; + ev->to_read -= next; + + return next; +} + +AVCodecParser ff_evc_parser = { + .codec_ids = { AV_CODEC_ID_EVC }, + .priv_data_size = sizeof(EVCParserContext), + .parser_init = evc_parser_init, + .parser_parse = evc_parse, + .parser_close = ff_parse_close, +}; diff --git a/libavcodec/libxevd.c b/libavcodec/libxevd.c new file mode 100644 index 0000000000..e8d4eae73a --- /dev/null +++ b/libavcodec/libxevd.c @@ -0,0 +1,443 @@ +/* + * libxevd decoder + * EVC (MPEG-5 Essential Video Coding) decoding using XEVD MPEG-5 EVC decoder library + * + * Copyright (C) 2021 Dawid Kozinski <d.kozinski@samsung.com> + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#if defined(_MSC_VER) +#define XEVD_API_IMPORTS 1 +#endif + +#include <float.h> +#include <stdlib.h> + +#include <xevd.h> + +#include "libavutil/internal.h" +#include "libavutil/common.h" +#include "libavutil/opt.h" +#include "libavutil/pixdesc.h" +#include "libavutil/pixfmt.h" +#include "libavutil/imgutils.h" + +#include "avcodec.h" +#include "internal.h" +#include "packet_internal.h" + +#define XEVD_PARAM_BAD_NAME -1 +#define XEVD_PARAM_BAD_VALUE -2 + +#define EVC_NAL_HEADER_SIZE 2 /* byte */ + +/** + * The structure stores all the states associated with the instance of Xeve MPEG-5 EVC decoder + * The first field is a pointer to an AVClass struct (@see https://ffmpeg.org/doxygen/trunk/structAVClass.html#details). + */ +typedef struct XevdContext { + const AVClass *class; + + XEVD id; // XEVD instance identifier @see xevd.h + XEVD_CDSC cdsc; // decoding parameters @see xevd.h + + int decod_frames; // number of decoded frames + int packet_count; // number of packets created by decoder + + // configuration parameters + AVDictionary *xevd_opts; // xevd configuration read from a : separated list of key=value parameters + +} XevdContext; + +/** + * The function returns a pointer to a variable of type XEVD_CDSC. + * XEVD_CDSC contains all decoder parameters that should be initialized before its use. + * + * The field values of the XEVD_CDSC structure are populated based on: + * - the corresponding field values of the AvCodecConetxt structure, + * - the xevd decoder specific option values, + * (the full list of options available for the xevd encoder is displayed after executing the command ./ffmpeg --help decoder = libxevd) + * - and the xevd encoder options specified as a list of key-value pairs following the xevd-params option + * + * Order of input processing and populating the XEVD_CDSC structure + * 1. first, the corresponding fields of the AVCodecContext structure are processed, (i.e -threads 4) + * 2. then xevd-specific options are added as AVOption to the xevd AVCodec implementation (i.e -threads 3) + * 3. finally, the options specified after the xevd-params option as the parameter list of type key value are processed (i.e -xevd-params "m=2") + * + * Some options can be set in different ways. In this case, please follow the above-mentioned order of processing. + * The most recent assignments overwrite the previous values. + * + * @param[in] avctx codec context + * @param[out] cdsc contains all encoder parameters that should be initialized before its use + * + * @return 0 on success, negative error code on failure + */ +static int get_conf(AVCodecContext *avctx, XEVD_CDSC *cdsc) +{ + int cpu_count = av_cpu_count(); + + /* clear XEVS_CDSC structure */ + memset(cdsc, 0, sizeof(XEVD_CDSC)); + + /* init XEVD_CDSC */ + if(avctx->thread_count <= 0) + cdsc->threads = (cpu_count < XEVD_MAX_TASK_CNT) ? cpu_count : XEVD_MAX_TASK_CNT; + else if(avctx->thread_count > XEVD_MAX_TASK_CNT) + cdsc->threads = XEVD_MAX_TASK_CNT; + else + cdsc->threads = avctx->thread_count; + + return XEVD_OK; +} + +/** + * Read NAL unit length + * @param bs input data (bitstream) + * @return the length of NAL unit on success, 0 value on failure + */ +static uint32_t read_nal_unit_length(const uint8_t *bs, int bs_size, AVCodecContext *avctx) +{ + uint32_t len = 0; + XEVD_INFO info; + int ret; + + if(bs_size == XEVD_NAL_UNIT_LENGTH_BYTE) { + ret = xevd_info((void *)bs, XEVD_NAL_UNIT_LENGTH_BYTE, 1, &info); + if (XEVD_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Cannot get bitstream information\n"); + return 0; + } + len = info.nalu_len; + if(len == 0) { + av_log(avctx, AV_LOG_ERROR, "Invalid bitstream size! [%d]\n", bs_size); + return 0; + } + } + + return len; +} + +/** + * @param[in] xectx the structure that stores all the state associated with the instance of Xeve MPEG-5 EVC decoder + * @param[out] avctx codec context + * @return 0 on success, negative value on failure + */ +static int export_stream_params(const XevdContext *xectx, AVCodecContext *avctx) +{ + int ret; + int size; + int color_space; + + avctx->pix_fmt = AV_PIX_FMT_YUV420P10; + + size = 4; + ret = xevd_config(xectx->id, XEVD_CFG_GET_CODED_WIDTH, &avctx->coded_width, &size); + if (XEVD_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to get coded_width\n"); + return -1; + } + + ret = xevd_config(xectx->id, XEVD_CFG_GET_CODED_HEIGHT, &avctx->coded_height, &size); + if (XEVD_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to get coded_height\n"); + return -1; + } + + ret = xevd_config(xectx->id, XEVD_CFG_GET_WIDTH, &avctx->width, &size); + if (XEVD_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to get width\n"); + return -1; + } + + ret = xevd_config(xectx->id, XEVD_CFG_GET_HEIGHT, &avctx->height, &size); + if (XEVD_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to get height\n"); + return -1; + } + + ret = xevd_config(xectx->id, XEVD_CFG_GET_COLOR_SPACE, &color_space, &size); + if (XEVD_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to get color_space\n"); + return -1; + } + switch(color_space) { + case XEVD_CS_YCBCR400_10LE: + avctx->pix_fmt = AV_PIX_FMT_GRAY10LE; + break; + case XEVD_CS_YCBCR420_10LE: + avctx->pix_fmt = AV_PIX_FMT_YUV420P10LE; + break; + case XEVD_CS_YCBCR422_10LE: + avctx->pix_fmt = AV_PIX_FMT_YUV422P10LE; + break; + case XEVD_CS_YCBCR444_10LE: + avctx->pix_fmt = AV_PIX_FMT_YUV444P10LE; + break; + default: + av_log(avctx, AV_LOG_ERROR, "Unknown color space\n"); + avctx->pix_fmt = AV_PIX_FMT_NONE; + return -1; + } + + // the function returns sps->num_reorder_pics + ret = xevd_config(xectx->id, XEVD_CFG_GET_MAX_CODING_DELAY, &avctx->max_b_frames, &size); + if (XEVD_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to get max_coding_delay\n"); + return -1; + } + + avctx->has_b_frames = (avctx->max_b_frames) ? 1 : 0; + + avctx->color_primaries = AVCOL_PRI_UNSPECIFIED; + avctx->color_trc = AVCOL_TRC_UNSPECIFIED; + avctx->colorspace = AVCOL_SPC_UNSPECIFIED; + + return 0; +} + +/** + * Initialize decoder + * Create a decoder instance and allocate all the needed resources + * + * @param avctx codec context + * @return 0 on success, negative error code on failure + */ +static av_cold int libxevd_init(AVCodecContext *avctx) +{ + XevdContext *xectx = avctx->priv_data; + int val = 0; + XEVD_CDSC *cdsc = &(xectx->cdsc); + + /* read configurations and set values for created descriptor (XEVD_CDSC) */ + val = get_conf(avctx, cdsc); + if (val != XEVD_OK) { + av_log(avctx, AV_LOG_ERROR, "Cannot get configuration\n"); + return -1; + } + + /* create decoder */ + xectx->id = xevd_create(&(xectx->cdsc), NULL); + if(xectx->id == NULL) { + av_log(avctx, AV_LOG_ERROR, "Cannot create XEVD encoder\n"); + return -1; + } + + xectx->packet_count = 0; + xectx->decod_frames = 0; + + return 0; +} + +/** + * Decode picture + * + * @param avctx codec context + * @param[out] data decoded frame + * @param[out] got_frame decoder sets to 0 or 1 to indicate that a + * non-empty frame or subtitle was returned in + * outdata. + * @param[in] pkt AVPacket containing encoded data to be decoded + * + * @return amount of bytes read from the packet on success, negative error + * code on failure + */ +static int libxevd_decode(AVCodecContext *avctx, void *data, int *got_frame, AVPacket *pkt) +{ + AVFrame *frame = data; + XevdContext *xectx = NULL; + XEVD_IMGB *imgb = NULL; + XEVD_STAT stat; + XEVD_BITB bitb; + int ret, nalu_size, bs_read_pos; + + if(avctx == NULL) { + av_log(avctx, AV_LOG_ERROR, "Invalid input parameter: AVCodecContext\n"); + return -1; + } + xectx = avctx->priv_data; + if(xectx == NULL) { + av_log(avctx, AV_LOG_ERROR, "Invalid XEVD context\n"); + return -1; + } + + if(pkt->size > 0) { + bs_read_pos = 0; + imgb = NULL; + while(pkt->size > (bs_read_pos + XEVD_NAL_UNIT_LENGTH_BYTE)) { + + memset(&stat, 0, sizeof(XEVD_STAT)); + memset(&bitb, 0, sizeof(XEVD_BITB)); + + nalu_size = read_nal_unit_length(pkt->data + bs_read_pos, XEVD_NAL_UNIT_LENGTH_BYTE, avctx); + if(nalu_size == 0) { + av_log(avctx, AV_LOG_ERROR, "Invalid bitstream\n"); + goto ERR; + } + bs_read_pos += XEVD_NAL_UNIT_LENGTH_BYTE; + + bitb.addr = pkt->data + bs_read_pos; + bitb.ssize = nalu_size; + + /* main decoding block */ + ret = xevd_decode(xectx->id, &bitb, &stat); + if(XEVD_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to decode bitstream\n"); + goto ERR; + } + + bs_read_pos += nalu_size; + + if(stat.nalu_type == XEVD_NUT_SPS) { // EVC stream parameters changed + if(export_stream_params(xectx, avctx) != 0) + goto ERR; + } + + if(stat.read != nalu_size) + av_log(avctx, AV_LOG_INFO, "Different reading of bitstream (in:%d, read:%d)\n,", nalu_size, stat.read); + if(stat.fnum >= 0) { + if (imgb) { /* already has a decoded image */ + imgb->release(imgb); + imgb = NULL; + } + ret = xevd_pull(xectx->id, &imgb); + if(XEVD_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to pull the decoded image (xevd error code: %d, frame#=%d)\n", ret, stat.fnum); + goto ERR; + } else if (ret == XEVD_OK_FRM_DELAYED) { + if(imgb) { + imgb->release(imgb); + imgb = NULL; + } + } + } + } + } else { // bumping + ret = xevd_pull(xectx->id, &(imgb)); + if(ret == XEVD_ERR_UNEXPECTED) { // bumping process completed + *got_frame = 0; + return 0; + } else if(XEVD_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to pull the decoded image (xevd error code: %d)\n", ret); + goto ERR; + } + } + + if(imgb) { + // @todo supports other color space and bit depth + if(imgb->cs != XEVD_CS_YCBCR420_10LE) { + av_log(avctx, AV_LOG_ERROR, "Not supported pixel format: %s\n", av_get_pix_fmt_name(avctx->pix_fmt)); + goto ERR; + } + + if (imgb->w[0] != avctx->width || imgb->h[0] != avctx->height) { // stream resolution changed + if(ff_set_dimensions(avctx, imgb->w[0], imgb->h[0]) < 0) { + av_log(avctx, AV_LOG_ERROR, "Cannot set new dimension\n"); + goto ERR; + } + } + + frame->coded_picture_number++; + frame->display_picture_number++; + frame->format = AV_PIX_FMT_YUV420P10LE; + + if (ff_get_buffer(avctx, frame, 0) < 0) { + av_log(avctx, AV_LOG_ERROR, "Cannot get AV buffer\n"); + goto ERR; + } + + frame->pts = pkt->pts; + + av_image_copy(frame->data, frame->linesize, (const uint8_t **)imgb->a, + imgb->s, avctx->pix_fmt, + imgb->w[0], imgb->h[0]); + + xectx->decod_frames++; + *got_frame = 1; + + imgb->release(imgb); + imgb = NULL; + } else + *got_frame = 0; + + xectx->packet_count++; + + return pkt->size; + +ERR: + if(imgb) { + imgb->release(imgb); + imgb = NULL; + } + *got_frame = 0; + + return -1; +} + +/** + * Destroy decoder + * + * @param avctx codec context + * @return 0 on success + */ +static av_cold int libxevd_close(AVCodecContext *avctx) +{ + XevdContext *xectx = avctx->priv_data; + if(xectx->id) { + xevd_delete(xectx->id); + xectx->id = NULL; + } + + return 0; +} + +#define OFFSET(x) offsetof(XevdContext, x) +#define VD AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_DECODING_PARAM + +// @todo consider using following options (./ffmpeg --help decoder=libxevd) +// +static const AVOption options[] = { + { "xevd-params", "override the xevd configuration using a : separated list of key=value parameters", OFFSET(xevd_opts), AV_OPT_TYPE_DICT, { 0 }, 0, 0, VD }, + { NULL } +}; + +static const AVClass xevd_class = { + .class_name = "libxevd", + .item_name = av_default_item_name, + .option = options, + .version = LIBAVUTIL_VERSION_INT, +}; + +static const AVCodecDefault xevd_defaults[] = { + { "b", "0" }, + { NULL }, +}; + +AVCodec ff_libxevd_decoder = { + .name = "evc", + .long_name = NULL_IF_CONFIG_SMALL("EVC / MPEG-5 Essential Video Coding (EVC)"), + .type = AVMEDIA_TYPE_VIDEO, + .id = AV_CODEC_ID_EVC, + .init = libxevd_init, + .decode = libxevd_decode, + .close = libxevd_close, + .priv_data_size = sizeof(XevdContext), + .priv_class = &xevd_class, + .defaults = xevd_defaults, + .capabilities = AV_CODEC_CAP_DELAY | AV_CODEC_CAP_AUTO_THREADS | AV_CODEC_CAP_AVOID_PROBING, + .wrapper_name = "libxevd", +}; diff --git a/libavcodec/libxeve.c b/libavcodec/libxeve.c new file mode 100644 index 0000000000..d04a12bc60 --- /dev/null +++ b/libavcodec/libxeve.c @@ -0,0 +1,1139 @@ +/* + * libxeve encoder + * EVC (MPEG-5 Essential Video Coding) encoding using XEVE MPEG-5 EVC encoder library + * + * Copyright (C) 2021 Dawid Kozinski <d.kozinski@samsung.com> + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#if defined(_MSC_VER) +#define XEVE_API_IMPORTS 1 +#endif + +#include <float.h> +#include <stdlib.h> + +#include <xeve.h> + +#include "libavutil/internal.h" +#include "libavutil/common.h" +#include "libavutil/opt.h" +#include "libavutil/pixdesc.h" +#include "libavutil/pixfmt.h" +#include "libavutil/time.h" + +#include "avcodec.h" +#include "internal.h" +#include "packet_internal.h" + +#define MAX_BS_BUF (16*1024*1024) + +/** + * Error codes + */ +#define XEVE_PARAM_BAD_NAME -100 +#define XEVE_PARAM_BAD_VALUE -200 + +/** + * Encoder states + * + * STATE_ENCODING - the encoder receives and processes input frames + * STATE_BUMPING - there are no more input frames, however the encoder still processes previously received data + * STATE_SKIPPING - skipping input frames + */ +typedef enum State { + STATE_ENCODING, + STATE_BUMPING, + STATE_SKIPPING +} State; + +/** + * The structure stores all the states associated with the instance of Xeve MPEG-5 EVC encoder + * The first field is a pointer to an AVClass struct (@see https://ffmpeg.org/doxygen/trunk/structAVClass.html#details). + */ +typedef struct XeveContext { + const AVClass *class; + + XEVE id; // XEVE instance identifier + XEVE_CDSC cdsc; // coding parameters i.e profile, width & height of input frame, num of therads, frame rate ... + XEVE_BITB bitb; // bitstream buffer (output) + XEVE_STAT stat; // encoding status (output) + XEVE_IMGB imgb; // image buffer (input) + + State state; // encoder state (skipping, encoding, bumping) + + int encod_frames; // num of encoded frames + double bytes_total; // encoded bitstream byte size + double bitrate; // bits per second + int packet_count; // num of packets created by encoder + + // Chroma subsampling + int width_luma; + int height_luma; + int width_chroma; + int height_chroma; + + int profile_id; // encoder profile (main, baseline) + int preset_id; // preset of xeve ( fast, medium, slow, placebo) + int tune_id; // tune of xeve (psnr, zerolatency) + int input_depth; // input bit-depth: 8bit, 10bit + int hash; // embed picture signature (HASH) for conformance checking in decoding + + /* variables for input parameter */ + char *op_preset; + char *op_tune; + int op_qp; + int op_crf; + + // configuration parameters + // xeve configuration read from a : separated list of key=value parameters + AVDictionary *xeve_params; +} XeveContext; + +/** + * Gets Xeve encoder pre-defined profile + * + * @param profile string describing Xeve encoder profile (baseline, main) + * @return XEVE pre-defined profile on success, negative value on failure + */ +static int get_profile_id(const char *profile) +{ + if (!strcmp(profile, "baseline")) + return XEVE_PROFILE_BASELINE; + else if (!strcmp(profile, "main")) + return XEVE_PROFILE_MAIN; + else + return -1; +} + +/** + * Gets Xeve pre-defined preset + * + * @param preset string describing Xeve encoder preset (fast, medium, slow, placebo) + * @return XEVE pre-defined profile on success, negative value on failure + */ +static int get_preset_id(const char *preset) +{ + if((!strcmp(preset, "fast"))) + return XEVE_PRESET_FAST; + else if (!strcmp(preset, "medium")) + return XEVE_PRESET_MEDIUM; + else if (!strcmp(preset, "slow")) + return XEVE_PRESET_SLOW; + else if (!strcmp(preset, "placebo")) + return XEVE_PRESET_PLACEBO; + else + return -1; +} + +/** + * Gets Xeve pre-defined tune id + * + * @param[in] tune string describing Xeve encoder tune (psnr, zerolatency) + * @return XEVE pre-defined profile on success, negative value on failure + */ +static int get_tune_id(const char *tune) +{ + if((!strcmp(tune, "psnr"))) + return XEVE_TUNE_PSNR; + else if (!strcmp(tune, "zerolatency")) + return XEVE_TUNE_ZEROLATENCY; + else + return -1; +} + +/** + * Convert FFmpeg pixel format (AVPixelFormat) to XEVE pre-defined color format + * + * @param[in] px_fmt pixel format (@see https://ffmpeg.org/doxygen/trunk/pixfmt_8h.html#a9a8e335cf3be472042bc9f0cf80cd4c5) + * @param[out] color_format XEVE pre-defined color format (@see xeve.h) + * @param[out] bit_depth bit depth + * + * @return 0 on success, negative value on failure + */ +static int get_pix_fmt(enum AVPixelFormat pix_fmt, int *color_format, int *bit_depth) +{ + switch (pix_fmt) { + case AV_PIX_FMT_YUV420P: + *color_format = XEVE_CF_YCBCR420; + *bit_depth = 8; + break; + case AV_PIX_FMT_YUV422P: + *color_format = XEVE_CF_YCBCR422; + *bit_depth = 8; + break; + case AV_PIX_FMT_YUV444P: + *color_format = XEVE_CF_YCBCR444; + *bit_depth = 8; + break; + case AV_PIX_FMT_YUV420P10: + *color_format = XEVE_CF_YCBCR420; + *bit_depth = 10; + break; + case AV_PIX_FMT_YUV422P10: + *color_format = XEVE_CF_YCBCR422; + *bit_depth = 10; + break; + case AV_PIX_FMT_YUV444P10: + *color_format = XEVE_CF_YCBCR444; + *bit_depth = 10; + break; + default: + *color_format = XEVE_CF_UNKNOWN; + return -1; + } + + return 0; +} + +static int kbps_str_to_int(char *str) +{ + int kbps = 0; + if (strchr(str, 'K') || strchr(str, 'k')) { + char *tmp = strtok(str, "Kk "); + kbps = (int)(atof(tmp)); + } else if (strchr(str, 'M') || strchr(str, 'm')) { + char *tmp = strtok(str, "Mm "); + kbps = (int)(atof(tmp) * 1000); + } else + kbps = atoi(str); + + return kbps; +} + +/** + * Parse : separated list of key=value parameters + * + * @param[in] avctx context for logger + * @param[in] key + * @param[in] value + * @param[out] cdsc contains all Xeve MPEG-5 EVC encoder encoder parameters that + * should be initialized before the encoder is use + * + * @return 0 on success, negative value on failure + */ +static int parse_xeve_params(AVCodecContext *avctx, const char *key, const char *value, XEVE_CDSC* cdsc) +{ + XeveContext *xectx = NULL; + xectx = avctx->priv_data; + + if(!key) { + av_log(avctx, AV_LOG_ERROR, "Ivalid argument: key string is NULL\n"); + return XEVE_PARAM_BAD_VALUE; + } + if(!value) { + if (strcmp(key, "hash") == 0) { + xectx->hash = 1; + av_log(avctx, AV_LOG_INFO, "embedding signature is enabled\n"); + } else { + av_log(avctx, AV_LOG_ERROR, "Ivalid argument: value string is NULL\n"); + return XEVE_PARAM_BAD_VALUE; + } + } else if (strcmp(key, "vbv-bufsize") == 0 ) { + cdsc->param.vbv_bufsize = kbps_str_to_int((char *)value); + av_log(avctx, AV_LOG_INFO, "VBV buffer size: %dkbits\n", cdsc->param.vbv_bufsize); + } else if (strcmp(key, "rc-type") == 0 ) { + int rc_type = atoi(value); + if(rc_type < 0 || rc_type > 2) { + av_log(avctx, AV_LOG_ERROR, "Rate control type [ 0(rc_off) / 1(CBR) ] bad value: %d\n", rc_type); + return XEVE_PARAM_BAD_VALUE; + } + cdsc->param.rc_type = rc_type; + av_log(avctx, AV_LOG_INFO, "Rate control type [ 0(rc_off) / 1(CBR) ] : %d\n", rc_type); + } else if (strcmp(key, "bframes") == 0 ) { + int bframes = atoi(value); + if(bframes < 0) { + av_log(avctx, AV_LOG_ERROR, "bframes: bad value: %d\n", bframes); + return XEVE_PARAM_BAD_VALUE; + } + cdsc->param.bframes = bframes; + av_log(avctx, AV_LOG_INFO, "bframes : %d\n", bframes); + } else if (strcmp(key, "profile") == 0 ) { + const char *profile = value; + int profile_id; + av_log(avctx, AV_LOG_INFO, "profile (baseline, main): %s\n", profile); + profile_id = get_profile_id(profile); + if (profile_id < 0) { + av_log(avctx, AV_LOG_ERROR, "Invalid xeve param: profile(%s)\n", profile); + return XEVE_PARAM_BAD_VALUE; + } + xectx->profile_id = profile_id; + } else if (strcmp(key, "preset") == 0 ) { + const char *preset = value; + int preset_id; + av_log(avctx, AV_LOG_INFO, "Preset of xeve (fast, medium, slow, placebo): %s\n", preset); + preset_id = get_preset_id(preset); + if( preset_id < 0) { + av_log(avctx, AV_LOG_ERROR, "Invalid xeve param: preset(%s)\n", preset); + return XEVE_PARAM_BAD_VALUE; + } + xectx->preset_id = preset_id; + } else if (strcmp(key, "tune") == 0 ) { + const char *tune = value; + int tune_id; + av_log(avctx, AV_LOG_INFO, "Tune of xeve (psnr, zerolatency): %s\n", tune); + tune_id = get_tune_id(tune); + if( tune_id < 0) { + av_log(avctx, AV_LOG_ERROR, "Invalid xeve param: tune(%s)\n", tune); + return XEVE_PARAM_BAD_VALUE; + } + xectx->tune_id = tune_id; + } else if (strcmp(key, "bitrate") == 0 ) { + cdsc->param.bitrate = kbps_str_to_int((char *)value); + av_log(avctx, AV_LOG_INFO, "Bitrate = %dkbps\n", cdsc->param.bitrate); + } else if (strcmp(key, "q") == 0 || strcmp(key, "qp") == 0) { + int qp = atoi(value); + if(qp < 0 || qp > 51) { + av_log(avctx, AV_LOG_ERROR, "Invalid QP value (0~51) :%d\n", qp); + return XEVE_PARAM_BAD_VALUE; + } + cdsc->param.qp = qp; + av_log(avctx, AV_LOG_INFO, "QP value (0~51): %d\n", cdsc->param.qp); + } else { + av_log(avctx, AV_LOG_ERROR, "Unknown xeve codec option: %s\n", key); + return XEVE_PARAM_BAD_NAME; + } + + return 0; +} + +/** + * The function returns a pointer to the object of the XEVE_CDSC type. + * XEVE_CDSC contains all encoder parameters that should be initialized before the encoder is used. + * + * The field values of the XEVE_CDSC structure are populated based on: + * - the corresponding field values of the AvCodecConetxt structure, + * - the xeve encoder specific option values, + * (the full list of options available for xeve encoder is displayed after executing the command ./ffmpeg --help encoder = libxeve) + * - and the xeve encoder options specified as a list of key-value pairs following the xeve-params option + * + * The order of processing input data and populating the XEVE_CDSC structure + * 1) first, the fields of the AVCodecContext structure corresponding to the provided input options are processed, + * (i.e -pix_fmt yuv420p -s:v 1920x1080 -r 30 -profile:v 0) + * 2) then xeve-specific options added as AVOption to the xeve AVCodec implementation + * (i.e -threads_cnt 3 -preset 0) + * 3) finally, the options specified after the xeve-params option as the parameter list of type key value are processed + * (i.e -xeve-params "m=2:q=17") + * + * Keep in mind that, there are options that can be set in different ways. + * In this case, please follow the above-mentioned order of processing. + * The most recent assignments overwrite the previous values. + * + * @param[in] avctx codec context (AVCodecContext) + * @param[out] cdsc contains all Xeve MPEG-5 EVC encoder encoder parameters that should be initialized before the encoder is use + * + * @return 0 on success, negative error code on failure + */ +static int get_conf(AVCodecContext *avctx, XEVE_CDSC *cdsc) +{ + XeveContext *xectx = NULL; + int color_format; + int cpu_count = av_cpu_count(); + int ret; + + xectx = avctx->priv_data; + xectx->hash = 0; + + /* initialize xeve_param struct with default values */ + ret = xeve_param_default(&cdsc->param); + if (XEVE_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Cannot set_default parameter\n"); + goto ERR; + } + + /* read options from AVCodecContext */ + if(avctx->width > 0) + cdsc->param.w = xectx->width_luma = avctx->width; + + if(avctx->height > 0) + cdsc->param.h = xectx->height_luma = avctx->height; + + if(avctx->framerate.num > 0) { + // fps can be float number, but xeve API doesn't support it + cdsc->param.fps = (int)(((float)avctx->framerate.num / avctx->framerate.den) + 0.5); + } + + if(avctx->gop_size >= 0) { // GOP size (key-frame interval) + cdsc->param.keyint = avctx->gop_size; // 0: only one I-frame at the first time; 1: every frame is coded in I-frame + } + + if (avctx->max_b_frames == 0 || avctx->max_b_frames == 1 || avctx->max_b_frames == 3 || + avctx->max_b_frames == 7 || avctx->max_b_frames == 15) { // number of b-frames + cdsc->param.bframes = avctx->max_b_frames; + } else { + av_log(avctx, AV_LOG_ERROR, "Incorrect value for maximum number of B frames: (%d) \n" + "Acceptable values for bf option (maximum number of B frames) are 0,1,3,7 or 15\n", avctx->max_b_frames); + goto ERR; + } + + if (avctx->level >= 0) + cdsc->param.level_idc = avctx->level; + + ret = get_pix_fmt(avctx->pix_fmt, &color_format, &xectx->input_depth); + + if (ret != 0) { + av_log(avctx, AV_LOG_ERROR, "Unsupported pixel format.\n"); + goto ERR; + } + cdsc->param.cs = XEVE_CS_SET(color_format, xectx->input_depth, 0); + + if (avctx->rc_buffer_size > 0) { // VBV buf size + cdsc->param.vbv_bufsize = (int)(avctx->rc_buffer_size / 1000); + } + + // rc_type: Rate control type [ 0(CQP) / 1(ABR) / 2(CRF) ] + if (avctx->bit_rate > 0) { + if (avctx->bit_rate / 1000 > INT_MAX || avctx->rc_max_rate / 1000 > INT_MAX) { + av_log(avctx, AV_LOG_ERROR, "Not supported bitrate bit_rate and rc_max_rate > %d000\n", INT_MAX); + goto ERR; + } + cdsc->param.bitrate = (int)(avctx->bit_rate / 1000); + cdsc->param.rc_type = XEVE_RC_ABR; + } + + if (xectx->op_crf >= 0) { + cdsc->param.crf = xectx->op_crf; + cdsc->param.rc_type = XEVE_RC_CRF; + } + + if(avctx->thread_count <= 0) { + cdsc->param.threads = (cpu_count < XEVE_MAX_THREADS) ? cpu_count : XEVE_MAX_THREADS; + } else if(avctx->thread_count > XEVE_MAX_THREADS) + cdsc->param.threads = XEVE_MAX_THREADS; + else + cdsc->param.threads = avctx->thread_count; + + cdsc->param.cs = XEVE_CS_SET(color_format, cdsc->param.codec_bit_depth, 0); + cdsc->max_bs_buf_size = MAX_BS_BUF; + + if(avctx->profile == FF_PROFILE_EVC_BASELINE) + xectx->profile_id = XEVE_PROFILE_BASELINE; + else if(avctx->profile == FF_PROFILE_EVC_MAIN) + xectx->profile_id = XEVE_PROFILE_MAIN; + else { + av_log(avctx, AV_LOG_ERROR, "Unknown encoder profile (%d)\n" + "Acceptable values for profile option are 0 and 1 (0: baseline profile; 1: main profile)\n", avctx->profile); + goto ERR; + } + + if (xectx->op_preset) { // preset + xectx->preset_id = get_preset_id(xectx->op_preset); + } + + if (xectx->op_tune) { // tune + xectx->tune_id = get_tune_id(xectx->op_tune); + } + + ret = xeve_param_ppt(&cdsc->param, xectx->profile_id, xectx->preset_id, xectx->tune_id); + if (XEVE_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Cannot set profile(%d), preset(%d), tune(%d)\n", xectx->profile_id, xectx->preset_id, xectx->tune_id); + goto ERR; + } + + /* parse : separated list of key=value parameters and set values for created descriptor (XEVE_CDSC) */ + { + AVDictionaryEntry *en = NULL; + + // Start to parse xeve_params + while ((en = av_dict_get(xectx->xeve_params, "", en, AV_DICT_IGNORE_SUFFIX))) { + int parse_ret = parse_xeve_params(avctx, en->key, en->value, cdsc); + + switch (parse_ret) { + case XEVE_PARAM_BAD_NAME: + av_log(avctx, AV_LOG_WARNING, "Unknown option: %s.\n", en->key); + break; + case XEVE_PARAM_BAD_VALUE: + av_log(avctx, AV_LOG_WARNING, "Invalid value for %s: %s.\n", en->key, en->value); + break; + default: + break; + } + } + } + + return 0; + +ERR: + return AVERROR(EINVAL); +} + +/** + * Check encoder configuration + * + * @param[in] avctx context for logger + * @param[in] cdsc contains all encoder parameters that should be initialized before its use + * + * @return 0 on success, negative error code on failure + */ +static int check_conf(AVCodecContext *avctx, XEVE_CDSC *cdsc) +{ + int ret = 0; + int min_block_size = 4; + int pic_m; + + if(cdsc->param.profile == XEVE_PROFILE_BASELINE) { + if (cdsc->param.tool_amvr == 1) { + av_log(avctx, AV_LOG_ERROR, "AMVR cannot be on in base profile\n"); + ret = -1; + } + if (cdsc->param.tool_mmvd == 1) { + av_log(avctx, AV_LOG_ERROR, "MMVD cannot be on in base profile\n"); + ret = -1; + } + if (cdsc->param.tool_affine == 1) { + av_log(avctx, AV_LOG_ERROR, "Affine cannot be on in base profile\n"); + ret = -1; + } + if (cdsc->param.tool_dmvr == 1) { + av_log(avctx, AV_LOG_ERROR, "DMVR cannot be on in base profile\n"); + ret = -1; + } + if (cdsc->param.tool_admvp == 1) { + av_log(avctx, AV_LOG_ERROR, "ADMVP cannot be on in base profile\n"); + ret = -1; + } + if (cdsc->param.tool_hmvp == 1) { + av_log(avctx, AV_LOG_ERROR, "HMVP cannot be on in base profile\n"); + ret = -1; + } + if (cdsc->param.tool_addb == 1) { + av_log(avctx, AV_LOG_ERROR, "ADDB cannot be on in base profile\n"); + ret = -1; + } + if (cdsc->param.tool_alf == 1) { + av_log(avctx, AV_LOG_ERROR, "ALF cannot be on in base profile\n"); + ret = -1; + } + if (cdsc->param.tool_htdf == 1) { + av_log(avctx, AV_LOG_ERROR, "HTDF cannot be on in base profile\n"); + ret = -1; + } + if (cdsc->param.btt == 1) { + av_log(avctx, AV_LOG_ERROR, "BTT cannot be on in base profile\n"); + ret = -1; + } + if (cdsc->param.suco == 1) { + av_log(avctx, AV_LOG_ERROR, "SUCO cannot be on in base profile\n"); + ret = -1; + } + if (cdsc->param.tool_eipd == 1) { + av_log(avctx, AV_LOG_ERROR, "EIPD cannot be on in base profile\n"); + ret = -1; + } + if (cdsc->param.tool_iqt == 1) { + av_log(avctx, AV_LOG_ERROR, "IQT cannot be on in base profile\n"); + ret = -1; + } + if (cdsc->param.tool_cm_init == 1) { + av_log(avctx, AV_LOG_ERROR, "CM_INIT cannot be on in base profile\n"); + ret = -1; + } + if (cdsc->param.tool_adcc == 1) { + av_log(avctx, AV_LOG_ERROR, "ADCC cannot be on in base profile\n"); + ret = -1; + } + if (cdsc->param.tool_ats == 1) { + av_log(avctx, AV_LOG_ERROR, "ATS_INTRA cannot be on in base profile\n"); + ret = -1; + } + if (cdsc->param.ibc_flag == 1) { + av_log(avctx, AV_LOG_ERROR, "IBC cannot be on in base profile\n"); + ret = -1; + } + if (cdsc->param.tool_rpl == 1) { + av_log(avctx, AV_LOG_ERROR, "RPL cannot be on in base profile\n"); + ret = -1; + } + if (cdsc->param.tool_pocs == 1) { + av_log(avctx, AV_LOG_ERROR, "POCS cannot be on in base profile\n"); + ret = -1; + } + } else { + if (cdsc->param.tool_admvp == 0 && cdsc->param.tool_affine == 1) { + av_log(avctx, AV_LOG_ERROR, "AFFINE cannot be on when ADMVP is off\n"); + ret = -1; + } + if (cdsc->param.tool_admvp == 0 && cdsc->param.tool_amvr == 1) { + av_log(avctx, AV_LOG_ERROR, "AMVR cannot be on when ADMVP is off\n"); + ret = -1; + } + if (cdsc->param.tool_admvp == 0 && cdsc->param.tool_dmvr == 1) { + av_log(avctx, AV_LOG_ERROR, "DMVR cannot be on when ADMVP is off\n"); + ret = -1; + } + if (cdsc->param.tool_admvp == 0 && cdsc->param.tool_mmvd == 1) { + av_log(avctx, AV_LOG_ERROR, "MMVD cannot be on when ADMVP is off\n"); + ret = -1; + } + if (cdsc->param.tool_eipd == 0 && cdsc->param.ibc_flag == 1) { + av_log(avctx, AV_LOG_ERROR, "IBC cannot be on when EIPD is off\n"); + ret = -1; + } + if (cdsc->param.tool_iqt == 0 && cdsc->param.tool_ats == 1) { + av_log(avctx, AV_LOG_ERROR, "ATS cannot be on when IQT is off\n"); + ret = -1; + } + if (cdsc->param.tool_cm_init == 0 && cdsc->param.tool_adcc == 1) { + av_log(avctx, AV_LOG_ERROR, "ADCC cannot be on when CM_INIT is off\n"); + ret = -1; + } + } + + if (cdsc->param.btt == 1) { + if (cdsc->param.framework_cb_max && cdsc->param.framework_cb_max < 5) { + av_log(avctx, AV_LOG_ERROR, "Maximun Coding Block size cannot be smaller than 5\n"); + ret = -1; + } + if (cdsc->param.framework_cb_max > 7) { + av_log(avctx, AV_LOG_ERROR, "Maximun Coding Block size cannot be greater than 7\n"); + ret = -1; + } + if (cdsc->param.framework_cb_min && cdsc->param.framework_cb_min < 2) { + av_log(avctx, AV_LOG_ERROR, "Minimum Coding Block size cannot be smaller than 2\n"); + ret = -1; + } + if ((cdsc->param.framework_cb_max || cdsc->param.framework_cb_min) && + cdsc->param.framework_cb_min > cdsc->param.framework_cb_max) { + av_log(avctx, AV_LOG_ERROR, "Minimum Coding Block size cannot be greater than Maximum coding Block size\n"); + ret = -1; + } + if (cdsc->param.framework_cu14_max > 6) { + av_log(avctx, AV_LOG_ERROR, "Maximun 1:4 Coding Block size cannot be greater than 6\n"); + ret = -1; + } + if ((cdsc->param.framework_cb_max || cdsc->param.framework_cu14_max) && + cdsc->param.framework_cu14_max > cdsc->param.framework_cb_max) { + av_log(avctx, AV_LOG_ERROR, "Maximun 1:4 Coding Block size cannot be greater than Maximum coding Block size\n"); + ret = -1; + } + if (cdsc->param.framework_tris_max > 6) { + av_log(avctx, AV_LOG_ERROR, "Maximun Tri-split Block size be greater than 6\n"); + ret = -1; + } + if ((cdsc->param.framework_tris_max || cdsc->param.framework_cb_max) && + cdsc->param.framework_tris_max > cdsc->param.framework_cb_max) { + av_log(avctx, AV_LOG_ERROR, "Maximun Tri-split Block size cannot be greater than Maximum coding Block size\n"); + ret = -1; + } + if ((cdsc->param.framework_tris_min || cdsc->param.framework_cb_min) && + cdsc->param.framework_tris_min < cdsc->param.framework_cb_min + 2) { + av_log(avctx, AV_LOG_ERROR, "Maximun Tri-split Block size cannot be smaller than Minimum Coding Block size plus two\n"); + ret = -1; + } + if(cdsc->param.framework_cb_min) min_block_size = 1 << cdsc->param.framework_cb_min; + else min_block_size = 8; + } + + if (cdsc->param.suco == 1) { + if (cdsc->param.framework_suco_max > 6) { + av_log(avctx, AV_LOG_ERROR, "Maximun SUCO size cannot be greater than 6\n"); + ret = -1; + } + if (cdsc->param.framework_cb_max && cdsc->param.framework_suco_max > cdsc->param.framework_cb_max) { + av_log(avctx, AV_LOG_ERROR, "Maximun SUCO size cannot be greater than Maximum coding Block size\n"); + ret = -1; + } + if (cdsc->param.framework_suco_min < 4) { + av_log(avctx, AV_LOG_ERROR, "Minimun SUCO size cannot be smaller than 4\n"); + ret = -1; + } + if (cdsc->param.framework_cb_min && cdsc->param.framework_suco_min < cdsc->param.framework_cb_min) { + av_log(avctx, AV_LOG_ERROR, "Minimun SUCO size cannot be smaller than Minimum coding Block size\n"); + ret = -1; + } + if (cdsc->param.framework_suco_min > cdsc->param.framework_suco_max) { + av_log(avctx, AV_LOG_ERROR, "Minimum SUCO size cannot be greater than Maximum SUCO size\n"); + ret = -1; + } + } + + pic_m = (8 > min_block_size) ? min_block_size : 8; + if ((cdsc->param.w & (pic_m - 1)) != 0) { + av_log(avctx, AV_LOG_ERROR, "Current encoder does not support picture width, not multiple of max(8, minimum CU size)\n"); + ret = -1; + } + if ((cdsc->param.h & (pic_m - 1)) != 0) { + av_log(avctx, AV_LOG_ERROR, "Current encoder does not support picture height, not multiple of max(8, minimum CU size)\n"); + ret = -1; + } + + return ret; +} + +/** + * Set XEVE_CFG_SET_USE_PIC_SIGNATURE for encoder + * + * @param[in] logger context + * @param[in] id XEVE encodec instance identifier + * @param[in] ctx the structure stores all the states associated with the instance of Xeve MPEG-5 EVC encoder + * + * @return 0 on success, negative error code on failure + */ +static int set_extra_config(AVCodecContext* avctx, XEVE id, XeveContext *ctx) +{ + int ret, size, value; + + if(ctx->hash) { + value = 1; + size = 4; + ret = xeve_config(id, XEVE_CFG_SET_USE_PIC_SIGNATURE, &value, &size); + if(XEVE_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to set config for picture signature\n"); + return -1; + } + } + + return 0; +} + +/** + * Convert FFmpeg pixel format (AVPixelFormat) into XEVE pre-defined color space + * + * @param[in] px_fmt pixel format (@see https://ffmpeg.org/doxygen/trunk/pixfmt_8h.html#a9a8e335cf3be472042bc9f0cf80cd4c5) + * + * @return XEVE pre-defined color space (@see xeve.h) on success, XEVE_CF_UNKNOWN on failure + */ +static int xeve_color_space(enum AVPixelFormat pix_fmt) +{ + /* color space of input image */ + int cs = XEVE_CF_UNKNOWN; + + switch (pix_fmt) { + case AV_PIX_FMT_YUV420P: + cs = XEVE_CS_YCBCR420; + break; + case AV_PIX_FMT_YUV420P10: +#if AV_HAVE_BIGENDIAN + cs = XEVE_CS_SET(XEVE_CF_YCBCR420, 10, 1); +#else + cs = XEVE_CS_YCBCR420_10LE; +#endif + + break; + case AV_PIX_FMT_YUV420P12: +#if AV_HAVE_BIGENDIAN + cs = XEVE_CS_SET(XEVE_CF_YCBCR420, 12, 1); +#else + cs = XEVE_CS_YCBCR420_12LE; +#endif + + break; + case AV_PIX_FMT_YUV422P: + cs = XEVE_CS_YCBCR422; + break; + case AV_PIX_FMT_YUV422P10: +#if AV_HAVE_BIGENDIAN + cs = XEVE_CS_SET(XEVE_CF_YCBCR422, 10, 1); +#else + cs = XEVE_CS_YCBCR422_10LE; +#endif + + break; + case AV_PIX_FMT_YUV422P12: +#if AV_HAVE_BIGENDIAN + cs = XEVE_CS_SET(XEVE_CF_YCBCR422, 12, 1); +#else + cs = XEVE_CS_SET(XEVE_CF_YCBCR422, 12, 0); +#endif + + break; + case AV_PIX_FMT_GBRP: + case AV_PIX_FMT_GBRP10: + case AV_PIX_FMT_GBRP12: + cs = XEVE_CF_UNKNOWN; + break; + case AV_PIX_FMT_YUV444P: + cs = XEVE_CF_YCBCR444; + break; + case AV_PIX_FMT_YUV444P10: +#if AV_HAVE_BIGENDIAN + cs = XEVE_CS_SET(XEVE_CF_YCBCR444, 10, 1); +#else + cs = XEVE_CS_YCBCR444_10LE; +#endif + + break; + case AV_PIX_FMT_YUV444P12: +#if AV_HAVE_BIGENDIAN + cs = XEVE_CS_SET(XEVE_CF_YCBCR444, 12, 1); +#else + cs = XEVE_CS_SET(XEVE_CF_YCBCR444, 12, 0); +#endif + + break; + case AV_PIX_FMT_GRAY8: + cs = XEVE_CF_YCBCR400; + break; + case AV_PIX_FMT_GRAY10: +#if AV_HAVE_BIGENDIAN + cs = XEVE_CS_SET(XEVE_CF_YCBCR400, 10, 1); +#else + cs = XEVE_CS_YCBCR400_10LE; +#endif + + break; + case AV_PIX_FMT_GRAY12: +#if AV_HAVE_BIGENDIAN + cs = XEVE_CS_SET(XEVE_CF_YCBCR400, 12, 1); +#else + cs = XEVE_CS_YCBCR400_12LE; +#endif + + break; + default: + cs = XEVE_CF_UNKNOWN; + break; + } + + return cs; +} + +/** + * @brief Switch encoder to bumping mode + * + * @param id XEVE encodec instance identifier + * @return 0 on success, negative error code on failure + */ +static int setup_bumping(XEVE id) +{ + int val, size; + val = 1; + size = sizeof(int); + if(XEVE_FAILED(xeve_config(id, XEVE_CFG_SET_FORCE_OUT, (void *)(&val), &size))) + return -1; + + return 0; +} + +/** + * @brief Initialize eXtra-fast Essential Video Encoder codec + * Create an encoder instance and allocate all the needed resources + * + * @param avctx codec context + * @return 0 on success, negative error code on failure + */ +static av_cold int libxeve_init(AVCodecContext *avctx) +{ + XeveContext *xectx = avctx->priv_data; + unsigned char *bs_buf = NULL; + int i, val = 0; + int shift_h = 0; + int shift_v = 0; + XEVE_IMGB *imgb = NULL; + + XEVE_CDSC *cdsc = &(xectx->cdsc); + + if(avctx->pix_fmt != AV_PIX_FMT_YUV420P && avctx->pix_fmt != AV_PIX_FMT_YUV420P10) { + av_log(avctx, AV_LOG_ERROR, "Invalid pixel format: %s\n", av_get_pix_fmt_name(avctx->pix_fmt)); + goto ERR; + } + + /* allocate bitstream buffer */ + bs_buf = (unsigned char *)malloc(MAX_BS_BUF); + if(bs_buf == NULL) { + av_log(avctx, AV_LOG_ERROR, "Cannot allocate bitstream buffer\n"); + goto ERR; + } + + /* read configurations and set values for created descriptor (XEVE_CDSC) */ + val = get_conf(avctx, cdsc); + if (val != XEVE_OK) { + av_log(avctx, AV_LOG_ERROR, "Cannot get configuration\n"); + goto ERR; + } + + if (check_conf(avctx, cdsc) != 0) { + av_log(avctx, AV_LOG_ERROR, "Invalid configuration\n"); + goto ERR; + } + + /* create encoder */ + xectx->id = xeve_create(cdsc, NULL); + if(xectx->id == NULL) { + av_log(avctx, AV_LOG_ERROR, "Cannot create XEVE encoder\n"); + goto ERR; + } + + if(set_extra_config(avctx, xectx->id, xectx)) { + av_log(avctx, AV_LOG_ERROR, "Cannot set extra configuration\n"); + goto ERR; + } + + xectx->bitb.addr = bs_buf; + xectx->bitb.bsize = MAX_BS_BUF; + + if(av_pix_fmt_get_chroma_sub_sample(avctx->pix_fmt, &shift_h, &shift_v)) { + av_log(avctx, AV_LOG_ERROR, "Failed to get chroma shift\n"); + goto ERR; + } + + // YUV format explanation + // shift_h == 1 && shift_v == 1 : YUV420 + // shift_h == 1 && shift_v == 0 : YUV422 + // shift_h == 0 && shift_v == 0 : YUV444 + // + xectx->width_chroma = AV_CEIL_RSHIFT(xectx->width_luma, shift_h); + xectx->height_chroma = AV_CEIL_RSHIFT(xectx->height_luma, shift_v); + + /* set default values for input image buffer */ + imgb = &xectx->imgb; + imgb->cs = xeve_color_space(avctx->pix_fmt); + imgb->np = 3; /* only for yuv420p, yuv420ple */ + + for (i = 0; i < imgb->np; i++) + imgb->x[i] = imgb->y[i] = 0; + + imgb->w[0] = imgb->aw[0] = xectx->width_luma; + imgb->w[1] = imgb->w[2] = imgb->aw[1] = imgb->aw[2] = xectx->width_chroma; + imgb->h[0] = imgb->ah[0] = xectx->height_luma; + imgb->h[1] = imgb->h[2] = imgb->ah[1] = imgb->ah[2] = xectx->height_chroma; + + xectx->encod_frames = 0; + xectx->bytes_total = 0; + xectx->state = STATE_ENCODING; + xectx->packet_count = 0; + xectx->bitrate = 0; + + return 0; + +ERR: + if(bs_buf) + free(bs_buf); + + return -1; +} + +/** + * Encode raw data frame into EVC packet + * + * @param[in] avctx codec context + * @param[out] pkt output AVPacket containing encoded data + * @param[in] frame AVFrame containing the raw data to be encoded + * @param[out] got_packet encoder sets to 0 or 1 to indicate that a + * non-empty packet was returned in pkt + * + * @return 0 on success, negative error code on failure + */ +static int libxeve_encode(AVCodecContext *avctx, AVPacket *pkt, + const AVFrame *frame, int *got_packet) +{ + XeveContext *xectx = NULL; + int ret = -1; + int xeve_cs; + if(avctx == NULL || pkt == NULL || got_packet == NULL) { + av_log(avctx, AV_LOG_ERROR, "Invalid arguments\n"); + return -1; + } + xectx = avctx->priv_data; + if(xectx == NULL) { + av_log(avctx, AV_LOG_ERROR, "Invalid XEVE context\n"); + return -1; + } + if(xectx->state == STATE_SKIPPING && frame ) { + xectx->state = STATE_ENCODING; // Entering encoding process + } else if(xectx->state == STATE_ENCODING && frame == NULL) { + if (setup_bumping(xectx->id) == 0) + xectx->state = STATE_BUMPING; // Entering bumping process + else { + av_log(avctx, AV_LOG_ERROR, "Failed to setup bumping\n"); + xectx->state = STATE_SKIPPING; + } + } + + if(xectx->state == STATE_ENCODING) { + const AVPixFmtDescriptor *pixel_fmt_desc = av_pix_fmt_desc_get (frame->format); + if(!pixel_fmt_desc) { + av_log(avctx, AV_LOG_ERROR, "Invalid pixel format descriptor for pixel format: %s\n", av_get_pix_fmt_name(avctx->pix_fmt)); + return -1; + } + + xeve_cs = xeve_color_space(avctx->pix_fmt); + if(xeve_cs != XEVE_CS_YCBCR420 && xeve_cs != XEVE_CS_YCBCR420_10LE) { + av_log(avctx, AV_LOG_ERROR, "Invalid pixel format: %s\n", av_get_pix_fmt_name(avctx->pix_fmt)); + return -1; + } + + { + int i; + XEVE_IMGB *imgb = NULL; + + imgb = &xectx->imgb; + + for (i = 0; i < imgb->np; i++) { + imgb->a[i] = frame->data[i]; + imgb->s[i] = frame->linesize[i]; + } + + if(xectx->id == NULL) { + av_log(avctx, AV_LOG_ERROR, "Invalid XEVE encoder\n"); + return -1; + } + + imgb->ts[0] = frame->pts; + imgb->ts[1] = 0; + imgb->ts[2] = 0; + imgb->ts[3] = 0; + + /* push image to encoder */ + ret = xeve_push(xectx->id, imgb); + if(XEVE_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "xeve_push() failed\n"); + return -1; + } + } + } + if(xectx->state == STATE_ENCODING || xectx->state == STATE_BUMPING) { + + /* encoding */ + ret = xeve_encode(xectx->id, &(xectx->bitb), &(xectx->stat)); + if(XEVE_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "xeve_encode() failed\n"); + return -1; + } + + xectx->encod_frames++; + + /* store bitstream */ + if (ret == XEVE_OK_OUT_NOT_AVAILABLE) { // Return OK but picture is not available yet + *got_packet = 0; + return 0; + } else if(ret == XEVE_OK) { + int av_pic_type; + + if(xectx->stat.write > 0) { + xectx->bytes_total += xectx->stat.write; + + ret = av_grow_packet(pkt, xectx->stat.write); + if (ret < 0) { + av_log(avctx, AV_LOG_ERROR, "Can't allocate memory for AVPacket data\n"); + return ret; + } + + memcpy(pkt->data, xectx->bitb.addr, xectx->stat.write); + + pkt->pts = xectx->bitb.ts[0]; + pkt->dts = xectx->bitb.ts[1]; + + xectx->bitrate += (xectx->stat.write - xectx->stat.sei_size); + + switch(xectx->stat.stype) { + case XEVE_ST_I: + av_pic_type = AV_PICTURE_TYPE_I; + pkt->flags |= AV_PKT_FLAG_KEY; + break; + case XEVE_ST_P: + av_pic_type = AV_PICTURE_TYPE_P; + break; + case XEVE_ST_B: + av_pic_type = AV_PICTURE_TYPE_B; + break; + case XEVE_ST_UNKNOWN: + av_log(avctx, AV_LOG_ERROR, "Unknown slice type\n"); + return -1; + } + + ff_side_data_set_encoder_stats(pkt, xectx->stat.qp * FF_QP2LAMBDA, NULL, 0, av_pic_type); + + xectx->bitrate += (xectx->stat.write - xectx->stat.sei_size); + + *got_packet = 1; + xectx->packet_count++; + } + } else if (ret == XEVE_OK_NO_MORE_FRM) { + // Return OK but no more frames + return 0; + } else { + av_log(avctx, AV_LOG_ERROR, "Invalid return value: %d\n", ret); + return -1; + } + } else { + av_log(avctx, AV_LOG_ERROR, "Udefined encoder state\n", xectx->state); + return -1; + } + + return 0; +} + +/** + * Destroy the encoder and release all the allocated resources + * + * @param avctx codec context + * @return 0 on success, negative error code on failure + */ +static av_cold int libxeve_close(AVCodecContext *avctx) +{ + XeveContext *xectx = avctx->priv_data; + + xeve_delete(xectx->id); + + if(xectx->bitb.addr) + free(xectx->bitb.addr); /* release bitstream buffer */ + + return 0; +} + +#define OFFSET(x) offsetof(XeveContext, x) +#define VE AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_ENCODING_PARAM + +// Example of using: ./ffmpeg -xeve-params "m=2:q=17" +// Consider using following options (./ffmpeg --help encoder=libxeve) +// +static const AVOption xeve_options[] = { + { "preset", "Encoding preset for setting encoding speed [fast, medium, slow, placebo]", OFFSET(op_preset), AV_OPT_TYPE_STRING, { .str = "medium" }, 0, 0, VE }, + { "tune", "Tuneing parameter for special purpose operation [psnr, zerolatency]", OFFSET(op_tune), AV_OPT_TYPE_STRING, { 0 }, 0, 0, VE}, + { "qp", "quantization parameter qp <0..51> [default: 32]", OFFSET(op_qp), AV_OPT_TYPE_INT, { .i64 = 32 }, 0, 51, VE }, + { "crf", "constant rate factor <-1..51> [default: 32]", OFFSET(op_crf), AV_OPT_TYPE_INT, { .i64 = -1 }, -1, 51, VE }, + { "xeve-params", "override the xeve configuration using a : separated list of key=value parameters", OFFSET(xeve_params), AV_OPT_TYPE_DICT, { 0 }, 0, 0, VE }, + { NULL } +}; + +static const AVClass xeve_class = { + .class_name = "libxeve", + .item_name = av_default_item_name, + .option = xeve_options, + .version = LIBAVUTIL_VERSION_INT, +}; + +/** + * libavcodec generic global options, which can be set on all the encoders and decoders + * @see https://www.ffmpeg.org/ffmpeg-codecs.html#Codec-Options + */ +static const AVCodecDefault xeve_defaults[] = { + { "b", "0" }, // bitrate + { "g", "0" }, // gop_size (key-frame interval 0: only one I-frame at the first time; 1: every frame is coded in I-frame) + { "bf", "15"}, // bframes (0: no B-frames) + { "profile", "0"}, // encoder codec profile (0: baselie; 1: main) + { "threads", "0"}, // number of threads to be used (0: automatically select the number of threads to set) + { NULL }, +}; + +AVCodec ff_libxeve_encoder = { + .name = "libxeve", + .long_name = NULL_IF_CONFIG_SMALL("libxeve MPEG-5 EVC"), + .type = AVMEDIA_TYPE_VIDEO, + .id = AV_CODEC_ID_EVC, + .init = libxeve_init, + .encode2 = libxeve_encode, + .close = libxeve_close, + .priv_data_size = sizeof(XeveContext), + .priv_class = &xeve_class, + .defaults = xeve_defaults, + .capabilities = AV_CODEC_CAP_DELAY | AV_CODEC_CAP_AUTO_THREADS | + AV_CODEC_CAP_ENCODER_REORDERED_OPAQUE, + .wrapper_name = "libxeve", +}; diff --git a/libavcodec/parsers.c b/libavcodec/parsers.c index 6b40c18d80..295fa33f52 100644 --- a/libavcodec/parsers.c +++ b/libavcodec/parsers.c @@ -73,6 +73,7 @@ extern const AVCodecParser ff_vp9_parser; extern const AVCodecParser ff_webp_parser; extern const AVCodecParser ff_xbm_parser; extern const AVCodecParser ff_xma_parser; +extern const AVCodecParser ff_evc_parser; #include "libavcodec/parser_list.c" -- 2.17.1 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2022-07-01 9:46 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
[not found] <CGME20220622064904eucas1p19bbbc922c4f1b734730c48169bada35a@eucas1p1.samsung.com>
2022-06-22 6:49 ` [FFmpeg-devel] [PATCH 2/3] Provided support for MPEG-5 EVC (Essential Video Coding) codec Dawid Kozinski
2022-07-01 9:46 ` Anton Khirnov
[not found] <CGME20220512060536eucas1p262e353ab046a9c672e44aec493ef7d4f@eucas1p2.samsung.com>
2022-05-12 6:05 ` Dawid Kozinski
2022-06-01 11:41 ` Andreas Rheinhardt
[not found] <CGME20220420123208eucas1p1359df4487ab7f4a7edde29b4f80cdfdc@eucas1p1.samsung.com>
2022-04-20 12:32 ` Dawid Kozinski
[not found] <CGME20220420095424eucas1p13b9dc5cacc7b763a05ac32ca25b14e69@eucas1p1.samsung.com>
2022-04-20 9:54 ` Dawid Kozinski
Git Inbox Mirror of the ffmpeg-devel mailing list - see https://ffmpeg.org/mailman/listinfo/ffmpeg-devel This inbox may be cloned and mirrored by anyone: git clone --mirror https://master.gitmailbox.com/ffmpegdev/0 ffmpegdev/git/0.git # If you have public-inbox 1.1+ installed, you may # initialize and index your mirror using the following commands: public-inbox-init -V2 ffmpegdev ffmpegdev/ https://master.gitmailbox.com/ffmpegdev \ ffmpegdev@gitmailbox.com public-inbox-index ffmpegdev Example config snippet for mirrors. AGPL code for this site: git clone https://public-inbox.org/public-inbox.git