* [FFmpeg-devel] [PATCH 1/2] Provided support for MPEG-5 EVC (Essential Video Coding) codec [not found] <CGME20220808123033eucas1p2c85ea9811cb74a92762a4c9a28564d7d@eucas1p2.samsung.com> @ 2022-08-08 12:30 ` Dawid Kozinski 2022-08-11 7:44 ` Andreas Rheinhardt 0 siblings, 1 reply; 13+ messages in thread From: Dawid Kozinski @ 2022-08-08 12:30 UTC (permalink / raw) To: ffmpeg-devel - Added new entry to codec IDs list - Added new entry to the codec descriptor list - Bumped libavcodec minor version - Changes in Changelog and MAINTAINERS files - Added xeve encoder wrapper - Added xevd dencoder wrapper - Added documentation for xeve and xevd wrappers - Added parser for EVC format - Changes in project configuration file and libavcodec Makefile Signed-off-by: Dawid Kozinski <d.kozinski@samsung.com> --- Changelog | 3 +- MAINTAINERS | 2 + configure | 8 + doc/decoders.texi | 24 ++ doc/encoders.texi | 92 +++++ doc/general_contents.texi | 19 + libavcodec/Makefile | 3 + libavcodec/allcodecs.c | 2 + libavcodec/avcodec.h | 3 + libavcodec/codec_desc.c | 8 + libavcodec/codec_id.h | 1 + libavcodec/evc_parser.c | 758 ++++++++++++++++++++++++++++++++++++++ libavcodec/libxevd.c | 419 +++++++++++++++++++++ libavcodec/libxeve.c | 597 ++++++++++++++++++++++++++++++ libavcodec/parsers.c | 1 + libavcodec/profiles.c | 6 + libavcodec/profiles.h | 1 + libavcodec/version.h | 2 +- 18 files changed, 1947 insertions(+), 2 deletions(-) create mode 100644 libavcodec/evc_parser.c create mode 100644 libavcodec/libxevd.c create mode 100644 libavcodec/libxeve.c diff --git a/Changelog b/Changelog index cc271c22bd..dbeb08c373 100644 --- a/Changelog +++ b/Changelog @@ -82,7 +82,8 @@ version 5.0: - VideoToolbox ProRes encoder - anlmf audio filter - IMF demuxer (experimental) - +- eXtra-fast Essential Video Encoder (XEVE) +- eXtra-fast Essential Video Decoder (XEVD) version 4.4: - AudioToolbox output device diff --git a/MAINTAINERS b/MAINTAINERS index 7ed15f96f6..7d4d543a13 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -206,6 +206,7 @@ Codecs: libvpx* James Zern libxavs.c Stefan Gehrer libxavs2.c Huiwen Ren + libxev*.c, evc_parser.c Dawid Kozinski libzvbi-teletextdec.c Marton Balint lzo.h, lzo.c Reimar Doeffinger mdec.c Michael Niedermayer @@ -426,6 +427,7 @@ Muxers/Demuxers: dv.c Roman Shaposhnik electronicarts.c Peter Ross epafdec.c Paul B Mahol + evcdec.c Dawid Kozinski ffm* Baptiste Coudurier flic.c Mike Melanson flvdec.c Michael Niedermayer diff --git a/configure b/configure index cbbb4dd9c8..e6582eae91 100755 --- a/configure +++ b/configure @@ -291,6 +291,8 @@ External library support: --enable-libwebp enable WebP encoding via libwebp [no] --enable-libx264 enable H.264 encoding via x264 [no] --enable-libx265 enable HEVC encoding via x265 [no] + --enable-libxeve enable EVC encoding via libxeve [no] + --enable-libxevd enable EVC decoding via libxevd [no] --enable-libxavs enable AVS encoding via xavs [no] --enable-libxavs2 enable AVS2 encoding via xavs2 [no] --enable-libxcb enable X11 grabbing using XCB [autodetect] @@ -1874,6 +1876,8 @@ EXTERNAL_LIBRARY_LIST=" libvorbis libvpx libwebp + libxevd + libxeve libxml2 libzimg libzmq @@ -3395,6 +3399,8 @@ libx264rgb_encoder_select="libx264_encoder" libx265_encoder_deps="libx265" libxavs_encoder_deps="libxavs" libxavs2_encoder_deps="libxavs2" +libxevd_decoder_deps="libxevd" +libxeve_encoder_deps="libxeve" libxvid_encoder_deps="libxvid" libzvbi_teletext_decoder_deps="libzvbi" vapoursynth_demuxer_deps="vapoursynth" @@ -6685,6 +6691,8 @@ enabled libx265 && require_pkg_config libx265 x265 x265.h x265_api_get require_cpp_condition libx265 x265.h "X265_BUILD >= 70" enabled libxavs && require libxavs "stdint.h xavs.h" xavs_encoder_encode "-lxavs $pthreads_extralibs $libm_extralibs" enabled libxavs2 && require_pkg_config libxavs2 "xavs2 >= 1.3.0" "stdint.h xavs2.h" xavs2_api_get +enabled libxevd && require_pkg_config libxevd "xevd >= 0.4.0" "xevd.h" xevd_decode +enabled libxeve && require_pkg_config libxeve "xeve >= 0.4.0" "xeve.h" xeve_encode enabled libxvid && require libxvid xvid.h xvid_global -lxvidcore enabled libzimg && require_pkg_config libzimg "zimg >= 2.7.0" zimg.h zimg_get_api_version enabled libzmq && require_pkg_config libzmq "libzmq >= 4.2.1" zmq.h zmq_ctx_new diff --git a/doc/decoders.texi b/doc/decoders.texi index e2fcbf5dc9..0bd9096114 100644 --- a/doc/decoders.texi +++ b/doc/decoders.texi @@ -126,6 +126,30 @@ Set amount of frame threads to use during decoding. The default value is 0 (auto @end table +@section libxevd + +eXtra-fast Essential Video Decoder (XEVD) MPEG-5 EVC decoder wrapper. + +This decoder requires the presence of the libxevd headers and library +during configuration. You need to explicitly configure the build with +@option{--enable-libxevd}. + +The xevd project website is at @url{https://github.com/mpeg5/xevd}. + +@subsection Options + +The following options are supported by the libxevd wrapper. +The xevd-equivalent options or values are listed in parentheses for easy migration. + +To get a more accurate and extensive documentation of the libxevd options, +invoke the command @code{xevd_app --help} or consult the libxevd documentation. + +@table @option +@item threads (@emph{threads}) +Force to use a specific number of threads + +@end table + @section QSV Decoders The family of Intel QuickSync Video decoders (VC1, MPEG-2, H.264, HEVC, diff --git a/doc/encoders.texi b/doc/encoders.texi index 6d73f74196..76f55e6721 100644 --- a/doc/encoders.texi +++ b/doc/encoders.texi @@ -2892,6 +2892,98 @@ ffmpeg -i input -c:v libxavs2 -xavs2-params RdoqLevel=0 output.avs2 @end example @end table +@section libxeve + +eXtra-fast Essential Video Encoder (XEVE) MPEG-5 EVC encoder wrapper. +The xeve-equivalent options or values are listed in parentheses for easy migration. + +This encoder requires the presence of the libxeve headers and library +during configuration. You need to explicitly configure the build with +@option{--enable-libxeve}. + +@float NOTE +Many libxeve encoder options are mapped to FFmpeg global codec options, +while unique encoder options are provided through private options. +Additionally the xeve-params private options allows one to pass a list +of key=value tuples as accepted by the libxeve @code{parse_xeve_params} function. +@end float + +The xeve project website is at @url{https://github.com/mpeg5/xeve}. + +@subsection Options + +The following options are supported by the libxeve wrapper. +The xeve-equivalent options or values are listed in parentheses for easy migration. + +@float NOTE +To reduce the duplication of documentation, only the private options +and some others requiring special attention are documented here. For +the documentation of the undocumented generic options, see +@ref{codec-options,,the Codec Options chapter}. +@end float + +@float NOTE +To get a more accurate and extensive documentation of the libxeve options, +invoke the command @code{xeve_app --help} or consult the libxeve documentation. +@end float + +@table @option +@item b (@emph{bitrate}) +Set target video bitrate in bits/s. +Note that FFmpeg's b option is expressed in bits/s, while xeve's bitrate is in kilobits/s. + +@item bf (@emph{bframes}) +Set the maximum number of B frames (1,3,7,15). + +@item g (@emph{keyint}) +Set the GOP size (I-picture period). + +@item preset (@emph{preset}) +Set the xeve preset. +Set the encoder preset value to determine encoding speed [fast, medium, slow, placebo] + +@item tune (@emph{tune}) +Set the encoder tune parameter [psnr, zerolatency] + +@item profile (@emph{profile}) +Set the encoder profile [0: baselie; 1: main] + +@item crf (@emph{crf}) +Set the quality for constant quality mode. +Constant rate factor <10..49> [default: 32] + +@item qp (@emph{qp}) +Set constant quantization rate control method parameter. +Quantization parameter qp <0..51> [default: 32] + +@item threads (@emph{threads}) +Force to use a specific number of threads + +@item xeve-params +Set xeve options using a list of @var{key}=@var{value} couples separated +by ":". + +For example to specify libxeve encoding options with @option{-xeve-params}: + +@example +ffmpeg -i input -c:v libxeve -xeve-params profile=main:tune=zerolatency:preset=fast:q=17:bframes=15:bitrate=100k output.mp4 +@end example +@end table + +@subsubsection Supported key values for xeve-params + +Set xeve options using a list of @var{key}=@var{value} couples separated + +@table @option + +@item hash +Embed the picture signature (HASH) for conformance checking in decoding, (0: Off, 1: On) + +@item vbv-bufsize +Set the VBV buffer size: Kbits(none,K,k), Mbits(M,m) + +@end table + @section libxvid Xvid MPEG-4 Part 2 encoder wrapper. diff --git a/doc/general_contents.texi b/doc/general_contents.texi index 86ec6d606b..89236047ea 100644 --- a/doc/general_contents.texi +++ b/doc/general_contents.texi @@ -343,6 +343,22 @@ libxavs2 is under the GNU Public License Version 2 or later details), you must upgrade FFmpeg's license to GPL in order to use it. @end float +@section eXtra-fast Essential Video Encoder (XEVE) + +FFmpeg can make use of the XEVE library for EVC video encoding. + +Go to @url{https://github.com/mpeg5/xeve} and follow the instructions for +installing the XEVE library. Then pass @code{--enable-libxeve} to configure to +enable it. + +@section eXtra-fast Essential Video Decoder (XEVD) + +FFmpeg can make use of the XEVD library for EVC video decoding. + +Go to @url{https://github.com/mpeg5/xevd} and follow the instructions for +installing the XEVD library. Then pass @code{--enable-libxevd} to configure to +enable it. + @section ZVBI ZVBI is a VBI decoding library which can be used by FFmpeg to decode DVB @@ -591,6 +607,7 @@ library: @item raw DTS @tab X @tab X @item raw DTS-HD @tab @tab X @item raw E-AC-3 @tab X @tab X +@item raw EVC @tab X @tab X @item raw FLAC @tab X @tab X @item raw GSM @tab @tab X @item raw H.261 @tab X @tab X @@ -930,6 +947,8 @@ following image formats are supported: @item Electronic Arts TQI video @tab @tab X @item Escape 124 @tab @tab X @item Escape 130 @tab @tab X +@item EVC / MPEG-5 Part 1 @tab X @tab X + @tab encoding and decoding supported through external libraries libxeve and libxevd @item FFmpeg video codec #1 @tab X @tab X @tab lossless codec (fourcc: FFV1) @item Flash Screen Video v1 @tab X @tab X diff --git a/libavcodec/Makefile b/libavcodec/Makefile index 83901bb52b..cfeacd96d1 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -1102,6 +1102,8 @@ OBJS-$(CONFIG_LIBX264_ENCODER) += libx264.o OBJS-$(CONFIG_LIBX265_ENCODER) += libx265.o OBJS-$(CONFIG_LIBXAVS_ENCODER) += libxavs.o OBJS-$(CONFIG_LIBXAVS2_ENCODER) += libxavs2.o +OBJS-$(CONFIG_LIBXEVD_DECODER) += libxevd.o +OBJS-$(CONFIG_LIBXEVE_ENCODER) += libxeve.o OBJS-$(CONFIG_LIBXVID_ENCODER) += libxvid.o OBJS-$(CONFIG_LIBZVBI_TELETEXT_DECODER) += libzvbi-teletextdec.o ass.o @@ -1129,6 +1131,7 @@ OBJS-$(CONFIG_DVAUDIO_PARSER) += dvaudio_parser.o OBJS-$(CONFIG_DVBSUB_PARSER) += dvbsub_parser.o OBJS-$(CONFIG_DVD_NAV_PARSER) += dvd_nav_parser.o OBJS-$(CONFIG_DVDSUB_PARSER) += dvdsub_parser.o +OBJS-$(CONFIG_EVC_PARSER) += evc_parser.o OBJS-$(CONFIG_FLAC_PARSER) += flac_parser.o flacdata.o flac.o OBJS-$(CONFIG_G723_1_PARSER) += g723_1_parser.o OBJS-$(CONFIG_G729_PARSER) += g729_parser.o diff --git a/libavcodec/allcodecs.c b/libavcodec/allcodecs.c index c94e2d5966..2526e11f70 100644 --- a/libavcodec/allcodecs.c +++ b/libavcodec/allcodecs.c @@ -800,6 +800,8 @@ extern LIBX264_CONST FFCodec ff_libx264_encoder; #endif extern const FFCodec ff_libx264rgb_encoder; extern FFCodec ff_libx265_encoder; +extern const FFCodec ff_libxeve_encoder; +extern const FFCodec ff_libxevd_decoder; extern const FFCodec ff_libxavs_encoder; extern const FFCodec ff_libxavs2_encoder; extern const FFCodec ff_libxvid_encoder; diff --git a/libavcodec/avcodec.h b/libavcodec/avcodec.h index 65c8535359..7d56a03fca 100644 --- a/libavcodec/avcodec.h +++ b/libavcodec/avcodec.h @@ -1674,6 +1674,9 @@ typedef struct AVCodecContext { #define FF_PROFILE_KLVA_SYNC 0 #define FF_PROFILE_KLVA_ASYNC 1 +#define FF_PROFILE_EVC_BASELINE 0 +#define FF_PROFILE_EVC_MAIN 1 + /** * level * - encoding: Set by user. diff --git a/libavcodec/codec_desc.c b/libavcodec/codec_desc.c index c1a177c22d..1a43ff8cb6 100644 --- a/libavcodec/codec_desc.c +++ b/libavcodec/codec_desc.c @@ -1907,6 +1907,14 @@ static const AVCodecDescriptor codec_descriptors[] = { .long_name = NULL_IF_CONFIG_SMALL("WBMP (Wireless Application Protocol Bitmap) image"), .props = AV_CODEC_PROP_INTRA_ONLY | AV_CODEC_PROP_LOSSLESS, }, + { + .id = AV_CODEC_ID_EVC, + .type = AVMEDIA_TYPE_VIDEO, + .name = "evc", + .long_name = NULL_IF_CONFIG_SMALL("MPEG-5 EVC (Essential Video Coding)"), + .props = AV_CODEC_PROP_LOSSY | AV_CODEC_PROP_REORDER, + .profiles = NULL_IF_CONFIG_SMALL(ff_evc_profiles), + }, /* various PCM "codecs" */ { diff --git a/libavcodec/codec_id.h b/libavcodec/codec_id.h index 386a00a7ef..02b60be690 100644 --- a/libavcodec/codec_id.h +++ b/libavcodec/codec_id.h @@ -314,6 +314,7 @@ enum AVCodecID { AV_CODEC_ID_PHM, AV_CODEC_ID_RADIANCE_HDR, AV_CODEC_ID_WBMP, + AV_CODEC_ID_EVC, /* various PCM "codecs" */ AV_CODEC_ID_FIRST_AUDIO = 0x10000, ///< A dummy id pointing at the start of audio codecs diff --git a/libavcodec/evc_parser.c b/libavcodec/evc_parser.c new file mode 100644 index 0000000000..0ae0eaa08a --- /dev/null +++ b/libavcodec/evc_parser.c @@ -0,0 +1,758 @@ +/* + * EVC format parser + * + * Copyright (C) 2021 Dawid Kozinski <d.kozinski@samsung.com> + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include <stdint.h> + +#include "libavutil/common.h" + +#include "parser.h" +#include "golomb.h" + +// The length field that indicates the length in bytes of the following NAL unit is configured to be of 4 bytes +#define EVC_NAL_UNIT_LENGTH_BYTE (4) /* byte */ + +#define EVC_NAL_HEADER_SIZE (2) /* byte */ +#define MAX_SPS_CNT (16) /* defined value in EVC standard */ +#define MAX_PPS_CNT (64) /* defined value in EVC standard */ + +#define MAX_NUM_TILES_ROW (22) +#define MAX_NUM_TILES_COL (20) + +// NALU types +// @see ISO_IEC_23094-1_2020 7.4.2.2 NAL unit header semantics +// +#define EVC_NUT_NONIDR (0) /* Coded slice of a non-IDR picture */ +#define EVC_NUT_IDR (1) /* Coded slice of an IDR picture */ +#define EVC_NUT_SPS (24) /* Sequence parameter set */ +#define EVC_NUT_PPS (25) /* Picture paremeter set */ +#define EVC_NUT_APS (26) /* Adaptation parameter set */ +#define EVC_NUT_FD (27) /* Filler data */ +#define EVC_NUT_SEI (28) /* Supplemental enhancement information */ + +// slice type +// @see ISO_IEC_23094-1_2020 7.4.5 Slice header semantics +// +#define SLICE_TYPE_UNKNOWN (-1) +#define SLICE_TYPE_B (0) +#define SLICE_TYPE_P (1) +#define SLICE_TYPE_I (2) + +#define MAX_NUM_RPLS (32) +#define MAX_NUM_REF_PICS (21) + +// rpl structure +typedef struct RefPicListStruct { + int poc; + int tid; + int ref_pic_num; + int ref_pic_active_num; + int ref_pics[MAX_NUM_REF_PICS]; + char pic_type; + +} RefPicListStruct; + +// The sturcture reflects SPS RBSP(raw byte sequence payload) layout +// @see ISO_IEC_23094-1 section 7.3.2.1 +// +// The following descriptors specify the parsing process of each element +// u(n) - unsigned integer using n bits +// ue(v) - unsigned integer 0-th order Exp_Golomb-coded syntax element with the left bit first +typedef struct EVCParserSPS { + int sps_seq_parameter_set_id; // ue(v) + int profile_idc; // u(8) + int level_idc; // u(8) + int toolset_idc_h; // u(32) + int toolset_idc_l; // u(32) + int chroma_format_idc; // ue(v) + int pic_width_in_luma_samples; // ue(v) + int pic_height_in_luma_samples; // ue(v) + int bit_depth_luma_minus8; // ue(v) + int bit_depth_chroma_minus8; // ue(v) + + int sps_btt_flag; // u(1) + int log2_ctu_size_minus5; // ue(v) + int log2_min_cb_size_minus2; // ue(v) + int log2_diff_ctu_max_14_cb_size; // ue(v) + int log2_diff_ctu_max_tt_cb_size; // ue(v) + int log2_diff_min_cb_min_tt_cb_size_minus2; // ue(v) + + int sps_suco_flag; // u(1) + int log2_diff_ctu_size_max_suco_cb_size; // ue(v) + int log2_diff_max_suco_min_suco_cb_size; // ue(v) + + int sps_admvp_flag; // u(1) + int sps_affine_flag; // u(1) + int sps_amvr_flag; // u(1) + int sps_dmvr_flag; // u(1) + int sps_mmvd_flag; // u(1) + int sps_hmvp_flag; // u(1) + + int sps_eipd_flag; // u(1) + int sps_ibc_flag; // u(1) + int log2_max_ibc_cand_size_minus2; // ue(v) + + int sps_cm_init_flag; // u(1) + int sps_adcc_flag; // u(1) + + int sps_iqt_flag; // u(1) + int sps_ats_flag; // u(1) + + int sps_addb_flag; // u(1) + int sps_alf_flag; // u(1) + int sps_htdf_flag; // u(1) + int sps_rpl_flag; // u(1) + int sps_pocs_flag; // u(1) + int sps_dquant_flag; // u(1) + int sps_dra_flag; // u(1) + + int log2_max_pic_order_cnt_lsb_minus4; // ue(v) + int log2_sub_gop_length; // ue(v) + int log2_ref_pic_gap_length; // ue(v) + + int max_num_tid0_ref_pics; // ue(v) + + int sps_max_dec_pic_buffering_minus1; // ue(v) + int long_term_ref_pic_flag; // u(1) + int rpl1_same_as_rpl0_flag; // u(1) + int num_ref_pic_list_in_sps[2]; // ue(v) + RefPicListStruct rpls[2][MAX_NUM_RPLS]; + + int picture_cropping_flag; // u(1) + int picture_crop_left_offset; // ue(v) + int picture_crop_right_offset; // ue(v) + int picture_crop_top_offset; // ue(v) + int picture_crop_bottom_offset; // ue(v) + + // @note + // Currently the structure does not reflect the entire SPS RBSP layout. + // It contains only the fields that are necessary to read from the NAL unit all the values + // necessary for the correct initialization of the AVCodecContext structure. + + // @note + // If necessary, add the missing fields to the structure to reflect + // the contents of the entire NAL unit of the SPS type + +} EVCParserSPS; + +typedef struct EVCParserPPS { + int pps_pic_parameter_set_id; // ue(v) + int pps_seq_parameter_set_id; // ue(v) + int num_ref_idx_default_active_minus1[2]; // ue(v) + int additional_lt_poc_lsb_len; // ue(v) + int rpl1_idx_present_flag; // u(1) + int single_tile_in_pic_flag; // u(1) + int num_tile_columns_minus1; // ue(v) + int num_tile_rows_minus1; // ue(v) + int uniform_tile_spacing_flag; // u(1) + int tile_column_width_minus1[MAX_NUM_TILES_ROW]; // ue(v) + int tile_row_height_minus1[MAX_NUM_TILES_COL]; // ue(v) + int loop_filter_across_tiles_enabled_flag; // u(1) + int tile_offset_len_minus1; // ue(v) + int tile_id_len_minus1; // ue(v) + int explicit_tile_id_flag; // u(1) + int tile_id_val[MAX_NUM_TILES_ROW][MAX_NUM_TILES_COL]; // u(v) + int pic_dra_enabled_flag; // u(1) + int pic_dra_aps_id; // u(5) + int arbitrary_slice_present_flag; // u(1) + int constrained_intra_pred_flag; // u(1) + int cu_qp_delta_enabled_flag; // u(1) + int log2_cu_qp_delta_area_minus6; // ue(v) + +} EVCParserPPS; + +// The sturcture reflects Slice Header RBSP(raw byte sequence payload) layout +// @see ISO_IEC_23094-1 section 7.3.2.6 +// +// The following descriptors specify the parsing process of each element +// u(n) - unsigned integer using n bits +// ue(v) - unsigned integer 0-th order Exp_Golomb-coded syntax element with the left bit first +// u(n) - unsigned integer using n bits. +// When n is "v" in the syntax table, the number of bits varies in a manner dependent on the value of other syntax elements. +typedef struct EVCParserSliceHeader { + int slice_pic_parameter_set_id; // ue(v) + int single_tile_in_slice_flag; // u(1) + int first_tile_id; // u(v) + int arbitrary_slice_flag; // u(1) + int last_tile_id; // u(v) + int num_remaining_tiles_in_slice_minus1; // ue(v) + int delta_tile_id_minus1[MAX_NUM_TILES_ROW * MAX_NUM_TILES_COL]; // ue(v) + + int slice_type; // ue(v) + + // @note + // Currently the structure does not reflect the entire Slice Header RBSP layout. + // It contains only the fields that are necessary to read from the NAL unit all the values + // necessary for the correct initialization of the AVCodecContext structure. + + // @note + // If necessary, add the missing fields to the structure to reflect + // the contents of the entire NAL unit of the SPS type + +} EVCParserSliceHeader; + +typedef struct EVCParserContext { + ParseContext pc; + EVCParserSPS sps[MAX_SPS_CNT]; + EVCParserPPS pps[MAX_PPS_CNT]; + EVCParserSliceHeader slice_header[MAX_PPS_CNT]; + int is_avc; + int nal_length_size; + int to_read; + int incomplete_nalu_prefix_read; // The flag is set to 1 when an incomplete NAL unit prefix has been read + +} EVCParserContext; + +static int get_nalu_type(const uint8_t *bits, int bits_size, AVCodecContext *avctx) +{ + int unit_type_plus1 = 0; + + if (bits_size >= EVC_NAL_HEADER_SIZE) { + unsigned char *p = (unsigned char *)bits; + // forbidden_zero_bit + if ((p[0] & 0x80) != 0) + return -1; + + // nal_unit_type + unit_type_plus1 = (p[0] >> 1) & 0x3F; + } + + return unit_type_plus1 - 1; +} + +static uint32_t read_nal_unit_length(const uint8_t *bits, int bits_size, AVCodecContext *avctx) +{ + uint32_t nalu_len = 0; + + if (bits_size >= EVC_NAL_UNIT_LENGTH_BYTE) { + + int t = 0; + unsigned char *p = (unsigned char *)bits; + + for (int i = 0; i < EVC_NAL_UNIT_LENGTH_BYTE; i++) + t = (t << 8) | p[i]; + + nalu_len = t; + if (nalu_len == 0) + return 0; + } + + return nalu_len; +} + +// @see ISO_IEC_23094-1 (7.3.2.1 SPS RBSP syntax) +static EVCParserSPS *parse_sps(const uint8_t *bs, int bs_size, EVCParserContext *ev) +{ + GetBitContext gb; + EVCParserSPS *sps; + int sps_seq_parameter_set_id; + + if (init_get_bits8(&gb, bs, bs_size) < 0) + return NULL; + + sps_seq_parameter_set_id = get_ue_golomb(&gb); + + if (sps_seq_parameter_set_id >= MAX_SPS_CNT) + return NULL; + + sps = &ev->sps[sps_seq_parameter_set_id]; + sps->sps_seq_parameter_set_id = sps_seq_parameter_set_id; + + // the Baseline profile is indicated by profile_idc eqal to 0 + // the Main profile is indicated by profile_idc eqal to 1 + sps->profile_idc = get_bits(&gb, 8); + + sps->level_idc = get_bits(&gb, 8); + + skip_bits_long(&gb, 32); /* skip toolset_idc_h */ + skip_bits_long(&gb, 32); /* skip toolset_idc_l */ + + // 0 - monochrome + // 1 - 4:2:0 + // 2 - 4:2:2 + // 3 - 4:4:4 + sps->chroma_format_idc = get_ue_golomb(&gb); + + sps->pic_width_in_luma_samples = get_ue_golomb(&gb); + sps->pic_height_in_luma_samples = get_ue_golomb(&gb); + + sps->bit_depth_luma_minus8 = get_ue_golomb(&gb); + sps->bit_depth_chroma_minus8 = get_ue_golomb(&gb); + + sps->sps_btt_flag = get_bits(&gb, 1); + if (sps->sps_btt_flag) { + sps->log2_ctu_size_minus5 = get_ue_golomb(&gb); + sps->log2_min_cb_size_minus2 = get_ue_golomb(&gb); + sps->log2_diff_ctu_max_14_cb_size = get_ue_golomb(&gb); + sps->log2_diff_ctu_max_tt_cb_size = get_ue_golomb(&gb); + sps->log2_diff_min_cb_min_tt_cb_size_minus2 = get_ue_golomb(&gb); + } + + sps->sps_suco_flag = get_bits(&gb, 1); + if (sps->sps_suco_flag) { + sps->log2_diff_ctu_size_max_suco_cb_size = get_ue_golomb(&gb); + sps->log2_diff_max_suco_min_suco_cb_size = get_ue_golomb(&gb); + } + + sps->sps_admvp_flag = get_bits(&gb, 1); + if (sps->sps_admvp_flag) { + sps->sps_affine_flag = get_bits(&gb, 1); + sps->sps_amvr_flag = get_bits(&gb, 1); + sps->sps_dmvr_flag = get_bits(&gb, 1); + sps->sps_mmvd_flag = get_bits(&gb, 1); + sps->sps_hmvp_flag = get_bits(&gb, 1); + } + + sps->sps_eipd_flag = get_bits(&gb, 1); + if (sps->sps_eipd_flag) { + sps->sps_ibc_flag = get_bits(&gb, 1); + if (sps->sps_ibc_flag) + sps->log2_max_ibc_cand_size_minus2 = get_ue_golomb(&gb); + } + + sps->sps_cm_init_flag = get_bits(&gb, 1); + if (sps->sps_cm_init_flag) + sps->sps_adcc_flag = get_bits(&gb, 1); + + sps->sps_iqt_flag = get_bits(&gb, 1); + if (sps->sps_iqt_flag) + sps->sps_ats_flag = get_bits(&gb, 1); + + sps->sps_addb_flag = get_bits(&gb, 1); + sps->sps_alf_flag = get_bits(&gb, 1); + sps->sps_htdf_flag = get_bits(&gb, 1); + sps->sps_rpl_flag = get_bits(&gb, 1); + sps->sps_pocs_flag = get_bits(&gb, 1); + sps->sps_dquant_flag = get_bits(&gb, 1); + sps->sps_dra_flag = get_bits(&gb, 1); + + if (sps->sps_pocs_flag) + sps->log2_max_pic_order_cnt_lsb_minus4 = get_ue_golomb(&gb); + + if (!sps->sps_pocs_flag || !sps->sps_rpl_flag) { + sps->log2_sub_gop_length = get_ue_golomb(&gb); + if (sps->log2_sub_gop_length == 0) + sps->log2_ref_pic_gap_length = get_ue_golomb(&gb); + } + + // @note + // If necessary, add the missing fields to the EVCParserSPS structure + // and then extend parser implementation + + return sps; +} + +// @see ISO_IEC_23094-1 (7.3.2.2 SPS RBSP syntax) +static EVCParserPPS *parse_pps(const uint8_t *bs, int bs_size, EVCParserContext *ev) +{ + GetBitContext gb; + EVCParserPPS *pps; + + int pps_pic_parameter_set_id; + + if (init_get_bits8(&gb, bs, bs_size) < 0) + return NULL; + + pps_pic_parameter_set_id = get_ue_golomb(&gb); + if (pps_pic_parameter_set_id > MAX_PPS_CNT) + return NULL; + + pps = &ev->pps[pps_pic_parameter_set_id]; + + pps->pps_pic_parameter_set_id = pps_pic_parameter_set_id; + + pps->pps_seq_parameter_set_id = get_ue_golomb(&gb); + if (pps->pps_seq_parameter_set_id >= MAX_SPS_CNT) + return NULL; + + pps->num_ref_idx_default_active_minus1[0] = get_ue_golomb(&gb); + pps->num_ref_idx_default_active_minus1[1] = get_ue_golomb(&gb); + pps->additional_lt_poc_lsb_len = get_ue_golomb(&gb); + pps->rpl1_idx_present_flag = get_bits(&gb, 1); + pps->single_tile_in_pic_flag = get_bits(&gb, 1); + + if (!pps->single_tile_in_pic_flag) { + pps->num_tile_columns_minus1 = get_ue_golomb(&gb); + pps->num_tile_rows_minus1 = get_ue_golomb(&gb); + pps->uniform_tile_spacing_flag = get_bits(&gb, 1); + + if (!pps->uniform_tile_spacing_flag) { + for (int i = 0; i < pps->num_tile_columns_minus1; i++) + pps->tile_column_width_minus1[i] = get_ue_golomb(&gb); + + for (int i = 0; i < pps->num_tile_rows_minus1; i++) + pps->tile_row_height_minus1[i] = get_ue_golomb(&gb); + } + pps->loop_filter_across_tiles_enabled_flag = get_bits(&gb, 1); + pps->tile_offset_len_minus1 = get_ue_golomb(&gb); + } + + pps->tile_id_len_minus1 = get_ue_golomb(&gb); + pps->explicit_tile_id_flag = get_bits(&gb, 1); + + if (pps->explicit_tile_id_flag) { + for (int i = 0; i <= pps->num_tile_rows_minus1; i++) { + for (int j = 0; j <= pps->num_tile_columns_minus1; j++) + pps->tile_id_val[i][j] = get_bits(&gb, pps->tile_id_len_minus1 + 1); + } + } + + pps->pic_dra_enabled_flag = 0; + pps->pic_dra_enabled_flag = get_bits(&gb, 1); + + if (pps->pic_dra_enabled_flag) + pps->pic_dra_aps_id = get_bits(&gb, 5); + + pps->arbitrary_slice_present_flag = get_bits(&gb, 1); + pps->constrained_intra_pred_flag = get_bits(&gb, 1); + pps->cu_qp_delta_enabled_flag = get_bits(&gb, 1); + + if (pps->cu_qp_delta_enabled_flag) + pps->log2_cu_qp_delta_area_minus6 = get_ue_golomb(&gb); + + return pps; +} + +// @see ISO_IEC_23094-1 (7.3.2.6 Slice layer RBSP syntax) +static EVCParserSliceHeader *parse_slice_header(const uint8_t *bs, int bs_size, EVCParserContext *ev) +{ + GetBitContext gb; + EVCParserSliceHeader *sh; + EVCParserPPS *pps; + int num_tiles_in_slice = 0; + int slice_pic_parameter_set_id; + + if (init_get_bits8(&gb, bs, bs_size) < 0) + return NULL; + + slice_pic_parameter_set_id = get_ue_golomb(&gb); + + if (slice_pic_parameter_set_id < 0 || slice_pic_parameter_set_id >= MAX_PPS_CNT) + return NULL; + + sh = &ev->slice_header[slice_pic_parameter_set_id]; + pps = &ev->pps[slice_pic_parameter_set_id]; + + sh->slice_pic_parameter_set_id = slice_pic_parameter_set_id; + + if (!pps->single_tile_in_pic_flag) { + sh->single_tile_in_slice_flag = get_bits(&gb, 1); + sh->first_tile_id = get_bits(&gb, pps->tile_id_len_minus1 + 1); + } else + sh->single_tile_in_slice_flag = 1; + + if (!sh->single_tile_in_slice_flag) { + if (pps->arbitrary_slice_present_flag) + sh->arbitrary_slice_flag = get_bits(&gb, 1); + + if (!sh->arbitrary_slice_flag) + sh->last_tile_id = get_bits(&gb, pps->tile_id_len_minus1 + 1); + else { + sh->num_remaining_tiles_in_slice_minus1 = get_ue_golomb(&gb); + num_tiles_in_slice = sh->num_remaining_tiles_in_slice_minus1 + 2; + for (int i = 0; i < num_tiles_in_slice - 1; ++i) + sh->delta_tile_id_minus1[i] = get_ue_golomb(&gb); + } + } + + sh->slice_type = get_ue_golomb(&gb); + + // @note + // If necessary, add the missing fields to the EVCParserSliceHeader structure + // and then extend parser implementation + + return sh; +} + +static int parse_nal_units(AVCodecParserContext *s, const uint8_t *bs, + int bs_size, AVCodecContext *avctx) +{ + EVCParserContext *ev = s->priv_data; + int nalu_type, nalu_size; + unsigned char *bits = (unsigned char *)bs; + int bits_size = bs_size; + + avctx->codec_id = AV_CODEC_ID_EVC; + s->pict_type = AV_PICTURE_TYPE_NONE; + s->picture_structure = AV_PICTURE_STRUCTURE_FRAME; + s->key_frame = -1; + + nalu_size = read_nal_unit_length(bits, bits_size, avctx); + if (nalu_size == 0) { + av_log(avctx, AV_LOG_ERROR, "Invalid NAL unit size: (%d)\n", nalu_size); + return -1; + } + + bits += EVC_NAL_UNIT_LENGTH_BYTE; + bits_size -= EVC_NAL_UNIT_LENGTH_BYTE; + + nalu_type = get_nalu_type(bits, bits_size, avctx); + + bits += EVC_NAL_HEADER_SIZE; + bits_size -= EVC_NAL_HEADER_SIZE; + + if (nalu_type == EVC_NUT_SPS) { // NAL Unit type: SPS (Sequence Parameter Set) + EVCParserSPS *sps; + + sps = parse_sps(bits, bits_size, ev); + if (!sps) { + av_log(avctx, AV_LOG_ERROR, "SPS parsing error\n"); + return -1; + } + + s->coded_width = sps->pic_width_in_luma_samples; + s->coded_height = sps->pic_height_in_luma_samples; + s->width = sps->pic_width_in_luma_samples; + s->height = sps->pic_height_in_luma_samples; + + if (sps->profile_idc == 1) avctx->profile = FF_PROFILE_EVC_MAIN; + else avctx->profile = FF_PROFILE_EVC_BASELINE; + + // Currently XEVD decoder supports ony YCBCR420_10LE chroma format for EVC stream + switch (sps->chroma_format_idc) { + case 0: /* YCBCR400_10LE */ + av_log(avctx, AV_LOG_ERROR, "YCBCR400_10LE: Not supported chroma format\n"); + s->format = AV_PIX_FMT_GRAY10LE; + return -1; + case 1: /* YCBCR420_10LE */ + s->format = AV_PIX_FMT_YUV420P10LE; + break; + case 2: /* YCBCR422_10LE */ + av_log(avctx, AV_LOG_ERROR, "YCBCR422_10LE: Not supported chroma format\n"); + s->format = AV_PIX_FMT_YUV422P10LE; + return -1; + case 3: /* YCBCR444_10LE */ + av_log(avctx, AV_LOG_ERROR, "YCBCR444_10LE: Not supported chroma format\n"); + s->format = AV_PIX_FMT_YUV444P10LE; + return -1; + default: + s->format = AV_PIX_FMT_NONE; + av_log(avctx, AV_LOG_ERROR, "Unknown supported chroma format\n"); + return -1; + } + + // @note + // The current implementation of parse_sps function doesn't handle VUI parameters parsing. + // If it will be needed, parse_sps function could be extended to handle VUI parameters parsing + // to initialize fields of the AVCodecContex i.e. color_primaries, color_trc,color_range + + } else if (nalu_type == EVC_NUT_PPS) { // NAL Unit type: PPS (Video Parameter Set) + EVCParserPPS *pps; + + pps = parse_pps(bits, bits_size, ev); + if (!pps) { + av_log(avctx, AV_LOG_ERROR, "PPS parsing error\n"); + return -1; + } + } else if (nalu_type == EVC_NUT_SEI) // NAL unit type: SEI (Supplemental Enhancement Information) + return 0; + else if (nalu_type == EVC_NUT_IDR || nalu_type == EVC_NUT_NONIDR) { // NAL Unit type: Coded slice of a IDR or non-IDR picture + EVCParserSliceHeader *sh; + + sh = parse_slice_header(bits, bits_size, ev); + if (!sh) { + av_log(avctx, AV_LOG_ERROR, "Slice header parsing error\n"); + return -1; + } + switch (sh->slice_type) { + case SLICE_TYPE_B: { + s->pict_type = AV_PICTURE_TYPE_B; + break; + } + case SLICE_TYPE_P: { + s->pict_type = AV_PICTURE_TYPE_P; + break; + } + case SLICE_TYPE_I: { + s->pict_type = AV_PICTURE_TYPE_I; + break; + } + default: { + s->pict_type = AV_PICTURE_TYPE_NONE; + } + } + s->key_frame = (nalu_type == EVC_NUT_IDR) ? 1 : 0; + } else { + av_log(avctx, AV_LOG_ERROR, "Invalid NAL unit type: %d\n", nalu_type); + return -1; + } + + return 0; +} + +/** + * Find the end of the current frame in the bitstream. + * @return the position of the first byte of the next frame, or END_NOT_FOUND + */ +static int evc_find_frame_end(AVCodecParserContext *s, const uint8_t *buf, + int buf_size, AVCodecContext *avctx) +{ + EVCParserContext *ev = s->priv_data; + + if (!ev->to_read) { + int nal_unit_size = 0; + int next = END_NOT_FOUND; + + // This is the case when buffer size is not enough for buffer to store NAL unit length + if (buf_size < EVC_NAL_UNIT_LENGTH_BYTE) { + ev->to_read = EVC_NAL_UNIT_LENGTH_BYTE; + ev->nal_length_size = buf_size; + ev->incomplete_nalu_prefix_read = 1; + + return END_NOT_FOUND; + } + + nal_unit_size = read_nal_unit_length(buf, buf_size, avctx); + ev->nal_length_size = EVC_NAL_UNIT_LENGTH_BYTE; + + next = nal_unit_size + EVC_NAL_UNIT_LENGTH_BYTE; + ev->to_read = next; + if (next < buf_size) + return next; + else + return END_NOT_FOUND; + } else if (ev->to_read > buf_size) + return END_NOT_FOUND; + else { + if (ev->incomplete_nalu_prefix_read == 1) { + EVCParserContext *ev = s->priv_data; + ParseContext *pc = &ev->pc; + uint8_t nalu_len[EVC_NAL_UNIT_LENGTH_BYTE] = {0}; + int nal_unit_size = 0; + + // 1. pc->buffer contains previously read bytes of NALU prefix + // 2. buf contains the rest of NAL unit prefix bytes + // + // ~~~~~~~ + // EXAMPLE + // ~~~~~~~ + // + // In the following example we assumed that the number of already read NAL Unit prefix bytes is equal 1 + // + // ---------- + // pc->buffer -> conatins already read bytes + // ---------- + // __ pc->index == 1 + // | + // V + // ------------------------------------------------------- + // | 0 | 1 | 2 | 3 | 4 | ... | N | + // ------------------------------------------------------- + // | 0x00 | 0xXX | 0xXX | 0xXX | 0xXX | ... | 0xXX | + // ------------------------------------------------------- + // + // ---------- + // buf -> contains newly read bytes + // ---------- + // ------------------------------------------------------- + // | 0 | 1 | 2 | 3 | 4 | ... | N | + // ------------------------------------------------------- + // | 0x00 | 0x00 | 0x3C | 0xXX | 0xXX | ... | 0xXX | + // ------------------------------------------------------- + // + for (int i = 0; i < EVC_NAL_UNIT_LENGTH_BYTE; i++) { + if (i < pc->index) + nalu_len[i] = pc->buffer[i]; + else + nalu_len[i] = buf[i - pc->index]; + } + + // ---------- + // nalu_len + // ---------- + // --------------------------------- + // | 0 | 1 | 2 | 3 | + // --------------------------------- + // | 0x00 | 0x00 | 0x00 | 0x3C | + // --------------------------------- + // | NALU LENGTH | + // --------------------------------- + // NAL Unit lenght = 60 (0x0000003C) + + nal_unit_size = read_nal_unit_length(nalu_len, EVC_NAL_UNIT_LENGTH_BYTE, avctx); + + ev->to_read = nal_unit_size + EVC_NAL_UNIT_LENGTH_BYTE - pc->index; + + ev->incomplete_nalu_prefix_read = 0; + + if (ev->to_read > buf_size) + return END_NOT_FOUND; + else + return ev->to_read; + } + return ev->to_read; + } + + return END_NOT_FOUND; +} + +static int evc_parser_init(AVCodecParserContext *s) +{ + EVCParserContext *ev = s->priv_data; + + ev->nal_length_size = EVC_NAL_UNIT_LENGTH_BYTE; + ev->incomplete_nalu_prefix_read = 0; + + return 0; +} + +static int evc_parse(AVCodecParserContext *s, AVCodecContext *avctx, + const uint8_t **poutbuf, int *poutbuf_size, + const uint8_t *buf, int buf_size) +{ + int next; + EVCParserContext *ev = s->priv_data; + ParseContext *pc = &ev->pc; + int is_dummy_buf = !buf_size; + const uint8_t *dummy_buf = buf; + + if (s->flags & PARSER_FLAG_COMPLETE_FRAMES) + next = buf_size; + else { + next = evc_find_frame_end(s, buf, buf_size, avctx); + if (ff_combine_frame(pc, next, &buf, &buf_size) < 0) { + *poutbuf = NULL; + *poutbuf_size = 0; + ev->to_read -= buf_size; + return buf_size; + } + } + + is_dummy_buf &= (dummy_buf == buf); + + if (!is_dummy_buf) + parse_nal_units(s, buf, buf_size, avctx); + + // poutbuf contains just one NAL unit + *poutbuf = buf; + *poutbuf_size = buf_size; + ev->to_read -= next; + + return next; +} + +const AVCodecParser ff_evc_parser = { + .codec_ids = { AV_CODEC_ID_EVC }, + .priv_data_size = sizeof(EVCParserContext), + .parser_init = evc_parser_init, + .parser_parse = evc_parse, + .parser_close = ff_parse_close, +}; diff --git a/libavcodec/libxevd.c b/libavcodec/libxevd.c new file mode 100644 index 0000000000..85f71affd3 --- /dev/null +++ b/libavcodec/libxevd.c @@ -0,0 +1,419 @@ +/* + * libxevd decoder + * EVC (MPEG-5 Essential Video Coding) decoding using XEVD MPEG-5 EVC decoder library + * + * Copyright (C) 2021 Dawid Kozinski <d.kozinski@samsung.com> + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include <float.h> +#include <stdlib.h> + +#include <xevd.h> + +#include "libavutil/internal.h" +#include "libavutil/common.h" +#include "libavutil/opt.h" +#include "libavutil/pixdesc.h" +#include "libavutil/pixfmt.h" +#include "libavutil/imgutils.h" +#include "libavutil/cpu.h" + +#include "avcodec.h" +#include "internal.h" +#include "packet_internal.h" +#include "codec_internal.h" + +#define XEVD_PARAM_BAD_NAME -1 +#define XEVD_PARAM_BAD_VALUE -2 + +#define EVC_NAL_HEADER_SIZE 2 /* byte */ + +/** + * The structure stores all the states associated with the instance of Xeve MPEG-5 EVC decoder + */ +typedef struct XevdContext { + const AVClass *class; + + XEVD id; // XEVD instance identifier @see xevd.h + XEVD_CDSC cdsc; // decoding parameters @see xevd.h +} XevdContext; + +/** + * The function populates the XEVD_CDSC structure. + * XEVD_CDSC contains all decoder parameters that should be initialized before its use. + * + * @param[in] avctx codec context + * @param[out] cdsc contains all decoder parameters that should be initialized before its use + * + */ +static void get_conf(AVCodecContext *avctx, XEVD_CDSC *cdsc) +{ + int cpu_count = av_cpu_count(); + + /* clear XEVS_CDSC structure */ + memset(cdsc, 0, sizeof(XEVD_CDSC)); + + /* init XEVD_CDSC */ + if (avctx->thread_count <= 0) + cdsc->threads = (cpu_count < XEVD_MAX_TASK_CNT) ? cpu_count : XEVD_MAX_TASK_CNT; + else if (avctx->thread_count > XEVD_MAX_TASK_CNT) + cdsc->threads = XEVD_MAX_TASK_CNT; + else + cdsc->threads = avctx->thread_count; +} + +/** + * Read NAL unit length + * @param bs input data (bitstream) + * @return the length of NAL unit on success, 0 value on failure + */ +static uint32_t read_nal_unit_length(const uint8_t *bs, int bs_size, AVCodecContext *avctx) +{ + uint32_t len = 0; + XEVD_INFO info; + int ret; + + if (bs_size == XEVD_NAL_UNIT_LENGTH_BYTE) { + ret = xevd_info((void *)bs, XEVD_NAL_UNIT_LENGTH_BYTE, 1, &info); + if (XEVD_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Cannot get bitstream information\n"); + return 0; + } + len = info.nalu_len; + if (len == 0) { + av_log(avctx, AV_LOG_ERROR, "Invalid bitstream size! [%d]\n", bs_size); + return 0; + } + } + + return len; +} + +/** + * @param[in] xectx the structure that stores all the state associated with the instance of Xeve MPEG-5 EVC decoder + * @param[out] avctx codec context + * @return 0 on success, negative value on failure + */ +static int export_stream_params(const XevdContext *xectx, AVCodecContext *avctx) +{ + int ret; + int size; + int color_space; + + avctx->pix_fmt = AV_PIX_FMT_YUV420P10; + + size = 4; + ret = xevd_config(xectx->id, XEVD_CFG_GET_CODED_WIDTH, &avctx->coded_width, &size); + if (XEVD_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to get coded_width\n"); + return AVERROR_EXTERNAL; + } + + ret = xevd_config(xectx->id, XEVD_CFG_GET_CODED_HEIGHT, &avctx->coded_height, &size); + if (XEVD_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to get coded_height\n"); + return AVERROR_EXTERNAL; + } + + ret = xevd_config(xectx->id, XEVD_CFG_GET_WIDTH, &avctx->width, &size); + if (XEVD_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to get width\n"); + return AVERROR_EXTERNAL; + } + + ret = xevd_config(xectx->id, XEVD_CFG_GET_HEIGHT, &avctx->height, &size); + if (XEVD_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to get height\n"); + return AVERROR_EXTERNAL; + } + + ret = xevd_config(xectx->id, XEVD_CFG_GET_COLOR_SPACE, &color_space, &size); + if (XEVD_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to get color_space\n"); + return AVERROR_EXTERNAL; + } + switch(color_space) { + case XEVD_CS_YCBCR400_10LE: + avctx->pix_fmt = AV_PIX_FMT_GRAY10LE; + break; + case XEVD_CS_YCBCR420_10LE: + avctx->pix_fmt = AV_PIX_FMT_YUV420P10LE; + break; + case XEVD_CS_YCBCR422_10LE: + avctx->pix_fmt = AV_PIX_FMT_YUV422P10LE; + break; + case XEVD_CS_YCBCR444_10LE: + avctx->pix_fmt = AV_PIX_FMT_YUV444P10LE; + break; + default: + av_log(avctx, AV_LOG_ERROR, "Unknown color space\n"); + avctx->pix_fmt = AV_PIX_FMT_NONE; + return AVERROR_INVALIDDATA; + } + + // the function returns sps->num_reorder_pics + ret = xevd_config(xectx->id, XEVD_CFG_GET_MAX_CODING_DELAY, &avctx->max_b_frames, &size); + if (XEVD_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to get max_coding_delay\n"); + return AVERROR_EXTERNAL; + } + + avctx->has_b_frames = (avctx->max_b_frames) ? 1 : 0; + + avctx->color_primaries = AVCOL_PRI_UNSPECIFIED; + avctx->color_trc = AVCOL_TRC_UNSPECIFIED; + avctx->colorspace = AVCOL_SPC_UNSPECIFIED; + + return 0; +} + +/** + * @brief Copy image in imgb to frame. + * + * @param avctx codec context + * @param[in] imgb + * @param[out] frame + * @return 0 on success, negative value on failure + */ +static int libxevd_image_copy(struct AVCodecContext *avctx, XEVD_IMGB *imgb, struct AVFrame *frame) +{ + int ret; + if (imgb->cs != XEVD_CS_YCBCR420_10LE) { + av_log(avctx, AV_LOG_ERROR, "Not supported pixel format: %s\n", av_get_pix_fmt_name(avctx->pix_fmt)); + return AVERROR_INVALIDDATA; + } + + if (imgb->w[0] != avctx->width || imgb->h[0] != avctx->height) { // stream resolution changed + if (ff_set_dimensions(avctx, imgb->w[0], imgb->h[0]) < 0) { + av_log(avctx, AV_LOG_ERROR, "Cannot set new dimension\n"); + return AVERROR_INVALIDDATA; + } + } + + if (ret = ff_get_buffer(avctx, frame, 0) < 0) { + return ret; + } + + av_image_copy(frame->data, frame->linesize, (const uint8_t **)imgb->a, + imgb->s, avctx->pix_fmt, + imgb->w[0], imgb->h[0]); + + return 0; +} + +/** + * Initialize decoder + * Create a decoder instance and allocate all the needed resources + * + * @param avctx codec context + * @return 0 on success, negative error code on failure + */ +static av_cold int libxevd_init(AVCodecContext *avctx) +{ + XevdContext *xectx = avctx->priv_data; + XEVD_CDSC *cdsc = &(xectx->cdsc); + + /* read configurations and set values for created descriptor (XEVD_CDSC) */ + get_conf(avctx, cdsc); + + /* create decoder */ + xectx->id = xevd_create(&(xectx->cdsc), NULL); + if (xectx->id == NULL) { + av_log(avctx, AV_LOG_ERROR, "Cannot create XEVD encoder\n"); + return AVERROR_EXTERNAL; + } + + return 0; +} + +/** + * Decode picture + * + * @param avctx codec context + * @param[out] frame decoded frame + * @param[out] got_frame_ptr decoder sets to 0 or 1 to indicate that a + * non-empty frame or subtitle was returned in + * outdata. + * @param[in] avpkt AVPacket containing encoded data to be decoded + * + * @return amount of bytes read from the packet on success, negative error + * code on failure + */ +static int libxevd_decode(struct AVCodecContext *avctx, struct AVFrame *frame, int *got_frame_ptr, AVPacket *avpkt) +{ + XevdContext *xectx = NULL; + XEVD_IMGB *imgb = NULL; + XEVD_STAT stat; + XEVD_BITB bitb; + int xevd_ret, nalu_size, bs_read_pos; + int ret = 0; + + xectx = avctx->priv_data; + + if (avpkt->size > 0) { + bs_read_pos = 0; + imgb = NULL; + while(avpkt->size > (bs_read_pos + XEVD_NAL_UNIT_LENGTH_BYTE)) { + memset(&stat, 0, sizeof(XEVD_STAT)); + memset(&bitb, 0, sizeof(XEVD_BITB)); + + nalu_size = read_nal_unit_length(avpkt->data + bs_read_pos, XEVD_NAL_UNIT_LENGTH_BYTE, avctx); + if (nalu_size == 0) { + av_log(avctx, AV_LOG_ERROR, "Invalid bitstream\n"); + ret = AVERROR_INVALIDDATA; + goto ERR; + } + bs_read_pos += XEVD_NAL_UNIT_LENGTH_BYTE; + + bitb.addr = avpkt->data + bs_read_pos; + bitb.ssize = nalu_size; + + /* main decoding block */ + xevd_ret = xevd_decode(xectx->id, &bitb, &stat); + if (XEVD_FAILED(xevd_ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to decode bitstream\n"); + ret = AVERROR_EXTERNAL; + goto ERR; + } + + bs_read_pos += nalu_size; + + if (stat.nalu_type == XEVD_NUT_SPS) { // EVC stream parameters changed + if ((ret = export_stream_params(xectx, avctx)) != 0) + goto ERR; + } + + if (stat.read != nalu_size) + av_log(avctx, AV_LOG_INFO, "Different reading of bitstream (in:%d, read:%d)\n,", nalu_size, stat.read); + if (stat.fnum >= 0) { + // already has a decoded image + if (imgb) { + // xevd_pull uses pool of objects of type XEVD_IMGB. + // The pool size is equal MAX_PB_SIZE (26), so release object when it is no more needed + imgb->release(imgb); + imgb = NULL; + } + xevd_ret = xevd_pull(xectx->id, &imgb); + if (XEVD_FAILED(xevd_ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to pull the decoded image (xevd error code: %d, frame#=%d)\n", xevd_ret, stat.fnum); + ret = AVERROR_EXTERNAL; + goto ERR; + } else if (xevd_ret == XEVD_OK_FRM_DELAYED) { + if (imgb) { + // xevd_pull uses pool of objects of type XEVD_IMGB. + // The pool size is equal MAX_PB_SIZE (26), so release object when it is no more needed + imgb->release(imgb); + imgb = NULL; + } + } + if (imgb) { + int ret = libxevd_image_copy(avctx, imgb, frame); + if(ret < 0) { + goto ERR; + } + + frame->pts = avpkt->pts; + *got_frame_ptr = 1; + + // xevd_pull uses pool of objects of type XEVD_IMGB. + // The pool size is equal MAX_PB_SIZE (26), so release object when it is no more needed + imgb->release(imgb); + imgb = NULL; + } else + *got_frame_ptr = 0; + } + } + } else { // bumping + xevd_ret = xevd_pull(xectx->id, &(imgb)); + if (xevd_ret == XEVD_ERR_UNEXPECTED) { // bumping process completed + *got_frame_ptr = 0; + return 0; + } else if (XEVD_FAILED(xevd_ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to pull the decoded image (xevd error code: %d)\n", xevd_ret); + ret = AVERROR_EXTERNAL; + goto ERR; + } + if (imgb) { + int ret = libxevd_image_copy(avctx, imgb, frame); + if(ret < 0) { + goto ERR; + } + + frame->pts = avpkt->pts; + *got_frame_ptr = 1; + + // xevd_pull uses pool of objects of type XEVD_IMGB. + // The pool size is equal MAX_PB_SIZE (26), so release object when it is no more needed + imgb->release(imgb); + imgb = NULL; + } else + *got_frame_ptr = 0; + } + + return avpkt->size; + +ERR: + if (imgb) { + imgb->release(imgb); + imgb = NULL; + } + *got_frame_ptr = 0; + + return ret; +} + +/** + * Destroy decoder + * + * @param avctx codec context + * @return 0 on success + */ +static av_cold int libxevd_close(AVCodecContext *avctx) +{ + XevdContext *xectx = avctx->priv_data; + if (xectx->id) { + xevd_delete(xectx->id); + xectx->id = NULL; + } + + return 0; +} + +#define OFFSET(x) offsetof(XevdContext, x) +#define VD AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_DECODING_PARAM + +static const AVClass libxevd_class = { + .class_name = "libxevd", + .item_name = av_default_item_name, + .version = LIBAVUTIL_VERSION_INT, +}; + +const FFCodec ff_libxevd_decoder = { + .p.name = "evc", + .p.long_name = NULL_IF_CONFIG_SMALL("EVC / MPEG-5 Essential Video Coding (EVC)"), + .p.type = AVMEDIA_TYPE_VIDEO, + .p.id = AV_CODEC_ID_EVC, + .init = libxevd_init, + FF_CODEC_DECODE_CB(libxevd_decode), + .close = libxevd_close, + .priv_data_size = sizeof(XevdContext), + .p.priv_class = &libxevd_class, + .p.capabilities = AV_CODEC_CAP_DELAY | AV_CODEC_CAP_OTHER_THREADS | AV_CODEC_CAP_AVOID_PROBING | AV_CODEC_CAP_DR1, + .p.wrapper_name = "libxevd", +}; diff --git a/libavcodec/libxeve.c b/libavcodec/libxeve.c new file mode 100644 index 0000000000..4808e6d853 --- /dev/null +++ b/libavcodec/libxeve.c @@ -0,0 +1,597 @@ +/* + * libxeve encoder + * EVC (MPEG-5 Essential Video Coding) encoding using XEVE MPEG-5 EVC encoder library + * + * Copyright (C) 2021 Dawid Kozinski <d.kozinski@samsung.com> + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include <float.h> +#include <stdlib.h> + +#include <xeve.h> + +#include "libavutil/internal.h" +#include "libavutil/common.h" +#include "libavutil/opt.h" +#include "libavutil/pixdesc.h" +#include "libavutil/pixfmt.h" +#include "libavutil/time.h" +#include "libavutil/cpu.h" +#include "libavutil/avstring.h" + +#include "avcodec.h" +#include "internal.h" +#include "packet_internal.h" +#include "codec_internal.h" +#include "encode.h" + +#define MAX_BS_BUF (16*1024*1024) + +/** + * Error codes + */ +#define XEVE_PARAM_BAD_NAME -100 +#define XEVE_PARAM_BAD_VALUE -200 + +/** + * Encoder states + * + * STATE_ENCODING - the encoder receives and processes input frames + * STATE_BUMPING - there are no more input frames, however the encoder still processes previously received data + */ +typedef enum State { + STATE_ENCODING, + STATE_BUMPING, +} State; + +/** + * The structure stores all the states associated with the instance of Xeve MPEG-5 EVC encoder + */ +typedef struct XeveContext { + const AVClass *class; + + XEVE id; // XEVE instance identifier + XEVE_CDSC cdsc; // coding parameters i.e profile, width & height of input frame, num of therads, frame rate ... + XEVE_BITB bitb; // bitstream buffer (output) + XEVE_STAT stat; // encoding status (output) + XEVE_IMGB imgb; // image buffer (input) + + State state; // encoder state (skipping, encoding, bumping) + + int profile_id; // encoder profile (main, baseline) + int preset_id; // preset of xeve ( fast, medium, slow, placebo) + int tune_id; // tune of xeve (psnr, zerolatency) + + // variables for rate control modes + int rc_mode; // Rate control mode [ 0(CQP) / 1(ABR) / 2(CRF) ] + int qp; // quantization parameter (QP) [0,51] + int crf; // constant rate factor (CRF) [10,49] + + int hash; // embed picture signature (HASH) for conformance checking in decoding + int sei_info; // embed Supplemental enhancement information while encoding + + int color_format; // input data color format: currently only XEVE_CF_YCBCR420 is supported +} XeveContext; + +/** + * Convert FFmpeg pixel format (AVPixelFormat) to XEVE pre-defined color format + * + * @param[in] av_pix_fmt pixel format (@see https://ffmpeg.org/doxygen/trunk/pixfmt_8h.html#a9a8e335cf3be472042bc9f0cf80cd4c5) + * @param[out] xeve_col_fmt XEVE pre-defined color format (@see xeve.h) + * + * @return 0 on success, negative value on failure + */ +static int libxeve_color_fmt(enum AVPixelFormat av_pix_fmt, int *xeve_col_fmt) +{ + switch (av_pix_fmt) { + case AV_PIX_FMT_YUV420P: + *xeve_col_fmt = XEVE_CF_YCBCR420; + break; + case AV_PIX_FMT_YUV420P10: + *xeve_col_fmt = XEVE_CF_YCBCR420; + break; + default: + *xeve_col_fmt = XEVE_CF_UNKNOWN; + return AVERROR_INVALIDDATA; + } + + return 0; +} + +/** + * Convert FFmpeg pixel format (AVPixelFormat) into XEVE pre-defined color space + * + * @param[in] px_fmt pixel format (@see https://ffmpeg.org/doxygen/trunk/pixfmt_8h.html#a9a8e335cf3be472042bc9f0cf80cd4c5) + * + * @return XEVE pre-defined color space (@see xeve.h) on success, XEVE_CF_UNKNOWN on failure + */ +static int libxeve_color_space(enum AVPixelFormat av_pix_fmt) +{ + /* color space of input image */ + int cs = XEVE_CF_UNKNOWN; + + switch (av_pix_fmt) { + case AV_PIX_FMT_YUV420P: + cs = XEVE_CS_YCBCR420; + break; + case AV_PIX_FMT_YUV420P10: +#if AV_HAVE_BIGENDIAN + cs = XEVE_CS_SET(XEVE_CF_YCBCR420, 10, 1); +#else + cs = XEVE_CS_YCBCR420_10LE; +#endif + + break; + default: + cs = XEVE_CF_UNKNOWN; + break; + } + + return cs; +} + +/** + * The function returns a pointer to the object of the XEVE_CDSC type. + * XEVE_CDSC contains all encoder parameters that should be initialized before the encoder is used. + * + * The field values of the XEVE_CDSC structure are populated based on: + * - the corresponding field values of the AvCodecConetxt structure, + * - the xeve encoder specific option values, + * (the full list of options available for xeve encoder is displayed after executing the command ./ffmpeg --help encoder = libxeve) + * - and the xeve encoder options specified as a list of key-value pairs following the xeve-params option + * + * The order of processing input data and populating the XEVE_CDSC structure + * 1) first, the fields of the AVCodecContext structure corresponding to the provided input options are processed, + * (i.e -pix_fmt yuv420p -s:v 1920x1080 -r 30 -profile:v 0) + * 2) then xeve-specific options added as AVOption to the xeve AVCodec implementation + * (i.e -preset 0) + * + * Keep in mind that, there are options that can be set in different ways. + * In this case, please follow the above-mentioned order of processing. + * The most recent assignments overwrite the previous values. + * + * @param[in] avctx codec context (AVCodecContext) + * @param[out] cdsc contains all Xeve MPEG-5 EVC encoder encoder parameters that should be initialized before the encoder is use + * + * @return 0 on success, negative error code on failure + */ +static int get_conf(AVCodecContext *avctx, XEVE_CDSC *cdsc) +{ + XeveContext *xectx = NULL; + int ret; + + xectx = avctx->priv_data; + + /* initialize xeve_param struct with default values */ + ret = xeve_param_default(&cdsc->param); + if (XEVE_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Cannot set_default parameter\n"); + return AVERROR_EXTERNAL; + } + + /* read options from AVCodecContext */ + if (avctx->width > 0) + cdsc->param.w = avctx->width; + + if (avctx->height > 0) + cdsc->param.h = avctx->height; + + if (avctx->framerate.num > 0) { + // fps can be float number, but xeve API doesn't support it + cdsc->param.fps = (int)(((float)avctx->framerate.num / avctx->framerate.den) + 0.5); + } + + if (avctx->gop_size >= 0) // GOP size (key-frame interval, I-picture period) + cdsc->param.keyint = avctx->gop_size; // 0: only one I-frame at the first time; 1: every frame is coded in I-frame + + if (avctx->max_b_frames == 0 || avctx->max_b_frames == 1 || avctx->max_b_frames == 3 || + avctx->max_b_frames == 7 || avctx->max_b_frames == 15) // number of b-frames + cdsc->param.bframes = avctx->max_b_frames; + else { + av_log(avctx, AV_LOG_ERROR, "Incorrect value for maximum number of B frames: (%d) \n" + "Acceptable values for bf option (maximum number of B frames) are 0,1,3,7 or 15\n", avctx->max_b_frames); + return AVERROR_INVALIDDATA; + } + + if (avctx->level >= 0) + cdsc->param.level_idc = avctx->level; + + if (avctx->rc_buffer_size) // VBV buf size + cdsc->param.vbv_bufsize = (int)(avctx->rc_buffer_size / 1000); + + cdsc->param.rc_type = xectx->rc_mode; + + if (xectx->rc_mode == XEVE_RC_CQP) { + cdsc->param.qp = xectx->qp; + } else if (xectx->rc_mode == XEVE_RC_ABR) { + if (avctx->bit_rate / 1000 > INT_MAX || avctx->rc_max_rate / 1000 > INT_MAX) { + av_log(avctx, AV_LOG_ERROR, "Not supported bitrate bit_rate and rc_max_rate > %d000\n", INT_MAX); + return AVERROR_INVALIDDATA; + } + cdsc->param.bitrate = (int)(avctx->bit_rate / 1000); + } else if (xectx->rc_mode == XEVE_RC_CRF) { + cdsc->param.crf = xectx->crf; + } else { + av_log(avctx, AV_LOG_ERROR, "Not supported rate control type: %d\n", xectx->rc_mode); + return AVERROR_INVALIDDATA; + } + + if (avctx->thread_count <= 0) { + int cpu_count = av_cpu_count(); + cdsc->param.threads = (cpu_count < XEVE_MAX_THREADS) ? cpu_count : XEVE_MAX_THREADS; + } else if (avctx->thread_count > XEVE_MAX_THREADS) + cdsc->param.threads = XEVE_MAX_THREADS; + else + cdsc->param.threads = avctx->thread_count; + + + libxeve_color_fmt(avctx->pix_fmt, &xectx->color_format); + + cdsc->param.cs = XEVE_CS_SET(xectx->color_format, cdsc->param.codec_bit_depth, AV_HAVE_BIGENDIAN); + + cdsc->max_bs_buf_size = MAX_BS_BUF; + + ret = xeve_param_ppt(&cdsc->param, xectx->profile_id, xectx->preset_id, xectx->tune_id); + if (XEVE_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Cannot set profile(%d), preset(%d), tune(%d)\n", xectx->profile_id, xectx->preset_id, xectx->tune_id); + return AVERROR_EXTERNAL; + } + + return 0; +} + +/** + * Set XEVE_CFG_SET_USE_PIC_SIGNATURE for encoder + * + * @param[in] logger context + * @param[in] id XEVE encodec instance identifier + * @param[in] ctx the structure stores all the states associated with the instance of Xeve MPEG-5 EVC encoder + * + * @return 0 on success, negative error code on failure + */ +static int set_extra_config(AVCodecContext *avctx, XEVE id, XeveContext *ctx) +{ + int ret, size; + size = 4; + + // embed SEI messages identifying encoder parameters and command line arguments + // - 0: off\n" + // - 1: emit sei info" + // + // SEI - Supplemental enhancement information contains information + // that is not necessary to decode the samples of coded pictures from VCL NAL units. + // Some SEI message information is required to check bitstream conformance + // and for output timing decoder conformance. + // @see ISO_IEC_23094-1_2020 7.4.3.5 + // @see ISO_IEC_23094-1_2020 Annex D + ret = xeve_config(id, XEVE_CFG_SET_SEI_CMD, &ctx->sei_info, &size); // sei_cmd_info + if (XEVE_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to set config for sei command info messages\n"); + return AVERROR_EXTERNAL; + } + + ret = xeve_config(id, XEVE_CFG_SET_USE_PIC_SIGNATURE, &ctx->hash, &size); + if (XEVE_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to set config for picture signature\n"); + return AVERROR_EXTERNAL; + } + + return 0; +} + +/** + * @brief Switch encoder to bumping mode + * + * @param id XEVE encodec instance identifier + * @return 0 on success, negative error code on failure + */ +static int setup_bumping(XEVE id) +{ + int val = 1; + int size = sizeof(int); + if (XEVE_FAILED(xeve_config(id, XEVE_CFG_SET_FORCE_OUT, (void *)(&val), &size))) + return AVERROR_EXTERNAL; + + return 0; +} + +/** + * @brief Initialize eXtra-fast Essential Video Encoder codec + * Create an encoder instance and allocate all the needed resources + * + * @param avctx codec context + * @return 0 on success, negative error code on failure + */ +static av_cold int libxeve_init(AVCodecContext *avctx) +{ + XeveContext *xectx = avctx->priv_data; + unsigned char *bs_buf = NULL; + int i; + int shift_h = 0; + int shift_v = 0; + int width_chroma = 0; + int height_chroma = 0; + XEVE_IMGB *imgb = NULL; + int ret = 0; + + XEVE_CDSC *cdsc = &(xectx->cdsc); + + /* allocate bitstream buffer */ + bs_buf = av_malloc(MAX_BS_BUF); + if (bs_buf == NULL) { + av_log(avctx, AV_LOG_ERROR, "Cannot allocate bitstream buffer\n"); + return AVERROR(ENOMEM); + } + xectx->bitb.addr = bs_buf; + xectx->bitb.bsize = MAX_BS_BUF; + + /* read configurations and set values for created descriptor (XEVE_CDSC) */ + if ((ret = get_conf(avctx, cdsc)) != 0) { + av_log(avctx, AV_LOG_ERROR, "Cannot get configuration\n"); + return AVERROR(EINVAL); + } + + if ((ret = xeve_param_check(&cdsc->param)) != 0) { + av_log(avctx, AV_LOG_ERROR, "Invalid configuration\n"); + return AVERROR(EINVAL); + } + + /* create encoder */ + xectx->id = xeve_create(cdsc, NULL); + if (xectx->id == NULL) { + av_log(avctx, AV_LOG_ERROR, "Cannot create XEVE encoder\n"); + return AVERROR_EXTERNAL; + } + + if ((ret = set_extra_config(avctx, xectx->id, xectx)) != 0) { + av_log(avctx, AV_LOG_ERROR, "Cannot set extra configuration\n"); + return AVERROR(EINVAL); + } + + if ((ret = av_pix_fmt_get_chroma_sub_sample(avctx->pix_fmt, &shift_h, &shift_v)) != 0) { + av_log(avctx, AV_LOG_ERROR, "Failed to get chroma shift\n"); + return AVERROR(EINVAL); + } + + // Chroma subsampling + // + // YUV format explanation + // shift_h == 1 && shift_v == 1 : YUV420 + // shift_h == 1 && shift_v == 0 : YUV422 + // shift_h == 0 && shift_v == 0 : YUV444 + // + width_chroma = AV_CEIL_RSHIFT(avctx->width, shift_h); + height_chroma = AV_CEIL_RSHIFT(avctx->height, shift_v); + + /* set default values for input image buffer */ + imgb = &xectx->imgb; + imgb->cs = libxeve_color_space(avctx->pix_fmt); + imgb->np = 3; /* only for yuv420p, yuv420ple */ + + for (i = 0; i < imgb->np; i++) + imgb->x[i] = imgb->y[i] = 0; + + imgb->w[0] = imgb->aw[0] = avctx->width; // width luma + imgb->w[1] = imgb->w[2] = imgb->aw[1] = imgb->aw[2] = width_chroma; + imgb->h[0] = imgb->ah[0] = avctx->height; // height luma + imgb->h[1] = imgb->h[2] = imgb->ah[1] = imgb->ah[2] = height_chroma; + + xectx->state = STATE_ENCODING; + + return 0; +} + +/** + * Encode raw data frame into EVC packet + * + * @param[in] avctx codec context + * @param[out] pkt output AVPacket containing encoded data + * @param[in] frame AVFrame containing the raw data to be encoded + * @param[out] got_packet encoder sets to 0 or 1 to indicate that a + * non-empty packet was returned in pkt + * + * @return 0 on success, negative error code on failure + */ +static int libxeve_encode(AVCodecContext *avctx, AVPacket *avpkt, + const AVFrame *frame, int *got_packet) +{ + XeveContext *xectx = avctx->priv_data; + int ret = -1; + + // No more input frames are available but encoder still can have some data in its internal buffer to process + // and some frames to dump. + if (xectx->state == STATE_ENCODING && frame == NULL) { + if (setup_bumping(xectx->id) == 0) + xectx->state = STATE_BUMPING; // Entering bumping process + else { + av_log(avctx, AV_LOG_ERROR, "Failed to setup bumping\n"); + return 0; + } + } + + if (xectx->state == STATE_ENCODING) { + int i; + XEVE_IMGB *imgb = NULL; + + imgb = &xectx->imgb; + + for (i = 0; i < imgb->np; i++) { + imgb->a[i] = frame->data[i]; + imgb->s[i] = frame->linesize[i]; + } + + imgb->ts[0] = frame->pts; + imgb->ts[1] = 0; + + /* push image to encoder */ + ret = xeve_push(xectx->id, imgb); + if (XEVE_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "xeve_push() failed\n"); + return AVERROR_EXTERNAL; + } + } + if (xectx->state == STATE_ENCODING || xectx->state == STATE_BUMPING) { + /* encoding */ + ret = xeve_encode(xectx->id, &(xectx->bitb), &(xectx->stat)); + if (XEVE_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "xeve_encode() failed\n"); + return AVERROR_EXTERNAL; + } + + /* store bitstream */ + if (ret == XEVE_OK_OUT_NOT_AVAILABLE) { // Return OK but picture is not available yet + *got_packet = 0; + return 0; + } else if (ret == XEVE_OK) { + int av_pic_type; + + if (xectx->stat.write > 0) { + + ret = ff_get_encode_buffer(avctx, avpkt, xectx->stat.write, 0); + if (ret < 0) + return ret; + + memcpy(avpkt->data, xectx->bitb.addr, xectx->stat.write); + + avpkt->pts = xectx->bitb.ts[0]; + avpkt->dts = xectx->bitb.ts[1]; + + switch(xectx->stat.stype) { + case XEVE_ST_I: + av_pic_type = AV_PICTURE_TYPE_I; + avpkt->flags |= AV_PKT_FLAG_KEY; + break; + case XEVE_ST_P: + av_pic_type = AV_PICTURE_TYPE_P; + break; + case XEVE_ST_B: + av_pic_type = AV_PICTURE_TYPE_B; + break; + case XEVE_ST_UNKNOWN: + av_log(avctx, AV_LOG_ERROR, "Unknown slice type\n"); + return AVERROR_INVALIDDATA; + } + + ff_side_data_set_encoder_stats(avpkt, xectx->stat.qp * FF_QP2LAMBDA, NULL, 0, av_pic_type); + + *got_packet = 1; + } + } else if (ret == XEVE_OK_NO_MORE_FRM) { + // Return OK but no more frames + return 0; + } else { + av_log(avctx, AV_LOG_ERROR, "Invalid return value: %d\n", ret); + return AVERROR_EXTERNAL; + } + } else { + av_log(avctx, AV_LOG_ERROR, "Udefined encoder state\n"); + return AVERROR_INVALIDDATA; + } + + return 0; +} + +/** + * Destroy the encoder and release all the allocated resources + * + * @param avctx codec context + * @return 0 on success, negative error code on failure + */ +static av_cold int libxeve_close(AVCodecContext *avctx) +{ + XeveContext *xectx = avctx->priv_data; + + xeve_delete(xectx->id); + + av_free(xectx->bitb.addr); /* release bitstream buffer */ + + return 0; +} + +#define OFFSET(x) offsetof(XeveContext, x) +#define VE AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_ENCODING_PARAM + +static const enum AVPixelFormat supported_pixel_formats[] = { + AV_PIX_FMT_YUV420P, + AV_PIX_FMT_YUV420P10, + AV_PIX_FMT_NONE +}; + +// Example of using: ./ffmpeg -xeve-params "m=2:q=17" +// Consider using following options (./ffmpeg --help encoder=libxeve) +// +static const AVOption libxeve_options[] = { + { "preset", "Encoding preset for setting encoding speed", OFFSET(preset_id), AV_OPT_TYPE_INT, { .i64 = XEVE_PRESET_MEDIUM }, XEVE_PRESET_DEFAULT, XEVE_PRESET_PLACEBO, VE, "preset" }, + { "default", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_PRESET_DEFAULT }, INT_MIN, INT_MAX, VE, "preset" }, + { "fast", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_PRESET_FAST }, INT_MIN, INT_MAX, VE, "preset" }, + { "medium", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_PRESET_MEDIUM }, INT_MIN, INT_MAX, VE, "preset" }, + { "slow", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_PRESET_SLOW }, INT_MIN, INT_MAX, VE, "preset" }, + { "placebo", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_PRESET_PLACEBO }, INT_MIN, INT_MAX, VE, "preset" }, + { "tune", "Tuning parameter for special purpose operation", OFFSET(tune_id), AV_OPT_TYPE_INT, { .i64 = XEVE_TUNE_NONE }, XEVE_TUNE_NONE, XEVE_TUNE_PSNR, VE, "tune"}, + { "none", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_TUNE_NONE }, INT_MIN, INT_MAX, VE, "tune" }, + { "zerolatency", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_TUNE_ZEROLATENCY }, INT_MIN, INT_MAX, VE, "tune" }, + { "psnr", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_TUNE_PSNR }, INT_MIN, INT_MAX, VE, "tune" }, + { "profile", "Encoding profile", OFFSET(profile_id), AV_OPT_TYPE_INT, { .i64 = XEVE_PROFILE_BASELINE }, XEVE_PROFILE_BASELINE, XEVE_PROFILE_MAIN, VE, "profile" }, + { "baseline", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_PROFILE_BASELINE }, INT_MIN, INT_MAX, VE, "profile" }, + { "main", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_PROFILE_MAIN }, INT_MIN, INT_MAX, VE, "profile" }, + { "rc_mode", "Rate control mode", OFFSET(rc_mode), AV_OPT_TYPE_INT, { .i64 = XEVE_RC_CQP }, XEVE_RC_CQP, XEVE_RC_CRF, VE, "rc_mode" }, + { "CQP", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_RC_CQP }, INT_MIN, INT_MAX, VE, "rc_mode" }, + { "ABR", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_RC_ABR }, INT_MIN, INT_MAX, VE, "rc_mode" }, + { "CRF", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_RC_CRF }, INT_MIN, INT_MAX, VE, "rc_mode" }, + { "qp", "Quantization parameter value for CQP rate control mode", OFFSET(qp), AV_OPT_TYPE_INT, { .i64 = 32 }, 0, 51, VE }, + { "crf", "Constant rate factor value for CRF rate control mode", OFFSET(crf), AV_OPT_TYPE_INT, { .i64 = 32 }, 10, 49, VE }, + { "hash", "Embed picture signature (HASH) for conformance checking in decoding", OFFSET(hash), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, 1, VE }, + { "sei_info", "Embed SEI messages identifying encoder parameters and command line arguments", OFFSET(sei_info), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, 1, VE }, + { NULL } +}; + +static const AVClass libxeve_class = { + .class_name = "libxeve", + .item_name = av_default_item_name, + .option = libxeve_options, + .version = LIBAVUTIL_VERSION_INT, +}; + +/** + * libavcodec generic global options, which can be set on all the encoders and decoders + * @see https://www.ffmpeg.org/ffmpeg-codecs.html#Codec-Options + */ +static const FFCodecDefault libxeve_defaults[] = { + { "b", "0" }, // bitrate in terms of kilo-bits per second + { "g", "0" }, // gop_size (key-frame interval 0: only one I-frame at the first time; 1: every frame is coded in I-frame) + { "bf", "15"}, // maximum number of B frames (0: no B-frames, 1,3,7,15) + { "threads", "0"}, // number of threads to be used (0: automatically select the number of threads to set) + { NULL }, +}; + +const FFCodec ff_libxeve_encoder = { + .p.name = "libxeve", + .p.long_name = NULL_IF_CONFIG_SMALL("libxeve MPEG-5 EVC"), + .p.type = AVMEDIA_TYPE_VIDEO, + .p.id = AV_CODEC_ID_EVC, + .init = libxeve_init, + FF_CODEC_ENCODE_CB(libxeve_encode), + .close = libxeve_close, + .priv_data_size = sizeof(XeveContext), + .p.priv_class = &libxeve_class, + .defaults = libxeve_defaults, + .p.capabilities = FF_CODEC_CAP_INIT_CLEANUP | AV_CODEC_CAP_DELAY | AV_CODEC_CAP_AUTO_THREADS | AV_CODEC_CAP_DR1, + .p.wrapper_name = "libxeve", + .p.pix_fmts = supported_pixel_formats, +}; diff --git a/libavcodec/parsers.c b/libavcodec/parsers.c index a8d52af6cb..ca00c63176 100644 --- a/libavcodec/parsers.c +++ b/libavcodec/parsers.c @@ -41,6 +41,7 @@ extern const AVCodecParser ff_dvaudio_parser; extern const AVCodecParser ff_dvbsub_parser; extern const AVCodecParser ff_dvdsub_parser; extern const AVCodecParser ff_dvd_nav_parser; +extern const AVCodecParser ff_evc_parser; extern const AVCodecParser ff_flac_parser; extern const AVCodecParser ff_g723_1_parser; extern const AVCodecParser ff_g729_parser; diff --git a/libavcodec/profiles.c b/libavcodec/profiles.c index 7af7fbeb13..a31244e0db 100644 --- a/libavcodec/profiles.c +++ b/libavcodec/profiles.c @@ -181,4 +181,10 @@ const AVProfile ff_arib_caption_profiles[] = { { FF_PROFILE_UNKNOWN } }; +const AVProfile ff_evc_profiles[] = { + { FF_PROFILE_EVC_BASELINE, "Baseline" }, + { FF_PROFILE_EVC_MAIN, "Main" }, + { FF_PROFILE_UNKNOWN }, +}; + #endif /* !CONFIG_SMALL */ diff --git a/libavcodec/profiles.h b/libavcodec/profiles.h index 41a19aa9ad..cf92b5f126 100644 --- a/libavcodec/profiles.h +++ b/libavcodec/profiles.h @@ -72,5 +72,6 @@ extern const AVProfile ff_sbc_profiles[]; extern const AVProfile ff_prores_profiles[]; extern const AVProfile ff_mjpeg_profiles[]; extern const AVProfile ff_arib_caption_profiles[]; +extern const AVProfile ff_evc_profiles[]; #endif /* AVCODEC_PROFILES_H */ diff --git a/libavcodec/version.h b/libavcodec/version.h index e488eee355..2328be4b26 100644 --- a/libavcodec/version.h +++ b/libavcodec/version.h @@ -29,7 +29,7 @@ #include "version_major.h" -#define LIBAVCODEC_VERSION_MINOR 42 +#define LIBAVCODEC_VERSION_MINOR 43 #define LIBAVCODEC_VERSION_MICRO 100 #define LIBAVCODEC_VERSION_INT AV_VERSION_INT(LIBAVCODEC_VERSION_MAJOR, \ -- 2.17.1 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [FFmpeg-devel] [PATCH 1/2] Provided support for MPEG-5 EVC (Essential Video Coding) codec 2022-08-08 12:30 ` [FFmpeg-devel] [PATCH 1/2] Provided support for MPEG-5 EVC (Essential Video Coding) codec Dawid Kozinski @ 2022-08-11 7:44 ` Andreas Rheinhardt 2022-08-11 11:28 ` Dawid Kozinski/Robot SDK (PLT) /SRPOL/Staff Engineer/삼성전자 2022-08-18 9:00 ` Dawid Kozinski/Robot SDK (PLT) /SRPOL/Staff Engineer/삼성전자 0 siblings, 2 replies; 13+ messages in thread From: Andreas Rheinhardt @ 2022-08-11 7:44 UTC (permalink / raw) To: ffmpeg-devel Dawid Kozinski: > - Added new entry to codec IDs list > - Added new entry to the codec descriptor list > - Bumped libavcodec minor version > - Changes in Changelog and MAINTAINERS files > - Added xeve encoder wrapper > - Added xevd dencoder wrapper > - Added documentation for xeve and xevd wrappers > - Added parser for EVC format > - Changes in project configuration file and libavcodec Makefile > > Signed-off-by: Dawid Kozinski <d.kozinski@samsung.com> > --- > Changelog | 3 +- > MAINTAINERS | 2 + > configure | 8 + > doc/decoders.texi | 24 ++ > doc/encoders.texi | 92 +++++ > doc/general_contents.texi | 19 + > libavcodec/Makefile | 3 + > libavcodec/allcodecs.c | 2 + > libavcodec/avcodec.h | 3 + > libavcodec/codec_desc.c | 8 + > libavcodec/codec_id.h | 1 + > libavcodec/evc_parser.c | 758 ++++++++++++++++++++++++++++++++++++++ > libavcodec/libxevd.c | 419 +++++++++++++++++++++ > libavcodec/libxeve.c | 597 ++++++++++++++++++++++++++++++ > libavcodec/parsers.c | 1 + > libavcodec/profiles.c | 6 + > libavcodec/profiles.h | 1 + > libavcodec/version.h | 2 +- > 18 files changed, 1947 insertions(+), 2 deletions(-) > create mode 100644 libavcodec/evc_parser.c > create mode 100644 libavcodec/libxevd.c > create mode 100644 libavcodec/libxeve.c > > diff --git a/Changelog b/Changelog > index cc271c22bd..dbeb08c373 100644 > --- a/Changelog > +++ b/Changelog > @@ -82,7 +82,8 @@ version 5.0: > - VideoToolbox ProRes encoder > - anlmf audio filter > - IMF demuxer (experimental) > - > +- eXtra-fast Essential Video Encoder (XEVE) > +- eXtra-fast Essential Video Decoder (XEVD) > > version 4.4: > - AudioToolbox output device > diff --git a/MAINTAINERS b/MAINTAINERS > index 7ed15f96f6..7d4d543a13 100644 > --- a/MAINTAINERS > +++ b/MAINTAINERS > @@ -206,6 +206,7 @@ Codecs: > libvpx* James Zern > libxavs.c Stefan Gehrer > libxavs2.c Huiwen Ren > + libxev*.c, evc_parser.c Dawid Kozinski > libzvbi-teletextdec.c Marton Balint > lzo.h, lzo.c Reimar Doeffinger > mdec.c Michael Niedermayer > @@ -426,6 +427,7 @@ Muxers/Demuxers: > dv.c Roman Shaposhnik > electronicarts.c Peter Ross > epafdec.c Paul B Mahol > + evcdec.c Dawid Kozinski > ffm* Baptiste Coudurier > flic.c Mike Melanson > flvdec.c Michael Niedermayer > diff --git a/configure b/configure > index cbbb4dd9c8..e6582eae91 100755 > --- a/configure > +++ b/configure > @@ -291,6 +291,8 @@ External library support: > --enable-libwebp enable WebP encoding via libwebp [no] > --enable-libx264 enable H.264 encoding via x264 [no] > --enable-libx265 enable HEVC encoding via x265 [no] > + --enable-libxeve enable EVC encoding via libxeve [no] > + --enable-libxevd enable EVC decoding via libxevd [no] > --enable-libxavs enable AVS encoding via xavs [no] > --enable-libxavs2 enable AVS2 encoding via xavs2 [no] > --enable-libxcb enable X11 grabbing using XCB [autodetect] > @@ -1874,6 +1876,8 @@ EXTERNAL_LIBRARY_LIST=" > libvorbis > libvpx > libwebp > + libxevd > + libxeve > libxml2 > libzimg > libzmq > @@ -3395,6 +3399,8 @@ libx264rgb_encoder_select="libx264_encoder" > libx265_encoder_deps="libx265" > libxavs_encoder_deps="libxavs" > libxavs2_encoder_deps="libxavs2" > +libxevd_decoder_deps="libxevd" > +libxeve_encoder_deps="libxeve" > libxvid_encoder_deps="libxvid" > libzvbi_teletext_decoder_deps="libzvbi" > vapoursynth_demuxer_deps="vapoursynth" > @@ -6685,6 +6691,8 @@ enabled libx265 && require_pkg_config libx265 x265 x265.h x265_api_get > require_cpp_condition libx265 x265.h "X265_BUILD >= 70" > enabled libxavs && require libxavs "stdint.h xavs.h" xavs_encoder_encode "-lxavs $pthreads_extralibs $libm_extralibs" > enabled libxavs2 && require_pkg_config libxavs2 "xavs2 >= 1.3.0" "stdint.h xavs2.h" xavs2_api_get > +enabled libxevd && require_pkg_config libxevd "xevd >= 0.4.0" "xevd.h" xevd_decode > +enabled libxeve && require_pkg_config libxeve "xeve >= 0.4.0" "xeve.h" xeve_encode > enabled libxvid && require libxvid xvid.h xvid_global -lxvidcore > enabled libzimg && require_pkg_config libzimg "zimg >= 2.7.0" zimg.h zimg_get_api_version > enabled libzmq && require_pkg_config libzmq "libzmq >= 4.2.1" zmq.h zmq_ctx_new > diff --git a/doc/decoders.texi b/doc/decoders.texi > index e2fcbf5dc9..0bd9096114 100644 > --- a/doc/decoders.texi > +++ b/doc/decoders.texi > @@ -126,6 +126,30 @@ Set amount of frame threads to use during decoding. The default value is 0 (auto > > @end table > > +@section libxevd > + > +eXtra-fast Essential Video Decoder (XEVD) MPEG-5 EVC decoder wrapper. > + > +This decoder requires the presence of the libxevd headers and library > +during configuration. You need to explicitly configure the build with > +@option{--enable-libxevd}. > + > +The xevd project website is at @url{https://github.com/mpeg5/xevd}. > + > +@subsection Options > + > +The following options are supported by the libxevd wrapper. > +The xevd-equivalent options or values are listed in parentheses for easy migration. > + > +To get a more accurate and extensive documentation of the libxevd options, > +invoke the command @code{xevd_app --help} or consult the libxevd documentation. > + > +@table @option > +@item threads (@emph{threads}) > +Force to use a specific number of threads > + > +@end table > + > @section QSV Decoders > > The family of Intel QuickSync Video decoders (VC1, MPEG-2, H.264, HEVC, > diff --git a/doc/encoders.texi b/doc/encoders.texi > index 6d73f74196..76f55e6721 100644 > --- a/doc/encoders.texi > +++ b/doc/encoders.texi > @@ -2892,6 +2892,98 @@ ffmpeg -i input -c:v libxavs2 -xavs2-params RdoqLevel=0 output.avs2 > @end example > @end table > > +@section libxeve > + > +eXtra-fast Essential Video Encoder (XEVE) MPEG-5 EVC encoder wrapper. > +The xeve-equivalent options or values are listed in parentheses for easy migration. > + > +This encoder requires the presence of the libxeve headers and library > +during configuration. You need to explicitly configure the build with > +@option{--enable-libxeve}. > + > +@float NOTE > +Many libxeve encoder options are mapped to FFmpeg global codec options, > +while unique encoder options are provided through private options. > +Additionally the xeve-params private options allows one to pass a list > +of key=value tuples as accepted by the libxeve @code{parse_xeve_params} function. > +@end float > + > +The xeve project website is at @url{https://github.com/mpeg5/xeve}. > + > +@subsection Options > + > +The following options are supported by the libxeve wrapper. > +The xeve-equivalent options or values are listed in parentheses for easy migration. > + > +@float NOTE > +To reduce the duplication of documentation, only the private options > +and some others requiring special attention are documented here. For > +the documentation of the undocumented generic options, see > +@ref{codec-options,,the Codec Options chapter}. > +@end float > + > +@float NOTE > +To get a more accurate and extensive documentation of the libxeve options, > +invoke the command @code{xeve_app --help} or consult the libxeve documentation. > +@end float > + > +@table @option > +@item b (@emph{bitrate}) > +Set target video bitrate in bits/s. > +Note that FFmpeg's b option is expressed in bits/s, while xeve's bitrate is in kilobits/s. > + > +@item bf (@emph{bframes}) > +Set the maximum number of B frames (1,3,7,15). > + > +@item g (@emph{keyint}) > +Set the GOP size (I-picture period). > + > +@item preset (@emph{preset}) > +Set the xeve preset. > +Set the encoder preset value to determine encoding speed [fast, medium, slow, placebo] > + > +@item tune (@emph{tune}) > +Set the encoder tune parameter [psnr, zerolatency] > + > +@item profile (@emph{profile}) > +Set the encoder profile [0: baselie; 1: main] > + > +@item crf (@emph{crf}) > +Set the quality for constant quality mode. > +Constant rate factor <10..49> [default: 32] > + > +@item qp (@emph{qp}) > +Set constant quantization rate control method parameter. > +Quantization parameter qp <0..51> [default: 32] > + > +@item threads (@emph{threads}) > +Force to use a specific number of threads > + > +@item xeve-params > +Set xeve options using a list of @var{key}=@var{value} couples separated > +by ":". > + > +For example to specify libxeve encoding options with @option{-xeve-params}: > + > +@example > +ffmpeg -i input -c:v libxeve -xeve-params profile=main:tune=zerolatency:preset=fast:q=17:bframes=15:bitrate=100k output.mp4 > +@end example > +@end table > + > +@subsubsection Supported key values for xeve-params > + > +Set xeve options using a list of @var{key}=@var{value} couples separated > + > +@table @option > + > +@item hash > +Embed the picture signature (HASH) for conformance checking in decoding, (0: Off, 1: On) > + > +@item vbv-bufsize > +Set the VBV buffer size: Kbits(none,K,k), Mbits(M,m) > + > +@end table > + > @section libxvid > > Xvid MPEG-4 Part 2 encoder wrapper. > diff --git a/doc/general_contents.texi b/doc/general_contents.texi > index 86ec6d606b..89236047ea 100644 > --- a/doc/general_contents.texi > +++ b/doc/general_contents.texi > @@ -343,6 +343,22 @@ libxavs2 is under the GNU Public License Version 2 or later > details), you must upgrade FFmpeg's license to GPL in order to use it. > @end float > > +@section eXtra-fast Essential Video Encoder (XEVE) > + > +FFmpeg can make use of the XEVE library for EVC video encoding. > + > +Go to @url{https://github.com/mpeg5/xeve} and follow the instructions for > +installing the XEVE library. Then pass @code{--enable-libxeve} to configure to > +enable it. > + > +@section eXtra-fast Essential Video Decoder (XEVD) > + > +FFmpeg can make use of the XEVD library for EVC video decoding. > + > +Go to @url{https://github.com/mpeg5/xevd} and follow the instructions for > +installing the XEVD library. Then pass @code{--enable-libxevd} to configure to > +enable it. > + > @section ZVBI > > ZVBI is a VBI decoding library which can be used by FFmpeg to decode DVB > @@ -591,6 +607,7 @@ library: > @item raw DTS @tab X @tab X > @item raw DTS-HD @tab @tab X > @item raw E-AC-3 @tab X @tab X > +@item raw EVC @tab X @tab X > @item raw FLAC @tab X @tab X > @item raw GSM @tab @tab X > @item raw H.261 @tab X @tab X > @@ -930,6 +947,8 @@ following image formats are supported: > @item Electronic Arts TQI video @tab @tab X > @item Escape 124 @tab @tab X > @item Escape 130 @tab @tab X > +@item EVC / MPEG-5 Part 1 @tab X @tab X > + @tab encoding and decoding supported through external libraries libxeve and libxevd > @item FFmpeg video codec #1 @tab X @tab X > @tab lossless codec (fourcc: FFV1) > @item Flash Screen Video v1 @tab X @tab X > diff --git a/libavcodec/Makefile b/libavcodec/Makefile > index 83901bb52b..cfeacd96d1 100644 > --- a/libavcodec/Makefile > +++ b/libavcodec/Makefile > @@ -1102,6 +1102,8 @@ OBJS-$(CONFIG_LIBX264_ENCODER) += libx264.o > OBJS-$(CONFIG_LIBX265_ENCODER) += libx265.o > OBJS-$(CONFIG_LIBXAVS_ENCODER) += libxavs.o > OBJS-$(CONFIG_LIBXAVS2_ENCODER) += libxavs2.o > +OBJS-$(CONFIG_LIBXEVD_DECODER) += libxevd.o > +OBJS-$(CONFIG_LIBXEVE_ENCODER) += libxeve.o > OBJS-$(CONFIG_LIBXVID_ENCODER) += libxvid.o > OBJS-$(CONFIG_LIBZVBI_TELETEXT_DECODER) += libzvbi-teletextdec.o ass.o > > @@ -1129,6 +1131,7 @@ OBJS-$(CONFIG_DVAUDIO_PARSER) += dvaudio_parser.o > OBJS-$(CONFIG_DVBSUB_PARSER) += dvbsub_parser.o > OBJS-$(CONFIG_DVD_NAV_PARSER) += dvd_nav_parser.o > OBJS-$(CONFIG_DVDSUB_PARSER) += dvdsub_parser.o > +OBJS-$(CONFIG_EVC_PARSER) += evc_parser.o > OBJS-$(CONFIG_FLAC_PARSER) += flac_parser.o flacdata.o flac.o > OBJS-$(CONFIG_G723_1_PARSER) += g723_1_parser.o > OBJS-$(CONFIG_G729_PARSER) += g729_parser.o > diff --git a/libavcodec/allcodecs.c b/libavcodec/allcodecs.c > index c94e2d5966..2526e11f70 100644 > --- a/libavcodec/allcodecs.c > +++ b/libavcodec/allcodecs.c > @@ -800,6 +800,8 @@ extern LIBX264_CONST FFCodec ff_libx264_encoder; > #endif > extern const FFCodec ff_libx264rgb_encoder; > extern FFCodec ff_libx265_encoder; > +extern const FFCodec ff_libxeve_encoder; > +extern const FFCodec ff_libxevd_decoder; > extern const FFCodec ff_libxavs_encoder; > extern const FFCodec ff_libxavs2_encoder; > extern const FFCodec ff_libxvid_encoder; > diff --git a/libavcodec/avcodec.h b/libavcodec/avcodec.h > index 65c8535359..7d56a03fca 100644 > --- a/libavcodec/avcodec.h > +++ b/libavcodec/avcodec.h > @@ -1674,6 +1674,9 @@ typedef struct AVCodecContext { > #define FF_PROFILE_KLVA_SYNC 0 > #define FF_PROFILE_KLVA_ASYNC 1 > > +#define FF_PROFILE_EVC_BASELINE 0 > +#define FF_PROFILE_EVC_MAIN 1 > + > /** > * level > * - encoding: Set by user. > diff --git a/libavcodec/codec_desc.c b/libavcodec/codec_desc.c > index c1a177c22d..1a43ff8cb6 100644 > --- a/libavcodec/codec_desc.c > +++ b/libavcodec/codec_desc.c > @@ -1907,6 +1907,14 @@ static const AVCodecDescriptor codec_descriptors[] = { > .long_name = NULL_IF_CONFIG_SMALL("WBMP (Wireless Application Protocol Bitmap) image"), > .props = AV_CODEC_PROP_INTRA_ONLY | AV_CODEC_PROP_LOSSLESS, > }, > + { > + .id = AV_CODEC_ID_EVC, > + .type = AVMEDIA_TYPE_VIDEO, > + .name = "evc", > + .long_name = NULL_IF_CONFIG_SMALL("MPEG-5 EVC (Essential Video Coding)"), > + .props = AV_CODEC_PROP_LOSSY | AV_CODEC_PROP_REORDER, > + .profiles = NULL_IF_CONFIG_SMALL(ff_evc_profiles), > + }, > > /* various PCM "codecs" */ > { > diff --git a/libavcodec/codec_id.h b/libavcodec/codec_id.h > index 386a00a7ef..02b60be690 100644 > --- a/libavcodec/codec_id.h > +++ b/libavcodec/codec_id.h > @@ -314,6 +314,7 @@ enum AVCodecID { > AV_CODEC_ID_PHM, > AV_CODEC_ID_RADIANCE_HDR, > AV_CODEC_ID_WBMP, > + AV_CODEC_ID_EVC, > > /* various PCM "codecs" */ > AV_CODEC_ID_FIRST_AUDIO = 0x10000, ///< A dummy id pointing at the start of audio codecs > diff --git a/libavcodec/evc_parser.c b/libavcodec/evc_parser.c > new file mode 100644 > index 0000000000..0ae0eaa08a > --- /dev/null > +++ b/libavcodec/evc_parser.c > @@ -0,0 +1,758 @@ > +/* > + * EVC format parser > + * > + * Copyright (C) 2021 Dawid Kozinski <d.kozinski@samsung.com> > + * > + * This file is part of FFmpeg. > + * > + * FFmpeg is free software; you can redistribute it and/or > + * modify it under the terms of the GNU Lesser General Public > + * License as published by the Free Software Foundation; either > + * version 2.1 of the License, or (at your option) any later version. > + * > + * FFmpeg is distributed in the hope that it will be useful, > + * but WITHOUT ANY WARRANTY; without even the implied warranty of > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > + * Lesser General Public License for more details. > + * > + * You should have received a copy of the GNU Lesser General Public > + * License along with FFmpeg; if not, write to the Free Software > + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA > + */ > + > +#include <stdint.h> > + > +#include "libavutil/common.h" > + > +#include "parser.h" > +#include "golomb.h" > + > +// The length field that indicates the length in bytes of the following NAL unit is configured to be of 4 bytes > +#define EVC_NAL_UNIT_LENGTH_BYTE (4) /* byte */ > + > +#define EVC_NAL_HEADER_SIZE (2) /* byte */ > +#define MAX_SPS_CNT (16) /* defined value in EVC standard */ > +#define MAX_PPS_CNT (64) /* defined value in EVC standard */ > + > +#define MAX_NUM_TILES_ROW (22) > +#define MAX_NUM_TILES_COL (20) > + > +// NALU types > +// @see ISO_IEC_23094-1_2020 7.4.2.2 NAL unit header semantics > +// > +#define EVC_NUT_NONIDR (0) /* Coded slice of a non-IDR picture */ > +#define EVC_NUT_IDR (1) /* Coded slice of an IDR picture */ > +#define EVC_NUT_SPS (24) /* Sequence parameter set */ > +#define EVC_NUT_PPS (25) /* Picture paremeter set */ > +#define EVC_NUT_APS (26) /* Adaptation parameter set */ > +#define EVC_NUT_FD (27) /* Filler data */ > +#define EVC_NUT_SEI (28) /* Supplemental enhancement information */ > + > +// slice type > +// @see ISO_IEC_23094-1_2020 7.4.5 Slice header semantics > +// > +#define SLICE_TYPE_UNKNOWN (-1) > +#define SLICE_TYPE_B (0) > +#define SLICE_TYPE_P (1) > +#define SLICE_TYPE_I (2) > + > +#define MAX_NUM_RPLS (32) > +#define MAX_NUM_REF_PICS (21) > + > +// rpl structure > +typedef struct RefPicListStruct { > + int poc; > + int tid; > + int ref_pic_num; > + int ref_pic_active_num; > + int ref_pics[MAX_NUM_REF_PICS]; > + char pic_type; > + > +} RefPicListStruct; > + > +// The sturcture reflects SPS RBSP(raw byte sequence payload) layout > +// @see ISO_IEC_23094-1 section 7.3.2.1 > +// > +// The following descriptors specify the parsing process of each element > +// u(n) - unsigned integer using n bits > +// ue(v) - unsigned integer 0-th order Exp_Golomb-coded syntax element with the left bit first > +typedef struct EVCParserSPS { > + int sps_seq_parameter_set_id; // ue(v) > + int profile_idc; // u(8) > + int level_idc; // u(8) > + int toolset_idc_h; // u(32) > + int toolset_idc_l; // u(32) > + int chroma_format_idc; // ue(v) > + int pic_width_in_luma_samples; // ue(v) > + int pic_height_in_luma_samples; // ue(v) > + int bit_depth_luma_minus8; // ue(v) > + int bit_depth_chroma_minus8; // ue(v) > + > + int sps_btt_flag; // u(1) > + int log2_ctu_size_minus5; // ue(v) > + int log2_min_cb_size_minus2; // ue(v) > + int log2_diff_ctu_max_14_cb_size; // ue(v) > + int log2_diff_ctu_max_tt_cb_size; // ue(v) > + int log2_diff_min_cb_min_tt_cb_size_minus2; // ue(v) > + > + int sps_suco_flag; // u(1) > + int log2_diff_ctu_size_max_suco_cb_size; // ue(v) > + int log2_diff_max_suco_min_suco_cb_size; // ue(v) > + > + int sps_admvp_flag; // u(1) > + int sps_affine_flag; // u(1) > + int sps_amvr_flag; // u(1) > + int sps_dmvr_flag; // u(1) > + int sps_mmvd_flag; // u(1) > + int sps_hmvp_flag; // u(1) > + > + int sps_eipd_flag; // u(1) > + int sps_ibc_flag; // u(1) > + int log2_max_ibc_cand_size_minus2; // ue(v) > + > + int sps_cm_init_flag; // u(1) > + int sps_adcc_flag; // u(1) > + > + int sps_iqt_flag; // u(1) > + int sps_ats_flag; // u(1) > + > + int sps_addb_flag; // u(1) > + int sps_alf_flag; // u(1) > + int sps_htdf_flag; // u(1) > + int sps_rpl_flag; // u(1) > + int sps_pocs_flag; // u(1) > + int sps_dquant_flag; // u(1) > + int sps_dra_flag; // u(1) > + > + int log2_max_pic_order_cnt_lsb_minus4; // ue(v) > + int log2_sub_gop_length; // ue(v) > + int log2_ref_pic_gap_length; // ue(v) > + > + int max_num_tid0_ref_pics; // ue(v) > + > + int sps_max_dec_pic_buffering_minus1; // ue(v) > + int long_term_ref_pic_flag; // u(1) > + int rpl1_same_as_rpl0_flag; // u(1) > + int num_ref_pic_list_in_sps[2]; // ue(v) > + RefPicListStruct rpls[2][MAX_NUM_RPLS]; > + > + int picture_cropping_flag; // u(1) > + int picture_crop_left_offset; // ue(v) > + int picture_crop_right_offset; // ue(v) > + int picture_crop_top_offset; // ue(v) > + int picture_crop_bottom_offset; // ue(v) > + > + // @note > + // Currently the structure does not reflect the entire SPS RBSP layout. > + // It contains only the fields that are necessary to read from the NAL unit all the values > + // necessary for the correct initialization of the AVCodecContext structure. > + > + // @note > + // If necessary, add the missing fields to the structure to reflect > + // the contents of the entire NAL unit of the SPS type > + > +} EVCParserSPS; > + > +typedef struct EVCParserPPS { > + int pps_pic_parameter_set_id; // ue(v) > + int pps_seq_parameter_set_id; // ue(v) > + int num_ref_idx_default_active_minus1[2]; // ue(v) > + int additional_lt_poc_lsb_len; // ue(v) > + int rpl1_idx_present_flag; // u(1) > + int single_tile_in_pic_flag; // u(1) > + int num_tile_columns_minus1; // ue(v) > + int num_tile_rows_minus1; // ue(v) > + int uniform_tile_spacing_flag; // u(1) > + int tile_column_width_minus1[MAX_NUM_TILES_ROW]; // ue(v) > + int tile_row_height_minus1[MAX_NUM_TILES_COL]; // ue(v) > + int loop_filter_across_tiles_enabled_flag; // u(1) > + int tile_offset_len_minus1; // ue(v) > + int tile_id_len_minus1; // ue(v) > + int explicit_tile_id_flag; // u(1) > + int tile_id_val[MAX_NUM_TILES_ROW][MAX_NUM_TILES_COL]; // u(v) > + int pic_dra_enabled_flag; // u(1) > + int pic_dra_aps_id; // u(5) > + int arbitrary_slice_present_flag; // u(1) > + int constrained_intra_pred_flag; // u(1) > + int cu_qp_delta_enabled_flag; // u(1) > + int log2_cu_qp_delta_area_minus6; // ue(v) > + > +} EVCParserPPS; > + > +// The sturcture reflects Slice Header RBSP(raw byte sequence payload) layout > +// @see ISO_IEC_23094-1 section 7.3.2.6 > +// > +// The following descriptors specify the parsing process of each element > +// u(n) - unsigned integer using n bits > +// ue(v) - unsigned integer 0-th order Exp_Golomb-coded syntax element with the left bit first > +// u(n) - unsigned integer using n bits. > +// When n is "v" in the syntax table, the number of bits varies in a manner dependent on the value of other syntax elements. > +typedef struct EVCParserSliceHeader { > + int slice_pic_parameter_set_id; // ue(v) > + int single_tile_in_slice_flag; // u(1) > + int first_tile_id; // u(v) > + int arbitrary_slice_flag; // u(1) > + int last_tile_id; // u(v) > + int num_remaining_tiles_in_slice_minus1; // ue(v) > + int delta_tile_id_minus1[MAX_NUM_TILES_ROW * MAX_NUM_TILES_COL]; // ue(v) > + > + int slice_type; // ue(v) > + > + // @note > + // Currently the structure does not reflect the entire Slice Header RBSP layout. > + // It contains only the fields that are necessary to read from the NAL unit all the values > + // necessary for the correct initialization of the AVCodecContext structure. > + > + // @note > + // If necessary, add the missing fields to the structure to reflect > + // the contents of the entire NAL unit of the SPS type > + > +} EVCParserSliceHeader; > + > +typedef struct EVCParserContext { > + ParseContext pc; > + EVCParserSPS sps[MAX_SPS_CNT]; > + EVCParserPPS pps[MAX_PPS_CNT]; > + EVCParserSliceHeader slice_header[MAX_PPS_CNT]; > + int is_avc; > + int nal_length_size; > + int to_read; > + int incomplete_nalu_prefix_read; // The flag is set to 1 when an incomplete NAL unit prefix has been read > + > +} EVCParserContext; > + > +static int get_nalu_type(const uint8_t *bits, int bits_size, AVCodecContext *avctx) > +{ > + int unit_type_plus1 = 0; > + > + if (bits_size >= EVC_NAL_HEADER_SIZE) { > + unsigned char *p = (unsigned char *)bits; > + // forbidden_zero_bit > + if ((p[0] & 0x80) != 0) > + return -1; > + > + // nal_unit_type > + unit_type_plus1 = (p[0] >> 1) & 0x3F; > + } > + > + return unit_type_plus1 - 1; > +} > + > +static uint32_t read_nal_unit_length(const uint8_t *bits, int bits_size, AVCodecContext *avctx) > +{ > + uint32_t nalu_len = 0; > + > + if (bits_size >= EVC_NAL_UNIT_LENGTH_BYTE) { > + > + int t = 0; > + unsigned char *p = (unsigned char *)bits; > + > + for (int i = 0; i < EVC_NAL_UNIT_LENGTH_BYTE; i++) > + t = (t << 8) | p[i]; > + > + nalu_len = t; > + if (nalu_len == 0) > + return 0; > + } > + > + return nalu_len; > +} > + > +// @see ISO_IEC_23094-1 (7.3.2.1 SPS RBSP syntax) > +static EVCParserSPS *parse_sps(const uint8_t *bs, int bs_size, EVCParserContext *ev) > +{ > + GetBitContext gb; > + EVCParserSPS *sps; > + int sps_seq_parameter_set_id; > + > + if (init_get_bits8(&gb, bs, bs_size) < 0) > + return NULL; > + > + sps_seq_parameter_set_id = get_ue_golomb(&gb); > + > + if (sps_seq_parameter_set_id >= MAX_SPS_CNT) > + return NULL; > + > + sps = &ev->sps[sps_seq_parameter_set_id]; > + sps->sps_seq_parameter_set_id = sps_seq_parameter_set_id; > + > + // the Baseline profile is indicated by profile_idc eqal to 0 > + // the Main profile is indicated by profile_idc eqal to 1 > + sps->profile_idc = get_bits(&gb, 8); > + > + sps->level_idc = get_bits(&gb, 8); > + > + skip_bits_long(&gb, 32); /* skip toolset_idc_h */ > + skip_bits_long(&gb, 32); /* skip toolset_idc_l */ > + > + // 0 - monochrome > + // 1 - 4:2:0 > + // 2 - 4:2:2 > + // 3 - 4:4:4 > + sps->chroma_format_idc = get_ue_golomb(&gb); > + > + sps->pic_width_in_luma_samples = get_ue_golomb(&gb); > + sps->pic_height_in_luma_samples = get_ue_golomb(&gb); > + > + sps->bit_depth_luma_minus8 = get_ue_golomb(&gb); > + sps->bit_depth_chroma_minus8 = get_ue_golomb(&gb); > + > + sps->sps_btt_flag = get_bits(&gb, 1); > + if (sps->sps_btt_flag) { > + sps->log2_ctu_size_minus5 = get_ue_golomb(&gb); > + sps->log2_min_cb_size_minus2 = get_ue_golomb(&gb); > + sps->log2_diff_ctu_max_14_cb_size = get_ue_golomb(&gb); > + sps->log2_diff_ctu_max_tt_cb_size = get_ue_golomb(&gb); > + sps->log2_diff_min_cb_min_tt_cb_size_minus2 = get_ue_golomb(&gb); > + } > + > + sps->sps_suco_flag = get_bits(&gb, 1); > + if (sps->sps_suco_flag) { > + sps->log2_diff_ctu_size_max_suco_cb_size = get_ue_golomb(&gb); > + sps->log2_diff_max_suco_min_suco_cb_size = get_ue_golomb(&gb); > + } > + > + sps->sps_admvp_flag = get_bits(&gb, 1); > + if (sps->sps_admvp_flag) { > + sps->sps_affine_flag = get_bits(&gb, 1); > + sps->sps_amvr_flag = get_bits(&gb, 1); > + sps->sps_dmvr_flag = get_bits(&gb, 1); > + sps->sps_mmvd_flag = get_bits(&gb, 1); > + sps->sps_hmvp_flag = get_bits(&gb, 1); > + } > + > + sps->sps_eipd_flag = get_bits(&gb, 1); > + if (sps->sps_eipd_flag) { > + sps->sps_ibc_flag = get_bits(&gb, 1); > + if (sps->sps_ibc_flag) > + sps->log2_max_ibc_cand_size_minus2 = get_ue_golomb(&gb); > + } > + > + sps->sps_cm_init_flag = get_bits(&gb, 1); > + if (sps->sps_cm_init_flag) > + sps->sps_adcc_flag = get_bits(&gb, 1); > + > + sps->sps_iqt_flag = get_bits(&gb, 1); > + if (sps->sps_iqt_flag) > + sps->sps_ats_flag = get_bits(&gb, 1); > + > + sps->sps_addb_flag = get_bits(&gb, 1); > + sps->sps_alf_flag = get_bits(&gb, 1); > + sps->sps_htdf_flag = get_bits(&gb, 1); > + sps->sps_rpl_flag = get_bits(&gb, 1); > + sps->sps_pocs_flag = get_bits(&gb, 1); > + sps->sps_dquant_flag = get_bits(&gb, 1); > + sps->sps_dra_flag = get_bits(&gb, 1); > + > + if (sps->sps_pocs_flag) > + sps->log2_max_pic_order_cnt_lsb_minus4 = get_ue_golomb(&gb); > + > + if (!sps->sps_pocs_flag || !sps->sps_rpl_flag) { > + sps->log2_sub_gop_length = get_ue_golomb(&gb); > + if (sps->log2_sub_gop_length == 0) > + sps->log2_ref_pic_gap_length = get_ue_golomb(&gb); > + } > + > + // @note > + // If necessary, add the missing fields to the EVCParserSPS structure > + // and then extend parser implementation > + > + return sps; > +} > + > +// @see ISO_IEC_23094-1 (7.3.2.2 SPS RBSP syntax) > +static EVCParserPPS *parse_pps(const uint8_t *bs, int bs_size, EVCParserContext *ev) > +{ > + GetBitContext gb; > + EVCParserPPS *pps; > + > + int pps_pic_parameter_set_id; > + > + if (init_get_bits8(&gb, bs, bs_size) < 0) > + return NULL; > + > + pps_pic_parameter_set_id = get_ue_golomb(&gb); > + if (pps_pic_parameter_set_id > MAX_PPS_CNT) > + return NULL; > + > + pps = &ev->pps[pps_pic_parameter_set_id]; > + > + pps->pps_pic_parameter_set_id = pps_pic_parameter_set_id; > + > + pps->pps_seq_parameter_set_id = get_ue_golomb(&gb); > + if (pps->pps_seq_parameter_set_id >= MAX_SPS_CNT) > + return NULL; > + > + pps->num_ref_idx_default_active_minus1[0] = get_ue_golomb(&gb); > + pps->num_ref_idx_default_active_minus1[1] = get_ue_golomb(&gb); > + pps->additional_lt_poc_lsb_len = get_ue_golomb(&gb); > + pps->rpl1_idx_present_flag = get_bits(&gb, 1); > + pps->single_tile_in_pic_flag = get_bits(&gb, 1); > + > + if (!pps->single_tile_in_pic_flag) { > + pps->num_tile_columns_minus1 = get_ue_golomb(&gb); > + pps->num_tile_rows_minus1 = get_ue_golomb(&gb); > + pps->uniform_tile_spacing_flag = get_bits(&gb, 1); > + > + if (!pps->uniform_tile_spacing_flag) { > + for (int i = 0; i < pps->num_tile_columns_minus1; i++) > + pps->tile_column_width_minus1[i] = get_ue_golomb(&gb); > + > + for (int i = 0; i < pps->num_tile_rows_minus1; i++) > + pps->tile_row_height_minus1[i] = get_ue_golomb(&gb); > + } > + pps->loop_filter_across_tiles_enabled_flag = get_bits(&gb, 1); > + pps->tile_offset_len_minus1 = get_ue_golomb(&gb); > + } > + > + pps->tile_id_len_minus1 = get_ue_golomb(&gb); > + pps->explicit_tile_id_flag = get_bits(&gb, 1); > + > + if (pps->explicit_tile_id_flag) { > + for (int i = 0; i <= pps->num_tile_rows_minus1; i++) { > + for (int j = 0; j <= pps->num_tile_columns_minus1; j++) > + pps->tile_id_val[i][j] = get_bits(&gb, pps->tile_id_len_minus1 + 1); > + } > + } > + > + pps->pic_dra_enabled_flag = 0; > + pps->pic_dra_enabled_flag = get_bits(&gb, 1); > + > + if (pps->pic_dra_enabled_flag) > + pps->pic_dra_aps_id = get_bits(&gb, 5); > + > + pps->arbitrary_slice_present_flag = get_bits(&gb, 1); > + pps->constrained_intra_pred_flag = get_bits(&gb, 1); > + pps->cu_qp_delta_enabled_flag = get_bits(&gb, 1); > + > + if (pps->cu_qp_delta_enabled_flag) > + pps->log2_cu_qp_delta_area_minus6 = get_ue_golomb(&gb); > + > + return pps; > +} > + > +// @see ISO_IEC_23094-1 (7.3.2.6 Slice layer RBSP syntax) > +static EVCParserSliceHeader *parse_slice_header(const uint8_t *bs, int bs_size, EVCParserContext *ev) > +{ > + GetBitContext gb; > + EVCParserSliceHeader *sh; > + EVCParserPPS *pps; > + int num_tiles_in_slice = 0; > + int slice_pic_parameter_set_id; > + > + if (init_get_bits8(&gb, bs, bs_size) < 0) > + return NULL; > + > + slice_pic_parameter_set_id = get_ue_golomb(&gb); > + > + if (slice_pic_parameter_set_id < 0 || slice_pic_parameter_set_id >= MAX_PPS_CNT) > + return NULL; > + > + sh = &ev->slice_header[slice_pic_parameter_set_id]; > + pps = &ev->pps[slice_pic_parameter_set_id]; > + > + sh->slice_pic_parameter_set_id = slice_pic_parameter_set_id; > + > + if (!pps->single_tile_in_pic_flag) { > + sh->single_tile_in_slice_flag = get_bits(&gb, 1); > + sh->first_tile_id = get_bits(&gb, pps->tile_id_len_minus1 + 1); > + } else > + sh->single_tile_in_slice_flag = 1; > + > + if (!sh->single_tile_in_slice_flag) { > + if (pps->arbitrary_slice_present_flag) > + sh->arbitrary_slice_flag = get_bits(&gb, 1); > + > + if (!sh->arbitrary_slice_flag) > + sh->last_tile_id = get_bits(&gb, pps->tile_id_len_minus1 + 1); > + else { > + sh->num_remaining_tiles_in_slice_minus1 = get_ue_golomb(&gb); > + num_tiles_in_slice = sh->num_remaining_tiles_in_slice_minus1 + 2; > + for (int i = 0; i < num_tiles_in_slice - 1; ++i) > + sh->delta_tile_id_minus1[i] = get_ue_golomb(&gb); > + } > + } > + > + sh->slice_type = get_ue_golomb(&gb); > + > + // @note > + // If necessary, add the missing fields to the EVCParserSliceHeader structure > + // and then extend parser implementation > + > + return sh; > +} > + > +static int parse_nal_units(AVCodecParserContext *s, const uint8_t *bs, > + int bs_size, AVCodecContext *avctx) > +{ > + EVCParserContext *ev = s->priv_data; > + int nalu_type, nalu_size; > + unsigned char *bits = (unsigned char *)bs; > + int bits_size = bs_size; > + > + avctx->codec_id = AV_CODEC_ID_EVC; > + s->pict_type = AV_PICTURE_TYPE_NONE; > + s->picture_structure = AV_PICTURE_STRUCTURE_FRAME; > + s->key_frame = -1; > + > + nalu_size = read_nal_unit_length(bits, bits_size, avctx); > + if (nalu_size == 0) { > + av_log(avctx, AV_LOG_ERROR, "Invalid NAL unit size: (%d)\n", nalu_size); > + return -1; > + } > + > + bits += EVC_NAL_UNIT_LENGTH_BYTE; > + bits_size -= EVC_NAL_UNIT_LENGTH_BYTE; > + > + nalu_type = get_nalu_type(bits, bits_size, avctx); > + > + bits += EVC_NAL_HEADER_SIZE; > + bits_size -= EVC_NAL_HEADER_SIZE; > + > + if (nalu_type == EVC_NUT_SPS) { // NAL Unit type: SPS (Sequence Parameter Set) > + EVCParserSPS *sps; > + > + sps = parse_sps(bits, bits_size, ev); > + if (!sps) { > + av_log(avctx, AV_LOG_ERROR, "SPS parsing error\n"); > + return -1; > + } > + > + s->coded_width = sps->pic_width_in_luma_samples; > + s->coded_height = sps->pic_height_in_luma_samples; > + s->width = sps->pic_width_in_luma_samples; > + s->height = sps->pic_height_in_luma_samples; > + > + if (sps->profile_idc == 1) avctx->profile = FF_PROFILE_EVC_MAIN; > + else avctx->profile = FF_PROFILE_EVC_BASELINE; > + > + // Currently XEVD decoder supports ony YCBCR420_10LE chroma format for EVC stream > + switch (sps->chroma_format_idc) { > + case 0: /* YCBCR400_10LE */ > + av_log(avctx, AV_LOG_ERROR, "YCBCR400_10LE: Not supported chroma format\n"); > + s->format = AV_PIX_FMT_GRAY10LE; > + return -1; > + case 1: /* YCBCR420_10LE */ > + s->format = AV_PIX_FMT_YUV420P10LE; > + break; > + case 2: /* YCBCR422_10LE */ > + av_log(avctx, AV_LOG_ERROR, "YCBCR422_10LE: Not supported chroma format\n"); > + s->format = AV_PIX_FMT_YUV422P10LE; > + return -1; > + case 3: /* YCBCR444_10LE */ > + av_log(avctx, AV_LOG_ERROR, "YCBCR444_10LE: Not supported chroma format\n"); > + s->format = AV_PIX_FMT_YUV444P10LE; > + return -1; > + default: > + s->format = AV_PIX_FMT_NONE; > + av_log(avctx, AV_LOG_ERROR, "Unknown supported chroma format\n"); > + return -1; > + } > + > + // @note > + // The current implementation of parse_sps function doesn't handle VUI parameters parsing. > + // If it will be needed, parse_sps function could be extended to handle VUI parameters parsing > + // to initialize fields of the AVCodecContex i.e. color_primaries, color_trc,color_range > + > + } else if (nalu_type == EVC_NUT_PPS) { // NAL Unit type: PPS (Video Parameter Set) > + EVCParserPPS *pps; > + > + pps = parse_pps(bits, bits_size, ev); > + if (!pps) { > + av_log(avctx, AV_LOG_ERROR, "PPS parsing error\n"); > + return -1; > + } > + } else if (nalu_type == EVC_NUT_SEI) // NAL unit type: SEI (Supplemental Enhancement Information) > + return 0; > + else if (nalu_type == EVC_NUT_IDR || nalu_type == EVC_NUT_NONIDR) { // NAL Unit type: Coded slice of a IDR or non-IDR picture > + EVCParserSliceHeader *sh; > + > + sh = parse_slice_header(bits, bits_size, ev); > + if (!sh) { > + av_log(avctx, AV_LOG_ERROR, "Slice header parsing error\n"); > + return -1; > + } > + switch (sh->slice_type) { > + case SLICE_TYPE_B: { > + s->pict_type = AV_PICTURE_TYPE_B; > + break; > + } > + case SLICE_TYPE_P: { > + s->pict_type = AV_PICTURE_TYPE_P; > + break; > + } > + case SLICE_TYPE_I: { > + s->pict_type = AV_PICTURE_TYPE_I; > + break; > + } > + default: { > + s->pict_type = AV_PICTURE_TYPE_NONE; > + } > + } > + s->key_frame = (nalu_type == EVC_NUT_IDR) ? 1 : 0; > + } else { > + av_log(avctx, AV_LOG_ERROR, "Invalid NAL unit type: %d\n", nalu_type); > + return -1; > + } > + > + return 0; > +} > + > +/** > + * Find the end of the current frame in the bitstream. > + * @return the position of the first byte of the next frame, or END_NOT_FOUND > + */ > +static int evc_find_frame_end(AVCodecParserContext *s, const uint8_t *buf, > + int buf_size, AVCodecContext *avctx) > +{ > + EVCParserContext *ev = s->priv_data; > + > + if (!ev->to_read) { > + int nal_unit_size = 0; > + int next = END_NOT_FOUND; > + > + // This is the case when buffer size is not enough for buffer to store NAL unit length > + if (buf_size < EVC_NAL_UNIT_LENGTH_BYTE) { > + ev->to_read = EVC_NAL_UNIT_LENGTH_BYTE; > + ev->nal_length_size = buf_size; > + ev->incomplete_nalu_prefix_read = 1; > + > + return END_NOT_FOUND; > + } > + > + nal_unit_size = read_nal_unit_length(buf, buf_size, avctx); > + ev->nal_length_size = EVC_NAL_UNIT_LENGTH_BYTE; > + > + next = nal_unit_size + EVC_NAL_UNIT_LENGTH_BYTE; > + ev->to_read = next; > + if (next < buf_size) > + return next; > + else > + return END_NOT_FOUND; > + } else if (ev->to_read > buf_size) > + return END_NOT_FOUND; > + else { > + if (ev->incomplete_nalu_prefix_read == 1) { > + EVCParserContext *ev = s->priv_data; > + ParseContext *pc = &ev->pc; > + uint8_t nalu_len[EVC_NAL_UNIT_LENGTH_BYTE] = {0}; > + int nal_unit_size = 0; > + > + // 1. pc->buffer contains previously read bytes of NALU prefix > + // 2. buf contains the rest of NAL unit prefix bytes > + // > + // ~~~~~~~ > + // EXAMPLE > + // ~~~~~~~ > + // > + // In the following example we assumed that the number of already read NAL Unit prefix bytes is equal 1 > + // > + // ---------- > + // pc->buffer -> conatins already read bytes > + // ---------- > + // __ pc->index == 1 > + // | > + // V > + // ------------------------------------------------------- > + // | 0 | 1 | 2 | 3 | 4 | ... | N | > + // ------------------------------------------------------- > + // | 0x00 | 0xXX | 0xXX | 0xXX | 0xXX | ... | 0xXX | > + // ------------------------------------------------------- > + // > + // ---------- > + // buf -> contains newly read bytes > + // ---------- > + // ------------------------------------------------------- > + // | 0 | 1 | 2 | 3 | 4 | ... | N | > + // ------------------------------------------------------- > + // | 0x00 | 0x00 | 0x3C | 0xXX | 0xXX | ... | 0xXX | > + // ------------------------------------------------------- > + // > + for (int i = 0; i < EVC_NAL_UNIT_LENGTH_BYTE; i++) { > + if (i < pc->index) > + nalu_len[i] = pc->buffer[i]; > + else > + nalu_len[i] = buf[i - pc->index]; > + } > + > + // ---------- > + // nalu_len > + // ---------- > + // --------------------------------- > + // | 0 | 1 | 2 | 3 | > + // --------------------------------- > + // | 0x00 | 0x00 | 0x00 | 0x3C | > + // --------------------------------- > + // | NALU LENGTH | > + // --------------------------------- > + // NAL Unit lenght = 60 (0x0000003C) > + > + nal_unit_size = read_nal_unit_length(nalu_len, EVC_NAL_UNIT_LENGTH_BYTE, avctx); > + > + ev->to_read = nal_unit_size + EVC_NAL_UNIT_LENGTH_BYTE - pc->index; > + > + ev->incomplete_nalu_prefix_read = 0; > + > + if (ev->to_read > buf_size) > + return END_NOT_FOUND; > + else > + return ev->to_read; > + } > + return ev->to_read; > + } > + > + return END_NOT_FOUND; > +} > + > +static int evc_parser_init(AVCodecParserContext *s) > +{ > + EVCParserContext *ev = s->priv_data; > + > + ev->nal_length_size = EVC_NAL_UNIT_LENGTH_BYTE; > + ev->incomplete_nalu_prefix_read = 0; > + > + return 0; > +} > + > +static int evc_parse(AVCodecParserContext *s, AVCodecContext *avctx, > + const uint8_t **poutbuf, int *poutbuf_size, > + const uint8_t *buf, int buf_size) > +{ > + int next; > + EVCParserContext *ev = s->priv_data; > + ParseContext *pc = &ev->pc; > + int is_dummy_buf = !buf_size; > + const uint8_t *dummy_buf = buf; > + > + if (s->flags & PARSER_FLAG_COMPLETE_FRAMES) > + next = buf_size; > + else { > + next = evc_find_frame_end(s, buf, buf_size, avctx); > + if (ff_combine_frame(pc, next, &buf, &buf_size) < 0) { > + *poutbuf = NULL; > + *poutbuf_size = 0; > + ev->to_read -= buf_size; > + return buf_size; > + } > + } > + > + is_dummy_buf &= (dummy_buf == buf); > + > + if (!is_dummy_buf) > + parse_nal_units(s, buf, buf_size, avctx); > + > + // poutbuf contains just one NAL unit > + *poutbuf = buf; > + *poutbuf_size = buf_size; > + ev->to_read -= next; > + > + return next; > +} > + > +const AVCodecParser ff_evc_parser = { > + .codec_ids = { AV_CODEC_ID_EVC }, > + .priv_data_size = sizeof(EVCParserContext), > + .parser_init = evc_parser_init, > + .parser_parse = evc_parse, > + .parser_close = ff_parse_close, > +}; > diff --git a/libavcodec/libxevd.c b/libavcodec/libxevd.c > new file mode 100644 > index 0000000000..85f71affd3 > --- /dev/null > +++ b/libavcodec/libxevd.c > @@ -0,0 +1,419 @@ > +/* > + * libxevd decoder > + * EVC (MPEG-5 Essential Video Coding) decoding using XEVD MPEG-5 EVC decoder library > + * > + * Copyright (C) 2021 Dawid Kozinski <d.kozinski@samsung.com> > + * > + * This file is part of FFmpeg. > + * > + * FFmpeg is free software; you can redistribute it and/or > + * modify it under the terms of the GNU Lesser General Public > + * License as published by the Free Software Foundation; either > + * version 2.1 of the License, or (at your option) any later version. > + * > + * FFmpeg is distributed in the hope that it will be useful, > + * but WITHOUT ANY WARRANTY; without even the implied warranty of > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > + * Lesser General Public License for more details. > + * > + * You should have received a copy of the GNU Lesser General Public > + * License along with FFmpeg; if not, write to the Free Software > + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA > + */ > + > +#include <float.h> > +#include <stdlib.h> > + > +#include <xevd.h> > + > +#include "libavutil/internal.h" > +#include "libavutil/common.h" > +#include "libavutil/opt.h" > +#include "libavutil/pixdesc.h" > +#include "libavutil/pixfmt.h" > +#include "libavutil/imgutils.h" > +#include "libavutil/cpu.h" > + > +#include "avcodec.h" > +#include "internal.h" > +#include "packet_internal.h" > +#include "codec_internal.h" > + > +#define XEVD_PARAM_BAD_NAME -1 > +#define XEVD_PARAM_BAD_VALUE -2 > + > +#define EVC_NAL_HEADER_SIZE 2 /* byte */ > + > +/** > + * The structure stores all the states associated with the instance of Xeve MPEG-5 EVC decoder > + */ > +typedef struct XevdContext { > + const AVClass *class; > + > + XEVD id; // XEVD instance identifier @see xevd.h > + XEVD_CDSC cdsc; // decoding parameters @see xevd.h > +} XevdContext; > + > +/** > + * The function populates the XEVD_CDSC structure. > + * XEVD_CDSC contains all decoder parameters that should be initialized before its use. > + * > + * @param[in] avctx codec context > + * @param[out] cdsc contains all decoder parameters that should be initialized before its use > + * > + */ > +static void get_conf(AVCodecContext *avctx, XEVD_CDSC *cdsc) > +{ > + int cpu_count = av_cpu_count(); > + > + /* clear XEVS_CDSC structure */ > + memset(cdsc, 0, sizeof(XEVD_CDSC)); > + > + /* init XEVD_CDSC */ > + if (avctx->thread_count <= 0) > + cdsc->threads = (cpu_count < XEVD_MAX_TASK_CNT) ? cpu_count : XEVD_MAX_TASK_CNT; > + else if (avctx->thread_count > XEVD_MAX_TASK_CNT) > + cdsc->threads = XEVD_MAX_TASK_CNT; > + else > + cdsc->threads = avctx->thread_count; > +} > + > +/** > + * Read NAL unit length > + * @param bs input data (bitstream) > + * @return the length of NAL unit on success, 0 value on failure > + */ > +static uint32_t read_nal_unit_length(const uint8_t *bs, int bs_size, AVCodecContext *avctx) > +{ > + uint32_t len = 0; > + XEVD_INFO info; > + int ret; > + > + if (bs_size == XEVD_NAL_UNIT_LENGTH_BYTE) { > + ret = xevd_info((void *)bs, XEVD_NAL_UNIT_LENGTH_BYTE, 1, &info); > + if (XEVD_FAILED(ret)) { > + av_log(avctx, AV_LOG_ERROR, "Cannot get bitstream information\n"); > + return 0; > + } > + len = info.nalu_len; > + if (len == 0) { > + av_log(avctx, AV_LOG_ERROR, "Invalid bitstream size! [%d]\n", bs_size); > + return 0; > + } > + } > + > + return len; > +} > + > +/** > + * @param[in] xectx the structure that stores all the state associated with the instance of Xeve MPEG-5 EVC decoder > + * @param[out] avctx codec context > + * @return 0 on success, negative value on failure > + */ > +static int export_stream_params(const XevdContext *xectx, AVCodecContext *avctx) > +{ > + int ret; > + int size; > + int color_space; > + > + avctx->pix_fmt = AV_PIX_FMT_YUV420P10; > + > + size = 4; > + ret = xevd_config(xectx->id, XEVD_CFG_GET_CODED_WIDTH, &avctx->coded_width, &size); > + if (XEVD_FAILED(ret)) { > + av_log(avctx, AV_LOG_ERROR, "Failed to get coded_width\n"); > + return AVERROR_EXTERNAL; > + } > + > + ret = xevd_config(xectx->id, XEVD_CFG_GET_CODED_HEIGHT, &avctx->coded_height, &size); > + if (XEVD_FAILED(ret)) { > + av_log(avctx, AV_LOG_ERROR, "Failed to get coded_height\n"); > + return AVERROR_EXTERNAL; > + } > + > + ret = xevd_config(xectx->id, XEVD_CFG_GET_WIDTH, &avctx->width, &size); > + if (XEVD_FAILED(ret)) { > + av_log(avctx, AV_LOG_ERROR, "Failed to get width\n"); > + return AVERROR_EXTERNAL; > + } > + > + ret = xevd_config(xectx->id, XEVD_CFG_GET_HEIGHT, &avctx->height, &size); > + if (XEVD_FAILED(ret)) { > + av_log(avctx, AV_LOG_ERROR, "Failed to get height\n"); > + return AVERROR_EXTERNAL; > + } > + > + ret = xevd_config(xectx->id, XEVD_CFG_GET_COLOR_SPACE, &color_space, &size); > + if (XEVD_FAILED(ret)) { > + av_log(avctx, AV_LOG_ERROR, "Failed to get color_space\n"); > + return AVERROR_EXTERNAL; > + } > + switch(color_space) { > + case XEVD_CS_YCBCR400_10LE: > + avctx->pix_fmt = AV_PIX_FMT_GRAY10LE; > + break; > + case XEVD_CS_YCBCR420_10LE: > + avctx->pix_fmt = AV_PIX_FMT_YUV420P10LE; > + break; > + case XEVD_CS_YCBCR422_10LE: > + avctx->pix_fmt = AV_PIX_FMT_YUV422P10LE; > + break; > + case XEVD_CS_YCBCR444_10LE: > + avctx->pix_fmt = AV_PIX_FMT_YUV444P10LE; > + break; > + default: > + av_log(avctx, AV_LOG_ERROR, "Unknown color space\n"); > + avctx->pix_fmt = AV_PIX_FMT_NONE; > + return AVERROR_INVALIDDATA; > + } > + > + // the function returns sps->num_reorder_pics > + ret = xevd_config(xectx->id, XEVD_CFG_GET_MAX_CODING_DELAY, &avctx->max_b_frames, &size); > + if (XEVD_FAILED(ret)) { > + av_log(avctx, AV_LOG_ERROR, "Failed to get max_coding_delay\n"); > + return AVERROR_EXTERNAL; > + } > + > + avctx->has_b_frames = (avctx->max_b_frames) ? 1 : 0; > + > + avctx->color_primaries = AVCOL_PRI_UNSPECIFIED; > + avctx->color_trc = AVCOL_TRC_UNSPECIFIED; > + avctx->colorspace = AVCOL_SPC_UNSPECIFIED; > + > + return 0; > +} > + > +/** > + * @brief Copy image in imgb to frame. > + * > + * @param avctx codec context > + * @param[in] imgb > + * @param[out] frame > + * @return 0 on success, negative value on failure > + */ > +static int libxevd_image_copy(struct AVCodecContext *avctx, XEVD_IMGB *imgb, struct AVFrame *frame) > +{ > + int ret; > + if (imgb->cs != XEVD_CS_YCBCR420_10LE) { > + av_log(avctx, AV_LOG_ERROR, "Not supported pixel format: %s\n", av_get_pix_fmt_name(avctx->pix_fmt)); > + return AVERROR_INVALIDDATA; > + } > + > + if (imgb->w[0] != avctx->width || imgb->h[0] != avctx->height) { // stream resolution changed > + if (ff_set_dimensions(avctx, imgb->w[0], imgb->h[0]) < 0) { > + av_log(avctx, AV_LOG_ERROR, "Cannot set new dimension\n"); > + return AVERROR_INVALIDDATA; > + } > + } > + > + if (ret = ff_get_buffer(avctx, frame, 0) < 0) { > + return ret; > + } > + > + av_image_copy(frame->data, frame->linesize, (const uint8_t **)imgb->a, > + imgb->s, avctx->pix_fmt, > + imgb->w[0], imgb->h[0]); > + > + return 0; > +} > + > +/** > + * Initialize decoder > + * Create a decoder instance and allocate all the needed resources > + * > + * @param avctx codec context > + * @return 0 on success, negative error code on failure > + */ > +static av_cold int libxevd_init(AVCodecContext *avctx) > +{ > + XevdContext *xectx = avctx->priv_data; > + XEVD_CDSC *cdsc = &(xectx->cdsc); > + > + /* read configurations and set values for created descriptor (XEVD_CDSC) */ > + get_conf(avctx, cdsc); > + > + /* create decoder */ > + xectx->id = xevd_create(&(xectx->cdsc), NULL); > + if (xectx->id == NULL) { > + av_log(avctx, AV_LOG_ERROR, "Cannot create XEVD encoder\n"); > + return AVERROR_EXTERNAL; > + } > + > + return 0; > +} > + > +/** > + * Decode picture > + * > + * @param avctx codec context > + * @param[out] frame decoded frame > + * @param[out] got_frame_ptr decoder sets to 0 or 1 to indicate that a > + * non-empty frame or subtitle was returned in > + * outdata. > + * @param[in] avpkt AVPacket containing encoded data to be decoded > + * > + * @return amount of bytes read from the packet on success, negative error > + * code on failure > + */ > +static int libxevd_decode(struct AVCodecContext *avctx, struct AVFrame *frame, int *got_frame_ptr, AVPacket *avpkt) > +{ > + XevdContext *xectx = NULL; > + XEVD_IMGB *imgb = NULL; > + XEVD_STAT stat; > + XEVD_BITB bitb; > + int xevd_ret, nalu_size, bs_read_pos; > + int ret = 0; > + > + xectx = avctx->priv_data; > + > + if (avpkt->size > 0) { > + bs_read_pos = 0; > + imgb = NULL; > + while(avpkt->size > (bs_read_pos + XEVD_NAL_UNIT_LENGTH_BYTE)) { > + memset(&stat, 0, sizeof(XEVD_STAT)); > + memset(&bitb, 0, sizeof(XEVD_BITB)); > + > + nalu_size = read_nal_unit_length(avpkt->data + bs_read_pos, XEVD_NAL_UNIT_LENGTH_BYTE, avctx); > + if (nalu_size == 0) { > + av_log(avctx, AV_LOG_ERROR, "Invalid bitstream\n"); > + ret = AVERROR_INVALIDDATA; > + goto ERR; > + } > + bs_read_pos += XEVD_NAL_UNIT_LENGTH_BYTE; > + > + bitb.addr = avpkt->data + bs_read_pos; > + bitb.ssize = nalu_size; > + > + /* main decoding block */ > + xevd_ret = xevd_decode(xectx->id, &bitb, &stat); > + if (XEVD_FAILED(xevd_ret)) { > + av_log(avctx, AV_LOG_ERROR, "Failed to decode bitstream\n"); > + ret = AVERROR_EXTERNAL; > + goto ERR; > + } > + > + bs_read_pos += nalu_size; > + > + if (stat.nalu_type == XEVD_NUT_SPS) { // EVC stream parameters changed > + if ((ret = export_stream_params(xectx, avctx)) != 0) > + goto ERR; > + } > + > + if (stat.read != nalu_size) > + av_log(avctx, AV_LOG_INFO, "Different reading of bitstream (in:%d, read:%d)\n,", nalu_size, stat.read); > + if (stat.fnum >= 0) { > + // already has a decoded image > + if (imgb) { > + // xevd_pull uses pool of objects of type XEVD_IMGB. > + // The pool size is equal MAX_PB_SIZE (26), so release object when it is no more needed > + imgb->release(imgb); > + imgb = NULL; > + } > + xevd_ret = xevd_pull(xectx->id, &imgb); > + if (XEVD_FAILED(xevd_ret)) { > + av_log(avctx, AV_LOG_ERROR, "Failed to pull the decoded image (xevd error code: %d, frame#=%d)\n", xevd_ret, stat.fnum); > + ret = AVERROR_EXTERNAL; > + goto ERR; > + } else if (xevd_ret == XEVD_OK_FRM_DELAYED) { > + if (imgb) { > + // xevd_pull uses pool of objects of type XEVD_IMGB. > + // The pool size is equal MAX_PB_SIZE (26), so release object when it is no more needed > + imgb->release(imgb); > + imgb = NULL; > + } > + } > + if (imgb) { > + int ret = libxevd_image_copy(avctx, imgb, frame); > + if(ret < 0) { > + goto ERR; > + } > + > + frame->pts = avpkt->pts; > + *got_frame_ptr = 1; > + > + // xevd_pull uses pool of objects of type XEVD_IMGB. > + // The pool size is equal MAX_PB_SIZE (26), so release object when it is no more needed > + imgb->release(imgb); > + imgb = NULL; > + } else > + *got_frame_ptr = 0; > + } > + } > + } else { // bumping > + xevd_ret = xevd_pull(xectx->id, &(imgb)); > + if (xevd_ret == XEVD_ERR_UNEXPECTED) { // bumping process completed > + *got_frame_ptr = 0; > + return 0; > + } else if (XEVD_FAILED(xevd_ret)) { > + av_log(avctx, AV_LOG_ERROR, "Failed to pull the decoded image (xevd error code: %d)\n", xevd_ret); > + ret = AVERROR_EXTERNAL; > + goto ERR; > + } > + if (imgb) { > + int ret = libxevd_image_copy(avctx, imgb, frame); > + if(ret < 0) { > + goto ERR; > + } > + > + frame->pts = avpkt->pts; > + *got_frame_ptr = 1; > + > + // xevd_pull uses pool of objects of type XEVD_IMGB. > + // The pool size is equal MAX_PB_SIZE (26), so release object when it is no more needed > + imgb->release(imgb); > + imgb = NULL; > + } else > + *got_frame_ptr = 0; > + } > + > + return avpkt->size; > + > +ERR: > + if (imgb) { > + imgb->release(imgb); > + imgb = NULL; > + } > + *got_frame_ptr = 0; > + > + return ret; > +} > + > +/** > + * Destroy decoder > + * > + * @param avctx codec context > + * @return 0 on success > + */ > +static av_cold int libxevd_close(AVCodecContext *avctx) > +{ > + XevdContext *xectx = avctx->priv_data; > + if (xectx->id) { > + xevd_delete(xectx->id); > + xectx->id = NULL; > + } > + > + return 0; > +} > + > +#define OFFSET(x) offsetof(XevdContext, x) > +#define VD AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_DECODING_PARAM > + > +static const AVClass libxevd_class = { > + .class_name = "libxevd", > + .item_name = av_default_item_name, > + .version = LIBAVUTIL_VERSION_INT, > +}; > + > +const FFCodec ff_libxevd_decoder = { > + .p.name = "evc", > + .p.long_name = NULL_IF_CONFIG_SMALL("EVC / MPEG-5 Essential Video Coding (EVC)"), > + .p.type = AVMEDIA_TYPE_VIDEO, > + .p.id = AV_CODEC_ID_EVC, > + .init = libxevd_init, > + FF_CODEC_DECODE_CB(libxevd_decode), > + .close = libxevd_close, > + .priv_data_size = sizeof(XevdContext), > + .p.priv_class = &libxevd_class, > + .p.capabilities = AV_CODEC_CAP_DELAY | AV_CODEC_CAP_OTHER_THREADS | AV_CODEC_CAP_AVOID_PROBING | AV_CODEC_CAP_DR1, > + .p.wrapper_name = "libxevd", > +}; > diff --git a/libavcodec/libxeve.c b/libavcodec/libxeve.c > new file mode 100644 > index 0000000000..4808e6d853 > --- /dev/null > +++ b/libavcodec/libxeve.c > @@ -0,0 +1,597 @@ > +/* > + * libxeve encoder > + * EVC (MPEG-5 Essential Video Coding) encoding using XEVE MPEG-5 EVC encoder library > + * > + * Copyright (C) 2021 Dawid Kozinski <d.kozinski@samsung.com> > + * > + * This file is part of FFmpeg. > + * > + * FFmpeg is free software; you can redistribute it and/or > + * modify it under the terms of the GNU Lesser General Public > + * License as published by the Free Software Foundation; either > + * version 2.1 of the License, or (at your option) any later version. > + * > + * FFmpeg is distributed in the hope that it will be useful, > + * but WITHOUT ANY WARRANTY; without even the implied warranty of > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > + * Lesser General Public License for more details. > + * > + * You should have received a copy of the GNU Lesser General Public > + * License along with FFmpeg; if not, write to the Free Software > + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA > + */ > + > +#include <float.h> > +#include <stdlib.h> > + > +#include <xeve.h> > + > +#include "libavutil/internal.h" > +#include "libavutil/common.h" > +#include "libavutil/opt.h" > +#include "libavutil/pixdesc.h" > +#include "libavutil/pixfmt.h" > +#include "libavutil/time.h" > +#include "libavutil/cpu.h" > +#include "libavutil/avstring.h" > + > +#include "avcodec.h" > +#include "internal.h" > +#include "packet_internal.h" > +#include "codec_internal.h" > +#include "encode.h" > + > +#define MAX_BS_BUF (16*1024*1024) > + > +/** > + * Error codes > + */ > +#define XEVE_PARAM_BAD_NAME -100 > +#define XEVE_PARAM_BAD_VALUE -200 > + > +/** > + * Encoder states > + * > + * STATE_ENCODING - the encoder receives and processes input frames > + * STATE_BUMPING - there are no more input frames, however the encoder still processes previously received data > + */ > +typedef enum State { > + STATE_ENCODING, > + STATE_BUMPING, > +} State; > + > +/** > + * The structure stores all the states associated with the instance of Xeve MPEG-5 EVC encoder > + */ > +typedef struct XeveContext { > + const AVClass *class; > + > + XEVE id; // XEVE instance identifier > + XEVE_CDSC cdsc; // coding parameters i.e profile, width & height of input frame, num of therads, frame rate ... > + XEVE_BITB bitb; // bitstream buffer (output) > + XEVE_STAT stat; // encoding status (output) > + XEVE_IMGB imgb; // image buffer (input) > + > + State state; // encoder state (skipping, encoding, bumping) > + > + int profile_id; // encoder profile (main, baseline) > + int preset_id; // preset of xeve ( fast, medium, slow, placebo) > + int tune_id; // tune of xeve (psnr, zerolatency) > + > + // variables for rate control modes > + int rc_mode; // Rate control mode [ 0(CQP) / 1(ABR) / 2(CRF) ] > + int qp; // quantization parameter (QP) [0,51] > + int crf; // constant rate factor (CRF) [10,49] > + > + int hash; // embed picture signature (HASH) for conformance checking in decoding > + int sei_info; // embed Supplemental enhancement information while encoding > + > + int color_format; // input data color format: currently only XEVE_CF_YCBCR420 is supported > +} XeveContext; > + > +/** > + * Convert FFmpeg pixel format (AVPixelFormat) to XEVE pre-defined color format > + * > + * @param[in] av_pix_fmt pixel format (@see https://ffmpeg.org/doxygen/trunk/pixfmt_8h.html#a9a8e335cf3be472042bc9f0cf80cd4c5) > + * @param[out] xeve_col_fmt XEVE pre-defined color format (@see xeve.h) > + * > + * @return 0 on success, negative value on failure > + */ > +static int libxeve_color_fmt(enum AVPixelFormat av_pix_fmt, int *xeve_col_fmt) > +{ > + switch (av_pix_fmt) { > + case AV_PIX_FMT_YUV420P: > + *xeve_col_fmt = XEVE_CF_YCBCR420; > + break; > + case AV_PIX_FMT_YUV420P10: > + *xeve_col_fmt = XEVE_CF_YCBCR420; > + break; > + default: > + *xeve_col_fmt = XEVE_CF_UNKNOWN; > + return AVERROR_INVALIDDATA; > + } > + > + return 0; > +} > + > +/** > + * Convert FFmpeg pixel format (AVPixelFormat) into XEVE pre-defined color space > + * > + * @param[in] px_fmt pixel format (@see https://ffmpeg.org/doxygen/trunk/pixfmt_8h.html#a9a8e335cf3be472042bc9f0cf80cd4c5) > + * > + * @return XEVE pre-defined color space (@see xeve.h) on success, XEVE_CF_UNKNOWN on failure > + */ > +static int libxeve_color_space(enum AVPixelFormat av_pix_fmt) > +{ > + /* color space of input image */ > + int cs = XEVE_CF_UNKNOWN; > + > + switch (av_pix_fmt) { > + case AV_PIX_FMT_YUV420P: > + cs = XEVE_CS_YCBCR420; > + break; > + case AV_PIX_FMT_YUV420P10: > +#if AV_HAVE_BIGENDIAN > + cs = XEVE_CS_SET(XEVE_CF_YCBCR420, 10, 1); > +#else > + cs = XEVE_CS_YCBCR420_10LE; > +#endif > + > + break; > + default: > + cs = XEVE_CF_UNKNOWN; > + break; > + } > + > + return cs; > +} > + > +/** > + * The function returns a pointer to the object of the XEVE_CDSC type. > + * XEVE_CDSC contains all encoder parameters that should be initialized before the encoder is used. > + * > + * The field values of the XEVE_CDSC structure are populated based on: > + * - the corresponding field values of the AvCodecConetxt structure, > + * - the xeve encoder specific option values, > + * (the full list of options available for xeve encoder is displayed after executing the command ./ffmpeg --help encoder = libxeve) > + * - and the xeve encoder options specified as a list of key-value pairs following the xeve-params option > + * > + * The order of processing input data and populating the XEVE_CDSC structure > + * 1) first, the fields of the AVCodecContext structure corresponding to the provided input options are processed, > + * (i.e -pix_fmt yuv420p -s:v 1920x1080 -r 30 -profile:v 0) > + * 2) then xeve-specific options added as AVOption to the xeve AVCodec implementation > + * (i.e -preset 0) > + * > + * Keep in mind that, there are options that can be set in different ways. > + * In this case, please follow the above-mentioned order of processing. > + * The most recent assignments overwrite the previous values. > + * > + * @param[in] avctx codec context (AVCodecContext) > + * @param[out] cdsc contains all Xeve MPEG-5 EVC encoder encoder parameters that should be initialized before the encoder is use > + * > + * @return 0 on success, negative error code on failure > + */ > +static int get_conf(AVCodecContext *avctx, XEVE_CDSC *cdsc) > +{ > + XeveContext *xectx = NULL; > + int ret; > + > + xectx = avctx->priv_data; > + > + /* initialize xeve_param struct with default values */ > + ret = xeve_param_default(&cdsc->param); > + if (XEVE_FAILED(ret)) { > + av_log(avctx, AV_LOG_ERROR, "Cannot set_default parameter\n"); > + return AVERROR_EXTERNAL; > + } > + > + /* read options from AVCodecContext */ > + if (avctx->width > 0) > + cdsc->param.w = avctx->width; > + > + if (avctx->height > 0) > + cdsc->param.h = avctx->height; > + > + if (avctx->framerate.num > 0) { > + // fps can be float number, but xeve API doesn't support it > + cdsc->param.fps = (int)(((float)avctx->framerate.num / avctx->framerate.den) + 0.5); > + } > + > + if (avctx->gop_size >= 0) // GOP size (key-frame interval, I-picture period) > + cdsc->param.keyint = avctx->gop_size; // 0: only one I-frame at the first time; 1: every frame is coded in I-frame > + > + if (avctx->max_b_frames == 0 || avctx->max_b_frames == 1 || avctx->max_b_frames == 3 || > + avctx->max_b_frames == 7 || avctx->max_b_frames == 15) // number of b-frames > + cdsc->param.bframes = avctx->max_b_frames; > + else { > + av_log(avctx, AV_LOG_ERROR, "Incorrect value for maximum number of B frames: (%d) \n" > + "Acceptable values for bf option (maximum number of B frames) are 0,1,3,7 or 15\n", avctx->max_b_frames); > + return AVERROR_INVALIDDATA; > + } > + > + if (avctx->level >= 0) > + cdsc->param.level_idc = avctx->level; > + > + if (avctx->rc_buffer_size) // VBV buf size > + cdsc->param.vbv_bufsize = (int)(avctx->rc_buffer_size / 1000); > + > + cdsc->param.rc_type = xectx->rc_mode; > + > + if (xectx->rc_mode == XEVE_RC_CQP) { > + cdsc->param.qp = xectx->qp; > + } else if (xectx->rc_mode == XEVE_RC_ABR) { > + if (avctx->bit_rate / 1000 > INT_MAX || avctx->rc_max_rate / 1000 > INT_MAX) { > + av_log(avctx, AV_LOG_ERROR, "Not supported bitrate bit_rate and rc_max_rate > %d000\n", INT_MAX); > + return AVERROR_INVALIDDATA; > + } > + cdsc->param.bitrate = (int)(avctx->bit_rate / 1000); > + } else if (xectx->rc_mode == XEVE_RC_CRF) { > + cdsc->param.crf = xectx->crf; > + } else { > + av_log(avctx, AV_LOG_ERROR, "Not supported rate control type: %d\n", xectx->rc_mode); > + return AVERROR_INVALIDDATA; > + } > + > + if (avctx->thread_count <= 0) { > + int cpu_count = av_cpu_count(); > + cdsc->param.threads = (cpu_count < XEVE_MAX_THREADS) ? cpu_count : XEVE_MAX_THREADS; > + } else if (avctx->thread_count > XEVE_MAX_THREADS) > + cdsc->param.threads = XEVE_MAX_THREADS; > + else > + cdsc->param.threads = avctx->thread_count; > + > + > + libxeve_color_fmt(avctx->pix_fmt, &xectx->color_format); > + > + cdsc->param.cs = XEVE_CS_SET(xectx->color_format, cdsc->param.codec_bit_depth, AV_HAVE_BIGENDIAN); > + > + cdsc->max_bs_buf_size = MAX_BS_BUF; > + > + ret = xeve_param_ppt(&cdsc->param, xectx->profile_id, xectx->preset_id, xectx->tune_id); > + if (XEVE_FAILED(ret)) { > + av_log(avctx, AV_LOG_ERROR, "Cannot set profile(%d), preset(%d), tune(%d)\n", xectx->profile_id, xectx->preset_id, xectx->tune_id); > + return AVERROR_EXTERNAL; > + } > + > + return 0; > +} > + > +/** > + * Set XEVE_CFG_SET_USE_PIC_SIGNATURE for encoder > + * > + * @param[in] logger context > + * @param[in] id XEVE encodec instance identifier > + * @param[in] ctx the structure stores all the states associated with the instance of Xeve MPEG-5 EVC encoder > + * > + * @return 0 on success, negative error code on failure > + */ > +static int set_extra_config(AVCodecContext *avctx, XEVE id, XeveContext *ctx) > +{ > + int ret, size; > + size = 4; > + > + // embed SEI messages identifying encoder parameters and command line arguments > + // - 0: off\n" > + // - 1: emit sei info" > + // > + // SEI - Supplemental enhancement information contains information > + // that is not necessary to decode the samples of coded pictures from VCL NAL units. > + // Some SEI message information is required to check bitstream conformance > + // and for output timing decoder conformance. > + // @see ISO_IEC_23094-1_2020 7.4.3.5 > + // @see ISO_IEC_23094-1_2020 Annex D > + ret = xeve_config(id, XEVE_CFG_SET_SEI_CMD, &ctx->sei_info, &size); // sei_cmd_info > + if (XEVE_FAILED(ret)) { > + av_log(avctx, AV_LOG_ERROR, "Failed to set config for sei command info messages\n"); > + return AVERROR_EXTERNAL; > + } > + > + ret = xeve_config(id, XEVE_CFG_SET_USE_PIC_SIGNATURE, &ctx->hash, &size); > + if (XEVE_FAILED(ret)) { > + av_log(avctx, AV_LOG_ERROR, "Failed to set config for picture signature\n"); > + return AVERROR_EXTERNAL; > + } > + > + return 0; > +} > + > +/** > + * @brief Switch encoder to bumping mode > + * > + * @param id XEVE encodec instance identifier > + * @return 0 on success, negative error code on failure > + */ > +static int setup_bumping(XEVE id) > +{ > + int val = 1; > + int size = sizeof(int); > + if (XEVE_FAILED(xeve_config(id, XEVE_CFG_SET_FORCE_OUT, (void *)(&val), &size))) > + return AVERROR_EXTERNAL; > + > + return 0; > +} > + > +/** > + * @brief Initialize eXtra-fast Essential Video Encoder codec > + * Create an encoder instance and allocate all the needed resources > + * > + * @param avctx codec context > + * @return 0 on success, negative error code on failure > + */ > +static av_cold int libxeve_init(AVCodecContext *avctx) > +{ > + XeveContext *xectx = avctx->priv_data; > + unsigned char *bs_buf = NULL; > + int i; > + int shift_h = 0; > + int shift_v = 0; > + int width_chroma = 0; > + int height_chroma = 0; > + XEVE_IMGB *imgb = NULL; > + int ret = 0; > + > + XEVE_CDSC *cdsc = &(xectx->cdsc); > + > + /* allocate bitstream buffer */ > + bs_buf = av_malloc(MAX_BS_BUF); > + if (bs_buf == NULL) { > + av_log(avctx, AV_LOG_ERROR, "Cannot allocate bitstream buffer\n"); > + return AVERROR(ENOMEM); > + } > + xectx->bitb.addr = bs_buf; > + xectx->bitb.bsize = MAX_BS_BUF; > + > + /* read configurations and set values for created descriptor (XEVE_CDSC) */ > + if ((ret = get_conf(avctx, cdsc)) != 0) { > + av_log(avctx, AV_LOG_ERROR, "Cannot get configuration\n"); > + return AVERROR(EINVAL); > + } > + > + if ((ret = xeve_param_check(&cdsc->param)) != 0) { > + av_log(avctx, AV_LOG_ERROR, "Invalid configuration\n"); > + return AVERROR(EINVAL); > + } > + > + /* create encoder */ > + xectx->id = xeve_create(cdsc, NULL); > + if (xectx->id == NULL) { > + av_log(avctx, AV_LOG_ERROR, "Cannot create XEVE encoder\n"); > + return AVERROR_EXTERNAL; > + } > + > + if ((ret = set_extra_config(avctx, xectx->id, xectx)) != 0) { > + av_log(avctx, AV_LOG_ERROR, "Cannot set extra configuration\n"); > + return AVERROR(EINVAL); > + } > + > + if ((ret = av_pix_fmt_get_chroma_sub_sample(avctx->pix_fmt, &shift_h, &shift_v)) != 0) { > + av_log(avctx, AV_LOG_ERROR, "Failed to get chroma shift\n"); > + return AVERROR(EINVAL); > + } > + > + // Chroma subsampling > + // > + // YUV format explanation > + // shift_h == 1 && shift_v == 1 : YUV420 > + // shift_h == 1 && shift_v == 0 : YUV422 > + // shift_h == 0 && shift_v == 0 : YUV444 > + // > + width_chroma = AV_CEIL_RSHIFT(avctx->width, shift_h); > + height_chroma = AV_CEIL_RSHIFT(avctx->height, shift_v); > + > + /* set default values for input image buffer */ > + imgb = &xectx->imgb; > + imgb->cs = libxeve_color_space(avctx->pix_fmt); > + imgb->np = 3; /* only for yuv420p, yuv420ple */ > + > + for (i = 0; i < imgb->np; i++) > + imgb->x[i] = imgb->y[i] = 0; > + > + imgb->w[0] = imgb->aw[0] = avctx->width; // width luma > + imgb->w[1] = imgb->w[2] = imgb->aw[1] = imgb->aw[2] = width_chroma; > + imgb->h[0] = imgb->ah[0] = avctx->height; // height luma > + imgb->h[1] = imgb->h[2] = imgb->ah[1] = imgb->ah[2] = height_chroma; > + > + xectx->state = STATE_ENCODING; > + > + return 0; > +} > + > +/** > + * Encode raw data frame into EVC packet > + * > + * @param[in] avctx codec context > + * @param[out] pkt output AVPacket containing encoded data > + * @param[in] frame AVFrame containing the raw data to be encoded > + * @param[out] got_packet encoder sets to 0 or 1 to indicate that a > + * non-empty packet was returned in pkt > + * > + * @return 0 on success, negative error code on failure > + */ > +static int libxeve_encode(AVCodecContext *avctx, AVPacket *avpkt, > + const AVFrame *frame, int *got_packet) > +{ > + XeveContext *xectx = avctx->priv_data; > + int ret = -1; > + > + // No more input frames are available but encoder still can have some data in its internal buffer to process > + // and some frames to dump. > + if (xectx->state == STATE_ENCODING && frame == NULL) { > + if (setup_bumping(xectx->id) == 0) > + xectx->state = STATE_BUMPING; // Entering bumping process > + else { > + av_log(avctx, AV_LOG_ERROR, "Failed to setup bumping\n"); > + return 0; > + } > + } > + > + if (xectx->state == STATE_ENCODING) { > + int i; > + XEVE_IMGB *imgb = NULL; > + > + imgb = &xectx->imgb; > + > + for (i = 0; i < imgb->np; i++) { > + imgb->a[i] = frame->data[i]; > + imgb->s[i] = frame->linesize[i]; > + } > + > + imgb->ts[0] = frame->pts; > + imgb->ts[1] = 0; > + > + /* push image to encoder */ > + ret = xeve_push(xectx->id, imgb); > + if (XEVE_FAILED(ret)) { > + av_log(avctx, AV_LOG_ERROR, "xeve_push() failed\n"); > + return AVERROR_EXTERNAL; > + } > + } > + if (xectx->state == STATE_ENCODING || xectx->state == STATE_BUMPING) { > + /* encoding */ > + ret = xeve_encode(xectx->id, &(xectx->bitb), &(xectx->stat)); > + if (XEVE_FAILED(ret)) { > + av_log(avctx, AV_LOG_ERROR, "xeve_encode() failed\n"); > + return AVERROR_EXTERNAL; > + } > + > + /* store bitstream */ > + if (ret == XEVE_OK_OUT_NOT_AVAILABLE) { // Return OK but picture is not available yet > + *got_packet = 0; > + return 0; > + } else if (ret == XEVE_OK) { > + int av_pic_type; > + > + if (xectx->stat.write > 0) { > + > + ret = ff_get_encode_buffer(avctx, avpkt, xectx->stat.write, 0); > + if (ret < 0) > + return ret; > + > + memcpy(avpkt->data, xectx->bitb.addr, xectx->stat.write); > + > + avpkt->pts = xectx->bitb.ts[0]; > + avpkt->dts = xectx->bitb.ts[1]; > + > + switch(xectx->stat.stype) { > + case XEVE_ST_I: > + av_pic_type = AV_PICTURE_TYPE_I; > + avpkt->flags |= AV_PKT_FLAG_KEY; > + break; > + case XEVE_ST_P: > + av_pic_type = AV_PICTURE_TYPE_P; > + break; > + case XEVE_ST_B: > + av_pic_type = AV_PICTURE_TYPE_B; > + break; > + case XEVE_ST_UNKNOWN: > + av_log(avctx, AV_LOG_ERROR, "Unknown slice type\n"); > + return AVERROR_INVALIDDATA; > + } > + > + ff_side_data_set_encoder_stats(avpkt, xectx->stat.qp * FF_QP2LAMBDA, NULL, 0, av_pic_type); > + > + *got_packet = 1; > + } > + } else if (ret == XEVE_OK_NO_MORE_FRM) { > + // Return OK but no more frames > + return 0; > + } else { > + av_log(avctx, AV_LOG_ERROR, "Invalid return value: %d\n", ret); > + return AVERROR_EXTERNAL; > + } > + } else { > + av_log(avctx, AV_LOG_ERROR, "Udefined encoder state\n"); > + return AVERROR_INVALIDDATA; > + } > + > + return 0; > +} > + > +/** > + * Destroy the encoder and release all the allocated resources > + * > + * @param avctx codec context > + * @return 0 on success, negative error code on failure > + */ > +static av_cold int libxeve_close(AVCodecContext *avctx) > +{ > + XeveContext *xectx = avctx->priv_data; > + > + xeve_delete(xectx->id); You intend to set the init-cleanup flag on this encoder. But then you have to account for the possibility of this function being called with a XeveContext that has not been successfully setup, i.e. xectx->id might be NULL. Is this allowed? > + > + av_free(xectx->bitb.addr); /* release bitstream buffer */ > + > + return 0; > +} > + > +#define OFFSET(x) offsetof(XeveContext, x) > +#define VE AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_ENCODING_PARAM > + > +static const enum AVPixelFormat supported_pixel_formats[] = { > + AV_PIX_FMT_YUV420P, > + AV_PIX_FMT_YUV420P10, > + AV_PIX_FMT_NONE > +}; > + > +// Example of using: ./ffmpeg -xeve-params "m=2:q=17" > +// Consider using following options (./ffmpeg --help encoder=libxeve) > +// > +static const AVOption libxeve_options[] = { > + { "preset", "Encoding preset for setting encoding speed", OFFSET(preset_id), AV_OPT_TYPE_INT, { .i64 = XEVE_PRESET_MEDIUM }, XEVE_PRESET_DEFAULT, XEVE_PRESET_PLACEBO, VE, "preset" }, > + { "default", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_PRESET_DEFAULT }, INT_MIN, INT_MAX, VE, "preset" }, > + { "fast", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_PRESET_FAST }, INT_MIN, INT_MAX, VE, "preset" }, > + { "medium", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_PRESET_MEDIUM }, INT_MIN, INT_MAX, VE, "preset" }, > + { "slow", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_PRESET_SLOW }, INT_MIN, INT_MAX, VE, "preset" }, > + { "placebo", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_PRESET_PLACEBO }, INT_MIN, INT_MAX, VE, "preset" }, > + { "tune", "Tuning parameter for special purpose operation", OFFSET(tune_id), AV_OPT_TYPE_INT, { .i64 = XEVE_TUNE_NONE }, XEVE_TUNE_NONE, XEVE_TUNE_PSNR, VE, "tune"}, > + { "none", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_TUNE_NONE }, INT_MIN, INT_MAX, VE, "tune" }, > + { "zerolatency", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_TUNE_ZEROLATENCY }, INT_MIN, INT_MAX, VE, "tune" }, > + { "psnr", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_TUNE_PSNR }, INT_MIN, INT_MAX, VE, "tune" }, > + { "profile", "Encoding profile", OFFSET(profile_id), AV_OPT_TYPE_INT, { .i64 = XEVE_PROFILE_BASELINE }, XEVE_PROFILE_BASELINE, XEVE_PROFILE_MAIN, VE, "profile" }, > + { "baseline", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_PROFILE_BASELINE }, INT_MIN, INT_MAX, VE, "profile" }, > + { "main", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_PROFILE_MAIN }, INT_MIN, INT_MAX, VE, "profile" }, > + { "rc_mode", "Rate control mode", OFFSET(rc_mode), AV_OPT_TYPE_INT, { .i64 = XEVE_RC_CQP }, XEVE_RC_CQP, XEVE_RC_CRF, VE, "rc_mode" }, > + { "CQP", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_RC_CQP }, INT_MIN, INT_MAX, VE, "rc_mode" }, > + { "ABR", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_RC_ABR }, INT_MIN, INT_MAX, VE, "rc_mode" }, > + { "CRF", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_RC_CRF }, INT_MIN, INT_MAX, VE, "rc_mode" }, > + { "qp", "Quantization parameter value for CQP rate control mode", OFFSET(qp), AV_OPT_TYPE_INT, { .i64 = 32 }, 0, 51, VE }, > + { "crf", "Constant rate factor value for CRF rate control mode", OFFSET(crf), AV_OPT_TYPE_INT, { .i64 = 32 }, 10, 49, VE }, > + { "hash", "Embed picture signature (HASH) for conformance checking in decoding", OFFSET(hash), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, 1, VE }, > + { "sei_info", "Embed SEI messages identifying encoder parameters and command line arguments", OFFSET(sei_info), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, 1, VE }, > + { NULL } > +}; > + > +static const AVClass libxeve_class = { > + .class_name = "libxeve", > + .item_name = av_default_item_name, > + .option = libxeve_options, > + .version = LIBAVUTIL_VERSION_INT, > +}; > + > +/** > + * libavcodec generic global options, which can be set on all the encoders and decoders > + * @see https://www.ffmpeg.org/ffmpeg-codecs.html#Codec-Options > + */ > +static const FFCodecDefault libxeve_defaults[] = { > + { "b", "0" }, // bitrate in terms of kilo-bits per second > + { "g", "0" }, // gop_size (key-frame interval 0: only one I-frame at the first time; 1: every frame is coded in I-frame) > + { "bf", "15"}, // maximum number of B frames (0: no B-frames, 1,3,7,15) > + { "threads", "0"}, // number of threads to be used (0: automatically select the number of threads to set) > + { NULL }, > +}; > + > +const FFCodec ff_libxeve_encoder = { > + .p.name = "libxeve", > + .p.long_name = NULL_IF_CONFIG_SMALL("libxeve MPEG-5 EVC"), > + .p.type = AVMEDIA_TYPE_VIDEO, > + .p.id = AV_CODEC_ID_EVC, > + .init = libxeve_init, > + FF_CODEC_ENCODE_CB(libxeve_encode), > + .close = libxeve_close, > + .priv_data_size = sizeof(XeveContext), > + .p.priv_class = &libxeve_class, > + .defaults = libxeve_defaults, > + .p.capabilities = FF_CODEC_CAP_INIT_CLEANUP | AV_CODEC_CAP_DELAY | AV_CODEC_CAP_AUTO_THREADS | AV_CODEC_CAP_DR1, 1. This is completely wring: FF_CODEC_CAP_INIT_CLEANUP is an internal capability flag, not something public. That's why it has an FF_ prefix. Internal capabilities should go into FFCodec.caps_internal. (In other words: s/.p.capabilities/.caps_internal/) 2. Due to 21b23ceab3cfcc2aa0a833bbff11cf38b2177800, a codec that has not the FF_CODEC_CAP_NOT_INIT_THREADSAFE internal cap set is presumed to be init-threadsafe (this changed since the last iteration of your patchset). Is it guaranteed and documented that libxeve and libxevd satisfy are init-threadsafe (i.e. there are no races if multiple encoders/decoders (each with their own context) are initialized concurrently)? > + .p.wrapper_name = "libxeve", > + .p.pix_fmts = supported_pixel_formats, > +}; > diff --git a/libavcodec/parsers.c b/libavcodec/parsers.c > index a8d52af6cb..ca00c63176 100644 > --- a/libavcodec/parsers.c > +++ b/libavcodec/parsers.c > @@ -41,6 +41,7 @@ extern const AVCodecParser ff_dvaudio_parser; > extern const AVCodecParser ff_dvbsub_parser; > extern const AVCodecParser ff_dvdsub_parser; > extern const AVCodecParser ff_dvd_nav_parser; > +extern const AVCodecParser ff_evc_parser; > extern const AVCodecParser ff_flac_parser; > extern const AVCodecParser ff_g723_1_parser; > extern const AVCodecParser ff_g729_parser; > diff --git a/libavcodec/profiles.c b/libavcodec/profiles.c > index 7af7fbeb13..a31244e0db 100644 > --- a/libavcodec/profiles.c > +++ b/libavcodec/profiles.c > @@ -181,4 +181,10 @@ const AVProfile ff_arib_caption_profiles[] = { > { FF_PROFILE_UNKNOWN } > }; > > +const AVProfile ff_evc_profiles[] = { > + { FF_PROFILE_EVC_BASELINE, "Baseline" }, > + { FF_PROFILE_EVC_MAIN, "Main" }, > + { FF_PROFILE_UNKNOWN }, > +}; > + > #endif /* !CONFIG_SMALL */ > diff --git a/libavcodec/profiles.h b/libavcodec/profiles.h > index 41a19aa9ad..cf92b5f126 100644 > --- a/libavcodec/profiles.h > +++ b/libavcodec/profiles.h > @@ -72,5 +72,6 @@ extern const AVProfile ff_sbc_profiles[]; > extern const AVProfile ff_prores_profiles[]; > extern const AVProfile ff_mjpeg_profiles[]; > extern const AVProfile ff_arib_caption_profiles[]; > +extern const AVProfile ff_evc_profiles[]; > > #endif /* AVCODEC_PROFILES_H */ > diff --git a/libavcodec/version.h b/libavcodec/version.h > index e488eee355..2328be4b26 100644 > --- a/libavcodec/version.h > +++ b/libavcodec/version.h > @@ -29,7 +29,7 @@ > > #include "version_major.h" > > -#define LIBAVCODEC_VERSION_MINOR 42 > +#define LIBAVCODEC_VERSION_MINOR 43 > #define LIBAVCODEC_VERSION_MICRO 100 > > #define LIBAVCODEC_VERSION_INT AV_VERSION_INT(LIBAVCODEC_VERSION_MAJOR, \ _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [FFmpeg-devel] [PATCH 1/2] Provided support for MPEG-5 EVC (Essential Video Coding) codec 2022-08-11 7:44 ` Andreas Rheinhardt @ 2022-08-11 11:28 ` Dawid Kozinski/Robot SDK (PLT) /SRPOL/Staff Engineer/삼성전자 2022-08-18 9:00 ` Dawid Kozinski/Robot SDK (PLT) /SRPOL/Staff Engineer/삼성전자 1 sibling, 0 replies; 13+ messages in thread From: Dawid Kozinski/Robot SDK (PLT) /SRPOL/Staff Engineer/삼성전자 @ 2022-08-11 11:28 UTC (permalink / raw) To: 'FFmpeg development discussions and patches' Hi Andreas, Thank you for your prompt response and code review. Please find below my comments, just right next to yours. Today, I'll prepare new patches and submit them to the mailing list. -----Original Message----- From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Andreas Rheinhardt Sent: Thursday, August 11, 2022 9:45 AM To: ffmpeg-devel@ffmpeg.org Subject: Re: [FFmpeg-devel] [PATCH 1/2] Provided support for MPEG-5 EVC (Essential Video Coding) codec Dawid Kozinski: > - Added new entry to codec IDs list > - Added new entry to the codec descriptor list > - Bumped libavcodec minor version > - Changes in Changelog and MAINTAINERS files > - Added xeve encoder wrapper > - Added xevd dencoder wrapper > - Added documentation for xeve and xevd wrappers > - Added parser for EVC format > - Changes in project configuration file and libavcodec Makefile > > Signed-off-by: Dawid Kozinski <d.kozinski@samsung.com> > --- > Changelog | 3 +- > MAINTAINERS | 2 + > configure | 8 + > doc/decoders.texi | 24 ++ > doc/encoders.texi | 92 +++++ > doc/general_contents.texi | 19 + > libavcodec/Makefile | 3 + > libavcodec/allcodecs.c | 2 + > libavcodec/avcodec.h | 3 + > libavcodec/codec_desc.c | 8 + > libavcodec/codec_id.h | 1 + > libavcodec/evc_parser.c | 758 ++++++++++++++++++++++++++++++++++++++ > libavcodec/libxevd.c | 419 +++++++++++++++++++++ > libavcodec/libxeve.c | 597 ++++++++++++++++++++++++++++++ > libavcodec/parsers.c | 1 + > libavcodec/profiles.c | 6 + > libavcodec/profiles.h | 1 + > libavcodec/version.h | 2 +- > 18 files changed, 1947 insertions(+), 2 deletions(-) > create mode 100644 libavcodec/evc_parser.c > create mode 100644 libavcodec/libxevd.c > create mode 100644 libavcodec/libxeve.c > > diff --git a/Changelog b/Changelog > index cc271c22bd..dbeb08c373 100644 > --- a/Changelog > +++ b/Changelog > @@ -82,7 +82,8 @@ version 5.0: > - VideoToolbox ProRes encoder > - anlmf audio filter > - IMF demuxer (experimental) > - > +- eXtra-fast Essential Video Encoder (XEVE) > +- eXtra-fast Essential Video Decoder (XEVD) > > version 4.4: > - AudioToolbox output device > diff --git a/MAINTAINERS b/MAINTAINERS > index 7ed15f96f6..7d4d543a13 100644 > --- a/MAINTAINERS > +++ b/MAINTAINERS > @@ -206,6 +206,7 @@ Codecs: > libvpx* James Zern > libxavs.c Stefan Gehrer > libxavs2.c Huiwen Ren > + libxev*.c, evc_parser.c Dawid Kozinski > libzvbi-teletextdec.c Marton Balint > lzo.h, lzo.c Reimar Doeffinger > mdec.c Michael Niedermayer > @@ -426,6 +427,7 @@ Muxers/Demuxers: > dv.c Roman Shaposhnik > electronicarts.c Peter Ross > epafdec.c Paul B Mahol > + evcdec.c Dawid Kozinski > ffm* Baptiste Coudurier > flic.c Mike Melanson > flvdec.c Michael Niedermayer > diff --git a/configure b/configure > index cbbb4dd9c8..e6582eae91 100755 > --- a/configure > +++ b/configure > @@ -291,6 +291,8 @@ External library support: > --enable-libwebp enable WebP encoding via libwebp [no] > --enable-libx264 enable H.264 encoding via x264 [no] > --enable-libx265 enable HEVC encoding via x265 [no] > + --enable-libxeve enable EVC encoding via libxeve [no] > + --enable-libxevd enable EVC decoding via libxevd [no] > --enable-libxavs enable AVS encoding via xavs [no] > --enable-libxavs2 enable AVS2 encoding via xavs2 [no] > --enable-libxcb enable X11 grabbing using XCB [autodetect] > @@ -1874,6 +1876,8 @@ EXTERNAL_LIBRARY_LIST=" > libvorbis > libvpx > libwebp > + libxevd > + libxeve > libxml2 > libzimg > libzmq > @@ -3395,6 +3399,8 @@ libx264rgb_encoder_select="libx264_encoder" > libx265_encoder_deps="libx265" > libxavs_encoder_deps="libxavs" > libxavs2_encoder_deps="libxavs2" > +libxevd_decoder_deps="libxevd" > +libxeve_encoder_deps="libxeve" > libxvid_encoder_deps="libxvid" > libzvbi_teletext_decoder_deps="libzvbi" > vapoursynth_demuxer_deps="vapoursynth" > @@ -6685,6 +6691,8 @@ enabled libx265 && require_pkg_config libx265 x265 x265.h x265_api_get > require_cpp_condition libx265 x265.h "X265_BUILD >= 70" > enabled libxavs && require libxavs "stdint.h xavs.h" xavs_encoder_encode "-lxavs $pthreads_extralibs $libm_extralibs" > enabled libxavs2 && require_pkg_config libxavs2 "xavs2 >= 1.3.0" "stdint.h xavs2.h" xavs2_api_get > +enabled libxevd && require_pkg_config libxevd "xevd >= 0.4.0" "xevd.h" xevd_decode > +enabled libxeve && require_pkg_config libxeve "xeve >= 0.4.0" "xeve.h" xeve_encode > enabled libxvid && require libxvid xvid.h xvid_global -lxvidcore > enabled libzimg && require_pkg_config libzimg "zimg >= 2.7.0" zimg.h zimg_get_api_version > enabled libzmq && require_pkg_config libzmq "libzmq >= 4.2.1" zmq.h zmq_ctx_new > diff --git a/doc/decoders.texi b/doc/decoders.texi > index e2fcbf5dc9..0bd9096114 100644 > --- a/doc/decoders.texi > +++ b/doc/decoders.texi > @@ -126,6 +126,30 @@ Set amount of frame threads to use during decoding. The default value is 0 (auto > > @end table > > +@section libxevd > + > +eXtra-fast Essential Video Decoder (XEVD) MPEG-5 EVC decoder wrapper. > + > +This decoder requires the presence of the libxevd headers and library > +during configuration. You need to explicitly configure the build with > +@option{--enable-libxevd}. > + > +The xevd project website is at @url{https://protect2.fireeye.com/v1/url?k=0cdf05ab-53443cce-0cde8ee4-000babff32e3-d7c41d94be1006e7&q=1&e=b504c66d-387e-4b60-8cbe-1f53ce8a1ea8&u=https%3A%2F%2Fgithub.com%2Fmpeg5%2Fxevd%257D. > + > +@subsection Options > + > +The following options are supported by the libxevd wrapper. > +The xevd-equivalent options or values are listed in parentheses for easy migration. > + > +To get a more accurate and extensive documentation of the libxevd options, > +invoke the command @code{xevd_app --help} or consult the libxevd documentation. > + > +@table @option > +@item threads (@emph{threads}) > +Force to use a specific number of threads > + > +@end table > + > @section QSV Decoders > > The family of Intel QuickSync Video decoders (VC1, MPEG-2, H.264, HEVC, > diff --git a/doc/encoders.texi b/doc/encoders.texi > index 6d73f74196..76f55e6721 100644 > --- a/doc/encoders.texi > +++ b/doc/encoders.texi > @@ -2892,6 +2892,98 @@ ffmpeg -i input -c:v libxavs2 -xavs2-params RdoqLevel=0 output.avs2 > @end example > @end table > > +@section libxeve > + > +eXtra-fast Essential Video Encoder (XEVE) MPEG-5 EVC encoder wrapper. > +The xeve-equivalent options or values are listed in parentheses for easy migration. > + > +This encoder requires the presence of the libxeve headers and library > +during configuration. You need to explicitly configure the build with > +@option{--enable-libxeve}. > + > +@float NOTE > +Many libxeve encoder options are mapped to FFmpeg global codec options, > +while unique encoder options are provided through private options. > +Additionally the xeve-params private options allows one to pass a list > +of key=value tuples as accepted by the libxeve @code{parse_xeve_params} function. > +@end float > + > +The xeve project website is at @url{https://protect2.fireeye.com/v1/url?k=a21df818-fd86c17d-a21c7357-000babff32e3-481c230dcbcfe69f&q=1&e=b504c66d-387e-4b60-8cbe-1f53ce8a1ea8&u=https%3A%2F%2Fgithub.com%2Fmpeg5%2Fxeve%257D. > + > +@subsection Options > + > +The following options are supported by the libxeve wrapper. > +The xeve-equivalent options or values are listed in parentheses for easy migration. > + > +@float NOTE > +To reduce the duplication of documentation, only the private options > +and some others requiring special attention are documented here. For > +the documentation of the undocumented generic options, see > +@ref{codec-options,,the Codec Options chapter}. > +@end float > + > +@float NOTE > +To get a more accurate and extensive documentation of the libxeve options, > +invoke the command @code{xeve_app --help} or consult the libxeve documentation. > +@end float > + > +@table @option > +@item b (@emph{bitrate}) > +Set target video bitrate in bits/s. > +Note that FFmpeg's b option is expressed in bits/s, while xeve's bitrate is in kilobits/s. > + > +@item bf (@emph{bframes}) > +Set the maximum number of B frames (1,3,7,15). > + > +@item g (@emph{keyint}) > +Set the GOP size (I-picture period). > + > +@item preset (@emph{preset}) > +Set the xeve preset. > +Set the encoder preset value to determine encoding speed [fast, medium, slow, placebo] > + > +@item tune (@emph{tune}) > +Set the encoder tune parameter [psnr, zerolatency] > + > +@item profile (@emph{profile}) > +Set the encoder profile [0: baselie; 1: main] > + > +@item crf (@emph{crf}) > +Set the quality for constant quality mode. > +Constant rate factor <10..49> [default: 32] > + > +@item qp (@emph{qp}) > +Set constant quantization rate control method parameter. > +Quantization parameter qp <0..51> [default: 32] > + > +@item threads (@emph{threads}) > +Force to use a specific number of threads > + > +@item xeve-params > +Set xeve options using a list of @var{key}=@var{value} couples separated > +by ":". > + > +For example to specify libxeve encoding options with @option{-xeve-params}: > + > +@example > +ffmpeg -i input -c:v libxeve -xeve-params profile=main:tune=zerolatency:preset=fast:q=17:bframes=15:bitrate=100k output.mp4 > +@end example > +@end table > + > +@subsubsection Supported key values for xeve-params > + > +Set xeve options using a list of @var{key}=@var{value} couples separated > + > +@table @option > + > +@item hash > +Embed the picture signature (HASH) for conformance checking in decoding, (0: Off, 1: On) > + > +@item vbv-bufsize > +Set the VBV buffer size: Kbits(none,K,k), Mbits(M,m) > + > +@end table > + > @section libxvid > > Xvid MPEG-4 Part 2 encoder wrapper. > diff --git a/doc/general_contents.texi b/doc/general_contents.texi > index 86ec6d606b..89236047ea 100644 > --- a/doc/general_contents.texi > +++ b/doc/general_contents.texi > @@ -343,6 +343,22 @@ libxavs2 is under the GNU Public License Version 2 or later > details), you must upgrade FFmpeg's license to GPL in order to use it. > @end float > > +@section eXtra-fast Essential Video Encoder (XEVE) > + > +FFmpeg can make use of the XEVE library for EVC video encoding. > + > +Go to @url{https://protect2.fireeye.com/v1/url?k=315aae41-6ec19724-315b250e-000babff32e3-ecfea0332f2dcd7d&q=1&e=b504c66d-387e-4b60-8cbe-1f53ce8a1ea8&u=https%3A%2F%2Fgithub.com%2Fmpeg5%2Fxeve%257D and follow the instructions for > +installing the XEVE library. Then pass @code{--enable-libxeve} to configure to > +enable it. > + > +@section eXtra-fast Essential Video Decoder (XEVD) > + > +FFmpeg can make use of the XEVD library for EVC video decoding. > + > +Go to @url{https://protect2.fireeye.com/v1/url?k=39d72b05-664c1260-39d6a04a-000babff32e3-5e00c6377e7cf34c&q=1&e=b504c66d-387e-4b60-8cbe-1f53ce8a1ea8&u=https%3A%2F%2Fgithub.com%2Fmpeg5%2Fxevd%257D and follow the instructions for > +installing the XEVD library. Then pass @code{--enable-libxevd} to configure to > +enable it. > + > @section ZVBI > > ZVBI is a VBI decoding library which can be used by FFmpeg to decode DVB > @@ -591,6 +607,7 @@ library: > @item raw DTS @tab X @tab X > @item raw DTS-HD @tab @tab X > @item raw E-AC-3 @tab X @tab X > +@item raw EVC @tab X @tab X > @item raw FLAC @tab X @tab X > @item raw GSM @tab @tab X > @item raw H.261 @tab X @tab X > @@ -930,6 +947,8 @@ following image formats are supported: > @item Electronic Arts TQI video @tab @tab X > @item Escape 124 @tab @tab X > @item Escape 130 @tab @tab X > +@item EVC / MPEG-5 Part 1 @tab X @tab X > + @tab encoding and decoding supported through external libraries libxeve and libxevd > @item FFmpeg video codec #1 @tab X @tab X > @tab lossless codec (fourcc: FFV1) > @item Flash Screen Video v1 @tab X @tab X > diff --git a/libavcodec/Makefile b/libavcodec/Makefile > index 83901bb52b..cfeacd96d1 100644 > --- a/libavcodec/Makefile > +++ b/libavcodec/Makefile > @@ -1102,6 +1102,8 @@ OBJS-$(CONFIG_LIBX264_ENCODER) += libx264.o > OBJS-$(CONFIG_LIBX265_ENCODER) += libx265.o > OBJS-$(CONFIG_LIBXAVS_ENCODER) += libxavs.o > OBJS-$(CONFIG_LIBXAVS2_ENCODER) += libxavs2.o > +OBJS-$(CONFIG_LIBXEVD_DECODER) += libxevd.o > +OBJS-$(CONFIG_LIBXEVE_ENCODER) += libxeve.o > OBJS-$(CONFIG_LIBXVID_ENCODER) += libxvid.o > OBJS-$(CONFIG_LIBZVBI_TELETEXT_DECODER) += libzvbi-teletextdec.o ass.o > > @@ -1129,6 +1131,7 @@ OBJS-$(CONFIG_DVAUDIO_PARSER) += dvaudio_parser.o > OBJS-$(CONFIG_DVBSUB_PARSER) += dvbsub_parser.o > OBJS-$(CONFIG_DVD_NAV_PARSER) += dvd_nav_parser.o > OBJS-$(CONFIG_DVDSUB_PARSER) += dvdsub_parser.o > +OBJS-$(CONFIG_EVC_PARSER) += evc_parser.o > OBJS-$(CONFIG_FLAC_PARSER) += flac_parser.o flacdata.o flac.o > OBJS-$(CONFIG_G723_1_PARSER) += g723_1_parser.o > OBJS-$(CONFIG_G729_PARSER) += g729_parser.o > diff --git a/libavcodec/allcodecs.c b/libavcodec/allcodecs.c > index c94e2d5966..2526e11f70 100644 > --- a/libavcodec/allcodecs.c > +++ b/libavcodec/allcodecs.c > @@ -800,6 +800,8 @@ extern LIBX264_CONST FFCodec ff_libx264_encoder; > #endif > extern const FFCodec ff_libx264rgb_encoder; > extern FFCodec ff_libx265_encoder; > +extern const FFCodec ff_libxeve_encoder; > +extern const FFCodec ff_libxevd_decoder; > extern const FFCodec ff_libxavs_encoder; > extern const FFCodec ff_libxavs2_encoder; > extern const FFCodec ff_libxvid_encoder; > diff --git a/libavcodec/avcodec.h b/libavcodec/avcodec.h > index 65c8535359..7d56a03fca 100644 > --- a/libavcodec/avcodec.h > +++ b/libavcodec/avcodec.h > @@ -1674,6 +1674,9 @@ typedef struct AVCodecContext { > #define FF_PROFILE_KLVA_SYNC 0 > #define FF_PROFILE_KLVA_ASYNC 1 > > +#define FF_PROFILE_EVC_BASELINE 0 > +#define FF_PROFILE_EVC_MAIN 1 > + > /** > * level > * - encoding: Set by user. > diff --git a/libavcodec/codec_desc.c b/libavcodec/codec_desc.c > index c1a177c22d..1a43ff8cb6 100644 > --- a/libavcodec/codec_desc.c > +++ b/libavcodec/codec_desc.c > @@ -1907,6 +1907,14 @@ static const AVCodecDescriptor codec_descriptors[] = { > .long_name = NULL_IF_CONFIG_SMALL("WBMP (Wireless Application Protocol Bitmap) image"), > .props = AV_CODEC_PROP_INTRA_ONLY | AV_CODEC_PROP_LOSSLESS, > }, > + { > + .id = AV_CODEC_ID_EVC, > + .type = AVMEDIA_TYPE_VIDEO, > + .name = "evc", > + .long_name = NULL_IF_CONFIG_SMALL("MPEG-5 EVC (Essential Video Coding)"), > + .props = AV_CODEC_PROP_LOSSY | AV_CODEC_PROP_REORDER, > + .profiles = NULL_IF_CONFIG_SMALL(ff_evc_profiles), > + }, > > /* various PCM "codecs" */ > { > diff --git a/libavcodec/codec_id.h b/libavcodec/codec_id.h > index 386a00a7ef..02b60be690 100644 > --- a/libavcodec/codec_id.h > +++ b/libavcodec/codec_id.h > @@ -314,6 +314,7 @@ enum AVCodecID { > AV_CODEC_ID_PHM, > AV_CODEC_ID_RADIANCE_HDR, > AV_CODEC_ID_WBMP, > + AV_CODEC_ID_EVC, > > /* various PCM "codecs" */ > AV_CODEC_ID_FIRST_AUDIO = 0x10000, ///< A dummy id pointing at the start of audio codecs > diff --git a/libavcodec/evc_parser.c b/libavcodec/evc_parser.c > new file mode 100644 > index 0000000000..0ae0eaa08a > --- /dev/null > +++ b/libavcodec/evc_parser.c > @@ -0,0 +1,758 @@ > +/* > + * EVC format parser > + * > + * Copyright (C) 2021 Dawid Kozinski <d.kozinski@samsung.com> > + * > + * This file is part of FFmpeg. > + * > + * FFmpeg is free software; you can redistribute it and/or > + * modify it under the terms of the GNU Lesser General Public > + * License as published by the Free Software Foundation; either > + * version 2.1 of the License, or (at your option) any later version. > + * > + * FFmpeg is distributed in the hope that it will be useful, > + * but WITHOUT ANY WARRANTY; without even the implied warranty of > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > + * Lesser General Public License for more details. > + * > + * You should have received a copy of the GNU Lesser General Public > + * License along with FFmpeg; if not, write to the Free Software > + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA > + */ > + > +#include <stdint.h> > + > +#include "libavutil/common.h" > + > +#include "parser.h" > +#include "golomb.h" > + > +// The length field that indicates the length in bytes of the following NAL unit is configured to be of 4 bytes > +#define EVC_NAL_UNIT_LENGTH_BYTE (4) /* byte */ > + > +#define EVC_NAL_HEADER_SIZE (2) /* byte */ > +#define MAX_SPS_CNT (16) /* defined value in EVC standard */ > +#define MAX_PPS_CNT (64) /* defined value in EVC standard */ > + > +#define MAX_NUM_TILES_ROW (22) > +#define MAX_NUM_TILES_COL (20) > + > +// NALU types > +// @see ISO_IEC_23094-1_2020 7.4.2.2 NAL unit header semantics > +// > +#define EVC_NUT_NONIDR (0) /* Coded slice of a non-IDR picture */ > +#define EVC_NUT_IDR (1) /* Coded slice of an IDR picture */ > +#define EVC_NUT_SPS (24) /* Sequence parameter set */ > +#define EVC_NUT_PPS (25) /* Picture paremeter set */ > +#define EVC_NUT_APS (26) /* Adaptation parameter set */ > +#define EVC_NUT_FD (27) /* Filler data */ > +#define EVC_NUT_SEI (28) /* Supplemental enhancement information */ > + > +// slice type > +// @see ISO_IEC_23094-1_2020 7.4.5 Slice header semantics > +// > +#define SLICE_TYPE_UNKNOWN (-1) > +#define SLICE_TYPE_B (0) > +#define SLICE_TYPE_P (1) > +#define SLICE_TYPE_I (2) > + > +#define MAX_NUM_RPLS (32) > +#define MAX_NUM_REF_PICS (21) > + > +// rpl structure > +typedef struct RefPicListStruct { > + int poc; > + int tid; > + int ref_pic_num; > + int ref_pic_active_num; > + int ref_pics[MAX_NUM_REF_PICS]; > + char pic_type; > + > +} RefPicListStruct; > + > +// The sturcture reflects SPS RBSP(raw byte sequence payload) layout > +// @see ISO_IEC_23094-1 section 7.3.2.1 > +// > +// The following descriptors specify the parsing process of each element > +// u(n) - unsigned integer using n bits > +// ue(v) - unsigned integer 0-th order Exp_Golomb-coded syntax element with the left bit first > +typedef struct EVCParserSPS { > + int sps_seq_parameter_set_id; // ue(v) > + int profile_idc; // u(8) > + int level_idc; // u(8) > + int toolset_idc_h; // u(32) > + int toolset_idc_l; // u(32) > + int chroma_format_idc; // ue(v) > + int pic_width_in_luma_samples; // ue(v) > + int pic_height_in_luma_samples; // ue(v) > + int bit_depth_luma_minus8; // ue(v) > + int bit_depth_chroma_minus8; // ue(v) > + > + int sps_btt_flag; // u(1) > + int log2_ctu_size_minus5; // ue(v) > + int log2_min_cb_size_minus2; // ue(v) > + int log2_diff_ctu_max_14_cb_size; // ue(v) > + int log2_diff_ctu_max_tt_cb_size; // ue(v) > + int log2_diff_min_cb_min_tt_cb_size_minus2; // ue(v) > + > + int sps_suco_flag; // u(1) > + int log2_diff_ctu_size_max_suco_cb_size; // ue(v) > + int log2_diff_max_suco_min_suco_cb_size; // ue(v) > + > + int sps_admvp_flag; // u(1) > + int sps_affine_flag; // u(1) > + int sps_amvr_flag; // u(1) > + int sps_dmvr_flag; // u(1) > + int sps_mmvd_flag; // u(1) > + int sps_hmvp_flag; // u(1) > + > + int sps_eipd_flag; // u(1) > + int sps_ibc_flag; // u(1) > + int log2_max_ibc_cand_size_minus2; // ue(v) > + > + int sps_cm_init_flag; // u(1) > + int sps_adcc_flag; // u(1) > + > + int sps_iqt_flag; // u(1) > + int sps_ats_flag; // u(1) > + > + int sps_addb_flag; // u(1) > + int sps_alf_flag; // u(1) > + int sps_htdf_flag; // u(1) > + int sps_rpl_flag; // u(1) > + int sps_pocs_flag; // u(1) > + int sps_dquant_flag; // u(1) > + int sps_dra_flag; // u(1) > + > + int log2_max_pic_order_cnt_lsb_minus4; // ue(v) > + int log2_sub_gop_length; // ue(v) > + int log2_ref_pic_gap_length; // ue(v) > + > + int max_num_tid0_ref_pics; // ue(v) > + > + int sps_max_dec_pic_buffering_minus1; // ue(v) > + int long_term_ref_pic_flag; // u(1) > + int rpl1_same_as_rpl0_flag; // u(1) > + int num_ref_pic_list_in_sps[2]; // ue(v) > + RefPicListStruct rpls[2][MAX_NUM_RPLS]; > + > + int picture_cropping_flag; // u(1) > + int picture_crop_left_offset; // ue(v) > + int picture_crop_right_offset; // ue(v) > + int picture_crop_top_offset; // ue(v) > + int picture_crop_bottom_offset; // ue(v) > + > + // @note > + // Currently the structure does not reflect the entire SPS RBSP layout. > + // It contains only the fields that are necessary to read from the NAL unit all the values > + // necessary for the correct initialization of the AVCodecContext structure. > + > + // @note > + // If necessary, add the missing fields to the structure to reflect > + // the contents of the entire NAL unit of the SPS type > + > +} EVCParserSPS; > + > +typedef struct EVCParserPPS { > + int pps_pic_parameter_set_id; // ue(v) > + int pps_seq_parameter_set_id; // ue(v) > + int num_ref_idx_default_active_minus1[2]; // ue(v) > + int additional_lt_poc_lsb_len; // ue(v) > + int rpl1_idx_present_flag; // u(1) > + int single_tile_in_pic_flag; // u(1) > + int num_tile_columns_minus1; // ue(v) > + int num_tile_rows_minus1; // ue(v) > + int uniform_tile_spacing_flag; // u(1) > + int tile_column_width_minus1[MAX_NUM_TILES_ROW]; // ue(v) > + int tile_row_height_minus1[MAX_NUM_TILES_COL]; // ue(v) > + int loop_filter_across_tiles_enabled_flag; // u(1) > + int tile_offset_len_minus1; // ue(v) > + int tile_id_len_minus1; // ue(v) > + int explicit_tile_id_flag; // u(1) > + int tile_id_val[MAX_NUM_TILES_ROW][MAX_NUM_TILES_COL]; // u(v) > + int pic_dra_enabled_flag; // u(1) > + int pic_dra_aps_id; // u(5) > + int arbitrary_slice_present_flag; // u(1) > + int constrained_intra_pred_flag; // u(1) > + int cu_qp_delta_enabled_flag; // u(1) > + int log2_cu_qp_delta_area_minus6; // ue(v) > + > +} EVCParserPPS; > + > +// The sturcture reflects Slice Header RBSP(raw byte sequence payload) layout > +// @see ISO_IEC_23094-1 section 7.3.2.6 > +// > +// The following descriptors specify the parsing process of each element > +// u(n) - unsigned integer using n bits > +// ue(v) - unsigned integer 0-th order Exp_Golomb-coded syntax element with the left bit first > +// u(n) - unsigned integer using n bits. > +// When n is "v" in the syntax table, the number of bits varies in a manner dependent on the value of other syntax elements. > +typedef struct EVCParserSliceHeader { > + int slice_pic_parameter_set_id; // ue(v) > + int single_tile_in_slice_flag; // u(1) > + int first_tile_id; // u(v) > + int arbitrary_slice_flag; // u(1) > + int last_tile_id; // u(v) > + int num_remaining_tiles_in_slice_minus1; // ue(v) > + int delta_tile_id_minus1[MAX_NUM_TILES_ROW * MAX_NUM_TILES_COL]; // ue(v) > + > + int slice_type; // ue(v) > + > + // @note > + // Currently the structure does not reflect the entire Slice Header RBSP layout. > + // It contains only the fields that are necessary to read from the NAL unit all the values > + // necessary for the correct initialization of the AVCodecContext structure. > + > + // @note > + // If necessary, add the missing fields to the structure to reflect > + // the contents of the entire NAL unit of the SPS type > + > +} EVCParserSliceHeader; > + > +typedef struct EVCParserContext { > + ParseContext pc; > + EVCParserSPS sps[MAX_SPS_CNT]; > + EVCParserPPS pps[MAX_PPS_CNT]; > + EVCParserSliceHeader slice_header[MAX_PPS_CNT]; > + int is_avc; > + int nal_length_size; > + int to_read; > + int incomplete_nalu_prefix_read; // The flag is set to 1 when an incomplete NAL unit prefix has been read > + > +} EVCParserContext; > + > +static int get_nalu_type(const uint8_t *bits, int bits_size, AVCodecContext *avctx) > +{ > + int unit_type_plus1 = 0; > + > + if (bits_size >= EVC_NAL_HEADER_SIZE) { > + unsigned char *p = (unsigned char *)bits; > + // forbidden_zero_bit > + if ((p[0] & 0x80) != 0) > + return -1; > + > + // nal_unit_type > + unit_type_plus1 = (p[0] >> 1) & 0x3F; > + } > + > + return unit_type_plus1 - 1; > +} > + > +static uint32_t read_nal_unit_length(const uint8_t *bits, int bits_size, AVCodecContext *avctx) > +{ > + uint32_t nalu_len = 0; > + > + if (bits_size >= EVC_NAL_UNIT_LENGTH_BYTE) { > + > + int t = 0; > + unsigned char *p = (unsigned char *)bits; > + > + for (int i = 0; i < EVC_NAL_UNIT_LENGTH_BYTE; i++) > + t = (t << 8) | p[i]; > + > + nalu_len = t; > + if (nalu_len == 0) > + return 0; > + } > + > + return nalu_len; > +} > + > +// @see ISO_IEC_23094-1 (7.3.2.1 SPS RBSP syntax) > +static EVCParserSPS *parse_sps(const uint8_t *bs, int bs_size, EVCParserContext *ev) > +{ > + GetBitContext gb; > + EVCParserSPS *sps; > + int sps_seq_parameter_set_id; > + > + if (init_get_bits8(&gb, bs, bs_size) < 0) > + return NULL; > + > + sps_seq_parameter_set_id = get_ue_golomb(&gb); > + > + if (sps_seq_parameter_set_id >= MAX_SPS_CNT) > + return NULL; > + > + sps = &ev->sps[sps_seq_parameter_set_id]; > + sps->sps_seq_parameter_set_id = sps_seq_parameter_set_id; > + > + // the Baseline profile is indicated by profile_idc eqal to 0 > + // the Main profile is indicated by profile_idc eqal to 1 > + sps->profile_idc = get_bits(&gb, 8); > + > + sps->level_idc = get_bits(&gb, 8); > + > + skip_bits_long(&gb, 32); /* skip toolset_idc_h */ > + skip_bits_long(&gb, 32); /* skip toolset_idc_l */ > + > + // 0 - monochrome > + // 1 - 4:2:0 > + // 2 - 4:2:2 > + // 3 - 4:4:4 > + sps->chroma_format_idc = get_ue_golomb(&gb); > + > + sps->pic_width_in_luma_samples = get_ue_golomb(&gb); > + sps->pic_height_in_luma_samples = get_ue_golomb(&gb); > + > + sps->bit_depth_luma_minus8 = get_ue_golomb(&gb); > + sps->bit_depth_chroma_minus8 = get_ue_golomb(&gb); > + > + sps->sps_btt_flag = get_bits(&gb, 1); > + if (sps->sps_btt_flag) { > + sps->log2_ctu_size_minus5 = get_ue_golomb(&gb); > + sps->log2_min_cb_size_minus2 = get_ue_golomb(&gb); > + sps->log2_diff_ctu_max_14_cb_size = get_ue_golomb(&gb); > + sps->log2_diff_ctu_max_tt_cb_size = get_ue_golomb(&gb); > + sps->log2_diff_min_cb_min_tt_cb_size_minus2 = get_ue_golomb(&gb); > + } > + > + sps->sps_suco_flag = get_bits(&gb, 1); > + if (sps->sps_suco_flag) { > + sps->log2_diff_ctu_size_max_suco_cb_size = get_ue_golomb(&gb); > + sps->log2_diff_max_suco_min_suco_cb_size = get_ue_golomb(&gb); > + } > + > + sps->sps_admvp_flag = get_bits(&gb, 1); > + if (sps->sps_admvp_flag) { > + sps->sps_affine_flag = get_bits(&gb, 1); > + sps->sps_amvr_flag = get_bits(&gb, 1); > + sps->sps_dmvr_flag = get_bits(&gb, 1); > + sps->sps_mmvd_flag = get_bits(&gb, 1); > + sps->sps_hmvp_flag = get_bits(&gb, 1); > + } > + > + sps->sps_eipd_flag = get_bits(&gb, 1); > + if (sps->sps_eipd_flag) { > + sps->sps_ibc_flag = get_bits(&gb, 1); > + if (sps->sps_ibc_flag) > + sps->log2_max_ibc_cand_size_minus2 = get_ue_golomb(&gb); > + } > + > + sps->sps_cm_init_flag = get_bits(&gb, 1); > + if (sps->sps_cm_init_flag) > + sps->sps_adcc_flag = get_bits(&gb, 1); > + > + sps->sps_iqt_flag = get_bits(&gb, 1); > + if (sps->sps_iqt_flag) > + sps->sps_ats_flag = get_bits(&gb, 1); > + > + sps->sps_addb_flag = get_bits(&gb, 1); > + sps->sps_alf_flag = get_bits(&gb, 1); > + sps->sps_htdf_flag = get_bits(&gb, 1); > + sps->sps_rpl_flag = get_bits(&gb, 1); > + sps->sps_pocs_flag = get_bits(&gb, 1); > + sps->sps_dquant_flag = get_bits(&gb, 1); > + sps->sps_dra_flag = get_bits(&gb, 1); > + > + if (sps->sps_pocs_flag) > + sps->log2_max_pic_order_cnt_lsb_minus4 = get_ue_golomb(&gb); > + > + if (!sps->sps_pocs_flag || !sps->sps_rpl_flag) { > + sps->log2_sub_gop_length = get_ue_golomb(&gb); > + if (sps->log2_sub_gop_length == 0) > + sps->log2_ref_pic_gap_length = get_ue_golomb(&gb); > + } > + > + // @note > + // If necessary, add the missing fields to the EVCParserSPS structure > + // and then extend parser implementation > + > + return sps; > +} > + > +// @see ISO_IEC_23094-1 (7.3.2.2 SPS RBSP syntax) > +static EVCParserPPS *parse_pps(const uint8_t *bs, int bs_size, EVCParserContext *ev) > +{ > + GetBitContext gb; > + EVCParserPPS *pps; > + > + int pps_pic_parameter_set_id; > + > + if (init_get_bits8(&gb, bs, bs_size) < 0) > + return NULL; > + > + pps_pic_parameter_set_id = get_ue_golomb(&gb); > + if (pps_pic_parameter_set_id > MAX_PPS_CNT) > + return NULL; > + > + pps = &ev->pps[pps_pic_parameter_set_id]; > + > + pps->pps_pic_parameter_set_id = pps_pic_parameter_set_id; > + > + pps->pps_seq_parameter_set_id = get_ue_golomb(&gb); > + if (pps->pps_seq_parameter_set_id >= MAX_SPS_CNT) > + return NULL; > + > + pps->num_ref_idx_default_active_minus1[0] = get_ue_golomb(&gb); > + pps->num_ref_idx_default_active_minus1[1] = get_ue_golomb(&gb); > + pps->additional_lt_poc_lsb_len = get_ue_golomb(&gb); > + pps->rpl1_idx_present_flag = get_bits(&gb, 1); > + pps->single_tile_in_pic_flag = get_bits(&gb, 1); > + > + if (!pps->single_tile_in_pic_flag) { > + pps->num_tile_columns_minus1 = get_ue_golomb(&gb); > + pps->num_tile_rows_minus1 = get_ue_golomb(&gb); > + pps->uniform_tile_spacing_flag = get_bits(&gb, 1); > + > + if (!pps->uniform_tile_spacing_flag) { > + for (int i = 0; i < pps->num_tile_columns_minus1; i++) > + pps->tile_column_width_minus1[i] = get_ue_golomb(&gb); > + > + for (int i = 0; i < pps->num_tile_rows_minus1; i++) > + pps->tile_row_height_minus1[i] = get_ue_golomb(&gb); > + } > + pps->loop_filter_across_tiles_enabled_flag = get_bits(&gb, 1); > + pps->tile_offset_len_minus1 = get_ue_golomb(&gb); > + } > + > + pps->tile_id_len_minus1 = get_ue_golomb(&gb); > + pps->explicit_tile_id_flag = get_bits(&gb, 1); > + > + if (pps->explicit_tile_id_flag) { > + for (int i = 0; i <= pps->num_tile_rows_minus1; i++) { > + for (int j = 0; j <= pps->num_tile_columns_minus1; j++) > + pps->tile_id_val[i][j] = get_bits(&gb, pps->tile_id_len_minus1 + 1); > + } > + } > + > + pps->pic_dra_enabled_flag = 0; > + pps->pic_dra_enabled_flag = get_bits(&gb, 1); > + > + if (pps->pic_dra_enabled_flag) > + pps->pic_dra_aps_id = get_bits(&gb, 5); > + > + pps->arbitrary_slice_present_flag = get_bits(&gb, 1); > + pps->constrained_intra_pred_flag = get_bits(&gb, 1); > + pps->cu_qp_delta_enabled_flag = get_bits(&gb, 1); > + > + if (pps->cu_qp_delta_enabled_flag) > + pps->log2_cu_qp_delta_area_minus6 = get_ue_golomb(&gb); > + > + return pps; > +} > + > +// @see ISO_IEC_23094-1 (7.3.2.6 Slice layer RBSP syntax) > +static EVCParserSliceHeader *parse_slice_header(const uint8_t *bs, int bs_size, EVCParserContext *ev) > +{ > + GetBitContext gb; > + EVCParserSliceHeader *sh; > + EVCParserPPS *pps; > + int num_tiles_in_slice = 0; > + int slice_pic_parameter_set_id; > + > + if (init_get_bits8(&gb, bs, bs_size) < 0) > + return NULL; > + > + slice_pic_parameter_set_id = get_ue_golomb(&gb); > + > + if (slice_pic_parameter_set_id < 0 || slice_pic_parameter_set_id >= MAX_PPS_CNT) > + return NULL; > + > + sh = &ev->slice_header[slice_pic_parameter_set_id]; > + pps = &ev->pps[slice_pic_parameter_set_id]; > + > + sh->slice_pic_parameter_set_id = slice_pic_parameter_set_id; > + > + if (!pps->single_tile_in_pic_flag) { > + sh->single_tile_in_slice_flag = get_bits(&gb, 1); > + sh->first_tile_id = get_bits(&gb, pps->tile_id_len_minus1 + 1); > + } else > + sh->single_tile_in_slice_flag = 1; > + > + if (!sh->single_tile_in_slice_flag) { > + if (pps->arbitrary_slice_present_flag) > + sh->arbitrary_slice_flag = get_bits(&gb, 1); > + > + if (!sh->arbitrary_slice_flag) > + sh->last_tile_id = get_bits(&gb, pps->tile_id_len_minus1 + 1); > + else { > + sh->num_remaining_tiles_in_slice_minus1 = get_ue_golomb(&gb); > + num_tiles_in_slice = sh->num_remaining_tiles_in_slice_minus1 + 2; > + for (int i = 0; i < num_tiles_in_slice - 1; ++i) > + sh->delta_tile_id_minus1[i] = get_ue_golomb(&gb); > + } > + } > + > + sh->slice_type = get_ue_golomb(&gb); > + > + // @note > + // If necessary, add the missing fields to the EVCParserSliceHeader structure > + // and then extend parser implementation > + > + return sh; > +} > + > +static int parse_nal_units(AVCodecParserContext *s, const uint8_t *bs, > + int bs_size, AVCodecContext *avctx) > +{ > + EVCParserContext *ev = s->priv_data; > + int nalu_type, nalu_size; > + unsigned char *bits = (unsigned char *)bs; > + int bits_size = bs_size; > + > + avctx->codec_id = AV_CODEC_ID_EVC; > + s->pict_type = AV_PICTURE_TYPE_NONE; > + s->picture_structure = AV_PICTURE_STRUCTURE_FRAME; > + s->key_frame = -1; > + > + nalu_size = read_nal_unit_length(bits, bits_size, avctx); > + if (nalu_size == 0) { > + av_log(avctx, AV_LOG_ERROR, "Invalid NAL unit size: (%d)\n", nalu_size); > + return -1; > + } > + > + bits += EVC_NAL_UNIT_LENGTH_BYTE; > + bits_size -= EVC_NAL_UNIT_LENGTH_BYTE; > + > + nalu_type = get_nalu_type(bits, bits_size, avctx); > + > + bits += EVC_NAL_HEADER_SIZE; > + bits_size -= EVC_NAL_HEADER_SIZE; > + > + if (nalu_type == EVC_NUT_SPS) { // NAL Unit type: SPS (Sequence Parameter Set) > + EVCParserSPS *sps; > + > + sps = parse_sps(bits, bits_size, ev); > + if (!sps) { > + av_log(avctx, AV_LOG_ERROR, "SPS parsing error\n"); > + return -1; > + } > + > + s->coded_width = sps->pic_width_in_luma_samples; > + s->coded_height = sps->pic_height_in_luma_samples; > + s->width = sps->pic_width_in_luma_samples; > + s->height = sps->pic_height_in_luma_samples; > + > + if (sps->profile_idc == 1) avctx->profile = FF_PROFILE_EVC_MAIN; > + else avctx->profile = FF_PROFILE_EVC_BASELINE; > + > + // Currently XEVD decoder supports ony YCBCR420_10LE chroma format for EVC stream > + switch (sps->chroma_format_idc) { > + case 0: /* YCBCR400_10LE */ > + av_log(avctx, AV_LOG_ERROR, "YCBCR400_10LE: Not supported chroma format\n"); > + s->format = AV_PIX_FMT_GRAY10LE; > + return -1; > + case 1: /* YCBCR420_10LE */ > + s->format = AV_PIX_FMT_YUV420P10LE; > + break; > + case 2: /* YCBCR422_10LE */ > + av_log(avctx, AV_LOG_ERROR, "YCBCR422_10LE: Not supported chroma format\n"); > + s->format = AV_PIX_FMT_YUV422P10LE; > + return -1; > + case 3: /* YCBCR444_10LE */ > + av_log(avctx, AV_LOG_ERROR, "YCBCR444_10LE: Not supported chroma format\n"); > + s->format = AV_PIX_FMT_YUV444P10LE; > + return -1; > + default: > + s->format = AV_PIX_FMT_NONE; > + av_log(avctx, AV_LOG_ERROR, "Unknown supported chroma format\n"); > + return -1; > + } > + > + // @note > + // The current implementation of parse_sps function doesn't handle VUI parameters parsing. > + // If it will be needed, parse_sps function could be extended to handle VUI parameters parsing > + // to initialize fields of the AVCodecContex i.e. color_primaries, color_trc,color_range > + > + } else if (nalu_type == EVC_NUT_PPS) { // NAL Unit type: PPS (Video Parameter Set) > + EVCParserPPS *pps; > + > + pps = parse_pps(bits, bits_size, ev); > + if (!pps) { > + av_log(avctx, AV_LOG_ERROR, "PPS parsing error\n"); > + return -1; > + } > + } else if (nalu_type == EVC_NUT_SEI) // NAL unit type: SEI (Supplemental Enhancement Information) > + return 0; > + else if (nalu_type == EVC_NUT_IDR || nalu_type == EVC_NUT_NONIDR) { // NAL Unit type: Coded slice of a IDR or non-IDR picture > + EVCParserSliceHeader *sh; > + > + sh = parse_slice_header(bits, bits_size, ev); > + if (!sh) { > + av_log(avctx, AV_LOG_ERROR, "Slice header parsing error\n"); > + return -1; > + } > + switch (sh->slice_type) { > + case SLICE_TYPE_B: { > + s->pict_type = AV_PICTURE_TYPE_B; > + break; > + } > + case SLICE_TYPE_P: { > + s->pict_type = AV_PICTURE_TYPE_P; > + break; > + } > + case SLICE_TYPE_I: { > + s->pict_type = AV_PICTURE_TYPE_I; > + break; > + } > + default: { > + s->pict_type = AV_PICTURE_TYPE_NONE; > + } > + } > + s->key_frame = (nalu_type == EVC_NUT_IDR) ? 1 : 0; > + } else { > + av_log(avctx, AV_LOG_ERROR, "Invalid NAL unit type: %d\n", nalu_type); > + return -1; > + } > + > + return 0; > +} > + > +/** > + * Find the end of the current frame in the bitstream. > + * @return the position of the first byte of the next frame, or END_NOT_FOUND > + */ > +static int evc_find_frame_end(AVCodecParserContext *s, const uint8_t *buf, > + int buf_size, AVCodecContext *avctx) > +{ > + EVCParserContext *ev = s->priv_data; > + > + if (!ev->to_read) { > + int nal_unit_size = 0; > + int next = END_NOT_FOUND; > + > + // This is the case when buffer size is not enough for buffer to store NAL unit length > + if (buf_size < EVC_NAL_UNIT_LENGTH_BYTE) { > + ev->to_read = EVC_NAL_UNIT_LENGTH_BYTE; > + ev->nal_length_size = buf_size; > + ev->incomplete_nalu_prefix_read = 1; > + > + return END_NOT_FOUND; > + } > + > + nal_unit_size = read_nal_unit_length(buf, buf_size, avctx); > + ev->nal_length_size = EVC_NAL_UNIT_LENGTH_BYTE; > + > + next = nal_unit_size + EVC_NAL_UNIT_LENGTH_BYTE; > + ev->to_read = next; > + if (next < buf_size) > + return next; > + else > + return END_NOT_FOUND; > + } else if (ev->to_read > buf_size) > + return END_NOT_FOUND; > + else { > + if (ev->incomplete_nalu_prefix_read == 1) { > + EVCParserContext *ev = s->priv_data; > + ParseContext *pc = &ev->pc; > + uint8_t nalu_len[EVC_NAL_UNIT_LENGTH_BYTE] = {0}; > + int nal_unit_size = 0; > + > + // 1. pc->buffer contains previously read bytes of NALU prefix > + // 2. buf contains the rest of NAL unit prefix bytes > + // > + // ~~~~~~~ > + // EXAMPLE > + // ~~~~~~~ > + // > + // In the following example we assumed that the number of already read NAL Unit prefix bytes is equal 1 > + // > + // ---------- > + // pc->buffer -> conatins already read bytes > + // ---------- > + // __ pc->index == 1 > + // | > + // V > + // ------------------------------------------------------- > + // | 0 | 1 | 2 | 3 | 4 | ... | N | > + // ------------------------------------------------------- > + // | 0x00 | 0xXX | 0xXX | 0xXX | 0xXX | ... | 0xXX | > + // ------------------------------------------------------- > + // > + // ---------- > + // buf -> contains newly read bytes > + // ---------- > + // ------------------------------------------------------- > + // | 0 | 1 | 2 | 3 | 4 | ... | N | > + // ------------------------------------------------------- > + // | 0x00 | 0x00 | 0x3C | 0xXX | 0xXX | ... | 0xXX | > + // ------------------------------------------------------- > + // > + for (int i = 0; i < EVC_NAL_UNIT_LENGTH_BYTE; i++) { > + if (i < pc->index) > + nalu_len[i] = pc->buffer[i]; > + else > + nalu_len[i] = buf[i - pc->index]; > + } > + > + // ---------- > + // nalu_len > + // ---------- > + // --------------------------------- > + // | 0 | 1 | 2 | 3 | > + // --------------------------------- > + // | 0x00 | 0x00 | 0x00 | 0x3C | > + // --------------------------------- > + // | NALU LENGTH | > + // --------------------------------- > + // NAL Unit lenght = 60 (0x0000003C) > + > + nal_unit_size = read_nal_unit_length(nalu_len, EVC_NAL_UNIT_LENGTH_BYTE, avctx); > + > + ev->to_read = nal_unit_size + EVC_NAL_UNIT_LENGTH_BYTE - pc->index; > + > + ev->incomplete_nalu_prefix_read = 0; > + > + if (ev->to_read > buf_size) > + return END_NOT_FOUND; > + else > + return ev->to_read; > + } > + return ev->to_read; > + } > + > + return END_NOT_FOUND; > +} > + > +static int evc_parser_init(AVCodecParserContext *s) > +{ > + EVCParserContext *ev = s->priv_data; > + > + ev->nal_length_size = EVC_NAL_UNIT_LENGTH_BYTE; > + ev->incomplete_nalu_prefix_read = 0; > + > + return 0; > +} > + > +static int evc_parse(AVCodecParserContext *s, AVCodecContext *avctx, > + const uint8_t **poutbuf, int *poutbuf_size, > + const uint8_t *buf, int buf_size) > +{ > + int next; > + EVCParserContext *ev = s->priv_data; > + ParseContext *pc = &ev->pc; > + int is_dummy_buf = !buf_size; > + const uint8_t *dummy_buf = buf; > + > + if (s->flags & PARSER_FLAG_COMPLETE_FRAMES) > + next = buf_size; > + else { > + next = evc_find_frame_end(s, buf, buf_size, avctx); > + if (ff_combine_frame(pc, next, &buf, &buf_size) < 0) { > + *poutbuf = NULL; > + *poutbuf_size = 0; > + ev->to_read -= buf_size; > + return buf_size; > + } > + } > + > + is_dummy_buf &= (dummy_buf == buf); > + > + if (!is_dummy_buf) > + parse_nal_units(s, buf, buf_size, avctx); > + > + // poutbuf contains just one NAL unit > + *poutbuf = buf; > + *poutbuf_size = buf_size; > + ev->to_read -= next; > + > + return next; > +} > + > +const AVCodecParser ff_evc_parser = { > + .codec_ids = { AV_CODEC_ID_EVC }, > + .priv_data_size = sizeof(EVCParserContext), > + .parser_init = evc_parser_init, > + .parser_parse = evc_parse, > + .parser_close = ff_parse_close, > +}; > diff --git a/libavcodec/libxevd.c b/libavcodec/libxevd.c > new file mode 100644 > index 0000000000..85f71affd3 > --- /dev/null > +++ b/libavcodec/libxevd.c > @@ -0,0 +1,419 @@ > +/* > + * libxevd decoder > + * EVC (MPEG-5 Essential Video Coding) decoding using XEVD MPEG-5 EVC decoder library > + * > + * Copyright (C) 2021 Dawid Kozinski <d.kozinski@samsung.com> > + * > + * This file is part of FFmpeg. > + * > + * FFmpeg is free software; you can redistribute it and/or > + * modify it under the terms of the GNU Lesser General Public > + * License as published by the Free Software Foundation; either > + * version 2.1 of the License, or (at your option) any later version. > + * > + * FFmpeg is distributed in the hope that it will be useful, > + * but WITHOUT ANY WARRANTY; without even the implied warranty of > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > + * Lesser General Public License for more details. > + * > + * You should have received a copy of the GNU Lesser General Public > + * License along with FFmpeg; if not, write to the Free Software > + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA > + */ > + > +#include <float.h> > +#include <stdlib.h> > + > +#include <xevd.h> > + > +#include "libavutil/internal.h" > +#include "libavutil/common.h" > +#include "libavutil/opt.h" > +#include "libavutil/pixdesc.h" > +#include "libavutil/pixfmt.h" > +#include "libavutil/imgutils.h" > +#include "libavutil/cpu.h" > + > +#include "avcodec.h" > +#include "internal.h" > +#include "packet_internal.h" > +#include "codec_internal.h" > + > +#define XEVD_PARAM_BAD_NAME -1 > +#define XEVD_PARAM_BAD_VALUE -2 > + > +#define EVC_NAL_HEADER_SIZE 2 /* byte */ > + > +/** > + * The structure stores all the states associated with the instance of Xeve MPEG-5 EVC decoder > + */ > +typedef struct XevdContext { > + const AVClass *class; > + > + XEVD id; // XEVD instance identifier @see xevd.h > + XEVD_CDSC cdsc; // decoding parameters @see xevd.h > +} XevdContext; > + > +/** > + * The function populates the XEVD_CDSC structure. > + * XEVD_CDSC contains all decoder parameters that should be initialized before its use. > + * > + * @param[in] avctx codec context > + * @param[out] cdsc contains all decoder parameters that should be initialized before its use > + * > + */ > +static void get_conf(AVCodecContext *avctx, XEVD_CDSC *cdsc) > +{ > + int cpu_count = av_cpu_count(); > + > + /* clear XEVS_CDSC structure */ > + memset(cdsc, 0, sizeof(XEVD_CDSC)); > + > + /* init XEVD_CDSC */ > + if (avctx->thread_count <= 0) > + cdsc->threads = (cpu_count < XEVD_MAX_TASK_CNT) ? cpu_count : XEVD_MAX_TASK_CNT; > + else if (avctx->thread_count > XEVD_MAX_TASK_CNT) > + cdsc->threads = XEVD_MAX_TASK_CNT; > + else > + cdsc->threads = avctx->thread_count; > +} > + > +/** > + * Read NAL unit length > + * @param bs input data (bitstream) > + * @return the length of NAL unit on success, 0 value on failure > + */ > +static uint32_t read_nal_unit_length(const uint8_t *bs, int bs_size, AVCodecContext *avctx) > +{ > + uint32_t len = 0; > + XEVD_INFO info; > + int ret; > + > + if (bs_size == XEVD_NAL_UNIT_LENGTH_BYTE) { > + ret = xevd_info((void *)bs, XEVD_NAL_UNIT_LENGTH_BYTE, 1, &info); > + if (XEVD_FAILED(ret)) { > + av_log(avctx, AV_LOG_ERROR, "Cannot get bitstream information\n"); > + return 0; > + } > + len = info.nalu_len; > + if (len == 0) { > + av_log(avctx, AV_LOG_ERROR, "Invalid bitstream size! [%d]\n", bs_size); > + return 0; > + } > + } > + > + return len; > +} > + > +/** > + * @param[in] xectx the structure that stores all the state associated with the instance of Xeve MPEG-5 EVC decoder > + * @param[out] avctx codec context > + * @return 0 on success, negative value on failure > + */ > +static int export_stream_params(const XevdContext *xectx, AVCodecContext *avctx) > +{ > + int ret; > + int size; > + int color_space; > + > + avctx->pix_fmt = AV_PIX_FMT_YUV420P10; > + > + size = 4; > + ret = xevd_config(xectx->id, XEVD_CFG_GET_CODED_WIDTH, &avctx->coded_width, &size); > + if (XEVD_FAILED(ret)) { > + av_log(avctx, AV_LOG_ERROR, "Failed to get coded_width\n"); > + return AVERROR_EXTERNAL; > + } > + > + ret = xevd_config(xectx->id, XEVD_CFG_GET_CODED_HEIGHT, &avctx->coded_height, &size); > + if (XEVD_FAILED(ret)) { > + av_log(avctx, AV_LOG_ERROR, "Failed to get coded_height\n"); > + return AVERROR_EXTERNAL; > + } > + > + ret = xevd_config(xectx->id, XEVD_CFG_GET_WIDTH, &avctx->width, &size); > + if (XEVD_FAILED(ret)) { > + av_log(avctx, AV_LOG_ERROR, "Failed to get width\n"); > + return AVERROR_EXTERNAL; > + } > + > + ret = xevd_config(xectx->id, XEVD_CFG_GET_HEIGHT, &avctx->height, &size); > + if (XEVD_FAILED(ret)) { > + av_log(avctx, AV_LOG_ERROR, "Failed to get height\n"); > + return AVERROR_EXTERNAL; > + } > + > + ret = xevd_config(xectx->id, XEVD_CFG_GET_COLOR_SPACE, &color_space, &size); > + if (XEVD_FAILED(ret)) { > + av_log(avctx, AV_LOG_ERROR, "Failed to get color_space\n"); > + return AVERROR_EXTERNAL; > + } > + switch(color_space) { > + case XEVD_CS_YCBCR400_10LE: > + avctx->pix_fmt = AV_PIX_FMT_GRAY10LE; > + break; > + case XEVD_CS_YCBCR420_10LE: > + avctx->pix_fmt = AV_PIX_FMT_YUV420P10LE; > + break; > + case XEVD_CS_YCBCR422_10LE: > + avctx->pix_fmt = AV_PIX_FMT_YUV422P10LE; > + break; > + case XEVD_CS_YCBCR444_10LE: > + avctx->pix_fmt = AV_PIX_FMT_YUV444P10LE; > + break; > + default: > + av_log(avctx, AV_LOG_ERROR, "Unknown color space\n"); > + avctx->pix_fmt = AV_PIX_FMT_NONE; > + return AVERROR_INVALIDDATA; > + } > + > + // the function returns sps->num_reorder_pics > + ret = xevd_config(xectx->id, XEVD_CFG_GET_MAX_CODING_DELAY, &avctx->max_b_frames, &size); > + if (XEVD_FAILED(ret)) { > + av_log(avctx, AV_LOG_ERROR, "Failed to get max_coding_delay\n"); > + return AVERROR_EXTERNAL; > + } > + > + avctx->has_b_frames = (avctx->max_b_frames) ? 1 : 0; > + > + avctx->color_primaries = AVCOL_PRI_UNSPECIFIED; > + avctx->color_trc = AVCOL_TRC_UNSPECIFIED; > + avctx->colorspace = AVCOL_SPC_UNSPECIFIED; > + > + return 0; > +} > + > +/** > + * @brief Copy image in imgb to frame. > + * > + * @param avctx codec context > + * @param[in] imgb > + * @param[out] frame > + * @return 0 on success, negative value on failure > + */ > +static int libxevd_image_copy(struct AVCodecContext *avctx, XEVD_IMGB *imgb, struct AVFrame *frame) > +{ > + int ret; > + if (imgb->cs != XEVD_CS_YCBCR420_10LE) { > + av_log(avctx, AV_LOG_ERROR, "Not supported pixel format: %s\n", av_get_pix_fmt_name(avctx->pix_fmt)); > + return AVERROR_INVALIDDATA; > + } > + > + if (imgb->w[0] != avctx->width || imgb->h[0] != avctx->height) { // stream resolution changed > + if (ff_set_dimensions(avctx, imgb->w[0], imgb->h[0]) < 0) { > + av_log(avctx, AV_LOG_ERROR, "Cannot set new dimension\n"); > + return AVERROR_INVALIDDATA; > + } > + } > + > + if (ret = ff_get_buffer(avctx, frame, 0) < 0) { > + return ret; > + } > + > + av_image_copy(frame->data, frame->linesize, (const uint8_t **)imgb->a, > + imgb->s, avctx->pix_fmt, > + imgb->w[0], imgb->h[0]); > + > + return 0; > +} > + > +/** > + * Initialize decoder > + * Create a decoder instance and allocate all the needed resources > + * > + * @param avctx codec context > + * @return 0 on success, negative error code on failure > + */ > +static av_cold int libxevd_init(AVCodecContext *avctx) > +{ > + XevdContext *xectx = avctx->priv_data; > + XEVD_CDSC *cdsc = &(xectx->cdsc); > + > + /* read configurations and set values for created descriptor (XEVD_CDSC) */ > + get_conf(avctx, cdsc); > + > + /* create decoder */ > + xectx->id = xevd_create(&(xectx->cdsc), NULL); > + if (xectx->id == NULL) { > + av_log(avctx, AV_LOG_ERROR, "Cannot create XEVD encoder\n"); > + return AVERROR_EXTERNAL; > + } > + > + return 0; > +} > + > +/** > + * Decode picture > + * > + * @param avctx codec context > + * @param[out] frame decoded frame > + * @param[out] got_frame_ptr decoder sets to 0 or 1 to indicate that a > + * non-empty frame or subtitle was returned in > + * outdata. > + * @param[in] avpkt AVPacket containing encoded data to be decoded > + * > + * @return amount of bytes read from the packet on success, negative error > + * code on failure > + */ > +static int libxevd_decode(struct AVCodecContext *avctx, struct AVFrame *frame, int *got_frame_ptr, AVPacket *avpkt) > +{ > + XevdContext *xectx = NULL; > + XEVD_IMGB *imgb = NULL; > + XEVD_STAT stat; > + XEVD_BITB bitb; > + int xevd_ret, nalu_size, bs_read_pos; > + int ret = 0; > + > + xectx = avctx->priv_data; > + > + if (avpkt->size > 0) { > + bs_read_pos = 0; > + imgb = NULL; > + while(avpkt->size > (bs_read_pos + XEVD_NAL_UNIT_LENGTH_BYTE)) { > + memset(&stat, 0, sizeof(XEVD_STAT)); > + memset(&bitb, 0, sizeof(XEVD_BITB)); > + > + nalu_size = read_nal_unit_length(avpkt->data + bs_read_pos, XEVD_NAL_UNIT_LENGTH_BYTE, avctx); > + if (nalu_size == 0) { > + av_log(avctx, AV_LOG_ERROR, "Invalid bitstream\n"); > + ret = AVERROR_INVALIDDATA; > + goto ERR; > + } > + bs_read_pos += XEVD_NAL_UNIT_LENGTH_BYTE; > + > + bitb.addr = avpkt->data + bs_read_pos; > + bitb.ssize = nalu_size; > + > + /* main decoding block */ > + xevd_ret = xevd_decode(xectx->id, &bitb, &stat); > + if (XEVD_FAILED(xevd_ret)) { > + av_log(avctx, AV_LOG_ERROR, "Failed to decode bitstream\n"); > + ret = AVERROR_EXTERNAL; > + goto ERR; > + } > + > + bs_read_pos += nalu_size; > + > + if (stat.nalu_type == XEVD_NUT_SPS) { // EVC stream parameters changed > + if ((ret = export_stream_params(xectx, avctx)) != 0) > + goto ERR; > + } > + > + if (stat.read != nalu_size) > + av_log(avctx, AV_LOG_INFO, "Different reading of bitstream (in:%d, read:%d)\n,", nalu_size, stat.read); > + if (stat.fnum >= 0) { > + // already has a decoded image > + if (imgb) { > + // xevd_pull uses pool of objects of type XEVD_IMGB. > + // The pool size is equal MAX_PB_SIZE (26), so release object when it is no more needed > + imgb->release(imgb); > + imgb = NULL; > + } > + xevd_ret = xevd_pull(xectx->id, &imgb); > + if (XEVD_FAILED(xevd_ret)) { > + av_log(avctx, AV_LOG_ERROR, "Failed to pull the decoded image (xevd error code: %d, frame#=%d)\n", xevd_ret, stat.fnum); > + ret = AVERROR_EXTERNAL; > + goto ERR; > + } else if (xevd_ret == XEVD_OK_FRM_DELAYED) { > + if (imgb) { > + // xevd_pull uses pool of objects of type XEVD_IMGB. > + // The pool size is equal MAX_PB_SIZE (26), so release object when it is no more needed > + imgb->release(imgb); > + imgb = NULL; > + } > + } > + if (imgb) { > + int ret = libxevd_image_copy(avctx, imgb, frame); > + if(ret < 0) { > + goto ERR; > + } > + > + frame->pts = avpkt->pts; > + *got_frame_ptr = 1; > + > + // xevd_pull uses pool of objects of type XEVD_IMGB. > + // The pool size is equal MAX_PB_SIZE (26), so release object when it is no more needed > + imgb->release(imgb); > + imgb = NULL; > + } else > + *got_frame_ptr = 0; > + } > + } > + } else { // bumping > + xevd_ret = xevd_pull(xectx->id, &(imgb)); > + if (xevd_ret == XEVD_ERR_UNEXPECTED) { // bumping process completed > + *got_frame_ptr = 0; > + return 0; > + } else if (XEVD_FAILED(xevd_ret)) { > + av_log(avctx, AV_LOG_ERROR, "Failed to pull the decoded image (xevd error code: %d)\n", xevd_ret); > + ret = AVERROR_EXTERNAL; > + goto ERR; > + } > + if (imgb) { > + int ret = libxevd_image_copy(avctx, imgb, frame); > + if(ret < 0) { > + goto ERR; > + } > + > + frame->pts = avpkt->pts; > + *got_frame_ptr = 1; > + > + // xevd_pull uses pool of objects of type XEVD_IMGB. > + // The pool size is equal MAX_PB_SIZE (26), so release object when it is no more needed > + imgb->release(imgb); > + imgb = NULL; > + } else > + *got_frame_ptr = 0; > + } > + > + return avpkt->size; > + > +ERR: > + if (imgb) { > + imgb->release(imgb); > + imgb = NULL; > + } > + *got_frame_ptr = 0; > + > + return ret; > +} > + > +/** > + * Destroy decoder > + * > + * @param avctx codec context > + * @return 0 on success > + */ > +static av_cold int libxevd_close(AVCodecContext *avctx) > +{ > + XevdContext *xectx = avctx->priv_data; > + if (xectx->id) { > + xevd_delete(xectx->id); > + xectx->id = NULL; > + } > + > + return 0; > +} > + > +#define OFFSET(x) offsetof(XevdContext, x) > +#define VD AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_DECODING_PARAM > + > +static const AVClass libxevd_class = { > + .class_name = "libxevd", > + .item_name = av_default_item_name, > + .version = LIBAVUTIL_VERSION_INT, > +}; > + > +const FFCodec ff_libxevd_decoder = { > + .p.name = "evc", > + .p.long_name = NULL_IF_CONFIG_SMALL("EVC / MPEG-5 Essential Video Coding (EVC)"), > + .p.type = AVMEDIA_TYPE_VIDEO, > + .p.id = AV_CODEC_ID_EVC, > + .init = libxevd_init, > + FF_CODEC_DECODE_CB(libxevd_decode), > + .close = libxevd_close, > + .priv_data_size = sizeof(XevdContext), > + .p.priv_class = &libxevd_class, > + .p.capabilities = AV_CODEC_CAP_DELAY | AV_CODEC_CAP_OTHER_THREADS | AV_CODEC_CAP_AVOID_PROBING | AV_CODEC_CAP_DR1, > + .p.wrapper_name = "libxevd", > +}; > diff --git a/libavcodec/libxeve.c b/libavcodec/libxeve.c > new file mode 100644 > index 0000000000..4808e6d853 > --- /dev/null > +++ b/libavcodec/libxeve.c > @@ -0,0 +1,597 @@ > +/* > + * libxeve encoder > + * EVC (MPEG-5 Essential Video Coding) encoding using XEVE MPEG-5 EVC encoder library > + * > + * Copyright (C) 2021 Dawid Kozinski <d.kozinski@samsung.com> > + * > + * This file is part of FFmpeg. > + * > + * FFmpeg is free software; you can redistribute it and/or > + * modify it under the terms of the GNU Lesser General Public > + * License as published by the Free Software Foundation; either > + * version 2.1 of the License, or (at your option) any later version. > + * > + * FFmpeg is distributed in the hope that it will be useful, > + * but WITHOUT ANY WARRANTY; without even the implied warranty of > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > + * Lesser General Public License for more details. > + * > + * You should have received a copy of the GNU Lesser General Public > + * License along with FFmpeg; if not, write to the Free Software > + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA > + */ > + > +#include <float.h> > +#include <stdlib.h> > + > +#include <xeve.h> > + > +#include "libavutil/internal.h" > +#include "libavutil/common.h" > +#include "libavutil/opt.h" > +#include "libavutil/pixdesc.h" > +#include "libavutil/pixfmt.h" > +#include "libavutil/time.h" > +#include "libavutil/cpu.h" > +#include "libavutil/avstring.h" > + > +#include "avcodec.h" > +#include "internal.h" > +#include "packet_internal.h" > +#include "codec_internal.h" > +#include "encode.h" > + > +#define MAX_BS_BUF (16*1024*1024) > + > +/** > + * Error codes > + */ > +#define XEVE_PARAM_BAD_NAME -100 > +#define XEVE_PARAM_BAD_VALUE -200 > + > +/** > + * Encoder states > + * > + * STATE_ENCODING - the encoder receives and processes input frames > + * STATE_BUMPING - there are no more input frames, however the encoder still processes previously received data > + */ > +typedef enum State { > + STATE_ENCODING, > + STATE_BUMPING, > +} State; > + > +/** > + * The structure stores all the states associated with the instance of Xeve MPEG-5 EVC encoder > + */ > +typedef struct XeveContext { > + const AVClass *class; > + > + XEVE id; // XEVE instance identifier > + XEVE_CDSC cdsc; // coding parameters i.e profile, width & height of input frame, num of therads, frame rate ... > + XEVE_BITB bitb; // bitstream buffer (output) > + XEVE_STAT stat; // encoding status (output) > + XEVE_IMGB imgb; // image buffer (input) > + > + State state; // encoder state (skipping, encoding, bumping) > + > + int profile_id; // encoder profile (main, baseline) > + int preset_id; // preset of xeve ( fast, medium, slow, placebo) > + int tune_id; // tune of xeve (psnr, zerolatency) > + > + // variables for rate control modes > + int rc_mode; // Rate control mode [ 0(CQP) / 1(ABR) / 2(CRF) ] > + int qp; // quantization parameter (QP) [0,51] > + int crf; // constant rate factor (CRF) [10,49] > + > + int hash; // embed picture signature (HASH) for conformance checking in decoding > + int sei_info; // embed Supplemental enhancement information while encoding > + > + int color_format; // input data color format: currently only XEVE_CF_YCBCR420 is supported > +} XeveContext; > + > +/** > + * Convert FFmpeg pixel format (AVPixelFormat) to XEVE pre-defined color format > + * > + * @param[in] av_pix_fmt pixel format (@see https://ffmpeg.org/doxygen/trunk/pixfmt_8h.html#a9a8e335cf3be472042bc9f0cf80cd4c5) > + * @param[out] xeve_col_fmt XEVE pre-defined color format (@see xeve.h) > + * > + * @return 0 on success, negative value on failure > + */ > +static int libxeve_color_fmt(enum AVPixelFormat av_pix_fmt, int *xeve_col_fmt) > +{ > + switch (av_pix_fmt) { > + case AV_PIX_FMT_YUV420P: > + *xeve_col_fmt = XEVE_CF_YCBCR420; > + break; > + case AV_PIX_FMT_YUV420P10: > + *xeve_col_fmt = XEVE_CF_YCBCR420; > + break; > + default: > + *xeve_col_fmt = XEVE_CF_UNKNOWN; > + return AVERROR_INVALIDDATA; > + } > + > + return 0; > +} > + > +/** > + * Convert FFmpeg pixel format (AVPixelFormat) into XEVE pre-defined color space > + * > + * @param[in] px_fmt pixel format (@see https://ffmpeg.org/doxygen/trunk/pixfmt_8h.html#a9a8e335cf3be472042bc9f0cf80cd4c5) > + * > + * @return XEVE pre-defined color space (@see xeve.h) on success, XEVE_CF_UNKNOWN on failure > + */ > +static int libxeve_color_space(enum AVPixelFormat av_pix_fmt) > +{ > + /* color space of input image */ > + int cs = XEVE_CF_UNKNOWN; > + > + switch (av_pix_fmt) { > + case AV_PIX_FMT_YUV420P: > + cs = XEVE_CS_YCBCR420; > + break; > + case AV_PIX_FMT_YUV420P10: > +#if AV_HAVE_BIGENDIAN > + cs = XEVE_CS_SET(XEVE_CF_YCBCR420, 10, 1); > +#else > + cs = XEVE_CS_YCBCR420_10LE; > +#endif > + > + break; > + default: > + cs = XEVE_CF_UNKNOWN; > + break; > + } > + > + return cs; > +} > + > +/** > + * The function returns a pointer to the object of the XEVE_CDSC type. > + * XEVE_CDSC contains all encoder parameters that should be initialized before the encoder is used. > + * > + * The field values of the XEVE_CDSC structure are populated based on: > + * - the corresponding field values of the AvCodecConetxt structure, > + * - the xeve encoder specific option values, > + * (the full list of options available for xeve encoder is displayed after executing the command ./ffmpeg --help encoder = libxeve) > + * - and the xeve encoder options specified as a list of key-value pairs following the xeve-params option > + * > + * The order of processing input data and populating the XEVE_CDSC structure > + * 1) first, the fields of the AVCodecContext structure corresponding to the provided input options are processed, > + * (i.e -pix_fmt yuv420p -s:v 1920x1080 -r 30 -profile:v 0) > + * 2) then xeve-specific options added as AVOption to the xeve AVCodec implementation > + * (i.e -preset 0) > + * > + * Keep in mind that, there are options that can be set in different ways. > + * In this case, please follow the above-mentioned order of processing. > + * The most recent assignments overwrite the previous values. > + * > + * @param[in] avctx codec context (AVCodecContext) > + * @param[out] cdsc contains all Xeve MPEG-5 EVC encoder encoder parameters that should be initialized before the encoder is use > + * > + * @return 0 on success, negative error code on failure > + */ > +static int get_conf(AVCodecContext *avctx, XEVE_CDSC *cdsc) > +{ > + XeveContext *xectx = NULL; > + int ret; > + > + xectx = avctx->priv_data; > + > + /* initialize xeve_param struct with default values */ > + ret = xeve_param_default(&cdsc->param); > + if (XEVE_FAILED(ret)) { > + av_log(avctx, AV_LOG_ERROR, "Cannot set_default parameter\n"); > + return AVERROR_EXTERNAL; > + } > + > + /* read options from AVCodecContext */ > + if (avctx->width > 0) > + cdsc->param.w = avctx->width; > + > + if (avctx->height > 0) > + cdsc->param.h = avctx->height; > + > + if (avctx->framerate.num > 0) { > + // fps can be float number, but xeve API doesn't support it > + cdsc->param.fps = (int)(((float)avctx->framerate.num / avctx->framerate.den) + 0.5); > + } > + > + if (avctx->gop_size >= 0) // GOP size (key-frame interval, I-picture period) > + cdsc->param.keyint = avctx->gop_size; // 0: only one I-frame at the first time; 1: every frame is coded in I-frame > + > + if (avctx->max_b_frames == 0 || avctx->max_b_frames == 1 || avctx->max_b_frames == 3 || > + avctx->max_b_frames == 7 || avctx->max_b_frames == 15) // number of b-frames > + cdsc->param.bframes = avctx->max_b_frames; > + else { > + av_log(avctx, AV_LOG_ERROR, "Incorrect value for maximum number of B frames: (%d) \n" > + "Acceptable values for bf option (maximum number of B frames) are 0,1,3,7 or 15\n", avctx->max_b_frames); > + return AVERROR_INVALIDDATA; > + } > + > + if (avctx->level >= 0) > + cdsc->param.level_idc = avctx->level; > + > + if (avctx->rc_buffer_size) // VBV buf size > + cdsc->param.vbv_bufsize = (int)(avctx->rc_buffer_size / 1000); > + > + cdsc->param.rc_type = xectx->rc_mode; > + > + if (xectx->rc_mode == XEVE_RC_CQP) { > + cdsc->param.qp = xectx->qp; > + } else if (xectx->rc_mode == XEVE_RC_ABR) { > + if (avctx->bit_rate / 1000 > INT_MAX || avctx->rc_max_rate / 1000 > INT_MAX) { > + av_log(avctx, AV_LOG_ERROR, "Not supported bitrate bit_rate and rc_max_rate > %d000\n", INT_MAX); > + return AVERROR_INVALIDDATA; > + } > + cdsc->param.bitrate = (int)(avctx->bit_rate / 1000); > + } else if (xectx->rc_mode == XEVE_RC_CRF) { > + cdsc->param.crf = xectx->crf; > + } else { > + av_log(avctx, AV_LOG_ERROR, "Not supported rate control type: %d\n", xectx->rc_mode); > + return AVERROR_INVALIDDATA; > + } > + > + if (avctx->thread_count <= 0) { > + int cpu_count = av_cpu_count(); > + cdsc->param.threads = (cpu_count < XEVE_MAX_THREADS) ? cpu_count : XEVE_MAX_THREADS; > + } else if (avctx->thread_count > XEVE_MAX_THREADS) > + cdsc->param.threads = XEVE_MAX_THREADS; > + else > + cdsc->param.threads = avctx->thread_count; > + > + > + libxeve_color_fmt(avctx->pix_fmt, &xectx->color_format); > + > + cdsc->param.cs = XEVE_CS_SET(xectx->color_format, cdsc->param.codec_bit_depth, AV_HAVE_BIGENDIAN); > + > + cdsc->max_bs_buf_size = MAX_BS_BUF; > + > + ret = xeve_param_ppt(&cdsc->param, xectx->profile_id, xectx->preset_id, xectx->tune_id); > + if (XEVE_FAILED(ret)) { > + av_log(avctx, AV_LOG_ERROR, "Cannot set profile(%d), preset(%d), tune(%d)\n", xectx->profile_id, xectx->preset_id, xectx->tune_id); > + return AVERROR_EXTERNAL; > + } > + > + return 0; > +} > + > +/** > + * Set XEVE_CFG_SET_USE_PIC_SIGNATURE for encoder > + * > + * @param[in] logger context > + * @param[in] id XEVE encodec instance identifier > + * @param[in] ctx the structure stores all the states associated with the instance of Xeve MPEG-5 EVC encoder > + * > + * @return 0 on success, negative error code on failure > + */ > +static int set_extra_config(AVCodecContext *avctx, XEVE id, XeveContext *ctx) > +{ > + int ret, size; > + size = 4; > + > + // embed SEI messages identifying encoder parameters and command line arguments > + // - 0: off\n" > + // - 1: emit sei info" > + // > + // SEI - Supplemental enhancement information contains information > + // that is not necessary to decode the samples of coded pictures from VCL NAL units. > + // Some SEI message information is required to check bitstream conformance > + // and for output timing decoder conformance. > + // @see ISO_IEC_23094-1_2020 7.4.3.5 > + // @see ISO_IEC_23094-1_2020 Annex D > + ret = xeve_config(id, XEVE_CFG_SET_SEI_CMD, &ctx->sei_info, &size); // sei_cmd_info > + if (XEVE_FAILED(ret)) { > + av_log(avctx, AV_LOG_ERROR, "Failed to set config for sei command info messages\n"); > + return AVERROR_EXTERNAL; > + } > + > + ret = xeve_config(id, XEVE_CFG_SET_USE_PIC_SIGNATURE, &ctx->hash, &size); > + if (XEVE_FAILED(ret)) { > + av_log(avctx, AV_LOG_ERROR, "Failed to set config for picture signature\n"); > + return AVERROR_EXTERNAL; > + } > + > + return 0; > +} > + > +/** > + * @brief Switch encoder to bumping mode > + * > + * @param id XEVE encodec instance identifier > + * @return 0 on success, negative error code on failure > + */ > +static int setup_bumping(XEVE id) > +{ > + int val = 1; > + int size = sizeof(int); > + if (XEVE_FAILED(xeve_config(id, XEVE_CFG_SET_FORCE_OUT, (void *)(&val), &size))) > + return AVERROR_EXTERNAL; > + > + return 0; > +} > + > +/** > + * @brief Initialize eXtra-fast Essential Video Encoder codec > + * Create an encoder instance and allocate all the needed resources > + * > + * @param avctx codec context > + * @return 0 on success, negative error code on failure > + */ > +static av_cold int libxeve_init(AVCodecContext *avctx) > +{ > + XeveContext *xectx = avctx->priv_data; > + unsigned char *bs_buf = NULL; > + int i; > + int shift_h = 0; > + int shift_v = 0; > + int width_chroma = 0; > + int height_chroma = 0; > + XEVE_IMGB *imgb = NULL; > + int ret = 0; > + > + XEVE_CDSC *cdsc = &(xectx->cdsc); > + > + /* allocate bitstream buffer */ > + bs_buf = av_malloc(MAX_BS_BUF); > + if (bs_buf == NULL) { > + av_log(avctx, AV_LOG_ERROR, "Cannot allocate bitstream buffer\n"); > + return AVERROR(ENOMEM); > + } > + xectx->bitb.addr = bs_buf; > + xectx->bitb.bsize = MAX_BS_BUF; > + > + /* read configurations and set values for created descriptor (XEVE_CDSC) */ > + if ((ret = get_conf(avctx, cdsc)) != 0) { > + av_log(avctx, AV_LOG_ERROR, "Cannot get configuration\n"); > + return AVERROR(EINVAL); > + } > + > + if ((ret = xeve_param_check(&cdsc->param)) != 0) { > + av_log(avctx, AV_LOG_ERROR, "Invalid configuration\n"); > + return AVERROR(EINVAL); > + } > + > + /* create encoder */ > + xectx->id = xeve_create(cdsc, NULL); > + if (xectx->id == NULL) { > + av_log(avctx, AV_LOG_ERROR, "Cannot create XEVE encoder\n"); > + return AVERROR_EXTERNAL; > + } > + > + if ((ret = set_extra_config(avctx, xectx->id, xectx)) != 0) { > + av_log(avctx, AV_LOG_ERROR, "Cannot set extra configuration\n"); > + return AVERROR(EINVAL); > + } > + > + if ((ret = av_pix_fmt_get_chroma_sub_sample(avctx->pix_fmt, &shift_h, &shift_v)) != 0) { > + av_log(avctx, AV_LOG_ERROR, "Failed to get chroma shift\n"); > + return AVERROR(EINVAL); > + } > + > + // Chroma subsampling > + // > + // YUV format explanation > + // shift_h == 1 && shift_v == 1 : YUV420 > + // shift_h == 1 && shift_v == 0 : YUV422 > + // shift_h == 0 && shift_v == 0 : YUV444 > + // > + width_chroma = AV_CEIL_RSHIFT(avctx->width, shift_h); > + height_chroma = AV_CEIL_RSHIFT(avctx->height, shift_v); > + > + /* set default values for input image buffer */ > + imgb = &xectx->imgb; > + imgb->cs = libxeve_color_space(avctx->pix_fmt); > + imgb->np = 3; /* only for yuv420p, yuv420ple */ > + > + for (i = 0; i < imgb->np; i++) > + imgb->x[i] = imgb->y[i] = 0; > + > + imgb->w[0] = imgb->aw[0] = avctx->width; // width luma > + imgb->w[1] = imgb->w[2] = imgb->aw[1] = imgb->aw[2] = width_chroma; > + imgb->h[0] = imgb->ah[0] = avctx->height; // height luma > + imgb->h[1] = imgb->h[2] = imgb->ah[1] = imgb->ah[2] = height_chroma; > + > + xectx->state = STATE_ENCODING; > + > + return 0; > +} > + > +/** > + * Encode raw data frame into EVC packet > + * > + * @param[in] avctx codec context > + * @param[out] pkt output AVPacket containing encoded data > + * @param[in] frame AVFrame containing the raw data to be encoded > + * @param[out] got_packet encoder sets to 0 or 1 to indicate that a > + * non-empty packet was returned in pkt > + * > + * @return 0 on success, negative error code on failure > + */ > +static int libxeve_encode(AVCodecContext *avctx, AVPacket *avpkt, > + const AVFrame *frame, int *got_packet) > +{ > + XeveContext *xectx = avctx->priv_data; > + int ret = -1; > + > + // No more input frames are available but encoder still can have some data in its internal buffer to process > + // and some frames to dump. > + if (xectx->state == STATE_ENCODING && frame == NULL) { > + if (setup_bumping(xectx->id) == 0) > + xectx->state = STATE_BUMPING; // Entering bumping process > + else { > + av_log(avctx, AV_LOG_ERROR, "Failed to setup bumping\n"); > + return 0; > + } > + } > + > + if (xectx->state == STATE_ENCODING) { > + int i; > + XEVE_IMGB *imgb = NULL; > + > + imgb = &xectx->imgb; > + > + for (i = 0; i < imgb->np; i++) { > + imgb->a[i] = frame->data[i]; > + imgb->s[i] = frame->linesize[i]; > + } > + > + imgb->ts[0] = frame->pts; > + imgb->ts[1] = 0; > + > + /* push image to encoder */ > + ret = xeve_push(xectx->id, imgb); > + if (XEVE_FAILED(ret)) { > + av_log(avctx, AV_LOG_ERROR, "xeve_push() failed\n"); > + return AVERROR_EXTERNAL; > + } > + } > + if (xectx->state == STATE_ENCODING || xectx->state == STATE_BUMPING) { > + /* encoding */ > + ret = xeve_encode(xectx->id, &(xectx->bitb), &(xectx->stat)); > + if (XEVE_FAILED(ret)) { > + av_log(avctx, AV_LOG_ERROR, "xeve_encode() failed\n"); > + return AVERROR_EXTERNAL; > + } > + > + /* store bitstream */ > + if (ret == XEVE_OK_OUT_NOT_AVAILABLE) { // Return OK but picture is not available yet > + *got_packet = 0; > + return 0; > + } else if (ret == XEVE_OK) { > + int av_pic_type; > + > + if (xectx->stat.write > 0) { > + > + ret = ff_get_encode_buffer(avctx, avpkt, xectx->stat.write, 0); > + if (ret < 0) > + return ret; > + > + memcpy(avpkt->data, xectx->bitb.addr, xectx->stat.write); > + > + avpkt->pts = xectx->bitb.ts[0]; > + avpkt->dts = xectx->bitb.ts[1]; > + > + switch(xectx->stat.stype) { > + case XEVE_ST_I: > + av_pic_type = AV_PICTURE_TYPE_I; > + avpkt->flags |= AV_PKT_FLAG_KEY; > + break; > + case XEVE_ST_P: > + av_pic_type = AV_PICTURE_TYPE_P; > + break; > + case XEVE_ST_B: > + av_pic_type = AV_PICTURE_TYPE_B; > + break; > + case XEVE_ST_UNKNOWN: > + av_log(avctx, AV_LOG_ERROR, "Unknown slice type\n"); > + return AVERROR_INVALIDDATA; > + } > + > + ff_side_data_set_encoder_stats(avpkt, xectx->stat.qp * FF_QP2LAMBDA, NULL, 0, av_pic_type); > + > + *got_packet = 1; > + } > + } else if (ret == XEVE_OK_NO_MORE_FRM) { > + // Return OK but no more frames > + return 0; > + } else { > + av_log(avctx, AV_LOG_ERROR, "Invalid return value: %d\n", ret); > + return AVERROR_EXTERNAL; > + } > + } else { > + av_log(avctx, AV_LOG_ERROR, "Udefined encoder state\n"); > + return AVERROR_INVALIDDATA; > + } > + > + return 0; > +} > + > +/** > + * Destroy the encoder and release all the allocated resources > + * > + * @param avctx codec context > + * @return 0 on success, negative error code on failure > + */ > +static av_cold int libxeve_close(AVCodecContext *avctx) > +{ > + XeveContext *xectx = avctx->priv_data; > + > + xeve_delete(xectx->id); You intend to set the init-cleanup flag on this encoder. But then you have to account for the possibility of this function being called with a XeveContext that has not been successfully setup, i.e. xectx->id might be NULL. Is this allowed? ### DK ### Of course, it's not allowed. Thank you for pointing me to this issue. It should be definitely checked whether xectx->id is NULL pointer before passing it to xeve_delete function. Otherwise, execution can be stopped due to assertion inside xeve_delete function. > + > + av_free(xectx->bitb.addr); /* release bitstream buffer */ > + > + return 0; > +} > + > +#define OFFSET(x) offsetof(XeveContext, x) > +#define VE AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_ENCODING_PARAM > + > +static const enum AVPixelFormat supported_pixel_formats[] = { > + AV_PIX_FMT_YUV420P, > + AV_PIX_FMT_YUV420P10, > + AV_PIX_FMT_NONE > +}; > + > +// Example of using: ./ffmpeg -xeve-params "m=2:q=17" > +// Consider using following options (./ffmpeg --help encoder=libxeve) > +// > +static const AVOption libxeve_options[] = { > + { "preset", "Encoding preset for setting encoding speed", OFFSET(preset_id), AV_OPT_TYPE_INT, { .i64 = XEVE_PRESET_MEDIUM }, XEVE_PRESET_DEFAULT, XEVE_PRESET_PLACEBO, VE, "preset" }, > + { "default", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_PRESET_DEFAULT }, INT_MIN, INT_MAX, VE, "preset" }, > + { "fast", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_PRESET_FAST }, INT_MIN, INT_MAX, VE, "preset" }, > + { "medium", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_PRESET_MEDIUM }, INT_MIN, INT_MAX, VE, "preset" }, > + { "slow", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_PRESET_SLOW }, INT_MIN, INT_MAX, VE, "preset" }, > + { "placebo", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_PRESET_PLACEBO }, INT_MIN, INT_MAX, VE, "preset" }, > + { "tune", "Tuning parameter for special purpose operation", OFFSET(tune_id), AV_OPT_TYPE_INT, { .i64 = XEVE_TUNE_NONE }, XEVE_TUNE_NONE, XEVE_TUNE_PSNR, VE, "tune"}, > + { "none", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_TUNE_NONE }, INT_MIN, INT_MAX, VE, "tune" }, > + { "zerolatency", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_TUNE_ZEROLATENCY }, INT_MIN, INT_MAX, VE, "tune" }, > + { "psnr", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_TUNE_PSNR }, INT_MIN, INT_MAX, VE, "tune" }, > + { "profile", "Encoding profile", OFFSET(profile_id), AV_OPT_TYPE_INT, { .i64 = XEVE_PROFILE_BASELINE }, XEVE_PROFILE_BASELINE, XEVE_PROFILE_MAIN, VE, "profile" }, > + { "baseline", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_PROFILE_BASELINE }, INT_MIN, INT_MAX, VE, "profile" }, > + { "main", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_PROFILE_MAIN }, INT_MIN, INT_MAX, VE, "profile" }, > + { "rc_mode", "Rate control mode", OFFSET(rc_mode), AV_OPT_TYPE_INT, { .i64 = XEVE_RC_CQP }, XEVE_RC_CQP, XEVE_RC_CRF, VE, "rc_mode" }, > + { "CQP", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_RC_CQP }, INT_MIN, INT_MAX, VE, "rc_mode" }, > + { "ABR", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_RC_ABR }, INT_MIN, INT_MAX, VE, "rc_mode" }, > + { "CRF", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_RC_CRF }, INT_MIN, INT_MAX, VE, "rc_mode" }, > + { "qp", "Quantization parameter value for CQP rate control mode", OFFSET(qp), AV_OPT_TYPE_INT, { .i64 = 32 }, 0, 51, VE }, > + { "crf", "Constant rate factor value for CRF rate control mode", OFFSET(crf), AV_OPT_TYPE_INT, { .i64 = 32 }, 10, 49, VE }, > + { "hash", "Embed picture signature (HASH) for conformance checking in decoding", OFFSET(hash), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, 1, VE }, > + { "sei_info", "Embed SEI messages identifying encoder parameters and command line arguments", OFFSET(sei_info), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, 1, VE }, > + { NULL } > +}; > + > +static const AVClass libxeve_class = { > + .class_name = "libxeve", > + .item_name = av_default_item_name, > + .option = libxeve_options, > + .version = LIBAVUTIL_VERSION_INT, > +}; > + > +/** > + * libavcodec generic global options, which can be set on all the encoders and decoders > + * @see https://www.ffmpeg.org/ffmpeg-codecs.html#Codec-Options > + */ > +static const FFCodecDefault libxeve_defaults[] = { > + { "b", "0" }, // bitrate in terms of kilo-bits per second > + { "g", "0" }, // gop_size (key-frame interval 0: only one I-frame at the first time; 1: every frame is coded in I-frame) > + { "bf", "15"}, // maximum number of B frames (0: no B-frames, 1,3,7,15) > + { "threads", "0"}, // number of threads to be used (0: automatically select the number of threads to set) > + { NULL }, > +}; > + > +const FFCodec ff_libxeve_encoder = { > + .p.name = "libxeve", > + .p.long_name = NULL_IF_CONFIG_SMALL("libxeve MPEG-5 EVC"), > + .p.type = AVMEDIA_TYPE_VIDEO, > + .p.id = AV_CODEC_ID_EVC, > + .init = libxeve_init, > + FF_CODEC_ENCODE_CB(libxeve_encode), > + .close = libxeve_close, > + .priv_data_size = sizeof(XeveContext), > + .p.priv_class = &libxeve_class, > + .defaults = libxeve_defaults, > + .p.capabilities = FF_CODEC_CAP_INIT_CLEANUP | AV_CODEC_CAP_DELAY | AV_CODEC_CAP_AUTO_THREADS | AV_CODEC_CAP_DR1, 1. This is completely wring: FF_CODEC_CAP_INIT_CLEANUP is an internal capability flag, not something public. That's why it has an FF_ prefix. Internal capabilities should go into FFCodec.caps_internal. (In other words: s/.p.capabilities/.caps_internal/) ### DK ### I'll move the flag to FFCodec.caps_internal. ######### 2. Due to 21b23ceab3cfcc2aa0a833bbff11cf38b2177800, a codec that has not the FF_CODEC_CAP_NOT_INIT_THREADSAFE internal cap set is presumed to be init-threadsafe (this changed since the last iteration of your patchset). Is it guaranteed and documented that libxeve and libxevd satisfy are init-threadsafe (i.e. there are no races if multiple encoders/decoders (each with their own context) are initialized concurrently)? ### DK ### You are absolutely right. There is no official/public information confirming that libxeve and libxevd libraries are thread safe. I will try to find out more but by the time I will use FF_CODEC_CAP_NOT_INIT_THREADSAFE flag for both encoder and decoder. ######### > + .p.wrapper_name = "libxeve", > + .p.pix_fmts = supported_pixel_formats, > +}; > diff --git a/libavcodec/parsers.c b/libavcodec/parsers.c > index a8d52af6cb..ca00c63176 100644 > --- a/libavcodec/parsers.c > +++ b/libavcodec/parsers.c > @@ -41,6 +41,7 @@ extern const AVCodecParser ff_dvaudio_parser; > extern const AVCodecParser ff_dvbsub_parser; > extern const AVCodecParser ff_dvdsub_parser; > extern const AVCodecParser ff_dvd_nav_parser; > +extern const AVCodecParser ff_evc_parser; > extern const AVCodecParser ff_flac_parser; > extern const AVCodecParser ff_g723_1_parser; > extern const AVCodecParser ff_g729_parser; > diff --git a/libavcodec/profiles.c b/libavcodec/profiles.c > index 7af7fbeb13..a31244e0db 100644 > --- a/libavcodec/profiles.c > +++ b/libavcodec/profiles.c > @@ -181,4 +181,10 @@ const AVProfile ff_arib_caption_profiles[] = { > { FF_PROFILE_UNKNOWN } > }; > > +const AVProfile ff_evc_profiles[] = { > + { FF_PROFILE_EVC_BASELINE, "Baseline" }, > + { FF_PROFILE_EVC_MAIN, "Main" }, > + { FF_PROFILE_UNKNOWN }, > +}; > + > #endif /* !CONFIG_SMALL */ > diff --git a/libavcodec/profiles.h b/libavcodec/profiles.h > index 41a19aa9ad..cf92b5f126 100644 > --- a/libavcodec/profiles.h > +++ b/libavcodec/profiles.h > @@ -72,5 +72,6 @@ extern const AVProfile ff_sbc_profiles[]; > extern const AVProfile ff_prores_profiles[]; > extern const AVProfile ff_mjpeg_profiles[]; > extern const AVProfile ff_arib_caption_profiles[]; > +extern const AVProfile ff_evc_profiles[]; > > #endif /* AVCODEC_PROFILES_H */ > diff --git a/libavcodec/version.h b/libavcodec/version.h > index e488eee355..2328be4b26 100644 > --- a/libavcodec/version.h > +++ b/libavcodec/version.h > @@ -29,7 +29,7 @@ > > #include "version_major.h" > > -#define LIBAVCODEC_VERSION_MINOR 42 > +#define LIBAVCODEC_VERSION_MINOR 43 > #define LIBAVCODEC_VERSION_MICRO 100 > > #define LIBAVCODEC_VERSION_INT AV_VERSION_INT(LIBAVCODEC_VERSION_MAJOR, \ _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [FFmpeg-devel] [PATCH 1/2] Provided support for MPEG-5 EVC (Essential Video Coding) codec 2022-08-11 7:44 ` Andreas Rheinhardt 2022-08-11 11:28 ` Dawid Kozinski/Robot SDK (PLT) /SRPOL/Staff Engineer/삼성전자 @ 2022-08-18 9:00 ` Dawid Kozinski/Robot SDK (PLT) /SRPOL/Staff Engineer/삼성전자 1 sibling, 0 replies; 13+ messages in thread From: Dawid Kozinski/Robot SDK (PLT) /SRPOL/Staff Engineer/삼성전자 @ 2022-08-18 9:00 UTC (permalink / raw) To: 'FFmpeg development discussions and patches' Hi, Thank you for your code review and for all your valuable comments and remarks. We have done a lot of changes to our EVC wrapper implementation following them, and I think each of these changes have moved our wrapper one step forwards toward a final community-acceptable solution. A few days ago I submitted new patches providing EVC wrapper implementation. If there is still something we have to change or improve, please let me know. I'm looking forward to receiving your feedback. As I mentioned before we would like to start getting closer to the point where we can be able to merge our code into the FFmpeg master branch. :) Once again, thank you for your effort and for supporting us and we are waiting for your feedback. Best regards Dawid -----Original Message----- From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Andreas Rheinhardt Sent: Thursday, August 11, 2022 9:45 AM To: ffmpeg-devel@ffmpeg.org Subject: Re: [FFmpeg-devel] [PATCH 1/2] Provided support for MPEG-5 EVC (Essential Video Coding) codec Dawid Kozinski: > - Added new entry to codec IDs list > - Added new entry to the codec descriptor list > - Bumped libavcodec minor version > - Changes in Changelog and MAINTAINERS files > - Added xeve encoder wrapper > - Added xevd dencoder wrapper > - Added documentation for xeve and xevd wrappers > - Added parser for EVC format > - Changes in project configuration file and libavcodec Makefile > > Signed-off-by: Dawid Kozinski <d.kozinski@samsung.com> > --- > Changelog | 3 +- > MAINTAINERS | 2 + > configure | 8 + > doc/decoders.texi | 24 ++ > doc/encoders.texi | 92 +++++ > doc/general_contents.texi | 19 + > libavcodec/Makefile | 3 + > libavcodec/allcodecs.c | 2 + > libavcodec/avcodec.h | 3 + > libavcodec/codec_desc.c | 8 + > libavcodec/codec_id.h | 1 + > libavcodec/evc_parser.c | 758 ++++++++++++++++++++++++++++++++++++++ > libavcodec/libxevd.c | 419 +++++++++++++++++++++ > libavcodec/libxeve.c | 597 ++++++++++++++++++++++++++++++ > libavcodec/parsers.c | 1 + > libavcodec/profiles.c | 6 + > libavcodec/profiles.h | 1 + > libavcodec/version.h | 2 +- > 18 files changed, 1947 insertions(+), 2 deletions(-) > create mode 100644 libavcodec/evc_parser.c > create mode 100644 libavcodec/libxevd.c > create mode 100644 libavcodec/libxeve.c > > diff --git a/Changelog b/Changelog > index cc271c22bd..dbeb08c373 100644 > --- a/Changelog > +++ b/Changelog > @@ -82,7 +82,8 @@ version 5.0: > - VideoToolbox ProRes encoder > - anlmf audio filter > - IMF demuxer (experimental) > - > +- eXtra-fast Essential Video Encoder (XEVE) > +- eXtra-fast Essential Video Decoder (XEVD) > > version 4.4: > - AudioToolbox output device > diff --git a/MAINTAINERS b/MAINTAINERS > index 7ed15f96f6..7d4d543a13 100644 > --- a/MAINTAINERS > +++ b/MAINTAINERS > @@ -206,6 +206,7 @@ Codecs: > libvpx* James Zern > libxavs.c Stefan Gehrer > libxavs2.c Huiwen Ren > + libxev*.c, evc_parser.c Dawid Kozinski > libzvbi-teletextdec.c Marton Balint > lzo.h, lzo.c Reimar Doeffinger > mdec.c Michael Niedermayer > @@ -426,6 +427,7 @@ Muxers/Demuxers: > dv.c Roman Shaposhnik > electronicarts.c Peter Ross > epafdec.c Paul B Mahol > + evcdec.c Dawid Kozinski > ffm* Baptiste Coudurier > flic.c Mike Melanson > flvdec.c Michael Niedermayer > diff --git a/configure b/configure > index cbbb4dd9c8..e6582eae91 100755 > --- a/configure > +++ b/configure > @@ -291,6 +291,8 @@ External library support: > --enable-libwebp enable WebP encoding via libwebp [no] > --enable-libx264 enable H.264 encoding via x264 [no] > --enable-libx265 enable HEVC encoding via x265 [no] > + --enable-libxeve enable EVC encoding via libxeve [no] > + --enable-libxevd enable EVC decoding via libxevd [no] > --enable-libxavs enable AVS encoding via xavs [no] > --enable-libxavs2 enable AVS2 encoding via xavs2 [no] > --enable-libxcb enable X11 grabbing using XCB [autodetect] > @@ -1874,6 +1876,8 @@ EXTERNAL_LIBRARY_LIST=" > libvorbis > libvpx > libwebp > + libxevd > + libxeve > libxml2 > libzimg > libzmq > @@ -3395,6 +3399,8 @@ libx264rgb_encoder_select="libx264_encoder" > libx265_encoder_deps="libx265" > libxavs_encoder_deps="libxavs" > libxavs2_encoder_deps="libxavs2" > +libxevd_decoder_deps="libxevd" > +libxeve_encoder_deps="libxeve" > libxvid_encoder_deps="libxvid" > libzvbi_teletext_decoder_deps="libzvbi" > vapoursynth_demuxer_deps="vapoursynth" > @@ -6685,6 +6691,8 @@ enabled libx265 && require_pkg_config libx265 x265 x265.h x265_api_get > require_cpp_condition libx265 x265.h "X265_BUILD >= 70" > enabled libxavs && require libxavs "stdint.h xavs.h" xavs_encoder_encode "-lxavs $pthreads_extralibs $libm_extralibs" > enabled libxavs2 && require_pkg_config libxavs2 "xavs2 >= 1.3.0" "stdint.h xavs2.h" xavs2_api_get > +enabled libxevd && require_pkg_config libxevd "xevd >= 0.4.0" "xevd.h" xevd_decode > +enabled libxeve && require_pkg_config libxeve "xeve >= 0.4.0" "xeve.h" xeve_encode > enabled libxvid && require libxvid xvid.h xvid_global -lxvidcore > enabled libzimg && require_pkg_config libzimg "zimg >= 2.7.0" zimg.h zimg_get_api_version > enabled libzmq && require_pkg_config libzmq "libzmq >= 4.2.1" zmq.h zmq_ctx_new > diff --git a/doc/decoders.texi b/doc/decoders.texi > index e2fcbf5dc9..0bd9096114 100644 > --- a/doc/decoders.texi > +++ b/doc/decoders.texi > @@ -126,6 +126,30 @@ Set amount of frame threads to use during decoding. The default value is 0 (auto > > @end table > > +@section libxevd > + > +eXtra-fast Essential Video Decoder (XEVD) MPEG-5 EVC decoder wrapper. > + > +This decoder requires the presence of the libxevd headers and library > +during configuration. You need to explicitly configure the build with > +@option{--enable-libxevd}. > + > +The xevd project website is at @url{https://protect2.fireeye.com/v1/url?k=0cdf05ab-53443cce-0cde8ee4-000babff32e3-d7c41d94be1006e7&q=1&e=b504c66d-387e-4b60-8cbe-1f53ce8a1ea8&u=https%3A%2F%2Fgithub.com%2Fmpeg5%2Fxevd%257D. > + > +@subsection Options > + > +The following options are supported by the libxevd wrapper. > +The xevd-equivalent options or values are listed in parentheses for easy migration. > + > +To get a more accurate and extensive documentation of the libxevd options, > +invoke the command @code{xevd_app --help} or consult the libxevd documentation. > + > +@table @option > +@item threads (@emph{threads}) > +Force to use a specific number of threads > + > +@end table > + > @section QSV Decoders > > The family of Intel QuickSync Video decoders (VC1, MPEG-2, H.264, HEVC, > diff --git a/doc/encoders.texi b/doc/encoders.texi > index 6d73f74196..76f55e6721 100644 > --- a/doc/encoders.texi > +++ b/doc/encoders.texi > @@ -2892,6 +2892,98 @@ ffmpeg -i input -c:v libxavs2 -xavs2-params RdoqLevel=0 output.avs2 > @end example > @end table > > +@section libxeve > + > +eXtra-fast Essential Video Encoder (XEVE) MPEG-5 EVC encoder wrapper. > +The xeve-equivalent options or values are listed in parentheses for easy migration. > + > +This encoder requires the presence of the libxeve headers and library > +during configuration. You need to explicitly configure the build with > +@option{--enable-libxeve}. > + > +@float NOTE > +Many libxeve encoder options are mapped to FFmpeg global codec options, > +while unique encoder options are provided through private options. > +Additionally the xeve-params private options allows one to pass a list > +of key=value tuples as accepted by the libxeve @code{parse_xeve_params} function. > +@end float > + > +The xeve project website is at @url{https://protect2.fireeye.com/v1/url?k=a21df818-fd86c17d-a21c7357-000babff32e3-481c230dcbcfe69f&q=1&e=b504c66d-387e-4b60-8cbe-1f53ce8a1ea8&u=https%3A%2F%2Fgithub.com%2Fmpeg5%2Fxeve%257D. > + > +@subsection Options > + > +The following options are supported by the libxeve wrapper. > +The xeve-equivalent options or values are listed in parentheses for easy migration. > + > +@float NOTE > +To reduce the duplication of documentation, only the private options > +and some others requiring special attention are documented here. For > +the documentation of the undocumented generic options, see > +@ref{codec-options,,the Codec Options chapter}. > +@end float > + > +@float NOTE > +To get a more accurate and extensive documentation of the libxeve options, > +invoke the command @code{xeve_app --help} or consult the libxeve documentation. > +@end float > + > +@table @option > +@item b (@emph{bitrate}) > +Set target video bitrate in bits/s. > +Note that FFmpeg's b option is expressed in bits/s, while xeve's bitrate is in kilobits/s. > + > +@item bf (@emph{bframes}) > +Set the maximum number of B frames (1,3,7,15). > + > +@item g (@emph{keyint}) > +Set the GOP size (I-picture period). > + > +@item preset (@emph{preset}) > +Set the xeve preset. > +Set the encoder preset value to determine encoding speed [fast, medium, slow, placebo] > + > +@item tune (@emph{tune}) > +Set the encoder tune parameter [psnr, zerolatency] > + > +@item profile (@emph{profile}) > +Set the encoder profile [0: baselie; 1: main] > + > +@item crf (@emph{crf}) > +Set the quality for constant quality mode. > +Constant rate factor <10..49> [default: 32] > + > +@item qp (@emph{qp}) > +Set constant quantization rate control method parameter. > +Quantization parameter qp <0..51> [default: 32] > + > +@item threads (@emph{threads}) > +Force to use a specific number of threads > + > +@item xeve-params > +Set xeve options using a list of @var{key}=@var{value} couples separated > +by ":". > + > +For example to specify libxeve encoding options with @option{-xeve-params}: > + > +@example > +ffmpeg -i input -c:v libxeve -xeve-params profile=main:tune=zerolatency:preset=fast:q=17:bframes=15:bitrate=100k output.mp4 > +@end example > +@end table > + > +@subsubsection Supported key values for xeve-params > + > +Set xeve options using a list of @var{key}=@var{value} couples separated > + > +@table @option > + > +@item hash > +Embed the picture signature (HASH) for conformance checking in decoding, (0: Off, 1: On) > + > +@item vbv-bufsize > +Set the VBV buffer size: Kbits(none,K,k), Mbits(M,m) > + > +@end table > + > @section libxvid > > Xvid MPEG-4 Part 2 encoder wrapper. > diff --git a/doc/general_contents.texi b/doc/general_contents.texi > index 86ec6d606b..89236047ea 100644 > --- a/doc/general_contents.texi > +++ b/doc/general_contents.texi > @@ -343,6 +343,22 @@ libxavs2 is under the GNU Public License Version 2 or later > details), you must upgrade FFmpeg's license to GPL in order to use it. > @end float > > +@section eXtra-fast Essential Video Encoder (XEVE) > + > +FFmpeg can make use of the XEVE library for EVC video encoding. > + > +Go to @url{https://protect2.fireeye.com/v1/url?k=315aae41-6ec19724-315b250e-000babff32e3-ecfea0332f2dcd7d&q=1&e=b504c66d-387e-4b60-8cbe-1f53ce8a1ea8&u=https%3A%2F%2Fgithub.com%2Fmpeg5%2Fxeve%257D and follow the instructions for > +installing the XEVE library. Then pass @code{--enable-libxeve} to configure to > +enable it. > + > +@section eXtra-fast Essential Video Decoder (XEVD) > + > +FFmpeg can make use of the XEVD library for EVC video decoding. > + > +Go to @url{https://protect2.fireeye.com/v1/url?k=39d72b05-664c1260-39d6a04a-000babff32e3-5e00c6377e7cf34c&q=1&e=b504c66d-387e-4b60-8cbe-1f53ce8a1ea8&u=https%3A%2F%2Fgithub.com%2Fmpeg5%2Fxevd%257D and follow the instructions for > +installing the XEVD library. Then pass @code{--enable-libxevd} to configure to > +enable it. > + > @section ZVBI > > ZVBI is a VBI decoding library which can be used by FFmpeg to decode DVB > @@ -591,6 +607,7 @@ library: > @item raw DTS @tab X @tab X > @item raw DTS-HD @tab @tab X > @item raw E-AC-3 @tab X @tab X > +@item raw EVC @tab X @tab X > @item raw FLAC @tab X @tab X > @item raw GSM @tab @tab X > @item raw H.261 @tab X @tab X > @@ -930,6 +947,8 @@ following image formats are supported: > @item Electronic Arts TQI video @tab @tab X > @item Escape 124 @tab @tab X > @item Escape 130 @tab @tab X > +@item EVC / MPEG-5 Part 1 @tab X @tab X > + @tab encoding and decoding supported through external libraries libxeve and libxevd > @item FFmpeg video codec #1 @tab X @tab X > @tab lossless codec (fourcc: FFV1) > @item Flash Screen Video v1 @tab X @tab X > diff --git a/libavcodec/Makefile b/libavcodec/Makefile > index 83901bb52b..cfeacd96d1 100644 > --- a/libavcodec/Makefile > +++ b/libavcodec/Makefile > @@ -1102,6 +1102,8 @@ OBJS-$(CONFIG_LIBX264_ENCODER) += libx264.o > OBJS-$(CONFIG_LIBX265_ENCODER) += libx265.o > OBJS-$(CONFIG_LIBXAVS_ENCODER) += libxavs.o > OBJS-$(CONFIG_LIBXAVS2_ENCODER) += libxavs2.o > +OBJS-$(CONFIG_LIBXEVD_DECODER) += libxevd.o > +OBJS-$(CONFIG_LIBXEVE_ENCODER) += libxeve.o > OBJS-$(CONFIG_LIBXVID_ENCODER) += libxvid.o > OBJS-$(CONFIG_LIBZVBI_TELETEXT_DECODER) += libzvbi-teletextdec.o ass.o > > @@ -1129,6 +1131,7 @@ OBJS-$(CONFIG_DVAUDIO_PARSER) += dvaudio_parser.o > OBJS-$(CONFIG_DVBSUB_PARSER) += dvbsub_parser.o > OBJS-$(CONFIG_DVD_NAV_PARSER) += dvd_nav_parser.o > OBJS-$(CONFIG_DVDSUB_PARSER) += dvdsub_parser.o > +OBJS-$(CONFIG_EVC_PARSER) += evc_parser.o > OBJS-$(CONFIG_FLAC_PARSER) += flac_parser.o flacdata.o flac.o > OBJS-$(CONFIG_G723_1_PARSER) += g723_1_parser.o > OBJS-$(CONFIG_G729_PARSER) += g729_parser.o > diff --git a/libavcodec/allcodecs.c b/libavcodec/allcodecs.c > index c94e2d5966..2526e11f70 100644 > --- a/libavcodec/allcodecs.c > +++ b/libavcodec/allcodecs.c > @@ -800,6 +800,8 @@ extern LIBX264_CONST FFCodec ff_libx264_encoder; > #endif > extern const FFCodec ff_libx264rgb_encoder; > extern FFCodec ff_libx265_encoder; > +extern const FFCodec ff_libxeve_encoder; > +extern const FFCodec ff_libxevd_decoder; > extern const FFCodec ff_libxavs_encoder; > extern const FFCodec ff_libxavs2_encoder; > extern const FFCodec ff_libxvid_encoder; > diff --git a/libavcodec/avcodec.h b/libavcodec/avcodec.h > index 65c8535359..7d56a03fca 100644 > --- a/libavcodec/avcodec.h > +++ b/libavcodec/avcodec.h > @@ -1674,6 +1674,9 @@ typedef struct AVCodecContext { > #define FF_PROFILE_KLVA_SYNC 0 > #define FF_PROFILE_KLVA_ASYNC 1 > > +#define FF_PROFILE_EVC_BASELINE 0 > +#define FF_PROFILE_EVC_MAIN 1 > + > /** > * level > * - encoding: Set by user. > diff --git a/libavcodec/codec_desc.c b/libavcodec/codec_desc.c > index c1a177c22d..1a43ff8cb6 100644 > --- a/libavcodec/codec_desc.c > +++ b/libavcodec/codec_desc.c > @@ -1907,6 +1907,14 @@ static const AVCodecDescriptor codec_descriptors[] = { > .long_name = NULL_IF_CONFIG_SMALL("WBMP (Wireless Application Protocol Bitmap) image"), > .props = AV_CODEC_PROP_INTRA_ONLY | AV_CODEC_PROP_LOSSLESS, > }, > + { > + .id = AV_CODEC_ID_EVC, > + .type = AVMEDIA_TYPE_VIDEO, > + .name = "evc", > + .long_name = NULL_IF_CONFIG_SMALL("MPEG-5 EVC (Essential Video Coding)"), > + .props = AV_CODEC_PROP_LOSSY | AV_CODEC_PROP_REORDER, > + .profiles = NULL_IF_CONFIG_SMALL(ff_evc_profiles), > + }, > > /* various PCM "codecs" */ > { > diff --git a/libavcodec/codec_id.h b/libavcodec/codec_id.h > index 386a00a7ef..02b60be690 100644 > --- a/libavcodec/codec_id.h > +++ b/libavcodec/codec_id.h > @@ -314,6 +314,7 @@ enum AVCodecID { > AV_CODEC_ID_PHM, > AV_CODEC_ID_RADIANCE_HDR, > AV_CODEC_ID_WBMP, > + AV_CODEC_ID_EVC, > > /* various PCM "codecs" */ > AV_CODEC_ID_FIRST_AUDIO = 0x10000, ///< A dummy id pointing at the start of audio codecs > diff --git a/libavcodec/evc_parser.c b/libavcodec/evc_parser.c > new file mode 100644 > index 0000000000..0ae0eaa08a > --- /dev/null > +++ b/libavcodec/evc_parser.c > @@ -0,0 +1,758 @@ > +/* > + * EVC format parser > + * > + * Copyright (C) 2021 Dawid Kozinski <d.kozinski@samsung.com> > + * > + * This file is part of FFmpeg. > + * > + * FFmpeg is free software; you can redistribute it and/or > + * modify it under the terms of the GNU Lesser General Public > + * License as published by the Free Software Foundation; either > + * version 2.1 of the License, or (at your option) any later version. > + * > + * FFmpeg is distributed in the hope that it will be useful, > + * but WITHOUT ANY WARRANTY; without even the implied warranty of > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > + * Lesser General Public License for more details. > + * > + * You should have received a copy of the GNU Lesser General Public > + * License along with FFmpeg; if not, write to the Free Software > + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA > + */ > + > +#include <stdint.h> > + > +#include "libavutil/common.h" > + > +#include "parser.h" > +#include "golomb.h" > + > +// The length field that indicates the length in bytes of the following NAL unit is configured to be of 4 bytes > +#define EVC_NAL_UNIT_LENGTH_BYTE (4) /* byte */ > + > +#define EVC_NAL_HEADER_SIZE (2) /* byte */ > +#define MAX_SPS_CNT (16) /* defined value in EVC standard */ > +#define MAX_PPS_CNT (64) /* defined value in EVC standard */ > + > +#define MAX_NUM_TILES_ROW (22) > +#define MAX_NUM_TILES_COL (20) > + > +// NALU types > +// @see ISO_IEC_23094-1_2020 7.4.2.2 NAL unit header semantics > +// > +#define EVC_NUT_NONIDR (0) /* Coded slice of a non-IDR picture */ > +#define EVC_NUT_IDR (1) /* Coded slice of an IDR picture */ > +#define EVC_NUT_SPS (24) /* Sequence parameter set */ > +#define EVC_NUT_PPS (25) /* Picture paremeter set */ > +#define EVC_NUT_APS (26) /* Adaptation parameter set */ > +#define EVC_NUT_FD (27) /* Filler data */ > +#define EVC_NUT_SEI (28) /* Supplemental enhancement information */ > + > +// slice type > +// @see ISO_IEC_23094-1_2020 7.4.5 Slice header semantics > +// > +#define SLICE_TYPE_UNKNOWN (-1) > +#define SLICE_TYPE_B (0) > +#define SLICE_TYPE_P (1) > +#define SLICE_TYPE_I (2) > + > +#define MAX_NUM_RPLS (32) > +#define MAX_NUM_REF_PICS (21) > + > +// rpl structure > +typedef struct RefPicListStruct { > + int poc; > + int tid; > + int ref_pic_num; > + int ref_pic_active_num; > + int ref_pics[MAX_NUM_REF_PICS]; > + char pic_type; > + > +} RefPicListStruct; > + > +// The sturcture reflects SPS RBSP(raw byte sequence payload) layout > +// @see ISO_IEC_23094-1 section 7.3.2.1 > +// > +// The following descriptors specify the parsing process of each element > +// u(n) - unsigned integer using n bits > +// ue(v) - unsigned integer 0-th order Exp_Golomb-coded syntax element with the left bit first > +typedef struct EVCParserSPS { > + int sps_seq_parameter_set_id; // ue(v) > + int profile_idc; // u(8) > + int level_idc; // u(8) > + int toolset_idc_h; // u(32) > + int toolset_idc_l; // u(32) > + int chroma_format_idc; // ue(v) > + int pic_width_in_luma_samples; // ue(v) > + int pic_height_in_luma_samples; // ue(v) > + int bit_depth_luma_minus8; // ue(v) > + int bit_depth_chroma_minus8; // ue(v) > + > + int sps_btt_flag; // u(1) > + int log2_ctu_size_minus5; // ue(v) > + int log2_min_cb_size_minus2; // ue(v) > + int log2_diff_ctu_max_14_cb_size; // ue(v) > + int log2_diff_ctu_max_tt_cb_size; // ue(v) > + int log2_diff_min_cb_min_tt_cb_size_minus2; // ue(v) > + > + int sps_suco_flag; // u(1) > + int log2_diff_ctu_size_max_suco_cb_size; // ue(v) > + int log2_diff_max_suco_min_suco_cb_size; // ue(v) > + > + int sps_admvp_flag; // u(1) > + int sps_affine_flag; // u(1) > + int sps_amvr_flag; // u(1) > + int sps_dmvr_flag; // u(1) > + int sps_mmvd_flag; // u(1) > + int sps_hmvp_flag; // u(1) > + > + int sps_eipd_flag; // u(1) > + int sps_ibc_flag; // u(1) > + int log2_max_ibc_cand_size_minus2; // ue(v) > + > + int sps_cm_init_flag; // u(1) > + int sps_adcc_flag; // u(1) > + > + int sps_iqt_flag; // u(1) > + int sps_ats_flag; // u(1) > + > + int sps_addb_flag; // u(1) > + int sps_alf_flag; // u(1) > + int sps_htdf_flag; // u(1) > + int sps_rpl_flag; // u(1) > + int sps_pocs_flag; // u(1) > + int sps_dquant_flag; // u(1) > + int sps_dra_flag; // u(1) > + > + int log2_max_pic_order_cnt_lsb_minus4; // ue(v) > + int log2_sub_gop_length; // ue(v) > + int log2_ref_pic_gap_length; // ue(v) > + > + int max_num_tid0_ref_pics; // ue(v) > + > + int sps_max_dec_pic_buffering_minus1; // ue(v) > + int long_term_ref_pic_flag; // u(1) > + int rpl1_same_as_rpl0_flag; // u(1) > + int num_ref_pic_list_in_sps[2]; // ue(v) > + RefPicListStruct rpls[2][MAX_NUM_RPLS]; > + > + int picture_cropping_flag; // u(1) > + int picture_crop_left_offset; // ue(v) > + int picture_crop_right_offset; // ue(v) > + int picture_crop_top_offset; // ue(v) > + int picture_crop_bottom_offset; // ue(v) > + > + // @note > + // Currently the structure does not reflect the entire SPS RBSP layout. > + // It contains only the fields that are necessary to read from the NAL unit all the values > + // necessary for the correct initialization of the AVCodecContext structure. > + > + // @note > + // If necessary, add the missing fields to the structure to reflect > + // the contents of the entire NAL unit of the SPS type > + > +} EVCParserSPS; > + > +typedef struct EVCParserPPS { > + int pps_pic_parameter_set_id; // ue(v) > + int pps_seq_parameter_set_id; // ue(v) > + int num_ref_idx_default_active_minus1[2]; // ue(v) > + int additional_lt_poc_lsb_len; // ue(v) > + int rpl1_idx_present_flag; // u(1) > + int single_tile_in_pic_flag; // u(1) > + int num_tile_columns_minus1; // ue(v) > + int num_tile_rows_minus1; // ue(v) > + int uniform_tile_spacing_flag; // u(1) > + int tile_column_width_minus1[MAX_NUM_TILES_ROW]; // ue(v) > + int tile_row_height_minus1[MAX_NUM_TILES_COL]; // ue(v) > + int loop_filter_across_tiles_enabled_flag; // u(1) > + int tile_offset_len_minus1; // ue(v) > + int tile_id_len_minus1; // ue(v) > + int explicit_tile_id_flag; // u(1) > + int tile_id_val[MAX_NUM_TILES_ROW][MAX_NUM_TILES_COL]; // u(v) > + int pic_dra_enabled_flag; // u(1) > + int pic_dra_aps_id; // u(5) > + int arbitrary_slice_present_flag; // u(1) > + int constrained_intra_pred_flag; // u(1) > + int cu_qp_delta_enabled_flag; // u(1) > + int log2_cu_qp_delta_area_minus6; // ue(v) > + > +} EVCParserPPS; > + > +// The sturcture reflects Slice Header RBSP(raw byte sequence payload) layout > +// @see ISO_IEC_23094-1 section 7.3.2.6 > +// > +// The following descriptors specify the parsing process of each element > +// u(n) - unsigned integer using n bits > +// ue(v) - unsigned integer 0-th order Exp_Golomb-coded syntax element with the left bit first > +// u(n) - unsigned integer using n bits. > +// When n is "v" in the syntax table, the number of bits varies in a manner dependent on the value of other syntax elements. > +typedef struct EVCParserSliceHeader { > + int slice_pic_parameter_set_id; // ue(v) > + int single_tile_in_slice_flag; // u(1) > + int first_tile_id; // u(v) > + int arbitrary_slice_flag; // u(1) > + int last_tile_id; // u(v) > + int num_remaining_tiles_in_slice_minus1; // ue(v) > + int delta_tile_id_minus1[MAX_NUM_TILES_ROW * MAX_NUM_TILES_COL]; // ue(v) > + > + int slice_type; // ue(v) > + > + // @note > + // Currently the structure does not reflect the entire Slice Header RBSP layout. > + // It contains only the fields that are necessary to read from the NAL unit all the values > + // necessary for the correct initialization of the AVCodecContext structure. > + > + // @note > + // If necessary, add the missing fields to the structure to reflect > + // the contents of the entire NAL unit of the SPS type > + > +} EVCParserSliceHeader; > + > +typedef struct EVCParserContext { > + ParseContext pc; > + EVCParserSPS sps[MAX_SPS_CNT]; > + EVCParserPPS pps[MAX_PPS_CNT]; > + EVCParserSliceHeader slice_header[MAX_PPS_CNT]; > + int is_avc; > + int nal_length_size; > + int to_read; > + int incomplete_nalu_prefix_read; // The flag is set to 1 when an incomplete NAL unit prefix has been read > + > +} EVCParserContext; > + > +static int get_nalu_type(const uint8_t *bits, int bits_size, AVCodecContext *avctx) > +{ > + int unit_type_plus1 = 0; > + > + if (bits_size >= EVC_NAL_HEADER_SIZE) { > + unsigned char *p = (unsigned char *)bits; > + // forbidden_zero_bit > + if ((p[0] & 0x80) != 0) > + return -1; > + > + // nal_unit_type > + unit_type_plus1 = (p[0] >> 1) & 0x3F; > + } > + > + return unit_type_plus1 - 1; > +} > + > +static uint32_t read_nal_unit_length(const uint8_t *bits, int bits_size, AVCodecContext *avctx) > +{ > + uint32_t nalu_len = 0; > + > + if (bits_size >= EVC_NAL_UNIT_LENGTH_BYTE) { > + > + int t = 0; > + unsigned char *p = (unsigned char *)bits; > + > + for (int i = 0; i < EVC_NAL_UNIT_LENGTH_BYTE; i++) > + t = (t << 8) | p[i]; > + > + nalu_len = t; > + if (nalu_len == 0) > + return 0; > + } > + > + return nalu_len; > +} > + > +// @see ISO_IEC_23094-1 (7.3.2.1 SPS RBSP syntax) > +static EVCParserSPS *parse_sps(const uint8_t *bs, int bs_size, EVCParserContext *ev) > +{ > + GetBitContext gb; > + EVCParserSPS *sps; > + int sps_seq_parameter_set_id; > + > + if (init_get_bits8(&gb, bs, bs_size) < 0) > + return NULL; > + > + sps_seq_parameter_set_id = get_ue_golomb(&gb); > + > + if (sps_seq_parameter_set_id >= MAX_SPS_CNT) > + return NULL; > + > + sps = &ev->sps[sps_seq_parameter_set_id]; > + sps->sps_seq_parameter_set_id = sps_seq_parameter_set_id; > + > + // the Baseline profile is indicated by profile_idc eqal to 0 > + // the Main profile is indicated by profile_idc eqal to 1 > + sps->profile_idc = get_bits(&gb, 8); > + > + sps->level_idc = get_bits(&gb, 8); > + > + skip_bits_long(&gb, 32); /* skip toolset_idc_h */ > + skip_bits_long(&gb, 32); /* skip toolset_idc_l */ > + > + // 0 - monochrome > + // 1 - 4:2:0 > + // 2 - 4:2:2 > + // 3 - 4:4:4 > + sps->chroma_format_idc = get_ue_golomb(&gb); > + > + sps->pic_width_in_luma_samples = get_ue_golomb(&gb); > + sps->pic_height_in_luma_samples = get_ue_golomb(&gb); > + > + sps->bit_depth_luma_minus8 = get_ue_golomb(&gb); > + sps->bit_depth_chroma_minus8 = get_ue_golomb(&gb); > + > + sps->sps_btt_flag = get_bits(&gb, 1); > + if (sps->sps_btt_flag) { > + sps->log2_ctu_size_minus5 = get_ue_golomb(&gb); > + sps->log2_min_cb_size_minus2 = get_ue_golomb(&gb); > + sps->log2_diff_ctu_max_14_cb_size = get_ue_golomb(&gb); > + sps->log2_diff_ctu_max_tt_cb_size = get_ue_golomb(&gb); > + sps->log2_diff_min_cb_min_tt_cb_size_minus2 = get_ue_golomb(&gb); > + } > + > + sps->sps_suco_flag = get_bits(&gb, 1); > + if (sps->sps_suco_flag) { > + sps->log2_diff_ctu_size_max_suco_cb_size = get_ue_golomb(&gb); > + sps->log2_diff_max_suco_min_suco_cb_size = get_ue_golomb(&gb); > + } > + > + sps->sps_admvp_flag = get_bits(&gb, 1); > + if (sps->sps_admvp_flag) { > + sps->sps_affine_flag = get_bits(&gb, 1); > + sps->sps_amvr_flag = get_bits(&gb, 1); > + sps->sps_dmvr_flag = get_bits(&gb, 1); > + sps->sps_mmvd_flag = get_bits(&gb, 1); > + sps->sps_hmvp_flag = get_bits(&gb, 1); > + } > + > + sps->sps_eipd_flag = get_bits(&gb, 1); > + if (sps->sps_eipd_flag) { > + sps->sps_ibc_flag = get_bits(&gb, 1); > + if (sps->sps_ibc_flag) > + sps->log2_max_ibc_cand_size_minus2 = get_ue_golomb(&gb); > + } > + > + sps->sps_cm_init_flag = get_bits(&gb, 1); > + if (sps->sps_cm_init_flag) > + sps->sps_adcc_flag = get_bits(&gb, 1); > + > + sps->sps_iqt_flag = get_bits(&gb, 1); > + if (sps->sps_iqt_flag) > + sps->sps_ats_flag = get_bits(&gb, 1); > + > + sps->sps_addb_flag = get_bits(&gb, 1); > + sps->sps_alf_flag = get_bits(&gb, 1); > + sps->sps_htdf_flag = get_bits(&gb, 1); > + sps->sps_rpl_flag = get_bits(&gb, 1); > + sps->sps_pocs_flag = get_bits(&gb, 1); > + sps->sps_dquant_flag = get_bits(&gb, 1); > + sps->sps_dra_flag = get_bits(&gb, 1); > + > + if (sps->sps_pocs_flag) > + sps->log2_max_pic_order_cnt_lsb_minus4 = get_ue_golomb(&gb); > + > + if (!sps->sps_pocs_flag || !sps->sps_rpl_flag) { > + sps->log2_sub_gop_length = get_ue_golomb(&gb); > + if (sps->log2_sub_gop_length == 0) > + sps->log2_ref_pic_gap_length = get_ue_golomb(&gb); > + } > + > + // @note > + // If necessary, add the missing fields to the EVCParserSPS structure > + // and then extend parser implementation > + > + return sps; > +} > + > +// @see ISO_IEC_23094-1 (7.3.2.2 SPS RBSP syntax) > +static EVCParserPPS *parse_pps(const uint8_t *bs, int bs_size, EVCParserContext *ev) > +{ > + GetBitContext gb; > + EVCParserPPS *pps; > + > + int pps_pic_parameter_set_id; > + > + if (init_get_bits8(&gb, bs, bs_size) < 0) > + return NULL; > + > + pps_pic_parameter_set_id = get_ue_golomb(&gb); > + if (pps_pic_parameter_set_id > MAX_PPS_CNT) > + return NULL; > + > + pps = &ev->pps[pps_pic_parameter_set_id]; > + > + pps->pps_pic_parameter_set_id = pps_pic_parameter_set_id; > + > + pps->pps_seq_parameter_set_id = get_ue_golomb(&gb); > + if (pps->pps_seq_parameter_set_id >= MAX_SPS_CNT) > + return NULL; > + > + pps->num_ref_idx_default_active_minus1[0] = get_ue_golomb(&gb); > + pps->num_ref_idx_default_active_minus1[1] = get_ue_golomb(&gb); > + pps->additional_lt_poc_lsb_len = get_ue_golomb(&gb); > + pps->rpl1_idx_present_flag = get_bits(&gb, 1); > + pps->single_tile_in_pic_flag = get_bits(&gb, 1); > + > + if (!pps->single_tile_in_pic_flag) { > + pps->num_tile_columns_minus1 = get_ue_golomb(&gb); > + pps->num_tile_rows_minus1 = get_ue_golomb(&gb); > + pps->uniform_tile_spacing_flag = get_bits(&gb, 1); > + > + if (!pps->uniform_tile_spacing_flag) { > + for (int i = 0; i < pps->num_tile_columns_minus1; i++) > + pps->tile_column_width_minus1[i] = get_ue_golomb(&gb); > + > + for (int i = 0; i < pps->num_tile_rows_minus1; i++) > + pps->tile_row_height_minus1[i] = get_ue_golomb(&gb); > + } > + pps->loop_filter_across_tiles_enabled_flag = get_bits(&gb, 1); > + pps->tile_offset_len_minus1 = get_ue_golomb(&gb); > + } > + > + pps->tile_id_len_minus1 = get_ue_golomb(&gb); > + pps->explicit_tile_id_flag = get_bits(&gb, 1); > + > + if (pps->explicit_tile_id_flag) { > + for (int i = 0; i <= pps->num_tile_rows_minus1; i++) { > + for (int j = 0; j <= pps->num_tile_columns_minus1; j++) > + pps->tile_id_val[i][j] = get_bits(&gb, pps->tile_id_len_minus1 + 1); > + } > + } > + > + pps->pic_dra_enabled_flag = 0; > + pps->pic_dra_enabled_flag = get_bits(&gb, 1); > + > + if (pps->pic_dra_enabled_flag) > + pps->pic_dra_aps_id = get_bits(&gb, 5); > + > + pps->arbitrary_slice_present_flag = get_bits(&gb, 1); > + pps->constrained_intra_pred_flag = get_bits(&gb, 1); > + pps->cu_qp_delta_enabled_flag = get_bits(&gb, 1); > + > + if (pps->cu_qp_delta_enabled_flag) > + pps->log2_cu_qp_delta_area_minus6 = get_ue_golomb(&gb); > + > + return pps; > +} > + > +// @see ISO_IEC_23094-1 (7.3.2.6 Slice layer RBSP syntax) > +static EVCParserSliceHeader *parse_slice_header(const uint8_t *bs, int bs_size, EVCParserContext *ev) > +{ > + GetBitContext gb; > + EVCParserSliceHeader *sh; > + EVCParserPPS *pps; > + int num_tiles_in_slice = 0; > + int slice_pic_parameter_set_id; > + > + if (init_get_bits8(&gb, bs, bs_size) < 0) > + return NULL; > + > + slice_pic_parameter_set_id = get_ue_golomb(&gb); > + > + if (slice_pic_parameter_set_id < 0 || slice_pic_parameter_set_id >= MAX_PPS_CNT) > + return NULL; > + > + sh = &ev->slice_header[slice_pic_parameter_set_id]; > + pps = &ev->pps[slice_pic_parameter_set_id]; > + > + sh->slice_pic_parameter_set_id = slice_pic_parameter_set_id; > + > + if (!pps->single_tile_in_pic_flag) { > + sh->single_tile_in_slice_flag = get_bits(&gb, 1); > + sh->first_tile_id = get_bits(&gb, pps->tile_id_len_minus1 + 1); > + } else > + sh->single_tile_in_slice_flag = 1; > + > + if (!sh->single_tile_in_slice_flag) { > + if (pps->arbitrary_slice_present_flag) > + sh->arbitrary_slice_flag = get_bits(&gb, 1); > + > + if (!sh->arbitrary_slice_flag) > + sh->last_tile_id = get_bits(&gb, pps->tile_id_len_minus1 + 1); > + else { > + sh->num_remaining_tiles_in_slice_minus1 = get_ue_golomb(&gb); > + num_tiles_in_slice = sh->num_remaining_tiles_in_slice_minus1 + 2; > + for (int i = 0; i < num_tiles_in_slice - 1; ++i) > + sh->delta_tile_id_minus1[i] = get_ue_golomb(&gb); > + } > + } > + > + sh->slice_type = get_ue_golomb(&gb); > + > + // @note > + // If necessary, add the missing fields to the EVCParserSliceHeader structure > + // and then extend parser implementation > + > + return sh; > +} > + > +static int parse_nal_units(AVCodecParserContext *s, const uint8_t *bs, > + int bs_size, AVCodecContext *avctx) > +{ > + EVCParserContext *ev = s->priv_data; > + int nalu_type, nalu_size; > + unsigned char *bits = (unsigned char *)bs; > + int bits_size = bs_size; > + > + avctx->codec_id = AV_CODEC_ID_EVC; > + s->pict_type = AV_PICTURE_TYPE_NONE; > + s->picture_structure = AV_PICTURE_STRUCTURE_FRAME; > + s->key_frame = -1; > + > + nalu_size = read_nal_unit_length(bits, bits_size, avctx); > + if (nalu_size == 0) { > + av_log(avctx, AV_LOG_ERROR, "Invalid NAL unit size: (%d)\n", nalu_size); > + return -1; > + } > + > + bits += EVC_NAL_UNIT_LENGTH_BYTE; > + bits_size -= EVC_NAL_UNIT_LENGTH_BYTE; > + > + nalu_type = get_nalu_type(bits, bits_size, avctx); > + > + bits += EVC_NAL_HEADER_SIZE; > + bits_size -= EVC_NAL_HEADER_SIZE; > + > + if (nalu_type == EVC_NUT_SPS) { // NAL Unit type: SPS (Sequence Parameter Set) > + EVCParserSPS *sps; > + > + sps = parse_sps(bits, bits_size, ev); > + if (!sps) { > + av_log(avctx, AV_LOG_ERROR, "SPS parsing error\n"); > + return -1; > + } > + > + s->coded_width = sps->pic_width_in_luma_samples; > + s->coded_height = sps->pic_height_in_luma_samples; > + s->width = sps->pic_width_in_luma_samples; > + s->height = sps->pic_height_in_luma_samples; > + > + if (sps->profile_idc == 1) avctx->profile = FF_PROFILE_EVC_MAIN; > + else avctx->profile = FF_PROFILE_EVC_BASELINE; > + > + // Currently XEVD decoder supports ony YCBCR420_10LE chroma format for EVC stream > + switch (sps->chroma_format_idc) { > + case 0: /* YCBCR400_10LE */ > + av_log(avctx, AV_LOG_ERROR, "YCBCR400_10LE: Not supported chroma format\n"); > + s->format = AV_PIX_FMT_GRAY10LE; > + return -1; > + case 1: /* YCBCR420_10LE */ > + s->format = AV_PIX_FMT_YUV420P10LE; > + break; > + case 2: /* YCBCR422_10LE */ > + av_log(avctx, AV_LOG_ERROR, "YCBCR422_10LE: Not supported chroma format\n"); > + s->format = AV_PIX_FMT_YUV422P10LE; > + return -1; > + case 3: /* YCBCR444_10LE */ > + av_log(avctx, AV_LOG_ERROR, "YCBCR444_10LE: Not supported chroma format\n"); > + s->format = AV_PIX_FMT_YUV444P10LE; > + return -1; > + default: > + s->format = AV_PIX_FMT_NONE; > + av_log(avctx, AV_LOG_ERROR, "Unknown supported chroma format\n"); > + return -1; > + } > + > + // @note > + // The current implementation of parse_sps function doesn't handle VUI parameters parsing. > + // If it will be needed, parse_sps function could be extended to handle VUI parameters parsing > + // to initialize fields of the AVCodecContex i.e. color_primaries, color_trc,color_range > + > + } else if (nalu_type == EVC_NUT_PPS) { // NAL Unit type: PPS (Video Parameter Set) > + EVCParserPPS *pps; > + > + pps = parse_pps(bits, bits_size, ev); > + if (!pps) { > + av_log(avctx, AV_LOG_ERROR, "PPS parsing error\n"); > + return -1; > + } > + } else if (nalu_type == EVC_NUT_SEI) // NAL unit type: SEI (Supplemental Enhancement Information) > + return 0; > + else if (nalu_type == EVC_NUT_IDR || nalu_type == EVC_NUT_NONIDR) { // NAL Unit type: Coded slice of a IDR or non-IDR picture > + EVCParserSliceHeader *sh; > + > + sh = parse_slice_header(bits, bits_size, ev); > + if (!sh) { > + av_log(avctx, AV_LOG_ERROR, "Slice header parsing error\n"); > + return -1; > + } > + switch (sh->slice_type) { > + case SLICE_TYPE_B: { > + s->pict_type = AV_PICTURE_TYPE_B; > + break; > + } > + case SLICE_TYPE_P: { > + s->pict_type = AV_PICTURE_TYPE_P; > + break; > + } > + case SLICE_TYPE_I: { > + s->pict_type = AV_PICTURE_TYPE_I; > + break; > + } > + default: { > + s->pict_type = AV_PICTURE_TYPE_NONE; > + } > + } > + s->key_frame = (nalu_type == EVC_NUT_IDR) ? 1 : 0; > + } else { > + av_log(avctx, AV_LOG_ERROR, "Invalid NAL unit type: %d\n", nalu_type); > + return -1; > + } > + > + return 0; > +} > + > +/** > + * Find the end of the current frame in the bitstream. > + * @return the position of the first byte of the next frame, or END_NOT_FOUND > + */ > +static int evc_find_frame_end(AVCodecParserContext *s, const uint8_t *buf, > + int buf_size, AVCodecContext *avctx) > +{ > + EVCParserContext *ev = s->priv_data; > + > + if (!ev->to_read) { > + int nal_unit_size = 0; > + int next = END_NOT_FOUND; > + > + // This is the case when buffer size is not enough for buffer to store NAL unit length > + if (buf_size < EVC_NAL_UNIT_LENGTH_BYTE) { > + ev->to_read = EVC_NAL_UNIT_LENGTH_BYTE; > + ev->nal_length_size = buf_size; > + ev->incomplete_nalu_prefix_read = 1; > + > + return END_NOT_FOUND; > + } > + > + nal_unit_size = read_nal_unit_length(buf, buf_size, avctx); > + ev->nal_length_size = EVC_NAL_UNIT_LENGTH_BYTE; > + > + next = nal_unit_size + EVC_NAL_UNIT_LENGTH_BYTE; > + ev->to_read = next; > + if (next < buf_size) > + return next; > + else > + return END_NOT_FOUND; > + } else if (ev->to_read > buf_size) > + return END_NOT_FOUND; > + else { > + if (ev->incomplete_nalu_prefix_read == 1) { > + EVCParserContext *ev = s->priv_data; > + ParseContext *pc = &ev->pc; > + uint8_t nalu_len[EVC_NAL_UNIT_LENGTH_BYTE] = {0}; > + int nal_unit_size = 0; > + > + // 1. pc->buffer contains previously read bytes of NALU prefix > + // 2. buf contains the rest of NAL unit prefix bytes > + // > + // ~~~~~~~ > + // EXAMPLE > + // ~~~~~~~ > + // > + // In the following example we assumed that the number of already read NAL Unit prefix bytes is equal 1 > + // > + // ---------- > + // pc->buffer -> conatins already read bytes > + // ---------- > + // __ pc->index == 1 > + // | > + // V > + // ------------------------------------------------------- > + // | 0 | 1 | 2 | 3 | 4 | ... | N | > + // ------------------------------------------------------- > + // | 0x00 | 0xXX | 0xXX | 0xXX | 0xXX | ... | 0xXX | > + // ------------------------------------------------------- > + // > + // ---------- > + // buf -> contains newly read bytes > + // ---------- > + // ------------------------------------------------------- > + // | 0 | 1 | 2 | 3 | 4 | ... | N | > + // ------------------------------------------------------- > + // | 0x00 | 0x00 | 0x3C | 0xXX | 0xXX | ... | 0xXX | > + // ------------------------------------------------------- > + // > + for (int i = 0; i < EVC_NAL_UNIT_LENGTH_BYTE; i++) { > + if (i < pc->index) > + nalu_len[i] = pc->buffer[i]; > + else > + nalu_len[i] = buf[i - pc->index]; > + } > + > + // ---------- > + // nalu_len > + // ---------- > + // --------------------------------- > + // | 0 | 1 | 2 | 3 | > + // --------------------------------- > + // | 0x00 | 0x00 | 0x00 | 0x3C | > + // --------------------------------- > + // | NALU LENGTH | > + // --------------------------------- > + // NAL Unit lenght = 60 (0x0000003C) > + > + nal_unit_size = read_nal_unit_length(nalu_len, EVC_NAL_UNIT_LENGTH_BYTE, avctx); > + > + ev->to_read = nal_unit_size + EVC_NAL_UNIT_LENGTH_BYTE - pc->index; > + > + ev->incomplete_nalu_prefix_read = 0; > + > + if (ev->to_read > buf_size) > + return END_NOT_FOUND; > + else > + return ev->to_read; > + } > + return ev->to_read; > + } > + > + return END_NOT_FOUND; > +} > + > +static int evc_parser_init(AVCodecParserContext *s) > +{ > + EVCParserContext *ev = s->priv_data; > + > + ev->nal_length_size = EVC_NAL_UNIT_LENGTH_BYTE; > + ev->incomplete_nalu_prefix_read = 0; > + > + return 0; > +} > + > +static int evc_parse(AVCodecParserContext *s, AVCodecContext *avctx, > + const uint8_t **poutbuf, int *poutbuf_size, > + const uint8_t *buf, int buf_size) > +{ > + int next; > + EVCParserContext *ev = s->priv_data; > + ParseContext *pc = &ev->pc; > + int is_dummy_buf = !buf_size; > + const uint8_t *dummy_buf = buf; > + > + if (s->flags & PARSER_FLAG_COMPLETE_FRAMES) > + next = buf_size; > + else { > + next = evc_find_frame_end(s, buf, buf_size, avctx); > + if (ff_combine_frame(pc, next, &buf, &buf_size) < 0) { > + *poutbuf = NULL; > + *poutbuf_size = 0; > + ev->to_read -= buf_size; > + return buf_size; > + } > + } > + > + is_dummy_buf &= (dummy_buf == buf); > + > + if (!is_dummy_buf) > + parse_nal_units(s, buf, buf_size, avctx); > + > + // poutbuf contains just one NAL unit > + *poutbuf = buf; > + *poutbuf_size = buf_size; > + ev->to_read -= next; > + > + return next; > +} > + > +const AVCodecParser ff_evc_parser = { > + .codec_ids = { AV_CODEC_ID_EVC }, > + .priv_data_size = sizeof(EVCParserContext), > + .parser_init = evc_parser_init, > + .parser_parse = evc_parse, > + .parser_close = ff_parse_close, > +}; > diff --git a/libavcodec/libxevd.c b/libavcodec/libxevd.c > new file mode 100644 > index 0000000000..85f71affd3 > --- /dev/null > +++ b/libavcodec/libxevd.c > @@ -0,0 +1,419 @@ > +/* > + * libxevd decoder > + * EVC (MPEG-5 Essential Video Coding) decoding using XEVD MPEG-5 EVC decoder library > + * > + * Copyright (C) 2021 Dawid Kozinski <d.kozinski@samsung.com> > + * > + * This file is part of FFmpeg. > + * > + * FFmpeg is free software; you can redistribute it and/or > + * modify it under the terms of the GNU Lesser General Public > + * License as published by the Free Software Foundation; either > + * version 2.1 of the License, or (at your option) any later version. > + * > + * FFmpeg is distributed in the hope that it will be useful, > + * but WITHOUT ANY WARRANTY; without even the implied warranty of > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > + * Lesser General Public License for more details. > + * > + * You should have received a copy of the GNU Lesser General Public > + * License along with FFmpeg; if not, write to the Free Software > + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA > + */ > + > +#include <float.h> > +#include <stdlib.h> > + > +#include <xevd.h> > + > +#include "libavutil/internal.h" > +#include "libavutil/common.h" > +#include "libavutil/opt.h" > +#include "libavutil/pixdesc.h" > +#include "libavutil/pixfmt.h" > +#include "libavutil/imgutils.h" > +#include "libavutil/cpu.h" > + > +#include "avcodec.h" > +#include "internal.h" > +#include "packet_internal.h" > +#include "codec_internal.h" > + > +#define XEVD_PARAM_BAD_NAME -1 > +#define XEVD_PARAM_BAD_VALUE -2 > + > +#define EVC_NAL_HEADER_SIZE 2 /* byte */ > + > +/** > + * The structure stores all the states associated with the instance of Xeve MPEG-5 EVC decoder > + */ > +typedef struct XevdContext { > + const AVClass *class; > + > + XEVD id; // XEVD instance identifier @see xevd.h > + XEVD_CDSC cdsc; // decoding parameters @see xevd.h > +} XevdContext; > + > +/** > + * The function populates the XEVD_CDSC structure. > + * XEVD_CDSC contains all decoder parameters that should be initialized before its use. > + * > + * @param[in] avctx codec context > + * @param[out] cdsc contains all decoder parameters that should be initialized before its use > + * > + */ > +static void get_conf(AVCodecContext *avctx, XEVD_CDSC *cdsc) > +{ > + int cpu_count = av_cpu_count(); > + > + /* clear XEVS_CDSC structure */ > + memset(cdsc, 0, sizeof(XEVD_CDSC)); > + > + /* init XEVD_CDSC */ > + if (avctx->thread_count <= 0) > + cdsc->threads = (cpu_count < XEVD_MAX_TASK_CNT) ? cpu_count : XEVD_MAX_TASK_CNT; > + else if (avctx->thread_count > XEVD_MAX_TASK_CNT) > + cdsc->threads = XEVD_MAX_TASK_CNT; > + else > + cdsc->threads = avctx->thread_count; > +} > + > +/** > + * Read NAL unit length > + * @param bs input data (bitstream) > + * @return the length of NAL unit on success, 0 value on failure > + */ > +static uint32_t read_nal_unit_length(const uint8_t *bs, int bs_size, AVCodecContext *avctx) > +{ > + uint32_t len = 0; > + XEVD_INFO info; > + int ret; > + > + if (bs_size == XEVD_NAL_UNIT_LENGTH_BYTE) { > + ret = xevd_info((void *)bs, XEVD_NAL_UNIT_LENGTH_BYTE, 1, &info); > + if (XEVD_FAILED(ret)) { > + av_log(avctx, AV_LOG_ERROR, "Cannot get bitstream information\n"); > + return 0; > + } > + len = info.nalu_len; > + if (len == 0) { > + av_log(avctx, AV_LOG_ERROR, "Invalid bitstream size! [%d]\n", bs_size); > + return 0; > + } > + } > + > + return len; > +} > + > +/** > + * @param[in] xectx the structure that stores all the state associated with the instance of Xeve MPEG-5 EVC decoder > + * @param[out] avctx codec context > + * @return 0 on success, negative value on failure > + */ > +static int export_stream_params(const XevdContext *xectx, AVCodecContext *avctx) > +{ > + int ret; > + int size; > + int color_space; > + > + avctx->pix_fmt = AV_PIX_FMT_YUV420P10; > + > + size = 4; > + ret = xevd_config(xectx->id, XEVD_CFG_GET_CODED_WIDTH, &avctx->coded_width, &size); > + if (XEVD_FAILED(ret)) { > + av_log(avctx, AV_LOG_ERROR, "Failed to get coded_width\n"); > + return AVERROR_EXTERNAL; > + } > + > + ret = xevd_config(xectx->id, XEVD_CFG_GET_CODED_HEIGHT, &avctx->coded_height, &size); > + if (XEVD_FAILED(ret)) { > + av_log(avctx, AV_LOG_ERROR, "Failed to get coded_height\n"); > + return AVERROR_EXTERNAL; > + } > + > + ret = xevd_config(xectx->id, XEVD_CFG_GET_WIDTH, &avctx->width, &size); > + if (XEVD_FAILED(ret)) { > + av_log(avctx, AV_LOG_ERROR, "Failed to get width\n"); > + return AVERROR_EXTERNAL; > + } > + > + ret = xevd_config(xectx->id, XEVD_CFG_GET_HEIGHT, &avctx->height, &size); > + if (XEVD_FAILED(ret)) { > + av_log(avctx, AV_LOG_ERROR, "Failed to get height\n"); > + return AVERROR_EXTERNAL; > + } > + > + ret = xevd_config(xectx->id, XEVD_CFG_GET_COLOR_SPACE, &color_space, &size); > + if (XEVD_FAILED(ret)) { > + av_log(avctx, AV_LOG_ERROR, "Failed to get color_space\n"); > + return AVERROR_EXTERNAL; > + } > + switch(color_space) { > + case XEVD_CS_YCBCR400_10LE: > + avctx->pix_fmt = AV_PIX_FMT_GRAY10LE; > + break; > + case XEVD_CS_YCBCR420_10LE: > + avctx->pix_fmt = AV_PIX_FMT_YUV420P10LE; > + break; > + case XEVD_CS_YCBCR422_10LE: > + avctx->pix_fmt = AV_PIX_FMT_YUV422P10LE; > + break; > + case XEVD_CS_YCBCR444_10LE: > + avctx->pix_fmt = AV_PIX_FMT_YUV444P10LE; > + break; > + default: > + av_log(avctx, AV_LOG_ERROR, "Unknown color space\n"); > + avctx->pix_fmt = AV_PIX_FMT_NONE; > + return AVERROR_INVALIDDATA; > + } > + > + // the function returns sps->num_reorder_pics > + ret = xevd_config(xectx->id, XEVD_CFG_GET_MAX_CODING_DELAY, &avctx->max_b_frames, &size); > + if (XEVD_FAILED(ret)) { > + av_log(avctx, AV_LOG_ERROR, "Failed to get max_coding_delay\n"); > + return AVERROR_EXTERNAL; > + } > + > + avctx->has_b_frames = (avctx->max_b_frames) ? 1 : 0; > + > + avctx->color_primaries = AVCOL_PRI_UNSPECIFIED; > + avctx->color_trc = AVCOL_TRC_UNSPECIFIED; > + avctx->colorspace = AVCOL_SPC_UNSPECIFIED; > + > + return 0; > +} > + > +/** > + * @brief Copy image in imgb to frame. > + * > + * @param avctx codec context > + * @param[in] imgb > + * @param[out] frame > + * @return 0 on success, negative value on failure > + */ > +static int libxevd_image_copy(struct AVCodecContext *avctx, XEVD_IMGB *imgb, struct AVFrame *frame) > +{ > + int ret; > + if (imgb->cs != XEVD_CS_YCBCR420_10LE) { > + av_log(avctx, AV_LOG_ERROR, "Not supported pixel format: %s\n", av_get_pix_fmt_name(avctx->pix_fmt)); > + return AVERROR_INVALIDDATA; > + } > + > + if (imgb->w[0] != avctx->width || imgb->h[0] != avctx->height) { // stream resolution changed > + if (ff_set_dimensions(avctx, imgb->w[0], imgb->h[0]) < 0) { > + av_log(avctx, AV_LOG_ERROR, "Cannot set new dimension\n"); > + return AVERROR_INVALIDDATA; > + } > + } > + > + if (ret = ff_get_buffer(avctx, frame, 0) < 0) { > + return ret; > + } > + > + av_image_copy(frame->data, frame->linesize, (const uint8_t **)imgb->a, > + imgb->s, avctx->pix_fmt, > + imgb->w[0], imgb->h[0]); > + > + return 0; > +} > + > +/** > + * Initialize decoder > + * Create a decoder instance and allocate all the needed resources > + * > + * @param avctx codec context > + * @return 0 on success, negative error code on failure > + */ > +static av_cold int libxevd_init(AVCodecContext *avctx) > +{ > + XevdContext *xectx = avctx->priv_data; > + XEVD_CDSC *cdsc = &(xectx->cdsc); > + > + /* read configurations and set values for created descriptor (XEVD_CDSC) */ > + get_conf(avctx, cdsc); > + > + /* create decoder */ > + xectx->id = xevd_create(&(xectx->cdsc), NULL); > + if (xectx->id == NULL) { > + av_log(avctx, AV_LOG_ERROR, "Cannot create XEVD encoder\n"); > + return AVERROR_EXTERNAL; > + } > + > + return 0; > +} > + > +/** > + * Decode picture > + * > + * @param avctx codec context > + * @param[out] frame decoded frame > + * @param[out] got_frame_ptr decoder sets to 0 or 1 to indicate that a > + * non-empty frame or subtitle was returned in > + * outdata. > + * @param[in] avpkt AVPacket containing encoded data to be decoded > + * > + * @return amount of bytes read from the packet on success, negative error > + * code on failure > + */ > +static int libxevd_decode(struct AVCodecContext *avctx, struct AVFrame *frame, int *got_frame_ptr, AVPacket *avpkt) > +{ > + XevdContext *xectx = NULL; > + XEVD_IMGB *imgb = NULL; > + XEVD_STAT stat; > + XEVD_BITB bitb; > + int xevd_ret, nalu_size, bs_read_pos; > + int ret = 0; > + > + xectx = avctx->priv_data; > + > + if (avpkt->size > 0) { > + bs_read_pos = 0; > + imgb = NULL; > + while(avpkt->size > (bs_read_pos + XEVD_NAL_UNIT_LENGTH_BYTE)) { > + memset(&stat, 0, sizeof(XEVD_STAT)); > + memset(&bitb, 0, sizeof(XEVD_BITB)); > + > + nalu_size = read_nal_unit_length(avpkt->data + bs_read_pos, XEVD_NAL_UNIT_LENGTH_BYTE, avctx); > + if (nalu_size == 0) { > + av_log(avctx, AV_LOG_ERROR, "Invalid bitstream\n"); > + ret = AVERROR_INVALIDDATA; > + goto ERR; > + } > + bs_read_pos += XEVD_NAL_UNIT_LENGTH_BYTE; > + > + bitb.addr = avpkt->data + bs_read_pos; > + bitb.ssize = nalu_size; > + > + /* main decoding block */ > + xevd_ret = xevd_decode(xectx->id, &bitb, &stat); > + if (XEVD_FAILED(xevd_ret)) { > + av_log(avctx, AV_LOG_ERROR, "Failed to decode bitstream\n"); > + ret = AVERROR_EXTERNAL; > + goto ERR; > + } > + > + bs_read_pos += nalu_size; > + > + if (stat.nalu_type == XEVD_NUT_SPS) { // EVC stream parameters changed > + if ((ret = export_stream_params(xectx, avctx)) != 0) > + goto ERR; > + } > + > + if (stat.read != nalu_size) > + av_log(avctx, AV_LOG_INFO, "Different reading of bitstream (in:%d, read:%d)\n,", nalu_size, stat.read); > + if (stat.fnum >= 0) { > + // already has a decoded image > + if (imgb) { > + // xevd_pull uses pool of objects of type XEVD_IMGB. > + // The pool size is equal MAX_PB_SIZE (26), so release object when it is no more needed > + imgb->release(imgb); > + imgb = NULL; > + } > + xevd_ret = xevd_pull(xectx->id, &imgb); > + if (XEVD_FAILED(xevd_ret)) { > + av_log(avctx, AV_LOG_ERROR, "Failed to pull the decoded image (xevd error code: %d, frame#=%d)\n", xevd_ret, stat.fnum); > + ret = AVERROR_EXTERNAL; > + goto ERR; > + } else if (xevd_ret == XEVD_OK_FRM_DELAYED) { > + if (imgb) { > + // xevd_pull uses pool of objects of type XEVD_IMGB. > + // The pool size is equal MAX_PB_SIZE (26), so release object when it is no more needed > + imgb->release(imgb); > + imgb = NULL; > + } > + } > + if (imgb) { > + int ret = libxevd_image_copy(avctx, imgb, frame); > + if(ret < 0) { > + goto ERR; > + } > + > + frame->pts = avpkt->pts; > + *got_frame_ptr = 1; > + > + // xevd_pull uses pool of objects of type XEVD_IMGB. > + // The pool size is equal MAX_PB_SIZE (26), so release object when it is no more needed > + imgb->release(imgb); > + imgb = NULL; > + } else > + *got_frame_ptr = 0; > + } > + } > + } else { // bumping > + xevd_ret = xevd_pull(xectx->id, &(imgb)); > + if (xevd_ret == XEVD_ERR_UNEXPECTED) { // bumping process completed > + *got_frame_ptr = 0; > + return 0; > + } else if (XEVD_FAILED(xevd_ret)) { > + av_log(avctx, AV_LOG_ERROR, "Failed to pull the decoded image (xevd error code: %d)\n", xevd_ret); > + ret = AVERROR_EXTERNAL; > + goto ERR; > + } > + if (imgb) { > + int ret = libxevd_image_copy(avctx, imgb, frame); > + if(ret < 0) { > + goto ERR; > + } > + > + frame->pts = avpkt->pts; > + *got_frame_ptr = 1; > + > + // xevd_pull uses pool of objects of type XEVD_IMGB. > + // The pool size is equal MAX_PB_SIZE (26), so release object when it is no more needed > + imgb->release(imgb); > + imgb = NULL; > + } else > + *got_frame_ptr = 0; > + } > + > + return avpkt->size; > + > +ERR: > + if (imgb) { > + imgb->release(imgb); > + imgb = NULL; > + } > + *got_frame_ptr = 0; > + > + return ret; > +} > + > +/** > + * Destroy decoder > + * > + * @param avctx codec context > + * @return 0 on success > + */ > +static av_cold int libxevd_close(AVCodecContext *avctx) > +{ > + XevdContext *xectx = avctx->priv_data; > + if (xectx->id) { > + xevd_delete(xectx->id); > + xectx->id = NULL; > + } > + > + return 0; > +} > + > +#define OFFSET(x) offsetof(XevdContext, x) > +#define VD AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_DECODING_PARAM > + > +static const AVClass libxevd_class = { > + .class_name = "libxevd", > + .item_name = av_default_item_name, > + .version = LIBAVUTIL_VERSION_INT, > +}; > + > +const FFCodec ff_libxevd_decoder = { > + .p.name = "evc", > + .p.long_name = NULL_IF_CONFIG_SMALL("EVC / MPEG-5 Essential Video Coding (EVC)"), > + .p.type = AVMEDIA_TYPE_VIDEO, > + .p.id = AV_CODEC_ID_EVC, > + .init = libxevd_init, > + FF_CODEC_DECODE_CB(libxevd_decode), > + .close = libxevd_close, > + .priv_data_size = sizeof(XevdContext), > + .p.priv_class = &libxevd_class, > + .p.capabilities = AV_CODEC_CAP_DELAY | AV_CODEC_CAP_OTHER_THREADS | AV_CODEC_CAP_AVOID_PROBING | AV_CODEC_CAP_DR1, > + .p.wrapper_name = "libxevd", > +}; > diff --git a/libavcodec/libxeve.c b/libavcodec/libxeve.c > new file mode 100644 > index 0000000000..4808e6d853 > --- /dev/null > +++ b/libavcodec/libxeve.c > @@ -0,0 +1,597 @@ > +/* > + * libxeve encoder > + * EVC (MPEG-5 Essential Video Coding) encoding using XEVE MPEG-5 EVC encoder library > + * > + * Copyright (C) 2021 Dawid Kozinski <d.kozinski@samsung.com> > + * > + * This file is part of FFmpeg. > + * > + * FFmpeg is free software; you can redistribute it and/or > + * modify it under the terms of the GNU Lesser General Public > + * License as published by the Free Software Foundation; either > + * version 2.1 of the License, or (at your option) any later version. > + * > + * FFmpeg is distributed in the hope that it will be useful, > + * but WITHOUT ANY WARRANTY; without even the implied warranty of > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > + * Lesser General Public License for more details. > + * > + * You should have received a copy of the GNU Lesser General Public > + * License along with FFmpeg; if not, write to the Free Software > + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA > + */ > + > +#include <float.h> > +#include <stdlib.h> > + > +#include <xeve.h> > + > +#include "libavutil/internal.h" > +#include "libavutil/common.h" > +#include "libavutil/opt.h" > +#include "libavutil/pixdesc.h" > +#include "libavutil/pixfmt.h" > +#include "libavutil/time.h" > +#include "libavutil/cpu.h" > +#include "libavutil/avstring.h" > + > +#include "avcodec.h" > +#include "internal.h" > +#include "packet_internal.h" > +#include "codec_internal.h" > +#include "encode.h" > + > +#define MAX_BS_BUF (16*1024*1024) > + > +/** > + * Error codes > + */ > +#define XEVE_PARAM_BAD_NAME -100 > +#define XEVE_PARAM_BAD_VALUE -200 > + > +/** > + * Encoder states > + * > + * STATE_ENCODING - the encoder receives and processes input frames > + * STATE_BUMPING - there are no more input frames, however the encoder still processes previously received data > + */ > +typedef enum State { > + STATE_ENCODING, > + STATE_BUMPING, > +} State; > + > +/** > + * The structure stores all the states associated with the instance of Xeve MPEG-5 EVC encoder > + */ > +typedef struct XeveContext { > + const AVClass *class; > + > + XEVE id; // XEVE instance identifier > + XEVE_CDSC cdsc; // coding parameters i.e profile, width & height of input frame, num of therads, frame rate ... > + XEVE_BITB bitb; // bitstream buffer (output) > + XEVE_STAT stat; // encoding status (output) > + XEVE_IMGB imgb; // image buffer (input) > + > + State state; // encoder state (skipping, encoding, bumping) > + > + int profile_id; // encoder profile (main, baseline) > + int preset_id; // preset of xeve ( fast, medium, slow, placebo) > + int tune_id; // tune of xeve (psnr, zerolatency) > + > + // variables for rate control modes > + int rc_mode; // Rate control mode [ 0(CQP) / 1(ABR) / 2(CRF) ] > + int qp; // quantization parameter (QP) [0,51] > + int crf; // constant rate factor (CRF) [10,49] > + > + int hash; // embed picture signature (HASH) for conformance checking in decoding > + int sei_info; // embed Supplemental enhancement information while encoding > + > + int color_format; // input data color format: currently only XEVE_CF_YCBCR420 is supported > +} XeveContext; > + > +/** > + * Convert FFmpeg pixel format (AVPixelFormat) to XEVE pre-defined color format > + * > + * @param[in] av_pix_fmt pixel format (@see https://ffmpeg.org/doxygen/trunk/pixfmt_8h.html#a9a8e335cf3be472042bc9f0cf80cd4c5) > + * @param[out] xeve_col_fmt XEVE pre-defined color format (@see xeve.h) > + * > + * @return 0 on success, negative value on failure > + */ > +static int libxeve_color_fmt(enum AVPixelFormat av_pix_fmt, int *xeve_col_fmt) > +{ > + switch (av_pix_fmt) { > + case AV_PIX_FMT_YUV420P: > + *xeve_col_fmt = XEVE_CF_YCBCR420; > + break; > + case AV_PIX_FMT_YUV420P10: > + *xeve_col_fmt = XEVE_CF_YCBCR420; > + break; > + default: > + *xeve_col_fmt = XEVE_CF_UNKNOWN; > + return AVERROR_INVALIDDATA; > + } > + > + return 0; > +} > + > +/** > + * Convert FFmpeg pixel format (AVPixelFormat) into XEVE pre-defined color space > + * > + * @param[in] px_fmt pixel format (@see https://ffmpeg.org/doxygen/trunk/pixfmt_8h.html#a9a8e335cf3be472042bc9f0cf80cd4c5) > + * > + * @return XEVE pre-defined color space (@see xeve.h) on success, XEVE_CF_UNKNOWN on failure > + */ > +static int libxeve_color_space(enum AVPixelFormat av_pix_fmt) > +{ > + /* color space of input image */ > + int cs = XEVE_CF_UNKNOWN; > + > + switch (av_pix_fmt) { > + case AV_PIX_FMT_YUV420P: > + cs = XEVE_CS_YCBCR420; > + break; > + case AV_PIX_FMT_YUV420P10: > +#if AV_HAVE_BIGENDIAN > + cs = XEVE_CS_SET(XEVE_CF_YCBCR420, 10, 1); > +#else > + cs = XEVE_CS_YCBCR420_10LE; > +#endif > + > + break; > + default: > + cs = XEVE_CF_UNKNOWN; > + break; > + } > + > + return cs; > +} > + > +/** > + * The function returns a pointer to the object of the XEVE_CDSC type. > + * XEVE_CDSC contains all encoder parameters that should be initialized before the encoder is used. > + * > + * The field values of the XEVE_CDSC structure are populated based on: > + * - the corresponding field values of the AvCodecConetxt structure, > + * - the xeve encoder specific option values, > + * (the full list of options available for xeve encoder is displayed after executing the command ./ffmpeg --help encoder = libxeve) > + * - and the xeve encoder options specified as a list of key-value pairs following the xeve-params option > + * > + * The order of processing input data and populating the XEVE_CDSC structure > + * 1) first, the fields of the AVCodecContext structure corresponding to the provided input options are processed, > + * (i.e -pix_fmt yuv420p -s:v 1920x1080 -r 30 -profile:v 0) > + * 2) then xeve-specific options added as AVOption to the xeve AVCodec implementation > + * (i.e -preset 0) > + * > + * Keep in mind that, there are options that can be set in different ways. > + * In this case, please follow the above-mentioned order of processing. > + * The most recent assignments overwrite the previous values. > + * > + * @param[in] avctx codec context (AVCodecContext) > + * @param[out] cdsc contains all Xeve MPEG-5 EVC encoder encoder parameters that should be initialized before the encoder is use > + * > + * @return 0 on success, negative error code on failure > + */ > +static int get_conf(AVCodecContext *avctx, XEVE_CDSC *cdsc) > +{ > + XeveContext *xectx = NULL; > + int ret; > + > + xectx = avctx->priv_data; > + > + /* initialize xeve_param struct with default values */ > + ret = xeve_param_default(&cdsc->param); > + if (XEVE_FAILED(ret)) { > + av_log(avctx, AV_LOG_ERROR, "Cannot set_default parameter\n"); > + return AVERROR_EXTERNAL; > + } > + > + /* read options from AVCodecContext */ > + if (avctx->width > 0) > + cdsc->param.w = avctx->width; > + > + if (avctx->height > 0) > + cdsc->param.h = avctx->height; > + > + if (avctx->framerate.num > 0) { > + // fps can be float number, but xeve API doesn't support it > + cdsc->param.fps = (int)(((float)avctx->framerate.num / avctx->framerate.den) + 0.5); > + } > + > + if (avctx->gop_size >= 0) // GOP size (key-frame interval, I-picture period) > + cdsc->param.keyint = avctx->gop_size; // 0: only one I-frame at the first time; 1: every frame is coded in I-frame > + > + if (avctx->max_b_frames == 0 || avctx->max_b_frames == 1 || avctx->max_b_frames == 3 || > + avctx->max_b_frames == 7 || avctx->max_b_frames == 15) // number of b-frames > + cdsc->param.bframes = avctx->max_b_frames; > + else { > + av_log(avctx, AV_LOG_ERROR, "Incorrect value for maximum number of B frames: (%d) \n" > + "Acceptable values for bf option (maximum number of B frames) are 0,1,3,7 or 15\n", avctx->max_b_frames); > + return AVERROR_INVALIDDATA; > + } > + > + if (avctx->level >= 0) > + cdsc->param.level_idc = avctx->level; > + > + if (avctx->rc_buffer_size) // VBV buf size > + cdsc->param.vbv_bufsize = (int)(avctx->rc_buffer_size / 1000); > + > + cdsc->param.rc_type = xectx->rc_mode; > + > + if (xectx->rc_mode == XEVE_RC_CQP) { > + cdsc->param.qp = xectx->qp; > + } else if (xectx->rc_mode == XEVE_RC_ABR) { > + if (avctx->bit_rate / 1000 > INT_MAX || avctx->rc_max_rate / 1000 > INT_MAX) { > + av_log(avctx, AV_LOG_ERROR, "Not supported bitrate bit_rate and rc_max_rate > %d000\n", INT_MAX); > + return AVERROR_INVALIDDATA; > + } > + cdsc->param.bitrate = (int)(avctx->bit_rate / 1000); > + } else if (xectx->rc_mode == XEVE_RC_CRF) { > + cdsc->param.crf = xectx->crf; > + } else { > + av_log(avctx, AV_LOG_ERROR, "Not supported rate control type: %d\n", xectx->rc_mode); > + return AVERROR_INVALIDDATA; > + } > + > + if (avctx->thread_count <= 0) { > + int cpu_count = av_cpu_count(); > + cdsc->param.threads = (cpu_count < XEVE_MAX_THREADS) ? cpu_count : XEVE_MAX_THREADS; > + } else if (avctx->thread_count > XEVE_MAX_THREADS) > + cdsc->param.threads = XEVE_MAX_THREADS; > + else > + cdsc->param.threads = avctx->thread_count; > + > + > + libxeve_color_fmt(avctx->pix_fmt, &xectx->color_format); > + > + cdsc->param.cs = XEVE_CS_SET(xectx->color_format, cdsc->param.codec_bit_depth, AV_HAVE_BIGENDIAN); > + > + cdsc->max_bs_buf_size = MAX_BS_BUF; > + > + ret = xeve_param_ppt(&cdsc->param, xectx->profile_id, xectx->preset_id, xectx->tune_id); > + if (XEVE_FAILED(ret)) { > + av_log(avctx, AV_LOG_ERROR, "Cannot set profile(%d), preset(%d), tune(%d)\n", xectx->profile_id, xectx->preset_id, xectx->tune_id); > + return AVERROR_EXTERNAL; > + } > + > + return 0; > +} > + > +/** > + * Set XEVE_CFG_SET_USE_PIC_SIGNATURE for encoder > + * > + * @param[in] logger context > + * @param[in] id XEVE encodec instance identifier > + * @param[in] ctx the structure stores all the states associated with the instance of Xeve MPEG-5 EVC encoder > + * > + * @return 0 on success, negative error code on failure > + */ > +static int set_extra_config(AVCodecContext *avctx, XEVE id, XeveContext *ctx) > +{ > + int ret, size; > + size = 4; > + > + // embed SEI messages identifying encoder parameters and command line arguments > + // - 0: off\n" > + // - 1: emit sei info" > + // > + // SEI - Supplemental enhancement information contains information > + // that is not necessary to decode the samples of coded pictures from VCL NAL units. > + // Some SEI message information is required to check bitstream conformance > + // and for output timing decoder conformance. > + // @see ISO_IEC_23094-1_2020 7.4.3.5 > + // @see ISO_IEC_23094-1_2020 Annex D > + ret = xeve_config(id, XEVE_CFG_SET_SEI_CMD, &ctx->sei_info, &size); // sei_cmd_info > + if (XEVE_FAILED(ret)) { > + av_log(avctx, AV_LOG_ERROR, "Failed to set config for sei command info messages\n"); > + return AVERROR_EXTERNAL; > + } > + > + ret = xeve_config(id, XEVE_CFG_SET_USE_PIC_SIGNATURE, &ctx->hash, &size); > + if (XEVE_FAILED(ret)) { > + av_log(avctx, AV_LOG_ERROR, "Failed to set config for picture signature\n"); > + return AVERROR_EXTERNAL; > + } > + > + return 0; > +} > + > +/** > + * @brief Switch encoder to bumping mode > + * > + * @param id XEVE encodec instance identifier > + * @return 0 on success, negative error code on failure > + */ > +static int setup_bumping(XEVE id) > +{ > + int val = 1; > + int size = sizeof(int); > + if (XEVE_FAILED(xeve_config(id, XEVE_CFG_SET_FORCE_OUT, (void *)(&val), &size))) > + return AVERROR_EXTERNAL; > + > + return 0; > +} > + > +/** > + * @brief Initialize eXtra-fast Essential Video Encoder codec > + * Create an encoder instance and allocate all the needed resources > + * > + * @param avctx codec context > + * @return 0 on success, negative error code on failure > + */ > +static av_cold int libxeve_init(AVCodecContext *avctx) > +{ > + XeveContext *xectx = avctx->priv_data; > + unsigned char *bs_buf = NULL; > + int i; > + int shift_h = 0; > + int shift_v = 0; > + int width_chroma = 0; > + int height_chroma = 0; > + XEVE_IMGB *imgb = NULL; > + int ret = 0; > + > + XEVE_CDSC *cdsc = &(xectx->cdsc); > + > + /* allocate bitstream buffer */ > + bs_buf = av_malloc(MAX_BS_BUF); > + if (bs_buf == NULL) { > + av_log(avctx, AV_LOG_ERROR, "Cannot allocate bitstream buffer\n"); > + return AVERROR(ENOMEM); > + } > + xectx->bitb.addr = bs_buf; > + xectx->bitb.bsize = MAX_BS_BUF; > + > + /* read configurations and set values for created descriptor (XEVE_CDSC) */ > + if ((ret = get_conf(avctx, cdsc)) != 0) { > + av_log(avctx, AV_LOG_ERROR, "Cannot get configuration\n"); > + return AVERROR(EINVAL); > + } > + > + if ((ret = xeve_param_check(&cdsc->param)) != 0) { > + av_log(avctx, AV_LOG_ERROR, "Invalid configuration\n"); > + return AVERROR(EINVAL); > + } > + > + /* create encoder */ > + xectx->id = xeve_create(cdsc, NULL); > + if (xectx->id == NULL) { > + av_log(avctx, AV_LOG_ERROR, "Cannot create XEVE encoder\n"); > + return AVERROR_EXTERNAL; > + } > + > + if ((ret = set_extra_config(avctx, xectx->id, xectx)) != 0) { > + av_log(avctx, AV_LOG_ERROR, "Cannot set extra configuration\n"); > + return AVERROR(EINVAL); > + } > + > + if ((ret = av_pix_fmt_get_chroma_sub_sample(avctx->pix_fmt, &shift_h, &shift_v)) != 0) { > + av_log(avctx, AV_LOG_ERROR, "Failed to get chroma shift\n"); > + return AVERROR(EINVAL); > + } > + > + // Chroma subsampling > + // > + // YUV format explanation > + // shift_h == 1 && shift_v == 1 : YUV420 > + // shift_h == 1 && shift_v == 0 : YUV422 > + // shift_h == 0 && shift_v == 0 : YUV444 > + // > + width_chroma = AV_CEIL_RSHIFT(avctx->width, shift_h); > + height_chroma = AV_CEIL_RSHIFT(avctx->height, shift_v); > + > + /* set default values for input image buffer */ > + imgb = &xectx->imgb; > + imgb->cs = libxeve_color_space(avctx->pix_fmt); > + imgb->np = 3; /* only for yuv420p, yuv420ple */ > + > + for (i = 0; i < imgb->np; i++) > + imgb->x[i] = imgb->y[i] = 0; > + > + imgb->w[0] = imgb->aw[0] = avctx->width; // width luma > + imgb->w[1] = imgb->w[2] = imgb->aw[1] = imgb->aw[2] = width_chroma; > + imgb->h[0] = imgb->ah[0] = avctx->height; // height luma > + imgb->h[1] = imgb->h[2] = imgb->ah[1] = imgb->ah[2] = height_chroma; > + > + xectx->state = STATE_ENCODING; > + > + return 0; > +} > + > +/** > + * Encode raw data frame into EVC packet > + * > + * @param[in] avctx codec context > + * @param[out] pkt output AVPacket containing encoded data > + * @param[in] frame AVFrame containing the raw data to be encoded > + * @param[out] got_packet encoder sets to 0 or 1 to indicate that a > + * non-empty packet was returned in pkt > + * > + * @return 0 on success, negative error code on failure > + */ > +static int libxeve_encode(AVCodecContext *avctx, AVPacket *avpkt, > + const AVFrame *frame, int *got_packet) > +{ > + XeveContext *xectx = avctx->priv_data; > + int ret = -1; > + > + // No more input frames are available but encoder still can have some data in its internal buffer to process > + // and some frames to dump. > + if (xectx->state == STATE_ENCODING && frame == NULL) { > + if (setup_bumping(xectx->id) == 0) > + xectx->state = STATE_BUMPING; // Entering bumping process > + else { > + av_log(avctx, AV_LOG_ERROR, "Failed to setup bumping\n"); > + return 0; > + } > + } > + > + if (xectx->state == STATE_ENCODING) { > + int i; > + XEVE_IMGB *imgb = NULL; > + > + imgb = &xectx->imgb; > + > + for (i = 0; i < imgb->np; i++) { > + imgb->a[i] = frame->data[i]; > + imgb->s[i] = frame->linesize[i]; > + } > + > + imgb->ts[0] = frame->pts; > + imgb->ts[1] = 0; > + > + /* push image to encoder */ > + ret = xeve_push(xectx->id, imgb); > + if (XEVE_FAILED(ret)) { > + av_log(avctx, AV_LOG_ERROR, "xeve_push() failed\n"); > + return AVERROR_EXTERNAL; > + } > + } > + if (xectx->state == STATE_ENCODING || xectx->state == STATE_BUMPING) { > + /* encoding */ > + ret = xeve_encode(xectx->id, &(xectx->bitb), &(xectx->stat)); > + if (XEVE_FAILED(ret)) { > + av_log(avctx, AV_LOG_ERROR, "xeve_encode() failed\n"); > + return AVERROR_EXTERNAL; > + } > + > + /* store bitstream */ > + if (ret == XEVE_OK_OUT_NOT_AVAILABLE) { // Return OK but picture is not available yet > + *got_packet = 0; > + return 0; > + } else if (ret == XEVE_OK) { > + int av_pic_type; > + > + if (xectx->stat.write > 0) { > + > + ret = ff_get_encode_buffer(avctx, avpkt, xectx->stat.write, 0); > + if (ret < 0) > + return ret; > + > + memcpy(avpkt->data, xectx->bitb.addr, xectx->stat.write); > + > + avpkt->pts = xectx->bitb.ts[0]; > + avpkt->dts = xectx->bitb.ts[1]; > + > + switch(xectx->stat.stype) { > + case XEVE_ST_I: > + av_pic_type = AV_PICTURE_TYPE_I; > + avpkt->flags |= AV_PKT_FLAG_KEY; > + break; > + case XEVE_ST_P: > + av_pic_type = AV_PICTURE_TYPE_P; > + break; > + case XEVE_ST_B: > + av_pic_type = AV_PICTURE_TYPE_B; > + break; > + case XEVE_ST_UNKNOWN: > + av_log(avctx, AV_LOG_ERROR, "Unknown slice type\n"); > + return AVERROR_INVALIDDATA; > + } > + > + ff_side_data_set_encoder_stats(avpkt, xectx->stat.qp * FF_QP2LAMBDA, NULL, 0, av_pic_type); > + > + *got_packet = 1; > + } > + } else if (ret == XEVE_OK_NO_MORE_FRM) { > + // Return OK but no more frames > + return 0; > + } else { > + av_log(avctx, AV_LOG_ERROR, "Invalid return value: %d\n", ret); > + return AVERROR_EXTERNAL; > + } > + } else { > + av_log(avctx, AV_LOG_ERROR, "Udefined encoder state\n"); > + return AVERROR_INVALIDDATA; > + } > + > + return 0; > +} > + > +/** > + * Destroy the encoder and release all the allocated resources > + * > + * @param avctx codec context > + * @return 0 on success, negative error code on failure > + */ > +static av_cold int libxeve_close(AVCodecContext *avctx) > +{ > + XeveContext *xectx = avctx->priv_data; > + > + xeve_delete(xectx->id); You intend to set the init-cleanup flag on this encoder. But then you have to account for the possibility of this function being called with a XeveContext that has not been successfully setup, i.e. xectx->id might be NULL. Is this allowed? > + > + av_free(xectx->bitb.addr); /* release bitstream buffer */ > + > + return 0; > +} > + > +#define OFFSET(x) offsetof(XeveContext, x) > +#define VE AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_ENCODING_PARAM > + > +static const enum AVPixelFormat supported_pixel_formats[] = { > + AV_PIX_FMT_YUV420P, > + AV_PIX_FMT_YUV420P10, > + AV_PIX_FMT_NONE > +}; > + > +// Example of using: ./ffmpeg -xeve-params "m=2:q=17" > +// Consider using following options (./ffmpeg --help encoder=libxeve) > +// > +static const AVOption libxeve_options[] = { > + { "preset", "Encoding preset for setting encoding speed", OFFSET(preset_id), AV_OPT_TYPE_INT, { .i64 = XEVE_PRESET_MEDIUM }, XEVE_PRESET_DEFAULT, XEVE_PRESET_PLACEBO, VE, "preset" }, > + { "default", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_PRESET_DEFAULT }, INT_MIN, INT_MAX, VE, "preset" }, > + { "fast", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_PRESET_FAST }, INT_MIN, INT_MAX, VE, "preset" }, > + { "medium", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_PRESET_MEDIUM }, INT_MIN, INT_MAX, VE, "preset" }, > + { "slow", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_PRESET_SLOW }, INT_MIN, INT_MAX, VE, "preset" }, > + { "placebo", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_PRESET_PLACEBO }, INT_MIN, INT_MAX, VE, "preset" }, > + { "tune", "Tuning parameter for special purpose operation", OFFSET(tune_id), AV_OPT_TYPE_INT, { .i64 = XEVE_TUNE_NONE }, XEVE_TUNE_NONE, XEVE_TUNE_PSNR, VE, "tune"}, > + { "none", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_TUNE_NONE }, INT_MIN, INT_MAX, VE, "tune" }, > + { "zerolatency", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_TUNE_ZEROLATENCY }, INT_MIN, INT_MAX, VE, "tune" }, > + { "psnr", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_TUNE_PSNR }, INT_MIN, INT_MAX, VE, "tune" }, > + { "profile", "Encoding profile", OFFSET(profile_id), AV_OPT_TYPE_INT, { .i64 = XEVE_PROFILE_BASELINE }, XEVE_PROFILE_BASELINE, XEVE_PROFILE_MAIN, VE, "profile" }, > + { "baseline", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_PROFILE_BASELINE }, INT_MIN, INT_MAX, VE, "profile" }, > + { "main", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_PROFILE_MAIN }, INT_MIN, INT_MAX, VE, "profile" }, > + { "rc_mode", "Rate control mode", OFFSET(rc_mode), AV_OPT_TYPE_INT, { .i64 = XEVE_RC_CQP }, XEVE_RC_CQP, XEVE_RC_CRF, VE, "rc_mode" }, > + { "CQP", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_RC_CQP }, INT_MIN, INT_MAX, VE, "rc_mode" }, > + { "ABR", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_RC_ABR }, INT_MIN, INT_MAX, VE, "rc_mode" }, > + { "CRF", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_RC_CRF }, INT_MIN, INT_MAX, VE, "rc_mode" }, > + { "qp", "Quantization parameter value for CQP rate control mode", OFFSET(qp), AV_OPT_TYPE_INT, { .i64 = 32 }, 0, 51, VE }, > + { "crf", "Constant rate factor value for CRF rate control mode", OFFSET(crf), AV_OPT_TYPE_INT, { .i64 = 32 }, 10, 49, VE }, > + { "hash", "Embed picture signature (HASH) for conformance checking in decoding", OFFSET(hash), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, 1, VE }, > + { "sei_info", "Embed SEI messages identifying encoder parameters and command line arguments", OFFSET(sei_info), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, 1, VE }, > + { NULL } > +}; > + > +static const AVClass libxeve_class = { > + .class_name = "libxeve", > + .item_name = av_default_item_name, > + .option = libxeve_options, > + .version = LIBAVUTIL_VERSION_INT, > +}; > + > +/** > + * libavcodec generic global options, which can be set on all the encoders and decoders > + * @see https://www.ffmpeg.org/ffmpeg-codecs.html#Codec-Options > + */ > +static const FFCodecDefault libxeve_defaults[] = { > + { "b", "0" }, // bitrate in terms of kilo-bits per second > + { "g", "0" }, // gop_size (key-frame interval 0: only one I-frame at the first time; 1: every frame is coded in I-frame) > + { "bf", "15"}, // maximum number of B frames (0: no B-frames, 1,3,7,15) > + { "threads", "0"}, // number of threads to be used (0: automatically select the number of threads to set) > + { NULL }, > +}; > + > +const FFCodec ff_libxeve_encoder = { > + .p.name = "libxeve", > + .p.long_name = NULL_IF_CONFIG_SMALL("libxeve MPEG-5 EVC"), > + .p.type = AVMEDIA_TYPE_VIDEO, > + .p.id = AV_CODEC_ID_EVC, > + .init = libxeve_init, > + FF_CODEC_ENCODE_CB(libxeve_encode), > + .close = libxeve_close, > + .priv_data_size = sizeof(XeveContext), > + .p.priv_class = &libxeve_class, > + .defaults = libxeve_defaults, > + .p.capabilities = FF_CODEC_CAP_INIT_CLEANUP | AV_CODEC_CAP_DELAY | AV_CODEC_CAP_AUTO_THREADS | AV_CODEC_CAP_DR1, 1. This is completely wring: FF_CODEC_CAP_INIT_CLEANUP is an internal capability flag, not something public. That's why it has an FF_ prefix. Internal capabilities should go into FFCodec.caps_internal. (In other words: s/.p.capabilities/.caps_internal/) 2. Due to 21b23ceab3cfcc2aa0a833bbff11cf38b2177800, a codec that has not the FF_CODEC_CAP_NOT_INIT_THREADSAFE internal cap set is presumed to be init-threadsafe (this changed since the last iteration of your patchset). Is it guaranteed and documented that libxeve and libxevd satisfy are init-threadsafe (i.e. there are no races if multiple encoders/decoders (each with their own context) are initialized concurrently)? > + .p.wrapper_name = "libxeve", > + .p.pix_fmts = supported_pixel_formats, > +}; > diff --git a/libavcodec/parsers.c b/libavcodec/parsers.c > index a8d52af6cb..ca00c63176 100644 > --- a/libavcodec/parsers.c > +++ b/libavcodec/parsers.c > @@ -41,6 +41,7 @@ extern const AVCodecParser ff_dvaudio_parser; > extern const AVCodecParser ff_dvbsub_parser; > extern const AVCodecParser ff_dvdsub_parser; > extern const AVCodecParser ff_dvd_nav_parser; > +extern const AVCodecParser ff_evc_parser; > extern const AVCodecParser ff_flac_parser; > extern const AVCodecParser ff_g723_1_parser; > extern const AVCodecParser ff_g729_parser; > diff --git a/libavcodec/profiles.c b/libavcodec/profiles.c > index 7af7fbeb13..a31244e0db 100644 > --- a/libavcodec/profiles.c > +++ b/libavcodec/profiles.c > @@ -181,4 +181,10 @@ const AVProfile ff_arib_caption_profiles[] = { > { FF_PROFILE_UNKNOWN } > }; > > +const AVProfile ff_evc_profiles[] = { > + { FF_PROFILE_EVC_BASELINE, "Baseline" }, > + { FF_PROFILE_EVC_MAIN, "Main" }, > + { FF_PROFILE_UNKNOWN }, > +}; > + > #endif /* !CONFIG_SMALL */ > diff --git a/libavcodec/profiles.h b/libavcodec/profiles.h > index 41a19aa9ad..cf92b5f126 100644 > --- a/libavcodec/profiles.h > +++ b/libavcodec/profiles.h > @@ -72,5 +72,6 @@ extern const AVProfile ff_sbc_profiles[]; > extern const AVProfile ff_prores_profiles[]; > extern const AVProfile ff_mjpeg_profiles[]; > extern const AVProfile ff_arib_caption_profiles[]; > +extern const AVProfile ff_evc_profiles[]; > > #endif /* AVCODEC_PROFILES_H */ > diff --git a/libavcodec/version.h b/libavcodec/version.h > index e488eee355..2328be4b26 100644 > --- a/libavcodec/version.h > +++ b/libavcodec/version.h > @@ -29,7 +29,7 @@ > > #include "version_major.h" > > -#define LIBAVCODEC_VERSION_MINOR 42 > +#define LIBAVCODEC_VERSION_MINOR 43 > #define LIBAVCODEC_VERSION_MICRO 100 > > #define LIBAVCODEC_VERSION_INT AV_VERSION_INT(LIBAVCODEC_VERSION_MAJOR, \ _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 13+ messages in thread
[parent not found: <CGME20220812101849eucas1p225445ac91388200262dbf697436f67f2@eucas1p2.samsung.com>]
* [FFmpeg-devel] [PATCH 1/2] Provided support for MPEG-5 EVC (Essential Video Coding) codec [not found] <CGME20220812101849eucas1p225445ac91388200262dbf697436f67f2@eucas1p2.samsung.com> @ 2022-08-12 10:18 ` Dawid Kozinski 0 siblings, 0 replies; 13+ messages in thread From: Dawid Kozinski @ 2022-08-12 10:18 UTC (permalink / raw) To: ffmpeg-devel - Added new entry to codec IDs list - Added new entry to the codec descriptor list - Bumped libavcodec minor version - Changes in Changelog and MAINTAINERS files - Added xeve encoder wrapper - Added xevd dencoder wrapper - Added documentation for xeve and xevd wrappers - Added parser for EVC format - Changes in project configuration file and libavcodec Makefilei Signed-off-by: Dawid Kozinski <d.kozinski@samsung.com> --- Changelog | 3 +- MAINTAINERS | 2 + configure | 8 + doc/decoders.texi | 24 ++ doc/encoders.texi | 92 +++++ doc/general_contents.texi | 19 + libavcodec/Makefile | 3 + libavcodec/allcodecs.c | 2 + libavcodec/avcodec.h | 3 + libavcodec/codec_desc.c | 8 + libavcodec/codec_id.h | 1 + libavcodec/evc_parser.c | 758 ++++++++++++++++++++++++++++++++++++++ libavcodec/libxevd.c | 420 +++++++++++++++++++++ libavcodec/libxeve.c | 601 ++++++++++++++++++++++++++++++ libavcodec/parsers.c | 1 + libavcodec/profiles.c | 6 + libavcodec/profiles.h | 1 + libavcodec/version.h | 2 +- 18 files changed, 1952 insertions(+), 2 deletions(-) create mode 100644 libavcodec/evc_parser.c create mode 100644 libavcodec/libxevd.c create mode 100644 libavcodec/libxeve.c diff --git a/Changelog b/Changelog index fa83786a20..f41e260247 100644 --- a/Changelog +++ b/Changelog @@ -82,7 +82,8 @@ version 5.0: - VideoToolbox ProRes encoder - anlmf audio filter - IMF demuxer (experimental) - +- eXtra-fast Essential Video Encoder (XEVE) +- eXtra-fast Essential Video Decoder (XEVD) version 4.4: - AudioToolbox output device diff --git a/MAINTAINERS b/MAINTAINERS index ed2ec0b90c..03c052e0bd 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -206,6 +206,7 @@ Codecs: libvpx* James Zern libxavs.c Stefan Gehrer libxavs2.c Huiwen Ren + libxev*.c, evc_parser.c Dawid Kozinski libzvbi-teletextdec.c Marton Balint lzo.h, lzo.c Reimar Doeffinger mdec.c Michael Niedermayer @@ -426,6 +427,7 @@ Muxers/Demuxers: dv.c Roman Shaposhnik electronicarts.c Peter Ross epafdec.c Paul B Mahol + evcdec.c Dawid Kozinski ffm* Baptiste Coudurier flic.c Mike Melanson flvdec.c Michael Niedermayer diff --git a/configure b/configure index 090ccf21a7..f294b09759 100755 --- a/configure +++ b/configure @@ -291,6 +291,8 @@ External library support: --enable-libwebp enable WebP encoding via libwebp [no] --enable-libx264 enable H.264 encoding via x264 [no] --enable-libx265 enable HEVC encoding via x265 [no] + --enable-libxeve enable EVC encoding via libxeve [no] + --enable-libxevd enable EVC decoding via libxevd [no] --enable-libxavs enable AVS encoding via xavs [no] --enable-libxavs2 enable AVS2 encoding via xavs2 [no] --enable-libxcb enable X11 grabbing using XCB [autodetect] @@ -1875,6 +1877,8 @@ EXTERNAL_LIBRARY_LIST=" libvorbis libvpx libwebp + libxevd + libxeve libxml2 libzimg libzmq @@ -3397,6 +3401,8 @@ libx264rgb_encoder_select="libx264_encoder" libx265_encoder_deps="libx265" libxavs_encoder_deps="libxavs" libxavs2_encoder_deps="libxavs2" +libxevd_decoder_deps="libxevd" +libxeve_encoder_deps="libxeve" libxvid_encoder_deps="libxvid" libzvbi_teletext_decoder_deps="libzvbi" vapoursynth_demuxer_deps="vapoursynth" @@ -6705,6 +6711,8 @@ enabled libx265 && require_pkg_config libx265 x265 x265.h x265_api_get require_cpp_condition libx265 x265.h "X265_BUILD >= 70" enabled libxavs && require libxavs "stdint.h xavs.h" xavs_encoder_encode "-lxavs $pthreads_extralibs $libm_extralibs" enabled libxavs2 && require_pkg_config libxavs2 "xavs2 >= 1.3.0" "stdint.h xavs2.h" xavs2_api_get +enabled libxevd && require_pkg_config libxevd "xevd >= 0.4.0" "xevd.h" xevd_decode +enabled libxeve && require_pkg_config libxeve "xeve >= 0.4.0" "xeve.h" xeve_encode enabled libxvid && require libxvid xvid.h xvid_global -lxvidcore enabled libzimg && require_pkg_config libzimg "zimg >= 2.7.0" zimg.h zimg_get_api_version enabled libzmq && require_pkg_config libzmq "libzmq >= 4.2.1" zmq.h zmq_ctx_new diff --git a/doc/decoders.texi b/doc/decoders.texi index e2fcbf5dc9..0bd9096114 100644 --- a/doc/decoders.texi +++ b/doc/decoders.texi @@ -126,6 +126,30 @@ Set amount of frame threads to use during decoding. The default value is 0 (auto @end table +@section libxevd + +eXtra-fast Essential Video Decoder (XEVD) MPEG-5 EVC decoder wrapper. + +This decoder requires the presence of the libxevd headers and library +during configuration. You need to explicitly configure the build with +@option{--enable-libxevd}. + +The xevd project website is at @url{https://github.com/mpeg5/xevd}. + +@subsection Options + +The following options are supported by the libxevd wrapper. +The xevd-equivalent options or values are listed in parentheses for easy migration. + +To get a more accurate and extensive documentation of the libxevd options, +invoke the command @code{xevd_app --help} or consult the libxevd documentation. + +@table @option +@item threads (@emph{threads}) +Force to use a specific number of threads + +@end table + @section QSV Decoders The family of Intel QuickSync Video decoders (VC1, MPEG-2, H.264, HEVC, diff --git a/doc/encoders.texi b/doc/encoders.texi index 6d73f74196..76f55e6721 100644 --- a/doc/encoders.texi +++ b/doc/encoders.texi @@ -2892,6 +2892,98 @@ ffmpeg -i input -c:v libxavs2 -xavs2-params RdoqLevel=0 output.avs2 @end example @end table +@section libxeve + +eXtra-fast Essential Video Encoder (XEVE) MPEG-5 EVC encoder wrapper. +The xeve-equivalent options or values are listed in parentheses for easy migration. + +This encoder requires the presence of the libxeve headers and library +during configuration. You need to explicitly configure the build with +@option{--enable-libxeve}. + +@float NOTE +Many libxeve encoder options are mapped to FFmpeg global codec options, +while unique encoder options are provided through private options. +Additionally the xeve-params private options allows one to pass a list +of key=value tuples as accepted by the libxeve @code{parse_xeve_params} function. +@end float + +The xeve project website is at @url{https://github.com/mpeg5/xeve}. + +@subsection Options + +The following options are supported by the libxeve wrapper. +The xeve-equivalent options or values are listed in parentheses for easy migration. + +@float NOTE +To reduce the duplication of documentation, only the private options +and some others requiring special attention are documented here. For +the documentation of the undocumented generic options, see +@ref{codec-options,,the Codec Options chapter}. +@end float + +@float NOTE +To get a more accurate and extensive documentation of the libxeve options, +invoke the command @code{xeve_app --help} or consult the libxeve documentation. +@end float + +@table @option +@item b (@emph{bitrate}) +Set target video bitrate in bits/s. +Note that FFmpeg's b option is expressed in bits/s, while xeve's bitrate is in kilobits/s. + +@item bf (@emph{bframes}) +Set the maximum number of B frames (1,3,7,15). + +@item g (@emph{keyint}) +Set the GOP size (I-picture period). + +@item preset (@emph{preset}) +Set the xeve preset. +Set the encoder preset value to determine encoding speed [fast, medium, slow, placebo] + +@item tune (@emph{tune}) +Set the encoder tune parameter [psnr, zerolatency] + +@item profile (@emph{profile}) +Set the encoder profile [0: baselie; 1: main] + +@item crf (@emph{crf}) +Set the quality for constant quality mode. +Constant rate factor <10..49> [default: 32] + +@item qp (@emph{qp}) +Set constant quantization rate control method parameter. +Quantization parameter qp <0..51> [default: 32] + +@item threads (@emph{threads}) +Force to use a specific number of threads + +@item xeve-params +Set xeve options using a list of @var{key}=@var{value} couples separated +by ":". + +For example to specify libxeve encoding options with @option{-xeve-params}: + +@example +ffmpeg -i input -c:v libxeve -xeve-params profile=main:tune=zerolatency:preset=fast:q=17:bframes=15:bitrate=100k output.mp4 +@end example +@end table + +@subsubsection Supported key values for xeve-params + +Set xeve options using a list of @var{key}=@var{value} couples separated + +@table @option + +@item hash +Embed the picture signature (HASH) for conformance checking in decoding, (0: Off, 1: On) + +@item vbv-bufsize +Set the VBV buffer size: Kbits(none,K,k), Mbits(M,m) + +@end table + @section libxvid Xvid MPEG-4 Part 2 encoder wrapper. diff --git a/doc/general_contents.texi b/doc/general_contents.texi index 86ec6d606b..89236047ea 100644 --- a/doc/general_contents.texi +++ b/doc/general_contents.texi @@ -343,6 +343,22 @@ libxavs2 is under the GNU Public License Version 2 or later details), you must upgrade FFmpeg's license to GPL in order to use it. @end float +@section eXtra-fast Essential Video Encoder (XEVE) + +FFmpeg can make use of the XEVE library for EVC video encoding. + +Go to @url{https://github.com/mpeg5/xeve} and follow the instructions for +installing the XEVE library. Then pass @code{--enable-libxeve} to configure to +enable it. + +@section eXtra-fast Essential Video Decoder (XEVD) + +FFmpeg can make use of the XEVD library for EVC video decoding. + +Go to @url{https://github.com/mpeg5/xevd} and follow the instructions for +installing the XEVD library. Then pass @code{--enable-libxevd} to configure to +enable it. + @section ZVBI ZVBI is a VBI decoding library which can be used by FFmpeg to decode DVB @@ -591,6 +607,7 @@ library: @item raw DTS @tab X @tab X @item raw DTS-HD @tab @tab X @item raw E-AC-3 @tab X @tab X +@item raw EVC @tab X @tab X @item raw FLAC @tab X @tab X @item raw GSM @tab @tab X @item raw H.261 @tab X @tab X @@ -930,6 +947,8 @@ following image formats are supported: @item Electronic Arts TQI video @tab @tab X @item Escape 124 @tab @tab X @item Escape 130 @tab @tab X +@item EVC / MPEG-5 Part 1 @tab X @tab X + @tab encoding and decoding supported through external libraries libxeve and libxevd @item FFmpeg video codec #1 @tab X @tab X @tab lossless codec (fourcc: FFV1) @item Flash Screen Video v1 @tab X @tab X diff --git a/libavcodec/Makefile b/libavcodec/Makefile index 83901bb52b..cfeacd96d1 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -1102,6 +1102,8 @@ OBJS-$(CONFIG_LIBX264_ENCODER) += libx264.o OBJS-$(CONFIG_LIBX265_ENCODER) += libx265.o OBJS-$(CONFIG_LIBXAVS_ENCODER) += libxavs.o OBJS-$(CONFIG_LIBXAVS2_ENCODER) += libxavs2.o +OBJS-$(CONFIG_LIBXEVD_DECODER) += libxevd.o +OBJS-$(CONFIG_LIBXEVE_ENCODER) += libxeve.o OBJS-$(CONFIG_LIBXVID_ENCODER) += libxvid.o OBJS-$(CONFIG_LIBZVBI_TELETEXT_DECODER) += libzvbi-teletextdec.o ass.o @@ -1129,6 +1131,7 @@ OBJS-$(CONFIG_DVAUDIO_PARSER) += dvaudio_parser.o OBJS-$(CONFIG_DVBSUB_PARSER) += dvbsub_parser.o OBJS-$(CONFIG_DVD_NAV_PARSER) += dvd_nav_parser.o OBJS-$(CONFIG_DVDSUB_PARSER) += dvdsub_parser.o +OBJS-$(CONFIG_EVC_PARSER) += evc_parser.o OBJS-$(CONFIG_FLAC_PARSER) += flac_parser.o flacdata.o flac.o OBJS-$(CONFIG_G723_1_PARSER) += g723_1_parser.o OBJS-$(CONFIG_G729_PARSER) += g729_parser.o diff --git a/libavcodec/allcodecs.c b/libavcodec/allcodecs.c index c94e2d5966..2526e11f70 100644 --- a/libavcodec/allcodecs.c +++ b/libavcodec/allcodecs.c @@ -800,6 +800,8 @@ extern LIBX264_CONST FFCodec ff_libx264_encoder; #endif extern const FFCodec ff_libx264rgb_encoder; extern FFCodec ff_libx265_encoder; +extern const FFCodec ff_libxeve_encoder; +extern const FFCodec ff_libxevd_decoder; extern const FFCodec ff_libxavs_encoder; extern const FFCodec ff_libxavs2_encoder; extern const FFCodec ff_libxvid_encoder; diff --git a/libavcodec/avcodec.h b/libavcodec/avcodec.h index 65c8535359..7d56a03fca 100644 --- a/libavcodec/avcodec.h +++ b/libavcodec/avcodec.h @@ -1674,6 +1674,9 @@ typedef struct AVCodecContext { #define FF_PROFILE_KLVA_SYNC 0 #define FF_PROFILE_KLVA_ASYNC 1 +#define FF_PROFILE_EVC_BASELINE 0 +#define FF_PROFILE_EVC_MAIN 1 + /** * level * - encoding: Set by user. diff --git a/libavcodec/codec_desc.c b/libavcodec/codec_desc.c index c1a177c22d..1a43ff8cb6 100644 --- a/libavcodec/codec_desc.c +++ b/libavcodec/codec_desc.c @@ -1907,6 +1907,14 @@ static const AVCodecDescriptor codec_descriptors[] = { .long_name = NULL_IF_CONFIG_SMALL("WBMP (Wireless Application Protocol Bitmap) image"), .props = AV_CODEC_PROP_INTRA_ONLY | AV_CODEC_PROP_LOSSLESS, }, + { + .id = AV_CODEC_ID_EVC, + .type = AVMEDIA_TYPE_VIDEO, + .name = "evc", + .long_name = NULL_IF_CONFIG_SMALL("MPEG-5 EVC (Essential Video Coding)"), + .props = AV_CODEC_PROP_LOSSY | AV_CODEC_PROP_REORDER, + .profiles = NULL_IF_CONFIG_SMALL(ff_evc_profiles), + }, /* various PCM "codecs" */ { diff --git a/libavcodec/codec_id.h b/libavcodec/codec_id.h index 386a00a7ef..02b60be690 100644 --- a/libavcodec/codec_id.h +++ b/libavcodec/codec_id.h @@ -314,6 +314,7 @@ enum AVCodecID { AV_CODEC_ID_PHM, AV_CODEC_ID_RADIANCE_HDR, AV_CODEC_ID_WBMP, + AV_CODEC_ID_EVC, /* various PCM "codecs" */ AV_CODEC_ID_FIRST_AUDIO = 0x10000, ///< A dummy id pointing at the start of audio codecs diff --git a/libavcodec/evc_parser.c b/libavcodec/evc_parser.c new file mode 100644 index 0000000000..0ae0eaa08a --- /dev/null +++ b/libavcodec/evc_parser.c @@ -0,0 +1,758 @@ +/* + * EVC format parser + * + * Copyright (C) 2021 Dawid Kozinski <d.kozinski@samsung.com> + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include <stdint.h> + +#include "libavutil/common.h" + +#include "parser.h" +#include "golomb.h" + +// The length field that indicates the length in bytes of the following NAL unit is configured to be of 4 bytes +#define EVC_NAL_UNIT_LENGTH_BYTE (4) /* byte */ + +#define EVC_NAL_HEADER_SIZE (2) /* byte */ +#define MAX_SPS_CNT (16) /* defined value in EVC standard */ +#define MAX_PPS_CNT (64) /* defined value in EVC standard */ + +#define MAX_NUM_TILES_ROW (22) +#define MAX_NUM_TILES_COL (20) + +// NALU types +// @see ISO_IEC_23094-1_2020 7.4.2.2 NAL unit header semantics +// +#define EVC_NUT_NONIDR (0) /* Coded slice of a non-IDR picture */ +#define EVC_NUT_IDR (1) /* Coded slice of an IDR picture */ +#define EVC_NUT_SPS (24) /* Sequence parameter set */ +#define EVC_NUT_PPS (25) /* Picture paremeter set */ +#define EVC_NUT_APS (26) /* Adaptation parameter set */ +#define EVC_NUT_FD (27) /* Filler data */ +#define EVC_NUT_SEI (28) /* Supplemental enhancement information */ + +// slice type +// @see ISO_IEC_23094-1_2020 7.4.5 Slice header semantics +// +#define SLICE_TYPE_UNKNOWN (-1) +#define SLICE_TYPE_B (0) +#define SLICE_TYPE_P (1) +#define SLICE_TYPE_I (2) + +#define MAX_NUM_RPLS (32) +#define MAX_NUM_REF_PICS (21) + +// rpl structure +typedef struct RefPicListStruct { + int poc; + int tid; + int ref_pic_num; + int ref_pic_active_num; + int ref_pics[MAX_NUM_REF_PICS]; + char pic_type; + +} RefPicListStruct; + +// The sturcture reflects SPS RBSP(raw byte sequence payload) layout +// @see ISO_IEC_23094-1 section 7.3.2.1 +// +// The following descriptors specify the parsing process of each element +// u(n) - unsigned integer using n bits +// ue(v) - unsigned integer 0-th order Exp_Golomb-coded syntax element with the left bit first +typedef struct EVCParserSPS { + int sps_seq_parameter_set_id; // ue(v) + int profile_idc; // u(8) + int level_idc; // u(8) + int toolset_idc_h; // u(32) + int toolset_idc_l; // u(32) + int chroma_format_idc; // ue(v) + int pic_width_in_luma_samples; // ue(v) + int pic_height_in_luma_samples; // ue(v) + int bit_depth_luma_minus8; // ue(v) + int bit_depth_chroma_minus8; // ue(v) + + int sps_btt_flag; // u(1) + int log2_ctu_size_minus5; // ue(v) + int log2_min_cb_size_minus2; // ue(v) + int log2_diff_ctu_max_14_cb_size; // ue(v) + int log2_diff_ctu_max_tt_cb_size; // ue(v) + int log2_diff_min_cb_min_tt_cb_size_minus2; // ue(v) + + int sps_suco_flag; // u(1) + int log2_diff_ctu_size_max_suco_cb_size; // ue(v) + int log2_diff_max_suco_min_suco_cb_size; // ue(v) + + int sps_admvp_flag; // u(1) + int sps_affine_flag; // u(1) + int sps_amvr_flag; // u(1) + int sps_dmvr_flag; // u(1) + int sps_mmvd_flag; // u(1) + int sps_hmvp_flag; // u(1) + + int sps_eipd_flag; // u(1) + int sps_ibc_flag; // u(1) + int log2_max_ibc_cand_size_minus2; // ue(v) + + int sps_cm_init_flag; // u(1) + int sps_adcc_flag; // u(1) + + int sps_iqt_flag; // u(1) + int sps_ats_flag; // u(1) + + int sps_addb_flag; // u(1) + int sps_alf_flag; // u(1) + int sps_htdf_flag; // u(1) + int sps_rpl_flag; // u(1) + int sps_pocs_flag; // u(1) + int sps_dquant_flag; // u(1) + int sps_dra_flag; // u(1) + + int log2_max_pic_order_cnt_lsb_minus4; // ue(v) + int log2_sub_gop_length; // ue(v) + int log2_ref_pic_gap_length; // ue(v) + + int max_num_tid0_ref_pics; // ue(v) + + int sps_max_dec_pic_buffering_minus1; // ue(v) + int long_term_ref_pic_flag; // u(1) + int rpl1_same_as_rpl0_flag; // u(1) + int num_ref_pic_list_in_sps[2]; // ue(v) + RefPicListStruct rpls[2][MAX_NUM_RPLS]; + + int picture_cropping_flag; // u(1) + int picture_crop_left_offset; // ue(v) + int picture_crop_right_offset; // ue(v) + int picture_crop_top_offset; // ue(v) + int picture_crop_bottom_offset; // ue(v) + + // @note + // Currently the structure does not reflect the entire SPS RBSP layout. + // It contains only the fields that are necessary to read from the NAL unit all the values + // necessary for the correct initialization of the AVCodecContext structure. + + // @note + // If necessary, add the missing fields to the structure to reflect + // the contents of the entire NAL unit of the SPS type + +} EVCParserSPS; + +typedef struct EVCParserPPS { + int pps_pic_parameter_set_id; // ue(v) + int pps_seq_parameter_set_id; // ue(v) + int num_ref_idx_default_active_minus1[2]; // ue(v) + int additional_lt_poc_lsb_len; // ue(v) + int rpl1_idx_present_flag; // u(1) + int single_tile_in_pic_flag; // u(1) + int num_tile_columns_minus1; // ue(v) + int num_tile_rows_minus1; // ue(v) + int uniform_tile_spacing_flag; // u(1) + int tile_column_width_minus1[MAX_NUM_TILES_ROW]; // ue(v) + int tile_row_height_minus1[MAX_NUM_TILES_COL]; // ue(v) + int loop_filter_across_tiles_enabled_flag; // u(1) + int tile_offset_len_minus1; // ue(v) + int tile_id_len_minus1; // ue(v) + int explicit_tile_id_flag; // u(1) + int tile_id_val[MAX_NUM_TILES_ROW][MAX_NUM_TILES_COL]; // u(v) + int pic_dra_enabled_flag; // u(1) + int pic_dra_aps_id; // u(5) + int arbitrary_slice_present_flag; // u(1) + int constrained_intra_pred_flag; // u(1) + int cu_qp_delta_enabled_flag; // u(1) + int log2_cu_qp_delta_area_minus6; // ue(v) + +} EVCParserPPS; + +// The sturcture reflects Slice Header RBSP(raw byte sequence payload) layout +// @see ISO_IEC_23094-1 section 7.3.2.6 +// +// The following descriptors specify the parsing process of each element +// u(n) - unsigned integer using n bits +// ue(v) - unsigned integer 0-th order Exp_Golomb-coded syntax element with the left bit first +// u(n) - unsigned integer using n bits. +// When n is "v" in the syntax table, the number of bits varies in a manner dependent on the value of other syntax elements. +typedef struct EVCParserSliceHeader { + int slice_pic_parameter_set_id; // ue(v) + int single_tile_in_slice_flag; // u(1) + int first_tile_id; // u(v) + int arbitrary_slice_flag; // u(1) + int last_tile_id; // u(v) + int num_remaining_tiles_in_slice_minus1; // ue(v) + int delta_tile_id_minus1[MAX_NUM_TILES_ROW * MAX_NUM_TILES_COL]; // ue(v) + + int slice_type; // ue(v) + + // @note + // Currently the structure does not reflect the entire Slice Header RBSP layout. + // It contains only the fields that are necessary to read from the NAL unit all the values + // necessary for the correct initialization of the AVCodecContext structure. + + // @note + // If necessary, add the missing fields to the structure to reflect + // the contents of the entire NAL unit of the SPS type + +} EVCParserSliceHeader; + +typedef struct EVCParserContext { + ParseContext pc; + EVCParserSPS sps[MAX_SPS_CNT]; + EVCParserPPS pps[MAX_PPS_CNT]; + EVCParserSliceHeader slice_header[MAX_PPS_CNT]; + int is_avc; + int nal_length_size; + int to_read; + int incomplete_nalu_prefix_read; // The flag is set to 1 when an incomplete NAL unit prefix has been read + +} EVCParserContext; + +static int get_nalu_type(const uint8_t *bits, int bits_size, AVCodecContext *avctx) +{ + int unit_type_plus1 = 0; + + if (bits_size >= EVC_NAL_HEADER_SIZE) { + unsigned char *p = (unsigned char *)bits; + // forbidden_zero_bit + if ((p[0] & 0x80) != 0) + return -1; + + // nal_unit_type + unit_type_plus1 = (p[0] >> 1) & 0x3F; + } + + return unit_type_plus1 - 1; +} + +static uint32_t read_nal_unit_length(const uint8_t *bits, int bits_size, AVCodecContext *avctx) +{ + uint32_t nalu_len = 0; + + if (bits_size >= EVC_NAL_UNIT_LENGTH_BYTE) { + + int t = 0; + unsigned char *p = (unsigned char *)bits; + + for (int i = 0; i < EVC_NAL_UNIT_LENGTH_BYTE; i++) + t = (t << 8) | p[i]; + + nalu_len = t; + if (nalu_len == 0) + return 0; + } + + return nalu_len; +} + +// @see ISO_IEC_23094-1 (7.3.2.1 SPS RBSP syntax) +static EVCParserSPS *parse_sps(const uint8_t *bs, int bs_size, EVCParserContext *ev) +{ + GetBitContext gb; + EVCParserSPS *sps; + int sps_seq_parameter_set_id; + + if (init_get_bits8(&gb, bs, bs_size) < 0) + return NULL; + + sps_seq_parameter_set_id = get_ue_golomb(&gb); + + if (sps_seq_parameter_set_id >= MAX_SPS_CNT) + return NULL; + + sps = &ev->sps[sps_seq_parameter_set_id]; + sps->sps_seq_parameter_set_id = sps_seq_parameter_set_id; + + // the Baseline profile is indicated by profile_idc eqal to 0 + // the Main profile is indicated by profile_idc eqal to 1 + sps->profile_idc = get_bits(&gb, 8); + + sps->level_idc = get_bits(&gb, 8); + + skip_bits_long(&gb, 32); /* skip toolset_idc_h */ + skip_bits_long(&gb, 32); /* skip toolset_idc_l */ + + // 0 - monochrome + // 1 - 4:2:0 + // 2 - 4:2:2 + // 3 - 4:4:4 + sps->chroma_format_idc = get_ue_golomb(&gb); + + sps->pic_width_in_luma_samples = get_ue_golomb(&gb); + sps->pic_height_in_luma_samples = get_ue_golomb(&gb); + + sps->bit_depth_luma_minus8 = get_ue_golomb(&gb); + sps->bit_depth_chroma_minus8 = get_ue_golomb(&gb); + + sps->sps_btt_flag = get_bits(&gb, 1); + if (sps->sps_btt_flag) { + sps->log2_ctu_size_minus5 = get_ue_golomb(&gb); + sps->log2_min_cb_size_minus2 = get_ue_golomb(&gb); + sps->log2_diff_ctu_max_14_cb_size = get_ue_golomb(&gb); + sps->log2_diff_ctu_max_tt_cb_size = get_ue_golomb(&gb); + sps->log2_diff_min_cb_min_tt_cb_size_minus2 = get_ue_golomb(&gb); + } + + sps->sps_suco_flag = get_bits(&gb, 1); + if (sps->sps_suco_flag) { + sps->log2_diff_ctu_size_max_suco_cb_size = get_ue_golomb(&gb); + sps->log2_diff_max_suco_min_suco_cb_size = get_ue_golomb(&gb); + } + + sps->sps_admvp_flag = get_bits(&gb, 1); + if (sps->sps_admvp_flag) { + sps->sps_affine_flag = get_bits(&gb, 1); + sps->sps_amvr_flag = get_bits(&gb, 1); + sps->sps_dmvr_flag = get_bits(&gb, 1); + sps->sps_mmvd_flag = get_bits(&gb, 1); + sps->sps_hmvp_flag = get_bits(&gb, 1); + } + + sps->sps_eipd_flag = get_bits(&gb, 1); + if (sps->sps_eipd_flag) { + sps->sps_ibc_flag = get_bits(&gb, 1); + if (sps->sps_ibc_flag) + sps->log2_max_ibc_cand_size_minus2 = get_ue_golomb(&gb); + } + + sps->sps_cm_init_flag = get_bits(&gb, 1); + if (sps->sps_cm_init_flag) + sps->sps_adcc_flag = get_bits(&gb, 1); + + sps->sps_iqt_flag = get_bits(&gb, 1); + if (sps->sps_iqt_flag) + sps->sps_ats_flag = get_bits(&gb, 1); + + sps->sps_addb_flag = get_bits(&gb, 1); + sps->sps_alf_flag = get_bits(&gb, 1); + sps->sps_htdf_flag = get_bits(&gb, 1); + sps->sps_rpl_flag = get_bits(&gb, 1); + sps->sps_pocs_flag = get_bits(&gb, 1); + sps->sps_dquant_flag = get_bits(&gb, 1); + sps->sps_dra_flag = get_bits(&gb, 1); + + if (sps->sps_pocs_flag) + sps->log2_max_pic_order_cnt_lsb_minus4 = get_ue_golomb(&gb); + + if (!sps->sps_pocs_flag || !sps->sps_rpl_flag) { + sps->log2_sub_gop_length = get_ue_golomb(&gb); + if (sps->log2_sub_gop_length == 0) + sps->log2_ref_pic_gap_length = get_ue_golomb(&gb); + } + + // @note + // If necessary, add the missing fields to the EVCParserSPS structure + // and then extend parser implementation + + return sps; +} + +// @see ISO_IEC_23094-1 (7.3.2.2 SPS RBSP syntax) +static EVCParserPPS *parse_pps(const uint8_t *bs, int bs_size, EVCParserContext *ev) +{ + GetBitContext gb; + EVCParserPPS *pps; + + int pps_pic_parameter_set_id; + + if (init_get_bits8(&gb, bs, bs_size) < 0) + return NULL; + + pps_pic_parameter_set_id = get_ue_golomb(&gb); + if (pps_pic_parameter_set_id > MAX_PPS_CNT) + return NULL; + + pps = &ev->pps[pps_pic_parameter_set_id]; + + pps->pps_pic_parameter_set_id = pps_pic_parameter_set_id; + + pps->pps_seq_parameter_set_id = get_ue_golomb(&gb); + if (pps->pps_seq_parameter_set_id >= MAX_SPS_CNT) + return NULL; + + pps->num_ref_idx_default_active_minus1[0] = get_ue_golomb(&gb); + pps->num_ref_idx_default_active_minus1[1] = get_ue_golomb(&gb); + pps->additional_lt_poc_lsb_len = get_ue_golomb(&gb); + pps->rpl1_idx_present_flag = get_bits(&gb, 1); + pps->single_tile_in_pic_flag = get_bits(&gb, 1); + + if (!pps->single_tile_in_pic_flag) { + pps->num_tile_columns_minus1 = get_ue_golomb(&gb); + pps->num_tile_rows_minus1 = get_ue_golomb(&gb); + pps->uniform_tile_spacing_flag = get_bits(&gb, 1); + + if (!pps->uniform_tile_spacing_flag) { + for (int i = 0; i < pps->num_tile_columns_minus1; i++) + pps->tile_column_width_minus1[i] = get_ue_golomb(&gb); + + for (int i = 0; i < pps->num_tile_rows_minus1; i++) + pps->tile_row_height_minus1[i] = get_ue_golomb(&gb); + } + pps->loop_filter_across_tiles_enabled_flag = get_bits(&gb, 1); + pps->tile_offset_len_minus1 = get_ue_golomb(&gb); + } + + pps->tile_id_len_minus1 = get_ue_golomb(&gb); + pps->explicit_tile_id_flag = get_bits(&gb, 1); + + if (pps->explicit_tile_id_flag) { + for (int i = 0; i <= pps->num_tile_rows_minus1; i++) { + for (int j = 0; j <= pps->num_tile_columns_minus1; j++) + pps->tile_id_val[i][j] = get_bits(&gb, pps->tile_id_len_minus1 + 1); + } + } + + pps->pic_dra_enabled_flag = 0; + pps->pic_dra_enabled_flag = get_bits(&gb, 1); + + if (pps->pic_dra_enabled_flag) + pps->pic_dra_aps_id = get_bits(&gb, 5); + + pps->arbitrary_slice_present_flag = get_bits(&gb, 1); + pps->constrained_intra_pred_flag = get_bits(&gb, 1); + pps->cu_qp_delta_enabled_flag = get_bits(&gb, 1); + + if (pps->cu_qp_delta_enabled_flag) + pps->log2_cu_qp_delta_area_minus6 = get_ue_golomb(&gb); + + return pps; +} + +// @see ISO_IEC_23094-1 (7.3.2.6 Slice layer RBSP syntax) +static EVCParserSliceHeader *parse_slice_header(const uint8_t *bs, int bs_size, EVCParserContext *ev) +{ + GetBitContext gb; + EVCParserSliceHeader *sh; + EVCParserPPS *pps; + int num_tiles_in_slice = 0; + int slice_pic_parameter_set_id; + + if (init_get_bits8(&gb, bs, bs_size) < 0) + return NULL; + + slice_pic_parameter_set_id = get_ue_golomb(&gb); + + if (slice_pic_parameter_set_id < 0 || slice_pic_parameter_set_id >= MAX_PPS_CNT) + return NULL; + + sh = &ev->slice_header[slice_pic_parameter_set_id]; + pps = &ev->pps[slice_pic_parameter_set_id]; + + sh->slice_pic_parameter_set_id = slice_pic_parameter_set_id; + + if (!pps->single_tile_in_pic_flag) { + sh->single_tile_in_slice_flag = get_bits(&gb, 1); + sh->first_tile_id = get_bits(&gb, pps->tile_id_len_minus1 + 1); + } else + sh->single_tile_in_slice_flag = 1; + + if (!sh->single_tile_in_slice_flag) { + if (pps->arbitrary_slice_present_flag) + sh->arbitrary_slice_flag = get_bits(&gb, 1); + + if (!sh->arbitrary_slice_flag) + sh->last_tile_id = get_bits(&gb, pps->tile_id_len_minus1 + 1); + else { + sh->num_remaining_tiles_in_slice_minus1 = get_ue_golomb(&gb); + num_tiles_in_slice = sh->num_remaining_tiles_in_slice_minus1 + 2; + for (int i = 0; i < num_tiles_in_slice - 1; ++i) + sh->delta_tile_id_minus1[i] = get_ue_golomb(&gb); + } + } + + sh->slice_type = get_ue_golomb(&gb); + + // @note + // If necessary, add the missing fields to the EVCParserSliceHeader structure + // and then extend parser implementation + + return sh; +} + +static int parse_nal_units(AVCodecParserContext *s, const uint8_t *bs, + int bs_size, AVCodecContext *avctx) +{ + EVCParserContext *ev = s->priv_data; + int nalu_type, nalu_size; + unsigned char *bits = (unsigned char *)bs; + int bits_size = bs_size; + + avctx->codec_id = AV_CODEC_ID_EVC; + s->pict_type = AV_PICTURE_TYPE_NONE; + s->picture_structure = AV_PICTURE_STRUCTURE_FRAME; + s->key_frame = -1; + + nalu_size = read_nal_unit_length(bits, bits_size, avctx); + if (nalu_size == 0) { + av_log(avctx, AV_LOG_ERROR, "Invalid NAL unit size: (%d)\n", nalu_size); + return -1; + } + + bits += EVC_NAL_UNIT_LENGTH_BYTE; + bits_size -= EVC_NAL_UNIT_LENGTH_BYTE; + + nalu_type = get_nalu_type(bits, bits_size, avctx); + + bits += EVC_NAL_HEADER_SIZE; + bits_size -= EVC_NAL_HEADER_SIZE; + + if (nalu_type == EVC_NUT_SPS) { // NAL Unit type: SPS (Sequence Parameter Set) + EVCParserSPS *sps; + + sps = parse_sps(bits, bits_size, ev); + if (!sps) { + av_log(avctx, AV_LOG_ERROR, "SPS parsing error\n"); + return -1; + } + + s->coded_width = sps->pic_width_in_luma_samples; + s->coded_height = sps->pic_height_in_luma_samples; + s->width = sps->pic_width_in_luma_samples; + s->height = sps->pic_height_in_luma_samples; + + if (sps->profile_idc == 1) avctx->profile = FF_PROFILE_EVC_MAIN; + else avctx->profile = FF_PROFILE_EVC_BASELINE; + + // Currently XEVD decoder supports ony YCBCR420_10LE chroma format for EVC stream + switch (sps->chroma_format_idc) { + case 0: /* YCBCR400_10LE */ + av_log(avctx, AV_LOG_ERROR, "YCBCR400_10LE: Not supported chroma format\n"); + s->format = AV_PIX_FMT_GRAY10LE; + return -1; + case 1: /* YCBCR420_10LE */ + s->format = AV_PIX_FMT_YUV420P10LE; + break; + case 2: /* YCBCR422_10LE */ + av_log(avctx, AV_LOG_ERROR, "YCBCR422_10LE: Not supported chroma format\n"); + s->format = AV_PIX_FMT_YUV422P10LE; + return -1; + case 3: /* YCBCR444_10LE */ + av_log(avctx, AV_LOG_ERROR, "YCBCR444_10LE: Not supported chroma format\n"); + s->format = AV_PIX_FMT_YUV444P10LE; + return -1; + default: + s->format = AV_PIX_FMT_NONE; + av_log(avctx, AV_LOG_ERROR, "Unknown supported chroma format\n"); + return -1; + } + + // @note + // The current implementation of parse_sps function doesn't handle VUI parameters parsing. + // If it will be needed, parse_sps function could be extended to handle VUI parameters parsing + // to initialize fields of the AVCodecContex i.e. color_primaries, color_trc,color_range + + } else if (nalu_type == EVC_NUT_PPS) { // NAL Unit type: PPS (Video Parameter Set) + EVCParserPPS *pps; + + pps = parse_pps(bits, bits_size, ev); + if (!pps) { + av_log(avctx, AV_LOG_ERROR, "PPS parsing error\n"); + return -1; + } + } else if (nalu_type == EVC_NUT_SEI) // NAL unit type: SEI (Supplemental Enhancement Information) + return 0; + else if (nalu_type == EVC_NUT_IDR || nalu_type == EVC_NUT_NONIDR) { // NAL Unit type: Coded slice of a IDR or non-IDR picture + EVCParserSliceHeader *sh; + + sh = parse_slice_header(bits, bits_size, ev); + if (!sh) { + av_log(avctx, AV_LOG_ERROR, "Slice header parsing error\n"); + return -1; + } + switch (sh->slice_type) { + case SLICE_TYPE_B: { + s->pict_type = AV_PICTURE_TYPE_B; + break; + } + case SLICE_TYPE_P: { + s->pict_type = AV_PICTURE_TYPE_P; + break; + } + case SLICE_TYPE_I: { + s->pict_type = AV_PICTURE_TYPE_I; + break; + } + default: { + s->pict_type = AV_PICTURE_TYPE_NONE; + } + } + s->key_frame = (nalu_type == EVC_NUT_IDR) ? 1 : 0; + } else { + av_log(avctx, AV_LOG_ERROR, "Invalid NAL unit type: %d\n", nalu_type); + return -1; + } + + return 0; +} + +/** + * Find the end of the current frame in the bitstream. + * @return the position of the first byte of the next frame, or END_NOT_FOUND + */ +static int evc_find_frame_end(AVCodecParserContext *s, const uint8_t *buf, + int buf_size, AVCodecContext *avctx) +{ + EVCParserContext *ev = s->priv_data; + + if (!ev->to_read) { + int nal_unit_size = 0; + int next = END_NOT_FOUND; + + // This is the case when buffer size is not enough for buffer to store NAL unit length + if (buf_size < EVC_NAL_UNIT_LENGTH_BYTE) { + ev->to_read = EVC_NAL_UNIT_LENGTH_BYTE; + ev->nal_length_size = buf_size; + ev->incomplete_nalu_prefix_read = 1; + + return END_NOT_FOUND; + } + + nal_unit_size = read_nal_unit_length(buf, buf_size, avctx); + ev->nal_length_size = EVC_NAL_UNIT_LENGTH_BYTE; + + next = nal_unit_size + EVC_NAL_UNIT_LENGTH_BYTE; + ev->to_read = next; + if (next < buf_size) + return next; + else + return END_NOT_FOUND; + } else if (ev->to_read > buf_size) + return END_NOT_FOUND; + else { + if (ev->incomplete_nalu_prefix_read == 1) { + EVCParserContext *ev = s->priv_data; + ParseContext *pc = &ev->pc; + uint8_t nalu_len[EVC_NAL_UNIT_LENGTH_BYTE] = {0}; + int nal_unit_size = 0; + + // 1. pc->buffer contains previously read bytes of NALU prefix + // 2. buf contains the rest of NAL unit prefix bytes + // + // ~~~~~~~ + // EXAMPLE + // ~~~~~~~ + // + // In the following example we assumed that the number of already read NAL Unit prefix bytes is equal 1 + // + // ---------- + // pc->buffer -> conatins already read bytes + // ---------- + // __ pc->index == 1 + // | + // V + // ------------------------------------------------------- + // | 0 | 1 | 2 | 3 | 4 | ... | N | + // ------------------------------------------------------- + // | 0x00 | 0xXX | 0xXX | 0xXX | 0xXX | ... | 0xXX | + // ------------------------------------------------------- + // + // ---------- + // buf -> contains newly read bytes + // ---------- + // ------------------------------------------------------- + // | 0 | 1 | 2 | 3 | 4 | ... | N | + // ------------------------------------------------------- + // | 0x00 | 0x00 | 0x3C | 0xXX | 0xXX | ... | 0xXX | + // ------------------------------------------------------- + // + for (int i = 0; i < EVC_NAL_UNIT_LENGTH_BYTE; i++) { + if (i < pc->index) + nalu_len[i] = pc->buffer[i]; + else + nalu_len[i] = buf[i - pc->index]; + } + + // ---------- + // nalu_len + // ---------- + // --------------------------------- + // | 0 | 1 | 2 | 3 | + // --------------------------------- + // | 0x00 | 0x00 | 0x00 | 0x3C | + // --------------------------------- + // | NALU LENGTH | + // --------------------------------- + // NAL Unit lenght = 60 (0x0000003C) + + nal_unit_size = read_nal_unit_length(nalu_len, EVC_NAL_UNIT_LENGTH_BYTE, avctx); + + ev->to_read = nal_unit_size + EVC_NAL_UNIT_LENGTH_BYTE - pc->index; + + ev->incomplete_nalu_prefix_read = 0; + + if (ev->to_read > buf_size) + return END_NOT_FOUND; + else + return ev->to_read; + } + return ev->to_read; + } + + return END_NOT_FOUND; +} + +static int evc_parser_init(AVCodecParserContext *s) +{ + EVCParserContext *ev = s->priv_data; + + ev->nal_length_size = EVC_NAL_UNIT_LENGTH_BYTE; + ev->incomplete_nalu_prefix_read = 0; + + return 0; +} + +static int evc_parse(AVCodecParserContext *s, AVCodecContext *avctx, + const uint8_t **poutbuf, int *poutbuf_size, + const uint8_t *buf, int buf_size) +{ + int next; + EVCParserContext *ev = s->priv_data; + ParseContext *pc = &ev->pc; + int is_dummy_buf = !buf_size; + const uint8_t *dummy_buf = buf; + + if (s->flags & PARSER_FLAG_COMPLETE_FRAMES) + next = buf_size; + else { + next = evc_find_frame_end(s, buf, buf_size, avctx); + if (ff_combine_frame(pc, next, &buf, &buf_size) < 0) { + *poutbuf = NULL; + *poutbuf_size = 0; + ev->to_read -= buf_size; + return buf_size; + } + } + + is_dummy_buf &= (dummy_buf == buf); + + if (!is_dummy_buf) + parse_nal_units(s, buf, buf_size, avctx); + + // poutbuf contains just one NAL unit + *poutbuf = buf; + *poutbuf_size = buf_size; + ev->to_read -= next; + + return next; +} + +const AVCodecParser ff_evc_parser = { + .codec_ids = { AV_CODEC_ID_EVC }, + .priv_data_size = sizeof(EVCParserContext), + .parser_init = evc_parser_init, + .parser_parse = evc_parse, + .parser_close = ff_parse_close, +}; diff --git a/libavcodec/libxevd.c b/libavcodec/libxevd.c new file mode 100644 index 0000000000..ea57be74f6 --- /dev/null +++ b/libavcodec/libxevd.c @@ -0,0 +1,420 @@ +/* + * libxevd decoder + * EVC (MPEG-5 Essential Video Coding) decoding using XEVD MPEG-5 EVC decoder library + * + * Copyright (C) 2021 Dawid Kozinski <d.kozinski@samsung.com> + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include <float.h> +#include <stdlib.h> + +#include <xevd.h> + +#include "libavutil/internal.h" +#include "libavutil/common.h" +#include "libavutil/opt.h" +#include "libavutil/pixdesc.h" +#include "libavutil/pixfmt.h" +#include "libavutil/imgutils.h" +#include "libavutil/cpu.h" + +#include "avcodec.h" +#include "internal.h" +#include "packet_internal.h" +#include "codec_internal.h" + +#define XEVD_PARAM_BAD_NAME -1 +#define XEVD_PARAM_BAD_VALUE -2 + +#define EVC_NAL_HEADER_SIZE 2 /* byte */ + +/** + * The structure stores all the states associated with the instance of Xeve MPEG-5 EVC decoder + */ +typedef struct XevdContext { + const AVClass *class; + + XEVD id; // XEVD instance identifier @see xevd.h + XEVD_CDSC cdsc; // decoding parameters @see xevd.h +} XevdContext; + +/** + * The function populates the XEVD_CDSC structure. + * XEVD_CDSC contains all decoder parameters that should be initialized before its use. + * + * @param[in] avctx codec context + * @param[out] cdsc contains all decoder parameters that should be initialized before its use + * + */ +static void get_conf(AVCodecContext *avctx, XEVD_CDSC *cdsc) +{ + int cpu_count = av_cpu_count(); + + /* clear XEVS_CDSC structure */ + memset(cdsc, 0, sizeof(XEVD_CDSC)); + + /* init XEVD_CDSC */ + if (avctx->thread_count <= 0) + cdsc->threads = (cpu_count < XEVD_MAX_TASK_CNT) ? cpu_count : XEVD_MAX_TASK_CNT; + else if (avctx->thread_count > XEVD_MAX_TASK_CNT) + cdsc->threads = XEVD_MAX_TASK_CNT; + else + cdsc->threads = avctx->thread_count; +} + +/** + * Read NAL unit length + * @param bs input data (bitstream) + * @return the length of NAL unit on success, 0 value on failure + */ +static uint32_t read_nal_unit_length(const uint8_t *bs, int bs_size, AVCodecContext *avctx) +{ + uint32_t len = 0; + XEVD_INFO info; + int ret; + + if (bs_size == XEVD_NAL_UNIT_LENGTH_BYTE) { + ret = xevd_info((void *)bs, XEVD_NAL_UNIT_LENGTH_BYTE, 1, &info); + if (XEVD_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Cannot get bitstream information\n"); + return 0; + } + len = info.nalu_len; + if (len == 0) { + av_log(avctx, AV_LOG_ERROR, "Invalid bitstream size! [%d]\n", bs_size); + return 0; + } + } + + return len; +} + +/** + * @param[in] xectx the structure that stores all the state associated with the instance of Xeve MPEG-5 EVC decoder + * @param[out] avctx codec context + * @return 0 on success, negative value on failure + */ +static int export_stream_params(const XevdContext *xectx, AVCodecContext *avctx) +{ + int ret; + int size; + int color_space; + + avctx->pix_fmt = AV_PIX_FMT_YUV420P10; + + size = 4; + ret = xevd_config(xectx->id, XEVD_CFG_GET_CODED_WIDTH, &avctx->coded_width, &size); + if (XEVD_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to get coded_width\n"); + return AVERROR_EXTERNAL; + } + + ret = xevd_config(xectx->id, XEVD_CFG_GET_CODED_HEIGHT, &avctx->coded_height, &size); + if (XEVD_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to get coded_height\n"); + return AVERROR_EXTERNAL; + } + + ret = xevd_config(xectx->id, XEVD_CFG_GET_WIDTH, &avctx->width, &size); + if (XEVD_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to get width\n"); + return AVERROR_EXTERNAL; + } + + ret = xevd_config(xectx->id, XEVD_CFG_GET_HEIGHT, &avctx->height, &size); + if (XEVD_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to get height\n"); + return AVERROR_EXTERNAL; + } + + ret = xevd_config(xectx->id, XEVD_CFG_GET_COLOR_SPACE, &color_space, &size); + if (XEVD_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to get color_space\n"); + return AVERROR_EXTERNAL; + } + switch(color_space) { + case XEVD_CS_YCBCR400_10LE: + avctx->pix_fmt = AV_PIX_FMT_GRAY10LE; + break; + case XEVD_CS_YCBCR420_10LE: + avctx->pix_fmt = AV_PIX_FMT_YUV420P10LE; + break; + case XEVD_CS_YCBCR422_10LE: + avctx->pix_fmt = AV_PIX_FMT_YUV422P10LE; + break; + case XEVD_CS_YCBCR444_10LE: + avctx->pix_fmt = AV_PIX_FMT_YUV444P10LE; + break; + default: + av_log(avctx, AV_LOG_ERROR, "Unknown color space\n"); + avctx->pix_fmt = AV_PIX_FMT_NONE; + return AVERROR_INVALIDDATA; + } + + // the function returns sps->num_reorder_pics + ret = xevd_config(xectx->id, XEVD_CFG_GET_MAX_CODING_DELAY, &avctx->max_b_frames, &size); + if (XEVD_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to get max_coding_delay\n"); + return AVERROR_EXTERNAL; + } + + avctx->has_b_frames = (avctx->max_b_frames) ? 1 : 0; + + avctx->color_primaries = AVCOL_PRI_UNSPECIFIED; + avctx->color_trc = AVCOL_TRC_UNSPECIFIED; + avctx->colorspace = AVCOL_SPC_UNSPECIFIED; + + return 0; +} + +/** + * @brief Copy image in imgb to frame. + * + * @param avctx codec context + * @param[in] imgb + * @param[out] frame + * @return 0 on success, negative value on failure + */ +static int libxevd_image_copy(struct AVCodecContext *avctx, XEVD_IMGB *imgb, struct AVFrame *frame) +{ + int ret; + if (imgb->cs != XEVD_CS_YCBCR420_10LE) { + av_log(avctx, AV_LOG_ERROR, "Not supported pixel format: %s\n", av_get_pix_fmt_name(avctx->pix_fmt)); + return AVERROR_INVALIDDATA; + } + + if (imgb->w[0] != avctx->width || imgb->h[0] != avctx->height) { // stream resolution changed + if (ff_set_dimensions(avctx, imgb->w[0], imgb->h[0]) < 0) { + av_log(avctx, AV_LOG_ERROR, "Cannot set new dimension\n"); + return AVERROR_INVALIDDATA; + } + } + + if (ret = ff_get_buffer(avctx, frame, 0) < 0) { + return ret; + } + + av_image_copy(frame->data, frame->linesize, (const uint8_t **)imgb->a, + imgb->s, avctx->pix_fmt, + imgb->w[0], imgb->h[0]); + + return 0; +} + +/** + * Initialize decoder + * Create a decoder instance and allocate all the needed resources + * + * @param avctx codec context + * @return 0 on success, negative error code on failure + */ +static av_cold int libxevd_init(AVCodecContext *avctx) +{ + XevdContext *xectx = avctx->priv_data; + XEVD_CDSC *cdsc = &(xectx->cdsc); + + /* read configurations and set values for created descriptor (XEVD_CDSC) */ + get_conf(avctx, cdsc); + + /* create decoder */ + xectx->id = xevd_create(&(xectx->cdsc), NULL); + if (xectx->id == NULL) { + av_log(avctx, AV_LOG_ERROR, "Cannot create XEVD encoder\n"); + return AVERROR_EXTERNAL; + } + + return 0; +} + +/** + * Decode picture + * + * @param avctx codec context + * @param[out] frame decoded frame + * @param[out] got_frame_ptr decoder sets to 0 or 1 to indicate that a + * non-empty frame or subtitle was returned in + * outdata. + * @param[in] avpkt AVPacket containing encoded data to be decoded + * + * @return amount of bytes read from the packet on success, negative error + * code on failure + */ +static int libxevd_decode(struct AVCodecContext *avctx, struct AVFrame *frame, int *got_frame_ptr, AVPacket *avpkt) +{ + XevdContext *xectx = NULL; + XEVD_IMGB *imgb = NULL; + XEVD_STAT stat; + XEVD_BITB bitb; + int xevd_ret, nalu_size, bs_read_pos; + int ret = 0; + + xectx = avctx->priv_data; + + if (avpkt->size > 0) { + bs_read_pos = 0; + imgb = NULL; + while(avpkt->size > (bs_read_pos + XEVD_NAL_UNIT_LENGTH_BYTE)) { + memset(&stat, 0, sizeof(XEVD_STAT)); + memset(&bitb, 0, sizeof(XEVD_BITB)); + + nalu_size = read_nal_unit_length(avpkt->data + bs_read_pos, XEVD_NAL_UNIT_LENGTH_BYTE, avctx); + if (nalu_size == 0) { + av_log(avctx, AV_LOG_ERROR, "Invalid bitstream\n"); + ret = AVERROR_INVALIDDATA; + goto ERR; + } + bs_read_pos += XEVD_NAL_UNIT_LENGTH_BYTE; + + bitb.addr = avpkt->data + bs_read_pos; + bitb.ssize = nalu_size; + + /* main decoding block */ + xevd_ret = xevd_decode(xectx->id, &bitb, &stat); + if (XEVD_FAILED(xevd_ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to decode bitstream\n"); + ret = AVERROR_EXTERNAL; + goto ERR; + } + + bs_read_pos += nalu_size; + + if (stat.nalu_type == XEVD_NUT_SPS) { // EVC stream parameters changed + if ((ret = export_stream_params(xectx, avctx)) != 0) + goto ERR; + } + + if (stat.read != nalu_size) + av_log(avctx, AV_LOG_INFO, "Different reading of bitstream (in:%d, read:%d)\n,", nalu_size, stat.read); + if (stat.fnum >= 0) { + // already has a decoded image + if (imgb) { + // xevd_pull uses pool of objects of type XEVD_IMGB. + // The pool size is equal MAX_PB_SIZE (26), so release object when it is no more needed + imgb->release(imgb); + imgb = NULL; + } + xevd_ret = xevd_pull(xectx->id, &imgb); + if (XEVD_FAILED(xevd_ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to pull the decoded image (xevd error code: %d, frame#=%d)\n", xevd_ret, stat.fnum); + ret = AVERROR_EXTERNAL; + goto ERR; + } else if (xevd_ret == XEVD_OK_FRM_DELAYED) { + if (imgb) { + // xevd_pull uses pool of objects of type XEVD_IMGB. + // The pool size is equal MAX_PB_SIZE (26), so release object when it is no more needed + imgb->release(imgb); + imgb = NULL; + } + } + if (imgb) { + int ret = libxevd_image_copy(avctx, imgb, frame); + if(ret < 0) { + goto ERR; + } + + frame->pts = avpkt->pts; + *got_frame_ptr = 1; + + // xevd_pull uses pool of objects of type XEVD_IMGB. + // The pool size is equal MAX_PB_SIZE (26), so release object when it is no more needed + imgb->release(imgb); + imgb = NULL; + } else + *got_frame_ptr = 0; + } + } + } else { // bumping + xevd_ret = xevd_pull(xectx->id, &(imgb)); + if (xevd_ret == XEVD_ERR_UNEXPECTED) { // bumping process completed + *got_frame_ptr = 0; + return 0; + } else if (XEVD_FAILED(xevd_ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to pull the decoded image (xevd error code: %d)\n", xevd_ret); + ret = AVERROR_EXTERNAL; + goto ERR; + } + if (imgb) { + int ret = libxevd_image_copy(avctx, imgb, frame); + if(ret < 0) { + goto ERR; + } + + frame->pts = avpkt->pts; + *got_frame_ptr = 1; + + // xevd_pull uses pool of objects of type XEVD_IMGB. + // The pool size is equal MAX_PB_SIZE (26), so release object when it is no more needed + imgb->release(imgb); + imgb = NULL; + } else + *got_frame_ptr = 0; + } + + return avpkt->size; + +ERR: + if (imgb) { + imgb->release(imgb); + imgb = NULL; + } + *got_frame_ptr = 0; + + return ret; +} + +/** + * Destroy decoder + * + * @param avctx codec context + * @return 0 on success + */ +static av_cold int libxevd_close(AVCodecContext *avctx) +{ + XevdContext *xectx = avctx->priv_data; + if (xectx->id) { + xevd_delete(xectx->id); + xectx->id = NULL; + } + + return 0; +} + +#define OFFSET(x) offsetof(XevdContext, x) +#define VD AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_DECODING_PARAM + +static const AVClass libxevd_class = { + .class_name = "libxevd", + .item_name = av_default_item_name, + .version = LIBAVUTIL_VERSION_INT, +}; + +const FFCodec ff_libxevd_decoder = { + .p.name = "evc", + .p.long_name = NULL_IF_CONFIG_SMALL("EVC / MPEG-5 Essential Video Coding (EVC)"), + .p.type = AVMEDIA_TYPE_VIDEO, + .p.id = AV_CODEC_ID_EVC, + .init = libxevd_init, + FF_CODEC_DECODE_CB(libxevd_decode), + .close = libxevd_close, + .priv_data_size = sizeof(XevdContext), + .p.priv_class = &libxevd_class, + .p.capabilities = AV_CODEC_CAP_DELAY | AV_CODEC_CAP_OTHER_THREADS | AV_CODEC_CAP_AVOID_PROBING | AV_CODEC_CAP_DR1, + .p.wrapper_name = "libxevd", + .caps_internal = FF_CODEC_CAP_INIT_CLEANUP | FF_CODEC_CAP_NOT_INIT_THREADSAFE, +}; diff --git a/libavcodec/libxeve.c b/libavcodec/libxeve.c new file mode 100644 index 0000000000..bb1d5e4bba --- /dev/null +++ b/libavcodec/libxeve.c @@ -0,0 +1,601 @@ +/* + * libxeve encoder + * EVC (MPEG-5 Essential Video Coding) encoding using XEVE MPEG-5 EVC encoder library + * + * Copyright (C) 2021 Dawid Kozinski <d.kozinski@samsung.com> + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include <float.h> +#include <stdlib.h> + +#include <xeve.h> + +#include "libavutil/internal.h" +#include "libavutil/common.h" +#include "libavutil/opt.h" +#include "libavutil/pixdesc.h" +#include "libavutil/pixfmt.h" +#include "libavutil/time.h" +#include "libavutil/cpu.h" +#include "libavutil/avstring.h" + +#include "avcodec.h" +#include "internal.h" +#include "packet_internal.h" +#include "codec_internal.h" +#include "encode.h" + +#define MAX_BS_BUF (16*1024*1024) + +/** + * Error codes + */ +#define XEVE_PARAM_BAD_NAME -100 +#define XEVE_PARAM_BAD_VALUE -200 + +/** + * Encoder states + * + * STATE_ENCODING - the encoder receives and processes input frames + * STATE_BUMPING - there are no more input frames, however the encoder still processes previously received data + */ +typedef enum State { + STATE_ENCODING, + STATE_BUMPING, +} State; + +/** + * The structure stores all the states associated with the instance of Xeve MPEG-5 EVC encoder + */ +typedef struct XeveContext { + const AVClass *class; + + XEVE id; // XEVE instance identifier + XEVE_CDSC cdsc; // coding parameters i.e profile, width & height of input frame, num of therads, frame rate ... + XEVE_BITB bitb; // bitstream buffer (output) + XEVE_STAT stat; // encoding status (output) + XEVE_IMGB imgb; // image buffer (input) + + State state; // encoder state (skipping, encoding, bumping) + + int profile_id; // encoder profile (main, baseline) + int preset_id; // preset of xeve ( fast, medium, slow, placebo) + int tune_id; // tune of xeve (psnr, zerolatency) + + // variables for rate control modes + int rc_mode; // Rate control mode [ 0(CQP) / 1(ABR) / 2(CRF) ] + int qp; // quantization parameter (QP) [0,51] + int crf; // constant rate factor (CRF) [10,49] + + int hash; // embed picture signature (HASH) for conformance checking in decoding + int sei_info; // embed Supplemental enhancement information while encoding + + int color_format; // input data color format: currently only XEVE_CF_YCBCR420 is supported +} XeveContext; + +/** + * Convert FFmpeg pixel format (AVPixelFormat) to XEVE pre-defined color format + * + * @param[in] av_pix_fmt pixel format (@see https://ffmpeg.org/doxygen/trunk/pixfmt_8h.html#a9a8e335cf3be472042bc9f0cf80cd4c5) + * @param[out] xeve_col_fmt XEVE pre-defined color format (@see xeve.h) + * + * @return 0 on success, negative value on failure + */ +static int libxeve_color_fmt(enum AVPixelFormat av_pix_fmt, int *xeve_col_fmt) +{ + switch (av_pix_fmt) { + case AV_PIX_FMT_YUV420P: + *xeve_col_fmt = XEVE_CF_YCBCR420; + break; + case AV_PIX_FMT_YUV420P10: + *xeve_col_fmt = XEVE_CF_YCBCR420; + break; + default: + *xeve_col_fmt = XEVE_CF_UNKNOWN; + return AVERROR_INVALIDDATA; + } + + return 0; +} + +/** + * Convert FFmpeg pixel format (AVPixelFormat) into XEVE pre-defined color space + * + * @param[in] px_fmt pixel format (@see https://ffmpeg.org/doxygen/trunk/pixfmt_8h.html#a9a8e335cf3be472042bc9f0cf80cd4c5) + * + * @return XEVE pre-defined color space (@see xeve.h) on success, XEVE_CF_UNKNOWN on failure + */ +static int libxeve_color_space(enum AVPixelFormat av_pix_fmt) +{ + /* color space of input image */ + int cs = XEVE_CF_UNKNOWN; + + switch (av_pix_fmt) { + case AV_PIX_FMT_YUV420P: + cs = XEVE_CS_YCBCR420; + break; + case AV_PIX_FMT_YUV420P10: +#if AV_HAVE_BIGENDIAN + cs = XEVE_CS_SET(XEVE_CF_YCBCR420, 10, 1); +#else + cs = XEVE_CS_YCBCR420_10LE; +#endif + + break; + default: + cs = XEVE_CF_UNKNOWN; + break; + } + + return cs; +} + +/** + * The function returns a pointer to the object of the XEVE_CDSC type. + * XEVE_CDSC contains all encoder parameters that should be initialized before the encoder is used. + * + * The field values of the XEVE_CDSC structure are populated based on: + * - the corresponding field values of the AvCodecConetxt structure, + * - the xeve encoder specific option values, + * (the full list of options available for xeve encoder is displayed after executing the command ./ffmpeg --help encoder = libxeve) + * - and the xeve encoder options specified as a list of key-value pairs following the xeve-params option + * + * The order of processing input data and populating the XEVE_CDSC structure + * 1) first, the fields of the AVCodecContext structure corresponding to the provided input options are processed, + * (i.e -pix_fmt yuv420p -s:v 1920x1080 -r 30 -profile:v 0) + * 2) then xeve-specific options added as AVOption to the xeve AVCodec implementation + * (i.e -preset 0) + * + * Keep in mind that, there are options that can be set in different ways. + * In this case, please follow the above-mentioned order of processing. + * The most recent assignments overwrite the previous values. + * + * @param[in] avctx codec context (AVCodecContext) + * @param[out] cdsc contains all Xeve MPEG-5 EVC encoder encoder parameters that should be initialized before the encoder is use + * + * @return 0 on success, negative error code on failure + */ +static int get_conf(AVCodecContext *avctx, XEVE_CDSC *cdsc) +{ + XeveContext *xectx = NULL; + int ret; + + xectx = avctx->priv_data; + + /* initialize xeve_param struct with default values */ + ret = xeve_param_default(&cdsc->param); + if (XEVE_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Cannot set_default parameter\n"); + return AVERROR_EXTERNAL; + } + + /* read options from AVCodecContext */ + if (avctx->width > 0) + cdsc->param.w = avctx->width; + + if (avctx->height > 0) + cdsc->param.h = avctx->height; + + if (avctx->framerate.num > 0) { + // fps can be float number, but xeve API doesn't support it + cdsc->param.fps = (int)(((float)avctx->framerate.num / avctx->framerate.den) + 0.5); + } + + if (avctx->gop_size >= 0) // GOP size (key-frame interval, I-picture period) + cdsc->param.keyint = avctx->gop_size; // 0: only one I-frame at the first time; 1: every frame is coded in I-frame + + if (avctx->max_b_frames == 0 || avctx->max_b_frames == 1 || avctx->max_b_frames == 3 || + avctx->max_b_frames == 7 || avctx->max_b_frames == 15) // number of b-frames + cdsc->param.bframes = avctx->max_b_frames; + else { + av_log(avctx, AV_LOG_ERROR, "Incorrect value for maximum number of B frames: (%d) \n" + "Acceptable values for bf option (maximum number of B frames) are 0,1,3,7 or 15\n", avctx->max_b_frames); + return AVERROR_INVALIDDATA; + } + + if (avctx->level >= 0) + cdsc->param.level_idc = avctx->level; + + if (avctx->rc_buffer_size) // VBV buf size + cdsc->param.vbv_bufsize = (int)(avctx->rc_buffer_size / 1000); + + cdsc->param.rc_type = xectx->rc_mode; + + if (xectx->rc_mode == XEVE_RC_CQP) { + cdsc->param.qp = xectx->qp; + } else if (xectx->rc_mode == XEVE_RC_ABR) { + if (avctx->bit_rate / 1000 > INT_MAX || avctx->rc_max_rate / 1000 > INT_MAX) { + av_log(avctx, AV_LOG_ERROR, "Not supported bitrate bit_rate and rc_max_rate > %d000\n", INT_MAX); + return AVERROR_INVALIDDATA; + } + cdsc->param.bitrate = (int)(avctx->bit_rate / 1000); + } else if (xectx->rc_mode == XEVE_RC_CRF) { + cdsc->param.crf = xectx->crf; + } else { + av_log(avctx, AV_LOG_ERROR, "Not supported rate control type: %d\n", xectx->rc_mode); + return AVERROR_INVALIDDATA; + } + + if (avctx->thread_count <= 0) { + int cpu_count = av_cpu_count(); + cdsc->param.threads = (cpu_count < XEVE_MAX_THREADS) ? cpu_count : XEVE_MAX_THREADS; + } else if (avctx->thread_count > XEVE_MAX_THREADS) + cdsc->param.threads = XEVE_MAX_THREADS; + else + cdsc->param.threads = avctx->thread_count; + + + libxeve_color_fmt(avctx->pix_fmt, &xectx->color_format); + + cdsc->param.cs = XEVE_CS_SET(xectx->color_format, cdsc->param.codec_bit_depth, AV_HAVE_BIGENDIAN); + + cdsc->max_bs_buf_size = MAX_BS_BUF; + + ret = xeve_param_ppt(&cdsc->param, xectx->profile_id, xectx->preset_id, xectx->tune_id); + if (XEVE_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Cannot set profile(%d), preset(%d), tune(%d)\n", xectx->profile_id, xectx->preset_id, xectx->tune_id); + return AVERROR_EXTERNAL; + } + + return 0; +} + +/** + * Set XEVE_CFG_SET_USE_PIC_SIGNATURE for encoder + * + * @param[in] logger context + * @param[in] id XEVE encodec instance identifier + * @param[in] ctx the structure stores all the states associated with the instance of Xeve MPEG-5 EVC encoder + * + * @return 0 on success, negative error code on failure + */ +static int set_extra_config(AVCodecContext *avctx, XEVE id, XeveContext *ctx) +{ + int ret, size; + size = 4; + + // embed SEI messages identifying encoder parameters and command line arguments + // - 0: off\n" + // - 1: emit sei info" + // + // SEI - Supplemental enhancement information contains information + // that is not necessary to decode the samples of coded pictures from VCL NAL units. + // Some SEI message information is required to check bitstream conformance + // and for output timing decoder conformance. + // @see ISO_IEC_23094-1_2020 7.4.3.5 + // @see ISO_IEC_23094-1_2020 Annex D + ret = xeve_config(id, XEVE_CFG_SET_SEI_CMD, &ctx->sei_info, &size); // sei_cmd_info + if (XEVE_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to set config for sei command info messages\n"); + return AVERROR_EXTERNAL; + } + + ret = xeve_config(id, XEVE_CFG_SET_USE_PIC_SIGNATURE, &ctx->hash, &size); + if (XEVE_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to set config for picture signature\n"); + return AVERROR_EXTERNAL; + } + + return 0; +} + +/** + * @brief Switch encoder to bumping mode + * + * @param id XEVE encodec instance identifier + * @return 0 on success, negative error code on failure + */ +static int setup_bumping(XEVE id) +{ + int val = 1; + int size = sizeof(int); + if (XEVE_FAILED(xeve_config(id, XEVE_CFG_SET_FORCE_OUT, (void *)(&val), &size))) + return AVERROR_EXTERNAL; + + return 0; +} + +/** + * @brief Initialize eXtra-fast Essential Video Encoder codec + * Create an encoder instance and allocate all the needed resources + * + * @param avctx codec context + * @return 0 on success, negative error code on failure + */ +static av_cold int libxeve_init(AVCodecContext *avctx) +{ + XeveContext *xectx = avctx->priv_data; + unsigned char *bs_buf = NULL; + int i; + int shift_h = 0; + int shift_v = 0; + int width_chroma = 0; + int height_chroma = 0; + XEVE_IMGB *imgb = NULL; + int ret = 0; + + XEVE_CDSC *cdsc = &(xectx->cdsc); + + /* allocate bitstream buffer */ + bs_buf = av_malloc(MAX_BS_BUF); + if (bs_buf == NULL) { + av_log(avctx, AV_LOG_ERROR, "Cannot allocate bitstream buffer\n"); + return AVERROR(ENOMEM); + } + xectx->bitb.addr = bs_buf; + xectx->bitb.bsize = MAX_BS_BUF; + + /* read configurations and set values for created descriptor (XEVE_CDSC) */ + if ((ret = get_conf(avctx, cdsc)) != 0) { + av_log(avctx, AV_LOG_ERROR, "Cannot get configuration\n"); + return AVERROR(EINVAL); + } + + if ((ret = xeve_param_check(&cdsc->param)) != 0) { + av_log(avctx, AV_LOG_ERROR, "Invalid configuration\n"); + return AVERROR(EINVAL); + } + + /* create encoder */ + xectx->id = xeve_create(cdsc, NULL); + if (xectx->id == NULL) { + av_log(avctx, AV_LOG_ERROR, "Cannot create XEVE encoder\n"); + return AVERROR_EXTERNAL; + } + + if ((ret = set_extra_config(avctx, xectx->id, xectx)) != 0) { + av_log(avctx, AV_LOG_ERROR, "Cannot set extra configuration\n"); + return AVERROR(EINVAL); + } + + if ((ret = av_pix_fmt_get_chroma_sub_sample(avctx->pix_fmt, &shift_h, &shift_v)) != 0) { + av_log(avctx, AV_LOG_ERROR, "Failed to get chroma shift\n"); + return AVERROR(EINVAL); + } + + // Chroma subsampling + // + // YUV format explanation + // shift_h == 1 && shift_v == 1 : YUV420 + // shift_h == 1 && shift_v == 0 : YUV422 + // shift_h == 0 && shift_v == 0 : YUV444 + // + width_chroma = AV_CEIL_RSHIFT(avctx->width, shift_h); + height_chroma = AV_CEIL_RSHIFT(avctx->height, shift_v); + + /* set default values for input image buffer */ + imgb = &xectx->imgb; + imgb->cs = libxeve_color_space(avctx->pix_fmt); + imgb->np = 3; /* only for yuv420p, yuv420ple */ + + for (i = 0; i < imgb->np; i++) + imgb->x[i] = imgb->y[i] = 0; + + imgb->w[0] = imgb->aw[0] = avctx->width; // width luma + imgb->w[1] = imgb->w[2] = imgb->aw[1] = imgb->aw[2] = width_chroma; + imgb->h[0] = imgb->ah[0] = avctx->height; // height luma + imgb->h[1] = imgb->h[2] = imgb->ah[1] = imgb->ah[2] = height_chroma; + + xectx->state = STATE_ENCODING; + + return 0; +} + +/** + * Encode raw data frame into EVC packet + * + * @param[in] avctx codec context + * @param[out] pkt output AVPacket containing encoded data + * @param[in] frame AVFrame containing the raw data to be encoded + * @param[out] got_packet encoder sets to 0 or 1 to indicate that a + * non-empty packet was returned in pkt + * + * @return 0 on success, negative error code on failure + */ +static int libxeve_encode(AVCodecContext *avctx, AVPacket *avpkt, + const AVFrame *frame, int *got_packet) +{ + XeveContext *xectx = avctx->priv_data; + int ret = -1; + + // No more input frames are available but encoder still can have some data in its internal buffer to process + // and some frames to dump. + if (xectx->state == STATE_ENCODING && frame == NULL) { + if (setup_bumping(xectx->id) == 0) + xectx->state = STATE_BUMPING; // Entering bumping process + else { + av_log(avctx, AV_LOG_ERROR, "Failed to setup bumping\n"); + return 0; + } + } + + if (xectx->state == STATE_ENCODING) { + int i; + XEVE_IMGB *imgb = NULL; + + imgb = &xectx->imgb; + + for (i = 0; i < imgb->np; i++) { + imgb->a[i] = frame->data[i]; + imgb->s[i] = frame->linesize[i]; + } + + imgb->ts[0] = frame->pts; + imgb->ts[1] = 0; + + /* push image to encoder */ + ret = xeve_push(xectx->id, imgb); + if (XEVE_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "xeve_push() failed\n"); + return AVERROR_EXTERNAL; + } + } + if (xectx->state == STATE_ENCODING || xectx->state == STATE_BUMPING) { + /* encoding */ + ret = xeve_encode(xectx->id, &(xectx->bitb), &(xectx->stat)); + if (XEVE_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "xeve_encode() failed\n"); + return AVERROR_EXTERNAL; + } + + /* store bitstream */ + if (ret == XEVE_OK_OUT_NOT_AVAILABLE) { // Return OK but picture is not available yet + *got_packet = 0; + return 0; + } else if (ret == XEVE_OK) { + int av_pic_type; + + if (xectx->stat.write > 0) { + + ret = ff_get_encode_buffer(avctx, avpkt, xectx->stat.write, 0); + if (ret < 0) + return ret; + + memcpy(avpkt->data, xectx->bitb.addr, xectx->stat.write); + + avpkt->pts = xectx->bitb.ts[0]; + avpkt->dts = xectx->bitb.ts[1]; + + switch(xectx->stat.stype) { + case XEVE_ST_I: + av_pic_type = AV_PICTURE_TYPE_I; + avpkt->flags |= AV_PKT_FLAG_KEY; + break; + case XEVE_ST_P: + av_pic_type = AV_PICTURE_TYPE_P; + break; + case XEVE_ST_B: + av_pic_type = AV_PICTURE_TYPE_B; + break; + case XEVE_ST_UNKNOWN: + av_log(avctx, AV_LOG_ERROR, "Unknown slice type\n"); + return AVERROR_INVALIDDATA; + } + + ff_side_data_set_encoder_stats(avpkt, xectx->stat.qp * FF_QP2LAMBDA, NULL, 0, av_pic_type); + + *got_packet = 1; + } + } else if (ret == XEVE_OK_NO_MORE_FRM) { + // Return OK but no more frames + return 0; + } else { + av_log(avctx, AV_LOG_ERROR, "Invalid return value: %d\n", ret); + return AVERROR_EXTERNAL; + } + } else { + av_log(avctx, AV_LOG_ERROR, "Udefined encoder state\n"); + return AVERROR_INVALIDDATA; + } + + return 0; +} + +/** + * Destroy the encoder and release all the allocated resources + * + * @param avctx codec context + * @return 0 on success, negative error code on failure + */ +static av_cold int libxeve_close(AVCodecContext *avctx) +{ + XeveContext *xectx = avctx->priv_data; + + if (xectx->id) { + xeve_delete(xectx->id); + xectx->id = NULL; + } + + av_free(xectx->bitb.addr); /* release bitstream buffer */ + + return 0; +} + +#define OFFSET(x) offsetof(XeveContext, x) +#define VE AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_ENCODING_PARAM + +static const enum AVPixelFormat supported_pixel_formats[] = { + AV_PIX_FMT_YUV420P, + AV_PIX_FMT_YUV420P10, + AV_PIX_FMT_NONE +}; + +// Example of using: ./ffmpeg -xeve-params "m=2:q=17" +// Consider using following options (./ffmpeg --help encoder=libxeve) +// +static const AVOption libxeve_options[] = { + { "preset", "Encoding preset for setting encoding speed", OFFSET(preset_id), AV_OPT_TYPE_INT, { .i64 = XEVE_PRESET_MEDIUM }, XEVE_PRESET_DEFAULT, XEVE_PRESET_PLACEBO, VE, "preset" }, + { "default", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_PRESET_DEFAULT }, INT_MIN, INT_MAX, VE, "preset" }, + { "fast", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_PRESET_FAST }, INT_MIN, INT_MAX, VE, "preset" }, + { "medium", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_PRESET_MEDIUM }, INT_MIN, INT_MAX, VE, "preset" }, + { "slow", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_PRESET_SLOW }, INT_MIN, INT_MAX, VE, "preset" }, + { "placebo", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_PRESET_PLACEBO }, INT_MIN, INT_MAX, VE, "preset" }, + { "tune", "Tuning parameter for special purpose operation", OFFSET(tune_id), AV_OPT_TYPE_INT, { .i64 = XEVE_TUNE_NONE }, XEVE_TUNE_NONE, XEVE_TUNE_PSNR, VE, "tune"}, + { "none", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_TUNE_NONE }, INT_MIN, INT_MAX, VE, "tune" }, + { "zerolatency", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_TUNE_ZEROLATENCY }, INT_MIN, INT_MAX, VE, "tune" }, + { "psnr", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_TUNE_PSNR }, INT_MIN, INT_MAX, VE, "tune" }, + { "profile", "Encoding profile", OFFSET(profile_id), AV_OPT_TYPE_INT, { .i64 = XEVE_PROFILE_BASELINE }, XEVE_PROFILE_BASELINE, XEVE_PROFILE_MAIN, VE, "profile" }, + { "baseline", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_PROFILE_BASELINE }, INT_MIN, INT_MAX, VE, "profile" }, + { "main", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_PROFILE_MAIN }, INT_MIN, INT_MAX, VE, "profile" }, + { "rc_mode", "Rate control mode", OFFSET(rc_mode), AV_OPT_TYPE_INT, { .i64 = XEVE_RC_CQP }, XEVE_RC_CQP, XEVE_RC_CRF, VE, "rc_mode" }, + { "CQP", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_RC_CQP }, INT_MIN, INT_MAX, VE, "rc_mode" }, + { "ABR", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_RC_ABR }, INT_MIN, INT_MAX, VE, "rc_mode" }, + { "CRF", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_RC_CRF }, INT_MIN, INT_MAX, VE, "rc_mode" }, + { "qp", "Quantization parameter value for CQP rate control mode", OFFSET(qp), AV_OPT_TYPE_INT, { .i64 = 32 }, 0, 51, VE }, + { "crf", "Constant rate factor value for CRF rate control mode", OFFSET(crf), AV_OPT_TYPE_INT, { .i64 = 32 }, 10, 49, VE }, + { "hash", "Embed picture signature (HASH) for conformance checking in decoding", OFFSET(hash), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, 1, VE }, + { "sei_info", "Embed SEI messages identifying encoder parameters and command line arguments", OFFSET(sei_info), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, 1, VE }, + { NULL } +}; + +static const AVClass libxeve_class = { + .class_name = "libxeve", + .item_name = av_default_item_name, + .option = libxeve_options, + .version = LIBAVUTIL_VERSION_INT, +}; + +/** + * libavcodec generic global options, which can be set on all the encoders and decoders + * @see https://www.ffmpeg.org/ffmpeg-codecs.html#Codec-Options + */ +static const FFCodecDefault libxeve_defaults[] = { + { "b", "0" }, // bitrate in terms of kilo-bits per second + { "g", "0" }, // gop_size (key-frame interval 0: only one I-frame at the first time; 1: every frame is coded in I-frame) + { "bf", "15"}, // maximum number of B frames (0: no B-frames, 1,3,7,15) + { "threads", "0"}, // number of threads to be used (0: automatically select the number of threads to set) + { NULL }, +}; + +const FFCodec ff_libxeve_encoder = { + .p.name = "libxeve", + .p.long_name = NULL_IF_CONFIG_SMALL("libxeve MPEG-5 EVC"), + .p.type = AVMEDIA_TYPE_VIDEO, + .p.id = AV_CODEC_ID_EVC, + .init = libxeve_init, + FF_CODEC_ENCODE_CB(libxeve_encode), + .close = libxeve_close, + .priv_data_size = sizeof(XeveContext), + .p.priv_class = &libxeve_class, + .defaults = libxeve_defaults, + .p.capabilities = AV_CODEC_CAP_DELAY | AV_CODEC_CAP_OTHER_THREADS | AV_CODEC_CAP_DR1, + .p.wrapper_name = "libxeve", + .p.pix_fmts = supported_pixel_formats, + .caps_internal = FF_CODEC_CAP_INIT_CLEANUP | FF_CODEC_CAP_NOT_INIT_THREADSAFE, +}; diff --git a/libavcodec/parsers.c b/libavcodec/parsers.c index a8d52af6cb..ca00c63176 100644 --- a/libavcodec/parsers.c +++ b/libavcodec/parsers.c @@ -41,6 +41,7 @@ extern const AVCodecParser ff_dvaudio_parser; extern const AVCodecParser ff_dvbsub_parser; extern const AVCodecParser ff_dvdsub_parser; extern const AVCodecParser ff_dvd_nav_parser; +extern const AVCodecParser ff_evc_parser; extern const AVCodecParser ff_flac_parser; extern const AVCodecParser ff_g723_1_parser; extern const AVCodecParser ff_g729_parser; diff --git a/libavcodec/profiles.c b/libavcodec/profiles.c index 7af7fbeb13..a31244e0db 100644 --- a/libavcodec/profiles.c +++ b/libavcodec/profiles.c @@ -181,4 +181,10 @@ const AVProfile ff_arib_caption_profiles[] = { { FF_PROFILE_UNKNOWN } }; +const AVProfile ff_evc_profiles[] = { + { FF_PROFILE_EVC_BASELINE, "Baseline" }, + { FF_PROFILE_EVC_MAIN, "Main" }, + { FF_PROFILE_UNKNOWN }, +}; + #endif /* !CONFIG_SMALL */ diff --git a/libavcodec/profiles.h b/libavcodec/profiles.h index 41a19aa9ad..cf92b5f126 100644 --- a/libavcodec/profiles.h +++ b/libavcodec/profiles.h @@ -72,5 +72,6 @@ extern const AVProfile ff_sbc_profiles[]; extern const AVProfile ff_prores_profiles[]; extern const AVProfile ff_mjpeg_profiles[]; extern const AVProfile ff_arib_caption_profiles[]; +extern const AVProfile ff_evc_profiles[]; #endif /* AVCODEC_PROFILES_H */ diff --git a/libavcodec/version.h b/libavcodec/version.h index 777be76edf..7e89daf017 100644 --- a/libavcodec/version.h +++ b/libavcodec/version.h @@ -29,7 +29,7 @@ #include "version_major.h" -#define LIBAVCODEC_VERSION_MINOR 42 +#define LIBAVCODEC_VERSION_MINOR 43 #define LIBAVCODEC_VERSION_MICRO 101 #define LIBAVCODEC_VERSION_INT AV_VERSION_INT(LIBAVCODEC_VERSION_MAJOR, \ -- 2.17.1 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 13+ messages in thread
[parent not found: <CGME20220811123643eucas1p116b41dea8d7d8e0fd1ebc79414364f8c@eucas1p1.samsung.com>]
* [FFmpeg-devel] [PATCH 1/2] Provided support for MPEG-5 EVC (Essential Video Coding) codec [not found] <CGME20220811123643eucas1p116b41dea8d7d8e0fd1ebc79414364f8c@eucas1p1.samsung.com> @ 2022-08-11 12:36 ` Dawid Kozinski 0 siblings, 0 replies; 13+ messages in thread From: Dawid Kozinski @ 2022-08-11 12:36 UTC (permalink / raw) To: ffmpeg-devel - Added new entry to codec IDs list - Added new entry to the codec descriptor list - Bumped libavcodec minor version - Changes in Changelog and MAINTAINERS files - Added xeve encoder wrapper - Added xevd dencoder wrapper - Added documentation for xeve and xevd wrappers - Added parser for EVC format - Changes in project configuration file and libavcodec Makefile Signed-off-by: Dawid Kozinski <d.kozinski@samsung.com> --- Changelog | 3 +- MAINTAINERS | 2 + configure | 8 + doc/decoders.texi | 24 ++ doc/encoders.texi | 92 +++++ doc/general_contents.texi | 19 + doc/muxers.texi | 6 + libavcodec/Makefile | 3 + libavcodec/allcodecs.c | 2 + libavcodec/avcodec.h | 3 + libavcodec/codec_desc.c | 8 + libavcodec/codec_id.h | 1 + libavcodec/evc_parser.c | 758 ++++++++++++++++++++++++++++++++++++++ libavcodec/libxevd.c | 420 +++++++++++++++++++++ libavcodec/libxeve.c | 601 ++++++++++++++++++++++++++++++ libavcodec/parsers.c | 1 + libavcodec/profiles.c | 6 + libavcodec/profiles.h | 1 + libavcodec/version.h | 2 +- libavformat/Makefile | 4 +- libavformat/allformats.c | 2 + libavformat/isom_tags.c | 2 + libavformat/mov.c | 2 +- libavformat/movenc.c | 35 +- libavformat/rawenc.c | 13 + 25 files changed, 2013 insertions(+), 5 deletions(-) create mode 100644 libavcodec/evc_parser.c create mode 100644 libavcodec/libxevd.c create mode 100644 libavcodec/libxeve.c diff --git a/Changelog b/Changelog index fa83786a20..f41e260247 100644 --- a/Changelog +++ b/Changelog @@ -82,7 +82,8 @@ version 5.0: - VideoToolbox ProRes encoder - anlmf audio filter - IMF demuxer (experimental) - +- eXtra-fast Essential Video Encoder (XEVE) +- eXtra-fast Essential Video Decoder (XEVD) version 4.4: - AudioToolbox output device diff --git a/MAINTAINERS b/MAINTAINERS index ed2ec0b90c..03c052e0bd 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -206,6 +206,7 @@ Codecs: libvpx* James Zern libxavs.c Stefan Gehrer libxavs2.c Huiwen Ren + libxev*.c, evc_parser.c Dawid Kozinski libzvbi-teletextdec.c Marton Balint lzo.h, lzo.c Reimar Doeffinger mdec.c Michael Niedermayer @@ -426,6 +427,7 @@ Muxers/Demuxers: dv.c Roman Shaposhnik electronicarts.c Peter Ross epafdec.c Paul B Mahol + evcdec.c Dawid Kozinski ffm* Baptiste Coudurier flic.c Mike Melanson flvdec.c Michael Niedermayer diff --git a/configure b/configure index cbbb4dd9c8..e6582eae91 100755 --- a/configure +++ b/configure @@ -291,6 +291,8 @@ External library support: --enable-libwebp enable WebP encoding via libwebp [no] --enable-libx264 enable H.264 encoding via x264 [no] --enable-libx265 enable HEVC encoding via x265 [no] + --enable-libxeve enable EVC encoding via libxeve [no] + --enable-libxevd enable EVC decoding via libxevd [no] --enable-libxavs enable AVS encoding via xavs [no] --enable-libxavs2 enable AVS2 encoding via xavs2 [no] --enable-libxcb enable X11 grabbing using XCB [autodetect] @@ -1874,6 +1876,8 @@ EXTERNAL_LIBRARY_LIST=" libvorbis libvpx libwebp + libxevd + libxeve libxml2 libzimg libzmq @@ -3395,6 +3399,8 @@ libx264rgb_encoder_select="libx264_encoder" libx265_encoder_deps="libx265" libxavs_encoder_deps="libxavs" libxavs2_encoder_deps="libxavs2" +libxevd_decoder_deps="libxevd" +libxeve_encoder_deps="libxeve" libxvid_encoder_deps="libxvid" libzvbi_teletext_decoder_deps="libzvbi" vapoursynth_demuxer_deps="vapoursynth" @@ -6685,6 +6691,8 @@ enabled libx265 && require_pkg_config libx265 x265 x265.h x265_api_get require_cpp_condition libx265 x265.h "X265_BUILD >= 70" enabled libxavs && require libxavs "stdint.h xavs.h" xavs_encoder_encode "-lxavs $pthreads_extralibs $libm_extralibs" enabled libxavs2 && require_pkg_config libxavs2 "xavs2 >= 1.3.0" "stdint.h xavs2.h" xavs2_api_get +enabled libxevd && require_pkg_config libxevd "xevd >= 0.4.0" "xevd.h" xevd_decode +enabled libxeve && require_pkg_config libxeve "xeve >= 0.4.0" "xeve.h" xeve_encode enabled libxvid && require libxvid xvid.h xvid_global -lxvidcore enabled libzimg && require_pkg_config libzimg "zimg >= 2.7.0" zimg.h zimg_get_api_version enabled libzmq && require_pkg_config libzmq "libzmq >= 4.2.1" zmq.h zmq_ctx_new diff --git a/doc/decoders.texi b/doc/decoders.texi index e2fcbf5dc9..0bd9096114 100644 --- a/doc/decoders.texi +++ b/doc/decoders.texi @@ -126,6 +126,30 @@ Set amount of frame threads to use during decoding. The default value is 0 (auto @end table +@section libxevd + +eXtra-fast Essential Video Decoder (XEVD) MPEG-5 EVC decoder wrapper. + +This decoder requires the presence of the libxevd headers and library +during configuration. You need to explicitly configure the build with +@option{--enable-libxevd}. + +The xevd project website is at @url{https://github.com/mpeg5/xevd}. + +@subsection Options + +The following options are supported by the libxevd wrapper. +The xevd-equivalent options or values are listed in parentheses for easy migration. + +To get a more accurate and extensive documentation of the libxevd options, +invoke the command @code{xevd_app --help} or consult the libxevd documentation. + +@table @option +@item threads (@emph{threads}) +Force to use a specific number of threads + +@end table + @section QSV Decoders The family of Intel QuickSync Video decoders (VC1, MPEG-2, H.264, HEVC, diff --git a/doc/encoders.texi b/doc/encoders.texi index 6d73f74196..76f55e6721 100644 --- a/doc/encoders.texi +++ b/doc/encoders.texi @@ -2892,6 +2892,98 @@ ffmpeg -i input -c:v libxavs2 -xavs2-params RdoqLevel=0 output.avs2 @end example @end table +@section libxeve + +eXtra-fast Essential Video Encoder (XEVE) MPEG-5 EVC encoder wrapper. +The xeve-equivalent options or values are listed in parentheses for easy migration. + +This encoder requires the presence of the libxeve headers and library +during configuration. You need to explicitly configure the build with +@option{--enable-libxeve}. + +@float NOTE +Many libxeve encoder options are mapped to FFmpeg global codec options, +while unique encoder options are provided through private options. +Additionally the xeve-params private options allows one to pass a list +of key=value tuples as accepted by the libxeve @code{parse_xeve_params} function. +@end float + +The xeve project website is at @url{https://github.com/mpeg5/xeve}. + +@subsection Options + +The following options are supported by the libxeve wrapper. +The xeve-equivalent options or values are listed in parentheses for easy migration. + +@float NOTE +To reduce the duplication of documentation, only the private options +and some others requiring special attention are documented here. For +the documentation of the undocumented generic options, see +@ref{codec-options,,the Codec Options chapter}. +@end float + +@float NOTE +To get a more accurate and extensive documentation of the libxeve options, +invoke the command @code{xeve_app --help} or consult the libxeve documentation. +@end float + +@table @option +@item b (@emph{bitrate}) +Set target video bitrate in bits/s. +Note that FFmpeg's b option is expressed in bits/s, while xeve's bitrate is in kilobits/s. + +@item bf (@emph{bframes}) +Set the maximum number of B frames (1,3,7,15). + +@item g (@emph{keyint}) +Set the GOP size (I-picture period). + +@item preset (@emph{preset}) +Set the xeve preset. +Set the encoder preset value to determine encoding speed [fast, medium, slow, placebo] + +@item tune (@emph{tune}) +Set the encoder tune parameter [psnr, zerolatency] + +@item profile (@emph{profile}) +Set the encoder profile [0: baselie; 1: main] + +@item crf (@emph{crf}) +Set the quality for constant quality mode. +Constant rate factor <10..49> [default: 32] + +@item qp (@emph{qp}) +Set constant quantization rate control method parameter. +Quantization parameter qp <0..51> [default: 32] + +@item threads (@emph{threads}) +Force to use a specific number of threads + +@item xeve-params +Set xeve options using a list of @var{key}=@var{value} couples separated +by ":". + +For example to specify libxeve encoding options with @option{-xeve-params}: + +@example +ffmpeg -i input -c:v libxeve -xeve-params profile=main:tune=zerolatency:preset=fast:q=17:bframes=15:bitrate=100k output.mp4 +@end example +@end table + +@subsubsection Supported key values for xeve-params + +Set xeve options using a list of @var{key}=@var{value} couples separated + +@table @option + +@item hash +Embed the picture signature (HASH) for conformance checking in decoding, (0: Off, 1: On) + +@item vbv-bufsize +Set the VBV buffer size: Kbits(none,K,k), Mbits(M,m) + +@end table + @section libxvid Xvid MPEG-4 Part 2 encoder wrapper. diff --git a/doc/general_contents.texi b/doc/general_contents.texi index 86ec6d606b..89236047ea 100644 --- a/doc/general_contents.texi +++ b/doc/general_contents.texi @@ -343,6 +343,22 @@ libxavs2 is under the GNU Public License Version 2 or later details), you must upgrade FFmpeg's license to GPL in order to use it. @end float +@section eXtra-fast Essential Video Encoder (XEVE) + +FFmpeg can make use of the XEVE library for EVC video encoding. + +Go to @url{https://github.com/mpeg5/xeve} and follow the instructions for +installing the XEVE library. Then pass @code{--enable-libxeve} to configure to +enable it. + +@section eXtra-fast Essential Video Decoder (XEVD) + +FFmpeg can make use of the XEVD library for EVC video decoding. + +Go to @url{https://github.com/mpeg5/xevd} and follow the instructions for +installing the XEVD library. Then pass @code{--enable-libxevd} to configure to +enable it. + @section ZVBI ZVBI is a VBI decoding library which can be used by FFmpeg to decode DVB @@ -591,6 +607,7 @@ library: @item raw DTS @tab X @tab X @item raw DTS-HD @tab @tab X @item raw E-AC-3 @tab X @tab X +@item raw EVC @tab X @tab X @item raw FLAC @tab X @tab X @item raw GSM @tab @tab X @item raw H.261 @tab X @tab X @@ -930,6 +947,8 @@ following image formats are supported: @item Electronic Arts TQI video @tab @tab X @item Escape 124 @tab @tab X @item Escape 130 @tab @tab X +@item EVC / MPEG-5 Part 1 @tab X @tab X + @tab encoding and decoding supported through external libraries libxeve and libxevd @item FFmpeg video codec #1 @tab X @tab X @tab lossless codec (fourcc: FFV1) @item Flash Screen Video v1 @tab X @tab X diff --git a/doc/muxers.texi b/doc/muxers.texi index b2f4326aae..08ab20c09e 100644 --- a/doc/muxers.texi +++ b/doc/muxers.texi @@ -2124,6 +2124,12 @@ DTS Coherent Acoustics (DCA) audio. Dolby Digital Plus, also known as Enhanced AC-3, audio. +@subsection evc + +MPEG-5 Essential Video Coding (EVC) / EVC / MPEG-5 Part 1 EVC video. + +Extensions: evc + @subsection g722 ITU-T G.722 audio. diff --git a/libavcodec/Makefile b/libavcodec/Makefile index 83901bb52b..cfeacd96d1 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -1102,6 +1102,8 @@ OBJS-$(CONFIG_LIBX264_ENCODER) += libx264.o OBJS-$(CONFIG_LIBX265_ENCODER) += libx265.o OBJS-$(CONFIG_LIBXAVS_ENCODER) += libxavs.o OBJS-$(CONFIG_LIBXAVS2_ENCODER) += libxavs2.o +OBJS-$(CONFIG_LIBXEVD_DECODER) += libxevd.o +OBJS-$(CONFIG_LIBXEVE_ENCODER) += libxeve.o OBJS-$(CONFIG_LIBXVID_ENCODER) += libxvid.o OBJS-$(CONFIG_LIBZVBI_TELETEXT_DECODER) += libzvbi-teletextdec.o ass.o @@ -1129,6 +1131,7 @@ OBJS-$(CONFIG_DVAUDIO_PARSER) += dvaudio_parser.o OBJS-$(CONFIG_DVBSUB_PARSER) += dvbsub_parser.o OBJS-$(CONFIG_DVD_NAV_PARSER) += dvd_nav_parser.o OBJS-$(CONFIG_DVDSUB_PARSER) += dvdsub_parser.o +OBJS-$(CONFIG_EVC_PARSER) += evc_parser.o OBJS-$(CONFIG_FLAC_PARSER) += flac_parser.o flacdata.o flac.o OBJS-$(CONFIG_G723_1_PARSER) += g723_1_parser.o OBJS-$(CONFIG_G729_PARSER) += g729_parser.o diff --git a/libavcodec/allcodecs.c b/libavcodec/allcodecs.c index c94e2d5966..2526e11f70 100644 --- a/libavcodec/allcodecs.c +++ b/libavcodec/allcodecs.c @@ -800,6 +800,8 @@ extern LIBX264_CONST FFCodec ff_libx264_encoder; #endif extern const FFCodec ff_libx264rgb_encoder; extern FFCodec ff_libx265_encoder; +extern const FFCodec ff_libxeve_encoder; +extern const FFCodec ff_libxevd_decoder; extern const FFCodec ff_libxavs_encoder; extern const FFCodec ff_libxavs2_encoder; extern const FFCodec ff_libxvid_encoder; diff --git a/libavcodec/avcodec.h b/libavcodec/avcodec.h index 65c8535359..7d56a03fca 100644 --- a/libavcodec/avcodec.h +++ b/libavcodec/avcodec.h @@ -1674,6 +1674,9 @@ typedef struct AVCodecContext { #define FF_PROFILE_KLVA_SYNC 0 #define FF_PROFILE_KLVA_ASYNC 1 +#define FF_PROFILE_EVC_BASELINE 0 +#define FF_PROFILE_EVC_MAIN 1 + /** * level * - encoding: Set by user. diff --git a/libavcodec/codec_desc.c b/libavcodec/codec_desc.c index c1a177c22d..1a43ff8cb6 100644 --- a/libavcodec/codec_desc.c +++ b/libavcodec/codec_desc.c @@ -1907,6 +1907,14 @@ static const AVCodecDescriptor codec_descriptors[] = { .long_name = NULL_IF_CONFIG_SMALL("WBMP (Wireless Application Protocol Bitmap) image"), .props = AV_CODEC_PROP_INTRA_ONLY | AV_CODEC_PROP_LOSSLESS, }, + { + .id = AV_CODEC_ID_EVC, + .type = AVMEDIA_TYPE_VIDEO, + .name = "evc", + .long_name = NULL_IF_CONFIG_SMALL("MPEG-5 EVC (Essential Video Coding)"), + .props = AV_CODEC_PROP_LOSSY | AV_CODEC_PROP_REORDER, + .profiles = NULL_IF_CONFIG_SMALL(ff_evc_profiles), + }, /* various PCM "codecs" */ { diff --git a/libavcodec/codec_id.h b/libavcodec/codec_id.h index 386a00a7ef..02b60be690 100644 --- a/libavcodec/codec_id.h +++ b/libavcodec/codec_id.h @@ -314,6 +314,7 @@ enum AVCodecID { AV_CODEC_ID_PHM, AV_CODEC_ID_RADIANCE_HDR, AV_CODEC_ID_WBMP, + AV_CODEC_ID_EVC, /* various PCM "codecs" */ AV_CODEC_ID_FIRST_AUDIO = 0x10000, ///< A dummy id pointing at the start of audio codecs diff --git a/libavcodec/evc_parser.c b/libavcodec/evc_parser.c new file mode 100644 index 0000000000..0ae0eaa08a --- /dev/null +++ b/libavcodec/evc_parser.c @@ -0,0 +1,758 @@ +/* + * EVC format parser + * + * Copyright (C) 2021 Dawid Kozinski <d.kozinski@samsung.com> + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include <stdint.h> + +#include "libavutil/common.h" + +#include "parser.h" +#include "golomb.h" + +// The length field that indicates the length in bytes of the following NAL unit is configured to be of 4 bytes +#define EVC_NAL_UNIT_LENGTH_BYTE (4) /* byte */ + +#define EVC_NAL_HEADER_SIZE (2) /* byte */ +#define MAX_SPS_CNT (16) /* defined value in EVC standard */ +#define MAX_PPS_CNT (64) /* defined value in EVC standard */ + +#define MAX_NUM_TILES_ROW (22) +#define MAX_NUM_TILES_COL (20) + +// NALU types +// @see ISO_IEC_23094-1_2020 7.4.2.2 NAL unit header semantics +// +#define EVC_NUT_NONIDR (0) /* Coded slice of a non-IDR picture */ +#define EVC_NUT_IDR (1) /* Coded slice of an IDR picture */ +#define EVC_NUT_SPS (24) /* Sequence parameter set */ +#define EVC_NUT_PPS (25) /* Picture paremeter set */ +#define EVC_NUT_APS (26) /* Adaptation parameter set */ +#define EVC_NUT_FD (27) /* Filler data */ +#define EVC_NUT_SEI (28) /* Supplemental enhancement information */ + +// slice type +// @see ISO_IEC_23094-1_2020 7.4.5 Slice header semantics +// +#define SLICE_TYPE_UNKNOWN (-1) +#define SLICE_TYPE_B (0) +#define SLICE_TYPE_P (1) +#define SLICE_TYPE_I (2) + +#define MAX_NUM_RPLS (32) +#define MAX_NUM_REF_PICS (21) + +// rpl structure +typedef struct RefPicListStruct { + int poc; + int tid; + int ref_pic_num; + int ref_pic_active_num; + int ref_pics[MAX_NUM_REF_PICS]; + char pic_type; + +} RefPicListStruct; + +// The sturcture reflects SPS RBSP(raw byte sequence payload) layout +// @see ISO_IEC_23094-1 section 7.3.2.1 +// +// The following descriptors specify the parsing process of each element +// u(n) - unsigned integer using n bits +// ue(v) - unsigned integer 0-th order Exp_Golomb-coded syntax element with the left bit first +typedef struct EVCParserSPS { + int sps_seq_parameter_set_id; // ue(v) + int profile_idc; // u(8) + int level_idc; // u(8) + int toolset_idc_h; // u(32) + int toolset_idc_l; // u(32) + int chroma_format_idc; // ue(v) + int pic_width_in_luma_samples; // ue(v) + int pic_height_in_luma_samples; // ue(v) + int bit_depth_luma_minus8; // ue(v) + int bit_depth_chroma_minus8; // ue(v) + + int sps_btt_flag; // u(1) + int log2_ctu_size_minus5; // ue(v) + int log2_min_cb_size_minus2; // ue(v) + int log2_diff_ctu_max_14_cb_size; // ue(v) + int log2_diff_ctu_max_tt_cb_size; // ue(v) + int log2_diff_min_cb_min_tt_cb_size_minus2; // ue(v) + + int sps_suco_flag; // u(1) + int log2_diff_ctu_size_max_suco_cb_size; // ue(v) + int log2_diff_max_suco_min_suco_cb_size; // ue(v) + + int sps_admvp_flag; // u(1) + int sps_affine_flag; // u(1) + int sps_amvr_flag; // u(1) + int sps_dmvr_flag; // u(1) + int sps_mmvd_flag; // u(1) + int sps_hmvp_flag; // u(1) + + int sps_eipd_flag; // u(1) + int sps_ibc_flag; // u(1) + int log2_max_ibc_cand_size_minus2; // ue(v) + + int sps_cm_init_flag; // u(1) + int sps_adcc_flag; // u(1) + + int sps_iqt_flag; // u(1) + int sps_ats_flag; // u(1) + + int sps_addb_flag; // u(1) + int sps_alf_flag; // u(1) + int sps_htdf_flag; // u(1) + int sps_rpl_flag; // u(1) + int sps_pocs_flag; // u(1) + int sps_dquant_flag; // u(1) + int sps_dra_flag; // u(1) + + int log2_max_pic_order_cnt_lsb_minus4; // ue(v) + int log2_sub_gop_length; // ue(v) + int log2_ref_pic_gap_length; // ue(v) + + int max_num_tid0_ref_pics; // ue(v) + + int sps_max_dec_pic_buffering_minus1; // ue(v) + int long_term_ref_pic_flag; // u(1) + int rpl1_same_as_rpl0_flag; // u(1) + int num_ref_pic_list_in_sps[2]; // ue(v) + RefPicListStruct rpls[2][MAX_NUM_RPLS]; + + int picture_cropping_flag; // u(1) + int picture_crop_left_offset; // ue(v) + int picture_crop_right_offset; // ue(v) + int picture_crop_top_offset; // ue(v) + int picture_crop_bottom_offset; // ue(v) + + // @note + // Currently the structure does not reflect the entire SPS RBSP layout. + // It contains only the fields that are necessary to read from the NAL unit all the values + // necessary for the correct initialization of the AVCodecContext structure. + + // @note + // If necessary, add the missing fields to the structure to reflect + // the contents of the entire NAL unit of the SPS type + +} EVCParserSPS; + +typedef struct EVCParserPPS { + int pps_pic_parameter_set_id; // ue(v) + int pps_seq_parameter_set_id; // ue(v) + int num_ref_idx_default_active_minus1[2]; // ue(v) + int additional_lt_poc_lsb_len; // ue(v) + int rpl1_idx_present_flag; // u(1) + int single_tile_in_pic_flag; // u(1) + int num_tile_columns_minus1; // ue(v) + int num_tile_rows_minus1; // ue(v) + int uniform_tile_spacing_flag; // u(1) + int tile_column_width_minus1[MAX_NUM_TILES_ROW]; // ue(v) + int tile_row_height_minus1[MAX_NUM_TILES_COL]; // ue(v) + int loop_filter_across_tiles_enabled_flag; // u(1) + int tile_offset_len_minus1; // ue(v) + int tile_id_len_minus1; // ue(v) + int explicit_tile_id_flag; // u(1) + int tile_id_val[MAX_NUM_TILES_ROW][MAX_NUM_TILES_COL]; // u(v) + int pic_dra_enabled_flag; // u(1) + int pic_dra_aps_id; // u(5) + int arbitrary_slice_present_flag; // u(1) + int constrained_intra_pred_flag; // u(1) + int cu_qp_delta_enabled_flag; // u(1) + int log2_cu_qp_delta_area_minus6; // ue(v) + +} EVCParserPPS; + +// The sturcture reflects Slice Header RBSP(raw byte sequence payload) layout +// @see ISO_IEC_23094-1 section 7.3.2.6 +// +// The following descriptors specify the parsing process of each element +// u(n) - unsigned integer using n bits +// ue(v) - unsigned integer 0-th order Exp_Golomb-coded syntax element with the left bit first +// u(n) - unsigned integer using n bits. +// When n is "v" in the syntax table, the number of bits varies in a manner dependent on the value of other syntax elements. +typedef struct EVCParserSliceHeader { + int slice_pic_parameter_set_id; // ue(v) + int single_tile_in_slice_flag; // u(1) + int first_tile_id; // u(v) + int arbitrary_slice_flag; // u(1) + int last_tile_id; // u(v) + int num_remaining_tiles_in_slice_minus1; // ue(v) + int delta_tile_id_minus1[MAX_NUM_TILES_ROW * MAX_NUM_TILES_COL]; // ue(v) + + int slice_type; // ue(v) + + // @note + // Currently the structure does not reflect the entire Slice Header RBSP layout. + // It contains only the fields that are necessary to read from the NAL unit all the values + // necessary for the correct initialization of the AVCodecContext structure. + + // @note + // If necessary, add the missing fields to the structure to reflect + // the contents of the entire NAL unit of the SPS type + +} EVCParserSliceHeader; + +typedef struct EVCParserContext { + ParseContext pc; + EVCParserSPS sps[MAX_SPS_CNT]; + EVCParserPPS pps[MAX_PPS_CNT]; + EVCParserSliceHeader slice_header[MAX_PPS_CNT]; + int is_avc; + int nal_length_size; + int to_read; + int incomplete_nalu_prefix_read; // The flag is set to 1 when an incomplete NAL unit prefix has been read + +} EVCParserContext; + +static int get_nalu_type(const uint8_t *bits, int bits_size, AVCodecContext *avctx) +{ + int unit_type_plus1 = 0; + + if (bits_size >= EVC_NAL_HEADER_SIZE) { + unsigned char *p = (unsigned char *)bits; + // forbidden_zero_bit + if ((p[0] & 0x80) != 0) + return -1; + + // nal_unit_type + unit_type_plus1 = (p[0] >> 1) & 0x3F; + } + + return unit_type_plus1 - 1; +} + +static uint32_t read_nal_unit_length(const uint8_t *bits, int bits_size, AVCodecContext *avctx) +{ + uint32_t nalu_len = 0; + + if (bits_size >= EVC_NAL_UNIT_LENGTH_BYTE) { + + int t = 0; + unsigned char *p = (unsigned char *)bits; + + for (int i = 0; i < EVC_NAL_UNIT_LENGTH_BYTE; i++) + t = (t << 8) | p[i]; + + nalu_len = t; + if (nalu_len == 0) + return 0; + } + + return nalu_len; +} + +// @see ISO_IEC_23094-1 (7.3.2.1 SPS RBSP syntax) +static EVCParserSPS *parse_sps(const uint8_t *bs, int bs_size, EVCParserContext *ev) +{ + GetBitContext gb; + EVCParserSPS *sps; + int sps_seq_parameter_set_id; + + if (init_get_bits8(&gb, bs, bs_size) < 0) + return NULL; + + sps_seq_parameter_set_id = get_ue_golomb(&gb); + + if (sps_seq_parameter_set_id >= MAX_SPS_CNT) + return NULL; + + sps = &ev->sps[sps_seq_parameter_set_id]; + sps->sps_seq_parameter_set_id = sps_seq_parameter_set_id; + + // the Baseline profile is indicated by profile_idc eqal to 0 + // the Main profile is indicated by profile_idc eqal to 1 + sps->profile_idc = get_bits(&gb, 8); + + sps->level_idc = get_bits(&gb, 8); + + skip_bits_long(&gb, 32); /* skip toolset_idc_h */ + skip_bits_long(&gb, 32); /* skip toolset_idc_l */ + + // 0 - monochrome + // 1 - 4:2:0 + // 2 - 4:2:2 + // 3 - 4:4:4 + sps->chroma_format_idc = get_ue_golomb(&gb); + + sps->pic_width_in_luma_samples = get_ue_golomb(&gb); + sps->pic_height_in_luma_samples = get_ue_golomb(&gb); + + sps->bit_depth_luma_minus8 = get_ue_golomb(&gb); + sps->bit_depth_chroma_minus8 = get_ue_golomb(&gb); + + sps->sps_btt_flag = get_bits(&gb, 1); + if (sps->sps_btt_flag) { + sps->log2_ctu_size_minus5 = get_ue_golomb(&gb); + sps->log2_min_cb_size_minus2 = get_ue_golomb(&gb); + sps->log2_diff_ctu_max_14_cb_size = get_ue_golomb(&gb); + sps->log2_diff_ctu_max_tt_cb_size = get_ue_golomb(&gb); + sps->log2_diff_min_cb_min_tt_cb_size_minus2 = get_ue_golomb(&gb); + } + + sps->sps_suco_flag = get_bits(&gb, 1); + if (sps->sps_suco_flag) { + sps->log2_diff_ctu_size_max_suco_cb_size = get_ue_golomb(&gb); + sps->log2_diff_max_suco_min_suco_cb_size = get_ue_golomb(&gb); + } + + sps->sps_admvp_flag = get_bits(&gb, 1); + if (sps->sps_admvp_flag) { + sps->sps_affine_flag = get_bits(&gb, 1); + sps->sps_amvr_flag = get_bits(&gb, 1); + sps->sps_dmvr_flag = get_bits(&gb, 1); + sps->sps_mmvd_flag = get_bits(&gb, 1); + sps->sps_hmvp_flag = get_bits(&gb, 1); + } + + sps->sps_eipd_flag = get_bits(&gb, 1); + if (sps->sps_eipd_flag) { + sps->sps_ibc_flag = get_bits(&gb, 1); + if (sps->sps_ibc_flag) + sps->log2_max_ibc_cand_size_minus2 = get_ue_golomb(&gb); + } + + sps->sps_cm_init_flag = get_bits(&gb, 1); + if (sps->sps_cm_init_flag) + sps->sps_adcc_flag = get_bits(&gb, 1); + + sps->sps_iqt_flag = get_bits(&gb, 1); + if (sps->sps_iqt_flag) + sps->sps_ats_flag = get_bits(&gb, 1); + + sps->sps_addb_flag = get_bits(&gb, 1); + sps->sps_alf_flag = get_bits(&gb, 1); + sps->sps_htdf_flag = get_bits(&gb, 1); + sps->sps_rpl_flag = get_bits(&gb, 1); + sps->sps_pocs_flag = get_bits(&gb, 1); + sps->sps_dquant_flag = get_bits(&gb, 1); + sps->sps_dra_flag = get_bits(&gb, 1); + + if (sps->sps_pocs_flag) + sps->log2_max_pic_order_cnt_lsb_minus4 = get_ue_golomb(&gb); + + if (!sps->sps_pocs_flag || !sps->sps_rpl_flag) { + sps->log2_sub_gop_length = get_ue_golomb(&gb); + if (sps->log2_sub_gop_length == 0) + sps->log2_ref_pic_gap_length = get_ue_golomb(&gb); + } + + // @note + // If necessary, add the missing fields to the EVCParserSPS structure + // and then extend parser implementation + + return sps; +} + +// @see ISO_IEC_23094-1 (7.3.2.2 SPS RBSP syntax) +static EVCParserPPS *parse_pps(const uint8_t *bs, int bs_size, EVCParserContext *ev) +{ + GetBitContext gb; + EVCParserPPS *pps; + + int pps_pic_parameter_set_id; + + if (init_get_bits8(&gb, bs, bs_size) < 0) + return NULL; + + pps_pic_parameter_set_id = get_ue_golomb(&gb); + if (pps_pic_parameter_set_id > MAX_PPS_CNT) + return NULL; + + pps = &ev->pps[pps_pic_parameter_set_id]; + + pps->pps_pic_parameter_set_id = pps_pic_parameter_set_id; + + pps->pps_seq_parameter_set_id = get_ue_golomb(&gb); + if (pps->pps_seq_parameter_set_id >= MAX_SPS_CNT) + return NULL; + + pps->num_ref_idx_default_active_minus1[0] = get_ue_golomb(&gb); + pps->num_ref_idx_default_active_minus1[1] = get_ue_golomb(&gb); + pps->additional_lt_poc_lsb_len = get_ue_golomb(&gb); + pps->rpl1_idx_present_flag = get_bits(&gb, 1); + pps->single_tile_in_pic_flag = get_bits(&gb, 1); + + if (!pps->single_tile_in_pic_flag) { + pps->num_tile_columns_minus1 = get_ue_golomb(&gb); + pps->num_tile_rows_minus1 = get_ue_golomb(&gb); + pps->uniform_tile_spacing_flag = get_bits(&gb, 1); + + if (!pps->uniform_tile_spacing_flag) { + for (int i = 0; i < pps->num_tile_columns_minus1; i++) + pps->tile_column_width_minus1[i] = get_ue_golomb(&gb); + + for (int i = 0; i < pps->num_tile_rows_minus1; i++) + pps->tile_row_height_minus1[i] = get_ue_golomb(&gb); + } + pps->loop_filter_across_tiles_enabled_flag = get_bits(&gb, 1); + pps->tile_offset_len_minus1 = get_ue_golomb(&gb); + } + + pps->tile_id_len_minus1 = get_ue_golomb(&gb); + pps->explicit_tile_id_flag = get_bits(&gb, 1); + + if (pps->explicit_tile_id_flag) { + for (int i = 0; i <= pps->num_tile_rows_minus1; i++) { + for (int j = 0; j <= pps->num_tile_columns_minus1; j++) + pps->tile_id_val[i][j] = get_bits(&gb, pps->tile_id_len_minus1 + 1); + } + } + + pps->pic_dra_enabled_flag = 0; + pps->pic_dra_enabled_flag = get_bits(&gb, 1); + + if (pps->pic_dra_enabled_flag) + pps->pic_dra_aps_id = get_bits(&gb, 5); + + pps->arbitrary_slice_present_flag = get_bits(&gb, 1); + pps->constrained_intra_pred_flag = get_bits(&gb, 1); + pps->cu_qp_delta_enabled_flag = get_bits(&gb, 1); + + if (pps->cu_qp_delta_enabled_flag) + pps->log2_cu_qp_delta_area_minus6 = get_ue_golomb(&gb); + + return pps; +} + +// @see ISO_IEC_23094-1 (7.3.2.6 Slice layer RBSP syntax) +static EVCParserSliceHeader *parse_slice_header(const uint8_t *bs, int bs_size, EVCParserContext *ev) +{ + GetBitContext gb; + EVCParserSliceHeader *sh; + EVCParserPPS *pps; + int num_tiles_in_slice = 0; + int slice_pic_parameter_set_id; + + if (init_get_bits8(&gb, bs, bs_size) < 0) + return NULL; + + slice_pic_parameter_set_id = get_ue_golomb(&gb); + + if (slice_pic_parameter_set_id < 0 || slice_pic_parameter_set_id >= MAX_PPS_CNT) + return NULL; + + sh = &ev->slice_header[slice_pic_parameter_set_id]; + pps = &ev->pps[slice_pic_parameter_set_id]; + + sh->slice_pic_parameter_set_id = slice_pic_parameter_set_id; + + if (!pps->single_tile_in_pic_flag) { + sh->single_tile_in_slice_flag = get_bits(&gb, 1); + sh->first_tile_id = get_bits(&gb, pps->tile_id_len_minus1 + 1); + } else + sh->single_tile_in_slice_flag = 1; + + if (!sh->single_tile_in_slice_flag) { + if (pps->arbitrary_slice_present_flag) + sh->arbitrary_slice_flag = get_bits(&gb, 1); + + if (!sh->arbitrary_slice_flag) + sh->last_tile_id = get_bits(&gb, pps->tile_id_len_minus1 + 1); + else { + sh->num_remaining_tiles_in_slice_minus1 = get_ue_golomb(&gb); + num_tiles_in_slice = sh->num_remaining_tiles_in_slice_minus1 + 2; + for (int i = 0; i < num_tiles_in_slice - 1; ++i) + sh->delta_tile_id_minus1[i] = get_ue_golomb(&gb); + } + } + + sh->slice_type = get_ue_golomb(&gb); + + // @note + // If necessary, add the missing fields to the EVCParserSliceHeader structure + // and then extend parser implementation + + return sh; +} + +static int parse_nal_units(AVCodecParserContext *s, const uint8_t *bs, + int bs_size, AVCodecContext *avctx) +{ + EVCParserContext *ev = s->priv_data; + int nalu_type, nalu_size; + unsigned char *bits = (unsigned char *)bs; + int bits_size = bs_size; + + avctx->codec_id = AV_CODEC_ID_EVC; + s->pict_type = AV_PICTURE_TYPE_NONE; + s->picture_structure = AV_PICTURE_STRUCTURE_FRAME; + s->key_frame = -1; + + nalu_size = read_nal_unit_length(bits, bits_size, avctx); + if (nalu_size == 0) { + av_log(avctx, AV_LOG_ERROR, "Invalid NAL unit size: (%d)\n", nalu_size); + return -1; + } + + bits += EVC_NAL_UNIT_LENGTH_BYTE; + bits_size -= EVC_NAL_UNIT_LENGTH_BYTE; + + nalu_type = get_nalu_type(bits, bits_size, avctx); + + bits += EVC_NAL_HEADER_SIZE; + bits_size -= EVC_NAL_HEADER_SIZE; + + if (nalu_type == EVC_NUT_SPS) { // NAL Unit type: SPS (Sequence Parameter Set) + EVCParserSPS *sps; + + sps = parse_sps(bits, bits_size, ev); + if (!sps) { + av_log(avctx, AV_LOG_ERROR, "SPS parsing error\n"); + return -1; + } + + s->coded_width = sps->pic_width_in_luma_samples; + s->coded_height = sps->pic_height_in_luma_samples; + s->width = sps->pic_width_in_luma_samples; + s->height = sps->pic_height_in_luma_samples; + + if (sps->profile_idc == 1) avctx->profile = FF_PROFILE_EVC_MAIN; + else avctx->profile = FF_PROFILE_EVC_BASELINE; + + // Currently XEVD decoder supports ony YCBCR420_10LE chroma format for EVC stream + switch (sps->chroma_format_idc) { + case 0: /* YCBCR400_10LE */ + av_log(avctx, AV_LOG_ERROR, "YCBCR400_10LE: Not supported chroma format\n"); + s->format = AV_PIX_FMT_GRAY10LE; + return -1; + case 1: /* YCBCR420_10LE */ + s->format = AV_PIX_FMT_YUV420P10LE; + break; + case 2: /* YCBCR422_10LE */ + av_log(avctx, AV_LOG_ERROR, "YCBCR422_10LE: Not supported chroma format\n"); + s->format = AV_PIX_FMT_YUV422P10LE; + return -1; + case 3: /* YCBCR444_10LE */ + av_log(avctx, AV_LOG_ERROR, "YCBCR444_10LE: Not supported chroma format\n"); + s->format = AV_PIX_FMT_YUV444P10LE; + return -1; + default: + s->format = AV_PIX_FMT_NONE; + av_log(avctx, AV_LOG_ERROR, "Unknown supported chroma format\n"); + return -1; + } + + // @note + // The current implementation of parse_sps function doesn't handle VUI parameters parsing. + // If it will be needed, parse_sps function could be extended to handle VUI parameters parsing + // to initialize fields of the AVCodecContex i.e. color_primaries, color_trc,color_range + + } else if (nalu_type == EVC_NUT_PPS) { // NAL Unit type: PPS (Video Parameter Set) + EVCParserPPS *pps; + + pps = parse_pps(bits, bits_size, ev); + if (!pps) { + av_log(avctx, AV_LOG_ERROR, "PPS parsing error\n"); + return -1; + } + } else if (nalu_type == EVC_NUT_SEI) // NAL unit type: SEI (Supplemental Enhancement Information) + return 0; + else if (nalu_type == EVC_NUT_IDR || nalu_type == EVC_NUT_NONIDR) { // NAL Unit type: Coded slice of a IDR or non-IDR picture + EVCParserSliceHeader *sh; + + sh = parse_slice_header(bits, bits_size, ev); + if (!sh) { + av_log(avctx, AV_LOG_ERROR, "Slice header parsing error\n"); + return -1; + } + switch (sh->slice_type) { + case SLICE_TYPE_B: { + s->pict_type = AV_PICTURE_TYPE_B; + break; + } + case SLICE_TYPE_P: { + s->pict_type = AV_PICTURE_TYPE_P; + break; + } + case SLICE_TYPE_I: { + s->pict_type = AV_PICTURE_TYPE_I; + break; + } + default: { + s->pict_type = AV_PICTURE_TYPE_NONE; + } + } + s->key_frame = (nalu_type == EVC_NUT_IDR) ? 1 : 0; + } else { + av_log(avctx, AV_LOG_ERROR, "Invalid NAL unit type: %d\n", nalu_type); + return -1; + } + + return 0; +} + +/** + * Find the end of the current frame in the bitstream. + * @return the position of the first byte of the next frame, or END_NOT_FOUND + */ +static int evc_find_frame_end(AVCodecParserContext *s, const uint8_t *buf, + int buf_size, AVCodecContext *avctx) +{ + EVCParserContext *ev = s->priv_data; + + if (!ev->to_read) { + int nal_unit_size = 0; + int next = END_NOT_FOUND; + + // This is the case when buffer size is not enough for buffer to store NAL unit length + if (buf_size < EVC_NAL_UNIT_LENGTH_BYTE) { + ev->to_read = EVC_NAL_UNIT_LENGTH_BYTE; + ev->nal_length_size = buf_size; + ev->incomplete_nalu_prefix_read = 1; + + return END_NOT_FOUND; + } + + nal_unit_size = read_nal_unit_length(buf, buf_size, avctx); + ev->nal_length_size = EVC_NAL_UNIT_LENGTH_BYTE; + + next = nal_unit_size + EVC_NAL_UNIT_LENGTH_BYTE; + ev->to_read = next; + if (next < buf_size) + return next; + else + return END_NOT_FOUND; + } else if (ev->to_read > buf_size) + return END_NOT_FOUND; + else { + if (ev->incomplete_nalu_prefix_read == 1) { + EVCParserContext *ev = s->priv_data; + ParseContext *pc = &ev->pc; + uint8_t nalu_len[EVC_NAL_UNIT_LENGTH_BYTE] = {0}; + int nal_unit_size = 0; + + // 1. pc->buffer contains previously read bytes of NALU prefix + // 2. buf contains the rest of NAL unit prefix bytes + // + // ~~~~~~~ + // EXAMPLE + // ~~~~~~~ + // + // In the following example we assumed that the number of already read NAL Unit prefix bytes is equal 1 + // + // ---------- + // pc->buffer -> conatins already read bytes + // ---------- + // __ pc->index == 1 + // | + // V + // ------------------------------------------------------- + // | 0 | 1 | 2 | 3 | 4 | ... | N | + // ------------------------------------------------------- + // | 0x00 | 0xXX | 0xXX | 0xXX | 0xXX | ... | 0xXX | + // ------------------------------------------------------- + // + // ---------- + // buf -> contains newly read bytes + // ---------- + // ------------------------------------------------------- + // | 0 | 1 | 2 | 3 | 4 | ... | N | + // ------------------------------------------------------- + // | 0x00 | 0x00 | 0x3C | 0xXX | 0xXX | ... | 0xXX | + // ------------------------------------------------------- + // + for (int i = 0; i < EVC_NAL_UNIT_LENGTH_BYTE; i++) { + if (i < pc->index) + nalu_len[i] = pc->buffer[i]; + else + nalu_len[i] = buf[i - pc->index]; + } + + // ---------- + // nalu_len + // ---------- + // --------------------------------- + // | 0 | 1 | 2 | 3 | + // --------------------------------- + // | 0x00 | 0x00 | 0x00 | 0x3C | + // --------------------------------- + // | NALU LENGTH | + // --------------------------------- + // NAL Unit lenght = 60 (0x0000003C) + + nal_unit_size = read_nal_unit_length(nalu_len, EVC_NAL_UNIT_LENGTH_BYTE, avctx); + + ev->to_read = nal_unit_size + EVC_NAL_UNIT_LENGTH_BYTE - pc->index; + + ev->incomplete_nalu_prefix_read = 0; + + if (ev->to_read > buf_size) + return END_NOT_FOUND; + else + return ev->to_read; + } + return ev->to_read; + } + + return END_NOT_FOUND; +} + +static int evc_parser_init(AVCodecParserContext *s) +{ + EVCParserContext *ev = s->priv_data; + + ev->nal_length_size = EVC_NAL_UNIT_LENGTH_BYTE; + ev->incomplete_nalu_prefix_read = 0; + + return 0; +} + +static int evc_parse(AVCodecParserContext *s, AVCodecContext *avctx, + const uint8_t **poutbuf, int *poutbuf_size, + const uint8_t *buf, int buf_size) +{ + int next; + EVCParserContext *ev = s->priv_data; + ParseContext *pc = &ev->pc; + int is_dummy_buf = !buf_size; + const uint8_t *dummy_buf = buf; + + if (s->flags & PARSER_FLAG_COMPLETE_FRAMES) + next = buf_size; + else { + next = evc_find_frame_end(s, buf, buf_size, avctx); + if (ff_combine_frame(pc, next, &buf, &buf_size) < 0) { + *poutbuf = NULL; + *poutbuf_size = 0; + ev->to_read -= buf_size; + return buf_size; + } + } + + is_dummy_buf &= (dummy_buf == buf); + + if (!is_dummy_buf) + parse_nal_units(s, buf, buf_size, avctx); + + // poutbuf contains just one NAL unit + *poutbuf = buf; + *poutbuf_size = buf_size; + ev->to_read -= next; + + return next; +} + +const AVCodecParser ff_evc_parser = { + .codec_ids = { AV_CODEC_ID_EVC }, + .priv_data_size = sizeof(EVCParserContext), + .parser_init = evc_parser_init, + .parser_parse = evc_parse, + .parser_close = ff_parse_close, +}; diff --git a/libavcodec/libxevd.c b/libavcodec/libxevd.c new file mode 100644 index 0000000000..ea57be74f6 --- /dev/null +++ b/libavcodec/libxevd.c @@ -0,0 +1,420 @@ +/* + * libxevd decoder + * EVC (MPEG-5 Essential Video Coding) decoding using XEVD MPEG-5 EVC decoder library + * + * Copyright (C) 2021 Dawid Kozinski <d.kozinski@samsung.com> + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include <float.h> +#include <stdlib.h> + +#include <xevd.h> + +#include "libavutil/internal.h" +#include "libavutil/common.h" +#include "libavutil/opt.h" +#include "libavutil/pixdesc.h" +#include "libavutil/pixfmt.h" +#include "libavutil/imgutils.h" +#include "libavutil/cpu.h" + +#include "avcodec.h" +#include "internal.h" +#include "packet_internal.h" +#include "codec_internal.h" + +#define XEVD_PARAM_BAD_NAME -1 +#define XEVD_PARAM_BAD_VALUE -2 + +#define EVC_NAL_HEADER_SIZE 2 /* byte */ + +/** + * The structure stores all the states associated with the instance of Xeve MPEG-5 EVC decoder + */ +typedef struct XevdContext { + const AVClass *class; + + XEVD id; // XEVD instance identifier @see xevd.h + XEVD_CDSC cdsc; // decoding parameters @see xevd.h +} XevdContext; + +/** + * The function populates the XEVD_CDSC structure. + * XEVD_CDSC contains all decoder parameters that should be initialized before its use. + * + * @param[in] avctx codec context + * @param[out] cdsc contains all decoder parameters that should be initialized before its use + * + */ +static void get_conf(AVCodecContext *avctx, XEVD_CDSC *cdsc) +{ + int cpu_count = av_cpu_count(); + + /* clear XEVS_CDSC structure */ + memset(cdsc, 0, sizeof(XEVD_CDSC)); + + /* init XEVD_CDSC */ + if (avctx->thread_count <= 0) + cdsc->threads = (cpu_count < XEVD_MAX_TASK_CNT) ? cpu_count : XEVD_MAX_TASK_CNT; + else if (avctx->thread_count > XEVD_MAX_TASK_CNT) + cdsc->threads = XEVD_MAX_TASK_CNT; + else + cdsc->threads = avctx->thread_count; +} + +/** + * Read NAL unit length + * @param bs input data (bitstream) + * @return the length of NAL unit on success, 0 value on failure + */ +static uint32_t read_nal_unit_length(const uint8_t *bs, int bs_size, AVCodecContext *avctx) +{ + uint32_t len = 0; + XEVD_INFO info; + int ret; + + if (bs_size == XEVD_NAL_UNIT_LENGTH_BYTE) { + ret = xevd_info((void *)bs, XEVD_NAL_UNIT_LENGTH_BYTE, 1, &info); + if (XEVD_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Cannot get bitstream information\n"); + return 0; + } + len = info.nalu_len; + if (len == 0) { + av_log(avctx, AV_LOG_ERROR, "Invalid bitstream size! [%d]\n", bs_size); + return 0; + } + } + + return len; +} + +/** + * @param[in] xectx the structure that stores all the state associated with the instance of Xeve MPEG-5 EVC decoder + * @param[out] avctx codec context + * @return 0 on success, negative value on failure + */ +static int export_stream_params(const XevdContext *xectx, AVCodecContext *avctx) +{ + int ret; + int size; + int color_space; + + avctx->pix_fmt = AV_PIX_FMT_YUV420P10; + + size = 4; + ret = xevd_config(xectx->id, XEVD_CFG_GET_CODED_WIDTH, &avctx->coded_width, &size); + if (XEVD_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to get coded_width\n"); + return AVERROR_EXTERNAL; + } + + ret = xevd_config(xectx->id, XEVD_CFG_GET_CODED_HEIGHT, &avctx->coded_height, &size); + if (XEVD_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to get coded_height\n"); + return AVERROR_EXTERNAL; + } + + ret = xevd_config(xectx->id, XEVD_CFG_GET_WIDTH, &avctx->width, &size); + if (XEVD_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to get width\n"); + return AVERROR_EXTERNAL; + } + + ret = xevd_config(xectx->id, XEVD_CFG_GET_HEIGHT, &avctx->height, &size); + if (XEVD_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to get height\n"); + return AVERROR_EXTERNAL; + } + + ret = xevd_config(xectx->id, XEVD_CFG_GET_COLOR_SPACE, &color_space, &size); + if (XEVD_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to get color_space\n"); + return AVERROR_EXTERNAL; + } + switch(color_space) { + case XEVD_CS_YCBCR400_10LE: + avctx->pix_fmt = AV_PIX_FMT_GRAY10LE; + break; + case XEVD_CS_YCBCR420_10LE: + avctx->pix_fmt = AV_PIX_FMT_YUV420P10LE; + break; + case XEVD_CS_YCBCR422_10LE: + avctx->pix_fmt = AV_PIX_FMT_YUV422P10LE; + break; + case XEVD_CS_YCBCR444_10LE: + avctx->pix_fmt = AV_PIX_FMT_YUV444P10LE; + break; + default: + av_log(avctx, AV_LOG_ERROR, "Unknown color space\n"); + avctx->pix_fmt = AV_PIX_FMT_NONE; + return AVERROR_INVALIDDATA; + } + + // the function returns sps->num_reorder_pics + ret = xevd_config(xectx->id, XEVD_CFG_GET_MAX_CODING_DELAY, &avctx->max_b_frames, &size); + if (XEVD_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to get max_coding_delay\n"); + return AVERROR_EXTERNAL; + } + + avctx->has_b_frames = (avctx->max_b_frames) ? 1 : 0; + + avctx->color_primaries = AVCOL_PRI_UNSPECIFIED; + avctx->color_trc = AVCOL_TRC_UNSPECIFIED; + avctx->colorspace = AVCOL_SPC_UNSPECIFIED; + + return 0; +} + +/** + * @brief Copy image in imgb to frame. + * + * @param avctx codec context + * @param[in] imgb + * @param[out] frame + * @return 0 on success, negative value on failure + */ +static int libxevd_image_copy(struct AVCodecContext *avctx, XEVD_IMGB *imgb, struct AVFrame *frame) +{ + int ret; + if (imgb->cs != XEVD_CS_YCBCR420_10LE) { + av_log(avctx, AV_LOG_ERROR, "Not supported pixel format: %s\n", av_get_pix_fmt_name(avctx->pix_fmt)); + return AVERROR_INVALIDDATA; + } + + if (imgb->w[0] != avctx->width || imgb->h[0] != avctx->height) { // stream resolution changed + if (ff_set_dimensions(avctx, imgb->w[0], imgb->h[0]) < 0) { + av_log(avctx, AV_LOG_ERROR, "Cannot set new dimension\n"); + return AVERROR_INVALIDDATA; + } + } + + if (ret = ff_get_buffer(avctx, frame, 0) < 0) { + return ret; + } + + av_image_copy(frame->data, frame->linesize, (const uint8_t **)imgb->a, + imgb->s, avctx->pix_fmt, + imgb->w[0], imgb->h[0]); + + return 0; +} + +/** + * Initialize decoder + * Create a decoder instance and allocate all the needed resources + * + * @param avctx codec context + * @return 0 on success, negative error code on failure + */ +static av_cold int libxevd_init(AVCodecContext *avctx) +{ + XevdContext *xectx = avctx->priv_data; + XEVD_CDSC *cdsc = &(xectx->cdsc); + + /* read configurations and set values for created descriptor (XEVD_CDSC) */ + get_conf(avctx, cdsc); + + /* create decoder */ + xectx->id = xevd_create(&(xectx->cdsc), NULL); + if (xectx->id == NULL) { + av_log(avctx, AV_LOG_ERROR, "Cannot create XEVD encoder\n"); + return AVERROR_EXTERNAL; + } + + return 0; +} + +/** + * Decode picture + * + * @param avctx codec context + * @param[out] frame decoded frame + * @param[out] got_frame_ptr decoder sets to 0 or 1 to indicate that a + * non-empty frame or subtitle was returned in + * outdata. + * @param[in] avpkt AVPacket containing encoded data to be decoded + * + * @return amount of bytes read from the packet on success, negative error + * code on failure + */ +static int libxevd_decode(struct AVCodecContext *avctx, struct AVFrame *frame, int *got_frame_ptr, AVPacket *avpkt) +{ + XevdContext *xectx = NULL; + XEVD_IMGB *imgb = NULL; + XEVD_STAT stat; + XEVD_BITB bitb; + int xevd_ret, nalu_size, bs_read_pos; + int ret = 0; + + xectx = avctx->priv_data; + + if (avpkt->size > 0) { + bs_read_pos = 0; + imgb = NULL; + while(avpkt->size > (bs_read_pos + XEVD_NAL_UNIT_LENGTH_BYTE)) { + memset(&stat, 0, sizeof(XEVD_STAT)); + memset(&bitb, 0, sizeof(XEVD_BITB)); + + nalu_size = read_nal_unit_length(avpkt->data + bs_read_pos, XEVD_NAL_UNIT_LENGTH_BYTE, avctx); + if (nalu_size == 0) { + av_log(avctx, AV_LOG_ERROR, "Invalid bitstream\n"); + ret = AVERROR_INVALIDDATA; + goto ERR; + } + bs_read_pos += XEVD_NAL_UNIT_LENGTH_BYTE; + + bitb.addr = avpkt->data + bs_read_pos; + bitb.ssize = nalu_size; + + /* main decoding block */ + xevd_ret = xevd_decode(xectx->id, &bitb, &stat); + if (XEVD_FAILED(xevd_ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to decode bitstream\n"); + ret = AVERROR_EXTERNAL; + goto ERR; + } + + bs_read_pos += nalu_size; + + if (stat.nalu_type == XEVD_NUT_SPS) { // EVC stream parameters changed + if ((ret = export_stream_params(xectx, avctx)) != 0) + goto ERR; + } + + if (stat.read != nalu_size) + av_log(avctx, AV_LOG_INFO, "Different reading of bitstream (in:%d, read:%d)\n,", nalu_size, stat.read); + if (stat.fnum >= 0) { + // already has a decoded image + if (imgb) { + // xevd_pull uses pool of objects of type XEVD_IMGB. + // The pool size is equal MAX_PB_SIZE (26), so release object when it is no more needed + imgb->release(imgb); + imgb = NULL; + } + xevd_ret = xevd_pull(xectx->id, &imgb); + if (XEVD_FAILED(xevd_ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to pull the decoded image (xevd error code: %d, frame#=%d)\n", xevd_ret, stat.fnum); + ret = AVERROR_EXTERNAL; + goto ERR; + } else if (xevd_ret == XEVD_OK_FRM_DELAYED) { + if (imgb) { + // xevd_pull uses pool of objects of type XEVD_IMGB. + // The pool size is equal MAX_PB_SIZE (26), so release object when it is no more needed + imgb->release(imgb); + imgb = NULL; + } + } + if (imgb) { + int ret = libxevd_image_copy(avctx, imgb, frame); + if(ret < 0) { + goto ERR; + } + + frame->pts = avpkt->pts; + *got_frame_ptr = 1; + + // xevd_pull uses pool of objects of type XEVD_IMGB. + // The pool size is equal MAX_PB_SIZE (26), so release object when it is no more needed + imgb->release(imgb); + imgb = NULL; + } else + *got_frame_ptr = 0; + } + } + } else { // bumping + xevd_ret = xevd_pull(xectx->id, &(imgb)); + if (xevd_ret == XEVD_ERR_UNEXPECTED) { // bumping process completed + *got_frame_ptr = 0; + return 0; + } else if (XEVD_FAILED(xevd_ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to pull the decoded image (xevd error code: %d)\n", xevd_ret); + ret = AVERROR_EXTERNAL; + goto ERR; + } + if (imgb) { + int ret = libxevd_image_copy(avctx, imgb, frame); + if(ret < 0) { + goto ERR; + } + + frame->pts = avpkt->pts; + *got_frame_ptr = 1; + + // xevd_pull uses pool of objects of type XEVD_IMGB. + // The pool size is equal MAX_PB_SIZE (26), so release object when it is no more needed + imgb->release(imgb); + imgb = NULL; + } else + *got_frame_ptr = 0; + } + + return avpkt->size; + +ERR: + if (imgb) { + imgb->release(imgb); + imgb = NULL; + } + *got_frame_ptr = 0; + + return ret; +} + +/** + * Destroy decoder + * + * @param avctx codec context + * @return 0 on success + */ +static av_cold int libxevd_close(AVCodecContext *avctx) +{ + XevdContext *xectx = avctx->priv_data; + if (xectx->id) { + xevd_delete(xectx->id); + xectx->id = NULL; + } + + return 0; +} + +#define OFFSET(x) offsetof(XevdContext, x) +#define VD AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_DECODING_PARAM + +static const AVClass libxevd_class = { + .class_name = "libxevd", + .item_name = av_default_item_name, + .version = LIBAVUTIL_VERSION_INT, +}; + +const FFCodec ff_libxevd_decoder = { + .p.name = "evc", + .p.long_name = NULL_IF_CONFIG_SMALL("EVC / MPEG-5 Essential Video Coding (EVC)"), + .p.type = AVMEDIA_TYPE_VIDEO, + .p.id = AV_CODEC_ID_EVC, + .init = libxevd_init, + FF_CODEC_DECODE_CB(libxevd_decode), + .close = libxevd_close, + .priv_data_size = sizeof(XevdContext), + .p.priv_class = &libxevd_class, + .p.capabilities = AV_CODEC_CAP_DELAY | AV_CODEC_CAP_OTHER_THREADS | AV_CODEC_CAP_AVOID_PROBING | AV_CODEC_CAP_DR1, + .p.wrapper_name = "libxevd", + .caps_internal = FF_CODEC_CAP_INIT_CLEANUP | FF_CODEC_CAP_NOT_INIT_THREADSAFE, +}; diff --git a/libavcodec/libxeve.c b/libavcodec/libxeve.c new file mode 100644 index 0000000000..bb1d5e4bba --- /dev/null +++ b/libavcodec/libxeve.c @@ -0,0 +1,601 @@ +/* + * libxeve encoder + * EVC (MPEG-5 Essential Video Coding) encoding using XEVE MPEG-5 EVC encoder library + * + * Copyright (C) 2021 Dawid Kozinski <d.kozinski@samsung.com> + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include <float.h> +#include <stdlib.h> + +#include <xeve.h> + +#include "libavutil/internal.h" +#include "libavutil/common.h" +#include "libavutil/opt.h" +#include "libavutil/pixdesc.h" +#include "libavutil/pixfmt.h" +#include "libavutil/time.h" +#include "libavutil/cpu.h" +#include "libavutil/avstring.h" + +#include "avcodec.h" +#include "internal.h" +#include "packet_internal.h" +#include "codec_internal.h" +#include "encode.h" + +#define MAX_BS_BUF (16*1024*1024) + +/** + * Error codes + */ +#define XEVE_PARAM_BAD_NAME -100 +#define XEVE_PARAM_BAD_VALUE -200 + +/** + * Encoder states + * + * STATE_ENCODING - the encoder receives and processes input frames + * STATE_BUMPING - there are no more input frames, however the encoder still processes previously received data + */ +typedef enum State { + STATE_ENCODING, + STATE_BUMPING, +} State; + +/** + * The structure stores all the states associated with the instance of Xeve MPEG-5 EVC encoder + */ +typedef struct XeveContext { + const AVClass *class; + + XEVE id; // XEVE instance identifier + XEVE_CDSC cdsc; // coding parameters i.e profile, width & height of input frame, num of therads, frame rate ... + XEVE_BITB bitb; // bitstream buffer (output) + XEVE_STAT stat; // encoding status (output) + XEVE_IMGB imgb; // image buffer (input) + + State state; // encoder state (skipping, encoding, bumping) + + int profile_id; // encoder profile (main, baseline) + int preset_id; // preset of xeve ( fast, medium, slow, placebo) + int tune_id; // tune of xeve (psnr, zerolatency) + + // variables for rate control modes + int rc_mode; // Rate control mode [ 0(CQP) / 1(ABR) / 2(CRF) ] + int qp; // quantization parameter (QP) [0,51] + int crf; // constant rate factor (CRF) [10,49] + + int hash; // embed picture signature (HASH) for conformance checking in decoding + int sei_info; // embed Supplemental enhancement information while encoding + + int color_format; // input data color format: currently only XEVE_CF_YCBCR420 is supported +} XeveContext; + +/** + * Convert FFmpeg pixel format (AVPixelFormat) to XEVE pre-defined color format + * + * @param[in] av_pix_fmt pixel format (@see https://ffmpeg.org/doxygen/trunk/pixfmt_8h.html#a9a8e335cf3be472042bc9f0cf80cd4c5) + * @param[out] xeve_col_fmt XEVE pre-defined color format (@see xeve.h) + * + * @return 0 on success, negative value on failure + */ +static int libxeve_color_fmt(enum AVPixelFormat av_pix_fmt, int *xeve_col_fmt) +{ + switch (av_pix_fmt) { + case AV_PIX_FMT_YUV420P: + *xeve_col_fmt = XEVE_CF_YCBCR420; + break; + case AV_PIX_FMT_YUV420P10: + *xeve_col_fmt = XEVE_CF_YCBCR420; + break; + default: + *xeve_col_fmt = XEVE_CF_UNKNOWN; + return AVERROR_INVALIDDATA; + } + + return 0; +} + +/** + * Convert FFmpeg pixel format (AVPixelFormat) into XEVE pre-defined color space + * + * @param[in] px_fmt pixel format (@see https://ffmpeg.org/doxygen/trunk/pixfmt_8h.html#a9a8e335cf3be472042bc9f0cf80cd4c5) + * + * @return XEVE pre-defined color space (@see xeve.h) on success, XEVE_CF_UNKNOWN on failure + */ +static int libxeve_color_space(enum AVPixelFormat av_pix_fmt) +{ + /* color space of input image */ + int cs = XEVE_CF_UNKNOWN; + + switch (av_pix_fmt) { + case AV_PIX_FMT_YUV420P: + cs = XEVE_CS_YCBCR420; + break; + case AV_PIX_FMT_YUV420P10: +#if AV_HAVE_BIGENDIAN + cs = XEVE_CS_SET(XEVE_CF_YCBCR420, 10, 1); +#else + cs = XEVE_CS_YCBCR420_10LE; +#endif + + break; + default: + cs = XEVE_CF_UNKNOWN; + break; + } + + return cs; +} + +/** + * The function returns a pointer to the object of the XEVE_CDSC type. + * XEVE_CDSC contains all encoder parameters that should be initialized before the encoder is used. + * + * The field values of the XEVE_CDSC structure are populated based on: + * - the corresponding field values of the AvCodecConetxt structure, + * - the xeve encoder specific option values, + * (the full list of options available for xeve encoder is displayed after executing the command ./ffmpeg --help encoder = libxeve) + * - and the xeve encoder options specified as a list of key-value pairs following the xeve-params option + * + * The order of processing input data and populating the XEVE_CDSC structure + * 1) first, the fields of the AVCodecContext structure corresponding to the provided input options are processed, + * (i.e -pix_fmt yuv420p -s:v 1920x1080 -r 30 -profile:v 0) + * 2) then xeve-specific options added as AVOption to the xeve AVCodec implementation + * (i.e -preset 0) + * + * Keep in mind that, there are options that can be set in different ways. + * In this case, please follow the above-mentioned order of processing. + * The most recent assignments overwrite the previous values. + * + * @param[in] avctx codec context (AVCodecContext) + * @param[out] cdsc contains all Xeve MPEG-5 EVC encoder encoder parameters that should be initialized before the encoder is use + * + * @return 0 on success, negative error code on failure + */ +static int get_conf(AVCodecContext *avctx, XEVE_CDSC *cdsc) +{ + XeveContext *xectx = NULL; + int ret; + + xectx = avctx->priv_data; + + /* initialize xeve_param struct with default values */ + ret = xeve_param_default(&cdsc->param); + if (XEVE_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Cannot set_default parameter\n"); + return AVERROR_EXTERNAL; + } + + /* read options from AVCodecContext */ + if (avctx->width > 0) + cdsc->param.w = avctx->width; + + if (avctx->height > 0) + cdsc->param.h = avctx->height; + + if (avctx->framerate.num > 0) { + // fps can be float number, but xeve API doesn't support it + cdsc->param.fps = (int)(((float)avctx->framerate.num / avctx->framerate.den) + 0.5); + } + + if (avctx->gop_size >= 0) // GOP size (key-frame interval, I-picture period) + cdsc->param.keyint = avctx->gop_size; // 0: only one I-frame at the first time; 1: every frame is coded in I-frame + + if (avctx->max_b_frames == 0 || avctx->max_b_frames == 1 || avctx->max_b_frames == 3 || + avctx->max_b_frames == 7 || avctx->max_b_frames == 15) // number of b-frames + cdsc->param.bframes = avctx->max_b_frames; + else { + av_log(avctx, AV_LOG_ERROR, "Incorrect value for maximum number of B frames: (%d) \n" + "Acceptable values for bf option (maximum number of B frames) are 0,1,3,7 or 15\n", avctx->max_b_frames); + return AVERROR_INVALIDDATA; + } + + if (avctx->level >= 0) + cdsc->param.level_idc = avctx->level; + + if (avctx->rc_buffer_size) // VBV buf size + cdsc->param.vbv_bufsize = (int)(avctx->rc_buffer_size / 1000); + + cdsc->param.rc_type = xectx->rc_mode; + + if (xectx->rc_mode == XEVE_RC_CQP) { + cdsc->param.qp = xectx->qp; + } else if (xectx->rc_mode == XEVE_RC_ABR) { + if (avctx->bit_rate / 1000 > INT_MAX || avctx->rc_max_rate / 1000 > INT_MAX) { + av_log(avctx, AV_LOG_ERROR, "Not supported bitrate bit_rate and rc_max_rate > %d000\n", INT_MAX); + return AVERROR_INVALIDDATA; + } + cdsc->param.bitrate = (int)(avctx->bit_rate / 1000); + } else if (xectx->rc_mode == XEVE_RC_CRF) { + cdsc->param.crf = xectx->crf; + } else { + av_log(avctx, AV_LOG_ERROR, "Not supported rate control type: %d\n", xectx->rc_mode); + return AVERROR_INVALIDDATA; + } + + if (avctx->thread_count <= 0) { + int cpu_count = av_cpu_count(); + cdsc->param.threads = (cpu_count < XEVE_MAX_THREADS) ? cpu_count : XEVE_MAX_THREADS; + } else if (avctx->thread_count > XEVE_MAX_THREADS) + cdsc->param.threads = XEVE_MAX_THREADS; + else + cdsc->param.threads = avctx->thread_count; + + + libxeve_color_fmt(avctx->pix_fmt, &xectx->color_format); + + cdsc->param.cs = XEVE_CS_SET(xectx->color_format, cdsc->param.codec_bit_depth, AV_HAVE_BIGENDIAN); + + cdsc->max_bs_buf_size = MAX_BS_BUF; + + ret = xeve_param_ppt(&cdsc->param, xectx->profile_id, xectx->preset_id, xectx->tune_id); + if (XEVE_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Cannot set profile(%d), preset(%d), tune(%d)\n", xectx->profile_id, xectx->preset_id, xectx->tune_id); + return AVERROR_EXTERNAL; + } + + return 0; +} + +/** + * Set XEVE_CFG_SET_USE_PIC_SIGNATURE for encoder + * + * @param[in] logger context + * @param[in] id XEVE encodec instance identifier + * @param[in] ctx the structure stores all the states associated with the instance of Xeve MPEG-5 EVC encoder + * + * @return 0 on success, negative error code on failure + */ +static int set_extra_config(AVCodecContext *avctx, XEVE id, XeveContext *ctx) +{ + int ret, size; + size = 4; + + // embed SEI messages identifying encoder parameters and command line arguments + // - 0: off\n" + // - 1: emit sei info" + // + // SEI - Supplemental enhancement information contains information + // that is not necessary to decode the samples of coded pictures from VCL NAL units. + // Some SEI message information is required to check bitstream conformance + // and for output timing decoder conformance. + // @see ISO_IEC_23094-1_2020 7.4.3.5 + // @see ISO_IEC_23094-1_2020 Annex D + ret = xeve_config(id, XEVE_CFG_SET_SEI_CMD, &ctx->sei_info, &size); // sei_cmd_info + if (XEVE_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to set config for sei command info messages\n"); + return AVERROR_EXTERNAL; + } + + ret = xeve_config(id, XEVE_CFG_SET_USE_PIC_SIGNATURE, &ctx->hash, &size); + if (XEVE_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to set config for picture signature\n"); + return AVERROR_EXTERNAL; + } + + return 0; +} + +/** + * @brief Switch encoder to bumping mode + * + * @param id XEVE encodec instance identifier + * @return 0 on success, negative error code on failure + */ +static int setup_bumping(XEVE id) +{ + int val = 1; + int size = sizeof(int); + if (XEVE_FAILED(xeve_config(id, XEVE_CFG_SET_FORCE_OUT, (void *)(&val), &size))) + return AVERROR_EXTERNAL; + + return 0; +} + +/** + * @brief Initialize eXtra-fast Essential Video Encoder codec + * Create an encoder instance and allocate all the needed resources + * + * @param avctx codec context + * @return 0 on success, negative error code on failure + */ +static av_cold int libxeve_init(AVCodecContext *avctx) +{ + XeveContext *xectx = avctx->priv_data; + unsigned char *bs_buf = NULL; + int i; + int shift_h = 0; + int shift_v = 0; + int width_chroma = 0; + int height_chroma = 0; + XEVE_IMGB *imgb = NULL; + int ret = 0; + + XEVE_CDSC *cdsc = &(xectx->cdsc); + + /* allocate bitstream buffer */ + bs_buf = av_malloc(MAX_BS_BUF); + if (bs_buf == NULL) { + av_log(avctx, AV_LOG_ERROR, "Cannot allocate bitstream buffer\n"); + return AVERROR(ENOMEM); + } + xectx->bitb.addr = bs_buf; + xectx->bitb.bsize = MAX_BS_BUF; + + /* read configurations and set values for created descriptor (XEVE_CDSC) */ + if ((ret = get_conf(avctx, cdsc)) != 0) { + av_log(avctx, AV_LOG_ERROR, "Cannot get configuration\n"); + return AVERROR(EINVAL); + } + + if ((ret = xeve_param_check(&cdsc->param)) != 0) { + av_log(avctx, AV_LOG_ERROR, "Invalid configuration\n"); + return AVERROR(EINVAL); + } + + /* create encoder */ + xectx->id = xeve_create(cdsc, NULL); + if (xectx->id == NULL) { + av_log(avctx, AV_LOG_ERROR, "Cannot create XEVE encoder\n"); + return AVERROR_EXTERNAL; + } + + if ((ret = set_extra_config(avctx, xectx->id, xectx)) != 0) { + av_log(avctx, AV_LOG_ERROR, "Cannot set extra configuration\n"); + return AVERROR(EINVAL); + } + + if ((ret = av_pix_fmt_get_chroma_sub_sample(avctx->pix_fmt, &shift_h, &shift_v)) != 0) { + av_log(avctx, AV_LOG_ERROR, "Failed to get chroma shift\n"); + return AVERROR(EINVAL); + } + + // Chroma subsampling + // + // YUV format explanation + // shift_h == 1 && shift_v == 1 : YUV420 + // shift_h == 1 && shift_v == 0 : YUV422 + // shift_h == 0 && shift_v == 0 : YUV444 + // + width_chroma = AV_CEIL_RSHIFT(avctx->width, shift_h); + height_chroma = AV_CEIL_RSHIFT(avctx->height, shift_v); + + /* set default values for input image buffer */ + imgb = &xectx->imgb; + imgb->cs = libxeve_color_space(avctx->pix_fmt); + imgb->np = 3; /* only for yuv420p, yuv420ple */ + + for (i = 0; i < imgb->np; i++) + imgb->x[i] = imgb->y[i] = 0; + + imgb->w[0] = imgb->aw[0] = avctx->width; // width luma + imgb->w[1] = imgb->w[2] = imgb->aw[1] = imgb->aw[2] = width_chroma; + imgb->h[0] = imgb->ah[0] = avctx->height; // height luma + imgb->h[1] = imgb->h[2] = imgb->ah[1] = imgb->ah[2] = height_chroma; + + xectx->state = STATE_ENCODING; + + return 0; +} + +/** + * Encode raw data frame into EVC packet + * + * @param[in] avctx codec context + * @param[out] pkt output AVPacket containing encoded data + * @param[in] frame AVFrame containing the raw data to be encoded + * @param[out] got_packet encoder sets to 0 or 1 to indicate that a + * non-empty packet was returned in pkt + * + * @return 0 on success, negative error code on failure + */ +static int libxeve_encode(AVCodecContext *avctx, AVPacket *avpkt, + const AVFrame *frame, int *got_packet) +{ + XeveContext *xectx = avctx->priv_data; + int ret = -1; + + // No more input frames are available but encoder still can have some data in its internal buffer to process + // and some frames to dump. + if (xectx->state == STATE_ENCODING && frame == NULL) { + if (setup_bumping(xectx->id) == 0) + xectx->state = STATE_BUMPING; // Entering bumping process + else { + av_log(avctx, AV_LOG_ERROR, "Failed to setup bumping\n"); + return 0; + } + } + + if (xectx->state == STATE_ENCODING) { + int i; + XEVE_IMGB *imgb = NULL; + + imgb = &xectx->imgb; + + for (i = 0; i < imgb->np; i++) { + imgb->a[i] = frame->data[i]; + imgb->s[i] = frame->linesize[i]; + } + + imgb->ts[0] = frame->pts; + imgb->ts[1] = 0; + + /* push image to encoder */ + ret = xeve_push(xectx->id, imgb); + if (XEVE_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "xeve_push() failed\n"); + return AVERROR_EXTERNAL; + } + } + if (xectx->state == STATE_ENCODING || xectx->state == STATE_BUMPING) { + /* encoding */ + ret = xeve_encode(xectx->id, &(xectx->bitb), &(xectx->stat)); + if (XEVE_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "xeve_encode() failed\n"); + return AVERROR_EXTERNAL; + } + + /* store bitstream */ + if (ret == XEVE_OK_OUT_NOT_AVAILABLE) { // Return OK but picture is not available yet + *got_packet = 0; + return 0; + } else if (ret == XEVE_OK) { + int av_pic_type; + + if (xectx->stat.write > 0) { + + ret = ff_get_encode_buffer(avctx, avpkt, xectx->stat.write, 0); + if (ret < 0) + return ret; + + memcpy(avpkt->data, xectx->bitb.addr, xectx->stat.write); + + avpkt->pts = xectx->bitb.ts[0]; + avpkt->dts = xectx->bitb.ts[1]; + + switch(xectx->stat.stype) { + case XEVE_ST_I: + av_pic_type = AV_PICTURE_TYPE_I; + avpkt->flags |= AV_PKT_FLAG_KEY; + break; + case XEVE_ST_P: + av_pic_type = AV_PICTURE_TYPE_P; + break; + case XEVE_ST_B: + av_pic_type = AV_PICTURE_TYPE_B; + break; + case XEVE_ST_UNKNOWN: + av_log(avctx, AV_LOG_ERROR, "Unknown slice type\n"); + return AVERROR_INVALIDDATA; + } + + ff_side_data_set_encoder_stats(avpkt, xectx->stat.qp * FF_QP2LAMBDA, NULL, 0, av_pic_type); + + *got_packet = 1; + } + } else if (ret == XEVE_OK_NO_MORE_FRM) { + // Return OK but no more frames + return 0; + } else { + av_log(avctx, AV_LOG_ERROR, "Invalid return value: %d\n", ret); + return AVERROR_EXTERNAL; + } + } else { + av_log(avctx, AV_LOG_ERROR, "Udefined encoder state\n"); + return AVERROR_INVALIDDATA; + } + + return 0; +} + +/** + * Destroy the encoder and release all the allocated resources + * + * @param avctx codec context + * @return 0 on success, negative error code on failure + */ +static av_cold int libxeve_close(AVCodecContext *avctx) +{ + XeveContext *xectx = avctx->priv_data; + + if (xectx->id) { + xeve_delete(xectx->id); + xectx->id = NULL; + } + + av_free(xectx->bitb.addr); /* release bitstream buffer */ + + return 0; +} + +#define OFFSET(x) offsetof(XeveContext, x) +#define VE AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_ENCODING_PARAM + +static const enum AVPixelFormat supported_pixel_formats[] = { + AV_PIX_FMT_YUV420P, + AV_PIX_FMT_YUV420P10, + AV_PIX_FMT_NONE +}; + +// Example of using: ./ffmpeg -xeve-params "m=2:q=17" +// Consider using following options (./ffmpeg --help encoder=libxeve) +// +static const AVOption libxeve_options[] = { + { "preset", "Encoding preset for setting encoding speed", OFFSET(preset_id), AV_OPT_TYPE_INT, { .i64 = XEVE_PRESET_MEDIUM }, XEVE_PRESET_DEFAULT, XEVE_PRESET_PLACEBO, VE, "preset" }, + { "default", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_PRESET_DEFAULT }, INT_MIN, INT_MAX, VE, "preset" }, + { "fast", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_PRESET_FAST }, INT_MIN, INT_MAX, VE, "preset" }, + { "medium", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_PRESET_MEDIUM }, INT_MIN, INT_MAX, VE, "preset" }, + { "slow", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_PRESET_SLOW }, INT_MIN, INT_MAX, VE, "preset" }, + { "placebo", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_PRESET_PLACEBO }, INT_MIN, INT_MAX, VE, "preset" }, + { "tune", "Tuning parameter for special purpose operation", OFFSET(tune_id), AV_OPT_TYPE_INT, { .i64 = XEVE_TUNE_NONE }, XEVE_TUNE_NONE, XEVE_TUNE_PSNR, VE, "tune"}, + { "none", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_TUNE_NONE }, INT_MIN, INT_MAX, VE, "tune" }, + { "zerolatency", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_TUNE_ZEROLATENCY }, INT_MIN, INT_MAX, VE, "tune" }, + { "psnr", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_TUNE_PSNR }, INT_MIN, INT_MAX, VE, "tune" }, + { "profile", "Encoding profile", OFFSET(profile_id), AV_OPT_TYPE_INT, { .i64 = XEVE_PROFILE_BASELINE }, XEVE_PROFILE_BASELINE, XEVE_PROFILE_MAIN, VE, "profile" }, + { "baseline", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_PROFILE_BASELINE }, INT_MIN, INT_MAX, VE, "profile" }, + { "main", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_PROFILE_MAIN }, INT_MIN, INT_MAX, VE, "profile" }, + { "rc_mode", "Rate control mode", OFFSET(rc_mode), AV_OPT_TYPE_INT, { .i64 = XEVE_RC_CQP }, XEVE_RC_CQP, XEVE_RC_CRF, VE, "rc_mode" }, + { "CQP", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_RC_CQP }, INT_MIN, INT_MAX, VE, "rc_mode" }, + { "ABR", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_RC_ABR }, INT_MIN, INT_MAX, VE, "rc_mode" }, + { "CRF", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_RC_CRF }, INT_MIN, INT_MAX, VE, "rc_mode" }, + { "qp", "Quantization parameter value for CQP rate control mode", OFFSET(qp), AV_OPT_TYPE_INT, { .i64 = 32 }, 0, 51, VE }, + { "crf", "Constant rate factor value for CRF rate control mode", OFFSET(crf), AV_OPT_TYPE_INT, { .i64 = 32 }, 10, 49, VE }, + { "hash", "Embed picture signature (HASH) for conformance checking in decoding", OFFSET(hash), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, 1, VE }, + { "sei_info", "Embed SEI messages identifying encoder parameters and command line arguments", OFFSET(sei_info), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, 1, VE }, + { NULL } +}; + +static const AVClass libxeve_class = { + .class_name = "libxeve", + .item_name = av_default_item_name, + .option = libxeve_options, + .version = LIBAVUTIL_VERSION_INT, +}; + +/** + * libavcodec generic global options, which can be set on all the encoders and decoders + * @see https://www.ffmpeg.org/ffmpeg-codecs.html#Codec-Options + */ +static const FFCodecDefault libxeve_defaults[] = { + { "b", "0" }, // bitrate in terms of kilo-bits per second + { "g", "0" }, // gop_size (key-frame interval 0: only one I-frame at the first time; 1: every frame is coded in I-frame) + { "bf", "15"}, // maximum number of B frames (0: no B-frames, 1,3,7,15) + { "threads", "0"}, // number of threads to be used (0: automatically select the number of threads to set) + { NULL }, +}; + +const FFCodec ff_libxeve_encoder = { + .p.name = "libxeve", + .p.long_name = NULL_IF_CONFIG_SMALL("libxeve MPEG-5 EVC"), + .p.type = AVMEDIA_TYPE_VIDEO, + .p.id = AV_CODEC_ID_EVC, + .init = libxeve_init, + FF_CODEC_ENCODE_CB(libxeve_encode), + .close = libxeve_close, + .priv_data_size = sizeof(XeveContext), + .p.priv_class = &libxeve_class, + .defaults = libxeve_defaults, + .p.capabilities = AV_CODEC_CAP_DELAY | AV_CODEC_CAP_OTHER_THREADS | AV_CODEC_CAP_DR1, + .p.wrapper_name = "libxeve", + .p.pix_fmts = supported_pixel_formats, + .caps_internal = FF_CODEC_CAP_INIT_CLEANUP | FF_CODEC_CAP_NOT_INIT_THREADSAFE, +}; diff --git a/libavcodec/parsers.c b/libavcodec/parsers.c index a8d52af6cb..ca00c63176 100644 --- a/libavcodec/parsers.c +++ b/libavcodec/parsers.c @@ -41,6 +41,7 @@ extern const AVCodecParser ff_dvaudio_parser; extern const AVCodecParser ff_dvbsub_parser; extern const AVCodecParser ff_dvdsub_parser; extern const AVCodecParser ff_dvd_nav_parser; +extern const AVCodecParser ff_evc_parser; extern const AVCodecParser ff_flac_parser; extern const AVCodecParser ff_g723_1_parser; extern const AVCodecParser ff_g729_parser; diff --git a/libavcodec/profiles.c b/libavcodec/profiles.c index 7af7fbeb13..a31244e0db 100644 --- a/libavcodec/profiles.c +++ b/libavcodec/profiles.c @@ -181,4 +181,10 @@ const AVProfile ff_arib_caption_profiles[] = { { FF_PROFILE_UNKNOWN } }; +const AVProfile ff_evc_profiles[] = { + { FF_PROFILE_EVC_BASELINE, "Baseline" }, + { FF_PROFILE_EVC_MAIN, "Main" }, + { FF_PROFILE_UNKNOWN }, +}; + #endif /* !CONFIG_SMALL */ diff --git a/libavcodec/profiles.h b/libavcodec/profiles.h index 41a19aa9ad..cf92b5f126 100644 --- a/libavcodec/profiles.h +++ b/libavcodec/profiles.h @@ -72,5 +72,6 @@ extern const AVProfile ff_sbc_profiles[]; extern const AVProfile ff_prores_profiles[]; extern const AVProfile ff_mjpeg_profiles[]; extern const AVProfile ff_arib_caption_profiles[]; +extern const AVProfile ff_evc_profiles[]; #endif /* AVCODEC_PROFILES_H */ diff --git a/libavcodec/version.h b/libavcodec/version.h index 777be76edf..7e89daf017 100644 --- a/libavcodec/version.h +++ b/libavcodec/version.h @@ -29,7 +29,7 @@ #include "version_major.h" -#define LIBAVCODEC_VERSION_MINOR 42 +#define LIBAVCODEC_VERSION_MINOR 43 #define LIBAVCODEC_VERSION_MICRO 101 #define LIBAVCODEC_VERSION_INT AV_VERSION_INT(LIBAVCODEC_VERSION_MAJOR, \ diff --git a/libavformat/Makefile b/libavformat/Makefile index e420384355..f5326aea79 100644 --- a/libavformat/Makefile +++ b/libavformat/Makefile @@ -249,6 +249,8 @@ OBJS-$(CONFIG_HCOM_DEMUXER) += hcom.o pcm.o OBJS-$(CONFIG_HDS_MUXER) += hdsenc.o OBJS-$(CONFIG_HEVC_DEMUXER) += hevcdec.o rawdec.o OBJS-$(CONFIG_HEVC_MUXER) += rawenc.o +OBJS-$(CONFIG_EVC_DEMUXER) += evcdec.o rawdec.o +OBJS-$(CONFIG_EVC_MUXER) += rawenc.o OBJS-$(CONFIG_HLS_DEMUXER) += hls.o hls_sample_encryption.o OBJS-$(CONFIG_HLS_MUXER) += hlsenc.o hlsplaylist.o avc.o OBJS-$(CONFIG_HNM_DEMUXER) += hnm.o @@ -358,7 +360,7 @@ OBJS-$(CONFIG_MOV_DEMUXER) += mov.o mov_chan.o mov_esds.o \ OBJS-$(CONFIG_MOV_MUXER) += movenc.o av1.o avc.o hevc.o vpcc.o \ movenchint.o mov_chan.o rtp.o \ movenccenc.o movenc_ttml.o rawutils.o \ - dovi_isom.o + dovi_isom.o evc.o OBJS-$(CONFIG_MP2_MUXER) += rawenc.o OBJS-$(CONFIG_MP3_DEMUXER) += mp3dec.o replaygain.o OBJS-$(CONFIG_MP3_MUXER) += mp3enc.o rawenc.o id3v2enc.o diff --git a/libavformat/allformats.c b/libavformat/allformats.c index ae4479fb7a..dfce49d149 100644 --- a/libavformat/allformats.c +++ b/libavformat/allformats.c @@ -148,6 +148,8 @@ extern const AVInputFormat ff_ea_cdata_demuxer; extern const AVInputFormat ff_eac3_demuxer; extern const AVOutputFormat ff_eac3_muxer; extern const AVInputFormat ff_epaf_demuxer; +extern const AVInputFormat ff_evc_demuxer; +extern const AVOutputFormat ff_evc_muxer; extern const AVOutputFormat ff_f4v_muxer; extern const AVInputFormat ff_ffmetadata_demuxer; extern const AVOutputFormat ff_ffmetadata_muxer; diff --git a/libavformat/isom_tags.c b/libavformat/isom_tags.c index 362cb77e8f..98f866094b 100644 --- a/libavformat/isom_tags.c +++ b/libavformat/isom_tags.c @@ -147,6 +147,8 @@ const AVCodecTag ff_codec_movvideo_tags[] = { { AV_CODEC_ID_H264, MKTAG('d', 'v', 'a', '1') }, /* AVC-based Dolby Vision derived from avc1 */ { AV_CODEC_ID_H264, MKTAG('d', 'v', 'a', 'v') }, /* AVC-based Dolby Vision derived from avc3 */ + { AV_CODEC_ID_EVC, MKTAG('e', 'v', 'c', '1') }, /* EVC/MPEG-5 */ + { AV_CODEC_ID_VP8, MKTAG('v', 'p', '0', '8') }, /* VP8 */ { AV_CODEC_ID_VP9, MKTAG('v', 'p', '0', '9') }, /* VP9 */ { AV_CODEC_ID_AV1, MKTAG('a', 'v', '0', '1') }, /* AV1 */ diff --git a/libavformat/mov.c b/libavformat/mov.c index 6ee6ed0950..28049e6790 100644 --- a/libavformat/mov.c +++ b/libavformat/mov.c @@ -9090,7 +9090,7 @@ const AVInputFormat ff_mov_demuxer = { .long_name = NULL_IF_CONFIG_SMALL("QuickTime / MOV"), .priv_class = &mov_class, .priv_data_size = sizeof(MOVContext), - .extensions = "mov,mp4,m4a,3gp,3g2,mj2,psp,m4b,ism,ismv,isma,f4v,avif", + .extensions = "mov,mp4,m4a,3gp,3g2,mj2,psp,m4b,ism,ismv,isma,f4v,avif,evc", .flags_internal = FF_FMT_INIT_CLEANUP, .read_probe = mov_probe, .read_header = mov_read_header, diff --git a/libavformat/movenc.c b/libavformat/movenc.c index c8b2e141cb..93e4f64a63 100644 --- a/libavformat/movenc.c +++ b/libavformat/movenc.c @@ -35,6 +35,7 @@ #include "isom.h" #include "av1.h" #include "avc.h" +#include "evc.h" #include "libavcodec/ac3_parser_internal.h" #include "libavcodec/dnxhddata.h" #include "libavcodec/flac.h" @@ -1392,6 +1393,21 @@ static int mov_write_hvcc_tag(AVIOContext *pb, MOVTrack *track) return update_size(pb, pos); } +static int mov_write_evcc_tag(AVIOContext *pb, MOVTrack *track) +{ + int64_t pos = avio_tell(pb); + + avio_wb32(pb, 0); + ffio_wfourcc(pb, "evcC"); + + if (track->tag == MKTAG('e','v','c','1')) + ff_isom_write_evcc(pb, track->vos_data, track->vos_len, 1); + else + ff_isom_write_evcc(pb, track->vos_data, track->vos_len, 0); + + return update_size(pb, pos); +} + /* also used by all avid codecs (dv, imx, meridien) and their variants */ static int mov_write_avid_tag(AVIOContext *pb, MOVTrack *track) { @@ -1641,6 +1657,16 @@ static int mov_get_h264_codec_tag(AVFormatContext *s, MOVTrack *track) return tag; } +static int mov_get_evc_codec_tag(AVFormatContext *s, MOVTrack *track) +{ + int tag = track->par->codec_tag; + + if (!tag) + tag = MKTAG('e', 'v', 'c', 'i'); + + return tag; +} + static const struct { enum AVPixelFormat pix_fmt; uint32_t tag; @@ -1722,6 +1748,8 @@ static unsigned int mov_get_codec_tag(AVFormatContext *s, MOVTrack *track) tag = mov_get_mpeg2_xdcam_codec_tag(s, track); else if (track->par->codec_id == AV_CODEC_ID_H264) tag = mov_get_h264_codec_tag(s, track); + else if (track->par->codec_id == AV_CODEC_ID_EVC) + tag = mov_get_evc_codec_tag(s, track); else if (track->par->codec_id == AV_CODEC_ID_DNXHD) tag = mov_get_dnxhd_codec_tag(s, track); else if (track->par->codec_type == AVMEDIA_TYPE_VIDEO) { @@ -2280,6 +2308,9 @@ static int mov_write_video_tag(AVFormatContext *s, AVIOContext *pb, MOVMuxContex mov_write_avcc_tag(pb, track); if (track->mode == MODE_IPOD) mov_write_uuid_tag_ipod(pb); + } + else if (track->par->codec_id ==AV_CODEC_ID_EVC) { + mov_write_evcc_tag(pb, track); } else if (track->par->codec_id == AV_CODEC_ID_VP9) { mov_write_vpcc_tag(mov->fc, pb, track); } else if (track->par->codec_id == AV_CODEC_ID_AV1) { @@ -6030,7 +6061,8 @@ int ff_mov_write_packet(AVFormatContext *s, AVPacket *pkt) if ((par->codec_id == AV_CODEC_ID_DNXHD || par->codec_id == AV_CODEC_ID_H264 || par->codec_id == AV_CODEC_ID_HEVC || - par->codec_id == AV_CODEC_ID_TRUEHD) && !trk->vos_len && + par->codec_id == AV_CODEC_ID_TRUEHD || + par->codec_id == AV_CODEC_ID_EVC) && !trk->vos_len && !TAG_IS_AVCI(trk->tag)) { /* copy frame to create needed atoms */ trk->vos_len = size; @@ -7689,6 +7721,7 @@ static const AVCodecTag codec_mp4_tags[] = { { AV_CODEC_ID_H264, MKTAG('a', 'v', 'c', '3') }, { AV_CODEC_ID_HEVC, MKTAG('h', 'e', 'v', '1') }, { AV_CODEC_ID_HEVC, MKTAG('h', 'v', 'c', '1') }, + { AV_CODEC_ID_EVC, MKTAG('e', 'v', 'c', '1') }, { AV_CODEC_ID_MPEG2VIDEO, MKTAG('m', 'p', '4', 'v') }, { AV_CODEC_ID_MPEG1VIDEO, MKTAG('m', 'p', '4', 'v') }, { AV_CODEC_ID_MJPEG, MKTAG('m', 'p', '4', 'v') }, diff --git a/libavformat/rawenc.c b/libavformat/rawenc.c index 267fce252d..b7b2aff453 100644 --- a/libavformat/rawenc.c +++ b/libavformat/rawenc.c @@ -401,6 +401,19 @@ const AVOutputFormat ff_hevc_muxer = { }; #endif +#if CONFIG_EVC_MUXER +AVOutputFormat ff_evc_muxer = { + .name = "evc", + .long_name = NULL_IF_CONFIG_SMALL("raw EVC video"), + .extensions = "evc", + .audio_codec = AV_CODEC_ID_NONE, + .video_codec = AV_CODEC_ID_EVC, + .write_header = force_one_stream, + .write_packet = ff_raw_write_packet, + .flags = AVFMT_NOTIMESTAMPS, +}; +#endif + #if CONFIG_M4V_MUXER const AVOutputFormat ff_m4v_muxer = { .name = "m4v", -- 2.17.1 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 13+ messages in thread
[parent not found: <CGME20220801092858eucas1p2eb2d2732543be955865cc083ebf582b3@eucas1p2.samsung.com>]
* [FFmpeg-devel] [PATCH 1/2] Provided support for MPEG-5 EVC (Essential Video Coding) codec [not found] <CGME20220801092858eucas1p2eb2d2732543be955865cc083ebf582b3@eucas1p2.samsung.com> @ 2022-08-01 9:28 ` Dawid Kozinski 2022-08-01 12:29 ` James Almer 2022-08-01 15:09 ` James Almer 0 siblings, 2 replies; 13+ messages in thread From: Dawid Kozinski @ 2022-08-01 9:28 UTC (permalink / raw) To: ffmpeg-devel - Added new entry to codec IDs list - Added new entry to the codec descriptor list - Bumped libavcodec minor version - Changes in Changelog and MAINTAINERS files - Added xeve encoder wrapper - Added xevd dencoder wrapper - Added documentation for xeve and xevd wrappers - Added parser for EVC format - Changes in project configuration file and libavcodec Makefile Signed-off-by: Dawid Kozinski <d.kozinski@samsung.com> --- Changelog | 3 +- MAINTAINERS | 2 + configure | 8 + doc/decoders.texi | 24 ++ doc/encoders.texi | 92 ++++++ doc/general_contents.texi | 19 ++ libavcodec/Makefile | 3 + libavcodec/allcodecs.c | 2 + libavcodec/avcodec.h | 3 + libavcodec/codec_desc.c | 8 + libavcodec/codec_id.h | 1 + libavcodec/evc_parser.c | 526 +++++++++++++++++++++++++++++++++ libavcodec/libxevd.c | 436 ++++++++++++++++++++++++++++ libavcodec/libxeve.c | 597 ++++++++++++++++++++++++++++++++++++++ libavcodec/parsers.c | 1 + libavcodec/profiles.c | 6 + libavcodec/profiles.h | 1 + libavcodec/version.h | 2 +- 18 files changed, 1732 insertions(+), 2 deletions(-) create mode 100644 libavcodec/evc_parser.c create mode 100644 libavcodec/libxevd.c create mode 100644 libavcodec/libxeve.c diff --git a/Changelog b/Changelog index 40ea7ccecf..5e14f0a8a4 100644 --- a/Changelog +++ b/Changelog @@ -80,7 +80,8 @@ version 5.0: - VideoToolbox ProRes encoder - anlmf audio filter - IMF demuxer (experimental) - +- eXtra-fast Essential Video Encoder (XEVE) +- eXtra-fast Essential Video Decoder (XEVD) version 4.4: - AudioToolbox output device diff --git a/MAINTAINERS b/MAINTAINERS index 274fc89203..4b33d3c1b2 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -205,6 +205,7 @@ Codecs: libvpx* James Zern libxavs.c Stefan Gehrer libxavs2.c Huiwen Ren + libxev*.c, evc_parser.c Dawid Kozinski libzvbi-teletextdec.c Marton Balint lzo.h, lzo.c Reimar Doeffinger mdec.c Michael Niedermayer @@ -425,6 +426,7 @@ Muxers/Demuxers: dv.c Roman Shaposhnik electronicarts.c Peter Ross epafdec.c Paul B Mahol + evcdec.c Dawid Kozinski ffm* Baptiste Coudurier flic.c Mike Melanson flvdec.c Michael Niedermayer diff --git a/configure b/configure index 8c7e8c9d1d..3c12140721 100755 --- a/configure +++ b/configure @@ -291,6 +291,8 @@ External library support: --enable-libwebp enable WebP encoding via libwebp [no] --enable-libx264 enable H.264 encoding via x264 [no] --enable-libx265 enable HEVC encoding via x265 [no] + --enable-libxeve enable EVC encoding via libxeve [no] + --enable-libxevd enable EVC decoding via libxevd [no] --enable-libxavs enable AVS encoding via xavs [no] --enable-libxavs2 enable AVS2 encoding via xavs2 [no] --enable-libxcb enable X11 grabbing using XCB [autodetect] @@ -1874,6 +1876,8 @@ EXTERNAL_LIBRARY_LIST=" libvorbis libvpx libwebp + libxevd + libxeve libxml2 libzimg libzmq @@ -3393,6 +3397,8 @@ libx264rgb_encoder_select="libx264_encoder" libx265_encoder_deps="libx265" libxavs_encoder_deps="libxavs" libxavs2_encoder_deps="libxavs2" +libxevd_decoder_deps="libxevd" +libxeve_encoder_deps="libxeve" libxvid_encoder_deps="libxvid" libzvbi_teletext_decoder_deps="libzvbi" vapoursynth_demuxer_deps="vapoursynth" @@ -6682,6 +6688,8 @@ enabled libx265 && require_pkg_config libx265 x265 x265.h x265_api_get require_cpp_condition libx265 x265.h "X265_BUILD >= 70" enabled libxavs && require libxavs "stdint.h xavs.h" xavs_encoder_encode "-lxavs $pthreads_extralibs $libm_extralibs" enabled libxavs2 && require_pkg_config libxavs2 "xavs2 >= 1.3.0" "stdint.h xavs2.h" xavs2_api_get +enabled libxevd && require_pkg_config libxevd "xevd >= 0.4.0" "xevd.h" xevd_decode +enabled libxeve && require_pkg_config libxeve "xeve >= 0.4.0" "xeve.h" xeve_encode enabled libxvid && require libxvid xvid.h xvid_global -lxvidcore enabled libzimg && require_pkg_config libzimg "zimg >= 2.7.0" zimg.h zimg_get_api_version enabled libzmq && require_pkg_config libzmq "libzmq >= 4.2.1" zmq.h zmq_ctx_new diff --git a/doc/decoders.texi b/doc/decoders.texi index e2fcbf5dc9..0bd9096114 100644 --- a/doc/decoders.texi +++ b/doc/decoders.texi @@ -126,6 +126,30 @@ Set amount of frame threads to use during decoding. The default value is 0 (auto @end table +@section libxevd + +eXtra-fast Essential Video Decoder (XEVD) MPEG-5 EVC decoder wrapper. + +This decoder requires the presence of the libxevd headers and library +during configuration. You need to explicitly configure the build with +@option{--enable-libxevd}. + +The xevd project website is at @url{https://github.com/mpeg5/xevd}. + +@subsection Options + +The following options are supported by the libxevd wrapper. +The xevd-equivalent options or values are listed in parentheses for easy migration. + +To get a more accurate and extensive documentation of the libxevd options, +invoke the command @code{xevd_app --help} or consult the libxevd documentation. + +@table @option +@item threads (@emph{threads}) +Force to use a specific number of threads + +@end table + @section QSV Decoders The family of Intel QuickSync Video decoders (VC1, MPEG-2, H.264, HEVC, diff --git a/doc/encoders.texi b/doc/encoders.texi index 6d73f74196..76f55e6721 100644 --- a/doc/encoders.texi +++ b/doc/encoders.texi @@ -2892,6 +2892,98 @@ ffmpeg -i input -c:v libxavs2 -xavs2-params RdoqLevel=0 output.avs2 @end example @end table +@section libxeve + +eXtra-fast Essential Video Encoder (XEVE) MPEG-5 EVC encoder wrapper. +The xeve-equivalent options or values are listed in parentheses for easy migration. + +This encoder requires the presence of the libxeve headers and library +during configuration. You need to explicitly configure the build with +@option{--enable-libxeve}. + +@float NOTE +Many libxeve encoder options are mapped to FFmpeg global codec options, +while unique encoder options are provided through private options. +Additionally the xeve-params private options allows one to pass a list +of key=value tuples as accepted by the libxeve @code{parse_xeve_params} function. +@end float + +The xeve project website is at @url{https://github.com/mpeg5/xeve}. + +@subsection Options + +The following options are supported by the libxeve wrapper. +The xeve-equivalent options or values are listed in parentheses for easy migration. + +@float NOTE +To reduce the duplication of documentation, only the private options +and some others requiring special attention are documented here. For +the documentation of the undocumented generic options, see +@ref{codec-options,,the Codec Options chapter}. +@end float + +@float NOTE +To get a more accurate and extensive documentation of the libxeve options, +invoke the command @code{xeve_app --help} or consult the libxeve documentation. +@end float + +@table @option +@item b (@emph{bitrate}) +Set target video bitrate in bits/s. +Note that FFmpeg's b option is expressed in bits/s, while xeve's bitrate is in kilobits/s. + +@item bf (@emph{bframes}) +Set the maximum number of B frames (1,3,7,15). + +@item g (@emph{keyint}) +Set the GOP size (I-picture period). + +@item preset (@emph{preset}) +Set the xeve preset. +Set the encoder preset value to determine encoding speed [fast, medium, slow, placebo] + +@item tune (@emph{tune}) +Set the encoder tune parameter [psnr, zerolatency] + +@item profile (@emph{profile}) +Set the encoder profile [0: baselie; 1: main] + +@item crf (@emph{crf}) +Set the quality for constant quality mode. +Constant rate factor <10..49> [default: 32] + +@item qp (@emph{qp}) +Set constant quantization rate control method parameter. +Quantization parameter qp <0..51> [default: 32] + +@item threads (@emph{threads}) +Force to use a specific number of threads + +@item xeve-params +Set xeve options using a list of @var{key}=@var{value} couples separated +by ":". + +For example to specify libxeve encoding options with @option{-xeve-params}: + +@example +ffmpeg -i input -c:v libxeve -xeve-params profile=main:tune=zerolatency:preset=fast:q=17:bframes=15:bitrate=100k output.mp4 +@end example +@end table + +@subsubsection Supported key values for xeve-params + +Set xeve options using a list of @var{key}=@var{value} couples separated + +@table @option + +@item hash +Embed the picture signature (HASH) for conformance checking in decoding, (0: Off, 1: On) + +@item vbv-bufsize +Set the VBV buffer size: Kbits(none,K,k), Mbits(M,m) + +@end table + @section libxvid Xvid MPEG-4 Part 2 encoder wrapper. diff --git a/doc/general_contents.texi b/doc/general_contents.texi index f25c784d3b..d72304dd82 100644 --- a/doc/general_contents.texi +++ b/doc/general_contents.texi @@ -343,6 +343,22 @@ libxavs2 is under the GNU Public License Version 2 or later details), you must upgrade FFmpeg's license to GPL in order to use it. @end float +@section eXtra-fast Essential Video Encoder (XEVE) + +FFmpeg can make use of the XEVE library for EVC video encoding. + +Go to @url{https://github.com/mpeg5/xeve} and follow the instructions for +installing the XEVE library. Then pass @code{--enable-libxeve} to configure to +enable it. + +@section eXtra-fast Essential Video Decoder (XEVD) + +FFmpeg can make use of the XEVD library for EVC video decoding. + +Go to @url{https://github.com/mpeg5/xevd} and follow the instructions for +installing the XEVD library. Then pass @code{--enable-libxevd} to configure to +enable it. + @section ZVBI ZVBI is a VBI decoding library which can be used by FFmpeg to decode DVB @@ -591,6 +607,7 @@ library: @item raw DTS @tab X @tab X @item raw DTS-HD @tab @tab X @item raw E-AC-3 @tab X @tab X +@item raw EVC @tab X @tab X @item raw FLAC @tab X @tab X @item raw GSM @tab @tab X @item raw H.261 @tab X @tab X @@ -928,6 +945,8 @@ following image formats are supported: @item Electronic Arts TQI video @tab @tab X @item Escape 124 @tab @tab X @item Escape 130 @tab @tab X +@item EVC / MPEG-5 Part 1 @tab X @tab X + @tab encoding and decoding supported through external libraries libxeve and libxevd @item FFmpeg video codec #1 @tab X @tab X @tab lossless codec (fourcc: FFV1) @item Flash Screen Video v1 @tab X @tab X diff --git a/libavcodec/Makefile b/libavcodec/Makefile index aff7752856..80adc7d54f 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -1109,6 +1109,8 @@ OBJS-$(CONFIG_LIBX264_ENCODER) += libx264.o OBJS-$(CONFIG_LIBX265_ENCODER) += libx265.o OBJS-$(CONFIG_LIBXAVS_ENCODER) += libxavs.o OBJS-$(CONFIG_LIBXAVS2_ENCODER) += libxavs2.o +OBJS-$(CONFIG_LIBXEVD_DECODER) += libxevd.o +OBJS-$(CONFIG_LIBXEVE_ENCODER) += libxeve.o OBJS-$(CONFIG_LIBXVID_ENCODER) += libxvid.o OBJS-$(CONFIG_LIBZVBI_TELETEXT_DECODER) += libzvbi-teletextdec.o ass.o @@ -1136,6 +1138,7 @@ OBJS-$(CONFIG_DVAUDIO_PARSER) += dvaudio_parser.o OBJS-$(CONFIG_DVBSUB_PARSER) += dvbsub_parser.o OBJS-$(CONFIG_DVD_NAV_PARSER) += dvd_nav_parser.o OBJS-$(CONFIG_DVDSUB_PARSER) += dvdsub_parser.o +OBJS-$(CONFIG_EVC_PARSER) += evc_parser.o OBJS-$(CONFIG_FLAC_PARSER) += flac_parser.o flacdata.o flac.o OBJS-$(CONFIG_G723_1_PARSER) += g723_1_parser.o OBJS-$(CONFIG_G729_PARSER) += g729_parser.o diff --git a/libavcodec/allcodecs.c b/libavcodec/allcodecs.c index 31d2c5979c..8a3ca15d36 100644 --- a/libavcodec/allcodecs.c +++ b/libavcodec/allcodecs.c @@ -798,6 +798,8 @@ extern LIBX264_CONST FFCodec ff_libx264_encoder; #endif extern const FFCodec ff_libx264rgb_encoder; extern FFCodec ff_libx265_encoder; +extern const FFCodec ff_libxeve_encoder; +extern const FFCodec ff_libxevd_decoder; extern const FFCodec ff_libxavs_encoder; extern const FFCodec ff_libxavs2_encoder; extern const FFCodec ff_libxvid_encoder; diff --git a/libavcodec/avcodec.h b/libavcodec/avcodec.h index 60b215d2e9..ba33c985cd 100644 --- a/libavcodec/avcodec.h +++ b/libavcodec/avcodec.h @@ -1671,6 +1671,9 @@ typedef struct AVCodecContext { #define FF_PROFILE_KLVA_SYNC 0 #define FF_PROFILE_KLVA_ASYNC 1 +#define FF_PROFILE_EVC_BASELINE 0 +#define FF_PROFILE_EVC_MAIN 1 + /** * level * - encoding: Set by user. diff --git a/libavcodec/codec_desc.c b/libavcodec/codec_desc.c index fdcf8198fe..ebeddd5fa6 100644 --- a/libavcodec/codec_desc.c +++ b/libavcodec/codec_desc.c @@ -1900,6 +1900,14 @@ static const AVCodecDescriptor codec_descriptors[] = { .long_name = NULL_IF_CONFIG_SMALL("HDR (Radiance RGBE format) image"), .props = AV_CODEC_PROP_INTRA_ONLY | AV_CODEC_PROP_LOSSY, }, + { + .id = AV_CODEC_ID_EVC, + .type = AVMEDIA_TYPE_VIDEO, + .name = "evc", + .long_name = NULL_IF_CONFIG_SMALL("MPEG-5 EVC (Essential Video Coding)"), + .props = AV_CODEC_PROP_LOSSY | AV_CODEC_PROP_REORDER, + .profiles = NULL_IF_CONFIG_SMALL(ff_evc_profiles), + }, /* various PCM "codecs" */ { diff --git a/libavcodec/codec_id.h b/libavcodec/codec_id.h index 27bf68ec1d..cebbf63245 100644 --- a/libavcodec/codec_id.h +++ b/libavcodec/codec_id.h @@ -313,6 +313,7 @@ enum AVCodecID { AV_CODEC_ID_QOI, AV_CODEC_ID_PHM, AV_CODEC_ID_RADIANCE_HDR, + AV_CODEC_ID_EVC, /* various PCM "codecs" */ AV_CODEC_ID_FIRST_AUDIO = 0x10000, ///< A dummy id pointing at the start of audio codecs diff --git a/libavcodec/evc_parser.c b/libavcodec/evc_parser.c new file mode 100644 index 0000000000..b2136a6b45 --- /dev/null +++ b/libavcodec/evc_parser.c @@ -0,0 +1,526 @@ +/* + * EVC format parser + * + * Copyright (C) 2021 Dawid Kozinski <d.kozinski@samsung.com> + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include <stdint.h> + +#include "libavutil/common.h" + +#include "parser.h" +#include "golomb.h" + +// The length field that indicates the length in bytes of the following NAL unit is configured to be of 4 bytes +#define EVC_NAL_UNIT_LENGTH_BYTE (4) /* byte */ + +#define EVC_NAL_HEADER_SIZE (2) /* byte */ +#define MAX_SPS_CNT (16) /* defined value in EVC standard */ + +// NALU types +// @see ISO_IEC_23094-1_2020 7.4.2.2 NAL unit header semantics +// +#define EVC_NUT_NONIDR (0) /* Coded slice of a non-IDR picture */ +#define EVC_NUT_IDR (1) /* Coded slice of an IDR picture */ +#define EVC_NUT_SPS (24) /* Sequence parameter set */ +#define EVC_NUT_PPS (25) /* Picture paremeter set */ +#define EVC_NUT_APS (26) /* Adaptation parameter set */ +#define EVC_NUT_FD (27) /* Filler data */ +#define EVC_NUT_SEI (28) /* Supplemental enhancement information */ + +// The sturcture reflects SPS RBSP(raw byte sequence payload) layout +// @see ISO_IEC_23094-1 section 7.3.2.1 +// +// The following descriptors specify the parsing process of each element +// u(n) - unsigned integer using n bits +// ue(v) - unsigned integer 0-th order Exp_Golomb-coded syntax element with the left bit first +typedef struct EVCParserSPS { + int sps_seq_parameter_set_id; // ue(v) + int profile_idc; // u(8) + int level_idc; // u(8) + int toolset_idc_h; // u(32) + int toolset_idc_l; // u(32) + int chroma_format_idc; // ue(v) + int pic_width_in_luma_samples; // ue(v) + int pic_height_in_luma_samples; // ue(v) + int bit_depth_luma_minus8; // ue(v) + int bit_depth_chroma_minus8; // ue(v) + + int sps_btt_flag; // u(1) + int log2_ctu_size_minus5; // ue(v) + int log2_min_cb_size_minus2; // ue(v) + int log2_diff_ctu_max_14_cb_size; // ue(v) + int log2_diff_ctu_max_tt_cb_size; // ue(v) + int log2_diff_min_cb_min_tt_cb_size_minus2; // ue(v) + + int sps_suco_flag; // u(1) + int log2_diff_ctu_size_max_suco_cb_size; // ue(v) + int log2_diff_max_suco_min_suco_cb_size; // ue(v) + + int sps_admvp_flag; // u(1) + int sps_affine_flag; // u(1) + int sps_amvr_flag; // u(1) + int sps_dmvr_flag; // u(1) + int sps_mmvd_flag; // u(1) + int sps_hmvp_flag; // u(1) + + int sps_eipd_flag; // u(1) + int sps_ibc_flag; // u(1) + int log2_max_ibc_cand_size_minus2; // ue(v) + + int sps_cm_init_flag; // u(1) + int sps_adcc_flag; // u(1) + + int sps_iqt_flag; // u(1) + int sps_ats_flag; // u(1) + + int sps_addb_flag; // u(1) + int sps_alf_flag; // u(1) + int sps_htdf_flag; // u(1) + int sps_rpl_flag; // u(1) + int sps_pocs_flag; // u(1) + int sps_dquant_flag; // u(1) + int sps_dra_flag; // u(1) + + int log2_max_pic_order_cnt_lsb_minus4; // ue(v) + int log2_sub_gop_length; // ue(v) + int log2_ref_pic_gap_length; // ue(v) + + int max_num_tid0_ref_pics; // ue(v) + + int sps_max_dec_pic_buffering_minus1; // ue(v) + int long_term_ref_pic_flag; // u(1) + int rpl1_same_as_rpl0_flag; // u(1) + int num_ref_pic_list_in_sps[2]; // ue(v) + + int picture_cropping_flag; // u(1) + int picture_crop_left_offset; // ue(v) + int picture_crop_right_offset; // ue(v) + int picture_crop_top_offset; // ue(v) + int picture_crop_bottom_offset; // ue(v) + +} EVCParserSPS; + +typedef struct EVCParserContext { + ParseContext pc; + EVCParserSPS sps[MAX_SPS_CNT]; + int is_avc; + int nal_length_size; + int to_read; + int incomplete_nalu_prefix_read; // The flag is set to 1 when an incomplete NAL unit prefix has been read + + int got_sps; + int got_pps; + int got_sei; + int got_slice; +} EVCParserContext; + +static int get_nalu_type(const uint8_t *bits, int bits_size, AVCodecContext *avctx) +{ + int unit_type_plus1 = 0; + + if(bits_size >= EVC_NAL_HEADER_SIZE) { + unsigned char *p = (unsigned char *)bits; + // forbidden_zero_bit + if ((p[0] & 0x80) != 0) { + av_log(avctx, AV_LOG_ERROR, "Cannot get bitstream information. Malformed bitstream.\n"); + return -1; + } + + // nal_unit_type + unit_type_plus1 = (p[0] >> 1) & 0x3F; + } + + return unit_type_plus1 - 1; +} + +static uint32_t read_nal_unit_length(const uint8_t *bits, int bits_size, AVCodecContext *avctx) +{ + uint32_t nalu_len = 0; + + if(bits_size >= EVC_NAL_UNIT_LENGTH_BYTE) { + + int t = 0; + unsigned char *p = (unsigned char *)bits; + + for(int i = 0; i < EVC_NAL_UNIT_LENGTH_BYTE; i++) + t = (t << 8) | p[i]; + + nalu_len = t; + if(nalu_len == 0) { + av_log(avctx, AV_LOG_ERROR, "Invalid bitstream size!\n"); + return 0; + } + } + + return nalu_len; +} + +// @see ISO_IEC_23094-1 (7.3.2.1 SPS RBSP syntax) +static EVCParserSPS *parse_sps(const uint8_t *bs, int bs_size, EVCParserContext *ev) +{ + GetBitContext gb; + EVCParserSPS *sps; + int sps_seq_parameter_set_id; + + if(init_get_bits8(&gb, bs, bs_size) < 0) + return NULL; + + sps_seq_parameter_set_id = get_ue_golomb(&gb); + + if(sps_seq_parameter_set_id >= MAX_SPS_CNT) + return NULL; + + sps = &ev->sps[sps_seq_parameter_set_id]; + sps->sps_seq_parameter_set_id = sps_seq_parameter_set_id; + + // the Baseline profile is indicated by profile_idc eqal to 0 + // the Main profile is indicated by profile_idc eqal to 1 + sps->profile_idc = get_bits(&gb, 8); + + sps->level_idc = get_bits(&gb, 8); + + skip_bits_long(&gb, 32); /* skip toolset_idc_h */ + skip_bits_long(&gb, 32); /* skip toolset_idc_l */ + + // 0 - monochrome + // 1 - 4:2:0 + // 2 - 4:2:2 + // 3 - 4:4:4 + sps->chroma_format_idc = get_ue_golomb(&gb); + + sps->pic_width_in_luma_samples = get_ue_golomb(&gb); + sps->pic_height_in_luma_samples = get_ue_golomb(&gb); + + sps->bit_depth_luma_minus8 = get_ue_golomb(&gb); + sps->bit_depth_chroma_minus8 = get_ue_golomb(&gb); + + sps->sps_btt_flag = get_bits(&gb, 1); + if(sps->sps_btt_flag) { + sps->log2_ctu_size_minus5 = get_ue_golomb(&gb); + sps->log2_min_cb_size_minus2 = get_ue_golomb(&gb); + sps->log2_diff_ctu_max_14_cb_size = get_ue_golomb(&gb); + sps->log2_diff_ctu_max_tt_cb_size = get_ue_golomb(&gb); + sps->log2_diff_min_cb_min_tt_cb_size_minus2 = get_ue_golomb(&gb); + } + + sps->sps_suco_flag = get_bits(&gb, 1); + if(sps->sps_suco_flag) { + sps->log2_diff_ctu_size_max_suco_cb_size = get_ue_golomb(&gb); + sps->log2_diff_max_suco_min_suco_cb_size = get_ue_golomb(&gb); + } + + sps->sps_admvp_flag = get_bits(&gb, 1); + if(sps->sps_admvp_flag) { + sps->sps_affine_flag = get_bits(&gb, 1); + sps->sps_amvr_flag = get_bits(&gb, 1); + sps->sps_dmvr_flag = get_bits(&gb, 1); + sps->sps_mmvd_flag = get_bits(&gb, 1); + sps->sps_hmvp_flag = get_bits(&gb, 1); + } + + sps->sps_eipd_flag = get_bits(&gb, 1); + if(sps->sps_eipd_flag) { + sps->sps_ibc_flag = get_bits(&gb, 1); + if(sps->sps_ibc_flag) + sps->log2_max_ibc_cand_size_minus2 = get_ue_golomb(&gb); + } + + sps->sps_cm_init_flag = get_bits(&gb, 1); + if(sps->sps_cm_init_flag) + sps->sps_adcc_flag = get_bits(&gb, 1); + + sps->sps_iqt_flag = get_bits(&gb, 1); + if(sps->sps_iqt_flag) + sps->sps_ats_flag = get_bits(&gb, 1); + + sps->sps_addb_flag = get_bits(&gb, 1); + sps->sps_alf_flag = get_bits(&gb, 1); + sps->sps_htdf_flag = get_bits(&gb, 1); + sps->sps_rpl_flag = get_bits(&gb, 1); + sps->sps_pocs_flag = get_bits(&gb, 1); + sps->sps_dquant_flag = get_bits(&gb, 1); + sps->sps_dra_flag = get_bits(&gb, 1); + + if(sps->sps_pocs_flag) + sps->log2_max_pic_order_cnt_lsb_minus4 = get_ue_golomb(&gb); + + if(!sps->sps_pocs_flag || !sps->sps_rpl_flag) { + sps->log2_sub_gop_length = get_ue_golomb(&gb); + if(sps->log2_sub_gop_length == 0) + sps->log2_ref_pic_gap_length = get_ue_golomb(&gb); + } + + return sps; +} + +static int parse_nal_units(AVCodecParserContext *s, const uint8_t *bs, + int bs_size, AVCodecContext *avctx) +{ + EVCParserContext *ev = s->priv_data; + int nalu_type, nalu_size; + unsigned char *bits = (unsigned char *)bs; + int bits_size = bs_size; + + avctx->codec_id = AV_CODEC_ID_EVC; + + nalu_size = read_nal_unit_length(bits, bits_size, avctx); + if(nalu_size == 0) { + av_log(avctx, AV_LOG_ERROR, "Invalid NAL unit size: (%d)\n", nalu_size); + return -1; + } + + bits += EVC_NAL_UNIT_LENGTH_BYTE; + bits_size -= EVC_NAL_UNIT_LENGTH_BYTE; + + nalu_type = get_nalu_type(bits, bits_size, avctx); + + bits += EVC_NAL_HEADER_SIZE; + bits_size -= EVC_NAL_HEADER_SIZE; + + if (nalu_type == EVC_NUT_SPS) { // NAL Unit type: SPS (Sequence Parameter Set) + EVCParserSPS *sps; + + sps = parse_sps(bits, bits_size, ev); + + avctx->coded_width = sps->pic_width_in_luma_samples; + avctx->coded_height = sps->pic_height_in_luma_samples; + avctx->width = sps->pic_width_in_luma_samples; + avctx->height = sps->pic_height_in_luma_samples; + + if(sps->profile_idc == 0) avctx->profile = FF_PROFILE_EVC_BASELINE; + else if (sps->profile_idc == 1) avctx->profile = FF_PROFILE_EVC_MAIN; + else { + av_log(avctx, AV_LOG_ERROR, "Not supported profile (%d)\n", sps->profile_idc); + return -1; + } + + // Currently XEVD decoder supports ony YCBCR420_10LE chroma format for EVC stream + switch(sps->chroma_format_idc) { + case 0: /* YCBCR400_10LE */ + av_log(avctx, AV_LOG_ERROR, "YCBCR400_10LE: Not supported chroma format\n"); + avctx->pix_fmt = AV_PIX_FMT_GRAY10LE; + return -1; + case 1: /* YCBCR420_10LE */ + avctx->pix_fmt = AV_PIX_FMT_YUV420P10LE; + break; + case 2: /* YCBCR422_10LE */ + av_log(avctx, AV_LOG_ERROR, "YCBCR422_10LE: Not supported chroma format\n"); + avctx->pix_fmt = AV_PIX_FMT_YUV422P10LE; + return -1; + case 3: /* YCBCR444_10LE */ + av_log(avctx, AV_LOG_ERROR, "YCBCR444_10LE: Not supported chroma format\n"); + avctx->pix_fmt = AV_PIX_FMT_YUV444P10LE; + return -1; + default: + avctx->pix_fmt = AV_PIX_FMT_NONE; + av_log(avctx, AV_LOG_ERROR, "Unknown supported chroma format\n"); + return -1; + } + + // The current implementation of parse_sps function doesn't handle VUI parameters parsing, + // so at the moment it's impossible to initialize has_b_frames and max_b_frames AVCodecContex fields here. + // Currently, initialization of has_b_frames and max_b_frames AVCodecContex fields have been moved to + // libxevd_decode function where we can use xevd_config function being a part of xevd library API + // to get the needed information. + // However, if it will be needed, parse_sps function should be extended to handle VUI parameters parsing + // and the following lines should be used to initialize has_b_frames and max_b_frames fields of the AVCodecContex. + // + // sps->vui_parameters.num_reorder_pics + // if (sps->bitstream_restriction_flag && sps->vui_parameters.num_reorder_pics) { + // avctx->has_b_frames = sps->vui_parameters.num_reorder_pics; + // } + + ev->got_sps = 1; + + } else if (nalu_type == EVC_NUT_PPS) // NAL Unit type: PPS (Video Parameter Set) + ev->got_pps = 1; + else if(nalu_type == EVC_NUT_SEI) // NAL unit type: SEI (Supplemental Enhancement Information) + ev->got_sei = 1; + else if (nalu_type == EVC_NUT_IDR || nalu_type == EVC_NUT_NONIDR) // NAL Unit type: Coded slice of a IDR or non-IDR picture + ev->got_slice++; + else { + av_log(avctx, AV_LOG_ERROR, "Invalid NAL unit type: %d\n", nalu_type); + return -1; + } + + return 0; +} + +/** + * Find the end of the current frame in the bitstream. + * @return the position of the first byte of the next frame, or END_NOT_FOUND + */ +static int evc_find_frame_end(AVCodecParserContext *s, const uint8_t *buf, + int buf_size, AVCodecContext *avctx) +{ + EVCParserContext *ev = s->priv_data; + + if(!ev->to_read) { + int nal_unit_size = 0; + int next = END_NOT_FOUND; + + // This is the case when buffer size is not enough for buffer to store NAL unit length + if(buf_size < EVC_NAL_UNIT_LENGTH_BYTE) { + ev->to_read = EVC_NAL_UNIT_LENGTH_BYTE; + ev->nal_length_size = buf_size; + ev->incomplete_nalu_prefix_read = 1; + + return END_NOT_FOUND; + } + + nal_unit_size = read_nal_unit_length(buf, buf_size, avctx); + ev->nal_length_size = EVC_NAL_UNIT_LENGTH_BYTE; + + next = nal_unit_size + EVC_NAL_UNIT_LENGTH_BYTE; + ev->to_read = next; + if(next < buf_size) + return next; + else + return END_NOT_FOUND; + } else if(ev->to_read > buf_size) + return END_NOT_FOUND; + else { + if(ev->incomplete_nalu_prefix_read == 1) { + EVCParserContext *ev = s->priv_data; + ParseContext *pc = &ev->pc; + uint8_t nalu_len[EVC_NAL_UNIT_LENGTH_BYTE] = {0}; + int nal_unit_size = 0; + + // 1. pc->buffer contains previously read bytes of NALU prefix + // 2. buf contains the rest of NAL unit prefix bytes + // + // ~~~~~~~ + // EXAMPLE + // ~~~~~~~ + // + // In the following example we assumed that the number of already read NAL Unit prefix bytes is equal 1 + // + // ---------- + // pc->buffer -> conatins already read bytes + // ---------- + // __ pc->index == 1 + // | + // V + // ------------------------------------------------------- + // | 0 | 1 | 2 | 3 | 4 | ... | N | + // ------------------------------------------------------- + // | 0x00 | 0xXX | 0xXX | 0xXX | 0xXX | ... | 0xXX | + // ------------------------------------------------------- + // + // ---------- + // buf -> contains newly read bytes + // ---------- + // ------------------------------------------------------- + // | 0 | 1 | 2 | 3 | 4 | ... | N | + // ------------------------------------------------------- + // | 0x00 | 0x00 | 0x3C | 0xXX | 0xXX | ... | 0xXX | + // ------------------------------------------------------- + // + for(int i = 0; i < EVC_NAL_UNIT_LENGTH_BYTE; i++) { + if(i < pc->index) + nalu_len[i] = pc->buffer[i]; + else + nalu_len[i] = buf[i - pc->index]; + } + + // ---------- + // nalu_len + // ---------- + // --------------------------------- + // | 0 | 1 | 2 | 3 | + // --------------------------------- + // | 0x00 | 0x00 | 0x00 | 0x3C | + // --------------------------------- + // | NALU LENGTH | + // --------------------------------- + // NAL Unit lenght = 60 (0x0000003C) + + nal_unit_size = read_nal_unit_length(nalu_len, EVC_NAL_UNIT_LENGTH_BYTE, avctx); + + ev->to_read = nal_unit_size + EVC_NAL_UNIT_LENGTH_BYTE - pc->index; + + ev->incomplete_nalu_prefix_read = 0; + + if(ev->to_read > buf_size) + return END_NOT_FOUND; + else + return ev->to_read; + } + return ev->to_read; + } + + return END_NOT_FOUND; +} + +static int evc_parser_init(AVCodecParserContext *s) +{ + EVCParserContext *ev = s->priv_data; + + ev->got_sps = 0; + ev->got_pps = 0; + ev->got_sei = 0; + ev->got_slice = 0; + ev->nal_length_size = EVC_NAL_UNIT_LENGTH_BYTE; + ev->incomplete_nalu_prefix_read = 0; + + return 0; +} + +static int evc_parse(AVCodecParserContext *s, AVCodecContext *avctx, + const uint8_t **poutbuf, int *poutbuf_size, + const uint8_t *buf, int buf_size) +{ + int next; + EVCParserContext *ev = s->priv_data; + ParseContext *pc = &ev->pc; + int is_dummy_buf = !buf_size; + const uint8_t *dummy_buf = buf; + + if (s->flags & PARSER_FLAG_COMPLETE_FRAMES) + next = buf_size; + else { + next = evc_find_frame_end(s, buf, buf_size, avctx); + if (ff_combine_frame(pc, next, &buf, &buf_size) < 0) { + *poutbuf = NULL; + *poutbuf_size = 0; + ev->to_read -= buf_size; + return buf_size; + } + } + + is_dummy_buf &= (dummy_buf == buf); + + if (!is_dummy_buf) + parse_nal_units(s, buf, buf_size, avctx); + + *poutbuf = buf; + *poutbuf_size = buf_size; + ev->to_read -= next; + + return next; +} + +const AVCodecParser ff_evc_parser = { + .codec_ids = { AV_CODEC_ID_EVC }, + .priv_data_size = sizeof(EVCParserContext), + .parser_init = evc_parser_init, + .parser_parse = evc_parse, + .parser_close = ff_parse_close, +}; diff --git a/libavcodec/libxevd.c b/libavcodec/libxevd.c new file mode 100644 index 0000000000..8ba2d89cbd --- /dev/null +++ b/libavcodec/libxevd.c @@ -0,0 +1,436 @@ +/* + * libxevd decoder + * EVC (MPEG-5 Essential Video Coding) decoding using XEVD MPEG-5 EVC decoder library + * + * Copyright (C) 2021 Dawid Kozinski <d.kozinski@samsung.com> + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include <float.h> +#include <stdlib.h> + +#include <xevd.h> + +#include "libavutil/internal.h" +#include "libavutil/common.h" +#include "libavutil/opt.h" +#include "libavutil/pixdesc.h" +#include "libavutil/pixfmt.h" +#include "libavutil/imgutils.h" +#include "libavutil/cpu.h" + +#include "avcodec.h" +#include "internal.h" +#include "packet_internal.h" +#include "codec_internal.h" + +#define XEVD_PARAM_BAD_NAME -1 +#define XEVD_PARAM_BAD_VALUE -2 + +#define EVC_NAL_HEADER_SIZE 2 /* byte */ + +/** + * The structure stores all the states associated with the instance of Xeve MPEG-5 EVC decoder + */ +typedef struct XevdContext { + const AVClass *class; + + XEVD id; // XEVD instance identifier @see xevd.h + XEVD_CDSC cdsc; // decoding parameters @see xevd.h + + int decod_frames; // number of decoded frames + int packet_count; // number of packets created by decoder +} XevdContext; + +/** + * The function returns a pointer to a variable of type XEVD_CDSC. + * XEVD_CDSC contains all decoder parameters that should be initialized before its use. + * + * The field values of the XEVD_CDSC structure are populated based on: + * - the corresponding field values of the AvCodecConetxt structure, + * - the xevd decoder specific option values, + * (the full list of options available for the xevd encoder is displayed after executing the command ./ffmpeg --help decoder = libxevd) + * - and the xevd encoder options specified as a list of key-value pairs following the xevd-params option + * + * Order of input processing and populating the XEVD_CDSC structure + * 1. first, the corresponding fields of the AVCodecContext structure are processed, (i.e -threads 4) + * 2. then xevd-specific options are added as AVOption to the xevd AVCodec implementation (i.e -threads 3) + * 3. finally, the options specified after the xevd-params option as the parameter list of type key value are processed (i.e -xevd-params "m=2") + * + * Some options can be set in different ways. In this case, please follow the above-mentioned order of processing. + * The most recent assignments overwrite the previous values. + * + * @param[in] avctx codec context + * @param[out] cdsc contains all encoder parameters that should be initialized before its use + * + * @return 0 on success, negative error code on failure + */ +static int get_conf(AVCodecContext *avctx, XEVD_CDSC *cdsc) +{ + int cpu_count = av_cpu_count(); + + /* clear XEVS_CDSC structure */ + memset(cdsc, 0, sizeof(XEVD_CDSC)); + + /* init XEVD_CDSC */ + if (avctx->thread_count <= 0) + cdsc->threads = (cpu_count < XEVD_MAX_TASK_CNT) ? cpu_count : XEVD_MAX_TASK_CNT; + else if (avctx->thread_count > XEVD_MAX_TASK_CNT) + cdsc->threads = XEVD_MAX_TASK_CNT; + else + cdsc->threads = avctx->thread_count; + + return 0; +} + +/** + * Read NAL unit length + * @param bs input data (bitstream) + * @return the length of NAL unit on success, 0 value on failure + */ +static uint32_t read_nal_unit_length(const uint8_t *bs, int bs_size, AVCodecContext *avctx) +{ + uint32_t len = 0; + XEVD_INFO info; + int ret; + + if (bs_size == XEVD_NAL_UNIT_LENGTH_BYTE) { + ret = xevd_info((void *)bs, XEVD_NAL_UNIT_LENGTH_BYTE, 1, &info); + if (XEVD_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Cannot get bitstream information\n"); + return 0; + } + len = info.nalu_len; + if (len == 0) { + av_log(avctx, AV_LOG_ERROR, "Invalid bitstream size! [%d]\n", bs_size); + return 0; + } + } + + return len; +} + +/** + * @param[in] xectx the structure that stores all the state associated with the instance of Xeve MPEG-5 EVC decoder + * @param[out] avctx codec context + * @return 0 on success, negative value on failure + */ +static int export_stream_params(const XevdContext *xectx, AVCodecContext *avctx) +{ + int ret; + int size; + int color_space; + + avctx->pix_fmt = AV_PIX_FMT_YUV420P10; + + size = 4; + ret = xevd_config(xectx->id, XEVD_CFG_GET_CODED_WIDTH, &avctx->coded_width, &size); + if (XEVD_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to get coded_width\n"); + return AVERROR_EXTERNAL; + } + + ret = xevd_config(xectx->id, XEVD_CFG_GET_CODED_HEIGHT, &avctx->coded_height, &size); + if (XEVD_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to get coded_height\n"); + return AVERROR_EXTERNAL; + } + + ret = xevd_config(xectx->id, XEVD_CFG_GET_WIDTH, &avctx->width, &size); + if (XEVD_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to get width\n"); + return AVERROR_EXTERNAL; + } + + ret = xevd_config(xectx->id, XEVD_CFG_GET_HEIGHT, &avctx->height, &size); + if (XEVD_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to get height\n"); + return AVERROR_EXTERNAL; + } + + ret = xevd_config(xectx->id, XEVD_CFG_GET_COLOR_SPACE, &color_space, &size); + if (XEVD_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to get color_space\n"); + return AVERROR_EXTERNAL; + } + switch(color_space) { + case XEVD_CS_YCBCR400_10LE: + avctx->pix_fmt = AV_PIX_FMT_GRAY10LE; + break; + case XEVD_CS_YCBCR420_10LE: + avctx->pix_fmt = AV_PIX_FMT_YUV420P10LE; + break; + case XEVD_CS_YCBCR422_10LE: + avctx->pix_fmt = AV_PIX_FMT_YUV422P10LE; + break; + case XEVD_CS_YCBCR444_10LE: + avctx->pix_fmt = AV_PIX_FMT_YUV444P10LE; + break; + default: + av_log(avctx, AV_LOG_ERROR, "Unknown color space\n"); + avctx->pix_fmt = AV_PIX_FMT_NONE; + return AVERROR_INVALIDDATA; + } + + // the function returns sps->num_reorder_pics + ret = xevd_config(xectx->id, XEVD_CFG_GET_MAX_CODING_DELAY, &avctx->max_b_frames, &size); + if (XEVD_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to get max_coding_delay\n"); + return AVERROR_EXTERNAL; + } + + avctx->has_b_frames = (avctx->max_b_frames) ? 1 : 0; + + avctx->color_primaries = AVCOL_PRI_UNSPECIFIED; + avctx->color_trc = AVCOL_TRC_UNSPECIFIED; + avctx->colorspace = AVCOL_SPC_UNSPECIFIED; + + return 0; +} + +/** + * Initialize decoder + * Create a decoder instance and allocate all the needed resources + * + * @param avctx codec context + * @return 0 on success, negative error code on failure + */ +static av_cold int libxevd_init(AVCodecContext *avctx) +{ + XevdContext *xectx = avctx->priv_data; + int ret = 0; + XEVD_CDSC *cdsc = &(xectx->cdsc); + + /* read configurations and set values for created descriptor (XEVD_CDSC) */ + ret = get_conf(avctx, cdsc); + if (ret != 0) { + av_log(avctx, AV_LOG_ERROR, "Cannot get configuration\n"); + return AVERROR_INVALIDDATA; + } + + /* create decoder */ + xectx->id = xevd_create(&(xectx->cdsc), NULL); + if (xectx->id == NULL) { + av_log(avctx, AV_LOG_ERROR, "Cannot create XEVD encoder\n"); + return AVERROR_EXTERNAL; + } + + xectx->packet_count = 0; + xectx->decod_frames = 0; + + return 0; +} + +/** + * Decode picture + * + * @param avctx codec context + * @param[out] frame decoded frame + * @param[out] got_frame_ptr decoder sets to 0 or 1 to indicate that a + * non-empty frame or subtitle was returned in + * outdata. + * @param[in] avpkt AVPacket containing encoded data to be decoded + * + * @return amount of bytes read from the packet on success, negative error + * code on failure + */ +static int libxevd_decode(struct AVCodecContext *avctx, struct AVFrame *frame, int *got_frame_ptr, AVPacket *avpkt) +{ + XevdContext *xectx = NULL; + XEVD_IMGB *imgb = NULL; + XEVD_STAT stat; + XEVD_BITB bitb; + int xevd_ret, nalu_size, bs_read_pos; + int ret = 0; + + if (avctx == NULL) { + av_log(avctx, AV_LOG_ERROR, "Invalid input parameter: AVCodecContext\n"); + return AVERROR(EINVAL); + } + xectx = avctx->priv_data; + if (xectx == NULL) { + av_log(avctx, AV_LOG_ERROR, "Invalid XEVD context\n"); + return AVERROR(EINVAL); + } + + if (avpkt->size > 0) { + bs_read_pos = 0; + imgb = NULL; + while(avpkt->size > (bs_read_pos + XEVD_NAL_UNIT_LENGTH_BYTE)) { + + memset(&stat, 0, sizeof(XEVD_STAT)); + memset(&bitb, 0, sizeof(XEVD_BITB)); + + nalu_size = read_nal_unit_length(avpkt->data + bs_read_pos, XEVD_NAL_UNIT_LENGTH_BYTE, avctx); + if (nalu_size == 0) { + av_log(avctx, AV_LOG_ERROR, "Invalid bitstream\n"); + ret = AVERROR_INVALIDDATA; + goto ERR; + } + bs_read_pos += XEVD_NAL_UNIT_LENGTH_BYTE; + + bitb.addr = avpkt->data + bs_read_pos; + bitb.ssize = nalu_size; + + /* main decoding block */ + xevd_ret = xevd_decode(xectx->id, &bitb, &stat); + if (XEVD_FAILED(xevd_ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to decode bitstream\n"); + ret = AVERROR_EXTERNAL; + goto ERR; + } + + bs_read_pos += nalu_size; + + if (stat.nalu_type == XEVD_NUT_SPS) { // EVC stream parameters changed + if ((ret = export_stream_params(xectx, avctx)) != 0) + goto ERR; + } + + if (stat.read != nalu_size) + av_log(avctx, AV_LOG_INFO, "Different reading of bitstream (in:%d, read:%d)\n,", nalu_size, stat.read); + if (stat.fnum >= 0) { + // already has a decoded image + if (imgb) { + // xevd_pull uses pool of objects of type XEVD_IMGB. + // The pool size is equal MAX_PB_SIZE (26), so release object when it is no more needed + imgb->release(imgb); + imgb = NULL; + } + xevd_ret = xevd_pull(xectx->id, &imgb); + if (XEVD_FAILED(xevd_ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to pull the decoded image (xevd error code: %d, frame#=%d)\n", xevd_ret, stat.fnum); + ret = AVERROR_EXTERNAL; + goto ERR; + } else if (xevd_ret == XEVD_OK_FRM_DELAYED) { + if (imgb) { + // xevd_pull uses pool of objects of type XEVD_IMGB. + // The pool size is equal MAX_PB_SIZE (26), so release object when it is no more needed + imgb->release(imgb); + imgb = NULL; + } + } + } + } + } else { // bumping + xevd_ret = xevd_pull(xectx->id, &(imgb)); + if (xevd_ret == XEVD_ERR_UNEXPECTED) { // bumping process completed + *got_frame_ptr = 0; + return 0; + } else if (XEVD_FAILED(xevd_ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to pull the decoded image (xevd error code: %d)\n", xevd_ret); + ret = AVERROR_EXTERNAL; + goto ERR; + } + } + + if (imgb) { + // @todo supports other color space and bit depth + if (imgb->cs != XEVD_CS_YCBCR420_10LE) { + av_log(avctx, AV_LOG_ERROR, "Not supported pixel format: %s\n", av_get_pix_fmt_name(avctx->pix_fmt)); + ret = AVERROR_INVALIDDATA; + goto ERR; + } + + if (imgb->w[0] != avctx->width || imgb->h[0] != avctx->height) { // stream resolution changed + if (ff_set_dimensions(avctx, imgb->w[0], imgb->h[0]) < 0) { + av_log(avctx, AV_LOG_ERROR, "Cannot set new dimension\n"); + ret = AVERROR_INVALIDDATA; + goto ERR; + } + } + + frame->coded_picture_number++; + frame->display_picture_number++; + frame->format = AV_PIX_FMT_YUV420P10LE; + + if (ff_get_buffer(avctx, frame, 0) < 0) { + av_log(avctx, AV_LOG_ERROR, "Cannot get AV buffer\n"); + ret = AVERROR(ENOMEM); + goto ERR; + } + + frame->pts = avpkt->pts; + + av_image_copy(frame->data, frame->linesize, (const uint8_t **)imgb->a, + imgb->s, avctx->pix_fmt, + imgb->w[0], imgb->h[0]); + + xectx->decod_frames++; + *got_frame_ptr = 1; + + // xevd_pull uses pool of objects of type XEVD_IMGB. + // The pool size is equal MAX_PB_SIZE (26), so release object when it is no more needed + imgb->release(imgb); + imgb = NULL; + } else + *got_frame_ptr = 0; + + xectx->packet_count++; + + return avpkt->size; + +ERR: + if (imgb) { + imgb->release(imgb); + imgb = NULL; + } + *got_frame_ptr = 0; + + return ret; +} + +/** + * Destroy decoder + * + * @param avctx codec context + * @return 0 on success + */ +static av_cold int libxevd_close(AVCodecContext *avctx) +{ + XevdContext *xectx = avctx->priv_data; + if (xectx->id) { + xevd_delete(xectx->id); + xectx->id = NULL; + } + + return 0; +} + +#define OFFSET(x) offsetof(XevdContext, x) +#define VD AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_DECODING_PARAM + +static const AVClass libxevd_class = { + .class_name = "libxevd", + .item_name = av_default_item_name, + .version = LIBAVUTIL_VERSION_INT, +}; + +const FFCodec ff_libxevd_decoder = { + .p.name = "evc", + .p.long_name = NULL_IF_CONFIG_SMALL("EVC / MPEG-5 Essential Video Coding (EVC)"), + .p.type = AVMEDIA_TYPE_VIDEO, + .p.id = AV_CODEC_ID_EVC, + .init = libxevd_init, + FF_CODEC_DECODE_CB(libxevd_decode), + .close = libxevd_close, + .priv_data_size = sizeof(XevdContext), + .p.priv_class = &libxevd_class, + .p.capabilities = AV_CODEC_CAP_DELAY | AV_CODEC_CAP_AUTO_THREADS | AV_CODEC_CAP_AVOID_PROBING | AV_CODEC_CAP_DR1, + .p.wrapper_name = "libxevd", +}; diff --git a/libavcodec/libxeve.c b/libavcodec/libxeve.c new file mode 100644 index 0000000000..4808e6d853 --- /dev/null +++ b/libavcodec/libxeve.c @@ -0,0 +1,597 @@ +/* + * libxeve encoder + * EVC (MPEG-5 Essential Video Coding) encoding using XEVE MPEG-5 EVC encoder library + * + * Copyright (C) 2021 Dawid Kozinski <d.kozinski@samsung.com> + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include <float.h> +#include <stdlib.h> + +#include <xeve.h> + +#include "libavutil/internal.h" +#include "libavutil/common.h" +#include "libavutil/opt.h" +#include "libavutil/pixdesc.h" +#include "libavutil/pixfmt.h" +#include "libavutil/time.h" +#include "libavutil/cpu.h" +#include "libavutil/avstring.h" + +#include "avcodec.h" +#include "internal.h" +#include "packet_internal.h" +#include "codec_internal.h" +#include "encode.h" + +#define MAX_BS_BUF (16*1024*1024) + +/** + * Error codes + */ +#define XEVE_PARAM_BAD_NAME -100 +#define XEVE_PARAM_BAD_VALUE -200 + +/** + * Encoder states + * + * STATE_ENCODING - the encoder receives and processes input frames + * STATE_BUMPING - there are no more input frames, however the encoder still processes previously received data + */ +typedef enum State { + STATE_ENCODING, + STATE_BUMPING, +} State; + +/** + * The structure stores all the states associated with the instance of Xeve MPEG-5 EVC encoder + */ +typedef struct XeveContext { + const AVClass *class; + + XEVE id; // XEVE instance identifier + XEVE_CDSC cdsc; // coding parameters i.e profile, width & height of input frame, num of therads, frame rate ... + XEVE_BITB bitb; // bitstream buffer (output) + XEVE_STAT stat; // encoding status (output) + XEVE_IMGB imgb; // image buffer (input) + + State state; // encoder state (skipping, encoding, bumping) + + int profile_id; // encoder profile (main, baseline) + int preset_id; // preset of xeve ( fast, medium, slow, placebo) + int tune_id; // tune of xeve (psnr, zerolatency) + + // variables for rate control modes + int rc_mode; // Rate control mode [ 0(CQP) / 1(ABR) / 2(CRF) ] + int qp; // quantization parameter (QP) [0,51] + int crf; // constant rate factor (CRF) [10,49] + + int hash; // embed picture signature (HASH) for conformance checking in decoding + int sei_info; // embed Supplemental enhancement information while encoding + + int color_format; // input data color format: currently only XEVE_CF_YCBCR420 is supported +} XeveContext; + +/** + * Convert FFmpeg pixel format (AVPixelFormat) to XEVE pre-defined color format + * + * @param[in] av_pix_fmt pixel format (@see https://ffmpeg.org/doxygen/trunk/pixfmt_8h.html#a9a8e335cf3be472042bc9f0cf80cd4c5) + * @param[out] xeve_col_fmt XEVE pre-defined color format (@see xeve.h) + * + * @return 0 on success, negative value on failure + */ +static int libxeve_color_fmt(enum AVPixelFormat av_pix_fmt, int *xeve_col_fmt) +{ + switch (av_pix_fmt) { + case AV_PIX_FMT_YUV420P: + *xeve_col_fmt = XEVE_CF_YCBCR420; + break; + case AV_PIX_FMT_YUV420P10: + *xeve_col_fmt = XEVE_CF_YCBCR420; + break; + default: + *xeve_col_fmt = XEVE_CF_UNKNOWN; + return AVERROR_INVALIDDATA; + } + + return 0; +} + +/** + * Convert FFmpeg pixel format (AVPixelFormat) into XEVE pre-defined color space + * + * @param[in] px_fmt pixel format (@see https://ffmpeg.org/doxygen/trunk/pixfmt_8h.html#a9a8e335cf3be472042bc9f0cf80cd4c5) + * + * @return XEVE pre-defined color space (@see xeve.h) on success, XEVE_CF_UNKNOWN on failure + */ +static int libxeve_color_space(enum AVPixelFormat av_pix_fmt) +{ + /* color space of input image */ + int cs = XEVE_CF_UNKNOWN; + + switch (av_pix_fmt) { + case AV_PIX_FMT_YUV420P: + cs = XEVE_CS_YCBCR420; + break; + case AV_PIX_FMT_YUV420P10: +#if AV_HAVE_BIGENDIAN + cs = XEVE_CS_SET(XEVE_CF_YCBCR420, 10, 1); +#else + cs = XEVE_CS_YCBCR420_10LE; +#endif + + break; + default: + cs = XEVE_CF_UNKNOWN; + break; + } + + return cs; +} + +/** + * The function returns a pointer to the object of the XEVE_CDSC type. + * XEVE_CDSC contains all encoder parameters that should be initialized before the encoder is used. + * + * The field values of the XEVE_CDSC structure are populated based on: + * - the corresponding field values of the AvCodecConetxt structure, + * - the xeve encoder specific option values, + * (the full list of options available for xeve encoder is displayed after executing the command ./ffmpeg --help encoder = libxeve) + * - and the xeve encoder options specified as a list of key-value pairs following the xeve-params option + * + * The order of processing input data and populating the XEVE_CDSC structure + * 1) first, the fields of the AVCodecContext structure corresponding to the provided input options are processed, + * (i.e -pix_fmt yuv420p -s:v 1920x1080 -r 30 -profile:v 0) + * 2) then xeve-specific options added as AVOption to the xeve AVCodec implementation + * (i.e -preset 0) + * + * Keep in mind that, there are options that can be set in different ways. + * In this case, please follow the above-mentioned order of processing. + * The most recent assignments overwrite the previous values. + * + * @param[in] avctx codec context (AVCodecContext) + * @param[out] cdsc contains all Xeve MPEG-5 EVC encoder encoder parameters that should be initialized before the encoder is use + * + * @return 0 on success, negative error code on failure + */ +static int get_conf(AVCodecContext *avctx, XEVE_CDSC *cdsc) +{ + XeveContext *xectx = NULL; + int ret; + + xectx = avctx->priv_data; + + /* initialize xeve_param struct with default values */ + ret = xeve_param_default(&cdsc->param); + if (XEVE_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Cannot set_default parameter\n"); + return AVERROR_EXTERNAL; + } + + /* read options from AVCodecContext */ + if (avctx->width > 0) + cdsc->param.w = avctx->width; + + if (avctx->height > 0) + cdsc->param.h = avctx->height; + + if (avctx->framerate.num > 0) { + // fps can be float number, but xeve API doesn't support it + cdsc->param.fps = (int)(((float)avctx->framerate.num / avctx->framerate.den) + 0.5); + } + + if (avctx->gop_size >= 0) // GOP size (key-frame interval, I-picture period) + cdsc->param.keyint = avctx->gop_size; // 0: only one I-frame at the first time; 1: every frame is coded in I-frame + + if (avctx->max_b_frames == 0 || avctx->max_b_frames == 1 || avctx->max_b_frames == 3 || + avctx->max_b_frames == 7 || avctx->max_b_frames == 15) // number of b-frames + cdsc->param.bframes = avctx->max_b_frames; + else { + av_log(avctx, AV_LOG_ERROR, "Incorrect value for maximum number of B frames: (%d) \n" + "Acceptable values for bf option (maximum number of B frames) are 0,1,3,7 or 15\n", avctx->max_b_frames); + return AVERROR_INVALIDDATA; + } + + if (avctx->level >= 0) + cdsc->param.level_idc = avctx->level; + + if (avctx->rc_buffer_size) // VBV buf size + cdsc->param.vbv_bufsize = (int)(avctx->rc_buffer_size / 1000); + + cdsc->param.rc_type = xectx->rc_mode; + + if (xectx->rc_mode == XEVE_RC_CQP) { + cdsc->param.qp = xectx->qp; + } else if (xectx->rc_mode == XEVE_RC_ABR) { + if (avctx->bit_rate / 1000 > INT_MAX || avctx->rc_max_rate / 1000 > INT_MAX) { + av_log(avctx, AV_LOG_ERROR, "Not supported bitrate bit_rate and rc_max_rate > %d000\n", INT_MAX); + return AVERROR_INVALIDDATA; + } + cdsc->param.bitrate = (int)(avctx->bit_rate / 1000); + } else if (xectx->rc_mode == XEVE_RC_CRF) { + cdsc->param.crf = xectx->crf; + } else { + av_log(avctx, AV_LOG_ERROR, "Not supported rate control type: %d\n", xectx->rc_mode); + return AVERROR_INVALIDDATA; + } + + if (avctx->thread_count <= 0) { + int cpu_count = av_cpu_count(); + cdsc->param.threads = (cpu_count < XEVE_MAX_THREADS) ? cpu_count : XEVE_MAX_THREADS; + } else if (avctx->thread_count > XEVE_MAX_THREADS) + cdsc->param.threads = XEVE_MAX_THREADS; + else + cdsc->param.threads = avctx->thread_count; + + + libxeve_color_fmt(avctx->pix_fmt, &xectx->color_format); + + cdsc->param.cs = XEVE_CS_SET(xectx->color_format, cdsc->param.codec_bit_depth, AV_HAVE_BIGENDIAN); + + cdsc->max_bs_buf_size = MAX_BS_BUF; + + ret = xeve_param_ppt(&cdsc->param, xectx->profile_id, xectx->preset_id, xectx->tune_id); + if (XEVE_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Cannot set profile(%d), preset(%d), tune(%d)\n", xectx->profile_id, xectx->preset_id, xectx->tune_id); + return AVERROR_EXTERNAL; + } + + return 0; +} + +/** + * Set XEVE_CFG_SET_USE_PIC_SIGNATURE for encoder + * + * @param[in] logger context + * @param[in] id XEVE encodec instance identifier + * @param[in] ctx the structure stores all the states associated with the instance of Xeve MPEG-5 EVC encoder + * + * @return 0 on success, negative error code on failure + */ +static int set_extra_config(AVCodecContext *avctx, XEVE id, XeveContext *ctx) +{ + int ret, size; + size = 4; + + // embed SEI messages identifying encoder parameters and command line arguments + // - 0: off\n" + // - 1: emit sei info" + // + // SEI - Supplemental enhancement information contains information + // that is not necessary to decode the samples of coded pictures from VCL NAL units. + // Some SEI message information is required to check bitstream conformance + // and for output timing decoder conformance. + // @see ISO_IEC_23094-1_2020 7.4.3.5 + // @see ISO_IEC_23094-1_2020 Annex D + ret = xeve_config(id, XEVE_CFG_SET_SEI_CMD, &ctx->sei_info, &size); // sei_cmd_info + if (XEVE_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to set config for sei command info messages\n"); + return AVERROR_EXTERNAL; + } + + ret = xeve_config(id, XEVE_CFG_SET_USE_PIC_SIGNATURE, &ctx->hash, &size); + if (XEVE_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to set config for picture signature\n"); + return AVERROR_EXTERNAL; + } + + return 0; +} + +/** + * @brief Switch encoder to bumping mode + * + * @param id XEVE encodec instance identifier + * @return 0 on success, negative error code on failure + */ +static int setup_bumping(XEVE id) +{ + int val = 1; + int size = sizeof(int); + if (XEVE_FAILED(xeve_config(id, XEVE_CFG_SET_FORCE_OUT, (void *)(&val), &size))) + return AVERROR_EXTERNAL; + + return 0; +} + +/** + * @brief Initialize eXtra-fast Essential Video Encoder codec + * Create an encoder instance and allocate all the needed resources + * + * @param avctx codec context + * @return 0 on success, negative error code on failure + */ +static av_cold int libxeve_init(AVCodecContext *avctx) +{ + XeveContext *xectx = avctx->priv_data; + unsigned char *bs_buf = NULL; + int i; + int shift_h = 0; + int shift_v = 0; + int width_chroma = 0; + int height_chroma = 0; + XEVE_IMGB *imgb = NULL; + int ret = 0; + + XEVE_CDSC *cdsc = &(xectx->cdsc); + + /* allocate bitstream buffer */ + bs_buf = av_malloc(MAX_BS_BUF); + if (bs_buf == NULL) { + av_log(avctx, AV_LOG_ERROR, "Cannot allocate bitstream buffer\n"); + return AVERROR(ENOMEM); + } + xectx->bitb.addr = bs_buf; + xectx->bitb.bsize = MAX_BS_BUF; + + /* read configurations and set values for created descriptor (XEVE_CDSC) */ + if ((ret = get_conf(avctx, cdsc)) != 0) { + av_log(avctx, AV_LOG_ERROR, "Cannot get configuration\n"); + return AVERROR(EINVAL); + } + + if ((ret = xeve_param_check(&cdsc->param)) != 0) { + av_log(avctx, AV_LOG_ERROR, "Invalid configuration\n"); + return AVERROR(EINVAL); + } + + /* create encoder */ + xectx->id = xeve_create(cdsc, NULL); + if (xectx->id == NULL) { + av_log(avctx, AV_LOG_ERROR, "Cannot create XEVE encoder\n"); + return AVERROR_EXTERNAL; + } + + if ((ret = set_extra_config(avctx, xectx->id, xectx)) != 0) { + av_log(avctx, AV_LOG_ERROR, "Cannot set extra configuration\n"); + return AVERROR(EINVAL); + } + + if ((ret = av_pix_fmt_get_chroma_sub_sample(avctx->pix_fmt, &shift_h, &shift_v)) != 0) { + av_log(avctx, AV_LOG_ERROR, "Failed to get chroma shift\n"); + return AVERROR(EINVAL); + } + + // Chroma subsampling + // + // YUV format explanation + // shift_h == 1 && shift_v == 1 : YUV420 + // shift_h == 1 && shift_v == 0 : YUV422 + // shift_h == 0 && shift_v == 0 : YUV444 + // + width_chroma = AV_CEIL_RSHIFT(avctx->width, shift_h); + height_chroma = AV_CEIL_RSHIFT(avctx->height, shift_v); + + /* set default values for input image buffer */ + imgb = &xectx->imgb; + imgb->cs = libxeve_color_space(avctx->pix_fmt); + imgb->np = 3; /* only for yuv420p, yuv420ple */ + + for (i = 0; i < imgb->np; i++) + imgb->x[i] = imgb->y[i] = 0; + + imgb->w[0] = imgb->aw[0] = avctx->width; // width luma + imgb->w[1] = imgb->w[2] = imgb->aw[1] = imgb->aw[2] = width_chroma; + imgb->h[0] = imgb->ah[0] = avctx->height; // height luma + imgb->h[1] = imgb->h[2] = imgb->ah[1] = imgb->ah[2] = height_chroma; + + xectx->state = STATE_ENCODING; + + return 0; +} + +/** + * Encode raw data frame into EVC packet + * + * @param[in] avctx codec context + * @param[out] pkt output AVPacket containing encoded data + * @param[in] frame AVFrame containing the raw data to be encoded + * @param[out] got_packet encoder sets to 0 or 1 to indicate that a + * non-empty packet was returned in pkt + * + * @return 0 on success, negative error code on failure + */ +static int libxeve_encode(AVCodecContext *avctx, AVPacket *avpkt, + const AVFrame *frame, int *got_packet) +{ + XeveContext *xectx = avctx->priv_data; + int ret = -1; + + // No more input frames are available but encoder still can have some data in its internal buffer to process + // and some frames to dump. + if (xectx->state == STATE_ENCODING && frame == NULL) { + if (setup_bumping(xectx->id) == 0) + xectx->state = STATE_BUMPING; // Entering bumping process + else { + av_log(avctx, AV_LOG_ERROR, "Failed to setup bumping\n"); + return 0; + } + } + + if (xectx->state == STATE_ENCODING) { + int i; + XEVE_IMGB *imgb = NULL; + + imgb = &xectx->imgb; + + for (i = 0; i < imgb->np; i++) { + imgb->a[i] = frame->data[i]; + imgb->s[i] = frame->linesize[i]; + } + + imgb->ts[0] = frame->pts; + imgb->ts[1] = 0; + + /* push image to encoder */ + ret = xeve_push(xectx->id, imgb); + if (XEVE_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "xeve_push() failed\n"); + return AVERROR_EXTERNAL; + } + } + if (xectx->state == STATE_ENCODING || xectx->state == STATE_BUMPING) { + /* encoding */ + ret = xeve_encode(xectx->id, &(xectx->bitb), &(xectx->stat)); + if (XEVE_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "xeve_encode() failed\n"); + return AVERROR_EXTERNAL; + } + + /* store bitstream */ + if (ret == XEVE_OK_OUT_NOT_AVAILABLE) { // Return OK but picture is not available yet + *got_packet = 0; + return 0; + } else if (ret == XEVE_OK) { + int av_pic_type; + + if (xectx->stat.write > 0) { + + ret = ff_get_encode_buffer(avctx, avpkt, xectx->stat.write, 0); + if (ret < 0) + return ret; + + memcpy(avpkt->data, xectx->bitb.addr, xectx->stat.write); + + avpkt->pts = xectx->bitb.ts[0]; + avpkt->dts = xectx->bitb.ts[1]; + + switch(xectx->stat.stype) { + case XEVE_ST_I: + av_pic_type = AV_PICTURE_TYPE_I; + avpkt->flags |= AV_PKT_FLAG_KEY; + break; + case XEVE_ST_P: + av_pic_type = AV_PICTURE_TYPE_P; + break; + case XEVE_ST_B: + av_pic_type = AV_PICTURE_TYPE_B; + break; + case XEVE_ST_UNKNOWN: + av_log(avctx, AV_LOG_ERROR, "Unknown slice type\n"); + return AVERROR_INVALIDDATA; + } + + ff_side_data_set_encoder_stats(avpkt, xectx->stat.qp * FF_QP2LAMBDA, NULL, 0, av_pic_type); + + *got_packet = 1; + } + } else if (ret == XEVE_OK_NO_MORE_FRM) { + // Return OK but no more frames + return 0; + } else { + av_log(avctx, AV_LOG_ERROR, "Invalid return value: %d\n", ret); + return AVERROR_EXTERNAL; + } + } else { + av_log(avctx, AV_LOG_ERROR, "Udefined encoder state\n"); + return AVERROR_INVALIDDATA; + } + + return 0; +} + +/** + * Destroy the encoder and release all the allocated resources + * + * @param avctx codec context + * @return 0 on success, negative error code on failure + */ +static av_cold int libxeve_close(AVCodecContext *avctx) +{ + XeveContext *xectx = avctx->priv_data; + + xeve_delete(xectx->id); + + av_free(xectx->bitb.addr); /* release bitstream buffer */ + + return 0; +} + +#define OFFSET(x) offsetof(XeveContext, x) +#define VE AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_ENCODING_PARAM + +static const enum AVPixelFormat supported_pixel_formats[] = { + AV_PIX_FMT_YUV420P, + AV_PIX_FMT_YUV420P10, + AV_PIX_FMT_NONE +}; + +// Example of using: ./ffmpeg -xeve-params "m=2:q=17" +// Consider using following options (./ffmpeg --help encoder=libxeve) +// +static const AVOption libxeve_options[] = { + { "preset", "Encoding preset for setting encoding speed", OFFSET(preset_id), AV_OPT_TYPE_INT, { .i64 = XEVE_PRESET_MEDIUM }, XEVE_PRESET_DEFAULT, XEVE_PRESET_PLACEBO, VE, "preset" }, + { "default", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_PRESET_DEFAULT }, INT_MIN, INT_MAX, VE, "preset" }, + { "fast", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_PRESET_FAST }, INT_MIN, INT_MAX, VE, "preset" }, + { "medium", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_PRESET_MEDIUM }, INT_MIN, INT_MAX, VE, "preset" }, + { "slow", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_PRESET_SLOW }, INT_MIN, INT_MAX, VE, "preset" }, + { "placebo", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_PRESET_PLACEBO }, INT_MIN, INT_MAX, VE, "preset" }, + { "tune", "Tuning parameter for special purpose operation", OFFSET(tune_id), AV_OPT_TYPE_INT, { .i64 = XEVE_TUNE_NONE }, XEVE_TUNE_NONE, XEVE_TUNE_PSNR, VE, "tune"}, + { "none", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_TUNE_NONE }, INT_MIN, INT_MAX, VE, "tune" }, + { "zerolatency", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_TUNE_ZEROLATENCY }, INT_MIN, INT_MAX, VE, "tune" }, + { "psnr", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_TUNE_PSNR }, INT_MIN, INT_MAX, VE, "tune" }, + { "profile", "Encoding profile", OFFSET(profile_id), AV_OPT_TYPE_INT, { .i64 = XEVE_PROFILE_BASELINE }, XEVE_PROFILE_BASELINE, XEVE_PROFILE_MAIN, VE, "profile" }, + { "baseline", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_PROFILE_BASELINE }, INT_MIN, INT_MAX, VE, "profile" }, + { "main", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_PROFILE_MAIN }, INT_MIN, INT_MAX, VE, "profile" }, + { "rc_mode", "Rate control mode", OFFSET(rc_mode), AV_OPT_TYPE_INT, { .i64 = XEVE_RC_CQP }, XEVE_RC_CQP, XEVE_RC_CRF, VE, "rc_mode" }, + { "CQP", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_RC_CQP }, INT_MIN, INT_MAX, VE, "rc_mode" }, + { "ABR", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_RC_ABR }, INT_MIN, INT_MAX, VE, "rc_mode" }, + { "CRF", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_RC_CRF }, INT_MIN, INT_MAX, VE, "rc_mode" }, + { "qp", "Quantization parameter value for CQP rate control mode", OFFSET(qp), AV_OPT_TYPE_INT, { .i64 = 32 }, 0, 51, VE }, + { "crf", "Constant rate factor value for CRF rate control mode", OFFSET(crf), AV_OPT_TYPE_INT, { .i64 = 32 }, 10, 49, VE }, + { "hash", "Embed picture signature (HASH) for conformance checking in decoding", OFFSET(hash), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, 1, VE }, + { "sei_info", "Embed SEI messages identifying encoder parameters and command line arguments", OFFSET(sei_info), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, 1, VE }, + { NULL } +}; + +static const AVClass libxeve_class = { + .class_name = "libxeve", + .item_name = av_default_item_name, + .option = libxeve_options, + .version = LIBAVUTIL_VERSION_INT, +}; + +/** + * libavcodec generic global options, which can be set on all the encoders and decoders + * @see https://www.ffmpeg.org/ffmpeg-codecs.html#Codec-Options + */ +static const FFCodecDefault libxeve_defaults[] = { + { "b", "0" }, // bitrate in terms of kilo-bits per second + { "g", "0" }, // gop_size (key-frame interval 0: only one I-frame at the first time; 1: every frame is coded in I-frame) + { "bf", "15"}, // maximum number of B frames (0: no B-frames, 1,3,7,15) + { "threads", "0"}, // number of threads to be used (0: automatically select the number of threads to set) + { NULL }, +}; + +const FFCodec ff_libxeve_encoder = { + .p.name = "libxeve", + .p.long_name = NULL_IF_CONFIG_SMALL("libxeve MPEG-5 EVC"), + .p.type = AVMEDIA_TYPE_VIDEO, + .p.id = AV_CODEC_ID_EVC, + .init = libxeve_init, + FF_CODEC_ENCODE_CB(libxeve_encode), + .close = libxeve_close, + .priv_data_size = sizeof(XeveContext), + .p.priv_class = &libxeve_class, + .defaults = libxeve_defaults, + .p.capabilities = FF_CODEC_CAP_INIT_CLEANUP | AV_CODEC_CAP_DELAY | AV_CODEC_CAP_AUTO_THREADS | AV_CODEC_CAP_DR1, + .p.wrapper_name = "libxeve", + .p.pix_fmts = supported_pixel_formats, +}; diff --git a/libavcodec/parsers.c b/libavcodec/parsers.c index a8d52af6cb..ca00c63176 100644 --- a/libavcodec/parsers.c +++ b/libavcodec/parsers.c @@ -41,6 +41,7 @@ extern const AVCodecParser ff_dvaudio_parser; extern const AVCodecParser ff_dvbsub_parser; extern const AVCodecParser ff_dvdsub_parser; extern const AVCodecParser ff_dvd_nav_parser; +extern const AVCodecParser ff_evc_parser; extern const AVCodecParser ff_flac_parser; extern const AVCodecParser ff_g723_1_parser; extern const AVCodecParser ff_g729_parser; diff --git a/libavcodec/profiles.c b/libavcodec/profiles.c index 7af7fbeb13..a31244e0db 100644 --- a/libavcodec/profiles.c +++ b/libavcodec/profiles.c @@ -181,4 +181,10 @@ const AVProfile ff_arib_caption_profiles[] = { { FF_PROFILE_UNKNOWN } }; +const AVProfile ff_evc_profiles[] = { + { FF_PROFILE_EVC_BASELINE, "Baseline" }, + { FF_PROFILE_EVC_MAIN, "Main" }, + { FF_PROFILE_UNKNOWN }, +}; + #endif /* !CONFIG_SMALL */ diff --git a/libavcodec/profiles.h b/libavcodec/profiles.h index 41a19aa9ad..cf92b5f126 100644 --- a/libavcodec/profiles.h +++ b/libavcodec/profiles.h @@ -72,5 +72,6 @@ extern const AVProfile ff_sbc_profiles[]; extern const AVProfile ff_prores_profiles[]; extern const AVProfile ff_mjpeg_profiles[]; extern const AVProfile ff_arib_caption_profiles[]; +extern const AVProfile ff_evc_profiles[]; #endif /* AVCODEC_PROFILES_H */ diff --git a/libavcodec/version.h b/libavcodec/version.h index 19f3f4a272..4a1205c718 100644 --- a/libavcodec/version.h +++ b/libavcodec/version.h @@ -29,7 +29,7 @@ #include "version_major.h" -#define LIBAVCODEC_VERSION_MINOR 40 +#define LIBAVCODEC_VERSION_MINOR 41 #define LIBAVCODEC_VERSION_MICRO 100 #define LIBAVCODEC_VERSION_INT AV_VERSION_INT(LIBAVCODEC_VERSION_MAJOR, \ -- 2.17.1 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [FFmpeg-devel] [PATCH 1/2] Provided support for MPEG-5 EVC (Essential Video Coding) codec 2022-08-01 9:28 ` Dawid Kozinski @ 2022-08-01 12:29 ` James Almer 2022-08-08 12:39 ` Dawid Kozinski/Robot SDK (PLT) /SRPOL/Staff Engineer/삼성전자 2022-08-01 15:09 ` James Almer 1 sibling, 1 reply; 13+ messages in thread From: James Almer @ 2022-08-01 12:29 UTC (permalink / raw) To: ffmpeg-devel On 8/1/2022 6:28 AM, Dawid Kozinski wrote: > diff --git a/libavcodec/evc_parser.c b/libavcodec/evc_parser.c > new file mode 100644 > index 0000000000..b2136a6b45 > --- /dev/null > +++ b/libavcodec/evc_parser.c > @@ -0,0 +1,526 @@ > +/* > + * EVC format parser > + * > + * Copyright (C) 2021 Dawid Kozinski<d.kozinski@samsung.com> > + * > + * This file is part of FFmpeg. > + * > + * FFmpeg is free software; you can redistribute it and/or > + * modify it under the terms of the GNU Lesser General Public > + * License as published by the Free Software Foundation; either > + * version 2.1 of the License, or (at your option) any later version. > + * > + * FFmpeg is distributed in the hope that it will be useful, > + * but WITHOUT ANY WARRANTY; without even the implied warranty of > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > + * Lesser General Public License for more details. > + * > + * You should have received a copy of the GNU Lesser General Public > + * License along with FFmpeg; if not, write to the Free Software > + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA > + */ > + > +#include <stdint.h> > + > +#include "libavutil/common.h" > + > +#include "parser.h" > +#include "golomb.h" > + > +// The length field that indicates the length in bytes of the following NAL unit is configured to be of 4 bytes > +#define EVC_NAL_UNIT_LENGTH_BYTE (4) /* byte */ > + > +#define EVC_NAL_HEADER_SIZE (2) /* byte */ > +#define MAX_SPS_CNT (16) /* defined value in EVC standard */ > + > +// NALU types > +// @see ISO_IEC_23094-1_2020 7.4.2.2 NAL unit header semantics > +// > +#define EVC_NUT_NONIDR (0) /* Coded slice of a non-IDR picture */ > +#define EVC_NUT_IDR (1) /* Coded slice of an IDR picture */ > +#define EVC_NUT_SPS (24) /* Sequence parameter set */ > +#define EVC_NUT_PPS (25) /* Picture paremeter set */ > +#define EVC_NUT_APS (26) /* Adaptation parameter set */ > +#define EVC_NUT_FD (27) /* Filler data */ > +#define EVC_NUT_SEI (28) /* Supplemental enhancement information */ > + > +// The sturcture reflects SPS RBSP(raw byte sequence payload) layout > +// @see ISO_IEC_23094-1 section 7.3.2.1 > +// > +// The following descriptors specify the parsing process of each element > +// u(n) - unsigned integer using n bits > +// ue(v) - unsigned integer 0-th order Exp_Golomb-coded syntax element with the left bit first > +typedef struct EVCParserSPS { > + int sps_seq_parameter_set_id; // ue(v) > + int profile_idc; // u(8) > + int level_idc; // u(8) > + int toolset_idc_h; // u(32) > + int toolset_idc_l; // u(32) > + int chroma_format_idc; // ue(v) > + int pic_width_in_luma_samples; // ue(v) > + int pic_height_in_luma_samples; // ue(v) > + int bit_depth_luma_minus8; // ue(v) > + int bit_depth_chroma_minus8; // ue(v) > + > + int sps_btt_flag; // u(1) > + int log2_ctu_size_minus5; // ue(v) > + int log2_min_cb_size_minus2; // ue(v) > + int log2_diff_ctu_max_14_cb_size; // ue(v) > + int log2_diff_ctu_max_tt_cb_size; // ue(v) > + int log2_diff_min_cb_min_tt_cb_size_minus2; // ue(v) > + > + int sps_suco_flag; // u(1) > + int log2_diff_ctu_size_max_suco_cb_size; // ue(v) > + int log2_diff_max_suco_min_suco_cb_size; // ue(v) > + > + int sps_admvp_flag; // u(1) > + int sps_affine_flag; // u(1) > + int sps_amvr_flag; // u(1) > + int sps_dmvr_flag; // u(1) > + int sps_mmvd_flag; // u(1) > + int sps_hmvp_flag; // u(1) > + > + int sps_eipd_flag; // u(1) > + int sps_ibc_flag; // u(1) > + int log2_max_ibc_cand_size_minus2; // ue(v) > + > + int sps_cm_init_flag; // u(1) > + int sps_adcc_flag; // u(1) > + > + int sps_iqt_flag; // u(1) > + int sps_ats_flag; // u(1) > + > + int sps_addb_flag; // u(1) > + int sps_alf_flag; // u(1) > + int sps_htdf_flag; // u(1) > + int sps_rpl_flag; // u(1) > + int sps_pocs_flag; // u(1) > + int sps_dquant_flag; // u(1) > + int sps_dra_flag; // u(1) > + > + int log2_max_pic_order_cnt_lsb_minus4; // ue(v) > + int log2_sub_gop_length; // ue(v) > + int log2_ref_pic_gap_length; // ue(v) > + > + int max_num_tid0_ref_pics; // ue(v) > + > + int sps_max_dec_pic_buffering_minus1; // ue(v) > + int long_term_ref_pic_flag; // u(1) > + int rpl1_same_as_rpl0_flag; // u(1) > + int num_ref_pic_list_in_sps[2]; // ue(v) > + > + int picture_cropping_flag; // u(1) > + int picture_crop_left_offset; // ue(v) > + int picture_crop_right_offset; // ue(v) > + int picture_crop_top_offset; // ue(v) > + int picture_crop_bottom_offset; // ue(v) > + > +} EVCParserSPS; > + > +typedef struct EVCParserContext { > + ParseContext pc; > + EVCParserSPS sps[MAX_SPS_CNT]; > + int is_avc; > + int nal_length_size; > + int to_read; > + int incomplete_nalu_prefix_read; // The flag is set to 1 when an incomplete NAL unit prefix has been read > + > + int got_sps; > + int got_pps; > + int got_sei; > + int got_slice; > +} EVCParserContext; > + > +static int get_nalu_type(const uint8_t *bits, int bits_size, AVCodecContext *avctx) > +{ > + int unit_type_plus1 = 0; > + > + if(bits_size >= EVC_NAL_HEADER_SIZE) { > + unsigned char *p = (unsigned char *)bits; > + // forbidden_zero_bit > + if ((p[0] & 0x80) != 0) { > + av_log(avctx, AV_LOG_ERROR, "Cannot get bitstream information. Malformed bitstream.\n"); > + return -1; > + } > + > + // nal_unit_type > + unit_type_plus1 = (p[0] >> 1) & 0x3F; > + } > + > + return unit_type_plus1 - 1; > +} > + > +static uint32_t read_nal_unit_length(const uint8_t *bits, int bits_size, AVCodecContext *avctx) > +{ > + uint32_t nalu_len = 0; > + > + if(bits_size >= EVC_NAL_UNIT_LENGTH_BYTE) { > + > + int t = 0; > + unsigned char *p = (unsigned char *)bits; > + > + for(int i = 0; i < EVC_NAL_UNIT_LENGTH_BYTE; i++) > + t = (t << 8) | p[i]; > + > + nalu_len = t; > + if(nalu_len == 0) { > + av_log(avctx, AV_LOG_ERROR, "Invalid bitstream size!\n"); > + return 0; > + } > + } > + > + return nalu_len; > +} > + > +// @see ISO_IEC_23094-1 (7.3.2.1 SPS RBSP syntax) > +static EVCParserSPS *parse_sps(const uint8_t *bs, int bs_size, EVCParserContext *ev) > +{ > + GetBitContext gb; > + EVCParserSPS *sps; > + int sps_seq_parameter_set_id; > + > + if(init_get_bits8(&gb, bs, bs_size) < 0) > + return NULL; > + > + sps_seq_parameter_set_id = get_ue_golomb(&gb); > + > + if(sps_seq_parameter_set_id >= MAX_SPS_CNT) > + return NULL; > + > + sps = &ev->sps[sps_seq_parameter_set_id]; > + sps->sps_seq_parameter_set_id = sps_seq_parameter_set_id; > + > + // the Baseline profile is indicated by profile_idc eqal to 0 > + // the Main profile is indicated by profile_idc eqal to 1 > + sps->profile_idc = get_bits(&gb, 8); > + > + sps->level_idc = get_bits(&gb, 8); > + > + skip_bits_long(&gb, 32); /* skip toolset_idc_h */ > + skip_bits_long(&gb, 32); /* skip toolset_idc_l */ > + > + // 0 - monochrome > + // 1 - 4:2:0 > + // 2 - 4:2:2 > + // 3 - 4:4:4 > + sps->chroma_format_idc = get_ue_golomb(&gb); > + > + sps->pic_width_in_luma_samples = get_ue_golomb(&gb); > + sps->pic_height_in_luma_samples = get_ue_golomb(&gb); > + > + sps->bit_depth_luma_minus8 = get_ue_golomb(&gb); > + sps->bit_depth_chroma_minus8 = get_ue_golomb(&gb); > + > + sps->sps_btt_flag = get_bits(&gb, 1); > + if(sps->sps_btt_flag) { > + sps->log2_ctu_size_minus5 = get_ue_golomb(&gb); > + sps->log2_min_cb_size_minus2 = get_ue_golomb(&gb); > + sps->log2_diff_ctu_max_14_cb_size = get_ue_golomb(&gb); > + sps->log2_diff_ctu_max_tt_cb_size = get_ue_golomb(&gb); > + sps->log2_diff_min_cb_min_tt_cb_size_minus2 = get_ue_golomb(&gb); > + } > + > + sps->sps_suco_flag = get_bits(&gb, 1); > + if(sps->sps_suco_flag) { > + sps->log2_diff_ctu_size_max_suco_cb_size = get_ue_golomb(&gb); > + sps->log2_diff_max_suco_min_suco_cb_size = get_ue_golomb(&gb); > + } > + > + sps->sps_admvp_flag = get_bits(&gb, 1); > + if(sps->sps_admvp_flag) { > + sps->sps_affine_flag = get_bits(&gb, 1); > + sps->sps_amvr_flag = get_bits(&gb, 1); > + sps->sps_dmvr_flag = get_bits(&gb, 1); > + sps->sps_mmvd_flag = get_bits(&gb, 1); > + sps->sps_hmvp_flag = get_bits(&gb, 1); > + } > + > + sps->sps_eipd_flag = get_bits(&gb, 1); > + if(sps->sps_eipd_flag) { > + sps->sps_ibc_flag = get_bits(&gb, 1); > + if(sps->sps_ibc_flag) > + sps->log2_max_ibc_cand_size_minus2 = get_ue_golomb(&gb); > + } > + > + sps->sps_cm_init_flag = get_bits(&gb, 1); > + if(sps->sps_cm_init_flag) > + sps->sps_adcc_flag = get_bits(&gb, 1); > + > + sps->sps_iqt_flag = get_bits(&gb, 1); > + if(sps->sps_iqt_flag) > + sps->sps_ats_flag = get_bits(&gb, 1); > + > + sps->sps_addb_flag = get_bits(&gb, 1); > + sps->sps_alf_flag = get_bits(&gb, 1); > + sps->sps_htdf_flag = get_bits(&gb, 1); > + sps->sps_rpl_flag = get_bits(&gb, 1); > + sps->sps_pocs_flag = get_bits(&gb, 1); > + sps->sps_dquant_flag = get_bits(&gb, 1); > + sps->sps_dra_flag = get_bits(&gb, 1); > + > + if(sps->sps_pocs_flag) > + sps->log2_max_pic_order_cnt_lsb_minus4 = get_ue_golomb(&gb); > + > + if(!sps->sps_pocs_flag || !sps->sps_rpl_flag) { > + sps->log2_sub_gop_length = get_ue_golomb(&gb); > + if(sps->log2_sub_gop_length == 0) > + sps->log2_ref_pic_gap_length = get_ue_golomb(&gb); > + } > + > + return sps; > +} > + > +static int parse_nal_units(AVCodecParserContext *s, const uint8_t *bs, > + int bs_size, AVCodecContext *avctx) > +{ > + EVCParserContext *ev = s->priv_data; > + int nalu_type, nalu_size; > + unsigned char *bits = (unsigned char *)bs; > + int bits_size = bs_size; > + > + avctx->codec_id = AV_CODEC_ID_EVC; > + > + nalu_size = read_nal_unit_length(bits, bits_size, avctx); > + if(nalu_size == 0) { > + av_log(avctx, AV_LOG_ERROR, "Invalid NAL unit size: (%d)\n", nalu_size); > + return -1; > + } > + > + bits += EVC_NAL_UNIT_LENGTH_BYTE; > + bits_size -= EVC_NAL_UNIT_LENGTH_BYTE; > + > + nalu_type = get_nalu_type(bits, bits_size, avctx); > + > + bits += EVC_NAL_HEADER_SIZE; > + bits_size -= EVC_NAL_HEADER_SIZE; > + > + if (nalu_type == EVC_NUT_SPS) { // NAL Unit type: SPS (Sequence Parameter Set) > + EVCParserSPS *sps; > + > + sps = parse_sps(bits, bits_size, ev); > + > + avctx->coded_width = sps->pic_width_in_luma_samples; > + avctx->coded_height = sps->pic_height_in_luma_samples; > + avctx->width = sps->pic_width_in_luma_samples; > + avctx->height = sps->pic_height_in_luma_samples; These should be set in the AVCodecParserContext, not the AVCodecContext. > + > + if(sps->profile_idc == 0) avctx->profile = FF_PROFILE_EVC_BASELINE; > + else if (sps->profile_idc == 1) avctx->profile = FF_PROFILE_EVC_MAIN; Setting these two is ok. > + else { > + av_log(avctx, AV_LOG_ERROR, "Not supported profile (%d)\n", sps->profile_idc); > + return -1; Do you really need to abort here? Not recognizing the profile can't possibly mean the bitstream can't be parsed further to fetch other information. > + } > + > + // Currently XEVD decoder supports ony YCBCR420_10LE chroma format for EVC stream > + switch(sps->chroma_format_idc) { > + case 0: /* YCBCR400_10LE */ > + av_log(avctx, AV_LOG_ERROR, "YCBCR400_10LE: Not supported chroma format\n"); > + avctx->pix_fmt = AV_PIX_FMT_GRAY10LE; > + return -1; > + case 1: /* YCBCR420_10LE */ > + avctx->pix_fmt = AV_PIX_FMT_YUV420P10LE; > + break; > + case 2: /* YCBCR422_10LE */ > + av_log(avctx, AV_LOG_ERROR, "YCBCR422_10LE: Not supported chroma format\n"); > + avctx->pix_fmt = AV_PIX_FMT_YUV422P10LE; > + return -1; > + case 3: /* YCBCR444_10LE */ > + av_log(avctx, AV_LOG_ERROR, "YCBCR444_10LE: Not supported chroma format\n"); > + avctx->pix_fmt = AV_PIX_FMT_YUV444P10LE; > + return -1; > + default: > + avctx->pix_fmt = AV_PIX_FMT_NONE; Fill s->format instead of avctx->pix_fmt. > + av_log(avctx, AV_LOG_ERROR, "Unknown supported chroma format\n"); > + return -1; > + } > + > + // The current implementation of parse_sps function doesn't handle VUI parameters parsing, > + // so at the moment it's impossible to initialize has_b_frames and max_b_frames AVCodecContex fields here. > + // Currently, initialization of has_b_frames and max_b_frames AVCodecContex fields have been moved to > + // libxevd_decode function where we can use xevd_config function being a part of xevd library API > + // to get the needed information. > + // However, if it will be needed, parse_sps function should be extended to handle VUI parameters parsing > + // and the following lines should be used to initialize has_b_frames and max_b_frames fields of the AVCodecContex. There's no need no fill any of those field in this parser. What you can fill is avctx->color{space,_primaries,_trc,_range}, so mention that to add VUI parsing at some point. > + // > + // sps->vui_parameters.num_reorder_pics > + // if (sps->bitstream_restriction_flag && sps->vui_parameters.num_reorder_pics) { > + // avctx->has_b_frames = sps->vui_parameters.num_reorder_pics; > + // } > + > + ev->got_sps = 1; > + > + } else if (nalu_type == EVC_NUT_PPS) // NAL Unit type: PPS (Video Parameter Set) > + ev->got_pps = 1; > + else if(nalu_type == EVC_NUT_SEI) // NAL unit type: SEI (Supplemental Enhancement Information) > + ev->got_sei = 1; These got_* variables seem to be write only. > + else if (nalu_type == EVC_NUT_IDR || nalu_type == EVC_NUT_NONIDR) // NAL Unit type: Coded slice of a IDR or non-IDR picture In here you should parse enough data to get the picture type (I, P, B) and set s->pict_type, and based on (i suppose) nalu_type, also set s->key_frame to 1 or 0. At the start of this function, you'd ideally set them to AV_PICTURE_TYPE_NONE and -1 respectively, as "safe" defaults. s->picture_structure should also be always set to AV_PICTURE_STRUCTURE_FRAME, as i assume this codec does not support interlacing. > + ev->got_slice++; > + else { > + av_log(avctx, AV_LOG_ERROR, "Invalid NAL unit type: %d\n", nalu_type); > + return -1; > + } > + > + return 0; > +} > + > +/** > + * Find the end of the current frame in the bitstream. > + * @return the position of the first byte of the next frame, or END_NOT_FOUND > + */ > +static int evc_find_frame_end(AVCodecParserContext *s, const uint8_t *buf, > + int buf_size, AVCodecContext *avctx) > +{ > + EVCParserContext *ev = s->priv_data; > + > + if(!ev->to_read) { > + int nal_unit_size = 0; > + int next = END_NOT_FOUND; > + > + // This is the case when buffer size is not enough for buffer to store NAL unit length > + if(buf_size < EVC_NAL_UNIT_LENGTH_BYTE) { > + ev->to_read = EVC_NAL_UNIT_LENGTH_BYTE; > + ev->nal_length_size = buf_size; > + ev->incomplete_nalu_prefix_read = 1; > + > + return END_NOT_FOUND; > + } > + > + nal_unit_size = read_nal_unit_length(buf, buf_size, avctx); > + ev->nal_length_size = EVC_NAL_UNIT_LENGTH_BYTE; > + > + next = nal_unit_size + EVC_NAL_UNIT_LENGTH_BYTE; > + ev->to_read = next; > + if(next < buf_size) > + return next; > + else > + return END_NOT_FOUND; > + } else if(ev->to_read > buf_size) > + return END_NOT_FOUND; > + else { > + if(ev->incomplete_nalu_prefix_read == 1) { > + EVCParserContext *ev = s->priv_data; > + ParseContext *pc = &ev->pc; > + uint8_t nalu_len[EVC_NAL_UNIT_LENGTH_BYTE] = {0}; > + int nal_unit_size = 0; > + > + // 1. pc->buffer contains previously read bytes of NALU prefix > + // 2. buf contains the rest of NAL unit prefix bytes > + // > + // ~~~~~~~ > + // EXAMPLE > + // ~~~~~~~ > + // > + // In the following example we assumed that the number of already read NAL Unit prefix bytes is equal 1 > + // > + // ---------- > + // pc->buffer -> conatins already read bytes > + // ---------- > + // __ pc->index == 1 > + // | > + // V > + // ------------------------------------------------------- > + // | 0 | 1 | 2 | 3 | 4 | ... | N | > + // ------------------------------------------------------- > + // | 0x00 | 0xXX | 0xXX | 0xXX | 0xXX | ... | 0xXX | > + // ------------------------------------------------------- > + // > + // ---------- > + // buf -> contains newly read bytes > + // ---------- > + // ------------------------------------------------------- > + // | 0 | 1 | 2 | 3 | 4 | ... | N | > + // ------------------------------------------------------- > + // | 0x00 | 0x00 | 0x3C | 0xXX | 0xXX | ... | 0xXX | > + // ------------------------------------------------------- > + // > + for(int i = 0; i < EVC_NAL_UNIT_LENGTH_BYTE; i++) { > + if(i < pc->index) > + nalu_len[i] = pc->buffer[i]; > + else > + nalu_len[i] = buf[i - pc->index]; > + } > + > + // ---------- > + // nalu_len > + // ---------- > + // --------------------------------- > + // | 0 | 1 | 2 | 3 | > + // --------------------------------- > + // | 0x00 | 0x00 | 0x00 | 0x3C | > + // --------------------------------- > + // | NALU LENGTH | > + // --------------------------------- > + // NAL Unit lenght = 60 (0x0000003C) > + > + nal_unit_size = read_nal_unit_length(nalu_len, EVC_NAL_UNIT_LENGTH_BYTE, avctx); > + > + ev->to_read = nal_unit_size + EVC_NAL_UNIT_LENGTH_BYTE - pc->index; > + > + ev->incomplete_nalu_prefix_read = 0; > + > + if(ev->to_read > buf_size) > + return END_NOT_FOUND; > + else > + return ev->to_read; > + } > + return ev->to_read; > + } > + > + return END_NOT_FOUND; > +} > + > +static int evc_parser_init(AVCodecParserContext *s) > +{ > + EVCParserContext *ev = s->priv_data; > + > + ev->got_sps = 0; > + ev->got_pps = 0; > + ev->got_sei = 0; > + ev->got_slice = 0; > + ev->nal_length_size = EVC_NAL_UNIT_LENGTH_BYTE; > + ev->incomplete_nalu_prefix_read = 0; > + > + return 0; > +} > + > +static int evc_parse(AVCodecParserContext *s, AVCodecContext *avctx, > + const uint8_t **poutbuf, int *poutbuf_size, > + const uint8_t *buf, int buf_size) > +{ > + int next; > + EVCParserContext *ev = s->priv_data; > + ParseContext *pc = &ev->pc; > + int is_dummy_buf = !buf_size; > + const uint8_t *dummy_buf = buf; > + > + if (s->flags & PARSER_FLAG_COMPLETE_FRAMES) > + next = buf_size; > + else { > + next = evc_find_frame_end(s, buf, buf_size, avctx); > + if (ff_combine_frame(pc, next, &buf, &buf_size) < 0) { > + *poutbuf = NULL; > + *poutbuf_size = 0; > + ev->to_read -= buf_size; > + return buf_size; > + } > + } > + > + is_dummy_buf &= (dummy_buf == buf); > + > + if (!is_dummy_buf) > + parse_nal_units(s, buf, buf_size, avctx); > + > + *poutbuf = buf; > + *poutbuf_size = buf_size; > + ev->to_read -= next; > + > + return next; > +} > + > +const AVCodecParser ff_evc_parser = { > + .codec_ids = { AV_CODEC_ID_EVC }, > + .priv_data_size = sizeof(EVCParserContext), > + .parser_init = evc_parser_init, > + .parser_parse = evc_parse, > + .parser_close = ff_parse_close, > +}; _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [FFmpeg-devel] [PATCH 1/2] Provided support for MPEG-5 EVC (Essential Video Coding) codec 2022-08-01 12:29 ` James Almer @ 2022-08-08 12:39 ` Dawid Kozinski/Robot SDK (PLT) /SRPOL/Staff Engineer/삼성전자 0 siblings, 0 replies; 13+ messages in thread From: Dawid Kozinski/Robot SDK (PLT) /SRPOL/Staff Engineer/삼성전자 @ 2022-08-08 12:39 UTC (permalink / raw) To: 'FFmpeg development discussions and patches' I've just submitted new patches that were made based on your requests and suggestions. Please find below my comments. -----Original Message----- From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of James Almer Sent: Monday, August 1, 2022 2:29 PM To: ffmpeg-devel@ffmpeg.org Subject: Re: [FFmpeg-devel] [PATCH 1/2] Provided support for MPEG-5 EVC (Essential Video Coding) codec On 8/1/2022 6:28 AM, Dawid Kozinski wrote: > diff --git a/libavcodec/evc_parser.c b/libavcodec/evc_parser.c > new file mode 100644 > index 0000000000..b2136a6b45 > --- /dev/null > +++ b/libavcodec/evc_parser.c > @@ -0,0 +1,526 @@ > +/* > + * EVC format parser > + * > + * Copyright (C) 2021 Dawid Kozinski<d.kozinski@samsung.com> > + * > + * This file is part of FFmpeg. > + * > + * FFmpeg is free software; you can redistribute it and/or > + * modify it under the terms of the GNU Lesser General Public > + * License as published by the Free Software Foundation; either > + * version 2.1 of the License, or (at your option) any later version. > + * > + * FFmpeg is distributed in the hope that it will be useful, > + * but WITHOUT ANY WARRANTY; without even the implied warranty of > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > + * Lesser General Public License for more details. > + * > + * You should have received a copy of the GNU Lesser General Public > + * License along with FFmpeg; if not, write to the Free Software > + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA > + */ > + > +#include <stdint.h> > + > +#include "libavutil/common.h" > + > +#include "parser.h" > +#include "golomb.h" > + > +// The length field that indicates the length in bytes of the following NAL unit is configured to be of 4 bytes > +#define EVC_NAL_UNIT_LENGTH_BYTE (4) /* byte */ > + > +#define EVC_NAL_HEADER_SIZE (2) /* byte */ > +#define MAX_SPS_CNT (16) /* defined value in EVC standard */ > + > +// NALU types > +// @see ISO_IEC_23094-1_2020 7.4.2.2 NAL unit header semantics > +// > +#define EVC_NUT_NONIDR (0) /* Coded slice of a non-IDR picture */ > +#define EVC_NUT_IDR (1) /* Coded slice of an IDR picture */ > +#define EVC_NUT_SPS (24) /* Sequence parameter set */ > +#define EVC_NUT_PPS (25) /* Picture paremeter set */ > +#define EVC_NUT_APS (26) /* Adaptation parameter set */ > +#define EVC_NUT_FD (27) /* Filler data */ > +#define EVC_NUT_SEI (28) /* Supplemental enhancement information */ > + > +// The sturcture reflects SPS RBSP(raw byte sequence payload) layout > +// @see ISO_IEC_23094-1 section 7.3.2.1 > +// > +// The following descriptors specify the parsing process of each element > +// u(n) - unsigned integer using n bits > +// ue(v) - unsigned integer 0-th order Exp_Golomb-coded syntax element with the left bit first > +typedef struct EVCParserSPS { > + int sps_seq_parameter_set_id; // ue(v) > + int profile_idc; // u(8) > + int level_idc; // u(8) > + int toolset_idc_h; // u(32) > + int toolset_idc_l; // u(32) > + int chroma_format_idc; // ue(v) > + int pic_width_in_luma_samples; // ue(v) > + int pic_height_in_luma_samples; // ue(v) > + int bit_depth_luma_minus8; // ue(v) > + int bit_depth_chroma_minus8; // ue(v) > + > + int sps_btt_flag; // u(1) > + int log2_ctu_size_minus5; // ue(v) > + int log2_min_cb_size_minus2; // ue(v) > + int log2_diff_ctu_max_14_cb_size; // ue(v) > + int log2_diff_ctu_max_tt_cb_size; // ue(v) > + int log2_diff_min_cb_min_tt_cb_size_minus2; // ue(v) > + > + int sps_suco_flag; // u(1) > + int log2_diff_ctu_size_max_suco_cb_size; // ue(v) > + int log2_diff_max_suco_min_suco_cb_size; // ue(v) > + > + int sps_admvp_flag; // u(1) > + int sps_affine_flag; // u(1) > + int sps_amvr_flag; // u(1) > + int sps_dmvr_flag; // u(1) > + int sps_mmvd_flag; // u(1) > + int sps_hmvp_flag; // u(1) > + > + int sps_eipd_flag; // u(1) > + int sps_ibc_flag; // u(1) > + int log2_max_ibc_cand_size_minus2; // ue(v) > + > + int sps_cm_init_flag; // u(1) > + int sps_adcc_flag; // u(1) > + > + int sps_iqt_flag; // u(1) > + int sps_ats_flag; // u(1) > + > + int sps_addb_flag; // u(1) > + int sps_alf_flag; // u(1) > + int sps_htdf_flag; // u(1) > + int sps_rpl_flag; // u(1) > + int sps_pocs_flag; // u(1) > + int sps_dquant_flag; // u(1) > + int sps_dra_flag; // u(1) > + > + int log2_max_pic_order_cnt_lsb_minus4; // ue(v) > + int log2_sub_gop_length; // ue(v) > + int log2_ref_pic_gap_length; // ue(v) > + > + int max_num_tid0_ref_pics; // ue(v) > + > + int sps_max_dec_pic_buffering_minus1; // ue(v) > + int long_term_ref_pic_flag; // u(1) > + int rpl1_same_as_rpl0_flag; // u(1) > + int num_ref_pic_list_in_sps[2]; // ue(v) > + > + int picture_cropping_flag; // u(1) > + int picture_crop_left_offset; // ue(v) > + int picture_crop_right_offset; // ue(v) > + int picture_crop_top_offset; // ue(v) > + int picture_crop_bottom_offset; // ue(v) > + > +} EVCParserSPS; > + > +typedef struct EVCParserContext { > + ParseContext pc; > + EVCParserSPS sps[MAX_SPS_CNT]; > + int is_avc; > + int nal_length_size; > + int to_read; > + int incomplete_nalu_prefix_read; // The flag is set to 1 when an incomplete NAL unit prefix has been read > + > + int got_sps; > + int got_pps; > + int got_sei; > + int got_slice; > +} EVCParserContext; > + > +static int get_nalu_type(const uint8_t *bits, int bits_size, AVCodecContext *avctx) > +{ > + int unit_type_plus1 = 0; > + > + if(bits_size >= EVC_NAL_HEADER_SIZE) { > + unsigned char *p = (unsigned char *)bits; > + // forbidden_zero_bit > + if ((p[0] & 0x80) != 0) { > + av_log(avctx, AV_LOG_ERROR, "Cannot get bitstream information. Malformed bitstream.\n"); > + return -1; > + } > + > + // nal_unit_type > + unit_type_plus1 = (p[0] >> 1) & 0x3F; > + } > + > + return unit_type_plus1 - 1; > +} > + > +static uint32_t read_nal_unit_length(const uint8_t *bits, int bits_size, AVCodecContext *avctx) > +{ > + uint32_t nalu_len = 0; > + > + if(bits_size >= EVC_NAL_UNIT_LENGTH_BYTE) { > + > + int t = 0; > + unsigned char *p = (unsigned char *)bits; > + > + for(int i = 0; i < EVC_NAL_UNIT_LENGTH_BYTE; i++) > + t = (t << 8) | p[i]; > + > + nalu_len = t; > + if(nalu_len == 0) { > + av_log(avctx, AV_LOG_ERROR, "Invalid bitstream size!\n"); > + return 0; > + } > + } > + > + return nalu_len; > +} > + > +// @see ISO_IEC_23094-1 (7.3.2.1 SPS RBSP syntax) > +static EVCParserSPS *parse_sps(const uint8_t *bs, int bs_size, EVCParserContext *ev) > +{ > + GetBitContext gb; > + EVCParserSPS *sps; > + int sps_seq_parameter_set_id; > + > + if(init_get_bits8(&gb, bs, bs_size) < 0) > + return NULL; > + > + sps_seq_parameter_set_id = get_ue_golomb(&gb); > + > + if(sps_seq_parameter_set_id >= MAX_SPS_CNT) > + return NULL; > + > + sps = &ev->sps[sps_seq_parameter_set_id]; > + sps->sps_seq_parameter_set_id = sps_seq_parameter_set_id; > + > + // the Baseline profile is indicated by profile_idc eqal to 0 > + // the Main profile is indicated by profile_idc eqal to 1 > + sps->profile_idc = get_bits(&gb, 8); > + > + sps->level_idc = get_bits(&gb, 8); > + > + skip_bits_long(&gb, 32); /* skip toolset_idc_h */ > + skip_bits_long(&gb, 32); /* skip toolset_idc_l */ > + > + // 0 - monochrome > + // 1 - 4:2:0 > + // 2 - 4:2:2 > + // 3 - 4:4:4 > + sps->chroma_format_idc = get_ue_golomb(&gb); > + > + sps->pic_width_in_luma_samples = get_ue_golomb(&gb); > + sps->pic_height_in_luma_samples = get_ue_golomb(&gb); > + > + sps->bit_depth_luma_minus8 = get_ue_golomb(&gb); > + sps->bit_depth_chroma_minus8 = get_ue_golomb(&gb); > + > + sps->sps_btt_flag = get_bits(&gb, 1); > + if(sps->sps_btt_flag) { > + sps->log2_ctu_size_minus5 = get_ue_golomb(&gb); > + sps->log2_min_cb_size_minus2 = get_ue_golomb(&gb); > + sps->log2_diff_ctu_max_14_cb_size = get_ue_golomb(&gb); > + sps->log2_diff_ctu_max_tt_cb_size = get_ue_golomb(&gb); > + sps->log2_diff_min_cb_min_tt_cb_size_minus2 = get_ue_golomb(&gb); > + } > + > + sps->sps_suco_flag = get_bits(&gb, 1); > + if(sps->sps_suco_flag) { > + sps->log2_diff_ctu_size_max_suco_cb_size = get_ue_golomb(&gb); > + sps->log2_diff_max_suco_min_suco_cb_size = get_ue_golomb(&gb); > + } > + > + sps->sps_admvp_flag = get_bits(&gb, 1); > + if(sps->sps_admvp_flag) { > + sps->sps_affine_flag = get_bits(&gb, 1); > + sps->sps_amvr_flag = get_bits(&gb, 1); > + sps->sps_dmvr_flag = get_bits(&gb, 1); > + sps->sps_mmvd_flag = get_bits(&gb, 1); > + sps->sps_hmvp_flag = get_bits(&gb, 1); > + } > + > + sps->sps_eipd_flag = get_bits(&gb, 1); > + if(sps->sps_eipd_flag) { > + sps->sps_ibc_flag = get_bits(&gb, 1); > + if(sps->sps_ibc_flag) > + sps->log2_max_ibc_cand_size_minus2 = get_ue_golomb(&gb); > + } > + > + sps->sps_cm_init_flag = get_bits(&gb, 1); > + if(sps->sps_cm_init_flag) > + sps->sps_adcc_flag = get_bits(&gb, 1); > + > + sps->sps_iqt_flag = get_bits(&gb, 1); > + if(sps->sps_iqt_flag) > + sps->sps_ats_flag = get_bits(&gb, 1); > + > + sps->sps_addb_flag = get_bits(&gb, 1); > + sps->sps_alf_flag = get_bits(&gb, 1); > + sps->sps_htdf_flag = get_bits(&gb, 1); > + sps->sps_rpl_flag = get_bits(&gb, 1); > + sps->sps_pocs_flag = get_bits(&gb, 1); > + sps->sps_dquant_flag = get_bits(&gb, 1); > + sps->sps_dra_flag = get_bits(&gb, 1); > + > + if(sps->sps_pocs_flag) > + sps->log2_max_pic_order_cnt_lsb_minus4 = get_ue_golomb(&gb); > + > + if(!sps->sps_pocs_flag || !sps->sps_rpl_flag) { > + sps->log2_sub_gop_length = get_ue_golomb(&gb); > + if(sps->log2_sub_gop_length == 0) > + sps->log2_ref_pic_gap_length = get_ue_golomb(&gb); > + } > + > + return sps; > +} > + > +static int parse_nal_units(AVCodecParserContext *s, const uint8_t *bs, > + int bs_size, AVCodecContext *avctx) > +{ > + EVCParserContext *ev = s->priv_data; > + int nalu_type, nalu_size; > + unsigned char *bits = (unsigned char *)bs; > + int bits_size = bs_size; > + > + avctx->codec_id = AV_CODEC_ID_EVC; > + > + nalu_size = read_nal_unit_length(bits, bits_size, avctx); > + if(nalu_size == 0) { > + av_log(avctx, AV_LOG_ERROR, "Invalid NAL unit size: (%d)\n", nalu_size); > + return -1; > + } > + > + bits += EVC_NAL_UNIT_LENGTH_BYTE; > + bits_size -= EVC_NAL_UNIT_LENGTH_BYTE; > + > + nalu_type = get_nalu_type(bits, bits_size, avctx); > + > + bits += EVC_NAL_HEADER_SIZE; > + bits_size -= EVC_NAL_HEADER_SIZE; > + > + if (nalu_type == EVC_NUT_SPS) { // NAL Unit type: SPS (Sequence Parameter Set) > + EVCParserSPS *sps; > + > + sps = parse_sps(bits, bits_size, ev); > + > + avctx->coded_width = sps->pic_width_in_luma_samples; > + avctx->coded_height = sps->pic_height_in_luma_samples; > + avctx->width = sps->pic_width_in_luma_samples; > + avctx->height = sps->pic_height_in_luma_samples; These should be set in the AVCodecParserContext, not the AVCodecContext. #################### DONE #################### > + > + if(sps->profile_idc == 0) avctx->profile = FF_PROFILE_EVC_BASELINE; > + else if (sps->profile_idc == 1) avctx->profile = FF_PROFILE_EVC_MAIN; Setting these two is ok. > + else { > + av_log(avctx, AV_LOG_ERROR, "Not supported profile (%d)\n", sps->profile_idc); > + return -1; Do you really need to abort here? Not recognizing the profile can't possibly mean the bitstream can't be parsed further to fetch other information. #################### DONE #################### > + } > + > + // Currently XEVD decoder supports ony YCBCR420_10LE chroma format for EVC stream > + switch(sps->chroma_format_idc) { > + case 0: /* YCBCR400_10LE */ > + av_log(avctx, AV_LOG_ERROR, "YCBCR400_10LE: Not supported chroma format\n"); > + avctx->pix_fmt = AV_PIX_FMT_GRAY10LE; > + return -1; > + case 1: /* YCBCR420_10LE */ > + avctx->pix_fmt = AV_PIX_FMT_YUV420P10LE; > + break; > + case 2: /* YCBCR422_10LE */ > + av_log(avctx, AV_LOG_ERROR, "YCBCR422_10LE: Not supported chroma format\n"); > + avctx->pix_fmt = AV_PIX_FMT_YUV422P10LE; > + return -1; > + case 3: /* YCBCR444_10LE */ > + av_log(avctx, AV_LOG_ERROR, "YCBCR444_10LE: Not supported chroma format\n"); > + avctx->pix_fmt = AV_PIX_FMT_YUV444P10LE; > + return -1; > + default: > + avctx->pix_fmt = AV_PIX_FMT_NONE; Fill s->format instead of avctx->pix_fmt. #################### DONE #################### > + av_log(avctx, AV_LOG_ERROR, "Unknown supported chroma format\n"); > + return -1; > + } > + > + // The current implementation of parse_sps function doesn't handle VUI parameters parsing, > + // so at the moment it's impossible to initialize has_b_frames and max_b_frames AVCodecContex fields here. > + // Currently, initialization of has_b_frames and max_b_frames AVCodecContex fields have been moved to > + // libxevd_decode function where we can use xevd_config function being a part of xevd library API > + // to get the needed information. > + // However, if it will be needed, parse_sps function should be extended to handle VUI parameters parsing > + // and the following lines should be used to initialize has_b_frames and max_b_frames fields of the AVCodecContex. There's no need no fill any of those field in this parser. What you can fill is avctx->color{space,_primaries,_trc,_range}, so mention that to add VUI parsing at some point. > + // > + // sps->vui_parameters.num_reorder_pics > + // if (sps->bitstream_restriction_flag && sps->vui_parameters.num_reorder_pics) { > + // avctx->has_b_frames = sps->vui_parameters.num_reorder_pics; > + // } > + > + ev->got_sps = 1; > + > + } else if (nalu_type == EVC_NUT_PPS) // NAL Unit type: PPS (Video Parameter Set) > + ev->got_pps = 1; > + else if(nalu_type == EVC_NUT_SEI) // NAL unit type: SEI (Supplemental Enhancement Information) > + ev->got_sei = 1; These got_* variables seem to be write only. #################### DONE #################### > + else if (nalu_type == EVC_NUT_IDR || nalu_type == EVC_NUT_NONIDR) // NAL Unit type: Coded slice of a IDR or non-IDR picture In here you should parse enough data to get the picture type (I, P, B) and set s->pict_type, and based on (i suppose) nalu_type, also set s->key_frame to 1 or 0. #################### DONE #################### At the start of this function, you'd ideally set them to AV_PICTURE_TYPE_NONE and -1 respectively, as "safe" defaults. #################### DONE #################### s->picture_structure should also be always set to AV_PICTURE_STRUCTURE_FRAME, as i assume this codec does not support interlacing. #################### DONE #################### > + ev->got_slice++; > + else { > + av_log(avctx, AV_LOG_ERROR, "Invalid NAL unit type: %d\n", nalu_type); > + return -1; > + } > + > + return 0; > +} > + > +/** > + * Find the end of the current frame in the bitstream. > + * @return the position of the first byte of the next frame, or END_NOT_FOUND > + */ > +static int evc_find_frame_end(AVCodecParserContext *s, const uint8_t *buf, > + int buf_size, AVCodecContext *avctx) > +{ > + EVCParserContext *ev = s->priv_data; > + > + if(!ev->to_read) { > + int nal_unit_size = 0; > + int next = END_NOT_FOUND; > + > + // This is the case when buffer size is not enough for buffer to store NAL unit length > + if(buf_size < EVC_NAL_UNIT_LENGTH_BYTE) { > + ev->to_read = EVC_NAL_UNIT_LENGTH_BYTE; > + ev->nal_length_size = buf_size; > + ev->incomplete_nalu_prefix_read = 1; > + > + return END_NOT_FOUND; > + } > + > + nal_unit_size = read_nal_unit_length(buf, buf_size, avctx); > + ev->nal_length_size = EVC_NAL_UNIT_LENGTH_BYTE; > + > + next = nal_unit_size + EVC_NAL_UNIT_LENGTH_BYTE; > + ev->to_read = next; > + if(next < buf_size) > + return next; > + else > + return END_NOT_FOUND; > + } else if(ev->to_read > buf_size) > + return END_NOT_FOUND; > + else { > + if(ev->incomplete_nalu_prefix_read == 1) { > + EVCParserContext *ev = s->priv_data; > + ParseContext *pc = &ev->pc; > + uint8_t nalu_len[EVC_NAL_UNIT_LENGTH_BYTE] = {0}; > + int nal_unit_size = 0; > + > + // 1. pc->buffer contains previously read bytes of NALU prefix > + // 2. buf contains the rest of NAL unit prefix bytes > + // > + // ~~~~~~~ > + // EXAMPLE > + // ~~~~~~~ > + // > + // In the following example we assumed that the number of already read NAL Unit prefix bytes is equal 1 > + // > + // ---------- > + // pc->buffer -> conatins already read bytes > + // ---------- > + // __ pc->index == 1 > + // | > + // V > + // ------------------------------------------------------- > + // | 0 | 1 | 2 | 3 | 4 | ... | N | > + // ------------------------------------------------------- > + // | 0x00 | 0xXX | 0xXX | 0xXX | 0xXX | ... | 0xXX | > + // ------------------------------------------------------- > + // > + // ---------- > + // buf -> contains newly read bytes > + // ---------- > + // ------------------------------------------------------- > + // | 0 | 1 | 2 | 3 | 4 | ... | N | > + // ------------------------------------------------------- > + // | 0x00 | 0x00 | 0x3C | 0xXX | 0xXX | ... | 0xXX | > + // ------------------------------------------------------- > + // > + for(int i = 0; i < EVC_NAL_UNIT_LENGTH_BYTE; i++) { > + if(i < pc->index) > + nalu_len[i] = pc->buffer[i]; > + else > + nalu_len[i] = buf[i - pc->index]; > + } > + > + // ---------- > + // nalu_len > + // ---------- > + // --------------------------------- > + // | 0 | 1 | 2 | 3 | > + // --------------------------------- > + // | 0x00 | 0x00 | 0x00 | 0x3C | > + // --------------------------------- > + // | NALU LENGTH | > + // --------------------------------- > + // NAL Unit lenght = 60 (0x0000003C) > + > + nal_unit_size = read_nal_unit_length(nalu_len, EVC_NAL_UNIT_LENGTH_BYTE, avctx); > + > + ev->to_read = nal_unit_size + EVC_NAL_UNIT_LENGTH_BYTE - pc->index; > + > + ev->incomplete_nalu_prefix_read = 0; > + > + if(ev->to_read > buf_size) > + return END_NOT_FOUND; > + else > + return ev->to_read; > + } > + return ev->to_read; > + } > + > + return END_NOT_FOUND; > +} > + > +static int evc_parser_init(AVCodecParserContext *s) > +{ > + EVCParserContext *ev = s->priv_data; > + > + ev->got_sps = 0; > + ev->got_pps = 0; > + ev->got_sei = 0; > + ev->got_slice = 0; > + ev->nal_length_size = EVC_NAL_UNIT_LENGTH_BYTE; > + ev->incomplete_nalu_prefix_read = 0; > + > + return 0; > +} > + > +static int evc_parse(AVCodecParserContext *s, AVCodecContext *avctx, > + const uint8_t **poutbuf, int *poutbuf_size, > + const uint8_t *buf, int buf_size) > +{ > + int next; > + EVCParserContext *ev = s->priv_data; > + ParseContext *pc = &ev->pc; > + int is_dummy_buf = !buf_size; > + const uint8_t *dummy_buf = buf; > + > + if (s->flags & PARSER_FLAG_COMPLETE_FRAMES) > + next = buf_size; > + else { > + next = evc_find_frame_end(s, buf, buf_size, avctx); > + if (ff_combine_frame(pc, next, &buf, &buf_size) < 0) { > + *poutbuf = NULL; > + *poutbuf_size = 0; > + ev->to_read -= buf_size; > + return buf_size; > + } > + } > + > + is_dummy_buf &= (dummy_buf == buf); > + > + if (!is_dummy_buf) > + parse_nal_units(s, buf, buf_size, avctx); > + > + *poutbuf = buf; > + *poutbuf_size = buf_size; > + ev->to_read -= next; > + > + return next; > +} > + > +const AVCodecParser ff_evc_parser = { > + .codec_ids = { AV_CODEC_ID_EVC }, > + .priv_data_size = sizeof(EVCParserContext), > + .parser_init = evc_parser_init, > + .parser_parse = evc_parse, > + .parser_close = ff_parse_close, > +}; _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [FFmpeg-devel] [PATCH 1/2] Provided support for MPEG-5 EVC (Essential Video Coding) codec 2022-08-01 9:28 ` Dawid Kozinski 2022-08-01 12:29 ` James Almer @ 2022-08-01 15:09 ` James Almer 2022-08-08 12:44 ` Dawid Kozinski/Robot SDK (PLT) /SRPOL/Staff Engineer/삼성전자 ` (2 more replies) 1 sibling, 3 replies; 13+ messages in thread From: James Almer @ 2022-08-01 15:09 UTC (permalink / raw) To: ffmpeg-devel On 8/1/2022 6:28 AM, Dawid Kozinski wrote: > diff --git a/libavcodec/libxevd.c b/libavcodec/libxevd.c > new file mode 100644 > index 0000000000..8ba2d89cbd > --- /dev/null > +++ b/libavcodec/libxevd.c > @@ -0,0 +1,436 @@ > +/* > + * libxevd decoder > + * EVC (MPEG-5 Essential Video Coding) decoding using XEVD MPEG-5 EVC decoder library > + * > + * Copyright (C) 2021 Dawid Kozinski<d.kozinski@samsung.com> > + * > + * This file is part of FFmpeg. > + * > + * FFmpeg is free software; you can redistribute it and/or > + * modify it under the terms of the GNU Lesser General Public > + * License as published by the Free Software Foundation; either > + * version 2.1 of the License, or (at your option) any later version. > + * > + * FFmpeg is distributed in the hope that it will be useful, > + * but WITHOUT ANY WARRANTY; without even the implied warranty of > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > + * Lesser General Public License for more details. > + * > + * You should have received a copy of the GNU Lesser General Public > + * License along with FFmpeg; if not, write to the Free Software > + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA > + */ > + > +#include <float.h> > +#include <stdlib.h> > + > +#include <xevd.h> > + > +#include "libavutil/internal.h" > +#include "libavutil/common.h" > +#include "libavutil/opt.h" > +#include "libavutil/pixdesc.h" > +#include "libavutil/pixfmt.h" > +#include "libavutil/imgutils.h" > +#include "libavutil/cpu.h" > + > +#include "avcodec.h" > +#include "internal.h" > +#include "packet_internal.h" > +#include "codec_internal.h" > + > +#define XEVD_PARAM_BAD_NAME -1 > +#define XEVD_PARAM_BAD_VALUE -2 > + > +#define EVC_NAL_HEADER_SIZE 2 /* byte */ > + > +/** > + * The structure stores all the states associated with the instance of Xeve MPEG-5 EVC decoder > + */ > +typedef struct XevdContext { > + const AVClass *class; > + > + XEVD id; // XEVD instance identifier @see xevd.h > + XEVD_CDSC cdsc; // decoding parameters @see xevd.h > + > + int decod_frames; // number of decoded frames > + int packet_count; // number of packets created by decoder > +} XevdContext; > + > +/** > + * The function returns a pointer to a variable of type XEVD_CDSC. This doesn't seem to be the case. > + * XEVD_CDSC contains all decoder parameters that should be initialized before its use. > + * > + * The field values of the XEVD_CDSC structure are populated based on: > + * - the corresponding field values of the AvCodecConetxt structure, > + * - the xevd decoder specific option values, > + * (the full list of options available for the xevd encoder is displayed after executing the command ./ffmpeg --help decoder = libxevd) > + * - and the xevd encoder options specified as a list of key-value pairs following the xevd-params option > + * > + * Order of input processing and populating the XEVD_CDSC structure > + * 1. first, the corresponding fields of the AVCodecContext structure are processed, (i.e -threads 4) > + * 2. then xevd-specific options are added as AVOption to the xevd AVCodec implementation (i.e -threads 3) > + * 3. finally, the options specified after the xevd-params option as the parameter list of type key value are processed (i.e -xevd-params "m=2") I don't see a xevd-params option. > + * > + * Some options can be set in different ways. In this case, please follow the above-mentioned order of processing. > + * The most recent assignments overwrite the previous values. > + * > + * @param[in] avctx codec context > + * @param[out] cdsc contains all encoder parameters that should be initialized before its use > + * > + * @return 0 on success, negative error code on failure I don't see a failure path below, so might as well just make this a void function. > + */ > +static int get_conf(AVCodecContext *avctx, XEVD_CDSC *cdsc) > +{ > + int cpu_count = av_cpu_count(); > + > + /* clear XEVS_CDSC structure */ > + memset(cdsc, 0, sizeof(XEVD_CDSC)); > + > + /* init XEVD_CDSC */ > + if (avctx->thread_count <= 0) > + cdsc->threads = (cpu_count < XEVD_MAX_TASK_CNT) ? cpu_count : XEVD_MAX_TASK_CNT; > + else if (avctx->thread_count > XEVD_MAX_TASK_CNT) > + cdsc->threads = XEVD_MAX_TASK_CNT; > + else > + cdsc->threads = avctx->thread_count; > + > + return 0; > +} > + > +/** > + * Read NAL unit length > + * @param bs input data (bitstream) > + * @return the length of NAL unit on success, 0 value on failure > + */ > +static uint32_t read_nal_unit_length(const uint8_t *bs, int bs_size, AVCodecContext *avctx) > +{ > + uint32_t len = 0; > + XEVD_INFO info; > + int ret; > + > + if (bs_size == XEVD_NAL_UNIT_LENGTH_BYTE) { > + ret = xevd_info((void *)bs, XEVD_NAL_UNIT_LENGTH_BYTE, 1, &info); > + if (XEVD_FAILED(ret)) { > + av_log(avctx, AV_LOG_ERROR, "Cannot get bitstream information\n"); > + return 0; > + } > + len = info.nalu_len; > + if (len == 0) { > + av_log(avctx, AV_LOG_ERROR, "Invalid bitstream size! [%d]\n", bs_size); > + return 0; > + } > + } > + > + return len; > +} > + > +/** > + * @param[in] xectx the structure that stores all the state associated with the instance of Xeve MPEG-5 EVC decoder > + * @param[out] avctx codec context > + * @return 0 on success, negative value on failure > + */ > +static int export_stream_params(const XevdContext *xectx, AVCodecContext *avctx) > +{ > + int ret; > + int size; > + int color_space; > + > + avctx->pix_fmt = AV_PIX_FMT_YUV420P10; > + > + size = 4; > + ret = xevd_config(xectx->id, XEVD_CFG_GET_CODED_WIDTH, &avctx->coded_width, &size); > + if (XEVD_FAILED(ret)) { > + av_log(avctx, AV_LOG_ERROR, "Failed to get coded_width\n"); > + return AVERROR_EXTERNAL; > + } > + > + ret = xevd_config(xectx->id, XEVD_CFG_GET_CODED_HEIGHT, &avctx->coded_height, &size); > + if (XEVD_FAILED(ret)) { > + av_log(avctx, AV_LOG_ERROR, "Failed to get coded_height\n"); > + return AVERROR_EXTERNAL; > + } > + > + ret = xevd_config(xectx->id, XEVD_CFG_GET_WIDTH, &avctx->width, &size); > + if (XEVD_FAILED(ret)) { > + av_log(avctx, AV_LOG_ERROR, "Failed to get width\n"); > + return AVERROR_EXTERNAL; > + } > + > + ret = xevd_config(xectx->id, XEVD_CFG_GET_HEIGHT, &avctx->height, &size); > + if (XEVD_FAILED(ret)) { > + av_log(avctx, AV_LOG_ERROR, "Failed to get height\n"); > + return AVERROR_EXTERNAL; > + } > + > + ret = xevd_config(xectx->id, XEVD_CFG_GET_COLOR_SPACE, &color_space, &size); > + if (XEVD_FAILED(ret)) { > + av_log(avctx, AV_LOG_ERROR, "Failed to get color_space\n"); > + return AVERROR_EXTERNAL; > + } > + switch(color_space) { > + case XEVD_CS_YCBCR400_10LE: > + avctx->pix_fmt = AV_PIX_FMT_GRAY10LE; > + break; > + case XEVD_CS_YCBCR420_10LE: > + avctx->pix_fmt = AV_PIX_FMT_YUV420P10LE; > + break; > + case XEVD_CS_YCBCR422_10LE: > + avctx->pix_fmt = AV_PIX_FMT_YUV422P10LE; > + break; > + case XEVD_CS_YCBCR444_10LE: > + avctx->pix_fmt = AV_PIX_FMT_YUV444P10LE; > + break; > + default: > + av_log(avctx, AV_LOG_ERROR, "Unknown color space\n"); > + avctx->pix_fmt = AV_PIX_FMT_NONE; > + return AVERROR_INVALIDDATA; > + } > + > + // the function returns sps->num_reorder_pics > + ret = xevd_config(xectx->id, XEVD_CFG_GET_MAX_CODING_DELAY, &avctx->max_b_frames, &size); > + if (XEVD_FAILED(ret)) { > + av_log(avctx, AV_LOG_ERROR, "Failed to get max_coding_delay\n"); > + return AVERROR_EXTERNAL; > + } > + > + avctx->has_b_frames = (avctx->max_b_frames) ? 1 : 0; > + > + avctx->color_primaries = AVCOL_PRI_UNSPECIFIED; > + avctx->color_trc = AVCOL_TRC_UNSPECIFIED; > + avctx->colorspace = AVCOL_SPC_UNSPECIFIED; > + > + return 0; > +} > + > +/** > + * Initialize decoder > + * Create a decoder instance and allocate all the needed resources > + * > + * @param avctx codec context > + * @return 0 on success, negative error code on failure > + */ > +static av_cold int libxevd_init(AVCodecContext *avctx) > +{ > + XevdContext *xectx = avctx->priv_data; > + int ret = 0; > + XEVD_CDSC *cdsc = &(xectx->cdsc); > + > + /* read configurations and set values for created descriptor (XEVD_CDSC) */ > + ret = get_conf(avctx, cdsc); > + if (ret != 0) { > + av_log(avctx, AV_LOG_ERROR, "Cannot get configuration\n"); > + return AVERROR_INVALIDDATA; > + } > + > + /* create decoder */ > + xectx->id = xevd_create(&(xectx->cdsc), NULL); > + if (xectx->id == NULL) { > + av_log(avctx, AV_LOG_ERROR, "Cannot create XEVD encoder\n"); > + return AVERROR_EXTERNAL; > + } > + > + xectx->packet_count = 0; > + xectx->decod_frames = 0; > + > + return 0; > +} > + > +/** > + * Decode picture > + * > + * @param avctx codec context > + * @param[out] frame decoded frame > + * @param[out] got_frame_ptr decoder sets to 0 or 1 to indicate that a > + * non-empty frame or subtitle was returned in > + * outdata. > + * @param[in] avpkt AVPacket containing encoded data to be decoded > + * > + * @return amount of bytes read from the packet on success, negative error > + * code on failure > + */ > +static int libxevd_decode(struct AVCodecContext *avctx, struct AVFrame *frame, int *got_frame_ptr, AVPacket *avpkt) > +{ > + XevdContext *xectx = NULL; > + XEVD_IMGB *imgb = NULL; > + XEVD_STAT stat; > + XEVD_BITB bitb; > + int xevd_ret, nalu_size, bs_read_pos; > + int ret = 0; > + > + if (avctx == NULL) { > + av_log(avctx, AV_LOG_ERROR, "Invalid input parameter: AVCodecContext\n"); > + return AVERROR(EINVAL); > + } > + xectx = avctx->priv_data; > + if (xectx == NULL) { > + av_log(avctx, AV_LOG_ERROR, "Invalid XEVD context\n"); > + return AVERROR(EINVAL); This can't happen, just remove the check. > + } > + > + if (avpkt->size > 0) { > + bs_read_pos = 0; > + imgb = NULL; > + while(avpkt->size > (bs_read_pos + XEVD_NAL_UNIT_LENGTH_BYTE)) { > + > + memset(&stat, 0, sizeof(XEVD_STAT)); > + memset(&bitb, 0, sizeof(XEVD_BITB)); > + > + nalu_size = read_nal_unit_length(avpkt->data + bs_read_pos, XEVD_NAL_UNIT_LENGTH_BYTE, avctx); > + if (nalu_size == 0) { > + av_log(avctx, AV_LOG_ERROR, "Invalid bitstream\n"); > + ret = AVERROR_INVALIDDATA; > + goto ERR; > + } > + bs_read_pos += XEVD_NAL_UNIT_LENGTH_BYTE; > + > + bitb.addr = avpkt->data + bs_read_pos; > + bitb.ssize = nalu_size; > + > + /* main decoding block */ > + xevd_ret = xevd_decode(xectx->id, &bitb, &stat); > + if (XEVD_FAILED(xevd_ret)) { > + av_log(avctx, AV_LOG_ERROR, "Failed to decode bitstream\n"); > + ret = AVERROR_EXTERNAL; > + goto ERR; > + } > + > + bs_read_pos += nalu_size; > + > + if (stat.nalu_type == XEVD_NUT_SPS) { // EVC stream parameters changed > + if ((ret = export_stream_params(xectx, avctx)) != 0) > + goto ERR; > + } > + > + if (stat.read != nalu_size) > + av_log(avctx, AV_LOG_INFO, "Different reading of bitstream (in:%d,read:%d)\n,", nalu_size, stat.read); > + if (stat.fnum >= 0) { > + // already has a decoded image > + if (imgb) { > + // xevd_pull uses pool of objects of type XEVD_IMGB. > + // The pool size is equal MAX_PB_SIZE (26), so release object when it is no more needed > + imgb->release(imgb); > + imgb = NULL; > + } > + xevd_ret = xevd_pull(xectx->id, &imgb); > + if (XEVD_FAILED(xevd_ret)) { > + av_log(avctx, AV_LOG_ERROR, "Failed to pull the decoded image (xevd error code: %d, frame#=%d)\n", xevd_ret, stat.fnum); > + ret = AVERROR_EXTERNAL; > + goto ERR; > + } else if (xevd_ret == XEVD_OK_FRM_DELAYED) { > + if (imgb) { > + // xevd_pull uses pool of objects of type XEVD_IMGB. > + // The pool size is equal MAX_PB_SIZE (26), so release object when it is no more needed > + imgb->release(imgb); > + imgb = NULL; > + } > + } So if the packet has more than one frame, you'll drop the oldest? Maybe you should write this decoder using the decouple Input/Output API, FF_CODEC_RECEIVE_FRAME_CB() callback instead of FF_CODEC_DECODE_CB(). With it, you can request a packet as needed with ff_decode_get_packet(), so for example only when libxevd has run out of frames to output you actually request another. > + } > + } > + } else { // bumping > + xevd_ret = xevd_pull(xectx->id, &(imgb)); > + if (xevd_ret == XEVD_ERR_UNEXPECTED) { // bumping process completed Shouldn't this be XEVD_OK_NO_MORE_FRM? > + *got_frame_ptr = 0; > + return 0; > + } else if (XEVD_FAILED(xevd_ret)) { > + av_log(avctx, AV_LOG_ERROR, "Failed to pull the decoded image (xevd error code: %d)\n", xevd_ret); > + ret = AVERROR_EXTERNAL; > + goto ERR; > + } > + } > + > + if (imgb) { > + // @todo supports other color space and bit depth > + if (imgb->cs != XEVD_CS_YCBCR420_10LE) { > + av_log(avctx, AV_LOG_ERROR, "Not supported pixel format: %s\n", av_get_pix_fmt_name(avctx->pix_fmt)); > + ret = AVERROR_INVALIDDATA; > + goto ERR; > + } > + > + if (imgb->w[0] != avctx->width || imgb->h[0] != avctx->height) { // stream resolution changed > + if (ff_set_dimensions(avctx, imgb->w[0], imgb->h[0]) < 0) { > + av_log(avctx, AV_LOG_ERROR, "Cannot set new dimension\n"); > + ret = AVERROR_INVALIDDATA; > + goto ERR; > + } > + } > + > + frame->coded_picture_number++; > + frame->display_picture_number++; As is, these two will always be 1. If you want to set these fields, you need to keep track of total decoded frames elsewhere. stat.fnum looks like what you need, or maybe xectx->decod_frames. > + frame->format = AV_PIX_FMT_YUV420P10LE; No need to set this here. ff_get_buffer() will handle it for you. > + > + if (ff_get_buffer(avctx, frame, 0) < 0) { > + av_log(avctx, AV_LOG_ERROR, "Cannot get AV buffer\n"); ff_get_buffer() already prints an error, so remove this one. > + ret = AVERROR(ENOMEM); It also returns an error code you should propagate. > + goto ERR; > + } > + > + frame->pts = avpkt->pts; > + > + av_image_copy(frame->data, frame->linesize, (const uint8_t **)imgb->a, > + imgb->s, avctx->pix_fmt, > + imgb->w[0], imgb->h[0]); > + > + xectx->decod_frames++; Write only variable. > + *got_frame_ptr = 1; > + > + // xevd_pull uses pool of objects of type XEVD_IMGB. > + // The pool size is equal MAX_PB_SIZE (26), so release object when it is no more needed > + imgb->release(imgb); > + imgb = NULL; > + } else > + *got_frame_ptr = 0; > + > + xectx->packet_count++; Same. > + > + return avpkt->size; > + > +ERR: > + if (imgb) { > + imgb->release(imgb); > + imgb = NULL; > + } > + *got_frame_ptr = 0; > + > + return ret; > +} > + > +/** > + * Destroy decoder > + * > + * @param avctx codec context > + * @return 0 on success > + */ > +static av_cold int libxevd_close(AVCodecContext *avctx) > +{ > + XevdContext *xectx = avctx->priv_data; > + if (xectx->id) { > + xevd_delete(xectx->id); > + xectx->id = NULL; > + } > + > + return 0; > +} > + > +#define OFFSET(x) offsetof(XevdContext, x) > +#define VD AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_DECODING_PARAM > + > +static const AVClass libxevd_class = { > + .class_name = "libxevd", > + .item_name = av_default_item_name, > + .version = LIBAVUTIL_VERSION_INT, > +}; > + > +const FFCodec ff_libxevd_decoder = { > + .p.name = "evc", > + .p.long_name = NULL_IF_CONFIG_SMALL("EVC / MPEG-5 Essential Video Coding (EVC)"), > + .p.type = AVMEDIA_TYPE_VIDEO, > + .p.id = AV_CODEC_ID_EVC, > + .init = libxevd_init, > + FF_CODEC_DECODE_CB(libxevd_decode), > + .close = libxevd_close, > + .priv_data_size = sizeof(XevdContext), > + .p.priv_class = &libxevd_class, > + .p.capabilities = AV_CODEC_CAP_DELAY | AV_CODEC_CAP_AUTO_THREADS | AV_CODEC_CAP_AVOID_PROBING | AV_CODEC_CAP_DR1, Use AV_CODEC_CAP_OTHER_THREADS, not AV_CODEC_CAP_AUTO_THREADS. And set caps_internal to FF_CODEC_CAP_AUTO_THREADS. > + .p.wrapper_name = "libxevd", > +}; _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [FFmpeg-devel] [PATCH 1/2] Provided support for MPEG-5 EVC (Essential Video Coding) codec 2022-08-01 15:09 ` James Almer @ 2022-08-08 12:44 ` Dawid Kozinski/Robot SDK (PLT) /SRPOL/Staff Engineer/삼성전자 2022-08-11 6:49 ` Dawid Kozinski/Robot SDK (PLT) /SRPOL/Staff Engineer/삼성전자 2022-08-18 8:59 ` Dawid Kozinski/Robot SDK (PLT) /SRPOL/Staff Engineer/삼성전자 2 siblings, 0 replies; 13+ messages in thread From: Dawid Kozinski/Robot SDK (PLT) /SRPOL/Staff Engineer/삼성전자 @ 2022-08-08 12:44 UTC (permalink / raw) To: 'FFmpeg development discussions and patches' I just uploaded new patches that were made based on your requests and suggestions. Please find below my comments. -----Original Message----- From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of James Almer Sent: Monday, August 1, 2022 5:09 PM To: ffmpeg-devel@ffmpeg.org Subject: Re: [FFmpeg-devel] [PATCH 1/2] Provided support for MPEG-5 EVC (Essential Video Coding) codec On 8/1/2022 6:28 AM, Dawid Kozinski wrote: > diff --git a/libavcodec/libxevd.c b/libavcodec/libxevd.c > new file mode 100644 > index 0000000000..8ba2d89cbd > --- /dev/null > +++ b/libavcodec/libxevd.c > @@ -0,0 +1,436 @@ > +/* > + * libxevd decoder > + * EVC (MPEG-5 Essential Video Coding) decoding using XEVD MPEG-5 EVC decoder library > + * > + * Copyright (C) 2021 Dawid Kozinski<d.kozinski@samsung.com> > + * > + * This file is part of FFmpeg. > + * > + * FFmpeg is free software; you can redistribute it and/or > + * modify it under the terms of the GNU Lesser General Public > + * License as published by the Free Software Foundation; either > + * version 2.1 of the License, or (at your option) any later version. > + * > + * FFmpeg is distributed in the hope that it will be useful, > + * but WITHOUT ANY WARRANTY; without even the implied warranty of > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > + * Lesser General Public License for more details. > + * > + * You should have received a copy of the GNU Lesser General Public > + * License along with FFmpeg; if not, write to the Free Software > + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA > + */ > + > +#include <float.h> > +#include <stdlib.h> > + > +#include <xevd.h> > + > +#include "libavutil/internal.h" > +#include "libavutil/common.h" > +#include "libavutil/opt.h" > +#include "libavutil/pixdesc.h" > +#include "libavutil/pixfmt.h" > +#include "libavutil/imgutils.h" > +#include "libavutil/cpu.h" > + > +#include "avcodec.h" > +#include "internal.h" > +#include "packet_internal.h" > +#include "codec_internal.h" > + > +#define XEVD_PARAM_BAD_NAME -1 > +#define XEVD_PARAM_BAD_VALUE -2 > + > +#define EVC_NAL_HEADER_SIZE 2 /* byte */ > + > +/** > + * The structure stores all the states associated with the instance of Xeve MPEG-5 EVC decoder > + */ > +typedef struct XevdContext { > + const AVClass *class; > + > + XEVD id; // XEVD instance identifier @see xevd.h > + XEVD_CDSC cdsc; // decoding parameters @see xevd.h > + > + int decod_frames; // number of decoded frames > + int packet_count; // number of packets created by decoder > +} XevdContext; > + > +/** > + * The function returns a pointer to a variable of type XEVD_CDSC. This doesn't seem to be the case. #################### DONE #################### > + * XEVD_CDSC contains all decoder parameters that should be initialized before its use. > + * > + * The field values of the XEVD_CDSC structure are populated based on: > + * - the corresponding field values of the AvCodecConetxt structure, > + * - the xevd decoder specific option values, > + * (the full list of options available for the xevd encoder is displayed after executing the command ./ffmpeg --help decoder = libxevd) > + * - and the xevd encoder options specified as a list of key-value pairs following the xevd-params option > + * > + * Order of input processing and populating the XEVD_CDSC structure > + * 1. first, the corresponding fields of the AVCodecContext structure are processed, (i.e -threads 4) > + * 2. then xevd-specific options are added as AVOption to the xevd AVCodec implementation (i.e -threads 3) > + * 3. finally, the options specified after the xevd-params option as the parameter list of type key value are processed (i.e -xevd-params "m=2") I don't see a xevd-params option. #################### DONE #################### > + * > + * Some options can be set in different ways. In this case, please follow the above-mentioned order of processing. > + * The most recent assignments overwrite the previous values. > + * > + * @param[in] avctx codec context > + * @param[out] cdsc contains all encoder parameters that should be initialized before its use > + * > + * @return 0 on success, negative error code on failure I don't see a failure path below, so might as well just make this a void function. #################### DONE #################### > + */ > +static int get_conf(AVCodecContext *avctx, XEVD_CDSC *cdsc) > +{ > + int cpu_count = av_cpu_count(); > + > + /* clear XEVS_CDSC structure */ > + memset(cdsc, 0, sizeof(XEVD_CDSC)); > + > + /* init XEVD_CDSC */ > + if (avctx->thread_count <= 0) > + cdsc->threads = (cpu_count < XEVD_MAX_TASK_CNT) ? cpu_count : XEVD_MAX_TASK_CNT; > + else if (avctx->thread_count > XEVD_MAX_TASK_CNT) > + cdsc->threads = XEVD_MAX_TASK_CNT; > + else > + cdsc->threads = avctx->thread_count; > + > + return 0; > +} > + > +/** > + * Read NAL unit length > + * @param bs input data (bitstream) > + * @return the length of NAL unit on success, 0 value on failure > + */ > +static uint32_t read_nal_unit_length(const uint8_t *bs, int bs_size, AVCodecContext *avctx) > +{ > + uint32_t len = 0; > + XEVD_INFO info; > + int ret; > + > + if (bs_size == XEVD_NAL_UNIT_LENGTH_BYTE) { > + ret = xevd_info((void *)bs, XEVD_NAL_UNIT_LENGTH_BYTE, 1, &info); > + if (XEVD_FAILED(ret)) { > + av_log(avctx, AV_LOG_ERROR, "Cannot get bitstream information\n"); > + return 0; > + } > + len = info.nalu_len; > + if (len == 0) { > + av_log(avctx, AV_LOG_ERROR, "Invalid bitstream size! [%d]\n", bs_size); > + return 0; > + } > + } > + > + return len; > +} > + > +/** > + * @param[in] xectx the structure that stores all the state associated with the instance of Xeve MPEG-5 EVC decoder > + * @param[out] avctx codec context > + * @return 0 on success, negative value on failure > + */ > +static int export_stream_params(const XevdContext *xectx, AVCodecContext *avctx) > +{ > + int ret; > + int size; > + int color_space; > + > + avctx->pix_fmt = AV_PIX_FMT_YUV420P10; > + > + size = 4; > + ret = xevd_config(xectx->id, XEVD_CFG_GET_CODED_WIDTH, &avctx->coded_width, &size); > + if (XEVD_FAILED(ret)) { > + av_log(avctx, AV_LOG_ERROR, "Failed to get coded_width\n"); > + return AVERROR_EXTERNAL; > + } > + > + ret = xevd_config(xectx->id, XEVD_CFG_GET_CODED_HEIGHT, &avctx->coded_height, &size); > + if (XEVD_FAILED(ret)) { > + av_log(avctx, AV_LOG_ERROR, "Failed to get coded_height\n"); > + return AVERROR_EXTERNAL; > + } > + > + ret = xevd_config(xectx->id, XEVD_CFG_GET_WIDTH, &avctx->width, &size); > + if (XEVD_FAILED(ret)) { > + av_log(avctx, AV_LOG_ERROR, "Failed to get width\n"); > + return AVERROR_EXTERNAL; > + } > + > + ret = xevd_config(xectx->id, XEVD_CFG_GET_HEIGHT, &avctx->height, &size); > + if (XEVD_FAILED(ret)) { > + av_log(avctx, AV_LOG_ERROR, "Failed to get height\n"); > + return AVERROR_EXTERNAL; > + } > + > + ret = xevd_config(xectx->id, XEVD_CFG_GET_COLOR_SPACE, &color_space, &size); > + if (XEVD_FAILED(ret)) { > + av_log(avctx, AV_LOG_ERROR, "Failed to get color_space\n"); > + return AVERROR_EXTERNAL; > + } > + switch(color_space) { > + case XEVD_CS_YCBCR400_10LE: > + avctx->pix_fmt = AV_PIX_FMT_GRAY10LE; > + break; > + case XEVD_CS_YCBCR420_10LE: > + avctx->pix_fmt = AV_PIX_FMT_YUV420P10LE; > + break; > + case XEVD_CS_YCBCR422_10LE: > + avctx->pix_fmt = AV_PIX_FMT_YUV422P10LE; > + break; > + case XEVD_CS_YCBCR444_10LE: > + avctx->pix_fmt = AV_PIX_FMT_YUV444P10LE; > + break; > + default: > + av_log(avctx, AV_LOG_ERROR, "Unknown color space\n"); > + avctx->pix_fmt = AV_PIX_FMT_NONE; > + return AVERROR_INVALIDDATA; > + } > + > + // the function returns sps->num_reorder_pics > + ret = xevd_config(xectx->id, XEVD_CFG_GET_MAX_CODING_DELAY, &avctx->max_b_frames, &size); > + if (XEVD_FAILED(ret)) { > + av_log(avctx, AV_LOG_ERROR, "Failed to get max_coding_delay\n"); > + return AVERROR_EXTERNAL; > + } > + > + avctx->has_b_frames = (avctx->max_b_frames) ? 1 : 0; > + > + avctx->color_primaries = AVCOL_PRI_UNSPECIFIED; > + avctx->color_trc = AVCOL_TRC_UNSPECIFIED; > + avctx->colorspace = AVCOL_SPC_UNSPECIFIED; > + > + return 0; > +} > + > +/** > + * Initialize decoder > + * Create a decoder instance and allocate all the needed resources > + * > + * @param avctx codec context > + * @return 0 on success, negative error code on failure > + */ > +static av_cold int libxevd_init(AVCodecContext *avctx) > +{ > + XevdContext *xectx = avctx->priv_data; > + int ret = 0; > + XEVD_CDSC *cdsc = &(xectx->cdsc); > + > + /* read configurations and set values for created descriptor (XEVD_CDSC) */ > + ret = get_conf(avctx, cdsc); > + if (ret != 0) { > + av_log(avctx, AV_LOG_ERROR, "Cannot get configuration\n"); > + return AVERROR_INVALIDDATA; > + } > + > + /* create decoder */ > + xectx->id = xevd_create(&(xectx->cdsc), NULL); > + if (xectx->id == NULL) { > + av_log(avctx, AV_LOG_ERROR, "Cannot create XEVD encoder\n"); > + return AVERROR_EXTERNAL; > + } > + > + xectx->packet_count = 0; > + xectx->decod_frames = 0; > + > + return 0; > +} > + > +/** > + * Decode picture > + * > + * @param avctx codec context > + * @param[out] frame decoded frame > + * @param[out] got_frame_ptr decoder sets to 0 or 1 to indicate that a > + * non-empty frame or subtitle was returned in > + * outdata. > + * @param[in] avpkt AVPacket containing encoded data to be decoded > + * > + * @return amount of bytes read from the packet on success, negative error > + * code on failure > + */ > +static int libxevd_decode(struct AVCodecContext *avctx, struct AVFrame *frame, int *got_frame_ptr, AVPacket *avpkt) > +{ > + XevdContext *xectx = NULL; > + XEVD_IMGB *imgb = NULL; > + XEVD_STAT stat; > + XEVD_BITB bitb; > + int xevd_ret, nalu_size, bs_read_pos; > + int ret = 0; > + > + if (avctx == NULL) { > + av_log(avctx, AV_LOG_ERROR, "Invalid input parameter: AVCodecContext\n"); > + return AVERROR(EINVAL); > + } > + xectx = avctx->priv_data; > + if (xectx == NULL) { > + av_log(avctx, AV_LOG_ERROR, "Invalid XEVD context\n"); > + return AVERROR(EINVAL); This can't happen, just remove the check. #################### DONE #################### > + } > + > + if (avpkt->size > 0) { > + bs_read_pos = 0; > + imgb = NULL; > + while(avpkt->size > (bs_read_pos + XEVD_NAL_UNIT_LENGTH_BYTE)) { > + > + memset(&stat, 0, sizeof(XEVD_STAT)); > + memset(&bitb, 0, sizeof(XEVD_BITB)); > + > + nalu_size = read_nal_unit_length(avpkt->data + bs_read_pos, XEVD_NAL_UNIT_LENGTH_BYTE, avctx); > + if (nalu_size == 0) { > + av_log(avctx, AV_LOG_ERROR, "Invalid bitstream\n"); > + ret = AVERROR_INVALIDDATA; > + goto ERR; > + } > + bs_read_pos += XEVD_NAL_UNIT_LENGTH_BYTE; > + > + bitb.addr = avpkt->data + bs_read_pos; > + bitb.ssize = nalu_size; > + > + /* main decoding block */ > + xevd_ret = xevd_decode(xectx->id, &bitb, &stat); > + if (XEVD_FAILED(xevd_ret)) { > + av_log(avctx, AV_LOG_ERROR, "Failed to decode bitstream\n"); > + ret = AVERROR_EXTERNAL; > + goto ERR; > + } > + > + bs_read_pos += nalu_size; > + > + if (stat.nalu_type == XEVD_NUT_SPS) { // EVC stream parameters changed > + if ((ret = export_stream_params(xectx, avctx)) != 0) > + goto ERR; > + } > + > + if (stat.read != nalu_size) > + av_log(avctx, AV_LOG_INFO, "Different reading of bitstream (in:%d,read:%d)\n,", nalu_size, stat.read); > + if (stat.fnum >= 0) { > + // already has a decoded image > + if (imgb) { > + // xevd_pull uses pool of objects of type XEVD_IMGB. > + // The pool size is equal MAX_PB_SIZE (26), so release object when it is no more needed > + imgb->release(imgb); > + imgb = NULL; > + } > + xevd_ret = xevd_pull(xectx->id, &imgb); > + if (XEVD_FAILED(xevd_ret)) { > + av_log(avctx, AV_LOG_ERROR, "Failed to pull the decoded image (xevd error code: %d, frame#=%d)\n", xevd_ret, stat.fnum); > + ret = AVERROR_EXTERNAL; > + goto ERR; > + } else if (xevd_ret == XEVD_OK_FRM_DELAYED) { > + if (imgb) { > + // xevd_pull uses pool of objects of type XEVD_IMGB. > + // The pool size is equal MAX_PB_SIZE (26), so release object when it is no more needed > + imgb->release(imgb); > + imgb = NULL; > + } > + } So if the packet has more than one frame, you'll drop the oldest? #################### DONE #################### Maybe you should write this decoder using the decouple Input/Output API, FF_CODEC_RECEIVE_FRAME_CB() callback instead of FF_CODEC_DECODE_CB(). With it, you can request a packet as needed with ff_decode_get_packet(), so for example only when libxevd has run out of frames to output you actually request another. > + } > + } > + } else { // bumping > + xevd_ret = xevd_pull(xectx->id, &(imgb)); > + if (xevd_ret == XEVD_ERR_UNEXPECTED) { // bumping process completed Shouldn't this be XEVD_OK_NO_MORE_FRM? -------------------- No. It should't be XEVD_OK_NO_MORE_FRM. It means that bumping process has completed -------------------- > + *got_frame_ptr = 0; > + return 0; > + } else if (XEVD_FAILED(xevd_ret)) { > + av_log(avctx, AV_LOG_ERROR, "Failed to pull the decoded image (xevd error code: %d)\n", xevd_ret); > + ret = AVERROR_EXTERNAL; > + goto ERR; > + } > + } > + > + if (imgb) { > + // @todo supports other color space and bit depth > + if (imgb->cs != XEVD_CS_YCBCR420_10LE) { > + av_log(avctx, AV_LOG_ERROR, "Not supported pixel format: %s\n", av_get_pix_fmt_name(avctx->pix_fmt)); > + ret = AVERROR_INVALIDDATA; > + goto ERR; > + } > + > + if (imgb->w[0] != avctx->width || imgb->h[0] != avctx->height) { // stream resolution changed > + if (ff_set_dimensions(avctx, imgb->w[0], imgb->h[0]) < 0) { > + av_log(avctx, AV_LOG_ERROR, "Cannot set new dimension\n"); > + ret = AVERROR_INVALIDDATA; > + goto ERR; > + } > + } > + > + frame->coded_picture_number++; > + frame->display_picture_number++; As is, these two will always be 1. If you want to set these fields, you need to keep track of total decoded frames elsewhere. stat.fnum looks like what you need, or maybe xectx->decod_frames. #################### DONE -------------------- Removed #################### > + frame->format = AV_PIX_FMT_YUV420P10LE; No need to set this here. ff_get_buffer() will handle it for you. #################### DONE -------------------- Removed #################### > + > + if (ff_get_buffer(avctx, frame, 0) < 0) { > + av_log(avctx, AV_LOG_ERROR, "Cannot get AV buffer\n"); ff_get_buffer() already prints an error, so remove this one. > + ret = AVERROR(ENOMEM); It also returns an error code you should propagate. #################### DONE #################### > + goto ERR; > + } > + > + frame->pts = avpkt->pts; > + > + av_image_copy(frame->data, frame->linesize, (const uint8_t **)imgb->a, > + imgb->s, avctx->pix_fmt, > + imgb->w[0], imgb->h[0]); > + > + xectx->decod_frames++; Write only variable. #################### DONE #################### > + *got_frame_ptr = 1; > + > + // xevd_pull uses pool of objects of type XEVD_IMGB. > + // The pool size is equal MAX_PB_SIZE (26), so release object when it is no more needed > + imgb->release(imgb); > + imgb = NULL; > + } else > + *got_frame_ptr = 0; > + > + xectx->packet_count++; Same. #################### DONE #################### > + > + return avpkt->size; > + > +ERR: > + if (imgb) { > + imgb->release(imgb); > + imgb = NULL; > + } > + *got_frame_ptr = 0; > + > + return ret; > +} > + > +/** > + * Destroy decoder > + * > + * @param avctx codec context > + * @return 0 on success > + */ > +static av_cold int libxevd_close(AVCodecContext *avctx) > +{ > + XevdContext *xectx = avctx->priv_data; > + if (xectx->id) { > + xevd_delete(xectx->id); > + xectx->id = NULL; > + } > + > + return 0; > +} > + > +#define OFFSET(x) offsetof(XevdContext, x) > +#define VD AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_DECODING_PARAM > + > +static const AVClass libxevd_class = { > + .class_name = "libxevd", > + .item_name = av_default_item_name, > + .version = LIBAVUTIL_VERSION_INT, > +}; > + > +const FFCodec ff_libxevd_decoder = { > + .p.name = "evc", > + .p.long_name = NULL_IF_CONFIG_SMALL("EVC / MPEG-5 Essential Video Coding (EVC)"), > + .p.type = AVMEDIA_TYPE_VIDEO, > + .p.id = AV_CODEC_ID_EVC, > + .init = libxevd_init, > + FF_CODEC_DECODE_CB(libxevd_decode), > + .close = libxevd_close, > + .priv_data_size = sizeof(XevdContext), > + .p.priv_class = &libxevd_class, > + .p.capabilities = AV_CODEC_CAP_DELAY | AV_CODEC_CAP_AUTO_THREADS | AV_CODEC_CAP_AVOID_PROBING | AV_CODEC_CAP_DR1, Use AV_CODEC_CAP_OTHER_THREADS, not AV_CODEC_CAP_AUTO_THREADS. And set caps_internal to FF_CODEC_CAP_AUTO_THREADS. #################### DONE #################### > + .p.wrapper_name = "libxevd", > +}; _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [FFmpeg-devel] [PATCH 1/2] Provided support for MPEG-5 EVC (Essential Video Coding) codec 2022-08-01 15:09 ` James Almer 2022-08-08 12:44 ` Dawid Kozinski/Robot SDK (PLT) /SRPOL/Staff Engineer/삼성전자 @ 2022-08-11 6:49 ` Dawid Kozinski/Robot SDK (PLT) /SRPOL/Staff Engineer/삼성전자 2022-08-18 8:59 ` Dawid Kozinski/Robot SDK (PLT) /SRPOL/Staff Engineer/삼성전자 2 siblings, 0 replies; 13+ messages in thread From: Dawid Kozinski/Robot SDK (PLT) /SRPOL/Staff Engineer/삼성전자 @ 2022-08-11 6:49 UTC (permalink / raw) To: 'FFmpeg development discussions and patches' Hi James Almer, Thank you for your last code review. I did a lot of changes to our EVC wrapper implementation following your comments from the last code review and I'm looking forward to receiving your feedback. We would like to start getting closer to the point where we can be able to merge our code into ffmpeg master branch. :) Best regards Dawid -----Original Message----- From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of James Almer Sent: Monday, August 1, 2022 5:09 PM To: ffmpeg-devel@ffmpeg.org Subject: Re: [FFmpeg-devel] [PATCH 1/2] Provided support for MPEG-5 EVC (Essential Video Coding) codec On 8/1/2022 6:28 AM, Dawid Kozinski wrote: > diff --git a/libavcodec/libxevd.c b/libavcodec/libxevd.c > new file mode 100644 > index 0000000000..8ba2d89cbd > --- /dev/null > +++ b/libavcodec/libxevd.c > @@ -0,0 +1,436 @@ > +/* > + * libxevd decoder > + * EVC (MPEG-5 Essential Video Coding) decoding using XEVD MPEG-5 EVC decoder library > + * > + * Copyright (C) 2021 Dawid Kozinski<d.kozinski@samsung.com> > + * > + * This file is part of FFmpeg. > + * > + * FFmpeg is free software; you can redistribute it and/or > + * modify it under the terms of the GNU Lesser General Public > + * License as published by the Free Software Foundation; either > + * version 2.1 of the License, or (at your option) any later version. > + * > + * FFmpeg is distributed in the hope that it will be useful, > + * but WITHOUT ANY WARRANTY; without even the implied warranty of > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > + * Lesser General Public License for more details. > + * > + * You should have received a copy of the GNU Lesser General Public > + * License along with FFmpeg; if not, write to the Free Software > + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA > + */ > + > +#include <float.h> > +#include <stdlib.h> > + > +#include <xevd.h> > + > +#include "libavutil/internal.h" > +#include "libavutil/common.h" > +#include "libavutil/opt.h" > +#include "libavutil/pixdesc.h" > +#include "libavutil/pixfmt.h" > +#include "libavutil/imgutils.h" > +#include "libavutil/cpu.h" > + > +#include "avcodec.h" > +#include "internal.h" > +#include "packet_internal.h" > +#include "codec_internal.h" > + > +#define XEVD_PARAM_BAD_NAME -1 > +#define XEVD_PARAM_BAD_VALUE -2 > + > +#define EVC_NAL_HEADER_SIZE 2 /* byte */ > + > +/** > + * The structure stores all the states associated with the instance of Xeve MPEG-5 EVC decoder > + */ > +typedef struct XevdContext { > + const AVClass *class; > + > + XEVD id; // XEVD instance identifier @see xevd.h > + XEVD_CDSC cdsc; // decoding parameters @see xevd.h > + > + int decod_frames; // number of decoded frames > + int packet_count; // number of packets created by decoder > +} XevdContext; > + > +/** > + * The function returns a pointer to a variable of type XEVD_CDSC. This doesn't seem to be the case. > + * XEVD_CDSC contains all decoder parameters that should be initialized before its use. > + * > + * The field values of the XEVD_CDSC structure are populated based on: > + * - the corresponding field values of the AvCodecConetxt structure, > + * - the xevd decoder specific option values, > + * (the full list of options available for the xevd encoder is displayed after executing the command ./ffmpeg --help decoder = libxevd) > + * - and the xevd encoder options specified as a list of key-value pairs following the xevd-params option > + * > + * Order of input processing and populating the XEVD_CDSC structure > + * 1. first, the corresponding fields of the AVCodecContext structure are processed, (i.e -threads 4) > + * 2. then xevd-specific options are added as AVOption to the xevd AVCodec implementation (i.e -threads 3) > + * 3. finally, the options specified after the xevd-params option as the parameter list of type key value are processed (i.e -xevd-params "m=2") I don't see a xevd-params option. > + * > + * Some options can be set in different ways. In this case, please follow the above-mentioned order of processing. > + * The most recent assignments overwrite the previous values. > + * > + * @param[in] avctx codec context > + * @param[out] cdsc contains all encoder parameters that should be initialized before its use > + * > + * @return 0 on success, negative error code on failure I don't see a failure path below, so might as well just make this a void function. > + */ > +static int get_conf(AVCodecContext *avctx, XEVD_CDSC *cdsc) > +{ > + int cpu_count = av_cpu_count(); > + > + /* clear XEVS_CDSC structure */ > + memset(cdsc, 0, sizeof(XEVD_CDSC)); > + > + /* init XEVD_CDSC */ > + if (avctx->thread_count <= 0) > + cdsc->threads = (cpu_count < XEVD_MAX_TASK_CNT) ? cpu_count : XEVD_MAX_TASK_CNT; > + else if (avctx->thread_count > XEVD_MAX_TASK_CNT) > + cdsc->threads = XEVD_MAX_TASK_CNT; > + else > + cdsc->threads = avctx->thread_count; > + > + return 0; > +} > + > +/** > + * Read NAL unit length > + * @param bs input data (bitstream) > + * @return the length of NAL unit on success, 0 value on failure > + */ > +static uint32_t read_nal_unit_length(const uint8_t *bs, int bs_size, AVCodecContext *avctx) > +{ > + uint32_t len = 0; > + XEVD_INFO info; > + int ret; > + > + if (bs_size == XEVD_NAL_UNIT_LENGTH_BYTE) { > + ret = xevd_info((void *)bs, XEVD_NAL_UNIT_LENGTH_BYTE, 1, &info); > + if (XEVD_FAILED(ret)) { > + av_log(avctx, AV_LOG_ERROR, "Cannot get bitstream information\n"); > + return 0; > + } > + len = info.nalu_len; > + if (len == 0) { > + av_log(avctx, AV_LOG_ERROR, "Invalid bitstream size! [%d]\n", bs_size); > + return 0; > + } > + } > + > + return len; > +} > + > +/** > + * @param[in] xectx the structure that stores all the state associated with the instance of Xeve MPEG-5 EVC decoder > + * @param[out] avctx codec context > + * @return 0 on success, negative value on failure > + */ > +static int export_stream_params(const XevdContext *xectx, AVCodecContext *avctx) > +{ > + int ret; > + int size; > + int color_space; > + > + avctx->pix_fmt = AV_PIX_FMT_YUV420P10; > + > + size = 4; > + ret = xevd_config(xectx->id, XEVD_CFG_GET_CODED_WIDTH, &avctx->coded_width, &size); > + if (XEVD_FAILED(ret)) { > + av_log(avctx, AV_LOG_ERROR, "Failed to get coded_width\n"); > + return AVERROR_EXTERNAL; > + } > + > + ret = xevd_config(xectx->id, XEVD_CFG_GET_CODED_HEIGHT, &avctx->coded_height, &size); > + if (XEVD_FAILED(ret)) { > + av_log(avctx, AV_LOG_ERROR, "Failed to get coded_height\n"); > + return AVERROR_EXTERNAL; > + } > + > + ret = xevd_config(xectx->id, XEVD_CFG_GET_WIDTH, &avctx->width, &size); > + if (XEVD_FAILED(ret)) { > + av_log(avctx, AV_LOG_ERROR, "Failed to get width\n"); > + return AVERROR_EXTERNAL; > + } > + > + ret = xevd_config(xectx->id, XEVD_CFG_GET_HEIGHT, &avctx->height, &size); > + if (XEVD_FAILED(ret)) { > + av_log(avctx, AV_LOG_ERROR, "Failed to get height\n"); > + return AVERROR_EXTERNAL; > + } > + > + ret = xevd_config(xectx->id, XEVD_CFG_GET_COLOR_SPACE, &color_space, &size); > + if (XEVD_FAILED(ret)) { > + av_log(avctx, AV_LOG_ERROR, "Failed to get color_space\n"); > + return AVERROR_EXTERNAL; > + } > + switch(color_space) { > + case XEVD_CS_YCBCR400_10LE: > + avctx->pix_fmt = AV_PIX_FMT_GRAY10LE; > + break; > + case XEVD_CS_YCBCR420_10LE: > + avctx->pix_fmt = AV_PIX_FMT_YUV420P10LE; > + break; > + case XEVD_CS_YCBCR422_10LE: > + avctx->pix_fmt = AV_PIX_FMT_YUV422P10LE; > + break; > + case XEVD_CS_YCBCR444_10LE: > + avctx->pix_fmt = AV_PIX_FMT_YUV444P10LE; > + break; > + default: > + av_log(avctx, AV_LOG_ERROR, "Unknown color space\n"); > + avctx->pix_fmt = AV_PIX_FMT_NONE; > + return AVERROR_INVALIDDATA; > + } > + > + // the function returns sps->num_reorder_pics > + ret = xevd_config(xectx->id, XEVD_CFG_GET_MAX_CODING_DELAY, &avctx->max_b_frames, &size); > + if (XEVD_FAILED(ret)) { > + av_log(avctx, AV_LOG_ERROR, "Failed to get max_coding_delay\n"); > + return AVERROR_EXTERNAL; > + } > + > + avctx->has_b_frames = (avctx->max_b_frames) ? 1 : 0; > + > + avctx->color_primaries = AVCOL_PRI_UNSPECIFIED; > + avctx->color_trc = AVCOL_TRC_UNSPECIFIED; > + avctx->colorspace = AVCOL_SPC_UNSPECIFIED; > + > + return 0; > +} > + > +/** > + * Initialize decoder > + * Create a decoder instance and allocate all the needed resources > + * > + * @param avctx codec context > + * @return 0 on success, negative error code on failure > + */ > +static av_cold int libxevd_init(AVCodecContext *avctx) > +{ > + XevdContext *xectx = avctx->priv_data; > + int ret = 0; > + XEVD_CDSC *cdsc = &(xectx->cdsc); > + > + /* read configurations and set values for created descriptor (XEVD_CDSC) */ > + ret = get_conf(avctx, cdsc); > + if (ret != 0) { > + av_log(avctx, AV_LOG_ERROR, "Cannot get configuration\n"); > + return AVERROR_INVALIDDATA; > + } > + > + /* create decoder */ > + xectx->id = xevd_create(&(xectx->cdsc), NULL); > + if (xectx->id == NULL) { > + av_log(avctx, AV_LOG_ERROR, "Cannot create XEVD encoder\n"); > + return AVERROR_EXTERNAL; > + } > + > + xectx->packet_count = 0; > + xectx->decod_frames = 0; > + > + return 0; > +} > + > +/** > + * Decode picture > + * > + * @param avctx codec context > + * @param[out] frame decoded frame > + * @param[out] got_frame_ptr decoder sets to 0 or 1 to indicate that a > + * non-empty frame or subtitle was returned in > + * outdata. > + * @param[in] avpkt AVPacket containing encoded data to be decoded > + * > + * @return amount of bytes read from the packet on success, negative error > + * code on failure > + */ > +static int libxevd_decode(struct AVCodecContext *avctx, struct AVFrame *frame, int *got_frame_ptr, AVPacket *avpkt) > +{ > + XevdContext *xectx = NULL; > + XEVD_IMGB *imgb = NULL; > + XEVD_STAT stat; > + XEVD_BITB bitb; > + int xevd_ret, nalu_size, bs_read_pos; > + int ret = 0; > + > + if (avctx == NULL) { > + av_log(avctx, AV_LOG_ERROR, "Invalid input parameter: AVCodecContext\n"); > + return AVERROR(EINVAL); > + } > + xectx = avctx->priv_data; > + if (xectx == NULL) { > + av_log(avctx, AV_LOG_ERROR, "Invalid XEVD context\n"); > + return AVERROR(EINVAL); This can't happen, just remove the check. > + } > + > + if (avpkt->size > 0) { > + bs_read_pos = 0; > + imgb = NULL; > + while(avpkt->size > (bs_read_pos + XEVD_NAL_UNIT_LENGTH_BYTE)) { > + > + memset(&stat, 0, sizeof(XEVD_STAT)); > + memset(&bitb, 0, sizeof(XEVD_BITB)); > + > + nalu_size = read_nal_unit_length(avpkt->data + bs_read_pos, XEVD_NAL_UNIT_LENGTH_BYTE, avctx); > + if (nalu_size == 0) { > + av_log(avctx, AV_LOG_ERROR, "Invalid bitstream\n"); > + ret = AVERROR_INVALIDDATA; > + goto ERR; > + } > + bs_read_pos += XEVD_NAL_UNIT_LENGTH_BYTE; > + > + bitb.addr = avpkt->data + bs_read_pos; > + bitb.ssize = nalu_size; > + > + /* main decoding block */ > + xevd_ret = xevd_decode(xectx->id, &bitb, &stat); > + if (XEVD_FAILED(xevd_ret)) { > + av_log(avctx, AV_LOG_ERROR, "Failed to decode bitstream\n"); > + ret = AVERROR_EXTERNAL; > + goto ERR; > + } > + > + bs_read_pos += nalu_size; > + > + if (stat.nalu_type == XEVD_NUT_SPS) { // EVC stream parameters changed > + if ((ret = export_stream_params(xectx, avctx)) != 0) > + goto ERR; > + } > + > + if (stat.read != nalu_size) > + av_log(avctx, AV_LOG_INFO, "Different reading of bitstream (in:%d,read:%d)\n,", nalu_size, stat.read); > + if (stat.fnum >= 0) { > + // already has a decoded image > + if (imgb) { > + // xevd_pull uses pool of objects of type XEVD_IMGB. > + // The pool size is equal MAX_PB_SIZE (26), so release object when it is no more needed > + imgb->release(imgb); > + imgb = NULL; > + } > + xevd_ret = xevd_pull(xectx->id, &imgb); > + if (XEVD_FAILED(xevd_ret)) { > + av_log(avctx, AV_LOG_ERROR, "Failed to pull the decoded image (xevd error code: %d, frame#=%d)\n", xevd_ret, stat.fnum); > + ret = AVERROR_EXTERNAL; > + goto ERR; > + } else if (xevd_ret == XEVD_OK_FRM_DELAYED) { > + if (imgb) { > + // xevd_pull uses pool of objects of type XEVD_IMGB. > + // The pool size is equal MAX_PB_SIZE (26), so release object when it is no more needed > + imgb->release(imgb); > + imgb = NULL; > + } > + } So if the packet has more than one frame, you'll drop the oldest? Maybe you should write this decoder using the decouple Input/Output API, FF_CODEC_RECEIVE_FRAME_CB() callback instead of FF_CODEC_DECODE_CB(). With it, you can request a packet as needed with ff_decode_get_packet(), so for example only when libxevd has run out of frames to output you actually request another. > + } > + } > + } else { // bumping > + xevd_ret = xevd_pull(xectx->id, &(imgb)); > + if (xevd_ret == XEVD_ERR_UNEXPECTED) { // bumping process completed Shouldn't this be XEVD_OK_NO_MORE_FRM? > + *got_frame_ptr = 0; > + return 0; > + } else if (XEVD_FAILED(xevd_ret)) { > + av_log(avctx, AV_LOG_ERROR, "Failed to pull the decoded image (xevd error code: %d)\n", xevd_ret); > + ret = AVERROR_EXTERNAL; > + goto ERR; > + } > + } > + > + if (imgb) { > + // @todo supports other color space and bit depth > + if (imgb->cs != XEVD_CS_YCBCR420_10LE) { > + av_log(avctx, AV_LOG_ERROR, "Not supported pixel format: %s\n", av_get_pix_fmt_name(avctx->pix_fmt)); > + ret = AVERROR_INVALIDDATA; > + goto ERR; > + } > + > + if (imgb->w[0] != avctx->width || imgb->h[0] != avctx->height) { // stream resolution changed > + if (ff_set_dimensions(avctx, imgb->w[0], imgb->h[0]) < 0) { > + av_log(avctx, AV_LOG_ERROR, "Cannot set new dimension\n"); > + ret = AVERROR_INVALIDDATA; > + goto ERR; > + } > + } > + > + frame->coded_picture_number++; > + frame->display_picture_number++; As is, these two will always be 1. If you want to set these fields, you need to keep track of total decoded frames elsewhere. stat.fnum looks like what you need, or maybe xectx->decod_frames. > + frame->format = AV_PIX_FMT_YUV420P10LE; No need to set this here. ff_get_buffer() will handle it for you. > + > + if (ff_get_buffer(avctx, frame, 0) < 0) { > + av_log(avctx, AV_LOG_ERROR, "Cannot get AV buffer\n"); ff_get_buffer() already prints an error, so remove this one. > + ret = AVERROR(ENOMEM); It also returns an error code you should propagate. > + goto ERR; > + } > + > + frame->pts = avpkt->pts; > + > + av_image_copy(frame->data, frame->linesize, (const uint8_t **)imgb->a, > + imgb->s, avctx->pix_fmt, > + imgb->w[0], imgb->h[0]); > + > + xectx->decod_frames++; Write only variable. > + *got_frame_ptr = 1; > + > + // xevd_pull uses pool of objects of type XEVD_IMGB. > + // The pool size is equal MAX_PB_SIZE (26), so release object when it is no more needed > + imgb->release(imgb); > + imgb = NULL; > + } else > + *got_frame_ptr = 0; > + > + xectx->packet_count++; Same. > + > + return avpkt->size; > + > +ERR: > + if (imgb) { > + imgb->release(imgb); > + imgb = NULL; > + } > + *got_frame_ptr = 0; > + > + return ret; > +} > + > +/** > + * Destroy decoder > + * > + * @param avctx codec context > + * @return 0 on success > + */ > +static av_cold int libxevd_close(AVCodecContext *avctx) > +{ > + XevdContext *xectx = avctx->priv_data; > + if (xectx->id) { > + xevd_delete(xectx->id); > + xectx->id = NULL; > + } > + > + return 0; > +} > + > +#define OFFSET(x) offsetof(XevdContext, x) > +#define VD AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_DECODING_PARAM > + > +static const AVClass libxevd_class = { > + .class_name = "libxevd", > + .item_name = av_default_item_name, > + .version = LIBAVUTIL_VERSION_INT, > +}; > + > +const FFCodec ff_libxevd_decoder = { > + .p.name = "evc", > + .p.long_name = NULL_IF_CONFIG_SMALL("EVC / MPEG-5 Essential Video Coding (EVC)"), > + .p.type = AVMEDIA_TYPE_VIDEO, > + .p.id = AV_CODEC_ID_EVC, > + .init = libxevd_init, > + FF_CODEC_DECODE_CB(libxevd_decode), > + .close = libxevd_close, > + .priv_data_size = sizeof(XevdContext), > + .p.priv_class = &libxevd_class, > + .p.capabilities = AV_CODEC_CAP_DELAY | AV_CODEC_CAP_AUTO_THREADS | AV_CODEC_CAP_AVOID_PROBING | AV_CODEC_CAP_DR1, Use AV_CODEC_CAP_OTHER_THREADS, not AV_CODEC_CAP_AUTO_THREADS. And set caps_internal to FF_CODEC_CAP_AUTO_THREADS. > + .p.wrapper_name = "libxevd", > +}; _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [FFmpeg-devel] [PATCH 1/2] Provided support for MPEG-5 EVC (Essential Video Coding) codec 2022-08-01 15:09 ` James Almer 2022-08-08 12:44 ` Dawid Kozinski/Robot SDK (PLT) /SRPOL/Staff Engineer/삼성전자 2022-08-11 6:49 ` Dawid Kozinski/Robot SDK (PLT) /SRPOL/Staff Engineer/삼성전자 @ 2022-08-18 8:59 ` Dawid Kozinski/Robot SDK (PLT) /SRPOL/Staff Engineer/삼성전자 2 siblings, 0 replies; 13+ messages in thread From: Dawid Kozinski/Robot SDK (PLT) /SRPOL/Staff Engineer/삼성전자 @ 2022-08-18 8:59 UTC (permalink / raw) To: 'FFmpeg development discussions and patches' Hi, Thank you for your code review and for all your valuable comments and remarks. We have done a lot of changes to our EVC wrapper implementation following them, and I think each of these changes have moved our wrapper one step forwards toward a final community-acceptable solution. A few days ago I submitted new patches providing EVC wrapper implementation. If there is still something we have to change or improve, please let me know. I'm looking forward to receiving your feedback. As I mentioned before we would like to start getting closer to the point where we can be able to merge our code into the FFmpeg master branch. :) Once again, thank you for your effort and for supporting us and we are waiting for your feedback. Best regards Dawid -----Original Message----- From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of James Almer Sent: Monday, August 1, 2022 5:09 PM To: ffmpeg-devel@ffmpeg.org Subject: Re: [FFmpeg-devel] [PATCH 1/2] Provided support for MPEG-5 EVC (Essential Video Coding) codec On 8/1/2022 6:28 AM, Dawid Kozinski wrote: > diff --git a/libavcodec/libxevd.c b/libavcodec/libxevd.c > new file mode 100644 > index 0000000000..8ba2d89cbd > --- /dev/null > +++ b/libavcodec/libxevd.c > @@ -0,0 +1,436 @@ > +/* > + * libxevd decoder > + * EVC (MPEG-5 Essential Video Coding) decoding using XEVD MPEG-5 EVC decoder library > + * > + * Copyright (C) 2021 Dawid Kozinski<d.kozinski@samsung.com> > + * > + * This file is part of FFmpeg. > + * > + * FFmpeg is free software; you can redistribute it and/or > + * modify it under the terms of the GNU Lesser General Public > + * License as published by the Free Software Foundation; either > + * version 2.1 of the License, or (at your option) any later version. > + * > + * FFmpeg is distributed in the hope that it will be useful, > + * but WITHOUT ANY WARRANTY; without even the implied warranty of > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > + * Lesser General Public License for more details. > + * > + * You should have received a copy of the GNU Lesser General Public > + * License along with FFmpeg; if not, write to the Free Software > + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA > + */ > + > +#include <float.h> > +#include <stdlib.h> > + > +#include <xevd.h> > + > +#include "libavutil/internal.h" > +#include "libavutil/common.h" > +#include "libavutil/opt.h" > +#include "libavutil/pixdesc.h" > +#include "libavutil/pixfmt.h" > +#include "libavutil/imgutils.h" > +#include "libavutil/cpu.h" > + > +#include "avcodec.h" > +#include "internal.h" > +#include "packet_internal.h" > +#include "codec_internal.h" > + > +#define XEVD_PARAM_BAD_NAME -1 > +#define XEVD_PARAM_BAD_VALUE -2 > + > +#define EVC_NAL_HEADER_SIZE 2 /* byte */ > + > +/** > + * The structure stores all the states associated with the instance of Xeve MPEG-5 EVC decoder > + */ > +typedef struct XevdContext { > + const AVClass *class; > + > + XEVD id; // XEVD instance identifier @see xevd.h > + XEVD_CDSC cdsc; // decoding parameters @see xevd.h > + > + int decod_frames; // number of decoded frames > + int packet_count; // number of packets created by decoder > +} XevdContext; > + > +/** > + * The function returns a pointer to a variable of type XEVD_CDSC. This doesn't seem to be the case. > + * XEVD_CDSC contains all decoder parameters that should be initialized before its use. > + * > + * The field values of the XEVD_CDSC structure are populated based on: > + * - the corresponding field values of the AvCodecConetxt structure, > + * - the xevd decoder specific option values, > + * (the full list of options available for the xevd encoder is displayed after executing the command ./ffmpeg --help decoder = libxevd) > + * - and the xevd encoder options specified as a list of key-value pairs following the xevd-params option > + * > + * Order of input processing and populating the XEVD_CDSC structure > + * 1. first, the corresponding fields of the AVCodecContext structure are processed, (i.e -threads 4) > + * 2. then xevd-specific options are added as AVOption to the xevd AVCodec implementation (i.e -threads 3) > + * 3. finally, the options specified after the xevd-params option as the parameter list of type key value are processed (i.e -xevd-params "m=2") I don't see a xevd-params option. > + * > + * Some options can be set in different ways. In this case, please follow the above-mentioned order of processing. > + * The most recent assignments overwrite the previous values. > + * > + * @param[in] avctx codec context > + * @param[out] cdsc contains all encoder parameters that should be initialized before its use > + * > + * @return 0 on success, negative error code on failure I don't see a failure path below, so might as well just make this a void function. > + */ > +static int get_conf(AVCodecContext *avctx, XEVD_CDSC *cdsc) > +{ > + int cpu_count = av_cpu_count(); > + > + /* clear XEVS_CDSC structure */ > + memset(cdsc, 0, sizeof(XEVD_CDSC)); > + > + /* init XEVD_CDSC */ > + if (avctx->thread_count <= 0) > + cdsc->threads = (cpu_count < XEVD_MAX_TASK_CNT) ? cpu_count : XEVD_MAX_TASK_CNT; > + else if (avctx->thread_count > XEVD_MAX_TASK_CNT) > + cdsc->threads = XEVD_MAX_TASK_CNT; > + else > + cdsc->threads = avctx->thread_count; > + > + return 0; > +} > + > +/** > + * Read NAL unit length > + * @param bs input data (bitstream) > + * @return the length of NAL unit on success, 0 value on failure > + */ > +static uint32_t read_nal_unit_length(const uint8_t *bs, int bs_size, AVCodecContext *avctx) > +{ > + uint32_t len = 0; > + XEVD_INFO info; > + int ret; > + > + if (bs_size == XEVD_NAL_UNIT_LENGTH_BYTE) { > + ret = xevd_info((void *)bs, XEVD_NAL_UNIT_LENGTH_BYTE, 1, &info); > + if (XEVD_FAILED(ret)) { > + av_log(avctx, AV_LOG_ERROR, "Cannot get bitstream information\n"); > + return 0; > + } > + len = info.nalu_len; > + if (len == 0) { > + av_log(avctx, AV_LOG_ERROR, "Invalid bitstream size! [%d]\n", bs_size); > + return 0; > + } > + } > + > + return len; > +} > + > +/** > + * @param[in] xectx the structure that stores all the state associated with the instance of Xeve MPEG-5 EVC decoder > + * @param[out] avctx codec context > + * @return 0 on success, negative value on failure > + */ > +static int export_stream_params(const XevdContext *xectx, AVCodecContext *avctx) > +{ > + int ret; > + int size; > + int color_space; > + > + avctx->pix_fmt = AV_PIX_FMT_YUV420P10; > + > + size = 4; > + ret = xevd_config(xectx->id, XEVD_CFG_GET_CODED_WIDTH, &avctx->coded_width, &size); > + if (XEVD_FAILED(ret)) { > + av_log(avctx, AV_LOG_ERROR, "Failed to get coded_width\n"); > + return AVERROR_EXTERNAL; > + } > + > + ret = xevd_config(xectx->id, XEVD_CFG_GET_CODED_HEIGHT, &avctx->coded_height, &size); > + if (XEVD_FAILED(ret)) { > + av_log(avctx, AV_LOG_ERROR, "Failed to get coded_height\n"); > + return AVERROR_EXTERNAL; > + } > + > + ret = xevd_config(xectx->id, XEVD_CFG_GET_WIDTH, &avctx->width, &size); > + if (XEVD_FAILED(ret)) { > + av_log(avctx, AV_LOG_ERROR, "Failed to get width\n"); > + return AVERROR_EXTERNAL; > + } > + > + ret = xevd_config(xectx->id, XEVD_CFG_GET_HEIGHT, &avctx->height, &size); > + if (XEVD_FAILED(ret)) { > + av_log(avctx, AV_LOG_ERROR, "Failed to get height\n"); > + return AVERROR_EXTERNAL; > + } > + > + ret = xevd_config(xectx->id, XEVD_CFG_GET_COLOR_SPACE, &color_space, &size); > + if (XEVD_FAILED(ret)) { > + av_log(avctx, AV_LOG_ERROR, "Failed to get color_space\n"); > + return AVERROR_EXTERNAL; > + } > + switch(color_space) { > + case XEVD_CS_YCBCR400_10LE: > + avctx->pix_fmt = AV_PIX_FMT_GRAY10LE; > + break; > + case XEVD_CS_YCBCR420_10LE: > + avctx->pix_fmt = AV_PIX_FMT_YUV420P10LE; > + break; > + case XEVD_CS_YCBCR422_10LE: > + avctx->pix_fmt = AV_PIX_FMT_YUV422P10LE; > + break; > + case XEVD_CS_YCBCR444_10LE: > + avctx->pix_fmt = AV_PIX_FMT_YUV444P10LE; > + break; > + default: > + av_log(avctx, AV_LOG_ERROR, "Unknown color space\n"); > + avctx->pix_fmt = AV_PIX_FMT_NONE; > + return AVERROR_INVALIDDATA; > + } > + > + // the function returns sps->num_reorder_pics > + ret = xevd_config(xectx->id, XEVD_CFG_GET_MAX_CODING_DELAY, &avctx->max_b_frames, &size); > + if (XEVD_FAILED(ret)) { > + av_log(avctx, AV_LOG_ERROR, "Failed to get max_coding_delay\n"); > + return AVERROR_EXTERNAL; > + } > + > + avctx->has_b_frames = (avctx->max_b_frames) ? 1 : 0; > + > + avctx->color_primaries = AVCOL_PRI_UNSPECIFIED; > + avctx->color_trc = AVCOL_TRC_UNSPECIFIED; > + avctx->colorspace = AVCOL_SPC_UNSPECIFIED; > + > + return 0; > +} > + > +/** > + * Initialize decoder > + * Create a decoder instance and allocate all the needed resources > + * > + * @param avctx codec context > + * @return 0 on success, negative error code on failure > + */ > +static av_cold int libxevd_init(AVCodecContext *avctx) > +{ > + XevdContext *xectx = avctx->priv_data; > + int ret = 0; > + XEVD_CDSC *cdsc = &(xectx->cdsc); > + > + /* read configurations and set values for created descriptor (XEVD_CDSC) */ > + ret = get_conf(avctx, cdsc); > + if (ret != 0) { > + av_log(avctx, AV_LOG_ERROR, "Cannot get configuration\n"); > + return AVERROR_INVALIDDATA; > + } > + > + /* create decoder */ > + xectx->id = xevd_create(&(xectx->cdsc), NULL); > + if (xectx->id == NULL) { > + av_log(avctx, AV_LOG_ERROR, "Cannot create XEVD encoder\n"); > + return AVERROR_EXTERNAL; > + } > + > + xectx->packet_count = 0; > + xectx->decod_frames = 0; > + > + return 0; > +} > + > +/** > + * Decode picture > + * > + * @param avctx codec context > + * @param[out] frame decoded frame > + * @param[out] got_frame_ptr decoder sets to 0 or 1 to indicate that a > + * non-empty frame or subtitle was returned in > + * outdata. > + * @param[in] avpkt AVPacket containing encoded data to be decoded > + * > + * @return amount of bytes read from the packet on success, negative error > + * code on failure > + */ > +static int libxevd_decode(struct AVCodecContext *avctx, struct AVFrame *frame, int *got_frame_ptr, AVPacket *avpkt) > +{ > + XevdContext *xectx = NULL; > + XEVD_IMGB *imgb = NULL; > + XEVD_STAT stat; > + XEVD_BITB bitb; > + int xevd_ret, nalu_size, bs_read_pos; > + int ret = 0; > + > + if (avctx == NULL) { > + av_log(avctx, AV_LOG_ERROR, "Invalid input parameter: AVCodecContext\n"); > + return AVERROR(EINVAL); > + } > + xectx = avctx->priv_data; > + if (xectx == NULL) { > + av_log(avctx, AV_LOG_ERROR, "Invalid XEVD context\n"); > + return AVERROR(EINVAL); This can't happen, just remove the check. > + } > + > + if (avpkt->size > 0) { > + bs_read_pos = 0; > + imgb = NULL; > + while(avpkt->size > (bs_read_pos + XEVD_NAL_UNIT_LENGTH_BYTE)) { > + > + memset(&stat, 0, sizeof(XEVD_STAT)); > + memset(&bitb, 0, sizeof(XEVD_BITB)); > + > + nalu_size = read_nal_unit_length(avpkt->data + bs_read_pos, XEVD_NAL_UNIT_LENGTH_BYTE, avctx); > + if (nalu_size == 0) { > + av_log(avctx, AV_LOG_ERROR, "Invalid bitstream\n"); > + ret = AVERROR_INVALIDDATA; > + goto ERR; > + } > + bs_read_pos += XEVD_NAL_UNIT_LENGTH_BYTE; > + > + bitb.addr = avpkt->data + bs_read_pos; > + bitb.ssize = nalu_size; > + > + /* main decoding block */ > + xevd_ret = xevd_decode(xectx->id, &bitb, &stat); > + if (XEVD_FAILED(xevd_ret)) { > + av_log(avctx, AV_LOG_ERROR, "Failed to decode bitstream\n"); > + ret = AVERROR_EXTERNAL; > + goto ERR; > + } > + > + bs_read_pos += nalu_size; > + > + if (stat.nalu_type == XEVD_NUT_SPS) { // EVC stream parameters changed > + if ((ret = export_stream_params(xectx, avctx)) != 0) > + goto ERR; > + } > + > + if (stat.read != nalu_size) > + av_log(avctx, AV_LOG_INFO, "Different reading of bitstream (in:%d,read:%d)\n,", nalu_size, stat.read); > + if (stat.fnum >= 0) { > + // already has a decoded image > + if (imgb) { > + // xevd_pull uses pool of objects of type XEVD_IMGB. > + // The pool size is equal MAX_PB_SIZE (26), so release object when it is no more needed > + imgb->release(imgb); > + imgb = NULL; > + } > + xevd_ret = xevd_pull(xectx->id, &imgb); > + if (XEVD_FAILED(xevd_ret)) { > + av_log(avctx, AV_LOG_ERROR, "Failed to pull the decoded image (xevd error code: %d, frame#=%d)\n", xevd_ret, stat.fnum); > + ret = AVERROR_EXTERNAL; > + goto ERR; > + } else if (xevd_ret == XEVD_OK_FRM_DELAYED) { > + if (imgb) { > + // xevd_pull uses pool of objects of type XEVD_IMGB. > + // The pool size is equal MAX_PB_SIZE (26), so release object when it is no more needed > + imgb->release(imgb); > + imgb = NULL; > + } > + } So if the packet has more than one frame, you'll drop the oldest? Maybe you should write this decoder using the decouple Input/Output API, FF_CODEC_RECEIVE_FRAME_CB() callback instead of FF_CODEC_DECODE_CB(). With it, you can request a packet as needed with ff_decode_get_packet(), so for example only when libxevd has run out of frames to output you actually request another. > + } > + } > + } else { // bumping > + xevd_ret = xevd_pull(xectx->id, &(imgb)); > + if (xevd_ret == XEVD_ERR_UNEXPECTED) { // bumping process completed Shouldn't this be XEVD_OK_NO_MORE_FRM? > + *got_frame_ptr = 0; > + return 0; > + } else if (XEVD_FAILED(xevd_ret)) { > + av_log(avctx, AV_LOG_ERROR, "Failed to pull the decoded image (xevd error code: %d)\n", xevd_ret); > + ret = AVERROR_EXTERNAL; > + goto ERR; > + } > + } > + > + if (imgb) { > + // @todo supports other color space and bit depth > + if (imgb->cs != XEVD_CS_YCBCR420_10LE) { > + av_log(avctx, AV_LOG_ERROR, "Not supported pixel format: %s\n", av_get_pix_fmt_name(avctx->pix_fmt)); > + ret = AVERROR_INVALIDDATA; > + goto ERR; > + } > + > + if (imgb->w[0] != avctx->width || imgb->h[0] != avctx->height) { // stream resolution changed > + if (ff_set_dimensions(avctx, imgb->w[0], imgb->h[0]) < 0) { > + av_log(avctx, AV_LOG_ERROR, "Cannot set new dimension\n"); > + ret = AVERROR_INVALIDDATA; > + goto ERR; > + } > + } > + > + frame->coded_picture_number++; > + frame->display_picture_number++; As is, these two will always be 1. If you want to set these fields, you need to keep track of total decoded frames elsewhere. stat.fnum looks like what you need, or maybe xectx->decod_frames. > + frame->format = AV_PIX_FMT_YUV420P10LE; No need to set this here. ff_get_buffer() will handle it for you. > + > + if (ff_get_buffer(avctx, frame, 0) < 0) { > + av_log(avctx, AV_LOG_ERROR, "Cannot get AV buffer\n"); ff_get_buffer() already prints an error, so remove this one. > + ret = AVERROR(ENOMEM); It also returns an error code you should propagate. > + goto ERR; > + } > + > + frame->pts = avpkt->pts; > + > + av_image_copy(frame->data, frame->linesize, (const uint8_t **)imgb->a, > + imgb->s, avctx->pix_fmt, > + imgb->w[0], imgb->h[0]); > + > + xectx->decod_frames++; Write only variable. > + *got_frame_ptr = 1; > + > + // xevd_pull uses pool of objects of type XEVD_IMGB. > + // The pool size is equal MAX_PB_SIZE (26), so release object when it is no more needed > + imgb->release(imgb); > + imgb = NULL; > + } else > + *got_frame_ptr = 0; > + > + xectx->packet_count++; Same. > + > + return avpkt->size; > + > +ERR: > + if (imgb) { > + imgb->release(imgb); > + imgb = NULL; > + } > + *got_frame_ptr = 0; > + > + return ret; > +} > + > +/** > + * Destroy decoder > + * > + * @param avctx codec context > + * @return 0 on success > + */ > +static av_cold int libxevd_close(AVCodecContext *avctx) > +{ > + XevdContext *xectx = avctx->priv_data; > + if (xectx->id) { > + xevd_delete(xectx->id); > + xectx->id = NULL; > + } > + > + return 0; > +} > + > +#define OFFSET(x) offsetof(XevdContext, x) > +#define VD AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_DECODING_PARAM > + > +static const AVClass libxevd_class = { > + .class_name = "libxevd", > + .item_name = av_default_item_name, > + .version = LIBAVUTIL_VERSION_INT, > +}; > + > +const FFCodec ff_libxevd_decoder = { > + .p.name = "evc", > + .p.long_name = NULL_IF_CONFIG_SMALL("EVC / MPEG-5 Essential Video Coding (EVC)"), > + .p.type = AVMEDIA_TYPE_VIDEO, > + .p.id = AV_CODEC_ID_EVC, > + .init = libxevd_init, > + FF_CODEC_DECODE_CB(libxevd_decode), > + .close = libxevd_close, > + .priv_data_size = sizeof(XevdContext), > + .p.priv_class = &libxevd_class, > + .p.capabilities = AV_CODEC_CAP_DELAY | AV_CODEC_CAP_AUTO_THREADS | AV_CODEC_CAP_AVOID_PROBING | AV_CODEC_CAP_DR1, Use AV_CODEC_CAP_OTHER_THREADS, not AV_CODEC_CAP_AUTO_THREADS. And set caps_internal to FF_CODEC_CAP_AUTO_THREADS. > + .p.wrapper_name = "libxevd", > +}; _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". ^ permalink raw reply [flat|nested] 13+ messages in thread
end of thread, other threads:[~2022-08-18 9:00 UTC | newest]
Thread overview: 13+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
[not found] <CGME20220808123033eucas1p2c85ea9811cb74a92762a4c9a28564d7d@eucas1p2.samsung.com>
2022-08-08 12:30 ` [FFmpeg-devel] [PATCH 1/2] Provided support for MPEG-5 EVC (Essential Video Coding) codec Dawid Kozinski
2022-08-11 7:44 ` Andreas Rheinhardt
2022-08-11 11:28 ` Dawid Kozinski/Robot SDK (PLT) /SRPOL/Staff Engineer/삼성전자
2022-08-18 9:00 ` Dawid Kozinski/Robot SDK (PLT) /SRPOL/Staff Engineer/삼성전자
[not found] <CGME20220812101849eucas1p225445ac91388200262dbf697436f67f2@eucas1p2.samsung.com>
2022-08-12 10:18 ` Dawid Kozinski
[not found] <CGME20220811123643eucas1p116b41dea8d7d8e0fd1ebc79414364f8c@eucas1p1.samsung.com>
2022-08-11 12:36 ` Dawid Kozinski
[not found] <CGME20220801092858eucas1p2eb2d2732543be955865cc083ebf582b3@eucas1p2.samsung.com>
2022-08-01 9:28 ` Dawid Kozinski
2022-08-01 12:29 ` James Almer
2022-08-08 12:39 ` Dawid Kozinski/Robot SDK (PLT) /SRPOL/Staff Engineer/삼성전자
2022-08-01 15:09 ` James Almer
2022-08-08 12:44 ` Dawid Kozinski/Robot SDK (PLT) /SRPOL/Staff Engineer/삼성전자
2022-08-11 6:49 ` Dawid Kozinski/Robot SDK (PLT) /SRPOL/Staff Engineer/삼성전자
2022-08-18 8:59 ` Dawid Kozinski/Robot SDK (PLT) /SRPOL/Staff Engineer/삼성전자
Git Inbox Mirror of the ffmpeg-devel mailing list - see https://ffmpeg.org/mailman/listinfo/ffmpeg-devel This inbox may be cloned and mirrored by anyone: git clone --mirror https://master.gitmailbox.com/ffmpegdev/0 ffmpegdev/git/0.git # If you have public-inbox 1.1+ installed, you may # initialize and index your mirror using the following commands: public-inbox-init -V2 ffmpegdev ffmpegdev/ https://master.gitmailbox.com/ffmpegdev \ ffmpegdev@gitmailbox.com public-inbox-index ffmpegdev Example config snippet for mirrors. AGPL code for this site: git clone https://public-inbox.org/public-inbox.git