* Re: [FFmpeg-devel] [PATCH v26 5/9] avcodec/evc_decoder: Provided support for EVC decoder
2023-07-26 15:45 ` James Almer
@ 2023-07-27 13:43 ` Dawid Kozinski/Multimedia (PLT) /SRPOL/Staff Engineer/Samsung Electronics
2023-07-31 13:24 ` Dawid Kozinski/Multimedia (PLT) /SRPOL/Staff Engineer/Samsung Electronics
2023-08-18 5:44 ` Dawid Kozinski/Multimedia (PLT) /SRPOL/Staff Engineer/Samsung Electronics
2 siblings, 0 replies; 5+ messages in thread
From: Dawid Kozinski/Multimedia (PLT) /SRPOL/Staff Engineer/Samsung Electronics @ 2023-07-27 13:43 UTC (permalink / raw)
To: 'FFmpeg development discussions and patches'
> -----Original Message-----
> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of James
> Almer
> Sent: środa, 26 lipca 2023 17:46
> To: ffmpeg-devel@ffmpeg.org
> Subject: Re: [FFmpeg-devel] [PATCH v26 5/9] avcodec/evc_decoder: Provided
> support for EVC decoder
>
> On 6/15/2023 8:48 AM, Dawid Kozinski wrote:
> > - Added EVC decoder wrapper
> > - Changes in project configuration file and libavcodec Makefile
> > - Added documentation for xevd wrapper
> >
> > Signed-off-by: Dawid Kozinski <d.kozinski@samsung.com>
>
> I'm getting
>
> [evc @ 000001ee1ba7e960] An invalid frame was output by a decoder. This is
a
> bug, please report it.
> [vist#0:0/evc @ 000001ee1d47f220] Decoding error: Internal bug, should not
> have happened
>
> With akiyo_cif.evc, so there's something wrong.
We have just started investigating the cause of this issue. The error
message always appears when the decoder receives the last data packet to be
decoded, just before transitioning to the draining state (when no new data
is coming but there are still some data that reside inside decoder and must
be processed).
After receiving the last data packet, during the data decoding process, the
frame_validate function (decode.c) is called, and it receives a pointer to
an AVFrame object. Inside this function, there is a check for the condition
frame->buf[0] != NULL, which is not met.
>
> > ---
> > configure | 4 +
> > doc/decoders.texi | 24 ++
> > doc/general_contents.texi | 10 +-
> > libavcodec/Makefile | 1 +
> > libavcodec/allcodecs.c | 1 +
> > libavcodec/libxevd.c | 479 ++++++++++++++++++++++++++++++++++++++
> > 6 files changed, 518 insertions(+), 1 deletion(-)
> > create mode 100644 libavcodec/libxevd.c
> >
> > diff --git a/configure b/configure
> > index e2370a23bb..cae3b666a5 100755
> > --- a/configure
> > +++ b/configure
> > @@ -293,6 +293,7 @@ External library support:
> > --enable-libx264 enable H.264 encoding via x264 [no]
> > --enable-libx265 enable HEVC encoding via x265 [no]
> > --enable-libxeve enable EVC encoding via libxeve [no]
> > + --enable-libxevd enable EVC decoding via libxevd [no]
> > --enable-libxavs enable AVS encoding via xavs [no]
> > --enable-libxavs2 enable AVS2 encoding via xavs2 [no]
> > --enable-libxcb enable X11 grabbing using XCB [autodetect]
> > @@ -1904,6 +1905,7 @@ EXTERNAL_LIBRARY_LIST="
> > libvorbis
> > libvpx
> > libwebp
> > + libxevd
> > libxeve
> > libxml2
> > libzimg
> > @@ -3460,6 +3462,7 @@ libx265_encoder_deps="libx265"
> > libx265_encoder_select="atsc_a53"
> > libxavs_encoder_deps="libxavs"
> > libxavs2_encoder_deps="libxavs2"
> > +libxevd_decoder_deps="libxevd"
> > libxeve_encoder_deps="libxeve"
> > libxvid_encoder_deps="libxvid"
> > libzvbi_teletext_decoder_deps="libzvbi"
> > @@ -6835,6 +6838,7 @@ enabled libx265 && require_pkg_config
> libx265 x265 x265.h x265_api_get
> > require_cpp_condition libx265 x265.h
"X265_BUILD >= 89"
> > enabled libxavs && require libxavs "stdint.h xavs.h"
> xavs_encoder_encode "-lxavs $pthreads_extralibs $libm_extralibs"
> > enabled libxavs2 && require_pkg_config libxavs2 "xavs2 >=
1.3.0"
> "stdint.h xavs2.h" xavs2_api_get
> > +enabled libxevd && require_pkg_config libxevd "xevd >= 0.4.1"
"xevd.h"
> xevd_decode
> > enabled libxeve && require_pkg_config libxeve "xeve >=
0.4.3" "xeve.h"
> xeve_encode
> > enabled libxvid && require libxvid xvid.h xvid_global
-lxvidcore
> > enabled libzimg && require_pkg_config libzimg "zimg >=
2.7.0" zimg.h
> zimg_get_api_version
> > diff --git a/doc/decoders.texi b/doc/decoders.texi index
> > 09b8314dd2..6311af229f 100644
> > --- a/doc/decoders.texi
> > +++ b/doc/decoders.texi
> > @@ -130,6 +130,30 @@ Set amount of frame threads to use during
> > decoding. The default value is 0 (auto
> >
> > @end table
> >
> > +@section libxevd
> > +
> > +eXtra-fast Essential Video Decoder (XEVD) MPEG-5 EVC decoder wrapper.
> > +
> > +This decoder requires the presence of the libxevd headers and library
> > +during configuration. You need to explicitly configure the build with
> > +@option{--enable-libxevd}.
> > +
> > +The xevd project website is at
> @url{https://protect2.fireeye.com/v1/url?k=b6ce3a07-d7452f31-b6cfb148-
> 74fe485cbff1-251589a888281453&q=1&e=49c754eb-4416-4e31-90e9-
> ee8e7d469d1f&u=https%3A%2F%2Fgithub.com%2Fmpeg5%2Fxevd%257D.
> > +
> > +@subsection Options
> > +
> > +The following options are supported by the libxevd wrapper.
> > +The xevd-equivalent options or values are listed in parentheses for
easy
> migration.
> > +
> > +To get a more accurate and extensive documentation of the libxevd
> > +options, invoke the command @code{xevd_app --help} or consult the
libxevd
> documentation.
> > +
> > +@table @option
> > +@item threads (@emph{threads})
> > +Force to use a specific number of threads
> > +
> > +@end table
> > +
> > @section QSV Decoders
> >
> > The family of Intel QuickSync Video decoders (VC1, MPEG-2, H.264,
> > HEVC, diff --git a/doc/general_contents.texi
> > b/doc/general_contents.texi index c6a997bfd6..8e08f5ebc3 100644
> > --- a/doc/general_contents.texi
> > +++ b/doc/general_contents.texi
> > @@ -351,6 +351,14 @@ Go to
> @url{https://protect2.fireeye.com/v1/url?k=76721d6d-17f9085b-76739622-
> 74fe485cbff1-6b070a322743d6be&q=1&e=49c754eb-4416-4e31-90e9-
> ee8e7d469d1f&u=https%3A%2F%2Fgithub.com%2Fmpeg5%2Fxeve%257D and
> follow the instructions for
> > installing the XEVE library. Then pass @code{--enable-libxeve} to
configure to
> > enable it.
> >
> > +@section eXtra-fast Essential Video Decoder (XEVD)
> > +
> > +FFmpeg can make use of the XEVD library for EVC video decoding.
> > +
> > +Go to
> > +@url{https://protect2.fireeye.com/v1/url?k=3be9a12a-5a62b41c-3be82a65
> > +-74fe485cbff1-6cd6af142a668aff&q=1&e=49c754eb-4416-4e31-90e9-
> ee8e7d469d1f&u=https%3A%2F%2Fgithub.com%2Fmpeg5%2Fxevd%257D and
> follow the instructions for installing the XEVD library. Then pass
@code{--
> enable-libxevd} to configure to enable it.
> > +
> > @section ZVBI
> >
> > ZVBI is a VBI decoding library which can be used by FFmpeg to decode
> > DVB @@ -953,7 +961,7 @@ following image formats are supported:
> > @item Escape 124 @tab @tab X
> > @item Escape 130 @tab @tab X
> > @item EVC / MPEG-5 Part 1 @tab X @tab X
> > - @tab encoding and decoding supported through external library
libxeve
> > + @tab encoding and decoding supported through external libraries
> > + libxeve and libxevd
> > @item FFmpeg video codec #1 @tab X @tab X
> > @tab lossless codec (fourcc: FFV1)
> > @item Flash Screen Video v1 @tab X @tab X diff --git
> > a/libavcodec/Makefile b/libavcodec/Makefile index
> > 2223e9f46c..d0b8438717 100644
> > --- a/libavcodec/Makefile
> > +++ b/libavcodec/Makefile
> > @@ -1142,6 +1142,7 @@ OBJS-$(CONFIG_LIBX264_ENCODER) +=
> libx264.o
> > OBJS-$(CONFIG_LIBX265_ENCODER) += libx265.o
> > OBJS-$(CONFIG_LIBXAVS_ENCODER) += libxavs.o
> > OBJS-$(CONFIG_LIBXAVS2_ENCODER) += libxavs2.o
> > +OBJS-$(CONFIG_LIBXEVD_DECODER) += libxevd.o
> > OBJS-$(CONFIG_LIBXEVE_ENCODER) += libxeve.o
> > OBJS-$(CONFIG_LIBXVID_ENCODER) += libxvid.o
> > OBJS-$(CONFIG_LIBZVBI_TELETEXT_DECODER) += libzvbi-teletextdec.o
> ass.o
> > diff --git a/libavcodec/allcodecs.c b/libavcodec/allcodecs.c index
> > c317dd6da3..eb0abbb1fd 100644
> > --- a/libavcodec/allcodecs.c
> > +++ b/libavcodec/allcodecs.c
> > @@ -819,6 +819,7 @@ extern LIBX264_CONST FFCodec ff_libx264_encoder;
> > extern const FFCodec ff_libx264rgb_encoder;
> > extern FFCodec ff_libx265_encoder;
> > extern const FFCodec ff_libxeve_encoder;
> > +extern const FFCodec ff_libxevd_decoder;
> > extern const FFCodec ff_libxavs_encoder;
> > extern const FFCodec ff_libxavs2_encoder;
> > extern const FFCodec ff_libxvid_encoder; diff --git
> > a/libavcodec/libxevd.c b/libavcodec/libxevd.c new file mode 100644
> > index 0000000000..7c4922d46e
> > --- /dev/null
> > +++ b/libavcodec/libxevd.c
> > @@ -0,0 +1,479 @@
> > +/*
> > + * libxevd decoder
> > + * EVC (MPEG-5 Essential Video Coding) decoding using XEVD MPEG-5 EVC
> > +decoder library
> > + *
> > + * Copyright (C) 2021 Dawid Kozinski <d.kozinski@samsung.com>
> > + *
> > + * This file is part of FFmpeg.
> > + *
> > + * FFmpeg is free software; you can redistribute it and/or
> > + * modify it under the terms of the GNU Lesser General Public
> > + * License as published by the Free Software Foundation; either
> > + * version 2.1 of the License, or (at your option) any later version.
> > + *
> > + * FFmpeg is distributed in the hope that it will be useful,
> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
> > + * Lesser General Public License for more details.
> > + *
> > + * You should have received a copy of the GNU Lesser General Public
> > + * License along with FFmpeg; if not, write to the Free Software
> > + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA
> > +02110-1301 USA */
> > +
> > +#include <float.h>
> > +#include <stdlib.h>
> > +
> > +#include <xevd.h>
> > +
> > +#include "libavutil/internal.h"
> > +#include "libavutil/common.h"
> > +#include "libavutil/opt.h"
> > +#include "libavutil/pixdesc.h"
> > +#include "libavutil/pixfmt.h"
> > +#include "libavutil/imgutils.h"
> > +#include "libavutil/cpu.h"
> > +
> > +#include "avcodec.h"
> > +#include "internal.h"
> > +#include "packet_internal.h"
> > +#include "codec_internal.h"
> > +#include "profiles.h"
> > +#include "decode.h"
> > +
> > +#define XEVD_PARAM_BAD_NAME -1
> > +#define XEVD_PARAM_BAD_VALUE -2
> > +
> > +#define EVC_NAL_HEADER_SIZE 2 /* byte */
> > +
> > +/**
> > + * The structure stores all the states associated with the instance
> > +of Xeve MPEG-5 EVC decoder */ typedef struct XevdContext {
> > + const AVClass *class;
> > +
> > + XEVD id; // XEVD instance identifier @see xevd.h
> > + XEVD_CDSC cdsc; // decoding parameters @see xevd.h
> > +
> > + // If end of stream occurs it is required "flushing" (aka draining)
the codec,
> > + // as the codec might buffer multiple frames or packets internally.
> > + int draining_mode; // The flag is set if codec enters draining
mode.
> > +
> > + AVPacket *pkt;
> > +} XevdContext;
> > +
> > +/**
> > + * The function populates the XEVD_CDSC structure.
> > + * XEVD_CDSC contains all decoder parameters that should be initialized
> before its use.
> > + *
> > + * @param[in] avctx codec context
> > + * @param[out] cdsc contains all decoder parameters that should be
> > +initialized before its use
> > + *
> > + */
> > +static void get_conf(AVCodecContext *avctx, XEVD_CDSC *cdsc) {
> > + int cpu_count = av_cpu_count();
> > +
> > + /* clear XEVS_CDSC structure */
> > + memset(cdsc, 0, sizeof(XEVD_CDSC));
> > +
> > + /* init XEVD_CDSC */
> > + if (avctx->thread_count <= 0)
> > + cdsc->threads = (cpu_count < XEVD_MAX_TASK_CNT) ? cpu_count :
> XEVD_MAX_TASK_CNT;
> > + else if (avctx->thread_count > XEVD_MAX_TASK_CNT)
> > + cdsc->threads = XEVD_MAX_TASK_CNT;
> > + else
> > + cdsc->threads = avctx->thread_count; }
> > +
> > +/**
> > + * Read NAL unit length
> > + * @param bs input data (bitstream)
> > + * @return the length of NAL unit on success, 0 value on failure */
> > +static uint32_t read_nal_unit_length(const uint8_t *bs, int bs_size,
> > +AVCodecContext *avctx) {
> > + uint32_t len = 0;
> > + XEVD_INFO info;
> > + int ret;
> > +
> > + if (bs_size == XEVD_NAL_UNIT_LENGTH_BYTE) {
> > + ret = xevd_info((void *)bs, XEVD_NAL_UNIT_LENGTH_BYTE, 1,
&info);
> > + if (XEVD_FAILED(ret)) {
> > + av_log(avctx, AV_LOG_ERROR, "Cannot get bitstream
information\n");
> > + return 0;
> > + }
> > + len = info.nalu_len;
> > + if (len == 0) {
> > + av_log(avctx, AV_LOG_ERROR, "Invalid bitstream size!
[%d]\n",
> bs_size);
> > + return 0;
> > + }
> > + }
> > +
> > + return len;
> > +}
> > +
> > +/**
> > + * @param[in] xectx the structure that stores all the state
> > +associated with the instance of Xeve MPEG-5 EVC decoder
> > + * @param[out] avctx codec context
> > + * @return 0 on success, negative value on failure */ static int
> > +export_stream_params(const XevdContext *xectx, AVCodecContext *avctx)
> > +{
> > + int ret;
> > + int size;
> > + int color_space;
> > +
> > + avctx->pix_fmt = AV_PIX_FMT_YUV420P10;
> > +
> > + size = 4;
> > + ret = xevd_config(xectx->id, XEVD_CFG_GET_CODED_WIDTH, &avctx-
> >coded_width, &size);
> > + if (XEVD_FAILED(ret)) {
> > + av_log(avctx, AV_LOG_ERROR, "Failed to get coded_width\n");
> > + return AVERROR_EXTERNAL;
> > + }
> > +
> > + ret = xevd_config(xectx->id, XEVD_CFG_GET_CODED_HEIGHT, &avctx-
> >coded_height, &size);
> > + if (XEVD_FAILED(ret)) {
> > + av_log(avctx, AV_LOG_ERROR, "Failed to get coded_height\n");
> > + return AVERROR_EXTERNAL;
> > + }
> > +
> > + ret = xevd_config(xectx->id, XEVD_CFG_GET_WIDTH, &avctx->width,
> &size);
> > + if (XEVD_FAILED(ret)) {
> > + av_log(avctx, AV_LOG_ERROR, "Failed to get width\n");
> > + return AVERROR_EXTERNAL;
> > + }
> > +
> > + ret = xevd_config(xectx->id, XEVD_CFG_GET_HEIGHT, &avctx->height,
> &size);
> > + if (XEVD_FAILED(ret)) {
> > + av_log(avctx, AV_LOG_ERROR, "Failed to get height\n");
> > + return AVERROR_EXTERNAL;
> > + }
> > +
> > + ret = xevd_config(xectx->id, XEVD_CFG_GET_COLOR_SPACE,
> &color_space, &size);
> > + if (XEVD_FAILED(ret)) {
> > + av_log(avctx, AV_LOG_ERROR, "Failed to get color_space\n");
> > + return AVERROR_EXTERNAL;
> > + }
> > + switch(color_space) {
> > + case XEVD_CS_YCBCR400_10LE:
> > + avctx->pix_fmt = AV_PIX_FMT_GRAY10LE;
> > + break;
> > + case XEVD_CS_YCBCR420_10LE:
> > + avctx->pix_fmt = AV_PIX_FMT_YUV420P10LE;
> > + break;
> > + case XEVD_CS_YCBCR422_10LE:
> > + avctx->pix_fmt = AV_PIX_FMT_YUV422P10LE;
> > + break;
> > + case XEVD_CS_YCBCR444_10LE:
> > + avctx->pix_fmt = AV_PIX_FMT_YUV444P10LE;
> > + break;
> > + default:
> > + av_log(avctx, AV_LOG_ERROR, "Unknown color space\n");
> > + avctx->pix_fmt = AV_PIX_FMT_NONE;
> > + return AVERROR_INVALIDDATA;
> > + }
> > +
> > + // the function returns sps->num_reorder_pics
> > + ret = xevd_config(xectx->id, XEVD_CFG_GET_MAX_CODING_DELAY,
> &avctx->max_b_frames, &size);
> > + if (XEVD_FAILED(ret)) {
> > + av_log(avctx, AV_LOG_ERROR, "Failed to get
max_coding_delay\n");
> > + return AVERROR_EXTERNAL;
> > + }
> > +
> > + avctx->has_b_frames = (avctx->max_b_frames) ? 1 : 0;
> > +
> > + return 0;
> > +}
> > +
> > +/**
> > + * @brief Copy image in imgb to frame.
> > + *
> > + * @param avctx codec context
> > + * @param[in] imgb
> > + * @param[out] frame
> > + * @return 0 on success, negative value on failure */ static int
> > +libxevd_image_copy(struct AVCodecContext *avctx, XEVD_IMGB *imgb,
> > +struct AVFrame *frame) {
> > + int ret;
> > + if (imgb->cs != XEVD_CS_YCBCR420_10LE) {
> > + av_log(avctx, AV_LOG_ERROR, "Not supported pixel format: %s\n",
> av_get_pix_fmt_name(avctx->pix_fmt));
> > + return AVERROR_INVALIDDATA;
> > + }
> > +
> > + if (imgb->w[0] != avctx->width || imgb->h[0] != avctx->height) { //
stream
> resolution changed
> > + if (ff_set_dimensions(avctx, imgb->w[0], imgb->h[0]) < 0) {
> > + av_log(avctx, AV_LOG_ERROR, "Cannot set new dimension\n");
> > + return AVERROR_INVALIDDATA;
> > + }
> > + }
> > +
> > + if (ret = ff_get_buffer(avctx, frame, 0) < 0)
> > + return ret;
> > +
> > + av_image_copy(frame->data, frame->linesize, (const uint8_t
**)imgb->a,
> > + imgb->s, avctx->pix_fmt,
> > + imgb->w[0], imgb->h[0]);
> > +
> > + return 0;
> > +}
> > +
> > +/**
> > + * Initialize decoder
> > + * Create a decoder instance and allocate all the needed resources
> > + *
> > + * @param avctx codec context
> > + * @return 0 on success, negative error code on failure */ static
> > +av_cold int libxevd_init(AVCodecContext *avctx) {
> > + XevdContext *xectx = avctx->priv_data;
> > + XEVD_CDSC *cdsc = &(xectx->cdsc);
> > +
> > + /* read configurations and set values for created descriptor
(XEVD_CDSC)
> */
> > + get_conf(avctx, cdsc);
> > +
> > + /* create decoder */
> > + xectx->id = xevd_create(&(xectx->cdsc), NULL);
> > + if (xectx->id == NULL) {
> > + av_log(avctx, AV_LOG_ERROR, "Cannot create XEVD encoder\n");
> > + return AVERROR_EXTERNAL;
> > + }
> > +
> > + xectx->draining_mode = 0;
> > + xectx->pkt = av_packet_alloc();
> > +
> > + return 0;
> > +}
> > +
> > +/**
> > + * Decode frame with decoupled packet/frame dataflow
> > + *
> > + * @param avctx codec context
> > + * @param[out] frame decoded frame
> > + *
> > + * @return 0 on success, negative error code on failure
> > + */
> > +static int libxevd_receive_frame(AVCodecContext *avctx, AVFrame
> > +*frame) {
> > + XevdContext *xectx = avctx->priv_data;
> > + AVPacket *pkt = xectx->pkt;
> > + XEVD_IMGB *imgb = NULL;
> > +
> > + int xevd_ret = 0;
> > + int ret = 0;
> > +
> > + if (!pkt)
> > + return AVERROR(ENOMEM);
> > +
> > + // obtain input data
> > + ret = ff_decode_get_packet(avctx, pkt);
> > + if (ret < 0 && ret != AVERROR_EOF) {
> > + av_packet_unref(pkt);
> > +
> > + return ret;
> > + } else if(ret == AVERROR_EOF && xectx->draining_mode == 0) { //
> > + End of stream situations. Enter draining mode
> > +
> > + xectx->draining_mode = 1;
> > + av_packet_unref(pkt);
> > +
> > + return 0;
> > + }
> > +
> > + if (pkt->size > 0) {
> > + int bs_read_pos = 0;
> > + XEVD_STAT stat;
> > + XEVD_BITB bitb;
> > + int nalu_size;
> > +
> > + imgb = NULL;
> > +
> > + while(pkt->size > (bs_read_pos + XEVD_NAL_UNIT_LENGTH_BYTE)) {
> > + memset(&stat, 0, sizeof(XEVD_STAT));
> > +
> > + nalu_size = read_nal_unit_length(pkt->data + bs_read_pos,
> XEVD_NAL_UNIT_LENGTH_BYTE, avctx);
> > + if (nalu_size == 0) {
> > + av_log(avctx, AV_LOG_ERROR, "Invalid bitstream\n");
> > + av_packet_unref(pkt);
> > + ret = AVERROR_INVALIDDATA;
> > +
> > + return ret;
> > + }
> > + bs_read_pos += XEVD_NAL_UNIT_LENGTH_BYTE;
> > +
> > + bitb.addr = pkt->data + bs_read_pos;
> > + bitb.ssize = nalu_size;
> > +
> > + /* main decoding block */
> > + xevd_ret = xevd_decode(xectx->id, &bitb, &stat);
> > + if (XEVD_FAILED(xevd_ret)) {
> > + av_log(avctx, AV_LOG_ERROR, "Failed to decode
bitstream\n");
> > + av_packet_unref(pkt);
> > + ret = AVERROR_EXTERNAL;
> > +
> > + return ret;
> > + }
> > +
> > + bs_read_pos += nalu_size;
> > +
> > + if (stat.nalu_type == XEVD_NUT_SPS) { // EVC stream
parameters
> changed
> > + if ((ret = export_stream_params(xectx, avctx)) != 0) {
> > + av_log(avctx, AV_LOG_ERROR, "Failed to export
stream
> params\n");
> > + av_packet_unref(pkt);
> > +
> > + return ret;
> > + }
> > + }
> > + if (stat.read != nalu_size)
> > + av_log(avctx, AV_LOG_INFO, "Different reading of
bitstream (in:%d,
> read:%d)\n,", nalu_size, stat.read);
> > + if (stat.fnum >= 0) {
> > +
> > + xevd_ret = xevd_pull(xectx->id, &imgb); // The
> > + function returns a valid image only if the return code is XEVD_OK
> > +
> > + if (XEVD_FAILED(xevd_ret)) {
> > + av_log(avctx, AV_LOG_ERROR, "Failed to pull the
decoded image
> (xevd error code: %d, frame#=%d)\n", xevd_ret, stat.fnum);
> > + ret = AVERROR_EXTERNAL;
> > + av_packet_unref(pkt);
> > +
> > + return ret;
> > + } else if (xevd_ret == XEVD_OK_FRM_DELAYED) {
> > + av_packet_unref(pkt);
>
> Was the packet fully consumed here? As in, is bs_read_pos == pkt->size at
this
> point? If not, you're dropping NALUs that should be passed to libxevd.
>
> > +
> > + return AVERROR(EAGAIN);
> > + } else { // XEVD_OK
> > + if (!imgb) {
> > + av_packet_unref(pkt);
>
> Same question here. And I assume this would happen with
> XEVD_OK_OUT_NOT_AVAILABLE?
>
> > +
> > + return AVERROR(EAGAIN);
> > + }
> > +
> > + // got frame
> > + ret = libxevd_image_copy(avctx, imgb, frame);
> > + if(ret < 0) {
> > + av_log(avctx, AV_LOG_ERROR, "Image copying
> > + error\n");
> > +
> > + imgb->release(imgb);
> > + imgb = NULL;
> > +
> > + av_packet_unref(pkt);
> > +
> > + return ret;
> > + }
> > +
> > + // use ff_decode_frame_props() to fill frame
properties
> > + ret = ff_decode_frame_props(avctx, frame);
>
> ff_get_buffer() calls this, so it's not needed.
> You however set the decoder as FF_CODEC_CAP_SETS_FRAME_PROPS, so
> ff_decode_frame_props_pkt() is not being called to fetch props from the
last
> packet.
>
> You probably need a FIFO of packet props and use that to fill frame props
as you
> return them. It's a pity you can't propagate an opaque pointer to
xevd_decode()
> and get it back in xevd_pull(). That would simplify things a lot.
>
> > + if (ret < 0) {
> > + imgb->release(imgb);
> > + imgb = NULL;
> > +
> > + av_packet_unref(pkt);
> > + av_frame_unref(frame);
> > +
> > + return ret;
> > + }
> > +
> > + frame->pkt_dts = pkt->dts;
> > +
> > + // xevd_pull uses pool of objects of type
XEVD_IMGB.
> > + // The pool size is equal MAX_PB_SIZE (26), so
release object when
> it is no more needed
> > + imgb->release(imgb);
> > + imgb = NULL;
> > +
> > + av_packet_unref(pkt);
>
> Again, were all the NALUs in the packet passed to libxevd?
>
> > + return 0;
> > + }
> > + }
> > + }
> > + } else { // decoder draining mode handling
> > +
> > + xevd_ret = xevd_pull(xectx->id, &imgb);
> > +
> > + if (xevd_ret == XEVD_ERR_UNEXPECTED) { // draining process
completed
> > + av_log(avctx, AV_LOG_DEBUG, "Draining process
completed\n");
> > + av_packet_unref(pkt);
> > +
> > + return AVERROR_EOF;
> > + } else if (XEVD_FAILED(xevd_ret)) { // handle all other errors
> > + av_log(avctx, AV_LOG_ERROR, "Failed to pull the decoded
image (xevd
> error code: %d)\n", xevd_ret);
> > + av_packet_unref(pkt);
> > +
> > + return AVERROR_EXTERNAL;
> > + } else { // XEVD_OK
> > + if (!imgb) {
> > + av_packet_unref(pkt);
> > +
> > + return AVERROR_EXTERNAL;
> > + }
> > + // got frame
> > + ret = libxevd_image_copy(avctx, imgb, frame);
> > + if(ret < 0) {
> > + imgb->release(imgb);
> > + imgb = NULL;
> > +
> > + av_packet_unref(pkt);
> > +
> > + return ret;
> > + }
> > +
> > + frame->pkt_dts = pkt->dts;
> > +
> > + av_packet_unref(pkt);
> > +
> > + // xevd_pull uses pool of objects of type XEVD_IMGB.
> > + // The pool size is equal MAX_PB_SIZE (26), so release
object when it is
> no more needed
> > + imgb->release(imgb);
> > + imgb = NULL;
> > +
> > + return 0;
> > + }
> > + }
> > +
> > + return ret;
> > +}
> > +
> > +/**
> > + * Destroy decoder
> > + *
> > + * @param avctx codec context
> > + * @return 0 on success
> > + */
> > +static av_cold int libxevd_close(AVCodecContext *avctx) {
> > + XevdContext *xectx = avctx->priv_data;
> > + if (xectx->id) {
> > + xevd_delete(xectx->id);
> > + xectx->id = NULL;
> > + }
> > +
> > + xectx->draining_mode = 0;
> > + av_packet_free(&xectx->pkt);
> > +
> > + return 0;
> > +}
> > +
> > +#define OFFSET(x) offsetof(XevdContext, x) #define VD
> > +AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_DECODING_PARAM
> > +
> > +static const AVClass libxevd_class = {
> > + .class_name = "libxevd",
> > + .item_name = av_default_item_name,
> > + .version = LIBAVUTIL_VERSION_INT,
> > +};
> > +
> > +const FFCodec ff_libxevd_decoder = {
> > + .p.name = "evc",
> > + .p.long_name = NULL_IF_CONFIG_SMALL("EVC / MPEG-5 Essential
> Video Coding (EVC)"),
> > + .p.type = AVMEDIA_TYPE_VIDEO,
> > + .p.id = AV_CODEC_ID_EVC,
> > + .init = libxevd_init,
> > + FF_CODEC_RECEIVE_FRAME_CB(libxevd_receive_frame),
> > + .close = libxevd_close,
> > + .priv_data_size = sizeof(XevdContext),
> > + .p.priv_class = &libxevd_class,
> > + .p.capabilities = AV_CODEC_CAP_DELAY |
> AV_CODEC_CAP_OTHER_THREADS | AV_CODEC_CAP_AVOID_PROBING,
> > + .p.profiles = NULL_IF_CONFIG_SMALL(ff_evc_profiles),
> > + .p.wrapper_name = "libxevd",
> > + .caps_internal = FF_CODEC_CAP_INIT_CLEANUP |
> FF_CODEC_CAP_NOT_INIT_THREADSAFE | FF_CODEC_CAP_SETS_PKT_DTS |
> FF_CODEC_CAP_SETS_FRAME_PROPS
> > +};
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://protect2.fireeye.com/v1/url?k=6cd54358-0d5e566e-6cd4c817-
> 74fe485cbff1-e18dc94cc35fe766&q=1&e=49c754eb-4416-4e31-90e9-
> ee8e7d469d1f&u=https%3A%2F%2Fffmpeg.org%2Fmailman%2Flistinfo%2Fffmp
> eg-devel
>
> To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org
> with subject "unsubscribe".
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
^ permalink raw reply [flat|nested] 5+ messages in thread* Re: [FFmpeg-devel] [PATCH v26 5/9] avcodec/evc_decoder: Provided support for EVC decoder
2023-07-26 15:45 ` James Almer
2023-07-27 13:43 ` Dawid Kozinski/Multimedia (PLT) /SRPOL/Staff Engineer/Samsung Electronics
@ 2023-07-31 13:24 ` Dawid Kozinski/Multimedia (PLT) /SRPOL/Staff Engineer/Samsung Electronics
2023-08-18 5:44 ` Dawid Kozinski/Multimedia (PLT) /SRPOL/Staff Engineer/Samsung Electronics
2 siblings, 0 replies; 5+ messages in thread
From: Dawid Kozinski/Multimedia (PLT) /SRPOL/Staff Engineer/Samsung Electronics @ 2023-07-31 13:24 UTC (permalink / raw)
To: 'FFmpeg development discussions and patches'
> -----Original Message-----
> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of James
> Almer
> Sent: środa, 26 lipca 2023 17:46
> To: ffmpeg-devel@ffmpeg.org
> Subject: Re: [FFmpeg-devel] [PATCH v26 5/9] avcodec/evc_decoder: Provided
> support for EVC decoder
>
> On 6/15/2023 8:48 AM, Dawid Kozinski wrote:
> > - Added EVC decoder wrapper
> > - Changes in project configuration file and libavcodec Makefile
> > - Added documentation for xevd wrapper
> >
> > Signed-off-by: Dawid Kozinski <d.kozinski@samsung.com>
>
> I'm getting
>
> [evc @ 000001ee1ba7e960] An invalid frame was output by a decoder. This is
a
> bug, please report it.
> [vist#0:0/evc @ 000001ee1d47f220] Decoding error: Internal bug, should not
> have happened
>
> With akiyo_cif.evc, so there's something wrong.
>
I have fixed it and the fix will be included in the next patchset along with
other changes mentioned below.
> > ---
> > configure | 4 +
> > doc/decoders.texi | 24 ++
> > doc/general_contents.texi | 10 +-
> > libavcodec/Makefile | 1 +
> > libavcodec/allcodecs.c | 1 +
> > libavcodec/libxevd.c | 479 ++++++++++++++++++++++++++++++++++++++
> > 6 files changed, 518 insertions(+), 1 deletion(-)
> > create mode 100644 libavcodec/libxevd.c
> >
> > diff --git a/configure b/configure
> > index e2370a23bb..cae3b666a5 100755
> > --- a/configure
> > +++ b/configure
> > @@ -293,6 +293,7 @@ External library support:
> > --enable-libx264 enable H.264 encoding via x264 [no]
> > --enable-libx265 enable HEVC encoding via x265 [no]
> > --enable-libxeve enable EVC encoding via libxeve [no]
> > + --enable-libxevd enable EVC decoding via libxevd [no]
> > --enable-libxavs enable AVS encoding via xavs [no]
> > --enable-libxavs2 enable AVS2 encoding via xavs2 [no]
> > --enable-libxcb enable X11 grabbing using XCB [autodetect]
> > @@ -1904,6 +1905,7 @@ EXTERNAL_LIBRARY_LIST="
> > libvorbis
> > libvpx
> > libwebp
> > + libxevd
> > libxeve
> > libxml2
> > libzimg
> > @@ -3460,6 +3462,7 @@ libx265_encoder_deps="libx265"
> > libx265_encoder_select="atsc_a53"
> > libxavs_encoder_deps="libxavs"
> > libxavs2_encoder_deps="libxavs2"
> > +libxevd_decoder_deps="libxevd"
> > libxeve_encoder_deps="libxeve"
> > libxvid_encoder_deps="libxvid"
> > libzvbi_teletext_decoder_deps="libzvbi"
> > @@ -6835,6 +6838,7 @@ enabled libx265 && require_pkg_config
> libx265 x265 x265.h x265_api_get
> > require_cpp_condition libx265 x265.h
"X265_BUILD >= 89"
> > enabled libxavs && require libxavs "stdint.h xavs.h"
> xavs_encoder_encode "-lxavs $pthreads_extralibs $libm_extralibs"
> > enabled libxavs2 && require_pkg_config libxavs2 "xavs2 >=
1.3.0"
> "stdint.h xavs2.h" xavs2_api_get
> > +enabled libxevd && require_pkg_config libxevd "xevd >= 0.4.1"
"xevd.h"
> xevd_decode
> > enabled libxeve && require_pkg_config libxeve "xeve >=
0.4.3" "xeve.h"
> xeve_encode
> > enabled libxvid && require libxvid xvid.h xvid_global
-lxvidcore
> > enabled libzimg && require_pkg_config libzimg "zimg >=
2.7.0" zimg.h
> zimg_get_api_version
> > diff --git a/doc/decoders.texi b/doc/decoders.texi index
> > 09b8314dd2..6311af229f 100644
> > --- a/doc/decoders.texi
> > +++ b/doc/decoders.texi
> > @@ -130,6 +130,30 @@ Set amount of frame threads to use during
> > decoding. The default value is 0 (auto
> >
> > @end table
> >
> > +@section libxevd
> > +
> > +eXtra-fast Essential Video Decoder (XEVD) MPEG-5 EVC decoder wrapper.
> > +
> > +This decoder requires the presence of the libxevd headers and library
> > +during configuration. You need to explicitly configure the build with
> > +@option{--enable-libxevd}.
> > +
> > +The xevd project website is at
> @url{https://protect2.fireeye.com/v1/url?k=b6ce3a07-d7452f31-b6cfb148-
> 74fe485cbff1-251589a888281453&q=1&e=49c754eb-4416-4e31-90e9-
> ee8e7d469d1f&u=https%3A%2F%2Fgithub.com%2Fmpeg5%2Fxevd%257D.
> > +
> > +@subsection Options
> > +
> > +The following options are supported by the libxevd wrapper.
> > +The xevd-equivalent options or values are listed in parentheses for
easy
> migration.
> > +
> > +To get a more accurate and extensive documentation of the libxevd
> > +options, invoke the command @code{xevd_app --help} or consult the
libxevd
> documentation.
> > +
> > +@table @option
> > +@item threads (@emph{threads})
> > +Force to use a specific number of threads
> > +
> > +@end table
> > +
> > @section QSV Decoders
> >
> > The family of Intel QuickSync Video decoders (VC1, MPEG-2, H.264,
> > HEVC, diff --git a/doc/general_contents.texi
> > b/doc/general_contents.texi index c6a997bfd6..8e08f5ebc3 100644
> > --- a/doc/general_contents.texi
> > +++ b/doc/general_contents.texi
> > @@ -351,6 +351,14 @@ Go to
> @url{https://protect2.fireeye.com/v1/url?k=76721d6d-17f9085b-76739622-
> 74fe485cbff1-6b070a322743d6be&q=1&e=49c754eb-4416-4e31-90e9-
> ee8e7d469d1f&u=https%3A%2F%2Fgithub.com%2Fmpeg5%2Fxeve%257D and
> follow the instructions for
> > installing the XEVE library. Then pass @code{--enable-libxeve} to
configure to
> > enable it.
> >
> > +@section eXtra-fast Essential Video Decoder (XEVD)
> > +
> > +FFmpeg can make use of the XEVD library for EVC video decoding.
> > +
> > +Go to
> > +@url{https://protect2.fireeye.com/v1/url?k=3be9a12a-5a62b41c-3be82a65
> > +-74fe485cbff1-6cd6af142a668aff&q=1&e=49c754eb-4416-4e31-90e9-
> ee8e7d469d1f&u=https%3A%2F%2Fgithub.com%2Fmpeg5%2Fxevd%257D and
> follow the instructions for installing the XEVD library. Then pass
@code{--
> enable-libxevd} to configure to enable it.
> > +
> > @section ZVBI
> >
> > ZVBI is a VBI decoding library which can be used by FFmpeg to decode
> > DVB @@ -953,7 +961,7 @@ following image formats are supported:
> > @item Escape 124 @tab @tab X
> > @item Escape 130 @tab @tab X
> > @item EVC / MPEG-5 Part 1 @tab X @tab X
> > - @tab encoding and decoding supported through external library
libxeve
> > + @tab encoding and decoding supported through external libraries
> > + libxeve and libxevd
> > @item FFmpeg video codec #1 @tab X @tab X
> > @tab lossless codec (fourcc: FFV1)
> > @item Flash Screen Video v1 @tab X @tab X diff --git
> > a/libavcodec/Makefile b/libavcodec/Makefile index
> > 2223e9f46c..d0b8438717 100644
> > --- a/libavcodec/Makefile
> > +++ b/libavcodec/Makefile
> > @@ -1142,6 +1142,7 @@ OBJS-$(CONFIG_LIBX264_ENCODER) +=
> libx264.o
> > OBJS-$(CONFIG_LIBX265_ENCODER) += libx265.o
> > OBJS-$(CONFIG_LIBXAVS_ENCODER) += libxavs.o
> > OBJS-$(CONFIG_LIBXAVS2_ENCODER) += libxavs2.o
> > +OBJS-$(CONFIG_LIBXEVD_DECODER) += libxevd.o
> > OBJS-$(CONFIG_LIBXEVE_ENCODER) += libxeve.o
> > OBJS-$(CONFIG_LIBXVID_ENCODER) += libxvid.o
> > OBJS-$(CONFIG_LIBZVBI_TELETEXT_DECODER) += libzvbi-teletextdec.o
> ass.o
> > diff --git a/libavcodec/allcodecs.c b/libavcodec/allcodecs.c index
> > c317dd6da3..eb0abbb1fd 100644
> > --- a/libavcodec/allcodecs.c
> > +++ b/libavcodec/allcodecs.c
> > @@ -819,6 +819,7 @@ extern LIBX264_CONST FFCodec ff_libx264_encoder;
> > extern const FFCodec ff_libx264rgb_encoder;
> > extern FFCodec ff_libx265_encoder;
> > extern const FFCodec ff_libxeve_encoder;
> > +extern const FFCodec ff_libxevd_decoder;
> > extern const FFCodec ff_libxavs_encoder;
> > extern const FFCodec ff_libxavs2_encoder;
> > extern const FFCodec ff_libxvid_encoder; diff --git
> > a/libavcodec/libxevd.c b/libavcodec/libxevd.c new file mode 100644
> > index 0000000000..7c4922d46e
> > --- /dev/null
> > +++ b/libavcodec/libxevd.c
> > @@ -0,0 +1,479 @@
> > +/*
> > + * libxevd decoder
> > + * EVC (MPEG-5 Essential Video Coding) decoding using XEVD MPEG-5 EVC
> > +decoder library
> > + *
> > + * Copyright (C) 2021 Dawid Kozinski <d.kozinski@samsung.com>
> > + *
> > + * This file is part of FFmpeg.
> > + *
> > + * FFmpeg is free software; you can redistribute it and/or
> > + * modify it under the terms of the GNU Lesser General Public
> > + * License as published by the Free Software Foundation; either
> > + * version 2.1 of the License, or (at your option) any later version.
> > + *
> > + * FFmpeg is distributed in the hope that it will be useful,
> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
> > + * Lesser General Public License for more details.
> > + *
> > + * You should have received a copy of the GNU Lesser General Public
> > + * License along with FFmpeg; if not, write to the Free Software
> > + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA
> > +02110-1301 USA */
> > +
> > +#include <float.h>
> > +#include <stdlib.h>
> > +
> > +#include <xevd.h>
> > +
> > +#include "libavutil/internal.h"
> > +#include "libavutil/common.h"
> > +#include "libavutil/opt.h"
> > +#include "libavutil/pixdesc.h"
> > +#include "libavutil/pixfmt.h"
> > +#include "libavutil/imgutils.h"
> > +#include "libavutil/cpu.h"
> > +
> > +#include "avcodec.h"
> > +#include "internal.h"
> > +#include "packet_internal.h"
> > +#include "codec_internal.h"
> > +#include "profiles.h"
> > +#include "decode.h"
> > +
> > +#define XEVD_PARAM_BAD_NAME -1
> > +#define XEVD_PARAM_BAD_VALUE -2
> > +
> > +#define EVC_NAL_HEADER_SIZE 2 /* byte */
> > +
> > +/**
> > + * The structure stores all the states associated with the instance
> > +of Xeve MPEG-5 EVC decoder */ typedef struct XevdContext {
> > + const AVClass *class;
> > +
> > + XEVD id; // XEVD instance identifier @see xevd.h
> > + XEVD_CDSC cdsc; // decoding parameters @see xevd.h
> > +
> > + // If end of stream occurs it is required "flushing" (aka draining)
the codec,
> > + // as the codec might buffer multiple frames or packets internally.
> > + int draining_mode; // The flag is set if codec enters draining
mode.
> > +
> > + AVPacket *pkt;
> > +} XevdContext;
> > +
> > +/**
> > + * The function populates the XEVD_CDSC structure.
> > + * XEVD_CDSC contains all decoder parameters that should be initialized
> before its use.
> > + *
> > + * @param[in] avctx codec context
> > + * @param[out] cdsc contains all decoder parameters that should be
> > +initialized before its use
> > + *
> > + */
> > +static void get_conf(AVCodecContext *avctx, XEVD_CDSC *cdsc) {
> > + int cpu_count = av_cpu_count();
> > +
> > + /* clear XEVS_CDSC structure */
> > + memset(cdsc, 0, sizeof(XEVD_CDSC));
> > +
> > + /* init XEVD_CDSC */
> > + if (avctx->thread_count <= 0)
> > + cdsc->threads = (cpu_count < XEVD_MAX_TASK_CNT) ? cpu_count :
> XEVD_MAX_TASK_CNT;
> > + else if (avctx->thread_count > XEVD_MAX_TASK_CNT)
> > + cdsc->threads = XEVD_MAX_TASK_CNT;
> > + else
> > + cdsc->threads = avctx->thread_count; }
> > +
> > +/**
> > + * Read NAL unit length
> > + * @param bs input data (bitstream)
> > + * @return the length of NAL unit on success, 0 value on failure */
> > +static uint32_t read_nal_unit_length(const uint8_t *bs, int bs_size,
> > +AVCodecContext *avctx) {
> > + uint32_t len = 0;
> > + XEVD_INFO info;
> > + int ret;
> > +
> > + if (bs_size == XEVD_NAL_UNIT_LENGTH_BYTE) {
> > + ret = xevd_info((void *)bs, XEVD_NAL_UNIT_LENGTH_BYTE, 1,
&info);
> > + if (XEVD_FAILED(ret)) {
> > + av_log(avctx, AV_LOG_ERROR, "Cannot get bitstream
information\n");
> > + return 0;
> > + }
> > + len = info.nalu_len;
> > + if (len == 0) {
> > + av_log(avctx, AV_LOG_ERROR, "Invalid bitstream size!
[%d]\n",
> bs_size);
> > + return 0;
> > + }
> > + }
> > +
> > + return len;
> > +}
> > +
> > +/**
> > + * @param[in] xectx the structure that stores all the state
> > +associated with the instance of Xeve MPEG-5 EVC decoder
> > + * @param[out] avctx codec context
> > + * @return 0 on success, negative value on failure */ static int
> > +export_stream_params(const XevdContext *xectx, AVCodecContext *avctx)
> > +{
> > + int ret;
> > + int size;
> > + int color_space;
> > +
> > + avctx->pix_fmt = AV_PIX_FMT_YUV420P10;
> > +
> > + size = 4;
> > + ret = xevd_config(xectx->id, XEVD_CFG_GET_CODED_WIDTH, &avctx-
> >coded_width, &size);
> > + if (XEVD_FAILED(ret)) {
> > + av_log(avctx, AV_LOG_ERROR, "Failed to get coded_width\n");
> > + return AVERROR_EXTERNAL;
> > + }
> > +
> > + ret = xevd_config(xectx->id, XEVD_CFG_GET_CODED_HEIGHT, &avctx-
> >coded_height, &size);
> > + if (XEVD_FAILED(ret)) {
> > + av_log(avctx, AV_LOG_ERROR, "Failed to get coded_height\n");
> > + return AVERROR_EXTERNAL;
> > + }
> > +
> > + ret = xevd_config(xectx->id, XEVD_CFG_GET_WIDTH, &avctx->width,
> &size);
> > + if (XEVD_FAILED(ret)) {
> > + av_log(avctx, AV_LOG_ERROR, "Failed to get width\n");
> > + return AVERROR_EXTERNAL;
> > + }
> > +
> > + ret = xevd_config(xectx->id, XEVD_CFG_GET_HEIGHT, &avctx->height,
> &size);
> > + if (XEVD_FAILED(ret)) {
> > + av_log(avctx, AV_LOG_ERROR, "Failed to get height\n");
> > + return AVERROR_EXTERNAL;
> > + }
> > +
> > + ret = xevd_config(xectx->id, XEVD_CFG_GET_COLOR_SPACE,
> &color_space, &size);
> > + if (XEVD_FAILED(ret)) {
> > + av_log(avctx, AV_LOG_ERROR, "Failed to get color_space\n");
> > + return AVERROR_EXTERNAL;
> > + }
> > + switch(color_space) {
> > + case XEVD_CS_YCBCR400_10LE:
> > + avctx->pix_fmt = AV_PIX_FMT_GRAY10LE;
> > + break;
> > + case XEVD_CS_YCBCR420_10LE:
> > + avctx->pix_fmt = AV_PIX_FMT_YUV420P10LE;
> > + break;
> > + case XEVD_CS_YCBCR422_10LE:
> > + avctx->pix_fmt = AV_PIX_FMT_YUV422P10LE;
> > + break;
> > + case XEVD_CS_YCBCR444_10LE:
> > + avctx->pix_fmt = AV_PIX_FMT_YUV444P10LE;
> > + break;
> > + default:
> > + av_log(avctx, AV_LOG_ERROR, "Unknown color space\n");
> > + avctx->pix_fmt = AV_PIX_FMT_NONE;
> > + return AVERROR_INVALIDDATA;
> > + }
> > +
> > + // the function returns sps->num_reorder_pics
> > + ret = xevd_config(xectx->id, XEVD_CFG_GET_MAX_CODING_DELAY,
> &avctx->max_b_frames, &size);
> > + if (XEVD_FAILED(ret)) {
> > + av_log(avctx, AV_LOG_ERROR, "Failed to get
max_coding_delay\n");
> > + return AVERROR_EXTERNAL;
> > + }
> > +
> > + avctx->has_b_frames = (avctx->max_b_frames) ? 1 : 0;
> > +
> > + return 0;
> > +}
> > +
> > +/**
> > + * @brief Copy image in imgb to frame.
> > + *
> > + * @param avctx codec context
> > + * @param[in] imgb
> > + * @param[out] frame
> > + * @return 0 on success, negative value on failure */ static int
> > +libxevd_image_copy(struct AVCodecContext *avctx, XEVD_IMGB *imgb,
> > +struct AVFrame *frame) {
> > + int ret;
> > + if (imgb->cs != XEVD_CS_YCBCR420_10LE) {
> > + av_log(avctx, AV_LOG_ERROR, "Not supported pixel format: %s\n",
> av_get_pix_fmt_name(avctx->pix_fmt));
> > + return AVERROR_INVALIDDATA;
> > + }
> > +
> > + if (imgb->w[0] != avctx->width || imgb->h[0] != avctx->height) { //
stream
> resolution changed
> > + if (ff_set_dimensions(avctx, imgb->w[0], imgb->h[0]) < 0) {
> > + av_log(avctx, AV_LOG_ERROR, "Cannot set new dimension\n");
> > + return AVERROR_INVALIDDATA;
> > + }
> > + }
> > +
> > + if (ret = ff_get_buffer(avctx, frame, 0) < 0)
> > + return ret;
> > +
> > + av_image_copy(frame->data, frame->linesize, (const uint8_t
**)imgb->a,
> > + imgb->s, avctx->pix_fmt,
> > + imgb->w[0], imgb->h[0]);
> > +
> > + return 0;
> > +}
> > +
> > +/**
> > + * Initialize decoder
> > + * Create a decoder instance and allocate all the needed resources
> > + *
> > + * @param avctx codec context
> > + * @return 0 on success, negative error code on failure */ static
> > +av_cold int libxevd_init(AVCodecContext *avctx) {
> > + XevdContext *xectx = avctx->priv_data;
> > + XEVD_CDSC *cdsc = &(xectx->cdsc);
> > +
> > + /* read configurations and set values for created descriptor
(XEVD_CDSC)
> */
> > + get_conf(avctx, cdsc);
> > +
> > + /* create decoder */
> > + xectx->id = xevd_create(&(xectx->cdsc), NULL);
> > + if (xectx->id == NULL) {
> > + av_log(avctx, AV_LOG_ERROR, "Cannot create XEVD encoder\n");
> > + return AVERROR_EXTERNAL;
> > + }
> > +
> > + xectx->draining_mode = 0;
> > + xectx->pkt = av_packet_alloc();
> > +
> > + return 0;
> > +}
> > +
> > +/**
> > + * Decode frame with decoupled packet/frame dataflow
> > + *
> > + * @param avctx codec context
> > + * @param[out] frame decoded frame
> > + *
> > + * @return 0 on success, negative error code on failure
> > + */
> > +static int libxevd_receive_frame(AVCodecContext *avctx, AVFrame
> > +*frame) {
> > + XevdContext *xectx = avctx->priv_data;
> > + AVPacket *pkt = xectx->pkt;
> > + XEVD_IMGB *imgb = NULL;
> > +
> > + int xevd_ret = 0;
> > + int ret = 0;
> > +
> > + if (!pkt)
> > + return AVERROR(ENOMEM);
> > +
> > + // obtain input data
> > + ret = ff_decode_get_packet(avctx, pkt);
> > + if (ret < 0 && ret != AVERROR_EOF) {
> > + av_packet_unref(pkt);
> > +
> > + return ret;
> > + } else if(ret == AVERROR_EOF && xectx->draining_mode == 0) { //
> > + End of stream situations. Enter draining mode
> > +
> > + xectx->draining_mode = 1;
> > + av_packet_unref(pkt);
> > +
> > + return 0;
> > + }
> > +
> > + if (pkt->size > 0) {
> > + int bs_read_pos = 0;
> > + XEVD_STAT stat;
> > + XEVD_BITB bitb;
> > + int nalu_size;
> > +
> > + imgb = NULL;
> > +
> > + while(pkt->size > (bs_read_pos + XEVD_NAL_UNIT_LENGTH_BYTE)) {
> > + memset(&stat, 0, sizeof(XEVD_STAT));
> > +
> > + nalu_size = read_nal_unit_length(pkt->data + bs_read_pos,
> XEVD_NAL_UNIT_LENGTH_BYTE, avctx);
> > + if (nalu_size == 0) {
> > + av_log(avctx, AV_LOG_ERROR, "Invalid bitstream\n");
> > + av_packet_unref(pkt);
> > + ret = AVERROR_INVALIDDATA;
> > +
> > + return ret;
> > + }
> > + bs_read_pos += XEVD_NAL_UNIT_LENGTH_BYTE;
> > +
> > + bitb.addr = pkt->data + bs_read_pos;
> > + bitb.ssize = nalu_size;
> > +
> > + /* main decoding block */
> > + xevd_ret = xevd_decode(xectx->id, &bitb, &stat);
> > + if (XEVD_FAILED(xevd_ret)) {
> > + av_log(avctx, AV_LOG_ERROR, "Failed to decode
bitstream\n");
> > + av_packet_unref(pkt);
> > + ret = AVERROR_EXTERNAL;
> > +
> > + return ret;
> > + }
> > +
> > + bs_read_pos += nalu_size;
> > +
> > + if (stat.nalu_type == XEVD_NUT_SPS) { // EVC stream
parameters
> changed
> > + if ((ret = export_stream_params(xectx, avctx)) != 0) {
> > + av_log(avctx, AV_LOG_ERROR, "Failed to export
stream
> params\n");
> > + av_packet_unref(pkt);
> > +
> > + return ret;
> > + }
> > + }
> > + if (stat.read != nalu_size)
> > + av_log(avctx, AV_LOG_INFO, "Different reading of
bitstream (in:%d,
> read:%d)\n,", nalu_size, stat.read);
> > + if (stat.fnum >= 0) {
> > +
> > + xevd_ret = xevd_pull(xectx->id, &imgb); // The
> > + function returns a valid image only if the return code is XEVD_OK
> > +
> > + if (XEVD_FAILED(xevd_ret)) {
> > + av_log(avctx, AV_LOG_ERROR, "Failed to pull the
decoded image
> (xevd error code: %d, frame#=%d)\n", xevd_ret, stat.fnum);
> > + ret = AVERROR_EXTERNAL;
> > + av_packet_unref(pkt);
> > +
> > + return ret;
> > + } else if (xevd_ret == XEVD_OK_FRM_DELAYED) {
> > + av_packet_unref(pkt);
>
> Was the packet fully consumed here? As in, is bs_read_pos == pkt->size at
this
> point? If not, you're dropping NALUs that should be passed to libxevd.
Fixed and will be included in the next patchset.
>
> > +
> > + return AVERROR(EAGAIN);
> > + } else { // XEVD_OK
> > + if (!imgb) {
> > + av_packet_unref(pkt);
>
> Same question here. And I assume this would happen with
> XEVD_OK_OUT_NOT_AVAILABLE?
>
> > +
> > + return AVERROR(EAGAIN);
> > + }
> > +
> > + // got frame
> > + ret = libxevd_image_copy(avctx, imgb, frame);
> > + if(ret < 0) {
> > + av_log(avctx, AV_LOG_ERROR, "Image copying
> > + error\n");
> > +
> > + imgb->release(imgb);
> > + imgb = NULL;
> > +
> > + av_packet_unref(pkt);
> > +
> > + return ret;
> > + }
> > +
> > + // use ff_decode_frame_props() to fill frame
properties
> > + ret = ff_decode_frame_props(avctx, frame);
>
> ff_get_buffer() calls this, so it's not needed.
> You however set the decoder as FF_CODEC_CAP_SETS_FRAME_PROPS, so
> ff_decode_frame_props_pkt() is not being called to fetch props from the
last
> packet.
>
> You probably need a FIFO of packet props and use that to fill frame props
as you
> return them. It's a pity you can't propagate an opaque pointer to
xevd_decode()
> and get it back in xevd_pull(). That would simplify things a lot.
>
It's not entirely like that.
I can do it. I mean that I can propagate an opaque pointer to xevd_decode()
and get it back in xevd_pull()..
The structure XEVD_BITB has an array called pdata intended for passing
arbitrary pointers (void *pdata[XEVD_PDATA_NUM]). A similar array is present
in the structure of type XEVE_IMGB.
A pointer to the XEVD_BITB object is passed to the function xevd_decode(),
and the XEVE_IMGB object is returned by the function xevd_pull().
So I can propagate an opaque pointer to xevd_decode() and get it back in
xevd_pull().
In conclusion, I can pass a pointer to AVPacket to the xevd_decode()
function, then in the xevd_pull() function, get that pointer and finally
call the ff_decode_frame_props_from_pkt(avctx, frame, pkt) function.
Is it the way, how it should be done?
> > + if (ret < 0) {
> > + imgb->release(imgb);
> > + imgb = NULL;
> > +
> > + av_packet_unref(pkt);
> > + av_frame_unref(frame);
> > +
> > + return ret;
> > + }
> > +
> > + frame->pkt_dts = pkt->dts;
> > +
> > + // xevd_pull uses pool of objects of type
XEVD_IMGB.
> > + // The pool size is equal MAX_PB_SIZE (26), so
release object when
> it is no more needed
> > + imgb->release(imgb);
> > + imgb = NULL;
> > +
> > + av_packet_unref(pkt);
>
> Again, were all the NALUs in the packet passed to libxevd?
Fixed and will be included in the next patchset.
>
> > + return 0;
> > + }
> > + }
> > + }
> > + } else { // decoder draining mode handling
> > +
> > + xevd_ret = xevd_pull(xectx->id, &imgb);
> > +
> > + if (xevd_ret == XEVD_ERR_UNEXPECTED) { // draining process
completed
> > + av_log(avctx, AV_LOG_DEBUG, "Draining process
completed\n");
> > + av_packet_unref(pkt);
> > +
> > + return AVERROR_EOF;
> > + } else if (XEVD_FAILED(xevd_ret)) { // handle all other errors
> > + av_log(avctx, AV_LOG_ERROR, "Failed to pull the decoded
image (xevd
> error code: %d)\n", xevd_ret);
> > + av_packet_unref(pkt);
> > +
> > + return AVERROR_EXTERNAL;
> > + } else { // XEVD_OK
> > + if (!imgb) {
> > + av_packet_unref(pkt);
> > +
> > + return AVERROR_EXTERNAL;
> > + }
> > + // got frame
> > + ret = libxevd_image_copy(avctx, imgb, frame);
> > + if(ret < 0) {
> > + imgb->release(imgb);
> > + imgb = NULL;
> > +
> > + av_packet_unref(pkt);
> > +
> > + return ret;
> > + }
> > +
> > + frame->pkt_dts = pkt->dts;
> > +
> > + av_packet_unref(pkt);
> > +
> > + // xevd_pull uses pool of objects of type XEVD_IMGB.
> > + // The pool size is equal MAX_PB_SIZE (26), so release
object when it is
> no more needed
> > + imgb->release(imgb);
> > + imgb = NULL;
> > +
> > + return 0;
> > + }
> > + }
> > +
> > + return ret;
> > +}
> > +
> > +/**
> > + * Destroy decoder
> > + *
> > + * @param avctx codec context
> > + * @return 0 on success
> > + */
> > +static av_cold int libxevd_close(AVCodecContext *avctx) {
> > + XevdContext *xectx = avctx->priv_data;
> > + if (xectx->id) {
> > + xevd_delete(xectx->id);
> > + xectx->id = NULL;
> > + }
> > +
> > + xectx->draining_mode = 0;
> > + av_packet_free(&xectx->pkt);
> > +
> > + return 0;
> > +}
> > +
> > +#define OFFSET(x) offsetof(XevdContext, x) #define VD
> > +AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_DECODING_PARAM
> > +
> > +static const AVClass libxevd_class = {
> > + .class_name = "libxevd",
> > + .item_name = av_default_item_name,
> > + .version = LIBAVUTIL_VERSION_INT,
> > +};
> > +
> > +const FFCodec ff_libxevd_decoder = {
> > + .p.name = "evc",
> > + .p.long_name = NULL_IF_CONFIG_SMALL("EVC / MPEG-5 Essential
> Video Coding (EVC)"),
> > + .p.type = AVMEDIA_TYPE_VIDEO,
> > + .p.id = AV_CODEC_ID_EVC,
> > + .init = libxevd_init,
> > + FF_CODEC_RECEIVE_FRAME_CB(libxevd_receive_frame),
> > + .close = libxevd_close,
> > + .priv_data_size = sizeof(XevdContext),
> > + .p.priv_class = &libxevd_class,
> > + .p.capabilities = AV_CODEC_CAP_DELAY |
> AV_CODEC_CAP_OTHER_THREADS | AV_CODEC_CAP_AVOID_PROBING,
> > + .p.profiles = NULL_IF_CONFIG_SMALL(ff_evc_profiles),
> > + .p.wrapper_name = "libxevd",
> > + .caps_internal = FF_CODEC_CAP_INIT_CLEANUP |
> FF_CODEC_CAP_NOT_INIT_THREADSAFE | FF_CODEC_CAP_SETS_PKT_DTS |
> FF_CODEC_CAP_SETS_FRAME_PROPS
> > +};
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://protect2.fireeye.com/v1/url?k=6cd54358-0d5e566e-6cd4c817-
> 74fe485cbff1-e18dc94cc35fe766&q=1&e=49c754eb-4416-4e31-90e9-
> ee8e7d469d1f&u=https%3A%2F%2Fffmpeg.org%2Fmailman%2Flistinfo%2Fffmp
> eg-devel
>
> To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org
> with subject "unsubscribe".
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
^ permalink raw reply [flat|nested] 5+ messages in thread* Re: [FFmpeg-devel] [PATCH v26 5/9] avcodec/evc_decoder: Provided support for EVC decoder
2023-07-26 15:45 ` James Almer
2023-07-27 13:43 ` Dawid Kozinski/Multimedia (PLT) /SRPOL/Staff Engineer/Samsung Electronics
2023-07-31 13:24 ` Dawid Kozinski/Multimedia (PLT) /SRPOL/Staff Engineer/Samsung Electronics
@ 2023-08-18 5:44 ` Dawid Kozinski/Multimedia (PLT) /SRPOL/Staff Engineer/Samsung Electronics
2 siblings, 0 replies; 5+ messages in thread
From: Dawid Kozinski/Multimedia (PLT) /SRPOL/Staff Engineer/Samsung Electronics @ 2023-08-18 5:44 UTC (permalink / raw)
To: 'FFmpeg development discussions and patches'
> -----Original Message-----
> From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of James
> Almer
> Sent: środa, 26 lipca 2023 17:46
> To: ffmpeg-devel@ffmpeg.org
> Subject: Re: [FFmpeg-devel] [PATCH v26 5/9] avcodec/evc_decoder: Provided
> support for EVC decoder
>
> On 6/15/2023 8:48 AM, Dawid Kozinski wrote:
> > - Added EVC decoder wrapper
> > - Changes in project configuration file and libavcodec Makefile
> > - Added documentation for xevd wrapper
> >
> > Signed-off-by: Dawid Kozinski <d.kozinski@samsung.com>
>
> I'm getting
>
> [evc @ 000001ee1ba7e960] An invalid frame was output by a decoder. This is
a
> bug, please report it.
> [vist#0:0/evc @ 000001ee1d47f220] Decoding error: Internal bug, should not
> have happened
>
> With akiyo_cif.evc, so there's something wrong.
Done.
I've fixed that. The change has been pushed to a new patchset (v27)
containing implementations of EVC encoder and decoder wrappers.
The remaining things you mentioned below have also been taken into account.
>
> > ---
> > configure | 4 +
> > doc/decoders.texi | 24 ++
> > doc/general_contents.texi | 10 +-
> > libavcodec/Makefile | 1 +
> > libavcodec/allcodecs.c | 1 +
> > libavcodec/libxevd.c | 479 ++++++++++++++++++++++++++++++++++++++
> > 6 files changed, 518 insertions(+), 1 deletion(-)
> > create mode 100644 libavcodec/libxevd.c
> >
> > diff --git a/configure b/configure
> > index e2370a23bb..cae3b666a5 100755
> > --- a/configure
> > +++ b/configure
> > @@ -293,6 +293,7 @@ External library support:
> > --enable-libx264 enable H.264 encoding via x264 [no]
> > --enable-libx265 enable HEVC encoding via x265 [no]
> > --enable-libxeve enable EVC encoding via libxeve [no]
> > + --enable-libxevd enable EVC decoding via libxevd [no]
> > --enable-libxavs enable AVS encoding via xavs [no]
> > --enable-libxavs2 enable AVS2 encoding via xavs2 [no]
> > --enable-libxcb enable X11 grabbing using XCB [autodetect]
> > @@ -1904,6 +1905,7 @@ EXTERNAL_LIBRARY_LIST="
> > libvorbis
> > libvpx
> > libwebp
> > + libxevd
> > libxeve
> > libxml2
> > libzimg
> > @@ -3460,6 +3462,7 @@ libx265_encoder_deps="libx265"
> > libx265_encoder_select="atsc_a53"
> > libxavs_encoder_deps="libxavs"
> > libxavs2_encoder_deps="libxavs2"
> > +libxevd_decoder_deps="libxevd"
> > libxeve_encoder_deps="libxeve"
> > libxvid_encoder_deps="libxvid"
> > libzvbi_teletext_decoder_deps="libzvbi"
> > @@ -6835,6 +6838,7 @@ enabled libx265 && require_pkg_config
> libx265 x265 x265.h x265_api_get
> > require_cpp_condition libx265 x265.h
"X265_BUILD >= 89"
> > enabled libxavs && require libxavs "stdint.h xavs.h"
> xavs_encoder_encode "-lxavs $pthreads_extralibs $libm_extralibs"
> > enabled libxavs2 && require_pkg_config libxavs2 "xavs2 >=
1.3.0"
> "stdint.h xavs2.h" xavs2_api_get
> > +enabled libxevd && require_pkg_config libxevd "xevd >= 0.4.1"
"xevd.h"
> xevd_decode
> > enabled libxeve && require_pkg_config libxeve "xeve >=
0.4.3" "xeve.h"
> xeve_encode
> > enabled libxvid && require libxvid xvid.h xvid_global
-lxvidcore
> > enabled libzimg && require_pkg_config libzimg "zimg >=
2.7.0" zimg.h
> zimg_get_api_version
> > diff --git a/doc/decoders.texi b/doc/decoders.texi index
> > 09b8314dd2..6311af229f 100644
> > --- a/doc/decoders.texi
> > +++ b/doc/decoders.texi
> > @@ -130,6 +130,30 @@ Set amount of frame threads to use during
> > decoding. The default value is 0 (auto
> >
> > @end table
> >
> > +@section libxevd
> > +
> > +eXtra-fast Essential Video Decoder (XEVD) MPEG-5 EVC decoder wrapper.
> > +
> > +This decoder requires the presence of the libxevd headers and library
> > +during configuration. You need to explicitly configure the build with
> > +@option{--enable-libxevd}.
> > +
> > +The xevd project website is at
> @url{https://protect2.fireeye.com/v1/url?k=b6ce3a07-d7452f31-b6cfb148-
> 74fe485cbff1-251589a888281453&q=1&e=49c754eb-4416-4e31-90e9-
> ee8e7d469d1f&u=https%3A%2F%2Fgithub.com%2Fmpeg5%2Fxevd%257D.
> > +
> > +@subsection Options
> > +
> > +The following options are supported by the libxevd wrapper.
> > +The xevd-equivalent options or values are listed in parentheses for
easy
> migration.
> > +
> > +To get a more accurate and extensive documentation of the libxevd
> > +options, invoke the command @code{xevd_app --help} or consult the
libxevd
> documentation.
> > +
> > +@table @option
> > +@item threads (@emph{threads})
> > +Force to use a specific number of threads
> > +
> > +@end table
> > +
> > @section QSV Decoders
> >
> > The family of Intel QuickSync Video decoders (VC1, MPEG-2, H.264,
> > HEVC, diff --git a/doc/general_contents.texi
> > b/doc/general_contents.texi index c6a997bfd6..8e08f5ebc3 100644
> > --- a/doc/general_contents.texi
> > +++ b/doc/general_contents.texi
> > @@ -351,6 +351,14 @@ Go to
> @url{https://protect2.fireeye.com/v1/url?k=76721d6d-17f9085b-76739622-
> 74fe485cbff1-6b070a322743d6be&q=1&e=49c754eb-4416-4e31-90e9-
> ee8e7d469d1f&u=https%3A%2F%2Fgithub.com%2Fmpeg5%2Fxeve%257D and
> follow the instructions for
> > installing the XEVE library. Then pass @code{--enable-libxeve} to
configure to
> > enable it.
> >
> > +@section eXtra-fast Essential Video Decoder (XEVD)
> > +
> > +FFmpeg can make use of the XEVD library for EVC video decoding.
> > +
> > +Go to
> > +@url{https://protect2.fireeye.com/v1/url?k=3be9a12a-5a62b41c-3be82a65
> > +-74fe485cbff1-6cd6af142a668aff&q=1&e=49c754eb-4416-4e31-90e9-
> ee8e7d469d1f&u=https%3A%2F%2Fgithub.com%2Fmpeg5%2Fxevd%257D and
> follow the instructions for installing the XEVD library. Then pass
@code{--
> enable-libxevd} to configure to enable it.
> > +
> > @section ZVBI
> >
> > ZVBI is a VBI decoding library which can be used by FFmpeg to decode
> > DVB @@ -953,7 +961,7 @@ following image formats are supported:
> > @item Escape 124 @tab @tab X
> > @item Escape 130 @tab @tab X
> > @item EVC / MPEG-5 Part 1 @tab X @tab X
> > - @tab encoding and decoding supported through external library
libxeve
> > + @tab encoding and decoding supported through external libraries
> > + libxeve and libxevd
> > @item FFmpeg video codec #1 @tab X @tab X
> > @tab lossless codec (fourcc: FFV1)
> > @item Flash Screen Video v1 @tab X @tab X diff --git
> > a/libavcodec/Makefile b/libavcodec/Makefile index
> > 2223e9f46c..d0b8438717 100644
> > --- a/libavcodec/Makefile
> > +++ b/libavcodec/Makefile
> > @@ -1142,6 +1142,7 @@ OBJS-$(CONFIG_LIBX264_ENCODER) +=
> libx264.o
> > OBJS-$(CONFIG_LIBX265_ENCODER) += libx265.o
> > OBJS-$(CONFIG_LIBXAVS_ENCODER) += libxavs.o
> > OBJS-$(CONFIG_LIBXAVS2_ENCODER) += libxavs2.o
> > +OBJS-$(CONFIG_LIBXEVD_DECODER) += libxevd.o
> > OBJS-$(CONFIG_LIBXEVE_ENCODER) += libxeve.o
> > OBJS-$(CONFIG_LIBXVID_ENCODER) += libxvid.o
> > OBJS-$(CONFIG_LIBZVBI_TELETEXT_DECODER) += libzvbi-teletextdec.o
> ass.o
> > diff --git a/libavcodec/allcodecs.c b/libavcodec/allcodecs.c index
> > c317dd6da3..eb0abbb1fd 100644
> > --- a/libavcodec/allcodecs.c
> > +++ b/libavcodec/allcodecs.c
> > @@ -819,6 +819,7 @@ extern LIBX264_CONST FFCodec ff_libx264_encoder;
> > extern const FFCodec ff_libx264rgb_encoder;
> > extern FFCodec ff_libx265_encoder;
> > extern const FFCodec ff_libxeve_encoder;
> > +extern const FFCodec ff_libxevd_decoder;
> > extern const FFCodec ff_libxavs_encoder;
> > extern const FFCodec ff_libxavs2_encoder;
> > extern const FFCodec ff_libxvid_encoder; diff --git
> > a/libavcodec/libxevd.c b/libavcodec/libxevd.c new file mode 100644
> > index 0000000000..7c4922d46e
> > --- /dev/null
> > +++ b/libavcodec/libxevd.c
> > @@ -0,0 +1,479 @@
> > +/*
> > + * libxevd decoder
> > + * EVC (MPEG-5 Essential Video Coding) decoding using XEVD MPEG-5 EVC
> > +decoder library
> > + *
> > + * Copyright (C) 2021 Dawid Kozinski <d.kozinski@samsung.com>
> > + *
> > + * This file is part of FFmpeg.
> > + *
> > + * FFmpeg is free software; you can redistribute it and/or
> > + * modify it under the terms of the GNU Lesser General Public
> > + * License as published by the Free Software Foundation; either
> > + * version 2.1 of the License, or (at your option) any later version.
> > + *
> > + * FFmpeg is distributed in the hope that it will be useful,
> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
> > + * Lesser General Public License for more details.
> > + *
> > + * You should have received a copy of the GNU Lesser General Public
> > + * License along with FFmpeg; if not, write to the Free Software
> > + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA
> > +02110-1301 USA */
> > +
> > +#include <float.h>
> > +#include <stdlib.h>
> > +
> > +#include <xevd.h>
> > +
> > +#include "libavutil/internal.h"
> > +#include "libavutil/common.h"
> > +#include "libavutil/opt.h"
> > +#include "libavutil/pixdesc.h"
> > +#include "libavutil/pixfmt.h"
> > +#include "libavutil/imgutils.h"
> > +#include "libavutil/cpu.h"
> > +
> > +#include "avcodec.h"
> > +#include "internal.h"
> > +#include "packet_internal.h"
> > +#include "codec_internal.h"
> > +#include "profiles.h"
> > +#include "decode.h"
> > +
> > +#define XEVD_PARAM_BAD_NAME -1
> > +#define XEVD_PARAM_BAD_VALUE -2
> > +
> > +#define EVC_NAL_HEADER_SIZE 2 /* byte */
> > +
> > +/**
> > + * The structure stores all the states associated with the instance
> > +of Xeve MPEG-5 EVC decoder */ typedef struct XevdContext {
> > + const AVClass *class;
> > +
> > + XEVD id; // XEVD instance identifier @see xevd.h
> > + XEVD_CDSC cdsc; // decoding parameters @see xevd.h
> > +
> > + // If end of stream occurs it is required "flushing" (aka draining)
the codec,
> > + // as the codec might buffer multiple frames or packets internally.
> > + int draining_mode; // The flag is set if codec enters draining
mode.
> > +
> > + AVPacket *pkt;
> > +} XevdContext;
> > +
> > +/**
> > + * The function populates the XEVD_CDSC structure.
> > + * XEVD_CDSC contains all decoder parameters that should be initialized
> before its use.
> > + *
> > + * @param[in] avctx codec context
> > + * @param[out] cdsc contains all decoder parameters that should be
> > +initialized before its use
> > + *
> > + */
> > +static void get_conf(AVCodecContext *avctx, XEVD_CDSC *cdsc) {
> > + int cpu_count = av_cpu_count();
> > +
> > + /* clear XEVS_CDSC structure */
> > + memset(cdsc, 0, sizeof(XEVD_CDSC));
> > +
> > + /* init XEVD_CDSC */
> > + if (avctx->thread_count <= 0)
> > + cdsc->threads = (cpu_count < XEVD_MAX_TASK_CNT) ? cpu_count :
> XEVD_MAX_TASK_CNT;
> > + else if (avctx->thread_count > XEVD_MAX_TASK_CNT)
> > + cdsc->threads = XEVD_MAX_TASK_CNT;
> > + else
> > + cdsc->threads = avctx->thread_count; }
> > +
> > +/**
> > + * Read NAL unit length
> > + * @param bs input data (bitstream)
> > + * @return the length of NAL unit on success, 0 value on failure */
> > +static uint32_t read_nal_unit_length(const uint8_t *bs, int bs_size,
> > +AVCodecContext *avctx) {
> > + uint32_t len = 0;
> > + XEVD_INFO info;
> > + int ret;
> > +
> > + if (bs_size == XEVD_NAL_UNIT_LENGTH_BYTE) {
> > + ret = xevd_info((void *)bs, XEVD_NAL_UNIT_LENGTH_BYTE, 1,
&info);
> > + if (XEVD_FAILED(ret)) {
> > + av_log(avctx, AV_LOG_ERROR, "Cannot get bitstream
information\n");
> > + return 0;
> > + }
> > + len = info.nalu_len;
> > + if (len == 0) {
> > + av_log(avctx, AV_LOG_ERROR, "Invalid bitstream size!
[%d]\n",
> bs_size);
> > + return 0;
> > + }
> > + }
> > +
> > + return len;
> > +}
> > +
> > +/**
> > + * @param[in] xectx the structure that stores all the state
> > +associated with the instance of Xeve MPEG-5 EVC decoder
> > + * @param[out] avctx codec context
> > + * @return 0 on success, negative value on failure */ static int
> > +export_stream_params(const XevdContext *xectx, AVCodecContext *avctx)
> > +{
> > + int ret;
> > + int size;
> > + int color_space;
> > +
> > + avctx->pix_fmt = AV_PIX_FMT_YUV420P10;
> > +
> > + size = 4;
> > + ret = xevd_config(xectx->id, XEVD_CFG_GET_CODED_WIDTH, &avctx-
> >coded_width, &size);
> > + if (XEVD_FAILED(ret)) {
> > + av_log(avctx, AV_LOG_ERROR, "Failed to get coded_width\n");
> > + return AVERROR_EXTERNAL;
> > + }
> > +
> > + ret = xevd_config(xectx->id, XEVD_CFG_GET_CODED_HEIGHT, &avctx-
> >coded_height, &size);
> > + if (XEVD_FAILED(ret)) {
> > + av_log(avctx, AV_LOG_ERROR, "Failed to get coded_height\n");
> > + return AVERROR_EXTERNAL;
> > + }
> > +
> > + ret = xevd_config(xectx->id, XEVD_CFG_GET_WIDTH, &avctx->width,
> &size);
> > + if (XEVD_FAILED(ret)) {
> > + av_log(avctx, AV_LOG_ERROR, "Failed to get width\n");
> > + return AVERROR_EXTERNAL;
> > + }
> > +
> > + ret = xevd_config(xectx->id, XEVD_CFG_GET_HEIGHT, &avctx->height,
> &size);
> > + if (XEVD_FAILED(ret)) {
> > + av_log(avctx, AV_LOG_ERROR, "Failed to get height\n");
> > + return AVERROR_EXTERNAL;
> > + }
> > +
> > + ret = xevd_config(xectx->id, XEVD_CFG_GET_COLOR_SPACE,
> &color_space, &size);
> > + if (XEVD_FAILED(ret)) {
> > + av_log(avctx, AV_LOG_ERROR, "Failed to get color_space\n");
> > + return AVERROR_EXTERNAL;
> > + }
> > + switch(color_space) {
> > + case XEVD_CS_YCBCR400_10LE:
> > + avctx->pix_fmt = AV_PIX_FMT_GRAY10LE;
> > + break;
> > + case XEVD_CS_YCBCR420_10LE:
> > + avctx->pix_fmt = AV_PIX_FMT_YUV420P10LE;
> > + break;
> > + case XEVD_CS_YCBCR422_10LE:
> > + avctx->pix_fmt = AV_PIX_FMT_YUV422P10LE;
> > + break;
> > + case XEVD_CS_YCBCR444_10LE:
> > + avctx->pix_fmt = AV_PIX_FMT_YUV444P10LE;
> > + break;
> > + default:
> > + av_log(avctx, AV_LOG_ERROR, "Unknown color space\n");
> > + avctx->pix_fmt = AV_PIX_FMT_NONE;
> > + return AVERROR_INVALIDDATA;
> > + }
> > +
> > + // the function returns sps->num_reorder_pics
> > + ret = xevd_config(xectx->id, XEVD_CFG_GET_MAX_CODING_DELAY,
> &avctx->max_b_frames, &size);
> > + if (XEVD_FAILED(ret)) {
> > + av_log(avctx, AV_LOG_ERROR, "Failed to get
max_coding_delay\n");
> > + return AVERROR_EXTERNAL;
> > + }
> > +
> > + avctx->has_b_frames = (avctx->max_b_frames) ? 1 : 0;
> > +
> > + return 0;
> > +}
> > +
> > +/**
> > + * @brief Copy image in imgb to frame.
> > + *
> > + * @param avctx codec context
> > + * @param[in] imgb
> > + * @param[out] frame
> > + * @return 0 on success, negative value on failure */ static int
> > +libxevd_image_copy(struct AVCodecContext *avctx, XEVD_IMGB *imgb,
> > +struct AVFrame *frame) {
> > + int ret;
> > + if (imgb->cs != XEVD_CS_YCBCR420_10LE) {
> > + av_log(avctx, AV_LOG_ERROR, "Not supported pixel format: %s\n",
> av_get_pix_fmt_name(avctx->pix_fmt));
> > + return AVERROR_INVALIDDATA;
> > + }
> > +
> > + if (imgb->w[0] != avctx->width || imgb->h[0] != avctx->height) { //
stream
> resolution changed
> > + if (ff_set_dimensions(avctx, imgb->w[0], imgb->h[0]) < 0) {
> > + av_log(avctx, AV_LOG_ERROR, "Cannot set new dimension\n");
> > + return AVERROR_INVALIDDATA;
> > + }
> > + }
> > +
> > + if (ret = ff_get_buffer(avctx, frame, 0) < 0)
> > + return ret;
> > +
> > + av_image_copy(frame->data, frame->linesize, (const uint8_t
**)imgb->a,
> > + imgb->s, avctx->pix_fmt,
> > + imgb->w[0], imgb->h[0]);
> > +
> > + return 0;
> > +}
> > +
> > +/**
> > + * Initialize decoder
> > + * Create a decoder instance and allocate all the needed resources
> > + *
> > + * @param avctx codec context
> > + * @return 0 on success, negative error code on failure */ static
> > +av_cold int libxevd_init(AVCodecContext *avctx) {
> > + XevdContext *xectx = avctx->priv_data;
> > + XEVD_CDSC *cdsc = &(xectx->cdsc);
> > +
> > + /* read configurations and set values for created descriptor
(XEVD_CDSC)
> */
> > + get_conf(avctx, cdsc);
> > +
> > + /* create decoder */
> > + xectx->id = xevd_create(&(xectx->cdsc), NULL);
> > + if (xectx->id == NULL) {
> > + av_log(avctx, AV_LOG_ERROR, "Cannot create XEVD encoder\n");
> > + return AVERROR_EXTERNAL;
> > + }
> > +
> > + xectx->draining_mode = 0;
> > + xectx->pkt = av_packet_alloc();
> > +
> > + return 0;
> > +}
> > +
> > +/**
> > + * Decode frame with decoupled packet/frame dataflow
> > + *
> > + * @param avctx codec context
> > + * @param[out] frame decoded frame
> > + *
> > + * @return 0 on success, negative error code on failure
> > + */
> > +static int libxevd_receive_frame(AVCodecContext *avctx, AVFrame
> > +*frame) {
> > + XevdContext *xectx = avctx->priv_data;
> > + AVPacket *pkt = xectx->pkt;
> > + XEVD_IMGB *imgb = NULL;
> > +
> > + int xevd_ret = 0;
> > + int ret = 0;
> > +
> > + if (!pkt)
> > + return AVERROR(ENOMEM);
> > +
> > + // obtain input data
> > + ret = ff_decode_get_packet(avctx, pkt);
> > + if (ret < 0 && ret != AVERROR_EOF) {
> > + av_packet_unref(pkt);
> > +
> > + return ret;
> > + } else if(ret == AVERROR_EOF && xectx->draining_mode == 0) { //
> > + End of stream situations. Enter draining mode
> > +
> > + xectx->draining_mode = 1;
> > + av_packet_unref(pkt);
> > +
> > + return 0;
> > + }
> > +
> > + if (pkt->size > 0) {
> > + int bs_read_pos = 0;
> > + XEVD_STAT stat;
> > + XEVD_BITB bitb;
> > + int nalu_size;
> > +
> > + imgb = NULL;
> > +
> > + while(pkt->size > (bs_read_pos + XEVD_NAL_UNIT_LENGTH_BYTE)) {
> > + memset(&stat, 0, sizeof(XEVD_STAT));
> > +
> > + nalu_size = read_nal_unit_length(pkt->data + bs_read_pos,
> XEVD_NAL_UNIT_LENGTH_BYTE, avctx);
> > + if (nalu_size == 0) {
> > + av_log(avctx, AV_LOG_ERROR, "Invalid bitstream\n");
> > + av_packet_unref(pkt);
> > + ret = AVERROR_INVALIDDATA;
> > +
> > + return ret;
> > + }
> > + bs_read_pos += XEVD_NAL_UNIT_LENGTH_BYTE;
> > +
> > + bitb.addr = pkt->data + bs_read_pos;
> > + bitb.ssize = nalu_size;
> > +
> > + /* main decoding block */
> > + xevd_ret = xevd_decode(xectx->id, &bitb, &stat);
> > + if (XEVD_FAILED(xevd_ret)) {
> > + av_log(avctx, AV_LOG_ERROR, "Failed to decode
bitstream\n");
> > + av_packet_unref(pkt);
> > + ret = AVERROR_EXTERNAL;
> > +
> > + return ret;
> > + }
> > +
> > + bs_read_pos += nalu_size;
> > +
> > + if (stat.nalu_type == XEVD_NUT_SPS) { // EVC stream
parameters
> changed
> > + if ((ret = export_stream_params(xectx, avctx)) != 0) {
> > + av_log(avctx, AV_LOG_ERROR, "Failed to export
stream
> params\n");
> > + av_packet_unref(pkt);
> > +
> > + return ret;
> > + }
> > + }
> > + if (stat.read != nalu_size)
> > + av_log(avctx, AV_LOG_INFO, "Different reading of
bitstream (in:%d,
> read:%d)\n,", nalu_size, stat.read);
> > + if (stat.fnum >= 0) {
> > +
> > + xevd_ret = xevd_pull(xectx->id, &imgb); // The
> > + function returns a valid image only if the return code is XEVD_OK
> > +
> > + if (XEVD_FAILED(xevd_ret)) {
> > + av_log(avctx, AV_LOG_ERROR, "Failed to pull the
decoded image
> (xevd error code: %d, frame#=%d)\n", xevd_ret, stat.fnum);
> > + ret = AVERROR_EXTERNAL;
> > + av_packet_unref(pkt);
> > +
> > + return ret;
> > + } else if (xevd_ret == XEVD_OK_FRM_DELAYED) {
> > + av_packet_unref(pkt);
>
> Was the packet fully consumed here? As in, is bs_read_pos == pkt->size at
this
> point? If not, you're dropping NALUs that should be passed to libxevd.
>
> > +
> > + return AVERROR(EAGAIN);
> > + } else { // XEVD_OK
> > + if (!imgb) {
> > + av_packet_unref(pkt);
>
> Same question here. And I assume this would happen with
> XEVD_OK_OUT_NOT_AVAILABLE?
>
> > +
> > + return AVERROR(EAGAIN);
> > + }
> > +
> > + // got frame
> > + ret = libxevd_image_copy(avctx, imgb, frame);
> > + if(ret < 0) {
> > + av_log(avctx, AV_LOG_ERROR, "Image copying
> > + error\n");
> > +
> > + imgb->release(imgb);
> > + imgb = NULL;
> > +
> > + av_packet_unref(pkt);
> > +
> > + return ret;
> > + }
> > +
> > + // use ff_decode_frame_props() to fill frame
properties
> > + ret = ff_decode_frame_props(avctx, frame);
>
> ff_get_buffer() calls this, so it's not needed.
> You however set the decoder as FF_CODEC_CAP_SETS_FRAME_PROPS, so
> ff_decode_frame_props_pkt() is not being called to fetch props from the
last
> packet.
>
> You probably need a FIFO of packet props and use that to fill frame props
as you
> return them. It's a pity you can't propagate an opaque pointer to
xevd_decode()
> and get it back in xevd_pull(). That would simplify things a lot.
>
> > + if (ret < 0) {
> > + imgb->release(imgb);
> > + imgb = NULL;
> > +
> > + av_packet_unref(pkt);
> > + av_frame_unref(frame);
> > +
> > + return ret;
> > + }
> > +
> > + frame->pkt_dts = pkt->dts;
> > +
> > + // xevd_pull uses pool of objects of type
XEVD_IMGB.
> > + // The pool size is equal MAX_PB_SIZE (26), so
release object when
> it is no more needed
> > + imgb->release(imgb);
> > + imgb = NULL;
> > +
> > + av_packet_unref(pkt);
>
> Again, were all the NALUs in the packet passed to libxevd?
>
> > + return 0;
> > + }
> > + }
> > + }
> > + } else { // decoder draining mode handling
> > +
> > + xevd_ret = xevd_pull(xectx->id, &imgb);
> > +
> > + if (xevd_ret == XEVD_ERR_UNEXPECTED) { // draining process
completed
> > + av_log(avctx, AV_LOG_DEBUG, "Draining process
completed\n");
> > + av_packet_unref(pkt);
> > +
> > + return AVERROR_EOF;
> > + } else if (XEVD_FAILED(xevd_ret)) { // handle all other errors
> > + av_log(avctx, AV_LOG_ERROR, "Failed to pull the decoded
image (xevd
> error code: %d)\n", xevd_ret);
> > + av_packet_unref(pkt);
> > +
> > + return AVERROR_EXTERNAL;
> > + } else { // XEVD_OK
> > + if (!imgb) {
> > + av_packet_unref(pkt);
> > +
> > + return AVERROR_EXTERNAL;
> > + }
> > + // got frame
> > + ret = libxevd_image_copy(avctx, imgb, frame);
> > + if(ret < 0) {
> > + imgb->release(imgb);
> > + imgb = NULL;
> > +
> > + av_packet_unref(pkt);
> > +
> > + return ret;
> > + }
> > +
> > + frame->pkt_dts = pkt->dts;
> > +
> > + av_packet_unref(pkt);
> > +
> > + // xevd_pull uses pool of objects of type XEVD_IMGB.
> > + // The pool size is equal MAX_PB_SIZE (26), so release
object when it is
> no more needed
> > + imgb->release(imgb);
> > + imgb = NULL;
> > +
> > + return 0;
> > + }
> > + }
> > +
> > + return ret;
> > +}
> > +
> > +/**
> > + * Destroy decoder
> > + *
> > + * @param avctx codec context
> > + * @return 0 on success
> > + */
> > +static av_cold int libxevd_close(AVCodecContext *avctx) {
> > + XevdContext *xectx = avctx->priv_data;
> > + if (xectx->id) {
> > + xevd_delete(xectx->id);
> > + xectx->id = NULL;
> > + }
> > +
> > + xectx->draining_mode = 0;
> > + av_packet_free(&xectx->pkt);
> > +
> > + return 0;
> > +}
> > +
> > +#define OFFSET(x) offsetof(XevdContext, x) #define VD
> > +AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_DECODING_PARAM
> > +
> > +static const AVClass libxevd_class = {
> > + .class_name = "libxevd",
> > + .item_name = av_default_item_name,
> > + .version = LIBAVUTIL_VERSION_INT,
> > +};
> > +
> > +const FFCodec ff_libxevd_decoder = {
> > + .p.name = "evc",
> > + .p.long_name = NULL_IF_CONFIG_SMALL("EVC / MPEG-5 Essential
> Video Coding (EVC)"),
> > + .p.type = AVMEDIA_TYPE_VIDEO,
> > + .p.id = AV_CODEC_ID_EVC,
> > + .init = libxevd_init,
> > + FF_CODEC_RECEIVE_FRAME_CB(libxevd_receive_frame),
> > + .close = libxevd_close,
> > + .priv_data_size = sizeof(XevdContext),
> > + .p.priv_class = &libxevd_class,
> > + .p.capabilities = AV_CODEC_CAP_DELAY |
> AV_CODEC_CAP_OTHER_THREADS | AV_CODEC_CAP_AVOID_PROBING,
> > + .p.profiles = NULL_IF_CONFIG_SMALL(ff_evc_profiles),
> > + .p.wrapper_name = "libxevd",
> > + .caps_internal = FF_CODEC_CAP_INIT_CLEANUP |
> FF_CODEC_CAP_NOT_INIT_THREADSAFE | FF_CODEC_CAP_SETS_PKT_DTS |
> FF_CODEC_CAP_SETS_FRAME_PROPS
> > +};
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://protect2.fireeye.com/v1/url?k=6cd54358-0d5e566e-6cd4c817-
> 74fe485cbff1-e18dc94cc35fe766&q=1&e=49c754eb-4416-4e31-90e9-
> ee8e7d469d1f&u=https%3A%2F%2Fffmpeg.org%2Fmailman%2Flistinfo%2Fffmp
> eg-devel
>
> To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org
> with subject "unsubscribe".
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
^ permalink raw reply [flat|nested] 5+ messages in thread