From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTP id 2FE7743EA4 for ; Tue, 18 Oct 2022 12:11:46 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 7CF2E68BCF7; Tue, 18 Oct 2022 15:11:44 +0300 (EEST) Received: from mail-ot1-f49.google.com (mail-ot1-f49.google.com [209.85.210.49]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 826C568BC99 for ; Tue, 18 Oct 2022 15:11:37 +0300 (EEST) Received: by mail-ot1-f49.google.com with SMTP id cb2-20020a056830618200b00661b6e5dcd8so7488633otb.8 for ; Tue, 18 Oct 2022 05:11:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:in-reply-to:from:content-language :references:to:subject:user-agent:mime-version:date:message-id:from :to:cc:subject:date:message-id:reply-to; bh=XJuoff1a1iv3WbLHWWRvWAb0sYrXxC5UC30oh6dwbzY=; b=p9avJPol5b/M+sAqa1GOi7rk5r9q+TdGae7fqwwMVwVgdg9zHmMzi32urbk1WDoMY7 xUALXygtu8ZSxn+E98jI5UW06I9PYW8eO/x4vzaISt7qaraqX7l/jgFM64EeNhD31zoc xWmpG/Y5fJgmi0M1W1YH/0OkgsiDqzbFvfSU9R7hXP7/Cj7+g2+aBbG8UtEwvi31svrK Pe3lA0kwzKhhnlTvQdfr8a2/87vp4QcohmWFlygtJ06emenPCK9L6S0HPz0NmFXnb5t0 TwydwA/SaGXRIY1OFZqvHSgYWzKshLvLkx2qeC6g8DjcaC1cV0+/gKu4qTv19XvA0zDY n6Dw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:in-reply-to:from:content-language :references:to:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=XJuoff1a1iv3WbLHWWRvWAb0sYrXxC5UC30oh6dwbzY=; b=4jBQRja2eYMOwabzbEwZ5udAxMpbbvdgtA3IiQnbknUSG4hTc0f/7Ec/hQXQm6Q20O AYR4IOD4q5MjYzfJeW5+ZLSSRIuz994zoS5XCvocpajA8oGl1vlKGUFK+vaD9scbg609 mUxm98TtjOrkWjFT7QwE9yYvZQNK/dgbjZqrWaN3HNeESvQvCgljo/6xLpC7uvui81vO rL4NcakObGWAZgkDHEgaZIKGB3uU72nU5PB1+R1/mrPbKSuNtf8S4Y2itBMTMPMUEhZU AIkdUMfxDELADVdxG5Vm+aFNRve+ac5pX+uLby/+KDqh82Tx+VK80GcdFZAonRlNkQSt U09Q== X-Gm-Message-State: ACrzQf1EjkuIJxwqFaQHwMt/Upo3StDLjzTgIpHUHnG+pXTs/iM/xbjZ iT197gfu7Up7KeiB5HXwsyTfvwwscBs= X-Google-Smtp-Source: AMsMyM6JgYmVL+L2/9usGCH8CUxOFGAiFwwQgdlRF2F6Dgp1uJoxOKGOmle2f6pG9knNdXYgYJcRLQ== X-Received: by 2002:a05:6830:1343:b0:661:9598:6786 with SMTP id r3-20020a056830134300b0066195986786mr1122849otq.354.1666095095554; Tue, 18 Oct 2022 05:11:35 -0700 (PDT) Received: from [192.168.0.13] ([191.97.187.183]) by smtp.gmail.com with ESMTPSA id v27-20020a05683018db00b00661a80d555fsm5887778ote.40.2022.10.18.05.11.33 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 18 Oct 2022 05:11:34 -0700 (PDT) Message-ID: <9eb483c2-4802-f013-0fcf-b2b073c9afe3@gmail.com> Date: Tue, 18 Oct 2022 09:11:39 -0300 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Thunderbird/102.3.3 To: ffmpeg-devel@ffmpeg.org References: <20221018115159.17780-1-anton@khirnov.net> <20221018115159.17780-3-anton@khirnov.net> Content-Language: en-US From: James Almer In-Reply-To: <20221018115159.17780-3-anton@khirnov.net> Subject: Re: [FFmpeg-devel] [PATCH 3/3] tools: add an AV_CODEC_CAP_ENCODER_RECON_FRAME test tool X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: On 10/18/2022 8:51 AM, Anton Khirnov wrote: > --- > Makefile | 2 + > tools/Makefile | 3 +- > tools/enc_recon_frame_test.c | 388 +++++++++++++++++++++++++++++++++++ > 3 files changed, 392 insertions(+), 1 deletion(-) > create mode 100644 tools/enc_recon_frame_test.c > > diff --git a/Makefile b/Makefile > index 1fb742f390..bf1b69f96b 100644 > --- a/Makefile > +++ b/Makefile > @@ -67,6 +67,8 @@ tools/target_io_dem_fuzzer$(EXESUF): tools/target_io_dem_fuzzer.o $(FF_DEP_LIBS) > > tools/enum_options$(EXESUF): ELIBS = $(FF_EXTRALIBS) > tools/enum_options$(EXESUF): $(FF_DEP_LIBS) > +tools/enc_recon_frame_test$(EXESUF): $(FF_DEP_LIBS) > +tools/enc_recon_frame_test$(EXESUF): ELIBS = $(FF_EXTRALIBS) > tools/scale_slice_test$(EXESUF): $(FF_DEP_LIBS) > tools/scale_slice_test$(EXESUF): ELIBS = $(FF_EXTRALIBS) > tools/sofa2wavs$(EXESUF): ELIBS = $(FF_EXTRALIBS) > diff --git a/tools/Makefile b/tools/Makefile > index 4afa23342d..dee6a41668 100644 > --- a/tools/Makefile > +++ b/tools/Makefile > @@ -1,4 +1,4 @@ > -TOOLS = enum_options qt-faststart scale_slice_test trasher uncoded_frame > +TOOLS = enc_recon_frame_test enum_options qt-faststart scale_slice_test trasher uncoded_frame > TOOLS-$(CONFIG_LIBMYSOFA) += sofa2wavs > TOOLS-$(CONFIG_ZLIB) += cws2fws > > @@ -17,6 +17,7 @@ tools/target_dem_fuzzer.o: tools/target_dem_fuzzer.c > tools/target_io_dem_fuzzer.o: tools/target_dem_fuzzer.c > $(COMPILE_C) -DIO_FLAT=0 > > +tools/enc_recon_frame_test$(EXESUF): tools/decode_simple.o > tools/venc_data_dump$(EXESUF): tools/decode_simple.o > tools/scale_slice_test$(EXESUF): tools/decode_simple.o > > diff --git a/tools/enc_recon_frame_test.c b/tools/enc_recon_frame_test.c > new file mode 100644 > index 0000000000..0d3b8e88e3 > --- /dev/null > +++ b/tools/enc_recon_frame_test.c > @@ -0,0 +1,388 @@ > +/* > + * copyright (c) 2022 Anton Khirnov > + * > + * This file is part of FFmpeg. > + * > + * FFmpeg is free software; you can redistribute it and/or > + * modify it under the terms of the GNU Lesser General Public > + * License as published by the Free Software Foundation; either > + * version 2.1 of the License, or (at your option) any later version. > + * > + * FFmpeg is distributed in the hope that it will be useful, > + * but WITHOUT ANY WARRANTY; without even the implied warranty of > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > + * Lesser General Public License for more details. > + * > + * You should have received a copy of the GNU Lesser General Public > + * License along with FFmpeg; if not, write to the Free Software > + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA > + */ > + > +/* A test for AV_CODEC_FLAG_RECON_FRAME > + * TODO: dump reconstructed frames to disk */ > + > +#include > +#include > +#include > + > +#include "decode_simple.h" > + > +#include "libavutil/adler32.h" > +#include "libavutil/common.h" > +#include "libavutil/error.h" > +#include "libavutil/frame.h" > +#include "libavutil/imgutils.h" > +#include "libavutil/opt.h" > + > +#include "libavformat/avformat.h" > + > +#include "libavcodec/avcodec.h" > +#include "libavcodec/codec.h" > + > +#include "libswscale/swscale.h" > + > +typedef struct FrameChecksum { > + int64_t ts; > + uint32_t checksum[4]; > +} FrameChecksum; > + > +typedef struct PrivData { > + AVCodecContext *enc; > + AVCodecContext *dec; > + > + AVPacket *pkt; > + AVFrame *frame, *frame_recon; > + > + struct SwsContext *scaler; > + > + FrameChecksum *checksums_decoded; > + size_t nb_checksums_decoded; > + FrameChecksum *checksums_recon; > + size_t nb_checksums_recon; > + > + int dump; > +} PrivData; > + > +static int frame_hash(FrameChecksum **pc, size_t *nb_c, int64_t ts, > + const AVFrame *frame) > +{ > + FrameChecksum *c; > + int shift_h, shift_v; > + > + c = av_realloc_array(*pc, *nb_c + 1, sizeof(*c)); > + if (!c) > + return AVERROR(ENOMEM); > + *pc = c; > + (*nb_c)++; > + > + c += *nb_c - 1; > + memset(c, 0, sizeof(*c)); > + > + av_pix_fmt_get_chroma_sub_sample(frame->format, &shift_h, &shift_v); > + > + c->ts = ts; > + for (int p = 0; frame->data[p]; p++) { > + const uint8_t *data = frame->data[p]; > + int linesize = av_image_get_linesize(frame->format, frame->width, p); > + uint32_t checksum = 0; > + > + for (int j = 0; j < frame->height >> shift_v; j++) { > + checksum = av_adler32_update(checksum, data, linesize); > + data += frame->linesize[p]; > + } > + > + c->checksum[p] = checksum; > + } > + > + return 0; > +} > + > +static int recon_frame_process(PrivData *pd, const AVPacket *pkt) > +{ > + AVFrame *f = pd->frame_recon; > + int ret; > + > + ret = avcodec_receive_frame(pd->enc, f); > + if (ret < 0) { > + fprintf(stderr, "Error retrieving a reconstructed frame\n"); > + return ret; > + } > + > + if (pd->dump) { > + } > + > + // the encoder's internal format (in which the reconsturcted frames are > + // exported) may be different from the user-facing pixel format > + if (f->format != pd->enc->pix_fmt) { > + if (!pd->scaler) { > + pd->scaler = sws_getContext(f->width, f->height, f->format, > + f->width, f->height, pd->enc->pix_fmt, > + SWS_BITEXACT, NULL, NULL, NULL); > + if (!pd->scaler) > + return AVERROR(ENOMEM); > + } > + > + ret = sws_scale_frame(pd->scaler, pd->frame, f); > + if (ret < 0) { > + fprintf(stderr, "Error converting pixel formats\n"); > + return ret; > + } > + > + av_frame_unref(f); > + f = pd->frame; > + } > + > + ret = frame_hash(&pd->checksums_recon, &pd->nb_checksums_recon, > + pkt->pts, f); > + av_frame_unref(f); > + > + return 0; > +} > + > +static int process_frame(DecodeContext *dc, AVFrame *frame) > +{ > + PrivData *pd = dc->opaque; > + int ret; > + > + if (!avcodec_is_open(pd->enc)) { > + if (!frame) { > + fprintf(stderr, "No frames were decoded\n"); > + return AVERROR_INVALIDDATA; > + } > + > + pd->enc->width = frame->width; > + pd->enc->height = frame->height; > + pd->enc->pix_fmt = frame->format; > + pd->enc->time_base = dc->stream->time_base; > + pd->enc->thread_count = dc->decoder->thread_count; > + pd->enc->thread_type = dc->decoder->thread_type; > + > + ret = avcodec_open2(pd->enc, NULL, NULL); > + if (ret < 0) { > + fprintf(stderr, "Error opening the encoder\n"); > + return ret; > + } > + } > + > + if (frame) > + frame->pict_type = AV_PICTURE_TYPE_NONE; > + > + ret = avcodec_send_frame(pd->enc, frame); > + if (ret < 0) { > + fprintf(stderr, "Error submitting a frame for encoding\n"); > + return ret; > + } > + > + while (1) { > + AVPacket *pkt = pd->pkt; > + > + ret = avcodec_receive_packet(pd->enc, pkt); > + if (ret == AVERROR(EAGAIN)) > + break; > + else if (ret == AVERROR_EOF) > + pkt = NULL; > + else if (ret < 0 && ret != AVERROR_EOF) { > + fprintf(stderr, "Error receiving a frame from the encoder\n"); > + return ret; > + } > + > + if (pkt) { > + ret = recon_frame_process(pd, pkt); > + if (ret < 0) > + return ret; > + } > + > + if (!avcodec_is_open(pd->dec)) { > + if (!pkt) { > + fprintf(stderr, "No packets were received from the encoder\n"); > + return AVERROR(EINVAL); > + } > + > + pd->dec->width = pd->enc->width; > + pd->dec->height = pd->enc->height; > + pd->dec->pix_fmt = pd->enc->pix_fmt; > + pd->dec->thread_count = dc->decoder->thread_count; > + pd->dec->thread_type = dc->decoder->thread_type; > + if (pd->enc->extradata_size) { > + pd->dec->extradata = av_memdup(pd->enc->extradata, > + pd->enc->extradata_size + AV_INPUT_BUFFER_PADDING_SIZE); > + if (!pd->dec->extradata) > + return AVERROR(ENOMEM); > + } > + > + ret = avcodec_open2(pd->dec, NULL, NULL); > + if (ret < 0) { > + fprintf(stderr, "Error opening the decoder\n"); > + return ret; > + } > + } > + > + ret = avcodec_send_packet(pd->dec, pkt); > + if (ret < 0) { > + fprintf(stderr, "Error sending a frame to decoder\n"); > + return ret; > + } > + > + while (1) { > + ret = avcodec_receive_frame(pd->dec, pd->frame); > + if (ret == AVERROR(EAGAIN)) > + break; > + else if (ret == AVERROR_EOF) > + return 0; > + else if (ret < 0) { > + fprintf(stderr, "Error receving a frame from decoder\n"); > + return ret; > + } > + > + ret = frame_hash(&pd->checksums_decoded, &pd->nb_checksums_decoded, > + pd->frame->pts, pd->frame); > + av_frame_unref(pd->frame); > + if (ret < 0) > + return ret; > + } > + > + } > + > + return 0; > +} > + > +static int frame_checksum_compare(const void *a, const void *b) > +{ > + const FrameChecksum *ca = a; > + const FrameChecksum *cb = b; > + if (ca->ts == cb->ts) > + return 0; > + return FFSIGN(ca->ts - cb->ts); > +} > + > +int main(int argc, char **argv) > +{ > + PrivData pd; > + DecodeContext dc; > + > + const char *filename, *enc_name, *enc_opts, *thread_type = NULL, *nb_threads = NULL; > + const AVCodec *enc, *dec; > + int ret = 0, max_frames = 0, dump = 0; > + > + if (argc <= 4) { > + fprintf(stderr, > + "Usage: %s " > + "[ ]\n", argv[0]); > + return 0; > + } > + > + filename = argv[1]; > + enc_name = argv[2]; > + enc_opts = argv[3]; > + max_frames = strtol(argv[4], NULL, 0); Maybe this should be optional too. If i want to test encoding an entire sample, and i don't know its exact length, then this option is not really helpful. > + if (argc >= 6) > + nb_threads = argv[5]; > + if (argc >= 7) > + thread_type = argv[6]; > + if (argc >= 8) > + dump = strtol(argv[7], NULL, 0); Probably best to not add the option until it's implemented. Ignoring it silently is not nice. > + > + memset(&dc, 0, sizeof(dc)); > + memset(&pd, 0, sizeof(pd)); > + > + pd.dump = dump; > + > + enc = avcodec_find_encoder_by_name(enc_name); > + if (!enc) { > + fprintf(stderr, "No such encoder: %s\n", enc_name); > + return 1; > + } > + if (!(enc->capabilities & AV_CODEC_CAP_ENCODER_RECON_FRAME)) { > + fprintf(stderr, "Encoder '%s' cannot ouput reconstructed frames\n", > + enc->name); > + return 1; > + } > + > + dec = avcodec_find_decoder(enc->id); > + if (!dec) { > + fprintf(stderr, "No decoder for: %s\n", avcodec_get_name(enc->id)); > + return 1; > + } > + > + pd.enc = avcodec_alloc_context3(enc); > + if (!pd.enc) { > + fprintf(stderr, "Error allocating encoder\n"); > + return 1; > + } > + > + ret = av_set_options_string(pd.enc, enc_opts, "=", ","); > + if (ret < 0) { > + fprintf(stderr, "Error setting encoder options\n"); > + goto fail; > + } > + pd.enc->flags |= AV_CODEC_FLAG_RECON_FRAME | AV_CODEC_FLAG_BITEXACT; > + > + pd.dec = avcodec_alloc_context3(dec); > + if (!pd.dec) { > + fprintf(stderr, "Error allocating decoder\n"); > + goto fail; > + } > + > + pd.dec->flags |= AV_CODEC_FLAG_BITEXACT; nit: Maybe enable AV_EF_CRCCHECK in err_recognition for extra checks in the decoder (It's not too important since no decoder for a codec with a currently supported recon encoder uses it right now). > + > + pd.frame = av_frame_alloc(); > + pd.frame_recon = av_frame_alloc(); > + pd.pkt = av_packet_alloc(); > + if (!pd.frame ||!pd.frame_recon || !pd.pkt) { > + ret = 1; > + goto fail; > + } > + > + ret = ds_open(&dc, filename, 0); > + if (ret < 0) { > + fprintf(stderr, "Error opening the file\n"); > + goto fail; > + } > + > + dc.process_frame = process_frame; > + dc.opaque = &pd; > + dc.max_frames = max_frames; > + > + ret = av_dict_set(&dc.decoder_opts, "threads", nb_threads, 0); > + ret |= av_dict_set(&dc.decoder_opts, "thread_type", thread_type, 0); > + > + ret = ds_run(&dc); > + if (ret < 0) > + goto fail; > + > + if (pd.nb_checksums_decoded != pd.nb_checksums_recon) { > + fprintf(stderr, "Mismatching frame counts: recon=%zu decoded=%zu\n", > + pd.nb_checksums_recon, pd.nb_checksums_decoded); > + ret = 1; > + goto fail; > + } > + > + // reconstructed frames are in coded order, sort them by pts into presentation order > + qsort(pd.checksums_recon, pd.nb_checksums_recon, sizeof(*pd.checksums_recon), > + frame_checksum_compare); > + > + for (size_t i = 0; i < pd.nb_checksums_decoded; i++) { > + const FrameChecksum *d = &pd.checksums_decoded[i]; > + const FrameChecksum *r = &pd.checksums_recon[i]; > + > + for (int p = 0; p < FF_ARRAY_ELEMS(d->checksum); p++) > + if (d->checksum[p] != r->checksum[p]) { > + fprintf(stderr, "Checksum mismatch in frame ts=%"PRId64", plane %d\n", > + d->ts, p); > + ret = 1; > + goto fail; > + } > + } > + fprintf(stderr, "All %zu encoded frames match\n", pd.nb_checksums_decoded); > + > +fail: > + avcodec_free_context(&pd.enc); > + avcodec_free_context(&pd.dec); > + av_freep(&pd.checksums_decoded); > + av_freep(&pd.checksums_recon); > + av_frame_free(&pd.frame); > + av_frame_free(&pd.frame_recon); > + av_packet_free(&pd.pkt); > + ds_free(&dc); > + return !!ret; > +} Tested and it seems to work, so LGTM. _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".