From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <ffmpeg-devel-bounces@ffmpeg.org>
Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100])
by master.gitmailbox.com (Postfix) with ESMTPS id E6F824C0A9
for <ffmpegdev@gitmailbox.com>; Tue, 1 Apr 2025 17:17:34 +0000 (UTC)
Received: from [127.0.1.1] (localhost [127.0.0.1])
by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 32F78687D8D;
Tue, 1 Apr 2025 20:16:47 +0300 (EEST)
Received: from mail-pl1-f181.google.com (mail-pl1-f181.google.com
[209.85.214.181])
by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 38D1A687D88
for <ffmpeg-devel@ffmpeg.org>; Tue, 1 Apr 2025 20:16:40 +0300 (EEST)
Received: by mail-pl1-f181.google.com with SMTP id
d9443c01a7336-2279915e06eso120031305ad.1
for <ffmpeg-devel@ffmpeg.org>; Tue, 01 Apr 2025 10:16:40 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=gmail.com; s=20230601; t=1743527798; x=1744132598; darn=ffmpeg.org;
h=content-transfer-encoding:mime-version:references:in-reply-to
:message-id:date:subject:cc:to:from:from:to:cc:subject:date
:message-id:reply-to;
bh=Ts29mMZFgGS/B/mFZur9LBlCp9f03NR95fihb/YnB6o=;
b=HC+QenpF5mjsSpsE1a4jjs7Z9Vt8+5JtBXRKgU8e/OmkkHMlyMIh7KgZ8uvpnmrpun
TaAm9oWMqHy9FTwaTM5e/dLZgB/fFtU735HrnJ+O+ms66wvd8w2cVC2NUqkohuxLTdEW
mxXFwtLwJMgQT/zrMaaVViRB1PMbNZz5bHpmZ1npZ8oJ/4cb0fu6HfVvLXALZL2B+K1u
XZVGAOQEiBhiLQJnLt+g+8zkRKVkUICCIylOCyuixANc2QXmHvL/DtoE2Z8CVAW84PzP
dsNWDh1T12OBMFh6MvkyZv+FPSrYLn6DaAUxvCtR2D5NxtmifgHfb9E6uvKl/ofiCkFV
Itew==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20230601; t=1743527798; x=1744132598;
h=content-transfer-encoding:mime-version:references:in-reply-to
:message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
:subject:date:message-id:reply-to;
bh=Ts29mMZFgGS/B/mFZur9LBlCp9f03NR95fihb/YnB6o=;
b=M/iR9hNZf83jLHbfcI7hq4dSWWc1Kv0IzFMsxgg19JzYZirKEeYNAwqIv+aC1CUD0x
dZ1XAM84636i6MQ1nnn/MGuiw7ezLvk47P4ARKmsTo17W+NFJbT5YhD2r1NP200mTOJJ
146Dg4irEnia26/ICHvY5wPiWkhMc0M3TNqiD9/z50AH23Ui69QscZM7QYi7nEWfYqWV
xdO464b8Ti30ww4BByr5zgKPzACFnEHoafQ6brCZ8UUMIfdYeMIu0Xe/1oKUJgAV2oSB
3wC+qlVnB1t0CYaBtZxs/rjuSmB9i95yUh8G7JS83MgKoxinGD5BuGViX6XBbpm+aGij
9ung==
X-Gm-Message-State: AOJu0YxADd8KqkIZQhFdws5Ku0dP/Aboig11btA/jRjaMUSr6YmW1QtU
9h+rDbZ8h+tme94YaKVMbEIHeaoYgWtBMqEu5czj9BXBW94Mm6Pn4ilvndaWMR0=
X-Gm-Gg: ASbGncs8NoujdGhD8kzv4oX3FatTnpqoVXm4iVVOTmPg3xKjwAUDsC7VBIqBspPNijW
yDb6YTrko14q5ipeVOkUQYS9uoBQHbC3BGadS+6fhxKbyE0fLWBl/axd+OD4D0JNyXNzBi6n5yz
RldND5vpTtqYpeKQ8jQoLzxd1mQUDrYwbw+wYDDFIO9+PA64Rm0lYIuE5sLYHtIEj6EfzUT/3xJ
M+n3LIsPDU8ziXy7W0N9453u1x7tCuyEJtHyzEd7e3Q/yRdyer7kiJpkF5tiTu75JFvTA9PIt8F
x52yoA1GLjW5mzZqCp43OExTwckcIzJ5mEEhDkI4h/NAapuox9jo+SDqeUiOtS0ZHg==
X-Google-Smtp-Source: AGHT+IEVNVOqXKyRqU12KhMlsS4HEB/Z+bQgR/m9Pk94LGEk0s596aDXL+MPKE1twDTsEACN2AQuhA==
X-Received: by 2002:a17:902:f682:b0:223:37ec:63d3 with SMTP id
d9443c01a7336-2292f95d75bmr184380665ad.18.1743527798283;
Tue, 01 Apr 2025 10:16:38 -0700 (PDT)
Received: from localhost.localdomain ([124.79.129.75])
by smtp.gmail.com with ESMTPSA id
d9443c01a7336-2291eee0bafsm90902495ad.90.2025.04.01.10.16.36
(version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
Tue, 01 Apr 2025 10:16:37 -0700 (PDT)
From: toqsxw@gmail.com
X-Google-Original-From: toqsxw@outlook.com
To: ffmpeg-devel@ffmpeg.org
Date: Wed, 2 Apr 2025 01:16:02 +0800
Message-ID: <20250401171616.1378-5-toqsxw@outlook.com>
X-Mailer: git-send-email 2.44.0.windows.1
In-Reply-To: <20250401171616.1378-1-toqsxw@outlook.com>
References: <20250401171616.1378-1-toqsxw@outlook.com>
MIME-Version: 1.0
Subject: [FFmpeg-devel] [PATCH v1 05/19] avcodec/vvc/dec: support applying
film grain by the decoder
X-BeenThere: ffmpeg-devel@ffmpeg.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: FFmpeg development discussions and patches <ffmpeg-devel.ffmpeg.org>
List-Unsubscribe: <https://ffmpeg.org/mailman/options/ffmpeg-devel>,
<mailto:ffmpeg-devel-request@ffmpeg.org?subject=unsubscribe>
List-Archive: <https://ffmpeg.org/pipermail/ffmpeg-devel>
List-Post: <mailto:ffmpeg-devel@ffmpeg.org>
List-Help: <mailto:ffmpeg-devel-request@ffmpeg.org?subject=help>
List-Subscribe: <https://ffmpeg.org/mailman/listinfo/ffmpeg-devel>,
<mailto:ffmpeg-devel-request@ffmpeg.org?subject=subscribe>
Reply-To: FFmpeg development discussions and patches <ffmpeg-devel@ffmpeg.org>
Cc: Wu Jianhua <toqsxw@outlook.com>
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Errors-To: ffmpeg-devel-bounces@ffmpeg.org
Sender: "ffmpeg-devel" <ffmpeg-devel-bounces@ffmpeg.org>
Archived-At: <https://master.gitmailbox.com/ffmpegdev/20250401171616.1378-5-toqsxw@outlook.com/>
List-Archive: <https://master.gitmailbox.com/ffmpegdev/>
List-Post: <mailto:ffmpegdev@gitmailbox.com>
From: Wu Jianhua <toqsxw@outlook.com>
Signed-off-by: Wu Jianhua <toqsxw@outlook.com>
---
libavcodec/Makefile | 2 +-
libavcodec/vvc/dec.c | 89 +++++++++++++++++++++++++++++++++++++++++--
libavcodec/vvc/dec.h | 8 +++-
libavcodec/vvc/refs.c | 11 +++++-
4 files changed, 104 insertions(+), 6 deletions(-)
diff --git a/libavcodec/Makefile b/libavcodec/Makefile
index 7467c883d3..e2d324202d 100644
--- a/libavcodec/Makefile
+++ b/libavcodec/Makefile
@@ -809,7 +809,7 @@ OBJS-$(CONFIG_VP9_V4L2M2M_DECODER) += v4l2_m2m_dec.o
OBJS-$(CONFIG_VQA_DECODER) += vqavideo.o
OBJS-$(CONFIG_VQC_DECODER) += vqcdec.o
OBJS-$(CONFIG_VVC_DECODER) += executor.o h2645data.o
-OBJS-$(CONFIG_VVC_SEI) += h2645_sei.o aom_film_grain.o
+OBJS-$(CONFIG_VVC_SEI) += h2645_sei.o aom_film_grain.o h274.o
OBJS-$(CONFIG_WADY_DPCM_DECODER) += dpcm.o
OBJS-$(CONFIG_WAVARC_DECODER) += wavarc.o
OBJS-$(CONFIG_WAVPACK_DECODER) += wavpack.o wavpackdata.o dsd.o
diff --git a/libavcodec/vvc/dec.c b/libavcodec/vvc/dec.c
index 8b1c2c751b..4c0d08440a 100644
--- a/libavcodec/vvc/dec.c
+++ b/libavcodec/vvc/dec.c
@@ -26,9 +26,12 @@
#include "libavcodec/hwconfig.h"
#include "libavcodec/profiles.h"
#include "libavutil/refstruct.h"
+#include "libavcodec/aom_film_grain.h"
+#include "libavcodec/thread.h"
#include "libavutil/cpu.h"
#include "libavutil/mem.h"
#include "libavutil/thread.h"
+#include "libavutil/film_grain_params.h"
#include "dec.h"
#include "ctu.h"
@@ -601,6 +604,14 @@ static int ref_frame(VVCFrame *dst, const VVCFrame *src)
av_refstruct_replace(&dst->sps, src->sps);
av_refstruct_replace(&dst->pps, src->pps);
+ if (src->needs_fg) {
+ ret = av_frame_ref(dst->frame_grain, src->frame_grain);
+ if (ret < 0)
+ return ret;
+
+ dst->needs_fg = src->needs_fg;
+ }
+
av_refstruct_replace(&dst->progress, src->progress);
av_refstruct_replace(&dst->tab_dmvr_mvf, src->tab_dmvr_mvf);
@@ -634,6 +645,7 @@ static av_cold void frame_context_free(VVCFrameContext *fc)
for (int i = 0; i < FF_ARRAY_ELEMS(fc->DPB); i++) {
ff_vvc_unref_frame(fc, &fc->DPB[i], ~0);
av_frame_free(&fc->DPB[i].frame);
+ av_frame_free(&fc->DPB[i].frame_grain);
}
ff_vvc_frame_thread_free(fc);
@@ -656,6 +668,10 @@ static av_cold int frame_context_init(VVCFrameContext *fc, AVCodecContext *avctx
fc->DPB[j].frame = av_frame_alloc();
if (!fc->DPB[j].frame)
return AVERROR(ENOMEM);
+
+ fc->DPB[j].frame_grain = av_frame_alloc();
+ if (!fc->DPB[j].frame_grain)
+ return AVERROR(ENOMEM);
}
fc->cu_pool = av_refstruct_pool_alloc(sizeof(CodingUnit), 0);
if (!fc->cu_pool)
@@ -726,6 +742,41 @@ static int set_side_data(VVCContext *s, VVCFrameContext *fc)
NULL, fc->ps.sps->bit_depth, fc->ps.sps->bit_depth, fc->ref->poc);
}
+static int check_film_grain(VVCContext *s, VVCFrameContext *fc)
+{
+ int ret;
+
+ fc->ref->needs_fg = (fc->sei.common.film_grain_characteristics &&
+ fc->sei.common.film_grain_characteristics->present ||
+ fc->sei.common.aom_film_grain.enable) &&
+ !(s->avctx->export_side_data & AV_CODEC_EXPORT_DATA_FILM_GRAIN) &&
+ !s->avctx->hwaccel;
+
+ if (fc->ref->needs_fg &&
+ (fc->sei.common.film_grain_characteristics->present &&
+ !ff_h274_film_grain_params_supported(fc->sei.common.film_grain_characteristics->model_id,
+ fc->ref->frame->format) ||
+ !av_film_grain_params_select(fc->ref->frame))) {
+ av_log_once(s->avctx, AV_LOG_WARNING, AV_LOG_DEBUG, &s->film_grain_warning_shown,
+ "Unsupported film grain parameters. Ignoring film grain.\n");
+ fc->ref->needs_fg = 0;
+ }
+
+ if (fc->ref->needs_fg) {
+ fc->ref->frame_grain->format = fc->ref->frame->format;
+ fc->ref->frame_grain->width = fc->ref->frame->width;
+ fc->ref->frame_grain->height = fc->ref->frame->height;
+
+ ret = ff_thread_get_buffer(s->avctx, fc->ref->frame_grain, 0);
+ if (ret < 0)
+ return ret;
+
+ return av_frame_copy_props(fc->ref->frame_grain, fc->ref->frame);
+ }
+
+ return 0;
+}
+
static int frame_start(VVCContext *s, VVCFrameContext *fc, SliceContext *sc)
{
const VVCPH *ph = &fc->ps.ph;
@@ -745,6 +796,10 @@ static int frame_start(VVCContext *s, VVCFrameContext *fc, SliceContext *sc)
if (ret < 0)
goto fail;
+ ret = check_film_grain(s, fc);
+ if (ret < 0)
+ goto fail;
+
if (!IS_IDR(s))
ff_vvc_bump_frame(s, fc);
@@ -1002,14 +1057,42 @@ fail:
return ret;
}
+static int frame_end(VVCContext *s, VVCFrameContext *fc)
+{
+ const AVFilmGrainParams *fgp;
+ int ret = 0;
+
+ if (fc->ref->needs_fg) {
+ av_assert0(fc->ref->frame_grain->buf[0]);
+ fgp = av_film_grain_params_select(fc->ref->frame);
+ switch (fgp->type) {
+ case AV_FILM_GRAIN_PARAMS_NONE:
+ av_assert0(0);
+ return AVERROR_BUG;
+ case AV_FILM_GRAIN_PARAMS_H274:
+ ret = ff_h274_apply_film_grain(fc->ref->frame_grain, fc->ref->frame,
+ &s->h274db, fgp);
+ break;
+ case AV_FILM_GRAIN_PARAMS_AV1:
+ ret = ff_aom_apply_film_grain(fc->ref->frame_grain, fc->ref->frame, fgp);
+ break;
+ }
+ }
+
+ return ret;
+}
+
static int wait_delayed_frame(VVCContext *s, AVFrame *output, int *got_output)
{
VVCFrameContext *delayed = get_frame_context(s, s->fcs, s->nb_frames - s->nb_delayed);
int ret = ff_vvc_frame_wait(s, delayed);
- if (!ret && delayed->output_frame->buf[0] && output) {
- av_frame_move_ref(output, delayed->output_frame);
- *got_output = 1;
+ if (!ret) {
+ ret = frame_end(s, delayed);
+ if (ret >= 0 && delayed->output_frame->buf[0] && output) {
+ av_frame_move_ref(output, delayed->output_frame);
+ *got_output = 1;
+ }
}
s->nb_delayed--;
diff --git a/libavcodec/vvc/dec.h b/libavcodec/vvc/dec.h
index df81a83489..b159863b31 100644
--- a/libavcodec/vvc/dec.h
+++ b/libavcodec/vvc/dec.h
@@ -26,6 +26,7 @@
#include "libavcodec/videodsp.h"
#include "libavcodec/vvc.h"
+#include "libavcodec/h274.h"
#include "ps.h"
#include "dsp.h"
@@ -71,12 +72,15 @@ typedef struct VVCWindow {
typedef struct VVCFrame {
struct AVFrame *frame;
-
+ struct AVFrame *frame_grain;
const VVCSPS *sps; ///< RefStruct reference
const VVCPPS *pps; ///< RefStruct reference
struct MvField *tab_dmvr_mvf; ///< RefStruct reference
RefPicListTab **rpl_tab; ///< RefStruct reference
RefPicListTab *rpl; ///< RefStruct reference
+
+ int needs_fg; ///< 1 if grain needs to be applied by the decoder
+
int nb_rpl_elems;
int ctb_count;
@@ -218,6 +222,7 @@ typedef struct VVCContext {
CodedBitstreamFragment current_frame;
VVCParamSets ps;
+ H274FilmGrainDatabase h274db;
int temporal_id; ///< temporal_id_plus1 - 1
int poc_tid0;
@@ -228,6 +233,7 @@ typedef struct VVCContext {
enum VVCNALUnitType vcl_unit_type;
int no_output_before_recovery_flag; ///< NoOutputBeforeRecoveryFlag
int gdr_recovery_point_poc; ///< recoveryPointPocVal
+ int film_grain_warning_shown;
/**
* Sequence counters for decoded and output frames, so that old
diff --git a/libavcodec/vvc/refs.c b/libavcodec/vvc/refs.c
index 1cfca48204..4cf44bcf46 100644
--- a/libavcodec/vvc/refs.c
+++ b/libavcodec/vvc/refs.c
@@ -52,6 +52,12 @@ void ff_vvc_unref_frame(VVCFrameContext *fc, VVCFrame *frame, int flags)
frame->flags = 0;
if (!frame->flags) {
av_frame_unref(frame->frame);
+
+ if (frame->needs_fg) {
+ av_frame_unref(frame->frame_grain);
+ frame->needs_fg = 0;
+ }
+
av_refstruct_unref(&frame->sps);
av_refstruct_unref(&frame->pps);
av_refstruct_unref(&frame->progress);
@@ -285,7 +291,10 @@ int ff_vvc_output_frame(VVCContext *s, VVCFrameContext *fc, AVFrame *out, const
if (frame->flags & VVC_FRAME_FLAG_CORRUPT)
frame->frame->flags |= AV_FRAME_FLAG_CORRUPT;
- ret = av_frame_ref(out, frame->frame);
+ ret = av_frame_ref(out, frame->needs_fg ? frame->frame_grain : frame->frame);
+ if (ret < 0)
+ return ret;
+
if (frame->flags & VVC_FRAME_FLAG_BUMPING)
ff_vvc_unref_frame(fc, frame, VVC_FRAME_FLAG_OUTPUT | VVC_FRAME_FLAG_BUMPING);
else
--
2.44.0.windows.1
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".