[FFmpeg-devel] [PATCH v4 00/13] AFGS1 film grain support

[FFmpeg-devel] [PATCH v4 00/13] AFGS1 film grain support

[Niklas Haas]

Changes since v3:

- Moved metadata into common AVFilmGrainParams struct, as per James
  Almer's recommendation
- Fixed width/height field validation
- Signal this metadata for AV1 (instead of setting it to unspecified)
- Implement ffprobe/showinfo support for the new fields

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH v4 01/13] avutil/film_grain_params: add metadata to common struct

[Niklas Haas]

From: Niklas Haas <git@haasn.dev>

This is needed for AV1 film grain as well, when using AFGS1 streams.
Also add extra width/height and subsampling information, which AFGS1
cares about, as part of the same API bump. (And in principle, H274
should also expose this information, since it is needed downstream to
correctly adjust the chroma grain frequency to the subsampling ratio)

Deprecate the equivalent H274-exclusive fields. To avoid breaking ABI,
add the new fields after the union; but with enough of a paper trail to
hopefully re-order them on the next bump.
---
 doc/APIchanges                |  6 ++++
 libavutil/film_grain_params.h | 57 +++++++++++++++++++++++++++++++++--
 libavutil/version.h           |  3 +-
 3 files changed, 62 insertions(+), 4 deletions(-)

diff --git a/doc/APIchanges b/doc/APIchanges
index a44c8e4f108..256d9c7757a 100644
--- a/doc/APIchanges
+++ b/doc/APIchanges
@@ -2,6 +2,12 @@ The last version increases of all libraries were on 2024-03-07
 
 API changes, most recent first:
 
+2024-03-xx - xxxxxxxxxx - lavu 59.3.100 - film_grain_params.h
+  Add AVFilmGrainParams.color_range, color_primaries, color_trc, color_space,
+  width, height, subsampling_x, subsampling_y, bit_depth_luma and
+  bit_depth_chroma. Deprecate the corresponding fields from
+  AVFilmGrainH274Params.
+
 2024-03-xx - xxxxxxxxxx - lavu 59.2.100 - channel_layout.h
   Add AV_CHANNEL_LAYOUT_RETYPE_FLAG_CANONICAL.
 
diff --git a/libavutil/film_grain_params.h b/libavutil/film_grain_params.h
index f3bd0a4a6a3..a9f243351c9 100644
--- a/libavutil/film_grain_params.h
+++ b/libavutil/film_grain_params.h
@@ -136,20 +136,42 @@ typedef struct AVFilmGrainH274Params {
      */
     int model_id;
 
-    /**
-     * Specifies the bit depth used for the luma component.
-     */
+#if FF_API_H274_FILM_GRAIN_VCS
+  /**
+   * TODO: On this ABI bump, please also re-order the fields in
+   * AVFilmGrainParams (see below)
+   */
+
+  /**
+   * Specifies the bit depth used for the luma component.
+   *
+   * @deprecated use AVFilmGrainParams.bit_depth_luma.
+   */
+    attribute_deprecated
     int bit_depth_luma;
 
     /**
      * Specifies the bit depth used for the chroma components.
+     *
+     * @deprecated use AVFilmGrainParams.bit_depth_chroma.
      */
+    attribute_deprecated
     int bit_depth_chroma;
 
+    /**
+     * Specifies the video signal characteristics.
+     *
+     * @deprecated use AVFilmGrainParams.color_{range,primaries,trc,space}.
+     */
+    attribute_deprecated
     enum AVColorRange                  color_range;
+    attribute_deprecated
     enum AVColorPrimaries              color_primaries;
+    attribute_deprecated
     enum AVColorTransferCharacteristic color_trc;
+    attribute_deprecated
     enum AVColorSpace                  color_space;
+#endif
 
     /**
      * Specifies the blending mode used to blend the simulated film grain
@@ -231,11 +253,40 @@ typedef struct AVFilmGrainParams {
      * Additional fields may be added both here and in any structure included.
      * If a codec's film grain structure differs slightly over another
      * codec's, fields within may change meaning depending on the type.
+     *
+     * TODO: Move this to the end of the structure, at the next ABI bump.
      */
     union {
         AVFilmGrainAOMParams aom;
         AVFilmGrainH274Params h274;
     } codec;
+
+    /**
+     * Intended display resolution. May be 0 if the codec does not specify
+     * any restrictions.
+     */
+
+    int width, height;
+
+    /**
+     * Intended subsampling ratio, or 0 for luma-only streams.
+     */
+    int subsampling_x, subsampling_y;
+
+    /**
+     * Intended video signal characteristics.
+     */
+    enum AVColorRange                  color_range;
+    enum AVColorPrimaries              color_primaries;
+    enum AVColorTransferCharacteristic color_trc;
+    enum AVColorSpace                  color_space;
+
+    /**
+     * Intended bit depth, or 0 for unknown/unspecified.
+     */
+    int bit_depth_luma;
+    int bit_depth_chroma;
+
 } AVFilmGrainParams;
 
 /**
diff --git a/libavutil/version.h b/libavutil/version.h
index 57cad02ec0a..23351316b58 100644
--- a/libavutil/version.h
+++ b/libavutil/version.h
@@ -79,7 +79,7 @@
  */
 
 #define LIBAVUTIL_VERSION_MAJOR  59
-#define LIBAVUTIL_VERSION_MINOR   2
+#define LIBAVUTIL_VERSION_MINOR   3
 #define LIBAVUTIL_VERSION_MICRO 100
 
 #define LIBAVUTIL_VERSION_INT   AV_VERSION_INT(LIBAVUTIL_VERSION_MAJOR, \
@@ -111,6 +111,7 @@
 #define FF_API_FRAME_KEY                (LIBAVUTIL_VERSION_MAJOR < 60)
 #define FF_API_PALETTE_HAS_CHANGED      (LIBAVUTIL_VERSION_MAJOR < 60)
 #define FF_API_VULKAN_CONTIGUOUS_MEMORY (LIBAVUTIL_VERSION_MAJOR < 60)
+#define FF_API_H274_FILM_GRAIN_VCS      (LIBAVUTIL_VERSION_MAJOR < 60)
 
 /**
  * @}
-- 
2.44.0

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH v4 02/13] avutil/film_grain_params: initialize VCS to UNSPECIFIED

[Niklas Haas]

From: Niklas Haas <git@haasn.dev>

---
 libavutil/film_grain_params.c | 11 +++++++++--
 1 file changed, 9 insertions(+), 2 deletions(-)

diff --git a/libavutil/film_grain_params.c b/libavutil/film_grain_params.c
index 930d23c7fe9..230ce8d701c 100644
--- a/libavutil/film_grain_params.c
+++ b/libavutil/film_grain_params.c
@@ -30,13 +30,20 @@ AVFilmGrainParams *av_film_grain_params_alloc(size_t *size)
 
 AVFilmGrainParams *av_film_grain_params_create_side_data(AVFrame *frame)
 {
+    AVFilmGrainParams *fgp;
     AVFrameSideData *side_data = av_frame_new_side_data(frame,
                                                         AV_FRAME_DATA_FILM_GRAIN_PARAMS,
                                                         sizeof(AVFilmGrainParams));
     if (!side_data)
         return NULL;
 
-    memset(side_data->data, 0, sizeof(AVFilmGrainParams));
+    fgp = (AVFilmGrainParams *) side_data->data;
+    *fgp = (AVFilmGrainParams) {
+        .color_range     = AVCOL_RANGE_UNSPECIFIED,
+        .color_primaries = AVCOL_PRI_UNSPECIFIED,
+        .color_trc       = AVCOL_TRC_UNSPECIFIED,
+        .color_space     = AVCOL_SPC_UNSPECIFIED,
+    };
 
-    return (AVFilmGrainParams *)side_data->data;
+    return fgp;
 }
-- 
2.44.0

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH v4 03/13] avfilter/vf_showinfo: adapt to new AVFilmGrainParams

[Niklas Haas]

From: Niklas Haas <git@haasn.dev>

---
 libavfilter/vf_showinfo.c | 26 +++++++++++++++-----------
 1 file changed, 15 insertions(+), 11 deletions(-)

diff --git a/libavfilter/vf_showinfo.c b/libavfilter/vf_showinfo.c
index 095cb22639f..28d8ea76e9b 100644
--- a/libavfilter/vf_showinfo.c
+++ b/libavfilter/vf_showinfo.c
@@ -452,6 +452,11 @@ static void dump_sei_film_grain_params_metadata(AVFilterContext *ctx, const AVFr
         [AV_FILM_GRAIN_PARAMS_H274] = "h274",
     };
 
+    const char *color_range_str     = av_color_range_name(fgp->color_range);
+    const char *color_primaries_str = av_color_primaries_name(fgp->color_primaries);
+    const char *color_trc_str       = av_color_transfer_name(fgp->color_trc);
+    const char *colorspace_str      = av_color_space_name(fgp->color_space);
+
     if (fgp->type >= FF_ARRAY_ELEMS(film_grain_type_names)) {
         av_log(ctx, AV_LOG_ERROR, "invalid data\n");
         return;
@@ -459,6 +464,16 @@ static void dump_sei_film_grain_params_metadata(AVFilterContext *ctx, const AVFr
 
     av_log(ctx, AV_LOG_INFO, "type %s; ", film_grain_type_names[fgp->type]);
     av_log(ctx, AV_LOG_INFO, "seed=%"PRIu64"; ", fgp->seed);
+    av_log(ctx, AV_LOG_INFO, "width=%d; ", fgp->width);
+    av_log(ctx, AV_LOG_INFO, "height=%d; ", fgp->height);
+    av_log(ctx, AV_LOG_INFO, "subsampling_x=%d; ", fgp->subsampling_x);
+    av_log(ctx, AV_LOG_INFO, "subsampling_y=%d; ", fgp->subsampling_y);
+    av_log(ctx, AV_LOG_INFO, "color_range=%s; ", color_range_str ? color_range_str : "unknown");
+    av_log(ctx, AV_LOG_INFO, "color_primaries=%s; ", color_primaries_str ? color_primaries_str : "unknown");
+    av_log(ctx, AV_LOG_INFO, "color_trc=%s; ", color_trc_str ? color_trc_str : "unknown");
+    av_log(ctx, AV_LOG_INFO, "color_space=%s; ", colorspace_str ? colorspace_str : "unknown");
+    av_log(ctx, AV_LOG_INFO, "bit_depth_luma=%d; ", fgp->bit_depth_luma);
+    av_log(ctx, AV_LOG_INFO, "bit_depth_chroma=%d; ", fgp->bit_depth_chroma);
 
     switch (fgp->type) {
     case AV_FILM_GRAIN_PARAMS_NONE:
@@ -504,18 +519,7 @@ static void dump_sei_film_grain_params_metadata(AVFilterContext *ctx, const AVFr
     }
     case AV_FILM_GRAIN_PARAMS_H274: {
         const AVFilmGrainH274Params *h274 = &fgp->codec.h274;
-        const char *color_range_str     = av_color_range_name(h274->color_range);
-        const char *color_primaries_str = av_color_primaries_name(h274->color_primaries);
-        const char *color_trc_str       = av_color_transfer_name(h274->color_trc);
-        const char *colorspace_str      = av_color_space_name(h274->color_space);
-
         av_log(ctx, AV_LOG_INFO, "model_id=%d; ", h274->model_id);
-        av_log(ctx, AV_LOG_INFO, "bit_depth_luma=%d; ", h274->bit_depth_luma);
-        av_log(ctx, AV_LOG_INFO, "bit_depth_chroma=%d; ", h274->bit_depth_chroma);
-        av_log(ctx, AV_LOG_INFO, "color_range=%s; ", color_range_str ? color_range_str : "unknown");
-        av_log(ctx, AV_LOG_INFO, "color_primaries=%s; ", color_primaries_str ? color_primaries_str : "unknown");
-        av_log(ctx, AV_LOG_INFO, "color_trc=%s; ", color_trc_str ? color_trc_str : "unknown");
-        av_log(ctx, AV_LOG_INFO, "color_space=%s; ", colorspace_str ? colorspace_str : "unknown");
         av_log(ctx, AV_LOG_INFO, "blending_mode_id=%d; ", h274->blending_mode_id);
         av_log(ctx, AV_LOG_INFO, "log2_scale_factor=%d; ", h274->log2_scale_factor);
 
-- 
2.44.0

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH v4 04/13] ffprobe: adapt to new AVFilmGrainParams

[Niklas Haas]

From: Niklas Haas <git@haasn.dev>

Follow the establish convention of printing the bit depth metadata
per-component.
---
 fftools/ffprobe.c | 41 ++++++++++++++++++++++++++---------------
 1 file changed, 26 insertions(+), 15 deletions(-)

diff --git a/fftools/ffprobe.c b/fftools/ffprobe.c
index fe2da82e24c..7d9998b428b 100644
--- a/fftools/ffprobe.c
+++ b/fftools/ffprobe.c
@@ -2402,22 +2402,41 @@ static void print_ambient_viewing_environment(WriterContext *w,
 static void print_film_grain_params(WriterContext *w,
                                     const AVFilmGrainParams *fgp)
 {
+    const char *color_range, *color_primaries, *color_trc, *color_space;
+    const char *const film_grain_type_names[] = {
+        [AV_FILM_GRAIN_PARAMS_NONE] = "none",
+        [AV_FILM_GRAIN_PARAMS_AV1]  = "av1",
+        [AV_FILM_GRAIN_PARAMS_H274] = "h274",
+    };
+
     AVBPrint pbuf;
-    if (!fgp)
+    if (!fgp || fgp->type >= FF_ARRAY_ELEMS(film_grain_type_names))
         return;
 
+    color_range     = av_color_range_name(fgp->color_range);
+    color_primaries = av_color_primaries_name(fgp->color_primaries);
+    color_trc       = av_color_transfer_name(fgp->color_trc);
+    color_space     = av_color_space_name(fgp->color_space);
+
     av_bprint_init(&pbuf, 1, AV_BPRINT_SIZE_UNLIMITED);
+    print_str("type", film_grain_type_names[fgp->type]);
+    print_fmt("seed", "%"PRIu64, fgp->seed);
+    print_int("width", fgp->width);
+    print_int("height", fgp->height);
+    print_int("subsampling_x", fgp->subsampling_x);
+    print_int("subsampling_y", fgp->subsampling_y);
+    print_str("color_range", color_range ? color_range : "unknown");
+    print_str("color_primaries", color_primaries ? color_primaries : "unknown");
+    print_str("color_trc", color_trc ? color_trc : "unknown");
+    print_str("color_space", color_space ? color_space : "unknown");
 
     switch (fgp->type) {
     case AV_FILM_GRAIN_PARAMS_NONE:
-        print_str("type", "none");
         break;
     case AV_FILM_GRAIN_PARAMS_AV1: {
         const AVFilmGrainAOMParams *aom = &fgp->codec.aom;
         const int num_ar_coeffs_y = 2 * aom->ar_coeff_lag * (aom->ar_coeff_lag + 1);
         const int num_ar_coeffs_uv = num_ar_coeffs_y + !!aom->num_y_points;
-        print_str("type", "av1");
-        print_fmt("seed", "%"PRIu64, fgp->seed);
         print_int("chroma_scaling_from_luma", aom->chroma_scaling_from_luma);
         print_int("scaling_shift", aom->scaling_shift);
         print_int("ar_coeff_lag", aom->ar_coeff_lag);
@@ -2431,6 +2450,7 @@ static void print_film_grain_params(WriterContext *w,
         if (aom->num_y_points) {
             writer_print_section_header(w, NULL, SECTION_ID_FRAME_SIDE_DATA_COMPONENT);
 
+            print_int("bit_depth_luma", fgp->bit_depth_luma);
             print_list_fmt("y_points_value", "%"PRIu8, aom->num_y_points, 1, aom->y_points[idx][0]);
             print_list_fmt("y_points_scaling", "%"PRIu8, aom->num_y_points, 1, aom->y_points[idx][1]);
             print_list_fmt("ar_coeffs_y", "%"PRId8, num_ar_coeffs_y, 1, aom->ar_coeffs_y[idx]);
@@ -2445,6 +2465,7 @@ static void print_film_grain_params(WriterContext *w,
 
             writer_print_section_header(w, NULL, SECTION_ID_FRAME_SIDE_DATA_COMPONENT);
 
+            print_int("bit_depth_chroma", fgp->bit_depth_chroma);
             print_list_fmt("uv_points_value", "%"PRIu8, aom->num_uv_points[uv], 1, aom->uv_points[uv][idx][0]);
             print_list_fmt("uv_points_scaling", "%"PRIu8, aom->num_uv_points[uv], 1, aom->uv_points[uv][idx][1]);
             print_list_fmt("ar_coeffs_uv", "%"PRId8, num_ar_coeffs_uv, 1, aom->ar_coeffs_uv[uv][idx]);
@@ -2462,17 +2483,7 @@ static void print_film_grain_params(WriterContext *w,
     }
     case AV_FILM_GRAIN_PARAMS_H274: {
         const AVFilmGrainH274Params *h274 = &fgp->codec.h274;
-        const char *color_range_str     = av_color_range_name(h274->color_range);
-        const char *color_primaries_str = av_color_primaries_name(h274->color_primaries);
-        const char *color_trc_str       = av_color_transfer_name(h274->color_trc);
-        const char *color_space_str     = av_color_space_name(h274->color_space);
-        print_str("type", "h274");
-        print_fmt("seed", "%"PRIu64, fgp->seed);
         print_int("model_id", h274->model_id);
-        print_str("color_range", color_range_str ? color_range_str : "unknown");
-        print_str("color_primaries", color_primaries_str ? color_primaries_str : "unknown");
-        print_str("color_trc", color_trc_str ? color_trc_str : "unknown");
-        print_str("color_space", color_space_str ? color_space_str : "unknown");
         print_int("blending_mode_id", h274->blending_mode_id);
         print_int("log2_scale_factor", h274->log2_scale_factor);
 
@@ -2483,7 +2494,7 @@ static void print_film_grain_params(WriterContext *w,
                 continue;
 
             writer_print_section_header(w, NULL, SECTION_ID_FRAME_SIDE_DATA_COMPONENT);
-            print_int(c ? "bit_depth_chroma" : "bit_depth_luma", c ? h274->bit_depth_chroma : h274->bit_depth_luma);
+            print_int(c ? "bit_depth_chroma" : "bit_depth_luma", c ? fgp->bit_depth_chroma : fgp->bit_depth_luma);
 
             writer_print_section_header(w, NULL, SECTION_ID_FRAME_SIDE_DATA_PIECE_LIST);
             for (int i = 0; i < h274->num_intensity_intervals[c]; i++) {
-- 
2.44.0

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH v4 05/13] avcodec/h2645_sei: signal new AVFilmGrainParams members

[Niklas Haas]

From: Niklas Haas <git@haasn.dev>

H.274 specifies that film grain parameters are signalled as intended for
4:4:4 frames, so we always signal this, regardless of the frame's actual
subsampling.
---
 libavcodec/h2645_sei.c | 46 +++++++++++++++++++++++++-----------------
 1 file changed, 28 insertions(+), 18 deletions(-)

diff --git a/libavcodec/h2645_sei.c b/libavcodec/h2645_sei.c
index e60606f43f9..73f894436fd 100644
--- a/libavcodec/h2645_sei.c
+++ b/libavcodec/h2645_sei.c
@@ -641,35 +641,45 @@ int ff_h2645_sei_to_frame(AVFrame *frame, H2645SEI *sei,
         h274      = &fgp->codec.h274;
 
         fgp->seed = seed;
+        fgp->width = frame->width;
+        fgp->height = frame->height;
+
+        /* H.274 mandates film grain be applied to 4:4:4 frames */
+        fgp->subsampling_x = fgp->subsampling_y = 0;
 
         h274->model_id = fgc->model_id;
         if (fgc->separate_colour_description_present_flag) {
-            h274->bit_depth_luma   = fgc->bit_depth_luma;
-            h274->bit_depth_chroma = fgc->bit_depth_chroma;
-            h274->color_range      = fgc->full_range + 1;
-            h274->color_primaries  = fgc->color_primaries;
-            h274->color_trc        = fgc->transfer_characteristics;
-            h274->color_space      = fgc->matrix_coeffs;
+            fgp->bit_depth_luma   = fgc->bit_depth_luma;
+            fgp->bit_depth_chroma = fgc->bit_depth_chroma;
+            fgp->color_range      = fgc->full_range + 1;
+            fgp->color_primaries  = fgc->color_primaries;
+            fgp->color_trc        = fgc->transfer_characteristics;
+            fgp->color_space      = fgc->matrix_coeffs;
         } else {
-            h274->bit_depth_luma   = bit_depth_luma;
-            h274->bit_depth_chroma = bit_depth_chroma;
+            fgp->bit_depth_luma   = bit_depth_luma;
+            fgp->bit_depth_chroma = bit_depth_chroma;
             if (vui->video_signal_type_present_flag)
-                h274->color_range = vui->video_full_range_flag + 1;
-            else
-                h274->color_range = AVCOL_RANGE_UNSPECIFIED;
+                fgp->color_range = vui->video_full_range_flag + 1;
             if (vui->colour_description_present_flag) {
-                h274->color_primaries = vui->colour_primaries;
-                h274->color_trc       = vui->transfer_characteristics;
-                h274->color_space     = vui->matrix_coeffs;
-            } else {
-                h274->color_primaries = AVCOL_PRI_UNSPECIFIED;
-                h274->color_trc       = AVCOL_TRC_UNSPECIFIED;
-                h274->color_space     = AVCOL_SPC_UNSPECIFIED;
+                fgp->color_primaries = vui->colour_primaries;
+                fgp->color_trc       = vui->transfer_characteristics;
+                fgp->color_space     = vui->matrix_coeffs;
             }
         }
         h274->blending_mode_id  = fgc->blending_mode_id;
         h274->log2_scale_factor = fgc->log2_scale_factor;
 
+#if FF_API_H274_FILM_GRAIN_VCS
+FF_DISABLE_DEPRECATION_WARNINGS
+        h274->bit_depth_luma   = fgp->bit_depth_luma;
+        h274->bit_depth_chroma = fgp->bit_depth_chroma;
+        h274->color_range      = fgp->color_range;
+        h274->color_primaries  = fgp->color_primaries;
+        h274->color_trc        = fgp->color_trc;
+        h274->color_space      = fgp->color_space;
+FF_ENABLE_DEPRECATION_WARNINGS
+#endif
+
         memcpy(&h274->component_model_present, &fgc->comp_model_present_flag,
                sizeof(h274->component_model_present));
         memcpy(&h274->num_intensity_intervals, &fgc->num_intensity_intervals,
-- 
2.44.0

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH v4 06/13] avcodec/av1dec: signal new AVFilmGrainParams members

[Niklas Haas]

From: Niklas Haas <git@haasn.dev>

Not directly signalled by AV1, but we should still set this accordingly
so that users will know what the original intended video characteristics
and chroma resolution were.
---
 libavcodec/av1dec.c | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/libavcodec/av1dec.c b/libavcodec/av1dec.c
index e6346b51dbe..56e4af278be 100644
--- a/libavcodec/av1dec.c
+++ b/libavcodec/av1dec.c
@@ -1072,9 +1072,11 @@ static int export_film_grain(AVCodecContext *avctx, AVFrame *frame)
 {
     AV1DecContext *s = avctx->priv_data;
     const AV1RawFilmGrainParams *film_grain = &s->cur_frame.film_grain;
+    const AVPixFmtDescriptor *pixdesc = av_pix_fmt_desc_get(frame->format);
     AVFilmGrainParams *fgp;
     AVFilmGrainAOMParams *aom;
 
+    av_assert0(pixdesc);
     if (!film_grain->apply_grain)
         return 0;
 
@@ -1084,6 +1086,14 @@ static int export_film_grain(AVCodecContext *avctx, AVFrame *frame)
 
     fgp->type = AV_FILM_GRAIN_PARAMS_AV1;
     fgp->seed = film_grain->grain_seed;
+    fgp->width = frame->width;
+    fgp->height = frame->height;
+    fgp->color_range = frame->color_range;
+    fgp->color_primaries = frame->color_primaries;
+    fgp->color_trc = frame->color_trc;
+    fgp->color_space = frame->colorspace;
+    fgp->subsampling_x = pixdesc->log2_chroma_w;
+    fgp->subsampling_y = pixdesc->log2_chroma_h;
 
     aom = &fgp->codec.aom;
     aom->chroma_scaling_from_luma = film_grain->chroma_scaling_from_luma;
-- 
2.44.0

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH v4 07/13] avcodec/libdavv1d: signal new AVFilmGrainParams members

[Niklas Haas]

From: Niklas Haas <git@haasn.dev>

Not directly signalled by AV1, but we should still set this accordingly
so that users will know what the original intended video characteristics
and chroma resolution were.
---
 libavcodec/libdav1d.c | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/libavcodec/libdav1d.c b/libavcodec/libdav1d.c
index 1aa2d1f3436..361b988825c 100644
--- a/libavcodec/libdav1d.c
+++ b/libavcodec/libdav1d.c
@@ -613,6 +613,8 @@ static int libdav1d_receive_frame(AVCodecContext *c, AVFrame *frame)
     if (p->frame_hdr->film_grain.present && (!dav1d->apply_grain ||
         (c->export_side_data & AV_CODEC_EXPORT_DATA_FILM_GRAIN))) {
         AVFilmGrainParams *fgp = av_film_grain_params_create_side_data(frame);
+        const AVPixFmtDescriptor *pixdesc = av_pix_fmt_desc_get(frame->format);
+        av_assert0(pixdesc);
         if (!fgp) {
             res = AVERROR(ENOMEM);
             goto fail;
@@ -620,6 +622,14 @@ static int libdav1d_receive_frame(AVCodecContext *c, AVFrame *frame)
 
         fgp->type = AV_FILM_GRAIN_PARAMS_AV1;
         fgp->seed = p->frame_hdr->film_grain.data.seed;
+        fgp->width = frame->width;
+        fgp->height = frame->height;
+        fgp->color_range = frame->color_range;
+        fgp->color_primaries = frame->color_primaries;
+        fgp->color_trc = frame->color_trc;
+        fgp->color_space = frame->colorspace;
+        fgp->subsampling_x = pixdesc->log2_chroma_w;
+        fgp->subsampling_y = pixdesc->log2_chroma_h;
         fgp->codec.aom.num_y_points = p->frame_hdr->film_grain.data.num_y_points;
         fgp->codec.aom.chroma_scaling_from_luma = p->frame_hdr->film_grain.data.chroma_scaling_from_luma;
         fgp->codec.aom.scaling_shift = p->frame_hdr->film_grain.data.scaling_shift;
-- 
2.44.0

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH v4 08/13] avutil/frame: clarify AV_FRAME_DATA_FILM_GRAIN_PARAMS usage

[Niklas Haas]

From: Niklas Haas <git@haasn.dev>

To allow for AFGS1 usage, which can expose multiple parameter sets for
a single frame.
---
 libavutil/frame.h | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/libavutil/frame.h b/libavutil/frame.h
index b94687941db..7f616488be8 100644
--- a/libavutil/frame.h
+++ b/libavutil/frame.h
@@ -180,6 +180,10 @@ enum AVFrameSideDataType {
     /**
      * Film grain parameters for a frame, described by AVFilmGrainParams.
      * Must be present for every frame which should have film grain applied.
+     *
+     * May be present multiple times, for example when there are multiple
+     * alternative parameter sets for different video signal characteristics.
+     * The user should select the most appropriate set for the application.
      */
     AV_FRAME_DATA_FILM_GRAIN_PARAMS,
 
-- 
2.44.0

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH v4 09/13] avutil/film_grain_params: add av_film_grain_params_select()

[Niklas Haas]

From: Niklas Haas <git@haasn.dev>

Common utility function that can be used by all codecs to select the
right (any valid) film grain parameter set. In particular, this is
useful for AFGS1, which has support for multiple parameters.

However, it also performs parameter validation for H274.
---
 doc/APIchanges                |  3 ++
 libavutil/film_grain_params.c | 61 +++++++++++++++++++++++++++++++++++
 libavutil/film_grain_params.h | 11 +++++++
 libavutil/version.h           |  2 +-
 4 files changed, 76 insertions(+), 1 deletion(-)

diff --git a/doc/APIchanges b/doc/APIchanges
index 256d9c7757a..f91f4fdcec4 100644
--- a/doc/APIchanges
+++ b/doc/APIchanges
@@ -2,6 +2,9 @@ The last version increases of all libraries were on 2024-03-07
 
 API changes, most recent first:
 
+2024-03-xx - xxxxxxxxxx - lavu 59.4.100 - film_grain_params.h
+  Add av_film_grain_params_select().
+
 2024-03-xx - xxxxxxxxxx - lavu 59.3.100 - film_grain_params.h
   Add AVFilmGrainParams.color_range, color_primaries, color_trc, color_space,
   width, height, subsampling_x, subsampling_y, bit_depth_luma and
diff --git a/libavutil/film_grain_params.c b/libavutil/film_grain_params.c
index 230ce8d701c..fff7252f2f5 100644
--- a/libavutil/film_grain_params.c
+++ b/libavutil/film_grain_params.c
@@ -17,6 +17,7 @@
  */
 
 #include "film_grain_params.h"
+#include "pixdesc.h"
 
 AVFilmGrainParams *av_film_grain_params_alloc(size_t *size)
 {
@@ -47,3 +48,63 @@ AVFilmGrainParams *av_film_grain_params_create_side_data(AVFrame *frame)
 
     return fgp;
 }
+
+const AVFilmGrainParams *av_film_grain_params_select(const AVFrame *frame)
+{
+    const AVFilmGrainParams *fgp, *best = NULL;
+    const AVPixFmtDescriptor *desc = av_pix_fmt_desc_get(frame->format);
+    const AVFilmGrainAOMParams *aom;
+    const AVFilmGrainH274Params *h274;
+    int bit_depth_luma, bit_depth_chroma;
+    if (!desc)
+        return NULL;
+
+    /* There are no YUV formats with different bit depth per component,
+     * so just check both against the first component for simplicity */
+    bit_depth_luma = bit_depth_chroma = desc->comp[0].depth;
+
+    for (int i = 0; i < frame->nb_side_data; i++) {
+        if (frame->side_data[i]->type != AV_FRAME_DATA_FILM_GRAIN_PARAMS)
+            continue;
+        fgp = (const AVFilmGrainParams*)frame->side_data[i]->data;
+        if (fgp->width  && fgp->width  > frame->width ||
+            fgp->height && fgp->height > frame->height)
+            continue;
+
+#define CHECK(a, b, unspec)                                     \
+    do {                                                        \
+        if ((a) != (unspec) && (b) != (unspec) && (a) != (b))   \
+            continue;                                           \
+    } while (0)
+
+        CHECK(fgp->bit_depth_luma,   bit_depth_luma,         0);
+        CHECK(fgp->bit_depth_chroma, bit_depth_chroma,       0);
+        CHECK(fgp->color_range,      frame->color_range,     AVCOL_RANGE_UNSPECIFIED);
+        CHECK(fgp->color_primaries,  frame->color_primaries, AVCOL_PRI_UNSPECIFIED);
+        CHECK(fgp->color_trc,        frame->color_trc,       AVCOL_TRC_UNSPECIFIED);
+        CHECK(fgp->color_space,      frame->colorspace,      AVCOL_SPC_UNSPECIFIED);
+
+        switch (fgp->type) {
+        case AV_FILM_GRAIN_PARAMS_NONE:
+            continue;
+        case AV_FILM_GRAIN_PARAMS_AV1:
+            aom = &fgp->codec.aom;
+            /* AOM FGS needs an exact match for the chroma resolution */
+            if (fgp->subsampling_x != desc->log2_chroma_w ||
+                fgp->subsampling_y != desc->log2_chroma_h)
+                continue;
+            break;
+        case AV_FILM_GRAIN_PARAMS_H274:
+            /* H.274 FGS can be adapted to any lower chroma resolution */
+            if (fgp->subsampling_x > desc->log2_chroma_w ||
+                fgp->subsampling_y > desc->log2_chroma_h)
+                continue;
+            break;
+        }
+
+        if (!best || best->width < fgp->width || best->height < fgp->height)
+            best = fgp;
+    }
+
+    return best;
+}
diff --git a/libavutil/film_grain_params.h b/libavutil/film_grain_params.h
index a9f243351c9..ccacab88fed 100644
--- a/libavutil/film_grain_params.h
+++ b/libavutil/film_grain_params.h
@@ -308,4 +308,15 @@ AVFilmGrainParams *av_film_grain_params_alloc(size_t *size);
  */
 AVFilmGrainParams *av_film_grain_params_create_side_data(AVFrame *frame);
 
+/**
+ * Select the most appropriate film grain parameters set for the frame,
+ * taking into account the frame's format, resolution and video signal
+ * characteristics.
+ *
+ * @note, for H.274, this may select a film grain parameter set with
+ * greater chroma resolution than the frame. Users should take care to
+ * correctly adjust the chroma grain frequency to the frame.
+ */
+const AVFilmGrainParams *av_film_grain_params_select(const AVFrame *frame);
+
 #endif /* AVUTIL_FILM_GRAIN_PARAMS_H */
diff --git a/libavutil/version.h b/libavutil/version.h
index 23351316b58..5248207ec7d 100644
--- a/libavutil/version.h
+++ b/libavutil/version.h
@@ -79,7 +79,7 @@
  */
 
 #define LIBAVUTIL_VERSION_MAJOR  59
-#define LIBAVUTIL_VERSION_MINOR   3
+#define LIBAVUTIL_VERSION_MINOR   4
 #define LIBAVUTIL_VERSION_MICRO 100
 
 #define LIBAVUTIL_VERSION_INT   AV_VERSION_INT(LIBAVUTIL_VERSION_MAJOR, \
-- 
2.44.0

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v4 09/13] avutil/film_grain_params: add av_film_grain_params_select()

[Anton Khirnov]

Quoting Niklas Haas (2024-03-18 17:54:17)
> +const AVFilmGrainParams *av_film_grain_params_select(const AVFrame *frame);

Could this benefit from a flags parameter for future extensions?

-- 
Anton Khirnov
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH v4 10/13] avcodec/aom_film_grain: add AOM film grain synthesis

[Niklas Haas]

From: Niklas Haas <git@haasn.dev>

Implementation copied wholesale from dav1d, sans SIMD, under permissive
license. This implementation was extensively verified to be bit-exact,
so it serves as a much better starting point than trying to re-engineer
this from scratch for no reason. (I also authored the original
implementation in dav1d, so any "clean room" implementation would end up
looking much the same, anyway)

The notable changes I had to make while adapting this from the dav1d
code-base to the FFmpeg codebase include:

- reordering variable declarations to avoid triggering warnings
- replacing several inline helpers by avutil equivalents
- changing code that accesses frame metadata
- replacing raw plane copying logic by av_image_copy_plane

Apart from this, the implementation is basically unmodified.
---
 libavcodec/aom_film_grain.c          | 310 ++++++++++++++
 libavcodec/aom_film_grain.h          |  38 ++
 libavcodec/aom_film_grain_template.c | 577 +++++++++++++++++++++++++++
 3 files changed, 925 insertions(+)
 create mode 100644 libavcodec/aom_film_grain.c
 create mode 100644 libavcodec/aom_film_grain.h
 create mode 100644 libavcodec/aom_film_grain_template.c

diff --git a/libavcodec/aom_film_grain.c b/libavcodec/aom_film_grain.c
new file mode 100644
index 00000000000..ffcd71b584b
--- /dev/null
+++ b/libavcodec/aom_film_grain.c
@@ -0,0 +1,310 @@
+/*
+ * AOM film grain synthesis
+ * Copyright (c) 2023 Niklas Haas <ffmpeg@haasn.xyz>
+ *
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with FFmpeg; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+/**
+ * @file
+ * AOM film grain synthesis.
+ * @author Niklas Haas <ffmpeg@haasn.xyz>
+ */
+
+#include "libavutil/avassert.h"
+#include "libavutil/imgutils.h"
+
+#include "aom_film_grain.h"
+
+// Common/shared helpers (not dependent on BIT_DEPTH)
+static inline int get_random_number(const int bits, unsigned *const state) {
+    const int r = *state;
+    unsigned bit = ((r >> 0) ^ (r >> 1) ^ (r >> 3) ^ (r >> 12)) & 1;
+    *state = (r >> 1) | (bit << 15);
+
+    return (*state >> (16 - bits)) & ((1 << bits) - 1);
+}
+
+static inline int round2(const int x, const uint64_t shift) {
+    return (x + ((1 << shift) >> 1)) >> shift;
+}
+
+enum {
+    GRAIN_WIDTH      = 82,
+    GRAIN_HEIGHT     = 73,
+    SUB_GRAIN_WIDTH  = 44,
+    SUB_GRAIN_HEIGHT = 38,
+    FG_BLOCK_SIZE    = 32,
+};
+
+static const int16_t gaussian_sequence[2048];
+
+#define BIT_DEPTH 16
+#include "aom_film_grain_template.c"
+#undef BIT_DEPTH
+
+#define BIT_DEPTH 8
+#include "aom_film_grain_template.c"
+#undef BIT_DEPTH
+
+
+int ff_aom_apply_film_grain(AVFrame *out, const AVFrame *in,
+                            const AVFilmGrainParams *params)
+{
+    const AVFilmGrainAOMParams *const data = &params->codec.aom;
+    const AVPixFmtDescriptor *desc = av_pix_fmt_desc_get(out->format);
+    const int subx = desc->log2_chroma_w, suby = desc->log2_chroma_h;
+    const int pxstep = desc->comp[0].step;
+
+    av_assert0(out->format == in->format);
+    av_assert0(params->type == AV_FILM_GRAIN_PARAMS_AV1);
+
+    // Copy over the non-modified planes
+    if (!params->codec.aom.num_y_points) {
+        av_image_copy_plane(out->data[0], out->linesize[0],
+                            in->data[0], in->linesize[0],
+                            out->width * pxstep, out->height);
+    }
+    for (int uv = 0; uv < 2; uv++) {
+        if (!data->num_uv_points[uv]) {
+            av_image_copy_plane(out->data[1+uv], out->linesize[1+uv],
+                                in->data[1+uv], in->linesize[1+uv],
+                                AV_CEIL_RSHIFT(out->width, subx) * pxstep,
+                                AV_CEIL_RSHIFT(out->height, suby));
+        }
+    }
+
+    switch (in->format) {
+    case AV_PIX_FMT_GRAY8:
+    case AV_PIX_FMT_YUV420P:
+    case AV_PIX_FMT_YUV422P:
+    case AV_PIX_FMT_YUV444P:
+    case AV_PIX_FMT_YUVJ420P:
+    case AV_PIX_FMT_YUVJ422P:
+    case AV_PIX_FMT_YUVJ444P:
+        return apply_film_grain_8(out, in, params);
+    case AV_PIX_FMT_GRAY9:
+    case AV_PIX_FMT_YUV420P9:
+    case AV_PIX_FMT_YUV422P9:
+    case AV_PIX_FMT_YUV444P9:
+        return apply_film_grain_16(out, in, params, 9);
+    case AV_PIX_FMT_GRAY10:
+    case AV_PIX_FMT_YUV420P10:
+    case AV_PIX_FMT_YUV422P10:
+    case AV_PIX_FMT_YUV444P10:
+        return apply_film_grain_16(out, in, params, 10);
+    case AV_PIX_FMT_GRAY12:
+    case AV_PIX_FMT_YUV420P12:
+    case AV_PIX_FMT_YUV422P12:
+    case AV_PIX_FMT_YUV444P12:
+        return apply_film_grain_16(out, in, params, 12);
+    }
+
+    /* The AV1 spec only defines film grain synthesis for these formats */
+    return AVERROR_INVALIDDATA;
+}
+
+// Taken from the AV1 spec. Range is [-2048, 2047], mean is 0 and stddev is 512
+static const int16_t gaussian_sequence[2048] = {
+    56,    568,   -180,  172,   124,   -84,   172,   -64,   -900,  24,   820,
+    224,   1248,  996,   272,   -8,    -916,  -388,  -732,  -104,  -188, 800,
+    112,   -652,  -320,  -376,  140,   -252,  492,   -168,  44,    -788, 588,
+    -584,  500,   -228,  12,    680,   272,   -476,  972,   -100,  652,  368,
+    432,   -196,  -720,  -192,  1000,  -332,  652,   -136,  -552,  -604, -4,
+    192,   -220,  -136,  1000,  -52,   372,   -96,   -624,  124,   -24,  396,
+    540,   -12,   -104,  640,   464,   244,   -208,  -84,   368,   -528, -740,
+    248,   -968,  -848,  608,   376,   -60,   -292,  -40,   -156,  252,  -292,
+    248,   224,   -280,  400,   -244,  244,   -60,   76,    -80,   212,  532,
+    340,   128,   -36,   824,   -352,  -60,   -264,  -96,   -612,  416,  -704,
+    220,   -204,  640,   -160,  1220,  -408,  900,   336,   20,    -336, -96,
+    -792,  304,   48,    -28,   -1232, -1172, -448,  104,   -292,  -520, 244,
+    60,    -948,  0,     -708,  268,   108,   356,   -548,  488,   -344, -136,
+    488,   -196,  -224,  656,   -236,  -1128, 60,    4,     140,   276,  -676,
+    -376,  168,   -108,  464,   8,     564,   64,    240,   308,   -300, -400,
+    -456,  -136,  56,    120,   -408,  -116,  436,   504,   -232,  328,  844,
+    -164,  -84,   784,   -168,  232,   -224,  348,   -376,  128,   568,  96,
+    -1244, -288,  276,   848,   832,   -360,  656,   464,   -384,  -332, -356,
+    728,   -388,  160,   -192,  468,   296,   224,   140,   -776,  -100, 280,
+    4,     196,   44,    -36,   -648,  932,   16,    1428,  28,    528,  808,
+    772,   20,    268,   88,    -332,  -284,  124,   -384,  -448,  208,  -228,
+    -1044, -328,  660,   380,   -148,  -300,  588,   240,   540,   28,   136,
+    -88,   -436,  256,   296,   -1000, 1400,  0,     -48,   1056,  -136, 264,
+    -528,  -1108, 632,   -484,  -592,  -344,  796,   124,   -668,  -768, 388,
+    1296,  -232,  -188,  -200,  -288,  -4,    308,   100,   -168,  256,  -500,
+    204,   -508,  648,   -136,  372,   -272,  -120,  -1004, -552,  -548, -384,
+    548,   -296,  428,   -108,  -8,    -912,  -324,  -224,  -88,   -112, -220,
+    -100,  996,   -796,  548,   360,   -216,  180,   428,   -200,  -212, 148,
+    96,    148,   284,   216,   -412,  -320,  120,   -300,  -384,  -604, -572,
+    -332,  -8,    -180,  -176,  696,   116,   -88,   628,   76,    44,   -516,
+    240,   -208,  -40,   100,   -592,  344,   -308,  -452,  -228,  20,   916,
+    -1752, -136,  -340,  -804,  140,   40,    512,   340,   248,   184,  -492,
+    896,   -156,  932,   -628,  328,   -688,  -448,  -616,  -752,  -100, 560,
+    -1020, 180,   -800,  -64,   76,    576,   1068,  396,   660,   552,  -108,
+    -28,   320,   -628,  312,   -92,   -92,   -472,  268,   16,    560,  516,
+    -672,  -52,   492,   -100,  260,   384,   284,   292,   304,   -148, 88,
+    -152,  1012,  1064,  -228,  164,   -376,  -684,  592,   -392,  156,  196,
+    -524,  -64,   -884,  160,   -176,  636,   648,   404,   -396,  -436, 864,
+    424,   -728,  988,   -604,  904,   -592,  296,   -224,  536,   -176, -920,
+    436,   -48,   1176,  -884,  416,   -776,  -824,  -884,  524,   -548, -564,
+    -68,   -164,  -96,   692,   364,   -692,  -1012, -68,   260,   -480, 876,
+    -1116, 452,   -332,  -352,  892,   -1088, 1220,  -676,  12,    -292, 244,
+    496,   372,   -32,   280,   200,   112,   -440,  -96,   24,    -644, -184,
+    56,    -432,  224,   -980,  272,   -260,  144,   -436,  420,   356,  364,
+    -528,  76,    172,   -744,  -368,  404,   -752,  -416,  684,   -688, 72,
+    540,   416,   92,    444,   480,   -72,   -1416, 164,   -1172, -68,  24,
+    424,   264,   1040,  128,   -912,  -524,  -356,  64,    876,   -12,  4,
+    -88,   532,   272,   -524,  320,   276,   -508,  940,   24,    -400, -120,
+    756,   60,    236,   -412,  100,   376,   -484,  400,   -100,  -740, -108,
+    -260,  328,   -268,  224,   -200,  -416,  184,   -604,  -564,  -20,  296,
+    60,    892,   -888,  60,    164,   68,    -760,  216,   -296,  904,  -336,
+    -28,   404,   -356,  -568,  -208,  -1480, -512,  296,   328,   -360, -164,
+    -1560, -776,  1156,  -428,  164,   -504,  -112,  120,   -216,  -148, -264,
+    308,   32,    64,    -72,   72,    116,   176,   -64,   -272,  460,  -536,
+    -784,  -280,  348,   108,   -752,  -132,  524,   -540,  -776,  116,  -296,
+    -1196, -288,  -560,  1040,  -472,  116,   -848,  -1116, 116,   636,  696,
+    284,   -176,  1016,  204,   -864,  -648,  -248,  356,   972,   -584, -204,
+    264,   880,   528,   -24,   -184,  116,   448,   -144,  828,   524,  212,
+    -212,  52,    12,    200,   268,   -488,  -404,  -880,  824,   -672, -40,
+    908,   -248,  500,   716,   -576,  492,   -576,  16,    720,   -108, 384,
+    124,   344,   280,   576,   -500,  252,   104,   -308,  196,   -188, -8,
+    1268,  296,   1032,  -1196, 436,   316,   372,   -432,  -200,  -660, 704,
+    -224,  596,   -132,  268,   32,    -452,  884,   104,   -1008, 424,  -1348,
+    -280,  4,     -1168, 368,   476,   696,   300,   -8,    24,    180,  -592,
+    -196,  388,   304,   500,   724,   -160,  244,   -84,   272,   -256, -420,
+    320,   208,   -144,  -156,  156,   364,   452,   28,    540,   316,  220,
+    -644,  -248,  464,   72,    360,   32,    -388,  496,   -680,  -48,  208,
+    -116,  -408,  60,    -604,  -392,  548,   -840,  784,   -460,  656,  -544,
+    -388,  -264,  908,   -800,  -628,  -612,  -568,  572,   -220,  164,  288,
+    -16,   -308,  308,   -112,  -636,  -760,  280,   -668,  432,   364,  240,
+    -196,  604,   340,   384,   196,   592,   -44,   -500,  432,   -580, -132,
+    636,   -76,   392,   4,     -412,  540,   508,   328,   -356,  -36,  16,
+    -220,  -64,   -248,  -60,   24,    -192,  368,   1040,  92,    -24,  -1044,
+    -32,   40,    104,   148,   192,   -136,  -520,  56,    -816,  -224, 732,
+    392,   356,   212,   -80,   -424,  -1008, -324,  588,   -1496, 576,  460,
+    -816,  -848,  56,    -580,  -92,   -1372, -112,  -496,  200,   364,  52,
+    -140,  48,    -48,   -60,   84,    72,    40,    132,   -356,  -268, -104,
+    -284,  -404,  732,   -520,  164,   -304,  -540,  120,   328,   -76,  -460,
+    756,   388,   588,   236,   -436,  -72,   -176,  -404,  -316,  -148, 716,
+    -604,  404,   -72,   -88,   -888,  -68,   944,   88,    -220,  -344, 960,
+    472,   460,   -232,  704,   120,   832,   -228,  692,   -508,  132,  -476,
+    844,   -748,  -364,  -44,   1116,  -1104, -1056, 76,    428,   552,  -692,
+    60,    356,   96,    -384,  -188,  -612,  -576,  736,   508,   892,  352,
+    -1132, 504,   -24,   -352,  324,   332,   -600,  -312,  292,   508,  -144,
+    -8,    484,   48,    284,   -260,  -240,  256,   -100,  -292,  -204, -44,
+    472,   -204,  908,   -188,  -1000, -256,  92,    1164,  -392,  564,  356,
+    652,   -28,   -884,  256,   484,   -192,  760,   -176,  376,   -524, -452,
+    -436,  860,   -736,  212,   124,   504,   -476,  468,   76,    -472, 552,
+    -692,  -944,  -620,  740,   -240,  400,   132,   20,    192,   -196, 264,
+    -668,  -1012, -60,   296,   -316,  -828,  76,    -156,  284,   -768, -448,
+    -832,  148,   248,   652,   616,   1236,  288,   -328,  -400,  -124, 588,
+    220,   520,   -696,  1032,  768,   -740,  -92,   -272,  296,   448,  -464,
+    412,   -200,  392,   440,   -200,  264,   -152,  -260,  320,   1032, 216,
+    320,   -8,    -64,   156,   -1016, 1084,  1172,  536,   484,   -432, 132,
+    372,   -52,   -256,  84,    116,   -352,  48,    116,   304,   -384, 412,
+    924,   -300,  528,   628,   180,   648,   44,    -980,  -220,  1320, 48,
+    332,   748,   524,   -268,  -720,  540,   -276,  564,   -344,  -208, -196,
+    436,   896,   88,    -392,  132,   80,    -964,  -288,  568,   56,   -48,
+    -456,  888,   8,     552,   -156,  -292,  948,   288,   128,   -716, -292,
+    1192,  -152,  876,   352,   -600,  -260,  -812,  -468,  -28,   -120, -32,
+    -44,   1284,  496,   192,   464,   312,   -76,   -516,  -380,  -456, -1012,
+    -48,   308,   -156,  36,    492,   -156,  -808,  188,   1652,  68,   -120,
+    -116,  316,   160,   -140,  352,   808,   -416,  592,   316,   -480, 56,
+    528,   -204,  -568,  372,   -232,  752,   -344,  744,   -4,    324,  -416,
+    -600,  768,   268,   -248,  -88,   -132,  -420,  -432,  80,    -288, 404,
+    -316,  -1216, -588,  520,   -108,  92,    -320,  368,   -480,  -216, -92,
+    1688,  -300,  180,   1020,  -176,  820,   -68,   -228,  -260,  436,  -904,
+    20,    40,    -508,  440,   -736,  312,   332,   204,   760,   -372, 728,
+    96,    -20,   -632,  -520,  -560,  336,   1076,  -64,   -532,  776,  584,
+    192,   396,   -728,  -520,  276,   -188,  80,    -52,   -612,  -252, -48,
+    648,   212,   -688,  228,   -52,   -260,  428,   -412,  -272,  -404, 180,
+    816,   -796,  48,    152,   484,   -88,   -216,  988,   696,   188,  -528,
+    648,   -116,  -180,  316,   476,   12,    -564,  96,    476,   -252, -364,
+    -376,  -392,  556,   -256,  -576,  260,   -352,  120,   -16,   -136, -260,
+    -492,  72,    556,   660,   580,   616,   772,   436,   424,   -32,  -324,
+    -1268, 416,   -324,  -80,   920,   160,   228,   724,   32,    -516, 64,
+    384,   68,    -128,  136,   240,   248,   -204,  -68,   252,   -932, -120,
+    -480,  -628,  -84,   192,   852,   -404,  -288,  -132,  204,   100,  168,
+    -68,   -196,  -868,  460,   1080,  380,   -80,   244,   0,     484,  -888,
+    64,    184,   352,   600,   460,   164,   604,   -196,  320,   -64,  588,
+    -184,  228,   12,    372,   48,    -848,  -344,  224,   208,   -200, 484,
+    128,   -20,   272,   -468,  -840,  384,   256,   -720,  -520,  -464, -580,
+    112,   -120,  644,   -356,  -208,  -608,  -528,  704,   560,   -424, 392,
+    828,   40,    84,    200,   -152,  0,     -144,  584,   280,   -120, 80,
+    -556,  -972,  -196,  -472,  724,   80,    168,   -32,   88,    160,  -688,
+    0,     160,   356,   372,   -776,  740,   -128,  676,   -248,  -480, 4,
+    -364,  96,    544,   232,   -1032, 956,   236,   356,   20,    -40,  300,
+    24,    -676,  -596,  132,   1120,  -104,  532,   -1096, 568,   648,  444,
+    508,   380,   188,   -376,  -604,  1488,  424,   24,    756,   -220, -192,
+    716,   120,   920,   688,   168,   44,    -460,  568,   284,   1144, 1160,
+    600,   424,   888,   656,   -356,  -320,  220,   316,   -176,  -724, -188,
+    -816,  -628,  -348,  -228,  -380,  1012,  -452,  -660,  736,   928,  404,
+    -696,  -72,   -268,  -892,  128,   184,   -344,  -780,  360,   336,  400,
+    344,   428,   548,   -112,  136,   -228,  -216,  -820,  -516,  340,  92,
+    -136,  116,   -300,  376,   -244,  100,   -316,  -520,  -284,  -12,  824,
+    164,   -548,  -180,  -128,  116,   -924,  -828,  268,   -368,  -580, 620,
+    192,   160,   0,     -1676, 1068,  424,   -56,   -360,  468,   -156, 720,
+    288,   -528,  556,   -364,  548,   -148,  504,   316,   152,   -648, -620,
+    -684,  -24,   -376,  -384,  -108,  -920,  -1032, 768,   180,   -264, -508,
+    -1268, -260,  -60,   300,   -240,  988,   724,   -376,  -576,  -212, -736,
+    556,   192,   1092,  -620,  -880,  376,   -56,   -4,    -216,  -32,  836,
+    268,   396,   1332,  864,   -600,  100,   56,    -412,  -92,   356,  180,
+    884,   -468,  -436,  292,   -388,  -804,  -704,  -840,  368,   -348, 140,
+    -724,  1536,  940,   372,   112,   -372,  436,   -480,  1136,  296,  -32,
+    -228,  132,   -48,   -220,  868,   -1016, -60,   -1044, -464,  328,  916,
+    244,   12,    -736,  -296,  360,   468,   -376,  -108,  -92,   788,  368,
+    -56,   544,   400,   -672,  -420,  728,   16,    320,   44,    -284, -380,
+    -796,  488,   132,   204,   -596,  -372,  88,    -152,  -908,  -636, -572,
+    -624,  -116,  -692,  -200,  -56,   276,   -88,   484,   -324,  948,  864,
+    1000,  -456,  -184,  -276,  292,   -296,  156,   676,   320,   160,  908,
+    -84,   -1236, -288,  -116,  260,   -372,  -644,  732,   -756,  -96,  84,
+    344,   -520,  348,   -688,  240,   -84,   216,   -1044, -136,  -676, -396,
+    -1500, 960,   -40,   176,   168,   1516,  420,   -504,  -344,  -364, -360,
+    1216,  -940,  -380,  -212,  252,   -660,  -708,  484,   -444,  -152, 928,
+    -120,  1112,  476,   -260,  560,   -148,  -344,  108,   -196,  228,  -288,
+    504,   560,   -328,  -88,   288,   -1008, 460,   -228,  468,   -836, -196,
+    76,    388,   232,   412,   -1168, -716,  -644,  756,   -172,  -356, -504,
+    116,   432,   528,   48,    476,   -168,  -608,  448,   160,   -532, -272,
+    28,    -676,  -12,   828,   980,   456,   520,   104,   -104,  256,  -344,
+    -4,    -28,   -368,  -52,   -524,  -572,  -556,  -200,  768,   1124, -208,
+    -512,  176,   232,   248,   -148,  -888,  604,   -600,  -304,  804,  -156,
+    -212,  488,   -192,  -804,  -256,  368,   -360,  -916,  -328,  228,  -240,
+    -448,  -472,  856,   -556,  -364,  572,   -12,   -156,  -368,  -340, 432,
+    252,   -752,  -152,  288,   268,   -580,  -848,  -592,  108,   -76,  244,
+    312,   -716,  592,   -80,   436,   360,   4,     -248,  160,   516,  584,
+    732,   44,    -468,  -280,  -292,  -156,  -588,  28,    308,   912,  24,
+    124,   156,   180,   -252,  944,   -924,  -772,  -520,  -428,  -624, 300,
+    -212,  -1144, 32,    -724,  800,   -1128, -212,  -1288, -848,  180,  -416,
+    440,   192,   -576,  -792,  -76,   -1080, 80,    -532,  -352,  -132, 380,
+    -820,  148,   1112,  128,   164,   456,   700,   -924,  144,   -668, -384,
+    648,   -832,  508,   552,   -52,   -100,  -656,  208,   -568,  748,  -88,
+    680,   232,   300,   192,   -408,  -1012, -152,  -252,  -268,  272,  -876,
+    -664,  -648,  -332,  -136,  16,    12,    1152,  -28,   332,   -536, 320,
+    -672,  -460,  -316,  532,   -260,  228,   -40,   1052,  -816,  180,  88,
+    -496,  -556,  -672,  -368,  428,   92,    356,   404,   -408,  252,  196,
+    -176,  -556,  792,   268,   32,    372,   40,    96,    -332,  328,  120,
+    372,   -900,  -40,   472,   -264,  -592,  952,   128,   656,   112,  664,
+    -232,  420,   4,     -344,  -464,  556,   244,   -416,  -32,   252,  0,
+    -412,  188,   -696,  508,   -476,  324,   -1096, 656,   -312,  560,  264,
+    -136,  304,   160,   -64,   -580,  248,   336,   -720,  560,   -348, -288,
+    -276,  -196,  -500,  852,   -544,  -236,  -1128, -992,  -776,  116,  56,
+    52,    860,   884,   212,   -12,   168,   1020,  512,   -552,  924,  -148,
+    716,   188,   164,   -340,  -520,  -184,  880,   -152,  -680,  -208, -1156,
+    -300,  -528,  -472,  364,   100,   -744,  -1056, -32,   540,   280,  144,
+    -676,  -32,   -232,  -280,  -224,  96,    568,   -76,   172,   148,  148,
+    104,   32,    -296,  -32,   788,   -80,   32,    -16,   280,   288,  944,
+    428,   -484
+};
diff --git a/libavcodec/aom_film_grain.h b/libavcodec/aom_film_grain.h
new file mode 100644
index 00000000000..5d772bd7d17
--- /dev/null
+++ b/libavcodec/aom_film_grain.h
@@ -0,0 +1,38 @@
+/*
+ * AOM film grain synthesis
+ * Copyright (c) 2021 Niklas Haas <ffmpeg@haasn.xyz>
+ *
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with FFmpeg; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+/**
+ * @file
+ * AOM film grain synthesis.
+ * @author Niklas Haas <ffmpeg@haasn.xyz>
+ */
+
+#ifndef AVCODEC_AOM_FILM_GRAIN_H
+#define AVCODEC_AOM_FILM_GRAIN_H
+
+#include "libavutil/film_grain_params.h"
+
+// Synthesizes film grain on top of `in` and stores the result to `out`. `out`
+// must already have been allocated and set to the same size and format as `in`.
+int ff_aom_apply_film_grain(AVFrame *out, const AVFrame *in,
+                            const AVFilmGrainParams *params);
+
+#endif /* AVCODEC_AOM_FILM_GRAIN_H */
diff --git a/libavcodec/aom_film_grain_template.c b/libavcodec/aom_film_grain_template.c
new file mode 100644
index 00000000000..5f9f29f1fab
--- /dev/null
+++ b/libavcodec/aom_film_grain_template.c
@@ -0,0 +1,577 @@
+/*
+ * AOM film grain synthesis
+ * Copyright (c) 2023 Niklas Haas <ffmpeg@haasn.xyz>
+ *
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with FFmpeg; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+/*
+ * Copyright © 2018, Niklas Haas
+ * Copyright © 2018, VideoLAN and dav1d authors
+ * Copyright © 2018, Two Orioles, LLC
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions are met:
+ *
+ * 1. Redistributions of source code must retain the above copyright notice, this
+ *    list of conditions and the following disclaimer.
+ *
+ * 2. Redistributions in binary form must reproduce the above copyright notice,
+ *    this list of conditions and the following disclaimer in the documentation
+ *    and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+ * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+ * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR
+ * ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
+ * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+ * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
+ * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
+ * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include "bit_depth_template.c"
+
+#undef entry
+#undef bitdepth
+#undef bitdepth_max
+#undef HBD_DECL
+#undef HBD_CALL
+#undef SCALING_SIZE
+
+#if BIT_DEPTH > 8
+# define entry int16_t
+# define bitdepth_max ((1 << bitdepth) - 1)
+# define HBD_DECL , const int bitdepth
+# define HBD_CALL , bitdepth
+# define SCALING_SIZE 4096
+#else
+# define entry int8_t
+# define bitdepth 8
+# define bitdepth_max UINT8_MAX
+# define HBD_DECL
+# define HBD_CALL
+# define SCALING_SIZE 256
+#endif
+
+static void FUNC(generate_grain_y_c)(entry buf[][GRAIN_WIDTH],
+                                     const AVFilmGrainParams *const params
+                                     HBD_DECL)
+{
+    const AVFilmGrainAOMParams *const data = &params->codec.aom;
+    const int bitdepth_min_8 = bitdepth - 8;
+    unsigned seed = params->seed;
+    const int shift = 4 - bitdepth_min_8 + data->grain_scale_shift;
+    const int grain_ctr = 128 << bitdepth_min_8;
+    const int grain_min = -grain_ctr, grain_max = grain_ctr - 1;
+
+    const int ar_pad = 3;
+    const int ar_lag = data->ar_coeff_lag;
+
+    for (int y = 0; y < GRAIN_HEIGHT; y++) {
+        for (int x = 0; x < GRAIN_WIDTH; x++) {
+            const int value = get_random_number(11, &seed);
+            buf[y][x] = round2(gaussian_sequence[ value ], shift);
+        }
+    }
+
+    for (int y = ar_pad; y < GRAIN_HEIGHT; y++) {
+        for (int x = ar_pad; x < GRAIN_WIDTH - ar_pad; x++) {
+            const int8_t *coeff = data->ar_coeffs_y;
+            int sum = 0, grain;
+            for (int dy = -ar_lag; dy <= 0; dy++) {
+                for (int dx = -ar_lag; dx <= ar_lag; dx++) {
+                    if (!dx && !dy)
+                        break;
+                    sum += *(coeff++) * buf[y + dy][x + dx];
+                }
+            }
+
+            grain = buf[y][x] + round2(sum, data->ar_coeff_shift);
+            buf[y][x] = av_clip(grain, grain_min, grain_max);
+        }
+    }
+}
+
+static void
+FUNC(generate_grain_uv_c)(entry buf[][GRAIN_WIDTH],
+                          const entry buf_y[][GRAIN_WIDTH],
+                          const AVFilmGrainParams *const params, const intptr_t uv,
+                          const int subx, const int suby HBD_DECL)
+{
+    const AVFilmGrainAOMParams *const data = &params->codec.aom;
+    const int bitdepth_min_8 = bitdepth - 8;
+    unsigned seed = params->seed ^ (uv ? 0x49d8 : 0xb524);
+    const int shift = 4 - bitdepth_min_8 + data->grain_scale_shift;
+    const int grain_ctr = 128 << bitdepth_min_8;
+    const int grain_min = -grain_ctr, grain_max = grain_ctr - 1;
+
+    const int chromaW = subx ? SUB_GRAIN_WIDTH  : GRAIN_WIDTH;
+    const int chromaH = suby ? SUB_GRAIN_HEIGHT : GRAIN_HEIGHT;
+
+    const int ar_pad = 3;
+    const int ar_lag = data->ar_coeff_lag;
+
+    for (int y = 0; y < chromaH; y++) {
+        for (int x = 0; x < chromaW; x++) {
+            const int value = get_random_number(11, &seed);
+            buf[y][x] = round2(gaussian_sequence[ value ], shift);
+        }
+    }
+
+    for (int y = ar_pad; y < chromaH; y++) {
+        for (int x = ar_pad; x < chromaW - ar_pad; x++) {
+            const int8_t *coeff = data->ar_coeffs_uv[uv];
+            int sum = 0, grain;
+            for (int dy = -ar_lag; dy <= 0; dy++) {
+                for (int dx = -ar_lag; dx <= ar_lag; dx++) {
+                    // For the final (current) pixel, we need to add in the
+                    // contribution from the luma grain texture
+                    if (!dx && !dy) {
+                        const int lumaX = ((x - ar_pad) << subx) + ar_pad;
+                        const int lumaY = ((y - ar_pad) << suby) + ar_pad;
+                        int luma = 0;
+                        if (!data->num_y_points)
+                            break;
+                        for (int i = 0; i <= suby; i++) {
+                            for (int j = 0; j <= subx; j++) {
+                                luma += buf_y[lumaY + i][lumaX + j];
+                            }
+                        }
+                        luma = round2(luma, subx + suby);
+                        sum += luma * (*coeff);
+                        break;
+                    }
+
+                    sum += *(coeff++) * buf[y + dy][x + dx];
+                }
+            }
+
+            grain = buf[y][x] + round2(sum, data->ar_coeff_shift);
+            buf[y][x] = av_clip(grain, grain_min, grain_max);
+        }
+    }
+}
+
+// samples from the correct block of a grain LUT, while taking into account the
+// offsets provided by the offsets cache
+static inline entry FUNC(sample_lut)(const entry grain_lut[][GRAIN_WIDTH],
+                                     const int offsets[2][2],
+                                     const int subx, const int suby,
+                                     const int bx, const int by,
+                                     const int x, const int y)
+{
+    const int randval = offsets[bx][by];
+    const int offx = 3 + (2 >> subx) * (3 + (randval >> 4));
+    const int offy = 3 + (2 >> suby) * (3 + (randval & 0xF));
+    return grain_lut[offy + y + (FG_BLOCK_SIZE >> suby) * by]
+                    [offx + x + (FG_BLOCK_SIZE >> subx) * bx];
+}
+
+static void FUNC(fgy_32x32xn_c)(pixel *const dst_row, const pixel *const src_row,
+                                const ptrdiff_t stride,
+                                const AVFilmGrainParams *const params, const size_t pw,
+                                const uint8_t scaling[SCALING_SIZE],
+                                const entry grain_lut[][GRAIN_WIDTH],
+                                const int bh, const int row_num HBD_DECL)
+{
+    const AVFilmGrainAOMParams *const data = &params->codec.aom;
+    const int rows = 1 + (data->overlap_flag && row_num > 0);
+    const int bitdepth_min_8 = bitdepth - 8;
+    const int grain_ctr = 128 << bitdepth_min_8;
+    const int grain_min = -grain_ctr, grain_max = grain_ctr - 1;
+    unsigned seed[2];
+    int offsets[2 /* col offset */][2 /* row offset */];
+
+    int min_value, max_value;
+    if (data->limit_output_range) {
+        min_value = 16 << bitdepth_min_8;
+        max_value = 235 << bitdepth_min_8;
+    } else {
+        min_value = 0;
+        max_value = bitdepth_max;
+    }
+
+    // seed[0] contains the current row, seed[1] contains the previous
+    for (int i = 0; i < rows; i++) {
+        seed[i] = params->seed;
+        seed[i] ^= (((row_num - i) * 37  + 178) & 0xFF) << 8;
+        seed[i] ^= (((row_num - i) * 173 + 105) & 0xFF);
+    }
+
+    av_assert1(stride % (FG_BLOCK_SIZE * sizeof(pixel)) == 0);
+
+    // process this row in FG_BLOCK_SIZE^2 blocks
+    for (unsigned bx = 0; bx < pw; bx += FG_BLOCK_SIZE) {
+        const int bw = FFMIN(FG_BLOCK_SIZE, (int) pw - bx);
+        const pixel *src;
+        pixel *dst;
+        int noise;
+
+        // x/y block offsets to compensate for overlapped regions
+        const int ystart = data->overlap_flag && row_num ? FFMIN(2, bh) : 0;
+        const int xstart = data->overlap_flag && bx      ? FFMIN(2, bw) : 0;
+
+        static const int w[2][2] = { { 27, 17 }, { 17, 27 } };
+
+        if (data->overlap_flag && bx) {
+            // shift previous offsets left
+            for (int i = 0; i < rows; i++)
+                offsets[1][i] = offsets[0][i];
+        }
+
+        // update current offsets
+        for (int i = 0; i < rows; i++)
+            offsets[0][i] = get_random_number(8, &seed[i]);
+
+#define add_noise_y(x, y, grain)                                                \
+        src = (const pixel*)((const char*)src_row + (y) * stride) + (x) + bx;   \
+        dst = (pixel*)((char*)dst_row + (y) * stride) + (x) + bx;               \
+        noise = round2(scaling[ *src ] * (grain), data->scaling_shift);         \
+        *dst = av_clip(*src + noise, min_value, max_value);
+
+        for (int y = ystart; y < bh; y++) {
+            // Non-overlapped image region (straightforward)
+            for (int x = xstart; x < bw; x++) {
+                int grain = FUNC(sample_lut)(grain_lut, offsets, 0, 0, 0, 0, x, y);
+                add_noise_y(x, y, grain);
+            }
+
+            // Special case for overlapped column
+            for (int x = 0; x < xstart; x++) {
+                int grain = FUNC(sample_lut)(grain_lut, offsets, 0, 0, 0, 0, x, y);
+                int old   = FUNC(sample_lut)(grain_lut, offsets, 0, 0, 1, 0, x, y);
+                grain = round2(old * w[x][0] + grain * w[x][1], 5);
+                grain = av_clip(grain, grain_min, grain_max);
+                add_noise_y(x, y, grain);
+            }
+        }
+
+        for (int y = 0; y < ystart; y++) {
+            // Special case for overlapped row (sans corner)
+            for (int x = xstart; x < bw; x++) {
+                int grain = FUNC(sample_lut)(grain_lut, offsets, 0, 0, 0, 0, x, y);
+                int old   = FUNC(sample_lut)(grain_lut, offsets, 0, 0, 0, 1, x, y);
+                grain = round2(old * w[y][0] + grain * w[y][1], 5);
+                grain = av_clip(grain, grain_min, grain_max);
+                add_noise_y(x, y, grain);
+            }
+
+            // Special case for doubly-overlapped corner
+            for (int x = 0; x < xstart; x++) {
+                int grain = FUNC(sample_lut)(grain_lut, offsets, 0, 0, 0, 0, x, y);
+                int top = FUNC(sample_lut)(grain_lut, offsets, 0, 0, 0, 1, x, y);
+                int old = FUNC(sample_lut)(grain_lut, offsets, 0, 0, 1, 1, x, y);
+
+                // Blend the top pixel with the top left block
+                top = round2(old * w[x][0] + top * w[x][1], 5);
+                top = av_clip(top, grain_min, grain_max);
+
+                // Blend the current pixel with the left block
+                old = FUNC(sample_lut)(grain_lut, offsets, 0, 0, 1, 0, x, y);
+                grain = round2(old * w[x][0] + grain * w[x][1], 5);
+                grain = av_clip(grain, grain_min, grain_max);
+
+                // Mix the row rows together and apply grain
+                grain = round2(top * w[y][0] + grain * w[y][1], 5);
+                grain = av_clip(grain, grain_min, grain_max);
+                add_noise_y(x, y, grain);
+            }
+        }
+    }
+}
+
+static void
+FUNC(fguv_32x32xn_c)(pixel *const dst_row, const pixel *const src_row,
+                     const ptrdiff_t stride, const AVFilmGrainParams *const params,
+                     const size_t pw, const uint8_t scaling[SCALING_SIZE],
+                     const entry grain_lut[][GRAIN_WIDTH], const int bh,
+                     const int row_num, const pixel *const luma_row,
+                     const ptrdiff_t luma_stride, const int uv, const int is_id,
+                     const int sx, const int sy HBD_DECL)
+{
+    const AVFilmGrainAOMParams *const data = &params->codec.aom;
+    const int rows = 1 + (data->overlap_flag && row_num > 0);
+    const int bitdepth_min_8 = bitdepth - 8;
+    const int grain_ctr = 128 << bitdepth_min_8;
+    const int grain_min = -grain_ctr, grain_max = grain_ctr - 1;
+    unsigned seed[2];
+    int offsets[2 /* col offset */][2 /* row offset */];
+
+    int min_value, max_value;
+    if (data->limit_output_range) {
+        min_value = 16 << bitdepth_min_8;
+        max_value = (is_id ? 235 : 240) << bitdepth_min_8;
+    } else {
+        min_value = 0;
+        max_value = bitdepth_max;
+    }
+
+    // seed[0] contains the current row, seed[1] contains the previous
+    for (int i = 0; i < rows; i++) {
+        seed[i] = params->seed;
+        seed[i] ^= (((row_num - i) * 37  + 178) & 0xFF) << 8;
+        seed[i] ^= (((row_num - i) * 173 + 105) & 0xFF);
+    }
+
+    av_assert1(stride % (FG_BLOCK_SIZE * sizeof(pixel)) == 0);
+
+    // process this row in FG_BLOCK_SIZE^2 blocks (subsampled)
+    for (unsigned bx = 0; bx < pw; bx += FG_BLOCK_SIZE >> sx) {
+        const int bw = FFMIN(FG_BLOCK_SIZE >> sx, (int)(pw - bx));
+        int val, lx, ly, noise;
+        const pixel *src, *luma;
+        pixel *dst, avg;
+
+        // x/y block offsets to compensate for overlapped regions
+        const int ystart = data->overlap_flag && row_num ? FFMIN(2 >> sy, bh) : 0;
+        const int xstart = data->overlap_flag && bx      ? FFMIN(2 >> sx, bw) : 0;
+
+        static const int w[2 /* sub */][2 /* off */][2] = {
+            { { 27, 17 }, { 17, 27 } },
+            { { 23, 22 } },
+        };
+
+        if (data->overlap_flag && bx) {
+            // shift previous offsets left
+            for (int i = 0; i < rows; i++)
+                offsets[1][i] = offsets[0][i];
+        }
+
+        // update current offsets
+        for (int i = 0; i < rows; i++)
+            offsets[0][i] = get_random_number(8, &seed[i]);
+
+#define add_noise_uv(x, y, grain)                                               \
+            lx = (bx + x) << sx;                                                \
+            ly = y << sy;                                                       \
+            luma = (const pixel*)((const char*)luma_row + ly * luma_stride) + lx;\
+            avg = luma[0];                                                      \
+            if (sx)                                                             \
+                avg = (avg + luma[1] + 1) >> 1;                                 \
+            src = (const pixel*)((const char *)src_row + (y) * stride) + bx + (x);\
+            dst = (pixel *) ((char *) dst_row + (y) * stride) + bx + (x);       \
+            val = avg;                                                          \
+            if (!data->chroma_scaling_from_luma) {                              \
+                const int combined = avg * data->uv_mult_luma[uv] +             \
+                                    *src * data->uv_mult[uv];                   \
+                val = av_clip( (combined >> 6) +                                \
+                               (data->uv_offset[uv] * (1 << bitdepth_min_8)),   \
+                               0, bitdepth_max );                               \
+            }                                                                   \
+            noise = round2(scaling[ val ] * (grain), data->scaling_shift);      \
+            *dst = av_clip(*src + noise, min_value, max_value);
+
+        for (int y = ystart; y < bh; y++) {
+            // Non-overlapped image region (straightforward)
+            for (int x = xstart; x < bw; x++) {
+                int grain = FUNC(sample_lut)(grain_lut, offsets, sx, sy, 0, 0, x, y);
+                add_noise_uv(x, y, grain);
+            }
+
+            // Special case for overlapped column
+            for (int x = 0; x < xstart; x++) {
+                int grain = FUNC(sample_lut)(grain_lut, offsets, sx, sy, 0, 0, x, y);
+                int old   = FUNC(sample_lut)(grain_lut, offsets, sx, sy, 1, 0, x, y);
+                grain = round2(old * w[sx][x][0] + grain * w[sx][x][1], 5);
+                grain = av_clip(grain, grain_min, grain_max);
+                add_noise_uv(x, y, grain);
+            }
+        }
+
+        for (int y = 0; y < ystart; y++) {
+            // Special case for overlapped row (sans corner)
+            for (int x = xstart; x < bw; x++) {
+                int grain = FUNC(sample_lut)(grain_lut, offsets, sx, sy, 0, 0, x, y);
+                int old   = FUNC(sample_lut)(grain_lut, offsets, sx, sy, 0, 1, x, y);
+                grain = round2(old * w[sy][y][0] + grain * w[sy][y][1], 5);
+                grain = av_clip(grain, grain_min, grain_max);
+                add_noise_uv(x, y, grain);
+            }
+
+            // Special case for doubly-overlapped corner
+            for (int x = 0; x < xstart; x++) {
+                int top = FUNC(sample_lut)(grain_lut, offsets, sx, sy, 0, 1, x, y);
+                int old = FUNC(sample_lut)(grain_lut, offsets, sx, sy, 1, 1, x, y);
+                int grain = FUNC(sample_lut)(grain_lut, offsets, sx, sy, 0, 0, x, y);
+
+                // Blend the top pixel with the top left block
+                top = round2(old * w[sx][x][0] + top * w[sx][x][1], 5);
+                top = av_clip(top, grain_min, grain_max);
+
+                // Blend the current pixel with the left block
+                old = FUNC(sample_lut)(grain_lut, offsets, sx, sy, 1, 0, x, y);
+                grain = round2(old * w[sx][x][0] + grain * w[sx][x][1], 5);
+                grain = av_clip(grain, grain_min, grain_max);
+
+                // Mix the row rows together and apply to image
+                grain = round2(top * w[sy][y][0] + grain * w[sy][y][1], 5);
+                grain = av_clip(grain, grain_min, grain_max);
+                add_noise_uv(x, y, grain);
+            }
+        }
+    }
+}
+
+static void FUNC(generate_scaling)(const uint8_t points[][2], const int num,
+                                   uint8_t scaling[SCALING_SIZE] HBD_DECL)
+{
+    const int shift_x = bitdepth - 8;
+    const int scaling_size = 1 << bitdepth;
+    const int max_value = points[num - 1][0] << shift_x;
+    av_assert0(scaling_size <= SCALING_SIZE);
+
+    if (num == 0) {
+        memset(scaling, 0, scaling_size);
+        return;
+    }
+
+    // Fill up the preceding entries with the initial value
+    memset(scaling, points[0][1], points[0][0] << shift_x);
+
+    // Linearly interpolate the values in the middle
+    for (int i = 0; i < num - 1; i++) {
+        const int bx = points[i][0];
+        const int by = points[i][1];
+        const int ex = points[i+1][0];
+        const int ey = points[i+1][1];
+        const int dx = ex - bx;
+        const int dy = ey - by;
+        const int delta = dy * ((0x10000 + (dx >> 1)) / dx);
+        av_assert1(dx > 0);
+        for (int x = 0, d = 0x8000; x < dx; x++) {
+            scaling[(bx + x) << shift_x] = by + (d >> 16);
+            d += delta;
+        }
+    }
+
+    // Fill up the remaining entries with the final value
+    memset(&scaling[max_value], points[num - 1][1], scaling_size - max_value);
+
+#if BIT_DEPTH != 8
+    for (int i = 0; i < num - 1; i++) {
+        const int pad = 1 << shift_x, rnd = pad >> 1;
+        const int bx = points[i][0] << shift_x;
+        const int ex = points[i+1][0] << shift_x;
+        const int dx = ex - bx;
+        for (int x = 0; x < dx; x += pad) {
+            const int range = scaling[bx + x + pad] - scaling[bx + x];
+            for (int n = 1, r = rnd; n < pad; n++) {
+                r += range;
+                scaling[bx + x + n] = scaling[bx + x] + (r >> shift_x);
+            }
+        }
+    }
+#endif
+}
+
+static av_always_inline void
+FUNC(apply_grain_row)(AVFrame *out, const AVFrame *in,
+                      const int ss_x, const int ss_y,
+                      const uint8_t scaling[3][SCALING_SIZE],
+                      const entry grain_lut[3][GRAIN_HEIGHT+1][GRAIN_WIDTH],
+                      const AVFilmGrainParams *params,
+                      const int row HBD_DECL)
+{
+    // Synthesize grain for the affected planes
+    const AVFilmGrainAOMParams *const data = &params->codec.aom;
+    const int cpw = (out->width + ss_x) >> ss_x;
+    const int is_id = out->colorspace == AVCOL_SPC_RGB;
+    const int bh = (FFMIN(out->height - row * FG_BLOCK_SIZE, FG_BLOCK_SIZE) + ss_y) >> ss_y;
+    const ptrdiff_t uv_off = row * FG_BLOCK_SIZE * out->linesize[1] >> ss_y;
+    pixel *const luma_src = (pixel *)
+        ((char *) in->data[0] + row * FG_BLOCK_SIZE * in->linesize[0]);
+
+    if (data->num_y_points) {
+        const int bh = FFMIN(out->height - row * FG_BLOCK_SIZE, FG_BLOCK_SIZE);
+        const ptrdiff_t off = row * FG_BLOCK_SIZE * out->linesize[0];
+        FUNC(fgy_32x32xn_c)((pixel *) ((char *) out->data[0] + off), luma_src,
+                            out->linesize[0], params, out->width, scaling[0],
+                            grain_lut[0], bh, row HBD_CALL);
+    }
+
+    if (!data->num_uv_points[0] && !data->num_uv_points[1] &&
+        !data->chroma_scaling_from_luma)
+    {
+        return;
+    }
+
+    // extend padding pixels
+    if (out->width & ss_x) {
+        pixel *ptr = luma_src;
+        for (int y = 0; y < bh; y++) {
+            ptr[out->width] = ptr[out->width - 1];
+            ptr = (pixel *) ((char *) ptr + (in->linesize[0] << ss_y));
+        }
+    }
+
+    if (data->chroma_scaling_from_luma) {
+        for (int pl = 0; pl < 2; pl++)
+            FUNC(fguv_32x32xn_c)((pixel *) ((char *) out->data[1 + pl] + uv_off),
+                                 (const pixel *) ((const char *) in->data[1 + pl] + uv_off),
+                                 in->linesize[1], params, cpw, scaling[0],
+                                 grain_lut[1 + pl], bh, row, luma_src,
+                                 in->linesize[0], pl, is_id, ss_x, ss_y HBD_CALL);
+    } else {
+        for (int pl = 0; pl < 2; pl++) {
+            if (data->num_uv_points[pl]) {
+                FUNC(fguv_32x32xn_c)((pixel *) ((char *) out->data[1 + pl] + uv_off),
+                                     (const pixel *) ((const char *) in->data[1 + pl] + uv_off),
+                                     in->linesize[1], params, cpw, scaling[1 + pl],
+                                     grain_lut[1 + pl], bh, row, luma_src,
+                                     in->linesize[0], pl, is_id, ss_x, ss_y HBD_CALL);
+            }
+        }
+    }
+}
+
+static int FUNC(apply_film_grain)(AVFrame *out_frame, const AVFrame *in_frame,
+                                  const AVFilmGrainParams *params HBD_DECL)
+{
+    entry grain_lut[3][GRAIN_HEIGHT + 1][GRAIN_WIDTH];
+    uint8_t scaling[3][SCALING_SIZE];
+
+    const AVFilmGrainAOMParams *const data = &params->codec.aom;
+    const AVPixFmtDescriptor *desc = av_pix_fmt_desc_get(out_frame->format);
+    const int rows = AV_CEIL_RSHIFT(out_frame->height, 5); /* log2(FG_BLOCK_SIZE) */
+    const int subx = desc->log2_chroma_w, suby = desc->log2_chroma_h;
+
+    // Generate grain LUTs as needed
+    FUNC(generate_grain_y_c)(grain_lut[0], params HBD_CALL);
+    if (data->num_uv_points[0] || data->chroma_scaling_from_luma)
+        FUNC(generate_grain_uv_c)(grain_lut[1], grain_lut[0], params, 0, subx, suby HBD_CALL);
+    if (data->num_uv_points[1] || data->chroma_scaling_from_luma)
+        FUNC(generate_grain_uv_c)(grain_lut[2], grain_lut[0], params, 1, subx, suby HBD_CALL);
+
+    // Generate scaling LUTs as needed
+    if (data->num_y_points || data->chroma_scaling_from_luma)
+        FUNC(generate_scaling)(data->y_points, data->num_y_points, scaling[0] HBD_CALL);
+    if (data->num_uv_points[0])
+        FUNC(generate_scaling)(data->uv_points[0], data->num_uv_points[0], scaling[1] HBD_CALL);
+    if (data->num_uv_points[1])
+        FUNC(generate_scaling)(data->uv_points[1], data->num_uv_points[1], scaling[2] HBD_CALL);
+
+    for (int row = 0; row < rows; row++) {
+        FUNC(apply_grain_row)(out_frame, in_frame, subx, suby, scaling, grain_lut,
+                              params, row HBD_CALL);
+    }
+
+    return 0;
+}
-- 
2.44.0

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH v4 11/13] avcodec/aom_film_grain: implement AFGS1 parsing

[Niklas Haas]

From: Niklas Haas <git@haasn.dev>

Based on the AOMedia Film Grain Synthesis 1 (AFGS1) spec:
  https://aomediacodec.github.io/afgs1-spec/

The parsing has been changed substantially relative to the AV1 film
grain OBU. In particular:

1. There is the possibility of maintaining multiple independent film
   grain parameter sets, and decoders/players are recommended to pick
   the one most appropriate for the intended display resolution. This
   could also be used to e.g. switch between different grain profiles
   without having to re-signal the appropriate coefficients.

2. Supporting this, it's possible to *predict* the grain coefficients
   from previously signalled parameter sets, transmitting only the
   residual.

3. When not predicting, the parameter sets are now stored as a series of
   increments, rather than being directly transmitted.

4. There are several new AFGS1-exclusive fields.

I placed this parser in its own file, rather than h2645_sei.c, since
nothing in the generic AFGS1 film grain payload is specific to T.35, and
to compartmentalize the code base.
---
 libavcodec/aom_film_grain.c | 238 ++++++++++++++++++++++++++++++++++++
 libavcodec/aom_film_grain.h |  13 ++
 2 files changed, 251 insertions(+)

diff --git a/libavcodec/aom_film_grain.c b/libavcodec/aom_film_grain.c
index ffcd71b584b..e302567ba5f 100644
--- a/libavcodec/aom_film_grain.c
+++ b/libavcodec/aom_film_grain.c
@@ -29,6 +29,7 @@
 #include "libavutil/imgutils.h"
 
 #include "aom_film_grain.h"
+#include "get_bits.h"
 
 // Common/shared helpers (not dependent on BIT_DEPTH)
 static inline int get_random_number(const int bits, unsigned *const state) {
@@ -118,6 +119,243 @@ int ff_aom_apply_film_grain(AVFrame *out, const AVFrame *in,
     return AVERROR_INVALIDDATA;
 }
 
+int ff_aom_parse_film_grain_sets(AVFilmGrainAFGS1Params *s,
+                                 const uint8_t *payload, int payload_size)
+{
+    GetBitContext gbc, *gb = &gbc;
+    AVFilmGrainAOMParams *aom;
+    AVFilmGrainParams *fgp, *ref = NULL;
+    int ret, num_sets, n, i, uv, num_y_coeffs, update_grain, luma_only;
+
+    ret = init_get_bits8(gb, payload, payload_size);
+    if (ret < 0)
+        return ret;
+
+    s->enable = get_bits1(gb);
+    if (!s->enable)
+        return 0;
+
+    skip_bits(gb, 4); // reserved
+    num_sets = get_bits(gb, 3) + 1;
+    for (n = 0; n < num_sets; n++) {
+        int payload_4byte, payload_size, set_idx, apply_units_log2, vsc_flag;
+        int predict_scaling, predict_y_scaling, predict_uv_scaling[2];
+        int payload_bits, start_position;
+
+        start_position = get_bits_count(gb);
+        payload_4byte = get_bits1(gb);
+        payload_size = get_bits(gb, payload_4byte ? 2 : 8);
+        set_idx = get_bits(gb, 3);
+        fgp = &s->sets[set_idx];
+        aom = &fgp->codec.aom;
+
+        fgp->type = get_bits1(gb) ? AV_FILM_GRAIN_PARAMS_AV1 : AV_FILM_GRAIN_PARAMS_NONE;
+        if (!fgp->type)
+            continue;
+
+        fgp->seed = get_bits(gb, 16);
+        update_grain = get_bits1(gb);
+        if (!update_grain)
+            continue;
+
+        apply_units_log2  = get_bits(gb, 4);
+        fgp->width  = get_bits(gb, 12) << apply_units_log2;
+        fgp->height = get_bits(gb, 12) << apply_units_log2;
+        luma_only = get_bits1(gb);
+        if (luma_only) {
+            fgp->subsampling_x = fgp->subsampling_y = 0;
+        } else {
+            fgp->subsampling_x = get_bits1(gb);
+            fgp->subsampling_y = get_bits1(gb);
+        }
+
+        fgp->bit_depth_luma  = fgp->bit_depth_chroma = 0;
+        fgp->color_primaries = AVCOL_PRI_UNSPECIFIED;
+        fgp->color_trc       = AVCOL_TRC_UNSPECIFIED;
+        fgp->color_space     = AVCOL_SPC_UNSPECIFIED;
+        fgp->color_range     = AVCOL_RANGE_UNSPECIFIED;
+
+        vsc_flag = get_bits1(gb); // video_signal_characteristics_flag
+        if (vsc_flag) {
+            int cicp_flag;
+            fgp->bit_depth_luma = get_bits(gb, 3) + 8;
+            if (!luma_only)
+                fgp->bit_depth_chroma = fgp->bit_depth_luma;
+            cicp_flag = get_bits1(gb);
+            if (cicp_flag) {
+                fgp->color_primaries = get_bits(gb, 8);
+                fgp->color_trc = get_bits(gb, 8);
+                fgp->color_space = get_bits(gb, 8);
+                fgp->color_range = get_bits1(gb) ? AVCOL_RANGE_JPEG : AVCOL_RANGE_MPEG;
+                if (fgp->color_primaries > AVCOL_PRI_NB ||
+                    fgp->color_primaries == AVCOL_PRI_RESERVED ||
+                    fgp->color_primaries == AVCOL_PRI_RESERVED0 ||
+                    fgp->color_trc > AVCOL_TRC_NB ||
+                    fgp->color_trc == AVCOL_TRC_RESERVED ||
+                    fgp->color_trc == AVCOL_TRC_RESERVED0 ||
+                    fgp->color_space > AVCOL_SPC_NB ||
+                    fgp->color_space == AVCOL_SPC_RESERVED)
+                    goto error;
+            }
+        }
+
+        predict_scaling = get_bits1(gb);
+        if (predict_scaling && (!ref || ref == fgp))
+            goto error; // prediction must be from valid, different set
+
+        predict_y_scaling = predict_scaling ? get_bits1(gb) : 0;
+        if (predict_y_scaling) {
+            int y_scale, y_offset, bits_res;
+            y_scale = get_bits(gb, 9) - 256;
+            y_offset = get_bits(gb, 9) - 256;
+            bits_res = get_bits(gb, 3);
+            if (bits_res) {
+                int res[14], pred, granularity;
+                aom->num_y_points = ref->codec.aom.num_y_points;
+                for (i = 0; i < aom->num_y_points; i++)
+                    res[i] = get_bits(gb, bits_res);
+                granularity = get_bits(gb, 3);
+                for (i = 0; i < aom->num_y_points; i++) {
+                    pred = ref->codec.aom.y_points[i][1];
+                    pred = ((pred * y_scale + 8) >> 4) + y_offset;
+                    pred += (res[i] - (1 << (bits_res - 1))) * granularity;
+                    aom->y_points[i][0] = ref->codec.aom.y_points[i][0];
+                    aom->y_points[i][1] = av_clip_uint8(pred);
+                }
+            }
+        } else {
+            aom->num_y_points = get_bits(gb, 4);
+            if (aom->num_y_points > 14) {
+                goto error;
+            } else if (aom->num_y_points) {
+                int bits_inc, bits_scaling;
+                int y_value = 0;
+                bits_inc = get_bits(gb, 3) + 1;
+                bits_scaling = get_bits(gb, 2) + 5;
+                for (i = 0; i < aom->num_y_points; i++) {
+                    y_value += get_bits(gb, bits_inc);
+                    if (y_value > UINT8_MAX)
+                        goto error;
+                    aom->y_points[i][0] = y_value;
+                    aom->y_points[i][1] = get_bits(gb, bits_scaling);
+                }
+            }
+        }
+
+        if (luma_only) {
+            aom->chroma_scaling_from_luma = 0;
+            aom->num_uv_points[0] = aom->num_uv_points[1] = 0;
+        } else {
+            aom->chroma_scaling_from_luma = get_bits1(gb);
+            if (aom->chroma_scaling_from_luma) {
+                aom->num_uv_points[0] = aom->num_uv_points[1] = 0;
+            } else {
+                for (uv = 0; uv < 2; uv++) {
+                    predict_uv_scaling[uv] = predict_scaling ? get_bits1(gb) : 0;
+                    if (predict_uv_scaling[uv]) {
+                        int uv_scale, uv_offset, bits_res;
+                        uv_scale = get_bits(gb, 9) - 256;
+                        uv_offset = get_bits(gb, 9) - 256;
+                        bits_res = get_bits(gb, 3);
+                        aom->uv_mult[uv] = ref->codec.aom.uv_mult[uv];
+                        aom->uv_mult_luma[uv] = ref->codec.aom.uv_mult_luma[uv];
+                        aom->uv_offset[uv] = ref->codec.aom.uv_offset[uv];
+                        if (bits_res) {
+                            int res[10], pred, granularity;
+                            aom->num_uv_points[uv] = ref->codec.aom.num_uv_points[uv];
+                            for (i = 0; i < aom->num_uv_points[uv]; i++)
+                                res[i] = get_bits(gb, bits_res);
+                            granularity = get_bits(gb, 3);
+                            for (i = 0; i < aom->num_uv_points[uv]; i++) {
+                                pred = ref->codec.aom.uv_points[uv][i][1];
+                                pred = ((pred * uv_scale + 8) >> 4) + uv_offset;
+                                pred += (res[i] - (1 << (bits_res - 1))) * granularity;
+                                aom->uv_points[uv][i][0] = ref->codec.aom.uv_points[uv][i][0];
+                                aom->uv_points[uv][i][1] = av_clip_uint8(pred);
+                            }
+                        }
+                    } else {
+                        int bits_inc, bits_scaling, uv_offset;
+                        int uv_value = 0;
+                        aom->num_uv_points[uv] = get_bits(gb, 4);
+                        if (aom->num_uv_points[uv] > 10)
+                            goto error;
+                        bits_inc = get_bits(gb, 3) + 1;
+                        bits_scaling = get_bits(gb, 2) + 5;
+                        uv_offset = get_bits(gb, 8);
+                        for (i = 0; i < aom->num_uv_points[uv]; i++) {
+                            uv_value += get_bits(gb, bits_inc);
+                            if (uv_value > UINT8_MAX)
+                                goto error;
+                            aom->uv_points[uv][i][0] = uv_value;
+                            aom->uv_points[uv][i][1] = get_bits(gb, bits_scaling) + uv_offset;
+                        }
+                    }
+                }
+            }
+        }
+
+        aom->scaling_shift = get_bits(gb, 2) + 8;
+        aom->ar_coeff_lag = get_bits(gb, 2);
+        num_y_coeffs = 2 * aom->ar_coeff_lag * (aom->ar_coeff_lag + 1);
+        if (aom->num_y_points) {
+            int ar_bits = get_bits(gb, 2) + 5;
+            for (i = 0; i < num_y_coeffs; i++)
+                aom->ar_coeffs_y[i] = get_bits(gb, ar_bits) - (1 << (ar_bits - 1));
+        }
+        for (uv = 0; uv < 2; uv++) {
+            if (aom->chroma_scaling_from_luma || aom->num_uv_points[uv]) {
+                int ar_bits = get_bits(gb, 2) + 5;
+                for (i = 0; i < num_y_coeffs + !!aom->num_y_points; i++)
+                    aom->ar_coeffs_uv[uv][i] = get_bits(gb, ar_bits) - (1 << (ar_bits - 1));
+            }
+        }
+        aom->ar_coeff_shift = get_bits(gb, 2) + 6;
+        aom->grain_scale_shift = get_bits(gb, 2);
+        for (uv = 0; uv < 2; uv++) {
+            if (aom->num_uv_points[uv] && !predict_uv_scaling[uv]) {
+                aom->uv_mult[uv]      = get_bits(gb, 8) - 128;
+                aom->uv_mult_luma[uv] = get_bits(gb, 8) - 128;
+                aom->uv_offset[uv]    = get_bits(gb, 9) - 256;
+            }
+        }
+        aom->overlap_flag = get_bits1(gb);
+        aom->limit_output_range = get_bits1(gb);
+
+        // use first set as reference only if it was fully transmitted
+        if (n == 0)
+            ref = fgp;
+
+        payload_bits = get_bits_count(gb) - start_position;
+        if (payload_bits > payload_size * 8)
+            goto error;
+        skip_bits(gb, payload_size * 8 - payload_bits);
+    }
+    return 0;
+
+error:
+    memset(s, 0, sizeof(*s));
+    return AVERROR_INVALIDDATA;
+}
+
+int ff_aom_attach_film_grain_sets(const AVFilmGrainAFGS1Params *s, AVFrame *frame)
+{
+    AVFilmGrainParams *fgp;
+    if (!s->enable)
+        return 0;
+
+    for (int i = 0; i < FF_ARRAY_ELEMS(s->sets); i++) {
+        if (s->sets[i].type != AV_FILM_GRAIN_PARAMS_AV1)
+            continue;
+        fgp = av_film_grain_params_create_side_data(frame);
+        if (!fgp)
+            return AVERROR(ENOMEM);
+        memcpy(fgp, &s->sets[i], sizeof(*fgp));
+    }
+
+    return 0;
+}
+
 // Taken from the AV1 spec. Range is [-2048, 2047], mean is 0 and stddev is 512
 static const int16_t gaussian_sequence[2048] = {
     56,    568,   -180,  172,   124,   -84,   172,   -64,   -900,  24,   820,
diff --git a/libavcodec/aom_film_grain.h b/libavcodec/aom_film_grain.h
index 5d772bd7d17..1f8c78f657a 100644
--- a/libavcodec/aom_film_grain.h
+++ b/libavcodec/aom_film_grain.h
@@ -30,9 +30,22 @@
 
 #include "libavutil/film_grain_params.h"
 
+typedef struct AVFilmGrainAFGS1Params {
+    int enable;
+    AVFilmGrainParams sets[8];
+} AVFilmGrainAFGS1Params;
+
 // Synthesizes film grain on top of `in` and stores the result to `out`. `out`
 // must already have been allocated and set to the same size and format as `in`.
 int ff_aom_apply_film_grain(AVFrame *out, const AVFrame *in,
                             const AVFilmGrainParams *params);
 
+// Parse AFGS1 parameter sets from an ITU-T T.35 payload. Returns 0 on success,
+// or a negative error code.
+int ff_aom_parse_film_grain_sets(AVFilmGrainAFGS1Params *s,
+                                 const uint8_t *payload, int payload_size);
+
+// Attach all valid film grain param sets to `frame`.
+int ff_aom_attach_film_grain_sets(const AVFilmGrainAFGS1Params *s, AVFrame *frame);
+
 #endif /* AVCODEC_AOM_FILM_GRAIN_H */
-- 
2.44.0

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH v4 12/13] avcodec/h2645_sei: decode AFGS1 T.35 SEI

[Niklas Haas]

From: Niklas Haas <git@haasn.dev>

I restricted this SEI to HEVC for now, until I see a H.264 sample.
---
 libavcodec/Makefile    |  2 +-
 libavcodec/h2645_sei.c | 25 +++++++++++++++++++++++++
 libavcodec/h2645_sei.h |  3 +++
 3 files changed, 29 insertions(+), 1 deletion(-)

diff --git a/libavcodec/Makefile b/libavcodec/Makefile
index 708434ac76c..824845276ae 100644
--- a/libavcodec/Makefile
+++ b/libavcodec/Makefile
@@ -105,7 +105,7 @@ OBJS-$(CONFIG_H264_SEI)                += h264_sei.o h2645_sei.o
 OBJS-$(CONFIG_HEVCPARSE)               += hevc_parse.o hevc_ps.o hevc_data.o \
                                           h2645data.o h2645_parse.o h2645_vui.o
 OBJS-$(CONFIG_HEVC_SEI)                += hevc_sei.o h2645_sei.o \
-                                          dynamic_hdr_vivid.o
+                                          dynamic_hdr_vivid.o aom_film_grain.o
 OBJS-$(CONFIG_HPELDSP)                 += hpeldsp.o
 OBJS-$(CONFIG_HUFFMAN)                 += huffman.o
 OBJS-$(CONFIG_HUFFYUVDSP)              += huffyuvdsp.o
diff --git a/libavcodec/h2645_sei.c b/libavcodec/h2645_sei.c
index 73f894436fd..ea6d596ba39 100644
--- a/libavcodec/h2645_sei.c
+++ b/libavcodec/h2645_sei.c
@@ -209,6 +209,24 @@ static int decode_registered_user_data(H2645SEI *h, GetByteContext *gb,
         }
         break;
     }
+    case 0x5890: { // aom_provider_code
+        const uint16_t aom_grain_provider_oriented_code = 0x0001;
+        uint16_t provider_oriented_code;
+
+        if (!IS_HEVC(codec_id))
+            goto unsupported_provider_code;
+
+        if (bytestream2_get_bytes_left(gb) < 2)
+            return AVERROR_INVALIDDATA;
+
+        provider_oriented_code = bytestream2_get_byteu(gb);
+        if (provider_oriented_code == aom_grain_provider_oriented_code) {
+            return ff_aom_parse_film_grain_sets(&h->aom_film_grain,
+                                                gb->buffer,
+                                                bytestream2_get_bytes_left(gb));
+        }
+        break;
+    }
     unsupported_provider_code:
 #endif
     default:
@@ -702,6 +720,12 @@ FF_ENABLE_DEPRECATION_WARNINGS
             avctx->properties |= FF_CODEC_PROPERTY_FILM_GRAIN;
     }
 
+#if CONFIG_HEVC_SEI
+    ret = ff_aom_attach_film_grain_sets(&sei->aom_film_grain, frame);
+    if (ret < 0)
+        return ret;
+#endif
+
     if (sei->ambient_viewing_environment.present) {
         H2645SEIAmbientViewingEnvironment *env =
             &sei->ambient_viewing_environment;
@@ -798,4 +822,5 @@ void ff_h2645_sei_reset(H2645SEI *s)
     s->ambient_viewing_environment.present = 0;
     s->mastering_display.present = 0;
     s->content_light.present = 0;
+    s->aom_film_grain.enable = 0;
 }
diff --git a/libavcodec/h2645_sei.h b/libavcodec/h2645_sei.h
index 0ebf48011af..b9a6c7587b1 100644
--- a/libavcodec/h2645_sei.h
+++ b/libavcodec/h2645_sei.h
@@ -23,7 +23,9 @@
 
 #include "libavutil/buffer.h"
 #include "libavutil/frame.h"
+#include "libavutil/film_grain_params.h"
 
+#include "aom_film_grain.h"
 #include "avcodec.h"
 #include "bytestream.h"
 #include "codec_id.h"
@@ -132,6 +134,7 @@ typedef struct H2645SEI {
     H2645SEIAmbientViewingEnvironment ambient_viewing_environment;
     H2645SEIMasteringDisplay mastering_display;
     H2645SEIContentLight content_light;
+    AVFilmGrainAFGS1Params aom_film_grain;
 } H2645SEI;
 
 enum {
-- 
2.44.0

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH v4 13/13] avcodec/hevcdec: apply AOM film grain synthesis

[Niklas Haas]

From: Niklas Haas <git@haasn.dev>

Following the usual logic for H.274 film grain.
---
 libavcodec/Makefile  |  2 +-
 libavcodec/hevcdec.c | 29 ++++++++++++++++++++++-------
 2 files changed, 23 insertions(+), 8 deletions(-)

diff --git a/libavcodec/Makefile b/libavcodec/Makefile
index 824845276ae..7ef2e03ca6a 100644
--- a/libavcodec/Makefile
+++ b/libavcodec/Makefile
@@ -432,7 +432,7 @@ OBJS-$(CONFIG_HDR_ENCODER)             += hdrenc.o
 OBJS-$(CONFIG_HEVC_DECODER)            += hevcdec.o hevc_mvs.o \
                                           hevc_cabac.o hevc_refs.o hevcpred.o    \
                                           hevcdsp.o hevc_filter.o hevc_data.o \
-                                          h274.o
+                                          h274.o aom_film_grain.o
 OBJS-$(CONFIG_HEVC_AMF_ENCODER)        += amfenc_hevc.o
 OBJS-$(CONFIG_HEVC_CUVID_DECODER)      += cuviddec.o
 OBJS-$(CONFIG_HEVC_MEDIACODEC_DECODER) += mediacodecdec.o
diff --git a/libavcodec/hevcdec.c b/libavcodec/hevcdec.c
index 76aa6b45882..575836e340c 100644
--- a/libavcodec/hevcdec.c
+++ b/libavcodec/hevcdec.c
@@ -35,6 +35,7 @@
 #include "libavutil/pixdesc.h"
 #include "libavutil/timecode.h"
 
+#include "aom_film_grain.h"
 #include "bswapdsp.h"
 #include "cabac_functions.h"
 #include "codec_internal.h"
@@ -388,7 +389,8 @@ static int export_stream_params_from_sei(HEVCContext *s)
         avctx->color_trc = s->sei.common.alternative_transfer.preferred_transfer_characteristics;
     }
 
-    if (s->sei.common.film_grain_characteristics.present)
+    if (s->sei.common.film_grain_characteristics.present ||
+        s->sei.common.aom_film_grain.enable)
         avctx->properties |= FF_CODEC_PROPERTY_FILM_GRAIN;
 
     return 0;
@@ -2885,11 +2887,13 @@ static int hevc_frame_start(HEVCContext *s)
     else
         s->ref->frame->flags &= ~AV_FRAME_FLAG_KEY;
 
-    s->ref->needs_fg = s->sei.common.film_grain_characteristics.present &&
+    s->ref->needs_fg = (s->sei.common.film_grain_characteristics.present ||
+                        s->sei.common.aom_film_grain.enable) &&
         !(s->avctx->export_side_data & AV_CODEC_EXPORT_DATA_FILM_GRAIN) &&
         !s->avctx->hwaccel;
 
     if (s->ref->needs_fg &&
+        s->sei.common.film_grain_characteristics.present &&
         !ff_h274_film_grain_params_supported(s->sei.common.film_grain_characteristics.model_id,
                                              s->ref->frame->format)) {
         av_log_once(s->avctx, AV_LOG_WARNING, AV_LOG_DEBUG, &s->film_grain_warning_shown,
@@ -2934,14 +2938,24 @@ fail:
 static int hevc_frame_end(HEVCContext *s)
 {
     HEVCFrame *out = s->ref;
-    const AVFrameSideData *sd;
+    const AVFilmGrainParams *fgp;
     av_unused int ret;
 
     if (out->needs_fg) {
-        sd = av_frame_get_side_data(out->frame, AV_FRAME_DATA_FILM_GRAIN_PARAMS);
-        av_assert0(out->frame_grain->buf[0] && sd);
-        ret = ff_h274_apply_film_grain(out->frame_grain, out->frame, &s->h274db,
-                                       (AVFilmGrainParams *) sd->data);
+        av_assert0(out->frame_grain->buf[0]);
+        fgp = av_film_grain_params_select(out->frame);
+        switch (fgp->type) {
+        case AV_FILM_GRAIN_PARAMS_NONE:
+            av_assert0(0);
+            return AVERROR_BUG;
+        case AV_FILM_GRAIN_PARAMS_H274:
+            ret = ff_h274_apply_film_grain(out->frame_grain, out->frame,
+                                           &s->h274db, fgp);
+            break;
+        case AV_FILM_GRAIN_PARAMS_AV1:
+            ret = ff_aom_apply_film_grain(out->frame_grain, out->frame, fgp);
+            break;
+        }
         av_assert1(ret >= 0);
     }
 
@@ -3596,6 +3610,7 @@ static int hevc_update_thread_context(AVCodecContext *dst,
     s->sei.common.alternative_transfer = s0->sei.common.alternative_transfer;
     s->sei.common.mastering_display    = s0->sei.common.mastering_display;
     s->sei.common.content_light        = s0->sei.common.content_light;
+    s->sei.common.aom_film_grain       = s0->sei.common.aom_film_grain;
 
     ret = export_stream_params_from_sei(s);
     if (ret < 0)
-- 
2.44.0

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH v4 00/13] AFGS1 film grain support

[Niklas Haas]

On Mon, 18 Mar 2024 17:54:08 +0100 Niklas Haas <ffmpeg@haasn.xyz> wrote:
> Changes since v3:
> 
> - Moved metadata into common AVFilmGrainParams struct, as per James
>   Almer's recommendation
> - Fixed width/height field validation
> - Signal this metadata for AV1 (instead of setting it to unspecified)
> - Implement ffprobe/showinfo support for the new fields

Will push this version at the end of the week if there are no
objections.
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".