[FFmpeg-devel] [PATCH 0/8] DoVi metadata compression

[FFmpeg-devel] [PATCH 0/8] DoVi metadata compression

[Niklas Haas]

Moves metadata compression out of the codecs and into a bitstream
filter. This is required by design, because keyframes must have
compression disabled (for obvious reasons). Therefore, we _must_
generate the dolby vision RPUs after encoding.

The downside of this approach is that it requires "redundantly" encoding
+ decoding RPUs in between the codec and the bsf. However, the tradeoff
is that it allows us to also strip/modify DV metadata when not
transcoding (-c:v copy). Such a feature has a legitimate use case, as it
allows enabling/disabling Dolby Vision compression on existing streams
(to e.g. minimize size, or to maximize compatibility).

If we wanted to avoid this overhead, we could hypothetically introduce
a new packet side data type for dolby vision metadata, and have the
codec merely copy the frame side data to the packet. But I am somewhat
unwilling to add yet more packet side data types that are aliases of
frame side data types, and I don't think the overhead is significant.
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH 1/8] avcodec/dovi_rpudec: clarify semantics

[Niklas Haas]

From: Niklas Haas <git@haasn.dev>

ff_dovi_rpu_parse() and ff_dovi_rpu_generate() are a bit inconsistent in
that they expect different levels of encapsulation, due to the nature of
how this is handled in the context of different APIs. Clarify the status
quo. (And fix an incorrect reference to the RPU payload bytes as 'RBSP')
---
 libavcodec/dovi_rpu.h    | 5 +++--
 libavcodec/dovi_rpudec.c | 2 +-
 2 files changed, 4 insertions(+), 3 deletions(-)

diff --git a/libavcodec/dovi_rpu.h b/libavcodec/dovi_rpu.h
index bfb118d6b5..205d16ffbc 100644
--- a/libavcodec/dovi_rpu.h
+++ b/libavcodec/dovi_rpu.h
@@ -95,8 +95,9 @@ void ff_dovi_ctx_unref(DOVIContext *s);
 void ff_dovi_ctx_flush(DOVIContext *s);
 
 /**
- * Parse the contents of a Dovi RPU NAL and update the parsed values in the
- * DOVIContext struct.
+ * Parse the contents of a Dolby Vision RPU and update the parsed values in the
+ * DOVIContext struct. This function should receive the decoded unit payload,
+ * without any T.35 or NAL unit headers.
  *
  * Returns 0 or an error code.
  *
diff --git a/libavcodec/dovi_rpudec.c b/libavcodec/dovi_rpudec.c
index cae4dc4c3c..b34c3116b7 100644
--- a/libavcodec/dovi_rpudec.c
+++ b/libavcodec/dovi_rpudec.c
@@ -360,7 +360,7 @@ int ff_dovi_rpu_parse(DOVIContext *s, const uint8_t *rpu, size_t rpu_size,
         emdf_protection = get_bits(gb, 5 + 12);
         VALIDATE(emdf_protection, 0x400, 0x400);
     } else {
-        /* NAL RBSP with prefix and trailing zeroes */
+        /* NAL unit with prefix and trailing zeroes */
         VALIDATE(rpu[0], 25, 25); /* NAL prefix */
         rpu++;
         rpu_size--;
-- 
2.45.1

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH 2/8] avcodec/dovi_rpuenc: also copy ext blocks to dovi ctx

[Niklas Haas]

From: Niklas Haas <git@haasn.dev>

As the comment implies, DOVIContext.ext_blocks should also reflect the
current state after ff_dovi_rpu_generate().

Fluff for now, but will be needed once we start implementing metadata
compression for extension blocks as well.
---
 libavcodec/dovi_rpuenc.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/libavcodec/dovi_rpuenc.c b/libavcodec/dovi_rpuenc.c
index a6262844d4..45fcd9a86c 100644
--- a/libavcodec/dovi_rpuenc.c
+++ b/libavcodec/dovi_rpuenc.c
@@ -506,6 +506,12 @@ int ff_dovi_rpu_generate(DOVIContext *s, const AVDOVIMetadata *metadata,
         }
     }
 
+    if (metadata->num_ext_blocks && !s->ext_blocks) {
+        s->ext_blocks = ff_refstruct_allocz(sizeof(AVDOVIDmData) * AV_DOVI_MAX_EXT_BLOCKS);
+        if (!s->ext_blocks)
+            return AVERROR(ENOMEM);
+    }
+
     vdr_dm_metadata_present = memcmp(color, &ff_dovi_color_default, sizeof(*color));
     use_prev_vdr_rpu = !memcmp(s->vdr[vdr_rpu_id], mapping, sizeof(*mapping));
     if (num_ext_blocks_v1 || num_ext_blocks_v2)
@@ -635,6 +641,7 @@ int ff_dovi_rpu_generate(DOVIContext *s, const AVDOVIMetadata *metadata,
     }
 
     if (vdr_dm_metadata_present) {
+        size_t ext_sz;
         const int denom = profile == 4 ? (1 << 30) : (1 << 28);
         set_ue_golomb(pb, color->dm_metadata_id); /* affected_dm_id */
         set_ue_golomb(pb, color->dm_metadata_id); /* current_dm_id */
@@ -672,6 +679,11 @@ int ff_dovi_rpu_generate(DOVIContext *s, const AVDOVIMetadata *metadata,
             for (int i = 0; i < metadata->num_ext_blocks; i++)
                 generate_ext_v2(pb, av_dovi_get_ext(metadata, i));
         }
+
+        ext_sz = FFMIN(sizeof(AVDOVIDmData), metadata->ext_block_size);
+        for (int i = 0; i < metadata->num_ext_blocks; i++)
+            memcpy(&s->ext_blocks[i], av_dovi_get_ext(metadata, i), ext_sz);
+        s->num_ext_blocks = metadata->num_ext_blocks;
     } else {
         s->color = &ff_dovi_color_default;
         s->num_ext_blocks = 0;
-- 
2.45.1

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH 3/8] avcodec/dovi_rpuenc: try to re-use existing vdr_rpu_id

[Niklas Haas]

From: Niklas Haas <git@haasn.dev>

And only override it if we either have an exact match, or if we still
have unused metadata slots (to avoid an overwrite).
---
 libavcodec/dovi_rpuenc.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/libavcodec/dovi_rpuenc.c b/libavcodec/dovi_rpuenc.c
index 45fcd9a86c..0d49a128fd 100644
--- a/libavcodec/dovi_rpuenc.c
+++ b/libavcodec/dovi_rpuenc.c
@@ -463,12 +463,12 @@ int ff_dovi_rpu_generate(DOVIContext *s, const AVDOVIMetadata *metadata,
         return AVERROR_INVALIDDATA;
     }
 
-    vdr_rpu_id = -1;
+    vdr_rpu_id = mapping->vdr_rpu_id;
     for (int i = 0; i <= DOVI_MAX_DM_ID; i++) {
         if (s->vdr[i] && !memcmp(s->vdr[i], mapping, sizeof(*mapping))) {
             vdr_rpu_id = i;
             break;
-        } else if (vdr_rpu_id < 0 && (!s->vdr[i] || i == DOVI_MAX_DM_ID)) {
+        } else if (s->vdr[vdr_rpu_id] && !s->vdr[i]) {
             vdr_rpu_id = i;
         }
     }
-- 
2.45.1

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 3/8] avcodec/dovi_rpuenc: try to re-use existing vdr_rpu_id

[Niklas Haas]

On Tue, 18 Jun 2024 21:35:33 +0200 Niklas Haas <ffmpeg@haasn.xyz> wrote:
> From: Niklas Haas <git@haasn.dev>
> 
> And only override it if we either have an exact match, or if we still
> have unused metadata slots (to avoid an overwrite).
> ---
>  libavcodec/dovi_rpuenc.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/libavcodec/dovi_rpuenc.c b/libavcodec/dovi_rpuenc.c
> index 45fcd9a86c..0d49a128fd 100644
> --- a/libavcodec/dovi_rpuenc.c
> +++ b/libavcodec/dovi_rpuenc.c
> @@ -463,12 +463,12 @@ int ff_dovi_rpu_generate(DOVIContext *s, const AVDOVIMetadata *metadata,
>          return AVERROR_INVALIDDATA;
>      }
>  
> -    vdr_rpu_id = -1;
> +    vdr_rpu_id = mapping->vdr_rpu_id;
>      for (int i = 0; i <= DOVI_MAX_DM_ID; i++) {
>          if (s->vdr[i] && !memcmp(s->vdr[i], mapping, sizeof(*mapping))) {
>              vdr_rpu_id = i;
>              break;
> -        } else if (vdr_rpu_id < 0 && (!s->vdr[i] || i == DOVI_MAX_DM_ID)) {
> +        } else if (s->vdr[vdr_rpu_id] && !s->vdr[i]) {
>              vdr_rpu_id = i;
>          }
>      }
> -- 
> 2.45.1
> 

I just noticed this check is wrong either way, because the `vdr_rpu_id`
is also included as part of `mapping`. If we reshuffle VDR IDs, we also
need to change the contents of `mapping` when updating the vdrs, as well
as omitting it when doing the `memcmp`.

Effectively this loop currently does nothing, since the only field that
can succeed is in the case `i == mapping->vdr_rpu_id`.

Will fix for v2.
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH 4/8] avcodec/dovi_rpuenc: add `flags` to ff_dovi_rpu_generate()

[Niklas Haas]

From: Niklas Haas <git@haasn.dev>

Will be used to control compression, encapsulation etc.
---
 libavcodec/dovi_rpu.h    | 2 +-
 libavcodec/dovi_rpuenc.c | 2 +-
 libavcodec/libaomenc.c   | 2 +-
 libavcodec/libsvtav1.c   | 2 +-
 libavcodec/libx265.c     | 3 ++-
 5 files changed, 6 insertions(+), 5 deletions(-)

diff --git a/libavcodec/dovi_rpu.h b/libavcodec/dovi_rpu.h
index 205d16ffbc..65a4529106 100644
--- a/libavcodec/dovi_rpu.h
+++ b/libavcodec/dovi_rpu.h
@@ -135,7 +135,7 @@ int ff_dovi_configure(DOVIContext *s, AVCodecContext *avctx);
  * including the EMDF header (profile 10) or NAL encapsulation (otherwise).
  */
 int ff_dovi_rpu_generate(DOVIContext *s, const AVDOVIMetadata *metadata,
-                         uint8_t **out_rpu, int *out_size);
+                         int flags, uint8_t **out_rpu, int *out_size);
 
 
 /***************************************************
diff --git a/libavcodec/dovi_rpuenc.c b/libavcodec/dovi_rpuenc.c
index 0d49a128fd..d8f113c5c4 100644
--- a/libavcodec/dovi_rpuenc.c
+++ b/libavcodec/dovi_rpuenc.c
@@ -435,7 +435,7 @@ static void generate_ext_v2(PutBitContext *pb, const AVDOVIDmData *dm)
 }
 
 int ff_dovi_rpu_generate(DOVIContext *s, const AVDOVIMetadata *metadata,
-                         uint8_t **out_rpu, int *out_size)
+                         int flags, uint8_t **out_rpu, int *out_size)
 {
     PutBitContext *pb = &(PutBitContext){0};
     const AVDOVIRpuDataHeader *hdr;
diff --git a/libavcodec/libaomenc.c b/libavcodec/libaomenc.c
index dec74ebecd..aa51c89e29 100644
--- a/libavcodec/libaomenc.c
+++ b/libavcodec/libaomenc.c
@@ -1294,7 +1294,7 @@ FF_ENABLE_DEPRECATION_WARNINGS
             const AVDOVIMetadata *metadata = (const AVDOVIMetadata *)sd->data;
             uint8_t *t35;
             int size;
-            if ((res = ff_dovi_rpu_generate(&ctx->dovi, metadata, &t35, &size)) < 0)
+            if ((res = ff_dovi_rpu_generate(&ctx->dovi, metadata, 0, &t35, &size)) < 0)
                 return res;
             res = aom_img_add_metadata(rawimg, OBU_METADATA_TYPE_ITUT_T35,
                                        t35, size, AOM_MIF_ANY_FRAME);
diff --git a/libavcodec/libsvtav1.c b/libavcodec/libsvtav1.c
index 2fef8c8971..b6db63fd7a 100644
--- a/libavcodec/libsvtav1.c
+++ b/libavcodec/libsvtav1.c
@@ -541,7 +541,7 @@ static int eb_send_frame(AVCodecContext *avctx, const AVFrame *frame)
         const AVDOVIMetadata *metadata = (const AVDOVIMetadata *)sd->data;
         uint8_t *t35;
         int size;
-        if ((ret = ff_dovi_rpu_generate(&svt_enc->dovi, metadata, &t35, &size)) < 0)
+        if ((ret = ff_dovi_rpu_generate(&svt_enc->dovi, metadata, 0, &t35, &size)) < 0)
             return ret;
         ret = svt_add_metadata(headerPtr, EB_AV1_METADATA_TYPE_ITUT_T35, t35, size);
         av_free(t35);
diff --git a/libavcodec/libx265.c b/libavcodec/libx265.c
index 0dc7ab6eeb..4302c3d587 100644
--- a/libavcodec/libx265.c
+++ b/libavcodec/libx265.c
@@ -783,7 +783,8 @@ static int libx265_encode_frame(AVCodecContext *avctx, AVPacket *pkt,
         sd = av_frame_get_side_data(pic, AV_FRAME_DATA_DOVI_METADATA);
         if (ctx->dovi.cfg.dv_profile && sd) {
             const AVDOVIMetadata *metadata = (const AVDOVIMetadata *)sd->data;
-            ret = ff_dovi_rpu_generate(&ctx->dovi, metadata, &x265pic.rpu.payload,
+            ret = ff_dovi_rpu_generate(&ctx->dovi, metadata, 0,
+                                       &x265pic.rpu.payload,
                                        &x265pic.rpu.payloadSize);
             if (ret < 0) {
                 free_picture(ctx, &x265pic);
-- 
2.45.1

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH 5/8] avcodec/dovi_rpuenc: make encapsulation optional

[Niklas Haas]

From: Niklas Haas <git@haasn.dev>

And move the choice of desired container to `flags`. This is needed to
handle differing API requirements (e.g. libx265 requires the NAL RBSP,
but CBS BSF requires the unescaped bytes).
---
 libavcodec/dovi_rpu.h    | 16 ++++++++++------
 libavcodec/dovi_rpuenc.c | 22 ++++++++++------------
 libavcodec/libaomenc.c   |  3 ++-
 libavcodec/libsvtav1.c   |  3 ++-
 libavcodec/libx265.c     |  2 +-
 5 files changed, 25 insertions(+), 21 deletions(-)

diff --git a/libavcodec/dovi_rpu.h b/libavcodec/dovi_rpu.h
index 65a4529106..226a769bff 100644
--- a/libavcodec/dovi_rpu.h
+++ b/libavcodec/dovi_rpu.h
@@ -123,16 +123,20 @@ int ff_dovi_attach_side_data(DOVIContext *s, AVFrame *frame);
  */
 int ff_dovi_configure(DOVIContext *s, AVCodecContext *avctx);
 
+enum {
+    FF_DOVI_WRAP_NAL        = 1 << 0, ///< wrap inside NAL RBSP
+    FF_DOVI_WRAP_T35        = 1 << 1, ///< wrap inside T.35+EMDF
+};
+
 /**
- * Synthesize a Dolby Vision RPU reflecting the current state. Note that this
- * assumes all previous calls to `ff_dovi_rpu_generate` have been appropriately
- * signalled, i.e. it will not re-send already transmitted redundant data.
+ * Synthesize a Dolby Vision RPU reflecting the current state. By default, the
+ * RPU is not encapsulated (see `flags` for more options). Note that this
+ * assumes all previous calls to `ff_dovi_rpu_generate` have been
+ * appropriately signalled, i.e. it will not re-send already transmitted
+ * redundant data.
  *
  * Mutates the internal state of DOVIContext to reflect the change.
  * Returns 0 or a negative error code.
- *
- * This generates a fully formed RPU ready for inclusion in the bitstream,
- * including the EMDF header (profile 10) or NAL encapsulation (otherwise).
  */
 int ff_dovi_rpu_generate(DOVIContext *s, const AVDOVIMetadata *metadata,
                          int flags, uint8_t **out_rpu, int *out_size);
diff --git a/libavcodec/dovi_rpuenc.c b/libavcodec/dovi_rpuenc.c
index d8f113c5c4..65c36ed0ae 100644
--- a/libavcodec/dovi_rpuenc.c
+++ b/libavcodec/dovi_rpuenc.c
@@ -697,9 +697,7 @@ int ff_dovi_rpu_generate(DOVIContext *s, const AVDOVIMetadata *metadata,
     flush_put_bits(pb);
 
     rpu_size = put_bytes_output(pb);
-    switch (s->cfg.dv_profile) {
-    case 10:
-        /* AV1 uses T.35 OBU with EMDF header */
+    if (flags & FF_DOVI_WRAP_T35) {
         *out_rpu = av_malloc(rpu_size + 15);
         if (!*out_rpu)
             return AVERROR(ENOMEM);
@@ -726,10 +724,8 @@ int ff_dovi_rpu_generate(DOVIContext *s, const AVDOVIMetadata *metadata,
         flush_put_bits(pb);
         *out_size = put_bytes_output(pb);
         return 0;
-
-    case 5:
-    case 8:
-        *out_rpu = dst = av_malloc(1 + rpu_size * 3 / 2); /* worst case */
+    } else if (flags & FF_DOVI_WRAP_NAL) {
+        *out_rpu = dst = av_malloc(4 + rpu_size * 3 / 2); /* worst case */
         if (!*out_rpu)
             return AVERROR(ENOMEM);
         *dst++ = 25; /* NAL prefix */
@@ -752,10 +748,12 @@ int ff_dovi_rpu_generate(DOVIContext *s, const AVDOVIMetadata *metadata,
         }
         *out_size = dst - *out_rpu;
         return 0;
-
-    default:
-        /* Should be unreachable */
-        av_assert0(0);
-        return AVERROR_BUG;
+    } else {
+        /* Return intermediate buffer directly */
+        *out_rpu = s->rpu_buf;
+        *out_size = rpu_size;
+        s->rpu_buf = NULL;
+        s->rpu_buf_sz = 0;
+        return 0;
     }
 }
diff --git a/libavcodec/libaomenc.c b/libavcodec/libaomenc.c
index aa51c89e29..fd9bea2505 100644
--- a/libavcodec/libaomenc.c
+++ b/libavcodec/libaomenc.c
@@ -1294,7 +1294,8 @@ FF_ENABLE_DEPRECATION_WARNINGS
             const AVDOVIMetadata *metadata = (const AVDOVIMetadata *)sd->data;
             uint8_t *t35;
             int size;
-            if ((res = ff_dovi_rpu_generate(&ctx->dovi, metadata, 0, &t35, &size)) < 0)
+            if ((res = ff_dovi_rpu_generate(&ctx->dovi, metadata, FF_DOVI_WRAP_T35,
+                                            &t35, &size)) < 0)
                 return res;
             res = aom_img_add_metadata(rawimg, OBU_METADATA_TYPE_ITUT_T35,
                                        t35, size, AOM_MIF_ANY_FRAME);
diff --git a/libavcodec/libsvtav1.c b/libavcodec/libsvtav1.c
index b6db63fd7a..e7b12fb488 100644
--- a/libavcodec/libsvtav1.c
+++ b/libavcodec/libsvtav1.c
@@ -541,7 +541,8 @@ static int eb_send_frame(AVCodecContext *avctx, const AVFrame *frame)
         const AVDOVIMetadata *metadata = (const AVDOVIMetadata *)sd->data;
         uint8_t *t35;
         int size;
-        if ((ret = ff_dovi_rpu_generate(&svt_enc->dovi, metadata, 0, &t35, &size)) < 0)
+        if ((ret = ff_dovi_rpu_generate(&svt_enc->dovi, metadata, FF_DOVI_WRAP_T35,
+                                        &t35, &size)) < 0)
             return ret;
         ret = svt_add_metadata(headerPtr, EB_AV1_METADATA_TYPE_ITUT_T35, t35, size);
         av_free(t35);
diff --git a/libavcodec/libx265.c b/libavcodec/libx265.c
index 4302c3d587..718bd21b20 100644
--- a/libavcodec/libx265.c
+++ b/libavcodec/libx265.c
@@ -783,7 +783,7 @@ static int libx265_encode_frame(AVCodecContext *avctx, AVPacket *pkt,
         sd = av_frame_get_side_data(pic, AV_FRAME_DATA_DOVI_METADATA);
         if (ctx->dovi.cfg.dv_profile && sd) {
             const AVDOVIMetadata *metadata = (const AVDOVIMetadata *)sd->data;
-            ret = ff_dovi_rpu_generate(&ctx->dovi, metadata, 0,
+            ret = ff_dovi_rpu_generate(&ctx->dovi, metadata, FF_DOVI_WRAP_NAL,
                                        &x265pic.rpu.payload,
                                        &x265pic.rpu.payloadSize);
             if (ret < 0) {
-- 
2.45.1

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH 6/8] avcodec/dovi_rpuenc: disable metadata compression by default

[Niklas Haas]

From: Niklas Haas <git@haasn.dev>

Keyframes must reset the metadata compression state, so we cannot enable
metadata compression inside the encoders. Solve this by adding a new
flag, rather than removing it entirely, because I plan on adding
a bitstream filter for metadata compression.
---
 libavcodec/dovi_rpu.h    |  3 +++
 libavcodec/dovi_rpuenc.c | 27 +++++++++++++++++++--------
 2 files changed, 22 insertions(+), 8 deletions(-)

diff --git a/libavcodec/dovi_rpu.h b/libavcodec/dovi_rpu.h
index 226a769bff..f0d9c24379 100644
--- a/libavcodec/dovi_rpu.h
+++ b/libavcodec/dovi_rpu.h
@@ -126,6 +126,9 @@ int ff_dovi_configure(DOVIContext *s, AVCodecContext *avctx);
 enum {
     FF_DOVI_WRAP_NAL        = 1 << 0, ///< wrap inside NAL RBSP
     FF_DOVI_WRAP_T35        = 1 << 1, ///< wrap inside T.35+EMDF
+
+    FF_DOVI_COMPRESS_VDR    = 1 << 2, ///< enable VDR RPU compression
+    FF_DOVI_COMPRESS_ALL    = FF_DOVI_COMPRESS_VDR,
 };
 
 /**
diff --git a/libavcodec/dovi_rpuenc.c b/libavcodec/dovi_rpuenc.c
index 65c36ed0ae..631733c620 100644
--- a/libavcodec/dovi_rpuenc.c
+++ b/libavcodec/dovi_rpuenc.c
@@ -20,6 +20,8 @@
  * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
  */
 
+#include <stdbool.h>
+
 #include "libavutil/avassert.h"
 #include "libavutil/crc.h"
 #include "libavutil/mem.h"
@@ -441,9 +443,10 @@ int ff_dovi_rpu_generate(DOVIContext *s, const AVDOVIMetadata *metadata,
     const AVDOVIRpuDataHeader *hdr;
     const AVDOVIDataMapping *mapping;
     const AVDOVIColorMetadata *color;
-    int vdr_dm_metadata_present, vdr_rpu_id, use_prev_vdr_rpu, profile,
+    int vdr_dm_metadata_present, vdr_rpu_id, profile,
         buffer_size, rpu_size, pad, zero_run;
     int num_ext_blocks_v1, num_ext_blocks_v2;
+    int use_prev_vdr_rpu = false;
     uint32_t crc;
     uint8_t *dst;
     if (!metadata) {
@@ -464,12 +467,21 @@ int ff_dovi_rpu_generate(DOVIContext *s, const AVDOVIMetadata *metadata,
     }
 
     vdr_rpu_id = mapping->vdr_rpu_id;
-    for (int i = 0; i <= DOVI_MAX_DM_ID; i++) {
-        if (s->vdr[i] && !memcmp(s->vdr[i], mapping, sizeof(*mapping))) {
-            vdr_rpu_id = i;
-            break;
-        } else if (s->vdr[vdr_rpu_id] && !s->vdr[i]) {
-            vdr_rpu_id = i;
+    if (flags & FF_DOVI_COMPRESS_VDR) {
+        for (int i = 0; i <= DOVI_MAX_DM_ID; i++) {
+            if (s->vdr[i] && !memcmp(s->vdr[i], mapping, sizeof(*mapping))) {
+                use_prev_vdr_rpu = true;
+                vdr_rpu_id = i;
+                break;
+            } else if (s->vdr[vdr_rpu_id] && !s->vdr[i]) {
+                vdr_rpu_id = i;
+            }
+        }
+    } else {
+        /* Flush VDRs to avoid leaking old state after keyframe */
+        for (int i = 0; i <= DOVI_MAX_DM_ID; i++) {
+            if (i != vdr_rpu_id)
+                ff_refstruct_unref(&s->vdr[i]);
         }
     }
 
@@ -513,7 +525,6 @@ int ff_dovi_rpu_generate(DOVIContext *s, const AVDOVIMetadata *metadata,
     }
 
     vdr_dm_metadata_present = memcmp(color, &ff_dovi_color_default, sizeof(*color));
-    use_prev_vdr_rpu = !memcmp(s->vdr[vdr_rpu_id], mapping, sizeof(*mapping));
     if (num_ext_blocks_v1 || num_ext_blocks_v2)
         vdr_dm_metadata_present = 1;
 
-- 
2.45.1

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH 7/8] avcodec/dovi_rpu: add ff_dovi_get_metadata()

[Niklas Haas]

From: Niklas Haas <git@haasn.dev>

Provides direct access to the AVDOVIMetadata without having to attach it
to a frame.
---
 libavcodec/dovi_rpu.h    |  9 +++++++++
 libavcodec/dovi_rpudec.c | 40 +++++++++++++++++++++++++++-------------
 2 files changed, 36 insertions(+), 13 deletions(-)

diff --git a/libavcodec/dovi_rpu.h b/libavcodec/dovi_rpu.h
index f0d9c24379..10d5c7f566 100644
--- a/libavcodec/dovi_rpu.h
+++ b/libavcodec/dovi_rpu.h
@@ -108,8 +108,17 @@ void ff_dovi_ctx_flush(DOVIContext *s);
 int ff_dovi_rpu_parse(DOVIContext *s, const uint8_t *rpu, size_t rpu_size,
                       int err_recognition);
 
+/**
+ * Get the decoded AVDOVIMetadata. Ownership passes to the caller.
+ *
+ * Returns the size of *out_metadata, a negative error code, or 0 if no
+ * metadata is available to return.
+ */
+int ff_dovi_get_metadata(DOVIContext *s, AVDOVIMetadata **out_metadata);
+
 /**
  * Attach the decoded AVDOVIMetadata as side data to an AVFrame.
+ * Returns 0 or a negative error code.
  */
 int ff_dovi_attach_side_data(DOVIContext *s, AVFrame *frame);
 
diff --git a/libavcodec/dovi_rpudec.c b/libavcodec/dovi_rpudec.c
index b34c3116b7..1e2ad4fd3d 100644
--- a/libavcodec/dovi_rpudec.c
+++ b/libavcodec/dovi_rpudec.c
@@ -30,10 +30,8 @@
 #include "get_bits.h"
 #include "refstruct.h"
 
-int ff_dovi_attach_side_data(DOVIContext *s, AVFrame *frame)
+int ff_dovi_get_metadata(DOVIContext *s, AVDOVIMetadata **out_metadata)
 {
-    AVFrameSideData *sd;
-    AVBufferRef *buf;
     AVDOVIMetadata *dovi;
     size_t dovi_size, ext_sz;
 
@@ -44,7 +42,32 @@ int ff_dovi_attach_side_data(DOVIContext *s, AVFrame *frame)
     if (!dovi)
         return AVERROR(ENOMEM);
 
-    buf = av_buffer_create((uint8_t *) dovi, dovi_size, NULL, NULL, 0);
+    /* Copy only the parts of these structs known to us at compiler-time. */
+#define COPY(t, a, b, last) memcpy(a, b, offsetof(t, last) + sizeof((b)->last))
+    COPY(AVDOVIRpuDataHeader, av_dovi_get_header(dovi), &s->header, disable_residual_flag);
+    COPY(AVDOVIDataMapping, av_dovi_get_mapping(dovi), s->mapping, nlq_pivots);
+    COPY(AVDOVIColorMetadata, av_dovi_get_color(dovi), s->color, source_diagonal);
+    ext_sz = FFMIN(sizeof(AVDOVIDmData), dovi->ext_block_size);
+    for (int i = 0; i < s->num_ext_blocks; i++)
+        memcpy(av_dovi_get_ext(dovi, i), &s->ext_blocks[i], ext_sz);
+    dovi->num_ext_blocks = s->num_ext_blocks;
+
+    *out_metadata = dovi;
+    return dovi_size;
+}
+
+int ff_dovi_attach_side_data(DOVIContext *s, AVFrame *frame)
+{
+    AVFrameSideData *sd;
+    AVDOVIMetadata *dovi;
+    AVBufferRef *buf;
+    int size;
+
+    size = ff_dovi_get_metadata(s, &dovi);
+    if (size <= 0)
+        return size;
+
+    buf = av_buffer_create((uint8_t *) dovi, size, NULL, NULL, 0);
     if (!buf) {
         av_free(dovi);
         return AVERROR(ENOMEM);
@@ -56,15 +79,6 @@ int ff_dovi_attach_side_data(DOVIContext *s, AVFrame *frame)
         return AVERROR(ENOMEM);
     }
 
-    /* Copy only the parts of these structs known to us at compiler-time. */
-#define COPY(t, a, b, last) memcpy(a, b, offsetof(t, last) + sizeof((b)->last))
-    COPY(AVDOVIRpuDataHeader, av_dovi_get_header(dovi), &s->header, disable_residual_flag);
-    COPY(AVDOVIDataMapping, av_dovi_get_mapping(dovi), s->mapping, nlq_pivots);
-    COPY(AVDOVIColorMetadata, av_dovi_get_color(dovi), s->color, source_diagonal);
-    ext_sz = FFMIN(sizeof(AVDOVIDmData), dovi->ext_block_size);
-    for (int i = 0; i < s->num_ext_blocks; i++)
-        memcpy(av_dovi_get_ext(dovi, i), &s->ext_blocks[i], ext_sz);
-    dovi->num_ext_blocks = s->num_ext_blocks;
     return 0;
 }
 
-- 
2.45.1

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH 8/8] avcodec/bsf/dovi_rpu: add new bitstream filter

[Niklas Haas]

From: Niklas Haas <git@haasn.dev>

This can be used to strip dovi metadata, or enable/disable dovi
metadata compression. Possibly more use cases in the future.
---
 configure                      |   1 +
 doc/bitstream_filters.texi     |  21 +++
 libavcodec/bitstream_filters.c |   1 +
 libavcodec/bsf/Makefile        |   1 +
 libavcodec/bsf/dovi_rpu.c      | 258 +++++++++++++++++++++++++++++++++
 5 files changed, 282 insertions(+)
 create mode 100644 libavcodec/bsf/dovi_rpu.c

diff --git a/configure b/configure
index 95565994fe..b432a1c11c 100755
--- a/configure
+++ b/configure
@@ -3437,6 +3437,7 @@ aac_adtstoasc_bsf_select="adts_header mpeg4audio"
 av1_frame_merge_bsf_select="cbs_av1"
 av1_frame_split_bsf_select="cbs_av1"
 av1_metadata_bsf_select="cbs_av1"
+dovi_rpu_bsf_select="cbs_h265 cbs_av1 dovi_rpudec dovi_rpuenc"
 dts2pts_bsf_select="cbs_h264 h264parse"
 eac3_core_bsf_select="ac3_parser"
 evc_frame_merge_bsf_select="evcparse"
diff --git a/doc/bitstream_filters.texi b/doc/bitstream_filters.texi
index c03f04f858..918735e8c5 100644
--- a/doc/bitstream_filters.texi
+++ b/doc/bitstream_filters.texi
@@ -101,6 +101,27 @@ Remove zero padding at the end of a packet.
 Extract the core from a DCA/DTS stream, dropping extensions such as
 DTS-HD.
 
+@section dovi_rpu
+
+Manipulate Dolby Vision metadata in a HEVC/AV1 bitstream, optionally enabling
+metadata compression.
+
+@table @option
+@item strip
+If enabled, strip all Dolby Vision metadata (configuration record + RPU data
+blocks) from the stream.
+@item compression
+A bit mask of compression methods to enable.
+@table @samp
+@item none
+No compression. Selected automatically for keyframes.
+@item vdr
+Compress VDR metadata (color reshaping / data mapping parameters).
+@item all
+Enable all implemented compression methods. This is the default.
+@end table
+@end table
+
 @section dump_extra
 
 Add extradata to the beginning of the filtered packets except when
diff --git a/libavcodec/bitstream_filters.c b/libavcodec/bitstream_filters.c
index 138246c50e..f923411bee 100644
--- a/libavcodec/bitstream_filters.c
+++ b/libavcodec/bitstream_filters.c
@@ -31,6 +31,7 @@ extern const FFBitStreamFilter ff_av1_metadata_bsf;
 extern const FFBitStreamFilter ff_chomp_bsf;
 extern const FFBitStreamFilter ff_dump_extradata_bsf;
 extern const FFBitStreamFilter ff_dca_core_bsf;
+extern const FFBitStreamFilter ff_dovi_rpu_bsf;
 extern const FFBitStreamFilter ff_dts2pts_bsf;
 extern const FFBitStreamFilter ff_dv_error_marker_bsf;
 extern const FFBitStreamFilter ff_eac3_core_bsf;
diff --git a/libavcodec/bsf/Makefile b/libavcodec/bsf/Makefile
index fb70ad0c21..40b7fc6e9b 100644
--- a/libavcodec/bsf/Makefile
+++ b/libavcodec/bsf/Makefile
@@ -19,6 +19,7 @@ OBJS-$(CONFIG_H264_MP4TOANNEXB_BSF)       += bsf/h264_mp4toannexb.o
 OBJS-$(CONFIG_H264_REDUNDANT_PPS_BSF)     += bsf/h264_redundant_pps.o
 OBJS-$(CONFIG_HAPQA_EXTRACT_BSF)          += bsf/hapqa_extract.o
 OBJS-$(CONFIG_HEVC_METADATA_BSF)          += bsf/h265_metadata.o
+OBJS-$(CONFIG_DOVI_RPU_BSF)               += bsf/dovi_rpu.o
 OBJS-$(CONFIG_HEVC_MP4TOANNEXB_BSF)       += bsf/hevc_mp4toannexb.o
 OBJS-$(CONFIG_IMX_DUMP_HEADER_BSF)        += bsf/imx_dump_header.o
 OBJS-$(CONFIG_MEDIA100_TO_MJPEGB_BSF)     += bsf/media100_to_mjpegb.o
diff --git a/libavcodec/bsf/dovi_rpu.c b/libavcodec/bsf/dovi_rpu.c
new file mode 100644
index 0000000000..c57c3d87dd
--- /dev/null
+++ b/libavcodec/bsf/dovi_rpu.c
@@ -0,0 +1,258 @@
+/*
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with FFmpeg; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+#include "libavutil/common.h"
+#include "libavutil/mem.h"
+#include "libavutil/opt.h"
+
+#include "bsf.h"
+#include "bsf_internal.h"
+#include "cbs.h"
+#include "cbs_bsf.h"
+#include "cbs_av1.h"
+#include "cbs_h265.h"
+#include "dovi_rpu.h"
+#include "h2645data.h"
+#include "h265_profile_level.h"
+#include "itut35.h"
+
+#include "hevc/hevc.h"
+
+typedef struct DoviRpuContext {
+    CBSBSFContext common;
+    DOVIContext dec;
+    DOVIContext enc;
+
+    int strip;
+    int compression;
+} DoviRpuContext;
+
+static int update_rpu(AVBSFContext *bsf, const AVPacket *pkt, int flags,
+                      const uint8_t *rpu, size_t rpu_size,
+                      uint8_t **out_rpu, int *out_size)
+{
+    DoviRpuContext *s = bsf->priv_data;
+    AVDOVIMetadata *metadata = NULL;
+    int ret;
+
+    ret = ff_dovi_rpu_parse(&s->dec, rpu, rpu_size, 0);
+    if (ret < 0) {
+        ff_dovi_ctx_flush(&s->dec);
+        return ret;
+    }
+
+    ret = ff_dovi_get_metadata(&s->dec, &metadata);
+    if (ret == 0 /* no metadata */) {
+        *out_rpu = NULL;
+        *out_size = 0;
+        return 0;
+    } else if (ret < 0) {
+        ff_dovi_ctx_flush(&s->dec);
+        return ret;
+    }
+
+    if (pkt && !(pkt->flags & AV_PKT_FLAG_KEY))
+        flags |= s->compression;
+    ret = ff_dovi_rpu_generate(&s->enc, metadata, flags, out_rpu, out_size);
+    av_free(metadata);
+    if (ret < 0)
+        ff_dovi_ctx_flush(&s->enc);
+
+    return ret;
+}
+
+static int dovi_rpu_update_fragment_hevc(AVBSFContext *bsf, AVPacket *pkt,
+                                         CodedBitstreamFragment *au)
+{
+    DoviRpuContext *s = bsf->priv_data;
+    CodedBitstreamUnit *nal = au->nb_units ? &au->units[au->nb_units - 1] : NULL;
+    uint8_t *rpu = NULL;
+    int rpu_size, ret;
+
+    if (!nal || nal->type != HEVC_NAL_UNSPEC62)
+        return 0;
+
+    if (s->strip) {
+        ff_cbs_delete_unit(au, au->nb_units - 1);
+        return 0;
+    }
+
+    ret = update_rpu(bsf, pkt, 0, nal->data + 2, nal->data_size - 2, &rpu, &rpu_size);
+    if (ret < 0)
+        return ret;
+
+    /* NAL unit header + NAL prefix */
+    if (rpu_size + 3 <= nal->data_size && av_buffer_is_writable(nal->data_ref)) {
+        memcpy(nal->data + 3, rpu, rpu_size);
+        av_free(rpu);
+        nal->data_size = rpu_size + 3;
+    } else {
+        AVBufferRef *ref = av_buffer_alloc(rpu_size + 3);
+        if (!ref) {
+            av_free(rpu);
+            return AVERROR(ENOMEM);
+        }
+
+        memcpy(ref->data, nal->data, 3);
+        memcpy(ref->data + 3, rpu, rpu_size);
+        av_buffer_unref(&nal->data_ref);
+        av_free(rpu);
+        nal->data = ref->data;
+        nal->data_size = rpu_size + 3;
+        nal->data_ref = ref;
+        nal->data_bit_padding = 0;
+    }
+
+    return 0;
+}
+
+static int dovi_rpu_update_fragment_av1(AVBSFContext *bsf, AVPacket *pkt,
+                                        CodedBitstreamFragment *frag)
+{
+    DoviRpuContext *s = bsf->priv_data;
+    int provider_code, provider_oriented_code, rpu_size, ret;
+    AVBufferRef *ref;
+    uint8_t *rpu;
+
+    for (int i = 0; i < frag->nb_units; i++) {
+        AV1RawOBU *obu = frag->units[i].content;
+        AV1RawMetadataITUTT35 *t35 = &obu->obu.metadata.metadata.itut_t35;
+        if (frag->units[i].type != AV1_OBU_METADATA ||
+            obu->obu.metadata.metadata_type != AV1_METADATA_TYPE_ITUT_T35 ||
+            t35->itu_t_t35_country_code != ITU_T_T35_COUNTRY_CODE_US ||
+            t35->payload_size < 6)
+            continue;
+
+        provider_code = AV_RB16(t35->payload);
+        provider_oriented_code = AV_RB32(t35->payload + 2);
+        if (provider_code != ITU_T_T35_PROVIDER_CODE_DOLBY ||
+            provider_oriented_code != 0x800)
+            continue;
+
+        if (s->strip) {
+            ff_cbs_delete_unit(frag, i);
+            return 0;
+        }
+
+        ret = update_rpu(bsf, pkt, FF_DOVI_WRAP_T35,
+                         t35->payload + 6, t35->payload_size - 6,
+                         &rpu, &rpu_size);
+        if (ret < 0)
+            return ret;
+
+        ref = av_buffer_create(rpu, rpu_size, av_buffer_default_free, NULL, 0);
+        if (!ref) {
+            av_free(rpu);
+            return AVERROR(ENOMEM);
+        }
+
+        av_buffer_unref(&t35->payload_ref);
+        t35->payload_ref = ref;
+        t35->payload = rpu + 1; /* skip country code */
+        t35->payload_size = rpu_size - 1;
+        break; /* should be only one RPU per packet */
+    }
+
+    return 0;
+}
+
+static const CBSBSFType dovi_rpu_hevc_type = {
+    .codec_id        = AV_CODEC_ID_HEVC,
+    .fragment_name   = "access unit",
+    .unit_name       = "NAL unit",
+    .update_fragment = &dovi_rpu_update_fragment_hevc,
+};
+
+static const CBSBSFType dovi_rpu_av1_type = {
+    .codec_id        = AV_CODEC_ID_AV1,
+    .fragment_name   = "temporal unit",
+    .unit_name       = "OBU",
+    .update_fragment = &dovi_rpu_update_fragment_av1,
+};
+
+static int dovi_rpu_init(AVBSFContext *bsf)
+{
+    DoviRpuContext *s = bsf->priv_data;
+
+    if (s->strip) {
+        av_packet_side_data_remove(bsf->par_out->coded_side_data,
+                                   &bsf->par_out->nb_coded_side_data,
+                                   AV_PKT_DATA_DOVI_CONF);
+    } else {
+        const AVPacketSideData *sd;
+        sd = av_packet_side_data_get(bsf->par_in->coded_side_data,
+                                     bsf->par_in->nb_coded_side_data,
+                                     AV_PKT_DATA_DOVI_CONF);
+        if (!sd) {
+            av_log(bsf, AV_LOG_ERROR, "No Dolby Vision configuration record found?\n");
+            return AVERROR(EINVAL);
+        }
+
+        s->dec.logctx = s->enc.logctx = bsf;
+        s->dec.cfg = s->enc.cfg = *(AVDOVIDecoderConfigurationRecord *) sd->data;
+    }
+
+    switch (bsf->par_in->codec_id) {
+    case AV_CODEC_ID_HEVC:
+        return ff_cbs_bsf_generic_init(bsf, &dovi_rpu_hevc_type);
+    case AV_CODEC_ID_AV1:
+        return ff_cbs_bsf_generic_init(bsf, &dovi_rpu_av1_type);
+    default:
+        return AVERROR_BUG;
+    }
+}
+
+static void dovi_rpu_close(AVBSFContext *bsf)
+{
+    DoviRpuContext *s = bsf->priv_data;
+    ff_dovi_ctx_unref(&s->dec);
+    ff_dovi_ctx_unref(&s->enc);
+    ff_cbs_bsf_generic_close(bsf);
+}
+
+#define OFFSET(x) offsetof(DoviRpuContext, x)
+#define FLAGS (AV_OPT_FLAG_VIDEO_PARAM|AV_OPT_FLAG_BSF_PARAM)
+static const AVOption dovi_rpu_options[] = {
+    { "strip", "Strip Dolby Vision metadata", OFFSET(strip), AV_OPT_TYPE_BOOL, { .i64 = 0 }, 0, 1, FLAGS },
+    { "compression", "DV metadata compression mode", OFFSET(compression), AV_OPT_TYPE_FLAGS, { .i64 = FF_DOVI_COMPRESS_ALL }, 0, FF_DOVI_COMPRESS_ALL, FLAGS, .unit = "compression" },
+        { "none", "Don't compress metadata", 0, AV_OPT_TYPE_CONST, {.i64 = 0}, 0, 0, FLAGS, .unit = "compression" },
+        { "vdr", "Compress VDR metadata", 0, AV_OPT_TYPE_CONST, {.i64 = FF_DOVI_COMPRESS_VDR}, 0, 0, FLAGS, .unit = "compression" },
+        { "all", "Compress all metadata", 0, AV_OPT_TYPE_CONST, {.i64 = FF_DOVI_COMPRESS_ALL}, 0, 0, FLAGS, .unit = "compression" },
+    { NULL }
+};
+
+static const AVClass dovi_rpu_class = {
+    .class_name = "dovi_rpu_bsf",
+    .item_name  = av_default_item_name,
+    .option     = dovi_rpu_options,
+    .version    = LIBAVUTIL_VERSION_INT,
+};
+
+static const enum AVCodecID dovi_rpu_codec_ids[] = {
+    AV_CODEC_ID_HEVC, AV_CODEC_ID_AV1, AV_CODEC_ID_NONE,
+};
+
+const FFBitStreamFilter ff_dovi_rpu_bsf = {
+    .p.name         = "dovi_rpu",
+    .p.codec_ids    = dovi_rpu_codec_ids,
+    .p.priv_class   = &dovi_rpu_class,
+    .priv_data_size = sizeof(DoviRpuContext),
+    .init           = &dovi_rpu_init,
+    .close          = &dovi_rpu_close,
+    .filter         = &ff_cbs_bsf_generic_filter,
+};
-- 
2.45.1

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".