[FFmpeg-devel] [PATCH 0/2] Fix Nikon HEIF decoding error

[FFmpeg-devel] [PATCH 0/2] Fix Nikon HEIF decoding error

[Eric Joyner]

This patch set was motivated by ffmpeg being unable to open .HIF files
from a Nikon Z6 III camera because it would always fail with an error
involving thumbnails; probably due to them including three.

With these changes, ffprobe/ffmpeg will no longer
error out on those types of files.

I ran "make fate" on these commits, and was able to extract the thumbnail
streams and encode them as pngs or jpegs, too.

Eric Joyner (2):
  libavformat: Support multiple thumbnails in HEIF
  libavformat: Enable jpeg streams in HEIF MOVContext

 libavformat/isom.h |  3 ++-
 libavformat/mov.c  | 25 ++++++++++++++++++-------
 2 files changed, 20 insertions(+), 8 deletions(-)

-- 
2.49.0.windows.1

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH 1/2] libavformat: Support multiple thumbnails in HEIF

[Eric Joyner]

Prevents ffmpeg/ffprobe from erroring out when reading an HEIF that
contains multiple hvcC thumbnails (e.g. from a Nikon Z6III camera).

Before, move_read_iref_thmb() would always override the stored
thmb_item_id in the MOVContext with each new read thumbnail, causing a
stream and item_id mismatch later in mov_parse_heif_items(), resulting
in the "HEIF thumbnail doesn't reference a stream" error message.

To solve this,

- Turn thmb_item_id into an array of IDs because multiple thumbnails can
  exist
- Change check in mov_parse_heif_items() to compare against all stored
  thumbnail IDs to see if any item missing a stream is in the list of
  thumbnail IDs.

Signed-off-by: Eric Joyner <erj@erj.cc>
---
 libavformat/isom.h |  3 ++-
 libavformat/mov.c  | 24 +++++++++++++++++-------
 2 files changed, 19 insertions(+), 8 deletions(-)

diff --git a/libavformat/isom.h b/libavformat/isom.h
index 10f882806e..94c9c65989 100644
--- a/libavformat/isom.h
+++ b/libavformat/isom.h
@@ -376,7 +376,8 @@ typedef struct MOVContext {
     int nb_heif_item;
     HEIFGrid *heif_grid;
     int nb_heif_grid;
-    int thmb_item_id;
+    int* thmb_item_id;
+    int nb_thmb_item;
     int64_t idat_offset;
     int interleaved_read;
 } MOVContext;
diff --git a/libavformat/mov.c b/libavformat/mov.c
index c935bbf0bf..7010e13b50 100644
--- a/libavformat/mov.c
+++ b/libavformat/mov.c
@@ -8972,6 +8972,7 @@ static int mov_read_iref_dimg(MOVContext *c, AVIOContext *pb, int version)
 
 static int mov_read_iref_thmb(MOVContext *c, AVIOContext *pb, int version)
 {
+    int *thmb_item_id;
     int entries;
     int to_item_id, from_item_id = version ? avio_rb32(pb) : avio_rb16(pb);
 
@@ -8986,10 +8987,15 @@ static int mov_read_iref_thmb(MOVContext *c, AVIOContext *pb, int version)
     if (to_item_id != c->primary_item_id)
         return 0;
 
-    c->thmb_item_id = from_item_id;
+    /* Put thumnbail IDs into an array */
+    thmb_item_id = av_dynarray2_add((void **)&c->thmb_item_id, &c->nb_thmb_item,
+                                    sizeof(*c->thmb_item_id),
+                                    (const void *)&from_item_id);
+    if (!thmb_item_id)
+        return AVERROR(ENOMEM);
 
-    av_log(c->fc, AV_LOG_TRACE, "thmb: from_item_id %d, entries %d\n",
-           from_item_id, entries);
+    av_log(c->fc, AV_LOG_TRACE, "thmb: from_item_id %d, entries %d, nb_thmb: %d\n",
+           from_item_id, entries, c->nb_thmb_item);
 
     return 0;
 }
@@ -9859,6 +9865,7 @@ static int mov_read_close(AVFormatContext *s)
         av_freep(&mov->heif_grid[i].tile_item_list);
     }
     av_freep(&mov->heif_grid);
+    av_freep(&mov->thmb_item_id);
 
     return 0;
 }
@@ -10323,9 +10330,12 @@ static int mov_parse_heif_items(AVFormatContext *s)
         if (!item)
             continue;
         if (!item->st) {
-            if (item->item_id == mov->thmb_item_id) {
-                av_log(s, AV_LOG_ERROR, "HEIF thumbnail doesn't reference a stream\n");
-                return AVERROR_INVALIDDATA;
+            for (int i = 0; i < mov->nb_thmb_item; i++) {
+                if (item->item_id == mov->thmb_item_id[i]) {
+                    av_log(s, AV_LOG_ERROR, "HEIF thumbnail ID %d doesn't reference a stream\n",
+                           item->item_id);
+                    return AVERROR_INVALIDDATA;
+                }
             }
             continue;
         }
@@ -10476,7 +10486,7 @@ static int mov_read_header(AVFormatContext *s)
 
     mov->fc = s;
     mov->trak_index = -1;
-    mov->thmb_item_id = -1;
+    mov->thmb_item_id = NULL;
     mov->primary_item_id = -1;
     mov->cur_item_id = -1;
     /* .mov and .mp4 aren't streamable anyway (only progressive download if moov is before mdat) */
-- 
2.49.0.windows.1

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

[FFmpeg-devel] [PATCH 2/2] libavformat: Enable jpeg streams in HEIF MOVContext

[Eric Joyner]

Nikon HEIFs from a camera or NX studio include a small jpeg thumbnail in addition to
the expected HEVC thumbnails; allowing jpegs allows all thumbnails to
have an associated stream for Nikon HEIF files.

With this, Nikon HEIFs can finally be decoded without failing and the
thumbnails can be extracted into their own files.

Signed-off-by: Eric Joyner <erj@erj.cc>
---
 libavformat/mov.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/libavformat/mov.c b/libavformat/mov.c
index 7010e13b50..19b31b032d 100644
--- a/libavformat/mov.c
+++ b/libavformat/mov.c
@@ -8840,6 +8840,7 @@ static int mov_read_infe(MOVContext *c, AVIOContext *pb, MOVAtom atom, int idx)
 
     switch (item_type) {
     case MKTAG('a','v','0','1'):
+    case MKTAG('j','p','e','g'):
     case MKTAG('h','v','c','1'):
         ret = heif_add_stream(c, c->heif_item[idx]);
         if (ret < 0)
-- 
2.49.0.windows.1

_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 2/2] libavformat: Enable jpeg streams in HEIF MOVContext

[Lynne]

Thanks. Patchset LGTM.
This also helps HEIF files generated by the S1II, but sadly they still fail:

 > [mjpeg @ 0x55a49a0e29c0] No JPEG data found in image

And for the main stream:

 > [hevc @ 0x55a49a0fe880] Failed to parse header of NALU (type 34): 
"Invalid data found when processing input". Skipping NALU.
 > [hevc @ 0x55a49a0fe880] Invalid NAL unit size (1229531648 > 102386).
 > [hevc @ 0x55a49a0fe880] Error splitting the input into NAL units.
 > [vist#0:0/hevc @ 0x55a49a0e5000] [dec:hevc @ 0x55a49a103a80] Decoding 
error: Invalid data found when processing input

I wonder what they're doing with the thumbnail and NALUs. If you'd like 
to take a look at it, here's a sample:
https://files.lynne.ee/P1001083.HIF

On 17/07/2025 11:30, Eric Joyner wrote:
> Nikon HEIFs from a camera or NX studio include a small jpeg thumbnail in addition to
> the expected HEVC thumbnails; allowing jpegs allows all thumbnails to
> have an associated stream for Nikon HEIF files.
> 
> With this, Nikon HEIFs can finally be decoded without failing and the
> thumbnails can be extracted into their own files.
> 
> Signed-off-by: Eric Joyner <erj@erj.cc>
> ---
>   libavformat/mov.c | 1 +
>   1 file changed, 1 insertion(+)
> 
> diff --git a/libavformat/mov.c b/libavformat/mov.c
> index 7010e13b50..19b31b032d 100644
> --- a/libavformat/mov.c
> +++ b/libavformat/mov.c
> @@ -8840,6 +8840,7 @@ static int mov_read_infe(MOVContext *c, AVIOContext *pb, MOVAtom atom, int idx)
>   
>       switch (item_type) {
>       case MKTAG('a','v','0','1'):
> +    case MKTAG('j','p','e','g'):
>       case MKTAG('h','v','c','1'):
>           ret = heif_add_stream(c, c->heif_item[idx]);
>           if (ret < 0)
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Re: [FFmpeg-devel] [PATCH 0/2] Fix Nikon HEIF decoding error

[compn]

On Wed, 16 Jul 2025 19:30:00 -0700, Eric Joyner wrote:

> This patch set was motivated by ffmpeg being unable to open .HIF files
> from a Nikon Z6 III camera because it would always fail with an error
> involving thumbnails; probably due to them including three.
> 
> With these changes, ffprobe/ffmpeg will no longer
> error out on those types of files.

nice!

do we have a nikon heif/ hif file sample for our samples
repository? http://samples.ffmpeg.org 

we like to keep some files around for testing , to make sure we dont
break them later.

-compn
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".