On 24/02/2024 13:26, Tomas Härdin wrote:
> [...]
>>> It should be possible to have ffmpeg set up the necessary plumbing
>>> for
>>> this.
>> But is it how it works elsewhere in FFmpeg? Would such complex and
>> deep
>> modifications be accepted by others?
> Good question. I would propose something like the following:
>
> 1) detect the use of SEPARATE_FIELDS and set a flag in AVStream


As in practice and in that case (2 jp2k codestreams per AVPacket) it is 
only a tip (because we can autodetect and there are many buggy files in 
the wild) for the jpeg2000 decoder, I was planning to add that later in 
a separate patch, but attached is a version with the flag.

> 2) allocate AVFrame for the size of the resulting *frame*
So keeping what is already there.

> 3a) if the codec is inherently interlaced, call the decoder once
> 3b) if the codec is not inherently interlaced, call the decoder twice,
> with appropriate stride, and keep track of the number of bytes decoded
> so far so we know what offset to start the second decode from


The place I see for that is in decode_simple_internal().
But it is a very hot place I don't like to modify, and it seems to me 
some extra code for 99.9999% (or even more 9s) of files which don't need 
such feature, with more risk to forget this specific feature during a 
future dev e.g. not obvious to change also in ff_thread_decode_frame 
when touching this part.
I also needed to add a dedicated AVStream field for saying that the 
decoder is able to manage this functionality (and is needed there).

What is the added value to call the decoder twice from decode.c rather 
than recursive call (or a master function in the decoder calling the 
current function twice, if preferred) inside the decoder only?
As far as I understand, it would not help for other formats (only the 
signaling propagation in AVStream helps and it is done by another 
AVStream field) and I personally highly prefer that such feature is as 
much as possible in a single place in each decoder rather than pieces a 
bit everywhere, and each decoder needs to be upgraded anyway.

> The codecs for which 3b) applies include at least:
>
> * jpeg2000

Our use case.

> * ffv1

FFV1 has its own flags internally for interlaced content (interleaved 
method only) and I expect no work for separated fields. the MXF/FFV1 
spec does not plan separated fields for FFV1, and there is no byte in 
the essence label for that.


> * rawvideo
> * tiff

I didn't find specifications for the essence label UL corresponding and 
I have no file for that, as far as I understand it is highly theoretical 
but if it appears would be only a matter of mapping the MXF signaling to 
the new AVStream field and supporting the feature in the decoders (even 
if we implement the idea of calling the decoder twice, the decoder needs 
to be expanded for this feature).
So IMO no dev to do there too for the moment.

Jérôme