Re: [FFmpeg-devel] [PATCH v3 2/2] lavc/vaapi_encode_h265: Set general_*_constriaint flags with profile

[Mark Thompson <sw@jkqxz.net>]

On 20/03/2024 08:44, Wang, Fei W wrote:
> On Mon, 2024-03-18 at 21:22 +0000, Mark Thompson wrote:
>> On 18/03/2024 04:21, fei.w.wang-at-intel.com@ffmpeg.org wrote:
>>> From: Fei Wang <fei.w.wang@intel.com>
>>>
>>> According to Table A.2 in spec.
>>>
>>> Signed-off-by: Fei Wang <fei.w.wang@intel.com>
>>> ---
>>>    libavcodec/vaapi_encode_h265.c | 176 +++++++++++++++++++++++-----
>>> -----
>>>    1 file changed, 123 insertions(+), 53 deletions(-)
>>>
>>> diff --git a/libavcodec/vaapi_encode_h265.c
>>> b/libavcodec/vaapi_encode_h265.c
>>> index 43755e2188..5ed317ce11 100644
>>> --- a/libavcodec/vaapi_encode_h265.c
>>> +++ b/libavcodec/vaapi_encode_h265.c
>>> @@ -258,6 +258,124 @@ fail:
>>>        return err;
>>>    }
>>>    
>>> +static int vaapi_encode_h265_init_ptl(AVCodecContext *avctx)
>>> +{
>>> +    VAAPIEncodeContext      *ctx = avctx->priv_data;
>>> +    VAAPIEncodeH265Context *priv = avctx->priv_data;
>>> +    H265RawVPS              *vps = &priv->raw_vps;
>>> +    H265RawProfileTierLevel *ptl = &vps->profile_tier_level;
>>> +
>>> +    ptl->general_profile_space = 0;
>>> +    ptl->general_profile_idc   = avctx->profile;
>>> +    ptl->general_tier_flag     = priv->tier;
>>> +
>>> +    ptl->general_profile_compatibility_flag[ptl-
>>>> general_profile_idc] = 1;
>>> +
>>> +    if (ptl->general_profile_compatibility_flag[1])
>>> +        ptl->general_profile_compatibility_flag[2] = 1;
>>> +    if (ptl->general_profile_compatibility_flag[3]) {
>>> +        ptl->general_profile_compatibility_flag[1] = 1;
>>> +        ptl->general_profile_compatibility_flag[2] = 1;
>>> +    }
>>> +
>>> +    ptl->general_progressive_source_flag    = 1;
>>> +    ptl->general_interlaced_source_flag     = 0;
>>> +    ptl->general_non_packed_constraint_flag = 1;
>>> +    ptl->general_frame_only_constraint_flag = 1;
>>> +
>>> +    if (avctx->profile >= 4) {
>>> +        ptl->general_intra_constraint_flag            = ctx-
>>>> gop_size == 1;
>>> +        ptl->general_one_picture_only_constraint_flag = 0;
>>> +        ptl->general_lower_bit_rate_constraint_flag   = 1;
>>> +        ptl->general_max_14bit_constraint_flag        = 0;
>>> +
>>> +        switch (ctx->va_profile) {
>>> +#if VA_CHECK_VERSION(1, 2, 0)
>>> +        case VAProfileHEVCMain12:
>>> +            // Main 12
>>> +            ptl->general_max_12bit_constraint_flag      = 1;
>>> +            ptl->general_max_10bit_constraint_flag      = 0;
>>> +            ptl->general_max_8bit_constraint_flag       = 0;
>>> +            ptl->general_max_422chroma_constraint_flag  = 1;
>>> +            ptl->general_max_420chroma_constraint_flag  = 1;
>>> +            ptl->general_max_monochrome_constraint_flag = 0;
>>> +            break;
>>> +        case VAProfileHEVCMain422_10:
>>> +            // Main 4:2:2 10
>>> +            ptl->general_max_12bit_constraint_flag      = 1;
>>> +            ptl->general_max_10bit_constraint_flag      = 1;
>>> +            ptl->general_max_8bit_constraint_flag       = 0;
>>> +            ptl->general_max_422chroma_constraint_flag  = 1;
>>> +            ptl->general_max_420chroma_constraint_flag  = 0;
>>> +            ptl->general_max_monochrome_constraint_flag = 0;
>>> +            break;
>>> +        case VAProfileHEVCMain422_12:
>>> +            // Main 4:2:2 12
>>> +            ptl->general_max_12bit_constraint_flag      = 1;
>>> +            ptl->general_max_10bit_constraint_flag      = 0;
>>> +            ptl->general_max_8bit_constraint_flag       = 0;
>>> +            ptl->general_max_422chroma_constraint_flag  = 1;
>>> +            ptl->general_max_420chroma_constraint_flag  = 0;
>>> +            ptl->general_max_monochrome_constraint_flag = 0;
>>> +            break;
>>> +        case VAProfileHEVCMain444:
>>> +            // Main 4:4:4
>>> +            ptl->general_max_12bit_constraint_flag      = 1;
>>> +            ptl->general_max_10bit_constraint_flag      = 1;
>>> +            ptl->general_max_8bit_constraint_flag       = 1;
>>> +            ptl->general_max_422chroma_constraint_flag  = 0;
>>> +            ptl->general_max_420chroma_constraint_flag  = 0;
>>> +            ptl->general_max_monochrome_constraint_flag = 0;
>>> +            break;
>>> +        case VAProfileHEVCMain444_10:
>>> +            // Main 4:4:4 10
>>> +            ptl->general_max_12bit_constraint_flag      = 1;
>>> +            ptl->general_max_10bit_constraint_flag      = 1;
>>> +            ptl->general_max_8bit_constraint_flag       = 0;
>>> +            ptl->general_max_422chroma_constraint_flag  = 0;
>>> +            ptl->general_max_420chroma_constraint_flag  = 0;
>>> +            ptl->general_max_monochrome_constraint_flag = 0;
>>> +            break;
>>> +        case VAProfileHEVCMain444_12:
>>> +            // Main 4:4:4 12
>>> +            ptl->general_max_12bit_constraint_flag      = 1;
>>> +            ptl->general_max_10bit_constraint_flag      = 0;
>>> +            ptl->general_max_8bit_constraint_flag       = 0;
>>> +            ptl->general_max_422chroma_constraint_flag  = 0;
>>> +            ptl->general_max_420chroma_constraint_flag  = 0;
>>> +            ptl->general_max_monochrome_constraint_flag = 0;
>>> +            break;
>>> +#endif
>>> +        default:
>>> +            av_log(avctx, AV_LOG_ERROR, "Unknown profile to init
>>> PTL.\n");
>>> +            return AVERROR(EINVAL);
>>> +        }
>>> +    }
>>
>> Why is this an improvement over the current code which sets the
>> constraint flags based on the actual content of the stream?
>>
>> Note the requirement in A.3.5 for decoder support:
>>
>> general_profile_idc is equal to 4 or
>> general_profile_compatibility_flag[ 4 ] is equal to 1 for the
>> bitstream, and
>> the value of each constraint flag listed in Table A.2 is greater than
>> or equal to the value(s) specified in the row
>> of Table A.2 for the format range extensions profile for which the
>> decoder conformance is evaluated.
>>
>> which says that decoders must be able to support streams which set
>> additional constraint flags beyond those which are specified for a
>> particular profile (in particular, an 8-bit 4:2:2 stream must be
>> decodable by any decoder supporting either "Main 4:2:2 10" or "Main
>> 4:4:4", so ideally it shouldn't be marked to require one of those in
>> particular).
> 
> That's a capability request for decoder. For encoder side, I'd prefer
> to use the most typical values. The constraint flags in Table A.2/3/5
> are strictly checked to get rext profile in ff_h265_get_profile(), and
> seems the constraint flags is the only way to distinguish profiles in
> rext(all the rext profiles shares same general_profile_idc or
> general_profile_compatibility_flag[ 4 ]).

Hmm.  I was missing that the VAAPI decoder has fixed matching - it only accepts things which exactly match the hull of each profile, not ones inside it which are compatible.  That seems worth fixing.

Do other decoder implementations do the same thing with strict matching?  libavcodec's software decoder doesn't - the profile isn't really relevant to it.  nvdec doesn't obviously do anything there either because it accepts all rext streams, though presumably some of them fail later.

Thanks,

- Mark
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".