From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTP id 7C68441D3D for ; Mon, 21 Feb 2022 12:06:50 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 8951368B185; Mon, 21 Feb 2022 14:06:47 +0200 (EET) Received: from mail-wr1-f42.google.com (mail-wr1-f42.google.com [209.85.221.42]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id C0BCA68B161 for ; Mon, 21 Feb 2022 14:06:40 +0200 (EET) Received: by mail-wr1-f42.google.com with SMTP id u1so26659892wrg.11 for ; Mon, 21 Feb 2022 04:06:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jkqxz-net.20210112.gappssmtp.com; s=20210112; h=message-id:date:mime-version:user-agent:subject:content-language:to :references:from:in-reply-to:content-transfer-encoding; bh=AaYUXE4987QWnksij19YXHSjv/WfR1B1VgvT2W1GE50=; b=BxpyDhNG2gXKmIk0gjIrZRRhv5yeAdP8IFgxLAe7zaUObt0ASiC2H+wn5e4y8AMIPu m6AIgE+i0agcEDXKQtV0gJ2qPPp4rZF3HR0QphKs2bGWrOyXXJPSlQEA3xEm58E61Lal wFPKU/gJ7J9VA/a474Ax2aYFz8n6ucUcip7p37DrC6VQlYWChFx7dpv/UAGZpMcZH07C SNxxtAKIywsQu3e08zs0u+rqiKZVZZr+8xn/2vTtJtSqlhw1pFRbDSLkvpsEqAOa+aZW 4fE/V3Cbw9VWjdUwbzTtl/97sMMOXf0zlgu4Lox0MSEZ2HqEDa+r4FCJRnSKgNHJQXkT 8VyQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:date:mime-version:user-agent:subject :content-language:to:references:from:in-reply-to :content-transfer-encoding; bh=AaYUXE4987QWnksij19YXHSjv/WfR1B1VgvT2W1GE50=; b=nNSYMgVBzVduIcp3i6ecNBqzVAvjRIyrtuggjMhz1AUgmrPZ2uD/0YS5kpP4ll1rLl iDQ0bnRlL+berGA87PZnfYjRg9B4Sz6hppcPBkBFTZLZJsreOYW4smRbY7Nbwb6OiHti q5CPPu09P3k4JoJ+z6OJT4JJ4ZXJDDEp+4yGS6X0Bn92R1GSe/AfXOMgcp+EkL8Lys+P WCeh0lgS7DAT7C9WyUjCv6Ybd5GkifWjfwTCLCA30iwm2Q5a/iu+jCXyBS+Rv7TZ4qoQ ZmtJaq9ac3S1601S0vexlBkC6MYydUuG3QxSPNO2MWIjL/NZeUedQcI5sse2nGZu0pt4 eHVw== X-Gm-Message-State: AOAM530dL57o2UupMfbvEDDYmnd7lx/KLGOQpmgYQdFvSPejvhII8O2M ah2uKUTsLMrlR37AWUMZvXAYQwZVfzle4A== X-Google-Smtp-Source: ABdhPJxKZT4pwhOMvvw3hMaKmVEHAy0dqy6nHiQo78qqwUhT2QXuJljD2ijsz1we8KGHtn8eyq1iog== X-Received: by 2002:a5d:6d05:0:b0:1e5:948e:8f58 with SMTP id e5-20020a5d6d05000000b001e5948e8f58mr15587257wrq.322.1645445200002; Mon, 21 Feb 2022 04:06:40 -0800 (PST) Received: from [192.168.0.11] (cpc91222-cmbg18-2-0-cust46.5-4.cable.virginm.net. [81.106.30.47]) by smtp.gmail.com with ESMTPSA id l7sm30851704wrs.46.2022.02.21.04.06.39 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 21 Feb 2022 04:06:39 -0800 (PST) Message-ID: <03ec8c31-088a-95b6-a514-6fbf14518537@jkqxz.net> Date: Mon, 21 Feb 2022 12:06:36 +0000 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.6.0 Content-Language: en-US To: ffmpeg-devel@ffmpeg.org References: <20220221021337.2190359-1-fei.w.wang@intel.com> From: Mark Thompson In-Reply-To: <20220221021337.2190359-1-fei.w.wang@intel.com> Subject: Re: [FFmpeg-devel] [PATCH v2 1/4] lavc/vaapi_encode_h265: Add P frame to GPB frame support for hevc_vaapi X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: On 21/02/2022 02:13, Fei Wang wrote: > From: Linjie Fu > > Use GPB frames to replace regular P frames if backend driver does not > support it. > > - GPB: > Generalized P and B picture. P frames replaced by B frames with > forward-predict only, L0 == L1. Normal B frames still have 2 > different ref_lists and allow bi-prediction > > Signed-off-by: Linjie Fu > Signed-off-by: Fei Wang > --- > libavcodec/vaapi_encode.c | 33 +++++++++++++++++++++++++++++++-- > libavcodec/vaapi_encode.h | 1 + > libavcodec/vaapi_encode_h265.c | 15 +++++++++++++++ > 3 files changed, 47 insertions(+), 2 deletions(-) This always fails immediately on current iHD (7e357b4bea76b2fe2522e6af41ae02ea69cec49e): $ ./ffmpeg_g -v 44 -y -hwaccel vaapi -hwaccel_output_format vaapi -i in.mp4 -an -c:v hevc_vaapi -low_power 1 out.mp4 ... [hevc_vaapi @ 0x560e81d45e80] Using input frames context (format vaapi) with hevc_vaapi encoder. [hevc_vaapi @ 0x560e81d45e80] Input surface format is nv12. [hevc_vaapi @ 0x560e81d45e80] Using VAAPI profile VAProfileHEVCMain (17). [hevc_vaapi @ 0x560e81d45e80] Using VAAPI entrypoint VAEntrypointEncSliceLP (8). [hevc_vaapi @ 0x560e81d45e80] Using VAAPI render target format YUV420 (0x1). [hevc_vaapi @ 0x560e81d45e80] Using CTU size 64x64, min CB size 8x8. [hevc_vaapi @ 0x560e81d45e80] No quality level set; using default (25). [hevc_vaapi @ 0x560e81d45e80] RC mode: CQP. [hevc_vaapi @ 0x560e81d45e80] RC quality: 25. [hevc_vaapi @ 0x560e81d45e80] RC framerate: 30000/1001 (29.97 fps). [hevc_vaapi @ 0x560e81d45e80] Use GPB B frames to replace regular P frames. [hevc_vaapi @ 0x560e81d45e80] Using intra, GPB-B-frames and B-frames (supported references: 3 / 3). [hevc_vaapi @ 0x560e81d45e80] All wanted packed headers available (wanted 0xd, found 0x1f). [hevc_vaapi @ 0x560e81d45e80] Using level 4. ... [hevc_vaapi @ 0x560e81d45e80] Failed to end picture encode issue: 24 (internal encoding error). [hevc_vaapi @ 0x560e81d45e80] Encode failed: -5. Video encoding failed ... $ cat /proc/cpuinfo | grep 'model name' | head -1 model name : Intel(R) Core(TM) i7-1065G7 CPU @ 1.30GHz $ uname -v #1 SMP PREEMPT Debian 5.16.7-2 (2022-02-09) Do you get this too, or is your setup different somehow? > diff --git a/libavcodec/vaapi_encode.c b/libavcodec/vaapi_encode.c > index 3bf379b1a0..95eca7c288 100644 > --- a/libavcodec/vaapi_encode.c > +++ b/libavcodec/vaapi_encode.c > @@ -1845,6 +1845,30 @@ static av_cold int vaapi_encode_init_gop_structure(AVCodecContext *avctx) > ref_l1 = attr.value >> 16 & 0xffff; > } > > + ctx->p_to_gpb = 0; > + > +#if VA_CHECK_VERSION(1, 9, 0) > + attr = (VAConfigAttrib) { VAConfigAttribPredictionDirection }; > + vas = vaGetConfigAttributes(ctx->hwctx->display, > + ctx->va_profile, > + ctx->va_entrypoint, > + &attr, 1); This probably shouldn't be done at all if the user has selected a codec without B-frames or a configuration which is intra-only, because the log message is confusing: [mjpeg_vaapi @ 0x55b90d72ee00] Driver does not report whether support GPB, use regular P frames. [mjpeg_vaapi @ 0x55b90d72ee00] Using intra frames only. > + if (vas != VA_STATUS_SUCCESS) { > + av_log(avctx, AV_LOG_WARNING, "Failed to query prediction direction " > + "attribute: %d (%s).\n", vas, vaErrorStr(vas)); And fail? > + } else if (attr.value == VA_ATTRIB_NOT_SUPPORTED) { > + av_log(avctx, AV_LOG_VERBOSE, "Driver does not report whether " > + "support GPB, use regular P frames.\n"); "support GPB" is a strange thing to say. It's a constraint - any driver which supports B-frames will let you have the same thing in both RefPicLists, but some require it because they don't support P-frames. So maybe something like "Driver does not report any additional prediction constraints, using P-frames."? > + } else { > + if (attr.value & VA_PREDICTION_DIRECTION_BI_NOT_EMPTY) { > + ctx->p_to_gpb = 1; > + av_log(avctx, AV_LOG_VERBOSE, "Use GPB B frames to replace " > + "regular P frames.\n"); Maybe "Driver does not support P-frames, replacing them with B-frames."? > + } else > + av_log(avctx, AV_LOG_VERBOSE, "Use regular P frames.\n"); > + } > +#endif > + > if (ctx->codec->flags & FLAG_INTRA_ONLY || > avctx->gop_size <= 1) { > av_log(avctx, AV_LOG_VERBOSE, "Using intra frames only.\n"); > @@ -1861,8 +1885,13 @@ static av_cold int vaapi_encode_init_gop_structure(AVCodecContext *avctx) > ctx->p_per_i = INT_MAX; > ctx->b_per_p = 0; > } else { > - av_log(avctx, AV_LOG_VERBOSE, "Using intra, P- and B-frames " > - "(supported references: %d / %d).\n", ref_l0, ref_l1); > + if (ctx->p_to_gpb) > + av_log(avctx, AV_LOG_VERBOSE, "Using intra, GPB-B-frames and " > + "B-frames (supported references: %d / %d).\n", > + ref_l0, ref_l1); Seems easier to just say intra and B-frames (though this isn't really adding anything to the message above - it's still effectively a P-frame referring to one previous frame, just using B-slices to do it in a strange way). > + else > + av_log(avctx, AV_LOG_VERBOSE, "Using intra, P- and B-frames " > + "(supported references: %d / %d).\n", ref_l0, ref_l1); > ctx->gop_size = avctx->gop_size; > ctx->p_per_i = INT_MAX; > ctx->b_per_p = avctx->max_b_frames; > diff --git a/libavcodec/vaapi_encode.h b/libavcodec/vaapi_encode.h > index b41604a883..61c5615eb8 100644 > --- a/libavcodec/vaapi_encode.h > +++ b/libavcodec/vaapi_encode.h > @@ -328,6 +328,7 @@ typedef struct VAAPIEncodeContext { > int idr_counter; > int gop_counter; > int end_of_stream; > + int p_to_gpb; > > // Whether the driver supports ROI at all. > int roi_allowed; > diff --git a/libavcodec/vaapi_encode_h265.c b/libavcodec/vaapi_encode_h265.c > index e1dc53dfa9..ea45893508 100644 > --- a/libavcodec/vaapi_encode_h265.c > +++ b/libavcodec/vaapi_encode_h265.c > @@ -886,6 +886,7 @@ static int vaapi_encode_h265_init_slice_params(AVCodecContext *avctx, > VAAPIEncodePicture *pic, > VAAPIEncodeSlice *slice) > { > + VAAPIEncodeContext *ctx = avctx->priv_data; > VAAPIEncodeH265Context *priv = avctx->priv_data; > VAAPIEncodeH265Picture *hpic = pic->priv_data; > const H265RawSPS *sps = &priv->raw_sps; > @@ -908,6 +909,9 @@ static int vaapi_encode_h265_init_slice_params(AVCodecContext *avctx, > > sh->slice_type = hpic->slice_type; > > + if (sh->slice_type == HEVC_SLICE_P && ctx->p_to_gpb) > + sh->slice_type = HEVC_SLICE_B; > + > sh->slice_pic_order_cnt_lsb = hpic->pic_order_cnt & > (1 << (sps->log2_max_pic_order_cnt_lsb_minus4 + 4)) - 1; > > @@ -1066,6 +1070,9 @@ static int vaapi_encode_h265_init_slice_params(AVCodecContext *avctx, > av_assert0(pic->type == PICTURE_TYPE_P || > pic->type == PICTURE_TYPE_B); > vslice->ref_pic_list0[0] = vpic->reference_frames[0]; > + if (ctx->p_to_gpb && pic->type == PICTURE_TYPE_P) > + // Reference for GPB B-frame, L0 == L1 > + vslice->ref_pic_list1[0] = vpic->reference_frames[0]; > } > if (pic->nb_refs >= 2) { > // Forward reference for B-frame. > @@ -1073,6 +1080,14 @@ static int vaapi_encode_h265_init_slice_params(AVCodecContext *avctx, > vslice->ref_pic_list1[0] = vpic->reference_frames[1]; > } > > + if (pic->type == PICTURE_TYPE_P && ctx->p_to_gpb) { > + vslice->slice_type = HEVC_SLICE_B; > + for (i = 0; i < FF_ARRAY_ELEMS(vslice->ref_pic_list0); i++) { > + vslice->ref_pic_list1[i].picture_id = vslice->ref_pic_list0[i].picture_id; > + vslice->ref_pic_list1[i].flags = vslice->ref_pic_list0[i].flags; > + } > + } Is it sensible to implement this properly in the generic code rather than having special ad-hoc code here? (Is there any chance this will also be a thing for e.g. H.264 or H.266 in future?) > + > return 0; > } > - Mark _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".