From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTP id 0C67A46106 for ; Mon, 5 Jun 2023 12:29:46 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 7B4CC68C404; Mon, 5 Jun 2023 15:29:43 +0300 (EEST) Received: from mail-oi1-f172.google.com (mail-oi1-f172.google.com [209.85.167.172]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 6A9DA68C3FB for ; Mon, 5 Jun 2023 15:29:37 +0300 (EEST) Received: by mail-oi1-f172.google.com with SMTP id 5614622812f47-39b86f95a66so170287b6e.1 for ; Mon, 05 Jun 2023 05:29:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=visionlabs-ai.20221208.gappssmtp.com; s=20221208; t=1685968175; x=1688560175; h=to:subject:message-id:date:from:in-reply-to:references:mime-version :from:to:cc:subject:date:message-id:reply-to; bh=oPTNWfOGe7ICrLMUpv5vOChQdgD1O+6kIiMTWC10M9c=; b=Ljws1ruVfQWVp/1Grha0YP8yKHR3KDLsr2yCkkCcyMHLHLhAsNTU0Y1aA0RTo5She4 tRgBxrYxVLUHFRFUABwIcU7ku99iIl9gXPBMjKoy4kwOvnGEauK6DSCyH/rci2thxxpu V+iwIveCBUF3R03SnbtC1/GaeR1vg5PAr9CjDvUm96hG2noIVZreWazc+LjoNB+Gioic vJmHWBiZN9UPh2iPY03JqZicQ2HE86LL80XRwbpUNnFHgkfjG7I8sk+JyBzRC3JmMWLl WcwtfM23xmnoPzsYw5RUlqbb0GcmqrmndBjhoRKsFvOk0tf2oUih2HMacYLXSjdM0jTx eYig== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1685968175; x=1688560175; h=to:subject:message-id:date:from:in-reply-to:references:mime-version :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=oPTNWfOGe7ICrLMUpv5vOChQdgD1O+6kIiMTWC10M9c=; b=jFW2+xrSOK74sVOIw4l+U/Bt9UtbPV5nJOHCjED4blzJ9RyER6opxvPl1+e9DdrRVa ZUPiCxWfB1aEiM7lKq0moF1I5mtu8vssBL2A2bvfz9TvxV5A+/LTYDBkC+5ORZYBy7RY 3pAQCgFivHjmE4aapa2x71bsQP5EV7dq5DGJRK9zyEEcz6caoM3skjT8guad2a740Ms/ 9CRvvoa/lE7/CGlInayGPfT68jPz5XdskZYsfTgt2gAzO9jV09fySv5G66S96KUo4fK4 6mX42sJOTcLcdYhzLr23xOyo8E6IlTRfYNUSxD2t4EnVNMhvwtI+GzhBQH4+7d4AvGPU 1naw== X-Gm-Message-State: AC+VfDwlAPDoOef/bnJ+f/RB+TjoKujRNuidM9+Xq9C6Jkmgn/QWWrCC 1fl32WLsX2iKhpeDShIaEZ5zk9giAYHCpvsLLkqQXGehw/VI1mM4xNM= X-Google-Smtp-Source: ACHHUZ5aQ7wjwBO9y6WW/6nqO0QxhMWELnyke1o7dLSbq+CMD+4cIydOtMZ3Ami+RKmzFetAtYa5QWjejg5tbPsrD1g= X-Received: by 2002:a54:4102:0:b0:398:1807:84bc with SMTP id l2-20020a544102000000b00398180784bcmr4167989oic.26.1685968174909; Mon, 05 Jun 2023 05:29:34 -0700 (PDT) MIME-Version: 1.0 References: <168596036850.3843.16759199081478040168@lain.khirnov.net> In-Reply-To: <168596036850.3843.16759199081478040168@lain.khirnov.net> From: Roman Arzumanyan Date: Mon, 5 Jun 2023 15:29:23 +0300 Message-ID: To: FFmpeg development discussions and patches Content-Type: multipart/mixed; boundary="0000000000002bad7c05fd610e7e" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 Subject: Re: [FFmpeg-devel] [PATCH] avcodec/cuviddec: update amount of decoder surfaces from within sequence decode callback X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: --0000000000002bad7c05fd610e7e Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Thanks for the review, Anton. Please find the updated patch attached. BTW - what do you mean by "you should also forward the actual error code"? Within the cuvid_handle_video_sequence() function in case of error, function still returns 0 and sets internal context error, so I kept behavior intact. Do I miss something? =D0=BF=D0=BD, 5 =D0=B8=D1=8E=D0=BD. 2023=E2=80=AF=D0=B3. =D0=B2 13:19, Anto= n Khirnov : > Quoting Roman Arzumanyan (2023-06-05 09:30:07) > > Hello, > > > > This patch reduces vRAM usage by cuvid decoder implementation. > > The number of surfaces used for decoding is updated within the parser > > sequence decode callback. > > Also the "surfaces" AVDictionary option specific to cuvid was removed i= n > > favor of "extra_hw_surfaces". > > This can break existing workflows, you should deprecated the option > instead and only remove it after some time has passed. > > > > > vRAM consumption was tested on various videos and savings are between 1= % > > for 360p resolution up to 21% for some 1080p H.264 videos. > > Decoding performance was tested on various H.264 and H.265 videos in > > different resolutions from 360p and higher, no performance penalty was > > found. > > > > From 32a1b016e88fa40b983318d4583750ef250a78d9 Mon Sep 17 00:00:00 2001 > > From: Roman Arzumanyan > > Date: Thu, 1 Jun 2023 11:17:39 +0300 > > Subject: [PATCH] libavcodec/cuviddec: determine DPB size from within > cuvid > > parser > > > > --- > > libavcodec/cuviddec.c | 29 +++++++++++++++++++++++++++-- > > 1 file changed, 27 insertions(+), 2 deletions(-) > > > > diff --git a/libavcodec/cuviddec.c b/libavcodec/cuviddec.c > > index 3d43bbd466..759ed49870 100644 > > --- a/libavcodec/cuviddec.c > > +++ b/libavcodec/cuviddec.c > > @@ -115,6 +115,12 @@ typedef struct CuvidParsedFrame > > > > #define CHECK_CU(x) FF_CUDA_CHECK_DL(avctx, ctx->cudl, x) > > > > +// NV recommends [2;4] range > > +#define CUVID_MAX_DISPLAY_DELAY (4) > > + > > +// Actual DPB size will be determined by parser. > > +#define CUVID_DEFAULT_NUM_SURFACES (CUVID_MAX_DISPLAY_DELAY + 1) > > + > > static int CUDAAPI cuvid_handle_video_sequence(void *opaque, > CUVIDEOFORMAT* format) > > { > > AVCodecContext *avctx =3D opaque; > > @@ -309,6 +315,25 @@ static int CUDAAPI cuvid_handle_video_sequence(voi= d > *opaque, CUVIDEOFORMAT* form > > return 0; > > } > > > > + if (ctx->nb_surfaces < format->min_num_decode_surfaces + 3) > > + ctx->nb_surfaces =3D format->min_num_decode_surfaces + 3; > > FFMAX() > > > + > > + if (avctx->extra_hw_frames > 0) > > + ctx->nb_surfaces +=3D avctx->extra_hw_frames; > > + > > + if (0 > av_fifo_realloc2(ctx->frame_queue, ctx->nb_surfaces * > sizeof(CuvidParsedFrame))) { > > this is the old deprecated AVFifoBuffer API, you cannot use it with > AVFifo objects > > you should also forward the actual error code > > > + av_log(avctx, AV_LOG_ERROR, "Failed to recreate frame queue on > video sequence callback\n"); > > + ctx->internal_error =3D AVERROR(EINVAL); > > + return 0; > > + } > > + > > + ctx->key_frame =3D av_realloc_array(ctx->key_frame, ctx->nb_surfac= es, > sizeof(int)); > > + if (!ctx->key_frame) { > > + av_log(avctx, AV_LOG_ERROR, "Failed to recreate key frame queu= e > on video sequence callback\n"); > > + ctx->internal_error =3D AVERROR(EINVAL); > > Leaks key_frame on failure and should be ENOMEM. > > -- > Anton Khirnov > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". > --0000000000002bad7c05fd610e7e Content-Type: text/x-patch; charset="US-ASCII"; name="0001-libavcodec-cuviddec-determine-amount-of-decoded-surf.patch" Content-Disposition: attachment; filename="0001-libavcodec-cuviddec-determine-amount-of-decoded-surf.patch" Content-Transfer-Encoding: base64 Content-ID: X-Attachment-Id: f_liitr3e00 RnJvbSA0YWVmZjM2YzlkNzljMDQ2ZTcyNmJjYzE4YTMxMTc1NGFjZTVjNDdlIE1vbiBTZXAgMTcg MDA6MDA6MDAgMjAwMQpGcm9tOiBSb21hbiBBcnp1bWFueWFuIDxyLmFyenVtYW55YW5AdmlzaW9u bGFicy5haT4KRGF0ZTogVGh1LCAxIEp1biAyMDIzIDExOjE3OjM5ICswMzAwClN1YmplY3Q6IFtQ QVRDSF0gbGliYXZjb2RlYy9jdXZpZGRlYzogZGV0ZXJtaW5lIGFtb3VudCBvZiBkZWNvZGVkIHN1 cmZhY2VzCiBmcm9tIHdpdGhpbiBjdXZpZCBwYXJzZXIKCi0tLQogbGliYXZjb2RlYy9jdXZpZGRl Yy5jIHwgMzQgKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKystLQogMSBmaWxlIGNoYW5n ZWQsIDMyIGluc2VydGlvbnMoKyksIDIgZGVsZXRpb25zKC0pCgpkaWZmIC0tZ2l0IGEvbGliYXZj b2RlYy9jdXZpZGRlYy5jIGIvbGliYXZjb2RlYy9jdXZpZGRlYy5jCmluZGV4IDNkNDNiYmQ0NjYu LmVmMjVjMWQ0NzAgMTAwNjQ0Ci0tLSBhL2xpYmF2Y29kZWMvY3V2aWRkZWMuYworKysgYi9saWJh dmNvZGVjL2N1dmlkZGVjLmMKQEAgLTExNSw2ICsxMTUsMTIgQEAgdHlwZWRlZiBzdHJ1Y3QgQ3V2 aWRQYXJzZWRGcmFtZQogCiAjZGVmaW5lIENIRUNLX0NVKHgpIEZGX0NVREFfQ0hFQ0tfREwoYXZj dHgsIGN0eC0+Y3VkbCwgeCkKIAorLy8gTlYgcmVjb21tZW5kcyBbMjs0XSByYW5nZQorI2RlZmlu ZSBDVVZJRF9NQVhfRElTUExBWV9ERUxBWSAoNCkKKworLy8gQWN0dWFsIHBvb2wgc2l6ZSB3aWxs IGJlIGRldGVybWluZWQgYnkgcGFyc2VyLgorI2RlZmluZSBDVVZJRF9ERUZBVUxUX05VTV9TVVJG QUNFUyAoQ1VWSURfTUFYX0RJU1BMQVlfREVMQVkgKyAxKQorCiBzdGF0aWMgaW50IENVREFBUEkg Y3V2aWRfaGFuZGxlX3ZpZGVvX3NlcXVlbmNlKHZvaWQgKm9wYXF1ZSwgQ1VWSURFT0ZPUk1BVCog Zm9ybWF0KQogewogICAgIEFWQ29kZWNDb250ZXh0ICphdmN0eCA9IG9wYXF1ZTsKQEAgLTEyNCw2 ICsxMzAsNyBAQCBzdGF0aWMgaW50IENVREFBUEkgY3V2aWRfaGFuZGxlX3ZpZGVvX3NlcXVlbmNl KHZvaWQgKm9wYXF1ZSwgQ1VWSURFT0ZPUk1BVCogZm9ybQogICAgIENVVklEREVDT0RFQ1JFQVRF SU5GTyBjdWluZm87CiAgICAgaW50IHN1cmZhY2VfZm10OwogICAgIGludCBjaHJvbWFfNDQ0Owor ICAgIGludCBmaWZvX3NpemVfaW5jOwogCiAgICAgaW50IG9sZF93aWR0aCA9IGF2Y3R4LT53aWR0 aDsKICAgICBpbnQgb2xkX2hlaWdodCA9IGF2Y3R4LT5oZWlnaHQ7CkBAIC0zMDksNiArMzE2LDI1 IEBAIHN0YXRpYyBpbnQgQ1VEQUFQSSBjdXZpZF9oYW5kbGVfdmlkZW9fc2VxdWVuY2Uodm9pZCAq b3BhcXVlLCBDVVZJREVPRk9STUFUKiBmb3JtCiAgICAgICAgIHJldHVybiAwOwogICAgIH0KIAor ICAgIGZpZm9fc2l6ZV9pbmMgPSBjdHgtPm5iX3N1cmZhY2VzOworICAgIGN0eC0+bmJfc3VyZmFj ZXMgPSBGRk1BWChjdHgtPm5iX3N1cmZhY2VzLCBmb3JtYXQtPm1pbl9udW1fZGVjb2RlX3N1cmZh Y2VzICsgMyk7CisKKyAgICBpZiAoMCA8IGF2Y3R4LT5leHRyYV9od19mcmFtZXMpCisgICAgICAg IGN0eC0+bmJfc3VyZmFjZXMgKz0gYXZjdHgtPmV4dHJhX2h3X2ZyYW1lczsKKworICAgIGZpZm9f c2l6ZV9pbmMgPSBjdHgtPm5iX3N1cmZhY2VzIC0gZmlmb19zaXplX2luYzsKKyAgICBpZiAoMCA8 IGZpZm9fc2l6ZV9pbmMgJiYgMCA+IGF2X2ZpZm9fZ3JvdzIoY3R4LT5mcmFtZV9xdWV1ZSwgZmlm b19zaXplX2luYykpIHsKKyAgICAgICAgYXZfbG9nKGF2Y3R4LCBBVl9MT0dfRVJST1IsICJGYWls ZWQgdG8gZ3JvdyBmcmFtZSBxdWV1ZSBvbiB2aWRlbyBzZXF1ZW5jZSBjYWxsYmFja1xuIik7Cisg ICAgICAgIGN0eC0+aW50ZXJuYWxfZXJyb3IgPSBBVkVSUk9SKEVOT01FTSk7CisgICAgICAgIHJl dHVybiAwOworICAgIH0KKworICAgIGlmICgwIDwgZmlmb19zaXplX2luYyAmJiAwID4gYXZfcmVh bGxvY3BfYXJyYXkoJmN0eC0+a2V5X2ZyYW1lLCBjdHgtPm5iX3N1cmZhY2VzLCBzaXplb2YoaW50 KSkpIHsKKyAgICAgICAgYXZfbG9nKGF2Y3R4LCBBVl9MT0dfRVJST1IsICJGYWlsZWQgdG8gZ3Jv dyBrZXkgZnJhbWUgYXJyYXkgb24gdmlkZW8gc2VxdWVuY2UgY2FsbGJhY2tcbiIpOworICAgICAg ICBjdHgtPmludGVybmFsX2Vycm9yID0gQVZFUlJPUihFTk9NRU0pOworICAgICAgICByZXR1cm4g MDsKKyAgICB9CisKICAgICBjdWluZm8udWxOdW1EZWNvZGVTdXJmYWNlcyA9IGN0eC0+bmJfc3Vy ZmFjZXM7CiAgICAgY3VpbmZvLnVsTnVtT3V0cHV0U3VyZmFjZXMgPSAxOwogICAgIGN1aW5mby51 bENyZWF0aW9uRmxhZ3MgPSBjdWRhVmlkZW9DcmVhdGVfUHJlZmVyQ1VWSUQ7CkBAIC04NDYsNiAr ODcyLDEwIEBAIHN0YXRpYyBhdl9jb2xkIGludCBjdXZpZF9kZWNvZGVfaW5pdChBVkNvZGVjQ29u dGV4dCAqYXZjdHgpCiAgICAgICAgIGdvdG8gZXJyb3I7CiAgICAgfQogCisgICAgLy8gcmVzcGVj dCB0aGUgZGVwcmVjYXRlZCAic3VyZmFjZXMiIG9wdGlvbiBpZiBub24tZGVmYXVsdCB2YWx1ZSBp cyBnaXZlbiBieSB1c2VyOworICAgIGlmKDI1ID09IGN0eC0+bmJfc3VyZmFjZXMpCisgICAgICAg IGN0eC0+bmJfc3VyZmFjZXMgPSBDVVZJRF9ERUZBVUxUX05VTV9TVVJGQUNFUzsKKwogICAgIGN0 eC0+ZnJhbWVfcXVldWUgPSBhdl9maWZvX2FsbG9jMihjdHgtPm5iX3N1cmZhY2VzLCBzaXplb2Yo Q3V2aWRQYXJzZWRGcmFtZSksIDApOwogICAgIGlmICghY3R4LT5mcmFtZV9xdWV1ZSkgewogICAg ICAgICByZXQgPSBBVkVSUk9SKEVOT01FTSk7CkBAIC05OTMsNyArMTAyMyw3IEBAIHN0YXRpYyBh dl9jb2xkIGludCBjdXZpZF9kZWNvZGVfaW5pdChBVkNvZGVjQ29udGV4dCAqYXZjdHgpCiAgICAg fQogCiAgICAgY3R4LT5jdXBhcnNlaW5mby51bE1heE51bURlY29kZVN1cmZhY2VzID0gY3R4LT5u Yl9zdXJmYWNlczsKLSAgICBjdHgtPmN1cGFyc2VpbmZvLnVsTWF4RGlzcGxheURlbGF5ID0gKGF2 Y3R4LT5mbGFncyAmIEFWX0NPREVDX0ZMQUdfTE9XX0RFTEFZKSA/IDAgOiA0OworICAgIGN0eC0+ Y3VwYXJzZWluZm8udWxNYXhEaXNwbGF5RGVsYXkgPSAoYXZjdHgtPmZsYWdzICYgQVZfQ09ERUNf RkxBR19MT1dfREVMQVkpID8gMCA6IENVVklEX01BWF9ESVNQTEFZX0RFTEFZOwogICAgIGN0eC0+ Y3VwYXJzZWluZm8ucFVzZXJEYXRhID0gYXZjdHg7CiAgICAgY3R4LT5jdXBhcnNlaW5mby5wZm5T ZXF1ZW5jZUNhbGxiYWNrID0gY3V2aWRfaGFuZGxlX3ZpZGVvX3NlcXVlbmNlOwogICAgIGN0eC0+ Y3VwYXJzZWluZm8ucGZuRGVjb2RlUGljdHVyZSA9IGN1dmlkX2hhbmRsZV9waWN0dXJlX2RlY29k ZTsKQEAgLTEwOTcsNyArMTEyNyw3IEBAIHN0YXRpYyBjb25zdCBBVk9wdGlvbiBvcHRpb25zW10g PSB7CiAgICAgeyAiYm9iIiwgICAgICAiQm9iIGRlaW50ZXJsYWNpbmciLCAgICAgICAgICAgICAg ICAgICAgICAgMCwgQVZfT1BUX1RZUEVfQ09OU1QsIHsgLmk2NCA9IGN1ZGFWaWRlb0RlaW50ZXJs YWNlTW9kZV9Cb2IgICAgICB9LCAwLCAwLCBWRCwgImRlaW50IiB9LAogICAgIHsgImFkYXB0aXZl IiwgIkFkYXB0aXZlIGRlaW50ZXJsYWNpbmciLCAgICAgICAgICAgICAgICAgIDAsIEFWX09QVF9U WVBFX0NPTlNULCB7IC5pNjQgPSBjdWRhVmlkZW9EZWludGVybGFjZU1vZGVfQWRhcHRpdmUgfSwg MCwgMCwgVkQsICJkZWludCIgfSwKICAgICB7ICJncHUiLCAgICAgICJHUFUgdG8gYmUgdXNlZCBm b3IgZGVjb2RpbmciLCBPRkZTRVQoY3VfZ3B1KSwgQVZfT1BUX1RZUEVfU1RSSU5HLCB7IC5zdHIg PSBOVUxMIH0sIDAsIDAsIFZEIH0sCi0gICAgeyAic3VyZmFjZXMiLCAiTWF4aW11bSBzdXJmYWNl cyB0byBiZSB1c2VkIGZvciBkZWNvZGluZyIsIE9GRlNFVChuYl9zdXJmYWNlcyksIEFWX09QVF9U WVBFX0lOVCwgeyAuaTY0ID0gMjUgfSwgMCwgSU5UX01BWCwgVkQgfSwKKyAgICB7ICJzdXJmYWNl cyIsICJNYXhpbXVtIHN1cmZhY2VzIHRvIGJlIHVzZWQgZm9yIGRlY29kaW5nIiwgT0ZGU0VUKG5i X3N1cmZhY2VzKSwgQVZfT1BUX1RZUEVfSU5ULCB7IC5pNjQgPSAyNSB9LCAwLCBJTlRfTUFYLCBW RCB8IEFWX09QVF9GTEFHX0RFUFJFQ0FURUQgfSwKICAgICB7ICJkcm9wX3NlY29uZF9maWVsZCIs ICJEcm9wIHNlY29uZCBmaWVsZCB3aGVuIGRlaW50ZXJsYWNpbmciLCBPRkZTRVQoZHJvcF9zZWNv bmRfZmllbGQpLCBBVl9PUFRfVFlQRV9CT09MLCB7IC5pNjQgPSAwIH0sIDAsIDEsIFZEIH0sCiAg ICAgeyAiY3JvcCIsICAgICAiQ3JvcCAodG9wKXgoYm90dG9tKXgobGVmdCl4KHJpZ2h0KSIsIE9G RlNFVChjcm9wX2V4cHIpLCBBVl9PUFRfVFlQRV9TVFJJTkcsIHsgLnN0ciA9IE5VTEwgfSwgMCwg MCwgVkQgfSwKICAgICB7ICJyZXNpemUiLCAgICJSZXNpemUgKHdpZHRoKXgoaGVpZ2h0KSIsIE9G RlNFVChyZXNpemVfZXhwciksIEFWX09QVF9UWVBFX1NUUklORywgeyAuc3RyID0gTlVMTCB9LCAw LCAwLCBWRCB9LAotLSAKMi4zNC4xCgo= --0000000000002bad7c05fd610e7e Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". --0000000000002bad7c05fd610e7e--