From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTP id 6226A4B219 for ; Mon, 1 Jul 2024 21:27:18 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 97C7768D7F0; Tue, 2 Jul 2024 00:27:14 +0300 (EEST) Received: from btbn.de (btbn.de [144.76.60.213]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 01A3968D681 for ; Tue, 2 Jul 2024 00:27:08 +0300 (EEST) Received: from [authenticated] by btbn.de (Postfix) with ESMTPSA id 0148A27FFD5A1 for ; Mon, 1 Jul 2024 23:27:06 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rothenpieler.org; s=mail; t=1719869227; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=u4M71xWPB6CtCch58q0P1jJpio/vLBGeS0usnhGeOtg=; b=QE2q+O89WCX0PUrS3tEKMV27WdGSh5H6vtEaUSoPammVnBcScEfUXaTkgL7StMM2261+Se xrgPdOd6Kcv4GTQezYreyAnG3TO8aohA4puBqkofjtSdwgbN1pftS0zESUYwuRTghi9DSn 0xdHOvPftId9RRy7fKJ0lWwwIxqUpGH1I4QYQM1hino22XnkEvQ6j5grr4tdsrpNZpxXlR MqUTe4Cs79vmQH6R/8CBVx4hLkcktMECdGhVpALsFA+diPeQTd6MeyqXZ8gRB2DVDbcwct fJGL+yuhN6M30o/X8bnqiRO/zlSLd08PSiIhOAUVUtTCgMWzAYEsVIxkq1G1NQ== Message-ID: Date: Mon, 1 Jul 2024 23:27:07 +0200 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird To: ffmpeg-devel@ffmpeg.org References: <20240621003355.3301-1-joshua.allmann@gmail.com> Content-Language: en-US From: Timo Rothenpieler In-Reply-To: Subject: Re: [FFmpeg-devel] [PATCH] avcodec/nvenc: Fix segfault with intra-only X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: On 01.07.2024 20:05, Dennis Mungai wrote: > On Mon, 1 Jul 2024 at 21:01, Timo Rothenpieler > wrote: > >> On 01.07.2024 18:52, Josh Allmann wrote: >>> On Thu, 20 Jun 2024 at 17:39, Josh Allmann >> wrote: >>>> >>>> In intra-only mode, frameIntervalP is 0, which means the frame >>>> data array is smaller than the number of surfaces. This causes a >>>> crash when closing the encoder. >>>> >>>> Fix this by making sure the frame data array is at least as big as >>>> the number of surfaces. >>>> --- >>>> libavcodec/nvenc.c | 2 +- >>>> 1 file changed, 1 insertion(+), 1 deletion(-) >>>> >>>> diff --git a/libavcodec/nvenc.c b/libavcodec/nvenc.c >>>> index a9945355ba..93e87b21db 100644 >>>> --- a/libavcodec/nvenc.c >>>> +++ b/libavcodec/nvenc.c >>>> @@ -1021,7 +1021,7 @@ static av_cold int >> nvenc_recalc_surfaces(AVCodecContext *avctx) >>>> >>>> // Output in the worst case will only start when the surface >> buffer is completely full. >>>> // Hence we need to keep at least the max amount of surfaces plus >> the max reorder delay around. >>>> - ctx->frame_data_array_nb = ctx->nb_surfaces + >> ctx->encode_config.frameIntervalP - 1; >>>> + ctx->frame_data_array_nb = FFMAX(ctx->nb_surfaces, >> ctx->nb_surfaces + ctx->encode_config.frameIntervalP - 1); >>>> >>>> return 0; >>>> } >>>> -- >>>> 2.39.2 >>>> >>> >>> Hello, >>> >>> Ping for review. This patch fixes an easily triggered crash with nvenc >>> in intra-only mode, eg >>> >>> ffmpeg -i -c:v h264_nvenc -g 0 >>> >>> Josh >> > > Can this also be back-ported to FFmpeg 7.x upon merging? Already is, applies to 7.0 and 6.x _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".