From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTP id 52B214906F for ; Mon, 1 Jul 2024 18:01:39 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 23C1368D72C; Mon, 1 Jul 2024 21:01:37 +0300 (EEST) Received: from btbn.de (btbn.de [144.76.60.213]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 590A968D5DE for ; Mon, 1 Jul 2024 21:01:31 +0300 (EEST) Received: from [authenticated] by btbn.de (Postfix) with ESMTPSA id 6CF7527D76BBD for ; Mon, 1 Jul 2024 20:01:30 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rothenpieler.org; s=mail; t=1719856890; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=lXPyT/v5/Y/5W81wr6oT7lDkowNDIvR3omI5esdDD8Q=; b=qByNAJC5Z2D84IgcPI9S6HERBbK2giIAE66z4dKl4YqE2lchduYvLvzOnrGl6cfDz35P1a DO0M60FFUoBorKQofvDhqLlIdDO1YTK3A+1w7Pz4PxMz3CdHUmBX9COwpet0ZPtpJtPfYA LyHXX2tbMTKpOrXuPSPucmw5HBgzy1apZth1fufXz3mtxXIPISc3hLvZg6FYmnXVeKq3JN gOyo7TAJS4w5mcoKfRctpCKGqdbJuWq2NRamN2K/JqNKJH41GYl4EOaqAF/R7X+VtzNFZU +xUt7UsGJYbhT2j+vJa53BAn0i8MxF7WiDNcmNqqoi6KyRitIhK+R9GvkJKFhw== Message-ID: Date: Mon, 1 Jul 2024 20:01:28 +0200 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird To: ffmpeg-devel@ffmpeg.org References: <20240621003355.3301-1-joshua.allmann@gmail.com> Content-Language: en-US From: Timo Rothenpieler In-Reply-To: Subject: Re: [FFmpeg-devel] [PATCH] avcodec/nvenc: Fix segfault with intra-only X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: On 01.07.2024 18:52, Josh Allmann wrote: > On Thu, 20 Jun 2024 at 17:39, Josh Allmann wrote: >> >> In intra-only mode, frameIntervalP is 0, which means the frame >> data array is smaller than the number of surfaces. This causes a >> crash when closing the encoder. >> >> Fix this by making sure the frame data array is at least as big as >> the number of surfaces. >> --- >> libavcodec/nvenc.c | 2 +- >> 1 file changed, 1 insertion(+), 1 deletion(-) >> >> diff --git a/libavcodec/nvenc.c b/libavcodec/nvenc.c >> index a9945355ba..93e87b21db 100644 >> --- a/libavcodec/nvenc.c >> +++ b/libavcodec/nvenc.c >> @@ -1021,7 +1021,7 @@ static av_cold int nvenc_recalc_surfaces(AVCodecContext *avctx) >> >> // Output in the worst case will only start when the surface buffer is completely full. >> // Hence we need to keep at least the max amount of surfaces plus the max reorder delay around. >> - ctx->frame_data_array_nb = ctx->nb_surfaces + ctx->encode_config.frameIntervalP - 1; >> + ctx->frame_data_array_nb = FFMAX(ctx->nb_surfaces, ctx->nb_surfaces + ctx->encode_config.frameIntervalP - 1); >> >> return 0; >> } >> -- >> 2.39.2 >> > > Hello, > > Ping for review. This patch fixes an easily triggered crash with nvenc > in intra-only mode, eg > > ffmpeg -i -c:v h264_nvenc -g 0 > > Josh > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe". It's the wrong fix for the crash, but definitely a sensible change in itself. Will amend with the actual fix (using the right size when deallocating that array) but keep this change as well, since it makes sense. _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".