From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTP id 994654B011 for ; Mon, 1 Jul 2024 16:53:04 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 48EDD68D656; Mon, 1 Jul 2024 19:53:01 +0300 (EEST) Received: from mail-ed1-f52.google.com (mail-ed1-f52.google.com [209.85.208.52]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id E3F9668D2F7 for ; Mon, 1 Jul 2024 19:52:53 +0300 (EEST) Received: by mail-ed1-f52.google.com with SMTP id 4fb4d7f45d1cf-57d1d614049so717812a12.1 for ; Mon, 01 Jul 2024 09:52:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1719852772; x=1720457572; darn=ffmpeg.org; h=to:subject:message-id:date:from:in-reply-to:references:mime-version :from:to:cc:subject:date:message-id:reply-to; bh=CcGdGPVQ6O1MglhBVG1YcNWN6v0agnwFWTiOkVwa1Ps=; b=mpU9JTtIcpmf8vdBjomZlAeKLmpEfzCcpVE+1Paq8rYMfkp8YDYm2nsIqZJTKb7+OV 0SybH/sxAFCgahM6a7gK3xiFiNV1smw2C8vJ4iVMfKNKau2v9bzQX4wIGwnYqgKg/bN7 4uVKByxgeQjVeAQ0nrB0HERq6kaMbZ378K4nBN/gWx2pfCiBcAd1CttFSSGGre2XsDzw 7y40yM3UfjU0rYchgB5eIRI05anrVwg0Jp+6E7QQIGxtjoxR/vIWazeZqEQShyvxsby+ HDKlRnjmGNO15/l56/WrzNDdbghp8HQEnvy199DjLlMvj8W3hE/4txJ6oMhlkmyPMtrC nGXg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1719852772; x=1720457572; h=to:subject:message-id:date:from:in-reply-to:references:mime-version :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=CcGdGPVQ6O1MglhBVG1YcNWN6v0agnwFWTiOkVwa1Ps=; b=pksZR7TBYR5WWZ5FNjF3/zLT3Q0waBcRfo4qPGAcuHJvXRRszZxLFakWGDbqRW20AH EGJaUnXll7d/RdjzVz5cV3pya8nyOxTnsuQAexnBbiH7CeLUp5Bh30bXCvOvgh70ViAu 43jPFo2Jrj3qI9xCY19dZHd0x6MDVSNpNfz7DcWY4YopDwX1jvNFw9k+7RGvVoEKwczz k+rzpe0dgHiIgxTH6svw3ecZWWCyw/0o8oLuUOAG2SEmJ4SZ3ebmFwEdPotSNsG4XXd4 KOhyJ9eH9DkF/Q6v/s/1uFgBm5mp3GQTVOTv398wJHH8PG7TP+WiKde3X8qpZ837QjAy 0AMw== X-Gm-Message-State: AOJu0YxWNBzSR5Brq9lWWhav9RajrQEPBMyV2xwWh1M9zNLWOhL7U4v4 EOaCCaJKRtaFsWLFx6CqEv3wsbTjunr9NV5qzo6Vuc2xBkyh6Hoa2ps6LbEJrpcboCd0Lu3ndZ9 Jr7BL9yZsC4SL8J9LnPo+D1lCbnFHVw== X-Google-Smtp-Source: AGHT+IHrXaBHfZKzJXuxd3hv0P7hXwKfEP7R5mZFVMV0inMVtaxKI95CLUvjwJHe/VdebORT0H/KsGtnXWbwl3JrBT4= X-Received: by 2002:a05:6402:40cd:b0:584:8feb:c3a1 with SMTP id 4fb4d7f45d1cf-5879ede2784mr6059603a12.1.1719852772297; Mon, 01 Jul 2024 09:52:52 -0700 (PDT) MIME-Version: 1.0 References: <20240621003355.3301-1-joshua.allmann@gmail.com> In-Reply-To: <20240621003355.3301-1-joshua.allmann@gmail.com> From: Josh Allmann Date: Mon, 1 Jul 2024 09:52:14 -0700 Message-ID: To: ffmpeg-devel@ffmpeg.org Subject: Re: [FFmpeg-devel] [PATCH] avcodec/nvenc: Fix segfault with intra-only X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: On Thu, 20 Jun 2024 at 17:39, Josh Allmann wrote: > > In intra-only mode, frameIntervalP is 0, which means the frame > data array is smaller than the number of surfaces. This causes a > crash when closing the encoder. > > Fix this by making sure the frame data array is at least as big as > the number of surfaces. > --- > libavcodec/nvenc.c | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/libavcodec/nvenc.c b/libavcodec/nvenc.c > index a9945355ba..93e87b21db 100644 > --- a/libavcodec/nvenc.c > +++ b/libavcodec/nvenc.c > @@ -1021,7 +1021,7 @@ static av_cold int nvenc_recalc_surfaces(AVCodecContext *avctx) > > // Output in the worst case will only start when the surface buffer is completely full. > // Hence we need to keep at least the max amount of surfaces plus the max reorder delay around. > - ctx->frame_data_array_nb = ctx->nb_surfaces + ctx->encode_config.frameIntervalP - 1; > + ctx->frame_data_array_nb = FFMAX(ctx->nb_surfaces, ctx->nb_surfaces + ctx->encode_config.frameIntervalP - 1); > > return 0; > } > -- > 2.39.2 > Hello, Ping for review. This patch fixes an easily triggered crash with nvenc in intra-only mode, eg ffmpeg -i -c:v h264_nvenc -g 0 Josh _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".