From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTP id 121AF48988 for ; Fri, 21 Jun 2024 00:39:20 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 6896E68D83C; Fri, 21 Jun 2024 03:39:17 +0300 (EEST) Received: from mail-pf1-f174.google.com (mail-pf1-f174.google.com [209.85.210.174]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 4BB7068D671 for ; Fri, 21 Jun 2024 03:39:10 +0300 (EEST) Received: by mail-pf1-f174.google.com with SMTP id d2e1a72fcca58-70635466708so1451618b3a.1 for ; Thu, 20 Jun 2024 17:39:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1718930347; x=1719535147; darn=ffmpeg.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=emYnUwUnpozwlPOmkFrZYJa+YCf3WdKYGGVPfpI07rc=; b=jFqTRHcd7qdT7qU31TM13IWt81b8C+zW5wYiNTvcMhsNfU2phR5CH3nJ+qrsnvTe3b v9DOFZC+0UZ7x5Hx7qbiAollOmZ2XOPA6O7HhnMlq7+NzlHj1PjLKqwnAAWGLtDanpc6 AqQHMq9D+1QbiMXZT4gg1glaLu0xocp95z2SLoEXYu5X82hGvn3vHTtNPlLl6UOHMF5Z OJ+K8ACDrY1zJnzCdbaGnuoPMChHW/bgdaLEVJFp3P94JAwbFBtqBi6FbK1AD5x09tXs wWT81pjZfd923oNHbnw/qjCrNElOYkKkal+9oJXrc2Mc++t5Ov0b1nIyiASg+NB0uuyi KdSw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718930347; x=1719535147; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=emYnUwUnpozwlPOmkFrZYJa+YCf3WdKYGGVPfpI07rc=; b=CY2anGR8jushE8t2yeBMRk0lchrGgwHKIr+5cDElhlioBQM3c4RllBMw4nPd2cQKwV Ch00KY3I56ukPgx/g6+WYzTCK4oyjvJ+2pAoS+DjBKEZyBuFGAbPxGOZB7bfPq0Eq7kn mNR2uR85iYdu91yd3giWJZo9pTlO0T+BmY0MNcAqxI3Q358StZIZXlK5N3bhGZ8QHpDG gPF+MY1Yd3oRi7RZcBn827qSLWr15ZfXi002GGhrvKApcVJ8v9fzorUqnOODanVskF15 MQ1U94smZkJzIrwhf1qxSd7DQ7CDdDo3ifTpSdtLHo4Rt3m4srzINs+oX35giV3E20F1 QZMA== X-Gm-Message-State: AOJu0Yw6pH90YPrJLARmYMXwU/ECxrdrIYMQp8NiVpiQ96Z9JsIdNCMg LVb57hA9igRHhrEb/2hiONqW2n9zyKxkUwSlJYKyFwT/Arh6AIgnn1TpnQ== X-Google-Smtp-Source: AGHT+IGkTocIHdZ8JohfRQACnavCllFevt+7eMsVGbY6PwtweaQDG3KvSvVaJldtFwwE9lyFghSnbg== X-Received: by 2002:a05:6a20:29d:b0:1b5:2fbb:2d78 with SMTP id adf61e73a8af0-1bcbb656593mr6247250637.48.1718930347168; Thu, 20 Jun 2024 17:39:07 -0700 (PDT) Received: from gaia.lan (syn-076-176-004-046.res.spectrum.com. [76.176.4.46]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-7065107b892sm250810b3a.14.2024.06.20.17.39.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 20 Jun 2024 17:39:06 -0700 (PDT) From: Josh Allmann To: ffmpeg-devel@ffmpeg.org Date: Thu, 20 Jun 2024 17:33:55 -0700 Message-Id: <20240621003355.3301-1-joshua.allmann@gmail.com> X-Mailer: git-send-email 2.37.1 (Apple Git-137.1) MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH] avcodec/nvenc: Fix segfault with intra-only X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Josh Allmann Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: In intra-only mode, frameIntervalP is 0, which means the frame data array is smaller than the number of surfaces. This causes a crash when closing the encoder. Fix this by making sure the frame data array is at least as big as the number of surfaces. --- libavcodec/nvenc.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/libavcodec/nvenc.c b/libavcodec/nvenc.c index a9945355ba..93e87b21db 100644 --- a/libavcodec/nvenc.c +++ b/libavcodec/nvenc.c @@ -1021,7 +1021,7 @@ static av_cold int nvenc_recalc_surfaces(AVCodecContext *avctx) // Output in the worst case will only start when the surface buffer is completely full. // Hence we need to keep at least the max amount of surfaces plus the max reorder delay around. - ctx->frame_data_array_nb = ctx->nb_surfaces + ctx->encode_config.frameIntervalP - 1; + ctx->frame_data_array_nb = FFMAX(ctx->nb_surfaces, ctx->nb_surfaces + ctx->encode_config.frameIntervalP - 1); return 0; } -- 2.39.2 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".