From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTP id 4607A44C64 for ; Sun, 12 Nov 2023 15:12:18 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id ACD4968CC49; Sun, 12 Nov 2023 17:12:08 +0200 (EET) Received: from alphacentauri.ramacher.at (alphacentauri.ramacher.at [195.201.139.148]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 55E3768CC4E for ; Sun, 12 Nov 2023 17:12:02 +0200 (EET) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=ramacher.at ; s=mail; h=MIME-Version:Message-ID:Date:Subject:Cc:To:From:References: In-Reply-To:Content-Type; bh=3gUv6zuCPuKoz/tQ3OpiXh3X/aAimop1rhJhIxv3eVI=; b= 3KMNq+rq5+Scxtn/UGbv8ccjZvWfEfdBH69/o2oM56tURGL5bx7utszlxRXK3UpZ2apW8qzhl2nFs PnjrpsvSD7GdvFerW5IRN8MxpCoWk37rYMlzkoHtIukfuy+5hSetdRwrk1k9U7VZKh0cFewn4Cu9N SmHNWL81PAwd2Hq1s=; From: Sebastian Ramacher To: ffmpeg-devel@ffmpeg.org Date: Sun, 12 Nov 2023 16:11:41 +0100 Message-ID: <20231112151144.2307049-1-sramacher@debian.org> X-Mailer: git-send-email 2.42.0 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 0/3] Fix invalid frees, segfaults and memory leaks in avcodec/fft wrappers X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Sebastian Ramacher Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: The wrappers in avcodec/fft which were introduced in 6.1 may lead to invalid frees, segfaults and memory leaks. Consider the following example program: #include int main() { FFTContext* fft = av_fft_init(11, 0); av_fft_end(fft); FFTContext* mdct = av_mdct_init(11, 0, 1.0); av_mdct_end(mdct); mdct = av_mdct_init(11, 1, 1.0); av_mdct_end(mdct); RDFTContext* rdft = av_rdft_init(11, DFT_R2C); av_rdft_end(rdft); DCTContext* dct = av_dct_init(11, DCT_II); av_dct_end(dct); } When executed under valgrind, one obtains: ==2300086== Memcheck, a memory error detector ==2300086== Copyright (C) 2002-2022, and GNU GPL'd, by Julian Seward et al. ==2300086== Using Valgrind-3.19.0 and LibVEX; rerun with -h for copyright info ==2300086== Command: ./a.out ==2300086== ==2300086== Conditional jump or move depends on uninitialised value(s) ==2300086== at 0x5FB6CBE: av_tx_uninit (in /usr/lib/x86_64-linux-gnu/libavutil.so.58.29.100) ==2300086== by 0x490B3AA: av_fft_end (in /usr/lib/x86_64-linux-gnu/libavcodec.so.60.31.102) ==2300086== by 0x1090D7: main (test.c:5) ==2300086== Uninitialised value was created by a heap allocation ==2300086== at 0x4845990: memalign (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so) ==2300086== by 0x4845AED: posix_memalign (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so) ==2300086== by 0x5FFAC14: av_malloc (in /usr/lib/x86_64-linux-gnu/libavutil.so.58.29.100) ==2300086== by 0x4A4B4A5: av_fft_init (in /usr/lib/x86_64-linux-gnu/libavcodec.so.60.31.102) ==2300086== by 0x1090CF: main (test.c:4) ==2300086== ==2300086== Conditional jump or move depends on uninitialised value(s) ==2300086== at 0x4843131: free (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so) ==2300086== by 0x490B4A5: av_dct_end (in /usr/lib/x86_64-linux-gnu/libavcodec.so.60.31.102) ==2300086== by 0x10913A: main (test.c:17) ==2300086== Uninitialised value was created by a heap allocation ==2300086== at 0x4845990: memalign (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so) ==2300086== by 0x4845AED: posix_memalign (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so) ==2300086== by 0x5FFAC14: av_malloc (in /usr/lib/x86_64-linux-gnu/libavutil.so.58.29.100) ==2300086== by 0x4A4B96F: av_dct_init (in /usr/lib/x86_64-linux-gnu/libavcodec.so.60.31.102) ==2300086== by 0x109132: main (test.c:16) ==2300086== ==2300086== ==2300086== HEAP SUMMARY: ==2300086== in use at exit: 66,528 bytes in 270 blocks ==2300086== total heap usage: 1,353 allocs, 1,083 frees, 386,566 bytes allocated ==2300086== ==2300086== 8,064 (640 direct, 7,424 indirect) bytes in 1 blocks are definitely lost in loss record 247 of 249 ==2300086== at 0x4845990: memalign (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so) ==2300086== by 0x4845AED: posix_memalign (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so) ==2300086== by 0x5FFAC14: av_malloc (in /usr/lib/x86_64-linux-gnu/libavutil.so.58.29.100) ==2300086== by 0x5FFAF80: av_mallocz (in /usr/lib/x86_64-linux-gnu/libavutil.so.58.29.100) ==2300086== by 0x5FB732D: ??? (in /usr/lib/x86_64-linux-gnu/libavutil.so.58.29.100) ==2300086== by 0x5FB7616: av_tx_init (in /usr/lib/x86_64-linux-gnu/libavutil.so.58.29.100) ==2300086== by 0x4A4B678: av_mdct_init (in /usr/lib/x86_64-linux-gnu/libavcodec.so.60.31.102) ==2300086== by 0x10910A: main (test.c:10) ==2300086== ==2300086== 8,192 bytes in 1 blocks are possibly lost in loss record 248 of 249 ==2300086== at 0x4845990: memalign (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so) ==2300086== by 0x4845AED: posix_memalign (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so) ==2300086== by 0x5FFAC14: av_malloc (in /usr/lib/x86_64-linux-gnu/libavutil.so.58.29.100) ==2300086== by 0x6030F60: ??? (in /usr/lib/x86_64-linux-gnu/libavutil.so.58.29.100) ==2300086== by 0x5FBC968: ??? (in /usr/lib/x86_64-linux-gnu/libavutil.so.58.29.100) ==2300086== by 0x5FB73C8: ??? (in /usr/lib/x86_64-linux-gnu/libavutil.so.58.29.100) ==2300086== by 0x5FB73C8: ??? (in /usr/lib/x86_64-linux-gnu/libavutil.so.58.29.100) ==2300086== by 0x5FB7616: av_tx_init (in /usr/lib/x86_64-linux-gnu/libavutil.so.58.29.100) ==2300086== by 0x4A4B678: av_mdct_init (in /usr/lib/x86_64-linux-gnu/libavcodec.so.60.31.102) ==2300086== by 0x10910A: main (test.c:10) ==2300086== ==2300086== LEAK SUMMARY: ==2300086== definitely lost: 640 bytes in 1 blocks ==2300086== indirectly lost: 7,424 bytes in 4 blocks ==2300086== possibly lost: 8,192 bytes in 1 blocks ==2300086== still reachable: 48,256 bytes in 243 blocks ==2300086== suppressed: 0 bytes in 0 blocks ==2300086== Reachable blocks (those to which a pointer was found) are not shown. ==2300086== To see them, rerun with: --leak-check=full --show-leak-kinds=all ==2300086== ==2300086== For lists of detected and suppressed errors, rerun with: -s ==2300086== ERROR SUMMARY: 4 errors from 4 contexts (suppressed: 0 from 0) This patch series fixes the above issues. The initial issue in av_fft_end was discuvered via the test suite of r-cran-av. Sebastian Ramacher (3): avcodec/fft: Do not uninit never initialized ctx2 avcodec/fft: Set potentially unused wrapper variables to avoid invalid free/uninit avcoded/fft: Fix memory leak if ctx2 is used libavcodec/avfft.c | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) -- 2.42.0 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".