From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTP id 010CB4324E for ; Wed, 25 May 2022 17:15:42 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 198D868B4C3; Wed, 25 May 2022 20:15:40 +0300 (EEST) Received: from mail-oa1-f44.google.com (mail-oa1-f44.google.com [209.85.160.44]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 404EB68B4C3 for ; Wed, 25 May 2022 20:15:33 +0300 (EEST) Received: by mail-oa1-f44.google.com with SMTP id 586e51a60fabf-e93bbb54f9so26744159fac.12 for ; Wed, 25 May 2022 10:15:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:subject:date:message-id:mime-version :content-transfer-encoding; bh=9B0SIHShbuKdqYuCzlY07jmMzwQJ3QPWFFrCToeavxs=; b=pm9wix6H4D9dtJa+vYHxHNMn7IrLQZqlMmuakHXT6akIm2wyq1XMfWm4+0rRnuTr4s gOUSdw7mgPhXIcyqLhpw5DWSRRLgien9DPKaueWjyKclbOEcbhQgfQ1RFgZGGFWbp5fn Vz51iBtI4WlFfNJGw2dhGnLQlQ8w2JBJ2ZzkeaguSv6Bzb6jTtEvCsjnKraTTsrw7dZW X8WqtwK9X7ADSXdgiCRmb8xMycOq5Y//K1b9ak339EmhM6P7Izpq7MqPvi0Nk79yYEhU Fsfj2G/byrjgvMiWEvxaXXsu6/44yAZzmBqkjWs8bcEVwGoupVj6OmsT3BeCgg7M0Osx QTmw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:mime-version :content-transfer-encoding; bh=9B0SIHShbuKdqYuCzlY07jmMzwQJ3QPWFFrCToeavxs=; b=zKYIiSkAR5QNIi6FdsWD2ueHcr3Kfitvo7I39dQJY6odHTIzfumeVcNXh+I1COC7tW 0xpi1rOV2Tg5Fj4NQoSNiPC23H2V7rF3t0Qwh5mYDpHgNxNvbMlBI1/NcwneiyCTfH05 EvOa4wPRpgOuOLwce5zem6tH4UPAZVH/NGsyyJo6j93gF6jSyzgs4xfFUDSKyvAt2wvg v3EYIrEcmgClMEQc8H/4+7KqaM85+FCesyGU5ZN1tsYTrg2sNMFW1+ahLjBbvUtWP5E9 thDfoUP7V8qbvnfNqfz5Og558EqJBh+Kyd6Hj4htnd5dVzx8NS8wsouUAFJ3y6WqSeSh uYAQ== X-Gm-Message-State: AOAM5320KfwpeRB0IdeEe3iJ1VhXFj0QLri+6lDuGWpWcU8gURGlylKx 7bYv8uuKOhutj8tEh8illP8pzdGBH2U= X-Google-Smtp-Source: ABdhPJy1WbgMNA6VRCJdvBM804AucQz14wHh70fuGaZYCIWmjeA/+hwZ9LRucy/N+BFUUhuHGK9LPQ== X-Received: by 2002:a05:6870:4596:b0:da:b3f:2b1d with SMTP id y22-20020a056870459600b000da0b3f2b1dmr6693256oao.188.1653498931396; Wed, 25 May 2022 10:15:31 -0700 (PDT) Received: from localhost.localdomain ([186.136.131.95]) by smtp.gmail.com with ESMTPSA id g6-20020aca3906000000b00325cda1ffb5sm6503185oia.52.2022.05.25.10.15.30 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 25 May 2022 10:15:30 -0700 (PDT) From: James Almer To: ffmpeg-devel@ffmpeg.org Date: Wed, 25 May 2022 14:15:16 -0300 Message-Id: <20220525171516.64893-1-jamrial@gmail.com> X-Mailer: git-send-email 2.36.1 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH] avutil/avframe: fix channel layout checks in av_frame_copy() X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: Normally, both the source and dest frame would have only the old API fields set, only the new API fields set, or both set. But in some cases, like when calling av_frame_ref() using a non reference counted source frame where only the old channel layout API fields were populated, the result would be the dst frame having both the new and old fields populated. This commit takes this into account and fixes the checks by calling av_channel_layout_compare() only if the source frame has the new API fields set, and doing sanity checks for the source frame old API fields if the new ones are not set. Signed-off-by: James Almer --- libavutil/frame.c | 19 +++++++++++++++---- 1 file changed, 15 insertions(+), 4 deletions(-) diff --git a/libavutil/frame.c b/libavutil/frame.c index fbb869fffa..0da4bb4768 100644 --- a/libavutil/frame.c +++ b/libavutil/frame.c @@ -727,19 +727,30 @@ static int frame_copy_audio(AVFrame *dst, const AVFrame *src) #if FF_API_OLD_CHANNEL_LAYOUT FF_DISABLE_DEPRECATION_WARNINGS + int channel_layout = dst->ch_layout.order == AV_CHANNEL_ORDER_NATIVE ? + dst->ch_layout.u.mask : 0; if (!channels) { - if (dst->channels != src->channels || - dst->channel_layout != src->channel_layout) - return AVERROR(EINVAL); channels = dst->channels; + channel_layout = dst->channel_layout; planes = planar ? channels : 1; + } + if (!src->ch_layout.nb_channels) { + if (channels != src->channels || + channel_layout != src->channel_layout) + return AVERROR(EINVAL); CHECK_CHANNELS_CONSISTENCY(src); } FF_ENABLE_DEPRECATION_WARNINGS #endif if (dst->nb_samples != src->nb_samples || +#if FF_API_OLD_CHANNEL_LAYOUT + (av_channel_layout_check(&src->ch_layout) && +#endif av_channel_layout_compare(&dst->ch_layout, &src->ch_layout)) +#if FF_API_OLD_CHANNEL_LAYOUT + ) +#endif return AVERROR(EINVAL); for (i = 0; i < planes; i++) @@ -763,7 +774,7 @@ FF_DISABLE_DEPRECATION_WARNINGS else if (dst->nb_samples > 0 && (av_channel_layout_check(&dst->ch_layout) #if FF_API_OLD_CHANNEL_LAYOUT - || dst->channel_layout || dst->channels + || dst->channels > 0 #endif )) return frame_copy_audio(dst, src); -- 2.36.1 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".