From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTP id 3F62140B26 for ; Mon, 27 Dec 2021 09:18:22 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 54D6468B037; Mon, 27 Dec 2021 11:18:20 +0200 (EET) Received: from mail-ot1-f48.google.com (mail-ot1-f48.google.com [209.85.210.48]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id DD03E68AEE5 for ; Mon, 27 Dec 2021 11:18:13 +0200 (EET) Received: by mail-ot1-f48.google.com with SMTP id 45-20020a9d0a30000000b0058f1a6df088so19985269otg.4 for ; Mon, 27 Dec 2021 01:18:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:date:mime-version:user-agent:subject:content-language:to :references:from:in-reply-to:content-transfer-encoding; bh=aB59MMbNCKc7E+tN9VaN4rfLKvqz+AGSjrAajr7pWfM=; b=EXkZHrxKBdeWA+JY8hdxsYO0frwkcId7nopvi2VtySFZaDoHNhsOdjjvJIFh+v3RCO xB7KToG+k5XGOah0hjBP+WK4JSTkEJZ5aBYI/bbYr6mFibaIwYptclVn4VbPmknWNalh 6ZrnHQhuDwT1PAOlJwryVhovlpiI0g/C6SyiId2hE4csHhKtnYXL4yMn5NofeqXPZmIQ 1v0vh98hTrphlaMpEaTp7Q72NMEanR+aVvLEXRtThM+hjnuEFW9Gm71gKX1MHrMQlvLH FuDuSHRqxFLWBpcYUQ8tsB4yteqyXgTj5Pm+srQj6TBc+HWVgAz7LN9SlWbLiIJjeGxL RTpQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:date:mime-version:user-agent:subject :content-language:to:references:from:in-reply-to :content-transfer-encoding; bh=aB59MMbNCKc7E+tN9VaN4rfLKvqz+AGSjrAajr7pWfM=; b=1/CCS3ZYqhBrlNPAK8rnCelBt4wVfOZJ0ipxUYqqpALLKSR9cA+nKHorkquf65VoR9 GYnTzSeiMitk5vprv1+PEBJavOBYxdI+WCBtExRa3GM6vFk2/a2VzBghUmSc5gU0kwgE D4o3CqGcSGkFBucG24bCrgwTKLKYOR8M1TiH5pLoOoDUYRv7oLTayAkN92WDra0XJeKR q5PZoKMiMLeuQisOcoYf7oVtgWW2rEX4SUcFbI8Mzvr0sr/IFUkCVDJXWefbefkU2eYE ZRnY7yskhfN7c9h6H0O6v2EkNd2tq3GqEw5keXY8nQQ1MHh55HBWh23YP5FXQorBUn3P 4+zQ== X-Gm-Message-State: AOAM5334AUS3K21Vo/wsv4U+/5Bt9ugZRTZEf/48Js1/u84h0cJwGK1s o/QBJXETTvmXlvp58zZ8bBs= X-Google-Smtp-Source: ABdhPJwc3gZTlnjY/1bbmIh7QI70PyWosFZzU2H4HxIKs4NGWrYjleyEpxwdlt54Zsi96AD9JgDspA== X-Received: by 2002:a9d:6d86:: with SMTP id x6mr11473681otp.263.1640596691596; Mon, 27 Dec 2021 01:18:11 -0800 (PST) Received: from ?IPV6:2600:1700:4830:3f7f:7aac:34da:1863:159e? ([2600:1700:4830:3f7f:7aac:34da:1863:159e]) by smtp.gmail.com with ESMTPSA id h3sm2559972ooe.13.2021.12.27.01.18.11 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 27 Dec 2021 01:18:11 -0800 (PST) Message-ID: <246eb92f-61c3-7158-5265-feb0cb0252af@gmail.com> Date: Mon, 27 Dec 2021 03:18:10 -0600 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.2.0 Content-Language: en-US To: Andriy Gelman , FFmpeg development discussions and patches References: <20211214021215.456493-1-aicommander@gmail.com> <20211227061738.mbkxlmm4deveyggf@jackie> From: Cameron Gutman In-Reply-To: <20211227061738.mbkxlmm4deveyggf@jackie> Subject: Re: [FFmpeg-devel] [PATCH] avcodec/v4l2_m2m_dec: dequeue frame if input isn't ready X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: On 12/27/21 00:17, Andriy Gelman wrote: > On Tue, 14. Dec 02:12, Cameron Gutman wrote: >> The V4L2M2M API operates asynchronously, so multiple packets can >> be enqueued before getting a batch of frames back. Since it was >> only possible to receive a frame by submitting another packet, >> there wasn't a way to drain those excess output frames from when >> avcodec_receive_frame() returned AVERROR(EAGAIN). >> > >> In my testing, this change reduced decode latency of a real-time >> 60 FPS H.264 stream by approximately 10x (200ms -> 20ms) on a >> Raspberry Pi 4. > > I was doing some more tests today, but didn't have any luck dequeuing a frame > if ff_decode_get_packet() returned EAGAIN. Hmm, maybe there is something different about your test harness? I'm receiving 720p 60 FPS H.264 ES in real-time from the network. For each H.264 encoded frame I receive off the network, my basic approach is like this (simplified for brevity): avcodec_send_packet(&pkt); do { frame = av_frame_alloc(); if ((err = avcodec_receive_frame(frame)) == 0) { render_frame(frame); } } while (err == 0); I'll usually get EAGAIN immediately for the first few frames I submit (so no output frame yet), but then I'll get a batch of output frames back after the first completed decode. That drains the excess latency from the pipeline to avoid always being behind. For cases where we want to prioritize latency over throughput, I've had success with this approach too: avcodec_send_packet(&pkt); while (avcodec_receive_frame(frame) == AVERROR(EAGAIN)) { msleep(1); } render_frame(frame); In this case, we can retry avcodec_receive_frame() until we get the frame back that we just submitted for decoding. The patch here enables both of these use-cases by allowing V4L2M2M to retry getting a decoded frame without new input data. Both of these also work with MMAL after the recent decoupled dataflow patch. > Could you share the dataset? > It is 720p60.h264 from here: https://onedrive.live.com/?authkey=%21ALoKfcPfFeKyhzs&id=C15BF9770619F56%21165617&cid=0C15BF9770619F56 > Thanks, > Thank you _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".