From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTP id 683AD40BFF for ; Tue, 28 Dec 2021 23:41:47 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id ABB5B68AEEA; Wed, 29 Dec 2021 01:41:44 +0200 (EET) Received: from mail-qv1-f53.google.com (mail-qv1-f53.google.com [209.85.219.53]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id E70A468A744 for ; Wed, 29 Dec 2021 01:41:38 +0200 (EET) Received: by mail-qv1-f53.google.com with SMTP id r6so17681693qvr.13 for ; Tue, 28 Dec 2021 15:41:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=sqw15rBwinkHieHa9bfVsy4jX6DKxHvv6pbgQlz6wBo=; b=YADaQ0H6/WLn3SmZ1O249dV2Q9MjkUmu1DW0uqH6BUD2r1SXztYq++xVP+5QG9U7p3 pNlFJdWcJx0DuH+sJv3ZTvtF/oYqwi0UBOdCMu6sqBjT8sMEuRHeVvz/LPQrtQHch/tf UsA8l73N8w40wrLjzfcrKV37LCzymZyBl42xdet7grXNDha5HWlakhtV5GTE4qOpJTTh wXu9tTRz1DgxRr5w07YMaXYM9coWFpHy2T4mfnG+YxkqRqEQ/aVHVlm5cDn8JBlWrIfn l84Q7iPCFrnQ2E7qSTGnGmKny67B13oQ1xpO08sB2hZlG9QgeMHjeEC6ctKBSH8Pdn2T R9bA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=sqw15rBwinkHieHa9bfVsy4jX6DKxHvv6pbgQlz6wBo=; b=fciMLHqMhQE24gRnUUyTn03RmbGFo+mMIDsjlGSmIYia83WPs4Etf2WVjuF+We/vO2 Op/Jx85Eqp8NzU2YMqaR4/ryLiBcGvoMhs/o2B6ua5ASnY6oXUWylQcZZ7rrsWNQegFn a47YuZB3Dxxam3xLA4YGZN3GRMHbTLczn+XPMC5/8tyQfuO0KyYngoOeeTE4herbLLEV 1BwdudMuyTQFXw2VJ3o6JZCDhHrSvRfa58ZdcVCIf7/UQW+ryNVeYC1g5p2gIIlP+QxG Yn+zlhNUU7xBqfwnLrl1MzqygMQDkTg0ZqB9ColtKpQWcooRpm80c7Ag+CBh5cftyuvZ 4/mQ== X-Gm-Message-State: AOAM530SIgfVudQHfIDRiAaVb4qlSkPPt7ZSmRGz9WmhqZ7ki2VSWTLS 4d2e1OFWnqjaXB3jcMTl9rM= X-Google-Smtp-Source: ABdhPJxsnvCKyvFuPgBcLHwX1y94QqbNnDwmpHpFnmUt5QqTlYU18+OddgsAPxMxdwtxqwNcOcYChQ== X-Received: by 2002:a05:6214:e4c:: with SMTP id o12mr10112631qvc.79.1640734897515; Tue, 28 Dec 2021 15:41:37 -0800 (PST) Received: from jackie (pool-173-48-211-61.bstnma.fios.verizon.net. [173.48.211.61]) by smtp.gmail.com with ESMTPSA id v21sm16836438qta.0.2021.12.28.15.41.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 28 Dec 2021 15:41:37 -0800 (PST) Date: Tue, 28 Dec 2021 18:41:35 -0500 From: Andriy Gelman To: Cameron Gutman , jc@kynesim.co.uk Message-ID: <20211228234135.p5rreo3orbo6axsr@jackie> References: <20211214021215.456493-1-aicommander@gmail.com> <20211227061738.mbkxlmm4deveyggf@jackie> <246eb92f-61c3-7158-5265-feb0cb0252af@gmail.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <246eb92f-61c3-7158-5265-feb0cb0252af@gmail.com> Subject: Re: [FFmpeg-devel] [PATCH] avcodec/v4l2_m2m_dec: dequeue frame if input isn't ready X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: FFmpeg development discussions and patches Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: On Mon, 27. Dec 03:18, Cameron Gutman wrote: > On 12/27/21 00:17, Andriy Gelman wrote: > > On Tue, 14. Dec 02:12, Cameron Gutman wrote: > >> The V4L2M2M API operates asynchronously, so multiple packets can > >> be enqueued before getting a batch of frames back. Since it was > >> only possible to receive a frame by submitting another packet, > >> there wasn't a way to drain those excess output frames from when > >> avcodec_receive_frame() returned AVERROR(EAGAIN). > >> > > > >> In my testing, this change reduced decode latency of a real-time > >> 60 FPS H.264 stream by approximately 10x (200ms -> 20ms) on a > >> Raspberry Pi 4. > > > > I was doing some more tests today, but didn't have any luck dequeuing a frame > > if ff_decode_get_packet() returned EAGAIN. > > Hmm, maybe there is something different about your test harness? > I'm receiving 720p 60 FPS H.264 ES in real-time from the network. I used ffmpeg, i.e. ./ffmpeg -codec:v h264_v4l2m2m -i 720p60.h264 -f null - Anyway, I applied your patch but just removed the comment on latency because I couldn't reproduce it. > > For each H.264 encoded frame I receive off the network, my basic > approach is like this (simplified for brevity): > > avcodec_send_packet(&pkt); > do { > frame = av_frame_alloc(); > if ((err = avcodec_receive_frame(frame)) == 0) { > render_frame(frame); > } > } while (err == 0); > > I'll usually get EAGAIN immediately for the first few frames I submit > (so no output frame yet), but then I'll get a batch of output frames > back after the first completed decode. That drains the excess latency > from the pipeline to avoid always being behind. > > For cases where we want to prioritize latency over throughput, I've > had success with this approach too: > > avcodec_send_packet(&pkt); > while (avcodec_receive_frame(frame) == AVERROR(EAGAIN)) { > msleep(1); > } > render_frame(frame); > > In this case, we can retry avcodec_receive_frame() until we get the > frame back that we just submitted for decoding. > > The patch here enables both of these use-cases by allowing V4L2M2M > to retry getting a decoded frame without new input data. Both of > these also work with MMAL after the recent decoupled dataflow patch. > > > Could you share the dataset? > > > > It is 720p60.h264 from here: > https://onedrive.live.com/?authkey=%21ALoKfcPfFeKyhzs&id=C15BF9770619F56%21165617&cid=0C15BF9770619F56 I could not decode this dataset on rpi4 (before or after the patch). Tried to upgrade kernel to 5.16, but still same problem. Odroid xu4 seems to work fine. I found a repo in github https://github.com/jc-kynesim/rpi-ffmpeg (I guess Rpi4 team), where the decoding is working. Doing a bisect points to this commit that fixes: https://github.com/jc-kynesim/rpi-ffmpeg/commit/3e60c08552110d9da1b3b5d3637c6bd8b53eec69 There are lots of changes in the commit, so I'm not sure what exactly fixed the problem or if it was by chance. It would be good to figure out. -- Andriy _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".