From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTP id 1A7FF40FF3 for ; Sun, 2 Jan 2022 16:56:53 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 05DA468B160; Sun, 2 Jan 2022 18:56:51 +0200 (EET) Received: from mail-qv1-f48.google.com (mail-qv1-f48.google.com [209.85.219.48]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id AF0226897E2 for ; Sun, 2 Jan 2022 18:56:44 +0200 (EET) Received: by mail-qv1-f48.google.com with SMTP id q3so29017726qvc.7 for ; Sun, 02 Jan 2022 08:56:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=m4avbLUh/J4+R22yBvwOPj38HzNYCvBXyKOuLat7gFU=; b=CJyeUxjVImXOyjbB7Uevpi4Jq1GsMFPEF9IvhbKN4CAfZT2A/sG+pGmvLtY+dqJuTJ DIDrb5i2SxQdBXX+JtCiHbZD6q7VntLW3En8VszHDJXDJRnqy3lIbV3AOBs7J6FtjtJU E57zZVDxuN/9M2AbDdWqJ2SEgVLv2ZNWGQkfq5YPancqNQX+Ax0nGtf9lT0EotMKGmTr Y2hrxHT7X+vT9gujU0aNTThNtO9vtkyDwiRB940K0q+TGJ3z07bOREYX/mdK3rL04XHR rczaNZQBrJ87FfBX1Dl7emxuTi94+hSwc8o5pcqD2HxuTfR9vDl+DVUY1jIa6Mo7yeDd iorg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=m4avbLUh/J4+R22yBvwOPj38HzNYCvBXyKOuLat7gFU=; b=ltrOxUGyLx5zW5tmqHDdpERDwC669gakLK7xf6n5AIxtrWswNqq3v+uwp83mdsB6jM SymRw+IKexMhDIQJlQ2VsGROZglVw8qsfQ8EPPt3qVD38axW8RekWFf0lJOhQ3aGMsEW /mjqL+G7AXTO23iGRgWtnPYC5rzPWpF99LdgUvAgiZP1XmUJg77CnBPz5z5v0jW+LoAW 7ucezHyyjL0ykMDYtoIRtP1sP67VRX07/wr6VAeeHqKcs+FoGkiiufSKwhsoBNV1RSKi nKbaH/Ojz5fpotwgWZy5XFaZGy2NRijb4jQlBrFF22sJHhbyb8KB2bQuo6XXIorpDWly rpnA== X-Gm-Message-State: AOAM530HZc7nbaJdZNY+IsR8h43VWNloRgCFbA1bl5ksYjAJmQE4kZZK RIt38Vg7+lrHoLu3lNwtj6JsKpXoyi82rA== X-Google-Smtp-Source: ABdhPJzkWBfHhWWdV2IGCqsR9lnlmYWksHSr6mS9nW21vZQwpu3wyDfgxMtBdO+rZo1LSlAqXOVftw== X-Received: by 2002:a05:6214:5096:: with SMTP id kk22mr39309190qvb.39.1641142603338; Sun, 02 Jan 2022 08:56:43 -0800 (PST) Received: from jackie (pool-108-20-185-127.bstnma.fios.verizon.net. [108.20.185.127]) by smtp.gmail.com with ESMTPSA id y12sm25814766qko.36.2022.01.02.08.56.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 02 Jan 2022 08:56:43 -0800 (PST) Date: Sun, 2 Jan 2022 11:56:41 -0500 From: Andriy Gelman To: Cameron Gutman , jc@kynesim.co.uk Message-ID: <20220102165641.5fewafbspqf7wo5n@jackie> References: <20211214021215.456493-1-aicommander@gmail.com> <20211227061738.mbkxlmm4deveyggf@jackie> <246eb92f-61c3-7158-5265-feb0cb0252af@gmail.com> <20211228234135.p5rreo3orbo6axsr@jackie> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20211228234135.p5rreo3orbo6axsr@jackie> Subject: Re: [FFmpeg-devel] [PATCH] avcodec/v4l2_m2m_dec: dequeue frame if input isn't ready X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: FFmpeg development discussions and patches Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: On Tue, 28. Dec 18:41, Andriy Gelman wrote: > On Mon, 27. Dec 03:18, Cameron Gutman wrote: > > On 12/27/21 00:17, Andriy Gelman wrote: > > > On Tue, 14. Dec 02:12, Cameron Gutman wrote: > > >> The V4L2M2M API operates asynchronously, so multiple packets can > > >> be enqueued before getting a batch of frames back. Since it was > > >> only possible to receive a frame by submitting another packet, > > >> there wasn't a way to drain those excess output frames from when > > >> avcodec_receive_frame() returned AVERROR(EAGAIN). > > >> > > > > > >> In my testing, this change reduced decode latency of a real-time > > >> 60 FPS H.264 stream by approximately 10x (200ms -> 20ms) on a > > >> Raspberry Pi 4. > > > > > > I was doing some more tests today, but didn't have any luck dequeuing a frame > > > if ff_decode_get_packet() returned EAGAIN. > > > > Hmm, maybe there is something different about your test harness? > > I'm receiving 720p 60 FPS H.264 ES in real-time from the network. > > I used ffmpeg, i.e. > ./ffmpeg -codec:v h264_v4l2m2m -i 720p60.h264 -f null - > > Anyway, I applied your patch but just removed the comment on latency > because I couldn't reproduce it. > > > > > For each H.264 encoded frame I receive off the network, my basic > > approach is like this (simplified for brevity): > > > > avcodec_send_packet(&pkt); > > do { > > frame = av_frame_alloc(); > > if ((err = avcodec_receive_frame(frame)) == 0) { > > render_frame(frame); > > } > > } while (err == 0); > > > > I'll usually get EAGAIN immediately for the first few frames I submit > > (so no output frame yet), but then I'll get a batch of output frames > > back after the first completed decode. That drains the excess latency > > from the pipeline to avoid always being behind. > > > > For cases where we want to prioritize latency over throughput, I've > > had success with this approach too: > > > > avcodec_send_packet(&pkt); > > while (avcodec_receive_frame(frame) == AVERROR(EAGAIN)) { > > msleep(1); > > } > > render_frame(frame); > > > > In this case, we can retry avcodec_receive_frame() until we get the > > frame back that we just submitted for decoding. > > > > > The patch here enables both of these use-cases by allowing V4L2M2M > > to retry getting a decoded frame without new input data. Both of > > these also work with MMAL after the recent decoupled dataflow patch. > > > > > Could you share the dataset? > > > > > > > > It is 720p60.h264 from here: > > https://onedrive.live.com/?authkey=%21ALoKfcPfFeKyhzs&id=C15BF9770619F56%21165617&cid=0C15BF9770619F56 > > I could not decode this dataset on rpi4 (before or after the patch). Tried to upgrade kernel > to 5.16, but still same problem. Odroid xu4 seems to work fine. This patch fixes decoding on the RPi4 for me: https://patchwork.ffmpeg.org/project/ffmpeg/patch/20210819085533.1174-2-ming.qian@nxp.com/ It would be nice to get the fix into the upcoming release. -- Andriy _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".