From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTP id 0FA044260D for ; Mon, 21 Mar 2022 07:27:47 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id A5F6D68B143; Mon, 21 Mar 2022 09:27:44 +0200 (EET) Received: from EUR04-DB3-obe.outbound.protection.outlook.com (mail-eopbgr60053.outbound.protection.outlook.com [40.107.6.53]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 9BF0D680867 for ; Mon, 21 Mar 2022 09:27:38 +0200 (EET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=VTJhR5s0sfwNM+YqTj8aQFhRI9H6O5rKU+SY+oRNiIVVUQdtFcjEwG/ox80Hmqo6rbFzks+B3+BOa0ReedPrC8pcewHz67wO+2IbvUC0451kLFwyGaj57dUIF2AHmtfLZDJkZoebAAECjHqDcUEsefX6w0JpUfqmb8MI/iPzjK91wLsAsB8fdXDP7atBoID08KVovA7ydQSw8hxXKqHHUYYVnAVbpPgqh0hnZowizOjkLWDq0HVEjm7GeIJQig9W+rMBcd04t+q+A3ZCBz63oIELIYpHTS38Po8iiBTHd8oY+LbPu5vu1ffp6iOmT4+ei2rqR/LfCxdzuFOm7emENQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=MwR+sFvy7Lwp6gHJ2K0MLAB4TTnlwYx/PkrkO8Hykf8=; b=CdmilZ9/daUxOEPwdC85mfZpOgolr355JgkjM8XgDdQMEbZiFQR9hSMDz28t3rt9k6eDDbIu6Kb7yovH5RCu0YxtKMYYrbX5F++z8hVbgbThtI1qV3T18hS9u67j27h+glb/bZnHfXRyJkdqArfWEbh6P/+BA12KO4Lc1yYOAJRxFz1/MyLu/lR/sLgS2BHN4iaPjypFKR/yLm9oDtIRCsTZ+b61jFYcm0g0s1wXQOv9VTjsllsksy53kiGzm3rYyzoF0SYNNCy0blDnFwJphuUqukDaiIjnkiMGnTjcDYRSqw5Ss7ZIkoqtZR2+UpbnHyDzVbT71rfbVA8Uhheimg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nxp.com; dmarc=pass action=none header.from=nxp.com; dkim=pass header.d=nxp.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nxp.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=MwR+sFvy7Lwp6gHJ2K0MLAB4TTnlwYx/PkrkO8Hykf8=; b=pP90UIXpjgTBBnBZrU3ih0dF7ouwJWo35gUWWZECU0+EJ45YM10RRTg8/2XLgTvRpAugRXDQW1jNzIbvqnFfkUBhoFyCs+ZcivXjcUcZf3uZjHuv8drfB9Ua01KYLpefY0lMTfi3P3R7yW7Yfv/ssKmvGTATcZQv4kBFDeB9GCc= Received: from AM6PR04MB6341.eurprd04.prod.outlook.com (2603:10a6:20b:d8::14) by PR3PR04MB7225.eurprd04.prod.outlook.com (2603:10a6:102:83::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5081.22; Mon, 21 Mar 2022 07:27:36 +0000 Received: from AM6PR04MB6341.eurprd04.prod.outlook.com ([fe80::c39:69cf:c4ea:967]) by AM6PR04MB6341.eurprd04.prod.outlook.com ([fe80::c39:69cf:c4ea:967%5]) with mapi id 15.20.5081.022; Mon, 21 Mar 2022 07:27:36 +0000 From: Ming Qian To: FFmpeg development discussions and patches Thread-Topic: [EXT] [FFmpeg-devel] [PATCH v5 7/7] avcodec/v4l2_m2m_dec: setup capture queue before enqueue the first frame Thread-Index: AQHYAUrA0YR8L2Ey40SyVAAg5vPQpKzJ560g Date: Mon, 21 Mar 2022 07:27:36 +0000 Message-ID: References: <20220104090836.31660-1-ming.qian@nxp.com> <20220104090836.31660-7-ming.qian@nxp.com> In-Reply-To: <20220104090836.31660-7-ming.qian@nxp.com> Accept-Language: zh-CN, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nxp.com; x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 2f0b6603-ae5b-4365-00bd-08da0b0c45d3 x-ms-traffictypediagnostic: PR3PR04MB7225:EE_ x-microsoft-antispam-prvs: x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: L8n7uKMLgfDEvvlXIt9FP6BWyxAssJNs2s7awCw1Z2kKpN/R/ibyLjkSm/cN+LfX3/++dHSsqlxblRfh9iMCCmq0KTLFv15XE8uqz6+nVqhCAG2N3p8tqKRpgOZduEGIk8Ob/1E5b+ltbj/0MZs7nkg9h8RKg/lPZ19jCsydg1RygHmvMZfDSTHz0yCDVXtJ8Hlv9v9xBsuaMJOicSWITdUvsVygYaSAQJsxMAOyi/GT2R/KRY8JOga8LLslz01uVjJpZL0svHrRSkm/dEYos6KOEsh+C1hVT4VRulbpARk1WAY98rbjB+9iRxd8em4Zkflyrzi7M495f7wsQL6udUc/RLV/ncNjWB7nNpJSaZcecWjlWKTTHxyWlCwFJdqiPYj+faAb1uOU1MGKCQ2spNGVl6/jJWjpSbLl+kyXKoKhZv03P3YQXmExub0eBU1471tGRA/tomcE9bG6/+dup3rUB8sLqtHRo3SIbUxQ2wNpmclyi5GQAKnOegbp489xHJUOxBP9AIxM1sEXSJnJFItCwULOKi2bdbq45aAl3W7l4OV+ITwfC/2xLsL3Qpr24/VR9rTNt8qxwhFCt6n14DraB38X0Bti7ny+dAesYhmtviGFu9x62V2QIVHgOehq7V1Uef+nKruYaOWXtOhsrKIeb9crXLRNW8Hk1RI/kOknKplcO/xkqQHw2sdmjP05IcyY/VmvgyLl9DRfw9/SztK4/y9X2F35Y9Rr+jHkjRwBwePuTKXKoi6Jpy5BHF7AZJEE0XeB6dQ4zwsz/BlbdRAWSQpvdgUjEfnDB0SzpNqBOmu87Hdjo7/aovKrxi1Z x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:AM6PR04MB6341.eurprd04.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230001)(4636009)(366004)(33656002)(8936002)(966005)(508600001)(83380400001)(86362001)(44832011)(52536014)(38100700002)(71200400001)(55016003)(6916009)(38070700005)(316002)(26005)(2906002)(186003)(66476007)(64756008)(66556008)(66446008)(8676002)(5660300002)(66946007)(53546011)(45080400002)(9686003)(122000001)(7696005)(6506007)(76116006)(70780200001); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?f7n7kNxQf2FUf0ui7o+d2rlIhDdFD938zPZO1Yl+6RRbISDC21+375/hfXs4?= =?us-ascii?Q?3NO7QfJfDAfd3IAViIvA6bnluhaDaBlPSHZ4J+rG5i7lJL1RuHgkhiFiB8TR?= =?us-ascii?Q?XwUz4aDHG0MaGAGUFlXefkbdZFXjE6KDfKKQFTA0BHejsiQS9M4cIzVa2PYL?= =?us-ascii?Q?wklhLGmxI/NYdEhtOcL307EMULs6uq5UlZ207RioB6RBq4yfS518pS/vF2X+?= =?us-ascii?Q?QgvHF8QeFbztxatFyOQEIPPWDSxx/guB3pPJ8zu1K4rdZ7uG1UjUGC1tYEUZ?= =?us-ascii?Q?lo+7HEaGIQdhj9ngLxaOr7M7b3Hy55ntE0inEexYUY32hXSCzUi9MnrmS36m?= =?us-ascii?Q?6rDWbmErTQxxhl9AmJ3d4u+9Qy+dHxTE1ZUo2gt5X19IgaNjJ5c+NZ1rSB3l?= =?us-ascii?Q?nQvzhxDLpRYMjiM/nK7N7yotCl250ZfetPfAn2Orz/Y5P6oVfRs+u+ZhDF9S?= =?us-ascii?Q?Vj7ldWRCEBbKMjO1L94oizv2cMW0XUooSmyfBgo8R4uSQ0i84A7sH2NRsFk1?= =?us-ascii?Q?F8yfJt2FIYywxtyAAXEcX8ewM0Rnuu9/cmOvl1+0CGoXMrFLPqWj5GltWw+u?= =?us-ascii?Q?2jnHAz2EAzNxzZjNQrkrJ2Bt+qoEuVY7KZP0l4doFUAEuKr+d0cibZ0jszrY?= =?us-ascii?Q?Rt8MfnNABBMg5k8fU3AJ2MnMT7Zetg112EYmzJTGdW8wq+Da8Zi+3aVhMNsE?= =?us-ascii?Q?qZM16h/AW21eZMmrq6s13xmyQ/ZQ+NruA9BlDh34bVQzQEZ/Vf9Tq3PZ77LR?= =?us-ascii?Q?3iiaAP0xauFT1rOkjDvBWMhRwStU8na6dSgWMtGZS5RYquX94pfN8WodgRhz?= =?us-ascii?Q?uO3pXCFGlykOPUSWfVSqq8RMbOnGCnn4MbF91ZKKjgUssFU8bq8v36DjV4n8?= =?us-ascii?Q?2YnQyMcmaM4CWkZH/uX9ntjtQrIGkEbyEsolZvgYlQTmilx+UlzH5Lcz6vK2?= =?us-ascii?Q?iCrQ20nFGHcnWxvKHNjAjGrMwJB4oSV1+I+Cm9WMfFjmYfk/R7xyn3DZ0XUC?= =?us-ascii?Q?uvVo+4r71j8NAf8N8A7NnsWYqq+2D9vjr4bbnCCxiSaFVZibrFIlfcwapk8W?= =?us-ascii?Q?fJAajeGeS7GvVFsPX417aZNH3Jk0f63pixgSxk5qQrCTYMLNP/JPKq/Bj5Zw?= =?us-ascii?Q?8y+R+BZr/uSk/oiFDobfyuRIlvD8BBWH8TC0oG6magfCVS6K/RSjzNkyHMAh?= =?us-ascii?Q?JJ+qy6qHUaPjeauDlvARQWkYfZrquNySRfJ4XrPDBiRxtKMWvbnpeJ9o0MAI?= =?us-ascii?Q?Gprl+KtAQguZfrOuH+jNTtuWxh6YKmRK6MPsNb3AKrNH4wnXUtwID9s7UjwD?= =?us-ascii?Q?/0JzWCt4ET4cw3votzBeScGv/6mvDX5/lTuaBzdsR7++kdMWW9g/+4HL2bdC?= =?us-ascii?Q?49KU6nb87tNg/SJs9SRuQvCR+vtinLT5IjWc5D56QJbIkBWHJwEJ3e5+C8MQ?= =?us-ascii?Q?zReJcD6HhF4=3D?= MIME-Version: 1.0 X-OriginatorOrg: nxp.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: AM6PR04MB6341.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 2f0b6603-ae5b-4365-00bd-08da0b0c45d3 X-MS-Exchange-CrossTenant-originalarrivaltime: 21 Mar 2022 07:27:36.4378 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 686ea1d3-bc2b-4c6f-a92c-d99c5c301635 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: xFwh0/K4ecmWlFzdkoAgNQTa72+Vsjt7yQLhi/vo6SH47KERnJkuGHvwjBYZsFXzD9mJwIc5tUxBlIidx6T76g== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PR3PR04MB7225 Subject: Re: [FFmpeg-devel] [EXT] [PATCH v5 7/7] avcodec/v4l2_m2m_dec: setup capture queue before enqueue the first frame X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: Hi Andriy, What do you think of this patch? The Initialization flow defined in linux/Documentation/userspace-api/media/v4l/dev-decoder.rst 1. Set the coded format on OUTPUT via VIDIOC_S_FMT(). 2. Allocate source (bytestream) buffers via VIDIOC_REQBUFS() on OUTPUT. 3. Start streaming on the OUTPUT queue via VIDIOC_STREAMON(). 4. This step only applies to coded formats that contain resolution information in the stream. Continue queuing/dequeuing bytestream buffers to/from the OUTPUT queue via VIDIOC_QBUF() and VIDIOC_DQBUF(). The buffers will be processed and returned to the client in order, until required metadata to configure the CAPTURE queue are found. This is indicated by the decoder sending a V4L2_EVENT_SOURCE_CHANGE event with changes set to V4L2_EVENT_SRC_CH_RESOLUTION. Note: A client capable of acquiring stream parameters from the bytestream on its own may attempt to set the width and height of the OUTPUT format to non-zero values matching the coded size of the stream, skip this step and continue with the Capture Setup sequence. 5. Continue with the Capture Setup sequence. In ffmpeg's implementation, ffmpeg will set non-zero width and height on output queue, so the step 4 should be skipped, and setup the capture queue directly. So the flow should be: 1. Set the coded format and valid resolution on OUTPUT via VIDIOC_S_FMT(). 2. Allocate source (bytestream) buffers via VIDIOC_REQBUFS() on OUTPUT. 3. Start streaming on the OUTPUT queue via VIDIOC_STREAMON(). 5. Continue with the Capture Setup sequence. And this patch is just following the above flow. Ming > From: ffmpeg-devel [mailto:ffmpeg-devel-bounces@ffmpeg.org] On Behalf Of > Ming Qian > Sent: Tuesday, January 4, 2022 5:09 PM > To: ffmpeg-devel@ffmpeg.org > Subject: [EXT] [FFmpeg-devel] [PATCH v5 7/7] avcodec/v4l2_m2m_dec: setup > capture queue before enqueue the first frame > > Caution: EXT Email > > there are two proper ways to setup capture queue. > 1. client wait the source change event, > then setup the capture queue and streamon 2. client setup the capture > queue in advance, > but to avoid time issues, client should start > the capture queue before it enqueue the sequence > header to decoder driver through output queue. > and the sequence header is always in the first > frame, so client should start capture before > enqueue the first frame. > > ffmpeg use the method 2 to setup capture queue, but currently ffmpeg > enqueue the first frame before starting the capture queue. > so in driver side, there are time issues. > when driver has parsed the resolution from sequence header, but the client > may not finished setup the capture. > so driver can't decide whether to notify a source change event to client. and > the following flow may be chaotic. > > And it's OK that client setup capture queue first, then enqueue the first frame. > > Signed-off-by: Ming Qian > --- > libavcodec/v4l2_m2m_dec.c | 18 ++++++++---------- > 1 file changed, 8 insertions(+), 10 deletions(-) > > diff --git a/libavcodec/v4l2_m2m_dec.c b/libavcodec/v4l2_m2m_dec.c index > b0c3d30ac8ae..e67758531ace 100644 > --- a/libavcodec/v4l2_m2m_dec.c > +++ b/libavcodec/v4l2_m2m_dec.c > @@ -153,6 +153,14 @@ static int v4l2_receive_frame(AVCodecContext *avctx, > AVFrame *frame) > if (s->draining) > goto dequeue; > > + ret = v4l2_try_start(avctx); > + if (ret) { > + /* can't recover */ > + if (ret != AVERROR(ENOMEM)) > + ret = 0; > + goto fail; > + } > + > ret = ff_v4l2_context_enqueue_packet(output, &s->buf_pkt); > if (ret < 0 && ret != AVERROR(EAGAIN)) > goto fail; > @@ -161,16 +169,6 @@ static int v4l2_receive_frame(AVCodecContext *avctx, > AVFrame *frame) > if (ret != AVERROR(EAGAIN)) > av_packet_unref(&s->buf_pkt); > > - if (!s->draining) { > - ret = v4l2_try_start(avctx); > - if (ret) { > - /* cant recover */ > - if (ret != AVERROR(ENOMEM)) > - ret = 0; > - goto fail; > - } > - } > - > dequeue: > return ff_v4l2_context_dequeue_frame(capture, frame, -1); > fail: > -- > 2.33.0 > > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fffmpeg. > org%2Fmailman%2Flistinfo%2Fffmpeg-devel&data=04%7C01%7Cming.qi > an%40nxp.com%7C6b9621a0804c45f9da9108d9cf61fb87%7C686ea1d3bc2b > 4c6fa92cd99c5c301635%7C0%7C1%7C637768841997715387%7CUnknown% > 7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwi > LCJXVCI6Mn0%3D%7C3000&sdata=E7V4kwlJtxbh6jnisMV8FWAwwDzksS > qLp82kapSW4FA%3D&reserved=0 > > To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org > with subject "unsubscribe". _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".