From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTP id 3F8E642193 for ; Tue, 29 Mar 2022 06:51:53 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 8714968B26C; Tue, 29 Mar 2022 09:51:51 +0300 (EEST) Received: from EUR03-VE1-obe.outbound.protection.outlook.com (unknown [40.107.5.48]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id A154C68B151 for ; Tue, 29 Mar 2022 09:51:44 +0300 (EEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=PFxjlrmZORn0VafopUhgGUfMqah/qnFgqQgq8nA/5+ggJKLRFS31DPFPLCKRSJpxaMrgVgj6x8vkj5xwpKCUGSEy0pK6KK/JMNSVYW2RBplv8NB/BVkUaPHd0yHpqSj9yZkxrIBqt+usCNLrmz/e5LlEmhkd5vWrqzU3U25HvOXuDs2EUDbMjkkuqJV7eKkqgWWh3SRA+O9/7nXWaGx7xVv9H/Qb83Vixfaarr9zKR6UEU/wjfmGXHB0ok/PBxA1V2qSDr2BHpJnNviB3A3GWc157JoS1oJvWkCMSAVe9N5AhdmPEpANeV4PFpkZBLDptWMfpgVCE84LkWBtAkLrIg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=MB2E15XY+dMOjbNE9ikc9NpnF+EdEYGipywDxKJFi8w=; b=I8Subk6/mf/QoHGEMqAm9K72Ja7lv52G0+0XiRQtP3wqIz8a52fz7T0OHrGILq4ZDXRBIPf6kMzovlygBFzBd8jjadAKvFuIBlDaElT6qReWDKb3qupV9wu9aC6NNc/50uEKDj5zZrvXbsBiYmfcLjNYCGRh9YPtt8uUa0E1VwLnaTxD6QqgjR28sOs/irYl+4NYHF4OzEWaIxRAgd+qGigtkj93X6pHxtnuWP/cDXt4d4dmqKYGuChXnrDdulUF0Y9D5MCDt3j1/5csSkBXSBnHmIsZmP0lXj5ofEtbdakeYQ2rXFkBBqCctLr7Fc5pxXJFoiwctMYF2cN5pMo9og== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nxp.com; dmarc=pass action=none header.from=nxp.com; dkim=pass header.d=nxp.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nxp.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=MB2E15XY+dMOjbNE9ikc9NpnF+EdEYGipywDxKJFi8w=; b=XJo9wHOhJvkdgUjGdNsTNyHyAeqzRIcd1y3CR5IIypHUKSB1v6sLhhGaunLiYAbLun5H5VMXP6BsCYYx8NwkJpwvn0ka1pI41yFXaxIBATB/MWVUnSXLO2Ir1IR9oOJZHijZdVDg3iIVGAmR5Gps6rwV2UPczjB00+4A6kltNi4= Received: from AM6PR04MB6341.eurprd04.prod.outlook.com (2603:10a6:20b:d8::14) by AM5PR04MB2947.eurprd04.prod.outlook.com (2603:10a6:206:6::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5102.23; Tue, 29 Mar 2022 06:51:43 +0000 Received: from AM6PR04MB6341.eurprd04.prod.outlook.com ([fe80::c39:69cf:c4ea:967]) by AM6PR04MB6341.eurprd04.prod.outlook.com ([fe80::c39:69cf:c4ea:967%5]) with mapi id 15.20.5102.023; Tue, 29 Mar 2022 06:51:43 +0000 From: Ming Qian To: Andriy Gelman , FFmpeg development discussions and patches Thread-Topic: [FFmpeg-devel] [EXT] [PATCH v5 7/7] avcodec/v4l2_m2m_dec: setup capture queue before enqueue the first frame Thread-Index: AQHYQSgmOmqMxYmBBUyFNpBgKQDarKzV1yrQ Date: Tue, 29 Mar 2022 06:51:43 +0000 Message-ID: References: <20220104090836.31660-1-ming.qian@nxp.com> <20220104090836.31660-7-ming.qian@nxp.com> <20220326154245.oy4usrr2fvymk2xx@jackie> In-Reply-To: <20220326154245.oy4usrr2fvymk2xx@jackie> Accept-Language: zh-CN, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nxp.com; x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 6da349c8-3e78-4a34-855c-08da115095b1 x-ms-traffictypediagnostic: AM5PR04MB2947:EE_ x-microsoft-antispam-prvs: x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: Q089c/a6YuRfSK4rYm5NeBIolAO9uOMcAiw5DhshRkGo5286AR7wlzhtRrsttnbHNVlQZDXfTbuA5TFCeQ480UQG0p7m/HivwUQxMPFj/gMzatdkcHTMi5YP2xAsnMPZcLT2wX+A5bXtrDl2ftR8r1QUHQwBb904szpi4Bu9zhqgxDHh+TuoDCuJ0kHKpZvX0nHOyDwyEe7S74VrxX9j3J5nFOVkh2sN9iRr36Psa4d5ceA2RY29BNGHZMIoAhGY3N6yXYmGr0lCAso/Kp62MRESbxVi11e3829Ip9ER4+TzvceDyiqvwZtwaP1R7utF4KMpWSuuGYIFAiJFfePe83Vx0FqsXAm//4ngVsLPVXx12xApP+IOhtRfSm6OirFAcTlfS4UbPIiAv53GW/AjxVyRhmjWQhc6VIWZfDI3AWZsoD4TnRRUgM1rZLUvAf/UUXC11T/Phisc4sSBaDLZT602mmX/U48ZPFu6Md0Q4oZZs2tetqblwIwyg8D8V5pSSgXnXV33Xcc9ChZwcjgKHUqoqjzf0Z2MPIkIoA0Ipdk20v9jr6/29CkPsZ3vug+YyOR7EUf8FyQ8Hjlf7sUVBXxAZ/oL6L5Hf92Um5PPvI44nLjY9B4P9nIq/mpo4p+yMZL9w46lNXZPplOeLaBGPZW9Hxezpsv+YspHrPggroz8f40oEmk14gsDbqf9qLYv5leHRjrv7ISH259pgpxTEciHFRUjGVv5XpKXNEwGJYKQEhZ/LNeasnKtnjO6gIAG x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:AM6PR04MB6341.eurprd04.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230001)(4636009)(366004)(53546011)(122000001)(45080400002)(76116006)(55016003)(71200400001)(86362001)(8676002)(2906002)(66946007)(38070700005)(83380400001)(38100700002)(316002)(7696005)(66556008)(5660300002)(66446008)(66476007)(508600001)(8936002)(52536014)(186003)(33656002)(110136005)(966005)(64756008)(6506007)(26005)(9686003)(44832011)(70780200001); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?04iDD+xzI7B3AH2LVukoM7pP/cg3KlRlOFenLckkv473pH7yWGRkDV//PMOE?= =?us-ascii?Q?aYa73uB4+Q6z8eJkypmCG8YlsWcoC87BsYtMdP0NKBsflcd6Otwm4lKvvDbB?= =?us-ascii?Q?yHAAf6aA7aT1mVvtD6d//w6ybrvWX7Npb62giUkLU0aCrVuz0d9s1qjzDpL1?= =?us-ascii?Q?UVTv4e0yNFV1gUZ8ualJpjdYrnrqbQ513j2RpRJ6qcmD9orT8vNTEvnueOdC?= =?us-ascii?Q?w37NZfTkxogGZ0xzWin7xQe3otAVF31sXw+MJr+A3ggd4n2iKF1gyUCrBkPS?= =?us-ascii?Q?96mne5uHE/0l4noiQp0TGdPfD6DwX+2gg8tBzWw3SZyHrBnBWpFitHaoy4JD?= =?us-ascii?Q?Qq6rgzLusGsaZO3wGzJcvLZmt2uM12dnWJPD3X29luBZLtfJbS07Yir+4lb6?= =?us-ascii?Q?yAilKqyI8N0lkUhiLZ/UrLTdWa/2mtQrem9tvAPVO9uoU0SxuezJ/XZti1wI?= =?us-ascii?Q?5ACrk8DjvakPwmPGst8YLrS4Pat+vt1ymennn5vqag9qEp3nEKYmsYtAB2AI?= =?us-ascii?Q?g3+XfSiWXwJLYym6tW7BLNJBLKN7CmB7ITPjIMqRpSVkxq0tLYRQ78s/vSHS?= =?us-ascii?Q?CTlK7wDCI3/1/GoLey9v0xRchKliupQ4vweqzsj1L89evJfZ0cKsap6r0ThQ?= =?us-ascii?Q?7dzftgog6MjwtzTIUIGGThiwkvbdUQ95F0K6Ai1iGzsM3g4uG0UWYeat4w6u?= =?us-ascii?Q?7nH3X291M5BeQlMrlzWEih5GB2TxQ9voXqXzPzFva+AmaDsOkTWCO0R3SsfJ?= =?us-ascii?Q?oGh18W/VeqxL8fyakoisKlC6flDpQNLOMUtex+DxOB1Ujip51lhLMHHOcukM?= =?us-ascii?Q?/YHpadwibPVQurDgV9yHsxYaI7quNd3+E5/TZ+WTCbzil7kyDNI995OiF0Il?= =?us-ascii?Q?vZzmtHruvXDvIPDuGQfiaGBHJIqM4/4dyHi5mLeDwZvhao578FTHMRE2yNS4?= =?us-ascii?Q?BaA6FEQTENTfIde0ibZrAaRQP4P8N6Fe4Oo50opVmaLr8VFQAeyd7EOb0QiC?= =?us-ascii?Q?Kua7FrYuTo6SgsWLutSIiIc8mppKX6lv/qTV2yI1Hb9r2ZxdbRUxNJDVIssh?= =?us-ascii?Q?8hm7rMf0G79wT8NFyYC0eQyAg2y/L26Lp/EZhedbvl9hJAvLBq+lHwckTQyP?= =?us-ascii?Q?ZcxrhJuFQStvREA8uFT/qr3Ym1aqEAEgOxDlWBVSAwsz0U3/SojNtrua5Ofu?= =?us-ascii?Q?WbnDx/qLugIl4bGSwZwqN2xKL5c8VI3RU77Ai2QHoBymVAhJvd4Cp78FWzL4?= =?us-ascii?Q?p/Nj5aTO15yRCA5tPhr0g/WeXdHldmJApcwCnVnK2pTxuTACEGOzJVTtPN8S?= =?us-ascii?Q?hKTvRaXPIHe3+NCMlIoqWa0k3dELehsXug5UICjzIz5fcR3iZxTv91eY3hIE?= =?us-ascii?Q?w7enONT3dV8FHWt86dr/5zELUXd38q+NVCpGG+h04Xj44CUIa+R3YqcC+Llo?= =?us-ascii?Q?/OC528G3hwi5xUK6NDN8sR+0Gstgur+G+RonGAt6faaOOjAFCcV/thYxhsqa?= =?us-ascii?Q?61PE6xHVQKwMsgJENcxZszC/38I/sxFGcKTcGOFvu8I7jfWLN6croULvZ1r2?= =?us-ascii?Q?9hadBJdu3M5zUzyiqBlfofiwPznLdYytKooM3VJUBC7iZLnynpeFbAWiSK6P?= =?us-ascii?Q?fgtR4eFfY1R+MtnCHMPQsdGdnzpSizsWunq470zEPxhizLr94I+Cab5OXCip?= =?us-ascii?Q?gSaPFxXDaxFbMhyZA2kGdmY8sLKXG18vwC08NqsDJ8hQuep8crDy9NoeGtJt?= =?us-ascii?Q?bmxBTNjqqA=3D=3D?= MIME-Version: 1.0 X-OriginatorOrg: nxp.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: AM6PR04MB6341.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 6da349c8-3e78-4a34-855c-08da115095b1 X-MS-Exchange-CrossTenant-originalarrivaltime: 29 Mar 2022 06:51:43.1904 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 686ea1d3-bc2b-4c6f-a92c-d99c5c301635 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: cesYaGJx0vM75kpo6Ye7ZbZgogv46rTeqgLw6ykQk/O8dMJ/odqs5Q/x5SJ8GrtKtdx/KDCONaLHlW4mDrVDmA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM5PR04MB2947 Subject: Re: [FFmpeg-devel] [EXT] [PATCH v5 7/7] avcodec/v4l2_m2m_dec: setup capture queue before enqueue the first frame X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: > From: Andriy Gelman [mailto:andriy.gelman@gmail.com] > Sent: Saturday, March 26, 2022 11:43 PM > To: FFmpeg development discussions and patches > Cc: Ming Qian > Subject: Re: [FFmpeg-devel] [EXT] [PATCH v5 7/7] avcodec/v4l2_m2m_dec: > setup capture queue before enqueue the first frame > > Caution: EXT Email > > Hi Ming, > > On Mon, 21. Mar 07:27, Ming Qian wrote: > > Hi Andriy, > > > > What do you think of this patch? > > > > The Initialization flow defined in > linux/Documentation/userspace-api/media/v4l/dev-decoder.rst > > 1. Set the coded format on OUTPUT via VIDIOC_S_FMT(). > > 2. Allocate source (bytestream) buffers via VIDIOC_REQBUFS() on OUTPUT. > > 3. Start streaming on the OUTPUT queue via VIDIOC_STREAMON(). > > 4. This step only applies to coded formats that contain resolution > information in the stream. Continue queuing/dequeuing bytestream buffers > to/from the OUTPUT queue via VIDIOC_QBUF() and VIDIOC_DQBUF(). The > buffers will be processed and returned to the client in order, until required > metadata to configure the CAPTURE queue are found. This is indicated by the > decoder sending a V4L2_EVENT_SOURCE_CHANGE event with changes set to > V4L2_EVENT_SRC_CH_RESOLUTION. > > Note: A client capable of acquiring stream parameters from the > bytestream on its own may attempt to set the width and height of the > OUTPUT format to non-zero values matching the coded size of the stream, skip > this step and continue with the Capture Setup sequence. > > > > 5. Continue with the Capture Setup sequence. > > > > In ffmpeg's implementation, ffmpeg will set non-zero width and height > on output queue, so the step 4 should be skipped, and setup the capture > queue directly. > > So the flow should be: > > 1. Set the coded format and valid resolution on OUTPUT via VIDIOC_S_FMT(). > > 2. Allocate source (bytestream) buffers via VIDIOC_REQBUFS() on OUTPUT. > > 3. Start streaming on the OUTPUT queue via VIDIOC_STREAMON(). > > 5. Continue with the Capture Setup sequence. > > > > And this patch is just following the above flow. > > > > Ming > > The v4 version didn't work for me on odroid xu4 > https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fffmpeg.o > rg%2Fpipermail%2Fffmpeg-devel%2F2022-January%2F290679.html&dat > a=04%7C01%7Cming.qian%40nxp.com%7Cca988d592253473cbac408da0f3f4 > 801%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C637839061704 > 093567%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2l > uMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=I80bpE70S6JI > 8A%2BaXQGq4JyxL0eBbtLN58p5SpqlA6s%3D&reserved=0 > > I haven't had time retest but don't think anything has changed. > > -- > Andriy Hi Andriy, I check the S5P MFC Video Codec driver, and indeed it requires the client enqueue the output buffer which contains the sequence header. And in vidioc_g_fmt(), driver will block and wait until the header is parsed. And indeed my patch will not work on it. But it seems don't meet the specification. For S5P MFC, the step 4 can't be skipped, and the client should wait the V4L2_EVENT_SOURCE_CHANGE event before the step 5 setting up capture queue. The ffmpeg v4l2 decoder don't skip step 4, but also doesn't wait the V4L2_EVENT_SOURCE_CHANGE event. So in current, we should make it work on more devices instead of more spec? The Initialization flow defined in linux/Documentation/userspace-api/media/v4l/dev-decoder.rst 1. Set the coded format on OUTPUT via VIDIOC_S_FMT(). 2. Allocate source (bytestream) buffers via VIDIOC_REQBUFS() on OUTPUT. 3. Start streaming on the OUTPUT queue via VIDIOC_STREAMON(). 4. This step only applies to coded formats that contain resolution information in the stream. Continue queuing/dequeuing bytestream buffers to/from the OUTPUT queue via VIDIOC_QBUF() and VIDIOC_DQBUF(). The buffers will be processed and returned to the client in order, until required metadata to configure the CAPTURE queue are found. This is indicated by the decoder sending a V4L2_EVENT_SOURCE_CHANGE event with changes set to V4L2_EVENT_SRC_CH_RESOLUTION. Note: A client capable of acquiring stream parameters from the bytestream on its own may attempt to set the width and height of the OUTPUT format to non-zero values matching the coded size of the stream, skip this step and continue with the Capture Setup sequence. 5. Continue with the Capture Setup sequence. Ming _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".