From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTP id E0E1240957 for ; Sun, 3 Apr 2022 22:29:36 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id A2AAE68B2AA; Mon, 4 Apr 2022 01:29:33 +0300 (EEST) Received: from mail-qk1-f172.google.com (mail-qk1-f172.google.com [209.85.222.172]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 6BEBE68991B for ; Mon, 4 Apr 2022 01:29:26 +0300 (EEST) Received: by mail-qk1-f172.google.com with SMTP id k125so6399370qkf.0 for ; Sun, 03 Apr 2022 15:29:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=SIchZGoptVlyiCoBOqXgXhsURj/48pExWUaaMCv3VJ8=; b=U5OeBwlJ9QUeenvOgKXiJs3OoWpfUO4o36Qx1IrWsuIo4ROANXDXAapMjOSfJB37ke 0irTCSL53n2xGOYDE6Aave0LLt546R7yndYarg3On1jcqwITeoaVcTtF0SpnEtp0gM6S YPHuzuTyoZ2Mz8RqdQHBouynHwhuXVe4iAykch6PigDBCZf0eGr4RXfwxTRMYJRZhDq0 sJDQMPuA23yA33bRSWDYkRGDIYrn0klewqh3sPqFR7UuiLUGxw84sPv308q7EATQmUCw 5Hyk8+FhyCD4er3UP+3vCn/MyjgpLDjqGWBYBFgQ1sspZVl7zwI7PGRqYW2wXVJT5k4K 7euQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=SIchZGoptVlyiCoBOqXgXhsURj/48pExWUaaMCv3VJ8=; b=06xSiJC9un1ykzWq7mQWRPnvNzKxyVcSYg619SvEE1vDnIXV4yfEinjyK40hh6kMsf woaO9eDL3DKc3TQ96zVBVFLDo5FpVYPyYJW/Mup4Lxe8KayT15whkQZuaE4zp0oSEcxB skjVL96W3VJC0fkLEqAx6T3seKmpQpzW9ig0llYOiFpZr8z2a3WiSFbBjVuApy9gFtPn 76/Kcn7NV2QAurrbGZNi16evmXRFsyEBe026kboXsnsVVjmWzJKwQzQY3d/sMAWuEnW5 7Tnx4PHSq0DQ/rio6Z38AdsHl6pHCkpIroo7J+Lp5FgpZy/HgUh0FtvUJdshCgUgNB3D w7Zg== X-Gm-Message-State: AOAM530LDQS+nR+Denjo1eDkh/SEBQauJjn9GaspsvVyO6eQ1whJQYJT cdfifjZ2WnNQLrgNVqQUPy4= X-Google-Smtp-Source: ABdhPJzbIoZbMpR2DRMgCffbsP/RNRChSJWbX4zxKV7SRyUaB4f9UoOaBAlm5YvPlibqcOjRg9sjMA== X-Received: by 2002:a05:620a:2484:b0:680:9f76:aa7a with SMTP id i4-20020a05620a248400b006809f76aa7amr12687785qkn.672.1649024964943; Sun, 03 Apr 2022 15:29:24 -0700 (PDT) Received: from jackie (pool-108-20-201-137.bstnma.fios.verizon.net. [108.20.201.137]) by smtp.gmail.com with ESMTPSA id g5-20020ac87f45000000b002e125ef0ba3sm7093282qtk.82.2022.04.03.15.29.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 03 Apr 2022 15:29:24 -0700 (PDT) Date: Sun, 3 Apr 2022 18:29:22 -0400 From: Andriy Gelman To: Ming Qian Message-ID: <20220403222922.ae5dxqaz3mt25myd@jackie> References: <20220104090836.31660-1-ming.qian@nxp.com> <20220104090836.31660-7-ming.qian@nxp.com> <20220326154245.oy4usrr2fvymk2xx@jackie> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Subject: Re: [FFmpeg-devel] [EXT] [PATCH v5 7/7] avcodec/v4l2_m2m_dec: setup capture queue before enqueue the first frame X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: FFmpeg development discussions and patches Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: Hi Ming, Sorry for the late reply. On Tue, 29. Mar 06:51, Ming Qian wrote: > > From: Andriy Gelman [mailto:andriy.gelman@gmail.com] > > Sent: Saturday, March 26, 2022 11:43 PM > > To: FFmpeg development discussions and patches > > Cc: Ming Qian > > Subject: Re: [FFmpeg-devel] [EXT] [PATCH v5 7/7] avcodec/v4l2_m2m_dec: > > setup capture queue before enqueue the first frame > > > > Caution: EXT Email > > > > Hi Ming, > > > > On Mon, 21. Mar 07:27, Ming Qian wrote: > > > Hi Andriy, > > > > > > What do you think of this patch? > > > > > > The Initialization flow defined in > > linux/Documentation/userspace-api/media/v4l/dev-decoder.rst > > > 1. Set the coded format on OUTPUT via VIDIOC_S_FMT(). > > > 2. Allocate source (bytestream) buffers via VIDIOC_REQBUFS() on OUTPUT. > > > 3. Start streaming on the OUTPUT queue via VIDIOC_STREAMON(). > > > 4. This step only applies to coded formats that contain resolution > > information in the stream. Continue queuing/dequeuing bytestream buffers > > to/from the OUTPUT queue via VIDIOC_QBUF() and VIDIOC_DQBUF(). The > > buffers will be processed and returned to the client in order, until required > > metadata to configure the CAPTURE queue are found. This is indicated by the > > decoder sending a V4L2_EVENT_SOURCE_CHANGE event with changes set to > > V4L2_EVENT_SRC_CH_RESOLUTION. > > > Note: A client capable of acquiring stream parameters from the > > bytestream on its own may attempt to set the width and height of the > > OUTPUT format to non-zero values matching the coded size of the stream, skip > > this step and continue with the Capture Setup sequence. > > > > > > 5. Continue with the Capture Setup sequence. > > > > > > In ffmpeg's implementation, ffmpeg will set non-zero width and height > > on output queue, so the step 4 should be skipped, and setup the capture > > queue directly. > > > So the flow should be: > > > 1. Set the coded format and valid resolution on OUTPUT via VIDIOC_S_FMT(). > > > 2. Allocate source (bytestream) buffers via VIDIOC_REQBUFS() on OUTPUT. > > > 3. Start streaming on the OUTPUT queue via VIDIOC_STREAMON(). > > > 5. Continue with the Capture Setup sequence. > > > > > > And this patch is just following the above flow. > > > > > > Ming > > > > The v4 version didn't work for me on odroid xu4 > > https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fffmpeg.o > > rg%2Fpipermail%2Fffmpeg-devel%2F2022-January%2F290679.html&dat > > a=04%7C01%7Cming.qian%40nxp.com%7Cca988d592253473cbac408da0f3f4 > > 801%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C637839061704 > > 093567%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2l > > uMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=I80bpE70S6JI > > 8A%2BaXQGq4JyxL0eBbtLN58p5SpqlA6s%3D&reserved=0 > > > > I haven't had time retest but don't think anything has changed. > > > > -- > > Andriy > > Hi Andriy, > I check the S5P MFC Video Codec driver, and indeed it requires the client enqueue the output buffer which contains the sequence header. And in vidioc_g_fmt(), driver will block and wait until the header is parsed. > And indeed my patch will not work on it. > But it seems don't meet the specification. > For S5P MFC, the step 4 can't be skipped, and the client should wait the V4L2_EVENT_SOURCE_CHANGE event before the step 5 setting up capture queue. > The ffmpeg v4l2 decoder don't skip step 4, but also doesn't wait the V4L2_EVENT_SOURCE_CHANGE event. > > So in current, we should make it work on more devices instead of more spec? Personally I'd prefer not to break users. Odroid xu4 is a popular board afaik. But I'm also not the maintainer of v4l2m2m. IMO it would be better the fix the problem in the driver first. > > The Initialization flow defined in linux/Documentation/userspace-api/media/v4l/dev-decoder.rst > 1. Set the coded format on OUTPUT via VIDIOC_S_FMT(). > 2. Allocate source (bytestream) buffers via VIDIOC_REQBUFS() on OUTPUT. > 3. Start streaming on the OUTPUT queue via VIDIOC_STREAMON(). > 4. This step only applies to coded formats that contain resolution information in the stream. Continue queuing/dequeuing bytestream buffers to/from the OUTPUT queue via VIDIOC_QBUF() and VIDIOC_DQBUF(). The buffers will be processed and returned to the client in order, until required metadata to configure the CAPTURE queue are found. This is indicated by the decoder sending a V4L2_EVENT_SOURCE_CHANGE event with changes set to V4L2_EVENT_SRC_CH_RESOLUTION. > Note: A client capable of acquiring stream parameters from the bytestream on its own may attempt to set the width and height of the OUTPUT format to non-zero values matching the coded size of the stream, skip this step and continue with the Capture Setup sequence. > 5. Continue with the Capture Setup sequence. -- Andriy _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".