From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTP id 05BCC446E7 for ; Thu, 24 Nov 2022 21:10:28 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 1F6AC68B42D; Thu, 24 Nov 2022 23:10:25 +0200 (EET) Received: from mail-wr1-f44.google.com (mail-wr1-f44.google.com [209.85.221.44]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id C7F63689A02 for ; Thu, 24 Nov 2022 23:10:18 +0200 (EET) Received: by mail-wr1-f44.google.com with SMTP id bs21so4098621wrb.4 for ; Thu, 24 Nov 2022 13:10:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jkqxz-net.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:in-reply-to:from:references:to :content-language:subject:user-agent:mime-version:date:message-id :from:to:cc:subject:date:message-id:reply-to; bh=jedoR+TYnvXOlVQIzY/gxPCSILplSnL3ZK3NJb17tvM=; b=ST/UfurFTbrLoUjLXoIg0Lv3vXI7IC7OyVYbb2r0xtOXF+TYCzu7NcBLJuURARJIWU ZzBeC/5Ld289F36+KDw0VzVHXxdJUby5wf6GHvM5RL+kcoUePWXChIkWVW6h9+Hmlv7P HXwbtm9jAiH9qoNuBUnE+DFKyJJ8E6vwwad/dvKmFvKHzTMQfVT3yfpiThwmFTN1P3+2 mgB/UbwVQbn/+xCr3nTu1jmRbNDC8Ry9CoPTECXxWYcPXcWSU6vGW7+IaV8jokwxFaZz PkqMQlwwcImWJV4PpelrsynTiFbsc6JHHYAFSCOsNmV8lqMwrhCdEDiS3rhKpBBbEzDf msXw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:in-reply-to:from:references:to :content-language:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=jedoR+TYnvXOlVQIzY/gxPCSILplSnL3ZK3NJb17tvM=; b=3ymj5KaAIUS7ScoobZ7KbCdsP/J+Yk5Z0ygggw06z265qn6LbFpG4XarsrhyZ112VL hSgDiuVC+HNcSJ8YFrcvHNXupKHcUV1he1uLqk9UWDMYx/qyXRSmrzhdmff+FnI27KSB KVqoJdHZCz2IeqFAUsMyI7kiIw8kH3d9BEsNPgiQA44+G4cSES2rjdlx026bH4/1zfe/ 245z36evtLF0ztNZPSnAFYuuvbe8kujHxMMWc1JsJnyWnQnHuxLPw8uHX64Pnce4aPEE X4w01AEXwPr9B0SZs6qI9l1DNDOgNcFVb8GJlTeGjLsFICyuyPi7NIyBdRWIeM7cgfZO SFEQ== X-Gm-Message-State: ANoB5pkD5qMCqN5QK3DadG8pf3hvP3gWeJX37TqFdTlZ2NjR5b/l/c3N 9PSQNDkaX80tRvuJrvrDQ76XuiDjNL920A== X-Google-Smtp-Source: AA0mqf7T8hhiR6dS/BAdWKV87tc7CDLeJ47FKgSYACUih9Cz14PxzrZgSRrDhYsN99xZx/kaHW2bUQ== X-Received: by 2002:adf:f60d:0:b0:22e:d91a:231b with SMTP id t13-20020adff60d000000b0022ed91a231bmr21916800wrp.78.1669324218106; Thu, 24 Nov 2022 13:10:18 -0800 (PST) Received: from [192.168.0.14] (cpc91224-cmbg18-2-0-cust209.5-4.cable.virginm.net. [81.106.228.210]) by smtp.gmail.com with ESMTPSA id j14-20020a05600c190e00b003b47e75b401sm8502119wmq.37.2022.11.24.13.10.17 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 24 Nov 2022 13:10:17 -0800 (PST) Message-ID: <0ae356e7-e413-ee32-b67b-5043476e705a@jkqxz.net> Date: Thu, 24 Nov 2022 21:09:56 +0000 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.4.1 Content-Language: en-US To: "Dong, Ruijing" , FFmpeg development discussions and patches References: <20221118153422.67632-1-ruijing.dong@amd.com> <20221120025914.39732-1-ruijing.dong@amd.com> <4776acec-357b-5c36-1292-7e5204c4759f@jkqxz.net> <4df30d96-5b77-bc2b-ce3a-4c253889a408@jkqxz.net> From: Mark Thompson In-Reply-To: Subject: Re: [FFmpeg-devel] [PATCH v2] avcodec/av1_vaapi: add direct film grain mode X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: On 24/11/2022 15:27, Dong, Ruijing wrote: > [AMD Official Use Only - General] > > I might have misunderstood some of the questions, and I would like to explain more about the issue from my perspective, please correct me if anything wrong. Are you perhaps missing that this is intended to be a common API between multiple implementations and users? The important feature is that once the API has specified the behaviour then both the driver implementers (such as Mesa, but also others) and the users (such as FFmpeg, but also others) don't need to know what is going on on the other side, because the other side always behaves in the same way. In this case, the API specifies that there are two output surfaces - one is written with the reference (pre-grain) frame, one is written with the display (post-grain) frame. The Mesa driver is incorrect because it writes the display frame to the reference surface, and doesn't bother to write the display surface at all. This is then very visible when any user tries to display the display surface, because it hasn't been written. > This patch is NOT a hack, like @Mark Thompson mentioned. Your proposed patch is trying to crystalise an alternative not-quite-VAAPI which uses the same library and appears compatible, but actually isn't because it differs in critical ways. I don't think it is unreasonable to characterise this as a hack. > Video codec, especially decoders will need to meet the requirements of video codecs first, if the reference picture management (DPB) was implemented wrongly, > then it could not meet the fundamental decoder criteria. From this point of view, different hardware will need to follow the same standard for the implementation > so that the decoders can generate the conformance outputs. > > The DPB is always an internal part of the decoder, the detail implementation could be differed with different benefits, if DPB is managed by the application, it can be > more flexible and easily maintained, the other way is the DPB is managed by the driver and hardware itself, it could have more space for the optimization, for > example the reference frame access, where the format of reference frames is NOT used for display output, and the display output cannot be used for the > reference frames neither because reference frames could use a different format, which is more efficient for reference access however not good for display. > From my point of view VAAPI supports both of the above two ideas, and it is not necessary to force one to follow another, because that is limited by the initial design > Idea. In this case, there is no pre-grain output in the former decoder, it has only one display output. Fair enough. Luckily, the API clearly states where that display output should be written: . You can get away with not writing the reference frame because it won't be displayed in this case, but you must write the display frame because that is the one the API says will be displayed to the user. The unwritten reference frame will still get used in references because that is how the API is defined, but since Mesa deals with this by attaching hidden metadata to the frames there is no problem. > From the other side, most of the AMD AV1 decoding issues are resolved from the community, the film grain problem becomes more noticeable. And generally speaking > it is usually a flexible part of the post processing phase after video decoding, and here it is strictly defined in AV1 spec, and it is part of the decoding standard. > It is not practical to make changes in the DPB design idea for resolving this issue from AMD decoder side. And naturally output the applied firm grain is just another > film grain process mode, I called it "direct film grain mode". > > I have asked the community to inspire me to have a better idea, and eventually I found out there is no good way other than to have the external user choice or detecting > AMD driver. I understand doing the string match to choose AMD driver is not a perfect idea, but we really need to have a method to resolve this issue. Please let me know > If a better way come to your mind, which can resolve this issue and in the meanwhile not affecting other AV1 implementation path. The proper fix is straighforward: correct the error in the Mesa driver so that it writes the display output to the display surface, as the API says it should be. Thanks, - Mark _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".