From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTP id D9C794087E for ; Mon, 1 Aug 2022 11:25:02 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id A1FCB68B88F; Mon, 1 Aug 2022 14:24:59 +0300 (EEST) Received: from haasn.dev (haasn.dev [78.46.187.166]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id CC80B68B7C9 for ; Mon, 1 Aug 2022 14:24:53 +0300 (EEST) Received: from haasn.dev (unknown [10.30.0.2]) by haasn.dev (Postfix) with ESMTP id AF0964AB6B for ; Mon, 1 Aug 2022 13:24:52 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=haasn.xyz; s=mail; t=1659353092; bh=4jc3+bSGjfFCqVrdpuEiPGt6pf+sACqcoqO7j5TFJCQ=; h=Date:From:To:Subject:References:From; b=V0oKByRYGKkC7Ww8Emn/xd0Je1o9LOQWKh2lv5CLj9orQ+BSUeEiYppGr9POdFY09 Hv3+NMyNqJlrDGwG6aunAhfqN9tRS70KlUCxNtE4113prYCATYoi8WJS4vdIXogfo1 wtzU0UFIZ/klODziDjqseSvoR+bUqBLHg+DplIRo= Date: Mon, 1 Aug 2022 13:24:52 +0200 Message-ID: <20220801132452.GB41618@haasn.xyz> From: Niklas Haas To: ffmpeg-devel@ffmpeg.org References: MIME-Version: 1.0 Content-Disposition: inline Subject: [FFmpeg-devel] Enhancement layers in FFmpeg X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: Hey, We need to think about possible ways to implement reasonably-transparent support for enhancement layers in FFmpeg. (SVC, Dolby Vision, ...). There are more open questions than answers here. >From what I can tell, these are basically separate bitstreams that carry some amount of auxiliary information needed to reconstruct the high-quality bitstream. That is, they are not independent, but need to be merged with the original bitstream somehow. How do we architecturally fit this into FFmpeg? Do we define a new codec ID for each (common/relevant) combination of base codec and enhancement layer, e.g. HEVC+DoVi, H.264+SVC, ..., or do we transparently handle it for the base codec ID and control it via a flag? Do the enhancement layer packets already make their way to the codec, and if not, how do we ensure that this is the case? Can the decoder itself recursively initialize a sub-decoder for the second bitstream? And if so, does the decoder apply the actual transformation, or does it merely attach the EL data to the AVFrame somehow in a way that can be used by further filters or end users? (What about the case of Dolby Vision, which iirc requires handling the DoVi RPU metadata before the EL can be applied? What about instances where the user wants the DoVi/EL application to happen on GPU, e.g. via libplacebo in mpv/vlc?) How does this metadata need to be attached? A second AVFrame reference inside the AVFrame? Raw data in a big side data struct? _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".