From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTP id A7BCB45737 for ; Fri, 21 Jul 2023 21:31:37 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id B2D4468C64A; Sat, 22 Jul 2023 00:31:11 +0300 (EEST) Received: from mail-qk1-f180.google.com (mail-qk1-f180.google.com [209.85.222.180]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 2B57268BA8D for ; Sat, 22 Jul 2023 00:31:04 +0300 (EEST) Received: by mail-qk1-f180.google.com with SMTP id af79cd13be357-76728ae3162so199661085a.3 for ; Fri, 21 Jul 2023 14:31:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ltnglobal-com.20221208.gappssmtp.com; s=20221208; t=1689975063; x=1690579863; h=references:in-reply-to:message-id:date:subject:cc:to:from:from:to :cc:subject:date:message-id:reply-to; bh=k27HggGtyh/7SQlalRdyM0KZT2OoJTKokQsrvPp4dcE=; b=YXUS9B+fw33LEH61elljofnFozKcX5u1Xfgt7ORzxd2RoNxZtHrfpVIjpeSVMNkMqL 6QedwBYXM5HPkFg8uONEzhD39EAGd7Ww1OxgQtzS+HnPc1jlXNKOCwktmIxvhCTC8FXu paWrhZvpnMCBSnty2wY0l3Hgjtmw7Y3jXX+2LUg7fjrVue6ZbAWp39xxdp32VBQXXVqY e2OB81C74cPptH6b1yf7kT4p4uR1bmqk3XSDLqC/tSu9voH+f1TpF+a0H0amVKe+IXI/ kDKDGrEIOSnDs+kEyAOUogJpywZfvovY/FA1SUCVoelPlN+DxkTWvgT7VjuQHtFBpSC0 jZKA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689975063; x=1690579863; h=references:in-reply-to:message-id:date:subject:cc:to:from :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=k27HggGtyh/7SQlalRdyM0KZT2OoJTKokQsrvPp4dcE=; b=DOO5ukWW2VWyqKbhjE/3JdgnvcbC3gt9MnVoJEfPuJE3fEYvKYVubO/yfgv3IMxEB3 mib+wBPAwY4NSDvTGGhXXJo+U1RxleA5Zx0BrDyxNSxsbv+l2pBj8TsqsA+Cr4PP00cz +g63U243Y9aXaE+NxuK25lxv15rioCPm1Xm8BwWKiulWYxn56yysU/6Ylvcxga3oOHRH WcbMADPeAuun6NqdVJZAgPxlUn8SUEUEs1f2WSnSB9LZObWNAMjQyU6fz03ks/myF1LM 2hN6OCK145SqMcaO4lT9nHeHRQvmWDTCzJhpZurfwlQEeMe/IrTuqx6DuaZ3OjDWP70s Et2Q== X-Gm-Message-State: ABy/qLatIxypzjxHSQ1zwwxhHktEmLqei1FQnOGtm2J6deIXYCqnHRYl EIYw6Db1DzOBRSUixf5hUqxul9amCcdDTiQXNxw= X-Google-Smtp-Source: APBJJlGpJMwA/MzQJiGYn/ojr1JagT2fW5hbeh/xZlAP9t+Z/7Z+FwyA6oRiapIRXkX2dxj2q67GjA== X-Received: by 2002:a05:620a:4501:b0:767:d6f9:452b with SMTP id t1-20020a05620a450100b00767d6f9452bmr1523390qkp.12.1689975062775; Fri, 21 Jul 2023 14:31:02 -0700 (PDT) Received: from ltnt-nyc-580testdevin.livetimenet.com (pool-71-105-132-214.nycmny.fios.verizon.net. [71.105.132.214]) by smtp.gmail.com with ESMTPSA id z21-20020ae9c115000000b0076738337cd1sm1406333qki.1.2023.07.21.14.31.02 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 21 Jul 2023 14:31:02 -0700 (PDT) From: Devin Heitmueller X-Google-Original-From: Devin Heitmueller To: ffmpeg-devel@ffmpeg.org Date: Fri, 21 Jul 2023 17:30:57 -0400 Message-Id: <1689975057-22226-4-git-send-email-dheitmueller@ltnglobal.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1689975057-22226-1-git-send-email-dheitmueller@ltnglobal.com> References: <1689975057-22226-1-git-send-email-dheitmueller@ltnglobal.com> Subject: [FFmpeg-devel] [RFC][PATCH 3/3] decklink: Add support for output of HDR metadata X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Devin Heitmueller MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: Add HDR support to the decklink output for cards that support such functionality. This includes setting the EOTF, the colorspace, the mastering info, and the content light level info. Both the Payload Identification HANC data as well as the SMPTE ST 2108-1 VANC data are being set. Tested with in-house content as well as samples from 4kmedia.org. Testing was done with the Decklink 8K Pro and the Duo2 with 12.5.1 firmware, as well as with the Duo2 with 10.11.2 (before it supported HDR) to ensure there are no regressions. Signed-off-by: Devin Heitmueller --- libavdevice/decklink_common.cpp | 12 +++ libavdevice/decklink_common.h | 2 + libavdevice/decklink_enc.cpp | 204 ++++++++++++++++++++++++++++++++++++++-- 3 files changed, 211 insertions(+), 7 deletions(-) diff --git a/libavdevice/decklink_common.cpp b/libavdevice/decklink_common.cpp index 47de7ef..c1bcb82 100644 --- a/libavdevice/decklink_common.cpp +++ b/libavdevice/decklink_common.cpp @@ -251,6 +251,18 @@ int ff_decklink_set_configs(AVFormatContext *avctx, } } + DECKLINK_BOOL hdr_supported; + if (ctx->attr->GetFlag(BMDDeckLinkSupportsHDRMetadata, &hdr_supported) == S_OK) { + if (hdr_supported) + ctx->supports_hdr = 1; + } + + DECKLINK_BOOL colorspace_supported; + if (ctx->attr->GetFlag(BMDDeckLinkSupportsColorspaceMetadata, &colorspace_supported) == S_OK) { + if (colorspace_supported) + ctx->supports_colorspace = 1; + } + return 0; } diff --git a/libavdevice/decklink_common.h b/libavdevice/decklink_common.h index 34ab1b9..d50007f 100644 --- a/libavdevice/decklink_common.h +++ b/libavdevice/decklink_common.h @@ -109,6 +109,8 @@ struct decklink_ctx { int bmd_height; int bmd_field_dominance; int supports_vanc; + int supports_hdr; + int supports_colorspace; /* Capture buffer queue */ DecklinkPacketQueue queue; diff --git a/libavdevice/decklink_enc.cpp b/libavdevice/decklink_enc.cpp index ffd0ad9..92d6bbe 100644 --- a/libavdevice/decklink_enc.cpp +++ b/libavdevice/decklink_enc.cpp @@ -35,6 +35,7 @@ extern "C" { #include "libavcodec/bytestream.h" #include "libavutil/internal.h" #include "libavutil/imgutils.h" +#include "libavutil/mastering_display_metadata.h" #include "avdevice.h" } @@ -47,13 +48,13 @@ extern "C" { #endif /* DeckLink callback class declaration */ -class decklink_frame : public IDeckLinkVideoFrame +class decklink_frame : public IDeckLinkVideoFrame, public IDeckLinkVideoFrameMetadataExtensions { public: decklink_frame(struct decklink_ctx *ctx, AVFrame *avframe, AVCodecID codec_id, int height, int width) : _ctx(ctx), _avframe(avframe), _avpacket(NULL), _codec_id(codec_id), _ancillary(NULL), _height(height), _width(width), _refs(1) { } decklink_frame(struct decklink_ctx *ctx, AVPacket *avpacket, AVCodecID codec_id, int height, int width) : - _ctx(ctx), _avframe(NULL), _avpacket(avpacket), _codec_id(codec_id), _ancillary(NULL), _height(height), _width(width), _refs(1) { } + _ctx(ctx), _avframe(NULL), _avpacket(avpacket), _codec_id(codec_id), _ancillary(NULL), _height(height), _width(width), _colorspace(AVCOL_SPC_BT709), _eotf(AVCOL_TRC_BT709), hdr(NULL), lighting(NULL), _refs(1) { } virtual long STDMETHODCALLTYPE GetWidth (void) { return _width; } virtual long STDMETHODCALLTYPE GetHeight (void) { return _height; } virtual long STDMETHODCALLTYPE GetRowBytes (void) @@ -72,10 +73,14 @@ public: } virtual BMDFrameFlags STDMETHODCALLTYPE GetFlags (void) { - if (_codec_id == AV_CODEC_ID_WRAPPED_AVFRAME) - return _avframe->linesize[0] < 0 ? bmdFrameFlagFlipVertical : bmdFrameFlagDefault; - else - return bmdFrameFlagDefault; + if (_codec_id == AV_CODEC_ID_WRAPPED_AVFRAME) { + return _avframe->linesize[0] < 0 ? bmdFrameFlagFlipVertical : bmdFrameFlagDefault; + } else { + if (_ctx->supports_hdr && (hdr || lighting)) + return bmdFrameFlagDefault | bmdFrameContainsHDRMetadata; + else + return bmdFrameFlagDefault; + } } virtual HRESULT STDMETHODCALLTYPE GetBytes (void **buffer) @@ -110,7 +115,176 @@ public: _ancillary->AddRef(); return S_OK; } - virtual HRESULT STDMETHODCALLTYPE QueryInterface(REFIID iid, LPVOID *ppv) { return E_NOINTERFACE; } + + virtual HRESULT STDMETHODCALLTYPE SetMetadata(enum AVColorSpace colorspace, enum AVColorTransferCharacteristic eotf) + { + _colorspace = colorspace; + _eotf = eotf; + return S_OK; + } + + // IDeckLinkVideoFrameMetadataExtensions interface + virtual HRESULT GetInt(BMDDeckLinkFrameMetadataID metadataID, int64_t* value) + { + HRESULT result = S_OK; + + switch (metadataID) { + case bmdDeckLinkFrameMetadataHDRElectroOpticalTransferFunc: + /* See CTA-861-G Sec 6.9 Dynamic Range and Mastering */ + + switch(_eotf) { + case AVCOL_TRC_SMPTEST2084: + /* PQ */ + *value = 2; + break; + case AVCOL_TRC_ARIB_STD_B67: + /* Also known as "HLG" */ + *value = 3; + break; + case AVCOL_TRC_SMPTE170M: + case AVCOL_TRC_SMPTE240M: + case AVCOL_TRC_BT709: + default: + /* SDR */ + *value = 0; + break; + } + break; + + case bmdDeckLinkFrameMetadataColorspace: + if (!_ctx->supports_colorspace) { + result = E_NOTIMPL; + break; + } + switch(_colorspace) { + case AVCOL_SPC_BT470BG: + case AVCOL_SPC_SMPTE170M: + case AVCOL_SPC_SMPTE240M: + *value = bmdColorspaceRec601; + break; + case AVCOL_SPC_BT2020_CL: + case AVCOL_SPC_BT2020_NCL: + *value = bmdColorspaceRec2020; + break; + case AVCOL_SPC_BT709: + default: + *value = bmdColorspaceRec709; + break; + } + break; + default: + result = E_INVALIDARG; + } + + return result; + } + virtual HRESULT GetFloat(BMDDeckLinkFrameMetadataID metadataID, double* value) + { + *value = 0; + + switch (metadataID) { + case bmdDeckLinkFrameMetadataHDRDisplayPrimariesRedX: + if (hdr && hdr->has_primaries) + *value = av_q2d(hdr->display_primaries[0][0]); + break; + case bmdDeckLinkFrameMetadataHDRDisplayPrimariesRedY: + if (hdr && hdr->has_primaries) + *value = av_q2d(hdr->display_primaries[0][1]); + break; + case bmdDeckLinkFrameMetadataHDRDisplayPrimariesGreenX: + if (hdr && hdr->has_primaries) + *value = av_q2d(hdr->display_primaries[1][0]); + break; + case bmdDeckLinkFrameMetadataHDRDisplayPrimariesGreenY: + if (hdr && hdr->has_primaries) + *value = av_q2d(hdr->display_primaries[1][1]); + break; + case bmdDeckLinkFrameMetadataHDRDisplayPrimariesBlueX: + if (hdr && hdr->has_primaries) + *value = av_q2d(hdr->display_primaries[2][0]); + break; + case bmdDeckLinkFrameMetadataHDRDisplayPrimariesBlueY: + if (hdr && hdr->has_primaries) + *value = av_q2d(hdr->display_primaries[2][1]); + break; + case bmdDeckLinkFrameMetadataHDRWhitePointX: + if (hdr && hdr->has_primaries) + *value = av_q2d(hdr->white_point[0]); + break; + case bmdDeckLinkFrameMetadataHDRWhitePointY: + if (hdr && hdr->has_primaries) + *value = av_q2d(hdr->white_point[1]); + break; + case bmdDeckLinkFrameMetadataHDRMaxDisplayMasteringLuminance: + if (hdr && hdr->has_luminance) + *value = av_q2d(hdr->max_luminance); + break; + case bmdDeckLinkFrameMetadataHDRMinDisplayMasteringLuminance: + if (hdr && hdr->has_luminance) + *value = av_q2d(hdr->min_luminance); + break; + case bmdDeckLinkFrameMetadataHDRMaximumContentLightLevel: + if (lighting) + *value = (float) lighting->MaxCLL; + else + *value = 0; + break; + case bmdDeckLinkFrameMetadataHDRMaximumFrameAverageLightLevel: + if (lighting) + *value = (float) lighting->MaxFALL; + else + *value = 0; + break; + default: + return E_INVALIDARG; + } + + return S_OK; + } + + virtual HRESULT GetFlag(BMDDeckLinkFrameMetadataID metadataID, bool* value) + { + *value = false; + return E_INVALIDARG; + } + virtual HRESULT GetString(BMDDeckLinkFrameMetadataID metadataID, const char** value) + { + *value = nullptr; + return E_INVALIDARG; + } + virtual HRESULT GetBytes(BMDDeckLinkFrameMetadataID metadataID, void* buffer, uint32_t* bufferSize) + { + *bufferSize = 0; + return E_INVALIDARG; + } + + virtual HRESULT STDMETHODCALLTYPE QueryInterface(REFIID iid, LPVOID *ppv) + { + CFUUIDBytes iunknown; + HRESULT result = S_OK; + + if (!ppv) + return E_INVALIDARG; + + *ppv = NULL; + + iunknown = CFUUIDGetUUIDBytes(IUnknownUUID); + if (memcmp(&iid, &iunknown, sizeof(REFIID)) == 0) { + *ppv = this; + AddRef(); + } else if (memcmp(&iid, &IID_IDeckLinkVideoFrame, sizeof(REFIID)) == 0) { + *ppv = static_cast(this); + AddRef(); + } else if (memcmp(&iid, &IID_IDeckLinkVideoFrameMetadataExtensions, sizeof(REFIID)) == 0) { + *ppv = static_cast(this); + AddRef(); + } else { + result = E_NOINTERFACE; + } + + return result; + } + virtual ULONG STDMETHODCALLTYPE AddRef(void) { return ++_refs; } virtual ULONG STDMETHODCALLTYPE Release(void) { @@ -132,6 +306,10 @@ public: IDeckLinkVideoFrameAncillary *_ancillary; int _height; int _width; + enum AVColorSpace _colorspace; + enum AVColorTransferCharacteristic _eotf; + const AVMasteringDisplayMetadata *hdr; + const AVContentLightMetadata *lighting; private: std::atomic _refs; @@ -726,6 +904,18 @@ static int decklink_write_video_packet(AVFormatContext *avctx, AVPacket *pkt) return AVERROR(EIO); } + /* Set frame metadata properties */ + size_t size; + const AVMasteringDisplayMetadata *hdr = (const AVMasteringDisplayMetadata *) av_packet_get_side_data(pkt, AV_PKT_DATA_MASTERING_DISPLAY_METADATA, &size); + if (hdr && size > 0) + frame->hdr = hdr; + + const AVContentLightMetadata *lighting = (const AVContentLightMetadata *) av_packet_get_side_data(pkt, AV_PKT_DATA_CONTENT_LIGHT_LEVEL, &size); + if (hdr && size > 0) + frame->lighting = lighting; + + frame->SetMetadata(st->codecpar->color_space, st->codecpar->color_trc); + /* Always keep at most one second of frames buffered. */ pthread_mutex_lock(&ctx->mutex); while (ctx->frames_buffer_available_spots == 0) { -- 1.8.3.1 _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".