From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTP id 8E8BB454D5 for ; Fri, 26 Apr 2024 19:29:18 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id EBF0E68D3C6; Fri, 26 Apr 2024 22:29:15 +0300 (EEST) Received: from mail-pl1-f172.google.com (mail-pl1-f172.google.com [209.85.214.172]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 1F69868D281 for ; Fri, 26 Apr 2024 22:29:09 +0300 (EEST) Received: by mail-pl1-f172.google.com with SMTP id d9443c01a7336-1e5715a9ebdso21793025ad.2 for ; Fri, 26 Apr 2024 12:29:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1714159746; x=1714764546; darn=ffmpeg.org; h=content-transfer-encoding:in-reply-to:from:content-language :references:to:subject:user-agent:mime-version:date:message-id:from :to:cc:subject:date:message-id:reply-to; bh=15tRcRofs+WDvqgWbGQ0KILaRatMq7/Unf63cdIVftY=; b=Lls3W3PGtpZRSbWQHp92tfFjizQf5+HUmhYRYdJl2jUkTWfz5l8APB/PNL66YPBJUp EIEH6vYtZxEn0aVGaJpAdFn8y2IZZlorF7ej4GbR8DnLi/U7gnRtDcmMbV98BKxjqbMJ h1llpvHtyjHKrZhZS6jLMe+5IM76aKFnjxkjS8aLpJY//w3/5THtDBaK2+WcVn5LkwZy 9rKnVFlb2cYcuZSajLVK4CvIECXMagtiHPLY0KXuUZElg1yiWYSey0mfGBn/LbXcrIb8 S3K6ywDKjjgXJnrEz6+UryDUX8KPK+r/6SMpSoQj5g10Y5coZWx+XA2aW8HjRABlXPNx sZgg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1714159746; x=1714764546; h=content-transfer-encoding:in-reply-to:from:content-language :references:to:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=15tRcRofs+WDvqgWbGQ0KILaRatMq7/Unf63cdIVftY=; b=cHEoFk7lSN8hFuufiAJkCVNKF/rWttBe1fj4uB3Z3/p68MAUd2nkdBePufoIDpQ7x/ HwlS+S56+4+6rMgintE+TDKTzlwO3w+8WunVUeFbqqe/+1bzKJILMUoRpj1j4Xv+5DoJ f6EDpsQhxcnQEAa/TKN3+5JUl9N31LZiHzbjdsjevxwCMyEPaPSDEw7YASgJP1knYEGx rPLp/ebO7a3u2jUM+ha0zz+YF8bTXpQC08u8/6HAmZfCq0DCZQ55RJSjzUBtWeyWEHzO BfA7SDDgHBJ/YptTCdjtUFfLDogC6+wU/3kiLimFkUkmiwkdQPhp/9H6aPnKvDS1JbpC loSQ== X-Gm-Message-State: AOJu0YyPTEnIDbCUDIw5Wq26ruDJ/cKVofIWwfktcrsQPXn8HhBC7hJb 2TE98pkKBAdIZKroiEHxIikPgthikRvwwvJbaL6TYl9IJSFQPSS3ROrKwg== X-Google-Smtp-Source: AGHT+IGAP9vaQI9iRyxqhOhV8ongCE4D+/Uf9Yy3n7FqXeip1EDtzOIe+VLmBrPZ2LLeuZo/jzeIwQ== X-Received: by 2002:a17:902:c412:b0:1e0:b62a:c0a2 with SMTP id k18-20020a170902c41200b001e0b62ac0a2mr686201plk.51.1714159745725; Fri, 26 Apr 2024 12:29:05 -0700 (PDT) Received: from [192.168.0.10] ([190.194.167.233]) by smtp.gmail.com with ESMTPSA id e8-20020a17090301c800b001e0f5034e95sm15774488plh.288.2024.04.26.12.29.04 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 26 Apr 2024 12:29:05 -0700 (PDT) Message-ID: <2d4bc05a-ff4e-4174-855e-65ef84a388f2@gmail.com> Date: Fri, 26 Apr 2024 16:29:03 -0300 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird To: ffmpeg-devel@ffmpeg.org References: <20240426122803.19967-1-ffmpeg@haasn.xyz> Content-Language: en-US From: James Almer In-Reply-To: <20240426122803.19967-1-ffmpeg@haasn.xyz> Subject: Re: [FFmpeg-devel] [PATCH 1/6] avutil/frame: add av_frame_remove_side_data_changed X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: On 4/26/2024 9:27 AM, Niklas Haas wrote: > From: Niklas Haas > > Many filters modify certain aspects of frame data, e.g. through resizing > (vf_*scale* family), color volume mapping (vf_lut*, vf_tonemap*), or > possibly others. > > When this happens, we should strip all frame side data that will no > longer be correct/relevant after the operation. For example, changing > the image size should invalidate AV_FRAME_DATA_PANSCAN because the crop > window (given in pixels) no longer corresponds to the actual image size. > For another example, tone-mapping filters (e.g. from HDR to SDR) should > strip all of the dynamic HDR related metadata. > > Since there are a lot of similar with basically similar operations, it > make sense to consolidate this stripping logic into a common helper > function. I decided to put it into libavutil as it may be useful for API > users as well, who often have their own internal processing and > filtering. Maybe instead of "changed", which is a concept that doesn't belong to a frame but to a process, use a name that references side data that depends on specific properties (dimensions, color props, etc). Also, don't make it depend on AVFrame, but AVFrameSideData instead, so it can be reused in other contexts (Use the av_frame_side_data_* namespace for it). > --- > doc/APIchanges | 3 +++ > libavutil/frame.c | 31 +++++++++++++++++++++++++++++++ > libavutil/frame.h | 14 ++++++++++++++ > libavutil/version.h | 2 +- > 4 files changed, 49 insertions(+), 1 deletion(-) > > diff --git a/doc/APIchanges b/doc/APIchanges > index 0566fcdcc5..26af725528 100644 > --- a/doc/APIchanges > +++ b/doc/APIchanges > @@ -2,6 +2,9 @@ The last version increases of all libraries were on 2024-03-07 > > API changes, most recent first: > > +2024-04-xx - xxxxxxxxxx - lavu 59.17.100 - frame.h > + Add av_frame_remove_side_data_changed(). > + > 2024-04-24 - 8616cfe0890 - lavu 59.16.100 - opt.h > Add AV_OPT_SERIALIZE_SEARCH_CHILDREN. > > diff --git a/libavutil/frame.c b/libavutil/frame.c > index 0775e2abd9..d5482c258e 100644 > --- a/libavutil/frame.c > +++ b/libavutil/frame.c > @@ -1015,6 +1015,37 @@ void av_frame_remove_side_data(AVFrame *frame, enum AVFrameSideDataType type) > remove_side_data(&frame->side_data, &frame->nb_side_data, type); > } > > +static const struct { > + enum AVFrameSideDataType type; > + int aspect; > +} side_data_aspects[] = { > + { AV_FRAME_DATA_PANSCAN, AV_FRAME_CHANGED_SIZE }, > + { AV_FRAME_DATA_MOTION_VECTORS, AV_FRAME_CHANGED_SIZE }, > + { AV_FRAME_DATA_MASTERING_DISPLAY_METADATA, AV_FRAME_CHANGED_COLOR_VOLUME }, > + > + { AV_FRAME_DATA_SPHERICAL, AV_FRAME_CHANGED_SIZE }, > + { AV_FRAME_DATA_CONTENT_LIGHT_LEVEL, AV_FRAME_CHANGED_COLOR_VOLUME }, > + { AV_FRAME_DATA_ICC_PROFILE, AV_FRAME_CHANGED_COLOR_VOLUME }, > + { AV_FRAME_DATA_DYNAMIC_HDR_PLUS, AV_FRAME_CHANGED_COLOR_VOLUME }, > + { AV_FRAME_DATA_REGIONS_OF_INTEREST, AV_FRAME_CHANGED_SIZE }, > + { AV_FRAME_DATA_DETECTION_BBOXES, AV_FRAME_CHANGED_SIZE }, > + { AV_FRAME_DATA_DOVI_RPU_BUFFER, AV_FRAME_CHANGED_COLOR_VOLUME }, > + { AV_FRAME_DATA_DOVI_METADATA, AV_FRAME_CHANGED_COLOR_VOLUME }, > + { AV_FRAME_DATA_DYNAMIC_HDR_VIVID, AV_FRAME_CHANGED_COLOR_VOLUME }, > + { AV_FRAME_DATA_VIDEO_HINT, AV_FRAME_CHANGED_SIZE }, > +}; > + > +void av_frame_remove_side_data_changed(AVFrame *frame, int changed_aspects) > +{ > + if (!changed_aspects) > + return; > + > + for (int i = 0; i < FF_ARRAY_ELEMS(side_data_aspects); i++) { > + if (changed_aspects & side_data_aspects[i].aspect) > + av_frame_remove_side_data(frame, side_data_aspects[i].type); > + } > +} > + > const AVSideDataDescriptor *av_frame_side_data_desc(enum AVFrameSideDataType type) > { > unsigned t = type; > diff --git a/libavutil/frame.h b/libavutil/frame.h > index 60bb966f8b..7e07ecf91f 100644 > --- a/libavutil/frame.h > +++ b/libavutil/frame.h > @@ -983,6 +983,20 @@ AVFrameSideData *av_frame_get_side_data(const AVFrame *frame, > */ > void av_frame_remove_side_data(AVFrame *frame, enum AVFrameSideDataType type); > > +/** > + * Flags for stripping side data based on changed aspects. > + */ > +enum { > + /* Video only */ > + AV_FRAME_CHANGED_SIZE = 1 << 0, ///< changed dimensions / crop > + AV_FRAME_CHANGED_COLOR_VOLUME = 1 << 1, ///< changed color volume > +}; > + > +/** > + * Remove all relevant side data after a corresponding change to the frame > + * data, based on the given bitset of frame aspects. > + */ > +void av_frame_remove_side_data_changed(AVFrame *frame, int changed_aspects); > > /** > * Flags for frame cropping. > diff --git a/libavutil/version.h b/libavutil/version.h > index ea289c406f..3b5a2e7aaa 100644 > --- a/libavutil/version.h > +++ b/libavutil/version.h > @@ -79,7 +79,7 @@ > */ > > #define LIBAVUTIL_VERSION_MAJOR 59 > -#define LIBAVUTIL_VERSION_MINOR 16 > +#define LIBAVUTIL_VERSION_MINOR 17 > #define LIBAVUTIL_VERSION_MICRO 100 > > #define LIBAVUTIL_VERSION_INT AV_VERSION_INT(LIBAVUTIL_VERSION_MAJOR, \ _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".