From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by master.gitmailbox.com (Postfix) with ESMTP id E1C32469EB for ; Wed, 2 Aug 2023 05:36:54 +0000 (UTC) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id C99D068C59A; Wed, 2 Aug 2023 08:36:51 +0300 (EEST) Received: from mail-ed1-f48.google.com (mail-ed1-f48.google.com [209.85.208.48]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 826F368C3A7 for ; Wed, 2 Aug 2023 08:36:45 +0300 (EEST) Received: by mail-ed1-f48.google.com with SMTP id 4fb4d7f45d1cf-52256241b76so8352357a12.1 for ; Tue, 01 Aug 2023 22:36:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1690954604; x=1691559404; h=user-agent:in-reply-to:content-transfer-encoding :content-disposition:mime-version:references:mail-followup-to :message-id:subject:to:from:date:from:to:cc:subject:date:message-id :reply-to; bh=D+yDXSINPbvOr3Afk85zLsXRK7k7D3MqcVWjKj3f4eI=; b=r6NmLzc1/tYMJ1Z884cXBBYtF/d+4gVTss2zw85V4tSuAvgO8jGtlut01hP3GprSmw e5tNILLD55ddrde3pBK94gx3TwoiAUXyKW5hFOmqn7F+UBMYk3ibx3VmWHEWRHdgFK0O F0nyu3cAGIdX8NMrfwStzBzPZvSggQ1lbZozJXbS+hjiTw25xpdYzd0KFJPSJJShN3rY aZLsDc+hGyZFvC94f4FM6tlik/Qw2fwrak84nMwO/R/hhgEoYHqDEv17VAyVTkbXjPF5 hp4RR7iwsw+ConJZ4F2C2Tp+KgVpCWwVnmCie0+NY7avJ6vP5Rd1hpIRhw2oJSlK0/bI IY2A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1690954604; x=1691559404; h=user-agent:in-reply-to:content-transfer-encoding :content-disposition:mime-version:references:mail-followup-to :message-id:subject:to:from:date:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=D+yDXSINPbvOr3Afk85zLsXRK7k7D3MqcVWjKj3f4eI=; b=LbZihyVY/lU6oZ/j+hhWJlmAvA6oYeoqw3JXu4qOr1F6itBkMuv90mCU93l5RQh2fw cygmrnSy/uAMdDWKRhIXI2f2z+nGg/nvREXCJKzeimyOy84gqo4Ds4DznU+wJkahp6rK j683TdYE/s/6LYPvrMzD5C5YHxh8ShvXBfB3+uN56VQpudUS4hDh4LcZpSbozJuKmm0D nkytHVCx8g9jU/OyqZ2mhmM3Bns/UhFYkf4ULWfdah1EUsNt62u7OHBUsWw9BzT9wy2l JRLsPDDTaQYhqWB9mV13fn5OdGAvH0t2x/XoZcXNEKSpZpqL/ZGHeds2wQ88rSZcdFYa vXKQ== X-Gm-Message-State: ABy/qLbrnMA1AIyKC+ytln6YOx7P5WcUhtUR2AeclL1UkD4jJETSRLg8 ty6y8va7OawJ5KJFqVS+rZE1ISlnHzM= X-Google-Smtp-Source: APBJJlF6nISBLlfoXNrg8iJ8mHdSHxiWZKHR7ZzVqzTWalGSLSpb4g1HGOHQmEosw+9MEwwYHStRuQ== X-Received: by 2002:a05:6402:1659:b0:522:cb97:f196 with SMTP id s25-20020a056402165900b00522cb97f196mr4186486edx.36.1690954603921; Tue, 01 Aug 2023 22:36:43 -0700 (PDT) Received: from mariano ([82.84.192.32]) by smtp.gmail.com with ESMTPSA id n17-20020aa7c691000000b0051de018af1esm7935209edq.59.2023.08.01.22.36.43 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 01 Aug 2023 22:36:43 -0700 (PDT) Received: by mariano (Postfix, from userid 1000) id 1AD5EBFB73; Wed, 2 Aug 2023 07:36:43 +0200 (CEST) Date: Wed, 2 Aug 2023 07:36:43 +0200 From: Stefano Sabatini To: FFmpeg development discussions and patches Message-ID: <20230802053643.GG10927@mariano> Mail-Followup-To: FFmpeg development discussions and patches References: <20230709110529.29490-1-anton@khirnov.net> <2ef05e377ecdcb086d2ec00a0c8d810ff962ca77.camel@amazon.it> <20230717221940.GC10400@mariano> <20230723232727.GB10927@mariano> <20230728074437.GC10927@mariano> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20230728074437.GC10927@mariano> User-Agent: Mutt/1.13.2 (2019-12-18) Subject: Re: [FFmpeg-devel] [PATCH] lavu: add AVVideoHint API X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Archived-At: List-Archive: List-Post: On date Friday 2023-07-28 09:44:37 +0200, Stefano Sabatini wrote: > On date Wednesday 2023-07-26 10:52:57 +0000, Carotti, Elias wrote: > > On Mon, 2023-07-24 at 01:27 +0200, Stefano Sabatini wrote: > > > CAUTION: This email originated from outside of the organization. Do > > > not click links or open attachments unless you can confirm the sender > > > and know the content is safe. > > > = > > > = > > > > = > > > > = > > > > sorry to nitpick, but if we have this information in the single > > > > rect > > > > should not we use that instead? > > > > = > > > > Basically I see two options: > > > > 1. generic VideoHint, storing single hint type and rects > > > > = > > > > for example we might have type=3Dchanged, and the rects storing only > > > > the > > > > changed rects > > > > = > > > > If we want to extend this then we need to add a new type. The > > > > problem > > > > might be if we want to store more than one VideoHint in the side > > > > data > > > > (it it possible to store more than one AVVideoHint, each one with a > > > > different type, as side data?). Suppose for example we want to add > > > > a > > > > new hint type, e.g. ROI, then we can add a new AVHideoHint with > > > > type=3Droi - but I don't know if we can have several frame hint side > > > > data (each one with different individual types). > > > > = > > > > 2. generic VideoHint, storing rects each one containing the hint > > > > type > > > > = > > > > for example we might have a list of rects, each one with > > > > type=3Dchanged|constant|roi. This would allow one to store different > > > > kind of hints in the same AVVideoHint structure, but at the same > > > > time > > > > would enable inconsistent data (for example we might have changed > > > > and > > > > unchanged rects in the same list of rects) > > > > = > > = > = > > I don't think it allowed such inconsistency the way it was made. > > Basically, the semantics was that the Hint type could be either > > CONSTANT or CHANGED and that was the basis to interpret the rects with > > type DAMAGE. = > = > Yes, but then there is no guarantee that you'll have all rects with > the same subtype, and you need to iterate and check each one to > check the subtype (so you might have mixed constant/changed and > unclear behavior to apply in this case). > = > > > > In each case, storing the type in the generic AVVideHint and in > > > > each > > > > rect might lead to inconsistent states (e.g. you might have generic > > > > type=3Ddamage, and have type=3Droi and type=3Dchanged in the rects)= , in > > > > this > > > > case having the type in both the general structure and in each > > > > rects > > > > seems redundant and leads to possible data inconsistencies. > > > > = > > > > ... > > > > = > > > = > > > > I would rather go to solution 1, but I'm not sure if having more > > > > than > > > > one hint in the side data (with different subtypes) is possible and > > > > wanted. If this is the case, probably the best solution would be to > > > > store more than one AVVideoHint (each one with an individual type > > > > and > > > > list of rects) in the single side data object. > > > = > > > Elaborating on this. > > > = > > > 1. Ideally we might want to extend the av_frame_get_side_data API to > > > be able to store more than one entry per side data, by adding a > > > unique > > > label to the side data, but this requires probably more effort and > > > extends the scope even further. > > > = > > > 2. Alternatively, we might extend the hint API like this: > > > = > > > AVVideoHint *av_video_hint_create_side_data(AVFrame *frame, > > > =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 size_t nb_rect= s); > > > = > > > AVVideoHint *av_video_hint_extend_side_data(AVFrame *frame, > > > =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 size_t nb_rect= s); > > > = > > > by marking the continuation on the previous hint (but this would > > > complicate deserialization as you need then to iterate to get all the > > > side data). > > > = > > > Or we could make a variadic function like this: > > > AVVideoHint *av_video_hint_create_side_data(AVFrame *frame, size_t > > > nb_rects, ...); > > > = > > > Alternatively, we might store in AVVideoHint more than one hint, but > > > this somehow conflict with the general design. > > > = > > > ... > > > = > > > As a low effort approach, I would keep the initial design of a single > > > hint type per side-data (dropping the possibility of having more than > > > one hint type in the same side data), which should be fixed > > > generically by implementing 1. > > = > > Agreed. > > For clarity I am attaching again the patch with Anton's changes just > > rebased on the current master branch. > > Anton, is this still ok to be merged? > = > I'm fine with the current patch, the behavior can then be extended > by extending side data with multiple-label for the same type. > = > Anton, if you are fine with it I would merge this one (and maybe get > included in the next release if we are not too late already). I'll push this in a few days unless there are more comments. _______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".