* [FFmpeg-devel] [PATCH v2 01/15] avfilter/formats: document ff_default_query_formats
2023-12-13 13:11 [FFmpeg-devel] [PATCH v2 00/15] YUV colorspace filter negotiation Niklas Haas
@ 2023-12-13 13:11 ` Niklas Haas
2023-12-13 13:11 ` [FFmpeg-devel] [PATCH v2 02/15] avfilter: always call ff_default_query_formats Niklas Haas
` (14 subsequent siblings)
15 siblings, 0 replies; 26+ messages in thread
From: Niklas Haas @ 2023-12-13 13:11 UTC (permalink / raw)
To: ffmpeg-devel; +Cc: Niklas Haas
From: Niklas Haas <git@haasn.dev>
In particular, make it clear that this function will not touch format
lists which were already set by the caller before calling into this
function.
---
libavfilter/formats.h | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/libavfilter/formats.h b/libavfilter/formats.h
index 22224dce2d..d44890109e 100644
--- a/libavfilter/formats.h
+++ b/libavfilter/formats.h
@@ -198,6 +198,10 @@ void ff_channel_layouts_unref(AVFilterChannelLayouts **ref);
void ff_channel_layouts_changeref(AVFilterChannelLayouts **oldref,
AVFilterChannelLayouts **newref);
+/**
+ * Sets all remaining unset filter lists for all inputs/outputs to their
+ * corresponding `ff_all_*()` lists.
+ */
av_warn_unused_result
int ff_default_query_formats(AVFilterContext *ctx);
--
2.43.0
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
^ permalink raw reply [flat|nested] 26+ messages in thread* [FFmpeg-devel] [PATCH v2 02/15] avfilter: always call ff_default_query_formats
2023-12-13 13:11 [FFmpeg-devel] [PATCH v2 00/15] YUV colorspace filter negotiation Niklas Haas
2023-12-13 13:11 ` [FFmpeg-devel] [PATCH v2 01/15] avfilter/formats: document ff_default_query_formats Niklas Haas
@ 2023-12-13 13:11 ` Niklas Haas
2023-12-13 13:12 ` [FFmpeg-devel] [PATCH v2 03/15] avfilter: add negotiation API for color space/range Niklas Haas
` (13 subsequent siblings)
15 siblings, 0 replies; 26+ messages in thread
From: Niklas Haas @ 2023-12-13 13:11 UTC (permalink / raw)
To: ffmpeg-devel; +Cc: Niklas Haas
From: Niklas Haas <git@haasn.dev>
Even if a query func is set. This is safe to do, because
ff_default_query_formats is documented not to touch any filter lists
that were already set by the query func.
The reason to do this is because it allows us to extend
AVFilterFormatsConfig without having to touch every filter in existence.
An alternative implementation of this commit would be to explicitly add
a `ff_default_query_formats` call at the end of every query_formats
function, but that would end up functionally equivalent to this change
while touching a whole lot more code paths for no reason.
As a bonus, eliminates some code/logic duplication from this function.
---
libavfilter/avfiltergraph.c | 39 ++++++++++++-------------------------
1 file changed, 12 insertions(+), 27 deletions(-)
diff --git a/libavfilter/avfiltergraph.c b/libavfilter/avfiltergraph.c
index 68daa93e61..625cbc022e 100644
--- a/libavfilter/avfiltergraph.c
+++ b/libavfilter/avfiltergraph.c
@@ -341,33 +341,21 @@ static int filter_check_formats(AVFilterContext *ctx)
static int filter_query_formats(AVFilterContext *ctx)
{
int ret;
- AVFilterFormats *formats;
- AVFilterChannelLayouts *chlayouts;
- enum AVMediaType type = ctx->inputs && ctx->inputs [0] ? ctx->inputs [0]->type :
- ctx->outputs && ctx->outputs[0] ? ctx->outputs[0]->type :
- AVMEDIA_TYPE_VIDEO;
-
- if ((ret = ctx->filter->formats.query_func(ctx)) < 0) {
- if (ret != AVERROR(EAGAIN))
- av_log(ctx, AV_LOG_ERROR, "Query format failed for '%s': %s\n",
- ctx->name, av_err2str(ret));
- return ret;
- }
- ret = filter_check_formats(ctx);
- if (ret < 0)
- return ret;
- formats = ff_all_formats(type);
- if ((ret = ff_set_common_formats(ctx, formats)) < 0)
- return ret;
- if (type == AVMEDIA_TYPE_AUDIO) {
- if ((ret = ff_set_common_all_samplerates(ctx)) < 0)
+ if (ctx->filter->formats_state == FF_FILTER_FORMATS_QUERY_FUNC) {
+ if ((ret = ctx->filter->formats.query_func(ctx)) < 0) {
+ if (ret != AVERROR(EAGAIN))
+ av_log(ctx, AV_LOG_ERROR, "Query format failed for '%s': %s\n",
+ ctx->name, av_err2str(ret));
return ret;
- chlayouts = ff_all_channel_layouts();
- if ((ret = ff_set_common_channel_layouts(ctx, chlayouts)) < 0)
+ }
+
+ ret = filter_check_formats(ctx);
+ if (ret < 0)
return ret;
}
- return 0;
+
+ return ff_default_query_formats(ctx);
}
static int formats_declared(AVFilterContext *f)
@@ -416,10 +404,7 @@ static int query_formats(AVFilterGraph *graph, void *log_ctx)
AVFilterContext *f = graph->filters[i];
if (formats_declared(f))
continue;
- if (f->filter->formats_state == FF_FILTER_FORMATS_QUERY_FUNC)
- ret = filter_query_formats(f);
- else
- ret = ff_default_query_formats(f);
+ ret = filter_query_formats(f);
if (ret < 0 && ret != AVERROR(EAGAIN))
return ret;
/* note: EAGAIN could indicate a partial success, not counted yet */
--
2.43.0
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
^ permalink raw reply [flat|nested] 26+ messages in thread* [FFmpeg-devel] [PATCH v2 03/15] avfilter: add negotiation API for color space/range
2023-12-13 13:11 [FFmpeg-devel] [PATCH v2 00/15] YUV colorspace filter negotiation Niklas Haas
2023-12-13 13:11 ` [FFmpeg-devel] [PATCH v2 01/15] avfilter/formats: document ff_default_query_formats Niklas Haas
2023-12-13 13:11 ` [FFmpeg-devel] [PATCH v2 02/15] avfilter: always call ff_default_query_formats Niklas Haas
@ 2023-12-13 13:12 ` Niklas Haas
2023-12-14 3:09 ` Michael Niedermayer
2024-01-03 3:04 ` [FFmpeg-devel] [PATCH v2 03/15] avfilter: add negotiation API for color space/range Xiang, Haihao
2023-12-13 13:12 ` [FFmpeg-devel] [PATCH v2 04/15] fftools/ffmpeg_filter: don't clear buffersrc params Niklas Haas
` (12 subsequent siblings)
15 siblings, 2 replies; 26+ messages in thread
From: Niklas Haas @ 2023-12-13 13:12 UTC (permalink / raw)
To: ffmpeg-devel; +Cc: Niklas Haas
From: Niklas Haas <git@haasn.dev>
Motivated by YUVJ removal. This change will allow full negotiation
between color ranges and matrices as needed. By default, all ranges and
matrices are marked as supported.
Because grayscale formats are currently handled very inconsistently (and
in particular, assumed as forced full-range by swscale), we exclude them
from negotiation altogether for the time being, to get this API merged.
After filter negotiation is available, we can relax the
grayscale-is-forced-jpeg restriction again, when it will be more
feasible to do so without breaking a million test cases.
Note that this commit updates one FATE test as a consequence of the
sanity fallback for non-YUV formats. In particular, the test case now
writes rgb24(pc, gbr/unspecified/unspecified) to the matroska file,
instead of rgb24(unspecified/unspecified/unspecified) as before.
---
doc/APIchanges | 3 +
libavfilter/avfilter.c | 17 +++-
libavfilter/avfilter.h | 28 ++++++
libavfilter/avfiltergraph.c | 173 +++++++++++++++++++++++++++++++++++-
libavfilter/formats.c | 122 ++++++++++++++++++++++++-
libavfilter/formats.h | 54 +++++++++++
libavfilter/internal.h | 6 ++
libavfilter/vaapi_vpp.c | 4 +
libavfilter/video.c | 2 +
tests/ref/fate/shortest-sub | 4 +-
10 files changed, 404 insertions(+), 9 deletions(-)
diff --git a/doc/APIchanges b/doc/APIchanges
index 4a2dc1c44f..2f6ea50f63 100644
--- a/doc/APIchanges
+++ b/doc/APIchanges
@@ -2,6 +2,9 @@ The last version increases of all libraries were on 2023-02-09
API changes, most recent first:
+2023-11-xx - xxxxxxxxxx - lavf 58.14.100 - avfilter.h
+ Add AVFilterLink.colorspace, AVFilterLink.color_range
+
2023-11-08 - b82957a66a7 - lavu 58.32.100 - channel_layout.h
Add AV_CH_LAYOUT_7POINT2POINT3 and AV_CHANNEL_LAYOUT_7POINT2POINT3.
Add AV_CH_LAYOUT_9POINT1POINT4_BACK and AV_CHANNEL_LAYOUT_9POINT1POINT4_BACK.
diff --git a/libavfilter/avfilter.c b/libavfilter/avfilter.c
index bde1c33d07..31300bb515 100644
--- a/libavfilter/avfilter.c
+++ b/libavfilter/avfilter.c
@@ -185,6 +185,7 @@ int avfilter_link(AVFilterContext *src, unsigned srcpad,
link->type = src->output_pads[srcpad].type;
av_assert0(AV_PIX_FMT_NONE == -1 && AV_SAMPLE_FMT_NONE == -1);
link->format = -1;
+ link->colorspace = AVCOL_SPC_UNSPECIFIED;
ff_framequeue_init(&link->fifo, &src->graph->internal->frame_queues);
return 0;
@@ -286,6 +287,12 @@ int avfilter_insert_filter(AVFilterLink *link, AVFilterContext *filt,
if (link->outcfg.formats)
ff_formats_changeref(&link->outcfg.formats,
&filt->outputs[filt_dstpad_idx]->outcfg.formats);
+ if (link->outcfg.color_spaces)
+ ff_formats_changeref(&link->outcfg.color_spaces,
+ &filt->outputs[filt_dstpad_idx]->outcfg.color_spaces);
+ if (link->outcfg.color_ranges)
+ ff_formats_changeref(&link->outcfg.color_ranges,
+ &filt->outputs[filt_dstpad_idx]->outcfg.color_ranges);
if (link->outcfg.samplerates)
ff_formats_changeref(&link->outcfg.samplerates,
&filt->outputs[filt_dstpad_idx]->outcfg.samplerates);
@@ -730,6 +737,10 @@ static void free_link(AVFilterLink *link)
ff_formats_unref(&link->incfg.formats);
ff_formats_unref(&link->outcfg.formats);
+ ff_formats_unref(&link->incfg.color_spaces);
+ ff_formats_unref(&link->outcfg.color_spaces);
+ ff_formats_unref(&link->incfg.color_ranges);
+ ff_formats_unref(&link->outcfg.color_ranges);
ff_formats_unref(&link->incfg.samplerates);
ff_formats_unref(&link->outcfg.samplerates);
ff_channel_layouts_unref(&link->incfg.channel_layouts);
@@ -987,9 +998,9 @@ int ff_filter_frame(AVFilterLink *link, AVFrame *frame)
strcmp(link->dst->filter->name, "idet") &&
strcmp(link->dst->filter->name, "null") &&
strcmp(link->dst->filter->name, "scale")) {
- av_assert1(frame->format == link->format);
- av_assert1(frame->width == link->w);
- av_assert1(frame->height == link->h);
+ av_assert1(frame->format == link->format);
+ av_assert1(frame->width == link->w);
+ av_assert1(frame->height == link->h);
}
frame->sample_aspect_ratio = link->sample_aspect_ratio;
diff --git a/libavfilter/avfilter.h b/libavfilter/avfilter.h
index d69381aed4..246d000251 100644
--- a/libavfilter/avfilter.h
+++ b/libavfilter/avfilter.h
@@ -301,6 +301,14 @@ typedef struct AVFilter {
* @ref AVFilterFormatsConfig.formats "incfg.formats"
* on every output link to a list of pixel/sample formats that the filter
* supports on that link.
+ * For video links, this filter may also set
+ * @ref AVFilterFormatsConfig.color_spaces "incfg.color_spaces"
+ * /
+ * @ref AVFilterFormatsConfig.color_spaces "outcfg.color_spaces"
+ * and @ref AVFilterFormatsConfig.color_ranges "incfg.color_ranges"
+ * /
+ * @ref AVFilterFormatsConfig.color_ranges "outcfg.color_ranges"
+ * analogously.
* For audio links, this filter must also set
* @ref AVFilterFormatsConfig.samplerates "incfg.samplerates"
* /
@@ -322,6 +330,10 @@ typedef struct AVFilter {
* to indicate that this filter supports each of these pixel formats,
* provided that all inputs and outputs use the same pixel format.
*
+ * In addition to that the generic code will mark all inputs
+ * and all outputs as supporting all color spaces and ranges, as
+ * long as all inputs and outputs use the same color space/range.
+ *
* This list must never be NULL if the union is in this state.
* The type of all inputs and outputs of filters using this must
* be AVMEDIA_TYPE_VIDEO.
@@ -514,6 +526,12 @@ typedef struct AVFilterFormatsConfig {
*/
AVFilterChannelLayouts *channel_layouts;
+ /**
+ * Lists of supported YUV color metadata, only for YUV video.
+ */
+ AVFilterFormats *color_spaces; ///< AVColorSpace
+ AVFilterFormats *color_ranges; ///< AVColorRange
+
} AVFilterFormatsConfig;
/**
@@ -565,6 +583,16 @@ struct AVFilterLink {
AVChannelLayout ch_layout; ///< channel layout of current buffer (see libavutil/channel_layout.h)
+ /**
+ * For non-YUV links, these are respectively set to fallback values (as
+ * appropriate for that colorspace).
+ *
+ * Note: This includes grayscale formats, as these are currently treated
+ * as forced full range always.
+ */
+ enum AVColorSpace colorspace; ///< agreed upon YUV color space
+ enum AVColorRange color_range; ///< agreed upon YUV color range
+
/*****************************************************************
* All fields below this line are not part of the public API. They
* may not be used outside of libavfilter and can be changed and
diff --git a/libavfilter/avfiltergraph.c b/libavfilter/avfiltergraph.c
index 625cbc022e..ef739735bd 100644
--- a/libavfilter/avfiltergraph.c
+++ b/libavfilter/avfiltergraph.c
@@ -27,6 +27,7 @@
#include "libavutil/avassert.h"
#include "libavutil/bprint.h"
#include "libavutil/channel_layout.h"
+#include "libavutil/hwcontext.h"
#include "libavutil/imgutils.h"
#include "libavutil/opt.h"
#include "libavutil/pixdesc.h"
@@ -298,7 +299,9 @@ static int filter_link_check_formats(void *log, AVFilterLink *link, AVFilterForm
switch (link->type) {
case AVMEDIA_TYPE_VIDEO:
- if ((ret = ff_formats_check_pixel_formats(log, cfg->formats)) < 0)
+ if ((ret = ff_formats_check_pixel_formats(log, cfg->formats)) < 0 ||
+ (ret = ff_formats_check_color_spaces(log, cfg->color_spaces)) < 0 ||
+ (ret = ff_formats_check_color_ranges(log, cfg->color_ranges)) < 0)
return ret;
break;
@@ -365,6 +368,10 @@ static int formats_declared(AVFilterContext *f)
for (i = 0; i < f->nb_inputs; i++) {
if (!f->inputs[i]->outcfg.formats)
return 0;
+ if (f->inputs[i]->type == AVMEDIA_TYPE_VIDEO &&
+ !(f->inputs[i]->outcfg.color_ranges &&
+ f->inputs[i]->outcfg.color_spaces))
+ return 0;
if (f->inputs[i]->type == AVMEDIA_TYPE_AUDIO &&
!(f->inputs[i]->outcfg.samplerates &&
f->inputs[i]->outcfg.channel_layouts))
@@ -373,6 +380,10 @@ static int formats_declared(AVFilterContext *f)
for (i = 0; i < f->nb_outputs; i++) {
if (!f->outputs[i]->incfg.formats)
return 0;
+ if (f->outputs[i]->type == AVMEDIA_TYPE_VIDEO &&
+ !(f->outputs[i]->incfg.color_ranges &&
+ f->outputs[i]->incfg.color_spaces))
+ return 0;
if (f->outputs[i]->type == AVMEDIA_TYPE_AUDIO &&
!(f->outputs[i]->incfg.samplerates &&
f->outputs[i]->incfg.channel_layouts))
@@ -493,7 +504,16 @@ static int query_formats(AVFilterGraph *graph, void *log_ctx)
av_assert0( inlink->outcfg.formats->refcount > 0);
av_assert0(outlink->incfg.formats->refcount > 0);
av_assert0(outlink->outcfg.formats->refcount > 0);
- if (outlink->type == AVMEDIA_TYPE_AUDIO) {
+ if (outlink->type == AVMEDIA_TYPE_VIDEO) {
+ av_assert0( inlink-> incfg.color_spaces->refcount > 0);
+ av_assert0( inlink->outcfg.color_spaces->refcount > 0);
+ av_assert0(outlink-> incfg.color_spaces->refcount > 0);
+ av_assert0(outlink->outcfg.color_spaces->refcount > 0);
+ av_assert0( inlink-> incfg.color_ranges->refcount > 0);
+ av_assert0( inlink->outcfg.color_ranges->refcount > 0);
+ av_assert0(outlink-> incfg.color_ranges->refcount > 0);
+ av_assert0(outlink->outcfg.color_ranges->refcount > 0);
+ } else if (outlink->type == AVMEDIA_TYPE_AUDIO) {
av_assert0( inlink-> incfg.samplerates->refcount > 0);
av_assert0( inlink->outcfg.samplerates->refcount > 0);
av_assert0(outlink-> incfg.samplerates->refcount > 0);
@@ -583,6 +603,30 @@ static enum AVSampleFormat find_best_sample_fmt_of_2(enum AVSampleFormat dst_fmt
return score1 < score2 ? dst_fmt1 : dst_fmt2;
}
+int ff_fmt_is_regular_yuv(enum AVPixelFormat fmt)
+{
+ const AVPixFmtDescriptor *desc = av_pix_fmt_desc_get(fmt);
+ if (!desc)
+ return 0;
+ if (desc->nb_components < 3)
+ return 0; /* Grayscale is explicitly full-range in swscale */
+ av_assert1(!(desc->flags & AV_PIX_FMT_FLAG_HWACCEL));
+ if (desc->flags & (AV_PIX_FMT_FLAG_RGB | AV_PIX_FMT_FLAG_PAL |
+ AV_PIX_FMT_FLAG_XYZ | AV_PIX_FMT_FLAG_FLOAT))
+ return 0;
+
+ switch (fmt) {
+ case AV_PIX_FMT_YUVJ420P:
+ case AV_PIX_FMT_YUVJ422P:
+ case AV_PIX_FMT_YUVJ444P:
+ case AV_PIX_FMT_YUVJ440P:
+ case AV_PIX_FMT_YUVJ411P:
+ return 0;
+ default:
+ return 1;
+ }
+}
+
static int pick_format(AVFilterLink *link, AVFilterLink *ref)
{
if (!link || !link->incfg.formats)
@@ -621,7 +665,46 @@ static int pick_format(AVFilterLink *link, AVFilterLink *ref)
link->incfg.formats->nb_formats = 1;
link->format = link->incfg.formats->formats[0];
- if (link->type == AVMEDIA_TYPE_AUDIO) {
+ if (link->type == AVMEDIA_TYPE_VIDEO) {
+ enum AVPixelFormat swfmt = link->format;
+ if (av_pix_fmt_desc_get(swfmt)->flags & AV_PIX_FMT_FLAG_HWACCEL) {
+ av_assert1(link->hw_frames_ctx);
+ swfmt = ((AVHWFramesContext *) link->hw_frames_ctx->data)->sw_format;
+ }
+
+ if (!ff_fmt_is_regular_yuv(swfmt)) {
+ const AVPixFmtDescriptor *desc = av_pix_fmt_desc_get(swfmt);
+ /* These fields are explicitly documented as affecting YUV only,
+ * so set them to sane values for other formats. */
+ if (desc->flags & AV_PIX_FMT_FLAG_FLOAT)
+ link->color_range = AVCOL_RANGE_UNSPECIFIED;
+ else
+ link->color_range = AVCOL_RANGE_JPEG;
+ if (desc->flags & (AV_PIX_FMT_FLAG_RGB | AV_PIX_FMT_FLAG_XYZ)) {
+ link->colorspace = AVCOL_SPC_RGB;
+ } else {
+ link->colorspace = AVCOL_SPC_UNSPECIFIED;
+ }
+ } else {
+ if (!link->incfg.color_spaces->nb_formats) {
+ av_log(link->src, AV_LOG_ERROR, "Cannot select color space for"
+ " the link between filters %s and %s.\n", link->src->name,
+ link->dst->name);
+ return AVERROR(EINVAL);
+ }
+ link->incfg.color_spaces->nb_formats = 1;
+ link->colorspace = link->incfg.color_spaces->formats[0];
+
+ if (!link->incfg.color_ranges->nb_formats) {
+ av_log(link->src, AV_LOG_ERROR, "Cannot select color range for"
+ " the link between filters %s and %s.\n", link->src->name,
+ link->dst->name);
+ return AVERROR(EINVAL);
+ }
+ link->incfg.color_ranges->nb_formats = 1;
+ link->color_range = link->incfg.color_ranges->formats[0];
+ }
+ } else if (link->type == AVMEDIA_TYPE_AUDIO) {
int ret;
if (!link->incfg.samplerates->nb_formats) {
@@ -661,6 +744,10 @@ FF_ENABLE_DEPRECATION_WARNINGS
ff_formats_unref(&link->outcfg.samplerates);
ff_channel_layouts_unref(&link->incfg.channel_layouts);
ff_channel_layouts_unref(&link->outcfg.channel_layouts);
+ ff_formats_unref(&link->incfg.color_spaces);
+ ff_formats_unref(&link->outcfg.color_spaces);
+ ff_formats_unref(&link->incfg.color_ranges);
+ ff_formats_unref(&link->outcfg.color_ranges);
return 0;
}
@@ -822,6 +909,82 @@ static void swap_samplerates(AVFilterGraph *graph)
swap_samplerates_on_filter(graph->filters[i]);
}
+static void swap_color_spaces_on_filter(AVFilterContext *filter)
+{
+ AVFilterLink *link = NULL;
+ enum AVColorSpace csp;
+ int i;
+
+ for (i = 0; i < filter->nb_inputs; i++) {
+ link = filter->inputs[i];
+ if (link->type == AVMEDIA_TYPE_VIDEO &&
+ link->outcfg.color_spaces->nb_formats == 1)
+ break;
+ }
+ if (i == filter->nb_inputs)
+ return;
+
+ csp = link->outcfg.color_spaces->formats[0];
+
+ for (i = 0; i < filter->nb_outputs; i++) {
+ AVFilterLink *outlink = filter->outputs[i];
+ if (outlink->type != AVMEDIA_TYPE_VIDEO)
+ continue;
+ /* there is no meaningful 'score' between different yuv matrices,
+ * so just prioritize an exact match if it exists */
+ for (int j = 0; j < outlink->incfg.color_spaces->nb_formats; j++) {
+ if (csp == outlink->incfg.color_spaces->formats[j]) {
+ FFSWAP(int, outlink->incfg.color_spaces->formats[0],
+ outlink->incfg.color_spaces->formats[j]);
+ break;
+ }
+ }
+ }
+}
+
+static void swap_color_spaces(AVFilterGraph *graph)
+{
+ for (int i = 0; i < graph->nb_filters; i++)
+ swap_color_spaces_on_filter(graph->filters[i]);
+}
+
+static void swap_color_ranges_on_filter(AVFilterContext *filter)
+{
+ AVFilterLink *link = NULL;
+ enum AVColorRange range;
+ int i;
+
+ for (i = 0; i < filter->nb_inputs; i++) {
+ link = filter->inputs[i];
+ if (link->type == AVMEDIA_TYPE_VIDEO &&
+ link->outcfg.color_ranges->nb_formats == 1)
+ break;
+ }
+ if (i == filter->nb_inputs)
+ return;
+
+ range = link->outcfg.color_ranges->formats[0];
+
+ for (i = 0; i < filter->nb_outputs; i++) {
+ AVFilterLink *outlink = filter->outputs[i];
+ if (outlink->type != AVMEDIA_TYPE_VIDEO)
+ continue;
+ for (int j = 0; j < outlink->incfg.color_ranges->nb_formats; j++) {
+ if (range == outlink->incfg.color_ranges->formats[j]) {
+ FFSWAP(int, outlink->incfg.color_ranges->formats[0],
+ outlink->incfg.color_ranges->formats[j]);
+ break;
+ }
+ }
+ }
+}
+
+static void swap_color_ranges(AVFilterGraph *graph)
+{
+ for (int i = 0; i < graph->nb_filters; i++)
+ swap_color_ranges_on_filter(graph->filters[i]);
+}
+
#define CH_CENTER_PAIR (AV_CH_FRONT_LEFT_OF_CENTER | AV_CH_FRONT_RIGHT_OF_CENTER)
#define CH_FRONT_PAIR (AV_CH_FRONT_LEFT | AV_CH_FRONT_RIGHT)
#define CH_STEREO_PAIR (AV_CH_STEREO_LEFT | AV_CH_STEREO_RIGHT)
@@ -1098,6 +1261,10 @@ static int graph_config_formats(AVFilterGraph *graph, void *log_ctx)
if ((ret = reduce_formats(graph)) < 0)
return ret;
+ /* for video filters, ensure that the best colorspace metadata is selected */
+ swap_color_spaces(graph);
+ swap_color_ranges(graph);
+
/* for audio filters, ensure the best format, sample rate and channel layout
* is selected */
swap_sample_fmts(graph);
diff --git a/libavfilter/formats.c b/libavfilter/formats.c
index d1c97daf64..20f00d2db4 100644
--- a/libavfilter/formats.c
+++ b/libavfilter/formats.c
@@ -321,12 +321,46 @@ static int merge_channel_layouts(void *a, void *b)
return merge_channel_layouts_internal(a, b, 0);
}
+static int merge_generic_internal(AVFilterFormats *a,
+ AVFilterFormats *b, int check)
+{
+ av_assert2(check || (a->refcount && b->refcount));
+
+ if (a == b)
+ return 1;
+
+ MERGE_FORMATS(a, b, formats, nb_formats, AVFilterFormats, check, 0);
+
+ return 1;
+}
+
+static int can_merge_generic(const void *a, const void *b)
+{
+ return merge_generic_internal((AVFilterFormats *)a,
+ (AVFilterFormats *)b, 1);
+}
+
+static int merge_generic(void *a, void *b)
+{
+ return merge_generic_internal(a, b, 0);
+}
+
static const AVFilterFormatsMerger mergers_video[] = {
{
.offset = offsetof(AVFilterFormatsConfig, formats),
.merge = merge_pix_fmts,
.can_merge = can_merge_pix_fmts,
},
+ {
+ .offset = offsetof(AVFilterFormatsConfig, color_spaces),
+ .merge = merge_generic,
+ .can_merge = can_merge_generic,
+ },
+ {
+ .offset = offsetof(AVFilterFormatsConfig, color_ranges),
+ .merge = merge_generic,
+ .can_merge = can_merge_generic,
+ },
};
static const AVFilterFormatsMerger mergers_audio[] = {
@@ -594,6 +628,33 @@ AVFilterChannelLayouts *ff_all_channel_counts(void)
return ret;
}
+AVFilterFormats *ff_all_color_spaces(void)
+{
+ AVFilterFormats *ret = NULL;
+ if (ff_add_format(&ret, AVCOL_SPC_UNSPECIFIED) < 0)
+ return NULL;
+ for (int csp = 0; csp < AVCOL_SPC_NB; csp++) {
+ if (csp == AVCOL_SPC_RESERVED ||
+ csp == AVCOL_SPC_UNSPECIFIED)
+ continue;
+ if (ff_add_format(&ret, csp) < 0)
+ return NULL;
+ }
+
+ return ret;
+}
+
+AVFilterFormats *ff_all_color_ranges(void)
+{
+ AVFilterFormats *ret = NULL;
+ for (int range = 0; range < AVCOL_RANGE_NB; range++) {
+ if (ff_add_format(&ret, range) < 0)
+ return NULL;
+ }
+
+ return ret;
+}
+
#define FORMATS_REF(f, ref, unref_fn) \
void *tmp; \
\
@@ -763,6 +824,42 @@ int ff_set_common_all_samplerates(AVFilterContext *ctx)
return ff_set_common_samplerates(ctx, ff_all_samplerates());
}
+int ff_set_common_color_spaces(AVFilterContext *ctx,
+ AVFilterFormats *color_spaces)
+{
+ SET_COMMON_FORMATS(ctx, color_spaces, AVMEDIA_TYPE_VIDEO,
+ ff_formats_ref, ff_formats_unref);
+}
+
+int ff_set_common_color_spaces_from_list(AVFilterContext *ctx,
+ const int *color_ranges)
+{
+ return ff_set_common_color_spaces(ctx, ff_make_format_list(color_ranges));
+}
+
+int ff_set_common_all_color_spaces(AVFilterContext *ctx)
+{
+ return ff_set_common_color_spaces(ctx, ff_all_color_spaces());
+}
+
+int ff_set_common_color_ranges(AVFilterContext *ctx,
+ AVFilterFormats *color_ranges)
+{
+ SET_COMMON_FORMATS(ctx, color_ranges, AVMEDIA_TYPE_VIDEO,
+ ff_formats_ref, ff_formats_unref);
+}
+
+int ff_set_common_color_ranges_from_list(AVFilterContext *ctx,
+ const int *color_ranges)
+{
+ return ff_set_common_color_ranges(ctx, ff_make_format_list(color_ranges));
+}
+
+int ff_set_common_all_color_ranges(AVFilterContext *ctx)
+{
+ return ff_set_common_color_ranges(ctx, ff_all_color_ranges());
+}
+
/**
* A helper for query_formats() which sets all links to the same list of
* formats. If there are no links hooked to this filter, the list of formats is
@@ -817,7 +914,14 @@ int ff_default_query_formats(AVFilterContext *ctx)
ret = ff_set_common_formats(ctx, formats);
if (ret < 0)
return ret;
- if (type == AVMEDIA_TYPE_AUDIO) {
+ if (type == AVMEDIA_TYPE_VIDEO) {
+ ret = ff_set_common_all_color_spaces(ctx);
+ if (ret < 0)
+ return ret;
+ ret = ff_set_common_all_color_ranges(ctx);
+ if (ret < 0)
+ return ret;
+ } else if (type == AVMEDIA_TYPE_AUDIO) {
ret = ff_set_common_all_channel_counts(ctx);
if (ret < 0)
return ret;
@@ -935,6 +1039,22 @@ int ff_formats_check_sample_rates(void *log, const AVFilterFormats *fmts)
return check_list(log, "sample rate", fmts);
}
+int ff_formats_check_color_spaces(void *log, const AVFilterFormats *fmts)
+{
+ for (int i = 0; fmts && i < fmts->nb_formats; i++) {
+ if (fmts->formats[i] == AVCOL_SPC_RESERVED) {
+ av_log(log, AV_LOG_ERROR, "Invalid color range\n");
+ return AVERROR(EINVAL);
+ }
+ }
+ return check_list(log, "color space", fmts);
+}
+
+int ff_formats_check_color_ranges(void *log, const AVFilterFormats *fmts)
+{
+ return check_list(log, "color range", fmts);
+}
+
static int layouts_compatible(const AVChannelLayout *a, const AVChannelLayout *b)
{
return !av_channel_layout_compare(a, b) ||
diff --git a/libavfilter/formats.h b/libavfilter/formats.h
index d44890109e..82b3af4be1 100644
--- a/libavfilter/formats.h
+++ b/libavfilter/formats.h
@@ -130,6 +130,20 @@ AVFilterChannelLayouts *ff_all_channel_counts(void);
av_warn_unused_result
AVFilterChannelLayouts *ff_make_channel_layout_list(const AVChannelLayout *fmts);
+/**
+ * Construct an AVFilterFormats representing all possible color spaces.
+ *
+ * Note: This list does not include AVCOL_SPC_RESERVED.
+ */
+av_warn_unused_result
+AVFilterFormats *ff_all_color_spaces(void);
+
+/**
+ * Construct an AVFilterFormats representing all possible color ranges.
+ */
+av_warn_unused_result
+AVFilterFormats *ff_all_color_ranges(void);
+
/**
* Helpers for query_formats() which set all free audio links to the same list
* of channel layouts/sample rates. If there are no links hooked to this list,
@@ -165,6 +179,38 @@ int ff_set_common_samplerates_from_list(AVFilterContext *ctx,
av_warn_unused_result
int ff_set_common_all_samplerates(AVFilterContext *ctx);
+av_warn_unused_result
+int ff_set_common_color_spaces(AVFilterContext *ctx,
+ AVFilterFormats *color_spaces);
+/**
+ * Equivalent to ff_set_common_color_spaces(ctx, ff_make_format_list(color_spaces))
+ */
+av_warn_unused_result
+int ff_set_common_color_spaces_from_list(AVFilterContext *ctx,
+ const int *color_spaces);
+
+/**
+ * Equivalent to ff_set_common_color_spaces(ctx, ff_all_color_spaces())
+ */
+av_warn_unused_result
+int ff_set_common_all_color_spaces(AVFilterContext *ctx);
+
+av_warn_unused_result
+int ff_set_common_color_ranges(AVFilterContext *ctx,
+ AVFilterFormats *color_ranges);
+/**
+ * Equivalent to ff_set_common_color_ranges(ctx, ff_make_format_list(color_ranges))
+ */
+av_warn_unused_result
+int ff_set_common_color_ranges_from_list(AVFilterContext *ctx,
+ const int *color_ranges);
+
+/**
+ * Equivalent to ff_set_common_color_ranges(ctx, ff_all_color_ranges())
+ */
+av_warn_unused_result
+int ff_set_common_all_color_ranges(AVFilterContext *ctx);
+
/**
* A helper for query_formats() which sets all links to the same list of
* formats. If there are no links hooked to this filter, the list of formats is
@@ -328,6 +374,14 @@ int ff_formats_check_sample_rates(void *log, const AVFilterFormats *fmts);
*/
int ff_formats_check_channel_layouts(void *log, const AVFilterChannelLayouts *fmts);
+/**
+ * Check that fmts is a valid formats list for YUV colorspace metadata.
+ *
+ * In particular, check for duplicates.
+ */
+int ff_formats_check_color_spaces(void *log, const AVFilterFormats *fmts);
+int ff_formats_check_color_ranges(void *log, const AVFilterFormats *fmts);
+
typedef struct AVFilterFormatMerger {
unsigned offset;
int (*merge)(void *a, void *b);
diff --git a/libavfilter/internal.h b/libavfilter/internal.h
index 2dbc5def0a..a6cdf9994c 100644
--- a/libavfilter/internal.h
+++ b/libavfilter/internal.h
@@ -203,6 +203,12 @@ enum FilterFormatsState {
*/
int ff_fmt_is_in(int fmt, const int *fmts);
+/**
+ * Returns true if a pixel format is "regular YUV", which includes all pixel
+ * formats that are affected by YUV colorspace negotiation.
+ */
+int ff_fmt_is_regular_yuv(enum AVPixelFormat fmt);
+
/* Functions to parse audio format arguments */
/**
diff --git a/libavfilter/vaapi_vpp.c b/libavfilter/vaapi_vpp.c
index cf2592e068..59961bfa4a 100644
--- a/libavfilter/vaapi_vpp.c
+++ b/libavfilter/vaapi_vpp.c
@@ -38,6 +38,10 @@ int ff_vaapi_vpp_query_formats(AVFilterContext *avctx)
&avctx->outputs[0]->incfg.formats)) < 0)
return err;
+ if ((err = ff_set_common_all_color_spaces(avctx)) < 0 ||
+ (err = ff_set_common_all_color_ranges(avctx)) < 0)
+ return err;
+
return 0;
}
diff --git a/libavfilter/video.c b/libavfilter/video.c
index 42eeb98c28..243762c8fd 100644
--- a/libavfilter/video.c
+++ b/libavfilter/video.c
@@ -96,6 +96,8 @@ AVFrame *ff_default_get_video_buffer2(AVFilterLink *link, int w, int h, int alig
return NULL;
frame->sample_aspect_ratio = link->sample_aspect_ratio;
+ frame->colorspace = link->colorspace;
+ frame->color_range = link->color_range;
return frame;
}
diff --git a/tests/ref/fate/shortest-sub b/tests/ref/fate/shortest-sub
index 9caee587ce..16b1324e83 100644
--- a/tests/ref/fate/shortest-sub
+++ b/tests/ref/fate/shortest-sub
@@ -1,5 +1,5 @@
-73d142a80965f9e0884a5863abde0dab *tests/data/fate/shortest-sub.matroska
-139249 tests/data/fate/shortest-sub.matroska
+d6608277c93097383e62388196dc62f0 *tests/data/fate/shortest-sub.matroska
+139260 tests/data/fate/shortest-sub.matroska
#extradata 1: 167, 0xf7272d5f
#tb 0: 1/1000
#media_type 0: video
--
2.43.0
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
^ permalink raw reply [flat|nested] 26+ messages in thread* Re: [FFmpeg-devel] [PATCH v2 03/15] avfilter: add negotiation API for color space/range
2023-12-13 13:12 ` [FFmpeg-devel] [PATCH v2 03/15] avfilter: add negotiation API for color space/range Niklas Haas
@ 2023-12-14 3:09 ` Michael Niedermayer
2023-12-14 14:09 ` Niklas Haas
2023-12-14 14:39 ` [FFmpeg-devel] [PATCH] avfilter/formats: set audio fmt lists for vaf filters Niklas Haas
2024-01-03 3:04 ` [FFmpeg-devel] [PATCH v2 03/15] avfilter: add negotiation API for color space/range Xiang, Haihao
1 sibling, 2 replies; 26+ messages in thread
From: Michael Niedermayer @ 2023-12-14 3:09 UTC (permalink / raw)
To: FFmpeg development discussions and patches
[-- Attachment #1.1: Type: text/plain, Size: 2556 bytes --]
On Wed, Dec 13, 2023 at 02:12:00PM +0100, Niklas Haas wrote:
> From: Niklas Haas <git@haasn.dev>
>
> Motivated by YUVJ removal. This change will allow full negotiation
> between color ranges and matrices as needed. By default, all ranges and
> matrices are marked as supported.
>
> Because grayscale formats are currently handled very inconsistently (and
> in particular, assumed as forced full-range by swscale), we exclude them
> from negotiation altogether for the time being, to get this API merged.
>
> After filter negotiation is available, we can relax the
> grayscale-is-forced-jpeg restriction again, when it will be more
> feasible to do so without breaking a million test cases.
>
> Note that this commit updates one FATE test as a consequence of the
> sanity fallback for non-YUV formats. In particular, the test case now
> writes rgb24(pc, gbr/unspecified/unspecified) to the matroska file,
> instead of rgb24(unspecified/unspecified/unspecified) as before.
> ---
> doc/APIchanges | 3 +
> libavfilter/avfilter.c | 17 +++-
> libavfilter/avfilter.h | 28 ++++++
> libavfilter/avfiltergraph.c | 173 +++++++++++++++++++++++++++++++++++-
> libavfilter/formats.c | 122 ++++++++++++++++++++++++-
> libavfilter/formats.h | 54 +++++++++++
> libavfilter/internal.h | 6 ++
> libavfilter/vaapi_vpp.c | 4 +
> libavfilter/video.c | 2 +
> tests/ref/fate/shortest-sub | 4 +-
> 10 files changed, 404 insertions(+), 9 deletions(-)
segfaults
./ffmpeg -f lavfi -i "amovie=fate-suite/wavpack/num_channels/eva_2.22_6.1_16bit-partial.wv,asplit=3[out1][a][b]; [a]showwaves=s=340x240,pad=iw:ih*2[waves]; [b]showspectrum=s=340x240[spectrum]; [waves][spectrum] overlay=0:h [out0]" -t 0.1 -qscale 2 -bitexact /tmp/file-waves.avi
Thread 1 "ffmpeg_g" received signal SIGSEGV, Segmentation fault.
0x00005555557a7c07 in query_formats ()
(gdb) bt
#0 0x00005555557a7c07 in query_formats ()
#1 0x00005555557a84d5 in avfilter_graph_config ()
#2 0x000055555569ac0a in lavfi_read_header ()
#3 0x0000555555a7bd61 in avformat_open_input ()
#4 0x000055555574cb43 in ifile_open ()
#5 0x0000555555763c49 in open_files.isra ()
#6 0x0000555555765327 in ffmpeg_parse_options ()
#7 0x00005555557442b4 in main ()
(i can provide better backtrace if you cannot reproduce ...)
thx
[...]
--
Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB
I have often repented speaking, but never of holding my tongue.
-- Xenocrates
[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]
[-- Attachment #2: Type: text/plain, Size: 251 bytes --]
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [FFmpeg-devel] [PATCH v2 03/15] avfilter: add negotiation API for color space/range
2023-12-14 3:09 ` Michael Niedermayer
@ 2023-12-14 14:09 ` Niklas Haas
2023-12-14 14:39 ` [FFmpeg-devel] [PATCH] avfilter/formats: set audio fmt lists for vaf filters Niklas Haas
1 sibling, 0 replies; 26+ messages in thread
From: Niklas Haas @ 2023-12-14 14:09 UTC (permalink / raw)
To: FFmpeg development discussions and patches
> segfaults
>
> ./ffmpeg -f lavfi -i "amovie=fate-suite/wavpack/num_channels/eva_2.22_6.1_16bit-partial.wv,asplit=3[out1][a][b]; [a]showwaves=s=340x240,pad=iw:ih*2[waves]; [b]showspectrum=s=340x240[spectrum]; [waves][spectrum] overlay=0:h [out0]" -t 0.1 -qscale 2 -bitexact /tmp/file-waves.avi
>
> Thread 1 "ffmpeg_g" received signal SIGSEGV, Segmentation fault.
> 0x00005555557a7c07 in query_formats ()
> (gdb) bt
> #0 0x00005555557a7c07 in query_formats ()
> #1 0x00005555557a84d5 in avfilter_graph_config ()
> #2 0x000055555569ac0a in lavfi_read_header ()
> #3 0x0000555555a7bd61 in avformat_open_input ()
> #4 0x000055555574cb43 in ifile_open ()
> #5 0x0000555555763c49 in open_files.isra ()
> #6 0x0000555555765327 in ffmpeg_parse_options ()
> #7 0x00005555557442b4 in main ()
>
> (i can provide better backtrace if you cannot reproduce ...)
The culprit appears to be from this logic in formats.c:
case FF_FILTER_FORMATS_PASSTHROUGH:
case FF_FILTER_FORMATS_QUERY_FUNC:
type = ctx->nb_inputs ? ctx->inputs [0]->type :
ctx->nb_outputs ? ctx->outputs[0]->type : AVMEDIA_TYPE_VIDEO;
formats = ff_all_formats(type);
break;
In the case of an a/v filter (like avf_showspectrum), this incorrectly
determines the filter type as an audio filter, and therefore does not
run the default configuration logic on the video *output*.
Indeed, this logic would appear to break also in the case of
a (hypothetical) video->audio conversion filter, in which case channel
counts and samplerates would similarly fail to get assigned, because the
filter type would be incorrectly determined to be a video filter.
I think the correct resolution here would be to simply remove these
conditionals and always run ff_set_common_* in ff_default_query_formats.
They already explicitly ignore inputs/outputs of mismatching type, and
are a no-op if there is nothing to ref.
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
^ permalink raw reply [flat|nested] 26+ messages in thread* [FFmpeg-devel] [PATCH] avfilter/formats: set audio fmt lists for vaf filters
2023-12-14 3:09 ` Michael Niedermayer
2023-12-14 14:09 ` Niklas Haas
@ 2023-12-14 14:39 ` Niklas Haas
2023-12-14 14:40 ` Niklas Haas
2023-12-19 15:03 ` Nicolas George
1 sibling, 2 replies; 26+ messages in thread
From: Niklas Haas @ 2023-12-14 14:39 UTC (permalink / raw)
To: ffmpeg-devel; +Cc: Niklas Haas
From: Niklas Haas <git@haasn.dev>
Currently, the logic inside the FF_FILTER_FORMATS_QUERY_FUNC branch
prevents this code from running in the event that we have a filter with
a single video input and a single audio output, as the resulting audio
output link will not have its channel counts / samplerates correctly
initialized to their default values, possibly triggering a segfault
downstream.
An example of such a filter is vaf_spectrumsynth. Although this
particular filter already sets up the channel counts and samplerates as
part of the query function and therefore avoids triggering this bug, the
bug still exists in principle. (And importantly, sets a wrong precedent)
---
libavfilter/formats.c | 9 +++++----
1 file changed, 5 insertions(+), 4 deletions(-)
diff --git a/libavfilter/formats.c b/libavfilter/formats.c
index d1c97daf64..114886aeb2 100644
--- a/libavfilter/formats.c
+++ b/libavfilter/formats.c
@@ -808,16 +808,17 @@ int ff_default_query_formats(AVFilterContext *ctx)
/* Intended fallthrough */
case FF_FILTER_FORMATS_PASSTHROUGH:
case FF_FILTER_FORMATS_QUERY_FUNC:
- type = ctx->nb_inputs ? ctx->inputs [0]->type :
- ctx->nb_outputs ? ctx->outputs[0]->type : AVMEDIA_TYPE_VIDEO;
- formats = ff_all_formats(type);
+ type = AVMEDIA_TYPE_UNKNOWN;
+ formats = ff_all_formats(ctx->nb_inputs ? ctx->inputs [0]->type :
+ ctx->nb_outputs ? ctx->outputs[0]->type :
+ AVMEDIA_TYPE_VIDEO);
break;
}
ret = ff_set_common_formats(ctx, formats);
if (ret < 0)
return ret;
- if (type == AVMEDIA_TYPE_AUDIO) {
+ if (type != AVMEDIA_TYPE_VIDEO) {
ret = ff_set_common_all_channel_counts(ctx);
if (ret < 0)
return ret;
--
2.43.0
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
^ permalink raw reply [flat|nested] 26+ messages in thread* Re: [FFmpeg-devel] [PATCH] avfilter/formats: set audio fmt lists for vaf filters
2023-12-14 14:39 ` [FFmpeg-devel] [PATCH] avfilter/formats: set audio fmt lists for vaf filters Niklas Haas
@ 2023-12-14 14:40 ` Niklas Haas
2023-12-14 19:18 ` Michael Niedermayer
2023-12-19 15:03 ` Nicolas George
1 sibling, 1 reply; 26+ messages in thread
From: Niklas Haas @ 2023-12-14 14:40 UTC (permalink / raw)
To: ffmpeg-devel; +Cc: Niklas Haas
On Thu, 14 Dec 2023 15:39:11 +0100 Niklas Haas <ffmpeg@haasn.xyz> wrote:
> From: Niklas Haas <git@haasn.dev>
>
> Currently, the logic inside the FF_FILTER_FORMATS_QUERY_FUNC branch
> prevents this code from running in the event that we have a filter with
> a single video input and a single audio output, as the resulting audio
> output link will not have its channel counts / samplerates correctly
> initialized to their default values, possibly triggering a segfault
> downstream.
>
> An example of such a filter is vaf_spectrumsynth. Although this
> particular filter already sets up the channel counts and samplerates as
> part of the query function and therefore avoids triggering this bug, the
> bug still exists in principle. (And importantly, sets a wrong precedent)
> ---
> libavfilter/formats.c | 9 +++++----
> 1 file changed, 5 insertions(+), 4 deletions(-)
>
> diff --git a/libavfilter/formats.c b/libavfilter/formats.c
> index d1c97daf64..114886aeb2 100644
> --- a/libavfilter/formats.c
> +++ b/libavfilter/formats.c
> @@ -808,16 +808,17 @@ int ff_default_query_formats(AVFilterContext *ctx)
> /* Intended fallthrough */
> case FF_FILTER_FORMATS_PASSTHROUGH:
> case FF_FILTER_FORMATS_QUERY_FUNC:
> - type = ctx->nb_inputs ? ctx->inputs [0]->type :
> - ctx->nb_outputs ? ctx->outputs[0]->type : AVMEDIA_TYPE_VIDEO;
> - formats = ff_all_formats(type);
> + type = AVMEDIA_TYPE_UNKNOWN;
> + formats = ff_all_formats(ctx->nb_inputs ? ctx->inputs [0]->type :
> + ctx->nb_outputs ? ctx->outputs[0]->type :
> + AVMEDIA_TYPE_VIDEO);
> break;
> }
>
> ret = ff_set_common_formats(ctx, formats);
> if (ret < 0)
> return ret;
> - if (type == AVMEDIA_TYPE_AUDIO) {
> + if (type != AVMEDIA_TYPE_VIDEO) {
> ret = ff_set_common_all_channel_counts(ctx);
> if (ret < 0)
> return ret;
> --
> 2.43.0
>
This patch fixes the underlying issue (alongside the corresponding
adjustment to the conditional from `type == AVMEDIA_TYPE_VIDEO` to `type
!= AVMEDIA_TYPE_AUDIO` in patch 03/15).
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
^ permalink raw reply [flat|nested] 26+ messages in thread* Re: [FFmpeg-devel] [PATCH] avfilter/formats: set audio fmt lists for vaf filters
2023-12-14 14:40 ` Niklas Haas
@ 2023-12-14 19:18 ` Michael Niedermayer
0 siblings, 0 replies; 26+ messages in thread
From: Michael Niedermayer @ 2023-12-14 19:18 UTC (permalink / raw)
To: FFmpeg development discussions and patches
[-- Attachment #1.1: Type: text/plain, Size: 2790 bytes --]
On Thu, Dec 14, 2023 at 03:40:36PM +0100, Niklas Haas wrote:
> On Thu, 14 Dec 2023 15:39:11 +0100 Niklas Haas <ffmpeg@haasn.xyz> wrote:
> > From: Niklas Haas <git@haasn.dev>
> >
> > Currently, the logic inside the FF_FILTER_FORMATS_QUERY_FUNC branch
> > prevents this code from running in the event that we have a filter with
> > a single video input and a single audio output, as the resulting audio
> > output link will not have its channel counts / samplerates correctly
> > initialized to their default values, possibly triggering a segfault
> > downstream.
> >
> > An example of such a filter is vaf_spectrumsynth. Although this
> > particular filter already sets up the channel counts and samplerates as
> > part of the query function and therefore avoids triggering this bug, the
> > bug still exists in principle. (And importantly, sets a wrong precedent)
> > ---
> > libavfilter/formats.c | 9 +++++----
> > 1 file changed, 5 insertions(+), 4 deletions(-)
> >
> > diff --git a/libavfilter/formats.c b/libavfilter/formats.c
> > index d1c97daf64..114886aeb2 100644
> > --- a/libavfilter/formats.c
> > +++ b/libavfilter/formats.c
> > @@ -808,16 +808,17 @@ int ff_default_query_formats(AVFilterContext *ctx)
> > /* Intended fallthrough */
> > case FF_FILTER_FORMATS_PASSTHROUGH:
> > case FF_FILTER_FORMATS_QUERY_FUNC:
> > - type = ctx->nb_inputs ? ctx->inputs [0]->type :
> > - ctx->nb_outputs ? ctx->outputs[0]->type : AVMEDIA_TYPE_VIDEO;
> > - formats = ff_all_formats(type);
> > + type = AVMEDIA_TYPE_UNKNOWN;
> > + formats = ff_all_formats(ctx->nb_inputs ? ctx->inputs [0]->type :
> > + ctx->nb_outputs ? ctx->outputs[0]->type :
> > + AVMEDIA_TYPE_VIDEO);
> > break;
> > }
> >
> > ret = ff_set_common_formats(ctx, formats);
> > if (ret < 0)
> > return ret;
> > - if (type == AVMEDIA_TYPE_AUDIO) {
> > + if (type != AVMEDIA_TYPE_VIDEO) {
> > ret = ff_set_common_all_channel_counts(ctx);
> > if (ret < 0)
> > return ret;
> > --
> > 2.43.0
> >
>
> This patch fixes the underlying issue (alongside the corresponding
> adjustment to the conditional from `type == AVMEDIA_TYPE_VIDEO` to `type
> != AVMEDIA_TYPE_AUDIO` in patch 03/15).
do you have some git branch i can test so i dont have to apply a patchset
with adjustments and hope i have the exact same code ?
thx
[...]
--
Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB
In fact, the RIAA has been known to suggest that students drop out
of college or go to community college in order to be able to afford
settlements. -- The RIAA
[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]
[-- Attachment #2: Type: text/plain, Size: 251 bytes --]
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [FFmpeg-devel] [PATCH] avfilter/formats: set audio fmt lists for vaf filters
2023-12-14 14:39 ` [FFmpeg-devel] [PATCH] avfilter/formats: set audio fmt lists for vaf filters Niklas Haas
2023-12-14 14:40 ` Niklas Haas
@ 2023-12-19 15:03 ` Nicolas George
1 sibling, 0 replies; 26+ messages in thread
From: Nicolas George @ 2023-12-19 15:03 UTC (permalink / raw)
To: FFmpeg development discussions and patches; +Cc: Niklas Haas
Niklas Haas (12023-12-14):
> From: Niklas Haas <git@haasn.dev>
>
> Currently, the logic inside the FF_FILTER_FORMATS_QUERY_FUNC branch
> prevents this code from running in the event that we have a filter with
> a single video input and a single audio output, as the resulting audio
> output link will not have its channel counts / samplerates correctly
> initialized to their default values, possibly triggering a segfault
> downstream.
>
> An example of such a filter is vaf_spectrumsynth. Although this
> particular filter already sets up the channel counts and samplerates as
> part of the query function and therefore avoids triggering this bug, the
> bug still exists in principle. (And importantly, sets a wrong precedent)
> ---
> libavfilter/formats.c | 9 +++++----
> 1 file changed, 5 insertions(+), 4 deletions(-)
You are asking the framework to guess the audio format when it has
NOTHING to base its guess on. This is a terrible idea.
When a filter has no audio input but must produce audio, only the filter
itself knows what kind of audio it will produce, and it should say so in
query_formats().
What happens in spectrumsynth is what MUST happens. It is not a wrong
precedent, it is the proper way of things.
The worry that a filter will forget to do so and result in a segfault
seems wrong to me: just testing the filter before committing would
reveal it, not even regression testing is necessary.
But if you really want, I would not object to a consistency check after
the call to query_formats().
Regards,
--
Nicolas George
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [FFmpeg-devel] [PATCH v2 03/15] avfilter: add negotiation API for color space/range
2023-12-13 13:12 ` [FFmpeg-devel] [PATCH v2 03/15] avfilter: add negotiation API for color space/range Niklas Haas
2023-12-14 3:09 ` Michael Niedermayer
@ 2024-01-03 3:04 ` Xiang, Haihao
1 sibling, 0 replies; 26+ messages in thread
From: Xiang, Haihao @ 2024-01-03 3:04 UTC (permalink / raw)
To: ffmpeg-devel; +Cc: git
On Wo, 2023-12-13 at 14:12 +0100, Niklas Haas wrote:
> From: Niklas Haas <git@haasn.dev>
>
> Motivated by YUVJ removal. This change will allow full negotiation
> between color ranges and matrices as needed. By default, all ranges and
> matrices are marked as supported.
>
> Because grayscale formats are currently handled very inconsistently (and
> in particular, assumed as forced full-range by swscale), we exclude them
> from negotiation altogether for the time being, to get this API merged.
>
> After filter negotiation is available, we can relax the
> grayscale-is-forced-jpeg restriction again, when it will be more
> feasible to do so without breaking a million test cases.
>
> Note that this commit updates one FATE test as a consequence of the
> sanity fallback for non-YUV formats. In particular, the test case now
> writes rgb24(pc, gbr/unspecified/unspecified) to the matroska file,
> instead of rgb24(unspecified/unspecified/unspecified) as before.
> ---
> doc/APIchanges | 3 +
> libavfilter/avfilter.c | 17 +++-
> libavfilter/avfilter.h | 28 ++++++
> libavfilter/avfiltergraph.c | 173 +++++++++++++++++++++++++++++++++++-
> libavfilter/formats.c | 122 ++++++++++++++++++++++++-
> libavfilter/formats.h | 54 +++++++++++
> libavfilter/internal.h | 6 ++
> libavfilter/vaapi_vpp.c | 4 +
> libavfilter/video.c | 2 +
> tests/ref/fate/shortest-sub | 4 +-
> 10 files changed, 404 insertions(+), 9 deletions(-)
It caused segfault when using hw accelerations, such as vulkan, vaapi, qsv.
$ffmpeg -init_hw_device qsv -f lavfi -i yuvtestsrc=duration=1,format=nv12 -vf 'hwupload=extra_hw_frames=8' -f null -
$ffmpeg -init_hw_device vaapi -f lavfi -i yuvtestsrc=duration=1,format=nv12 -vf 'hwupload=extra_hw_frames=8' -f null -
$ffmpeg -init_hw_device vulkan -f lavfi -i yuvtestsrc=duration=1,format=nv12 -vf 'hwupload=extra_hw_frames=8' -f null -
Thread 36 "vf#0:0" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fffc4fe1640 (LWP 3795664)]
pick_format (link=0x7fffb8004100, ref=0x0) at libavfilter/avfiltergraph.c:671
671 swfmt = ((AVHWFramesContext *) link->hw_frames_ctx->data)->sw_format;
(gdb) bt
#0 pick_format (link=0x7fffb8004100, ref=0x0) at libavfilter/avfiltergraph.c:671
#1 0x00007ffff75d5b57 in pick_formats (graph=0x7fffb8000ff0) at libavfilter/avfiltergraph.c:1213
#2 0x00007ffff75d5ded in graph_config_formats (graph=0x7fffb8000ff0, log_ctx=0x0) at libavfilter/avfiltergraph.c:1273
#3 0x00007ffff75d60c3 in avfilter_graph_config (graphctx=0x7fffb8000ff0, log_ctx=0x0) at libavfilter/avfiltergraph.c:1326
#4 0x00005555555786c8 in configure_filtergraph (fg=0x555555666e60, fgt=0x7fffc4fe0b30) at fftools/ffmpeg_filter.c:1758
#5 0x000055555557b4f5 in send_frame (fg=0x555555666e60, fgt=0x7fffc4fe0b30, ifilter=0x555555663b80, frame=0x7fffb8000b70) at fftools/ffmpeg_filter.c:2652
#6 0x000055555557bdce in filter_thread (arg=0x555555666e60) at fftools/ffmpeg_filter.c:2813
#7 0x000055555559d8cd in task_wrapper (arg=0x555555663df8) at fftools/ffmpeg_sched.c:2200
#8 0x00007ffff4094b43 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442
#9 0x00007ffff4126a00 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81
(gdb) p link->hw_frames_ctx
$1 = (AVBufferRef *) 0x0
link->hw_frames_ctx is set after graph_config_formats()
Thanks
Haihao
>
> diff --git a/doc/APIchanges b/doc/APIchanges
> index 4a2dc1c44f..2f6ea50f63 100644
> --- a/doc/APIchanges
> +++ b/doc/APIchanges
> @@ -2,6 +2,9 @@ The last version increases of all libraries were on 2023-02-09
>
> API changes, most recent first:
>
> +2023-11-xx - xxxxxxxxxx - lavf 58.14.100 - avfilter.h
> + Add AVFilterLink.colorspace, AVFilterLink.color_range
> +
> 2023-11-08 - b82957a66a7 - lavu 58.32.100 - channel_layout.h
> Add AV_CH_LAYOUT_7POINT2POINT3 and AV_CHANNEL_LAYOUT_7POINT2POINT3.
> Add AV_CH_LAYOUT_9POINT1POINT4_BACK and
> AV_CHANNEL_LAYOUT_9POINT1POINT4_BACK.
> diff --git a/libavfilter/avfilter.c b/libavfilter/avfilter.c
> index bde1c33d07..31300bb515 100644
> --- a/libavfilter/avfilter.c
> +++ b/libavfilter/avfilter.c
> @@ -185,6 +185,7 @@ int avfilter_link(AVFilterContext *src, unsigned srcpad,
> link->type = src->output_pads[srcpad].type;
> av_assert0(AV_PIX_FMT_NONE == -1 && AV_SAMPLE_FMT_NONE == -1);
> link->format = -1;
> + link->colorspace = AVCOL_SPC_UNSPECIFIED;
> ff_framequeue_init(&link->fifo, &src->graph->internal->frame_queues);
>
> return 0;
> @@ -286,6 +287,12 @@ int avfilter_insert_filter(AVFilterLink *link,
> AVFilterContext *filt,
> if (link->outcfg.formats)
> ff_formats_changeref(&link->outcfg.formats,
> &filt->outputs[filt_dstpad_idx]-
> >outcfg.formats);
> + if (link->outcfg.color_spaces)
> + ff_formats_changeref(&link->outcfg.color_spaces,
> + &filt->outputs[filt_dstpad_idx]-
> >outcfg.color_spaces);
> + if (link->outcfg.color_ranges)
> + ff_formats_changeref(&link->outcfg.color_ranges,
> + &filt->outputs[filt_dstpad_idx]-
> >outcfg.color_ranges);
> if (link->outcfg.samplerates)
> ff_formats_changeref(&link->outcfg.samplerates,
> &filt->outputs[filt_dstpad_idx]-
> >outcfg.samplerates);
> @@ -730,6 +737,10 @@ static void free_link(AVFilterLink *link)
>
> ff_formats_unref(&link->incfg.formats);
> ff_formats_unref(&link->outcfg.formats);
> + ff_formats_unref(&link->incfg.color_spaces);
> + ff_formats_unref(&link->outcfg.color_spaces);
> + ff_formats_unref(&link->incfg.color_ranges);
> + ff_formats_unref(&link->outcfg.color_ranges);
> ff_formats_unref(&link->incfg.samplerates);
> ff_formats_unref(&link->outcfg.samplerates);
> ff_channel_layouts_unref(&link->incfg.channel_layouts);
> @@ -987,9 +998,9 @@ int ff_filter_frame(AVFilterLink *link, AVFrame *frame)
> strcmp(link->dst->filter->name, "idet") &&
> strcmp(link->dst->filter->name, "null") &&
> strcmp(link->dst->filter->name, "scale")) {
> - av_assert1(frame->format == link->format);
> - av_assert1(frame->width == link->w);
> - av_assert1(frame->height == link->h);
> + av_assert1(frame->format == link->format);
> + av_assert1(frame->width == link->w);
> + av_assert1(frame->height == link->h);
> }
>
> frame->sample_aspect_ratio = link->sample_aspect_ratio;
> diff --git a/libavfilter/avfilter.h b/libavfilter/avfilter.h
> index d69381aed4..246d000251 100644
> --- a/libavfilter/avfilter.h
> +++ b/libavfilter/avfilter.h
> @@ -301,6 +301,14 @@ typedef struct AVFilter {
> * @ref AVFilterFormatsConfig.formats "incfg.formats"
> * on every output link to a list of pixel/sample formats that the
> filter
> * supports on that link.
> + * For video links, this filter may also set
> + * @ref AVFilterFormatsConfig.color_spaces "incfg.color_spaces"
> + * /
> + * @ref AVFilterFormatsConfig.color_spaces "outcfg.color_spaces"
> + * and @ref AVFilterFormatsConfig.color_ranges "incfg.color_ranges"
> + * /
> + * @ref AVFilterFormatsConfig.color_ranges "outcfg.color_ranges"
> + * analogously.
> * For audio links, this filter must also set
> * @ref AVFilterFormatsConfig.samplerates "incfg.samplerates"
> * /
> @@ -322,6 +330,10 @@ typedef struct AVFilter {
> * to indicate that this filter supports each of these pixel formats,
> * provided that all inputs and outputs use the same pixel format.
> *
> + * In addition to that the generic code will mark all inputs
> + * and all outputs as supporting all color spaces and ranges, as
> + * long as all inputs and outputs use the same color space/range.
> + *
> * This list must never be NULL if the union is in this state.
> * The type of all inputs and outputs of filters using this must
> * be AVMEDIA_TYPE_VIDEO.
> @@ -514,6 +526,12 @@ typedef struct AVFilterFormatsConfig {
> */
> AVFilterChannelLayouts *channel_layouts;
>
> + /**
> + * Lists of supported YUV color metadata, only for YUV video.
> + */
> + AVFilterFormats *color_spaces; ///< AVColorSpace
> + AVFilterFormats *color_ranges; ///< AVColorRange
> +
> } AVFilterFormatsConfig;
>
> /**
> @@ -565,6 +583,16 @@ struct AVFilterLink {
>
> AVChannelLayout ch_layout; ///< channel layout of current buffer (see
> libavutil/channel_layout.h)
>
> + /**
> + * For non-YUV links, these are respectively set to fallback values (as
> + * appropriate for that colorspace).
> + *
> + * Note: This includes grayscale formats, as these are currently treated
> + * as forced full range always.
> + */
> + enum AVColorSpace colorspace; ///< agreed upon YUV color space
> + enum AVColorRange color_range; ///< agreed upon YUV color range
> +
> /*****************************************************************
> * All fields below this line are not part of the public API. They
> * may not be used outside of libavfilter and can be changed and
> diff --git a/libavfilter/avfiltergraph.c b/libavfilter/avfiltergraph.c
> index 625cbc022e..ef739735bd 100644
> --- a/libavfilter/avfiltergraph.c
> +++ b/libavfilter/avfiltergraph.c
> @@ -27,6 +27,7 @@
> #include "libavutil/avassert.h"
> #include "libavutil/bprint.h"
> #include "libavutil/channel_layout.h"
> +#include "libavutil/hwcontext.h"
> #include "libavutil/imgutils.h"
> #include "libavutil/opt.h"
> #include "libavutil/pixdesc.h"
> @@ -298,7 +299,9 @@ static int filter_link_check_formats(void *log,
> AVFilterLink *link, AVFilterForm
> switch (link->type) {
>
> case AVMEDIA_TYPE_VIDEO:
> - if ((ret = ff_formats_check_pixel_formats(log, cfg->formats)) < 0)
> + if ((ret = ff_formats_check_pixel_formats(log, cfg->formats)) < 0 ||
> + (ret = ff_formats_check_color_spaces(log, cfg->color_spaces)) < 0
> ||
> + (ret = ff_formats_check_color_ranges(log, cfg->color_ranges)) <
> 0)
> return ret;
> break;
>
> @@ -365,6 +368,10 @@ static int formats_declared(AVFilterContext *f)
> for (i = 0; i < f->nb_inputs; i++) {
> if (!f->inputs[i]->outcfg.formats)
> return 0;
> + if (f->inputs[i]->type == AVMEDIA_TYPE_VIDEO &&
> + !(f->inputs[i]->outcfg.color_ranges &&
> + f->inputs[i]->outcfg.color_spaces))
> + return 0;
> if (f->inputs[i]->type == AVMEDIA_TYPE_AUDIO &&
> !(f->inputs[i]->outcfg.samplerates &&
> f->inputs[i]->outcfg.channel_layouts))
> @@ -373,6 +380,10 @@ static int formats_declared(AVFilterContext *f)
> for (i = 0; i < f->nb_outputs; i++) {
> if (!f->outputs[i]->incfg.formats)
> return 0;
> + if (f->outputs[i]->type == AVMEDIA_TYPE_VIDEO &&
> + !(f->outputs[i]->incfg.color_ranges &&
> + f->outputs[i]->incfg.color_spaces))
> + return 0;
> if (f->outputs[i]->type == AVMEDIA_TYPE_AUDIO &&
> !(f->outputs[i]->incfg.samplerates &&
> f->outputs[i]->incfg.channel_layouts))
> @@ -493,7 +504,16 @@ static int query_formats(AVFilterGraph *graph, void
> *log_ctx)
> av_assert0( inlink->outcfg.formats->refcount > 0);
> av_assert0(outlink->incfg.formats->refcount > 0);
> av_assert0(outlink->outcfg.formats->refcount > 0);
> - if (outlink->type == AVMEDIA_TYPE_AUDIO) {
> + if (outlink->type == AVMEDIA_TYPE_VIDEO) {
> + av_assert0( inlink-> incfg.color_spaces->refcount > 0);
> + av_assert0( inlink->outcfg.color_spaces->refcount > 0);
> + av_assert0(outlink-> incfg.color_spaces->refcount > 0);
> + av_assert0(outlink->outcfg.color_spaces->refcount > 0);
> + av_assert0( inlink-> incfg.color_ranges->refcount > 0);
> + av_assert0( inlink->outcfg.color_ranges->refcount > 0);
> + av_assert0(outlink-> incfg.color_ranges->refcount > 0);
> + av_assert0(outlink->outcfg.color_ranges->refcount > 0);
> + } else if (outlink->type == AVMEDIA_TYPE_AUDIO) {
> av_assert0( inlink-> incfg.samplerates->refcount > 0);
> av_assert0( inlink->outcfg.samplerates->refcount > 0);
> av_assert0(outlink-> incfg.samplerates->refcount > 0);
> @@ -583,6 +603,30 @@ static enum AVSampleFormat find_best_sample_fmt_of_2(enum
> AVSampleFormat dst_fmt
> return score1 < score2 ? dst_fmt1 : dst_fmt2;
> }
>
> +int ff_fmt_is_regular_yuv(enum AVPixelFormat fmt)
> +{
> + const AVPixFmtDescriptor *desc = av_pix_fmt_desc_get(fmt);
> + if (!desc)
> + return 0;
> + if (desc->nb_components < 3)
> + return 0; /* Grayscale is explicitly full-range in swscale */
> + av_assert1(!(desc->flags & AV_PIX_FMT_FLAG_HWACCEL));
> + if (desc->flags & (AV_PIX_FMT_FLAG_RGB | AV_PIX_FMT_FLAG_PAL |
> + AV_PIX_FMT_FLAG_XYZ | AV_PIX_FMT_FLAG_FLOAT))
> + return 0;
> +
> + switch (fmt) {
> + case AV_PIX_FMT_YUVJ420P:
> + case AV_PIX_FMT_YUVJ422P:
> + case AV_PIX_FMT_YUVJ444P:
> + case AV_PIX_FMT_YUVJ440P:
> + case AV_PIX_FMT_YUVJ411P:
> + return 0;
> + default:
> + return 1;
> + }
> +}
> +
> static int pick_format(AVFilterLink *link, AVFilterLink *ref)
> {
> if (!link || !link->incfg.formats)
> @@ -621,7 +665,46 @@ static int pick_format(AVFilterLink *link, AVFilterLink
> *ref)
> link->incfg.formats->nb_formats = 1;
> link->format = link->incfg.formats->formats[0];
>
> - if (link->type == AVMEDIA_TYPE_AUDIO) {
> + if (link->type == AVMEDIA_TYPE_VIDEO) {
> + enum AVPixelFormat swfmt = link->format;
> + if (av_pix_fmt_desc_get(swfmt)->flags & AV_PIX_FMT_FLAG_HWACCEL) {
> + av_assert1(link->hw_frames_ctx);
> + swfmt = ((AVHWFramesContext *) link->hw_frames_ctx->data)-
> >sw_format;
> + }
> +
> + if (!ff_fmt_is_regular_yuv(swfmt)) {
> + const AVPixFmtDescriptor *desc = av_pix_fmt_desc_get(swfmt);
> + /* These fields are explicitly documented as affecting YUV only,
> + * so set them to sane values for other formats. */
> + if (desc->flags & AV_PIX_FMT_FLAG_FLOAT)
> + link->color_range = AVCOL_RANGE_UNSPECIFIED;
> + else
> + link->color_range = AVCOL_RANGE_JPEG;
> + if (desc->flags & (AV_PIX_FMT_FLAG_RGB | AV_PIX_FMT_FLAG_XYZ)) {
> + link->colorspace = AVCOL_SPC_RGB;
> + } else {
> + link->colorspace = AVCOL_SPC_UNSPECIFIED;
> + }
> + } else {
> + if (!link->incfg.color_spaces->nb_formats) {
> + av_log(link->src, AV_LOG_ERROR, "Cannot select color space
> for"
> + " the link between filters %s and %s.\n", link->src-
> >name,
> + link->dst->name);
> + return AVERROR(EINVAL);
> + }
> + link->incfg.color_spaces->nb_formats = 1;
> + link->colorspace = link->incfg.color_spaces->formats[0];
> +
> + if (!link->incfg.color_ranges->nb_formats) {
> + av_log(link->src, AV_LOG_ERROR, "Cannot select color range
> for"
> + " the link between filters %s and %s.\n", link->src-
> >name,
> + link->dst->name);
> + return AVERROR(EINVAL);
> + }
> + link->incfg.color_ranges->nb_formats = 1;
> + link->color_range = link->incfg.color_ranges->formats[0];
> + }
> + } else if (link->type == AVMEDIA_TYPE_AUDIO) {
> int ret;
>
> if (!link->incfg.samplerates->nb_formats) {
> @@ -661,6 +744,10 @@ FF_ENABLE_DEPRECATION_WARNINGS
> ff_formats_unref(&link->outcfg.samplerates);
> ff_channel_layouts_unref(&link->incfg.channel_layouts);
> ff_channel_layouts_unref(&link->outcfg.channel_layouts);
> + ff_formats_unref(&link->incfg.color_spaces);
> + ff_formats_unref(&link->outcfg.color_spaces);
> + ff_formats_unref(&link->incfg.color_ranges);
> + ff_formats_unref(&link->outcfg.color_ranges);
>
> return 0;
> }
> @@ -822,6 +909,82 @@ static void swap_samplerates(AVFilterGraph *graph)
> swap_samplerates_on_filter(graph->filters[i]);
> }
>
> +static void swap_color_spaces_on_filter(AVFilterContext *filter)
> +{
> + AVFilterLink *link = NULL;
> + enum AVColorSpace csp;
> + int i;
> +
> + for (i = 0; i < filter->nb_inputs; i++) {
> + link = filter->inputs[i];
> + if (link->type == AVMEDIA_TYPE_VIDEO &&
> + link->outcfg.color_spaces->nb_formats == 1)
> + break;
> + }
> + if (i == filter->nb_inputs)
> + return;
> +
> + csp = link->outcfg.color_spaces->formats[0];
> +
> + for (i = 0; i < filter->nb_outputs; i++) {
> + AVFilterLink *outlink = filter->outputs[i];
> + if (outlink->type != AVMEDIA_TYPE_VIDEO)
> + continue;
> + /* there is no meaningful 'score' between different yuv matrices,
> + * so just prioritize an exact match if it exists */
> + for (int j = 0; j < outlink->incfg.color_spaces->nb_formats; j++) {
> + if (csp == outlink->incfg.color_spaces->formats[j]) {
> + FFSWAP(int, outlink->incfg.color_spaces->formats[0],
> + outlink->incfg.color_spaces->formats[j]);
> + break;
> + }
> + }
> + }
> +}
> +
> +static void swap_color_spaces(AVFilterGraph *graph)
> +{
> + for (int i = 0; i < graph->nb_filters; i++)
> + swap_color_spaces_on_filter(graph->filters[i]);
> +}
> +
> +static void swap_color_ranges_on_filter(AVFilterContext *filter)
> +{
> + AVFilterLink *link = NULL;
> + enum AVColorRange range;
> + int i;
> +
> + for (i = 0; i < filter->nb_inputs; i++) {
> + link = filter->inputs[i];
> + if (link->type == AVMEDIA_TYPE_VIDEO &&
> + link->outcfg.color_ranges->nb_formats == 1)
> + break;
> + }
> + if (i == filter->nb_inputs)
> + return;
> +
> + range = link->outcfg.color_ranges->formats[0];
> +
> + for (i = 0; i < filter->nb_outputs; i++) {
> + AVFilterLink *outlink = filter->outputs[i];
> + if (outlink->type != AVMEDIA_TYPE_VIDEO)
> + continue;
> + for (int j = 0; j < outlink->incfg.color_ranges->nb_formats; j++) {
> + if (range == outlink->incfg.color_ranges->formats[j]) {
> + FFSWAP(int, outlink->incfg.color_ranges->formats[0],
> + outlink->incfg.color_ranges->formats[j]);
> + break;
> + }
> + }
> + }
> +}
> +
> +static void swap_color_ranges(AVFilterGraph *graph)
> +{
> + for (int i = 0; i < graph->nb_filters; i++)
> + swap_color_ranges_on_filter(graph->filters[i]);
> +}
> +
> #define CH_CENTER_PAIR (AV_CH_FRONT_LEFT_OF_CENTER |
> AV_CH_FRONT_RIGHT_OF_CENTER)
> #define CH_FRONT_PAIR (AV_CH_FRONT_LEFT | AV_CH_FRONT_RIGHT)
> #define CH_STEREO_PAIR (AV_CH_STEREO_LEFT | AV_CH_STEREO_RIGHT)
> @@ -1098,6 +1261,10 @@ static int graph_config_formats(AVFilterGraph *graph,
> void *log_ctx)
> if ((ret = reduce_formats(graph)) < 0)
> return ret;
>
> + /* for video filters, ensure that the best colorspace metadata is
> selected */
> + swap_color_spaces(graph);
> + swap_color_ranges(graph);
> +
> /* for audio filters, ensure the best format, sample rate and channel
> layout
> * is selected */
> swap_sample_fmts(graph);
> diff --git a/libavfilter/formats.c b/libavfilter/formats.c
> index d1c97daf64..20f00d2db4 100644
> --- a/libavfilter/formats.c
> +++ b/libavfilter/formats.c
> @@ -321,12 +321,46 @@ static int merge_channel_layouts(void *a, void *b)
> return merge_channel_layouts_internal(a, b, 0);
> }
>
> +static int merge_generic_internal(AVFilterFormats *a,
> + AVFilterFormats *b, int check)
> +{
> + av_assert2(check || (a->refcount && b->refcount));
> +
> + if (a == b)
> + return 1;
> +
> + MERGE_FORMATS(a, b, formats, nb_formats, AVFilterFormats, check, 0);
> +
> + return 1;
> +}
> +
> +static int can_merge_generic(const void *a, const void *b)
> +{
> + return merge_generic_internal((AVFilterFormats *)a,
> + (AVFilterFormats *)b, 1);
> +}
> +
> +static int merge_generic(void *a, void *b)
> +{
> + return merge_generic_internal(a, b, 0);
> +}
> +
> static const AVFilterFormatsMerger mergers_video[] = {
> {
> .offset = offsetof(AVFilterFormatsConfig, formats),
> .merge = merge_pix_fmts,
> .can_merge = can_merge_pix_fmts,
> },
> + {
> + .offset = offsetof(AVFilterFormatsConfig, color_spaces),
> + .merge = merge_generic,
> + .can_merge = can_merge_generic,
> + },
> + {
> + .offset = offsetof(AVFilterFormatsConfig, color_ranges),
> + .merge = merge_generic,
> + .can_merge = can_merge_generic,
> + },
> };
>
> static const AVFilterFormatsMerger mergers_audio[] = {
> @@ -594,6 +628,33 @@ AVFilterChannelLayouts *ff_all_channel_counts(void)
> return ret;
> }
>
> +AVFilterFormats *ff_all_color_spaces(void)
> +{
> + AVFilterFormats *ret = NULL;
> + if (ff_add_format(&ret, AVCOL_SPC_UNSPECIFIED) < 0)
> + return NULL;
> + for (int csp = 0; csp < AVCOL_SPC_NB; csp++) {
> + if (csp == AVCOL_SPC_RESERVED ||
> + csp == AVCOL_SPC_UNSPECIFIED)
> + continue;
> + if (ff_add_format(&ret, csp) < 0)
> + return NULL;
> + }
> +
> + return ret;
> +}
> +
> +AVFilterFormats *ff_all_color_ranges(void)
> +{
> + AVFilterFormats *ret = NULL;
> + for (int range = 0; range < AVCOL_RANGE_NB; range++) {
> + if (ff_add_format(&ret, range) < 0)
> + return NULL;
> + }
> +
> + return ret;
> +}
> +
> #define FORMATS_REF(f, ref,
> unref_fn) \
> void
> *tmp; \
>
> \
> @@ -763,6 +824,42 @@ int ff_set_common_all_samplerates(AVFilterContext *ctx)
> return ff_set_common_samplerates(ctx, ff_all_samplerates());
> }
>
> +int ff_set_common_color_spaces(AVFilterContext *ctx,
> + AVFilterFormats *color_spaces)
> +{
> + SET_COMMON_FORMATS(ctx, color_spaces, AVMEDIA_TYPE_VIDEO,
> + ff_formats_ref, ff_formats_unref);
> +}
> +
> +int ff_set_common_color_spaces_from_list(AVFilterContext *ctx,
> + const int *color_ranges)
> +{
> + return ff_set_common_color_spaces(ctx,
> ff_make_format_list(color_ranges));
> +}
> +
> +int ff_set_common_all_color_spaces(AVFilterContext *ctx)
> +{
> + return ff_set_common_color_spaces(ctx, ff_all_color_spaces());
> +}
> +
> +int ff_set_common_color_ranges(AVFilterContext *ctx,
> + AVFilterFormats *color_ranges)
> +{
> + SET_COMMON_FORMATS(ctx, color_ranges, AVMEDIA_TYPE_VIDEO,
> + ff_formats_ref, ff_formats_unref);
> +}
> +
> +int ff_set_common_color_ranges_from_list(AVFilterContext *ctx,
> + const int *color_ranges)
> +{
> + return ff_set_common_color_ranges(ctx,
> ff_make_format_list(color_ranges));
> +}
> +
> +int ff_set_common_all_color_ranges(AVFilterContext *ctx)
> +{
> + return ff_set_common_color_ranges(ctx, ff_all_color_ranges());
> +}
> +
> /**
> * A helper for query_formats() which sets all links to the same list of
> * formats. If there are no links hooked to this filter, the list of formats
> is
> @@ -817,7 +914,14 @@ int ff_default_query_formats(AVFilterContext *ctx)
> ret = ff_set_common_formats(ctx, formats);
> if (ret < 0)
> return ret;
> - if (type == AVMEDIA_TYPE_AUDIO) {
> + if (type == AVMEDIA_TYPE_VIDEO) {
> + ret = ff_set_common_all_color_spaces(ctx);
> + if (ret < 0)
> + return ret;
> + ret = ff_set_common_all_color_ranges(ctx);
> + if (ret < 0)
> + return ret;
> + } else if (type == AVMEDIA_TYPE_AUDIO) {
> ret = ff_set_common_all_channel_counts(ctx);
> if (ret < 0)
> return ret;
> @@ -935,6 +1039,22 @@ int ff_formats_check_sample_rates(void *log, const
> AVFilterFormats *fmts)
> return check_list(log, "sample rate", fmts);
> }
>
> +int ff_formats_check_color_spaces(void *log, const AVFilterFormats *fmts)
> +{
> + for (int i = 0; fmts && i < fmts->nb_formats; i++) {
> + if (fmts->formats[i] == AVCOL_SPC_RESERVED) {
> + av_log(log, AV_LOG_ERROR, "Invalid color range\n");
> + return AVERROR(EINVAL);
> + }
> + }
> + return check_list(log, "color space", fmts);
> +}
> +
> +int ff_formats_check_color_ranges(void *log, const AVFilterFormats *fmts)
> +{
> + return check_list(log, "color range", fmts);
> +}
> +
> static int layouts_compatible(const AVChannelLayout *a, const AVChannelLayout
> *b)
> {
> return !av_channel_layout_compare(a, b) ||
> diff --git a/libavfilter/formats.h b/libavfilter/formats.h
> index d44890109e..82b3af4be1 100644
> --- a/libavfilter/formats.h
> +++ b/libavfilter/formats.h
> @@ -130,6 +130,20 @@ AVFilterChannelLayouts *ff_all_channel_counts(void);
> av_warn_unused_result
> AVFilterChannelLayouts *ff_make_channel_layout_list(const AVChannelLayout
> *fmts);
>
> +/**
> + * Construct an AVFilterFormats representing all possible color spaces.
> + *
> + * Note: This list does not include AVCOL_SPC_RESERVED.
> + */
> +av_warn_unused_result
> +AVFilterFormats *ff_all_color_spaces(void);
> +
> +/**
> + * Construct an AVFilterFormats representing all possible color ranges.
> + */
> +av_warn_unused_result
> +AVFilterFormats *ff_all_color_ranges(void);
> +
> /**
> * Helpers for query_formats() which set all free audio links to the same
> list
> * of channel layouts/sample rates. If there are no links hooked to this
> list,
> @@ -165,6 +179,38 @@ int ff_set_common_samplerates_from_list(AVFilterContext
> *ctx,
> av_warn_unused_result
> int ff_set_common_all_samplerates(AVFilterContext *ctx);
>
> +av_warn_unused_result
> +int ff_set_common_color_spaces(AVFilterContext *ctx,
> + AVFilterFormats *color_spaces);
> +/**
> + * Equivalent to ff_set_common_color_spaces(ctx,
> ff_make_format_list(color_spaces))
> + */
> +av_warn_unused_result
> +int ff_set_common_color_spaces_from_list(AVFilterContext *ctx,
> + const int *color_spaces);
> +
> +/**
> + * Equivalent to ff_set_common_color_spaces(ctx, ff_all_color_spaces())
> + */
> +av_warn_unused_result
> +int ff_set_common_all_color_spaces(AVFilterContext *ctx);
> +
> +av_warn_unused_result
> +int ff_set_common_color_ranges(AVFilterContext *ctx,
> + AVFilterFormats *color_ranges);
> +/**
> + * Equivalent to ff_set_common_color_ranges(ctx,
> ff_make_format_list(color_ranges))
> + */
> +av_warn_unused_result
> +int ff_set_common_color_ranges_from_list(AVFilterContext *ctx,
> + const int *color_ranges);
> +
> +/**
> + * Equivalent to ff_set_common_color_ranges(ctx, ff_all_color_ranges())
> + */
> +av_warn_unused_result
> +int ff_set_common_all_color_ranges(AVFilterContext *ctx);
> +
> /**
> * A helper for query_formats() which sets all links to the same list of
> * formats. If there are no links hooked to this filter, the list of formats
> is
> @@ -328,6 +374,14 @@ int ff_formats_check_sample_rates(void *log, const
> AVFilterFormats *fmts);
> */
> int ff_formats_check_channel_layouts(void *log, const AVFilterChannelLayouts
> *fmts);
>
> +/**
> + * Check that fmts is a valid formats list for YUV colorspace metadata.
> + *
> + * In particular, check for duplicates.
> + */
> +int ff_formats_check_color_spaces(void *log, const AVFilterFormats *fmts);
> +int ff_formats_check_color_ranges(void *log, const AVFilterFormats *fmts);
> +
> typedef struct AVFilterFormatMerger {
> unsigned offset;
> int (*merge)(void *a, void *b);
> diff --git a/libavfilter/internal.h b/libavfilter/internal.h
> index 2dbc5def0a..a6cdf9994c 100644
> --- a/libavfilter/internal.h
> +++ b/libavfilter/internal.h
> @@ -203,6 +203,12 @@ enum FilterFormatsState {
> */
> int ff_fmt_is_in(int fmt, const int *fmts);
>
> +/**
> + * Returns true if a pixel format is "regular YUV", which includes all pixel
> + * formats that are affected by YUV colorspace negotiation.
> + */
> +int ff_fmt_is_regular_yuv(enum AVPixelFormat fmt);
> +
> /* Functions to parse audio format arguments */
>
> /**
> diff --git a/libavfilter/vaapi_vpp.c b/libavfilter/vaapi_vpp.c
> index cf2592e068..59961bfa4a 100644
> --- a/libavfilter/vaapi_vpp.c
> +++ b/libavfilter/vaapi_vpp.c
> @@ -38,6 +38,10 @@ int ff_vaapi_vpp_query_formats(AVFilterContext *avctx)
> &avctx->outputs[0]->incfg.formats)) < 0)
> return err;
>
> + if ((err = ff_set_common_all_color_spaces(avctx)) < 0 ||
> + (err = ff_set_common_all_color_ranges(avctx)) < 0)
> + return err;
> +
> return 0;
> }
>
> diff --git a/libavfilter/video.c b/libavfilter/video.c
> index 42eeb98c28..243762c8fd 100644
> --- a/libavfilter/video.c
> +++ b/libavfilter/video.c
> @@ -96,6 +96,8 @@ AVFrame *ff_default_get_video_buffer2(AVFilterLink *link,
> int w, int h, int alig
> return NULL;
>
> frame->sample_aspect_ratio = link->sample_aspect_ratio;
> + frame->colorspace = link->colorspace;
> + frame->color_range = link->color_range;
>
> return frame;
> }
> diff --git a/tests/ref/fate/shortest-sub b/tests/ref/fate/shortest-sub
> index 9caee587ce..16b1324e83 100644
> --- a/tests/ref/fate/shortest-sub
> +++ b/tests/ref/fate/shortest-sub
> @@ -1,5 +1,5 @@
> -73d142a80965f9e0884a5863abde0dab *tests/data/fate/shortest-sub.matroska
> -139249 tests/data/fate/shortest-sub.matroska
> +d6608277c93097383e62388196dc62f0 *tests/data/fate/shortest-sub.matroska
> +139260 tests/data/fate/shortest-sub.matroska
> #extradata 1: 167, 0xf7272d5f
> #tb 0: 1/1000
> #media_type 0: video
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
^ permalink raw reply [flat|nested] 26+ messages in thread
* [FFmpeg-devel] [PATCH v2 04/15] fftools/ffmpeg_filter: don't clear buffersrc params
2023-12-13 13:11 [FFmpeg-devel] [PATCH v2 00/15] YUV colorspace filter negotiation Niklas Haas
` (2 preceding siblings ...)
2023-12-13 13:12 ` [FFmpeg-devel] [PATCH v2 03/15] avfilter: add negotiation API for color space/range Niklas Haas
@ 2023-12-13 13:12 ` Niklas Haas
2023-12-13 13:12 ` [FFmpeg-devel] [PATCH v2 05/15] avfilter/buffersrc: add color_space/range parameters Niklas Haas
` (11 subsequent siblings)
15 siblings, 0 replies; 26+ messages in thread
From: Niklas Haas @ 2023-12-13 13:12 UTC (permalink / raw)
To: ffmpeg-devel; +Cc: Niklas Haas
From: Niklas Haas <git@haasn.dev>
This memset is bogus, it accomplishes nothing in the best case and
regresses future additions to AVBufferSrcParameters in the worst case.
---
fftools/ffmpeg_filter.c | 3 ---
1 file changed, 3 deletions(-)
diff --git a/fftools/ffmpeg_filter.c b/fftools/ffmpeg_filter.c
index ada235b084..e5b00123f2 100644
--- a/fftools/ffmpeg_filter.c
+++ b/fftools/ffmpeg_filter.c
@@ -1461,11 +1461,8 @@ static int configure_input_video_filter(FilterGraph *fg, InputFilter *ifilter,
int ret, pad_idx = 0;
int64_t tsoffset = 0;
AVBufferSrcParameters *par = av_buffersrc_parameters_alloc();
-
if (!par)
return AVERROR(ENOMEM);
- memset(par, 0, sizeof(*par));
- par->format = AV_PIX_FMT_NONE;
if (ist->dec_ctx->codec_type == AVMEDIA_TYPE_AUDIO) {
av_log(fg, AV_LOG_ERROR, "Cannot connect video filter to audio input\n");
--
2.43.0
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
^ permalink raw reply [flat|nested] 26+ messages in thread* [FFmpeg-devel] [PATCH v2 05/15] avfilter/buffersrc: add color_space/range parameters
2023-12-13 13:11 [FFmpeg-devel] [PATCH v2 00/15] YUV colorspace filter negotiation Niklas Haas
` (3 preceding siblings ...)
2023-12-13 13:12 ` [FFmpeg-devel] [PATCH v2 04/15] fftools/ffmpeg_filter: don't clear buffersrc params Niklas Haas
@ 2023-12-13 13:12 ` Niklas Haas
2023-12-13 13:12 ` [FFmpeg-devel] [PATCH v2 06/15] fftools/ffmpeg_filter: configure buffersrc with csp/range Niklas Haas
` (10 subsequent siblings)
15 siblings, 0 replies; 26+ messages in thread
From: Niklas Haas @ 2023-12-13 13:12 UTC (permalink / raw)
To: ffmpeg-devel; +Cc: Niklas Haas
From: Niklas Haas <git@haasn.dev>
To allow adding proper negotiation, in particular, to fftools.
These values will simply be negotiated downstream for YUV formats, and
ignored otherwise.
---
doc/filters.texi | 10 ++++++
libavfilter/buffersrc.c | 74 ++++++++++++++++++++++++++++++++++++-----
libavfilter/buffersrc.h | 6 ++++
3 files changed, 82 insertions(+), 8 deletions(-)
diff --git a/doc/filters.texi b/doc/filters.texi
index 6d00ba2c3f..f182dc2ddb 100644
--- a/doc/filters.texi
+++ b/doc/filters.texi
@@ -28025,6 +28025,16 @@ Specify the timebase assumed by the timestamps of the buffered frames.
@item frame_rate
Specify the frame rate expected for the video stream.
+@item colorspace
+A string representing the color space of the buffered video frames.
+It may be a number corresponding to a color space, or a color space
+name.
+
+@item range
+A string representing the color range of the buffered video frames.
+It may be a number corresponding to a color range, or a color range
+name.
+
@item pixel_aspect, sar
The sample (pixel) aspect ratio of the input video.
diff --git a/libavfilter/buffersrc.c b/libavfilter/buffersrc.c
index b0a905d455..1c5db94696 100644
--- a/libavfilter/buffersrc.c
+++ b/libavfilter/buffersrc.c
@@ -28,6 +28,7 @@
#include "libavutil/channel_layout.h"
#include "libavutil/common.h"
#include "libavutil/frame.h"
+#include "libavutil/hwcontext.h"
#include "libavutil/imgutils.h"
#include "libavutil/internal.h"
#include "libavutil/opt.h"
@@ -50,6 +51,8 @@ typedef struct BufferSourceContext {
/* video only */
int w, h;
enum AVPixelFormat pix_fmt;
+ enum AVColorSpace color_space;
+ enum AVColorRange color_range;
AVRational pixel_aspect;
AVBufferRef *hw_frames_ctx;
@@ -65,10 +68,13 @@ typedef struct BufferSourceContext {
int64_t last_pts;
} BufferSourceContext;
-#define CHECK_VIDEO_PARAM_CHANGE(s, c, width, height, format, pts)\
- if (c->w != width || c->h != height || c->pix_fmt != format) {\
- av_log(s, AV_LOG_INFO, "filter context - w: %d h: %d fmt: %d, incoming frame - w: %d h: %d fmt: %d pts_time: %s\n",\
- c->w, c->h, c->pix_fmt, width, height, format, av_ts2timestr(pts, &s->outputs[0]->time_base));\
+#define CHECK_VIDEO_PARAM_CHANGE(s, c, width, height, format, csp, range, pts)\
+ if (c->w != width || c->h != height || c->pix_fmt != format ||\
+ c->color_space != csp || c->color_range != range) {\
+ av_log(s, AV_LOG_INFO, "filter context - w: %d h: %d fmt: %d csp: %s range: %s, incoming frame - w: %d h: %d fmt: %d csp: %s range: %s pts_time: %s\n",\
+ c->w, c->h, c->pix_fmt, av_color_space_name(c->color_space), av_color_range_name(c->color_range),\
+ width, height, format, av_color_space_name(csp), av_color_range_name(range),\
+ av_ts2timestr(pts, &s->outputs[0]->time_base));\
av_log(s, AV_LOG_WARNING, "Changing video frame properties on the fly is not supported by all filters.\n");\
}
@@ -89,6 +95,8 @@ AVBufferSrcParameters *av_buffersrc_parameters_alloc(void)
return NULL;
par->format = -1;
+ par->color_range = AVCOL_RANGE_UNSPECIFIED;
+ par->color_space = AVCOL_SPC_UNSPECIFIED;
return par;
}
@@ -119,6 +127,10 @@ int av_buffersrc_parameters_set(AVFilterContext *ctx, AVBufferSrcParameters *par
if (!s->hw_frames_ctx)
return AVERROR(ENOMEM);
}
+ if (param->color_space != AVCOL_SPC_UNSPECIFIED)
+ s->color_space = param->color_space;
+ if (param->color_range != AVCOL_RANGE_UNSPECIFIED)
+ s->color_range = param->color_range;
break;
case AVMEDIA_TYPE_AUDIO:
if (param->format != AV_SAMPLE_FMT_NONE) {
@@ -206,7 +218,8 @@ FF_ENABLE_DEPRECATION_WARNINGS
switch (ctx->outputs[0]->type) {
case AVMEDIA_TYPE_VIDEO:
CHECK_VIDEO_PARAM_CHANGE(ctx, s, frame->width, frame->height,
- frame->format, frame->pts);
+ frame->format, frame->colorspace,
+ frame->color_range, frame->pts);
break;
case AVMEDIA_TYPE_AUDIO:
/* For layouts unknown on input but known on link after negotiation. */
@@ -303,10 +316,11 @@ static av_cold int init_video(AVFilterContext *ctx)
return AVERROR(EINVAL);
}
- av_log(ctx, AV_LOG_VERBOSE, "w:%d h:%d pixfmt:%s tb:%d/%d fr:%d/%d sar:%d/%d\n",
+ av_log(ctx, AV_LOG_VERBOSE, "w:%d h:%d pixfmt:%s tb:%d/%d fr:%d/%d sar:%d/%d csp:%s range:%s\n",
c->w, c->h, av_get_pix_fmt_name(c->pix_fmt),
c->time_base.num, c->time_base.den, c->frame_rate.num, c->frame_rate.den,
- c->pixel_aspect.num, c->pixel_aspect.den);
+ c->pixel_aspect.num, c->pixel_aspect.den,
+ av_color_space_name(c->color_space), av_color_range_name(c->color_range));
return 0;
}
@@ -329,6 +343,30 @@ static const AVOption buffer_options[] = {
{ "pixel_aspect", "sample aspect ratio", OFFSET(pixel_aspect), AV_OPT_TYPE_RATIONAL, { .dbl = 0 }, 0, DBL_MAX, V },
{ "time_base", NULL, OFFSET(time_base), AV_OPT_TYPE_RATIONAL, { .dbl = 0 }, 0, DBL_MAX, V },
{ "frame_rate", NULL, OFFSET(frame_rate), AV_OPT_TYPE_RATIONAL, { .dbl = 0 }, 0, DBL_MAX, V },
+ { "colorspace", "select colorspace", OFFSET(color_space), AV_OPT_TYPE_INT, {.i64=AVCOL_SPC_UNSPECIFIED}, 0, AVCOL_SPC_NB-1, V, "colorspace"},
+ { "gbr", NULL, 0, AV_OPT_TYPE_CONST, {.i64=AVCOL_SPC_RGB}, INT_MIN, INT_MAX, V, "colorspace"},
+ { "bt709", NULL, 0, AV_OPT_TYPE_CONST, {.i64=AVCOL_SPC_BT709}, INT_MIN, INT_MAX, V, "colorspace"},
+ { "unknown", NULL, 0, AV_OPT_TYPE_CONST, {.i64=AVCOL_SPC_UNSPECIFIED}, INT_MIN, INT_MAX, V, "colorspace"},
+ { "fcc", NULL, 0, AV_OPT_TYPE_CONST, {.i64=AVCOL_SPC_FCC}, INT_MIN, INT_MAX, V, "colorspace"},
+ { "bt470bg", NULL, 0, AV_OPT_TYPE_CONST, {.i64=AVCOL_SPC_BT470BG}, INT_MIN, INT_MAX, V, "colorspace"},
+ { "smpte170m", NULL, 0, AV_OPT_TYPE_CONST, {.i64=AVCOL_SPC_SMPTE170M}, INT_MIN, INT_MAX, V, "colorspace"},
+ { "smpte240m", NULL, 0, AV_OPT_TYPE_CONST, {.i64=AVCOL_SPC_SMPTE240M}, INT_MIN, INT_MAX, V, "colorspace"},
+ { "ycgco", NULL, 0, AV_OPT_TYPE_CONST, {.i64=AVCOL_SPC_YCGCO}, INT_MIN, INT_MAX, V, "colorspace"},
+ { "bt2020nc", NULL, 0, AV_OPT_TYPE_CONST, {.i64=AVCOL_SPC_BT2020_NCL}, INT_MIN, INT_MAX, V, "colorspace"},
+ { "bt2020c", NULL, 0, AV_OPT_TYPE_CONST, {.i64=AVCOL_SPC_BT2020_CL}, INT_MIN, INT_MAX, V, "colorspace"},
+ { "smpte2085", NULL, 0, AV_OPT_TYPE_CONST, {.i64=AVCOL_SPC_SMPTE2085}, INT_MIN, INT_MAX, V, "colorspace"},
+ { "chroma-derived-nc", NULL, 0, AV_OPT_TYPE_CONST, {.i64=AVCOL_SPC_CHROMA_DERIVED_NCL},INT_MIN, INT_MAX, V, "colorspace"},
+ { "chroma-derived-c", NULL, 0, AV_OPT_TYPE_CONST, {.i64=AVCOL_SPC_CHROMA_DERIVED_CL}, INT_MIN, INT_MAX, V, "colorspace"},
+ { "ictcp", NULL, 0, AV_OPT_TYPE_CONST, {.i64=AVCOL_SPC_ICTCP}, INT_MIN, INT_MAX, V, "colorspace"},
+ { "range", "select color range", OFFSET(color_range), AV_OPT_TYPE_INT, {.i64=AVCOL_RANGE_UNSPECIFIED}, 0, AVCOL_RANGE_NB-1, V, "range"},
+ { "unspecified", NULL, 0, AV_OPT_TYPE_CONST, {.i64=AVCOL_RANGE_UNSPECIFIED}, 0, 0, V, "range"},
+ { "unknown", NULL, 0, AV_OPT_TYPE_CONST, {.i64=AVCOL_RANGE_UNSPECIFIED}, 0, 0, V, "range"},
+ { "limited", NULL, 0, AV_OPT_TYPE_CONST, {.i64=AVCOL_RANGE_MPEG}, 0, 0, V, "range"},
+ { "tv", NULL, 0, AV_OPT_TYPE_CONST, {.i64=AVCOL_RANGE_MPEG}, 0, 0, V, "range"},
+ { "mpeg", NULL, 0, AV_OPT_TYPE_CONST, {.i64=AVCOL_RANGE_MPEG}, 0, 0, V, "range"},
+ { "full", NULL, 0, AV_OPT_TYPE_CONST, {.i64=AVCOL_RANGE_JPEG}, 0, 0, V, "range"},
+ { "pc", NULL, 0, AV_OPT_TYPE_CONST, {.i64=AVCOL_RANGE_JPEG}, 0, 0, V, "range"},
+ { "jpeg", NULL, 0, AV_OPT_TYPE_CONST, {.i64=AVCOL_RANGE_JPEG}, 0, 0, V, "range"},
{ NULL },
};
@@ -426,13 +464,33 @@ static int query_formats(AVFilterContext *ctx)
AVFilterChannelLayouts *channel_layouts = NULL;
AVFilterFormats *formats = NULL;
AVFilterFormats *samplerates = NULL;
+ AVFilterFormats *color_spaces = NULL;
+ AVFilterFormats *color_ranges = NULL;
int ret;
switch (ctx->outputs[0]->type) {
- case AVMEDIA_TYPE_VIDEO:
+ case AVMEDIA_TYPE_VIDEO:;
+ enum AVPixelFormat swfmt = c->pix_fmt;
+ if (av_pix_fmt_desc_get(swfmt)->flags & AV_PIX_FMT_FLAG_HWACCEL) {
+ if (!c->hw_frames_ctx) {
+ av_log(ctx, AV_LOG_ERROR, "Setting BufferSourceContext.pix_fmt "
+ "to a HW format requires hw_frames_ctx to be non-NULL!\n");
+ return AVERROR(EINVAL);
+ }
+ swfmt = ((AVHWFramesContext *) c->hw_frames_ctx->data)->sw_format;
+ }
if ((ret = ff_add_format (&formats, c->pix_fmt)) < 0 ||
(ret = ff_set_common_formats (ctx , formats )) < 0)
return ret;
+ /* force specific colorspace/range downstream only for ordinary YUV */
+ if (ff_fmt_is_regular_yuv(swfmt)) {
+ if ((ret = ff_add_format(&color_spaces, c->color_space)) < 0 ||
+ (ret = ff_set_common_color_spaces(ctx, color_spaces)) < 0)
+ return ret;
+ if ((ret = ff_add_format(&color_ranges, c->color_range)) < 0 ||
+ (ret = ff_set_common_color_ranges(ctx, color_ranges)) < 0)
+ return ret;
+ }
break;
case AVMEDIA_TYPE_AUDIO:
if ((ret = ff_add_format (&formats , c->sample_fmt )) < 0 ||
diff --git a/libavfilter/buffersrc.h b/libavfilter/buffersrc.h
index 3b248b37cd..1e29f0340e 100644
--- a/libavfilter/buffersrc.h
+++ b/libavfilter/buffersrc.h
@@ -123,6 +123,12 @@ typedef struct AVBufferSrcParameters {
* Audio only, the audio channel layout
*/
AVChannelLayout ch_layout;
+
+ /**
+ * Video only, the YUV colorspace and range.
+ */
+ enum AVColorSpace color_space;
+ enum AVColorRange color_range;
} AVBufferSrcParameters;
/**
--
2.43.0
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
^ permalink raw reply [flat|nested] 26+ messages in thread* [FFmpeg-devel] [PATCH v2 06/15] fftools/ffmpeg_filter: configure buffersrc with csp/range
2023-12-13 13:11 [FFmpeg-devel] [PATCH v2 00/15] YUV colorspace filter negotiation Niklas Haas
` (4 preceding siblings ...)
2023-12-13 13:12 ` [FFmpeg-devel] [PATCH v2 05/15] avfilter/buffersrc: add color_space/range parameters Niklas Haas
@ 2023-12-13 13:12 ` Niklas Haas
2023-12-13 13:12 ` [FFmpeg-devel] [PATCH v2 07/15] vf_scale: use colorspace negotiation API Niklas Haas
` (9 subsequent siblings)
15 siblings, 0 replies; 26+ messages in thread
From: Niklas Haas @ 2023-12-13 13:12 UTC (permalink / raw)
To: ffmpeg-devel; +Cc: Niklas Haas
From: Niklas Haas <git@haasn.dev>
Propagates input metadata to the input filter graph.
---
fftools/ffmpeg_filter.c | 24 +++++++++++++++++++++---
1 file changed, 21 insertions(+), 3 deletions(-)
diff --git a/fftools/ffmpeg_filter.c b/fftools/ffmpeg_filter.c
index e5b00123f2..2da6d307a0 100644
--- a/fftools/ffmpeg_filter.c
+++ b/fftools/ffmpeg_filter.c
@@ -126,6 +126,8 @@ typedef struct InputFilterPriv {
int width, height;
AVRational sample_aspect_ratio;
+ enum AVColorSpace color_space;
+ enum AVColorRange color_range;
int sample_rate;
AVChannelLayout ch_layout;
@@ -146,6 +148,8 @@ typedef struct InputFilterPriv {
int width;
int height;
AVRational sample_aspect_ratio;
+ enum AVColorSpace color_space;
+ enum AVColorRange color_range;
int sample_rate;
AVChannelLayout ch_layout;
@@ -257,6 +261,8 @@ static int sub2video_get_blank_frame(InputFilterPriv *ifp)
frame->width = ifp->width;
frame->height = ifp->height;
frame->format = ifp->format;
+ frame->colorspace = ifp->color_space;
+ frame->color_range = ifp->color_range;
ret = av_frame_get_buffer(frame, 0);
if (ret < 0)
@@ -833,6 +839,8 @@ static InputFilter *ifilter_alloc(FilterGraph *fg)
ifp->index = fg->nb_inputs - 1;
ifp->format = -1;
ifp->fallback.format = -1;
+ ifp->color_space = ifp->fallback.color_space = AVCOL_SPC_UNSPECIFIED;
+ ifp->color_range = ifp->fallback.color_range = AVCOL_RANGE_UNSPECIFIED;
ifp->frame_queue = av_fifo_alloc2(8, sizeof(AVFrame*), AV_FIFO_FLAG_AUTO_GROW);
if (!ifp->frame_queue)
@@ -1485,9 +1493,11 @@ static int configure_input_video_filter(FilterGraph *fg, InputFilter *ifilter,
av_bprint_init(&args, 0, AV_BPRINT_SIZE_AUTOMATIC);
av_bprintf(&args,
"video_size=%dx%d:pix_fmt=%d:time_base=%d/%d:"
- "pixel_aspect=%d/%d",
+ "pixel_aspect=%d/%d:colorspace=%s:range=%s",
ifp->width, ifp->height, ifp->format,
- ifp->time_base.num, ifp->time_base.den, sar.num, sar.den);
+ ifp->time_base.num, ifp->time_base.den, sar.num, sar.den,
+ av_color_space_name(ifp->color_space),
+ av_color_range_name(ifp->color_range));
if (fr.num && fr.den)
av_bprintf(&args, ":frame_rate=%d/%d", fr.num, fr.den);
snprintf(name, sizeof(name), "graph %d input from stream %d:%d", fg->index,
@@ -1831,6 +1841,8 @@ int ifilter_parameters_from_dec(InputFilter *ifilter, const AVCodecContext *dec)
ifp->fallback.width = dec->width;
ifp->fallback.height = dec->height;
ifp->fallback.sample_aspect_ratio = dec->sample_aspect_ratio;
+ ifp->fallback.color_space = dec->colorspace;
+ ifp->fallback.color_range = dec->color_range;
} else if (dec->codec_type == AVMEDIA_TYPE_AUDIO) {
int ret;
@@ -1871,6 +1883,8 @@ static int ifilter_parameters_from_frame(InputFilter *ifilter, const AVFrame *fr
ifp->width = frame->width;
ifp->height = frame->height;
ifp->sample_aspect_ratio = frame->sample_aspect_ratio;
+ ifp->color_space = frame->colorspace;
+ ifp->color_range = frame->color_range;
ifp->sample_rate = frame->sample_rate;
ret = av_channel_layout_copy(&ifp->ch_layout, &frame->ch_layout);
@@ -2543,6 +2557,8 @@ static int send_eof(FilterGraphThread *fgt, InputFilter *ifilter,
ifp->width = ifp->fallback.width;
ifp->height = ifp->fallback.height;
ifp->sample_aspect_ratio = ifp->fallback.sample_aspect_ratio;
+ ifp->color_space = ifp->fallback.color_space;
+ ifp->color_range = ifp->fallback.color_range;
ret = av_channel_layout_copy(&ifp->ch_layout,
&ifp->fallback.ch_layout);
@@ -2586,7 +2602,9 @@ static int send_frame(FilterGraph *fg, FilterGraphThread *fgt,
break;
case AVMEDIA_TYPE_VIDEO:
need_reinit |= ifp->width != frame->width ||
- ifp->height != frame->height;
+ ifp->height != frame->height ||
+ ifp->color_space != frame->colorspace ||
+ ifp->color_range != frame->color_range;
break;
}
--
2.43.0
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
^ permalink raw reply [flat|nested] 26+ messages in thread* [FFmpeg-devel] [PATCH v2 07/15] vf_scale: use colorspace negotiation API
2023-12-13 13:11 [FFmpeg-devel] [PATCH v2 00/15] YUV colorspace filter negotiation Niklas Haas
` (5 preceding siblings ...)
2023-12-13 13:12 ` [FFmpeg-devel] [PATCH v2 06/15] fftools/ffmpeg_filter: configure buffersrc with csp/range Niklas Haas
@ 2023-12-13 13:12 ` Niklas Haas
2023-12-13 13:12 ` [FFmpeg-devel] [PATCH v2 08/15] avfilter/vf_zscale: remove unused variables Niklas Haas
` (8 subsequent siblings)
15 siblings, 0 replies; 26+ messages in thread
From: Niklas Haas @ 2023-12-13 13:12 UTC (permalink / raw)
To: ffmpeg-devel; +Cc: Niklas Haas
From: Niklas Haas <git@haasn.dev>
This filter will always accept any input format, even if the user sets
a specific in_range/in_color_matrix. This is to preserve status quo with
current behavior, where passing a specific in_color_matrix merely
overrides the incoming frames' attributes. (Use `vf_format` to force
a specific input range)
Because changing colorspace and color_range now requires reconfiguring
the link, we can lift sws_setColorspaceDetails out of scale_frame and
into config_props. (This will also get re-called if the input frame
properties change)
---
libavfilter/vf_scale.c | 188 ++++++++++++++++++++++-------------------
1 file changed, 103 insertions(+), 85 deletions(-)
diff --git a/libavfilter/vf_scale.c b/libavfilter/vf_scale.c
index b45035907b..44dc1f2bf8 100644
--- a/libavfilter/vf_scale.c
+++ b/libavfilter/vf_scale.c
@@ -143,7 +143,6 @@ typedef struct ScaleContext {
int out_color_matrix;
int in_range;
- int in_frame_range;
int out_range;
int out_h_chr_pos;
@@ -292,6 +291,18 @@ static av_cold int preinit(AVFilterContext *ctx)
return 0;
}
+static const int sws_colorspaces[] = {
+ AVCOL_SPC_UNSPECIFIED,
+ AVCOL_SPC_RGB,
+ AVCOL_SPC_BT709,
+ AVCOL_SPC_BT470BG,
+ AVCOL_SPC_SMPTE170M,
+ AVCOL_SPC_FCC,
+ AVCOL_SPC_SMPTE240M,
+ AVCOL_SPC_BT2020_NCL,
+ -1
+};
+
static av_cold int init(AVFilterContext *ctx)
{
ScaleContext *scale = ctx->priv;
@@ -332,6 +343,19 @@ static av_cold int init(AVFilterContext *ctx)
if (ret < 0)
return ret;
+ if (scale->in_color_matrix != -1 &&
+ !ff_fmt_is_in(scale->in_color_matrix, sws_colorspaces)) {
+ av_log(ctx, AV_LOG_ERROR, "Unsupported input color matrix '%s'\n",
+ av_color_space_name(scale->in_color_matrix));
+ return AVERROR(EINVAL);
+ }
+
+ if (!ff_fmt_is_in(scale->out_color_matrix, sws_colorspaces)) {
+ av_log(ctx, AV_LOG_ERROR, "Unsupported output color matrix '%s'\n",
+ av_color_space_name(scale->out_color_matrix));
+ return AVERROR(EINVAL);
+ }
+
av_log(ctx, AV_LOG_VERBOSE, "w:%s h:%s flags:'%s' interl:%d\n",
scale->w_expr, scale->h_expr, (char *)av_x_if_null(scale->flags_str, ""), scale->interlaced);
@@ -356,8 +380,6 @@ static av_cold int init(AVFilterContext *ctx)
if (!threads)
av_opt_set_int(scale->sws_opts, "threads", ff_filter_get_nb_threads(ctx), 0);
- scale->in_frame_range = AVCOL_RANGE_UNSPECIFIED;
-
return 0;
}
@@ -376,6 +398,7 @@ static av_cold void uninit(AVFilterContext *ctx)
static int query_formats(AVFilterContext *ctx)
{
+ ScaleContext *scale = ctx->priv;
AVFilterFormats *formats;
const AVPixFmtDescriptor *desc;
enum AVPixelFormat pix_fmt;
@@ -407,6 +430,28 @@ static int query_formats(AVFilterContext *ctx)
if ((ret = ff_formats_ref(formats, &ctx->outputs[0]->incfg.formats)) < 0)
return ret;
+ /* accept all supported inputs, even if user overrides their properties */
+ if ((ret = ff_formats_ref(ff_make_format_list(sws_colorspaces),
+ &ctx->inputs[0]->outcfg.color_spaces)) < 0)
+ return ret;
+
+ if ((ret = ff_formats_ref(ff_all_color_ranges(),
+ &ctx->inputs[0]->outcfg.color_ranges)) < 0)
+ return ret;
+
+ /* propagate output properties if overridden */
+ formats = scale->out_color_matrix != AVCOL_SPC_UNSPECIFIED
+ ? ff_make_formats_list_singleton(scale->out_color_matrix)
+ : ff_make_format_list(sws_colorspaces);
+ if ((ret = ff_formats_ref(formats, &ctx->outputs[0]->incfg.color_spaces)) < 0)
+ return ret;
+
+ formats = scale->out_range != AVCOL_RANGE_UNSPECIFIED
+ ? ff_make_formats_list_singleton(scale->out_range)
+ : ff_all_color_ranges();
+ if ((ret = ff_formats_ref(formats, &ctx->outputs[0]->incfg.color_ranges)) < 0)
+ return ret;
+
return 0;
}
@@ -497,6 +542,7 @@ static int config_props(AVFilterLink *outlink)
const AVPixFmtDescriptor *outdesc = av_pix_fmt_desc_get(outfmt);
ScaleContext *scale = ctx->priv;
uint8_t *flags_val = NULL;
+ int in_range, in_colorspace;
int ret;
if ((ret = scale_eval_dimensions(ctx)) < 0)
@@ -521,6 +567,14 @@ static int config_props(AVFilterLink *outlink)
if (outfmt == AV_PIX_FMT_PAL8) outfmt = AV_PIX_FMT_BGR8;
scale->output_is_pal = av_pix_fmt_desc_get(outfmt)->flags & AV_PIX_FMT_FLAG_PAL;
+ in_range = scale->in_range;
+ if (in_range == AVCOL_RANGE_UNSPECIFIED)
+ in_range = inlink0->color_range;
+
+ in_colorspace = scale->in_color_matrix;
+ if (in_colorspace == -1 /* auto */)
+ in_colorspace = inlink0->colorspace;
+
if (scale->sws)
sws_freeContext(scale->sws);
if (scale->isws[0])
@@ -530,8 +584,8 @@ static int config_props(AVFilterLink *outlink)
scale->isws[0] = scale->isws[1] = scale->sws = NULL;
if (inlink0->w == outlink->w &&
inlink0->h == outlink->h &&
- scale->out_color_matrix == AVCOL_SPC_UNSPECIFIED &&
- scale->in_range == scale->out_range &&
+ in_range == outlink->color_range &&
+ in_colorspace == outlink->colorspace &&
inlink0->format == outlink->format)
;
else {
@@ -540,6 +594,8 @@ static int config_props(AVFilterLink *outlink)
for (i = 0; i < 3; i++) {
int in_v_chr_pos = scale->in_v_chr_pos, out_v_chr_pos = scale->out_v_chr_pos;
+ int in_full, out_full, brightness, contrast, saturation;
+ const int *inv_table, *table;
struct SwsContext *const s = sws_alloc_context();
if (!s)
return AVERROR(ENOMEM);
@@ -555,15 +611,12 @@ static int config_props(AVFilterLink *outlink)
av_opt_set_int(s, "dstw", outlink->w, 0);
av_opt_set_int(s, "dsth", outlink->h >> !!i, 0);
av_opt_set_int(s, "dst_format", outfmt, 0);
- if (scale->in_range != AVCOL_RANGE_UNSPECIFIED)
- av_opt_set_int(s, "src_range",
- scale->in_range == AVCOL_RANGE_JPEG, 0);
- else if (scale->in_frame_range != AVCOL_RANGE_UNSPECIFIED)
+ if (in_range != AVCOL_RANGE_UNSPECIFIED)
av_opt_set_int(s, "src_range",
- scale->in_frame_range == AVCOL_RANGE_JPEG, 0);
- if (scale->out_range != AVCOL_RANGE_UNSPECIFIED)
+ in_range == AVCOL_RANGE_JPEG, 0);
+ if (outlink->color_range != AVCOL_RANGE_UNSPECIFIED)
av_opt_set_int(s, "dst_range",
- scale->out_range == AVCOL_RANGE_JPEG, 0);
+ outlink->color_range == AVCOL_RANGE_JPEG, 0);
/* Override chroma location default settings to have the correct
* chroma positions. MPEG chroma positions are used by convention.
@@ -584,6 +637,24 @@ static int config_props(AVFilterLink *outlink)
if ((ret = sws_init_context(s, NULL, NULL)) < 0)
return ret;
+
+ sws_getColorspaceDetails(s, (int **)&inv_table, &in_full,
+ (int **)&table, &out_full,
+ &brightness, &contrast, &saturation);
+
+ if (scale->in_color_matrix == -1 /* auto */)
+ inv_table = sws_getCoefficients(inlink0->colorspace);
+ else if (scale->in_color_matrix != AVCOL_SPC_UNSPECIFIED)
+ inv_table = sws_getCoefficients(scale->in_color_matrix);
+ if (outlink->colorspace != AVCOL_SPC_UNSPECIFIED)
+ table = sws_getCoefficients(outlink->colorspace);
+ else if (scale->in_color_matrix != AVCOL_SPC_UNSPECIFIED)
+ table = inv_table;
+
+ sws_setColorspaceDetails(s, inv_table, in_full,
+ table, out_full,
+ brightness, contrast, saturation);
+
if (!scale->interlaced)
break;
}
@@ -597,10 +668,12 @@ static int config_props(AVFilterLink *outlink)
if (scale->sws)
av_opt_get(scale->sws, "sws_flags", 0, &flags_val);
- av_log(ctx, AV_LOG_VERBOSE, "w:%d h:%d fmt:%s sar:%d/%d -> w:%d h:%d fmt:%s sar:%d/%d flags:%s\n",
+ av_log(ctx, AV_LOG_VERBOSE, "w:%d h:%d fmt:%s csp:%s range:%s sar:%d/%d -> w:%d h:%d fmt:%s csp:%s range:%s sar:%d/%d flags:%s\n",
inlink ->w, inlink ->h, av_get_pix_fmt_name( inlink->format),
+ av_color_space_name(inlink->colorspace), av_color_range_name(inlink->color_range),
inlink->sample_aspect_ratio.num, inlink->sample_aspect_ratio.den,
outlink->w, outlink->h, av_get_pix_fmt_name(outlink->format),
+ av_color_space_name(outlink->colorspace), av_color_range_name(outlink->color_range),
outlink->sample_aspect_ratio.num, outlink->sample_aspect_ratio.den,
flags_val);
av_freep(&flags_val);
@@ -620,6 +693,8 @@ static int config_props_ref(AVFilterLink *outlink)
outlink->sample_aspect_ratio = inlink->sample_aspect_ratio;
outlink->time_base = inlink->time_base;
outlink->frame_rate = inlink->frame_rate;
+ outlink->colorspace = inlink->colorspace;
+ outlink->color_range = inlink->color_range;
return 0;
}
@@ -693,7 +768,6 @@ static int scale_frame(AVFilterLink *link, AVFrame *in, AVFrame **frame_out)
const AVPixFmtDescriptor *desc = av_pix_fmt_desc_get(link->format);
char buf[32];
int ret;
- int in_range;
int frame_changed;
*frame_out = NULL;
@@ -704,14 +778,9 @@ static int scale_frame(AVFilterLink *link, AVFrame *in, AVFrame **frame_out)
in->height != link->h ||
in->format != link->format ||
in->sample_aspect_ratio.den != link->sample_aspect_ratio.den ||
- in->sample_aspect_ratio.num != link->sample_aspect_ratio.num;
-
- if (in->color_range != AVCOL_RANGE_UNSPECIFIED &&
- scale->in_range == AVCOL_RANGE_UNSPECIFIED &&
- in->color_range != scale->in_frame_range) {
- scale->in_frame_range = in->color_range;
- frame_changed = 1;
- }
+ in->sample_aspect_ratio.num != link->sample_aspect_ratio.num ||
+ in->colorspace != link->colorspace ||
+ in->color_range != link->color_range;
if (scale->eval_mode == EVAL_MODE_FRAME || frame_changed) {
unsigned vars_w[VARS_NB] = { 0 }, vars_h[VARS_NB] = { 0 };
@@ -768,9 +837,11 @@ FF_ENABLE_DEPRECATION_WARNINGS
#endif
}
- link->dst->inputs[0]->format = in->format;
- link->dst->inputs[0]->w = in->width;
- link->dst->inputs[0]->h = in->height;
+ link->dst->inputs[0]->format = in->format;
+ link->dst->inputs[0]->w = in->width;
+ link->dst->inputs[0]->h = in->height;
+ link->dst->inputs[0]->colorspace = in->colorspace;
+ link->dst->inputs[0]->color_range = in->color_range;
link->dst->inputs[0]->sample_aspect_ratio.den = in->sample_aspect_ratio.den;
link->dst->inputs[0]->sample_aspect_ratio.num = in->sample_aspect_ratio.num;
@@ -798,69 +869,12 @@ scale:
av_frame_copy_props(out, in);
out->width = outlink->w;
out->height = outlink->h;
+ out->color_range = outlink->color_range;
+ out->colorspace = outlink->colorspace;
if (scale->output_is_pal)
avpriv_set_systematic_pal2((uint32_t*)out->data[1], outlink->format == AV_PIX_FMT_PAL8 ? AV_PIX_FMT_BGR8 : outlink->format);
- in_range = in->color_range;
-
- if ( scale->in_color_matrix != AVCOL_SPC_UNSPECIFIED
- || scale->out_color_matrix != AVCOL_SPC_UNSPECIFIED
- || scale-> in_range != AVCOL_RANGE_UNSPECIFIED
- || in_range != AVCOL_RANGE_UNSPECIFIED
- || scale->out_range != AVCOL_RANGE_UNSPECIFIED) {
- int in_full, out_full, brightness, contrast, saturation;
- const int *inv_table, *table;
-
- sws_getColorspaceDetails(scale->sws, (int **)&inv_table, &in_full,
- (int **)&table, &out_full,
- &brightness, &contrast, &saturation);
-
- if (scale->in_color_matrix == -1 /* auto */)
- inv_table = sws_getCoefficients(in->colorspace);
- else if (scale->in_color_matrix != AVCOL_SPC_UNSPECIFIED)
- inv_table = sws_getCoefficients(scale->in_color_matrix);
- if (scale->out_color_matrix != AVCOL_SPC_UNSPECIFIED)
- table = sws_getCoefficients(scale->out_color_matrix);
- else if (scale->in_color_matrix != AVCOL_SPC_UNSPECIFIED)
- table = inv_table;
-
- if (scale-> in_range != AVCOL_RANGE_UNSPECIFIED)
- in_full = (scale-> in_range == AVCOL_RANGE_JPEG);
- else if (in_range != AVCOL_RANGE_UNSPECIFIED)
- in_full = (in_range == AVCOL_RANGE_JPEG);
- if (scale->out_range != AVCOL_RANGE_UNSPECIFIED)
- out_full = (scale->out_range == AVCOL_RANGE_JPEG);
-
- sws_setColorspaceDetails(scale->sws, inv_table, in_full,
- table, out_full,
- brightness, contrast, saturation);
- if (scale->isws[0])
- sws_setColorspaceDetails(scale->isws[0], inv_table, in_full,
- table, out_full,
- brightness, contrast, saturation);
- if (scale->isws[1])
- sws_setColorspaceDetails(scale->isws[1], inv_table, in_full,
- table, out_full,
- brightness, contrast, saturation);
-
- out->color_range = out_full ? AVCOL_RANGE_JPEG : AVCOL_RANGE_MPEG;
- if (scale->out_color_matrix != AVCOL_SPC_UNSPECIFIED)
- out->colorspace = scale->out_color_matrix;
- }
-
- // Sanity checks:
- // 1. If the output is RGB, set the matrix coefficients to RGB.
- // 2. If the output is not RGB and we've got the RGB/XYZ (identity)
- // matrix configured, unset the matrix.
- // In theory these should be in swscale itself as the AVFrame
- // based API gets in, so that not every swscale API user has
- // to go through duplicating such sanity checks.
- if (av_pix_fmt_desc_get(out->format)->flags & AV_PIX_FMT_FLAG_RGB)
- out->colorspace = AVCOL_SPC_RGB;
- else if (out->colorspace == AVCOL_SPC_RGB)
- out->colorspace = AVCOL_SPC_UNSPECIFIED;
-
av_reduce(&out->sample_aspect_ratio.num, &out->sample_aspect_ratio.den,
(int64_t)in->sample_aspect_ratio.num * outlink->h * link->w,
(int64_t)in->sample_aspect_ratio.den * outlink->w * link->h,
@@ -905,7 +919,9 @@ static int filter_frame_ref(AVFilterLink *link, AVFrame *in)
in->height != link->h ||
in->format != link->format ||
in->sample_aspect_ratio.den != link->sample_aspect_ratio.den ||
- in->sample_aspect_ratio.num != link->sample_aspect_ratio.num;
+ in->sample_aspect_ratio.num != link->sample_aspect_ratio.num ||
+ in->colorspace != link->colorspace ||
+ in->color_range != link->color_range;
if (frame_changed) {
link->format = in->format;
@@ -913,6 +929,8 @@ static int filter_frame_ref(AVFilterLink *link, AVFrame *in)
link->h = in->height;
link->sample_aspect_ratio.num = in->sample_aspect_ratio.num;
link->sample_aspect_ratio.den = in->sample_aspect_ratio.den;
+ link->colorspace = in->colorspace;
+ link->color_range = in->color_range;
config_props_ref(outlink);
}
--
2.43.0
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
^ permalink raw reply [flat|nested] 26+ messages in thread* [FFmpeg-devel] [PATCH v2 08/15] avfilter/vf_zscale: remove unused variables
2023-12-13 13:11 [FFmpeg-devel] [PATCH v2 00/15] YUV colorspace filter negotiation Niklas Haas
` (6 preceding siblings ...)
2023-12-13 13:12 ` [FFmpeg-devel] [PATCH v2 07/15] vf_scale: use colorspace negotiation API Niklas Haas
@ 2023-12-13 13:12 ` Niklas Haas
2023-12-13 13:12 ` [FFmpeg-devel] [PATCH v2 09/15] avfilter/vf_zscale: switch to colorspace negotiation API Niklas Haas
` (7 subsequent siblings)
15 siblings, 0 replies; 26+ messages in thread
From: Niklas Haas @ 2023-12-13 13:12 UTC (permalink / raw)
To: ffmpeg-devel; +Cc: Niklas Haas
From: Niklas Haas <git@haasn.dev>
Only assigned, never read.
---
libavfilter/vf_zscale.c | 15 ---------------
1 file changed, 15 deletions(-)
diff --git a/libavfilter/vf_zscale.c b/libavfilter/vf_zscale.c
index 788870ffaf..f76c9954cd 100644
--- a/libavfilter/vf_zscale.c
+++ b/libavfilter/vf_zscale.c
@@ -132,12 +132,6 @@ typedef struct ZScaleContext {
zimg_graph_builder_params alpha_params, params;
zimg_graph_builder_params alpha_params_tmp, params_tmp;
zimg_filter_graph *alpha_graph[MAX_THREADS], *graph[MAX_THREADS];
-
- enum AVColorSpace in_colorspace, out_colorspace;
- enum AVColorTransferCharacteristic in_trc, out_trc;
- enum AVColorPrimaries in_primaries, out_primaries;
- enum AVColorRange in_range, out_range;
- enum AVChromaLocation in_chromal, out_chromal;
} ZScaleContext;
typedef struct ThreadData {
@@ -816,15 +810,6 @@ static int filter_frame(AVFilterLink *link, AVFrame *in)
link->dst->inputs[0]->h = in->height;
s->nb_threads = av_clip(FFMIN(ff_filter_get_nb_threads(ctx), FFMIN(link->h, outlink->h) / MIN_TILESIZE), 1, MAX_THREADS);
- s->in_colorspace = in->colorspace;
- s->in_trc = in->color_trc;
- s->in_primaries = in->color_primaries;
- s->in_range = in->color_range;
- s->out_colorspace = out->colorspace;
- s->out_trc = out->color_trc;
- s->out_primaries = out->color_primaries;
- s->out_range = out->color_range;
-
slice_params(s, out->height, in->height);
zimg_image_format_default(&s->src_format, ZIMG_API_VERSION);
--
2.43.0
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
^ permalink raw reply [flat|nested] 26+ messages in thread* [FFmpeg-devel] [PATCH v2 09/15] avfilter/vf_zscale: switch to colorspace negotiation API
2023-12-13 13:11 [FFmpeg-devel] [PATCH v2 00/15] YUV colorspace filter negotiation Niklas Haas
` (7 preceding siblings ...)
2023-12-13 13:12 ` [FFmpeg-devel] [PATCH v2 08/15] avfilter/vf_zscale: remove unused variables Niklas Haas
@ 2023-12-13 13:12 ` Niklas Haas
2023-12-13 13:12 ` [FFmpeg-devel] [PATCH v2 10/15] avfilter/vf_libplacebo: don't force dovi frames to bt.2020-ncl Niklas Haas
` (6 subsequent siblings)
15 siblings, 0 replies; 26+ messages in thread
From: Niklas Haas @ 2023-12-13 13:12 UTC (permalink / raw)
To: ffmpeg-devel; +Cc: Niklas Haas
From: Niklas Haas <git@haasn.dev>
Following the same design as vf_scale.
---
libavfilter/vf_zscale.c | 44 +++++++++++++++++++++++++++++++----------
1 file changed, 34 insertions(+), 10 deletions(-)
diff --git a/libavfilter/vf_zscale.c b/libavfilter/vf_zscale.c
index f76c9954cd..3b14ce4f33 100644
--- a/libavfilter/vf_zscale.c
+++ b/libavfilter/vf_zscale.c
@@ -187,8 +187,12 @@ static av_cold int init(AVFilterContext *ctx)
return 0;
}
+static enum AVColorRange convert_range_from_zimg(enum zimg_pixel_range_e color_range);
+
static int query_formats(AVFilterContext *ctx)
{
+ ZScaleContext *s = ctx->priv;
+ AVFilterFormats *formats;
static const enum AVPixelFormat pixel_fmts[] = {
AV_PIX_FMT_YUV410P, AV_PIX_FMT_YUV411P,
AV_PIX_FMT_YUV420P, AV_PIX_FMT_YUV422P,
@@ -217,7 +221,27 @@ static int query_formats(AVFilterContext *ctx)
ret = ff_formats_ref(ff_make_format_list(pixel_fmts), &ctx->inputs[0]->outcfg.formats);
if (ret < 0)
return ret;
- return ff_formats_ref(ff_make_format_list(pixel_fmts), &ctx->outputs[0]->incfg.formats);
+ ret = ff_formats_ref(ff_make_format_list(pixel_fmts), &ctx->outputs[0]->incfg.formats);
+ if (ret < 0)
+ return ret;
+
+ if ((ret = ff_formats_ref(ff_all_color_spaces(), &ctx->inputs[0]->outcfg.formats)) < 0 ||
+ (ret = ff_formats_ref(ff_all_color_ranges(), &ctx->inputs[0]->outcfg.formats)) < 0)
+ return ret;
+
+ formats = s->colorspace != ZIMG_MATRIX_UNSPECIFIED && s->colorspace > 0
+ ? ff_make_formats_list_singleton(s->colorspace)
+ : ff_all_color_spaces();
+ if ((ret = ff_formats_ref(formats, &ctx->outputs[0]->incfg.formats)) < 0)
+ return ret;
+
+ formats = s->range != -1
+ ? ff_make_formats_list_singleton(convert_range_from_zimg(s->range))
+ : ff_all_color_ranges();
+ if ((ret = ff_formats_ref(formats, &ctx->outputs[0]->incfg.formats)) < 0)
+ return ret;
+
+ return 0;
}
static void slice_params(ZScaleContext *s, int out_h, int in_h)
@@ -678,15 +702,9 @@ fail:
static void update_output_color_information(ZScaleContext *s, AVFrame *frame)
{
- if (s->colorspace != -1)
- frame->colorspace = (int)s->dst_format.matrix_coefficients;
-
if (s->primaries != -1)
frame->color_primaries = (int)s->dst_format.color_primaries;
- if (s->range != -1)
- frame->color_range = convert_range_from_zimg(s->dst_format.pixel_range);
-
if (s->trc != -1)
frame->color_trc = (int)s->dst_format.transfer_characteristics;
@@ -775,6 +793,8 @@ static int filter_frame(AVFilterLink *link, AVFrame *in)
if ((link->format != outlink->format) ||
(link->w != outlink->w) ||
(link->h != outlink->h) ||
+ (link->colorspace != outlink->colorspace) ||
+ (link->color_range != outlink->color_range) ||
s->first_time ||
(s->src_format.chroma_location != s->dst_format.chroma_location) ||
(s->src_format.color_family !=s->dst_format.color_family) ||
@@ -796,6 +816,8 @@ static int filter_frame(AVFilterLink *link, AVFrame *in)
goto fail;
av_frame_copy_props(out, in);
+ out->colorspace = outlink->colorspace;
+ out->color_range = outlink->color_range;
if ((ret = realign_frame(desc, &in, 1)) < 0)
goto fail;
@@ -805,9 +827,11 @@ static int filter_frame(AVFilterLink *link, AVFrame *in)
snprintf(buf, sizeof(buf)-1, "%d", outlink->h);
av_opt_set(s, "h", buf, 0);
- link->dst->inputs[0]->format = in->format;
- link->dst->inputs[0]->w = in->width;
- link->dst->inputs[0]->h = in->height;
+ link->dst->inputs[0]->format = in->format;
+ link->dst->inputs[0]->w = in->width;
+ link->dst->inputs[0]->h = in->height;
+ link->dst->inputs[0]->colorspace = in->colorspace;
+ link->dst->inputs[0]->color_range = in->color_range;
s->nb_threads = av_clip(FFMIN(ff_filter_get_nb_threads(ctx), FFMIN(link->h, outlink->h) / MIN_TILESIZE), 1, MAX_THREADS);
slice_params(s, out->height, in->height);
--
2.43.0
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
^ permalink raw reply [flat|nested] 26+ messages in thread* [FFmpeg-devel] [PATCH v2 10/15] avfilter/vf_libplacebo: don't force dovi frames to bt.2020-ncl
2023-12-13 13:11 [FFmpeg-devel] [PATCH v2 00/15] YUV colorspace filter negotiation Niklas Haas
` (8 preceding siblings ...)
2023-12-13 13:12 ` [FFmpeg-devel] [PATCH v2 09/15] avfilter/vf_zscale: switch to colorspace negotiation API Niklas Haas
@ 2023-12-13 13:12 ` Niklas Haas
2023-12-13 13:12 ` [FFmpeg-devel] [PATCH v2 11/15] avfilter/vf_libplacebo: switch to colorspace negotiation API Niklas Haas
` (5 subsequent siblings)
15 siblings, 0 replies; 26+ messages in thread
From: Niklas Haas @ 2023-12-13 13:12 UTC (permalink / raw)
To: ffmpeg-devel; +Cc: Niklas Haas
From: Niklas Haas <git@haasn.dev>
This is at odds with the YUV matrix negotiation API, in which such
dynamic changes in YUV encoding are no longer easily possible. There is
also no really strong motivating reason to do this, since the choice of
YUV matrix is essentially arbitrary and not actually related to the
Dolby Vision decoding process.
---
libavfilter/vf_libplacebo.c | 1 -
1 file changed, 1 deletion(-)
diff --git a/libavfilter/vf_libplacebo.c b/libavfilter/vf_libplacebo.c
index 8534945ba6..b6e82a61e0 100644
--- a/libavfilter/vf_libplacebo.c
+++ b/libavfilter/vf_libplacebo.c
@@ -888,7 +888,6 @@ static int output_frame(AVFilterContext *ctx, int64_t pts)
if (s->apply_dovi && av_frame_get_side_data(ref, AV_FRAME_DATA_DOVI_METADATA)) {
/* Output of dovi reshaping is always BT.2020+PQ, so infer the correct
* output colorspace defaults */
- out->colorspace = AVCOL_SPC_BT2020_NCL;
out->color_primaries = AVCOL_PRI_BT2020;
out->color_trc = AVCOL_TRC_SMPTE2084;
}
--
2.43.0
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
^ permalink raw reply [flat|nested] 26+ messages in thread* [FFmpeg-devel] [PATCH v2 11/15] avfilter/vf_libplacebo: switch to colorspace negotiation API
2023-12-13 13:11 [FFmpeg-devel] [PATCH v2 00/15] YUV colorspace filter negotiation Niklas Haas
` (9 preceding siblings ...)
2023-12-13 13:12 ` [FFmpeg-devel] [PATCH v2 10/15] avfilter/vf_libplacebo: don't force dovi frames to bt.2020-ncl Niklas Haas
@ 2023-12-13 13:12 ` Niklas Haas
2023-12-13 13:12 ` [FFmpeg-devel] [PATCH v2 12/15] avfilter/buffersink: add color space/range accessors Niklas Haas
` (4 subsequent siblings)
15 siblings, 0 replies; 26+ messages in thread
From: Niklas Haas @ 2023-12-13 13:12 UTC (permalink / raw)
To: ffmpeg-devel; +Cc: Niklas Haas
From: Niklas Haas <git@haasn.dev>
---
libavfilter/vf_libplacebo.c | 25 ++++++++++++++-----------
1 file changed, 14 insertions(+), 11 deletions(-)
diff --git a/libavfilter/vf_libplacebo.c b/libavfilter/vf_libplacebo.c
index b6e82a61e0..a9a3d884ce 100644
--- a/libavfilter/vf_libplacebo.c
+++ b/libavfilter/vf_libplacebo.c
@@ -882,6 +882,8 @@ static int output_frame(AVFilterContext *ctx, int64_t pts)
out->pts = pts;
out->width = outlink->w;
out->height = outlink->h;
+ out->colorspace = outlink->colorspace;
+ out->color_range = outlink->color_range;
if (s->fps.num)
out->duration = 1;
@@ -892,22 +894,11 @@ static int output_frame(AVFilterContext *ctx, int64_t pts)
out->color_trc = AVCOL_TRC_SMPTE2084;
}
- if (s->colorspace >= 0)
- out->colorspace = s->colorspace;
- if (s->color_range >= 0)
- out->color_range = s->color_range;
if (s->color_trc >= 0)
out->color_trc = s->color_trc;
if (s->color_primaries >= 0)
out->color_primaries = s->color_primaries;
- /* Sanity colorspace overrides */
- if (outdesc->flags & AV_PIX_FMT_FLAG_RGB) {
- out->colorspace = AVCOL_SPC_RGB;
- } else if (out->colorspace == AVCOL_SPC_RGB) {
- out->colorspace = AVCOL_SPC_UNSPECIFIED;
- }
-
changed_csp = ref->colorspace != out->colorspace ||
ref->color_range != out->color_range ||
ref->color_trc != out->color_trc ||
@@ -1203,6 +1194,18 @@ static int libplacebo_query_format(AVFilterContext *ctx)
for (int i = 0; i < s->nb_inputs; i++)
RET(ff_formats_ref(infmts, &ctx->inputs[i]->outcfg.formats));
RET(ff_formats_ref(outfmts, &ctx->outputs[0]->incfg.formats));
+
+ /* Set colorspace properties */
+ RET(ff_formats_ref(ff_all_color_spaces(), &ctx->inputs[0]->outcfg.color_spaces));
+ RET(ff_formats_ref(ff_all_color_ranges(), &ctx->inputs[0]->outcfg.color_ranges));
+
+ outfmts = s->colorspace > 0 ? ff_make_formats_list_singleton(s->colorspace)
+ : ff_all_color_spaces();
+ RET(ff_formats_ref(outfmts, &ctx->outputs[0]->incfg.color_spaces));
+
+ outfmts = s->color_range > 0 ? ff_make_formats_list_singleton(s->color_range)
+ : ff_all_color_ranges();
+ RET(ff_formats_ref(outfmts, &ctx->outputs[0]->incfg.color_ranges));
return 0;
fail:
--
2.43.0
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
^ permalink raw reply [flat|nested] 26+ messages in thread* [FFmpeg-devel] [PATCH v2 12/15] avfilter/buffersink: add color space/range accessors
2023-12-13 13:11 [FFmpeg-devel] [PATCH v2 00/15] YUV colorspace filter negotiation Niklas Haas
` (10 preceding siblings ...)
2023-12-13 13:12 ` [FFmpeg-devel] [PATCH v2 11/15] avfilter/vf_libplacebo: switch to colorspace negotiation API Niklas Haas
@ 2023-12-13 13:12 ` Niklas Haas
2023-12-13 13:12 ` [FFmpeg-devel] [PATCH v2 13/15] avfilter/vf_format: re-use AVFilterFormats for pix_fmt parsing Niklas Haas
` (3 subsequent siblings)
15 siblings, 0 replies; 26+ messages in thread
From: Niklas Haas @ 2023-12-13 13:12 UTC (permalink / raw)
To: ffmpeg-devel; +Cc: Niklas Haas
From: Niklas Haas <git@haasn.dev>
Needed for fftools.
---
doc/APIchanges | 3 +++
libavfilter/buffersink.c | 2 ++
libavfilter/buffersink.h | 2 ++
libavfilter/version.h | 2 +-
4 files changed, 8 insertions(+), 1 deletion(-)
diff --git a/doc/APIchanges b/doc/APIchanges
index 2f6ea50f63..578c2c4641 100644
--- a/doc/APIchanges
+++ b/doc/APIchanges
@@ -2,6 +2,9 @@ The last version increases of all libraries were on 2023-02-09
API changes, most recent first:
+2023-11-xx - xxxxxxxxxx - lavf 58.15.100 - buffersink.h
+ Add av_buffersink_get_colorspace and av_buffersink_get_color_range.
+
2023-11-xx - xxxxxxxxxx - lavf 58.14.100 - avfilter.h
Add AVFilterLink.colorspace, AVFilterLink.color_range
diff --git a/libavfilter/buffersink.c b/libavfilter/buffersink.c
index 3da3331159..6ba2970dd5 100644
--- a/libavfilter/buffersink.c
+++ b/libavfilter/buffersink.c
@@ -209,6 +209,8 @@ MAKE_AVFILTERLINK_ACCESSOR(AVRational , frame_rate )
MAKE_AVFILTERLINK_ACCESSOR(int , w )
MAKE_AVFILTERLINK_ACCESSOR(int , h )
MAKE_AVFILTERLINK_ACCESSOR(AVRational , sample_aspect_ratio)
+MAKE_AVFILTERLINK_ACCESSOR(enum AVColorSpace, colorspace)
+MAKE_AVFILTERLINK_ACCESSOR(enum AVColorRange, color_range)
#if FF_API_OLD_CHANNEL_LAYOUT
FF_DISABLE_DEPRECATION_WARNINGS
diff --git a/libavfilter/buffersink.h b/libavfilter/buffersink.h
index 64e08de53e..a9374b8b4f 100644
--- a/libavfilter/buffersink.h
+++ b/libavfilter/buffersink.h
@@ -117,6 +117,8 @@ AVRational av_buffersink_get_frame_rate (const AVFilterContext *c
int av_buffersink_get_w (const AVFilterContext *ctx);
int av_buffersink_get_h (const AVFilterContext *ctx);
AVRational av_buffersink_get_sample_aspect_ratio (const AVFilterContext *ctx);
+enum AVColorSpace av_buffersink_get_colorspace (const AVFilterContext *ctx);
+enum AVColorRange av_buffersink_get_color_range (const AVFilterContext *ctx);
int av_buffersink_get_channels (const AVFilterContext *ctx);
#if FF_API_OLD_CHANNEL_LAYOUT
diff --git a/libavfilter/version.h b/libavfilter/version.h
index 7642b670d1..59330858bd 100644
--- a/libavfilter/version.h
+++ b/libavfilter/version.h
@@ -31,7 +31,7 @@
#include "version_major.h"
-#define LIBAVFILTER_VERSION_MINOR 14
+#define LIBAVFILTER_VERSION_MINOR 15
#define LIBAVFILTER_VERSION_MICRO 100
--
2.43.0
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
^ permalink raw reply [flat|nested] 26+ messages in thread* [FFmpeg-devel] [PATCH v2 13/15] avfilter/vf_format: re-use AVFilterFormats for pix_fmt parsing
2023-12-13 13:11 [FFmpeg-devel] [PATCH v2 00/15] YUV colorspace filter negotiation Niklas Haas
` (11 preceding siblings ...)
2023-12-13 13:12 ` [FFmpeg-devel] [PATCH v2 12/15] avfilter/buffersink: add color space/range accessors Niklas Haas
@ 2023-12-13 13:12 ` Niklas Haas
2023-12-13 13:12 ` [FFmpeg-devel] [PATCH v2 14/15] avfilter/vf_format: add color_ranges/spaces Niklas Haas
` (2 subsequent siblings)
15 siblings, 0 replies; 26+ messages in thread
From: Niklas Haas @ 2023-12-13 13:12 UTC (permalink / raw)
To: ffmpeg-devel; +Cc: Niklas Haas
From: Niklas Haas <git@haasn.dev>
Rewrite the format parsing code to make it more easily generalizable. In
particular, `invert_formats` does not depend on the type of format list
passed to it, which allows me to re-use this helper in an upcoming
commit.
Slightly shortens the code, at the sole cost of doing several malloc
(ff_add_format) instead of a single malloc.
---
libavfilter/vf_format.c | 103 +++++++++++++++++-----------------------
1 file changed, 43 insertions(+), 60 deletions(-)
diff --git a/libavfilter/vf_format.c b/libavfilter/vf_format.c
index 1189bd61c2..59832b7768 100644
--- a/libavfilter/vf_format.c
+++ b/libavfilter/vf_format.c
@@ -41,25 +41,48 @@ typedef struct FormatContext {
const AVClass *class;
char *pix_fmts;
- /**
- * pix_fmts parsed into AVPixelFormats and terminated with
- * AV_PIX_FMT_NONE
- */
- enum AVPixelFormat *formats;
+ AVFilterFormats *formats; ///< parsed from `pix_fmts`
} FormatContext;
static av_cold void uninit(AVFilterContext *ctx)
{
FormatContext *s = ctx->priv;
- av_freep(&s->formats);
+ ff_formats_unref(&s->formats);
+}
+
+static av_cold int invert_formats(AVFilterFormats **fmts,
+ AVFilterFormats *allfmts)
+{
+ if (!allfmts)
+ return AVERROR(ENOMEM);
+ if (!*fmts) {
+ /* empty fmt list means no restriction, regardless of filter type */
+ ff_formats_unref(&allfmts);
+ return 0;
+ }
+
+ for (int i = 0; i < allfmts->nb_formats; i++) {
+ for (int j = 0; j < (*fmts)->nb_formats; j++) {
+ if (allfmts->formats[i] == (*fmts)->formats[j]) {
+ /* format is forbidden, remove it from allfmts list */
+ memmove(&allfmts->formats[i], &allfmts->formats[i+1],
+ (allfmts->nb_formats - (i+1)) * sizeof(*allfmts->formats));
+ allfmts->nb_formats--;
+ i--; /* repeat loop with same idx */
+ break;
+ }
+ }
+ }
+
+ ff_formats_unref(fmts);
+ *fmts = allfmts;
+ return 0;
}
static av_cold int init(AVFilterContext *ctx)
{
FormatContext *s = ctx->priv;
- char *cur, *sep;
- int nb_formats = 1;
- int i;
+ enum AVPixelFormat pix_fmt;
int ret;
if (!s->pix_fmts) {
@@ -67,64 +90,24 @@ static av_cold int init(AVFilterContext *ctx)
return AVERROR(EINVAL);
}
- /* count the formats */
- cur = s->pix_fmts;
- while ((cur = strchr(cur, '|'))) {
- nb_formats++;
- if (*cur)
- cur++;
- }
-
- s->formats = av_malloc_array(nb_formats + 1, sizeof(*s->formats));
- if (!s->formats)
- return AVERROR(ENOMEM);
-
- /* parse the list of formats */
- cur = s->pix_fmts;
- for (i = 0; i < nb_formats; i++) {
+ for (char *sep, *cur = s->pix_fmts; cur; cur = sep) {
sep = strchr(cur, '|');
- if (sep)
+ if (sep && *sep)
*sep++ = 0;
-
- if ((ret = ff_parse_pixel_format(&s->formats[i], cur, ctx)) < 0)
+ if ((ret = ff_parse_pixel_format(&pix_fmt, cur, ctx)) < 0 ||
+ (ret = ff_add_format(&s->formats, pix_fmt)) < 0)
return ret;
-
- cur = sep;
}
- s->formats[nb_formats] = AV_PIX_FMT_NONE;
if (!strcmp(ctx->filter->name, "noformat")) {
- const AVPixFmtDescriptor *desc = NULL;
- enum AVPixelFormat *formats_allowed;
- int nb_formats_lavu = 0, nb_formats_allowed = 0;
-
- /* count the formats known to lavu */
- while ((desc = av_pix_fmt_desc_next(desc)))
- nb_formats_lavu++;
-
- formats_allowed = av_malloc_array(nb_formats_lavu + 1, sizeof(*formats_allowed));
- if (!formats_allowed)
- return AVERROR(ENOMEM);
-
- /* for each format known to lavu, check if it's in the list of
- * forbidden formats */
- while ((desc = av_pix_fmt_desc_next(desc))) {
- enum AVPixelFormat pix_fmt = av_pix_fmt_desc_get_id(desc);
-
- for (i = 0; i < nb_formats; i++) {
- if (s->formats[i] == pix_fmt)
- break;
- }
- if (i < nb_formats)
- continue;
-
- formats_allowed[nb_formats_allowed++] = pix_fmt;
- }
- formats_allowed[nb_formats_allowed] = AV_PIX_FMT_NONE;
- av_freep(&s->formats);
- s->formats = formats_allowed;
+ if ((ret = invert_formats(&s->formats, ff_all_formats(AVMEDIA_TYPE_VIDEO))) < 0)
+ return ret;
}
+ /* hold on to a ref for the lifetime of the filter */
+ if ((ret = ff_formats_ref(s->formats, &s->formats)) < 0)
+ return ret;
+
return 0;
}
@@ -132,7 +115,7 @@ static int query_formats(AVFilterContext *ctx)
{
FormatContext *s = ctx->priv;
- return ff_set_common_formats_from_list(ctx, s->formats);
+ return ff_set_common_formats(ctx, s->formats);
}
--
2.43.0
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
^ permalink raw reply [flat|nested] 26+ messages in thread* [FFmpeg-devel] [PATCH v2 14/15] avfilter/vf_format: add color_ranges/spaces
2023-12-13 13:11 [FFmpeg-devel] [PATCH v2 00/15] YUV colorspace filter negotiation Niklas Haas
` (12 preceding siblings ...)
2023-12-13 13:12 ` [FFmpeg-devel] [PATCH v2 13/15] avfilter/vf_format: re-use AVFilterFormats for pix_fmt parsing Niklas Haas
@ 2023-12-13 13:12 ` Niklas Haas
2023-12-13 13:12 ` [FFmpeg-devel] [PATCH v2 15/15] avfilter/vf_format: allow empty pix_fmts list Niklas Haas
2023-12-24 18:35 ` [FFmpeg-devel] [PATCH v2 00/15] YUV colorspace filter negotiation Niklas Haas
15 siblings, 0 replies; 26+ messages in thread
From: Niklas Haas @ 2023-12-13 13:12 UTC (permalink / raw)
To: ffmpeg-devel; +Cc: Niklas Haas
From: Niklas Haas <git@haasn.dev>
Needed for fftools/ffmpeg_filter to be able to force specific output
formats via the filter chain.
---
doc/filters.texi | 8 ++++++++
libavfilter/vf_format.c | 42 ++++++++++++++++++++++++++++++++++++++---
2 files changed, 47 insertions(+), 3 deletions(-)
diff --git a/doc/filters.texi b/doc/filters.texi
index f182dc2ddb..4be4dbf270 100644
--- a/doc/filters.texi
+++ b/doc/filters.texi
@@ -14323,6 +14323,14 @@ It accepts the following parameters:
A '|'-separated list of pixel format names, such as
"pix_fmts=yuv420p|monow|rgb24".
+@item color_spaces
+A '|'-separated list of color space names, such as
+"color_spaces=bt709|bt470bg|bt2020nc".
+
+@item color_ranges
+A '|'-separated list of color range names, such as
+"color_spaces=tv|pc".
+
@end table
@subsection Examples
diff --git a/libavfilter/vf_format.c b/libavfilter/vf_format.c
index 59832b7768..d1bb9daa97 100644
--- a/libavfilter/vf_format.c
+++ b/libavfilter/vf_format.c
@@ -40,14 +40,20 @@
typedef struct FormatContext {
const AVClass *class;
char *pix_fmts;
+ char *csps;
+ char *ranges;
AVFilterFormats *formats; ///< parsed from `pix_fmts`
+ AVFilterFormats *color_spaces; ///< parsed from `csps`
+ AVFilterFormats *color_ranges; ///< parsed from `ranges`
} FormatContext;
static av_cold void uninit(AVFilterContext *ctx)
{
FormatContext *s = ctx->priv;
ff_formats_unref(&s->formats);
+ ff_formats_unref(&s->color_spaces);
+ ff_formats_unref(&s->color_ranges);
}
static av_cold int invert_formats(AVFilterFormats **fmts,
@@ -99,13 +105,35 @@ static av_cold int init(AVFilterContext *ctx)
return ret;
}
+ for (char *sep, *cur = s->csps; cur; cur = sep) {
+ sep = strchr(cur, '|');
+ if (sep && *sep)
+ *sep++ = 0;
+ if ((ret = av_color_space_from_name(cur)) < 0 ||
+ (ret = ff_add_format(&s->color_spaces, ret)) < 0)
+ return ret;
+ }
+
+ for (char *sep, *cur = s->ranges; cur; cur = sep) {
+ sep = strchr(cur, '|');
+ if (sep && *sep)
+ *sep++ = 0;
+ if ((ret = av_color_range_from_name(cur)) < 0 ||
+ (ret = ff_add_format(&s->color_ranges, ret)) < 0)
+ return ret;
+ }
+
if (!strcmp(ctx->filter->name, "noformat")) {
- if ((ret = invert_formats(&s->formats, ff_all_formats(AVMEDIA_TYPE_VIDEO))) < 0)
+ if ((ret = invert_formats(&s->formats, ff_all_formats(AVMEDIA_TYPE_VIDEO))) < 0 ||
+ (ret = invert_formats(&s->color_spaces, ff_all_color_spaces())) < 0 ||
+ (ret = invert_formats(&s->color_ranges, ff_all_color_ranges())) < 0)
return ret;
}
/* hold on to a ref for the lifetime of the filter */
- if ((ret = ff_formats_ref(s->formats, &s->formats)) < 0)
+ if ((ret = ff_formats_ref(s->formats, &s->formats)) < 0 ||
+ s->color_spaces && (ret = ff_formats_ref(s->color_spaces, &s->color_spaces)) < 0 ||
+ s->color_ranges && (ret = ff_formats_ref(s->color_ranges, &s->color_ranges)) < 0)
return ret;
return 0;
@@ -114,14 +142,22 @@ static av_cold int init(AVFilterContext *ctx)
static int query_formats(AVFilterContext *ctx)
{
FormatContext *s = ctx->priv;
+ int ret;
+
+ if ((ret = ff_set_common_formats(ctx, s->formats)) < 0 ||
+ s->color_spaces && (ret = ff_set_common_color_spaces(ctx, s->color_spaces)) < 0 ||
+ s->color_ranges && (ret = ff_set_common_color_ranges(ctx, s->color_ranges)) < 0)
+ return ret;
- return ff_set_common_formats(ctx, s->formats);
+ return 0;
}
#define OFFSET(x) offsetof(FormatContext, x)
static const AVOption options[] = {
{ "pix_fmts", "A '|'-separated list of pixel formats", OFFSET(pix_fmts), AV_OPT_TYPE_STRING, .flags = AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_FILTERING_PARAM },
+ { "color_spaces", "A '|'-separated list of color spaces", OFFSET(csps), AV_OPT_TYPE_STRING, .flags = AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_FILTERING_PARAM },
+ { "color_ranges", "A '|'-separated list of color ranges", OFFSET(ranges), AV_OPT_TYPE_STRING, .flags = AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_FILTERING_PARAM },
{ NULL }
};
--
2.43.0
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
^ permalink raw reply [flat|nested] 26+ messages in thread* [FFmpeg-devel] [PATCH v2 15/15] avfilter/vf_format: allow empty pix_fmts list
2023-12-13 13:11 [FFmpeg-devel] [PATCH v2 00/15] YUV colorspace filter negotiation Niklas Haas
` (13 preceding siblings ...)
2023-12-13 13:12 ` [FFmpeg-devel] [PATCH v2 14/15] avfilter/vf_format: add color_ranges/spaces Niklas Haas
@ 2023-12-13 13:12 ` Niklas Haas
2023-12-24 18:35 ` [FFmpeg-devel] [PATCH v2 00/15] YUV colorspace filter negotiation Niklas Haas
15 siblings, 0 replies; 26+ messages in thread
From: Niklas Haas @ 2023-12-13 13:12 UTC (permalink / raw)
To: ffmpeg-devel; +Cc: Niklas Haas
From: Niklas Haas <git@haasn.dev>
Which will impose no restriction. This makes sense when using e.g.
`color_ranges=pc` to limit the color range, without also limiting the
pixel format.
---
libavfilter/vf_format.c | 9 ++-------
1 file changed, 2 insertions(+), 7 deletions(-)
diff --git a/libavfilter/vf_format.c b/libavfilter/vf_format.c
index d1bb9daa97..3a353bbb0d 100644
--- a/libavfilter/vf_format.c
+++ b/libavfilter/vf_format.c
@@ -91,11 +91,6 @@ static av_cold int init(AVFilterContext *ctx)
enum AVPixelFormat pix_fmt;
int ret;
- if (!s->pix_fmts) {
- av_log(ctx, AV_LOG_ERROR, "Empty output format string.\n");
- return AVERROR(EINVAL);
- }
-
for (char *sep, *cur = s->pix_fmts; cur; cur = sep) {
sep = strchr(cur, '|');
if (sep && *sep)
@@ -131,7 +126,7 @@ static av_cold int init(AVFilterContext *ctx)
}
/* hold on to a ref for the lifetime of the filter */
- if ((ret = ff_formats_ref(s->formats, &s->formats)) < 0 ||
+ if (s->formats && (ret = ff_formats_ref(s->formats, &s->formats)) < 0 ||
s->color_spaces && (ret = ff_formats_ref(s->color_spaces, &s->color_spaces)) < 0 ||
s->color_ranges && (ret = ff_formats_ref(s->color_ranges, &s->color_ranges)) < 0)
return ret;
@@ -144,7 +139,7 @@ static int query_formats(AVFilterContext *ctx)
FormatContext *s = ctx->priv;
int ret;
- if ((ret = ff_set_common_formats(ctx, s->formats)) < 0 ||
+ if (s->formats && (ret = ff_set_common_formats(ctx, s->formats)) < 0 ||
s->color_spaces && (ret = ff_set_common_color_spaces(ctx, s->color_spaces)) < 0 ||
s->color_ranges && (ret = ff_set_common_color_ranges(ctx, s->color_ranges)) < 0)
return ret;
--
2.43.0
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
^ permalink raw reply [flat|nested] 26+ messages in thread* Re: [FFmpeg-devel] [PATCH v2 00/15] YUV colorspace filter negotiation
2023-12-13 13:11 [FFmpeg-devel] [PATCH v2 00/15] YUV colorspace filter negotiation Niklas Haas
` (14 preceding siblings ...)
2023-12-13 13:12 ` [FFmpeg-devel] [PATCH v2 15/15] avfilter/vf_format: allow empty pix_fmts list Niklas Haas
@ 2023-12-24 18:35 ` Niklas Haas
2023-12-31 21:50 ` Niklas Haas
15 siblings, 1 reply; 26+ messages in thread
From: Niklas Haas @ 2023-12-24 18:35 UTC (permalink / raw)
To: ffmpeg-devel
On Wed, 13 Dec 2023 14:11:57 +0100 Niklas Haas <ffmpeg@haasn.xyz> wrote:
> Split off from my YUVJ removal series. This implements all of the
> libavfilter changes needed to fully deprecate YUVJ, but does not yet
> remove YUVJ, nor add an AVCodec API for advertising colorspace support.
>
> Update includes all of the feedback that was brought up by Anton. the
> major change from v1 is that YUV metadata configured on a link is no
> longer strictly required to be consistent with the frames, simply to
> avoid breaking users that don't set this link metadata. Effectively,
> this allows us to have a grace period where the field functions more
> like a hint than an authoritative field.
Ping, any further feedback? If not, I will merge this sometime after Christmas.
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
^ permalink raw reply [flat|nested] 26+ messages in thread* Re: [FFmpeg-devel] [PATCH v2 00/15] YUV colorspace filter negotiation
2023-12-24 18:35 ` [FFmpeg-devel] [PATCH v2 00/15] YUV colorspace filter negotiation Niklas Haas
@ 2023-12-31 21:50 ` Niklas Haas
2024-01-04 19:32 ` Dong, Ruijing via ffmpeg-devel
0 siblings, 1 reply; 26+ messages in thread
From: Niklas Haas @ 2023-12-31 21:50 UTC (permalink / raw)
To: ffmpeg-devel
On Sun, 24 Dec 2023 10:35:22 -0800 Niklas Haas <ffmpeg@haasn.xyz> wrote:
> On Wed, 13 Dec 2023 14:11:57 +0100 Niklas Haas <ffmpeg@haasn.xyz> wrote:
> > Split off from my YUVJ removal series. This implements all of the
> > libavfilter changes needed to fully deprecate YUVJ, but does not yet
> > remove YUVJ, nor add an AVCodec API for advertising colorspace support.
> >
> > Update includes all of the feedback that was brought up by Anton. the
> > major change from v1 is that YUV metadata configured on a link is no
> > longer strictly required to be consistent with the frames, simply to
> > avoid breaking users that don't set this link metadata. Effectively,
> > this allows us to have a grace period where the field functions more
> > like a hint than an authoritative field.
>
> Ping, any further feedback? If not, I will merge this sometime after Christmas.
Merged as 1b0ca477..44a33fccd
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [FFmpeg-devel] [PATCH v2 00/15] YUV colorspace filter negotiation
2023-12-31 21:50 ` Niklas Haas
@ 2024-01-04 19:32 ` Dong, Ruijing via ffmpeg-devel
0 siblings, 0 replies; 26+ messages in thread
From: Dong, Ruijing via ffmpeg-devel @ 2024-01-04 19:32 UTC (permalink / raw)
To: FFmpeg development discussions and patches; +Cc: Dong, Ruijing
[AMD Official Use Only - General]
I found out this change caused page fault as well.
--------
8c7934f73ab6c568acaa47c821a6833f9145fdbb is the first bad commit
commit 8c7934f73ab6c568acaa47c821a6833f9145fdbb
Author: Niklas Haas <git@haasn.dev>
Date: Sun Dec 31 13:35:03 2023 -0800
avfilter: add negotiation API for color space/range
Motivated by YUVJ removal. This change will allow full negotiation
between color ranges and matrices as needed. By default, all ranges and
matrices are marked as supported.
Because grayscale formats are currently handled very inconsistently (and
in particular, assumed as forced full-range by swscale), we exclude them
from negotiation altogether for the time being, to get this API merged.
After filter negotiation is available, we can relax the
grayscale-is-forced-jpeg restriction again, when it will be more
feasible to do so without breaking a million test cases.
Note that this commit updates one FATE test as a consequence of the
sanity fallback for non-YUV formats. In particular, the test case now
writes rgb24(pc, gbr/unspecified/unspecified) to the matroska file,
instead of rgb24(unspecified/unspecified/unspecified) as before.
doc/APIchanges | 3 +
libavfilter/avfilter.c | 17 ++++-
libavfilter/avfilter.h | 28 +++++++
libavfilter/avfiltergraph.c | 173 +++++++++++++++++++++++++++++++++++++++++++-
libavfilter/formats.c | 121 +++++++++++++++++++++++++++++++
libavfilter/formats.h | 54 ++++++++++++++
libavfilter/internal.h | 6 ++
libavfilter/vaapi_vpp.c | 4 +
libavfilter/version.h | 4 +-
libavfilter/video.c | 2 +
tests/ref/fate/shortest-sub | 4 +-
11 files changed, 406 insertions(+), 10 deletions(-)
ffmpeg -v trace -hwaccel vaapi -vaapi_device /dev/dri/renderD128 -i ./example_720p30.y4m -vf 'format=nv12|vaapi, hwupload' -c:v h264_vaapi ./out_720p30.264 -y
Successfully opened the file.
Stream mapping:
Stream #0:0 -> #0:0 (rawvideo (native) -> h264 (h264_vaapi))
[vost#0:0/h264_vaapi @ 0x55a86a145c70] Starting thread...
[vf#0:0 @ 0x55a86a104290] Starting thread...
[vist#0:0/rawvideo @ 0x55a86a105cc0] Starting thread...
[in#0/yuv4mpegpipe @ 0x55a86a11d250] Starting thread...
Press [q] to stop, [?] for help
[rawvideo @ 0x55a86a144d40] PACKET SIZE: 1382400, STRIDE: 1920
[AVFilterGraph @ 0x7fa5e0000ff0] Setting 'pix_fmts' to value 'nv12|vaapi'
detected 24 logical cores
[graph 0 input from stream 0:0 @ 0x7fa5e0003ab0] Setting 'video_size' to value '1280x720'
[graph 0 input from stream 0:0 @ 0x7fa5e0003ab0] Setting 'pix_fmt' to value '0'
[graph 0 input from stream 0:0 @ 0x7fa5e0003ab0] Setting 'time_base' to value '1/30'
[graph 0 input from stream 0:0 @ 0x7fa5e0003ab0] Setting 'pixel_aspect' to value '0/1'
[graph 0 input from stream 0:0 @ 0x7fa5e0003ab0] Setting 'colorspace' to value 'unknown'
[graph 0 input from stream 0:0 @ 0x7fa5e0003ab0] Setting 'range' to value 'unknown'
[graph 0 input from stream 0:0 @ 0x7fa5e0003ab0] Setting 'frame_rate' to value '30/1'
[rawvideo @ 0x55a86a144d40] PACKET SIZE: 1382400, STRIDE: 1920
[graph 0 input from stream 0:0 @ 0x7fa5e0003ab0] w:1280 h:720 pixfmt:yuv420p tb:1/30 fr:30/1 sar:0/1 csp:unknown range:unknown
[format @ 0x7fa5e0004390] Setting 'pix_fmts' to value 'vaapi'
[auto_scale_0 @ 0x7fa5e00055d0] w:iw h:ih flags:'' interl:0
[Parsed_format_0 @ 0x7fa5e0003450] auto-inserting filter 'auto_scale_0' between the filter 'graph 0 input from stream 0:0' and the filter 'Parsed_format_0'
[AVFilterGraph @ 0x7fa5e0000ff0] query_formats: 5 queried, 9 merged, 3 already done, 0 delayed
Segmentation fault (core dumped)
--------
-----Original Message-----
From: ffmpeg-devel <ffmpeg-devel-bounces@ffmpeg.org> On Behalf Of Niklas Haas
Sent: Sunday, December 31, 2023 4:51 PM
To: ffmpeg-devel@ffmpeg.org
Subject: Re: [FFmpeg-devel] [PATCH v2 00/15] YUV colorspace filter negotiation
On Sun, 24 Dec 2023 10:35:22 -0800 Niklas Haas <ffmpeg@haasn.xyz> wrote:
> On Wed, 13 Dec 2023 14:11:57 +0100 Niklas Haas <ffmpeg@haasn.xyz> wrote:
> > Split off from my YUVJ removal series. This implements all of the
> > libavfilter changes needed to fully deprecate YUVJ, but does not yet
> > remove YUVJ, nor add an AVCodec API for advertising colorspace support.
> >
> > Update includes all of the feedback that was brought up by Anton.
> > the major change from v1 is that YUV metadata configured on a link
> > is no longer strictly required to be consistent with the frames,
> > simply to avoid breaking users that don't set this link metadata.
> > Effectively, this allows us to have a grace period where the field
> > functions more like a hint than an authoritative field.
>
> Ping, any further feedback? If not, I will merge this sometime after Christmas.
Merged as 1b0ca477..44a33fccd
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
^ permalink raw reply [flat|nested] 26+ messages in thread