From patchwork Wed Dec 13 13:12:02 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Niklas Haas X-Patchwork-Id: 45105 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:1225:b0:181:818d:5e7f with SMTP id v37csp4264048pzf; Wed, 13 Dec 2023 05:17:06 -0800 (PST) X-Google-Smtp-Source: AGHT+IG2+riBboLA/83N0uEsYyIZV4VQFYHuWCilMAXqq7A2M/DjPp1s50x/RZqYonlsI5V0E3R0 X-Received: by 2002:a05:6402:1e86:b0:54f:55ad:f4e6 with SMTP id f6-20020a0564021e8600b0054f55adf4e6mr2274777edf.70.1702473426668; Wed, 13 Dec 2023 05:17:06 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1702473426; cv=none; d=google.com; s=arc-20160816; b=EPITOu5kn1MxCOwQyvx4CbLh+L6YoVwWwp1dTlSuVPD1ONze15FBu0WFFSGyUkeIX1 d+if+2PuSD8MU/zTJvxxFFfuxRNpYQKmzYh26oGraTSaK8vMGlVW84WOu+RFda9wpwuJ K/X2qkiFElU6xG+etP5ObJfaKlW8ruOlHA2UcY8r//2KR8SrYTvVqNhDuAe5tMkXCvAb ZNW3Lp89wm1+x8sEl5VpbKjtwY/DybF+30KxeLr+eG+Tux/tgVe89zHsB5pICv7ytINv oftMFC/TBUiixzk07yi0Gl1u+IDUDnC5r1ZOUgGd5Ho3YTLdeSAmCUVUFjZ6Vmh91M9P X6ZQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=wHsqU2hcWHInJ1JSDaXXltHeXXBO9XksTm1pFNO7vXs=; fh=xmAeKtysnShNOmkhiJmYkS30uw4Fu2hvBJ7qlIwukxQ=; b=bxLJvBVbXzVU0mdcyObzR5E3hy1xqh8LjhZBF5QGsXi96oWxw2KtSGXNuxu3llawYu 12l93Fz4EGqos4BQXCpYzQIvKw/PPJGkAGNV1NbITj/F6h2sVL6ZkOsQwpUHdpXrPEbl izvvdJjT0mX/tYACvoivWtpL9eTzCUDzjOWrJW0v8+wBlhMiAvFi23F3gdZkop3sxuB6 18AiF2PjYwLf/0WzpaWEFLvp73cuTM6I5xVn3YP3CUH7oJdmjhnz3zSeEDgIDJ3VcClG iELk1tl4dCeNwY3ki3X+Ju9eQbB7YmCsI0ycG/25HENPqus1zAZZ4QxW4CbUrXEOqDyX Wj6Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@haasn.xyz header.s=mail header.b=SIvoA5wD; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id s30-20020a50ab1e000000b0054f58424532si4799153edc.421.2023.12.13.05.17.05; Wed, 13 Dec 2023 05:17:06 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@haasn.xyz header.s=mail header.b=SIvoA5wD; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 0311068D201; Wed, 13 Dec 2023 15:15:55 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from haasn.dev (haasn.dev [78.46.187.166]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 074A168D1DD for ; Wed, 13 Dec 2023 15:15:44 +0200 (EET) Received: from haasn.dev (unknown [10.30.0.2]) by haasn.dev (Postfix) with ESMTP id 2F80D4BCE9; Wed, 13 Dec 2023 14:15:40 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=haasn.xyz; s=mail; t=1702473340; bh=ZcYFxTklFesptx2PECHMCGDTCO2G3Yq5crfJkSJxChU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=SIvoA5wDvNhKsyY736h4XPuqDi5bXRL3IR2qYNCzYuGZOgnWbg0KgtDINzrYRcyq+ R/QEr+3QYqQLvYv0kkprSqQBIlxYD5fonk3Zqj0pnBL06z0ERK3gqQm9O7sBd3h9nW jhKPgsWIY2xg1GLxa8YZk+rr/J2iVy7Hi2YgK4UQ= From: Niklas Haas To: ffmpeg-devel@ffmpeg.org Date: Wed, 13 Dec 2023 14:12:02 +0100 Message-ID: <20231213131536.10242-6-ffmpeg@haasn.xyz> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20231213131536.10242-1-ffmpeg@haasn.xyz> References: <20231213131536.10242-1-ffmpeg@haasn.xyz> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 05/15] avfilter/buffersrc: add color_space/range parameters X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Niklas Haas Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: i9r2WJ0s1GXM From: Niklas Haas To allow adding proper negotiation, in particular, to fftools. These values will simply be negotiated downstream for YUV formats, and ignored otherwise. --- doc/filters.texi | 10 ++++++ libavfilter/buffersrc.c | 74 ++++++++++++++++++++++++++++++++++++----- libavfilter/buffersrc.h | 6 ++++ 3 files changed, 82 insertions(+), 8 deletions(-) diff --git a/doc/filters.texi b/doc/filters.texi index 6d00ba2c3f..f182dc2ddb 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -28025,6 +28025,16 @@ Specify the timebase assumed by the timestamps of the buffered frames. @item frame_rate Specify the frame rate expected for the video stream. +@item colorspace +A string representing the color space of the buffered video frames. +It may be a number corresponding to a color space, or a color space +name. + +@item range +A string representing the color range of the buffered video frames. +It may be a number corresponding to a color range, or a color range +name. + @item pixel_aspect, sar The sample (pixel) aspect ratio of the input video. diff --git a/libavfilter/buffersrc.c b/libavfilter/buffersrc.c index b0a905d455..1c5db94696 100644 --- a/libavfilter/buffersrc.c +++ b/libavfilter/buffersrc.c @@ -28,6 +28,7 @@ #include "libavutil/channel_layout.h" #include "libavutil/common.h" #include "libavutil/frame.h" +#include "libavutil/hwcontext.h" #include "libavutil/imgutils.h" #include "libavutil/internal.h" #include "libavutil/opt.h" @@ -50,6 +51,8 @@ typedef struct BufferSourceContext { /* video only */ int w, h; enum AVPixelFormat pix_fmt; + enum AVColorSpace color_space; + enum AVColorRange color_range; AVRational pixel_aspect; AVBufferRef *hw_frames_ctx; @@ -65,10 +68,13 @@ typedef struct BufferSourceContext { int64_t last_pts; } BufferSourceContext; -#define CHECK_VIDEO_PARAM_CHANGE(s, c, width, height, format, pts)\ - if (c->w != width || c->h != height || c->pix_fmt != format) {\ - av_log(s, AV_LOG_INFO, "filter context - w: %d h: %d fmt: %d, incoming frame - w: %d h: %d fmt: %d pts_time: %s\n",\ - c->w, c->h, c->pix_fmt, width, height, format, av_ts2timestr(pts, &s->outputs[0]->time_base));\ +#define CHECK_VIDEO_PARAM_CHANGE(s, c, width, height, format, csp, range, pts)\ + if (c->w != width || c->h != height || c->pix_fmt != format ||\ + c->color_space != csp || c->color_range != range) {\ + av_log(s, AV_LOG_INFO, "filter context - w: %d h: %d fmt: %d csp: %s range: %s, incoming frame - w: %d h: %d fmt: %d csp: %s range: %s pts_time: %s\n",\ + c->w, c->h, c->pix_fmt, av_color_space_name(c->color_space), av_color_range_name(c->color_range),\ + width, height, format, av_color_space_name(csp), av_color_range_name(range),\ + av_ts2timestr(pts, &s->outputs[0]->time_base));\ av_log(s, AV_LOG_WARNING, "Changing video frame properties on the fly is not supported by all filters.\n");\ } @@ -89,6 +95,8 @@ AVBufferSrcParameters *av_buffersrc_parameters_alloc(void) return NULL; par->format = -1; + par->color_range = AVCOL_RANGE_UNSPECIFIED; + par->color_space = AVCOL_SPC_UNSPECIFIED; return par; } @@ -119,6 +127,10 @@ int av_buffersrc_parameters_set(AVFilterContext *ctx, AVBufferSrcParameters *par if (!s->hw_frames_ctx) return AVERROR(ENOMEM); } + if (param->color_space != AVCOL_SPC_UNSPECIFIED) + s->color_space = param->color_space; + if (param->color_range != AVCOL_RANGE_UNSPECIFIED) + s->color_range = param->color_range; break; case AVMEDIA_TYPE_AUDIO: if (param->format != AV_SAMPLE_FMT_NONE) { @@ -206,7 +218,8 @@ FF_ENABLE_DEPRECATION_WARNINGS switch (ctx->outputs[0]->type) { case AVMEDIA_TYPE_VIDEO: CHECK_VIDEO_PARAM_CHANGE(ctx, s, frame->width, frame->height, - frame->format, frame->pts); + frame->format, frame->colorspace, + frame->color_range, frame->pts); break; case AVMEDIA_TYPE_AUDIO: /* For layouts unknown on input but known on link after negotiation. */ @@ -303,10 +316,11 @@ static av_cold int init_video(AVFilterContext *ctx) return AVERROR(EINVAL); } - av_log(ctx, AV_LOG_VERBOSE, "w:%d h:%d pixfmt:%s tb:%d/%d fr:%d/%d sar:%d/%d\n", + av_log(ctx, AV_LOG_VERBOSE, "w:%d h:%d pixfmt:%s tb:%d/%d fr:%d/%d sar:%d/%d csp:%s range:%s\n", c->w, c->h, av_get_pix_fmt_name(c->pix_fmt), c->time_base.num, c->time_base.den, c->frame_rate.num, c->frame_rate.den, - c->pixel_aspect.num, c->pixel_aspect.den); + c->pixel_aspect.num, c->pixel_aspect.den, + av_color_space_name(c->color_space), av_color_range_name(c->color_range)); return 0; } @@ -329,6 +343,30 @@ static const AVOption buffer_options[] = { { "pixel_aspect", "sample aspect ratio", OFFSET(pixel_aspect), AV_OPT_TYPE_RATIONAL, { .dbl = 0 }, 0, DBL_MAX, V }, { "time_base", NULL, OFFSET(time_base), AV_OPT_TYPE_RATIONAL, { .dbl = 0 }, 0, DBL_MAX, V }, { "frame_rate", NULL, OFFSET(frame_rate), AV_OPT_TYPE_RATIONAL, { .dbl = 0 }, 0, DBL_MAX, V }, + { "colorspace", "select colorspace", OFFSET(color_space), AV_OPT_TYPE_INT, {.i64=AVCOL_SPC_UNSPECIFIED}, 0, AVCOL_SPC_NB-1, V, "colorspace"}, + { "gbr", NULL, 0, AV_OPT_TYPE_CONST, {.i64=AVCOL_SPC_RGB}, INT_MIN, INT_MAX, V, "colorspace"}, + { "bt709", NULL, 0, AV_OPT_TYPE_CONST, {.i64=AVCOL_SPC_BT709}, INT_MIN, INT_MAX, V, "colorspace"}, + { "unknown", NULL, 0, AV_OPT_TYPE_CONST, {.i64=AVCOL_SPC_UNSPECIFIED}, INT_MIN, INT_MAX, V, "colorspace"}, + { "fcc", NULL, 0, AV_OPT_TYPE_CONST, {.i64=AVCOL_SPC_FCC}, INT_MIN, INT_MAX, V, "colorspace"}, + { "bt470bg", NULL, 0, AV_OPT_TYPE_CONST, {.i64=AVCOL_SPC_BT470BG}, INT_MIN, INT_MAX, V, "colorspace"}, + { "smpte170m", NULL, 0, AV_OPT_TYPE_CONST, {.i64=AVCOL_SPC_SMPTE170M}, INT_MIN, INT_MAX, V, "colorspace"}, + { "smpte240m", NULL, 0, AV_OPT_TYPE_CONST, {.i64=AVCOL_SPC_SMPTE240M}, INT_MIN, INT_MAX, V, "colorspace"}, + { "ycgco", NULL, 0, AV_OPT_TYPE_CONST, {.i64=AVCOL_SPC_YCGCO}, INT_MIN, INT_MAX, V, "colorspace"}, + { "bt2020nc", NULL, 0, AV_OPT_TYPE_CONST, {.i64=AVCOL_SPC_BT2020_NCL}, INT_MIN, INT_MAX, V, "colorspace"}, + { "bt2020c", NULL, 0, AV_OPT_TYPE_CONST, {.i64=AVCOL_SPC_BT2020_CL}, INT_MIN, INT_MAX, V, "colorspace"}, + { "smpte2085", NULL, 0, AV_OPT_TYPE_CONST, {.i64=AVCOL_SPC_SMPTE2085}, INT_MIN, INT_MAX, V, "colorspace"}, + { "chroma-derived-nc", NULL, 0, AV_OPT_TYPE_CONST, {.i64=AVCOL_SPC_CHROMA_DERIVED_NCL},INT_MIN, INT_MAX, V, "colorspace"}, + { "chroma-derived-c", NULL, 0, AV_OPT_TYPE_CONST, {.i64=AVCOL_SPC_CHROMA_DERIVED_CL}, INT_MIN, INT_MAX, V, "colorspace"}, + { "ictcp", NULL, 0, AV_OPT_TYPE_CONST, {.i64=AVCOL_SPC_ICTCP}, INT_MIN, INT_MAX, V, "colorspace"}, + { "range", "select color range", OFFSET(color_range), AV_OPT_TYPE_INT, {.i64=AVCOL_RANGE_UNSPECIFIED}, 0, AVCOL_RANGE_NB-1, V, "range"}, + { "unspecified", NULL, 0, AV_OPT_TYPE_CONST, {.i64=AVCOL_RANGE_UNSPECIFIED}, 0, 0, V, "range"}, + { "unknown", NULL, 0, AV_OPT_TYPE_CONST, {.i64=AVCOL_RANGE_UNSPECIFIED}, 0, 0, V, "range"}, + { "limited", NULL, 0, AV_OPT_TYPE_CONST, {.i64=AVCOL_RANGE_MPEG}, 0, 0, V, "range"}, + { "tv", NULL, 0, AV_OPT_TYPE_CONST, {.i64=AVCOL_RANGE_MPEG}, 0, 0, V, "range"}, + { "mpeg", NULL, 0, AV_OPT_TYPE_CONST, {.i64=AVCOL_RANGE_MPEG}, 0, 0, V, "range"}, + { "full", NULL, 0, AV_OPT_TYPE_CONST, {.i64=AVCOL_RANGE_JPEG}, 0, 0, V, "range"}, + { "pc", NULL, 0, AV_OPT_TYPE_CONST, {.i64=AVCOL_RANGE_JPEG}, 0, 0, V, "range"}, + { "jpeg", NULL, 0, AV_OPT_TYPE_CONST, {.i64=AVCOL_RANGE_JPEG}, 0, 0, V, "range"}, { NULL }, }; @@ -426,13 +464,33 @@ static int query_formats(AVFilterContext *ctx) AVFilterChannelLayouts *channel_layouts = NULL; AVFilterFormats *formats = NULL; AVFilterFormats *samplerates = NULL; + AVFilterFormats *color_spaces = NULL; + AVFilterFormats *color_ranges = NULL; int ret; switch (ctx->outputs[0]->type) { - case AVMEDIA_TYPE_VIDEO: + case AVMEDIA_TYPE_VIDEO:; + enum AVPixelFormat swfmt = c->pix_fmt; + if (av_pix_fmt_desc_get(swfmt)->flags & AV_PIX_FMT_FLAG_HWACCEL) { + if (!c->hw_frames_ctx) { + av_log(ctx, AV_LOG_ERROR, "Setting BufferSourceContext.pix_fmt " + "to a HW format requires hw_frames_ctx to be non-NULL!\n"); + return AVERROR(EINVAL); + } + swfmt = ((AVHWFramesContext *) c->hw_frames_ctx->data)->sw_format; + } if ((ret = ff_add_format (&formats, c->pix_fmt)) < 0 || (ret = ff_set_common_formats (ctx , formats )) < 0) return ret; + /* force specific colorspace/range downstream only for ordinary YUV */ + if (ff_fmt_is_regular_yuv(swfmt)) { + if ((ret = ff_add_format(&color_spaces, c->color_space)) < 0 || + (ret = ff_set_common_color_spaces(ctx, color_spaces)) < 0) + return ret; + if ((ret = ff_add_format(&color_ranges, c->color_range)) < 0 || + (ret = ff_set_common_color_ranges(ctx, color_ranges)) < 0) + return ret; + } break; case AVMEDIA_TYPE_AUDIO: if ((ret = ff_add_format (&formats , c->sample_fmt )) < 0 || diff --git a/libavfilter/buffersrc.h b/libavfilter/buffersrc.h index 3b248b37cd..1e29f0340e 100644 --- a/libavfilter/buffersrc.h +++ b/libavfilter/buffersrc.h @@ -123,6 +123,12 @@ typedef struct AVBufferSrcParameters { * Audio only, the audio channel layout */ AVChannelLayout ch_layout; + + /** + * Video only, the YUV colorspace and range. + */ + enum AVColorSpace color_space; + enum AVColorRange color_range; } AVBufferSrcParameters; /**