From patchwork Sun Jul 7 19:26:50 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Thompson X-Patchwork-Id: 13847 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id D3D6E4492AF for ; Sun, 7 Jul 2019 22:27:02 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id B94E268AE5A; Sun, 7 Jul 2019 22:27:02 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wm1-f48.google.com (mail-wm1-f48.google.com [209.85.128.48]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id D9CC568A9BE for ; Sun, 7 Jul 2019 22:26:55 +0300 (EEST) Received: by mail-wm1-f48.google.com with SMTP id a15so13663985wmj.5 for ; Sun, 07 Jul 2019 12:26:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jkqxz-net.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=Ona1DG09YBXvb7rssQYifvDXCJTzv9NMP05kzby75YE=; b=cg2i8YMXwlayKmJAEsbFUYcQ2/vcpMq7qjuOK5TGU+K6T9cNLEyB71GfRkBjnBk6W8 zV5hGlqhJ9a/tdRPm4PAIEBt0qUDIyS+j0rEK+GQAU+ysQikCZowWt4eQuTeMMslnuwY EtNbPSErXOuUkIosJtkSMXvTBQP6pzMuhzVVKHXQNmLwVpQcepCf7gMfYcO5g8TSj7Hv ziqClwcwWdMJRFQLXjDW/8gh+6DE5qVngShW92gzJvkEhuFF/E/9S4Tt2o8HvuDYdbua IhJA/5ea/nAWfrBe9l1o6Dn1FlIT7l4LH00aBbEE0IdGhxee4hf576HpGGp0OycDVNzP N6Dw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Ona1DG09YBXvb7rssQYifvDXCJTzv9NMP05kzby75YE=; b=dcZmvsk12DPVTIF70yT6SVLuVoeVbRUSJog2VloxkQXlJZI8q6L9HqasTx+rG3cEVM bWRiqGdMssiZxQa1hATDLw/U1QN2gs++TzpUfZgFdQ3MrWyHHJnz/PpeTcLM5BOwP161 O2vySzEVqo1YHJ7n3ZIQZ6bFREatqoBSgBd5z7pfeudpIYslip9F3xeh6Wn2WGQGKGtW sCvvlMMrSk7DpqNlHTV3gG3n3spZ77VkqN8gQAn4H4oRTAPrckRA3vCA82IVVDQC434g erZx/HW481mLEcikoMwlnlJAeajMP1QOLSPhT1nKPV8BFAjGGBd1Ir5zAcwk4qhb4aey 2Arw== X-Gm-Message-State: APjAAAVtvzt/mblU8kAAwIq7cPWz4wRqf9cvrqYbA/l4HSFD/7BHTJdO hdU2yB2b6U5qs1e3fbb7yTmOWSv4Qi8= X-Google-Smtp-Source: APXvYqxUR4t7wYJiGv2QUecS646IMHY/zyzqBS3d0bdcoy0Ir5uVrniFD8E2nM6yDjhmnI8l7LBP7Q== X-Received: by 2002:a1c:7015:: with SMTP id l21mr12482768wmc.82.1562527614883; Sun, 07 Jul 2019 12:26:54 -0700 (PDT) Received: from rywe.jkqxz.net (cpc91242-cmbg18-2-0-cust650.5-4.cable.virginm.net. [82.8.130.139]) by smtp.gmail.com with ESMTPSA id z25sm17057904wmf.38.2019.07.07.12.26.54 for (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Sun, 07 Jul 2019 12:26:54 -0700 (PDT) From: Mark Thompson To: ffmpeg-devel@ffmpeg.org Date: Sun, 7 Jul 2019 20:26:50 +0100 Message-Id: <20190707192650.7161-2-sw@jkqxz.net> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190707192650.7161-1-sw@jkqxz.net> References: <20190707192650.7161-1-sw@jkqxz.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v4 2/2] lavfi: addroi filter X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" This can be used to add region of interest side data to video frames. --- doc/filters.texi | 73 +++++++++++ libavfilter/Makefile | 1 + libavfilter/allfilters.c | 1 + libavfilter/vf_addroi.c | 269 +++++++++++++++++++++++++++++++++++++++ 4 files changed, 344 insertions(+) create mode 100644 libavfilter/vf_addroi.c diff --git a/doc/filters.texi b/doc/filters.texi index ee6a93ffbf..f7145532c7 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -5876,6 +5876,79 @@ build. Below is a description of the currently available video filters. +@section addroi + +Mark a region of interest in a video frame. + +The frame data is passed through unchanged, but metadata is attached +to the frame indicating regions of interest which can affect the +behaviour of later encoding. Multiple regions can be marked by +applying the filter multiple times. + +@table @option +@item x +Region distance in pixels from the left edge of the frame. +@item y +Region distance in pixels from the top edge of the frame. +@item w +Region width in pixels. +@item h +Region height in pixels. + +The parameters @var{x}, @var{y}, @var{w} and @var{h} are expressions, +and may contain the following variables: +@table @option +@item iw +Width of the input frame. +@item ih +Height of the input frame. +@end table + +@item qoffset +Quantisation offset to apply within the region. + +This must be a real value in the range -1 to +1. A value of zero +indicates no quality change. A negative value asks for better quality +(less quantisation), while a positive value asks for worse quality +(greater quantisation). + +The range is calibrated so that the extreme values indicate the +largest possible offset - if the rest of the frame is encoded with the +worst possible quality, an offset of -1 indicates that this region +should be encoded with the best possible quality anyway. Intermediate +values are then interpolated in some codec-dependent way. + +For example, in 10-bit H.264 the quantisation parameter varies between +-12 and 51. A typical qoffset value of -1/10 therefore indicates that +this region should be encoded with a QP around one-tenth of the full +range better than the rest of the frame. So, if most of the frame +were to be encoded with a QP of around 30, this region would get a QP +of around 24 (an offset of approximately -1/10 * (51 - -12) = -6.3). +An extreme value of -1 would indicate that this region should be +encoded with the best possible quality regardless of the treatment of +the rest of the frame - that is, should be encoded at a QP of -12. +@item clear +If set to true, remove any existing regions of interest marked on the +frame before adding the new one. +@end table + +@subsection Examples + +@itemize +@item +Mark the centre quarter of the frame as interesting. +@example +addroi=iw/4:ih/4:iw/2:ih/2:-1/10 +@end example +@item +Mark the 100-pixel-wide region on the left edge of the frame as very +uninteresting (to be encoded at much lower quality than the rest of +the frame). +@example +addroi=0:0:100:ih:+1/5 +@end example +@end itemize + @section alphaextract Extract the alpha component from the input as a grayscale video. This diff --git a/libavfilter/Makefile b/libavfilter/Makefile index 455c809b15..00c402c3bc 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -152,6 +152,7 @@ OBJS-$(CONFIG_SINE_FILTER) += asrc_sine.o OBJS-$(CONFIG_ANULLSINK_FILTER) += asink_anullsink.o # video filters +OBJS-$(CONFIG_ADDROI_FILTER) += vf_addroi.o OBJS-$(CONFIG_ALPHAEXTRACT_FILTER) += vf_extractplanes.o OBJS-$(CONFIG_ALPHAMERGE_FILTER) += vf_alphamerge.o OBJS-$(CONFIG_AMPLIFY_FILTER) += vf_amplify.o diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index 04a3df7d56..abd726d616 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -143,6 +143,7 @@ extern AVFilter ff_asrc_sine; extern AVFilter ff_asink_anullsink; +extern AVFilter ff_vf_addroi; extern AVFilter ff_vf_alphaextract; extern AVFilter ff_vf_alphamerge; extern AVFilter ff_vf_amplify; diff --git a/libavfilter/vf_addroi.c b/libavfilter/vf_addroi.c new file mode 100644 index 0000000000..489998ce73 --- /dev/null +++ b/libavfilter/vf_addroi.c @@ -0,0 +1,269 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "libavutil/avassert.h" +#include "libavutil/eval.h" +#include "libavutil/opt.h" +#include "avfilter.h" +#include "internal.h" + +enum { + X, Y, W, H, + NB_PARAMS, +}; +static const char *addroi_param_names[] = { + "x", "y", "w", "h", +}; + +enum { + VAR_IW, + VAR_IH, + NB_VARS, +}; +static const char *const addroi_var_names[] = { + "iw", + "ih", +}; + +typedef struct AddROIContext { + const AVClass *class; + + char *region_str[NB_PARAMS]; + AVExpr *region_expr[NB_PARAMS]; + + int region[NB_PARAMS]; + AVRational qoffset; + + int clear; +} AddROIContext; + +static int addroi_config_input(AVFilterLink *inlink) +{ + AVFilterContext *avctx = inlink->dst; + AddROIContext *ctx = avctx->priv; + int i; + double vars[NB_VARS]; + double val; + + vars[VAR_IW] = inlink->w; + vars[VAR_IH] = inlink->h; + + for (i = 0; i < NB_PARAMS; i++) { + int max_value; + switch (i) { + case X: max_value = inlink->w; break; + case Y: max_value = inlink->h; break; + case W: max_value = inlink->w - ctx->region[X]; break; + case H: max_value = inlink->h - ctx->region[Y]; break; + } + + val = av_expr_eval(ctx->region_expr[i], vars, NULL); + if (val < 0.0) { + av_log(avctx, AV_LOG_WARNING, "Calculated value %g for %s is " + "less than zero - using zero instead.\n", val, + addroi_param_names[i]); + val = 0.0; + } else if (val > max_value) { + av_log(avctx, AV_LOG_WARNING, "Calculated value %g for %s is " + "greater than maximum allowed value %d - " + "using %d instead.\n", val, addroi_param_names[i], + max_value, max_value); + val = max_value; + } + ctx->region[i] = val; + } + + return 0; +} + +static int addroi_filter_frame(AVFilterLink *inlink, AVFrame *frame) +{ + AVFilterContext *avctx = inlink->dst; + AVFilterLink *outlink = avctx->outputs[0]; + AddROIContext *ctx = avctx->priv; + AVRegionOfInterest *roi; + AVFrameSideData *sd; + int err; + + if (ctx->clear) { + av_frame_remove_side_data(frame, AV_FRAME_DATA_REGIONS_OF_INTEREST); + sd = NULL; + } else { + sd = av_frame_get_side_data(frame, AV_FRAME_DATA_REGIONS_OF_INTEREST); + } + if (sd) { + const AVRegionOfInterest *old_roi; + uint32_t old_roi_size; + AVBufferRef *roi_ref; + int nb_roi, i; + + old_roi = (const AVRegionOfInterest*)sd->data; + old_roi_size = old_roi->self_size; + av_assert0(old_roi_size && sd->size % old_roi_size == 0); + nb_roi = sd->size / old_roi_size + 1; + + roi_ref = av_buffer_alloc(sizeof(*roi) * nb_roi); + if (!roi_ref) { + err = AVERROR(ENOMEM); + goto fail; + } + roi = (AVRegionOfInterest*)roi_ref->data; + + for (i = 0; i < nb_roi - 1; i++) { + old_roi = (const AVRegionOfInterest*) + (sd->data + old_roi_size * i); + + roi[i] = (AVRegionOfInterest) { + .self_size = sizeof(*roi), + .top = old_roi->top, + .bottom = old_roi->bottom, + .left = old_roi->left, + .right = old_roi->right, + .qoffset = old_roi->qoffset, + }; + } + + roi[nb_roi - 1] = (AVRegionOfInterest) { + .self_size = sizeof(*roi), + .top = ctx->region[Y], + .bottom = ctx->region[Y] + ctx->region[H], + .left = ctx->region[X], + .right = ctx->region[X] + ctx->region[W], + .qoffset = ctx->qoffset, + }; + + av_frame_remove_side_data(frame, AV_FRAME_DATA_REGIONS_OF_INTEREST); + + sd = av_frame_new_side_data_from_buf(frame, + AV_FRAME_DATA_REGIONS_OF_INTEREST, + roi_ref); + if (!sd) { + av_buffer_unref(&roi_ref); + err = AVERROR(ENOMEM); + goto fail; + } + + } else { + sd = av_frame_new_side_data(frame, AV_FRAME_DATA_REGIONS_OF_INTEREST, + sizeof(AVRegionOfInterest)); + if (!sd) { + err = AVERROR(ENOMEM); + goto fail; + } + roi = (AVRegionOfInterest*)sd->data; + *roi = (AVRegionOfInterest) { + .self_size = sizeof(*roi), + .top = ctx->region[Y], + .bottom = ctx->region[Y] + ctx->region[H], + .left = ctx->region[X], + .right = ctx->region[X] + ctx->region[W], + .qoffset = ctx->qoffset, + }; + } + + return ff_filter_frame(outlink, frame); + +fail: + av_frame_free(&frame); + return err; +} + +static av_cold int addroi_init(AVFilterContext *avctx) +{ + AddROIContext *ctx = avctx->priv; + int i, err; + + for (i = 0; i < NB_PARAMS; i++) { + err = av_expr_parse(&ctx->region_expr[i], ctx->region_str[i], + addroi_var_names, NULL, NULL, NULL, NULL, + 0, avctx); + if (err < 0) { + av_log(ctx, AV_LOG_ERROR, + "Error parsing %s expression '%s'.\n", + addroi_param_names[i], ctx->region_str[i]); + return err; + } + } + + return 0; +} + +static av_cold void addroi_uninit(AVFilterContext *avctx) +{ + AddROIContext *ctx = avctx->priv; + int i; + + for (i = 0; i < NB_PARAMS; i++) { + av_expr_free(ctx->region_expr[i]); + ctx->region_expr[i] = NULL; + } +} + +#define OFFSET(x) offsetof(AddROIContext, x) +#define FLAGS AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_FILTERING_PARAM +static const AVOption addroi_options[] = { + { "x", "Region distance from left edge of frame.", + OFFSET(region_str[X]), AV_OPT_TYPE_STRING, { .str = "0" }, .flags = FLAGS }, + { "y", "Region distance from top edge of frame.", + OFFSET(region_str[Y]), AV_OPT_TYPE_STRING, { .str = "0" }, .flags = FLAGS }, + { "w", "Region width.", + OFFSET(region_str[W]), AV_OPT_TYPE_STRING, { .str = "0" }, .flags = FLAGS }, + { "h", "Region height.", + OFFSET(region_str[H]), AV_OPT_TYPE_STRING, { .str = "0" }, .flags = FLAGS }, + + { "qoffset", "Quantisation offset to apply in the region.", + OFFSET(qoffset), AV_OPT_TYPE_RATIONAL, { .dbl = -0.1 }, -1, +1, FLAGS }, + + { "clear", "Remove any existing regions of interest before adding the new one.", + OFFSET(clear), AV_OPT_TYPE_BOOL, { .i64 = 0 }, 0, 1, FLAGS }, + + { NULL } +}; + +AVFILTER_DEFINE_CLASS(addroi); + +static const AVFilterPad addroi_inputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + .config_props = addroi_config_input, + .filter_frame = addroi_filter_frame, + }, + { NULL } +}; + +static const AVFilterPad addroi_outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + }, + { NULL } +}; + +AVFilter ff_vf_addroi = { + .name = "addroi", + .description = NULL_IF_CONFIG_SMALL("Add region of interest to frame."), + .init = addroi_init, + .uninit = addroi_uninit, + + .priv_size = sizeof(AddROIContext), + .priv_class = &addroi_class, + + .inputs = addroi_inputs, + .outputs = addroi_outputs, +};