[FFmpeg-devel,5/5] lavfi: addroi filter

Submitted by Mark Thompson on Feb. 27, 2019, 10 p.m.

Details

Message ID 20190227220023.16040-5-sw@jkqxz.net
State New
Headers show

Commit Message

Mark Thompson Feb. 27, 2019, 10 p.m.
This can be used to add region of interest side data to video frames.
---
 libavfilter/Makefile     |   1 +
 libavfilter/allfilters.c |   1 +
 libavfilter/vf_addroi.c  | 237 +++++++++++++++++++++++++++++++++++++++
 3 files changed, 239 insertions(+)
 create mode 100644 libavfilter/vf_addroi.c

Comments

Moritz Barsnick Feb. 27, 2019, 11:42 p.m.
On Wed, Feb 27, 2019 at 22:00:23 +0000, Mark Thompson wrote:
> +static const AVOption addroi_options[] = {
> +    { "top",    "Region distance from top edge of frame",
> +      OFFSET(region_str[TOP]),    AV_OPT_TYPE_STRING, { .str = "0" }, .flags = FLAGS },
> +    { "bottom", "Region distance from bottom edge of frame",
> +      OFFSET(region_str[BOTTOM]), AV_OPT_TYPE_STRING, { .str = "0" }, .flags = FLAGS },
> +    { "left",   "Region distance from left edge of frame",
> +      OFFSET(region_str[LEFT]),   AV_OPT_TYPE_STRING, { .str = "0" }, .flags = FLAGS },
> +    { "right",  "Region distance from right edge of frame",
> +      OFFSET(region_str[RIGHT]),  AV_OPT_TYPE_STRING, { .str = "0" }, .flags = FLAGS },

My comment may not apply here, but I sometimes wish all rectangle
defining filters had compatible options.

I.e. drawbox and crop and delogo (which I use interchangably - e.g.
first the one to visually determine the other's dimensions) aren't
compatible with respect to order of unnamed options, and to names of
variables in their expression (w, iw, ow, ...).

Can ROIs not also be defined as x, y, w, h?

Moritz
Mark Thompson Feb. 28, 2019, 1:18 a.m.
On 27/02/2019 23:42, Moritz Barsnick wrote:
> On Wed, Feb 27, 2019 at 22:00:23 +0000, Mark Thompson wrote:
>> +static const AVOption addroi_options[] = {
>> +    { "top",    "Region distance from top edge of frame",
>> +      OFFSET(region_str[TOP]),    AV_OPT_TYPE_STRING, { .str = "0" }, .flags = FLAGS },
>> +    { "bottom", "Region distance from bottom edge of frame",
>> +      OFFSET(region_str[BOTTOM]), AV_OPT_TYPE_STRING, { .str = "0" }, .flags = FLAGS },
>> +    { "left",   "Region distance from left edge of frame",
>> +      OFFSET(region_str[LEFT]),   AV_OPT_TYPE_STRING, { .str = "0" }, .flags = FLAGS },
>> +    { "right",  "Region distance from right edge of frame",
>> +      OFFSET(region_str[RIGHT]),  AV_OPT_TYPE_STRING, { .str = "0" }, .flags = FLAGS },
> 
> My comment may not apply here, but I sometimes wish all rectangle
> defining filters had compatible options.
> 
> I.e. drawbox and crop and delogo (which I use interchangably - e.g.
> first the one to visually determine the other's dimensions) aren't
> compatible with respect to order of unnamed options, and to names of
> variables in their expression (w, iw, ow, ...).
> 
> Can ROIs not also be defined as x, y, w, h?

Sure, it would be easy to change if people prefer that.  This was chosen pretty much entirely because it matches the structure definition (where I think the symmetric style is indeed cleanest?  I guess you could have the same debate there if you like...).  I certainly was making use of the order of options when using it, though - e.g. "addroi=h/4:h/4:w/4:w/4:-1/5" to highlight the central quarter of the frame.

Wrt variables, I don't have 'i'/'o' prefixed names because the input and output are the same frame.  Would "iw"/"ih" be better, or would that just be more confusing?

Thanks,

- Mark
Moritz Barsnick Feb. 28, 2019, 1:53 p.m.
On Thu, Feb 28, 2019 at 01:18:06 +0000, Mark Thompson wrote:
> > Can ROIs not also be defined as x, y, w, h?
> 
> Sure, it would be easy to change if people prefer that.

It was just my personal observation (and preference)

> This was chosen pretty much entirely because it matches the structure
> definition

Oh yeah, I thought so.

> Would "iw"/"ih" be better, or would that just be more confusing?

I'm not sure, I was just thinking consistence with other filters. You
do have an "outgoing" size, that's the size of the ROI. So perhaps
iw/ih also applies.

Just my €0,02,
Moritz
Marton Balint Feb. 28, 2019, 5:35 p.m.
On Wed, 27 Feb 2019, Mark Thompson wrote:

> This can be used to add region of interest side data to video frames.
> ---
> libavfilter/Makefile     |   1 +
> libavfilter/allfilters.c |   1 +
> libavfilter/vf_addroi.c  | 237 +++++++++++++++++++++++++++++++++++++++
> 3 files changed, 239 insertions(+)
> create mode 100644 libavfilter/vf_addroi.c

[...]

> +static const AVFilterPad addroi_inputs[] = {
> +    {
> +        .name         = "default",
> +        .type         = AVMEDIA_TYPE_VIDEO,
> +        .config_props = addroi_config_input,
> +        .filter_frame = addroi_filter_frame,

Please use the activate callback of AVFilter context instead.

Thanks,
Marton
Nicolas George Feb. 28, 2019, 5:45 p.m.
Marton Balint (12019-02-28):
> Please use the activate callback of AVFilter context instead.

If it is one in, one out, it is not really necessary.

Regards,

Patch hide | download patch | download mbox

diff --git a/libavfilter/Makefile b/libavfilter/Makefile
index fef6ec5c55..31ae738a50 100644
--- a/libavfilter/Makefile
+++ b/libavfilter/Makefile
@@ -149,6 +149,7 @@  OBJS-$(CONFIG_SINE_FILTER)                   += asrc_sine.o
 OBJS-$(CONFIG_ANULLSINK_FILTER)              += asink_anullsink.o
 
 # video filters
+OBJS-$(CONFIG_ADDROI_FILTER)                 += vf_addroi.o
 OBJS-$(CONFIG_ALPHAEXTRACT_FILTER)           += vf_extractplanes.o
 OBJS-$(CONFIG_ALPHAMERGE_FILTER)             += vf_alphamerge.o
 OBJS-$(CONFIG_AMPLIFY_FILTER)                += vf_amplify.o
diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c
index c51ae0f3c7..52413c8f0d 100644
--- a/libavfilter/allfilters.c
+++ b/libavfilter/allfilters.c
@@ -140,6 +140,7 @@  extern AVFilter ff_asrc_sine;
 
 extern AVFilter ff_asink_anullsink;
 
+extern AVFilter ff_vf_addroi;
 extern AVFilter ff_vf_alphaextract;
 extern AVFilter ff_vf_alphamerge;
 extern AVFilter ff_vf_amplify;
diff --git a/libavfilter/vf_addroi.c b/libavfilter/vf_addroi.c
new file mode 100644
index 0000000000..c8d323e660
--- /dev/null
+++ b/libavfilter/vf_addroi.c
@@ -0,0 +1,237 @@ 
+/*
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with FFmpeg; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+#include "libavutil/eval.h"
+#include "libavutil/opt.h"
+#include "avfilter.h"
+#include "internal.h"
+
+enum {
+    TOP,
+    BOTTOM,
+    LEFT,
+    RIGHT,
+};
+
+static const char *const addroi_var_names[] = {
+    "w",
+    "h",
+};
+
+enum {
+    VAR_W,
+    VAR_H,
+    NB_VARS,
+};
+
+typedef struct AddROIContext {
+    const AVClass *class;
+
+    char *region_str[4];
+
+    AVExpr *region_expr[4];
+    double vars[NB_VARS];
+
+    int region[4];
+    AVRational qoffset;
+} AddROIContext;
+
+static int addroi_config_input(AVFilterLink *inlink)
+{
+    AVFilterContext *avctx = inlink->dst;
+    AddROIContext     *ctx = avctx->priv;
+    int i;
+    double val;
+
+    ctx->vars[VAR_W] = inlink->w;
+    ctx->vars[VAR_H] = inlink->h;
+
+    for (i = 0; i < 4; i++) {
+        val = av_expr_eval(ctx->region_expr[i], ctx->vars, NULL);
+        ctx->region[i] = av_clip(rint(val), 0,
+                                 i < 2 ? inlink->h : inlink->w);
+    }
+
+    return 0;
+}
+
+static int addroi_filter_frame(AVFilterLink *inlink, AVFrame *frame)
+{
+    AVFilterContext *avctx = inlink->dst;
+    AVFilterLink  *outlink = avctx->outputs[0];
+    AddROIContext     *ctx = avctx->priv;
+    AVRegionOfInterest *roi;
+    AVFrameSideData *sd;
+    int err;
+
+    sd = av_frame_get_side_data(frame, AV_FRAME_DATA_REGIONS_OF_INTEREST);
+    if (sd) {
+        const AVRegionOfInterest *old_roi;
+        AVBufferRef *roi_ref;
+        int nb_roi, i;
+
+        old_roi = (const AVRegionOfInterest*)sd->data;
+        nb_roi = sd->size / old_roi->self_size + 1;
+
+        roi_ref = av_buffer_alloc(sizeof(*roi) * nb_roi);
+        if (!roi_ref) {
+            err = AVERROR(ENOMEM);
+            goto fail;
+        }
+        roi = (AVRegionOfInterest*)roi_ref->data;
+
+        for (i = 0; i < nb_roi - 1; i++) {
+            old_roi = (const AVRegionOfInterest*)
+                (sd->data + old_roi->self_size * i);
+
+            roi[i] = (AVRegionOfInterest) {
+                .self_size = sizeof(*roi),
+                .top       = old_roi->top,
+                .bottom    = old_roi->bottom,
+                .left      = old_roi->left,
+                .right     = old_roi->right,
+                .qoffset   = old_roi->qoffset,
+            };
+        }
+
+        roi[nb_roi - 1] = (AVRegionOfInterest) {
+            .self_size = sizeof(*roi),
+            .top       = ctx->region[TOP],
+            .bottom    = ctx->region[BOTTOM],
+            .left      = ctx->region[LEFT],
+            .right     = ctx->region[RIGHT],
+            .qoffset   = ctx->qoffset,
+        };
+
+        av_frame_remove_side_data(frame, AV_FRAME_DATA_REGIONS_OF_INTEREST);
+
+        sd = av_frame_new_side_data_from_buf(frame,
+                                             AV_FRAME_DATA_REGIONS_OF_INTEREST,
+                                             roi_ref);
+        if (!sd) {
+            av_buffer_unref(&roi_ref);
+            err = AVERROR(ENOMEM);
+            goto fail;
+        }
+
+    } else {
+        sd = av_frame_new_side_data(frame, AV_FRAME_DATA_REGIONS_OF_INTEREST,
+                                    sizeof(AVRegionOfInterest));
+        if (!sd) {
+            err = AVERROR(ENOMEM);
+            goto fail;
+        }
+        roi = (AVRegionOfInterest*)sd->data;
+        *roi = (AVRegionOfInterest) {
+            .self_size = sizeof(*roi),
+            .top       = ctx->region[TOP],
+            .bottom    = ctx->region[BOTTOM],
+            .left      = ctx->region[LEFT],
+            .right     = ctx->region[RIGHT],
+            .qoffset   = ctx->qoffset,
+        };
+    }
+
+    return ff_filter_frame(outlink, frame);
+
+fail:
+    av_frame_free(&frame);
+    return err;
+}
+
+static av_cold int addroi_init(AVFilterContext *avctx)
+{
+    AddROIContext *ctx = avctx->priv;
+    int i, err;
+
+    for (i = 0; i < 4; i++) {
+        err = av_expr_parse(&ctx->region_expr[i], ctx->region_str[i],
+                            addroi_var_names, NULL, NULL, NULL, NULL,
+                            0, avctx);
+        if (err < 0) {
+            av_log(ctx, AV_LOG_ERROR,
+                   "Error parsing region distance expression '%s'.\n",
+                   ctx->region_str[i]);
+            return err;
+        }
+    }
+
+    return 0;
+}
+
+static av_cold void addroi_uninit(AVFilterContext *avctx)
+{
+    AddROIContext *ctx = avctx->priv;
+    int i;
+
+    for (i = 0; i < 4; i++) {
+        av_expr_free(ctx->region_expr[i]);
+        ctx->region_expr[i] = NULL;
+    }
+}
+
+#define OFFSET(x) offsetof(AddROIContext, x)
+#define FLAGS AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_FILTERING_PARAM
+static const AVOption addroi_options[] = {
+    { "top",    "Region distance from top edge of frame",
+      OFFSET(region_str[TOP]),    AV_OPT_TYPE_STRING, { .str = "0" }, .flags = FLAGS },
+    { "bottom", "Region distance from bottom edge of frame",
+      OFFSET(region_str[BOTTOM]), AV_OPT_TYPE_STRING, { .str = "0" }, .flags = FLAGS },
+    { "left",   "Region distance from left edge of frame",
+      OFFSET(region_str[LEFT]),   AV_OPT_TYPE_STRING, { .str = "0" }, .flags = FLAGS },
+    { "right",  "Region distance from right edge of frame",
+      OFFSET(region_str[RIGHT]),  AV_OPT_TYPE_STRING, { .str = "0" }, .flags = FLAGS },
+
+    { "qoffset", "Quantisation offset to apply in the region.",
+      OFFSET(qoffset), AV_OPT_TYPE_RATIONAL, { .dbl = 0.0 }, -1, +1, .flags = FLAGS },
+
+    { NULL }
+};
+
+AVFILTER_DEFINE_CLASS(addroi);
+
+static const AVFilterPad addroi_inputs[] = {
+    {
+        .name         = "default",
+        .type         = AVMEDIA_TYPE_VIDEO,
+        .config_props = addroi_config_input,
+        .filter_frame = addroi_filter_frame,
+    },
+    { NULL }
+};
+
+static const AVFilterPad addroi_outputs[] = {
+    {
+        .name = "default",
+        .type = AVMEDIA_TYPE_VIDEO,
+    },
+    { NULL }
+};
+
+AVFilter ff_vf_addroi = {
+    .name        = "addroi",
+    .description = NULL_IF_CONFIG_SMALL("Add region of interest to frame."),
+    .init        = addroi_init,
+    .uninit      = addroi_uninit,
+
+    .priv_size   = sizeof(AddROIContext),
+    .priv_class  = &addroi_class,
+
+    .inputs      = addroi_inputs,
+    .outputs     = addroi_outputs,
+};