[FFmpeg-devel] avfilter/vf_tile: add skip option

Submitted by Paul B Mahol on Nov. 7, 2017, 7:34 p.m.

Details

Message ID 20171107193441.28526-1-onemda@gmail.com
State New
Headers show

Commit Message

Paul B Mahol Nov. 7, 2017, 7:34 p.m.
Signed-off-by: Paul B Mahol <onemda@gmail.com>
---
 doc/filters.texi      |  5 +++++
 libavfilter/vf_tile.c | 53 ++++++++++++++++++++++++++++++++++++++++++---------
 2 files changed, 49 insertions(+), 9 deletions(-)

Comments

Paul B Mahol Nov. 9, 2017, 9:28 a.m.
On 11/7/17, Paul B Mahol <onemda@gmail.com> wrote:
> Signed-off-by: Paul B Mahol <onemda@gmail.com>
> ---
>  doc/filters.texi      |  5 +++++
>  libavfilter/vf_tile.c | 53
> ++++++++++++++++++++++++++++++++++++++++++---------
>  2 files changed, 49 insertions(+), 9 deletions(-)
>

ping
Nicolas George Nov. 9, 2017, 9:48 a.m.
Le nonidi 19 brumaire, an CCXXVI, Paul B Mahol a écrit :
> ping

Please, it has not yet been 48 hours. I cannot promise to have time to
look at it carefully enough before Sunday.

Regards,
Nicolas George Nov. 10, 2017, 9:16 a.m.
Le septidi 17 brumaire, an CCXXVI, Paul B Mahol a écrit :
> Signed-off-by: Paul B Mahol <onemda@gmail.com>
> ---
>  doc/filters.texi      |  5 +++++
>  libavfilter/vf_tile.c | 53 ++++++++++++++++++++++++++++++++++++++++++---------
>  2 files changed, 49 insertions(+), 9 deletions(-)
> 
> diff --git a/doc/filters.texi b/doc/filters.texi
> index 2ba6e04166..a6d537485b 100644
> --- a/doc/filters.texi
> +++ b/doc/filters.texi
> @@ -14439,6 +14439,11 @@ refer to the pad video filter.
>  Specify the color of the unused area. For the syntax of this option, check the
>  "Color" section in the ffmpeg-utils manual. The default value of @var{color}
>  is "black".
> +

> +@item skip
> +Set the number of frames to skip from input before rendering output frame.
> +The default value is @code{0}, meaning skip @var{nb_frames} frames. If it is more
> +than @var{nb_frames} its is set to @var{nb_frames}.

I had trouble understanding the logic of the code, until I realized the
problem was here: I think the documentation does not explain well what
the option is meant to do.

If I understand correctly, using an example: if a 5x4 tile, the normal
mapping is:

out  0 <- in  0 .. in 19
out  1 <- in 20 .. in 39
out  2 <- in 40 .. in 59
out  3 <- in 60 .. in 79

But if "skip" is set to 15, it becomes:

out  0 <- in  0 .. in 19
out  1 <- in 15 .. in 34
out  2 <- in 30 .. in 49
out  3 <- in 45 .. in 64

In other words, the output frames overlap by nb_frames - skip input
frames.

Is it what you intended?

If so, then the doc needs to be clarified. And I think the name of the
option is misleading. What about "step"? Or maybe "period"?

Or possibly even better: reverse the direction of the option "overlap=5"
instead of "step=15".

Regards,
Paul B Mahol Nov. 10, 2017, 9:25 a.m.
On 11/10/17, Nicolas George <george@nsup.org> wrote:
> Le septidi 17 brumaire, an CCXXVI, Paul B Mahol a ecrit :
>> Signed-off-by: Paul B Mahol <onemda@gmail.com>
>> ---
>>  doc/filters.texi      |  5 +++++
>>  libavfilter/vf_tile.c | 53
>> ++++++++++++++++++++++++++++++++++++++++++---------
>>  2 files changed, 49 insertions(+), 9 deletions(-)
>>
>> diff --git a/doc/filters.texi b/doc/filters.texi
>> index 2ba6e04166..a6d537485b 100644
>> --- a/doc/filters.texi
>> +++ b/doc/filters.texi
>> @@ -14439,6 +14439,11 @@ refer to the pad video filter.
>>  Specify the color of the unused area. For the syntax of this option,
>> check the
>>  "Color" section in the ffmpeg-utils manual. The default value of
>> @var{color}
>>  is "black".
>> +
>
>> +@item skip
>> +Set the number of frames to skip from input before rendering output
>> frame.
>> +The default value is @code{0}, meaning skip @var{nb_frames} frames. If it
>> is more
>> +than @var{nb_frames} its is set to @var{nb_frames}.
>
> I had trouble understanding the logic of the code, until I realized the
> problem was here: I think the documentation does not explain well what
> the option is meant to do.
>
> If I understand correctly, using an example: if a 5x4 tile, the normal
> mapping is:
>
> out  0 <- in  0 .. in 19
> out  1 <- in 20 .. in 39
> out  2 <- in 40 .. in 59
> out  3 <- in 60 .. in 79
>
> But if "skip" is set to 15, it becomes:
>
> out  0 <- in  0 .. in 19
> out  1 <- in 15 .. in 34
> out  2 <- in 30 .. in 49
> out  3 <- in 45 .. in 64
>
> In other words, the output frames overlap by nb_frames - skip input
> frames.
>
> Is it what you intended?

Yes.

>
> If so, then the doc needs to be clarified. And I think the name of the
> option is misleading. What about "step"? Or maybe "period"?

It could be...

>
> Or possibly even better: reverse the direction of the option "overlap=5"
> instead of "step=15".

What if user sets overlap to same or greater than nb_frames, just error out?
Nicolas George Nov. 10, 2017, 9:27 a.m.
Le decadi 20 brumaire, an CCXXVI, Paul B Mahol a écrit :
> Yes.

Ok, I will look at the code with that in mind.

> What if user sets overlap to same or greater than nb_frames, just error out?

That seems reasonable. Clamping the value with a warning would be ok
too, I think. Both of these solutions allow extension.

Regards,
Nicolas George Nov. 11, 2017, 10:27 a.m.
Le septidi 17 brumaire, an CCXXVI, Paul B Mahol a écrit :
> Signed-off-by: Paul B Mahol <onemda@gmail.com>
> ---
>  doc/filters.texi      |  5 +++++
>  libavfilter/vf_tile.c | 53 ++++++++++++++++++++++++++++++++++++++++++---------
>  2 files changed, 49 insertions(+), 9 deletions(-)

The logic of the code looks ok to me.

Regards,

Patch hide | download patch | download mbox

diff --git a/doc/filters.texi b/doc/filters.texi
index 2ba6e04166..a6d537485b 100644
--- a/doc/filters.texi
+++ b/doc/filters.texi
@@ -14439,6 +14439,11 @@  refer to the pad video filter.
 Specify the color of the unused area. For the syntax of this option, check the
 "Color" section in the ffmpeg-utils manual. The default value of @var{color}
 is "black".
+
+@item skip
+Set the number of frames to skip from input before rendering output frame.
+The default value is @code{0}, meaning skip @var{nb_frames} frames. If it is more
+than @var{nb_frames} its is set to @var{nb_frames}.
 @end table
 
 @subsection Examples
diff --git a/libavfilter/vf_tile.c b/libavfilter/vf_tile.c
index e9d246fc73..089872831f 100644
--- a/libavfilter/vf_tile.c
+++ b/libavfilter/vf_tile.c
@@ -37,11 +37,13 @@  typedef struct TileContext {
     unsigned w, h;
     unsigned margin;
     unsigned padding;
+    unsigned skip;
     unsigned current;
     unsigned nb_frames;
     FFDrawContext draw;
     FFDrawColor blank;
     AVFrame *out_ref;
+    AVFrame *prev_out_ref;
     uint8_t rgba_color[4];
 } TileContext;
 
@@ -58,6 +60,8 @@  static const AVOption tile_options[] = {
     { "padding", "set inner border thickness in pixels", OFFSET(padding),
         AV_OPT_TYPE_INT, {.i64 = 0}, 0, 1024, FLAGS },
     { "color",   "set the color of the unused area", OFFSET(rgba_color), AV_OPT_TYPE_COLOR, {.str = "black"}, .flags = FLAGS },
+    { "skip", "set how many frames to skip for each render", OFFSET(skip),
+        AV_OPT_TYPE_INT, {.i64 = 0}, 0, INT_MAX, FLAGS },
     { NULL }
 };
 
@@ -90,6 +94,10 @@  static av_cold int init(AVFilterContext *ctx)
         return AVERROR(EINVAL);
     }
 
+    if (!tile->skip)
+        tile->skip = tile->nb_frames;
+    tile->skip = FFMIN(tile->skip, tile->nb_frames);
+
     return 0;
 }
 
@@ -120,19 +128,19 @@  static int config_props(AVFilterLink *outlink)
     outlink->h = tile->h * inlink->h + total_margin_h;
     outlink->sample_aspect_ratio = inlink->sample_aspect_ratio;
     outlink->frame_rate = av_mul_q(inlink->frame_rate,
-                                   av_make_q(1, tile->nb_frames));
+                                   av_make_q(1, tile->skip));
     ff_draw_init(&tile->draw, inlink->format, 0);
     ff_draw_color(&tile->draw, &tile->blank, tile->rgba_color);
 
     return 0;
 }
 
-static void get_current_tile_pos(AVFilterContext *ctx, unsigned *x, unsigned *y)
+static void get_tile_pos(AVFilterContext *ctx, unsigned *x, unsigned *y, unsigned current)
 {
     TileContext *tile    = ctx->priv;
     AVFilterLink *inlink = ctx->inputs[0];
-    const unsigned tx = tile->current % tile->w;
-    const unsigned ty = tile->current / tile->w;
+    const unsigned tx = current % tile->w;
+    const unsigned ty = current / tile->w;
 
     *x = tile->margin + (inlink->w + tile->padding) * tx;
     *y = tile->margin + (inlink->h + tile->padding) * ty;
@@ -144,7 +152,7 @@  static void draw_blank_frame(AVFilterContext *ctx, AVFrame *out_buf)
     AVFilterLink *inlink = ctx->inputs[0];
     unsigned x0, y0;
 
-    get_current_tile_pos(ctx, &x0, &y0);
+    get_tile_pos(ctx, &x0, &y0, tile->current);
     ff_fill_rectangle(&tile->draw, &tile->blank,
                       out_buf->data, out_buf->linesize,
                       x0, y0, inlink->w, inlink->h);
@@ -160,8 +168,13 @@  static int end_last_frame(AVFilterContext *ctx)
 
     while (tile->current < tile->nb_frames)
         draw_blank_frame(ctx, out_buf);
+    tile->current = tile->nb_frames - tile->skip;
+    if (tile->current) {
+        av_frame_free(&tile->prev_out_ref);
+        tile->prev_out_ref = av_frame_clone(out_buf);
+    }
     ret = ff_filter_frame(outlink, out_buf);
-    tile->current = 0;
+    tile->out_ref = NULL;
     return ret;
 }
 
@@ -176,7 +189,7 @@  static int filter_frame(AVFilterLink *inlink, AVFrame *picref)
     AVFilterLink *outlink = ctx->outputs[0];
     unsigned x0, y0;
 
-    if (!tile->current) {
+    if (!tile->out_ref) {
         tile->out_ref = ff_get_video_buffer(outlink, outlink->w, outlink->h);
         if (!tile->out_ref) {
             av_frame_free(&picref);
@@ -194,7 +207,21 @@  static int filter_frame(AVFilterLink *inlink, AVFrame *picref)
                               0, 0, outlink->w, outlink->h);
     }
 
-    get_current_tile_pos(ctx, &x0, &y0);
+    if (tile->prev_out_ref) {
+        unsigned x1, y1, i;
+
+        for (i = tile->skip; i < tile->nb_frames; i++) {
+            get_tile_pos(ctx, &x1, &y1, i);
+            get_tile_pos(ctx, &x0, &y0, i - tile->skip);
+            ff_copy_rectangle2(&tile->draw,
+                               tile->out_ref->data, tile->out_ref->linesize,
+                               tile->prev_out_ref->data, tile->prev_out_ref->linesize,
+                               x0, y0, x1, y1, inlink->w, inlink->h);
+
+        }
+    }
+
+    get_tile_pos(ctx, &x0, &y0, tile->current);
     ff_copy_rectangle2(&tile->draw,
                        tile->out_ref->data, tile->out_ref->linesize,
                        picref->data, picref->linesize,
@@ -215,11 +242,18 @@  static int request_frame(AVFilterLink *outlink)
     int r;
 
     r = ff_request_frame(inlink);
-    if (r == AVERROR_EOF && tile->current)
+    if (r == AVERROR_EOF && tile->current && tile->out_ref)
         r = end_last_frame(ctx);
     return r;
 }
 
+static av_cold void uninit(AVFilterContext *ctx)
+{
+    TileContext *tile = ctx->priv;
+
+    av_frame_free(&tile->prev_out_ref);
+}
+
 static const AVFilterPad tile_inputs[] = {
     {
         .name         = "default",
@@ -243,6 +277,7 @@  AVFilter ff_vf_tile = {
     .name          = "tile",
     .description   = NULL_IF_CONFIG_SMALL("Tile several successive frames together."),
     .init          = init,
+    .uninit        = uninit,
     .query_formats = query_formats,
     .priv_size     = sizeof(TileContext),
     .inputs        = tile_inputs,