diff mbox series

[FFmpeg-devel,v2,4/4] avfilter/vf_drawbox: support draw specific face by facedetect metadata

Message ID 1589382763-28061-4-git-send-email-lance.lmwang@gmail.com
State Superseded
Headers show
Series [FFmpeg-devel,v2,1/4] doc/filters: add anchor for addroi
Related show

Checks

Context Check Description
andriy/default pending
andriy/make success Make finished
andriy/make_fate success Make fate finished

Commit Message

Limin Wang May 13, 2020, 3:12 p.m. UTC
From: Limin Wang <lance.lmwang@gmail.com>

Signed-off-by: Limin Wang <lance.lmwang@gmail.com>
---
 doc/filters.texi         | 10 ++++++++++
 libavfilter/vf_drawbox.c | 27 +++++++++++++++++++++++++++
 2 files changed, 37 insertions(+)

Comments

Marton Balint May 13, 2020, 5:16 p.m. UTC | #1
On Wed, 13 May 2020, lance.lmwang@gmail.com wrote:

> From: Limin Wang <lance.lmwang@gmail.com>
>
> Signed-off-by: Limin Wang <lance.lmwang@gmail.com>
> ---
> doc/filters.texi         | 10 ++++++++++
> libavfilter/vf_drawbox.c | 27 +++++++++++++++++++++++++++
> 2 files changed, 37 insertions(+)
>
> diff --git a/doc/filters.texi b/doc/filters.texi
> index 76e12ef..bf9043c 100644
> --- a/doc/filters.texi
> +++ b/doc/filters.texi
> @@ -9337,6 +9337,10 @@ See below for the list of accepted constants.
> Applicable if the input has alpha. With value @code{1}, the pixels of the painted box
> will overwrite the video's color and alpha pixels.
> Default is @code{0}, which composites the box onto the input, leaving the video's alpha intact.
> +
> +@item face
> +Draw the box by the facedetect metadata for the specific face.

@item face @var{integer}
Draw the box onto the position of the nth face as detected by the 
ocv filter's facedetect mode. If no face detection metadata exists
then the filter will use the specified box parameters instead.

> +
> @end table
> 
> The parameters for @var{x}, @var{y}, @var{w} and @var{h} and @var{t} are expressions containing the
> @@ -9405,6 +9409,12 @@ Draw a 2-pixel red 2.40:1 mask:
> @example
> drawbox=x=-t:y=0.5*(ih-iw/2.4)-t:w=iw+t*2:h=iw/2.4+t*2:t=2:c=red
> @end example
> +
> +@item
> +draw the box with red color for the first face by metadata if its postion is detected:

Draw a red box onto the position of the first face as detected by the ocv 
filter's facedetect method.

> +@example
> +ocv=filter_name=facedetect:filter_params=facedetect=./haarcascade_frontalface_alt.xml,drawbox=face=0:color=red
> +@end example
> @end itemize
> 
> @subsection Commands
> diff --git a/libavfilter/vf_drawbox.c b/libavfilter/vf_drawbox.c
> index 21d520e..239a149 100644
> --- a/libavfilter/vf_drawbox.c
> +++ b/libavfilter/vf_drawbox.c
> @@ -81,6 +81,7 @@ typedef struct DrawBoxContext {
>     char *t_expr;          ///< expression for thickness
>     int have_alpha;
>     int replace;
> +    int face;
> } DrawBoxContext;
> 
> static const int NUM_EXPR_EVALS = 5;
> @@ -220,6 +221,31 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *frame)
>     int plane, x, y, xb = s->x, yb = s->y;
>     unsigned char *row[4];
> 
> +    if (s->face >= 0) {
> +        AVDictionaryEntry *ex, *ey, *ew, *eh;
> +        char key2[128];
> +        AVDictionary *metadata = frame->metadata;
> +
> +        snprintf(key2, sizeof(key2), "lavfi.facedetect.%d.%s", s->face, "x");
> +        ex = av_dict_get(metadata, key2, NULL, AV_DICT_MATCH_CASE);
> +
> +        snprintf(key2, sizeof(key2), "lavfi.facedetect.%d.%s", s->face, "y");
> +        ey = av_dict_get(metadata, key2, NULL, AV_DICT_MATCH_CASE);
> +
> +        snprintf(key2, sizeof(key2), "lavfi.facedetect.%d.%s", s->face, "w");
> +        ew = av_dict_get(metadata, key2, NULL, AV_DICT_MATCH_CASE);
> +
> +        snprintf(key2, sizeof(key2), "lavfi.facedetect.%d.%s", s->face, "h");
> +        eh = av_dict_get(metadata, key2, NULL, AV_DICT_MATCH_CASE);
> +
> +        if (ex && ey && ew && eh) {
> +            xb = s->x = strtol(ex->value, NULL, 10);
> +            yb = s->y = strtol(ey->value, NULL, 10);
> +            s->w = strtol(ew->value, NULL, 10);
> +            s->h = strtol(eh->value, NULL, 10);
> +        }
> +    }
> +
>     if (s->have_alpha && s->replace) {
>         for (y = FFMAX(yb, 0); y < frame->height && y < (yb + s->h); y++) {
>             row[0] = frame->data[0] + y * frame->linesize[0];
> @@ -323,6 +349,7 @@ static const AVOption drawbox_options[] = {
>     { "thickness", "set the box thickness",                        OFFSET(t_expr),    AV_OPT_TYPE_STRING, { .str="3" },       0, 0, FLAGS },
>     { "t",         "set the box thickness",                        OFFSET(t_expr),    AV_OPT_TYPE_STRING, { .str="3" },       0, 0, FLAGS },
>     { "replace",   "replace color & alpha",                        OFFSET(replace),   AV_OPT_TYPE_BOOL,   { .i64=0 },         0,        1,        FLAGS },
> +    { "face",      "set which face to draw with metadata",         OFFSET(face),      AV_OPT_TYPE_INT,    { .i64=-1 },        -1,     256,        FLAGS },
>     { NULL }
> };
>

It is a bit more work, but have you considered making the face position 
available to the filter as a variable to the expressions? Also possibly 
you could add a new mode to the filter to select when to evaluate the 
x/y/width/height/tickness parameters, at initalization time, or for each 
frame. Several filters alrady have such selectors (e.g.: see the the eval 
parameter of vf_eq).

I don't oppose the patch as is, but it is something worth considering.

Regards,
Marton
Limin Wang May 13, 2020, 11:42 p.m. UTC | #2
On Wed, May 13, 2020 at 07:16:47PM +0200, Marton Balint wrote:
> 
> 
> On Wed, 13 May 2020, lance.lmwang@gmail.com wrote:
> 
> > From: Limin Wang <lance.lmwang@gmail.com>
> > 
> > Signed-off-by: Limin Wang <lance.lmwang@gmail.com>
> > ---
> > doc/filters.texi         | 10 ++++++++++
> > libavfilter/vf_drawbox.c | 27 +++++++++++++++++++++++++++
> > 2 files changed, 37 insertions(+)
> > 
> > diff --git a/doc/filters.texi b/doc/filters.texi
> > index 76e12ef..bf9043c 100644
> > --- a/doc/filters.texi
> > +++ b/doc/filters.texi
> > @@ -9337,6 +9337,10 @@ See below for the list of accepted constants.
> > Applicable if the input has alpha. With value @code{1}, the pixels of the painted box
> > will overwrite the video's color and alpha pixels.
> > Default is @code{0}, which composites the box onto the input, leaving the video's alpha intact.
> > +
> > +@item face
> > +Draw the box by the facedetect metadata for the specific face.
> 
> @item face @var{integer}
> Draw the box onto the position of the nth face as detected by the ocv
> filter's facedetect mode. If no face detection metadata exists
> then the filter will use the specified box parameters instead.
thanks, it's more clear, will update.

> 
> > +
> > @end table
> > 
> > The parameters for @var{x}, @var{y}, @var{w} and @var{h} and @var{t} are expressions containing the
> > @@ -9405,6 +9409,12 @@ Draw a 2-pixel red 2.40:1 mask:
> > @example
> > drawbox=x=-t:y=0.5*(ih-iw/2.4)-t:w=iw+t*2:h=iw/2.4+t*2:t=2:c=red
> > @end example
> > +
> > +@item
> > +draw the box with red color for the first face by metadata if its postion is detected:
> 
> Draw a red box onto the position of the first face as detected by the ocv
> filter's facedetect method.
will updat

> 
> > +@example
> > +ocv=filter_name=facedetect:filter_params=facedetect=./haarcascade_frontalface_alt.xml,drawbox=face=0:color=red
> > +@end example
> > @end itemize
> > 
> > @subsection Commands
> > diff --git a/libavfilter/vf_drawbox.c b/libavfilter/vf_drawbox.c
> > index 21d520e..239a149 100644
> > --- a/libavfilter/vf_drawbox.c
> > +++ b/libavfilter/vf_drawbox.c
> > @@ -81,6 +81,7 @@ typedef struct DrawBoxContext {
> >     char *t_expr;          ///< expression for thickness
> >     int have_alpha;
> >     int replace;
> > +    int face;
> > } DrawBoxContext;
> > 
> > static const int NUM_EXPR_EVALS = 5;
> > @@ -220,6 +221,31 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *frame)
> >     int plane, x, y, xb = s->x, yb = s->y;
> >     unsigned char *row[4];
> > 
> > +    if (s->face >= 0) {
> > +        AVDictionaryEntry *ex, *ey, *ew, *eh;
> > +        char key2[128];
> > +        AVDictionary *metadata = frame->metadata;
> > +
> > +        snprintf(key2, sizeof(key2), "lavfi.facedetect.%d.%s", s->face, "x");
> > +        ex = av_dict_get(metadata, key2, NULL, AV_DICT_MATCH_CASE);
> > +
> > +        snprintf(key2, sizeof(key2), "lavfi.facedetect.%d.%s", s->face, "y");
> > +        ey = av_dict_get(metadata, key2, NULL, AV_DICT_MATCH_CASE);
> > +
> > +        snprintf(key2, sizeof(key2), "lavfi.facedetect.%d.%s", s->face, "w");
> > +        ew = av_dict_get(metadata, key2, NULL, AV_DICT_MATCH_CASE);
> > +
> > +        snprintf(key2, sizeof(key2), "lavfi.facedetect.%d.%s", s->face, "h");
> > +        eh = av_dict_get(metadata, key2, NULL, AV_DICT_MATCH_CASE);
> > +
> > +        if (ex && ey && ew && eh) {
> > +            xb = s->x = strtol(ex->value, NULL, 10);
> > +            yb = s->y = strtol(ey->value, NULL, 10);
> > +            s->w = strtol(ew->value, NULL, 10);
> > +            s->h = strtol(eh->value, NULL, 10);
> > +        }
> > +    }
> > +
> >     if (s->have_alpha && s->replace) {
> >         for (y = FFMAX(yb, 0); y < frame->height && y < (yb + s->h); y++) {
> >             row[0] = frame->data[0] + y * frame->linesize[0];
> > @@ -323,6 +349,7 @@ static const AVOption drawbox_options[] = {
> >     { "thickness", "set the box thickness",                        OFFSET(t_expr),    AV_OPT_TYPE_STRING, { .str="3" },       0, 0, FLAGS },
> >     { "t",         "set the box thickness",                        OFFSET(t_expr),    AV_OPT_TYPE_STRING, { .str="3" },       0, 0, FLAGS },
> >     { "replace",   "replace color & alpha",                        OFFSET(replace),   AV_OPT_TYPE_BOOL,   { .i64=0 },         0,        1,        FLAGS },
> > +    { "face",      "set which face to draw with metadata",         OFFSET(face),      AV_OPT_TYPE_INT,    { .i64=-1 },        -1,     256,        FLAGS },
> >     { NULL }
> > };
> > 
> 
> It is a bit more work, but have you considered making the face position
> available to the filter as a variable to the expressions? Also possibly you
> could add a new mode to the filter to select when to evaluate the
> x/y/width/height/tickness parameters, at initalization time, or for each
> frame. Several filters alrady have such selectors (e.g.: see the the eval
> parameter of vf_eq).
> 
> I don't oppose the patch as is, but it is something worth considering.

It's good idea, I'm considering to add opencv drawbox filter, for now, 
I prefer to keep the change simple for this stage. I'll do more in future.


> 
> Regards,
> Marton
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> 
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
diff mbox series

Patch

diff --git a/doc/filters.texi b/doc/filters.texi
index 76e12ef..bf9043c 100644
--- a/doc/filters.texi
+++ b/doc/filters.texi
@@ -9337,6 +9337,10 @@  See below for the list of accepted constants.
 Applicable if the input has alpha. With value @code{1}, the pixels of the painted box
 will overwrite the video's color and alpha pixels.
 Default is @code{0}, which composites the box onto the input, leaving the video's alpha intact.
+
+@item face
+Draw the box by the facedetect metadata for the specific face.
+
 @end table
 
 The parameters for @var{x}, @var{y}, @var{w} and @var{h} and @var{t} are expressions containing the
@@ -9405,6 +9409,12 @@  Draw a 2-pixel red 2.40:1 mask:
 @example
 drawbox=x=-t:y=0.5*(ih-iw/2.4)-t:w=iw+t*2:h=iw/2.4+t*2:t=2:c=red
 @end example
+
+@item
+draw the box with red color for the first face by metadata if its postion is detected:
+@example
+ocv=filter_name=facedetect:filter_params=facedetect=./haarcascade_frontalface_alt.xml,drawbox=face=0:color=red
+@end example
 @end itemize
 
 @subsection Commands
diff --git a/libavfilter/vf_drawbox.c b/libavfilter/vf_drawbox.c
index 21d520e..239a149 100644
--- a/libavfilter/vf_drawbox.c
+++ b/libavfilter/vf_drawbox.c
@@ -81,6 +81,7 @@  typedef struct DrawBoxContext {
     char *t_expr;          ///< expression for thickness
     int have_alpha;
     int replace;
+    int face;
 } DrawBoxContext;
 
 static const int NUM_EXPR_EVALS = 5;
@@ -220,6 +221,31 @@  static int filter_frame(AVFilterLink *inlink, AVFrame *frame)
     int plane, x, y, xb = s->x, yb = s->y;
     unsigned char *row[4];
 
+    if (s->face >= 0) {
+        AVDictionaryEntry *ex, *ey, *ew, *eh;
+        char key2[128];
+        AVDictionary *metadata = frame->metadata;
+
+        snprintf(key2, sizeof(key2), "lavfi.facedetect.%d.%s", s->face, "x");
+        ex = av_dict_get(metadata, key2, NULL, AV_DICT_MATCH_CASE);
+
+        snprintf(key2, sizeof(key2), "lavfi.facedetect.%d.%s", s->face, "y");
+        ey = av_dict_get(metadata, key2, NULL, AV_DICT_MATCH_CASE);
+
+        snprintf(key2, sizeof(key2), "lavfi.facedetect.%d.%s", s->face, "w");
+        ew = av_dict_get(metadata, key2, NULL, AV_DICT_MATCH_CASE);
+
+        snprintf(key2, sizeof(key2), "lavfi.facedetect.%d.%s", s->face, "h");
+        eh = av_dict_get(metadata, key2, NULL, AV_DICT_MATCH_CASE);
+
+        if (ex && ey && ew && eh) {
+            xb = s->x = strtol(ex->value, NULL, 10);
+            yb = s->y = strtol(ey->value, NULL, 10);
+            s->w = strtol(ew->value, NULL, 10);
+            s->h = strtol(eh->value, NULL, 10);
+        }
+    }
+
     if (s->have_alpha && s->replace) {
         for (y = FFMAX(yb, 0); y < frame->height && y < (yb + s->h); y++) {
             row[0] = frame->data[0] + y * frame->linesize[0];
@@ -323,6 +349,7 @@  static const AVOption drawbox_options[] = {
     { "thickness", "set the box thickness",                        OFFSET(t_expr),    AV_OPT_TYPE_STRING, { .str="3" },       0, 0, FLAGS },
     { "t",         "set the box thickness",                        OFFSET(t_expr),    AV_OPT_TYPE_STRING, { .str="3" },       0, 0, FLAGS },
     { "replace",   "replace color & alpha",                        OFFSET(replace),   AV_OPT_TYPE_BOOL,   { .i64=0 },         0,        1,        FLAGS },
+    { "face",      "set which face to draw with metadata",         OFFSET(face),      AV_OPT_TYPE_INT,    { .i64=-1 },        -1,     256,        FLAGS },
     { NULL }
 };