From patchwork Wed May 13 15:12:40 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lance Wang X-Patchwork-Id: 19669 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 28C8D44B25E for ; Wed, 13 May 2020 18:13:25 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id ECC3B687F88; Wed, 13 May 2020 18:13:24 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pg1-f193.google.com (mail-pg1-f193.google.com [209.85.215.193]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 037B2680922 for ; Wed, 13 May 2020 18:13:18 +0300 (EEST) Received: by mail-pg1-f193.google.com with SMTP id f4so911198pgi.10 for ; Wed, 13 May 2020 08:13:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id; bh=V+0g+vF/IqAfTled6o+MF1hun5SLjO1RvD4ENuMjHzo=; b=m0fBgjnNrobkSZY4qVBxWNfF818NsTAqRdBfnyEqi8M8G7PUA9jphdkklj9FLb6kIP mQupy/gN81WoW3SPgodOz7iS51rDIJO3JZRIbEiPLUIUhUrNckDomWkQBS2bgUMzc23s k8NwIEtacDUdqp3m9qds0VrTl8MEtEu1dEis38V/X3O/tYSc2CMAkcW/TYF+PDvFb3pd oJ02uCsn6F9fN9KTZEWbRrUAj2v6TcvQj6Y1qZ8sM1Hon5vjStyGlLrqMRjFTTrcEJEl aitZ5L1FS07wjk6j95OfJdsjR3BJFFgVQ6hp9mK67yVbU0TZJwkHdAEpdRhuIkhbws94 8UoA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=V+0g+vF/IqAfTled6o+MF1hun5SLjO1RvD4ENuMjHzo=; b=SS/zAwyk4Ix1Z88ismiWteQniVOiSdpFIJX9TGCbRK468nskyfcBWhcTAJioABm56h GDn/pktP7jJBRofNw+ESQFUfe+nJD6hz0z8Q+9tJw4yAPErb6kQSvxZUp0HeLs0UgZ+p +H43xZfHr7ofBqG0R+nx0e8A0Olm2g/IWuGsNum7Xd6CvY7V4GoE3Oqe71k2whfqrRMR WZ/2QG+2dO7+UXTp3Flow0HUYxgZHMGHOWBy2gpssKjbm2+SdyZSVUB5LpjDc3JiqbLs zRoklGbdFr7irRqP8aUlmQB2d38l1YmHiRTY9eeKiYv9dr3UH7PskJ9bmrXw4cQ9nOvB /b5g== X-Gm-Message-State: AGi0Pua5DBh78bf9AW/YTPm9SCcyNcaCu9NGJ/0gZ//Nzh8z7p9cWEmn tJZBxJyzy7UbGOLMxqBJwugd/49F X-Google-Smtp-Source: APiQypK8EDR2qL/0sF7cRU3uzVIi0u2bBZQEqocmGadrk4HZV2JOBVB26w8MVdCQtu0jN0SAaeVKeA== X-Received: by 2002:a63:4665:: with SMTP id v37mr23507816pgk.297.1589382796073; Wed, 13 May 2020 08:13:16 -0700 (PDT) Received: from vpn2.localdomain ([161.117.202.209]) by smtp.gmail.com with ESMTPSA id t64sm43309pgd.24.2020.05.13.08.13.14 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 13 May 2020 08:13:15 -0700 (PDT) From: lance.lmwang@gmail.com To: ffmpeg-devel@ffmpeg.org Date: Wed, 13 May 2020 23:12:40 +0800 Message-Id: <1589382763-28061-1-git-send-email-lance.lmwang@gmail.com> X-Mailer: git-send-email 1.8.3.1 Subject: [FFmpeg-devel] [PATCH v2 1/4] doc/filters: add anchor for addroi X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Limin Wang MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" From: Limin Wang Signed-off-by: Limin Wang --- doc/filters.texi | 1 + 1 file changed, 1 insertion(+) diff --git a/doc/filters.texi b/doc/filters.texi index d19fd34..c128033 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -6294,6 +6294,7 @@ build. Below is a description of the currently available video filters. +@anchor{addroi} @section addroi Mark a region of interest in a video frame. From patchwork Wed May 13 15:12:41 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lance Wang X-Patchwork-Id: 19670 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 1734444B25E for ; Wed, 13 May 2020 18:13:27 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id EF4A7680B8C; Wed, 13 May 2020 18:13:26 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pl1-f169.google.com (mail-pl1-f169.google.com [209.85.214.169]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 4E1F968803F for ; Wed, 13 May 2020 18:13:20 +0300 (EEST) Received: by mail-pl1-f169.google.com with SMTP id u10so6938056pls.8 for ; Wed, 13 May 2020 08:13:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=eSSBSaC3/9jOSAeK+vNkCHFbuTtrwCM9tjeTdhOhBxE=; b=INhExhOL18bulcfBAVMuB4QeKo6pcaXeCPx72mNDJdYT0fKkgyJZOHLj3MvZrknNKN QAjqHws93Z0IjWaDMDFugQATisS9MruhZSabAnBU3+n4zxdBomaLcd6fpSsRhXRx3SYA XxdOmF8dxOPz4VIOyI/Hv2W96qjtRGXwa3virfbpcn/SqJ6wCeel6bwbFhusEU2do6BL 3iOq63C1eU5tHujoLvM6CuDWkkgTrY8no1nlG0MngZT43ibppNlJv6s/3mffUxa61U/e e7e3CUp5eqYFh31PNW1qHCvLpqlKaajjgSB4hmtmJhZAdrtGPJbZUAb3VDBBsPNCVYT4 7Pmw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=eSSBSaC3/9jOSAeK+vNkCHFbuTtrwCM9tjeTdhOhBxE=; b=ZM7qK51+MVcxtHvUXA2vrpnwpmAoqLB/6QQjs6lxFe31X90GCp3Wq2d6Xi4qsMhC6K D0T1rNkGmcYxgmb/nqreAp1FSWaBYBCIbDHmMH/BbedRyy6xkjdZ0ssEPwaNCUNt1v+F jnX3lRXKMNgQHS/b2U+WCQ3e+s5nCgSY+7j8cKOTpc9Y1tEZJuW7Lju4mBdDDMf2uqEZ liwO0lPyfWRuwjs6VJHp38kIJKSanbkjDidUaldU1PAzcC7AkQA/dH6jxHtwkWJ/bj02 DkMRZiv56495zwZYwqyO4ppinpVqV0+hAsXnIa7SWeqb9nW2J/YDE0sHOd6UWS53Yj4w D0dg== X-Gm-Message-State: AOAM532BJ4NVYADjRj06MKrgtZBoMacK3wkipwdPnpYodX4GMJ5z7rAN XQmUlwyuvtccX4/aZDNgwBzzKAhw X-Google-Smtp-Source: ABdhPJxliwuGrL2e7NLfhe0wOU45RqQR5wlw+B7iKKQdQSy7OBpw2IlQoETu+zHxjLJcTg5F3VNUUg== X-Received: by 2002:a17:90a:33e8:: with SMTP id n95mr2947333pjb.219.1589382797909; Wed, 13 May 2020 08:13:17 -0700 (PDT) Received: from vpn2.localdomain ([161.117.202.209]) by smtp.gmail.com with ESMTPSA id t64sm43309pgd.24.2020.05.13.08.13.16 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 13 May 2020 08:13:17 -0700 (PDT) From: lance.lmwang@gmail.com To: ffmpeg-devel@ffmpeg.org Date: Wed, 13 May 2020 23:12:41 +0800 Message-Id: <1589382763-28061-2-git-send-email-lance.lmwang@gmail.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1589382763-28061-1-git-send-email-lance.lmwang@gmail.com> References: <1589382763-28061-1-git-send-email-lance.lmwang@gmail.com> Subject: [FFmpeg-devel] [PATCH v2 2/4] avfilter/vf_libopencv: add opencv HaarCascade classifier simple face detection filter X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Limin Wang MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" From: Limin Wang Signed-off-by: Limin Wang --- configure | 1 + doc/filters.texi | 29 ++++++++ libavfilter/vf_libopencv.c | 164 ++++++++++++++++++++++++++++++++++++++++++++- 3 files changed, 191 insertions(+), 3 deletions(-) diff --git a/configure b/configure index a45c0fb..99d008a 100755 --- a/configure +++ b/configure @@ -2123,6 +2123,7 @@ HEADERS_LIST=" machine_ioctl_meteor_h malloc_h opencv2_core_core_c_h + opencv2_objdetect_objdetect_c_h OpenGL_gl3_h poll_h sys_param_h diff --git a/doc/filters.texi b/doc/filters.texi index c128033..76e12ef 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -14177,6 +14177,35 @@ other parameters is 0. These parameters correspond to the parameters assigned to the libopencv function @code{cvSmooth}. +@subsection facedetect +Face detection using Haar Feature-based Cascade Classifiers. + +The filter takes the following parameters: +@var{xml_model}|@var{qoffset}. + +@var{xml_model} is the path of pre-trained classifiers, The C API still +does not support the newer cascade format, please use the old format +haarcascade_frontalface_alt.xml which type_id is opencv-haar-classifier. + +@var{qoffset} +If you want export the detected faces by ROI side data in frame, please set the +parameters, See also the @ref{addroi} filter. The range of qoffset is from [-1.0, 1.0] + +By default the filter will report these metadata values if face are +detected: +@table @option +@item lavfi.facedetect.nb_faces +Display the detected face number + +@item lavfi.facedetect.face_id.x, lavfi.facedetect.face_id.y +Display x and y of every faces, face_id is the face index which is range +from [0, nb_faces-1] + +@item lavfi.facedetect.face_id.w, lavfi.facedetect.face_id.h +Display width and height of every faces, face_id is the face index +which is range from [0, nb_faces-1] +@end table + @section oscilloscope 2D Video Oscilloscope. diff --git a/libavfilter/vf_libopencv.c b/libavfilter/vf_libopencv.c index 8128030..141d696 100644 --- a/libavfilter/vf_libopencv.c +++ b/libavfilter/vf_libopencv.c @@ -1,5 +1,6 @@ /* * Copyright (c) 2010 Stefano Sabatini + * Copyright (c) 2020 Limin Wang * * This file is part of FFmpeg. * @@ -27,10 +28,16 @@ #if HAVE_OPENCV2_CORE_CORE_C_H #include #include +#if HAVE_OPENCV2_OBJECTDETECT_OBJECTDETECT_C_H +#include +#else +#include +#endif #else #include #include #endif + #include "libavutil/avstring.h" #include "libavutil/common.h" #include "libavutil/file.h" @@ -82,6 +89,7 @@ typedef struct OCVContext { int (*init)(AVFilterContext *ctx, const char *args); void (*uninit)(AVFilterContext *ctx); void (*end_frame_filter)(AVFilterContext *ctx, IplImage *inimg, IplImage *outimg); + void (*update_metadata)(AVFilterContext *ctx, AVFrame *frame); void *priv; } OCVContext; @@ -326,18 +334,152 @@ static void erode_end_frame_filter(AVFilterContext *ctx, IplImage *inimg, IplIma cvErode(inimg, outimg, dilate->kernel, dilate->nb_iterations); } +typedef struct FaceDetectContext { + char *xml_model; + CvHaarClassifierCascade* cascade; + CvMemStorage* storage; + int nb_faces; + CvSeq *faces; + int add_roi; + AVRational qoffset; +} FaceDetectContext; + +static av_cold int facedetect_init(AVFilterContext *ctx, const char *args) +{ + OCVContext *s = ctx->priv; + FaceDetectContext *facedetect = s->priv; + const char *buf = args; + double qoffset; + + if (args) { + facedetect->xml_model = av_get_token(&buf, "|"); + if (!facedetect->xml_model) { + av_log(ctx, AV_LOG_ERROR, "failed to get %s, %s\n", args, facedetect->xml_model); + return AVERROR(EINVAL); + } + + if (buf && sscanf(buf, "|%lf", &qoffset) == 1) { + if (qoffset < -1.0 || qoffset > 1.0) { + av_log(ctx, AV_LOG_ERROR, "failed to get valid qoffset(%f))\n", qoffset); + return AVERROR(EINVAL); + } + facedetect->add_roi = 1; + facedetect->qoffset = av_d2q(qoffset, 255); + } + } else { + av_log(ctx, AV_LOG_ERROR, "failed to get haarcascade_frontalface_alt.xml model file\n"); + return AVERROR(EINVAL); + } + + av_log(ctx, AV_LOG_VERBOSE, "xml_model: %s add_roi: %d qoffset: %d/%d\n", + facedetect->xml_model, facedetect->add_roi, facedetect->qoffset.num, facedetect->qoffset.den); + + facedetect->storage = cvCreateMemStorage(0); + if (!facedetect->storage) { + av_log(ctx, AV_LOG_ERROR, "cvCreateMemStorage() failed\n"); + return AVERROR(EINVAL); + } + cvClearMemStorage(facedetect->storage); + + facedetect->cascade = (CvHaarClassifierCascade*)cvLoad( facedetect->xml_model, NULL, NULL, NULL ); + if (!facedetect->cascade) { + av_log(ctx, AV_LOG_ERROR, "failed to load classifier cascade: %s \n", facedetect->xml_model); + return AVERROR(EINVAL); + } + + return 0; +} + +static av_cold void facedetect_uninit(AVFilterContext *ctx) +{ + OCVContext *s = ctx->priv; + FaceDetectContext *facedetect = s->priv; + + if (facedetect->cascade) + cvReleaseHaarClassifierCascade(&facedetect->cascade); + if (facedetect->storage) + cvReleaseMemStorage(&facedetect->storage); +} + +static void set_meta_int(AVDictionary **metadata, const char *key, int idx, int d) +{ + char value[128]; + char key2[128]; + + snprintf(value, sizeof(value), "%d", d); + snprintf(key2, sizeof(key2), "lavfi.facedetect.%d.%s", idx, key); + av_dict_set(metadata, key2, value, 0); +} + +static void facedetect_end_frame_filter(AVFilterContext *ctx, IplImage *inimg, IplImage *outimg) +{ + OCVContext *s = ctx->priv; + FaceDetectContext *facedetect = s->priv; + + facedetect->faces = cvHaarDetectObjects(inimg, facedetect->cascade, facedetect->storage, + 1.25, 3, CV_HAAR_DO_CANNY_PRUNING, + cvSize(inimg->width/16,inimg->height/16), cvSize(0,0)); + + facedetect->nb_faces = facedetect->faces ? facedetect->faces->total : 0; +} + +static void facedetect_update_metadata(AVFilterContext *ctx, AVFrame *out) +{ + OCVContext *s = ctx->priv; + FaceDetectContext *facedetect = s->priv; + AVRegionOfInterest *roi; + AVFrameSideData *sd; + AVBufferRef *roi_buf; + int i; + + if (facedetect->add_roi && facedetect->nb_faces > 0) { + sd = av_frame_new_side_data(out, AV_FRAME_DATA_REGIONS_OF_INTEREST, + facedetect->nb_faces * sizeof(AVRegionOfInterest)); + if (!sd) { + return AVERROR(ENOMEM); + } + roi = (AVRegionOfInterest*)sd->data; + for(i = 0; i < facedetect->nb_faces; i++ ) { + CvRect *r = (CvRect*) cvGetSeqElem(facedetect->faces, i); + + roi[i] = (AVRegionOfInterest) { + .self_size = sizeof(*roi), + .top = r->y, + .bottom = r->y + r->height, + .left = r->x, + .right = r->x + r->width, + .qoffset = facedetect->qoffset, + }; + } + } + + if (facedetect->nb_faces > 0) + av_dict_set_int(&out->metadata, "lavfi.facedetect.nb_faces", facedetect->nb_faces, 0); + + for(i = 0; i < facedetect->nb_faces; i++ ) { + CvRect *r = (CvRect*) cvGetSeqElem(facedetect->faces, i); + + set_meta_int(&out->metadata, "x", i, r->x); + set_meta_int(&out->metadata, "y", i, r->y); + set_meta_int(&out->metadata, "w", i, r->width); + set_meta_int(&out->metadata, "h", i, r->height); + } +} + typedef struct OCVFilterEntry { const char *name; size_t priv_size; int (*init)(AVFilterContext *ctx, const char *args); void (*uninit)(AVFilterContext *ctx); void (*end_frame_filter)(AVFilterContext *ctx, IplImage *inimg, IplImage *outimg); + void (*update_metadata)(AVFilterContext *ctx, AVFrame *frame); } OCVFilterEntry; static const OCVFilterEntry ocv_filter_entries[] = { - { "dilate", sizeof(DilateContext), dilate_init, dilate_uninit, dilate_end_frame_filter }, - { "erode", sizeof(DilateContext), dilate_init, dilate_uninit, erode_end_frame_filter }, - { "smooth", sizeof(SmoothContext), smooth_init, NULL, smooth_end_frame_filter }, + { "dilate", sizeof(DilateContext), dilate_init, dilate_uninit, dilate_end_frame_filter, NULL }, + { "erode", sizeof(DilateContext), dilate_init, dilate_uninit, erode_end_frame_filter, NULL }, + { "smooth", sizeof(SmoothContext), smooth_init, NULL, smooth_end_frame_filter, NULL }, + { "facedetect", sizeof(FaceDetectContext), facedetect_init, facedetect_uninit, facedetect_end_frame_filter, facedetect_update_metadata }, }; static av_cold int init(AVFilterContext *ctx) @@ -355,6 +497,7 @@ static av_cold int init(AVFilterContext *ctx) s->init = entry->init; s->uninit = entry->uninit; s->end_frame_filter = entry->end_frame_filter; + s->update_metadata = entry->update_metadata; if (!(s->priv = av_mallocz(entry->priv_size))) return AVERROR(ENOMEM); @@ -383,18 +526,33 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *in) AVFrame *out; IplImage inimg, outimg; + /* facedetect filter will passthrought the input frame */ + if (strcmp(s->name, "facedetect")) { out = ff_get_video_buffer(outlink, outlink->w, outlink->h); if (!out) { av_frame_free(&in); return AVERROR(ENOMEM); } av_frame_copy_props(out, in); + } else { + out = in; + } fill_iplimage_from_frame(&inimg , in , inlink->format); + + if (strcmp(s->name, "facedetect")) { fill_iplimage_from_frame(&outimg, out, inlink->format); s->end_frame_filter(ctx, &inimg, &outimg); fill_frame_from_iplimage(out, &outimg, inlink->format); + } else { + s->end_frame_filter(ctx, &inimg, NULL); + } + + if (s->update_metadata) { + s->update_metadata(ctx, out); + } + if (out != in) av_frame_free(&in); return ff_filter_frame(outlink, out); From patchwork Wed May 13 15:12:42 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lance Wang X-Patchwork-Id: 19671 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 2934544B25E for ; Wed, 13 May 2020 18:13:28 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 1833B68813D; Wed, 13 May 2020 18:13:28 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pl1-f193.google.com (mail-pl1-f193.google.com [209.85.214.193]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 48288680ACD for ; Wed, 13 May 2020 18:13:21 +0300 (EEST) Received: by mail-pl1-f193.google.com with SMTP id m7so6937050plt.5 for ; Wed, 13 May 2020 08:13:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=kaymcUny1zKjdL3nVr5ltWEXNHzmUa4p6DNUv8wezBg=; b=XYx9WRrzLIavCFg7q22JY5iuZCxIGfM+QU7u5h4tVjZTA6j4/yU9ucQ1mfQa0g3iHL Y5/2zmgOxcf80kl6a8cHWBUhqkdYyUl+17KQw/qjI8lCbsSeH/20rntDI+d9pO8ZzliB NeaMM2hoUmeWcv0E5oQWU6beIqNePYoN1zR66n684jd3wErD6DPJRvuVx/N07/XPYHa2 cNCsLv9AKjVhS789Ktt6Dhv+RIhLvpZxIGwddugrWV1LSZTnayJ77l8hd1uu4Gzoxsfr qClKo+ZOlaykIjRTAP0Iz2SAWCHMki+U2ZMhFXafoPlZo9Jv2UNMbnWc7Bi+wnvlx8v6 j3Xw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=kaymcUny1zKjdL3nVr5ltWEXNHzmUa4p6DNUv8wezBg=; b=Rn1YdJPE/B3ctny+YGTvn33Dw9UxtLgWzYqq8DM3szpbD+MmcmREDFRfSYy7iR1Y+Y +K+T9BiGI9+JVIqE4r5IGdOoKbdQcSu5549gkLYuNIwIKDlch09XAB1kLSJdFMIoK9uk r+bTR2aXIHybtMpnitijBVcXr6KSEUTDi7/ous+E7k+44ywV3VLoNcWhU8TnuDTbCpy1 umAOWMIo2NHL+K3J5YWX569fbxT/K/3Q5RH5E8rGau6uObrkyBLzgGXPkFjDzmrX8k4z RXPrWjAdl2RLPgTuQrkH8y+vIJzbKO3oUMCQRzvdovdXXIvmVTghlrwKoBm+tLNfeCqF 0kDw== X-Gm-Message-State: AOAM533+qVjCNbFeOwAaZUiytqF2FpPu8hkYmE247MFKcEJmNuJQx8w/ Q5fAYK2cSRCrB71MQ3XvQ2IouL7v X-Google-Smtp-Source: ABdhPJyGg1U9HP5t/6JLfSgzEYf4Z8wi2l6Svn0KRAX01nYKREqd3XuvojBq9PUIL99+MICyy8Gx4g== X-Received: by 2002:a17:90a:8818:: with SMTP id s24mr4060198pjn.131.1589382799535; Wed, 13 May 2020 08:13:19 -0700 (PDT) Received: from vpn2.localdomain ([161.117.202.209]) by smtp.gmail.com with ESMTPSA id t64sm43309pgd.24.2020.05.13.08.13.18 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 13 May 2020 08:13:19 -0700 (PDT) From: lance.lmwang@gmail.com To: ffmpeg-devel@ffmpeg.org Date: Wed, 13 May 2020 23:12:42 +0800 Message-Id: <1589382763-28061-3-git-send-email-lance.lmwang@gmail.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1589382763-28061-1-git-send-email-lance.lmwang@gmail.com> References: <1589382763-28061-1-git-send-email-lance.lmwang@gmail.com> Subject: [FFmpeg-devel] [PATCH v2 3/4] avfilter/vf_libopencv: reindent after last commit X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Limin Wang MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" From: Limin Wang Signed-off-by: Limin Wang --- libavfilter/vf_libopencv.c | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/libavfilter/vf_libopencv.c b/libavfilter/vf_libopencv.c index 141d696..5899867 100644 --- a/libavfilter/vf_libopencv.c +++ b/libavfilter/vf_libopencv.c @@ -528,12 +528,12 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *in) /* facedetect filter will passthrought the input frame */ if (strcmp(s->name, "facedetect")) { - out = ff_get_video_buffer(outlink, outlink->w, outlink->h); - if (!out) { - av_frame_free(&in); - return AVERROR(ENOMEM); - } - av_frame_copy_props(out, in); + out = ff_get_video_buffer(outlink, outlink->w, outlink->h); + if (!out) { + av_frame_free(&in); + return AVERROR(ENOMEM); + } + av_frame_copy_props(out, in); } else { out = in; } @@ -541,9 +541,9 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *in) fill_iplimage_from_frame(&inimg , in , inlink->format); if (strcmp(s->name, "facedetect")) { - fill_iplimage_from_frame(&outimg, out, inlink->format); - s->end_frame_filter(ctx, &inimg, &outimg); - fill_frame_from_iplimage(out, &outimg, inlink->format); + fill_iplimage_from_frame(&outimg, out, inlink->format); + s->end_frame_filter(ctx, &inimg, &outimg); + fill_frame_from_iplimage(out, &outimg, inlink->format); } else { s->end_frame_filter(ctx, &inimg, NULL); } @@ -553,7 +553,7 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *in) } if (out != in) - av_frame_free(&in); + av_frame_free(&in); return ff_filter_frame(outlink, out); } From patchwork Wed May 13 15:12:43 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lance Wang X-Patchwork-Id: 19672 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id E567F44B25E for ; Wed, 13 May 2020 18:13:29 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id CC5FF689A6C; Wed, 13 May 2020 18:13:29 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pl1-f195.google.com (mail-pl1-f195.google.com [209.85.214.195]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 1BE27689963 for ; Wed, 13 May 2020 18:13:23 +0300 (EEST) Received: by mail-pl1-f195.google.com with SMTP id b12so5024109plz.13 for ; Wed, 13 May 2020 08:13:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=JPwcg53FbneaQjCfSlrDH5tBtlOjR0xig7Ug/rbahXI=; b=fvnGeWqbS2LQWZkoId7gLHvbBBQ7ovBNvndSsBMLXle10Q5w3uraHRWo11i+XQjwqZ 70Hs5h5OSDt0yppha13tDEQIh9SCM0CLhS+ErS8xOZRzHewKJeJBrT2YzQxFlTWpGrht zLMBrdWVjSnmpz2QtRysua5sWXEd3ymcMWD7WqIfV2J68XNK3YfXVCJtYgHB2wLPcBFA S8mRY/IJZjAV9rScD3coqecLEgs/WIRu6ZOQDCLeBEeOC0RDqsdXZBWDp4ylDipHXPii MH2L2kPo06ejToCP00j70FveDPjo1GG5d06Ni0EN+4g2MAxPhkorW7KqICj9EVNRfJWd Riuw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=JPwcg53FbneaQjCfSlrDH5tBtlOjR0xig7Ug/rbahXI=; b=qvbsmc1EvQ35GeGK7cRImgVR28/P2icbJSORjR7SE/reRJqoCLZI9uhNlPbdt4oWML r7Fi0a5rqT/lM1yeMkLXJRd3RnZbJ0IbO9Bes1GPLjrQK+ffIZFI0R7rqF9JsLsMq25c lX/7Lqyr67Rd2zbZFClGxvgNEhZUKYBok0mamSqSt6AEPfaqpMMzB+n+uQhyS9DeJ9dK ODY6/xpc0+sgzZzfc4KP/sAepSH9rAh+/7INsKjXcrJyCeIXQ8UgcPMU4es99yF/ttyQ 0oA1UyKnRXtvcJAOE3KqbVOKCAlMIx/StbVRqSvC0pkOTu/7Uywhx6PbAf/sOuWxLwuO 4HQA== X-Gm-Message-State: AOAM533Z+SmCvlf3RcZxox4DM685e5TRiPD8IAVDnRkQ674mFVBbjt5R d2Q3TTpSYgwZBO9yc5uGECKdv1BX X-Google-Smtp-Source: ABdhPJyaJ+UNdlxLdkbQUJYm/B4iHiv232E96qyOytDGMkjSF4GRNgZ+qOpOv0hT9skVGviefyhvxA== X-Received: by 2002:a17:90a:db0f:: with SMTP id g15mr2911502pjv.8.1589382801200; Wed, 13 May 2020 08:13:21 -0700 (PDT) Received: from vpn2.localdomain ([161.117.202.209]) by smtp.gmail.com with ESMTPSA id t64sm43309pgd.24.2020.05.13.08.13.19 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 13 May 2020 08:13:20 -0700 (PDT) From: lance.lmwang@gmail.com To: ffmpeg-devel@ffmpeg.org Date: Wed, 13 May 2020 23:12:43 +0800 Message-Id: <1589382763-28061-4-git-send-email-lance.lmwang@gmail.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1589382763-28061-1-git-send-email-lance.lmwang@gmail.com> References: <1589382763-28061-1-git-send-email-lance.lmwang@gmail.com> Subject: [FFmpeg-devel] [PATCH v2 4/4] avfilter/vf_drawbox: support draw specific face by facedetect metadata X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Limin Wang MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" From: Limin Wang Signed-off-by: Limin Wang --- doc/filters.texi | 10 ++++++++++ libavfilter/vf_drawbox.c | 27 +++++++++++++++++++++++++++ 2 files changed, 37 insertions(+) diff --git a/doc/filters.texi b/doc/filters.texi index 76e12ef..bf9043c 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -9337,6 +9337,10 @@ See below for the list of accepted constants. Applicable if the input has alpha. With value @code{1}, the pixels of the painted box will overwrite the video's color and alpha pixels. Default is @code{0}, which composites the box onto the input, leaving the video's alpha intact. + +@item face +Draw the box by the facedetect metadata for the specific face. + @end table The parameters for @var{x}, @var{y}, @var{w} and @var{h} and @var{t} are expressions containing the @@ -9405,6 +9409,12 @@ Draw a 2-pixel red 2.40:1 mask: @example drawbox=x=-t:y=0.5*(ih-iw/2.4)-t:w=iw+t*2:h=iw/2.4+t*2:t=2:c=red @end example + +@item +draw the box with red color for the first face by metadata if its postion is detected: +@example +ocv=filter_name=facedetect:filter_params=facedetect=./haarcascade_frontalface_alt.xml,drawbox=face=0:color=red +@end example @end itemize @subsection Commands diff --git a/libavfilter/vf_drawbox.c b/libavfilter/vf_drawbox.c index 21d520e..239a149 100644 --- a/libavfilter/vf_drawbox.c +++ b/libavfilter/vf_drawbox.c @@ -81,6 +81,7 @@ typedef struct DrawBoxContext { char *t_expr; ///< expression for thickness int have_alpha; int replace; + int face; } DrawBoxContext; static const int NUM_EXPR_EVALS = 5; @@ -220,6 +221,31 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *frame) int plane, x, y, xb = s->x, yb = s->y; unsigned char *row[4]; + if (s->face >= 0) { + AVDictionaryEntry *ex, *ey, *ew, *eh; + char key2[128]; + AVDictionary *metadata = frame->metadata; + + snprintf(key2, sizeof(key2), "lavfi.facedetect.%d.%s", s->face, "x"); + ex = av_dict_get(metadata, key2, NULL, AV_DICT_MATCH_CASE); + + snprintf(key2, sizeof(key2), "lavfi.facedetect.%d.%s", s->face, "y"); + ey = av_dict_get(metadata, key2, NULL, AV_DICT_MATCH_CASE); + + snprintf(key2, sizeof(key2), "lavfi.facedetect.%d.%s", s->face, "w"); + ew = av_dict_get(metadata, key2, NULL, AV_DICT_MATCH_CASE); + + snprintf(key2, sizeof(key2), "lavfi.facedetect.%d.%s", s->face, "h"); + eh = av_dict_get(metadata, key2, NULL, AV_DICT_MATCH_CASE); + + if (ex && ey && ew && eh) { + xb = s->x = strtol(ex->value, NULL, 10); + yb = s->y = strtol(ey->value, NULL, 10); + s->w = strtol(ew->value, NULL, 10); + s->h = strtol(eh->value, NULL, 10); + } + } + if (s->have_alpha && s->replace) { for (y = FFMAX(yb, 0); y < frame->height && y < (yb + s->h); y++) { row[0] = frame->data[0] + y * frame->linesize[0]; @@ -323,6 +349,7 @@ static const AVOption drawbox_options[] = { { "thickness", "set the box thickness", OFFSET(t_expr), AV_OPT_TYPE_STRING, { .str="3" }, 0, 0, FLAGS }, { "t", "set the box thickness", OFFSET(t_expr), AV_OPT_TYPE_STRING, { .str="3" }, 0, 0, FLAGS }, { "replace", "replace color & alpha", OFFSET(replace), AV_OPT_TYPE_BOOL, { .i64=0 }, 0, 1, FLAGS }, + { "face", "set which face to draw with metadata", OFFSET(face), AV_OPT_TYPE_INT, { .i64=-1 }, -1, 256, FLAGS }, { NULL } };