From patchwork Wed Feb 10 04:59:02 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Guo, Yejun" X-Patchwork-Id: 25537 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id EC47F44BDDC for ; Wed, 10 Feb 2021 07:09:07 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id D225A68A56C; Wed, 10 Feb 2021 07:09:07 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id E7C52689D9F for ; Wed, 10 Feb 2021 07:09:05 +0200 (EET) IronPort-SDR: 8t3IpMxAxbF0AjBGDLrhf4TgunosIXFHmJXHyeDjEFoYyve0pZ5aP1GE0mssIPJKp6LK2Im1Dx ZlEYeHOe30Dg== X-IronPort-AV: E=McAfee;i="6000,8403,9890"; a="266854832" X-IronPort-AV: E=Sophos;i="5.81,167,1610438400"; d="scan'208";a="266854832" Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Feb 2021 21:09:04 -0800 IronPort-SDR: pPzsuTdPUI+gX0gXUNYEI4+yPJLnzBFPXRIpGgIYdRquKFo11SJNdv8pxLTDn55ihjNuvF21w2 C5qPi8/wqeaQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.81,167,1610438400"; d="scan'208";a="587228263" Received: from yguo18-skl-u1604.sh.intel.com ([10.239.159.53]) by fmsmga005.fm.intel.com with ESMTP; 09 Feb 2021 21:09:03 -0800 From: "Guo, Yejun" To: ffmpeg-devel@ffmpeg.org Date: Wed, 10 Feb 2021 12:59:02 +0800 Message-Id: <20210210045902.15013-1-yejun.guo@intel.com> X-Mailer: git-send-email 2.17.1 Subject: [FFmpeg-devel] [PATCH 09/10] libavfilter: show side data of dnn bounding box X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: yejun.guo@intel.com MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Signed-off-by: Guo, Yejun --- libavfilter/f_sidedata.c | 2 ++ libavfilter/vf_showinfo.c | 34 ++++++++++++++++++++++++++++++++++ 2 files changed, 36 insertions(+) diff --git a/libavfilter/f_sidedata.c b/libavfilter/f_sidedata.c index 3757723375..08ee96c4ef 100644 --- a/libavfilter/f_sidedata.c +++ b/libavfilter/f_sidedata.c @@ -71,6 +71,7 @@ static const AVOption filt_name##_options[] = { \ { "S12M_TIMECOD", "", 0, AV_OPT_TYPE_CONST, {.i64 = AV_FRAME_DATA_S12M_TIMECODE }, 0, 0, FLAGS, "type" }, \ { "DYNAMIC_HDR_PLUS", "", 0, AV_OPT_TYPE_CONST, {.i64 = AV_FRAME_DATA_DYNAMIC_HDR_PLUS }, 0, 0, FLAGS, "type" }, \ { "REGIONS_OF_INTEREST", "", 0, AV_OPT_TYPE_CONST, {.i64 = AV_FRAME_DATA_REGIONS_OF_INTEREST }, 0, 0, FLAGS, "type" }, \ + { "DNN_BOUNDING_BOXES", "", 0, AV_OPT_TYPE_CONST, {.i64 = AV_FRAME_DATA_DNN_BBOXES }, 0, 0, FLAGS, "type" }, \ { "SEI_UNREGISTERED", "", 0, AV_OPT_TYPE_CONST, {.i64 = AV_FRAME_DATA_SEI_UNREGISTERED }, 0, 0, FLAGS, "type" }, \ { NULL } \ } @@ -100,6 +101,7 @@ static const AVOption filt_name##_options[] = { \ { "S12M_TIMECOD", "", 0, AV_OPT_TYPE_CONST, {.i64 = AV_FRAME_DATA_S12M_TIMECODE }, 0, 0, FLAGS, "type" }, \ { "DYNAMIC_HDR_PLUS", "", 0, AV_OPT_TYPE_CONST, {.i64 = AV_FRAME_DATA_DYNAMIC_HDR_PLUS }, 0, 0, FLAGS, "type" }, \ { "REGIONS_OF_INTEREST", "", 0, AV_OPT_TYPE_CONST, {.i64 = AV_FRAME_DATA_REGIONS_OF_INTEREST }, 0, 0, FLAGS, "type" }, \ + { "DNN_BOUNDING_BOXES", "", 0, AV_OPT_TYPE_CONST, {.i64 = AV_FRAME_DATA_DNN_BBOXES }, 0, 0, FLAGS, "type" }, \ { "SEI_UNREGISTERED", "", 0, AV_OPT_TYPE_CONST, {.i64 = AV_FRAME_DATA_SEI_UNREGISTERED }, 0, 0, FLAGS, "type" }, \ { NULL } \ } diff --git a/libavfilter/vf_showinfo.c b/libavfilter/vf_showinfo.c index 6208892005..67653b17ec 100644 --- a/libavfilter/vf_showinfo.c +++ b/libavfilter/vf_showinfo.c @@ -38,6 +38,7 @@ #include "libavutil/timecode.h" #include "libavutil/mastering_display_metadata.h" #include "libavutil/video_enc_params.h" +#include "libavutil/dnn_bbox.h" #include "avfilter.h" #include "internal.h" @@ -153,6 +154,36 @@ static void dump_roi(AVFilterContext *ctx, const AVFrameSideData *sd) } } +static void dump_dnnbbox(AVFilterContext *ctx, const AVFrameSideData *sd) +{ + int nb_bbox; + const AVDnnBoundingBox *bbox; + uint32_t bbox_size; + + bbox = (const AVDnnBoundingBox *)sd->data; + bbox_size = bbox->self_size; + if (!bbox_size || sd->size % bbox_size != 0 || bbox_size != sizeof(*bbox)) { + av_log(ctx, AV_LOG_ERROR, "Invalid AVDnnBoundingBox.self_size.\n"); + return; + } + nb_bbox = sd->size / bbox_size; + + av_log(ctx, AV_LOG_INFO, "Dnn bounding boxes:\n"); + for (int i = 0; i < nb_bbox; i++) { + bbox = (const AVDnnBoundingBox *)(sd->data + bbox_size * i); + av_log(ctx, AV_LOG_INFO, "index: %d, region: (%d/%d, %d/%d) -> (%d/%d, %d/%d), label: %d, confidence: %d/%d.\n", + i, bbox->left, bbox->model_input_width, bbox->top, bbox->model_input_height, + bbox->right, bbox->model_input_width, bbox->bottom, bbox->model_input_height, + bbox->detect_label, bbox->detect_conf.num, bbox->detect_conf.den); + if (bbox->classify_count > 0) { + for (int j = 0; j < bbox->classify_count; j++) { + av_log(ctx, AV_LOG_INFO, "\t\tclassify: label: %d, confidence: %d/%d.\n", + bbox->classify_labels[j], bbox->classify_confs[j].num, bbox->classify_confs[j].den); + } + } + } +} + static void dump_mastering_display(AVFilterContext *ctx, const AVFrameSideData *sd) { const AVMasteringDisplayMetadata *mastering_display; @@ -494,6 +525,9 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *frame) case AV_FRAME_DATA_REGIONS_OF_INTEREST: dump_roi(ctx, sd); break; + case AV_FRAME_DATA_DNN_BBOXES: + dump_dnnbbox(ctx, sd); + break; case AV_FRAME_DATA_MASTERING_DISPLAY_METADATA: dump_mastering_display(ctx, sd); break;