From patchwork Wed Feb 10 09:34:31 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Guo, Yejun" X-Patchwork-Id: 25545 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 9A8C5449CE2 for ; Wed, 10 Feb 2021 11:44:55 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 8855368A640; Wed, 10 Feb 2021 11:44:55 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id B9C8368A57E for ; Wed, 10 Feb 2021 11:44:52 +0200 (EET) IronPort-SDR: cSA1oSUupvYyF14r2/dRsH0kR4DY62tNrfOhtgnVS2UeUptu8dTZlmcNbgMUkAKy/3Jgx3Mo+H 21Mm4Lyl98Pg== X-IronPort-AV: E=McAfee;i="6000,8403,9890"; a="201144887" X-IronPort-AV: E=Sophos;i="5.81,167,1610438400"; d="scan'208";a="201144887" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Feb 2021 01:44:42 -0800 IronPort-SDR: SdwRWPQwdpXiVrIzZ8UUFpJ56LBWXeo7gV9YsOx7a1Rs1ghLxtrMeVLX7gszo2HerU+lYRyYh0 HBAYhvtuUvkQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.81,167,1610438400"; d="scan'208";a="488706586" Received: from yguo18-skl-u1604.sh.intel.com ([10.239.159.53]) by fmsmga001.fm.intel.com with ESMTP; 10 Feb 2021 01:44:42 -0800 From: "Guo, Yejun" To: ffmpeg-devel@ffmpeg.org Date: Wed, 10 Feb 2021 17:34:31 +0800 Message-Id: <20210210093432.9135-9-yejun.guo@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210210093432.9135-1-yejun.guo@intel.com> References: <20210210093432.9135-1-yejun.guo@intel.com> Subject: [FFmpeg-devel] [PATCH V2 09/10] libavfilter: show side data of dnn bounding box X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: yejun.guo@intel.com MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Signed-off-by: Guo, Yejun --- libavfilter/f_sidedata.c | 2 ++ libavfilter/vf_showinfo.c | 34 ++++++++++++++++++++++++++++++++++ 2 files changed, 36 insertions(+) diff --git a/libavfilter/f_sidedata.c b/libavfilter/f_sidedata.c index 3757723375..08ee96c4ef 100644 --- a/libavfilter/f_sidedata.c +++ b/libavfilter/f_sidedata.c @@ -71,6 +71,7 @@ static const AVOption filt_name##_options[] = { \ { "S12M_TIMECOD", "", 0, AV_OPT_TYPE_CONST, {.i64 = AV_FRAME_DATA_S12M_TIMECODE }, 0, 0, FLAGS, "type" }, \ { "DYNAMIC_HDR_PLUS", "", 0, AV_OPT_TYPE_CONST, {.i64 = AV_FRAME_DATA_DYNAMIC_HDR_PLUS }, 0, 0, FLAGS, "type" }, \ { "REGIONS_OF_INTEREST", "", 0, AV_OPT_TYPE_CONST, {.i64 = AV_FRAME_DATA_REGIONS_OF_INTEREST }, 0, 0, FLAGS, "type" }, \ + { "DNN_BOUNDING_BOXES", "", 0, AV_OPT_TYPE_CONST, {.i64 = AV_FRAME_DATA_DNN_BBOXES }, 0, 0, FLAGS, "type" }, \ { "SEI_UNREGISTERED", "", 0, AV_OPT_TYPE_CONST, {.i64 = AV_FRAME_DATA_SEI_UNREGISTERED }, 0, 0, FLAGS, "type" }, \ { NULL } \ } @@ -100,6 +101,7 @@ static const AVOption filt_name##_options[] = { \ { "S12M_TIMECOD", "", 0, AV_OPT_TYPE_CONST, {.i64 = AV_FRAME_DATA_S12M_TIMECODE }, 0, 0, FLAGS, "type" }, \ { "DYNAMIC_HDR_PLUS", "", 0, AV_OPT_TYPE_CONST, {.i64 = AV_FRAME_DATA_DYNAMIC_HDR_PLUS }, 0, 0, FLAGS, "type" }, \ { "REGIONS_OF_INTEREST", "", 0, AV_OPT_TYPE_CONST, {.i64 = AV_FRAME_DATA_REGIONS_OF_INTEREST }, 0, 0, FLAGS, "type" }, \ + { "DNN_BOUNDING_BOXES", "", 0, AV_OPT_TYPE_CONST, {.i64 = AV_FRAME_DATA_DNN_BBOXES }, 0, 0, FLAGS, "type" }, \ { "SEI_UNREGISTERED", "", 0, AV_OPT_TYPE_CONST, {.i64 = AV_FRAME_DATA_SEI_UNREGISTERED }, 0, 0, FLAGS, "type" }, \ { NULL } \ } diff --git a/libavfilter/vf_showinfo.c b/libavfilter/vf_showinfo.c index 6208892005..67653b17ec 100644 --- a/libavfilter/vf_showinfo.c +++ b/libavfilter/vf_showinfo.c @@ -38,6 +38,7 @@ #include "libavutil/timecode.h" #include "libavutil/mastering_display_metadata.h" #include "libavutil/video_enc_params.h" +#include "libavutil/dnn_bbox.h" #include "avfilter.h" #include "internal.h" @@ -153,6 +154,36 @@ static void dump_roi(AVFilterContext *ctx, const AVFrameSideData *sd) } } +static void dump_dnnbbox(AVFilterContext *ctx, const AVFrameSideData *sd) +{ + int nb_bbox; + const AVDnnBoundingBox *bbox; + uint32_t bbox_size; + + bbox = (const AVDnnBoundingBox *)sd->data; + bbox_size = bbox->self_size; + if (!bbox_size || sd->size % bbox_size != 0 || bbox_size != sizeof(*bbox)) { + av_log(ctx, AV_LOG_ERROR, "Invalid AVDnnBoundingBox.self_size.\n"); + return; + } + nb_bbox = sd->size / bbox_size; + + av_log(ctx, AV_LOG_INFO, "Dnn bounding boxes:\n"); + for (int i = 0; i < nb_bbox; i++) { + bbox = (const AVDnnBoundingBox *)(sd->data + bbox_size * i); + av_log(ctx, AV_LOG_INFO, "index: %d, region: (%d/%d, %d/%d) -> (%d/%d, %d/%d), label: %d, confidence: %d/%d.\n", + i, bbox->left, bbox->model_input_width, bbox->top, bbox->model_input_height, + bbox->right, bbox->model_input_width, bbox->bottom, bbox->model_input_height, + bbox->detect_label, bbox->detect_conf.num, bbox->detect_conf.den); + if (bbox->classify_count > 0) { + for (int j = 0; j < bbox->classify_count; j++) { + av_log(ctx, AV_LOG_INFO, "\t\tclassify: label: %d, confidence: %d/%d.\n", + bbox->classify_labels[j], bbox->classify_confs[j].num, bbox->classify_confs[j].den); + } + } + } +} + static void dump_mastering_display(AVFilterContext *ctx, const AVFrameSideData *sd) { const AVMasteringDisplayMetadata *mastering_display; @@ -494,6 +525,9 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *frame) case AV_FRAME_DATA_REGIONS_OF_INTEREST: dump_roi(ctx, sd); break; + case AV_FRAME_DATA_DNN_BBOXES: + dump_dnnbbox(ctx, sd); + break; case AV_FRAME_DATA_MASTERING_DISPLAY_METADATA: dump_mastering_display(ctx, sd); break;