From patchwork Fri Feb 19 06:59:44 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Guo, Yejun" X-Patchwork-Id: 25783 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 6A2014499CD for ; Fri, 19 Feb 2021 09:10:16 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 5CEF168A181; Fri, 19 Feb 2021 09:10:16 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id BD368680A4F for ; Fri, 19 Feb 2021 09:10:08 +0200 (EET) IronPort-SDR: U10XhHRHGjor6SvqAa39lCShdWh6aPUddlImcgV1fdKc3/bNCf9fy0J/c1kpJFbQonQYpt/NBY ohOwbBVQKVXw== X-IronPort-AV: E=McAfee;i="6000,8403,9899"; a="183826765" X-IronPort-AV: E=Sophos;i="5.81,189,1610438400"; d="scan'208";a="183826765" Received: from orsmga002.jf.intel.com ([10.7.209.21]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 18 Feb 2021 23:10:05 -0800 IronPort-SDR: 4hEliV4lRXkdUveMXxJsv28ueeU8MnRxNiB9ySh0oFjxvy6wxysEg+1NbIQ0QPXl3no8vMMS/b HPe5nzLxiGEQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.81,189,1610438400"; d="scan'208";a="378718807" Received: from yguo18-skl-u1604.sh.intel.com ([10.239.159.53]) by orsmga002.jf.intel.com with ESMTP; 18 Feb 2021 23:10:04 -0800 From: "Guo, Yejun" To: ffmpeg-devel@ffmpeg.org Date: Fri, 19 Feb 2021 14:59:44 +0800 Message-Id: <20210219065945.15594-3-yejun.guo@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210219065945.15594-1-yejun.guo@intel.com> References: <20210219065945.15594-1-yejun.guo@intel.com> Subject: [FFmpeg-devel] [PATCH 3/4] libavfilter: show side data of bounding box X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: yejun.guo@intel.com MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Signed-off-by: Guo, Yejun --- libavfilter/f_sidedata.c | 2 ++ libavfilter/vf_showinfo.c | 38 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 40 insertions(+) diff --git a/libavfilter/f_sidedata.c b/libavfilter/f_sidedata.c index 3757723375..912fb0d881 100644 --- a/libavfilter/f_sidedata.c +++ b/libavfilter/f_sidedata.c @@ -71,6 +71,7 @@ static const AVOption filt_name##_options[] = { \ { "S12M_TIMECOD", "", 0, AV_OPT_TYPE_CONST, {.i64 = AV_FRAME_DATA_S12M_TIMECODE }, 0, 0, FLAGS, "type" }, \ { "DYNAMIC_HDR_PLUS", "", 0, AV_OPT_TYPE_CONST, {.i64 = AV_FRAME_DATA_DYNAMIC_HDR_PLUS }, 0, 0, FLAGS, "type" }, \ { "REGIONS_OF_INTEREST", "", 0, AV_OPT_TYPE_CONST, {.i64 = AV_FRAME_DATA_REGIONS_OF_INTEREST }, 0, 0, FLAGS, "type" }, \ + { "BOUNDING_BOXES", "", 0, AV_OPT_TYPE_CONST, {.i64 = AV_FRAME_DATA_BOUNDING_BOXES }, 0, 0, FLAGS, "type" }, \ { "SEI_UNREGISTERED", "", 0, AV_OPT_TYPE_CONST, {.i64 = AV_FRAME_DATA_SEI_UNREGISTERED }, 0, 0, FLAGS, "type" }, \ { NULL } \ } @@ -100,6 +101,7 @@ static const AVOption filt_name##_options[] = { \ { "S12M_TIMECOD", "", 0, AV_OPT_TYPE_CONST, {.i64 = AV_FRAME_DATA_S12M_TIMECODE }, 0, 0, FLAGS, "type" }, \ { "DYNAMIC_HDR_PLUS", "", 0, AV_OPT_TYPE_CONST, {.i64 = AV_FRAME_DATA_DYNAMIC_HDR_PLUS }, 0, 0, FLAGS, "type" }, \ { "REGIONS_OF_INTEREST", "", 0, AV_OPT_TYPE_CONST, {.i64 = AV_FRAME_DATA_REGIONS_OF_INTEREST }, 0, 0, FLAGS, "type" }, \ + { "BOUNDING_BOXES", "", 0, AV_OPT_TYPE_CONST, {.i64 = AV_FRAME_DATA_BOUNDING_BOXES }, 0, 0, FLAGS, "type" }, \ { "SEI_UNREGISTERED", "", 0, AV_OPT_TYPE_CONST, {.i64 = AV_FRAME_DATA_SEI_UNREGISTERED }, 0, 0, FLAGS, "type" }, \ { NULL } \ } diff --git a/libavfilter/vf_showinfo.c b/libavfilter/vf_showinfo.c index 6208892005..f23518fdd5 100644 --- a/libavfilter/vf_showinfo.c +++ b/libavfilter/vf_showinfo.c @@ -38,6 +38,7 @@ #include "libavutil/timecode.h" #include "libavutil/mastering_display_metadata.h" #include "libavutil/video_enc_params.h" +#include "bbox.h" #include "avfilter.h" #include "internal.h" @@ -153,6 +154,40 @@ static void dump_roi(AVFilterContext *ctx, const AVFrameSideData *sd) } } +static void dump_boundingbox(AVFilterContext *ctx, const AVFrameSideData *sd) +{ + int nb_bbox; + const BoundingBoxHeader *header; + const BoundingBox *bbox; + uint32_t bbox_size, array_size; + + header = (const BoundingBoxHeader *)sd->data; + bbox_size = header->bbox_size; + array_size = sd->size - sizeof(*header); + if (!bbox_size || array_size % bbox_size != 0 || bbox_size != sizeof(*bbox)) { + av_log(ctx, AV_LOG_ERROR, "Invalid BoundingBoxHeader.bbox_size.\n"); + return; + } + + nb_bbox = array_size / bbox_size; + bbox = (const BoundingBox *)(header + 1); + + av_log(ctx, AV_LOG_INFO, "bounding boxes:\n"); + av_log(ctx, AV_LOG_INFO, "source: %s\n", header->source); + for (int i = 0; i < nb_bbox; i++) { + av_log(ctx, AV_LOG_INFO, "index: %d, region: (%d, %d) -> (%d, %d), label: %s, confidence: %d/%d.\n", + i, bbox->left, bbox->top, bbox->right, bbox->bottom, + bbox->detect_label, bbox->detect_confidence.num, bbox->detect_confidence.den); + if (bbox->classify_count > 0) { + for (int j = 0; j < bbox->classify_count; j++) { + av_log(ctx, AV_LOG_INFO, "\t\tclassify: label: %s, confidence: %d/%d.\n", + bbox->classify_labels[j], bbox->classify_confidences[j].num, bbox->classify_confidences[j].den); + } + } + bbox++; + } +} + static void dump_mastering_display(AVFilterContext *ctx, const AVFrameSideData *sd) { const AVMasteringDisplayMetadata *mastering_display; @@ -494,6 +529,9 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *frame) case AV_FRAME_DATA_REGIONS_OF_INTEREST: dump_roi(ctx, sd); break; + case AV_FRAME_DATA_BOUNDING_BOXES: + dump_boundingbox(ctx, sd); + break; case AV_FRAME_DATA_MASTERING_DISPLAY_METADATA: dump_mastering_display(ctx, sd); break;