From patchwork Mon Jun 27 10:02:45 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ting Fu X-Patchwork-Id: 36486 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:8b27:b0:88:1bbf:7fd2 with SMTP id l39csp1923501pzh; Mon, 27 Jun 2022 03:18:49 -0700 (PDT) X-Google-Smtp-Source: AGRyM1uHLVyQqW77CTC7w0UlNqKQGcfXJIOgquY6PpkPxMH371HV2G2/gC+8Axlfk/7G/zIx6LYs X-Received: by 2002:a05:6402:358c:b0:435:9daf:e825 with SMTP id y12-20020a056402358c00b004359dafe825mr15703817edc.375.1656325129574; Mon, 27 Jun 2022 03:18:49 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1656325129; cv=none; d=google.com; s=arc-20160816; b=pafZwHIjQIDcJ3NLfOVm3k4TRJbm1DPC+bajAajPmDhQBZKVpSdcbb0QBa9cxCOb2n DzAlnJJWoiMElfZwXxs8hPs6w3uImnf+e0GxalmOyXgf+JhEokrMegxY9hurnHV56Kps l6FzSyL7NJfiu8Bf0h/dPQOWYTReC0RJpsGk2dcnvVY0dqRGKUR/X/PAWRuJXvaBqY9P PelbbJqlBdZj/uYBBoL5FK6J7i6i6mORVe+1bhWq4ehfWa6IYQV5pJBqYzYZBUNrx9MK iYzo6dRLbrYuWG7iZ2F+h/8Pwfk/gLjKgCP3tqQqFBSul09ZbHqKMkPL/GxSagynBVYo 0x9w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:message-id:date:to:from:dkim-signature :delivered-to; bh=QX4LZ4GA48sl8F6jlxd7oahMLfzDicE0NJbxPaEFiJ4=; b=X0ja6HBn2SiDLsAcWTg+ibzPgBxvHyT6UfgCAweINYva+heJc/iBAEPliSBdILIbn/ 1yRJ766xkaCncYOj/60M+3lY8VyNS7GXdqkUN2TxUnxCRegapXmo5Jl4+jJMMDFKXLIo HDpxSLFh9JFoIykFeUkkAImLwxRWf5fl1hs9AOfVbSS13mNaXEjJxlrCH1YynpQI8dQR 67HhB6yKxoGnVT/4ctjvQd7xUCIR7aITVbKMV3T2C0ogngy8CwgfZh+8R/x16xlAHyac 4MhyeOC6uMFQv0mXo9ROiyfz3CwU5tEpKGtZDLQW/bMXIDfFiVdkP8nIc50VGPanQXqT /PtQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=gMXShUIX; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id dr8-20020a170907720800b00715790f718dsi12730389ejc.318.2022.06.27.03.18.49; Mon, 27 Jun 2022 03:18:49 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@intel.com header.s=Intel header.b=gMXShUIX; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id DED6B68B845; Mon, 27 Jun 2022 13:18:46 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id BC84068B865 for ; Mon, 27 Jun 2022 13:18:39 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1656325125; x=1687861125; h=from:to:subject:date:message-id; bh=Hfkm5lc7jDaIkDf/heLoovSnhY30qAwWyy8AD8r8Y4U=; b=gMXShUIXG+Vq0S04caIHqco410JCU3jt5MXJtt56/gBa4UjN/ouiksUs 6GhBAwKL328csCkv9YHCgx0faCSZhg/o7qbo4IUkXNHZkbfuT2f29bUTE aBK9A1H2ztGw/oZsr7xJ5Gc3zCyvHUTpPWr+YR/azPuQeFicvPw6a4IPb YH6Tl6yS3f2Jn5fLoxWvopUQ7HXCUYvMcI137FxV+R5wzJoD1qR37iV6x AS19jBMAiMw56hHH5cLNs9TFku1q6Bum2+bY9MWJv9NZmIZTJd4Ra/AUx hwXT6PEGrQ9wsZOLO7sSu1YVGeaiV83/OJoUfP7ZMZFElHQdnIf+K+Dfv Q==; X-IronPort-AV: E=McAfee;i="6400,9594,10390"; a="281456836" X-IronPort-AV: E=Sophos;i="5.92,226,1650956400"; d="scan'208";a="281456836" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Jun 2022 03:18:38 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.92,226,1650956400"; d="scan'208";a="732258062" Received: from semmer-ubuntu.sh.intel.com ([10.239.159.83]) by fmsmga001.fm.intel.com with ESMTP; 27 Jun 2022 03:18:37 -0700 From: Ting Fu To: ffmpeg-devel@ffmpeg.org Date: Mon, 27 Jun 2022 18:02:45 +0800 Message-Id: <20220627100245.31981-1-ting.fu@intel.com> X-Mailer: git-send-email 2.17.1 Subject: [FFmpeg-devel] [PATCH] lavf/dnn: refine the error log of OpenVINO incorrect input/output name. X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: GIt9d+dnmr1c Refine the error report of OpenVINO backend incorrect input or output node name, help the users to locate issue. Signed-off-by: Ting Fu --- libavfilter/dnn/dnn_backend_openvino.c | 54 ++++++++++++++++++++++---- 1 file changed, 47 insertions(+), 7 deletions(-) diff --git a/libavfilter/dnn/dnn_backend_openvino.c b/libavfilter/dnn/dnn_backend_openvino.c index cf012aca4c..02d40327a5 100644 --- a/libavfilter/dnn/dnn_backend_openvino.c +++ b/libavfilter/dnn/dnn_backend_openvino.c @@ -58,6 +58,8 @@ typedef struct OVModel{ SafeQueue *request_queue; // holds OVRequestItem Queue *task_queue; // holds TaskItem Queue *lltask_queue; // holds LastLevelTaskItem + char *all_input_names; + char *all_output_names; } OVModel; // one request for one call to openvino @@ -336,13 +338,23 @@ static int init_model_ov(OVModel *ov_model, const char *input_name, const char * // while we pass NHWC data from FFmpeg to openvino status = ie_network_set_input_layout(ov_model->network, input_name, NHWC); if (status != OK) { - av_log(ctx, AV_LOG_ERROR, "Failed to set layout as NHWC for input %s\n", input_name); + if (status == NOT_FOUND) { + av_log(ctx, AV_LOG_ERROR, "Could not find \"%s\" in model, failed to set input layout as NHWC, "\ + "all input(s) are: \"%s\"\n", input_name, ov_model->all_input_names); + } else{ + av_log(ctx, AV_LOG_ERROR, "Failed to set layout as NHWC for input %s\n", input_name); + } ret = DNN_GENERIC_ERROR; goto err; } status = ie_network_set_output_layout(ov_model->network, output_name, NHWC); if (status != OK) { - av_log(ctx, AV_LOG_ERROR, "Failed to set layout as NHWC for output %s\n", output_name); + if (status == NOT_FOUND) { + av_log(ctx, AV_LOG_ERROR, "Could not find \"%s\" in model, failed to set output layout as NHWC, "\ + "all output(s) are: \"%s\"\n", input_name, ov_model->all_output_names); + } else{ + av_log(ctx, AV_LOG_ERROR, "Failed to set layout as NHWC for output %s\n", output_name); + } ret = DNN_GENERIC_ERROR; goto err; } @@ -505,7 +517,6 @@ static int get_input_ov(void *model, DNNData *input, const char *input_name) OVModel *ov_model = model; OVContext *ctx = &ov_model->ctx; char *model_input_name = NULL; - char *all_input_names = NULL; IEStatusCode status; size_t model_input_count = 0; dimensions_t dims; @@ -538,15 +549,12 @@ static int get_input_ov(void *model, DNNData *input, const char *input_name) input->width = input_resizable ? -1 : dims.dims[3]; input->dt = precision_to_datatype(precision); return 0; - } else { - //incorrect input name - APPEND_STRING(all_input_names, model_input_name) } ie_network_name_free(&model_input_name); } - av_log(ctx, AV_LOG_ERROR, "Could not find \"%s\" in model, all input(s) are: \"%s\"\n", input_name, all_input_names); + av_log(ctx, AV_LOG_ERROR, "Could not find \"%s\" in model, all input(s) are: \"%s\"\n", input_name, ov_model->all_input_names); return AVERROR(EINVAL); } @@ -729,6 +737,8 @@ DNNModel *ff_dnn_load_model_ov(const char *model_filename, DNNFunctionType func_ OVModel *ov_model = NULL; OVContext *ctx = NULL; IEStatusCode status; + size_t node_count = 0; + char *node_name = NULL; model = av_mallocz(sizeof(DNNModel)); if (!model){ @@ -744,6 +754,8 @@ DNNModel *ff_dnn_load_model_ov(const char *model_filename, DNNFunctionType func_ ov_model->model = model; ov_model->ctx.class = &dnn_openvino_class; ctx = &ov_model->ctx; + ov_model->all_input_names = NULL; + ov_model->all_output_names = NULL; //parse options av_opt_set_defaults(ctx); @@ -767,6 +779,34 @@ DNNModel *ff_dnn_load_model_ov(const char *model_filename, DNNFunctionType func_ goto err; } + //get all the input and output names + status = ie_network_get_inputs_number(ov_model->network, &node_count); + if (status != OK) { + av_log(ctx, AV_LOG_ERROR, "Failed to get input count\n"); + goto err; + } + for (size_t i = 0; i < node_count; i++) { + status = ie_network_get_input_name(ov_model->network, i, &node_name); + if (status != OK) { + av_log(ctx, AV_LOG_ERROR, "Failed to get No.%d input's name\n", (int)i); + goto err; + } + APPEND_STRING(ov_model->all_input_names, node_name) + } + status = ie_network_get_outputs_number(ov_model->network, &node_count); + if (status != OK) { + av_log(ctx, AV_LOG_ERROR, "Failed to get output count\n"); + goto err; + } + for (size_t i = 0; i < node_count; i++) { + status = ie_network_get_output_name(ov_model->network, i, &node_name); + if (status != OK) { + av_log(ctx, AV_LOG_ERROR, "Failed to get No.%d output's name\n", (int)i); + goto err; + } + APPEND_STRING(ov_model->all_output_names, node_name) + } + model->get_input = &get_input_ov; model->get_output = &get_output_ov; model->options = options;