From patchwork Thu Apr 29 13:36:52 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Guo, Yejun" X-Patchwork-Id: 27479 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a11:4023:0:0:0:0 with SMTP id ky35csp1492491pxb; Thu, 29 Apr 2021 06:49:25 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxst+nGImN6OWjk0ehZ9C7ep16go0us+9z2qsfntWuGcU0p+QrKufwkD6Ey86sXeg840GZ+ X-Received: by 2002:a05:6402:708:: with SMTP id w8mr16033289edx.49.1619704165334; Thu, 29 Apr 2021 06:49:25 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1619704165; cv=none; d=google.com; s=arc-20160816; b=Bn4wiboOMIp4cWIPNMFzJOTy8fBqKsXxE8bv069cvzbQcIGkLZVAZypQCw37ZtgjYt 6NVSxiDIY6ehnJCzK+Nj+4gJqv2/edmNh5tIRigVcAsgFajo2m2eeop03zSK8KEaiuL4 flsx70uAao+4qy41pZJqpc9Mf2PfCYSsKK31YvGNdSBM1Jpv/xRrJHk+g7btYTSURSNc 5pTb0bup9jB+Nf/0t83/HCtOUuLyH/IDThkpZSvk1DULwMs90CdIrC7rVQ2l/EpdoitE 5LP0KU8M8Fovxqjr2eNk9wOmRp7/LeGUD7mhAPuwyy1tDxUFgRhOP+5hyZ42CSMX3c+9 VmgQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:message-id:date:to:from:ironport-sdr :ironport-sdr:delivered-to; bh=j9gA5EKgOScTWgb42TaV7aSbh1fcIhLr9og75ANqlmg=; b=KiVK1S9DncS7/HYovCKkFTA0yqfmloPtzEnLcSfWxkbd7aQZ/1k3Pnuvs8XMIQjGay Btpolxyy0dCe74NOrfFYKa4HqzLXEExpelP+CxqYRM49eD7ZJZRog3fP6H+GSPic7dnw HJ98WQxhpfYxNtTaMLTuMzTUKb83dXKkQIFretuDIfq7Np75LCAZu5WjEm2MCsnnuUAz K/q2kjdYCHDU+FKzgA5Jt8cX8cjYKAR4GsJPQvVpRrVae+/nIIhWX1OLKETlx7PF0uAo pqDL+CT8WIgusNdaRECzwBckSoR9PsZa18fp/RlpkdxyKrl7dg2S/oLubX0WYMXwaQ/v 8AuA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id eb8si3577168edb.49.2021.04.29.06.49.24; Thu, 29 Apr 2021 06:49:25 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 1AC2968A0B7; Thu, 29 Apr 2021 16:49:20 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id E8B08680545 for ; Thu, 29 Apr 2021 16:49:12 +0300 (EEST) IronPort-SDR: Txo32bYbGSUNyuZQLXeUWztdE9ZKl5xw8y735IsnbQpEM77qTpthl03CQLucIYM427xYDNPKlI lomMi7rOJlUw== X-IronPort-AV: E=McAfee;i="6200,9189,9969"; a="260956485" X-IronPort-AV: E=Sophos;i="5.82,259,1613462400"; d="scan'208";a="260956485" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Apr 2021 06:49:08 -0700 IronPort-SDR: KH1JG41OoD2GNe5M7DwBqIANpt4/pwQ+kQKEbrafUb0LCU2YMLKMC4BPfR1kMbkiv8XrI70Beg ZeFMQw8pK3HA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.82,259,1613462400"; d="scan'208";a="424096031" Received: from yguo18-skl-u1604.sh.intel.com ([10.239.159.53]) by fmsmga008.fm.intel.com with ESMTP; 29 Apr 2021 06:49:05 -0700 From: "Guo, Yejun" To: ffmpeg-devel@ffmpeg.org Date: Thu, 29 Apr 2021 21:36:52 +0800 Message-Id: <20210429133657.23076-1-yejun.guo@intel.com> X-Mailer: git-send-email 2.17.1 Subject: [FFmpeg-devel] [PATCH V2 1/6] lavfi/dnn_backend_openvino.c: unify code for infer request for sync/async X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: yejun.guo@intel.com MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: mXmC6vayGqwK --- the main change of V2 in this patch set is to rebase with latest code by resolving the conflicts libavfilter/dnn/dnn_backend_openvino.c | 49 +++++++++++--------------- 1 file changed, 21 insertions(+), 28 deletions(-) diff --git a/libavfilter/dnn/dnn_backend_openvino.c b/libavfilter/dnn/dnn_backend_openvino.c index a8032fe56b..267c154c87 100644 --- a/libavfilter/dnn/dnn_backend_openvino.c +++ b/libavfilter/dnn/dnn_backend_openvino.c @@ -52,9 +52,6 @@ typedef struct OVModel{ ie_core_t *core; ie_network_t *network; ie_executable_network_t *exe_network; - ie_infer_request_t *infer_request; - - /* for async execution */ SafeQueue *request_queue; // holds RequestItem Queue *task_queue; // holds TaskItem } OVModel; @@ -269,12 +266,9 @@ static void infer_completion_callback(void *args) ie_blob_free(&output_blob); request->task_count = 0; - - if (task->async) { - if (ff_safe_queue_push_back(requestq, request) < 0) { - av_log(ctx, AV_LOG_ERROR, "Failed to push back request_queue.\n"); - return; - } + if (ff_safe_queue_push_back(requestq, request) < 0) { + av_log(ctx, AV_LOG_ERROR, "Failed to push back request_queue.\n"); + return; } } @@ -347,11 +341,6 @@ static DNNReturnType init_model_ov(OVModel *ov_model, const char *input_name, co goto err; } - // create infer_request for sync execution - status = ie_exec_network_create_infer_request(ov_model->exe_network, &ov_model->infer_request); - if (status != OK) - goto err; - // create infer_requests for async execution if (ctx->options.nireq <= 0) { // the default value is a rough estimation @@ -502,10 +491,9 @@ static DNNReturnType get_output_ov(void *model, const char *input_name, int inpu OVModel *ov_model = model; OVContext *ctx = &ov_model->ctx; TaskItem task; - RequestItem request; + RequestItem *request; AVFrame *in_frame = NULL; AVFrame *out_frame = NULL; - TaskItem *ptask = &task; IEStatusCode status; input_shapes_t input_shapes; @@ -557,11 +545,16 @@ static DNNReturnType get_output_ov(void *model, const char *input_name, int inpu task.out_frame = out_frame; task.ov_model = ov_model; - request.infer_request = ov_model->infer_request; - request.task_count = 1; - request.tasks = &ptask; + request = ff_safe_queue_pop_front(ov_model->request_queue); + if (!request) { + av_frame_free(&out_frame); + av_frame_free(&in_frame); + av_log(ctx, AV_LOG_ERROR, "unable to get infer request.\n"); + return DNN_ERROR; + } + request->tasks[request->task_count++] = &task; - ret = execute_model_ov(&request); + ret = execute_model_ov(request); *output_width = out_frame->width; *output_height = out_frame->height; @@ -633,8 +626,7 @@ DNNReturnType ff_dnn_execute_model_ov(const DNNModel *model, const char *input_n OVModel *ov_model = model->model; OVContext *ctx = &ov_model->ctx; TaskItem task; - RequestItem request; - TaskItem *ptask = &task; + RequestItem *request; if (!in_frame) { av_log(ctx, AV_LOG_ERROR, "in frame is NULL when execute model.\n"); @@ -674,11 +666,14 @@ DNNReturnType ff_dnn_execute_model_ov(const DNNModel *model, const char *input_n task.out_frame = out_frame; task.ov_model = ov_model; - request.infer_request = ov_model->infer_request; - request.task_count = 1; - request.tasks = &ptask; + request = ff_safe_queue_pop_front(ov_model->request_queue); + if (!request) { + av_log(ctx, AV_LOG_ERROR, "unable to get infer request.\n"); + return DNN_ERROR; + } + request->tasks[request->task_count++] = &task; - return execute_model_ov(&request); + return execute_model_ov(request); } DNNReturnType ff_dnn_execute_model_async_ov(const DNNModel *model, const char *input_name, AVFrame *in_frame, @@ -821,8 +816,6 @@ void ff_dnn_free_model_ov(DNNModel **model) } ff_queue_destroy(ov_model->task_queue); - if (ov_model->infer_request) - ie_infer_request_free(&ov_model->infer_request); if (ov_model->exe_network) ie_exec_network_free(&ov_model->exe_network); if (ov_model->network)