From patchwork Sat Jun 5 18:08:02 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shubhanshu Saxena X-Patchwork-Id: 28104 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a6b:b214:0:0:0:0:0 with SMTP id b20csp2155688iof; Sat, 5 Jun 2021 11:38:28 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxd/YVd2sezQWsuJR3HVLWZN5QhQsJTh2/mqdVF/d+pM8V1iZVTgvnrzxY70m77Iw+mg+/O X-Received: by 2002:a17:906:3912:: with SMTP id f18mr10128730eje.161.1622918308094; Sat, 05 Jun 2021 11:38:28 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1622918308; cv=none; d=google.com; s=arc-20160816; b=NYr8qEbJJ52JTC7k/v5z4IBPb59pw1V3iqTDGOp+VOjlThKFX7ykOGzjJaBHRPVgc/ b2csv6HFQsFkCUaN0D6W91F0JCMVxVECApvd1WNcDHQRi/jmKTi5px0L/s6I7O1tOHO/ nO5b7G4v5iMHVbhbYhFHmlaTgrgS91jF3fZVBCJDp0jFN/qygi5Ukn1VZ/urIZJmcw6w PWigmpNT0AyF9Wjbfa6IwnFh902K8rFieCMpgsvVxVf94mmYaY9/GD5rCw4Ws9h7jsOB 9qtOPHuKCLD3pEnyV47lVkR+xQdlOnuFAVRYuDgJIldnsBSkzeAhKEVq+HdxKMzciLBP I12g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:message-id:date:to:from :dkim-signature:delivered-to; bh=GEo/0/1XrAdrKdht1l/eg+5SqgjoYss5ufrRkgIYRus=; b=ZjRUCJxl/ywEcuLkNavlDcqqhDIcT1hA4tc4Bkhqba8hWj7HWKNCd0mgDQpSixUcyy KboOxPGIOAUY0Zso2mreP2EsVYO7R8S/4abZpK2OsbVgqD5hrgD/vZlaG+Jd4sJmASe3 ImQwkfii/oG2DzHBe3b2Xdh4cnfkGzTc1fk4tn0JJ/qa9FsxEl1L4cYEu5e+JK/cvtWG cM+htOnxkHJv3SbNzdId4RCkAcnK6OlzvHdMO9Jb9jswxBmsGqostYHTlx6qgrS3ApQq qh0ClgeHA/915yAlcOYpqRdBqDbXpUV7HnbrI2IWCTlUu2tsU0FF4G2eZ6NkBc/nVRwt mnmQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b=M5yk9kRy; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id m4si9090037eja.301.2021.06.05.11.38.27; Sat, 05 Jun 2021 11:38:28 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b=M5yk9kRy; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 1F2C868A62B; Sat, 5 Jun 2021 21:38:24 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-oi1-f174.google.com (mail-oi1-f174.google.com [209.85.167.174]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 87EC068A4B0 for ; Sat, 5 Jun 2021 21:38:17 +0300 (EEST) Received: by mail-oi1-f174.google.com with SMTP id m137so9537009oig.6 for ; Sat, 05 Jun 2021 11:38:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=erF11YD3py2f2l5ecb/WCbFCl37zNobmVZU30NJq9VU=; b=M5yk9kRyEvgntDvk1tsLJtbv4RkTaww9PZBm/rF5a8m0TPAN1ctplkTSRcqjKsYwOP a1YsDEMhOCtRA3AHWzJqrh2hgjLpn3gEibrHdBpNP1A0m/xzneWfw6JeobR+ar4Ql5m+ UfCogFdO6W9PtZncJHLK7OdnMlK4xqoYr9MGTHHsJMzIKaXLf7gCaaU2NY0+tGpQSYGR P1sxJyWqDbhhmptoPoGhBdlyBRs1srfCG6A2MbL5Y8GjPKfQ07la7JB3I+JXIfoCoEa4 V+VGb0BcMSDcGgjN863+JWS0oi3ojSOEuH/pJn7JRii5K9Bkub6EaaNelIQOAYFCDtW8 HyTw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=erF11YD3py2f2l5ecb/WCbFCl37zNobmVZU30NJq9VU=; b=Xn/uJLeQuopAq/XGO9snEfjw7DE+NnIuyWe+wRZNYSJLnPl4F6LZXt12Wx/rhymFW7 jAsFyxtB5UxinxCWG/4W+PHGaR3M9r3wyHfTvtx6wH+4m/SASEZIfA4cp0doRd1LyPhH Bxc7dxjz8wxqcx/bo3+f5xWySo0qPpglsPCQfF3PAeyFfUNa6SxcfAAP0j0byTo8YaZ3 oMNwMWz/+gWzIOrDNzyan0ZL2l7OtvNp1VAzwY2w0V2/cvwbctUG0oub907iPVwJYTg9 wZ+i5I0kfs9kU+if5MjvVf2XMhyUMv0Hs7jpg9SjnpPZW6ljPMfhQFkCGueF6x7wanfr f+BA== X-Gm-Message-State: AOAM533cwg+sitwI7YHvf5J50UEOlh72xDdGTaTX+Kr5gmN2qPyPdK2c +/Fxu4HUmPJHiwEjVjFNEv0XAzy6HWqNxg== X-Received: by 2002:a17:90b:2250:: with SMTP id hk16mr22832480pjb.95.1622916503546; Sat, 05 Jun 2021 11:08:23 -0700 (PDT) Received: from Pavilion-x360.bbrouter ([103.157.221.215]) by smtp.googlemail.com with ESMTPSA id w14sm7818176pjf.12.2021.06.05.11.08.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 05 Jun 2021 11:08:23 -0700 (PDT) From: Shubhanshu Saxena To: ffmpeg-devel@ffmpeg.org Date: Sat, 5 Jun 2021 23:38:02 +0530 Message-Id: <20210605180806.65831-1-shubhanshu.e01@gmail.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH V2 1/5] lavfi/dnn: Extract TaskItem and InferenceItem from OpenVino Backend X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Shubhanshu Saxena Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: AHo8zwlXqkzk Extract TaskItem and InferenceItem from OpenVino backend and convert ov_model to void in TaskItem. Signed-off-by: Shubhanshu Saxena --- libavfilter/dnn/dnn_backend_common.h | 19 +++++++++ libavfilter/dnn/dnn_backend_openvino.c | 58 ++++++++++---------------- 2 files changed, 40 insertions(+), 37 deletions(-) diff --git a/libavfilter/dnn/dnn_backend_common.h b/libavfilter/dnn/dnn_backend_common.h index cd9c0f5339..0c043e51f0 100644 --- a/libavfilter/dnn/dnn_backend_common.h +++ b/libavfilter/dnn/dnn_backend_common.h @@ -26,6 +26,25 @@ #include "../dnn_interface.h" +// one task for one function call from dnn interface +typedef struct TaskItem { + void *model; // model for the backend + AVFrame *in_frame; + AVFrame *out_frame; + const char *input_name; + const char *output_name; + int async; + int do_ioproc; + uint32_t inference_todo; + uint32_t inference_done; +} TaskItem; + +// one task might have multiple inferences +typedef struct InferenceItem { + TaskItem *task; + uint32_t bbox_index; +} InferenceItem; + int ff_check_exec_params(void *ctx, DNNBackendType backend, DNNFunctionType func_type, DNNExecBaseParams *exec_params); #endif diff --git a/libavfilter/dnn/dnn_backend_openvino.c b/libavfilter/dnn/dnn_backend_openvino.c index 58c4ec9c9b..a84370d689 100644 --- a/libavfilter/dnn/dnn_backend_openvino.c +++ b/libavfilter/dnn/dnn_backend_openvino.c @@ -59,25 +59,6 @@ typedef struct OVModel{ Queue *inference_queue; // holds InferenceItem } OVModel; -// one task for one function call from dnn interface -typedef struct TaskItem { - OVModel *ov_model; - const char *input_name; - AVFrame *in_frame; - const char *output_name; - AVFrame *out_frame; - int do_ioproc; - int async; - uint32_t inference_todo; - uint32_t inference_done; -} TaskItem; - -// one task might have multiple inferences -typedef struct InferenceItem { - TaskItem *task; - uint32_t bbox_index; -} InferenceItem; - // one request for one call to openvino typedef struct RequestItem { ie_infer_request_t *infer_request; @@ -184,7 +165,7 @@ static DNNReturnType fill_model_input_ov(OVModel *ov_model, RequestItem *request request->inferences[i] = inference; request->inference_count = i + 1; task = inference->task; - switch (task->ov_model->model->func_type) { + switch (ov_model->model->func_type) { case DFT_PROCESS_FRAME: if (task->do_ioproc) { if (ov_model->model->frame_pre_proc != NULL) { @@ -220,11 +201,12 @@ static void infer_completion_callback(void *args) RequestItem *request = args; InferenceItem *inference = request->inferences[0]; TaskItem *task = inference->task; - SafeQueue *requestq = task->ov_model->request_queue; + OVModel *ov_model = task->model; + SafeQueue *requestq = ov_model->request_queue; ie_blob_t *output_blob = NULL; ie_blob_buffer_t blob_buffer; DNNData output; - OVContext *ctx = &task->ov_model->ctx; + OVContext *ctx = &ov_model->ctx; status = ie_infer_request_get_blob(request->infer_request, task->output_name, &output_blob); if (status != OK) { @@ -233,9 +215,9 @@ static void infer_completion_callback(void *args) char *all_output_names = NULL; size_t model_output_count = 0; av_log(ctx, AV_LOG_ERROR, "Failed to get model output data\n"); - status = ie_network_get_outputs_number(task->ov_model->network, &model_output_count); + status = ie_network_get_outputs_number(ov_model->network, &model_output_count); for (size_t i = 0; i < model_output_count; i++) { - status = ie_network_get_output_name(task->ov_model->network, i, &model_output_name); + status = ie_network_get_output_name(ov_model->network, i, &model_output_name); APPEND_STRING(all_output_names, model_output_name) } av_log(ctx, AV_LOG_ERROR, @@ -271,11 +253,11 @@ static void infer_completion_callback(void *args) task = request->inferences[i]->task; task->inference_done++; - switch (task->ov_model->model->func_type) { + switch (ov_model->model->func_type) { case DFT_PROCESS_FRAME: if (task->do_ioproc) { - if (task->ov_model->model->frame_post_proc != NULL) { - task->ov_model->model->frame_post_proc(task->out_frame, &output, task->ov_model->model->filter_ctx); + if (ov_model->model->frame_post_proc != NULL) { + ov_model->model->frame_post_proc(task->out_frame, &output, ov_model->model->filter_ctx); } else { ff_proc_from_dnn_to_frame(task->out_frame, &output, ctx); } @@ -285,18 +267,18 @@ static void infer_completion_callback(void *args) } break; case DFT_ANALYTICS_DETECT: - if (!task->ov_model->model->detect_post_proc) { + if (!ov_model->model->detect_post_proc) { av_log(ctx, AV_LOG_ERROR, "detect filter needs to provide post proc\n"); return; } - task->ov_model->model->detect_post_proc(task->out_frame, &output, 1, task->ov_model->model->filter_ctx); + ov_model->model->detect_post_proc(task->out_frame, &output, 1, ov_model->model->filter_ctx); break; case DFT_ANALYTICS_CLASSIFY: - if (!task->ov_model->model->classify_post_proc) { + if (!ov_model->model->classify_post_proc) { av_log(ctx, AV_LOG_ERROR, "classify filter needs to provide post proc\n"); return; } - task->ov_model->model->classify_post_proc(task->out_frame, &output, request->inferences[i]->bbox_index, task->ov_model->model->filter_ctx); + ov_model->model->classify_post_proc(task->out_frame, &output, request->inferences[i]->bbox_index, ov_model->model->filter_ctx); break; default: av_assert0(!"should not reach here"); @@ -445,6 +427,7 @@ static DNNReturnType execute_model_ov(RequestItem *request, Queue *inferenceq) InferenceItem *inference; TaskItem *task; OVContext *ctx; + OVModel *ov_model; if (ff_queue_size(inferenceq) == 0) { return DNN_SUCCESS; @@ -452,10 +435,11 @@ static DNNReturnType execute_model_ov(RequestItem *request, Queue *inferenceq) inference = ff_queue_peek_front(inferenceq); task = inference->task; - ctx = &task->ov_model->ctx; + ov_model = task->model; + ctx = &ov_model->ctx; if (task->async) { - ret = fill_model_input_ov(task->ov_model, request); + ret = fill_model_input_ov(ov_model, request); if (ret != DNN_SUCCESS) { return ret; } @@ -471,7 +455,7 @@ static DNNReturnType execute_model_ov(RequestItem *request, Queue *inferenceq) } return DNN_SUCCESS; } else { - ret = fill_model_input_ov(task->ov_model, request); + ret = fill_model_input_ov(ov_model, request); if (ret != DNN_SUCCESS) { return ret; } @@ -694,7 +678,7 @@ static DNNReturnType get_output_ov(void *model, const char *input_name, int inpu task.in_frame = in_frame; task.output_name = output_name; task.out_frame = out_frame; - task.ov_model = ov_model; + task.model = ov_model; if (extract_inference_from_task(ov_model->model->func_type, &task, ov_model->inference_queue, NULL) != DNN_SUCCESS) { av_frame_free(&out_frame); @@ -814,7 +798,7 @@ DNNReturnType ff_dnn_execute_model_ov(const DNNModel *model, DNNExecBaseParams * task.in_frame = exec_params->in_frame; task.output_name = exec_params->output_names[0]; task.out_frame = exec_params->out_frame ? exec_params->out_frame : exec_params->in_frame; - task.ov_model = ov_model; + task.model = ov_model; if (extract_inference_from_task(ov_model->model->func_type, &task, ov_model->inference_queue, exec_params) != DNN_SUCCESS) { av_log(ctx, AV_LOG_ERROR, "unable to extract inference from task.\n"); @@ -861,7 +845,7 @@ DNNReturnType ff_dnn_execute_model_async_ov(const DNNModel *model, DNNExecBasePa task->in_frame = exec_params->in_frame; task->output_name = exec_params->output_names[0]; task->out_frame = exec_params->out_frame ? exec_params->out_frame : exec_params->in_frame; - task->ov_model = ov_model; + task->model = ov_model; if (ff_queue_push_back(ov_model->task_queue, task) < 0) { av_freep(&task); av_log(ctx, AV_LOG_ERROR, "unable to push back task_queue.\n"); From patchwork Sat Jun 5 18:08:03 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shubhanshu Saxena X-Patchwork-Id: 28102 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a6b:b214:0:0:0:0:0 with SMTP id b20csp2139767iof; Sat, 5 Jun 2021 11:08:55 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxACJ4F6k/iVTmOzKtLfK6lYmXdsW3GaHb/8gxXUmROLnTsIRD+frBCPGuvJFLp3cfj017W X-Received: by 2002:a17:907:7b91:: with SMTP id ne17mr4232697ejc.58.1622916534826; Sat, 05 Jun 2021 11:08:54 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1622916534; cv=none; d=google.com; s=arc-20160816; b=hWh51oVnY4zFkl2luOH26pjrpciHyDLpoIqHgF9Pq3tgkBSI+MkDROCxtRLTvC4fYe 1FmPGL2r4RjAQOLOhfhl9WwVrkuGOJ296lvBvElrvU4kU2oNrGY8+UBXJlLbpqYxjZ6Q ptkuaM9QJGoUF4ye5OHAHCjYnib2g13yAXavo8eJw1scogVj6K/NxuwlruLKDdRBnJOj BsPApCrLgsbWDS4Q86MXKe30iY3vlVPYoVmLHmOZ/XdGaoHXJmfQigRu1yKL68rxWmd/ uYi6I7RsEw/R7o2/ePetGHsdfNrxiQ3X0elxFS/b/uMYpxAa8jGLIwZM8mDO+2D5grPX cbTg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=CCaIVmiHRqb3kTamUpgAh59WtS2XXtnqUQZx5GjaVbY=; b=QVcCl60S0R/Wa+HNB2RI1qxmJnoSRZUIK1VvuWIB/mybJ63Dp31830tcaX/FsvsZOJ vLw0G36bOt5ULNohCYy/ccs9QWPSKY7td7PYG/NSDaKvq3R3ffTrhOgJGhJ94c31L+oo gLinZjV+VUTl25wqsokbutPerX6FqlFReWU6ErXHOGRrE590mgEoeFjCzqKkSrHnUINL egctMKFgPV+6hfDH4wcLMn/RX8aj9Gj0YH/41kjxAI2zs+a0oGpRiUQTd3a/6KcAlTKW xFI65GT0xx+BwwRp9zS1aiRQQvzFfF4NsOL6FLk+5iloemllFe3Flu2BKMu0kp9rAIni G4BA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b=CRHYrtMA; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id d10si7775940edq.351.2021.06.05.11.08.54; Sat, 05 Jun 2021 11:08:54 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b=CRHYrtMA; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 1214C68073F; Sat, 5 Jun 2021 21:08:51 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pf1-f170.google.com (mail-pf1-f170.google.com [209.85.210.170]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 0FEDE68073F for ; Sat, 5 Jun 2021 21:08:45 +0300 (EEST) Received: by mail-pf1-f170.google.com with SMTP id d16so9841157pfn.12 for ; Sat, 05 Jun 2021 11:08:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=4gxJawNDd2iWOL99+S3dGdz3E8dlMirBOuo6hxiai7Q=; b=CRHYrtMAi2VuGeXng45OhMDT2TZoDpTTsSnW+0TmuBMaM+5GPsZymL0i7Ft9ks1+sx F5KLDKdogp8LImPxf+fCzrxr6F7iFM5OXD2ZmA2X8DjIZWwGygZ05arKBCgg3UFIL2Yy fm0fYnYVJxJuwkTUhIWygPb91w71qAp/Hm3vGaLf2WJNgnTk+lfIZc8qUEizpyaW17z+ djT62prEbs2zxlg1+YSdQMyCKb5/LzV9H9ZrMF8Cr7o3NqHQPsnn7dBJ9rVB0lelUCua R1Q53YFicEM53BSyui2Qxh2v0JqDoyi/IJGhVb3s8TeLx/a3/w1T6Nu1gijHTblEZ7wN pibw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=4gxJawNDd2iWOL99+S3dGdz3E8dlMirBOuo6hxiai7Q=; b=b5C+8jdYumAvXwhzJhucugD6w3ONBesEZzAClngdqTt9vlYftUloEw7JN2cbHcdbfP c8WWhgmPcZa5PthJ5jKkzB/VvLSDK2rm02q6o1mZsQNY0HdxfoePGOY0c8NaC7X2QDMp EcS1SV+CMQSVKn67NTMo/1ut+DkTbltPFQXUpHc7NwnSq3S/Y7bHjlSySd9rm4LxX6+W d6VmKe/e5zYHrV9zOGl6Eo9AtBdta9ft8oB/MpC12+mbqiaa/7Jduxf10vv5N/+h4ejS wos5MbQ7tEHojuhfSNLEQBmQbKkKI7+sJw26qCYEUdChKC782EaZ/yl9jCxItz3nNcSh yp4w== X-Gm-Message-State: AOAM533O9k45w85DV1a2EEjyXMOizqb/Oebhis/1V4bFqkU7oC2zdWtl TjgCqIYcCD5dCoXJbNZubMEFNC9B515IgQ== X-Received: by 2002:a63:471f:: with SMTP id u31mr10738851pga.85.1622916523140; Sat, 05 Jun 2021 11:08:43 -0700 (PDT) Received: from Pavilion-x360.bbrouter ([103.157.221.215]) by smtp.googlemail.com with ESMTPSA id w14sm7818176pjf.12.2021.06.05.11.08.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 05 Jun 2021 11:08:42 -0700 (PDT) From: Shubhanshu Saxena To: ffmpeg-devel@ffmpeg.org Date: Sat, 5 Jun 2021 23:38:03 +0530 Message-Id: <20210605180806.65831-2-shubhanshu.e01@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210605180806.65831-1-shubhanshu.e01@gmail.com> References: <20210605180806.65831-1-shubhanshu.e01@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH V2 2/5] lavfi/dnn: Convert output_name to char** in TaskItem X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Shubhanshu Saxena Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 4dZDB4nb4naj Convert output_name to char **output_names in TaskItem and use it as a pointer to array of output names in the DNN backend. Signed-off-by: Shubhanshu Saxena --- libavfilter/dnn/dnn_backend_common.h | 2 +- libavfilter/dnn/dnn_backend_openvino.c | 10 +++++----- 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/libavfilter/dnn/dnn_backend_common.h b/libavfilter/dnn/dnn_backend_common.h index 0c043e51f0..f76a05026d 100644 --- a/libavfilter/dnn/dnn_backend_common.h +++ b/libavfilter/dnn/dnn_backend_common.h @@ -32,7 +32,7 @@ typedef struct TaskItem { AVFrame *in_frame; AVFrame *out_frame; const char *input_name; - const char *output_name; + const char **output_names; int async; int do_ioproc; uint32_t inference_todo; diff --git a/libavfilter/dnn/dnn_backend_openvino.c b/libavfilter/dnn/dnn_backend_openvino.c index a84370d689..0f3b235820 100644 --- a/libavfilter/dnn/dnn_backend_openvino.c +++ b/libavfilter/dnn/dnn_backend_openvino.c @@ -208,7 +208,7 @@ static void infer_completion_callback(void *args) DNNData output; OVContext *ctx = &ov_model->ctx; - status = ie_infer_request_get_blob(request->infer_request, task->output_name, &output_blob); + status = ie_infer_request_get_blob(request->infer_request, task->output_names[0], &output_blob); if (status != OK) { //incorrect output name char *model_output_name = NULL; @@ -222,7 +222,7 @@ static void infer_completion_callback(void *args) } av_log(ctx, AV_LOG_ERROR, "output \"%s\" may not correct, all output(s) are: \"%s\"\n", - task->output_name, all_output_names); + task->output_names[0], all_output_names); return; } @@ -676,7 +676,7 @@ static DNNReturnType get_output_ov(void *model, const char *input_name, int inpu task.async = 0; task.input_name = input_name; task.in_frame = in_frame; - task.output_name = output_name; + task.output_names = &output_name; task.out_frame = out_frame; task.model = ov_model; @@ -796,7 +796,7 @@ DNNReturnType ff_dnn_execute_model_ov(const DNNModel *model, DNNExecBaseParams * task.async = 0; task.input_name = exec_params->input_name; task.in_frame = exec_params->in_frame; - task.output_name = exec_params->output_names[0]; + task.output_names = &exec_params->output_names[0]; task.out_frame = exec_params->out_frame ? exec_params->out_frame : exec_params->in_frame; task.model = ov_model; @@ -843,7 +843,7 @@ DNNReturnType ff_dnn_execute_model_async_ov(const DNNModel *model, DNNExecBasePa task->async = 1; task->input_name = exec_params->input_name; task->in_frame = exec_params->in_frame; - task->output_name = exec_params->output_names[0]; + task->output_names = &exec_params->output_names[0]; task->out_frame = exec_params->out_frame ? exec_params->out_frame : exec_params->in_frame; task->model = ov_model; if (ff_queue_push_back(ov_model->task_queue, task) < 0) { From patchwork Sat Jun 5 18:08:04 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shubhanshu Saxena X-Patchwork-Id: 28105 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a6b:b214:0:0:0:0:0 with SMTP id b20csp2157352iof; Sat, 5 Jun 2021 11:41:59 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwh4P3ADq9ZAv7RhTStDaosOfa1LHjvNeOFwpaJo6MJn4aXGzaiB0gAyE56av45ClZMRorC X-Received: by 2002:a17:906:6d43:: with SMTP id a3mr10381515ejt.142.1622918519244; Sat, 05 Jun 2021 11:41:59 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1622918519; cv=none; d=google.com; s=arc-20160816; b=NLJNdUD3Xb7CqmqnPD4dvl1SbeaC2E0ahJEzzoZKKp8PHO7TZyqLWFkGHjvxXg1F94 WTLI/PLM4P8z3yvm64UhnyZXwX8aO1CJJYubgt7CE9nH5lF5+4flCa+mofWrbG8TXKI9 AVTA6ay5r9Rn/pkDwVIi5/llB/oDCMkSIG2s894JxcnFb/4qSe9yS5gmt5oHzTU/Yv95 vvmawHAO8hmFuqiYgfugEQBpatQgdJjxqxjAIf9Y1fO0FMbmVlUOEkeCDMFI9uipqqDF oirS6h9z2EMjY6QIU4PosZ4bQZHKFkxezQsAVYSXmFyao+JFpN/RPVDfUD0xxtzfXlNV jDNw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=emymKabnFv5Ls4izyJf7J1g0ntecFJkic0zFGt1/Iy0=; b=gL+XUwKTXwBZ9vB4vqKGTzp9Q6sb8dWBQmaIAOLEmHGSIF7G2DzwXLIJn+pkI3OfjE SKrG898ook704U5LFcUydvmacXZ31Ch5BQYsQlUQfL68WWp//b89EskPxFCwvE7aFH9p Ykl5JgeIXaD+Z1d7oxWTb88YypHZs6o5WuvnVIErQCQp0yht5wvVCd2IA04kPmEg9FTW fxmv1KR83kpcE0yLr7664se1IvJDzEAg16sY91/tbRGXhQ1ZvQNCKcBIjBhlESU3XTpP zTnIzuxDlE6jUcAu6tzXXQfteK09QFRAwg0j2DlkHR1dsBrMioGDWD/fcU1IPBF/HRZ3 kSjA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b=qbAbgD3W; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id c10si9059426edn.209.2021.06.05.11.41.58; Sat, 05 Jun 2021 11:41:59 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b=qbAbgD3W; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 6EB3168A636; Sat, 5 Jun 2021 21:41:56 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pf1-f170.google.com (mail-pf1-f170.google.com [209.85.210.170]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id ED86568A3AF for ; Sat, 5 Jun 2021 21:41:49 +0300 (EEST) Received: by mail-pf1-f170.google.com with SMTP id h12so7080974pfe.2 for ; Sat, 05 Jun 2021 11:41:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=mFZCaPaO/WEp3G0w6D9geo+ojGtEwjtWaW8n6GNXQj0=; b=qbAbgD3WMJVWlU20cjM25+cXB5WRT9EitvlDov/oHtrgP+e9/Ur8Fsf1PMznFqHwZk 7Jv0nP5V/dEvHiPgIipGaiDpG1ly+N3FP7Kx8m1xWBai4xlG2XM94QSPBLNJYgVHdAq2 OATqFsi9b8JOYkeEwNvd582hz0nmk0k7EeWboD2Pf3vot6FEvY8l1qxWnlfQfYKIZG1L 6+HyKzRK/bNCLaHAcw8JkwcWD77vsdFt08t7SJVFrdLKaww9X83T6Xqi5yk7mu9xEX8q NnCVBODmiO7j1qhup0V7aQyPTyl1yjR9Lf8LWY/i/fvSp2PmtzRQZH07RgB4SnR6BIO8 itzg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=mFZCaPaO/WEp3G0w6D9geo+ojGtEwjtWaW8n6GNXQj0=; b=eUytHWH7cd3ajhZ8ZYOX6iiwe7ukEQMbdQ90Jx62aI312x/YkNJCPN8rgn+XpUYW5f PUj16d+ChxIY4DELZSiZdP1gR8o0kyHBHSub9yOLWp18uqMBSdcYGLo/rM561a//uHCy 5RH32qEjnLGe1t4rvlJM49dsX/GMfEdH3HEHLbukHuZ0yWt4RQX3a6ndEZo18UpZ/AJH ggnAjFtuRi5JP33josOm+HAAm45ncHleCRqHcqlDYzLhT6zWY+PAxBhQnA38CwddV7Dl Z/ShY5CFznuLBk07+ejg4wSzSQkSIA2w6PklDpGdoTRlsTFdfmd8mDfbsur1MH/SqLzd LauA== X-Gm-Message-State: AOAM533O2QdzZhhhVkz7pnsisiGam0w+XgpekIr0jDWBL93DQY+onsCz zIpoxwMiuKcBABNfxKyAY6kULIMU3wr0eA== X-Received: by 2002:a63:4145:: with SMTP id o66mr10842578pga.4.1622916524885; Sat, 05 Jun 2021 11:08:44 -0700 (PDT) Received: from Pavilion-x360.bbrouter ([103.157.221.215]) by smtp.googlemail.com with ESMTPSA id w14sm7818176pjf.12.2021.06.05.11.08.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 05 Jun 2021 11:08:44 -0700 (PDT) From: Shubhanshu Saxena To: ffmpeg-devel@ffmpeg.org Date: Sat, 5 Jun 2021 23:38:04 +0530 Message-Id: <20210605180806.65831-3-shubhanshu.e01@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210605180806.65831-1-shubhanshu.e01@gmail.com> References: <20210605180806.65831-1-shubhanshu.e01@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH V2 3/5] lavfi/dnn: Add nb_output to TaskItem X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Shubhanshu Saxena Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: WgJ9V+A1wyQl Add nb_output property to TaskItem for use in TensorFlow backend and Native backend. Signed-off-by: Shubhanshu Saxena --- libavfilter/dnn/dnn_backend_common.h | 1 + libavfilter/dnn/dnn_backend_openvino.c | 3 +++ 2 files changed, 4 insertions(+) diff --git a/libavfilter/dnn/dnn_backend_common.h b/libavfilter/dnn/dnn_backend_common.h index f76a05026d..704cf921f1 100644 --- a/libavfilter/dnn/dnn_backend_common.h +++ b/libavfilter/dnn/dnn_backend_common.h @@ -35,6 +35,7 @@ typedef struct TaskItem { const char **output_names; int async; int do_ioproc; + uint32_t nb_output; uint32_t inference_todo; uint32_t inference_done; } TaskItem; diff --git a/libavfilter/dnn/dnn_backend_openvino.c b/libavfilter/dnn/dnn_backend_openvino.c index 0f3b235820..c2487c35be 100644 --- a/libavfilter/dnn/dnn_backend_openvino.c +++ b/libavfilter/dnn/dnn_backend_openvino.c @@ -678,6 +678,7 @@ static DNNReturnType get_output_ov(void *model, const char *input_name, int inpu task.in_frame = in_frame; task.output_names = &output_name; task.out_frame = out_frame; + task.nb_output = 1; task.model = ov_model; if (extract_inference_from_task(ov_model->model->func_type, &task, ov_model->inference_queue, NULL) != DNN_SUCCESS) { @@ -798,6 +799,7 @@ DNNReturnType ff_dnn_execute_model_ov(const DNNModel *model, DNNExecBaseParams * task.in_frame = exec_params->in_frame; task.output_names = &exec_params->output_names[0]; task.out_frame = exec_params->out_frame ? exec_params->out_frame : exec_params->in_frame; + task.nb_output = exec_params->nb_output; task.model = ov_model; if (extract_inference_from_task(ov_model->model->func_type, &task, ov_model->inference_queue, exec_params) != DNN_SUCCESS) { @@ -845,6 +847,7 @@ DNNReturnType ff_dnn_execute_model_async_ov(const DNNModel *model, DNNExecBasePa task->in_frame = exec_params->in_frame; task->output_names = &exec_params->output_names[0]; task->out_frame = exec_params->out_frame ? exec_params->out_frame : exec_params->in_frame; + task->nb_output = exec_params->nb_output; task->model = ov_model; if (ff_queue_push_back(ov_model->task_queue, task) < 0) { av_freep(&task); From patchwork Sat Jun 5 18:08:05 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shubhanshu Saxena X-Patchwork-Id: 28099 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a6b:b214:0:0:0:0:0 with SMTP id b20csp2139862iof; Sat, 5 Jun 2021 11:09:03 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxwAzKj+cD5+cxCCwOfazvLHpPhvYeaIxI8v/BkceBb/4nZ1JOMYT1pK41cjue1uYYNTH+m X-Received: by 2002:a17:906:bcf9:: with SMTP id op25mr10450117ejb.453.1622916543683; Sat, 05 Jun 2021 11:09:03 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1622916543; cv=none; d=google.com; s=arc-20160816; b=d7qQ0fCsmczppHFfSHpVs8rOT2VihmMZxHFBRNswan2tee9+hnnCkdYLoLOl08F+ng XK/TCX/DTy6qy7d3OksBv6OQShfWsuw6D7Hz6GckGA/++/8nn5sa9AgrTKeOkrUio8H4 59x7zZ1y8/Dz0/OaqkIjlR0cQg/Y+O7xRMn7UoaANcLvmbKkR4JuhHlM8Tu1B1APH5Wm 5hnUJvLHsA2Hq3CusX/s/jdmO932zOwwG0T1JEjWdPrPykvS9B76TFwKrFiBqPMPfpJF ebS47fevKu/3o2ROytnCAqvofOc0+ykuI7Cso3983Y6xyEJHgZEB6NB7arOQxGcb6yJ+ LigQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=TxcIZCuAqi/BslpgzEwZRB9LDj4xcaJSc2mP4bD8kvI=; b=f7XUTXjICA90SY/TX9BEkvCYlE/l0woifI8csDdKbN4YeJZgV7krjJqqYjq8FOFUaa rlB71w9uSE6lbaKrdvN4Qg9W2V/0uWFi8g3KhG8sor5E29MW1g/Hh5NteLiNTut+G2pr gEFm037crdZtkzceEtEtkj/Opj+p9yBqc3crgmqW6MJrAQDRsT49vcWz1Qd8lp0BXtWu TDSUMbqTIv0d7yNiIellbYmLdhGra9yLa6Pa5/PR6ecPUzoWTxWU7mwsResZd7/xc21l 6Q87XOBc1m6bawf7yIJ2K6mhTthYTYBrouAgewwCKiQPNIRNV4BKyBLs+6nDm5TJkCgL o8Qw== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b=oGTBetqB; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id jz3si8031679ejb.9.2021.06.05.11.09.03; Sat, 05 Jun 2021 11:09:03 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b=oGTBetqB; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 1DD5D68A5F0; Sat, 5 Jun 2021 21:08:55 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pl1-f173.google.com (mail-pl1-f173.google.com [209.85.214.173]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 87ECC68A5D1 for ; Sat, 5 Jun 2021 21:08:48 +0300 (EEST) Received: by mail-pl1-f173.google.com with SMTP id u9so6247086plr.1 for ; Sat, 05 Jun 2021 11:08:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=MVayHqylz7jNiZJI9/HCLY9SxiXlUMBUuBz4k4NNPG8=; b=oGTBetqBPnbbtaubKMFGaCCVYtkv6HaW5sXwNRu4AtlStvgMnYxYKcxmQum6yGCo5c PQE2PTXdOXWMz5B+8hMuCKCDIe8rn/e0k8T2eYLMOdZdry7HK/y3pVwkXP8Eu0AqP93X jo2+MthpH8Zg/WVvA+Gzle1zhW9AopR2Q5ePh34O0Lb8rhy7mklbQXnlbXnXqmYrVpNF v4G1ya9UVcXAF3xvDLOv7iMF7o8laOqrFBOi2z048xPHjV0CtAcaDP5TApI7YlwZqgbY yJ0IieYpx7h9splVcQSZiQmWTUYKnByp/6RFJB40Gvp5AhGv5wh9V0OEPU0ABk2Dqik0 QPTA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=MVayHqylz7jNiZJI9/HCLY9SxiXlUMBUuBz4k4NNPG8=; b=Qf5rj+BWRMwQu3HJL162YPe3W+4ZKNhobn6+gJPST6/i4cwa/OzGNB/8QFzLuIrE7B WcW4pFV+hFEhVb4MartTE0a4/gSjf1Dja1vyiZUG8cFC7lkX4qnZnGdSIEV0A8inMCWm UR122ymqlsRZ5fQhbqtL1FMD9VdL8F5D/7OU+HsWMWTNNKBErCSBAcXjo934ABns4LVz fsCgW2lp4wgzOsfBsV9zE/TPiwInKzE+eHs2x5CoOzFKfN3UkTUYbaG667Q9I8QdzD76 5nWSMhpjL1mn2JdijKZrfJckxj7PdF0Y/qzP4/UqW4yOdaJIlZ1qQsoLYcrYDDJjTh3C Ul/A== X-Gm-Message-State: AOAM530Xxwg+xuAXb6dRnexsstk13Wufk5G/gcTzM4QOLItMBZoW7OvB pH+jB36uloHwXa9a8ysFJt2vT9/cQgHi5w== X-Received: by 2002:a17:90b:318:: with SMTP id ay24mr22422049pjb.150.1622916526676; Sat, 05 Jun 2021 11:08:46 -0700 (PDT) Received: from Pavilion-x360.bbrouter ([103.157.221.215]) by smtp.googlemail.com with ESMTPSA id w14sm7818176pjf.12.2021.06.05.11.08.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 05 Jun 2021 11:08:46 -0700 (PDT) From: Shubhanshu Saxena To: ffmpeg-devel@ffmpeg.org Date: Sat, 5 Jun 2021 23:38:05 +0530 Message-Id: <20210605180806.65831-4-shubhanshu.e01@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210605180806.65831-1-shubhanshu.e01@gmail.com> References: <20210605180806.65831-1-shubhanshu.e01@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH V2 4/5] lavfi/dnn: Use uint8_t for async and do_ioproc in TaskItems X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Shubhanshu Saxena Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: BxqA0xVAnwbj These properties have values either 0 or 1, so using uint8_t is a better option as compared to int. Signed-off-by: Shubhanshu Saxena --- libavfilter/dnn/dnn_backend_common.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/libavfilter/dnn/dnn_backend_common.h b/libavfilter/dnn/dnn_backend_common.h index 704cf921f1..d962312c16 100644 --- a/libavfilter/dnn/dnn_backend_common.h +++ b/libavfilter/dnn/dnn_backend_common.h @@ -33,8 +33,8 @@ typedef struct TaskItem { AVFrame *out_frame; const char *input_name; const char **output_names; - int async; - int do_ioproc; + uint8_t async; + uint8_t do_ioproc; uint32_t nb_output; uint32_t inference_todo; uint32_t inference_done; From patchwork Sat Jun 5 18:08:06 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shubhanshu Saxena X-Patchwork-Id: 28101 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a6b:b214:0:0:0:0:0 with SMTP id b20csp2139965iof; Sat, 5 Jun 2021 11:09:11 -0700 (PDT) X-Google-Smtp-Source: ABdhPJylE797AOrTIIXtL2U9k2LbeeZQDArJ/dtmqXP1cmeOi1tjRizgj9SIs5yLt8VyFRSOwsm7 X-Received: by 2002:a17:907:9721:: with SMTP id jg33mr507525ejc.64.1622916551681; Sat, 05 Jun 2021 11:09:11 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1622916551; cv=none; d=google.com; s=arc-20160816; b=f55RATGd8yFokhjBRHlNOIxVMVy2adfuLgya+Gzys/1M4d1PPUZ4h/mths/IC/VioK 3jPuBeALA670HdBqR/jOBxgx6zOiMebMOLvjsPz2ZP3TsgXlWiaJIAA4ceaPFIFgJocL JXj6Q3U5oUaFmEZhWEhfDnP57M28CHyU4L8a6Ki842j3TJvCwdt1Gasw8SwFradKlKIn B/Kh5VXrBzkkeLvG5bk+TGe3X8SC0+5CT/Tc+yHmYzjgy74MBUcLYgW6Po6aH8aG2REA dDz825rrohRrl0AD/0voAklZmKte1/7qRnVmLPqMsJLhynQet3s/2UDaU9sWoC5rtmKK RW0w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=enkXEmoLATIGnVyFqWx1KfSuHunLl0HGnAhTwRKtEhI=; b=d4bIPCx/22WlX3KSspvestJZSEuMsrLrZTMDcot1SxLu3xQlzbLXWtHlufYIqMKAoK vW3QQrYluqMrSFPv6PuR89qKdmZE/rt2TzM51FjHlUOIqrQQKptsjZzMLdKREsKBAfIt E7vTOj8T3bXUvOkGuu2uu8WFiVvbxTsn4Y5XOhxaRtwdC5pI19spBCimgTtYoX4tv0r5 ndM11Mvn09hw3TlAJR1WgQWBXEbdp5ojibBx0j2z3whsiZW1+VYcbK7tk7zLije6uC8u giq1fCO7iGqbewYSfZs6BhPiO0VJStk6az2D36v8sE9jKmQPJCtJGBA5nu/+T1eDm9uz jkwQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b="WcLiAx//"; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id t12si9427156edc.333.2021.06.05.11.09.11; Sat, 05 Jun 2021 11:09:11 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b="WcLiAx//"; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 3027068A601; Sat, 5 Jun 2021 21:08:56 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pf1-f170.google.com (mail-pf1-f170.google.com [209.85.210.170]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 1C8FC68A5EC for ; Sat, 5 Jun 2021 21:08:50 +0300 (EEST) Received: by mail-pf1-f170.google.com with SMTP id h12so7040846pfe.2 for ; Sat, 05 Jun 2021 11:08:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=NiWqJtj+P+9dijQdiV2TF8bwccUw6X6PrwK9rCOzSHQ=; b=WcLiAx//o/VmWtaCn9ocwbazTgg+T/YcfVfSX+RA46LMPFXIAYJcrt/0Ul7jrQ5NnU /lcMucHfgY1C4iPaAY8bZDvlUcaf14rRZ69fKVnA3T8myRs00eRerI0oGmCskXh4i3eS O7mszDQHAppLLja0rmIhxZyuULMznVE+D5abwHX8ttSPpVD+2XGRNxpu81dSq5+QGGj7 VZrb7ehL2Ybnj3GVSF2KkXE4zWFQrjSXnD8HoGju96VChzlP7kiKuFlISaxx5iJRR2O8 SRvAVZ35gtREHhVxGu0MCnu0fU1/wnvij5JpwIRwJ3GRS+xi6/mlfmm9lS3R+JQevAMr 74iA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=NiWqJtj+P+9dijQdiV2TF8bwccUw6X6PrwK9rCOzSHQ=; b=sKky8xJQCN6jB+hDLycyVEf8ghyXrHTkdTCjZrpz6Sr9Nm9LjQY+Tm6wKE7ZZGKlmN ebJ+Lj5LEjVt4mWlj8tSvRLAdF0xjZ/yE0c90vr+9Wr/xVAQ8P1OXYpnPIib97HOlKw/ m4cMqpLnksCu+iNplzH6ynUlaV+YdOsxZG/+tB4RzvWxBStRLq3GJRrVbOI3EOA3BtVV Hwkv0hwAgyZh7T1lx/F5IN00L38gQINRZTQ2htIy4kS+C/dJzW3k3ueg3X6Nh6vHLPYJ PVLE9GvPK8YUQthxoC6ABsvCdYdx29QIg0dvuGTihP/SdA/rAfY7jjOcDKBbUiIVNVAI vNxA== X-Gm-Message-State: AOAM533nShQW9z9VrnvutacqPMRQrspffHs0mRpXmapDlzXjmUnqDfmm p44d0+pTPHtQm4K1p+bng0vkpSfEZPveDg== X-Received: by 2002:a62:87c9:0:b029:2ea:572c:e4b1 with SMTP id i192-20020a6287c90000b02902ea572ce4b1mr10320923pfe.34.1622916528390; Sat, 05 Jun 2021 11:08:48 -0700 (PDT) Received: from Pavilion-x360.bbrouter ([103.157.221.215]) by smtp.googlemail.com with ESMTPSA id w14sm7818176pjf.12.2021.06.05.11.08.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 05 Jun 2021 11:08:48 -0700 (PDT) From: Shubhanshu Saxena To: ffmpeg-devel@ffmpeg.org Date: Sat, 5 Jun 2021 23:38:06 +0530 Message-Id: <20210605180806.65831-5-shubhanshu.e01@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210605180806.65831-1-shubhanshu.e01@gmail.com> References: <20210605180806.65831-1-shubhanshu.e01@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH V2 5/5] lavfi/dnn: Fill Task using Common Function X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Shubhanshu Saxena Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: sy5UcST6usRy This commit adds a common function for filling the TaskItems in all three backends. Signed-off-by: Shubhanshu Saxena --- libavfilter/dnn/dnn_backend_common.c | 20 ++++++++++++++++++++ libavfilter/dnn/dnn_backend_common.h | 15 +++++++++++++++ libavfilter/dnn/dnn_backend_openvino.c | 23 +++++++---------------- 3 files changed, 42 insertions(+), 16 deletions(-) diff --git a/libavfilter/dnn/dnn_backend_common.c b/libavfilter/dnn/dnn_backend_common.c index a522ab5650..4d9d3f79b1 100644 --- a/libavfilter/dnn/dnn_backend_common.c +++ b/libavfilter/dnn/dnn_backend_common.c @@ -49,3 +49,23 @@ int ff_check_exec_params(void *ctx, DNNBackendType backend, DNNFunctionType func return 0; } + +DNNReturnType ff_dnn_fill_task(TaskItem *task, DNNExecBaseParams *exec_params, void *backend_model, int async, int do_ioproc) { + if (task == NULL || exec_params == NULL || backend_model == NULL) + return DNN_ERROR; + if (do_ioproc != 0 && do_ioproc != 1) + return DNN_ERROR; + if (async != 0 && async != 1) + return DNN_ERROR; + + task->do_ioproc = do_ioproc; + task->async = async; + task->input_name = exec_params->input_name; + task->in_frame = exec_params->in_frame; + task->out_frame = exec_params->out_frame; + task->model = backend_model; + task->nb_output = exec_params->nb_output; + task->output_names = exec_params->output_names; + + return DNN_SUCCESS; +} diff --git a/libavfilter/dnn/dnn_backend_common.h b/libavfilter/dnn/dnn_backend_common.h index d962312c16..df59615f40 100644 --- a/libavfilter/dnn/dnn_backend_common.h +++ b/libavfilter/dnn/dnn_backend_common.h @@ -48,4 +48,19 @@ typedef struct InferenceItem { int ff_check_exec_params(void *ctx, DNNBackendType backend, DNNFunctionType func_type, DNNExecBaseParams *exec_params); +/** + * Fill the Task for Backend Execution. It should be called after + * checking execution parameters using ff_check_exec_params. + * + * @param task pointer to the allocated task + * @param exec_param pointer to execution parameters + * @param backend_model void pointer to the backend model + * @param async flag for async execution. Must be 0 or 1 + * @param do_ioproc flag for IO processing. Must be 0 or 1 + * + * @retval DNN_SUCCESS if successful + * @retval DNN_ERROR if flags are invalid or any parameter is NULL + */ +DNNReturnType ff_dnn_fill_task(TaskItem *task, DNNExecBaseParams *exec_params, void *backend_model, int async, int do_ioproc); + #endif diff --git a/libavfilter/dnn/dnn_backend_openvino.c b/libavfilter/dnn/dnn_backend_openvino.c index c2487c35be..709a772a4d 100644 --- a/libavfilter/dnn/dnn_backend_openvino.c +++ b/libavfilter/dnn/dnn_backend_openvino.c @@ -793,14 +793,9 @@ DNNReturnType ff_dnn_execute_model_ov(const DNNModel *model, DNNExecBaseParams * } } - task.do_ioproc = 1; - task.async = 0; - task.input_name = exec_params->input_name; - task.in_frame = exec_params->in_frame; - task.output_names = &exec_params->output_names[0]; - task.out_frame = exec_params->out_frame ? exec_params->out_frame : exec_params->in_frame; - task.nb_output = exec_params->nb_output; - task.model = ov_model; + if (ff_dnn_fill_task(&task, exec_params, ov_model, 0, 1) != DNN_SUCCESS) { + return DNN_ERROR; + } if (extract_inference_from_task(ov_model->model->func_type, &task, ov_model->inference_queue, exec_params) != DNN_SUCCESS) { av_log(ctx, AV_LOG_ERROR, "unable to extract inference from task.\n"); @@ -841,14 +836,10 @@ DNNReturnType ff_dnn_execute_model_async_ov(const DNNModel *model, DNNExecBasePa return DNN_ERROR; } - task->do_ioproc = 1; - task->async = 1; - task->input_name = exec_params->input_name; - task->in_frame = exec_params->in_frame; - task->output_names = &exec_params->output_names[0]; - task->out_frame = exec_params->out_frame ? exec_params->out_frame : exec_params->in_frame; - task->nb_output = exec_params->nb_output; - task->model = ov_model; + if (ff_dnn_fill_task(task, exec_params, ov_model, 1, 1) != DNN_SUCCESS) { + return DNN_ERROR; + } + if (ff_queue_push_back(ov_model->task_queue, task) < 0) { av_freep(&task); av_log(ctx, AV_LOG_ERROR, "unable to push back task_queue.\n");