From patchwork Thu Apr 29 13:36:55 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Guo, Yejun" X-Patchwork-Id: 27480 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a11:4023:0:0:0:0 with SMTP id ky35csp1492953pxb; Thu, 29 Apr 2021 06:49:56 -0700 (PDT) X-Google-Smtp-Source: ABdhPJx80ESDI17MK1OF9EnXppvPsivjmvXG1kvfUXPF5EVzFeho9y6Xz8/gtZ7m7EqQwzuX8I1w X-Received: by 2002:a17:906:c7d1:: with SMTP id dc17mr3566231ejb.111.1619704195848; Thu, 29 Apr 2021 06:49:55 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1619704195; cv=none; d=google.com; s=arc-20160816; b=e4o+X/Grlj6Fa8GxjEIFGeMyDxKzu7PUo5GAohNwMQTfqWEwozQI82/ibra6zDqipF fIu42vBJIOgMJJVp+bET1Rerbwmpxmu6XNOcGQ0weSsLCThNZKAGbVW2KOd9qFcss4dT 06irNSbo6snuVTK0Xtpkg8tkqH5ewSw/prbGWWvNqJxKjqkL/+ZxjzdjPSc6POiFh2pI 1nkfY1UBWXK+/0ALgyy9jKMrovST4K5PSQUfkmWhYoSu8QLq9//yJgXnX7tP38ccp042 SUhX478jLhnF8P0r/estMIf/A/ycT76nAkxQPQl3e9QK2yO7WScU6cN8k8QXX+B2EFyB T46g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:ironport-sdr:ironport-sdr:delivered-to; bh=2ZIlAhlmPNPQcy32QVhq/8hQV0VnxHly5TdtoHHHQsQ=; b=Q8BtIZXEdOI16skOkXDks8ZA78+zizDFOUOwrWIkLT2gCj8xHMIPetSdXSblqJAmaj dyfQmQ3tpQ6n3qS0HQD8tqhVUSg8zL5cXDNT4Pz7rU2hfMgkmNFJIr/P2pc0uEr8oiFA KX6W+tgOMaqtNObpLVoL6hyxc0Lo+24pmUIfO1BaAL2Doumhi8eWXnYF1swaw53WEf3X g8FT7hEI/OPXc1FRAp2HOCb5TfROAAIYWfSJgTKLL1cp+Qv42I1RAeqEPs4zXqPEHUeK jcxNp9opjtLca3vQ01rdzJQWXvOiALn7vAEnYDO1O7FrHwd3cznsPCdwm/377oIokY2Q NMJg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id c9si3112700edv.186.2021.04.29.06.49.55; Thu, 29 Apr 2021 06:49:55 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 4870568A11C; Thu, 29 Apr 2021 16:49:28 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 43D6D68A0CD for ; Thu, 29 Apr 2021 16:49:20 +0300 (EEST) IronPort-SDR: 6Qc7dcFajGRb/Cscs/DEXLa9+MuP8B59lpBM1ELF9Y7ZS3eSQbzSAbFW5IUOo/iOhhSLK7PUXD wU0WFbhYG2OA== X-IronPort-AV: E=McAfee;i="6200,9189,9969"; a="260956499" X-IronPort-AV: E=Sophos;i="5.82,259,1613462400"; d="scan'208";a="260956499" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Apr 2021 06:49:11 -0700 IronPort-SDR: WTdA76CCmm3unrFQ/eG+J+WltmIPpuRprbMNS8my3f1ZjtbGUYun/DOZDDtW+bU+PVeQVLmQwQ SDTeammHuX6Q== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.82,259,1613462400"; d="scan'208";a="424096061" Received: from yguo18-skl-u1604.sh.intel.com ([10.239.159.53]) by fmsmga008.fm.intel.com with ESMTP; 29 Apr 2021 06:49:10 -0700 From: "Guo, Yejun" To: ffmpeg-devel@ffmpeg.org Date: Thu, 29 Apr 2021 21:36:55 +0800 Message-Id: <20210429133657.23076-4-yejun.guo@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210429133657.23076-1-yejun.guo@intel.com> References: <20210429133657.23076-1-yejun.guo@intel.com> Subject: [FFmpeg-devel] [PATCH V2 4/6] lavfi/dnn: refine dnn interface to add DNNExecBaseParams X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: yejun.guo@intel.com MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: nDXtQVvjJqpE Different function type of model requires different parameters, for example, object detection detects lots of objects (cat/dog/...) in the frame, and classifcation needs to know which object (cat or dog) it is going to classify. The current interface needs to add a new function with more parameters to support new requirement, with this change, we can just add a new struct (for example DNNExecClassifyParams) based on DNNExecBaseParams, and so we can continue to use the current interface execute_model just with params changed. --- libavfilter/dnn/Makefile | 1 + libavfilter/dnn/dnn_backend_common.c | 51 ++++++++++++++++++++++++++ libavfilter/dnn/dnn_backend_common.h | 31 ++++++++++++++++ libavfilter/dnn/dnn_backend_native.c | 15 +++----- libavfilter/dnn/dnn_backend_native.h | 3 +- libavfilter/dnn/dnn_backend_openvino.c | 50 ++++++++----------------- libavfilter/dnn/dnn_backend_openvino.h | 6 +-- libavfilter/dnn/dnn_backend_tf.c | 18 +++------ libavfilter/dnn/dnn_backend_tf.h | 3 +- libavfilter/dnn_filter_common.c | 20 ++++++++-- libavfilter/dnn_interface.h | 14 +++++-- 11 files changed, 139 insertions(+), 73 deletions(-) create mode 100644 libavfilter/dnn/dnn_backend_common.c create mode 100644 libavfilter/dnn/dnn_backend_common.h diff --git a/libavfilter/dnn/Makefile b/libavfilter/dnn/Makefile index d6d58f4b61..4cfbce0efc 100644 --- a/libavfilter/dnn/Makefile +++ b/libavfilter/dnn/Makefile @@ -2,6 +2,7 @@ OBJS-$(CONFIG_DNN) += dnn/dnn_interface.o OBJS-$(CONFIG_DNN) += dnn/dnn_io_proc.o OBJS-$(CONFIG_DNN) += dnn/queue.o OBJS-$(CONFIG_DNN) += dnn/safe_queue.o +OBJS-$(CONFIG_DNN) += dnn/dnn_backend_common.o OBJS-$(CONFIG_DNN) += dnn/dnn_backend_native.o OBJS-$(CONFIG_DNN) += dnn/dnn_backend_native_layers.o OBJS-$(CONFIG_DNN) += dnn/dnn_backend_native_layer_avgpool.o diff --git a/libavfilter/dnn/dnn_backend_common.c b/libavfilter/dnn/dnn_backend_common.c new file mode 100644 index 0000000000..a522ab5650 --- /dev/null +++ b/libavfilter/dnn/dnn_backend_common.c @@ -0,0 +1,51 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * DNN common functions different backends. + */ + +#include "dnn_backend_common.h" + +int ff_check_exec_params(void *ctx, DNNBackendType backend, DNNFunctionType func_type, DNNExecBaseParams *exec_params) +{ + if (!exec_params) { + av_log(ctx, AV_LOG_ERROR, "exec_params is null when execute model.\n"); + return AVERROR(EINVAL); + } + + if (!exec_params->in_frame) { + av_log(ctx, AV_LOG_ERROR, "in frame is NULL when execute model.\n"); + return AVERROR(EINVAL); + } + + if (!exec_params->out_frame) { + av_log(ctx, AV_LOG_ERROR, "out frame is NULL when execute model.\n"); + return AVERROR(EINVAL); + } + + if (exec_params->nb_output != 1 && backend != DNN_TF) { + // currently, the filter does not need multiple outputs, + // so we just pending the support until we really need it. + avpriv_report_missing_feature(ctx, "multiple outputs"); + return AVERROR(EINVAL); + } + + return 0; +} diff --git a/libavfilter/dnn/dnn_backend_common.h b/libavfilter/dnn/dnn_backend_common.h new file mode 100644 index 0000000000..cd9c0f5339 --- /dev/null +++ b/libavfilter/dnn/dnn_backend_common.h @@ -0,0 +1,31 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * DNN common functions different backends. + */ + +#ifndef AVFILTER_DNN_DNN_BACKEND_COMMON_H +#define AVFILTER_DNN_DNN_BACKEND_COMMON_H + +#include "../dnn_interface.h" + +int ff_check_exec_params(void *ctx, DNNBackendType backend, DNNFunctionType func_type, DNNExecBaseParams *exec_params); + +#endif diff --git a/libavfilter/dnn/dnn_backend_native.c b/libavfilter/dnn/dnn_backend_native.c index d9762eeaf6..b5f1c16538 100644 --- a/libavfilter/dnn/dnn_backend_native.c +++ b/libavfilter/dnn/dnn_backend_native.c @@ -28,6 +28,7 @@ #include "dnn_backend_native_layer_conv2d.h" #include "dnn_backend_native_layers.h" #include "dnn_io_proc.h" +#include "dnn_backend_common.h" #define OFFSET(x) offsetof(NativeContext, x) #define FLAGS AV_OPT_FLAG_FILTERING_PARAM @@ -372,23 +373,17 @@ static DNNReturnType execute_model_native(const DNNModel *model, const char *inp return DNN_SUCCESS; } -DNNReturnType ff_dnn_execute_model_native(const DNNModel *model, const char *input_name, AVFrame *in_frame, - const char **output_names, uint32_t nb_output, AVFrame *out_frame) +DNNReturnType ff_dnn_execute_model_native(const DNNModel *model, DNNExecBaseParams *exec_params) { NativeModel *native_model = model->model; NativeContext *ctx = &native_model->ctx; - if (!in_frame) { - av_log(ctx, AV_LOG_ERROR, "in frame is NULL when execute model.\n"); - return DNN_ERROR; - } - - if (!out_frame) { - av_log(ctx, AV_LOG_ERROR, "out frame is NULL when execute model.\n"); + if (ff_check_exec_params(ctx, DNN_NATIVE, model->func_type, exec_params) != 0) { return DNN_ERROR; } - return execute_model_native(model, input_name, in_frame, output_names, nb_output, out_frame, 1); + return execute_model_native(model, exec_params->input_name, exec_params->in_frame, + exec_params->output_names, exec_params->nb_output, exec_params->out_frame, 1); } int32_t ff_calculate_operand_dims_count(const DnnOperand *oprd) diff --git a/libavfilter/dnn/dnn_backend_native.h b/libavfilter/dnn/dnn_backend_native.h index d313c48f3a..89bcb8e358 100644 --- a/libavfilter/dnn/dnn_backend_native.h +++ b/libavfilter/dnn/dnn_backend_native.h @@ -130,8 +130,7 @@ typedef struct NativeModel{ DNNModel *ff_dnn_load_model_native(const char *model_filename, DNNFunctionType func_type, const char *options, AVFilterContext *filter_ctx); -DNNReturnType ff_dnn_execute_model_native(const DNNModel *model, const char *input_name, AVFrame *in_frame, - const char **output_names, uint32_t nb_output, AVFrame *out_frame); +DNNReturnType ff_dnn_execute_model_native(const DNNModel *model, DNNExecBaseParams *exec_params); void ff_dnn_free_model_native(DNNModel **model); diff --git a/libavfilter/dnn/dnn_backend_openvino.c b/libavfilter/dnn/dnn_backend_openvino.c index 9f3c696e0a..4e58ff6d9c 100644 --- a/libavfilter/dnn/dnn_backend_openvino.c +++ b/libavfilter/dnn/dnn_backend_openvino.c @@ -33,6 +33,7 @@ #include "queue.h" #include "safe_queue.h" #include +#include "dnn_backend_common.h" typedef struct OVOptions{ char *device_type; @@ -678,28 +679,14 @@ err: return NULL; } -DNNReturnType ff_dnn_execute_model_ov(const DNNModel *model, const char *input_name, AVFrame *in_frame, - const char **output_names, uint32_t nb_output, AVFrame *out_frame) +DNNReturnType ff_dnn_execute_model_ov(const DNNModel *model, DNNExecBaseParams *exec_params) { OVModel *ov_model = model->model; OVContext *ctx = &ov_model->ctx; TaskItem task; RequestItem *request; - if (!in_frame) { - av_log(ctx, AV_LOG_ERROR, "in frame is NULL when execute model.\n"); - return DNN_ERROR; - } - - if (!out_frame && model->func_type == DFT_PROCESS_FRAME) { - av_log(ctx, AV_LOG_ERROR, "out frame is NULL when execute model.\n"); - return DNN_ERROR; - } - - if (nb_output != 1) { - // currently, the filter does not need multiple outputs, - // so we just pending the support until we really need it. - avpriv_report_missing_feature(ctx, "multiple outputs"); + if (ff_check_exec_params(ctx, DNN_OV, model->func_type, exec_params) != 0) { return DNN_ERROR; } @@ -709,7 +696,7 @@ DNNReturnType ff_dnn_execute_model_ov(const DNNModel *model, const char *input_n } if (!ov_model->exe_network) { - if (init_model_ov(ov_model, input_name, output_names[0]) != DNN_SUCCESS) { + if (init_model_ov(ov_model, exec_params->input_name, exec_params->output_names[0]) != DNN_SUCCESS) { av_log(ctx, AV_LOG_ERROR, "Failed init OpenVINO exectuable network or inference request\n"); return DNN_ERROR; } @@ -717,10 +704,10 @@ DNNReturnType ff_dnn_execute_model_ov(const DNNModel *model, const char *input_n task.do_ioproc = 1; task.async = 0; - task.input_name = input_name; - task.in_frame = in_frame; - task.output_name = output_names[0]; - task.out_frame = out_frame; + task.input_name = exec_params->input_name; + task.in_frame = exec_params->in_frame; + task.output_name = exec_params->output_names[0]; + task.out_frame = exec_params->out_frame ? exec_params->out_frame : exec_params->in_frame; task.ov_model = ov_model; if (extract_inference_from_task(ov_model->model->func_type, &task, ov_model->inference_queue) != DNN_SUCCESS) { @@ -737,26 +724,19 @@ DNNReturnType ff_dnn_execute_model_ov(const DNNModel *model, const char *input_n return execute_model_ov(request, ov_model->inference_queue); } -DNNReturnType ff_dnn_execute_model_async_ov(const DNNModel *model, const char *input_name, AVFrame *in_frame, - const char **output_names, uint32_t nb_output, AVFrame *out_frame) +DNNReturnType ff_dnn_execute_model_async_ov(const DNNModel *model, DNNExecBaseParams *exec_params) { OVModel *ov_model = model->model; OVContext *ctx = &ov_model->ctx; RequestItem *request; TaskItem *task; - if (!in_frame) { - av_log(ctx, AV_LOG_ERROR, "in frame is NULL when async execute model.\n"); - return DNN_ERROR; - } - - if (!out_frame && model->func_type == DFT_PROCESS_FRAME) { - av_log(ctx, AV_LOG_ERROR, "out frame is NULL when async execute model.\n"); + if (ff_check_exec_params(ctx, DNN_OV, model->func_type, exec_params) != 0) { return DNN_ERROR; } if (!ov_model->exe_network) { - if (init_model_ov(ov_model, input_name, output_names[0]) != DNN_SUCCESS) { + if (init_model_ov(ov_model, exec_params->input_name, exec_params->output_names[0]) != DNN_SUCCESS) { av_log(ctx, AV_LOG_ERROR, "Failed init OpenVINO exectuable network or inference request\n"); return DNN_ERROR; } @@ -770,10 +750,10 @@ DNNReturnType ff_dnn_execute_model_async_ov(const DNNModel *model, const char *i task->do_ioproc = 1; task->async = 1; - task->input_name = input_name; - task->in_frame = in_frame; - task->output_name = output_names[0]; - task->out_frame = out_frame; + task->input_name = exec_params->input_name; + task->in_frame = exec_params->in_frame; + task->output_name = exec_params->output_names[0]; + task->out_frame = exec_params->out_frame ? exec_params->out_frame : exec_params->in_frame; task->ov_model = ov_model; if (ff_queue_push_back(ov_model->task_queue, task) < 0) { av_freep(&task); diff --git a/libavfilter/dnn/dnn_backend_openvino.h b/libavfilter/dnn/dnn_backend_openvino.h index a484a7be32..046d0c5b5a 100644 --- a/libavfilter/dnn/dnn_backend_openvino.h +++ b/libavfilter/dnn/dnn_backend_openvino.h @@ -31,10 +31,8 @@ DNNModel *ff_dnn_load_model_ov(const char *model_filename, DNNFunctionType func_type, const char *options, AVFilterContext *filter_ctx); -DNNReturnType ff_dnn_execute_model_ov(const DNNModel *model, const char *input_name, AVFrame *in_frame, - const char **output_names, uint32_t nb_output, AVFrame *out_frame); -DNNReturnType ff_dnn_execute_model_async_ov(const DNNModel *model, const char *input_name, AVFrame *in_frame, - const char **output_names, uint32_t nb_output, AVFrame *out_frame); +DNNReturnType ff_dnn_execute_model_ov(const DNNModel *model, DNNExecBaseParams *exec_params); +DNNReturnType ff_dnn_execute_model_async_ov(const DNNModel *model, DNNExecBaseParams *exec_params); DNNAsyncStatusType ff_dnn_get_async_result_ov(const DNNModel *model, AVFrame **in, AVFrame **out); DNNReturnType ff_dnn_flush_ov(const DNNModel *model); diff --git a/libavfilter/dnn/dnn_backend_tf.c b/libavfilter/dnn/dnn_backend_tf.c index 076dd3d6a9..03fe310b03 100644 --- a/libavfilter/dnn/dnn_backend_tf.c +++ b/libavfilter/dnn/dnn_backend_tf.c @@ -34,7 +34,7 @@ #include "dnn_backend_native_layer_pad.h" #include "dnn_backend_native_layer_maximum.h" #include "dnn_io_proc.h" - +#include "dnn_backend_common.h" #include typedef struct TFOptions{ @@ -814,23 +814,17 @@ static DNNReturnType execute_model_tf(const DNNModel *model, const char *input_n return DNN_SUCCESS; } -DNNReturnType ff_dnn_execute_model_tf(const DNNModel *model, const char *input_name, AVFrame *in_frame, - const char **output_names, uint32_t nb_output, AVFrame *out_frame) +DNNReturnType ff_dnn_execute_model_tf(const DNNModel *model, DNNExecBaseParams *exec_params) { TFModel *tf_model = model->model; TFContext *ctx = &tf_model->ctx; - if (!in_frame) { - av_log(ctx, AV_LOG_ERROR, "in frame is NULL when execute model.\n"); - return DNN_ERROR; - } - - if (!out_frame) { - av_log(ctx, AV_LOG_ERROR, "out frame is NULL when execute model.\n"); - return DNN_ERROR; + if (ff_check_exec_params(ctx, DNN_TF, model->func_type, exec_params) != 0) { + return DNN_ERROR; } - return execute_model_tf(model, input_name, in_frame, output_names, nb_output, out_frame, 1); + return execute_model_tf(model, exec_params->input_name, exec_params->in_frame, + exec_params->output_names, exec_params->nb_output, exec_params->out_frame, 1); } void ff_dnn_free_model_tf(DNNModel **model) diff --git a/libavfilter/dnn/dnn_backend_tf.h b/libavfilter/dnn/dnn_backend_tf.h index 8cec04748e..3dfd6e4280 100644 --- a/libavfilter/dnn/dnn_backend_tf.h +++ b/libavfilter/dnn/dnn_backend_tf.h @@ -31,8 +31,7 @@ DNNModel *ff_dnn_load_model_tf(const char *model_filename, DNNFunctionType func_type, const char *options, AVFilterContext *filter_ctx); -DNNReturnType ff_dnn_execute_model_tf(const DNNModel *model, const char *input_name, AVFrame *in_frame, - const char **output_names, uint32_t nb_output, AVFrame *out_frame); +DNNReturnType ff_dnn_execute_model_tf(const DNNModel *model, DNNExecBaseParams *exec_params); void ff_dnn_free_model_tf(DNNModel **model); diff --git a/libavfilter/dnn_filter_common.c b/libavfilter/dnn_filter_common.c index 1b922455a3..c085884eb4 100644 --- a/libavfilter/dnn_filter_common.c +++ b/libavfilter/dnn_filter_common.c @@ -90,14 +90,26 @@ DNNReturnType ff_dnn_get_output(DnnContext *ctx, int input_width, int input_heig DNNReturnType ff_dnn_execute_model(DnnContext *ctx, AVFrame *in_frame, AVFrame *out_frame) { - return (ctx->dnn_module->execute_model)(ctx->model, ctx->model_inputname, in_frame, - (const char **)&ctx->model_outputname, 1, out_frame); + DNNExecBaseParams exec_params = { + .input_name = ctx->model_inputname, + .output_names = (const char **)&ctx->model_outputname, + .nb_output = 1, + .in_frame = in_frame, + .out_frame = out_frame, + }; + return (ctx->dnn_module->execute_model)(ctx->model, &exec_params); } DNNReturnType ff_dnn_execute_model_async(DnnContext *ctx, AVFrame *in_frame, AVFrame *out_frame) { - return (ctx->dnn_module->execute_model_async)(ctx->model, ctx->model_inputname, in_frame, - (const char **)&ctx->model_outputname, 1, out_frame); + DNNExecBaseParams exec_params = { + .input_name = ctx->model_inputname, + .output_names = (const char **)&ctx->model_outputname, + .nb_output = 1, + .in_frame = in_frame, + .out_frame = out_frame, + }; + return (ctx->dnn_module->execute_model_async)(ctx->model, &exec_params); } DNNAsyncStatusType ff_dnn_get_async_result(DnnContext *ctx, AVFrame **in_frame, AVFrame **out_frame) diff --git a/libavfilter/dnn_interface.h b/libavfilter/dnn_interface.h index ae5a488341..941670675d 100644 --- a/libavfilter/dnn_interface.h +++ b/libavfilter/dnn_interface.h @@ -63,6 +63,14 @@ typedef struct DNNData{ DNNColorOrder order; } DNNData; +typedef struct DNNExecBaseParams { + const char *input_name; + const char **output_names; + uint32_t nb_output; + AVFrame *in_frame; + AVFrame *out_frame; +} DNNExecBaseParams; + typedef int (*FramePrePostProc)(AVFrame *frame, DNNData *model, AVFilterContext *filter_ctx); typedef int (*DetectPostProc)(AVFrame *frame, DNNData *output, uint32_t nb, AVFilterContext *filter_ctx); @@ -96,11 +104,9 @@ typedef struct DNNModule{ // Loads model and parameters from given file. Returns NULL if it is not possible. DNNModel *(*load_model)(const char *model_filename, DNNFunctionType func_type, const char *options, AVFilterContext *filter_ctx); // Executes model with specified input and output. Returns DNN_ERROR otherwise. - DNNReturnType (*execute_model)(const DNNModel *model, const char *input_name, AVFrame *in_frame, - const char **output_names, uint32_t nb_output, AVFrame *out_frame); + DNNReturnType (*execute_model)(const DNNModel *model, DNNExecBaseParams *exec_params); // Executes model with specified input and output asynchronously. Returns DNN_ERROR otherwise. - DNNReturnType (*execute_model_async)(const DNNModel *model, const char *input_name, AVFrame *in_frame, - const char **output_names, uint32_t nb_output, AVFrame *out_frame); + DNNReturnType (*execute_model_async)(const DNNModel *model, DNNExecBaseParams *exec_params); // Retrieve inference result. DNNAsyncStatusType (*get_async_result)(const DNNModel *model, AVFrame **in, AVFrame **out); // Flush all the pending tasks.