From patchwork Fri Sep 20 03:56:10 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Guo, Yejun" X-Patchwork-Id: 15176 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 66EF24469F0 for ; Fri, 20 Sep 2019 07:01:08 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 4EFE9689F1C; Fri, 20 Sep 2019 07:01:08 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 5CECB689A28 for ; Fri, 20 Sep 2019 07:01:07 +0300 (EEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga003.jf.intel.com ([10.7.209.27]) by orsmga102.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 19 Sep 2019 21:01:04 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.64,527,1559545200"; d="scan'208";a="189820156" Received: from yguo18-skl-u1604.sh.intel.com ([10.239.13.25]) by orsmga003.jf.intel.com with ESMTP; 19 Sep 2019 21:01:02 -0700 From: "Guo, Yejun" To: ffmpeg-devel@ffmpeg.org Date: Fri, 20 Sep 2019 11:56:10 +0800 Message-Id: <1568951770-6165-1-git-send-email-yejun.guo@intel.com> X-Mailer: git-send-email 2.7.4 Subject: [FFmpeg-devel] [PATCH 4/4] libavfilter/dnn: support multiple outputs for native mode X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: yejun.guo@intel.com MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Signed-off-by: Guo, Yejun --- libavfilter/dnn/dnn_backend_native.c | 43 +++++++++++++++++++++++++++--------- libavfilter/dnn/dnn_backend_native.h | 2 ++ 2 files changed, 34 insertions(+), 11 deletions(-) diff --git a/libavfilter/dnn/dnn_backend_native.c b/libavfilter/dnn/dnn_backend_native.c index 1b0aea2..68fca50 100644 --- a/libavfilter/dnn/dnn_backend_native.c +++ b/libavfilter/dnn/dnn_backend_native.c @@ -38,6 +38,7 @@ static DNNReturnType set_input_output_native(void *model, DNNInputData *input, c if (network->layers_num <= 0 || network->operands_num <= 0) return DNN_ERROR; + /* inputs */ av_assert0(input->dt == DNN_FLOAT); for (int i = 0; i < network->operands_num; ++i) { oprd = &network->operands[i]; @@ -64,6 +65,28 @@ static DNNReturnType set_input_output_native(void *model, DNNInputData *input, c return DNN_ERROR; input->data = oprd->data; + + /* outputs */ + network->nb_output = 0; + av_freep(&network->output_indexes); + network->output_indexes = av_mallocz_array(nb_output, sizeof(*network->output_indexes)); + if (!network->output_indexes) + return DNN_ERROR; + + for (uint32_t i = 0; i < nb_output; ++i) { + const char *output_name = output_names[i]; + for (int j = 0; j < network->operands_num; ++j) { + oprd = &network->operands[j]; + if (strcmp(oprd->name, output_name) == 0) { + network->output_indexes[network->nb_output++] = j; + break; + } + } + } + + if (network->nb_output != nb_output) + return DNN_ERROR; + return DNN_SUCCESS; } @@ -315,6 +338,7 @@ DNNReturnType ff_dnn_execute_model_native(const DNNModel *model, DNNData *output DepthToSpaceParams *depth_to_space_params; LayerPadParams *pad_params; DnnLayerMaximumParams *maximum_params; + uint32_t nb = FFMIN(nb_output, network->nb_output); if (network->layers_num <= 0 || network->operands_num <= 0) return DNN_ERROR; @@ -348,17 +372,13 @@ DNNReturnType ff_dnn_execute_model_native(const DNNModel *model, DNNData *output } } - // native mode does not support multiple outputs yet - if (nb_output > 1) - return DNN_ERROR; - - /** - * as the first step, suppose network->operands[network->operands_num - 1] is the output operand. - */ - outputs[0].data = network->operands[network->operands_num - 1].data; - outputs[0].height = network->operands[network->operands_num - 1].dims[1]; - outputs[0].width = network->operands[network->operands_num - 1].dims[2]; - outputs[0].channels = network->operands[network->operands_num - 1].dims[3]; + for (uint32_t i = 0; i < nb; ++i) { + DnnOperand *oprd = &network->operands[network->output_indexes[i]]; + outputs[i].data = oprd->data; + outputs[i].height = oprd->dims[1]; + outputs[i].width = oprd->dims[2]; + outputs[i].channels = oprd->dims[3]; + } return DNN_SUCCESS; } @@ -401,6 +421,7 @@ void ff_dnn_free_model_native(DNNModel **model) av_freep(&network->operands[operand].data); av_freep(&network->operands); + av_freep(&network->output_indexes); av_freep(&network); av_freep(model); } diff --git a/libavfilter/dnn/dnn_backend_native.h b/libavfilter/dnn/dnn_backend_native.h index b238d18..3f2840c 100644 --- a/libavfilter/dnn/dnn_backend_native.h +++ b/libavfilter/dnn/dnn_backend_native.h @@ -96,6 +96,8 @@ typedef struct ConvolutionalNetwork{ int32_t layers_num; DnnOperand *operands; int32_t operands_num; + int32_t *output_indexes; + uint32_t nb_output; } ConvolutionalNetwork; DNNModel *ff_dnn_load_model_native(const char *model_filename);