From patchwork Sat Jul 31 05:35:15 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shubhanshu Saxena X-Patchwork-Id: 29144 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a5d:965a:0:0:0:0:0 with SMTP id d26csp132895ios; Fri, 30 Jul 2021 22:36:20 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyLgND+p0zaumOLsYPXWAX4cgFS/d5/iNCLfFyMKFg095046VWIH+5DkbLq3ciEX03vIlV2 X-Received: by 2002:a05:6402:22b0:: with SMTP id cx16mr7397915edb.185.1627709780299; Fri, 30 Jul 2021 22:36:20 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1627709780; cv=none; d=google.com; s=arc-20160816; b=MxrfJjFKDy2QdKBSfmej58NVwCNvoMTApTHXQ/LntAAiDL6SI5gBQG/ihtofgyVJ0/ 3TMsG7fLbH2okeypLUaZe2F5E1LLnlk69b1FC3HCFEH0fUyr6XWqSybPZ8VEsYDVpkmi fZcZ8VMz40nJlOGW/uUhMLdiqJsCuQXNq45JEs/oxm8EP7vKo7u+ccnRZGfjLRbviaiK nN4j69In3f9OkM/RfTdwDZFqgrujGxBJTn6R3bp2VIuncQiSGjuwf81pJU3kYI5zc4Av E1uONvXoAqwqkueF452qQneJM6pvugMbsEWWW8XLH4bPYslPSj0dz88tZz9XJKOkKnJ/ f6fQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=gRyICjugDiayJLj2iEvxw20o6U4kNhcX198/MHGBwuo=; b=VMhdhPvlLhripJkIUtctH5NjrGiBWs7OYLteX1+5x5ILjTkDHbIoMTRrRfeteiCcAz 7uB/h6lgoj5S84cHyfuR/Bci7Y/RJm6ZcBuV06gNekQz9lob8g+3p8JeJdBn+FySs4zu MkiaS0vQl1BGQXW13WKyyBKZzOGVLE7y0+uTwpvn4gIAtWiY5y+zI8nwmjjrIZ9mTL6E XrwUuFHryguwzyjulD85giFGScenrYcD7NmICSUDBQt7IEnPuuFnRUprgfwi7iTJ9mOX OQOXgruHiECQmjB2XrKKaEwusNA4pStASpeO0BN9PD/NRmsKDfnpGnSIDy1eh8zARTbG QHWA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b=o1kTUNcc; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id lv9si3887292ejb.470.2021.07.30.22.36.19; Fri, 30 Jul 2021 22:36:20 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b=o1kTUNcc; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id B53F3689B83; Sat, 31 Jul 2021 08:36:15 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pl1-f169.google.com (mail-pl1-f169.google.com [209.85.214.169]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id E6C5A689F02 for ; Sat, 31 Jul 2021 08:36:08 +0300 (EEST) Received: by mail-pl1-f169.google.com with SMTP id e21so13557422pla.5 for ; Fri, 30 Jul 2021 22:36:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=D4wOKHSoELtJQgdBzUqXxpwkADqTvb2S78r/abmBbV4=; b=o1kTUNccN8GyPgj/GPS6gu/akMkYPFxKVpA7BXs8Cbtz6fqU7ozjF9uX/H1KUVpFtN IEDi7nbnjh4uWstoEwEJp0au8lDFITVkjG9DqhMha2m+VegCh/+0h9IEYhOLzcyrgCzF CrVWb8pGng8qws5GQIFe7TmgEzh9eF1rDooHo8zX/oqp+XVim4nlWeeSwMfde9ibICAZ XUFdDGhWSpMWbe+MsdyMGXQHHDANYgM3PhpTyMBopDj68JipHMOlFr9WbhpDitNQ2H5A A8AZS/D7GVPGmzSSA43pRTdOHc9PjQ6ZdeCAoUEnB21ctFISHRdBwGnn62keg+PnmJT1 VIWg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=D4wOKHSoELtJQgdBzUqXxpwkADqTvb2S78r/abmBbV4=; b=pWRWaZG4dZH4xY0UBhMdj/40nIDn54t0pxirD6qDt2zcRfHpGO/K8okOYlfOOGG/g5 3I8w7m8Gu02yxB/psjJ1tiVP08rPQ5BxhLQDP6Hvt5wFlSODg99GzOGHoru/T5/Wwpge LBGiVW/US4W4km+zGrSX1nU2i8tSGvdYT6NTlOklalHnHfTvBJRBcpZqVXyfdXHNjIEj 7ncLRInH8FbpB8+mmIxTwK8zdJ8egub78lWvCZsWswTlZliD8dB5p5elR/dNPWNn9x9f /03/AgAuDfcTMZd+cb55q0w3A9Q5T2+9OfqMnx/++8YFgOPohEsZe+zQNs21FxD8ezRL uxeA== X-Gm-Message-State: AOAM531CEEX6g7ryb7msue7iiemGLm+mRpIlVJH2eu886i5kOYFWJ/Xx FklIGccOHZ2SHI7A1a3b0X6MCU0WQ1PlWw== X-Received: by 2002:a05:6a00:d5f:b029:32c:7264:2f65 with SMTP id n31-20020a056a000d5fb029032c72642f65mr6115128pfv.40.1627709767067; Fri, 30 Jul 2021 22:36:07 -0700 (PDT) Received: from localhost.localdomain ([103.157.220.250]) by smtp.googlemail.com with ESMTPSA id p34sm4312266pfh.172.2021.07.30.22.36.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Jul 2021 22:36:06 -0700 (PDT) From: Shubhanshu Saxena To: ffmpeg-devel@ffmpeg.org Date: Sat, 31 Jul 2021 11:05:15 +0530 Message-Id: <20210731053516.7700-7-shubhanshu.e01@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210731053516.7700-1-shubhanshu.e01@gmail.com> References: <20210731053516.7700-1-shubhanshu.e01@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 7/8] lavfi/dnn: Extract Common Parts from get_output functions X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Shubhanshu Saxena Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: +R1hvivr8MTL The frame allocation and filling the TaskItem with execution parameters is common in the three backends. This commit shifts this logic to dnn_backend_common. Signed-off-by: Shubhanshu Saxena --- libavfilter/dnn/dnn_backend_common.c | 26 +++++++++++++ libavfilter/dnn/dnn_backend_common.h | 16 ++++++++ libavfilter/dnn/dnn_backend_openvino.c | 51 +++++++++----------------- libavfilter/dnn/dnn_backend_tf.c | 42 +++++++-------------- 4 files changed, 72 insertions(+), 63 deletions(-) diff --git a/libavfilter/dnn/dnn_backend_common.c b/libavfilter/dnn/dnn_backend_common.c index df4bab85e0..470fffa2ae 100644 --- a/libavfilter/dnn/dnn_backend_common.c +++ b/libavfilter/dnn/dnn_backend_common.c @@ -142,3 +142,29 @@ DNNAsyncStatusType ff_dnn_get_async_result_common(Queue *task_queue, AVFrame **i return DAST_SUCCESS; } + +DNNReturnType ff_dnn_fill_gettingoutput_task(TaskItem *task, DNNExecBaseParams *exec_params, void *backend_model, int input_height, int input_width, void *ctx) +{ + AVFrame *in_frame = NULL; + AVFrame *out_frame = NULL; + + in_frame = av_frame_alloc(); + if (!in_frame) { + av_log(ctx, AV_LOG_ERROR, "Failed to allocate memory for input frame\n"); + return DNN_ERROR; + } + + out_frame = av_frame_alloc(); + if (!out_frame) { + av_frame_free(&in_frame); + av_log(ctx, AV_LOG_ERROR, "Failed to allocate memory for output frame\n"); + return DNN_ERROR; + } + + in_frame->width = input_width; + in_frame->height = input_height; + exec_params->in_frame = in_frame; + exec_params->out_frame = out_frame; + + return ff_dnn_fill_task(task, exec_params, backend_model, 0, 0); +} diff --git a/libavfilter/dnn/dnn_backend_common.h b/libavfilter/dnn/dnn_backend_common.h index e1d678b230..604c1d3bd7 100644 --- a/libavfilter/dnn/dnn_backend_common.h +++ b/libavfilter/dnn/dnn_backend_common.h @@ -137,4 +137,20 @@ DNNReturnType ff_dnn_start_inference_async(void *ctx, DNNAsyncExecModule *async_ */ DNNAsyncStatusType ff_dnn_get_async_result_common(Queue *task_queue, AVFrame **in, AVFrame **out); +/** + * Allocate input and output frames and fill the Task + * with execution parameters. + * + * @param task pointer to the allocated task + * @param exec_params pointer to execution parameters + * @param backend_model void pointer to the backend model + * @param input_height height of input frame + * @param input_width width of input frame + * @param ctx pointer to the backend context + * + * @retval DNN_SUCCESS if successful + * @retval DNN_ERROR if allocation fails + */ +DNNReturnType ff_dnn_fill_gettingoutput_task(TaskItem *task, DNNExecBaseParams *exec_params, void *backend_model, int input_height, int input_width, void *ctx); + #endif diff --git a/libavfilter/dnn/dnn_backend_openvino.c b/libavfilter/dnn/dnn_backend_openvino.c index 47cbd48ba2..c825e70c82 100644 --- a/libavfilter/dnn/dnn_backend_openvino.c +++ b/libavfilter/dnn/dnn_backend_openvino.c @@ -640,10 +640,15 @@ static DNNReturnType get_output_ov(void *model, const char *input_name, int inpu OVContext *ctx = &ov_model->ctx; TaskItem task; OVRequestItem *request; - AVFrame *in_frame = NULL; - AVFrame *out_frame = NULL; IEStatusCode status; input_shapes_t input_shapes; + DNNExecBaseParams exec_params = { + .input_name = input_name, + .output_names = &output_name, + .nb_output = 1, + .in_frame = NULL, + .out_frame = NULL, + }; if (ov_model->model->func_type != DFT_PROCESS_FRAME) { av_log(ctx, AV_LOG_ERROR, "Get output dim only when processing frame.\n"); @@ -669,51 +674,29 @@ static DNNReturnType get_output_ov(void *model, const char *input_name, int inpu } } - in_frame = av_frame_alloc(); - if (!in_frame) { - av_log(ctx, AV_LOG_ERROR, "Failed to allocate memory for input frame\n"); + if (ff_dnn_fill_gettingoutput_task(&task, &exec_params, ov_model, input_height, input_width, ctx) != DNN_SUCCESS) { return DNN_ERROR; } - in_frame->width = input_width; - in_frame->height = input_height; - - out_frame = av_frame_alloc(); - if (!out_frame) { - av_log(ctx, AV_LOG_ERROR, "Failed to allocate memory for output frame\n"); - av_frame_free(&in_frame); - return DNN_ERROR; - } - - task.do_ioproc = 0; - task.async = 0; - task.input_name = input_name; - task.in_frame = in_frame; - task.output_names = &output_name; - task.out_frame = out_frame; - task.nb_output = 1; - task.model = ov_model; if (extract_inference_from_task(ov_model->model->func_type, &task, ov_model->inference_queue, NULL) != DNN_SUCCESS) { - av_frame_free(&out_frame); - av_frame_free(&in_frame); av_log(ctx, AV_LOG_ERROR, "unable to extract inference from task.\n"); - return DNN_ERROR; + ret = DNN_ERROR; + goto err; } request = ff_safe_queue_pop_front(ov_model->request_queue); if (!request) { - av_frame_free(&out_frame); - av_frame_free(&in_frame); av_log(ctx, AV_LOG_ERROR, "unable to get infer request.\n"); - return DNN_ERROR; + ret = DNN_ERROR; + goto err; } ret = execute_model_ov(request, ov_model->inference_queue); - *output_width = out_frame->width; - *output_height = out_frame->height; - - av_frame_free(&out_frame); - av_frame_free(&in_frame); + *output_width = task.out_frame->width; + *output_height = task.out_frame->height; +err: + av_frame_free(&task.out_frame); + av_frame_free(&task.in_frame); return ret; } diff --git a/libavfilter/dnn/dnn_backend_tf.c b/libavfilter/dnn/dnn_backend_tf.c index c3425e01a8..9c96868fab 100644 --- a/libavfilter/dnn/dnn_backend_tf.c +++ b/libavfilter/dnn/dnn_backend_tf.c @@ -309,36 +309,20 @@ static DNNReturnType get_output_tf(void *model, const char *input_name, int inpu DNNReturnType ret; TFModel *tf_model = model; TFContext *ctx = &tf_model->ctx; - AVFrame *in_frame = av_frame_alloc(); - AVFrame *out_frame = NULL; TaskItem task; TFRequestItem *request; - - if (!in_frame) { - av_log(ctx, AV_LOG_ERROR, "Failed to allocate memory for input frame\n"); - ret = DNN_ERROR; - goto err; - } - - out_frame = av_frame_alloc(); - if (!out_frame) { - av_log(ctx, AV_LOG_ERROR, "Failed to allocate memory for output frame\n"); - ret = DNN_ERROR; + DNNExecBaseParams exec_params = { + .input_name = input_name, + .output_names = &output_name, + .nb_output = 1, + .in_frame = NULL, + .out_frame = NULL, + }; + + if (ff_dnn_fill_gettingoutput_task(&task, &exec_params, tf_model, input_height, input_width, ctx) != DNN_SUCCESS) { goto err; } - in_frame->width = input_width; - in_frame->height = input_height; - - task.do_ioproc = 0; - task.async = 0; - task.input_name = input_name; - task.in_frame = in_frame; - task.output_names = &output_name; - task.out_frame = out_frame; - task.model = tf_model; - task.nb_output = 1; - if (extract_inference_from_task(&task, tf_model->inference_queue) != DNN_SUCCESS) { av_log(ctx, AV_LOG_ERROR, "unable to extract inference from task.\n"); ret = DNN_ERROR; @@ -353,12 +337,12 @@ static DNNReturnType get_output_tf(void *model, const char *input_name, int inpu } ret = execute_model_tf(request, tf_model->inference_queue); - *output_width = out_frame->width; - *output_height = out_frame->height; + *output_width = task.out_frame->width; + *output_height = task.out_frame->height; err: - av_frame_free(&out_frame); - av_frame_free(&in_frame); + av_frame_free(&task.out_frame); + av_frame_free(&task.in_frame); return ret; }