From patchwork Wed Aug 4 11:51:37 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shubhanshu Saxena X-Patchwork-Id: 29234 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a6b:6c0f:0:0:0:0:0 with SMTP id a15csp3303907ioh; Wed, 4 Aug 2021 04:52:35 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwtZCjxa7SUP9dNPGqA/EmdVDzBA/fw+rWcEdtz0B2rjiVQf9tdw9Vyvy/AGPImebFclwqQ X-Received: by 2002:a17:907:6218:: with SMTP id ms24mr25690582ejc.367.1628077955654; Wed, 04 Aug 2021 04:52:35 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1628077955; cv=none; d=google.com; s=arc-20160816; b=0QBV9aMvbNm5NwYxjpOnygu4K4/pZZ/q2HzA0alzEqSsHRfrAO2zfFjRWQpWIb0y5U GAHbVg8fqh82iHWtMVHGUVq8qNsJWkQP6pJyszRAOfdams+O4owquLxqlQ6euoeuiUKP oF3PyRUSLmQq1QaE58sLUua9NE8hiyTUJeAizTk1+AFOdHbS0xHgqSlMDLMfsBo1nBPj xSnljiaVK7BneKo4V7cJKFvr3cReYpHyBd8VRHQtTFWF5pSq1QpmYXqL3TfWVhYBjjOB qAvROIuh3JTuJWsZUYVuRsioeRo7k/PtpkJsF2yoNSo6fyvfJBgYTgvy65FWOrK4bRmn 8ksg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=gRyICjugDiayJLj2iEvxw20o6U4kNhcX198/MHGBwuo=; b=rkAwgUyzYs5rgTFycmNoX+ohm3Ua/KKw5gNw/hLb8GwPhmoBdPH3GAJAzbe5QGOJzX 0p9E2NJmyU0FAcs7w7Y/9scEO7A2/DQdsRQO6z11hq5BZy/b3OOLA537rLvb+EPQcXYL yT1jIVmIqkLouduee7ACEUV1uyaKSnPon5nCluXR86DK6FSOfIA7ubD5ZzwOOcBy0S7O fouLtah4UHFx8W+eJTC9/KxVUueJYSqaaebXeIn2J0a0jqTQPnbEXZHcbq1Oe7yz6SKB rZx92sDXtSTrn51PiibVRf389nSJpTJjXYuWAEjozF+iFszvjhv5gc6Ek81rxoxa1rXv 7JJw== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b=ia9SUHWI; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id gt14si1709359ejb.752.2021.08.04.04.52.35; Wed, 04 Aug 2021 04:52:35 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b=ia9SUHWI; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 2F831689F8F; Wed, 4 Aug 2021 14:52:20 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pj1-f44.google.com (mail-pj1-f44.google.com [209.85.216.44]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 18252680C5A for ; Wed, 4 Aug 2021 14:52:13 +0300 (EEST) Received: by mail-pj1-f44.google.com with SMTP id dw2-20020a17090b0942b0290177cb475142so8312186pjb.2 for ; Wed, 04 Aug 2021 04:52:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=D4wOKHSoELtJQgdBzUqXxpwkADqTvb2S78r/abmBbV4=; b=ia9SUHWIHAFoTrKommIN2HFRileqP6GbicQ3569xymt3TjKndNTJiFgSW6Qv4ygixE DJM7zadgf86JhZXPE9Y0wJV/7AofoseZg3oWnK01lzmTIF9rV0Oc2r3Y3oHG331ScXji NfiYm1EFvi/wd1vP+Trqbojtq/sMQHrUNiKowHVolyR9pFr+/XbTnL4UkH96WAEbFD7k V6onO/av9ZJsHgV4QD9L3QQbiZAUUpzsVqCLsfstkjowGtf4m0bwyjkVZcAEVS4tndlE Hjm8KmNVulX7lgm4fUgtoSNllddv8xjfQ+WY+mUyYRKhUvXczG4kaD4Wz0b+EpVvTTQP 5k0w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=D4wOKHSoELtJQgdBzUqXxpwkADqTvb2S78r/abmBbV4=; b=r4fIPvl4LCUxQReaBnyfwAxXGLJpwOpRanHs08gYMxVsPxCCFLlJdYUzkdk3gYPwmZ 3Z6G7ZTrHrpRmK9F4ckLb7B3NS2oGm6xNOJdnB0D7058rHnis84/rQPeoAfIypFzzJOM BN4NMAGNAYK7VIh7bgULXpWUkQi4Z+qqf2Zfpzrnd7i1Wc7u1lVIe+WBdo7v8C+aZ1lP GaGzsbV0eDVQSMnkBDRGv71Hv/CN/ZtJRX7XAbvfYseLBFMMTFUECTfkHXtcvFsTVwQx 3c6lImpEhSktNZCSSaDTpKbP4oNatGjqPtBXRXdDSJ6pb0P4NccUknmkHapTyj9t6zXL e2ZQ== X-Gm-Message-State: AOAM533D+yH2wuyFRyUQu1m1spN9qm0d01ZnE2QjVf90n1U016nHY6hM BOK0X5aLT+9gu1HkBMACAcnxU+kPOU20+A== X-Received: by 2002:a17:90a:fc6:: with SMTP id 64mr27819050pjz.1.1628077931295; Wed, 04 Aug 2021 04:52:11 -0700 (PDT) Received: from Pavilion-x360.bbrouter ([103.133.121.9]) by smtp.googlemail.com with ESMTPSA id ms8sm6040305pjb.36.2021.08.04.04.52.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 04 Aug 2021 04:52:10 -0700 (PDT) From: Shubhanshu Saxena To: ffmpeg-devel@ffmpeg.org Date: Wed, 4 Aug 2021 17:21:37 +0530 Message-Id: <20210804115138.64475-7-shubhanshu.e01@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210804115138.64475-1-shubhanshu.e01@gmail.com> References: <20210804115138.64475-1-shubhanshu.e01@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH V2 7/8] [GSoC] lavfi/dnn: Extract Common Parts from get_output functions X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Shubhanshu Saxena Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: hf+9KNUO+Alq The frame allocation and filling the TaskItem with execution parameters is common in the three backends. This commit shifts this logic to dnn_backend_common. Signed-off-by: Shubhanshu Saxena --- libavfilter/dnn/dnn_backend_common.c | 26 +++++++++++++ libavfilter/dnn/dnn_backend_common.h | 16 ++++++++ libavfilter/dnn/dnn_backend_openvino.c | 51 +++++++++----------------- libavfilter/dnn/dnn_backend_tf.c | 42 +++++++-------------- 4 files changed, 72 insertions(+), 63 deletions(-) diff --git a/libavfilter/dnn/dnn_backend_common.c b/libavfilter/dnn/dnn_backend_common.c index df4bab85e0..470fffa2ae 100644 --- a/libavfilter/dnn/dnn_backend_common.c +++ b/libavfilter/dnn/dnn_backend_common.c @@ -142,3 +142,29 @@ DNNAsyncStatusType ff_dnn_get_async_result_common(Queue *task_queue, AVFrame **i return DAST_SUCCESS; } + +DNNReturnType ff_dnn_fill_gettingoutput_task(TaskItem *task, DNNExecBaseParams *exec_params, void *backend_model, int input_height, int input_width, void *ctx) +{ + AVFrame *in_frame = NULL; + AVFrame *out_frame = NULL; + + in_frame = av_frame_alloc(); + if (!in_frame) { + av_log(ctx, AV_LOG_ERROR, "Failed to allocate memory for input frame\n"); + return DNN_ERROR; + } + + out_frame = av_frame_alloc(); + if (!out_frame) { + av_frame_free(&in_frame); + av_log(ctx, AV_LOG_ERROR, "Failed to allocate memory for output frame\n"); + return DNN_ERROR; + } + + in_frame->width = input_width; + in_frame->height = input_height; + exec_params->in_frame = in_frame; + exec_params->out_frame = out_frame; + + return ff_dnn_fill_task(task, exec_params, backend_model, 0, 0); +} diff --git a/libavfilter/dnn/dnn_backend_common.h b/libavfilter/dnn/dnn_backend_common.h index e1d678b230..604c1d3bd7 100644 --- a/libavfilter/dnn/dnn_backend_common.h +++ b/libavfilter/dnn/dnn_backend_common.h @@ -137,4 +137,20 @@ DNNReturnType ff_dnn_start_inference_async(void *ctx, DNNAsyncExecModule *async_ */ DNNAsyncStatusType ff_dnn_get_async_result_common(Queue *task_queue, AVFrame **in, AVFrame **out); +/** + * Allocate input and output frames and fill the Task + * with execution parameters. + * + * @param task pointer to the allocated task + * @param exec_params pointer to execution parameters + * @param backend_model void pointer to the backend model + * @param input_height height of input frame + * @param input_width width of input frame + * @param ctx pointer to the backend context + * + * @retval DNN_SUCCESS if successful + * @retval DNN_ERROR if allocation fails + */ +DNNReturnType ff_dnn_fill_gettingoutput_task(TaskItem *task, DNNExecBaseParams *exec_params, void *backend_model, int input_height, int input_width, void *ctx); + #endif diff --git a/libavfilter/dnn/dnn_backend_openvino.c b/libavfilter/dnn/dnn_backend_openvino.c index 47cbd48ba2..c825e70c82 100644 --- a/libavfilter/dnn/dnn_backend_openvino.c +++ b/libavfilter/dnn/dnn_backend_openvino.c @@ -640,10 +640,15 @@ static DNNReturnType get_output_ov(void *model, const char *input_name, int inpu OVContext *ctx = &ov_model->ctx; TaskItem task; OVRequestItem *request; - AVFrame *in_frame = NULL; - AVFrame *out_frame = NULL; IEStatusCode status; input_shapes_t input_shapes; + DNNExecBaseParams exec_params = { + .input_name = input_name, + .output_names = &output_name, + .nb_output = 1, + .in_frame = NULL, + .out_frame = NULL, + }; if (ov_model->model->func_type != DFT_PROCESS_FRAME) { av_log(ctx, AV_LOG_ERROR, "Get output dim only when processing frame.\n"); @@ -669,51 +674,29 @@ static DNNReturnType get_output_ov(void *model, const char *input_name, int inpu } } - in_frame = av_frame_alloc(); - if (!in_frame) { - av_log(ctx, AV_LOG_ERROR, "Failed to allocate memory for input frame\n"); + if (ff_dnn_fill_gettingoutput_task(&task, &exec_params, ov_model, input_height, input_width, ctx) != DNN_SUCCESS) { return DNN_ERROR; } - in_frame->width = input_width; - in_frame->height = input_height; - - out_frame = av_frame_alloc(); - if (!out_frame) { - av_log(ctx, AV_LOG_ERROR, "Failed to allocate memory for output frame\n"); - av_frame_free(&in_frame); - return DNN_ERROR; - } - - task.do_ioproc = 0; - task.async = 0; - task.input_name = input_name; - task.in_frame = in_frame; - task.output_names = &output_name; - task.out_frame = out_frame; - task.nb_output = 1; - task.model = ov_model; if (extract_inference_from_task(ov_model->model->func_type, &task, ov_model->inference_queue, NULL) != DNN_SUCCESS) { - av_frame_free(&out_frame); - av_frame_free(&in_frame); av_log(ctx, AV_LOG_ERROR, "unable to extract inference from task.\n"); - return DNN_ERROR; + ret = DNN_ERROR; + goto err; } request = ff_safe_queue_pop_front(ov_model->request_queue); if (!request) { - av_frame_free(&out_frame); - av_frame_free(&in_frame); av_log(ctx, AV_LOG_ERROR, "unable to get infer request.\n"); - return DNN_ERROR; + ret = DNN_ERROR; + goto err; } ret = execute_model_ov(request, ov_model->inference_queue); - *output_width = out_frame->width; - *output_height = out_frame->height; - - av_frame_free(&out_frame); - av_frame_free(&in_frame); + *output_width = task.out_frame->width; + *output_height = task.out_frame->height; +err: + av_frame_free(&task.out_frame); + av_frame_free(&task.in_frame); return ret; } diff --git a/libavfilter/dnn/dnn_backend_tf.c b/libavfilter/dnn/dnn_backend_tf.c index c3425e01a8..9c96868fab 100644 --- a/libavfilter/dnn/dnn_backend_tf.c +++ b/libavfilter/dnn/dnn_backend_tf.c @@ -309,36 +309,20 @@ static DNNReturnType get_output_tf(void *model, const char *input_name, int inpu DNNReturnType ret; TFModel *tf_model = model; TFContext *ctx = &tf_model->ctx; - AVFrame *in_frame = av_frame_alloc(); - AVFrame *out_frame = NULL; TaskItem task; TFRequestItem *request; - - if (!in_frame) { - av_log(ctx, AV_LOG_ERROR, "Failed to allocate memory for input frame\n"); - ret = DNN_ERROR; - goto err; - } - - out_frame = av_frame_alloc(); - if (!out_frame) { - av_log(ctx, AV_LOG_ERROR, "Failed to allocate memory for output frame\n"); - ret = DNN_ERROR; + DNNExecBaseParams exec_params = { + .input_name = input_name, + .output_names = &output_name, + .nb_output = 1, + .in_frame = NULL, + .out_frame = NULL, + }; + + if (ff_dnn_fill_gettingoutput_task(&task, &exec_params, tf_model, input_height, input_width, ctx) != DNN_SUCCESS) { goto err; } - in_frame->width = input_width; - in_frame->height = input_height; - - task.do_ioproc = 0; - task.async = 0; - task.input_name = input_name; - task.in_frame = in_frame; - task.output_names = &output_name; - task.out_frame = out_frame; - task.model = tf_model; - task.nb_output = 1; - if (extract_inference_from_task(&task, tf_model->inference_queue) != DNN_SUCCESS) { av_log(ctx, AV_LOG_ERROR, "unable to extract inference from task.\n"); ret = DNN_ERROR; @@ -353,12 +337,12 @@ static DNNReturnType get_output_tf(void *model, const char *input_name, int inpu } ret = execute_model_tf(request, tf_model->inference_queue); - *output_width = out_frame->width; - *output_height = out_frame->height; + *output_width = task.out_frame->width; + *output_height = task.out_frame->height; err: - av_frame_free(&out_frame); - av_frame_free(&in_frame); + av_frame_free(&task.out_frame); + av_frame_free(&task.in_frame); return ret; }