From patchwork Sun Aug 8 10:55:33 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shubhanshu Saxena X-Patchwork-Id: 29349 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a6b:6c0f:0:0:0:0:0 with SMTP id a15csp1980913ioh; Sun, 8 Aug 2021 03:56:38 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzU8Ig27UConUtiLrqpnGJiPTi1dndqsm1bx1DL/RrVoJiBYKMZc+NycTPFlRNPBnIPiYG5 X-Received: by 2002:a05:6402:184b:: with SMTP id v11mr23581950edy.267.1628420198747; Sun, 08 Aug 2021 03:56:38 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1628420198; cv=none; d=google.com; s=arc-20160816; b=BKAFsRSLlRoWWBNcg9HBhsqkzjk0IVBEQvHD3mpqn9w4vS/KkfPcMLbDBZ/4ay8qhq V2jsdtiorOZbbS1ZrAd48tPnBzjLDI8LMUBUmbwSSDCqVxgBI4HJu1MmuTtZqVsz624D SsLsY3ybx9K7vijOGTH5462q1vgzLoKvSRvsXVMdO1WIApP2KWRcSTOTzJGXgbmDlK7c mw0gFD7q4r7mrTBIpPiqaO76VMpYlx7FMeX+A931z9S5DOP3OOBhlOkRSCrxZNVaz6dr 9wBb7SvNOhb4qL6NZvFvulE9SIf1EqNKM9LzOXKW6B2c1cCg+PDqJWjxx9kKxadTcD8w 3Ulw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=YMUTaUfel1J7hb5Ymw4unP1mrHBl8tlnblqLWs6WETQ=; b=B8l93QioJ7+QYHvdugPg64qEDcvjTmhIVT4fpW9DJBllshuND58kNIcqRDS1uZIgWE aoUGkS8kiuW9sfw5+Zc3UFKElXk/I7NJ86s8FDGEqiRk1ljH/3DBXA8Uhxn99M0/BcOW sPw0EBuSrhqUE9iCFjCMoG9LXWsjvJv/QHu4sCUDmiXLTTj78FKul9T2DvfSq+fBjOPV a7gV+KCVCyVdYigAJner+49ymWnqKENZ5aqjSQFtEWzzosRFwF8nmQlb6TFUjeOp/pSP 1Z3VvzB8vE8sZv/enSOOsI3eLY/xVSJLjCDqpnKGivr+XL6PAeCxLTa03xUfANibFj4S MfOA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b=hyNrRVMT; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id f14si14103833ejx.618.2021.08.08.03.56.38; Sun, 08 Aug 2021 03:56:38 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b=hyNrRVMT; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id C24FF68A466; Sun, 8 Aug 2021 13:56:19 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pj1-f51.google.com (mail-pj1-f51.google.com [209.85.216.51]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id C1D7968A457 for ; Sun, 8 Aug 2021 13:56:12 +0300 (EEST) Received: by mail-pj1-f51.google.com with SMTP id mt6so23460593pjb.1 for ; Sun, 08 Aug 2021 03:56:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=coEvR7NJ3/O2PciedLn3eoYxS1n8ONZdZczMZWtj/AQ=; b=hyNrRVMTlteS8BQm30ovMJJRh11ucBFIZQLOESisZV+B31f7i34AHDSAKUBQdYPk/G jyd3F2SksvdQsSsuDD5KhRXf5zkMeFh/i+aAK1aXGvssQhEyRkcX9qvMpwLyQvnHi4P7 KGP+A2e9JzdW8n04OwMc9nvyQWV03LvwFFmiE2zgCssxq2QUwwjrhJxUrUhuVdw4HxDJ CQnEzoCz+N543ynK2aj5Zmjiyl77EEwWvNNPOs8PSO6UlhGNgMwC8r2Kpwn8MB/K/IZm g6EN83563qHwDR/nLDTuUsf8vRRL/TQxc9q6+KhATD4A0wFK0E3GfJLL9CUNWL25wNR/ 8Scg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=coEvR7NJ3/O2PciedLn3eoYxS1n8ONZdZczMZWtj/AQ=; b=HxFeMBm2o+12Mbru7JjkPsCJZyj2OJmcjOuGycMkU5+R/lD7DG77ckZuY0IO1UhEWX koKRPPNKE8F1Qi3oHmvMz0j3CWeT2cBCZ0z58mVBsp/O8U64MKLIAepDT4i4b3AHUUGy UkubG+mk80m9ZQfuM/FandS2q27+GrXKojWNyNJwA6VKBw7voqn6QVnXMyseAKF6CkRp C9pWAp4cd4LJUDXGM1+tGjMGNX9XDsPu+oyujaF9nvs2nTXxMMDVKHqM/F9yWBirf98v AhXUxmyUK5tmzaTOU/vmzsWkzqBWB5g32c45Mx7tOdCXGlyk3PEBc+VzgfPtKkDsAOSq qg1Q== X-Gm-Message-State: AOAM53062nfKlaCm0NC44aQUjIwqc5b8uLQzGG0mzh6kAkNnBCUe7fKI k+WkwC9PICnp7n8D7s/3dO4F9gWlzysGIw== X-Received: by 2002:a17:902:8642:b029:12c:dda2:30c3 with SMTP id y2-20020a1709028642b029012cdda230c3mr8989635plt.84.1628420171099; Sun, 08 Aug 2021 03:56:11 -0700 (PDT) Received: from localhost.localdomain ([103.133.123.69]) by smtp.googlemail.com with ESMTPSA id r18sm20917405pgk.54.2021.08.08.03.56.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 08 Aug 2021 03:56:10 -0700 (PDT) From: Shubhanshu Saxena To: ffmpeg-devel@ffmpeg.org Date: Sun, 8 Aug 2021 16:25:33 +0530 Message-Id: <20210808105539.18822-3-shubhanshu.e01@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210808105539.18822-1-shubhanshu.e01@gmail.com> References: <20210808105539.18822-1-shubhanshu.e01@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v3 3/9] [GSoC] lavfi/dnn_backend_tf: TFInferRequest Execution and Documentation X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Shubhanshu Saxena Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: UuermNQl0qKl This commit adds a function for execution of TFInferRequest and documentation for functions related to TFInferRequest. Signed-off-by: Shubhanshu Saxena --- libavfilter/dnn/dnn_backend_tf.c | 45 ++++++++++++++++++++++++++++++++ 1 file changed, 45 insertions(+) diff --git a/libavfilter/dnn/dnn_backend_tf.c b/libavfilter/dnn/dnn_backend_tf.c index ac609ab9cb..939afec41d 100644 --- a/libavfilter/dnn/dnn_backend_tf.c +++ b/libavfilter/dnn/dnn_backend_tf.c @@ -95,6 +95,13 @@ static void free_buffer(void *data, size_t length) av_freep(&data); } +/** + * Free the contents of TensorFlow inference request. + * It does not free the TFInferRequest instance. + * + * @param request pointer to TFInferRequest instance. + * NULL pointer is allowed. + */ static void tf_free_request(TFInferRequest *request) { if (!request) @@ -117,6 +124,12 @@ static void tf_free_request(TFInferRequest *request) } } +/** + * Create a TensorFlow inference request. All properties + * are initially unallocated and set as NULL. + * + * @return pointer to the allocated TFInferRequest instance. + */ static TFInferRequest *tf_create_inference_request(void) { TFInferRequest *infer_request = av_malloc(sizeof(TFInferRequest)); @@ -127,6 +140,38 @@ static TFInferRequest *tf_create_inference_request(void) return infer_request; } +/** + * Start synchronous inference for the TensorFlow model. + * + * @param request pointer to the TFRequestItem for inference + * @retval DNN_SUCCESS if execution is successful + * @retval DNN_ERROR if execution fails + */ +static DNNReturnType tf_start_inference(void *args) +{ + TFRequestItem *request = args; + TFInferRequest *infer_request = request->infer_request; + InferenceItem *inference = request->inference; + TaskItem *task = inference->task; + TFModel *tf_model = task->model; + + if (!request) { + av_log(&tf_model->ctx, AV_LOG_ERROR, "TFRequestItem is NULL\n"); + return DNN_ERROR; + } + + TF_SessionRun(tf_model->session, NULL, + infer_request->tf_input, &infer_request->input_tensor, 1, + infer_request->tf_outputs, infer_request->output_tensors, + task->nb_output, NULL, 0, NULL, + tf_model->status); + if (TF_GetCode(tf_model->status) != TF_OK) { + av_log(&tf_model->ctx, AV_LOG_ERROR, "%s", TF_Message(tf_model->status)); + return DNN_ERROR; + } + return DNN_SUCCESS; +} + static DNNReturnType extract_inference_from_task(TaskItem *task, Queue *inference_queue) { TFModel *tf_model = task->model;