From patchwork Sat Jul 31 05:35:11 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shubhanshu Saxena X-Patchwork-Id: 29150 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a5d:965a:0:0:0:0:0 with SMTP id d26csp143702ios; Fri, 30 Jul 2021 22:59:49 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyJsgRAkgzXZkeii0nV25g1/FMUXX0a9mrx27CYluQ/D7V7d9JOrGuVNXwiFNjg94U3ofSv X-Received: by 2002:a17:906:86c4:: with SMTP id j4mr6207762ejy.431.1627711189758; Fri, 30 Jul 2021 22:59:49 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1627711189; cv=none; d=google.com; s=arc-20160816; b=OC8n7o1atmdJ/6U0Q3742E5O3AmVpo929RMivmjq/WzAQpmBbzxmTrwmtDQTDfUGuR kFFiWi+oJD7eLoyEZAwwE7Q2DgkI6xw5i2y60YsS0LixuCEVAHnTzTbKjdZmpdf8TBcq zygC+zfsFxDODXycavO4SeftWCaJfXwvifnHGl5d7z136ZS+lbt5dxcxugw5jRAVoYlO Y0HEXaQuH5nGEXCny5MtoT4hcb21iXhuf7XSZiE68/HJLNq/8REkqbNMDc00MHqTnJxP bbX/nB7WxGkB+Jeya167ThBYnQbx29M3Yfud9OAHg17H3rjcS0xsQYdEq4sF/DCIVBGa Y3GQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=gZq5amk4Fenj6Z1yPnJLFOm0T+gDdQEBhnvRZMDobOw=; b=EBKxssXNhlCNBnka7aEjP8AVqLcJrvW1gn76UORLEPsO2lknBnIw/4p6WedSrw68XP CI7rsj9xbOHkk4ALAwjkjF9p43pfIkt9Jgqkhpkdk7XLxPeYYA4y3LbiOQOLSMzmT9Go EAH3jPZ1/SL9b/LJwS3jVEcAt9KTBJWzb2Ovs9UK2U14M0D2nnkocX7iJAUgHqM3RXjV Z7grkalhMFaj08uKS7jCDGMxzLmY2b7DVVvJ+QY1jO1CNNpyZG6iH/6RdyOH2H5fv3Wu m8tk6PmZ+6rhBKLjdMsDk6zeZfsS4WqIxg87f96huIw9hNpYiQhiQhkhJb0qbGDsYbfD 16Pg== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b=hfWHOY2B; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id b11si3918912ejc.345.2021.07.30.22.59.49; Fri, 30 Jul 2021 22:59:49 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b=hfWHOY2B; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 1424368A3BF; Sat, 31 Jul 2021 08:59:46 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pj1-f45.google.com (mail-pj1-f45.google.com [209.85.216.45]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id A31126808F2 for ; Sat, 31 Jul 2021 08:59:39 +0300 (EEST) Received: by mail-pj1-f45.google.com with SMTP id mz5-20020a17090b3785b0290176ecf64922so23850039pjb.3 for ; Fri, 30 Jul 2021 22:59:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=cmzj2KvWrvFM25vmhuO/8auwNOwo3QxtZvI7+o3Mr1g=; b=hfWHOY2B/kUX6uAXpPGQ5ES8h5aydjCxiY9mMBjgWJP+AuZMuRkQmyTlPSmPxGBj1L /6jrWnirJI4Zw6F8PGr2d4K0C8fkP4LDccx6Pqjt5Og9V/Zh5b27RVEyem7bSyMgfbCI xNdHocnGLM8zjQVCKo/pLDi6atNSIjiOHf8ag5mwxHoCsEAm2Uzd3Am8xR574/n+3zLG qZ0Kw/3Ul27qxOyF+fU212XXEXGfoWLqw6mP3NlpKdEkAjLJu+UGSZcPBjZkLaAWv0Kj QZlEtuGHdkiayT5Bf60FOxYsOR1dqISXLeN/xrGoseQfkRhVqDwEmEOI5ZtamZ1xI6gE oRBw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=cmzj2KvWrvFM25vmhuO/8auwNOwo3QxtZvI7+o3Mr1g=; b=UQ9pjO+V87GbGoKutDt7oRfQsVytq9uAdYO8C71Z9oaBNHTTMJpd3k318iH4euG4Ck Az5W/G+BokFwCym53iKpSmqYyEgaXcQNbwU3U6sM9OKhahHo/xmX0hMts6llUzPXH3Lq hL+PrBp8/lgxwPDtUHbphpphB+MDDSM4tCNPTkvKO0EMOwiPGrvNfjSKuPtoHbzmgDdl Ot5mm4f116fKU/yumTuB2TMnr1PosHOMJmYwmJs6WS9mxklIfPFMweYir4JwnRKmDoVN Z67Z971xZxipxrRLj+s2r3nd0YW7OOzpKq7xXFYrookaBzhlqoKsD/VXVJ8Nk5zu3UBR o+jA== X-Gm-Message-State: AOAM532YZVCKh6QheY8ajx0HgPZoZQciXIp2c+EpsQ0jwTGF96ZW4MMS TUvvXG0jgmUfqHB7o9zuK5zMkGIjO0iOyg== X-Received: by 2002:a63:e24a:: with SMTP id y10mr5348686pgj.176.1627709759752; Fri, 30 Jul 2021 22:35:59 -0700 (PDT) Received: from localhost.localdomain ([103.157.220.250]) by smtp.googlemail.com with ESMTPSA id p34sm4312266pfh.172.2021.07.30.22.35.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Jul 2021 22:35:59 -0700 (PDT) From: Shubhanshu Saxena To: ffmpeg-devel@ffmpeg.org Date: Sat, 31 Jul 2021 11:05:11 +0530 Message-Id: <20210731053516.7700-3-shubhanshu.e01@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210731053516.7700-1-shubhanshu.e01@gmail.com> References: <20210731053516.7700-1-shubhanshu.e01@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 3/8] lavfi/dnn_backend_tf: TFInferRequest Execution and Documentation X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Shubhanshu Saxena Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: zjyKX1AnfpjE This commit adds a function for execution of TFInferRequest and documentation for functions related to TFInferRequest. Signed-off-by: Shubhanshu Saxena --- libavfilter/dnn/dnn_backend_tf.c | 45 ++++++++++++++++++++++++++++++++ 1 file changed, 45 insertions(+) diff --git a/libavfilter/dnn/dnn_backend_tf.c b/libavfilter/dnn/dnn_backend_tf.c index 6443c2fd1d..805a0328b6 100644 --- a/libavfilter/dnn/dnn_backend_tf.c +++ b/libavfilter/dnn/dnn_backend_tf.c @@ -94,6 +94,13 @@ static void free_buffer(void *data, size_t length) av_freep(&data); } +/** + * Free the contents of TensorFlow inference request. + * It does not free the TFInferRequest instance. + * + * @param request pointer to TFInferRequest instance. + * NULL pointer is allowed. + */ static void tf_free_request(TFInferRequest *request) { if (!request) @@ -116,6 +123,12 @@ static void tf_free_request(TFInferRequest *request) } } +/** + * Create a TensorFlow inference request. All properties + * are initially unallocated and set as NULL. + * + * @return pointer to the allocated TFInferRequest instance. + */ static TFInferRequest *tf_create_inference_request(void) { TFInferRequest *infer_request = av_malloc(sizeof(TFInferRequest)); @@ -126,6 +139,38 @@ static TFInferRequest *tf_create_inference_request(void) return infer_request; } +/** + * Start synchronous inference for the TensorFlow model. + * + * @param request pointer to the TFRequestItem for inference + * @retval DNN_SUCCESS if execution is successful + * @retval DNN_ERROR if execution fails + */ +static DNNReturnType tf_start_inference(void *args) +{ + TFRequestItem *request = args; + TFInferRequest *infer_request = request->infer_request; + InferenceItem *inference = request->inference; + TaskItem *task = inference->task; + TFModel *tf_model = task->model; + + if (!request) { + av_log(&tf_model->ctx, AV_LOG_ERROR, "TFRequestItem is NULL\n"); + return DNN_ERROR; + } + + TF_SessionRun(tf_model->session, NULL, + infer_request->tf_input, &infer_request->input_tensor, 1, + infer_request->tf_outputs, infer_request->output_tensors, + task->nb_output, NULL, 0, NULL, + tf_model->status); + if (TF_GetCode(tf_model->status) != TF_OK) { + av_log(&tf_model->ctx, AV_LOG_ERROR, "%s", TF_Message(tf_model->status)); + return DNN_ERROR; + } + return DNN_SUCCESS; +} + static DNNReturnType extract_inference_from_task(TaskItem *task, Queue *inference_queue) { TFModel *tf_model = task->model;