From patchwork Wed Aug 4 11:51:35 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shubhanshu Saxena X-Patchwork-Id: 29236 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a6b:6c0f:0:0:0:0:0 with SMTP id a15csp3308099ioh; Wed, 4 Aug 2021 04:59:17 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzOlzQ6IQtJyPE4gRhHuHaIJbEjfZYhP/azpeyvxESgV0oUeyv3jB7KAnyVWcI3BKaGuNlD X-Received: by 2002:a17:907:9d2:: with SMTP id bx18mr8396215ejc.117.1628078356844; Wed, 04 Aug 2021 04:59:16 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1628078356; cv=none; d=google.com; s=arc-20160816; b=lztH29bQZOnXrsPdX1O+Elbx+06CQQpEEhd30RcBG4vOkytX+WFy3CVt0OEtIThxqO ZlB11qSeD9T7DejgG/6g4Xuq44QSx1qrGXB9eBoLYlTAq9+cOUgW7T5EhpZtgrqOf+Wv q+WrUDN7TvJvTNVJPeQnVcRe9cRK+Y1VhEYwUVgMHR/MbK5b83xNucofXRfPtxZRutBV +LZq6kjmzyAfLX0bXr+5VFCbEyZgjCwPeQp4EEA6zBhtzZ0DTzCbejYkozW4RohXybCv kQMVURPzrBU0i6fmMlXOefDWFMH3q2ILVJEAA1w3SQ0HUASHSNuRaYaYizOJSqsAfeNj 0SbA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=PquoaGuVaabw19GMBimaFa/zV1bHx/p/nPndb+VBQ9c=; b=f0ECIfQylskEeqvg5RXwTtThZg7oYdxTm/bcZTZqWnrSfbIWcg2iT/bDkREwhoh1ko s2WO0iV7QzGiCU98aXebuHR8Lq0iGKiivOEQs+GJM34RXZX9WIY+na1xnCYeMaO0OXHt 1FWnyTy/bEWKVAG1qIJ4TLBu/Ams8gqDruW6TZLeD1dnP8wOTozmsibV2JAwXk5H0XeI EhF7ZIrPGqUHMrbsCEVqWFLDxufOFHOLTM5y+jUjEgHBLpucDYI/GHrO5JqZq6nCGHnj mj2Uz7DGKSY5KTihviBf/Vaeo7VjlvjGZvGNxxhAzAxGsogXf9n41sVKT+RSl7awis+E 42rA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b=N6g3Z14M; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id q13si298493ejz.268.2021.08.04.04.59.16; Wed, 04 Aug 2021 04:59:16 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b=N6g3Z14M; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id DFC2368A1AB; Wed, 4 Aug 2021 14:59:10 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pj1-f41.google.com (mail-pj1-f41.google.com [209.85.216.41]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 95F736807DC for ; Wed, 4 Aug 2021 14:59:04 +0300 (EEST) Received: by mail-pj1-f41.google.com with SMTP id m10-20020a17090a34cab0290176b52c60ddso3152905pjf.4 for ; Wed, 04 Aug 2021 04:59:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=vauU9bb9s1tBNjhXdaNfdF9AKP8/RU27dqP7XBh53/Q=; b=N6g3Z14MYn7acY2ITm/J89d0Zq2oGEFQmUwo+ZvKL3gryFB7p5HbA8irRu9a2Z1OUs 4ZQVnrsAZMXpP8m2MJoYiEYZ8xRlCRnIYP5HzfNYalp9KYE4Ni5OgFpCzsmEkr2JlzhM Sjh5Pz6x4wZu5oFis+zsbJTBHrWEKjGDe3xm/+c3lZq/Tt9X+fcue6Hn/rI1lINRd9Y3 MjW0+ZyArYzTJwu7FdFUyHY54FcUl3fkHiDyrnRrCa1AQ1TtR1oHYA45bXqABAM8t65C Fe3kvgIkXe6GLDhEdA5xtPiU5rX8FCWbqwVX19kd5ktqT6svzcOdrwgoOLovAUXH9MR1 fDRQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=vauU9bb9s1tBNjhXdaNfdF9AKP8/RU27dqP7XBh53/Q=; b=EgY83Tzs0LcXWcqbNewatfvjhTr8uNrZS3XZWD5C+ydQdZa7v6/Qo/+MAeISwwzhrV qCHIPZ6LBV6BuIDWLZAHn8XrHdbgSp4ZkwQP5Qqv85Wv6pxK6ysLDL64g38bL3LIHH3w 492UaAKjzQ2bx4h+lqw49FKTTINWH5rG379mnZPRVZZX+NspVUMDWu4akueoZnaDALFg uhcyQGq3FcFkCIAwHR9fhDrjLMHtrt9Nq3Ji7AzIhrbrbIxmcycC7BUQUQJWbXxsZxZu USw5lvMEoVCwVrB9pM8G1jhQ2vizMVmO6hnH5OKJdow28CV+Ink6WhA7bgUlod/jNKG+ H3bw== X-Gm-Message-State: AOAM531ogFqzduw/it1WcQ9PhrPpP7g4LUFcPei5K6y5gCXZpdm8IBg8 zQF6DANo+elDK5DI60bpEU10WGxqq3CARw== X-Received: by 2002:a63:1656:: with SMTP id 22mr335628pgw.163.1628077927387; Wed, 04 Aug 2021 04:52:07 -0700 (PDT) Received: from Pavilion-x360.bbrouter ([103.133.121.9]) by smtp.googlemail.com with ESMTPSA id ms8sm6040305pjb.36.2021.08.04.04.52.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 04 Aug 2021 04:52:07 -0700 (PDT) From: Shubhanshu Saxena To: ffmpeg-devel@ffmpeg.org Date: Wed, 4 Aug 2021 17:21:35 +0530 Message-Id: <20210804115138.64475-5-shubhanshu.e01@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210804115138.64475-1-shubhanshu.e01@gmail.com> References: <20210804115138.64475-1-shubhanshu.e01@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH V2 5/8] [GSoC] lavfi/dnn_backend_tf: Error Handling for execute_model_tf X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Shubhanshu Saxena Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: hCjswgMwU7sC This patch adds error handling for cases where the execute_model_tf fails, clears the used memory in the TFRequestItem and finally pushes it back to the request queue. Signed-off-by: Shubhanshu Saxena --- libavfilter/dnn/dnn_backend_tf.c | 52 ++++++++++++++++++++++---------- 1 file changed, 36 insertions(+), 16 deletions(-) diff --git a/libavfilter/dnn/dnn_backend_tf.c b/libavfilter/dnn/dnn_backend_tf.c index d3658c3308..102e91a667 100644 --- a/libavfilter/dnn/dnn_backend_tf.c +++ b/libavfilter/dnn/dnn_backend_tf.c @@ -172,6 +172,24 @@ static DNNReturnType tf_start_inference(void *args) return DNN_SUCCESS; } +/** + * Free the TFRequestItem completely. + * + * @param arg Address of the TFInferRequest instance. + */ +static inline void destroy_request_item(TFRequestItem **arg) { + TFRequestItem *request; + if (!arg) { + return; + } + request = *arg; + tf_free_request(request->infer_request); + av_freep(&request->infer_request); + av_freep(&request->inference); + ff_dnn_async_module_cleanup(&request->exec_module); + av_freep(arg); +} + static DNNReturnType extract_inference_from_task(TaskItem *task, Queue *inference_queue) { TFModel *tf_model = task->model; @@ -880,6 +898,7 @@ DNNModel *ff_dnn_load_model_tf(const char *model_filename, DNNFunctionType func_ if (!item) { goto err; } + item->inference = NULL; item->infer_request = tf_create_inference_request(); if (!item->infer_request) { av_log(ctx, AV_LOG_ERROR, "Failed to allocate memory for TensorFlow inference request\n"); @@ -891,8 +910,7 @@ DNNModel *ff_dnn_load_model_tf(const char *model_filename, DNNFunctionType func_ item->exec_module.args = item; if (ff_safe_queue_push_back(tf_model->request_queue, item) < 0) { - av_freep(&item->infer_request); - av_freep(&item); + destroy_request_item(&item); goto err; } } @@ -1059,8 +1077,7 @@ err: av_freep(&outputs); if (ff_safe_queue_push_back(tf_model->request_queue, request) < 0) { - av_freep(&request->infer_request); - av_freep(&request); + destroy_request_item(&request); av_log(ctx, AV_LOG_ERROR, "Failed to push back request_queue.\n"); } } @@ -1072,28 +1089,35 @@ static DNNReturnType execute_model_tf(TFRequestItem *request, Queue *inference_q InferenceItem *inference; TaskItem *task; - inference = ff_queue_peek_front(inference_queue); - if (!inference) { - av_log(NULL, AV_LOG_ERROR, "Failed to get inference item\n"); - return DNN_ERROR; + if (ff_queue_size(inference_queue) == 0) { + destroy_request_item(&request); + return DNN_SUCCESS; } + + inference = ff_queue_peek_front(inference_queue); task = inference->task; tf_model = task->model; ctx = &tf_model->ctx; if (fill_model_input_tf(tf_model, request) != DNN_SUCCESS) { - return DNN_ERROR; + goto err; } if (task->async) { return ff_dnn_start_inference_async(ctx, &request->exec_module); } else { if (tf_start_inference(request) != DNN_SUCCESS) { - return DNN_ERROR; + goto err; } infer_completion_callback(request); return (task->inference_done == task->inference_todo) ? DNN_SUCCESS : DNN_ERROR; } +err: + tf_free_request(request->infer_request); + if (ff_safe_queue_push_back(tf_model->request_queue, request) < 0) { + destroy_request_item(&request); + } + return DNN_ERROR; } DNNReturnType ff_dnn_execute_model_tf(const DNNModel *model, DNNExecBaseParams *exec_params) @@ -1193,8 +1217,7 @@ DNNReturnType ff_dnn_flush_tf(const DNNModel *model) if (ret != DNN_SUCCESS) { av_log(ctx, AV_LOG_ERROR, "Failed to fill model input.\n"); if (ff_safe_queue_push_back(tf_model->request_queue, request) < 0) { - av_freep(&request->infer_request); - av_freep(&request); + destroy_request_item(&request); } return ret; } @@ -1210,10 +1233,7 @@ void ff_dnn_free_model_tf(DNNModel **model) tf_model = (*model)->model; while (ff_safe_queue_size(tf_model->request_queue) != 0) { TFRequestItem *item = ff_safe_queue_pop_front(tf_model->request_queue); - ff_dnn_async_module_cleanup(&item->exec_module); - tf_free_request(item->infer_request); - av_freep(&item->infer_request); - av_freep(&item); + destroy_request_item(&item); } ff_safe_queue_destroy(tf_model->request_queue);