From patchwork Mon Jul 5 10:30:58 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shubhanshu Saxena X-Patchwork-Id: 28780 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a5d:965a:0:0:0:0:0 with SMTP id d26csp4026084ios; Mon, 5 Jul 2021 03:31:52 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyMRFhVo8xL789x5Xoo752ISWkb2WHi0MpTFW+KG27HcIdOe19UjDix1DlxeM8lMv6F6DJF X-Received: by 2002:a05:6402:c10:: with SMTP id co16mr14928065edb.192.1625481112286; Mon, 05 Jul 2021 03:31:52 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1625481112; cv=none; d=google.com; s=arc-20160816; b=IBnoKjqoNWyES0t/nG32uxtqhXlyDVjYu4kQyqvm1nagRUSDQN7/MEZZfxHTsDsWcM ICvAErcbrZSB8rXTioAZiyhGGo/yvy5TKojX695+xpL93mXDYvSRqrIY2wJMKRUJadzo +GElsNADVVm2uPumk50DhZ1qLrVqxLuxM5dwPqve3/hSDF2EapLpe94Jk/zE6BoHKWXr 8SlIpZqBa488AeqLPs3DmCtZ95e2CffO5RyJk5GZW0XCy73rEmfB7pcsgVAqaPLozCTI OtOZzhfKXL56XTqm2fV7N8yzMEaHDsAou6DmbeD8izXlumVHLUEDyAK88Tz3q+4KPMUy It5w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=mIMhwtjHiVHbpK0BhayJoeUOgQfwKS9uvXqCWlZVIZg=; b=fBwL85JYml8MgLOpcb8YxgNY5bsQj7ZD/8UDu+FO1KVjzJovwrbnb+MT3xO9rkGe7/ T25o4Lo3pFuSsVPcbxnox8Pf9SA+2aD8suUUOtt8UBtSGt746/2zJDbT0rK+tJiqB8ts b5LpQEKbRqHihWGk92Tnpz7VUY7+u+Ksvf7aA9uy6PJrf43FzWMhg+qOJ2Gw+FR9jS99 tabdrzqVFgt9dWvaqJcjwsFjjSQGlrws7IQJ3Akj1Vc/a2VwURmOocnpuinsuYJ0dIiz ONEMXDu928c3CaiYyQYYbTPOw4Rggvx53URxBSTJMxIIV1mYuaSw++GrnEmFYOW8l5f7 ytMA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b=IhmCiI1O; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 23si12041251ejg.6.2021.07.05.03.31.51; Mon, 05 Jul 2021 03:31:52 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b=IhmCiI1O; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 94C4E68A5EC; Mon, 5 Jul 2021 13:31:46 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pl1-f175.google.com (mail-pl1-f175.google.com [209.85.214.175]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id B9F5D68A65A for ; Mon, 5 Jul 2021 13:31:39 +0300 (EEST) Received: by mail-pl1-f175.google.com with SMTP id i13so9976046plb.10 for ; Mon, 05 Jul 2021 03:31:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=OLi0SAuWaDS7Gdtroll0XRBFMltoJ2tkXwM36oYVMKs=; b=IhmCiI1OBTbV2g10elGRnsPISd1T5GiGRl/iNo9EerlV4MqNSCW24z80gcoiLJuh4d yFwyHhC/i7PBEJg2vvRsf4lVRyoyd5o0DRLD3Me/E9VU4ktofddjIsPx6o9wWYuUmARg ne3+3yw8dnqq6uqEcBlsSdoQeKoxhLEkVp5Kz2MRqZRnJy5fOp1GZEsx+Jh1agHyqElp IsCvpXe3jOHqLvyyhFShWtRDov7MtZGOohjMGNA8l4iAlEuNh76362tz/gIf5OuSPPX7 mN0CpvS2Yqf2FbkKPDpTHo47KhmwVV3fGhfsaJSo+kFEchQwWXUHQri11WblxdcRKFVC A35A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=OLi0SAuWaDS7Gdtroll0XRBFMltoJ2tkXwM36oYVMKs=; b=BminwVeCMFNLWDXLejnlLwLkwtUEbGEmCP1NU7HjBrfQpMp+0FW5i2cRqh/njr2V8t xOLlCCa9QcKFG+5+Rjj6ZWO6ESXXoD1IqGDM7CGivtjOBZyqVmLMISdLG4KQmtCJnfjH tan5lHPh9+q6FDvipO9SPAQ0nyS/q8X7ooEHYSavHbc8WG/yQPWodqidrJVbVGoz9hZR kr8ykdDYh2xq6Jnt8vRpYDEEvhYtHFS5oElMVu5Swn+u0+inXWijmpy7voauF+viabAg jzE1Xv8S0jAojKpjJVVuhs60BsYfW7A6e6dU2lkOSWJ181Lfmf2e+gGJT70zuhmfoaAM DEGg== X-Gm-Message-State: AOAM532/rsXfsbDD4akTiQ7VZa+ouwVgwSzfhxs/NPmQ5y+z4LymD63t een9fQLwCiqQ6oqiR/Gb2atu2TcXDbHVmA== X-Received: by 2002:a17:90a:f68f:: with SMTP id cl15mr14350137pjb.219.1625481097190; Mon, 05 Jul 2021 03:31:37 -0700 (PDT) Received: from Pavilion-x360.bbrouter ([103.133.121.113]) by smtp.googlemail.com with ESMTPSA id cx5sm13174242pjb.1.2021.07.05.03.31.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 05 Jul 2021 03:31:36 -0700 (PDT) From: Shubhanshu Saxena To: ffmpeg-devel@ffmpeg.org Date: Mon, 5 Jul 2021 16:00:58 +0530 Message-Id: <20210705103057.42309-6-shubhanshu.e01@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210705103057.42309-1-shubhanshu.e01@gmail.com> References: <20210705103057.42309-1-shubhanshu.e01@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH V2 6/6] lavfi/dnn_backend_tf: Error Handling X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Shubhanshu Saxena Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: a7WSC5fJXUB9 This commit adds handling for cases where an error may occur, clearing the allocated memory resources. Signed-off-by: Shubhanshu Saxena --- libavfilter/dnn/dnn_backend_tf.c | 80 ++++++++++++++++++++++++-------- 1 file changed, 60 insertions(+), 20 deletions(-) diff --git a/libavfilter/dnn/dnn_backend_tf.c b/libavfilter/dnn/dnn_backend_tf.c index 6664d7194b..1d4ca516c1 100644 --- a/libavfilter/dnn/dnn_backend_tf.c +++ b/libavfilter/dnn/dnn_backend_tf.c @@ -127,14 +127,18 @@ static TFInferRequest *tf_create_inference_request(void) static DNNReturnType extract_inference_from_task(TaskItem *task, Queue *inference_queue) { + TFModel *tf_model = task->model; + TFContext *ctx = &tf_model->ctx; InferenceItem *inference = av_malloc(sizeof(*inference)); if (!inference) { + av_log(ctx, AV_LOG_ERROR, "Unable to allocate space for InferenceItem\n"); return DNN_ERROR; } task->inference_todo = 1; task->inference_done = 0; inference->task = task; if (ff_queue_push_back(inference_queue, inference) < 0) { + av_log(ctx, AV_LOG_ERROR, "Failed to push back inference_queue.\n"); av_freep(&inference); return DNN_ERROR; } @@ -245,14 +249,15 @@ static DNNReturnType get_output_tf(void *model, const char *input_name, int inpu if (!in_frame) { av_log(ctx, AV_LOG_ERROR, "Failed to allocate memory for input frame\n"); - return DNN_ERROR; + ret = DNN_ERROR; + goto err; } out_frame = av_frame_alloc(); if (!out_frame) { av_log(ctx, AV_LOG_ERROR, "Failed to allocate memory for output frame\n"); - av_frame_free(&in_frame); - return DNN_ERROR; + ret = DNN_ERROR; + goto err; } in_frame->width = input_width; @@ -269,19 +274,22 @@ static DNNReturnType get_output_tf(void *model, const char *input_name, int inpu if (extract_inference_from_task(&task, tf_model->inference_queue) != DNN_SUCCESS) { av_log(ctx, AV_LOG_ERROR, "unable to extract inference from task.\n"); - return DNN_ERROR; + ret = DNN_ERROR; + goto err; } request = ff_safe_queue_pop_front(tf_model->request_queue); if (!request) { av_log(ctx, AV_LOG_ERROR, "unable to get infer request.\n"); - return DNN_ERROR; + ret = DNN_ERROR; + goto err; } ret = execute_model_tf(request, tf_model->inference_queue); *output_width = out_frame->width; *output_height = out_frame->height; +err: av_frame_free(&out_frame); av_frame_free(&in_frame); return ret; @@ -802,17 +810,12 @@ DNNModel *ff_dnn_load_model_tf(const char *model_filename, DNNFunctionType func_ av_opt_set_defaults(ctx); if (av_opt_set_from_string(ctx, options, NULL, "=", "&") < 0) { av_log(ctx, AV_LOG_ERROR, "Failed to parse options \"%s\"\n", options); - av_freep(&tf_model); - av_freep(&model); - return NULL; + goto err; } if (load_tf_model(tf_model, model_filename) != DNN_SUCCESS){ if (load_native_model(tf_model, model_filename) != DNN_SUCCESS){ - av_freep(&tf_model); - av_freep(&model); - - return NULL; + goto err; } } @@ -821,14 +824,34 @@ DNNModel *ff_dnn_load_model_tf(const char *model_filename, DNNFunctionType func_ } tf_model->request_queue = ff_safe_queue_create(); + if (!tf_model->request_queue) { + goto err; + } for (int i = 0; i < ctx->options.nireq; i++) { TFRequestItem *item = av_mallocz(sizeof(*item)); + if (!item) { + goto err; + } item->infer_request = tf_create_inference_request(); - ff_safe_queue_push_back(tf_model->request_queue, item); + if (!item->infer_request) { + av_log(ctx, AV_LOG_ERROR, "Failed to allocate memory for TensorFlow inference request\n"); + av_freep(&item); + goto err; + } + + if (ff_safe_queue_push_back(tf_model->request_queue, item) < 0) { + av_freep(&item->infer_request); + av_freep(&item); + goto err; + } } tf_model->inference_queue = ff_queue_create(); + if (!tf_model->inference_queue) { + goto err; + } + model->model = tf_model; model->get_input = &get_input_tf; model->get_output = &get_output_tf; @@ -837,6 +860,9 @@ DNNModel *ff_dnn_load_model_tf(const char *model_filename, DNNFunctionType func_ model->func_type = func_type; return model; +err: + ff_dnn_free_model_tf(&model); + return NULL; } static DNNReturnType fill_model_input_tf(TFModel *tf_model, TFRequestItem *request) { @@ -851,24 +877,31 @@ static DNNReturnType fill_model_input_tf(TFModel *tf_model, TFRequestItem *reque task = inference->task; request->inference = inference; - if (get_input_tf(tf_model, &input, task->input_name) != DNN_SUCCESS) - return DNN_ERROR; + if (get_input_tf(tf_model, &input, task->input_name) != DNN_SUCCESS) { + goto err; + } infer_request = request->infer_request; input.height = task->in_frame->height; input.width = task->in_frame->width; infer_request->tf_input = av_malloc(sizeof(TF_Output)); + if (!infer_request->tf_input) { + av_log(ctx, AV_LOG_ERROR, "Failed to allocate memory for input tensor\n"); + goto err; + } + infer_request->tf_input->oper = TF_GraphOperationByName(tf_model->graph, task->input_name); if (!infer_request->tf_input->oper){ av_log(ctx, AV_LOG_ERROR, "Could not find \"%s\" in model\n", task->input_name); - return DNN_ERROR; + goto err; } infer_request->tf_input->index = 0; + infer_request->input_tensor = allocate_input_tensor(&input); if (!infer_request->input_tensor){ av_log(ctx, AV_LOG_ERROR, "Failed to allocate memory for input tensor\n"); - return DNN_ERROR; + goto err; } input.data = (float *)TF_TensorData(infer_request->input_tensor); @@ -893,13 +926,13 @@ static DNNReturnType fill_model_input_tf(TFModel *tf_model, TFRequestItem *reque infer_request->tf_outputs = av_malloc_array(task->nb_output, sizeof(TF_Output)); if (infer_request->tf_outputs == NULL) { av_log(ctx, AV_LOG_ERROR, "Failed to allocate memory for *tf_outputs\n"); - return DNN_ERROR; + goto err; } infer_request->output_tensors = av_mallocz_array(task->nb_output, sizeof(*infer_request->output_tensors)); if (!infer_request->output_tensors) { av_log(ctx, AV_LOG_ERROR, "Failed to allocate memory for output tensor\n"); - return DNN_ERROR; + goto err; } for (int i = 0; i < task->nb_output; ++i) { @@ -907,12 +940,15 @@ static DNNReturnType fill_model_input_tf(TFModel *tf_model, TFRequestItem *reque infer_request->tf_outputs[i].oper = TF_GraphOperationByName(tf_model->graph, task->output_names[i]); if (!infer_request->tf_outputs[i].oper) { av_log(ctx, AV_LOG_ERROR, "Could not find output \"%s\" in model\n", task->output_names[i]); - return DNN_ERROR; + goto err; } infer_request->tf_outputs[i].index = 0; } return DNN_SUCCESS; +err: + tf_free_request(infer_request); + return DNN_ERROR; } static void infer_completion_callback(void *args) { @@ -983,6 +1019,10 @@ static DNNReturnType execute_model_tf(TFRequestItem *request, Queue *inference_q TaskItem *task; inference = ff_queue_peek_front(inference_queue); + if (!inference) { + av_log(NULL, AV_LOG_ERROR, "Failed to get inference item\n"); + return DNN_ERROR; + } task = inference->task; tf_model = task->model; ctx = &tf_model->ctx;