From patchwork Mon Jul 5 10:30:55 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shubhanshu Saxena X-Patchwork-Id: 28785 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a5d:965a:0:0:0:0:0 with SMTP id d26csp4042383ios; Mon, 5 Jul 2021 03:56:22 -0700 (PDT) X-Google-Smtp-Source: ABdhPJw6SP0Og0yATGsZS1tfo5e8EYuDC8Lh4yu/mLsAHctfZ5IgbCVS9wmOr52cseDYzrgDr/Xf X-Received: by 2002:a17:907:2ce1:: with SMTP id hz1mr12473538ejc.376.1625482581987; Mon, 05 Jul 2021 03:56:21 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1625482581; cv=none; d=google.com; s=arc-20160816; b=UyImzY7XF+Tu3TWq9zsC35jrWreRF4DUxeLDeUeJVADZx+jS4iO9c1IDv7W3lOx0Hw YoEYpDV66Oc5RsOmwpOiN+TGDgNLOQrsYgTK/yKtftAzaVkeSJ8ZRtpZ/CudRBP6Rfvt XM6MXPTI5Rm4DonkhmJojHiOvNET3zCjNUiL1mWSkoaXg+TSfJDJ4puAP+5juD5iKolc q3aurNw2A9bjsTeLlVXtbnJ2hkLsdkhSOQISNEB4Mje7NTskqxmVeIEFiIxV0kPehC8U ki4Aw6R6mrXdQUGjM8UTH1WJR6tIxs4X4Vo28MO5H2aCgJVo8wV8XBbDXIL3TlRtUTD6 zkVw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=re+dprN96msf+Gdm9Xs7Rk1DPh2ULmJUlEuYnCaj3+w=; b=y7M59O0p8XocOvDiwSzPw1sHnx0IuDiRoz86m5hrJGO7KSmyVXYrTpNBdnI07p2wOA 9veUIzb66w/fgRCtiFKdjbjHH7Ukz52o+h4BZyWQ6KB/5GHxTwdvwGA7GJlWHDIklxmA G/qmN4Z1FsSKW/ds2j55PlY8EfVObTu5KrhIEp1w9rMuOwBncaS2BjysVN0vDewCd935 ivlXjIbKwncDOTDsdpXNWlLU0iL2ng5t+1wlHbjLThzwLwIE6Fsiawfcwr7boNunx2fV hB90hNCSV70/xc5mOrwK9nVGqrIrHxw1wOAtBSUioKa8vW7mp8UfGsEVS5AMgGtGlGgX 6rOg== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b=G1S4c578; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id co5si10812928edb.500.2021.07.05.03.56.21; Mon, 05 Jul 2021 03:56:21 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b=G1S4c578; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id EB71A68A66B; Mon, 5 Jul 2021 13:56:17 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pf1-f172.google.com (mail-pf1-f172.google.com [209.85.210.172]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 64AC068A334 for ; Mon, 5 Jul 2021 13:56:11 +0300 (EEST) Received: by mail-pf1-f172.google.com with SMTP id w22so12721930pff.5 for ; Mon, 05 Jul 2021 03:56:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=CsuiDPyHVptShydgmMhhYcg4B0oOyOEf2isfwjRBjOc=; b=G1S4c578MO9HZnnqgQHlAhrWx8y5PmVpD8NQ4t4qAzM5ZspNWcHvGSjU7uLJQKBc7y SKyLCPrCN91TiV7nL7xD+hHZHTfPHPDaayIaMDi1igjz1aMBakxywKo81mXoaemXfzKN R1SqfCRcD/VQVKpuDjDzGcC/JQBUm3D8VLjL8wHd4ECQnMA4mxUs2M7DifSA4M/39a9a YSnnV95uBrfwLmWPAXfg8l7VGPz1oyS7kyPjp7dCB1LquT8RVOGWifgXpMPTSuSU24fc dEmn7NrH6wnE0lgLOjVX9UPKjQ2d+hVznZwptdAb1sRzsYay7vqFeW1f5wpQbEXchoMU 3Gag== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=CsuiDPyHVptShydgmMhhYcg4B0oOyOEf2isfwjRBjOc=; b=AdNi5apXu5KhGAC7Vml0EsacSLXIQY2AhG8+uu6FfFTNn5BRqkd8Pra2mAL1qgM9F9 ilws7LfCna7vp0obVIHJdwhS3s3WspnMH2a0V3WJaU9DX3aY3J9Y/eFRvb71iG/bLeVQ Za6hY+bpI31ti1rsjUZ/dWmS3C/t2EDsZGjkdUQDJnv9AxT2yRArSiq8G/iMAuzgDAP7 SFAOzeJmxIkjCj9ZxtJjg+uiRDsvzp+v+kP8pQiaXVgZ9lJZjUQ7TNwgM+mQGaCtljmZ cEH0A3kmnz5R1XA2rHmnv7WhQ/I8HZMVWA+8jwGBQ7P1biEei6DDMSVR48fSrzq0qExG x/lw== X-Gm-Message-State: AOAM532qCpu6Jcy9+6L8GvP/7IrP059DUFWtYNdgnNzxkQNVDyycGc2u ZA+tXyCdLlmKJt5/olnYSUC7oHOkHhMO0A== X-Received: by 2002:a63:383:: with SMTP id 125mr9799643pgd.208.1625481091429; Mon, 05 Jul 2021 03:31:31 -0700 (PDT) Received: from Pavilion-x360.bbrouter ([103.133.121.113]) by smtp.googlemail.com with ESMTPSA id cx5sm13174242pjb.1.2021.07.05.03.31.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 05 Jul 2021 03:31:31 -0700 (PDT) From: Shubhanshu Saxena To: ffmpeg-devel@ffmpeg.org Date: Mon, 5 Jul 2021 16:00:55 +0530 Message-Id: <20210705103057.42309-3-shubhanshu.e01@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210705103057.42309-1-shubhanshu.e01@gmail.com> References: <20210705103057.42309-1-shubhanshu.e01@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH V2 3/6] lavfi/dnn_backend_tf: Request-based Execution X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Shubhanshu Saxena Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: N5BRkDqQF/Jy This commit uses TFRequestItem and the existing sync execution mechanism to use request-based execution. It will help in adding async functionality to the TensorFlow backend later. Signed-off-by: Shubhanshu Saxena --- libavfilter/dnn/dnn_backend_common.h | 3 + libavfilter/dnn/dnn_backend_openvino.c | 2 +- libavfilter/dnn/dnn_backend_tf.c | 156 ++++++++++++++----------- 3 files changed, 91 insertions(+), 70 deletions(-) diff --git a/libavfilter/dnn/dnn_backend_common.h b/libavfilter/dnn/dnn_backend_common.h index df59615f40..5281fdfed1 100644 --- a/libavfilter/dnn/dnn_backend_common.h +++ b/libavfilter/dnn/dnn_backend_common.h @@ -26,6 +26,9 @@ #include "../dnn_interface.h" +#define DNN_BACKEND_COMMON_OPTIONS \ + { "nireq", "number of request", OFFSET(options.nireq), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, INT_MAX, FLAGS }, + // one task for one function call from dnn interface typedef struct TaskItem { void *model; // model for the backend diff --git a/libavfilter/dnn/dnn_backend_openvino.c b/libavfilter/dnn/dnn_backend_openvino.c index 3295fc79d3..f34b8150f5 100644 --- a/libavfilter/dnn/dnn_backend_openvino.c +++ b/libavfilter/dnn/dnn_backend_openvino.c @@ -75,7 +75,7 @@ typedef struct RequestItem { #define FLAGS AV_OPT_FLAG_FILTERING_PARAM static const AVOption dnn_openvino_options[] = { { "device", "device to run model", OFFSET(options.device_type), AV_OPT_TYPE_STRING, { .str = "CPU" }, 0, 0, FLAGS }, - { "nireq", "number of request", OFFSET(options.nireq), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, INT_MAX, FLAGS }, + DNN_BACKEND_COMMON_OPTIONS { "batch_size", "batch size per request", OFFSET(options.batch_size), AV_OPT_TYPE_INT, { .i64 = 1 }, 1, 1000, FLAGS}, { "input_resizable", "can input be resizable or not", OFFSET(options.input_resizable), AV_OPT_TYPE_BOOL, { .i64 = 0 }, 0, 1, FLAGS }, { NULL } diff --git a/libavfilter/dnn/dnn_backend_tf.c b/libavfilter/dnn/dnn_backend_tf.c index 578748eb35..e8007406c8 100644 --- a/libavfilter/dnn/dnn_backend_tf.c +++ b/libavfilter/dnn/dnn_backend_tf.c @@ -35,11 +35,13 @@ #include "dnn_backend_native_layer_maximum.h" #include "dnn_io_proc.h" #include "dnn_backend_common.h" +#include "safe_queue.h" #include "queue.h" #include typedef struct TFOptions{ char *sess_config; + uint32_t nireq; } TFOptions; typedef struct TFContext { @@ -53,6 +55,7 @@ typedef struct TFModel{ TF_Graph *graph; TF_Session *session; TF_Status *status; + SafeQueue *request_queue; Queue *inference_queue; } TFModel; @@ -77,12 +80,13 @@ typedef struct TFRequestItem { #define FLAGS AV_OPT_FLAG_FILTERING_PARAM static const AVOption dnn_tensorflow_options[] = { { "sess_config", "config for SessionOptions", OFFSET(options.sess_config), AV_OPT_TYPE_STRING, { .str = NULL }, 0, 0, FLAGS }, + DNN_BACKEND_COMMON_OPTIONS { NULL } }; AVFILTER_DEFINE_CLASS(dnn_tensorflow); -static DNNReturnType execute_model_tf(Queue *inference_queue); +static DNNReturnType execute_model_tf(TFRequestItem *request, Queue *inference_queue); static void free_buffer(void *data, size_t length) { @@ -237,6 +241,7 @@ static DNNReturnType get_output_tf(void *model, const char *input_name, int inpu AVFrame *in_frame = av_frame_alloc(); AVFrame *out_frame = NULL; TaskItem task; + TFRequestItem *request; if (!in_frame) { av_log(ctx, AV_LOG_ERROR, "Failed to allocate memory for input frame\n"); @@ -267,7 +272,13 @@ static DNNReturnType get_output_tf(void *model, const char *input_name, int inpu return DNN_ERROR; } - ret = execute_model_tf(tf_model->inference_queue); + request = ff_safe_queue_pop_front(tf_model->request_queue); + if (!request) { + av_log(ctx, AV_LOG_ERROR, "unable to get infer request.\n"); + return DNN_ERROR; + } + + ret = execute_model_tf(request, tf_model->inference_queue); *output_width = out_frame->width; *output_height = out_frame->height; @@ -771,6 +782,7 @@ DNNModel *ff_dnn_load_model_tf(const char *model_filename, DNNFunctionType func_ { DNNModel *model = NULL; TFModel *tf_model = NULL; + TFContext *ctx = NULL; model = av_mallocz(sizeof(DNNModel)); if (!model){ @@ -782,13 +794,14 @@ DNNModel *ff_dnn_load_model_tf(const char *model_filename, DNNFunctionType func_ av_freep(&model); return NULL; } - tf_model->ctx.class = &dnn_tensorflow_class; tf_model->model = model; + ctx = &tf_model->ctx; + ctx->class = &dnn_tensorflow_class; //parse options - av_opt_set_defaults(&tf_model->ctx); - if (av_opt_set_from_string(&tf_model->ctx, options, NULL, "=", "&") < 0) { - av_log(&tf_model->ctx, AV_LOG_ERROR, "Failed to parse options \"%s\"\n", options); + av_opt_set_defaults(ctx); + if (av_opt_set_from_string(ctx, options, NULL, "=", "&") < 0) { + av_log(ctx, AV_LOG_ERROR, "Failed to parse options \"%s\"\n", options); av_freep(&tf_model); av_freep(&model); return NULL; @@ -803,6 +816,18 @@ DNNModel *ff_dnn_load_model_tf(const char *model_filename, DNNFunctionType func_ } } + if (ctx->options.nireq <= 0) { + ctx->options.nireq = av_cpu_count() / 2 + 1; + } + + tf_model->request_queue = ff_safe_queue_create(); + + for (int i = 0; i < ctx->options.nireq; i++) { + TFRequestItem *item = av_mallocz(sizeof(*item)); + item->infer_request = tf_create_inference_request(); + ff_safe_queue_push_back(tf_model->request_queue, item); + } + tf_model->inference_queue = ff_queue_create(); model->model = tf_model; model->get_input = &get_input_tf; @@ -814,42 +839,42 @@ DNNModel *ff_dnn_load_model_tf(const char *model_filename, DNNFunctionType func_ return model; } -static DNNReturnType execute_model_tf(Queue *inference_queue) +static DNNReturnType execute_model_tf(TFRequestItem *request, Queue *inference_queue) { - TF_Output *tf_outputs; TFModel *tf_model; TFContext *ctx; + TFInferRequest *infer_request; InferenceItem *inference; TaskItem *task; DNNData input, *outputs; - TF_Tensor **output_tensors; - TF_Output tf_input; - TF_Tensor *input_tensor; inference = ff_queue_pop_front(inference_queue); av_assert0(inference); task = inference->task; tf_model = task->model; ctx = &tf_model->ctx; + request->inference = inference; if (get_input_tf(tf_model, &input, task->input_name) != DNN_SUCCESS) return DNN_ERROR; + infer_request = request->infer_request; input.height = task->in_frame->height; input.width = task->in_frame->width; - tf_input.oper = TF_GraphOperationByName(tf_model->graph, task->input_name); - if (!tf_input.oper){ + infer_request->tf_input = av_malloc(sizeof(TF_Output)); + infer_request->tf_input->oper = TF_GraphOperationByName(tf_model->graph, task->input_name); + if (!infer_request->tf_input->oper){ av_log(ctx, AV_LOG_ERROR, "Could not find \"%s\" in model\n", task->input_name); return DNN_ERROR; } - tf_input.index = 0; - input_tensor = allocate_input_tensor(&input); - if (!input_tensor){ + infer_request->tf_input->index = 0; + infer_request->input_tensor = allocate_input_tensor(&input); + if (!infer_request->input_tensor){ av_log(ctx, AV_LOG_ERROR, "Failed to allocate memory for input tensor\n"); return DNN_ERROR; } - input.data = (float *)TF_TensorData(input_tensor); + input.data = (float *)TF_TensorData(infer_request->input_tensor); switch (tf_model->model->func_type) { case DFT_PROCESS_FRAME: @@ -869,60 +894,52 @@ static DNNReturnType execute_model_tf(Queue *inference_queue) break; } - tf_outputs = av_malloc_array(task->nb_output, sizeof(TF_Output)); - if (tf_outputs == NULL) { - TF_DeleteTensor(input_tensor); - av_log(ctx, AV_LOG_ERROR, "Failed to allocate memory for *tf_outputs\n"); \ + infer_request->tf_outputs = av_malloc_array(task->nb_output, sizeof(TF_Output)); + if (infer_request->tf_outputs == NULL) { + av_log(ctx, AV_LOG_ERROR, "Failed to allocate memory for *tf_outputs\n"); return DNN_ERROR; } - output_tensors = av_mallocz_array(task->nb_output, sizeof(*output_tensors)); - if (!output_tensors) { - TF_DeleteTensor(input_tensor); - av_freep(&tf_outputs); - av_log(ctx, AV_LOG_ERROR, "Failed to allocate memory for output tensor\n"); \ + infer_request->output_tensors = av_mallocz_array(task->nb_output, sizeof(*infer_request->output_tensors)); + if (!infer_request->output_tensors) { + av_log(ctx, AV_LOG_ERROR, "Failed to allocate memory for output tensor\n"); return DNN_ERROR; } for (int i = 0; i < task->nb_output; ++i) { - tf_outputs[i].oper = TF_GraphOperationByName(tf_model->graph, task->output_names[i]); - if (!tf_outputs[i].oper) { - TF_DeleteTensor(input_tensor); - av_freep(&tf_outputs); - av_freep(&output_tensors); - av_log(ctx, AV_LOG_ERROR, "Could not find output \"%s\" in model\n", task->output_names[i]); \ + infer_request->output_tensors[i] = NULL; + infer_request->tf_outputs[i].oper = TF_GraphOperationByName(tf_model->graph, task->output_names[i]); + if (!infer_request->tf_outputs[i].oper) { + av_log(ctx, AV_LOG_ERROR, "Could not find output \"%s\" in model\n", task->output_names[i]); return DNN_ERROR; } - tf_outputs[i].index = 0; + infer_request->tf_outputs[i].index = 0; } TF_SessionRun(tf_model->session, NULL, - &tf_input, &input_tensor, 1, - tf_outputs, output_tensors, task->nb_output, - NULL, 0, NULL, tf_model->status); + infer_request->tf_input, &infer_request->input_tensor, 1, + infer_request->tf_outputs, infer_request->output_tensors, + task->nb_output, NULL, 0, NULL, + tf_model->status); if (TF_GetCode(tf_model->status) != TF_OK) { - TF_DeleteTensor(input_tensor); - av_freep(&tf_outputs); - av_freep(&output_tensors); - av_log(ctx, AV_LOG_ERROR, "Failed to run session when executing model\n"); - return DNN_ERROR; + tf_free_request(infer_request); + av_log(ctx, AV_LOG_ERROR, "Failed to run session when executing model\n"); + return DNN_ERROR; } outputs = av_malloc_array(task->nb_output, sizeof(*outputs)); if (!outputs) { - TF_DeleteTensor(input_tensor); - av_freep(&tf_outputs); - av_freep(&output_tensors); - av_log(ctx, AV_LOG_ERROR, "Failed to allocate memory for *outputs\n"); \ + tf_free_request(infer_request); + av_log(ctx, AV_LOG_ERROR, "Failed to allocate memory for *outputs\n"); return DNN_ERROR; } for (uint32_t i = 0; i < task->nb_output; ++i) { - outputs[i].height = TF_Dim(output_tensors[i], 1); - outputs[i].width = TF_Dim(output_tensors[i], 2); - outputs[i].channels = TF_Dim(output_tensors[i], 3); - outputs[i].data = TF_TensorData(output_tensors[i]); - outputs[i].dt = TF_TensorType(output_tensors[i]); + outputs[i].height = TF_Dim(infer_request->output_tensors[i], 1); + outputs[i].width = TF_Dim(infer_request->output_tensors[i], 2); + outputs[i].channels = TF_Dim(infer_request->output_tensors[i], 3); + outputs[i].data = TF_TensorData(infer_request->output_tensors[i]); + outputs[i].dt = TF_TensorType(infer_request->output_tensors[i]); } switch (tf_model->model->func_type) { case DFT_PROCESS_FRAME: @@ -946,30 +963,15 @@ static DNNReturnType execute_model_tf(Queue *inference_queue) tf_model->model->detect_post_proc(task->out_frame, outputs, task->nb_output, tf_model->model->filter_ctx); break; default: - for (uint32_t i = 0; i < task->nb_output; ++i) { - if (output_tensors[i]) { - TF_DeleteTensor(output_tensors[i]); - } - } - TF_DeleteTensor(input_tensor); - av_freep(&output_tensors); - av_freep(&tf_outputs); - av_freep(&outputs); + tf_free_request(infer_request); av_log(ctx, AV_LOG_ERROR, "Tensorflow backend does not support this kind of dnn filter now\n"); return DNN_ERROR; } - for (uint32_t i = 0; i < task->nb_output; ++i) { - if (output_tensors[i]) { - TF_DeleteTensor(output_tensors[i]); - } - } task->inference_done++; - TF_DeleteTensor(input_tensor); - av_freep(&output_tensors); - av_freep(&tf_outputs); + tf_free_request(infer_request); av_freep(&outputs); - return DNN_SUCCESS; + ff_safe_queue_push_back(tf_model->request_queue, request); return (task->inference_done == task->inference_todo) ? DNN_SUCCESS : DNN_ERROR; } @@ -978,6 +980,7 @@ DNNReturnType ff_dnn_execute_model_tf(const DNNModel *model, DNNExecBaseParams * TFModel *tf_model = model->model; TFContext *ctx = &tf_model->ctx; TaskItem task; + TFRequestItem *request; if (ff_check_exec_params(ctx, DNN_TF, model->func_type, exec_params) != 0) { return DNN_ERROR; @@ -991,7 +994,14 @@ DNNReturnType ff_dnn_execute_model_tf(const DNNModel *model, DNNExecBaseParams * av_log(ctx, AV_LOG_ERROR, "unable to extract inference from task.\n"); return DNN_ERROR; } - return execute_model_tf(tf_model->inference_queue); + + request = ff_safe_queue_pop_front(tf_model->request_queue); + if (!request) { + av_log(ctx, AV_LOG_ERROR, "unable to get infer request.\n"); + return DNN_ERROR; + } + + return execute_model_tf(request, tf_model->inference_queue); } void ff_dnn_free_model_tf(DNNModel **model) @@ -1000,6 +1010,14 @@ void ff_dnn_free_model_tf(DNNModel **model) if (*model){ tf_model = (*model)->model; + while (ff_safe_queue_size(tf_model->request_queue) != 0) { + TFRequestItem *item = ff_safe_queue_pop_front(tf_model->request_queue); + tf_free_request(item->infer_request); + av_freep(&item->infer_request); + av_freep(&item); + } + ff_safe_queue_destroy(tf_model->request_queue); + while (ff_queue_size(tf_model->inference_queue) != 0) { InferenceItem *item = ff_queue_pop_front(tf_model->inference_queue); av_freep(&item);