From patchwork Mon Feb 8 12:23:24 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andreas Rheinhardt X-Patchwork-Id: 25504 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 1C34C44A631 for ; Mon, 8 Feb 2021 14:24:09 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id F3DDE688385; Mon, 8 Feb 2021 14:24:08 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-ej1-f45.google.com (mail-ej1-f45.google.com [209.85.218.45]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 6A5EA68832B for ; Mon, 8 Feb 2021 14:24:03 +0200 (EET) Received: by mail-ej1-f45.google.com with SMTP id bl23so24292833ejb.5 for ; Mon, 08 Feb 2021 04:24:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references:reply-to :mime-version:content-transfer-encoding; bh=ERZN+7p1WNQQ+Va7GqMs2VtXeI3il3mfVwABgz4MybQ=; b=sRDwvdakmE4lWXs4UNatWnRFRbwvHJDQ2RrWlNThbGDiv/t66lHIvXpkO+JGT11kDY gqrfBjNPsAtUulpyLdrzYMO7AxpLVEtdoCfwKql957zEUDMh9oE4vR1bNJ4tlnZesGUs dYpZFWyhUaTCtB/mP9Uweu9jlx01W95gIuitwotD0gTbdaENKfPR7QkpHtvcJbAaAKHv 12jiihoLmnklD5Jtdeb/C1Eh4eFvY3ahrqe8791HDYjG2U5vGpmWPSpMI7kzWqAFhpOr 4JVW2cFz5Xzf84Xd7aJSig25FwmGq7BKtZrrgY+cpeWmceRugg2vm+aeitFmdJcRPPMJ RG1Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:reply-to:mime-version:content-transfer-encoding; bh=ERZN+7p1WNQQ+Va7GqMs2VtXeI3il3mfVwABgz4MybQ=; b=jRbTeLrTpbxdmfMxGyT3VZY6i/AFYwvWT7yHwiJsxgZ6AbdIU05DKQYg5vDGRHN1xP V/xM13IBKQ+aFJ567BhE2KtMARdcmV0nh9mTHlFdeW8ZNSBVc19jyONft8RXAqczvWvL YVD2ZqnrnRmqTptOZ7AegIR3ontofnMmsN/RLlLNv1ZFFSRi15+XXAsxaeClKnuYXTI2 fUxR+kjGhKXrr2+M78HcfhUSgNZXeYs+NvUUACHpQHeKkvZSbsMU6WEs2hGqMpvtyclQ K3F2JJHwuLp993ipWQLilz04ulu0QtJo/Y+lCx0r+Y0bkDBA1EAeeMCz0iW8MgUDocJn vJUg== X-Gm-Message-State: AOAM530P1ECmY6dmnhUu4n8eZzoXWSm5uaretvxltMJ/hW6IlcphKUj8 sG+bRPlc+DIvXsHvLD/zB+UjBuHsC1Q= X-Google-Smtp-Source: ABdhPJxqUpCmG/UNAgAKse/ekMHe6qgWn03ly+RCd6mTeNZDC4EDuMMkrbHZ63RHBjeyCFEWvnMHbw== X-Received: by 2002:a17:906:1e87:: with SMTP id e7mr16590685ejj.322.1612787042712; Mon, 08 Feb 2021 04:24:02 -0800 (PST) Received: from sblaptop.fritz.box (ipbcc1aa4b.dynamic.kabel-deutschland.de. [188.193.170.75]) by smtp.gmail.com with ESMTPSA id ah20sm8486820ejc.99.2021.02.08.04.24.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 08 Feb 2021 04:24:02 -0800 (PST) From: Andreas Rheinhardt To: ffmpeg-devel@ffmpeg.org Date: Mon, 8 Feb 2021 13:23:24 +0100 Message-Id: <20210208122330.555354-2-andreas.rheinhardt@gmail.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210208122330.555354-1-andreas.rheinhardt@gmail.com> References: <20210208122330.555354-1-andreas.rheinhardt@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 2/8] avcodec/frame_thread_encoder: Fix segfault on allocation error X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Andreas Rheinhardt Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Fixes a segfault from av_fifo_size(NULL) that happens in ff_frame_thread_encoder_free if the fifo couldn't be allocted; furthermore the mutexes and conditions that are destroyed in ff_frame_thread_encoder_free are not even initialized at this point, so don't call said function. Signed-off-by: Andreas Rheinhardt --- libavcodec/frame_thread_encoder.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/libavcodec/frame_thread_encoder.c b/libavcodec/frame_thread_encoder.c index ee289c90e3..9ca34e7ffb 100644 --- a/libavcodec/frame_thread_encoder.c +++ b/libavcodec/frame_thread_encoder.c @@ -182,8 +182,10 @@ int ff_frame_thread_encoder_init(AVCodecContext *avctx, AVDictionary *options){ c->parent_avctx = avctx; c->task_fifo = av_fifo_alloc_array(BUFFER_SIZE, sizeof(Task)); - if(!c->task_fifo) - goto fail; + if (!c->task_fifo) { + av_freep(&avctx->internal->frame_thread_encoder); + return AVERROR(ENOMEM); + } pthread_mutex_init(&c->task_fifo_mutex, NULL); pthread_mutex_init(&c->finished_task_mutex, NULL);