From patchwork Tue Mar 16 20:30:57 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andreas Rheinhardt X-Patchwork-Id: 26417 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 703D54487CA for ; Tue, 16 Mar 2021 22:31:36 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 53EAB68B018; Tue, 16 Mar 2021 22:31:36 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-ed1-f53.google.com (mail-ed1-f53.google.com [209.85.208.53]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id AFBFA68AB4D for ; Tue, 16 Mar 2021 22:31:29 +0200 (EET) Received: by mail-ed1-f53.google.com with SMTP id dm8so23151595edb.2 for ; Tue, 16 Mar 2021 13:31:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references:reply-to :mime-version:content-transfer-encoding; bh=Ig4p3oL9r4fywjprxMA0e01fN5n9scNyhBRN+YqphVA=; b=eguSPF+liNRhg0+u0O5Yia5XbDjmGMYpEY7HXFUVi6S3pl/UktbKrbQyx2kLGJvPl0 StGhoh5c4lo0sOOX6H4gtfxo27P8p3ebrtoUNVStVoKn8i1/l+wIAwgRDqIwxifuFGZS LJSuuF1MgOer0q4QHZroJoUaAzhnauJj7Ha3fmVLn1SWX7bhCFFbt7Q3PRrUwhGM/Ltl QHbJjx7q2Ujg3gB5FZma1G+TZslrpzxvDLBszqv97cmK+3RSsmVxcPWhXlfJTXRv6pbg yOX67C/uhJqfmPVygJQYLQ8sZ7u8gJfI2ka36tLqxIYoDEjNllQ9TxU+wLMWQ1GDZnzW cBWw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:reply-to:mime-version:content-transfer-encoding; bh=Ig4p3oL9r4fywjprxMA0e01fN5n9scNyhBRN+YqphVA=; b=cFlDfJHZKxZc2TvS8mtJ4HHOk5g/fg+44aUSuksBspJxNVPJROil6l/pAZ2C8lOb+A rwU4RpRZbo7C0EveiRPTZJD5K8ugbjxU6NH+glEzzJcCgVRy8algYPRNyMaOY8skWfmT SVi81H14uB7onMkJ61zKA0puxurV6+Fx7p+FZJmsdSyIhW3bbYhK/Jd68mFwqiSHyksf I2gN/hTnkVsGN8FJsf/RD4GhfR8buaMPcrRF+XksgUZle3Z748ND+vngJ5ph7mpUofKt QsLdHGCn2Ihkv1dJ3UDenzW3Pz80hY1fjsMZxkWJ0/jz0KiTX17BMt89SF3rICT/036/ z40g== X-Gm-Message-State: AOAM531NL4xZ2RpGrrnpcIswmXAy/p4VFOuWPBIRHzq+JZwkiug0WlTE 59RVjCKeEv71W/AFHJRVUevTTjnE18prAA== X-Google-Smtp-Source: ABdhPJyCsLvR/TfZqrUzqsSji0zLOwmN6xsIhZpl8DveAcn/+j03yEWOBG97yjDZThmlkyBCV4z70g== X-Received: by 2002:a05:6402:180b:: with SMTP id g11mr37629460edy.195.1615926689110; Tue, 16 Mar 2021 13:31:29 -0700 (PDT) Received: from sblaptop.fritz.box (ipbcc08960.dynamic.kabel-deutschland.de. [188.192.137.96]) by smtp.gmail.com with ESMTPSA id a22sm10281908ejr.89.2021.03.16.13.31.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 Mar 2021 13:31:28 -0700 (PDT) From: Andreas Rheinhardt To: ffmpeg-devel@ffmpeg.org Date: Tue, 16 Mar 2021 21:30:57 +0100 Message-Id: <20210316203100.528950-3-andreas.rheinhardt@gmail.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210316203100.528950-1-andreas.rheinhardt@gmail.com> References: <20210316203100.528950-1-andreas.rheinhardt@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 3/6] avcodec/utils: Sanitize options before using them X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Andreas Rheinhardt Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" This is how it is supposed to happen, yet when using frame threading, the codec's init function has been called before preinit. This can lead to crashes when e.g. using unsupported lowres values for decoders together with frame threading. Signed-off-by: Andreas Rheinhardt --- libavcodec/utils.c | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/libavcodec/utils.c b/libavcodec/utils.c index 3629813387..7d4ad113df 100644 --- a/libavcodec/utils.c +++ b/libavcodec/utils.c @@ -716,6 +716,13 @@ int attribute_align_arg avcodec_open2(AVCodecContext *avctx, const AVCodec *code avctx->time_base.den = avctx->sample_rate; } + if (av_codec_is_encoder(avctx->codec)) + ret = ff_encode_preinit(avctx); + else + ret = ff_decode_preinit(avctx); + if (ret < 0) + goto free_and_end; + if (!HAVE_THREADS) av_log(avctx, AV_LOG_WARNING, "Warning: not compiled with thread support, using thread emulation\n"); @@ -737,13 +744,6 @@ int attribute_align_arg avcodec_open2(AVCodecContext *avctx, const AVCodec *code if (!HAVE_THREADS && !(codec->caps_internal & FF_CODEC_CAP_AUTO_THREADS)) avctx->thread_count = 1; - if (av_codec_is_encoder(avctx->codec)) - ret = ff_encode_preinit(avctx); - else - ret = ff_decode_preinit(avctx); - if (ret < 0) - goto free_and_end; - if ( avctx->codec->init && (!(avctx->active_thread_type&FF_THREAD_FRAME) || avci->frame_thread_encoder)) { ret = avctx->codec->init(avctx);