From patchwork Fri Dec 25 15:47:18 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andreas Rheinhardt X-Patchwork-Id: 24655 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 56454449BE4 for ; Fri, 25 Dec 2020 18:54:24 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 2584268AE81; Fri, 25 Dec 2020 18:54:24 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-lf1-f45.google.com (mail-lf1-f45.google.com [209.85.167.45]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 352AA68ADB0 for ; Fri, 25 Dec 2020 18:54:18 +0200 (EET) Received: by mail-lf1-f45.google.com with SMTP id y19so10725873lfa.13 for ; Fri, 25 Dec 2020 08:54:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:reply-to:mime-version :content-transfer-encoding; bh=nqua6AO8dTo5ND9go4tmZaZV046SAobbUKUstSIhIxo=; b=PPFU02MbOzrH6WD03LgAZfyb0I28HAi7XF1iW6Bdk5DxFZDkxNDlrApWZvCy2YxqhY sdpMoz6r1kFsDctWuttUmEYtT6IYGZWHtBa906H/vcZi9OFZdggYz0rYPi0DVPK3btxE En8gjPcLS5tDLjxowzdBpJuEM5Yltn8kJS0OZf+/IoZPjdzRZu1CWbkVlYlpr16QqNIc GnKBNqIDh4pP9u7/+zYEg+02KHUD4t/2lNlcx4SCfR/mHxaBBIjTZb/HustV0t1gV6Ro GrlLSz/i58/9QZqX8F+TsRqqhTtKxPgLd8Kso257/+rZZIwEnU0RsLkv1QC8c8XgDXvB ZO9g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:reply-to :mime-version:content-transfer-encoding; bh=nqua6AO8dTo5ND9go4tmZaZV046SAobbUKUstSIhIxo=; b=jMXd17rYTI/QIvdqCs2biSK2H9EcKhpSKBLJ0NMtLCwn/BabVeMetXz6jZ1hM4uX3x dswJ1RPwmY4KBkct6acrVk84yJy8S8bM0fZeIgHJXzbqz5K1y5M1Lx61wRnjJ+RIuuSt PpYneGYUfIOI49Kz7bCC2E7sxU1Byzl0tUUBmuawVzZ9Boy9lqiVYdrK617Ty7gBYaMO rgPa/+N8xPaaERXGI+PgWLfqiV569AAC6KutHRecXWMRuahFfYekZWM560jZbJ220jC7 lkzeVtJKbuA5ZN1gdBAO8INUKG5FHA+EVLVJyP+9Mj5GkaXO+MX6Yfzgs3pXzJaf68Dn QLZQ== X-Gm-Message-State: AOAM530s8GF4q2uUre5XYj1Fj4c8qMwRhBV7Xefq5RN97jj4sA12JIdg 81AqKeEF7Lovybo7vYzN+/xb39o5VLY= X-Google-Smtp-Source: ABdhPJwz0kKbCOTaxFUXwL/4BbFtDJYQrDQwXROgIJRJEwGga2IVGa14uuzJqxH+mVkJomxK4R3e1Q== X-Received: by 2002:a17:906:378f:: with SMTP id n15mr31413146ejc.263.1608911252598; Fri, 25 Dec 2020 07:47:32 -0800 (PST) Received: from sblaptop.fritz.box (ipbcc1aa4b.dynamic.kabel-deutschland.de. [188.193.170.75]) by smtp.gmail.com with ESMTPSA id ho12sm13733010ejc.45.2020.12.25.07.47.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 25 Dec 2020 07:47:32 -0800 (PST) From: Andreas Rheinhardt To: ffmpeg-devel@ffmpeg.org Date: Fri, 25 Dec 2020 16:47:18 +0100 Message-Id: <20201225154724.287465-1-andreas.rheinhardt@gmail.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 1/7] avcodec/mpeg12dec: Remove update_thread_context X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Andreas Rheinhardt Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" No decoder here supports frame threading. Signed-off-by: Andreas Rheinhardt --- libavcodec/mpeg12dec.c | 28 ---------------------------- 1 file changed, 28 deletions(-) diff --git a/libavcodec/mpeg12dec.c b/libavcodec/mpeg12dec.c index 6d0e9fc7ed..66556c7a82 100644 --- a/libavcodec/mpeg12dec.c +++ b/libavcodec/mpeg12dec.c @@ -1074,33 +1074,6 @@ static av_cold int mpeg_decode_init(AVCodecContext *avctx) return 0; } -#if HAVE_THREADS -static int mpeg_decode_update_thread_context(AVCodecContext *avctx, - const AVCodecContext *avctx_from) -{ - Mpeg1Context *ctx = avctx->priv_data, *ctx_from = avctx_from->priv_data; - MpegEncContext *s = &ctx->mpeg_enc_ctx, *s1 = &ctx_from->mpeg_enc_ctx; - int err; - - if (avctx == avctx_from || - !ctx_from->mpeg_enc_ctx_allocated || - !s1->context_initialized) - return 0; - - err = ff_mpeg_update_thread_context(avctx, avctx_from); - if (err) - return err; - - if (!ctx->mpeg_enc_ctx_allocated) - memcpy(s + 1, s1 + 1, sizeof(Mpeg1Context) - sizeof(MpegEncContext)); - - if (!(s->pict_type == AV_PICTURE_TYPE_B || s->low_delay)) - s->picture_number++; - - return 0; -} -#endif - static void quant_matrix_rebuild(uint16_t *matrix, const uint8_t *old_perm, const uint8_t *new_perm) { @@ -2902,7 +2875,6 @@ AVCodec ff_mpeg1video_decoder = { .caps_internal = FF_CODEC_CAP_SKIP_FRAME_FILL_PARAM | FF_CODEC_CAP_INIT_CLEANUP, .flush = flush, .max_lowres = 3, - .update_thread_context = ONLY_IF_THREADS_ENABLED(mpeg_decode_update_thread_context), .hw_configs = (const AVCodecHWConfigInternal*[]) { #if CONFIG_MPEG1_NVDEC_HWACCEL HWACCEL_NVDEC(mpeg1), From patchwork Fri Dec 25 15:47:19 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andreas Rheinhardt X-Patchwork-Id: 24649 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 357E544B6ED for ; Fri, 25 Dec 2020 17:53:05 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 018D768AE63; Fri, 25 Dec 2020 17:53:05 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-ed1-f53.google.com (mail-ed1-f53.google.com [209.85.208.53]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 453B968AA85 for ; Fri, 25 Dec 2020 17:52:58 +0200 (EET) Received: by mail-ed1-f53.google.com with SMTP id r5so4355356eda.12 for ; Fri, 25 Dec 2020 07:52:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references:reply-to :mime-version:content-transfer-encoding; bh=XGwW8OLPuFMM76bgFaXS/6HXtxInUziDe2TO4WwUC3s=; b=I6WJAspIZzekcs1Nkl2M/3sAsSz2FeKpAlcc3w6R2EsK2H8c4u0qrE4tNiSS1WoDYl F8JxY5taGkNuFYxr+tO141mhFiF8QhOPqHSjoqJboGMl3Sa68h+60uYeYq8XFs/kNSmm w07Hieg43EfM0Vb7UEQ+AgHfa8rG9/CXvC2m1KpQwJuWQgUQMAPcIrVkiprQsU+hLjQs yq9//q5iNqGjBKE2fbjesh2/2dE8YIk4V+k2qAriZEL4+lZi9YNIomjYlkxT7oofec8l yZSWHHD6fFImp0znlng5kLVRnRfEMezZW0l0EcrlX2W3aleuf5m3sSQ4JA9YD11lqy3c g+ow== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:reply-to:mime-version:content-transfer-encoding; bh=XGwW8OLPuFMM76bgFaXS/6HXtxInUziDe2TO4WwUC3s=; b=ctxpkNTguc7lIeVjFu4kLaW0mX3Hrc4uBOEbesbevQAWmoYSKOQ21fsFKvWKkrNB/4 Vcn20gnExOIXPd/69oea2Un2UYvj2YmdqksEfR3XTjPiEGc9kRMWC60yzSptrrnK3pLn 4NVaCMe9idy0F98pQZmMjSJa51kJh/qJRqbppbXrP21/sre+6PmNOQTaGH0imBcCYh3Z /kJHy/22HJj+CljINpjLMpRmvIykxy7Leu+m5+2dW077kdRR+Ri7qzNjqAq7/D7nRZ2N Uw9MGutcw0Ysq8UeBDhM3pkrXwCrhawoZXaYTt+jiSNfE0x3F9wQChLoJapo6knWhzBS N4Rg== X-Gm-Message-State: AOAM532IY7mEMdlRiNrXP7VadDibxf6wmMIkht1i/7CgSc1KoFS3GxTg 7kxfpDLXuydyvi43SKR5/lijnPNSKpw= X-Google-Smtp-Source: ABdhPJwF/7+FEtvdvVfEn/4nL0lrjQPBMOGEs+G+iITXJyFtjjD1f+bTKLIIF825iskDkITcCf2qGA== X-Received: by 2002:a05:6402:1041:: with SMTP id e1mr33723457edu.54.1608911577151; Fri, 25 Dec 2020 07:52:57 -0800 (PST) Received: from sblaptop.fritz.box (ipbcc1aa4b.dynamic.kabel-deutschland.de. [188.193.170.75]) by smtp.gmail.com with ESMTPSA id ho12sm13733010ejc.45.2020.12.25.07.52.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 25 Dec 2020 07:52:56 -0800 (PST) From: Andreas Rheinhardt To: ffmpeg-devel@ffmpeg.org Date: Fri, 25 Dec 2020 16:47:19 +0100 Message-Id: <20201225154724.287465-2-andreas.rheinhardt@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20201225154724.287465-1-andreas.rheinhardt@gmail.com> References: <20201225154724.287465-1-andreas.rheinhardt@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 2/7] avcodec/vc1dec: Postpone allocating sprite frame to avoid segfault X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Andreas Rheinhardt Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Up until now, the VC-1 decoders allocated an AVFrame for usage with sprites during vc1_decode_init(); yet said AVFrame can be freed if (re)initializing the context (which happens ordinarily during decoding) fails. The AVFrame does not get allocated again lateron in this case, leading to segfaults. Fix this by moving the allocation of said frame immediately before it is used (this also means that said frame won't be allocated at all any more in case of a regular (i.e. non-image) stream). Signed-off-by: Andreas Rheinhardt --- libavcodec/vc1dec.c | 20 +++++++------------- 1 file changed, 7 insertions(+), 13 deletions(-) diff --git a/libavcodec/vc1dec.c b/libavcodec/vc1dec.c index 7809234ff7..5cdf197da7 100644 --- a/libavcodec/vc1dec.c +++ b/libavcodec/vc1dec.c @@ -539,12 +539,6 @@ static av_cold int vc1_decode_init(AVCodecContext *avctx) ff_h264chroma_init(&v->h264chroma, 8); ff_qpeldsp_init(&s->qdsp); - // Must happen after calling ff_vc1_decode_end - // to avoid de-allocating the sprite_output_frame - v->sprite_output_frame = av_frame_alloc(); - if (!v->sprite_output_frame) - return AVERROR(ENOMEM); - avctx->has_b_frames = !!avctx->max_b_frames; if (v->color_prim == 1 || v->color_prim == 5 || v->color_prim == 6) @@ -577,20 +571,15 @@ static av_cold int vc1_decode_init(AVCodecContext *avctx) v->sprite_height > 1 << 14 || v->output_width > 1 << 14 || v->output_height > 1 << 14) { - ret = AVERROR_INVALIDDATA; - goto error; + return AVERROR_INVALIDDATA; } if ((v->sprite_width&1) || (v->sprite_height&1)) { avpriv_request_sample(avctx, "odd sprites support"); - ret = AVERROR_PATCHWELCOME; - goto error; + return AVERROR_PATCHWELCOME; } } return 0; -error: - av_frame_free(&v->sprite_output_frame); - return ret; } /** Close a VC1/WMV3 decoder @@ -1147,6 +1136,11 @@ image: avctx->height = avctx->coded_height = v->output_height; if (avctx->skip_frame >= AVDISCARD_NONREF) goto end; + if (!v->sprite_output_frame && + !(v->sprite_output_frame = av_frame_alloc())) { + ret = AVERROR(ENOMEM); + goto err; + } #if CONFIG_WMV3IMAGE_DECODER || CONFIG_VC1IMAGE_DECODER if ((ret = vc1_decode_sprites(v, &s->gb)) < 0) goto err; From patchwork Fri Dec 25 15:47:20 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andreas Rheinhardt X-Patchwork-Id: 24650 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 9249544B6ED for ; Fri, 25 Dec 2020 17:53:06 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 6BBA868AE78; Fri, 25 Dec 2020 17:53:06 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-ed1-f53.google.com (mail-ed1-f53.google.com [209.85.208.53]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 0222C68AA85 for ; Fri, 25 Dec 2020 17:52:58 +0200 (EET) Received: by mail-ed1-f53.google.com with SMTP id p22so4354082edu.11 for ; Fri, 25 Dec 2020 07:52:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references:reply-to :mime-version:content-transfer-encoding; bh=fOXJZb++16Bsh6gaKnoCflciacN1UetOSGCMH1gLLmc=; b=ndcn5ZHHvX5kd1plVb5hEDkG7VCaNSeYKX7xT4s9nT364ZfefllBtjfp78Jth1v1Mr ksBF2LgssfvSJqAAJZfIrjogDUq1Pddea9NDKJiCM8rCT5i3ZQdN9pgLNDEIQOoxe0Xj 5JHypjIxreKqqemmWr3g2FFhPd9xsqDN4UQ/9BH1Y1KGf3o6yqZ5ptdgcopqh29StV/8 LGMuWQR5xpAvuUGHxzoA/6DosDEnfJoOkdPZ9EIsQ5glWnR6BvhE5xUekVvarenbZ2xy 55FRRahX8nv+IadP7lT0f1Xsm0hvfD4lPvy6mHmcU/xKKVTz9St+QKOIz4JbFFIaah+t /OEg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:reply-to:mime-version:content-transfer-encoding; bh=fOXJZb++16Bsh6gaKnoCflciacN1UetOSGCMH1gLLmc=; b=frQDeYuNPcc0DaVLRDpWWi4JZ4mtBo8ae3Tkc8lvPnwFtptJ3HLIlZ/6xzag4HIvjV t3HBPdaBRTMZYWhVk0BbqfTPxziecgZAqCQvQToYQTVcYlTE2NwicV+u1BydTxRdw2xB fNrECdDc3eWdiH9NCsQBgel5puP83bMhjI7Ldg23mCxWjDEJa+pbEg/nQknuHFB9uust Npprz+Ie0GtPNQ8R7uhlEMCedjfpvt/Q+UQyKJgRu+au7nve65FvFMox8LbhvpXEposP DJKmvN79bO6plCB6WCxSd1UqxuV5ZTLjohjGNgIIuuFrthFRSQg3fua5eXZBvnvvEqm2 mXTg== X-Gm-Message-State: AOAM531CzSgl0apSRfcEcyO6SbMzYMtxQ4qzkFENfDWgglaKHU8mLy9d FcemwKf9/8tFGws+Q7gGYvtvS7NYVGk= X-Google-Smtp-Source: ABdhPJzi1ScdGzBi7KQdn5e1pizL0QFRY9JWSrFiiEJNId0snfIdg/3lQit+DXAzsa230fJkkoXQiA== X-Received: by 2002:aa7:d3d4:: with SMTP id o20mr32333152edr.190.1608911578340; Fri, 25 Dec 2020 07:52:58 -0800 (PST) Received: from sblaptop.fritz.box (ipbcc1aa4b.dynamic.kabel-deutschland.de. [188.193.170.75]) by smtp.gmail.com with ESMTPSA id ho12sm13733010ejc.45.2020.12.25.07.52.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 25 Dec 2020 07:52:57 -0800 (PST) From: Andreas Rheinhardt To: ffmpeg-devel@ffmpeg.org Date: Fri, 25 Dec 2020 16:47:20 +0100 Message-Id: <20201225154724.287465-3-andreas.rheinhardt@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20201225154724.287465-1-andreas.rheinhardt@gmail.com> References: <20201225154724.287465-1-andreas.rheinhardt@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 3/7] avcodec/vc1: Don't pretend ff_vc1_init_common() can fail X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Andreas Rheinhardt Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Signed-off-by: Andreas Rheinhardt --- libavcodec/mss2.c | 3 +-- libavcodec/vc1.c | 4 +--- libavcodec/vc1.h | 2 +- libavcodec/vc1_parser.c | 3 ++- libavcodec/vc1dec.c | 3 +-- 5 files changed, 6 insertions(+), 9 deletions(-) diff --git a/libavcodec/mss2.c b/libavcodec/mss2.c index 9434a740a7..3e3205ae92 100644 --- a/libavcodec/mss2.c +++ b/libavcodec/mss2.c @@ -751,8 +751,7 @@ static av_cold int wmv9_init(AVCodecContext *avctx) v->s.avctx = avctx; - if ((ret = ff_vc1_init_common(v)) < 0) - return ret; + ff_vc1_init_common(v); ff_vc1dsp_init(&v->vc1dsp); v->profile = PROFILE_MAIN; diff --git a/libavcodec/vc1.c b/libavcodec/vc1.c index cd9975d8cf..5d854b35d2 100644 --- a/libavcodec/vc1.c +++ b/libavcodec/vc1.c @@ -1695,7 +1695,7 @@ static av_cold void vc1_init_static(void) * @param v The VC1Context to initialize * @return Status */ -av_cold int ff_vc1_init_common(VC1Context *v) +av_cold void ff_vc1_init_common(VC1Context *v) { static AVOnce init_static_once = AV_ONCE_INIT; @@ -1709,6 +1709,4 @@ av_cold int ff_vc1_init_common(VC1Context *v) /* VLC tables */ ff_thread_once(&init_static_once, vc1_init_static); - - return 0; } diff --git a/libavcodec/vc1.h b/libavcodec/vc1.h index 4559a06cb6..3e5368b891 100644 --- a/libavcodec/vc1.h +++ b/libavcodec/vc1.h @@ -413,7 +413,7 @@ int ff_vc1_decode_entry_point(AVCodecContext *avctx, VC1Context *v, GetBitContex int ff_vc1_parse_frame_header (VC1Context *v, GetBitContext *gb); int ff_vc1_parse_frame_header_adv(VC1Context *v, GetBitContext *gb); -int ff_vc1_init_common(VC1Context *v); +void ff_vc1_init_common(VC1Context *v); int ff_vc1_decode_init_alloc_tables(VC1Context *v); void ff_vc1_init_transposed_scantables(VC1Context *v); diff --git a/libavcodec/vc1_parser.c b/libavcodec/vc1_parser.c index 493ffde611..1a9d3c0140 100644 --- a/libavcodec/vc1_parser.c +++ b/libavcodec/vc1_parser.c @@ -283,7 +283,8 @@ static av_cold int vc1_parse_init(AVCodecParserContext *s) vpc->bytes_to_skip = 0; vpc->unesc_index = 0; vpc->search_state = NO_MATCH; - return ff_vc1_init_common(&vpc->v); + ff_vc1_init_common(&vpc->v); + return 0; } AVCodecParser ff_vc1_parser = { diff --git a/libavcodec/vc1dec.c b/libavcodec/vc1dec.c index 5cdf197da7..78988b740c 100644 --- a/libavcodec/vc1dec.c +++ b/libavcodec/vc1dec.c @@ -434,8 +434,7 @@ static av_cold int vc1_decode_init(AVCodecContext *avctx) return AVERROR_INVALIDDATA; v->s.avctx = avctx; - if ((ret = ff_vc1_init_common(v)) < 0) - return ret; + ff_vc1_init_common(v); if (avctx->codec_id == AV_CODEC_ID_WMV3 || avctx->codec_id == AV_CODEC_ID_WMV3IMAGE) { int count = 0; From patchwork Fri Dec 25 15:47:21 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andreas Rheinhardt X-Patchwork-Id: 24651 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 8897844B6ED for ; Fri, 25 Dec 2020 17:53:08 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 6C65268AE7E; Fri, 25 Dec 2020 17:53:08 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-ej1-f42.google.com (mail-ej1-f42.google.com [209.85.218.42]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 2082B68AE6B for ; Fri, 25 Dec 2020 17:53:00 +0200 (EET) Received: by mail-ej1-f42.google.com with SMTP id q22so6653173eja.2 for ; Fri, 25 Dec 2020 07:53:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references:reply-to :mime-version:content-transfer-encoding; bh=lj4WFLhKWVcGbNNy6wOB2O5lauzhjsst2W/2MTfk+WE=; b=gw3dIBkW/S+9XXpt4cqhfLDpeKSIhgDCQ4aUuMo0lbd7dTthU4txWdjBfaPnfZ+eVs kjlDgJoPhi8GGGslHYF2jWAj/iz+ymm2HsxEaSQiLtQ/07auyX0HHIVhZytAVkk1QreA 79ojpt6ycG8VVV/S4H+J+OkFe44wC3o/uJXw2+vDXa2h4KMC/oHaPVxNAVPL4QqrmMBp r+Jq9vJIMBwzbshdDJK9v2OP7d+gRVAtcu2kn0yTC4UGWTVg+2La1gmQ0Js0zQNegGNl IHCp+0UMZMNUyjdTHhyPiUG+IW0hvZRx1Y/ln663VYlAcYrdqJPhFrWYkYyccJ8+ua77 9D6w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:reply-to:mime-version:content-transfer-encoding; bh=lj4WFLhKWVcGbNNy6wOB2O5lauzhjsst2W/2MTfk+WE=; b=MRsnJAXj+R1uopB2RkOQmVeoV/fk89W2FWs3lJFOUvkeIO8blIyLDgFjJIeNoTJEsa Rd+xfO0LdYw0Zf0XRVOsroOG9YA5SMuvUjd21LDiZQqXVeUsxqv0V5Q96nMrk/OT6GaH BD8bN//EDiCwgE8EufUL6BM4JSrKRbncLJnKbZTObrEdH0IoElCSl0WCUGGfRMRfIfdE rerUMbEXnJwfvp85KEaiJ9l0PJCO/Z9YWkb6wO97uaL5VRgNZVe2dUfnT0AgrUtvNQx4 +goUGvIElbT/e5qHKxUbeTYwV9YaF4yyZTLrYNHejNpgZXQcG92uLIECTEQgAsli00ex E14Q== X-Gm-Message-State: AOAM5303NkRD9EjftoyKBbQXEEQIbGvkE6ALBuw+7nFSZ4kfNkgc31It iwQtK8+L1MBEcO5iNNZ17nJl/c62Ihc= X-Google-Smtp-Source: ABdhPJw6tClvd4dc+z/QU3lmIw8dR5kugmrchRp26Zb+ZR/g+FdltYKxrXYvkCktQ+HQh71EquM3gA== X-Received: by 2002:a17:906:5857:: with SMTP id h23mr14289483ejs.465.1608911579402; Fri, 25 Dec 2020 07:52:59 -0800 (PST) Received: from sblaptop.fritz.box (ipbcc1aa4b.dynamic.kabel-deutschland.de. [188.193.170.75]) by smtp.gmail.com with ESMTPSA id ho12sm13733010ejc.45.2020.12.25.07.52.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 25 Dec 2020 07:52:58 -0800 (PST) From: Andreas Rheinhardt To: ffmpeg-devel@ffmpeg.org Date: Fri, 25 Dec 2020 16:47:21 +0100 Message-Id: <20201225154724.287465-4-andreas.rheinhardt@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20201225154724.287465-1-andreas.rheinhardt@gmail.com> References: <20201225154724.287465-1-andreas.rheinhardt@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 4/7] avcodec/h261dec: Remove parse_context cruft X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Andreas Rheinhardt Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" The H.261 decoder doesn't use the ParseContext of its MpegEncContext since e7316976650b429345da619c3acff38004aaf6b8. Signed-off-by: Andreas Rheinhardt --- Does actually anybody use the AV_CODEC_FLAG_TRUNCATED (MpegEncContext contains a ParseContext because of this flag)? libavcodec/h261dec.c | 3 --- 1 file changed, 3 deletions(-) diff --git a/libavcodec/h261dec.c b/libavcodec/h261dec.c index 8a49e7d894..4f1f22b279 100644 --- a/libavcodec/h261dec.c +++ b/libavcodec/h261dec.c @@ -613,10 +613,7 @@ retry: } if (s->width != avctx->coded_width || s->height != avctx->coded_height) { - ParseContext pc = s->parse_context; // FIXME move this demuxing hack to libavformat - s->parse_context.buffer = 0; ff_mpv_common_end(s); - s->parse_context = pc; } if (!s->context_initialized) { From patchwork Fri Dec 25 15:47:22 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andreas Rheinhardt X-Patchwork-Id: 24654 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 4CB7B44AF05 for ; Fri, 25 Dec 2020 18:01:23 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 3267B68AE70; Fri, 25 Dec 2020 18:01:23 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wr1-f41.google.com (mail-wr1-f41.google.com [209.85.221.41]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id A52DA68AE66 for ; Fri, 25 Dec 2020 18:01:17 +0200 (EET) Received: by mail-wr1-f41.google.com with SMTP id r3so4664357wrt.2 for ; Fri, 25 Dec 2020 08:01:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references:reply-to :mime-version:content-transfer-encoding; bh=4uN4lwrJlubge8Vdjc4G2LJHYo9sZCU7tA9n3+PRkBM=; b=VmkXWSEMzDbikv2LSi2/2Rk9Ex1WJC+zgRgahPnW3TRaMJlWnbGFZhihl7FCxW0Zgq TywoEB82QcCmP4HNq5ZwvTIRWtAZitRHYw/4dm4IygRFGyjF4ZhOK/y73fHwe1AopeQb nQ8V+0eIzk00nPfVqTSlBLGDfDrGzYyfP4wddYwwHDWi046Ba4/akk3r6AsacpehXl/I NBVf2ryLw+KLN1BMkp4Yi7IBdslSL2BM8MGSMj6iA/Mgsf4Narc5oswY9uIovx8W88rK Tc65NAxhlMFitQGwEQBH413AmThjkT/dfXSuTlTeuTqWSUNNgcznM8OpT9m7mt+xLKyX R/XQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:reply-to:mime-version:content-transfer-encoding; bh=4uN4lwrJlubge8Vdjc4G2LJHYo9sZCU7tA9n3+PRkBM=; b=pI9qUOPD3SlcQVYZKO/0fKXuak/r53HzmFRLEw8yS0n+PDGAF5LumWtBrOrhRDzwqu V8Humb5EFFXEzLR59VzZwfUBGNPmMWEC2aC0jSTFwNq9jHDCJ3RoMnUUEQvupHZgLfxL hjGAtEKnup0G/VTz2iX7HcxSv8WYDzNsW1NdPbF/3XYUjsEpPgJwIzVp3wvdsc2rPVfE aAERc1spUZE/umOzYmJf+VElrBMiAuRfE8DGieK/EB4j0BCoDaKDUuky+yV06SJrYbGz F6aIBnXz1aDLAqaWzllLo35IDxFwKfA3xdvxqd16PtWMM/C8HVU37buRdhzp6E/AnQYb kjdQ== X-Gm-Message-State: AOAM5327cTNr2b8lz82skVwq4YiNAuhA9055nW2cEx920HpU4SqKYJww ic+0fESTj0yZX/k/It8hCRXanMXBSb4= X-Google-Smtp-Source: ABdhPJzI+MtrDVlG11wonujP7QX4Nxd3vHDLX+tmWXCa5c7bJGWyMWX/79S7BtiPGOoJ0XdQnarimQ== X-Received: by 2002:a50:b282:: with SMTP id p2mr33721334edd.210.1608911580396; Fri, 25 Dec 2020 07:53:00 -0800 (PST) Received: from sblaptop.fritz.box (ipbcc1aa4b.dynamic.kabel-deutschland.de. [188.193.170.75]) by smtp.gmail.com with ESMTPSA id ho12sm13733010ejc.45.2020.12.25.07.52.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 25 Dec 2020 07:52:59 -0800 (PST) From: Andreas Rheinhardt To: ffmpeg-devel@ffmpeg.org Date: Fri, 25 Dec 2020 16:47:22 +0100 Message-Id: <20201225154724.287465-5-andreas.rheinhardt@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20201225154724.287465-1-andreas.rheinhardt@gmail.com> References: <20201225154724.287465-1-andreas.rheinhardt@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 5/7] Revert "avcodec: add FF_CODEC_CAP_INIT_CLEANUP for all codecs which use ff_mpv_common_init()" X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Andreas Rheinhardt Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" This mostly reverts commit 4b2863ff01b1fe93d9a518523c9098d17a9d8c6f. Said commit removed the freeing code from ff_mpv_common_init(), ff_mpv_common_frame_size_change() and ff_mpeg_framesize_alloc() and instead added the FF_CODEC_CAP_INIT_CLEANUP to several codecs that use ff_mpv_common_init(). This introduced several bugs: a) Several decoders using ff_mpv_common_init() in their init function were forgotten: This affected FLV, Intel H.263, RealVideo 3.0 and V4.0 as well as VC-1/WMV3. b) ff_mpv_common_init() is not only called from the init function of codecs, it is also called from AVCodec.decode functions. If an error happens after an allocation has succeeded, it can lead to memleaks; furthermore, it is now possible for the MpegEncContext to be marked as initialized even when ff_mpv_common_init() returns an error and this can lead to segfaults because decoders that call ff_mpv_common_init() when decoding a frame can mistakenly think that the MpegEncContext has been properly initialized. This can e.g. happen with H.261 or MPEG-4. c) Removing code for freeing from ff_mpeg_framesize_alloc() (which can't be called from any init function) can lead to segfaults because the check for whether it needs to called consists of checking whether the first of the buffers allocated there has been allocated. d) ff_mpv_common_frame_size_change() can also not be reached from any AVCodec.init function; yet the changes can e.g. lead to segfaults with decoders using ff_h263_decode_frame() upon allocation failure, because the MpegEncContext will upon return be flagged as both initialized and not in need of reinitialization (granted, the fact that ff_h263_decode_frame() clears context_reinit before the context has been reinited is a bug in itself). With the earlier version, the context would be cleaned upon failure and it would be attempted to initialize the context again in the next call to ff_h263_decode_frame(). While a) could be fixed by adding the missing FF_CODEC_CAP_INIT_CLEANUP, keeping the current approach would entail adding cleanup code to several other places because of b). Therefore ff_mpv_common_init() is again made to clean up after itself; the changes to the wmv2 decoder and the SVQ1 encoder have not been reverted: The former fixed a memleak, the latter allowed to remove cleanup code. Signed-off-by: Andreas Rheinhardt --- libavcodec/h261dec.c | 1 - libavcodec/h263dec.c | 4 ++-- libavcodec/mpeg12dec.c | 9 +++++---- libavcodec/mpeg4videodec.c | 3 +-- libavcodec/mpegpicture.c | 4 +++- libavcodec/mpegvideo.c | 31 ++++++++++++++++++++----------- libavcodec/msmpeg4dec.c | 8 ++++---- libavcodec/rv10.c | 2 -- 8 files changed, 35 insertions(+), 27 deletions(-) diff --git a/libavcodec/h261dec.c b/libavcodec/h261dec.c index 4f1f22b279..6c54ca79fe 100644 --- a/libavcodec/h261dec.c +++ b/libavcodec/h261dec.c @@ -683,6 +683,5 @@ AVCodec ff_h261_decoder = { .close = h261_decode_end, .decode = h261_decode_frame, .capabilities = AV_CODEC_CAP_DR1, - .caps_internal = FF_CODEC_CAP_INIT_CLEANUP, .max_lowres = 3, }; diff --git a/libavcodec/h263dec.c b/libavcodec/h263dec.c index 32e26a57de..39881dc713 100644 --- a/libavcodec/h263dec.c +++ b/libavcodec/h263dec.c @@ -770,7 +770,7 @@ AVCodec ff_h263_decoder = { .decode = ff_h263_decode_frame, .capabilities = AV_CODEC_CAP_DRAW_HORIZ_BAND | AV_CODEC_CAP_DR1 | AV_CODEC_CAP_TRUNCATED | AV_CODEC_CAP_DELAY, - .caps_internal = FF_CODEC_CAP_SKIP_FRAME_FILL_PARAM | FF_CODEC_CAP_INIT_CLEANUP, + .caps_internal = FF_CODEC_CAP_SKIP_FRAME_FILL_PARAM, .flush = ff_mpeg_flush, .max_lowres = 3, .pix_fmts = ff_h263_hwaccel_pixfmt_list_420, @@ -788,7 +788,7 @@ AVCodec ff_h263p_decoder = { .decode = ff_h263_decode_frame, .capabilities = AV_CODEC_CAP_DRAW_HORIZ_BAND | AV_CODEC_CAP_DR1 | AV_CODEC_CAP_TRUNCATED | AV_CODEC_CAP_DELAY, - .caps_internal = FF_CODEC_CAP_SKIP_FRAME_FILL_PARAM | FF_CODEC_CAP_INIT_CLEANUP, + .caps_internal = FF_CODEC_CAP_SKIP_FRAME_FILL_PARAM, .flush = ff_mpeg_flush, .max_lowres = 3, .pix_fmts = ff_h263_hwaccel_pixfmt_list_420, diff --git a/libavcodec/mpeg12dec.c b/libavcodec/mpeg12dec.c index 66556c7a82..ebafac209f 100644 --- a/libavcodec/mpeg12dec.c +++ b/libavcodec/mpeg12dec.c @@ -2855,7 +2855,8 @@ static av_cold int mpeg_decode_end(AVCodecContext *avctx) { Mpeg1Context *s = avctx->priv_data; - ff_mpv_common_end(&s->mpeg_enc_ctx); + if (s->mpeg_enc_ctx_allocated) + ff_mpv_common_end(&s->mpeg_enc_ctx); av_buffer_unref(&s->a53_buf_ref); return 0; } @@ -2872,7 +2873,7 @@ AVCodec ff_mpeg1video_decoder = { .capabilities = AV_CODEC_CAP_DRAW_HORIZ_BAND | AV_CODEC_CAP_DR1 | AV_CODEC_CAP_TRUNCATED | AV_CODEC_CAP_DELAY | AV_CODEC_CAP_SLICE_THREADS, - .caps_internal = FF_CODEC_CAP_SKIP_FRAME_FILL_PARAM | FF_CODEC_CAP_INIT_CLEANUP, + .caps_internal = FF_CODEC_CAP_SKIP_FRAME_FILL_PARAM, .flush = flush, .max_lowres = 3, .hw_configs = (const AVCodecHWConfigInternal*[]) { @@ -2904,7 +2905,7 @@ AVCodec ff_mpeg2video_decoder = { .capabilities = AV_CODEC_CAP_DRAW_HORIZ_BAND | AV_CODEC_CAP_DR1 | AV_CODEC_CAP_TRUNCATED | AV_CODEC_CAP_DELAY | AV_CODEC_CAP_SLICE_THREADS, - .caps_internal = FF_CODEC_CAP_SKIP_FRAME_FILL_PARAM | FF_CODEC_CAP_INIT_CLEANUP, + .caps_internal = FF_CODEC_CAP_SKIP_FRAME_FILL_PARAM, .flush = flush, .max_lowres = 3, .profiles = NULL_IF_CONFIG_SMALL(ff_mpeg2_video_profiles), @@ -2948,7 +2949,7 @@ AVCodec ff_mpegvideo_decoder = { .close = mpeg_decode_end, .decode = mpeg_decode_frame, .capabilities = AV_CODEC_CAP_DRAW_HORIZ_BAND | AV_CODEC_CAP_DR1 | AV_CODEC_CAP_TRUNCATED | AV_CODEC_CAP_DELAY | AV_CODEC_CAP_SLICE_THREADS, - .caps_internal = FF_CODEC_CAP_SKIP_FRAME_FILL_PARAM | FF_CODEC_CAP_INIT_CLEANUP, + .caps_internal = FF_CODEC_CAP_SKIP_FRAME_FILL_PARAM, .flush = flush, .max_lowres = 3, }; diff --git a/libavcodec/mpeg4videodec.c b/libavcodec/mpeg4videodec.c index d85109c857..5b1de03e1c 100644 --- a/libavcodec/mpeg4videodec.c +++ b/libavcodec/mpeg4videodec.c @@ -3585,8 +3585,7 @@ AVCodec ff_mpeg4_decoder = { AV_CODEC_CAP_TRUNCATED | AV_CODEC_CAP_DELAY | AV_CODEC_CAP_FRAME_THREADS, .caps_internal = FF_CODEC_CAP_SKIP_FRAME_FILL_PARAM | - FF_CODEC_CAP_ALLOCATE_PROGRESS | - FF_CODEC_CAP_INIT_CLEANUP, + FF_CODEC_CAP_ALLOCATE_PROGRESS, .flush = ff_mpeg_flush, .max_lowres = 3, .pix_fmts = ff_h263_hwaccel_pixfmt_list_420, diff --git a/libavcodec/mpegpicture.c b/libavcodec/mpegpicture.c index 13c11ec492..2aeabf221a 100644 --- a/libavcodec/mpegpicture.c +++ b/libavcodec/mpegpicture.c @@ -79,8 +79,10 @@ int ff_mpeg_framesize_alloc(AVCodecContext *avctx, MotionEstContext *me, // linesize * interlaced * MBsize // we also use this buffer for encoding in encode_mb_internal() needig an additional 32 lines if (!FF_ALLOCZ_TYPED_ARRAY(sc->edge_emu_buffer, alloc_size * EMU_EDGE_HEIGHT) || - !FF_ALLOCZ_TYPED_ARRAY(me->scratchpad, alloc_size * 4 * 16 * 2)) + !FF_ALLOCZ_TYPED_ARRAY(me->scratchpad, alloc_size * 4 * 16 * 2)) { + av_freep(&sc->edge_emu_buffer); return AVERROR(ENOMEM); + } me->temp = me->scratchpad; sc->rd_scratchpad = me->scratchpad; sc->b_scratchpad = me->scratchpad; diff --git a/libavcodec/mpegvideo.c b/libavcodec/mpegvideo.c index c28d1adef7..1bf25566b8 100644 --- a/libavcodec/mpegvideo.c +++ b/libavcodec/mpegvideo.c @@ -933,17 +933,17 @@ av_cold int ff_mpv_common_init(MpegEncContext *s) for (i = 0; i < MAX_PICTURE_COUNT; i++) { s->picture[i].f = av_frame_alloc(); if (!s->picture[i].f) - return AVERROR(ENOMEM); + goto fail_nomem; } if (!(s->next_picture.f = av_frame_alloc()) || !(s->last_picture.f = av_frame_alloc()) || !(s->current_picture.f = av_frame_alloc()) || !(s->new_picture.f = av_frame_alloc())) - return AVERROR(ENOMEM); + goto fail_nomem; if ((ret = init_context_frame(s))) - return AVERROR(ENOMEM); + goto fail; s->parse_context.state = -1; @@ -957,10 +957,10 @@ av_cold int ff_mpv_common_init(MpegEncContext *s) if (i) { s->thread_context[i] = av_memdup(s, sizeof(MpegEncContext)); if (!s->thread_context[i]) - return AVERROR(ENOMEM); + goto fail_nomem; } if ((ret = init_duplicate_context(s->thread_context[i])) < 0) - return ret; + goto fail; s->thread_context[i]->start_mb_y = (s->mb_height * (i) + nb_slices / 2) / nb_slices; s->thread_context[i]->end_mb_y = @@ -968,7 +968,7 @@ av_cold int ff_mpv_common_init(MpegEncContext *s) } } else { if ((ret = init_duplicate_context(s)) < 0) - return ret; + goto fail; s->start_mb_y = 0; s->end_mb_y = s->mb_height; } @@ -976,6 +976,11 @@ av_cold int ff_mpv_common_init(MpegEncContext *s) // } return 0; + fail_nomem: + ret = AVERROR(ENOMEM); + fail: + ff_mpv_common_end(s); + return ret; } /** @@ -1068,10 +1073,10 @@ int ff_mpv_common_frame_size_change(MpegEncContext *s) if ((s->width || s->height) && (err = av_image_check_size(s->width, s->height, 0, s->avctx)) < 0) - return err; + goto fail; if ((err = init_context_frame(s))) - return err; + goto fail; memset(s->thread_context, 0, sizeof(s->thread_context)); s->thread_context[0] = s; @@ -1083,11 +1088,12 @@ int ff_mpv_common_frame_size_change(MpegEncContext *s) if (i) { s->thread_context[i] = av_memdup(s, sizeof(MpegEncContext)); if (!s->thread_context[i]) { - return AVERROR(ENOMEM); + err = AVERROR(ENOMEM); + goto fail; } } if ((err = init_duplicate_context(s->thread_context[i])) < 0) - return err; + goto fail; s->thread_context[i]->start_mb_y = (s->mb_height * (i) + nb_slices / 2) / nb_slices; s->thread_context[i]->end_mb_y = @@ -1096,7 +1102,7 @@ int ff_mpv_common_frame_size_change(MpegEncContext *s) } else { err = init_duplicate_context(s); if (err < 0) - return err; + goto fail; s->start_mb_y = 0; s->end_mb_y = s->mb_height; } @@ -1104,6 +1110,9 @@ int ff_mpv_common_frame_size_change(MpegEncContext *s) } return 0; + fail: + ff_mpv_common_end(s); + return err; } /* init common structure for both encoder and decoder */ diff --git a/libavcodec/msmpeg4dec.c b/libavcodec/msmpeg4dec.c index 49df06a9d7..16b67192b5 100644 --- a/libavcodec/msmpeg4dec.c +++ b/libavcodec/msmpeg4dec.c @@ -888,7 +888,7 @@ AVCodec ff_msmpeg4v1_decoder = { .close = ff_h263_decode_end, .decode = ff_h263_decode_frame, .capabilities = AV_CODEC_CAP_DRAW_HORIZ_BAND | AV_CODEC_CAP_DR1, - .caps_internal = FF_CODEC_CAP_SKIP_FRAME_FILL_PARAM | FF_CODEC_CAP_INIT_CLEANUP, + .caps_internal = FF_CODEC_CAP_SKIP_FRAME_FILL_PARAM, .max_lowres = 3, .pix_fmts = (const enum AVPixelFormat[]) { AV_PIX_FMT_YUV420P, @@ -906,7 +906,7 @@ AVCodec ff_msmpeg4v2_decoder = { .close = ff_h263_decode_end, .decode = ff_h263_decode_frame, .capabilities = AV_CODEC_CAP_DRAW_HORIZ_BAND | AV_CODEC_CAP_DR1, - .caps_internal = FF_CODEC_CAP_SKIP_FRAME_FILL_PARAM | FF_CODEC_CAP_INIT_CLEANUP, + .caps_internal = FF_CODEC_CAP_SKIP_FRAME_FILL_PARAM, .max_lowres = 3, .pix_fmts = (const enum AVPixelFormat[]) { AV_PIX_FMT_YUV420P, @@ -924,7 +924,7 @@ AVCodec ff_msmpeg4v3_decoder = { .close = ff_h263_decode_end, .decode = ff_h263_decode_frame, .capabilities = AV_CODEC_CAP_DRAW_HORIZ_BAND | AV_CODEC_CAP_DR1, - .caps_internal = FF_CODEC_CAP_SKIP_FRAME_FILL_PARAM | FF_CODEC_CAP_INIT_CLEANUP, + .caps_internal = FF_CODEC_CAP_SKIP_FRAME_FILL_PARAM, .max_lowres = 3, .pix_fmts = (const enum AVPixelFormat[]) { AV_PIX_FMT_YUV420P, @@ -942,7 +942,7 @@ AVCodec ff_wmv1_decoder = { .close = ff_h263_decode_end, .decode = ff_h263_decode_frame, .capabilities = AV_CODEC_CAP_DRAW_HORIZ_BAND | AV_CODEC_CAP_DR1, - .caps_internal = FF_CODEC_CAP_SKIP_FRAME_FILL_PARAM | FF_CODEC_CAP_INIT_CLEANUP, + .caps_internal = FF_CODEC_CAP_SKIP_FRAME_FILL_PARAM, .max_lowres = 3, .pix_fmts = (const enum AVPixelFormat[]) { AV_PIX_FMT_YUV420P, diff --git a/libavcodec/rv10.c b/libavcodec/rv10.c index d118515c3b..dec7bbea58 100644 --- a/libavcodec/rv10.c +++ b/libavcodec/rv10.c @@ -688,7 +688,6 @@ AVCodec ff_rv10_decoder = { .close = rv10_decode_end, .decode = rv10_decode_frame, .capabilities = AV_CODEC_CAP_DR1, - .caps_internal = FF_CODEC_CAP_INIT_CLEANUP, .max_lowres = 3, .pix_fmts = (const enum AVPixelFormat[]) { AV_PIX_FMT_YUV420P, @@ -706,7 +705,6 @@ AVCodec ff_rv20_decoder = { .close = rv10_decode_end, .decode = rv10_decode_frame, .capabilities = AV_CODEC_CAP_DR1 | AV_CODEC_CAP_DELAY, - .caps_internal = FF_CODEC_CAP_INIT_CLEANUP, .flush = ff_mpeg_flush, .max_lowres = 3, .pix_fmts = (const enum AVPixelFormat[]) { From patchwork Fri Dec 25 15:47:23 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andreas Rheinhardt X-Patchwork-Id: 24652 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 9A15744B6ED for ; Fri, 25 Dec 2020 17:53:10 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 7E75868AE8C; Fri, 25 Dec 2020 17:53:10 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-ej1-f52.google.com (mail-ej1-f52.google.com [209.85.218.52]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 472AC68AE74 for ; Fri, 25 Dec 2020 17:53:02 +0200 (EET) Received: by mail-ej1-f52.google.com with SMTP id g20so6660208ejb.1 for ; Fri, 25 Dec 2020 07:53:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references:reply-to :mime-version:content-transfer-encoding; bh=ZUtEg9Yn/Kpn/iT1TOyhlDIlpavP32iMofyrKY4qfaQ=; b=JvIQmQkBXEqHiL2wrSFl72ieDpgJW0L7LrdL1d+I6TaKom7ljBK0zvarvFQiZtJN7c G3t6gaEW04JqPuMoEwwHK7XcXZGuop1KSvqcyY7XXwoVRrvMVhfI5ett8qV2HOBI8MYV UEzJBlL+LXARe/+/q+t+jp11WPW3VERazJdeui6p5QjNuSImDhAH+ev9uFbbIuen9U9P TCGXAUexGk1RZcYSVKhRAwDGcvS2fjPzR7WoKrrNR8T8QR5vX1AdbJoDgLtmsRw9JkGK N9fijcscMSCthWlg33I0kXkd5SCRr4I14e6saG0FUDNOp2aDdZFAiSqipovOUtiExtQz BaMQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:reply-to:mime-version:content-transfer-encoding; bh=ZUtEg9Yn/Kpn/iT1TOyhlDIlpavP32iMofyrKY4qfaQ=; b=gTagFMO3Vp7WW2QWnFBmNbftwEs8aTKYVhoZN6kR3r1WNVNK2Xt4ugCBg1GJa67SYE 13k5WEmH6iUbuip7EvXWrRClSqVXEBbrRf5EonWhqdSdAka8ZSpawLfvmAF/2ZoAdMyo MWqpRcYXj1AnnWy63UQ1TRItwOjQIVjMO7xwQgdq2dj9PF/2lkJOIbPZn4KE0JhIPonH GHFnwokJO5t0eZTVDopj0c94scs0gsRt4gBAXgG35ODu84K0TxT4/m6jkk6Pjv5ga2g2 CNX6QdXImHsohLoA2bpxQMELqgCUdUnhmN83YdfLIggjjwmaqQrcXfMzUFdd0FVRkfKO 2J8A== X-Gm-Message-State: AOAM532Z0NsKSApT0q+OLOklL5AQ/E4vDT7HDyh7xJ2NSBWeZM4bD+2y 93ASMq83xHcp2JpGs/gCBSoV5Fpec+M= X-Google-Smtp-Source: ABdhPJzRQzqiAEkuynTh+CwK272zU8fOJ60IGz6d6f9E5A4Cs88imGTuxFfuDrf1cYu/5pmHB5dh0Q== X-Received: by 2002:a17:906:1151:: with SMTP id i17mr32539761eja.250.1608911581449; Fri, 25 Dec 2020 07:53:01 -0800 (PST) Received: from sblaptop.fritz.box (ipbcc1aa4b.dynamic.kabel-deutschland.de. [188.193.170.75]) by smtp.gmail.com with ESMTPSA id ho12sm13733010ejc.45.2020.12.25.07.53.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 25 Dec 2020 07:53:00 -0800 (PST) From: Andreas Rheinhardt To: ffmpeg-devel@ffmpeg.org Date: Fri, 25 Dec 2020 16:47:23 +0100 Message-Id: <20201225154724.287465-6-andreas.rheinhardt@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20201225154724.287465-1-andreas.rheinhardt@gmail.com> References: <20201225154724.287465-1-andreas.rheinhardt@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 6/7] avcodec/mpegvideo: Fix memleak upon allocation error X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Andreas Rheinhardt Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" When slice-threading is used, ff_mpv_common_init() duplicates the first MpegEncContext and allocates some buffers for each MpegEncContext (the first as well as the copies). But the count of allocated MpegEncContexts is not updated until after everything has been allocated and if an error happens after the first one has been allocated, only the first one is freed; the others leak. This commit fixes this: The count is now set before the copies are allocated. Furthermore, the copies are now created and initialized before the first MpegEncContext, so that the buffers exclusively owned by each MpegEncContext are still NULL in the src MpegEncContext so that no double-free happens upon allocation failure. Given that this effectively touches every line of the init code, it has also been factored out in a function of its own in order to remove code duplication with the same code in ff_mpv_common_frame_size_change() (which was never called when using more than one slice (and if it were, there would be potential double-frees)). Signed-off-by: Andreas Rheinhardt --- libavcodec/mpegvideo.c | 89 +++++++++++++++++------------------------- 1 file changed, 36 insertions(+), 53 deletions(-) diff --git a/libavcodec/mpegvideo.c b/libavcodec/mpegvideo.c index 1bf25566b8..3daccb816f 100644 --- a/libavcodec/mpegvideo.c +++ b/libavcodec/mpegvideo.c @@ -364,13 +364,6 @@ static int init_duplicate_context(MpegEncContext *s) if (s->mb_height & 1) yc_size += 2*s->b8_stride + 2*s->mb_stride; - s->sc.edge_emu_buffer = - s->me.scratchpad = - s->me.temp = - s->sc.rd_scratchpad = - s->sc.b_scratchpad = - s->sc.obmc_scratchpad = NULL; - if (s->encoding) { if (!FF_ALLOCZ_TYPED_ARRAY(s->me.map, ME_MAP_SIZE) || !FF_ALLOCZ_TYPED_ARRAY(s->me.score_map, ME_MAP_SIZE)) @@ -411,6 +404,35 @@ static int init_duplicate_context(MpegEncContext *s) return 0; } +/** + * Initialize an MpegEncContext's thread contexts. Presumes that + * slice_context_count is already set and that all the fields + * that are freed/reset in free_duplicate_context() are NULL. + */ +static int init_duplicate_contexts(MpegEncContext *s) +{ + int nb_slices = s->slice_context_count, ret; + + /* We initialize the copies before the original so that + * fields allocated in init_duplicate_context are NULL after + * copying. This prevents double-frees upon allocation error. */ + for (int i = 1; i < nb_slices; i++) { + s->thread_context[i] = av_memdup(s, sizeof(MpegEncContext)); + if (!s->thread_context[i]) + return AVERROR(ENOMEM); + if ((ret = init_duplicate_context(s->thread_context[i])) < 0) + return ret; + s->thread_context[i]->start_mb_y = + (s->mb_height * (i ) + nb_slices / 2) / nb_slices; + s->thread_context[i]->end_mb_y = + (s->mb_height * (i + 1) + nb_slices / 2) / nb_slices; + } + s->start_mb_y = 0; + s->end_mb_y = nb_slices > 1 ? (s->mb_height + nb_slices / 2) / nb_slices + : s->mb_height; + return init_duplicate_context(s); +} + static void free_duplicate_context(MpegEncContext *s) { if (!s) @@ -950,29 +972,12 @@ av_cold int ff_mpv_common_init(MpegEncContext *s) s->context_initialized = 1; memset(s->thread_context, 0, sizeof(s->thread_context)); s->thread_context[0] = s; + s->slice_context_count = nb_slices; // if (s->width && s->height) { - if (nb_slices > 1) { - for (i = 0; i < nb_slices; i++) { - if (i) { - s->thread_context[i] = av_memdup(s, sizeof(MpegEncContext)); - if (!s->thread_context[i]) - goto fail_nomem; - } - if ((ret = init_duplicate_context(s->thread_context[i])) < 0) - goto fail; - s->thread_context[i]->start_mb_y = - (s->mb_height * (i) + nb_slices / 2) / nb_slices; - s->thread_context[i]->end_mb_y = - (s->mb_height * (i + 1) + nb_slices / 2) / nb_slices; - } - } else { - if ((ret = init_duplicate_context(s)) < 0) - goto fail; - s->start_mb_y = 0; - s->end_mb_y = s->mb_height; - } - s->slice_context_count = nb_slices; + ret = init_duplicate_contexts(s); + if (ret < 0) + goto fail; // } return 0; @@ -1082,31 +1087,9 @@ int ff_mpv_common_frame_size_change(MpegEncContext *s) s->thread_context[0] = s; if (s->width && s->height) { - int nb_slices = s->slice_context_count; - if (nb_slices > 1) { - for (i = 0; i < nb_slices; i++) { - if (i) { - s->thread_context[i] = av_memdup(s, sizeof(MpegEncContext)); - if (!s->thread_context[i]) { - err = AVERROR(ENOMEM); - goto fail; - } - } - if ((err = init_duplicate_context(s->thread_context[i])) < 0) - goto fail; - s->thread_context[i]->start_mb_y = - (s->mb_height * (i) + nb_slices / 2) / nb_slices; - s->thread_context[i]->end_mb_y = - (s->mb_height * (i + 1) + nb_slices / 2) / nb_slices; - } - } else { - err = init_duplicate_context(s); - if (err < 0) - goto fail; - s->start_mb_y = 0; - s->end_mb_y = s->mb_height; - } - s->slice_context_count = nb_slices; + err = init_duplicate_contexts(s); + if (err < 0) + goto fail; } return 0; From patchwork Fri Dec 25 15:47:24 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andreas Rheinhardt X-Patchwork-Id: 24653 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id B0C0B44B6ED for ; Fri, 25 Dec 2020 17:53:11 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 9FBD368AE80; Fri, 25 Dec 2020 17:53:11 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-ej1-f51.google.com (mail-ej1-f51.google.com [209.85.218.51]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 1F37468AE71 for ; Fri, 25 Dec 2020 17:53:03 +0200 (EET) Received: by mail-ej1-f51.google.com with SMTP id lt17so6638763ejb.3 for ; Fri, 25 Dec 2020 07:53:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references:reply-to :mime-version:content-transfer-encoding; bh=B+OpOm4tQ+SqlMrmIVwFhYfx8xtMxyhzQFBAQunaUj4=; b=KNXrr5DLV4pEVI4OwOsIAhH5CDmHly4JK4ADKwoy53wKaF6oEkhvaNY9Q1tAI11htx f40ejLng/I5tWk7Ursw64gL/aVyyscCG2Zap8ux+g2m5DvfRUcY50NfwdmvFLVN18FfY C1Y00Cd3AIvA2b7uEsCUHZYmCs3N12w8b5KqyAsQQ4duYUeu97sx9cf/fzxKmLEUL4ow +ZuLxDvt1X3kyGmh6xwHaWDuq0RDAcmPfT7I+iwbMHUDcstWf9Bths5EMOeKDGa3aEFK 4+OSd6aBJs/1Cew+R3rqhQoODixAanYfB7pIzznhpoL37zT5Vk3W6DsELJnhPlUShZrI D21A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:reply-to:mime-version:content-transfer-encoding; bh=B+OpOm4tQ+SqlMrmIVwFhYfx8xtMxyhzQFBAQunaUj4=; b=ZUdxz15lSJL9HXgHo9X/WK9LwSp5JsO3hNlBJ8rACkNGH9Ee+CqUhUg/SSRG6A2Ud4 39aI/QKph0IrNlr7SWryeH2Fo0qeiDrMR9qs0+yu8no5bSlurCgmR7GIueRtxKCW0x5l iZfcukBhqeGeabZwErw+qu9gVMdy65lolFZg6uoA9yj0p1gZet7gWJ2mATuUiVmpoRvf ohUXw4qtISvm/0IyWSTn0vXHAVI4O9Js9GcXPiwc2q6usJPM9yAja6/O5AHrmWXW2FHD lY7qF4cWehB3468Qok1urvtXxkn47bpGRSitZr0AYEfB4N/B8wfXRxwBeXR4jQWjWzOV tqCw== X-Gm-Message-State: AOAM530zFyn+WiyDG/cLa3/iZPWbLz80xUaRrA0XHw67hehMbLtwP5sk baig2qYhBuDulh3vuD1eZNbJMaRxROE= X-Google-Smtp-Source: ABdhPJxBuGBH5TxpBl2GXGbLLEufaCTCstZwJ+v+b6z/doL/4S1MnKH4tQW3/u0iX8Hwxw9nQJ6C8A== X-Received: by 2002:a17:906:5293:: with SMTP id c19mr32487322ejm.72.1608911582421; Fri, 25 Dec 2020 07:53:02 -0800 (PST) Received: from sblaptop.fritz.box (ipbcc1aa4b.dynamic.kabel-deutschland.de. [188.193.170.75]) by smtp.gmail.com with ESMTPSA id ho12sm13733010ejc.45.2020.12.25.07.53.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 25 Dec 2020 07:53:01 -0800 (PST) From: Andreas Rheinhardt To: ffmpeg-devel@ffmpeg.org Date: Fri, 25 Dec 2020 16:47:24 +0100 Message-Id: <20201225154724.287465-7-andreas.rheinhardt@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20201225154724.287465-1-andreas.rheinhardt@gmail.com> References: <20201225154724.287465-1-andreas.rheinhardt@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 7/7] avcodec/mpegvideo: Factor common freeing code out X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Andreas Rheinhardt Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Signed-off-by: Andreas Rheinhardt --- libavcodec/mpegvideo.c | 30 ++++++++++++------------------ 1 file changed, 12 insertions(+), 18 deletions(-) diff --git a/libavcodec/mpegvideo.c b/libavcodec/mpegvideo.c index 3daccb816f..dd50c03306 100644 --- a/libavcodec/mpegvideo.c +++ b/libavcodec/mpegvideo.c @@ -455,6 +455,15 @@ static void free_duplicate_context(MpegEncContext *s) s->block = NULL; } +static void free_duplicate_contexts(MpegEncContext *s) +{ + for (int i = 1; i < s->slice_context_count; i++) { + free_duplicate_context(s->thread_context[i]); + av_freep(&s->thread_context[i]); + } + free_duplicate_context(s); +} + static void backup_duplicate_context(MpegEncContext *bak, MpegEncContext *src) { #define COPY(a) bak->a = src->a @@ -1049,16 +1058,7 @@ int ff_mpv_common_frame_size_change(MpegEncContext *s) if (!s->context_initialized) return AVERROR(EINVAL); - if (s->slice_context_count > 1) { - for (i = 0; i < s->slice_context_count; i++) { - free_duplicate_context(s->thread_context[i]); - } - for (i = 1; i < s->slice_context_count; i++) { - av_freep(&s->thread_context[i]); - } - } else - free_duplicate_context(s); - + free_duplicate_contexts(s); free_context_frame(s); if (s->picture) @@ -1106,15 +1106,9 @@ void ff_mpv_common_end(MpegEncContext *s) if (!s) return; - if (s->slice_context_count > 1) { - for (i = 0; i < s->slice_context_count; i++) { - free_duplicate_context(s->thread_context[i]); - } - for (i = 1; i < s->slice_context_count; i++) { - av_freep(&s->thread_context[i]); - } + free_duplicate_contexts(s); + if (s->slice_context_count > 1) s->slice_context_count = 1; - } else free_duplicate_context(s); av_freep(&s->parse_context.buffer); s->parse_context.buffer_size = 0;