From patchwork Fri Dec 4 22:57:38 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andreas Rheinhardt X-Patchwork-Id: 24347 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 77F11449191 for ; Sat, 5 Dec 2020 00:57:56 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 53C9168A5A8; Sat, 5 Dec 2020 00:57:56 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-ej1-f66.google.com (mail-ej1-f66.google.com [209.85.218.66]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 51B77687FD2 for ; Sat, 5 Dec 2020 00:57:49 +0200 (EET) Received: by mail-ej1-f66.google.com with SMTP id ce23so7215879ejb.8 for ; Fri, 04 Dec 2020 14:57:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:reply-to:mime-version :content-transfer-encoding; bh=Myfd0zv+v0PisVaIbM+Ga3Rbp6MfZq3pZnc3UFiAULU=; b=YGc+vhzRTJpJ05+TBzPQiMVwjYVwHb8jYhMLVmSid3J4rJqt9YPNb8eWEtQmpdiovG /A/18AL6i0eBPE1+1N5ZjQIbIATznqbKiZr96WwJK5vUW18qfNF3CVM9l9HmDa95t4Gx 7np0OU5QvX1mf6raYIgHkRqM65Bdn61aQyXiqF1vU0qioneb0CJFsNov4mlGIr8RlDEa UJLK0dn5pbh5iHUZUDdlw63l41Fz40pqOL1WCdBcXxGtN8gSXJ8v0stbKCg/jQRyHP6Z JhQcjmq+S4tbYY+9YtkSE7MNtzf/HgJhqMFcDDc4R+P4RGjXx5D2RDgS+Qgri4asZNbQ qf+Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:reply-to :mime-version:content-transfer-encoding; bh=Myfd0zv+v0PisVaIbM+Ga3Rbp6MfZq3pZnc3UFiAULU=; b=F1MJda5BBopUHgT1PD/DaqXA6hTC/vZ5SvboBqSGjoIEqoCxQIcn3CdqVUxkMtIk3F T7gO9/4ilVjLtm/GYq5vDHeyGGpeSYwvQYpUD/+H9P0qm6OTSXuMPZy0ZurBocXTb8fb +qOhL/Arl39UA1bqglrXb6nk1+5jRrY/uJc3/UUcTkyzTxdGWzOQ0zihq+81HVZrCiUi EPe9G1LlhHOBWsikuq/6/J8OrOpvqzn32dH1Y6eqXzrMOpq+5ZCjvbMBmMCNKjH0RakS XKWbnFhgon5lJe8xeFu7LH90aDXTPoDMT3tOq+/YmtgzURXr0ddCQoR5G+siYqfcWeNO zpsg== X-Gm-Message-State: AOAM533gaynwzX7ummWa48h3aCwXSHc/zcS9GDLj4Xvs9hNUU01fBV9z RNj8i00iUKF79w1oij39ewUOZGLRiO9mng== X-Google-Smtp-Source: ABdhPJyWlIN1FeyjGXOMs1C5PRUiiKi/dPp/cZazQNd5IcUjX3gbZMlA67xclpFBK1Hq+6dIu2kO2Q== X-Received: by 2002:a17:907:1004:: with SMTP id ox4mr9132779ejb.240.1607122668444; Fri, 04 Dec 2020 14:57:48 -0800 (PST) Received: from sblaptop.fritz.box (ipbcc1aa4b.dynamic.kabel-deutschland.de. [188.193.170.75]) by smtp.gmail.com with ESMTPSA id s6sm3941116ejb.122.2020.12.04.14.57.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 04 Dec 2020 14:57:47 -0800 (PST) From: Andreas Rheinhardt To: ffmpeg-devel@ffmpeg.org Date: Fri, 4 Dec 2020 23:57:38 +0100 Message-Id: <20201204225740.1506052-1-andreas.rheinhardt@gmail.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 1/3] avcodec/av1dec: Fix leak in case of failure X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Andreas Rheinhardt Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" A reference to an AV1RawFrameHeader and consequently the AV1RawFrameHeader itself and everything it has a reference to leak if the hardware has no AV1 decoding capabilities. It happens e.g. in the cbs-av1-av1-1-b8-02-allintra FATE-test; it has just been masked because the return value of ffmpeg (which indicates failure when using Valgrind or ASAN) is ignored when doing tests of type md5. Signed-off-by: Andreas Rheinhardt --- I switched the av_buffer_ref() and update_context_with_frame_header() because the latter does not need any cleanup on failure. Also notice that actual decoding with this patch applied is completely untested. libavcodec/av1dec.c | 20 +++++++++++--------- 1 file changed, 11 insertions(+), 9 deletions(-) diff --git a/libavcodec/av1dec.c b/libavcodec/av1dec.c index 1589b8f0c0..d7b2ac9d46 100644 --- a/libavcodec/av1dec.c +++ b/libavcodec/av1dec.c @@ -674,20 +674,20 @@ static int av1_frame_alloc(AVCodecContext *avctx, AV1Frame *f) AVFrame *frame; int ret; - f->header_ref = av_buffer_ref(s->header_ref); - if (!f->header_ref) - return AVERROR(ENOMEM); - - f->raw_frame_header = s->raw_frame_header; - ret = update_context_with_frame_header(avctx, header); if (ret < 0) { av_log(avctx, AV_LOG_ERROR, "Failed to update context with frame header\n"); return ret; } + f->header_ref = av_buffer_ref(s->header_ref); + if (!f->header_ref) + return AVERROR(ENOMEM); + + f->raw_frame_header = s->raw_frame_header; + if ((ret = ff_thread_get_buffer(avctx, &f->tf, AV_GET_BUFFER_FLAG_REF)) < 0) - return ret; + goto fail; frame = f->tf.f; frame->key_frame = header->frame_type == AV1_FRAME_KEY; @@ -710,8 +710,10 @@ static int av1_frame_alloc(AVCodecContext *avctx, AV1Frame *f) if (hwaccel->frame_priv_data_size) { f->hwaccel_priv_buf = av_buffer_allocz(hwaccel->frame_priv_data_size); - if (!f->hwaccel_priv_buf) + if (!f->hwaccel_priv_buf) { + ret = AVERROR(ENOMEM); goto fail; + } f->hwaccel_picture_private = f->hwaccel_priv_buf->data; } } @@ -719,7 +721,7 @@ static int av1_frame_alloc(AVCodecContext *avctx, AV1Frame *f) fail: av1_frame_unref(avctx, f); - return AVERROR(ENOMEM); + return ret; } static int set_output_frame(AVCodecContext *avctx, AVFrame *frame,