From patchwork Mon Nov 1 22:03:50 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Almer X-Patchwork-Id: 31273 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a5e:a610:0:0:0:0:0 with SMTP id q16csp3202491ioi; Mon, 1 Nov 2021 15:05:02 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyP0oQPQEEdxXIYUtA+ljcaWqN+smMZXjsggRKNcfnybgWeoM8T5jxTGs9fZbGgQCOgnh/i X-Received: by 2002:a17:906:26c4:: with SMTP id u4mr39515678ejc.511.1635804302633; Mon, 01 Nov 2021 15:05:02 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1635804302; cv=none; d=google.com; s=arc-20160816; b=V+qsxtMcOK4MdRc29SYT0eAhRJUwaQVKX9CMuxBO922qFDtjeJbCzZQZnOLWcFnp3M kmHKOlpOLYLnvT9hzIRspJh4ONBiyL/CfOgJwjrBByWik0c2ntYvgBGWoYdz0G91swdT 00t2ofwHArxpEHMf0SzR31rJ3E450GNbd/JHg2KPuBFy6jS82xWDOTfGNsyjjrQ28HZl 2shcg6QmsVQK2BnYCh78B82i67oVOXlkESPSQlsGIhL1RnT2icFYmX6LATSsVRecAT/D dnZUfrvjzE+Izhdt8BXd8arsHsr0JBzwWWmxJNAP83BqC6yy1oefAsMRLdRlYRXKywtT tsHg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:message-id:date:to:from :dkim-signature:delivered-to; bh=k5CY26k+FtD8mINlfyw5pLTR/dxWr7d5ou0gcnc1MMo=; b=r2UNGQLj2aMPSWpxkEMoHi+3OzH3IOGDE6VrDPN7qnYI27qucw04OCQdREGFxiteEd obBBw5NIz1z25rasdqAcE8zytHJG1t3WYUa31jG6IYd/tSAjHYHPBrPTTQKFBxCPEaln fzEhQloAhlT9t5lOVX32WEDwYCZTr+ENzVzIf5lJhQeByODs30rUMgERlgh5SkzQQIpZ IzJ8a6SPl8vsBVYTq1ZMQcE7g1s4FzCxQUEfGjBfyM359NR3azylc1QqXjYtBanZvS0L 3uejIVBrrudxt4fWs6gyYOBwWpoWTWobdqC8dsO8FEHUN7iIHnXYGIgRAdSaEq96ShBb E9zg== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b="a/ivznqx"; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id sg12si37438321ejc.729.2021.11.01.15.05.01; Mon, 01 Nov 2021 15:05:02 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b="a/ivznqx"; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 0F88A689247; Tue, 2 Nov 2021 00:04:58 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-qt1-f170.google.com (mail-qt1-f170.google.com [209.85.160.170]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id E329E68819D for ; Tue, 2 Nov 2021 00:04:50 +0200 (EET) Received: by mail-qt1-f170.google.com with SMTP id t40so17246927qtc.6 for ; Mon, 01 Nov 2021 15:04:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:subject:date:message-id:mime-version :content-transfer-encoding; bh=mq07/bAS2xqwJDt6AzOjMs29P0twQzHEi+6aKntRU/Q=; b=a/ivznqxsfxAxiHgW6CQgXC7C1gUafec89JEiJFxGt64hiDkPftCH2+z1Y5b2nzjmZ L+1q6DADYPenle+gZKWf0TOTBtL11Ruhmmh1QyNEl49NrHwfEn512WH8fzaYdlSwy9gL DKcG7U2bYsqs1bldAnRLxwz+cH0dnuoazsmHAXVVbJVh9k+FHu57uxXyeKhZ4D7MihGN GOKoWXXAGmabDTKK3Ir8UuFQ2PGyM46820i6+TyW8qVhzY7SZt5Ebx2jdAM6S+T0GP7Z 8QpAtnLX/tQB7RPM3toeQg4dsTaEyXS8Z5d1CbLhm+XPMV4Sjw6C1fbzFU5evdE4BiuI snmQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:mime-version :content-transfer-encoding; bh=mq07/bAS2xqwJDt6AzOjMs29P0twQzHEi+6aKntRU/Q=; b=rklsq8sm+ByDRHTSu/eMDqQaaLN1er60WXqgIcUr7yyfY4Ari4QvV8ldJLc/TZqPH9 Sc06j0yg6CM9hxdtOVjagYEETwy6foKcuH7P4eLgvWs5DqMm26a/oVfb/6Kf7JxZW3rn aMLrInGT2ub1tD8Y8KVOgLxa5y01wAtumow3H+nz/VfZFsF/wROW3WvE5qwggN6IlO/r e8EtniuaaPFTp5Ca8Ww54L8RGL7Q6oTvFZYcDmtXipaI1eICXWqMEsMJDAwIZUf8q6m/ mae4Cm7geZ+WIY9rJbWuYeupA4ijQg904erGdHhcM5mDlsRS3ZBolVuTPFXrnB+fDOFN WXBA== X-Gm-Message-State: AOAM533Q05+1CpLytFoHkrymXJp+1e7URB3I8qq420W23bQMoTMS7Ca5 E/hWy8AKdlO1Kmr0oygrXaxFMFtZ62VP0w== X-Received: by 2002:a05:622a:134e:: with SMTP id w14mr13295187qtk.33.1635804289154; Mon, 01 Nov 2021 15:04:49 -0700 (PDT) Received: from localhost.localdomain ([191.83.220.232]) by smtp.gmail.com with ESMTPSA id d11sm10223609qtx.81.2021.11.01.15.04.47 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 01 Nov 2021 15:04:48 -0700 (PDT) From: James Almer To: ffmpeg-devel@ffmpeg.org Date: Mon, 1 Nov 2021 19:03:50 -0300 Message-Id: <20211101220350.700-1-jamrial@gmail.com> X-Mailer: git-send-email 2.33.0 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH] avcodec/libx264: fix sei payload leaks on error X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: e67xQpPjLmdf Signed-off-by: James Almer --- libavcodec/libx264.c | 23 +++++++++++++++++++---- 1 file changed, 19 insertions(+), 4 deletions(-) diff --git a/libavcodec/libx264.c b/libavcodec/libx264.c index 21f434d06d..0766b4a950 100644 --- a/libavcodec/libx264.c +++ b/libavcodec/libx264.c @@ -293,6 +293,18 @@ static void reconfig_encoder(AVCodecContext *ctx, const AVFrame *frame) } } +static void free_picture(AVCodecContext *ctx) +{ + X264Context *x4 = ctx->priv_data; + x264_picture_t *pic = &x4->pic; + + for (int i = 0; i < pic->extra_sei.num_payloads; i++) + av_free(pic->extra_sei.payloads[i].payload); + av_freep(&pic->extra_sei.payloads); + av_freep(&pic->prop.quant_offsets); + pic->extra_sei.num_payloads = 0; +} + static int X264_frame(AVCodecContext *ctx, AVPacket *pkt, const AVFrame *frame, int *got_packet) { @@ -396,15 +408,17 @@ static int X264_frame(AVCodecContext *ctx, AVPacket *pkt, const AVFrame *frame, roi = (const AVRegionOfInterest*)sd->data; roi_size = roi->self_size; if (!roi_size || sd->size % roi_size != 0) { + free_picture(ctx); av_log(ctx, AV_LOG_ERROR, "Invalid AVRegionOfInterest.self_size.\n"); return AVERROR(EINVAL); } nb_rois = sd->size / roi_size; qoffsets = av_calloc(mbx * mby, sizeof(*qoffsets)); - if (!qoffsets) + if (!qoffsets) { + free_picture(ctx); return AVERROR(ENOMEM); - + } // This list must be iterated in reverse because the first // region in the list applies when regions overlap. for (int i = nb_rois - 1; i >= 0; i--) { @@ -420,6 +434,7 @@ static int X264_frame(AVCodecContext *ctx, AVPacket *pkt, const AVFrame *frame, if (roi->qoffset.den == 0) { av_free(qoffsets); + free_picture(ctx); av_log(ctx, AV_LOG_ERROR, "AVRegionOfInterest.qoffset.den must not be zero.\n"); return AVERROR(EINVAL); } @@ -452,7 +467,7 @@ static int X264_frame(AVCodecContext *ctx, AVPacket *pkt, const AVFrame *frame, continue; tmp = av_fast_realloc(sei->payloads, &sei_data_size, (sei->num_payloads + 1) * sizeof(*sei_payload)); if (!tmp) { - av_freep(&x4->pic.prop.quant_offsets); + free_picture(ctx); return AVERROR(ENOMEM); } sei->payloads = tmp; @@ -460,7 +475,7 @@ static int X264_frame(AVCodecContext *ctx, AVPacket *pkt, const AVFrame *frame, sei_payload = &sei->payloads[sei->num_payloads]; sei_payload->payload = av_memdup(side_data->data, side_data->size); if (!sei_payload->payload) { - av_freep(&x4->pic.prop.quant_offsets); + free_picture(ctx); return AVERROR(ENOMEM); } sei_payload->payload_size = side_data->size;