From patchwork Sun Nov 27 17:03:36 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Khirnov X-Patchwork-Id: 39505 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a21:999a:b0:a4:2148:650a with SMTP id ve26csp6960391pzb; Sun, 27 Nov 2022 09:08:02 -0800 (PST) X-Google-Smtp-Source: AA0mqf59k2ej7ewPxNMux6Uf+twb5KnCOjg82f+fNLYJF7Rr2OdqsojEUDnS0tenedp1rm4QCnD0 X-Received: by 2002:a05:6402:5305:b0:467:69e3:c25b with SMTP id eo5-20020a056402530500b0046769e3c25bmr43800612edb.3.1669568882228; Sun, 27 Nov 2022 09:08:02 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1669568882; cv=none; d=google.com; s=arc-20160816; b=cVAFI75FZnuKl93KiwARbXvy9SkoqldaN0OaqVdrwbdheUUECwUS9Mz+Satzf9/xj5 5Q7PTfkM3w5McbEWSJJBrXrsOQPhqgjfgByrNGg1hwBcDqAl93pURl3Z5Gq6JqjVW8BX LfEoLEyremQ+LP1DUTOA8YzGSH25roGvw9ajtOiH28dQlRABh1JS1hJKzbFgY0O/w1Tl nnRMAKR4QRQOYCZvuSwn+wKpdsgCYe3CP2skphI46+PAf/OA9nJKGmV5IHjpE0fUi9j3 0O11WwBfrQVZuUCOyqp8DGN/QJA5YyAdjGZKgKM41zwbx5GgfsL019KSUlik1VaeGQR1 zJfA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:delivered-to; bh=q/wG3FkxZ9huquzE+k8mfISv7OAr8QViFVM8j7cEOew=; b=A2/Zxqf9EujxlMnHM2KtSezUZu+POsKR9ZxSW+L/NMpFyvIGEVXisLy2Whv3bz29/H pj1OQYd2SZpb92Pl/NiPEX4PSIJzudPXM1duY+TEpeFrl3D6euzDND0GnVcgiuGw/xWo GAylcX78So5AcbYa1YrUcLbghEI0B164cVFk0IXI6hYoNyPikASHkxkX+Kjp44aso9A5 y3usvKhNgKIg5UD66lg6d73qcfPM+KXKmtOBoKEFUCF1POdnuMvCJHoRc4KIgCKR+wB3 gNW12r03jUoifknCzZGD00o85Y0YVjSGMMjqBVqTre3NTZNAeQmT2jBeDG986FL+pWsR 6P5g== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id hc43-20020a17090716ab00b007ae17fd2f2dsi9285342ejc.996.2022.11.27.09.08.01; Sun, 27 Nov 2022 09:08:02 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 31D9968BB22; Sun, 27 Nov 2022 19:06:12 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail0.khirnov.net (red.khirnov.net [176.97.15.12]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 84B1F68B720 for ; Sun, 27 Nov 2022 19:06:00 +0200 (EET) Received: from localhost (localhost [IPv6:::1]) by mail0.khirnov.net (Postfix) with ESMTP id ED3DD240499 for ; Sun, 27 Nov 2022 18:05:59 +0100 (CET) Received: from mail0.khirnov.net ([IPv6:::1]) by localhost (mail0.khirnov.net [IPv6:::1]) (amavisd-new, port 10024) with ESMTP id 7mRJSo62cKjY for ; Sun, 27 Nov 2022 18:05:58 +0100 (CET) Received: from libav.khirnov.net (libav.khirnov.net [IPv6:2a00:c500:561:201::7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "libav.khirnov.net", Issuer "smtp.khirnov.net SMTP CA" (verified OK)) by mail0.khirnov.net (Postfix) with ESMTPS id D84DF240D03 for ; Sun, 27 Nov 2022 18:05:51 +0100 (CET) Received: from libav.khirnov.net (libav.khirnov.net [IPv6:::1]) by libav.khirnov.net (Postfix) with ESMTP id 7385B3A25D4 for ; Sun, 27 Nov 2022 18:05:46 +0100 (CET) From: Anton Khirnov To: ffmpeg-devel@ffmpeg.org Date: Sun, 27 Nov 2022 18:03:36 +0100 Message-Id: <20221127170351.11477-15-anton@khirnov.net> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20221127170351.11477-1-anton@khirnov.net> References: <20221127170351.11477-1-anton@khirnov.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 15/30] lavc/libx265: restructure handling reordered_opaque X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: g7qJ/uFdDJ98 Current code stores a pointer to allocated data in libx265 and frees it when the encoded packet is retrieved. This will leak if the packet is never retrieved, e.g. if the encoder is closed without being flushed. Restructure the code such that only indices to an array stored in our private data are given to libx265. This ensures no allocated memory can be lost. --- libavcodec/libx265.c | 85 +++++++++++++++++++++++++++++++++++++------- 1 file changed, 73 insertions(+), 12 deletions(-) diff --git a/libavcodec/libx265.c b/libavcodec/libx265.c index e87dea604d..25de3c669b 100644 --- a/libavcodec/libx265.c +++ b/libavcodec/libx265.c @@ -27,6 +27,7 @@ #include #include +#include "libavutil/avassert.h" #include "libavutil/internal.h" #include "libavutil/common.h" #include "libavutil/opt.h" @@ -39,6 +40,12 @@ #include "atsc_a53.h" #include "sei.h" +typedef struct ReorderedData { + int64_t reordered_opaque; + + int in_use; +} ReorderedData; + typedef struct libx265Context { const AVClass *class; @@ -59,6 +66,9 @@ typedef struct libx265Context { int udu_sei; int a53_cc; + ReorderedData *rd; + int nb_rd; + /** * If the encoder does not support ROI then warn the first time we * encounter a frame with ROI side data. @@ -81,6 +91,40 @@ static int is_keyframe(NalUnitType naltype) } } +static int rd_get(libx265Context *ctx) +{ + const int add = 16; + + ReorderedData *tmp; + int idx; + + for (int i = 0; i < ctx->nb_rd; i++) + if (!ctx->rd[i].in_use) { + ctx->rd[i].in_use = 1; + return i; + } + + tmp = av_realloc_array(ctx->rd, ctx->nb_rd + add, sizeof(*ctx->rd)); + if (!tmp) + return AVERROR(ENOMEM); + memset(tmp + ctx->nb_rd, 0, sizeof(*tmp) * add); + + ctx->rd = tmp; + ctx->nb_rd += add; + + idx = ctx->nb_rd - add; + ctx->rd[idx].in_use = 1; + + return idx; +} + +static void rd_release(libx265Context *ctx, int idx) +{ + av_assert0(idx >= 0 && idx < ctx->nb_rd); + + memset(&ctx->rd[idx], 0, sizeof(ctx->rd[idx])); +} + static av_cold int libx265_encode_close(AVCodecContext *avctx) { libx265Context *ctx = avctx->priv_data; @@ -88,6 +132,8 @@ static av_cold int libx265_encode_close(AVCodecContext *avctx) ctx->api->param_free(ctx->params); av_freep(&ctx->sei_data); + av_freep(&ctx->rd); + if (ctx->encoder) ctx->api->encoder_close(ctx->encoder); @@ -499,12 +545,18 @@ static av_cold int libx265_encode_set_roi(libx265Context *ctx, const AVFrame *fr return 0; } -static void free_picture(x265_picture *pic) +static void free_picture(libx265Context *ctx, x265_picture *pic) { x265_sei *sei = &pic->userSEI; for (int i = 0; i < sei->numPayloads; i++) av_free(sei->payloads[i].payload); - av_freep(&pic->userData); + + if (pic->userData) { + int idx = (int)(intptr_t)pic->userData - 1; + rd_release(ctx, idx); + pic->userData = NULL; + } + av_freep(&pic->quantOffsets); sei->numPayloads = 0; } @@ -549,13 +601,18 @@ static int libx265_encode_frame(AVCodecContext *avctx, AVPacket *pkt, return ret; if (pic->reordered_opaque) { - x265pic.userData = av_malloc(sizeof(pic->reordered_opaque)); - if (!x265pic.userData) { - free_picture(&x265pic); - return AVERROR(ENOMEM); + ReorderedData *rd; + int rd_idx = rd_get(ctx); + + if (rd_idx < 0) { + free_picture(ctx, &x265pic); + return rd_idx; } - memcpy(x265pic.userData, &pic->reordered_opaque, sizeof(pic->reordered_opaque)); + x265pic.userData = (void*)(intptr_t)(rd_idx + 1); + + rd = &ctx->rd[rd_idx]; + rd->reordered_opaque = pic->reordered_opaque; } if (ctx->a53_cc) { @@ -574,7 +631,7 @@ static int libx265_encode_frame(AVCodecContext *avctx, AVPacket *pkt, (sei->numPayloads + 1) * sizeof(*sei_payload)); if (!tmp) { av_free(sei_data); - free_picture(&x265pic); + free_picture(ctx, &x265pic); return AVERROR(ENOMEM); } ctx->sei_data = tmp; @@ -600,7 +657,7 @@ static int libx265_encode_frame(AVCodecContext *avctx, AVPacket *pkt, &ctx->sei_data_size, (sei->numPayloads + 1) * sizeof(*sei_payload)); if (!tmp) { - free_picture(&x265pic); + free_picture(ctx, &x265pic); return AVERROR(ENOMEM); } ctx->sei_data = tmp; @@ -608,7 +665,7 @@ static int libx265_encode_frame(AVCodecContext *avctx, AVPacket *pkt, sei_payload = &sei->payloads[sei->numPayloads]; sei_payload->payload = av_memdup(side_data->data, side_data->size); if (!sei_payload->payload) { - free_picture(&x265pic); + free_picture(ctx, &x265pic); return AVERROR(ENOMEM); } sei_payload->payloadSize = side_data->size; @@ -680,8 +737,12 @@ static int libx265_encode_frame(AVCodecContext *avctx, AVPacket *pkt, ff_side_data_set_encoder_stats(pkt, x265pic_out.frameData.qp * FF_QP2LAMBDA, NULL, 0, pict_type); if (x265pic_out.userData) { - memcpy(&avctx->reordered_opaque, x265pic_out.userData, sizeof(avctx->reordered_opaque)); - av_freep(&x265pic_out.userData); + int idx = (int)(intptr_t)x265pic_out.userData - 1; + ReorderedData *rd = &ctx->rd[idx]; + + avctx->reordered_opaque = rd->reordered_opaque; + + rd_release(ctx, idx); } else avctx->reordered_opaque = 0;