From patchwork Sun Jul 7 10:21:32 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhao Zhili X-Patchwork-Id: 50382 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a59:cc64:0:b0:482:c625:d099 with SMTP id k4csp5227770vqv; Sun, 7 Jul 2024 03:22:19 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCXVClf0xPzvxunttjXIeoj7kRr9cIcol9KJ9476hF+xz4Qb5UukqV6E3W/9T0ySeW3R91BAWSm64GFgylX3NZ9KnqPhMHTCDDtQ5g== X-Google-Smtp-Source: AGHT+IGEr9H56dw1iAcHy8QDXj5DyHQSO5R8bY+FqNNtYfbwaM+K5TKAG5usJxuBT+sLQQAf3Psd X-Received: by 2002:a17:907:2da1:b0:a77:eddf:a49c with SMTP id a640c23a62f3a-a77eddfbf14mr140915766b.41.1720347739168; Sun, 07 Jul 2024 03:22:19 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1720347739; cv=none; d=google.com; s=arc-20160816; b=ekmwEwUZSX/Z1Nz4eP3NpyLwU8ACruS5jL0spazP7NEGEqKH7JB1MWETC29Op90BS0 EI8Ux2PdUlpfRpkhhUigsuPldn3dx7pI6U3TJYpjRD0c/WOImhEVLTsUqXqLpgMaFDTS bgsFd9KRjVCzzMELE/W2m4keHwD5GLflPsMVcRXXUoh3op3o9tDxnfNsaUCDB2EK8SQY E0gn2XdHsJpKcI2rxVO3ny1b4Zm3thiqoDscVwQfhPjDH7y7aoC/y9COdXPUvi/MaHj6 Rz+qMusgHSZ7eF9H+qvsEpZyfqipG/4Ykx+bFVpo+8bXO9XjMNka2WJ6SWNq3vCqvk4e fkeQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to:date :to:from:message-id:dkim-signature:delivered-to; bh=32DRynPwoKCeU/wcZHa1wpCewt5MgWDXpDWvFAtm64A=; fh=HnHYuZ9XgUo86ZRXTLWWmQxhslYEI9B9taZ5X1DLFfc=; b=liiRLX9L+vvIryxAhjLsnIbtTIonerfUdemrSd5o9aXJVDl4wKlIH+aHTa7lAUZdKA zrwQ3TG0kBe400ZCGoYzdK2we6q0v0y5bHsed28GkO3EYGdCOaGWriYKH4+9uh8RHY8V iAExzY5b4h1AqC5m0Wd8WMgWp7hf6F1ulMCx21nfzPOraB6c9zoxM6lTbQUO03uMH8wI iYOult/ISPk8WdHW6n/Y6EzFBh71auSM0FWbZsUm6jiyAm7SjL6dev3eKr8AATqI6nr1 BBTUBq92f3r5pYpzIcrWKWJ7c9o7mILV78VindfBCNI0oz0uKQn3HfFpO91J1hvx/E53 Lbiw==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@foxmail.com header.s=s201512 header.b=qk5ueFY4; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=foxmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id a640c23a62f3a-a72ab04a376si1013496966b.318.2024.07.07.03.22.18; Sun, 07 Jul 2024 03:22:19 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@foxmail.com header.s=s201512 header.b=qk5ueFY4; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=foxmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 7652B68DADE; Sun, 7 Jul 2024 13:22:01 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from out162-62-58-216.mail.qq.com (out162-62-58-216.mail.qq.com [162.62.58.216]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 37E5F68DA5C for ; Sun, 7 Jul 2024 13:21:45 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=foxmail.com; s=s201512; t=1720347697; bh=J+H6R9iG5ROOJS/kL0K56cUnaOs3Pmi00FtqlA5Hu6k=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=qk5ueFY4aNpOhqUxfBude+c7ZdQW7I2CC6gLD8lJNhW4Qgfewj6Mm7jRAB03vGUTb c8DhQVfS/OIV3/E+P0VpkvpRH8STCYEgylodHtIgud4TuH/BQdzpaFiMGkMIO5otm4 7hKcARsoe9gMVgh52ApEW+PecC+rOz63p8Cxx2YM= Received: from ZHILIZHAO-MB1.tencent.com ([113.116.28.29]) by newxmesmtplogicsvrsza10-0.qq.com (NewEsmtp) with SMTP id 5633A86C; Sun, 07 Jul 2024 18:21:35 +0800 X-QQ-mid: xmsmtpt1720347696tdirvyhvk Message-ID: X-QQ-XMAILINFO: NvKyM24IHTKS1fop19uWb4kYzGGXwNsEg0SyhhV5xwJuDI7Lks0hoIiLkxfN8c NhLB/eWts+jDMdysZUuzcAdsHgZ2Fcnd9dJbn95LW7jP34Lnsc7XspGFHxco1zP7qeKxtcXpfRC2 syAzReEBiK5K9h7qAdnidKapYzuwTxRpairKlqNqce1CAfOqAUy3vaTF5p5GtQaxcCytCk8kBtPL 3uhT8qzc1BLOZS5KuK2Aek1iE8zr5ARp0eI2RqGfF0dCjq/rqa53HLkPeVfpZyvWHkM8Lggt77yw PZGHoxZJ7HemZsOHI/WuW3Qj/bGmKzQn/+y9c2iV21IaHPkqt7sBq8LVevx1n9gmOa5t+nviQmUj JIcL85IeskaqtzDW0AfI8J35+2AH/d5+V7H++VWD2fyD3czU+VtNHc8cK1YXfYAazR7+n2PyWck/ 2n0QshguP0uQxTBXmbSomHUFjm5tOqJVoAGVAvzqYo8Asla0IbAk4mUYmNd4r7XRepCvilOAe5p4 jr2cexFHyRgi+7J/0lJZcicg5XWC5649fH7eEuy082S3EqjSE73/u836hVq3sHaYT2SX+z3rdlH8 dbVLLyD9heSJ2uRgjDRToUbJliYUEFirfW+B33ZNOF6gAPYOaW7Itk2SHS8ACYBQE24Z4eEIRD3s 6MWs3ujqSJHN5j28EZYuNa7BTFVIjD0T0EbhzGYxJNgml45GlfPZYRurljPS058cnWfrHdnJUp22 bJ826ZMOGZTB+JanKlpQ4OTf4LfysojYg5Pr+lAnfy80ahvU+sxQo8Sv4qFJGD/RGCW7IWyDQTeU O48Fj6rKO5C3RjwxC1+l9tWDY8vBmfUqVKnNmKicBPxBEpYgb3sn2QNIZojvNt16SQSH9S6Sqkoj eypQ5YBGIS+9VyNn0xFODAlLDQhSE8zJMkFdHdLAeusMHigwOcABUmnAv6MEmPMUOxh+rUZ0SOwu FrmCUzzf16SrraaLaLHgslIUxlLx6ASDfPB+BH7KY+OkW1Sl4dd+EX4nxsJCotVwDY5rh9A4g= X-QQ-XMRINFO: Nq+8W0+stu50PRdwbJxPCL0= From: Zhao Zhili To: ffmpeg-devel@ffmpeg.org Date: Sun, 7 Jul 2024 18:21:32 +0800 X-OQ-MSGID: <20240707102134.93935-3-quinkblack@foxmail.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20240707102134.93935-1-quinkblack@foxmail.com> References: <20240707102134.93935-1-quinkblack@foxmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 3/5] avcodec/videotoolboxenc: Use BufNode as sourceFrameRefcon X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Zhao Zhili Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: i4j52zcmupRG From: Zhao Zhili ExtraSEI is used as the sourceFrameRefcon of VTCompressionSessionEncodeFrame. It cannot hold other information which is necessary to fix another issue in the following patch. This patch also fixed leak of ExtraSEI on the error path of vtenc_output_callback. --- libavcodec/videotoolboxenc.c | 77 ++++++++++++++++++++++++------------ 1 file changed, 51 insertions(+), 26 deletions(-) diff --git a/libavcodec/videotoolboxenc.c b/libavcodec/videotoolboxenc.c index 03e3c9ad42..213bfa8b49 100644 --- a/libavcodec/videotoolboxenc.c +++ b/libavcodec/videotoolboxenc.c @@ -347,8 +347,7 @@ static void set_async_error(VTEncContext *vtctx, int err) while (info) { BufNode *next = info->next; - CFRelease(info->cm_buffer); - av_free(info); + vtenc_free_buf_node(info); info = next; } @@ -438,19 +437,8 @@ static int vtenc_q_pop(VTEncContext *vtctx, bool wait, CMSampleBufferRef *buf, E return 0; } -static void vtenc_q_push(VTEncContext *vtctx, CMSampleBufferRef buffer, ExtraSEI *sei) +static void vtenc_q_push(VTEncContext *vtctx, BufNode *info) { - BufNode *info = av_malloc(sizeof(BufNode)); - if (!info) { - set_async_error(vtctx, AVERROR(ENOMEM)); - return; - } - - CFRetain(buffer); - info->cm_buffer = buffer; - info->sei = sei; - info->next = NULL; - pthread_mutex_lock(&vtctx->lock); if (!vtctx->q_head) { @@ -726,6 +714,22 @@ static int set_extradata(AVCodecContext *avctx, CMSampleBufferRef sample_buffer) return 0; } +static void vtenc_free_buf_node(BufNode *info) +{ + if (!info) + return; + + if (info->sei) { + av_free(info->sei->data); + av_free(info->sei); + } + + if (info->cm_buffer) + CFRelease(info->cm_buffer); + + av_free(info); +} + static void vtenc_output_callback( void *ctx, void *sourceFrameCtx, @@ -735,13 +739,15 @@ static void vtenc_output_callback( { AVCodecContext *avctx = ctx; VTEncContext *vtctx = avctx->priv_data; - ExtraSEI *sei = sourceFrameCtx; + BufNode *info = sourceFrameCtx; if (vtctx->async_error) { + vtenc_free_buf_node(info); return; } if (status) { + vtenc_free_buf_node(info); av_log(avctx, AV_LOG_ERROR, "Error encoding frame: %d\n", (int)status); set_async_error(vtctx, AVERROR_EXTERNAL); return; @@ -751,15 +757,19 @@ static void vtenc_output_callback( return; } + CFRetain(sample_buffer); + info->cm_buffer = sample_buffer; + if (!avctx->extradata && (avctx->flags & AV_CODEC_FLAG_GLOBAL_HEADER)) { int set_status = set_extradata(avctx, sample_buffer); if (set_status) { + vtenc_free_buf_node(info); set_async_error(vtctx, set_status); return; } } - vtenc_q_push(vtctx, sample_buffer, sei); + vtenc_q_push(vtctx, info); } static int get_length_code_size( @@ -2565,19 +2575,23 @@ static int vtenc_send_frame(AVCodecContext *avctx, const AVFrame *frame) { CMTime time; - CFDictionaryRef frame_dict; + CFDictionaryRef frame_dict = NULL; CVPixelBufferRef cv_img = NULL; AVFrameSideData *side_data = NULL; + BufNode *node = av_mallocz(sizeof(*node)); ExtraSEI *sei = NULL; - int status = create_cv_pixel_buffer(avctx, frame, &cv_img); + int status; - if (status) return status; + if (!node) + return AVERROR(ENOMEM); + + status = create_cv_pixel_buffer(avctx, frame, &cv_img); + if (status) + goto out; status = create_encoder_dict_h264(frame, &frame_dict); - if (status) { - CFRelease(cv_img); - return status; - } + if (status) + goto out; #if CONFIG_ATSC_A53 side_data = av_frame_get_side_data(frame, AV_FRAME_DATA_A53_CC); @@ -2587,6 +2601,7 @@ static int vtenc_send_frame(AVCodecContext *avctx, status = AVERROR(ENOMEM); goto out; } + node->sei = sei; status = ff_alloc_a53_sei(frame, 0, &sei->data, &sei->size); if (status < 0) { av_free(sei); @@ -2602,7 +2617,7 @@ static int vtenc_send_frame(AVCodecContext *avctx, time, kCMTimeInvalid, frame_dict, - sei, + node, NULL ); @@ -2616,7 +2631,10 @@ static int vtenc_send_frame(AVCodecContext *avctx, out: if (frame_dict) CFRelease(frame_dict); - CFRelease(cv_img); + if (cv_img) + CFRelease(cv_img); + if (status) + vtenc_free_buf_node(node); return status; } @@ -2701,6 +2719,10 @@ static int vtenc_populate_extradata(AVCodecContext *avctx, CVPixelBufferRef pix_buf = NULL; CMTime time; CMSampleBufferRef buf = NULL; + BufNode *node = av_mallocz(sizeof(*node)); + + if (!node) + return AVERROR(ENOMEM); status = vtenc_create_encoder(avctx, codec_type, @@ -2736,7 +2758,7 @@ static int vtenc_populate_extradata(AVCodecContext *avctx, time, kCMTimeInvalid, NULL, - NULL, + node, NULL); if (status) { @@ -2747,6 +2769,7 @@ static int vtenc_populate_extradata(AVCodecContext *avctx, status = AVERROR_EXTERNAL; goto pe_cleanup; } + node = NULL; //Populates extradata - output frames are flushed and param sets are available. status = VTCompressionSessionCompleteFrames(vtctx->session, @@ -2773,6 +2796,8 @@ pe_cleanup: vtctx->frame_ct_out = 0; av_assert0(status != 0 || (avctx->extradata && avctx->extradata_size > 0)); + if (!status) + vtenc_free_buf_node(node); return status; }