From patchwork Fri Jun 21 06:08:20 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hao Guan X-Patchwork-Id: 50030 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a59:ae71:0:b0:482:c625:d099 with SMTP id w17csp318680vqz; Thu, 20 Jun 2024 23:17:48 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCVFpUy1VAI3ixtK1v9HS4ZXPBOb7YgYkFfFsC+oJBaq4skJQaGZhZJsn/Gb77QPLEpIy+fvbGQlZ8jNeJ5+00FBcRTItVVaklhz4g== X-Google-Smtp-Source: AGHT+IEUlhuTuu6TIIcYttvbpFEbhfmc6RQIap/yJD70KT6wuQ6WZ1//9yw8TKdviJT+nMPMB5mL X-Received: by 2002:a2e:860e:0:b0:2eb:d963:d8cc with SMTP id 38308e7fff4ca-2ec3cffc54dmr55971421fa.49.1718950667708; Thu, 20 Jun 2024 23:17:47 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1718950667; cv=none; d=google.com; s=arc-20160816; b=O7ENL3oZHY6r8gUcknMihFvbwJzmPVTmcJKUWFsaW5r5ImePQzhY5o8kYcxyTq+Op3 65NkgYXGRk48CFESEQkGoyNlddWOiaLgC+DFvhMOKVZvtAy3aakbg548AOSFNfulSLrK +v639HlFw532uO5p4osJErchmrKQwCMBJZ0Q4Id8OCHTKWphHxHitqopHf0D0BFUxGkd YwM0cy72ad1dcqSoqZ00FvpKZV/AU9D+6XG9GwXBGfTyaY7aD7PTCFkk3QwmWPQmzIq0 tah/KrG67DHydvSO7zD+3IhA3ngFHneva1q6UvCWn1WC5FCnWYIy3IG/xnKvbMcrSwIB z4Cw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:message-id:date:to:from :dkim-signature:delivered-to; bh=KfBLTIAnWXgMWS2FxmRMi/PYt69ijGFcu8QmyP4X7BU=; fh=IxUZHPIsbVwrh/Hr4VLcP/wrcyIjUWwlHo8JLF4FHWQ=; b=mcpu7rHF1CWGR6vRVTH8nQdWd5k7/4g/SKKRJuQ3x6E3hdMnWf9IfWl0lIgnlQLuQt Me2AlcO9RIcHVjePtZIOrbKdO/1UcbrM219VNWcUylgP4juZNzxMOnXkwvTxeYHqvjlZ fmAIA1+6+xl/WnuIa6yfkzLcBqu2EWjpdVJGvv3o45CesQfhgd6JMWMTcJ7LY7d47wLH hKKtpf3ZhFyFAZXQcDZBA1dVcTPkZ/Tb1n4W59ShRWxGq2QbfSUR/dODzi51Ejq5KqTM uI2pfxzBb7lNK1jBN4P+y0iNEVfVP91pf7L3e93MR/tqhbEBkKs0501TUHkyKsPuT+t5 En9Q==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=A+0NOjXL; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id a640c23a62f3a-a6fcf45326fsi50705166b.5.2024.06.20.23.17.47; Thu, 20 Jun 2024 23:17:47 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=A+0NOjXL; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id A286F68D83F; Fri, 21 Jun 2024 09:08:34 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pl1-f182.google.com (mail-pl1-f182.google.com [209.85.214.182]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 746E268D57C for ; Fri, 21 Jun 2024 09:08:28 +0300 (EEST) Received: by mail-pl1-f182.google.com with SMTP id d9443c01a7336-1f9b523a15cso12920395ad.0 for ; Thu, 20 Jun 2024 23:08:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1718950106; x=1719554906; darn=ffmpeg.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=XXIOyjE7WlHld10fpdX1t+tEG9US3S1e9Cr+9hymr7A=; b=A+0NOjXLNuaQ2ORnDNcfejIucdeyuQnsKQG55gbRwTHOCyQ5iQbB4FcVHzQNekhFms iLEwOptHnYtGgkofHzyE/oHLFtzhd8PG6RxIjY2I8Iar8cAqObmBTnzLg9VrTV21b1No P6Y84yb+/vekw3Sf9mXGwLVYwW8in8j3WjgVvM46WfRjyILb0UjIAv79pInj+dQAC2o4 iUf8aUwUioasoUxPkqA4dYLKD2xJ0vAD+A0Pd7SjakxjLNSuyWe5P10auhYUl0A1oAgR 5hudJRNRUDhX52D1uq6Emk4/+71gtGICu+MxmCpuBLreJXF8Fb6UvaDUfrgHPqNLIQEC 6Frw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718950106; x=1719554906; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=XXIOyjE7WlHld10fpdX1t+tEG9US3S1e9Cr+9hymr7A=; b=sBYGnesky6ZhMQkrsgDM564iuPT+bm1N4U+OUz7vRPF5VorbCoYei933U2Fcm6Y08o Iet6DEHIK5JQsSNRmxrWU/CQqXcA4FmLA60OqpmL0XooXspVD2Bi7jJ3pQ9LTzSHSZci 3vutxzhbGXP099CHW26yumqMjXsjfc4dxVMqStxjahD2WyMWlDAOnGwwMkyyWTRE9i2n tQjWImK3TZQ/WVoR7oxpfDeEeYEFWzmobEhAhrmamWWRNdEVHJgIlSoVhQdGzeUKA18b yZ2XifwGE3lj/RgLNdpX02sJGXcnBC+3Zv5kUtWbU8qMP4UL2P2G9fhrU4D6fzysOWMu R2JA== X-Gm-Message-State: AOJu0Yw7KNrARSqHgYlmYNRkdsr2TTMYpBloK+HU+RQi4qCFbsZsFN8O BXSD2FJNucLt51S/S9gsVNVvZtttHDOXWoT/Mhpb1ZwzLa+qGnlLTfjSpg== X-Received: by 2002:a17:903:1cb:b0:1f6:f4c4:d700 with SMTP id d9443c01a7336-1f98b22ddb8mr152395725ad.10.1718950106095; Thu, 20 Jun 2024 23:08:26 -0700 (PDT) Received: from localhost.localdomain (n058152048004.netvigator.com. [58.152.48.4]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-1f9eb3c6bbdsm6186245ad.132.2024.06.20.23.08.25 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Thu, 20 Jun 2024 23:08:25 -0700 (PDT) From: Hao Guan To: ffmpeg-devel@ffmpeg.org Date: Fri, 21 Jun 2024 14:08:20 +0800 Message-Id: <20240621060820.3215-1-hguandl@gmail.com> X-Mailer: git-send-email 2.39.3 (Apple Git-146) MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH] avcodec/videotoolboxenc: fix vtctx reset condition X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Hao Guan Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: vqlp7MTwFa2V In vtenc_populate_extradata, the cleanup function vtenc_reset should not be used when no error occurs, otherwise some color information is lost (#11036). This patch checks the status code and conducts the correct cleanup. Signed-off-by: Hao Guan --- libavcodec/videotoolboxenc.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/libavcodec/videotoolboxenc.c b/libavcodec/videotoolboxenc.c index 15c34d59c3..0aa4885232 100644 --- a/libavcodec/videotoolboxenc.c +++ b/libavcodec/videotoolboxenc.c @@ -2766,7 +2766,14 @@ static int vtenc_populate_extradata(AVCodecContext *avctx, pe_cleanup: CVPixelBufferRelease(pix_buf); - vtenc_reset(vtctx); + + if (status) { + vtenc_reset(vtctx); + } else if (vtctx->session) { + CFRelease(vtctx->session); + vtctx->session = NULL; + } + vtctx->frame_ct_out = 0; av_assert0(status != 0 || (avctx->extradata && avctx->extradata_size > 0));