From patchwork Sun Jul 7 10:21:33 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhao Zhili X-Patchwork-Id: 50381 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a59:cc64:0:b0:482:c625:d099 with SMTP id k4csp5227667vqv; Sun, 7 Jul 2024 03:21:59 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCXfJ3fl2J3HE6fJ7WpkCl0ag/AoPx0plr9+OOM8vKu6BUxc+GU/Df8B1oJY6HSn41fb9X1EQfYkiV0E0jsRzcAuQfYAuwjKUsfWFw== X-Google-Smtp-Source: AGHT+IE+5YAQduHLQjbAK5n2cs097SE0Uvk5X790l5qhB38/2ihYEoef5H+qJAWoUENb+W6OY4ZD X-Received: by 2002:a17:906:dcf:b0:a77:e0ac:2a74 with SMTP id a640c23a62f3a-a77e0ac2bcamr266560966b.17.1720347719369; Sun, 07 Jul 2024 03:21:59 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1720347719; cv=none; d=google.com; s=arc-20160816; b=W2B2Yo5qVmruPd8fCKWtrVbO/FgL673g7dK5wFehtl/D7QDTBq6KjbgF9JQZVXnnfg JqvBPQQijbvK87in45HqtmMIxSA54rdqD+VTMdKmsnjdpRqH1c/E/yi4AA/4r+4Ex4Qs rvz2iKHzW9y8HYL3fLMDX+5ZN1gSq61WjogUeUk9ucRHsdTxuVN8a/iMMPY4xCcS1jA2 m8Sos9OVWSzpFtfOJrc0xdtG87x+mZ81l10BT8dbksWBZcl+TLqWAdBaN1QJM8VSQ6vz oLDnIcZOe8/+aobtj/dtZclyAPwmk6aFtUDRq5AA7O5WiqnywVfplkekbPr1/X5A7qw4 87eQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to:date :to:from:message-id:dkim-signature:delivered-to; bh=wxb0w+aiVaIX9ZT0BjflIekUsM7+EngegWK1Bbmoq0g=; fh=HnHYuZ9XgUo86ZRXTLWWmQxhslYEI9B9taZ5X1DLFfc=; b=sPaLYysm9TnZZ/EK98oTtwlFsvItvLAU7RKHGh5jMRlgSAvFyJWNLtQ9+cJe7xWNtV d83Oml9aV9nSDaUm6WcqHOhQHMBCtG8t7I8mlfrNV9gfXSbweReJA317iq95+YCoU7tf /DwToIr3Kqqn8Fz7X+kXgqOUBYtOHakVfG62WM3tFBbqQgU7s2DbS8PzAHM2JJPRYVwP lOiPYUs6h16F3VZ61eUJGEDSBhHs3rKKE7TVQAgSNE8Fe5H7OD/At9nlXgbxY/qIc7cQ anFpJ1qp/UeDJ+CADBjv1TjH5vD4gCSnFrVZTwcVyir1mij9qDXO5pxcoheMdIiKNHaU iEKA==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@foxmail.com header.s=s201512 header.b=xL8SDhAp; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=foxmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id a640c23a62f3a-a77d8214c48si231726366b.542.2024.07.07.03.21.58; Sun, 07 Jul 2024 03:21:59 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@foxmail.com header.s=s201512 header.b=xL8SDhAp; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=foxmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 963B968DA0B; Sun, 7 Jul 2024 13:21:55 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from out162-62-57-252.mail.qq.com (out162-62-57-252.mail.qq.com [162.62.57.252]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 349DF68DA30 for ; Sun, 7 Jul 2024 13:21:45 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=foxmail.com; s=s201512; t=1720347698; bh=gumDTTp1lo/TxZt7SLG49JJnYf+Y5Y9RSBzcSuR3LFU=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=xL8SDhAp+D4gW0JKLDdpwJLWI6jvOCZE2FrB9h4uwBK2al3oBz/yRmwStLtdjm1Ts yHRaW/wehs1i7zouR6mqGb1k68YccwlUsWTEiCbmJ3FyxKHR3SFhj+jzqkMPEA4fwq /WfMeCigINTKENn4+lv1cp6CSRFGvtHmLju5QR38= Received: from ZHILIZHAO-MB1.tencent.com ([113.116.28.29]) by newxmesmtplogicsvrsza10-0.qq.com (NewEsmtp) with SMTP id 5633A86C; Sun, 07 Jul 2024 18:21:35 +0800 X-QQ-mid: xmsmtpt1720347697tqlu9k5r9 Message-ID: X-QQ-XMAILINFO: M6T0E+nC5suJo0iPnt9YcvOvVA9lyTMPuVe9E9Q9cIipashwNcFK/AYIobHTh6 OscCdflE408eXpvq8AL32NXH79S6WJ0K+1+dHHRjVHPY5xkixNaOaNGUE3VqfdhYjSVarvcWmVZA 3j6kgRil6y3ew/7Yd31os2iWKlGI+p5Cneh11yTYN0y8bquoMOkWMRTIqlS7fuBmSQccK8GspEPc RQXY5kRNCy9uRfRDCUlhyRV+j4JoGmIMhJxAoF/EXXBI4MLnMibCt8KLB4YvdOh1EQAbzqPR1tke 56nubovCRhNwyJiwNcz89De7Bp3nXoQXCrR5FSfX6iHzmm/NPd6u2AskEm2htdo4YhKke7mHX+m6 ywuo2+j5Ip5pCx7sEVhW2U5C78YRhZhlYRV8fPf+3dX86N2arR1UH3A01nGOmCDVRL2LwFX8TPKG lqh2xkQUDuHRzC2r27/IkFoXzvTDjustRUDuPEqzoRXmAii5XfZZhyPH3auo0GvkS++G60JerVAu YexwXb9cZq3oKE33cUZnK4/4WdcvCHhnRUKEvoSZJ5IxAfwUwqsu+JiEYvoptfndI7PTLGeS1vWn k7sQF6SE9MZ4nEczmNiwDC+b5q2kxaTrcauS6XGsHOxE5g4xh57QSw+oTz7oYOQqgL0FKIOFJExV Y2jvQPOd7qHNJ96ULcJp0Q+96jQjlxBCccP3bW+d7pHaVg0/wVYYLetqEV58M7+HZUJwicZsIZqx h4sqMIEXSOgPvlSqPIuqPCu8c3998npkLILO6OcD2MC5eJzVA3/QIgMWsN9jlF4k3kGdSvQuLu7x +9d4+zUHqKTancad9AqIZpAeUlroDYy9xKxDAn/4/EjrWTAN43tyeBz1dG0xxog3ETKyVZt60JSC oWejaWNMx+hlSBWcYOQ5qtHurIok5Vh4zB/pC1wjsM8vrwGzmwNrol7pcGmjVL/gp3j+t9jpvbdD dpYJAuo1i/I8hDPzssqCD+cO6LL0GLCsDEGiIQG+VuUCdUNyWvZc9SsZPq21RXf3IC2WMvQZ+nQv y9blwk1lxNXyr3i7huHndz6EiYYBUXMcsBUUXnOeIcEdn/pRQB X-QQ-XMRINFO: Nq+8W0+stu50PRdwbJxPCL0= From: Zhao Zhili To: ffmpeg-devel@ffmpeg.org Date: Sun, 7 Jul 2024 18:21:33 +0800 X-OQ-MSGID: <20240707102134.93935-4-quinkblack@foxmail.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20240707102134.93935-1-quinkblack@foxmail.com> References: <20240707102134.93935-1-quinkblack@foxmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 4/5] avcodec/videotoolboxenc: Fix concurrent access to CVPixelBufferRef X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Zhao Zhili Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: fukU3CGP0Yze From: Zhao Zhili For a frame comes from AV_HWDEVICE_TYPE_VIDEOTOOLBOX, it's CVPixelBufferRef is maintained by a pool. CVPixelBufferRef returned to the pool when frame buffer reference reached to zero. However, VTCompressionSessionEncodeFrame also hold a reference to the CVPixelBufferRef. So a new frame get from av_hwframe_get_buffer may access a CVPixelBufferRef which still used by the encoder. It's only after vtenc_output_callback that we can make sure CVPixelBufferRef has been released by the encoder. The issue can be tested with sample from trac #10884. ffmpeg -hwaccel videotoolbox \ -hwaccel_output_format videotoolbox_vld \ -i input.mp4 \ -c:v hevc_videotoolbox \ -profile:v main \ -b:v 3M \ -vf scale_vt=w=iw/2:h=ih/2:color_matrix=bt709:color_primaries=bt709:color_transfer=bt709 \ -c:a copy \ -tag:v hvc1 \ output.mp4 Withtout the patch, there are some out of order images in output.mp4. --- libavcodec/videotoolboxenc.c | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-) diff --git a/libavcodec/videotoolboxenc.c b/libavcodec/videotoolboxenc.c index 213bfa8b49..9bb9b0b457 100644 --- a/libavcodec/videotoolboxenc.c +++ b/libavcodec/videotoolboxenc.c @@ -228,6 +228,7 @@ typedef struct ExtraSEI { typedef struct BufNode { CMSampleBufferRef cm_buffer; ExtraSEI *sei; + AVBufferRef *frame_buf; struct BufNode* next; } BufNode; @@ -727,6 +728,7 @@ static void vtenc_free_buf_node(BufNode *info) if (info->cm_buffer) CFRelease(info->cm_buffer); + av_buffer_unref(&info->frame_buf); av_free(info); } @@ -741,6 +743,7 @@ static void vtenc_output_callback( VTEncContext *vtctx = avctx->priv_data; BufNode *info = sourceFrameCtx; + av_buffer_unref(&info->frame_buf); if (vtctx->async_error) { vtenc_free_buf_node(info); return; @@ -2459,7 +2462,8 @@ static int copy_avframe_to_pixel_buffer(AVCodecContext *avctx, static int create_cv_pixel_buffer(AVCodecContext *avctx, const AVFrame *frame, - CVPixelBufferRef *cv_img) + CVPixelBufferRef *cv_img, + BufNode *node) { int plane_count; int color; @@ -2478,6 +2482,12 @@ static int create_cv_pixel_buffer(AVCodecContext *avctx, av_assert0(*cv_img); CFRetain(*cv_img); + if (frame->buf[0]) { + node->frame_buf = av_buffer_ref(frame->buf[0]); + if (!node->frame_buf) + return AVERROR(ENOMEM); + } + return 0; } @@ -2585,7 +2595,7 @@ static int vtenc_send_frame(AVCodecContext *avctx, if (!node) return AVERROR(ENOMEM); - status = create_cv_pixel_buffer(avctx, frame, &cv_img); + status = create_cv_pixel_buffer(avctx, frame, &cv_img, node); if (status) goto out;