From patchwork Thu Sep 15 13:56:27 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Khirnov X-Patchwork-Id: 37928 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:3b1c:b0:96:9ee8:5cfd with SMTP id c28csp329161pzh; Thu, 15 Sep 2022 06:59:31 -0700 (PDT) X-Google-Smtp-Source: AMsMyM4WjncOYLhlNYUfLz/fO2Wkza/gVLMgIb0OI/61gsTNX5ZL+jTitkUZzCjx9FejPKR1sVoB X-Received: by 2002:a17:906:db02:b0:780:24e:cf9 with SMTP id xj2-20020a170906db0200b00780024e0cf9mr92345ejb.460.1663250370942; Thu, 15 Sep 2022 06:59:30 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1663250370; cv=none; d=google.com; s=arc-20160816; b=zltVVN0SJht3ADcT75IBmj3v2fLIa7NNfEaBbfV6lAU4ESS7JoJuyV8elE9QgGBhzp RRKHo3Q6i/M0e56Ns4/5V5KcklybflIUFcv4z1aJdO0Pyj1jmx0hGmT99fFl31Ot4IL1 kKVea8nfc44lv/Dgy/uYm7FIPP5sxYd7RtMve9yQ4aUTG8FtbckzYcJNtPUl15e8JnXU I8hajAuWzFc2KovvuKUDavHOO3LQZ+7+aH+nKtZs85M92S8pXLhSGW/8/fi+PVsTrZ2x 921PSBLCtaZwruJ4VjW87QBRKJ/ppZR3obCHgpQ5OQck+EF1cE6CXFHw91o/+yMbV8wQ 7Xmw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:message-id:date:to:from :delivered-to; bh=BUGR8G8I8fg6LtIb1IwKTkE9EEVsNiyAj4y0CmZO4lc=; b=Ia2wTjz2slf/RXw/r+MZaPorm+PzKeTCXVCO7TbamLJqZvsBdzPCpWWKXYTx6JbDkj BggNL6fuRJs084FKy2B86B7oMvqPcjWqvLLtoG/QoRYhAQ/BHVnZbjHSmt8H5mDXRPFn XrTjzvBA+IVcl1+D7KljNfma453enRvO5dTG7uukU07Gn1JgqQ5P+vIcDdUeDwcUYw7r 1iSzThXJsazY/9MdSpphAV7hgyon/wdfdlYh1tllEuhqQQLgfMwTyo3X3CTSFfcayJa1 Ig+t4+qBJYrzlgzK2rOQmSMvPjMfhX9m+5+CojIPYPsrn3ugHjXHiYuOQD0ZE7LCZ+yp 0Ahg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id gs6-20020a1709072d0600b0073da774ff40si18598165ejc.122.2022.09.15.06.59.30; Thu, 15 Sep 2022 06:59:30 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 5220168BBB3; Thu, 15 Sep 2022 16:59:17 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail0.khirnov.net (red.khirnov.net [176.97.15.12]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 593D168BB84 for ; Thu, 15 Sep 2022 16:59:10 +0300 (EEST) Received: from localhost (localhost [IPv6:::1]) by mail0.khirnov.net (Postfix) with ESMTP id 173C5240D03 for ; Thu, 15 Sep 2022 15:59:10 +0200 (CEST) Received: from mail0.khirnov.net ([IPv6:::1]) by localhost (mail0.khirnov.net [IPv6:::1]) (amavisd-new, port 10024) with ESMTP id 9eWgJ7zkgv3J for ; Thu, 15 Sep 2022 15:59:07 +0200 (CEST) Received: from libav.khirnov.net (libav.khirnov.net [IPv6:2a00:c500:561:201::7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "libav.khirnov.net", Issuer "smtp.khirnov.net SMTP CA" (verified OK)) by mail0.khirnov.net (Postfix) with ESMTPS id D55A4240175 for ; Thu, 15 Sep 2022 15:59:07 +0200 (CEST) Received: by libav.khirnov.net (Postfix, from userid 1000) id 643663A0520; Thu, 15 Sep 2022 15:58:58 +0200 (CEST) From: Anton Khirnov To: ffmpeg-devel@ffmpeg.org Date: Thu, 15 Sep 2022 15:56:27 +0200 Message-Id: <20220915135627.32230-1-anton@khirnov.net> X-Mailer: git-send-email 2.35.1 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 1/2] lavc/videotoolbox: do not pass AVCodecContext to decoder output callback X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: SfITcH8eqYDB The opaque parameter for the callback is set in videotoolbox_start(), called when the hwaccel is initialized. When frame threading is used, avctx will be the context corresponding to the frame thread currently doing the decoding. Using this same codec context in all subsequent invocations of the decoder callback (even those triggered by a different frame thread) is unsafe, and broken after cc867f2c09d2b69cee8a0eccd62aff002cbbfe11, since each frame thread now cleans up its hwaccel state after decoding each frame. Fix this by passing hwaccel_priv_data as the opaque parameter, which exists in a single instance forwarded between all frame threads. The only other use of AVCodecContext in the decoder output callback is as a logging context. For this purpose, store a logging context in hwaccel_priv_data. --- Initial version of this patch tested by Marvin on IRC. Someone please test this final version, as I don't have the HW to do it myself. --- libavcodec/videotoolbox.c | 10 ++++++---- libavcodec/vt_internal.h | 2 ++ 2 files changed, 8 insertions(+), 4 deletions(-) diff --git a/libavcodec/videotoolbox.c b/libavcodec/videotoolbox.c index ce83c2594a..d61d310600 100644 --- a/libavcodec/videotoolbox.c +++ b/libavcodec/videotoolbox.c @@ -690,8 +690,7 @@ static void videotoolbox_decoder_callback(void *opaque, CMTime pts, CMTime duration) { - AVCodecContext *avctx = opaque; - VTContext *vtctx = avctx->internal->hwaccel_priv_data; + VTContext *vtctx = opaque; if (vtctx->frame) { CVPixelBufferRelease(vtctx->frame); @@ -699,7 +698,8 @@ static void videotoolbox_decoder_callback(void *opaque, } if (!image_buffer) { - av_log(avctx, status ? AV_LOG_WARNING : AV_LOG_DEBUG, "vt decoder cb: output image buffer is null: %i\n", status); + av_log(vtctx->logctx, status ? AV_LOG_WARNING : AV_LOG_DEBUG, + "vt decoder cb: output image buffer is null: %i\n", status); return; } @@ -949,7 +949,7 @@ static int videotoolbox_start(AVCodecContext *avctx) videotoolbox->cv_pix_fmt_type); decoder_cb.decompressionOutputCallback = videotoolbox_decoder_callback; - decoder_cb.decompressionOutputRefCon = avctx; + decoder_cb.decompressionOutputRefCon = avctx->internal->hwaccel_priv_data; status = VTDecompressionSessionCreate(NULL, // allocator videotoolbox->cm_fmt_desc, // videoFormatDescription @@ -1179,6 +1179,8 @@ int ff_videotoolbox_common_init(AVCodecContext *avctx) AVHWFramesContext *hw_frames; int err; + vtctx->logctx = avctx; + // Old API - do nothing. if (avctx->hwaccel_context) return 0; diff --git a/libavcodec/vt_internal.h b/libavcodec/vt_internal.h index 54a11fd1b5..9502d7c7dc 100644 --- a/libavcodec/vt_internal.h +++ b/libavcodec/vt_internal.h @@ -45,6 +45,8 @@ typedef struct VTContext { // Current H264 parameters (used to trigger decoder restart on SPS changes). uint8_t sps[3]; bool reconfig_needed; + + void *logctx; } VTContext; int ff_videotoolbox_alloc_frame(AVCodecContext *avctx, AVFrame *frame);