From patchwork Thu Sep 15 13:56:27 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Khirnov X-Patchwork-Id: 37928 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:3b1c:b0:96:9ee8:5cfd with SMTP id c28csp329161pzh; Thu, 15 Sep 2022 06:59:31 -0700 (PDT) X-Google-Smtp-Source: AMsMyM4WjncOYLhlNYUfLz/fO2Wkza/gVLMgIb0OI/61gsTNX5ZL+jTitkUZzCjx9FejPKR1sVoB X-Received: by 2002:a17:906:db02:b0:780:24e:cf9 with SMTP id xj2-20020a170906db0200b00780024e0cf9mr92345ejb.460.1663250370942; Thu, 15 Sep 2022 06:59:30 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1663250370; cv=none; d=google.com; s=arc-20160816; b=zltVVN0SJht3ADcT75IBmj3v2fLIa7NNfEaBbfV6lAU4ESS7JoJuyV8elE9QgGBhzp RRKHo3Q6i/M0e56Ns4/5V5KcklybflIUFcv4z1aJdO0Pyj1jmx0hGmT99fFl31Ot4IL1 kKVea8nfc44lv/Dgy/uYm7FIPP5sxYd7RtMve9yQ4aUTG8FtbckzYcJNtPUl15e8JnXU I8hajAuWzFc2KovvuKUDavHOO3LQZ+7+aH+nKtZs85M92S8pXLhSGW/8/fi+PVsTrZ2x 921PSBLCtaZwruJ4VjW87QBRKJ/ppZR3obCHgpQ5OQck+EF1cE6CXFHw91o/+yMbV8wQ 7Xmw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:message-id:date:to:from :delivered-to; bh=BUGR8G8I8fg6LtIb1IwKTkE9EEVsNiyAj4y0CmZO4lc=; b=Ia2wTjz2slf/RXw/r+MZaPorm+PzKeTCXVCO7TbamLJqZvsBdzPCpWWKXYTx6JbDkj BggNL6fuRJs084FKy2B86B7oMvqPcjWqvLLtoG/QoRYhAQ/BHVnZbjHSmt8H5mDXRPFn XrTjzvBA+IVcl1+D7KljNfma453enRvO5dTG7uukU07Gn1JgqQ5P+vIcDdUeDwcUYw7r 1iSzThXJsazY/9MdSpphAV7hgyon/wdfdlYh1tllEuhqQQLgfMwTyo3X3CTSFfcayJa1 Ig+t4+qBJYrzlgzK2rOQmSMvPjMfhX9m+5+CojIPYPsrn3ugHjXHiYuOQD0ZE7LCZ+yp 0Ahg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id gs6-20020a1709072d0600b0073da774ff40si18598165ejc.122.2022.09.15.06.59.30; Thu, 15 Sep 2022 06:59:30 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 5220168BBB3; Thu, 15 Sep 2022 16:59:17 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail0.khirnov.net (red.khirnov.net [176.97.15.12]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 593D168BB84 for ; Thu, 15 Sep 2022 16:59:10 +0300 (EEST) Received: from localhost (localhost [IPv6:::1]) by mail0.khirnov.net (Postfix) with ESMTP id 173C5240D03 for ; Thu, 15 Sep 2022 15:59:10 +0200 (CEST) Received: from mail0.khirnov.net ([IPv6:::1]) by localhost (mail0.khirnov.net [IPv6:::1]) (amavisd-new, port 10024) with ESMTP id 9eWgJ7zkgv3J for ; Thu, 15 Sep 2022 15:59:07 +0200 (CEST) Received: from libav.khirnov.net (libav.khirnov.net [IPv6:2a00:c500:561:201::7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "libav.khirnov.net", Issuer "smtp.khirnov.net SMTP CA" (verified OK)) by mail0.khirnov.net (Postfix) with ESMTPS id D55A4240175 for ; Thu, 15 Sep 2022 15:59:07 +0200 (CEST) Received: by libav.khirnov.net (Postfix, from userid 1000) id 643663A0520; Thu, 15 Sep 2022 15:58:58 +0200 (CEST) From: Anton Khirnov To: ffmpeg-devel@ffmpeg.org Date: Thu, 15 Sep 2022 15:56:27 +0200 Message-Id: <20220915135627.32230-1-anton@khirnov.net> X-Mailer: git-send-email 2.35.1 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 1/2] lavc/videotoolbox: do not pass AVCodecContext to decoder output callback X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: SfITcH8eqYDB The opaque parameter for the callback is set in videotoolbox_start(), called when the hwaccel is initialized. When frame threading is used, avctx will be the context corresponding to the frame thread currently doing the decoding. Using this same codec context in all subsequent invocations of the decoder callback (even those triggered by a different frame thread) is unsafe, and broken after cc867f2c09d2b69cee8a0eccd62aff002cbbfe11, since each frame thread now cleans up its hwaccel state after decoding each frame. Fix this by passing hwaccel_priv_data as the opaque parameter, which exists in a single instance forwarded between all frame threads. The only other use of AVCodecContext in the decoder output callback is as a logging context. For this purpose, store a logging context in hwaccel_priv_data. --- Initial version of this patch tested by Marvin on IRC. Someone please test this final version, as I don't have the HW to do it myself. --- libavcodec/videotoolbox.c | 10 ++++++---- libavcodec/vt_internal.h | 2 ++ 2 files changed, 8 insertions(+), 4 deletions(-) diff --git a/libavcodec/videotoolbox.c b/libavcodec/videotoolbox.c index ce83c2594a..d61d310600 100644 --- a/libavcodec/videotoolbox.c +++ b/libavcodec/videotoolbox.c @@ -690,8 +690,7 @@ static void videotoolbox_decoder_callback(void *opaque, CMTime pts, CMTime duration) { - AVCodecContext *avctx = opaque; - VTContext *vtctx = avctx->internal->hwaccel_priv_data; + VTContext *vtctx = opaque; if (vtctx->frame) { CVPixelBufferRelease(vtctx->frame); @@ -699,7 +698,8 @@ static void videotoolbox_decoder_callback(void *opaque, } if (!image_buffer) { - av_log(avctx, status ? AV_LOG_WARNING : AV_LOG_DEBUG, "vt decoder cb: output image buffer is null: %i\n", status); + av_log(vtctx->logctx, status ? AV_LOG_WARNING : AV_LOG_DEBUG, + "vt decoder cb: output image buffer is null: %i\n", status); return; } @@ -949,7 +949,7 @@ static int videotoolbox_start(AVCodecContext *avctx) videotoolbox->cv_pix_fmt_type); decoder_cb.decompressionOutputCallback = videotoolbox_decoder_callback; - decoder_cb.decompressionOutputRefCon = avctx; + decoder_cb.decompressionOutputRefCon = avctx->internal->hwaccel_priv_data; status = VTDecompressionSessionCreate(NULL, // allocator videotoolbox->cm_fmt_desc, // videoFormatDescription @@ -1179,6 +1179,8 @@ int ff_videotoolbox_common_init(AVCodecContext *avctx) AVHWFramesContext *hw_frames; int err; + vtctx->logctx = avctx; + // Old API - do nothing. if (avctx->hwaccel_context) return 0; diff --git a/libavcodec/vt_internal.h b/libavcodec/vt_internal.h index 54a11fd1b5..9502d7c7dc 100644 --- a/libavcodec/vt_internal.h +++ b/libavcodec/vt_internal.h @@ -45,6 +45,8 @@ typedef struct VTContext { // Current H264 parameters (used to trigger decoder restart on SPS changes). uint8_t sps[3]; bool reconfig_needed; + + void *logctx; } VTContext; int ff_videotoolbox_alloc_frame(AVCodecContext *avctx, AVFrame *frame); From patchwork Thu Sep 15 13:56:28 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Khirnov X-Patchwork-Id: 37927 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:3b1c:b0:96:9ee8:5cfd with SMTP id c28csp329058pzh; Thu, 15 Sep 2022 06:59:21 -0700 (PDT) X-Google-Smtp-Source: AMsMyM4K+VS7M96grHpTjOjaQ8dCM2FWfWCt9BZtLVkZnkJATO/AZLu0WPyAluUVodPQjjzFQrDg X-Received: by 2002:a05:6402:190:b0:453:35b:413a with SMTP id r16-20020a056402019000b00453035b413amr15102edv.362.1663250360959; Thu, 15 Sep 2022 06:59:20 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1663250360; cv=none; d=google.com; s=arc-20160816; b=jxCdw8Atn2ZKNMJeHVmE3Fmqh6WOvp88CC5/GVz6jWkROKSjN1WE6RdLw47job3m3K qRdmCRUReM6u+qbCVtMLVFLQMumMudfiA/C4CPiOLu0j7gk7s7L9QHYePshkpmTSDDHu a9+PW3+RbqgI4ZrXrwy/zxFgqBdAIu/HO+acK5Jd7mJ90FYFS7OhAc9hbjLnSqd/9AEY ASU/OzQtpYBMajtow4lGh8rcwHJxIqzlwN08w5z2Jn3BZ9feoqXXUmHF/JW2X0+yLPmj fOKB2AqnYosJgdIuRyrIPNV21Np0hoF6u9lpqbeUfc4N9tzGeQx58ph5xemHHwB3TLYO fWww== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:delivered-to; bh=9bphRa0kOzFwE82SJQpOIrSUmLzSnEl3KfNdOrkBJgg=; b=BBz1FDYuIhfAqoBYiLxR58BqLEtKs3QsXehKpqEgwGaHMNen9MiLVQzTdXKFWfp0PW j1WXnaGBnl9VlTeJc8bvkMgTYgfD2JYHETA+7rpzcUxyayjcs4JaCdyncWCAzvJsotMX 7mM3l1jZBl020G2jINRSTWfm/Jat9W/vh2qbEHq0u/F+8QmqvT1YqAAkmujFqsgs5wkA SvMtAZ8vjugwI9I1kc8jHrJHn9G5+FSPpWARhE3Jy0d7bDaAJh0VruSZpXifIXSYs55W dPauDGQOyMRHi+DLmMuM4prmKPiudNPUbSJWw4rY1CqxP8/q5yU1VnFt78ABIgbNPYKs m7jw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id d11-20020a50cd4b000000b004472b35d7a5si14357369edj.309.2022.09.15.06.59.19; Thu, 15 Sep 2022 06:59:20 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 2146F68BBA7; Thu, 15 Sep 2022 16:59:16 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail0.khirnov.net (red.khirnov.net [176.97.15.12]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id DF91068B7DE for ; Thu, 15 Sep 2022 16:59:09 +0300 (EEST) Received: from localhost (localhost [IPv6:::1]) by mail0.khirnov.net (Postfix) with ESMTP id 0247E240D1A for ; Thu, 15 Sep 2022 15:59:09 +0200 (CEST) Received: from mail0.khirnov.net ([IPv6:::1]) by localhost (mail0.khirnov.net [IPv6:::1]) (amavisd-new, port 10024) with ESMTP id lLTeIa_TCzzv for ; Thu, 15 Sep 2022 15:59:08 +0200 (CEST) Received: from libav.khirnov.net (libav.khirnov.net [IPv6:2a00:c500:561:201::7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "libav.khirnov.net", Issuer "smtp.khirnov.net SMTP CA" (verified OK)) by mail0.khirnov.net (Postfix) with ESMTPS id D7544240D03 for ; Thu, 15 Sep 2022 15:59:07 +0200 (CEST) Received: by libav.khirnov.net (Postfix, from userid 1000) id 67DE53A0D7F; Thu, 15 Sep 2022 15:58:58 +0200 (CEST) From: Anton Khirnov To: ffmpeg-devel@ffmpeg.org Date: Thu, 15 Sep 2022 15:56:28 +0200 Message-Id: <20220915135627.32230-2-anton@khirnov.net> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220915135627.32230-1-anton@khirnov.net> References: <20220915135627.32230-1-anton@khirnov.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 2/2] lavc/videotoolbox: deprecate write-only output_callback X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: HxQZ4Grpv86U This field has never been used for anything, so stop setting it and deprecate it. --- libavcodec/version_major.h | 1 + libavcodec/videotoolbox.c | 2 -- libavcodec/videotoolbox.h | 5 +++++ 3 files changed, 6 insertions(+), 2 deletions(-) diff --git a/libavcodec/version_major.h b/libavcodec/version_major.h index 1ec815a7bc..d9386792de 100644 --- a/libavcodec/version_major.h +++ b/libavcodec/version_major.h @@ -51,5 +51,6 @@ #define FF_API_IDCT_NONE (LIBAVCODEC_VERSION_MAJOR < 60) #define FF_API_SVTAV1_OPTS (LIBAVCODEC_VERSION_MAJOR < 60) #define FF_API_AYUV_CODECID (LIBAVCODEC_VERSION_MAJOR < 60) +#define FF_API_VT_OUTPUT_CALLBACK (LIBAVCODEC_VERSION_MAJOR < 60) #endif /* AVCODEC_VERSION_MAJOR_H */ diff --git a/libavcodec/videotoolbox.c b/libavcodec/videotoolbox.c index d61d310600..1b1be8ddb4 100644 --- a/libavcodec/videotoolbox.c +++ b/libavcodec/videotoolbox.c @@ -1377,8 +1377,6 @@ static AVVideotoolboxContext *av_videotoolbox_alloc_context_with_pix_fmt(enum AV AVVideotoolboxContext *ret = av_mallocz(sizeof(*ret)); if (ret) { - ret->output_callback = videotoolbox_decoder_callback; - OSType cv_pix_fmt_type = av_map_videotoolbox_format_from_pixfmt2(pix_fmt, full_range); if (cv_pix_fmt_type == 0) { cv_pix_fmt_type = kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange; diff --git a/libavcodec/videotoolbox.h b/libavcodec/videotoolbox.h index af2db0d580..fd8a5b7982 100644 --- a/libavcodec/videotoolbox.h +++ b/libavcodec/videotoolbox.h @@ -37,6 +37,8 @@ #include "libavcodec/avcodec.h" +#include "libavutil/attributes.h" + /** * This struct holds all the information that needs to be passed * between the caller and libavcodec for initializing Videotoolbox decoding. @@ -50,11 +52,14 @@ typedef struct AVVideotoolboxContext { */ VTDecompressionSessionRef session; +#if FF_API_VT_OUTPUT_CALLBACK /** * The output callback that must be passed to the session. * Set by av_videottoolbox_default_init() */ + attribute_deprecated VTDecompressionOutputCallback output_callback; +#endif /** * CVPixelBuffer Format Type that Videotoolbox will use for decoded frames.