From patchwork Sat Sep 3 18:39:29 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Khirnov X-Patchwork-Id: 37631 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:139a:b0:8f:1db5:eae2 with SMTP id w26csp1708471pzh; Sat, 3 Sep 2022 11:40:54 -0700 (PDT) X-Google-Smtp-Source: AA6agR6FGCI/jLvA7/ERv/iqbEPx83YH6w94OLdNYZdGOEnGbKCfIgB2JIDfkUv+SGze74IWxXa7 X-Received: by 2002:a05:6402:350b:b0:43e:f4be:c447 with SMTP id b11-20020a056402350b00b0043ef4bec447mr38449626edd.427.1662230454568; Sat, 03 Sep 2022 11:40:54 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1662230454; cv=none; d=google.com; s=arc-20160816; b=VMS06pZ84DhRYdWCDqKSyA8aR0dfRYKpNZsOwSWwu701VrdHdWtLIQX8W0NLNkjd8S oljcIf9XEdhnKTGfvV5xLEYixvy0bA49QjmToJUa/OMNN/3/lMXM/alsNrdu2h3G3isZ JhNXdqjA7nNhFPhaTBdPiZTpnwpeDWum+qPQndfb8hAZvigNwVAZ3GzjfjguvIaIHH2X hSByqyge93HCgLmsVlRCCJ32JFfMD66PaOBbtcu5A63INJuX4P7kXnbiKwKxT+g1wMoo fk4HkHQXrKCrYelcpztnpcAhaSzzAZNoYEIicTyaxSi4RIu6CHmijCQY9dnF3jsXPBau 5npg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:delivered-to; bh=DhTWwmDMW7Ihjd6xPH11EjI4zcbJbNHya4Y6YejpPOc=; b=eBaxLVUj4mLZjdYQ/fq3V9EKdezXGKQ/6HwQBqZiqxeq050mbnVTfyjbRCcWk+w9JN xSodKEqPi5ASJcOktmEqcHluWTSHH17ZlQcW8WUEZQbGGql2VAg5x/URFEn24/Us1rM/ Q1lARybZLtdOcldOg4XCVMlLtwuH3QECg3+HTlRCQoHJ3RFwiKpYAicxAUsWp5jU1O9g 4hucFhW4K1VV1JCeolhRIh3ZX3KsNhjxM2OmcYG4m3W+JW4MMvnfhLChV0MQPMgB5KDE POLrjAzTwlKGYReFOXb767c03qSWLlmjej4S0NVQYwvqDED+zU+TNYAudF30DOOfH5uu i+QA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id i24-20020a0564020f1800b0044e68f06cadsi20014eda.290.2022.09.03.11.40.53; Sat, 03 Sep 2022 11:40:54 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id E061868B9C3; Sat, 3 Sep 2022 21:40:50 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail0.khirnov.net (red.khirnov.net [176.97.15.12]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 0E59C68B455 for ; Sat, 3 Sep 2022 21:40:44 +0300 (EEST) Received: from localhost (localhost [IPv6:::1]) by mail0.khirnov.net (Postfix) with ESMTP id 8A5B4240D18 for ; Sat, 3 Sep 2022 20:40:43 +0200 (CEST) Received: from mail0.khirnov.net ([IPv6:::1]) by localhost (mail0.khirnov.net [IPv6:::1]) (amavisd-new, port 10024) with ESMTP id owz5nUQQ-LV2 for ; Sat, 3 Sep 2022 20:40:42 +0200 (CEST) Received: from libav.khirnov.net (libav.khirnov.net [IPv6:2a00:c500:561:201::7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "libav.khirnov.net", Issuer "smtp.khirnov.net SMTP CA" (verified OK)) by mail0.khirnov.net (Postfix) with ESMTPS id 2CEAE240D03 for ; Sat, 3 Sep 2022 20:40:42 +0200 (CEST) Received: by libav.khirnov.net (Postfix, from userid 1000) id 3E5263A0520; Sat, 3 Sep 2022 20:40:42 +0200 (CEST) From: Anton Khirnov To: ffmpeg-devel@ffmpeg.org Date: Sat, 3 Sep 2022 20:39:29 +0200 Message-Id: <20220903183930.9223-1-anton@khirnov.net> X-Mailer: git-send-email 2.35.1 In-Reply-To: <166222954842.3205.17186561679857350925@lain.khirnov.net> References: <166222954842.3205.17186561679857350925@lain.khirnov.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 1/2] lavc: fix and extend AVCodecContext.hwaccel_context doxy X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: KiL8FspeabWz Mention: - that it is legacy and optional (every hwaccel that uses it can also work with hwcontext, though some optional information can only be signalled throught hwaccel_context) - that it can be used for encoders (only qsvenc currently) - ownership and lifetime --- libavcodec/avcodec.h | 27 ++++++++++++++++++++------- 1 file changed, 20 insertions(+), 7 deletions(-) diff --git a/libavcodec/avcodec.h b/libavcodec/avcodec.h index 65c8535359..7db5d1b1c5 100644 --- a/libavcodec/avcodec.h +++ b/libavcodec/avcodec.h @@ -1388,13 +1388,26 @@ typedef struct AVCodecContext { const struct AVHWAccel *hwaccel; /** - * Hardware accelerator context. - * For some hardware accelerators, a global context needs to be - * provided by the user. In that case, this holds display-dependent - * data FFmpeg cannot instantiate itself. Please refer to the - * FFmpeg HW accelerator documentation to know how to fill this. - * - encoding: unused - * - decoding: Set by user + * Legacy hardware accelerator context. + * + * For some hardware acceleration methods, the caller may use this field to + * signal hwaccel-specific data to the codec. The struct pointed to by this + * pointer is hwaccel-dependent and defined in the respective header. Please + * refer to the FFmpeg HW accelerator documentation to know how to fill + * this. + * + * In most cases this field is optional - the necessary information may also + * be provided to libavcodec through @ref hw_frames_ctx or @ref + * hw_device_ctx (see avcodec_get_hw_config()). However, in some cases it + * may be the only method of signalling some (optional) information. + * + * The struct and its contents are owned by the caller. + * + * - encoding: May be set by the caller before avcodec_open2(). Must remain + * valid until avcodec_free_context(). + * - decoding: May be set by the caller in the get_format() callback. + * Must remain valid until the next get_format() call, + * or avcodec_free_context() (whichever comes first). */ void *hwaccel_context; From patchwork Sat Sep 3 18:39:30 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Khirnov X-Patchwork-Id: 37632 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:139a:b0:8f:1db5:eae2 with SMTP id w26csp1708521pzh; Sat, 3 Sep 2022 11:41:02 -0700 (PDT) X-Google-Smtp-Source: AA6agR6VtWD8Hm3xKpVPhlR9V848RiZryk7/padiwCXg/EWS48HJGyy5KYKcE8pcmVEHvwa2SQFG X-Received: by 2002:a05:6402:3552:b0:448:e395:dc06 with SMTP id f18-20020a056402355200b00448e395dc06mr18180561edd.136.1662230462389; Sat, 03 Sep 2022 11:41:02 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1662230462; cv=none; d=google.com; s=arc-20160816; b=mKpBrmhRPgQQdPjUNvqwRhV9bCt7bHx94CZK80KZGpOkj6/TL884LOqT/aG3KOqnHI ZS00ar3fk94BVCsvxZuWiKQv+xXssYpyPmYG5ErZ5zX4Qghh7IgI2CUeyUJvK7/k4Eds MHl6B+myGUSN6ccBMGCf2a0iwHVB7wvv2YPGIyqK9rW207/0cGoX9TFMRCqwud/j8yh6 EN7yfzQ+Xf1GB0mXXJSCyoTGS1OK8Rdy9CMjfRxOuZ/T+pTOVSwNIkokUzknsk3+4aBk zPu9xAoD16/yZONMTDGwQIeM0GsFdP/UerbjIGdVCqAio8jJCuze6bRPaf7zfO4oxI4W JOFA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:delivered-to; bh=yOsb6p/bB/u9piaT0GY1UW9AN55K4w9Gh1vJh588h4c=; b=QBZ4/uh3/+TuUpJ+tR+YYoTPhEXcsCi8dL+CbeknEwGGgTE0p3Dh+OD7VPOc0PZ67u Bj3NmqyFcFY1LWp9kNRS209HM9Hk8u9TsZ/YsgW4ebgE1fyvY2V5FPJzgeidXo49DZHo 0PgM613qAZLdZpI1srxoFzv/b74XegVqyPxOzZ33xYGKon1eTNC9QAcRu4oYVztqP7Xn fm0DTik8rfi2rMLjsnbWaR0DUz6FLdhtrzD0jvoQZ8tWT8LMjNkpRd/CCWbS0leTk3Yj 4HF4CW9ibN2uDjC4W7i4tDKzh3aF8LrcuiImmsEmpvb92VyODQVmraH6zGlbwNQU5rWP VBOg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id q15-20020a056402518f00b004464ae6552dsi5409744edd.236.2022.09.03.11.41.02; Sat, 03 Sep 2022 11:41:02 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id D9F0D68B980; Sat, 3 Sep 2022 21:40:51 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail0.khirnov.net (red.khirnov.net [176.97.15.12]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id BED3368B980 for ; Sat, 3 Sep 2022 21:40:44 +0300 (EEST) Received: from localhost (localhost [IPv6:::1]) by mail0.khirnov.net (Postfix) with ESMTP id 4D9F0240D03 for ; Sat, 3 Sep 2022 20:40:44 +0200 (CEST) Received: from mail0.khirnov.net ([IPv6:::1]) by localhost (mail0.khirnov.net [IPv6:::1]) (amavisd-new, port 10024) with ESMTP id aD4BUFeHmMvX for ; Sat, 3 Sep 2022 20:40:42 +0200 (CEST) Received: from libav.khirnov.net (libav.khirnov.net [IPv6:2a00:c500:561:201::7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "libav.khirnov.net", Issuer "smtp.khirnov.net SMTP CA" (verified OK)) by mail0.khirnov.net (Postfix) with ESMTPS id 42C03240D0E for ; Sat, 3 Sep 2022 20:40:42 +0200 (CEST) Received: by libav.khirnov.net (Postfix, from userid 1000) id 405553A03F2; Sat, 3 Sep 2022 20:40:42 +0200 (CEST) From: Anton Khirnov To: ffmpeg-devel@ffmpeg.org Date: Sat, 3 Sep 2022 20:39:30 +0200 Message-Id: <20220903183930.9223-2-anton@khirnov.net> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220903183930.9223-1-anton@khirnov.net> References: <166222954842.3205.17186561679857350925@lain.khirnov.net> <20220903183930.9223-1-anton@khirnov.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 2/2] lavc/pthread_frame: avoid leaving stale hwaccel state in worker threads X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: A7w1WuIPTplA This state is not refcounted, so make sure it always has a well-defined owner. Remove the block added in 091341f2ab5bd35ca1a2aae90503adc74f8d3523, as this commit also solves that issue in a more general way. --- libavcodec/pthread_frame.c | 47 ++++++++++++++++++++++++++++---------- 1 file changed, 35 insertions(+), 12 deletions(-) diff --git a/libavcodec/pthread_frame.c b/libavcodec/pthread_frame.c index 08a6f98898..066269621d 100644 --- a/libavcodec/pthread_frame.c +++ b/libavcodec/pthread_frame.c @@ -148,6 +148,12 @@ typedef struct FrameThreadContext { * Set for the first N packets, where N is the number of threads. * While it is set, ff_thread_en/decode_frame won't return any results. */ + + /* hwaccel state is temporarily stored here in order to transfer its ownership + * to the next decoding thread without the need for extra synchronization */ + const AVHWAccel *stash_hwaccel; + void *stash_hwaccel_context; + void *stash_hwaccel_priv; } FrameThreadContext; #if FF_API_THREAD_SAFE_CALLBACKS @@ -228,9 +234,17 @@ FF_ENABLE_DEPRECATION_WARNINGS ff_thread_finish_setup(avctx); if (p->hwaccel_serializing) { + /* wipe hwaccel state to avoid stale pointers lying around; + * the state was transferred to FrameThreadContext in + * ff_thread_finish_setup(), so nothing is leaked */ + avctx->hwaccel = NULL; + avctx->hwaccel_context = NULL; + avctx->internal->hwaccel_priv_data = NULL; + p->hwaccel_serializing = 0; pthread_mutex_unlock(&p->parent->hwaccel_mutex); } + av_assert0(!avctx->hwaccel); if (p->async_serializing) { p->async_serializing = 0; @@ -294,9 +308,6 @@ static int update_context_from_thread(AVCodecContext *dst, AVCodecContext *src, dst->color_range = src->color_range; dst->chroma_sample_location = src->chroma_sample_location; - dst->hwaccel = src->hwaccel; - dst->hwaccel_context = src->hwaccel_context; - dst->sample_rate = src->sample_rate; dst->sample_fmt = src->sample_fmt; #if FF_API_OLD_CHANNEL_LAYOUT @@ -309,8 +320,6 @@ FF_ENABLE_DEPRECATION_WARNINGS if (err < 0) return err; - dst->internal->hwaccel_priv_data = src->internal->hwaccel_priv_data; - if (!!dst->hw_frames_ctx != !!src->hw_frames_ctx || (dst->hw_frames_ctx && dst->hw_frames_ctx->data != src->hw_frames_ctx->data)) { av_buffer_unref(&dst->hw_frames_ctx); @@ -450,6 +459,12 @@ static int submit_packet(PerThreadContext *p, AVCodecContext *user_avctx, pthread_mutex_unlock(&p->mutex); return err; } + + /* transfer hwaccel state stashed from previous thread, if any */ + av_assert0(!p->avctx->hwaccel); + FFSWAP(const AVHWAccel*, p->avctx->hwaccel, fctx->stash_hwaccel); + FFSWAP(void*, p->avctx->hwaccel_context, fctx->stash_hwaccel_context); + FFSWAP(void*, p->avctx->internal->hwaccel_priv_data, fctx->stash_hwaccel_priv); } av_packet_unref(p->avpkt); @@ -655,6 +670,14 @@ void ff_thread_finish_setup(AVCodecContext *avctx) { async_lock(p->parent); } + /* save hwaccel state for passing to the next thread; + * this is done here so that this worker thread can wipe its own hwaccel + * state after decoding, without requiring synchronization */ + av_assert0(!p->parent->stash_hwaccel); + p->parent->stash_hwaccel = avctx->hwaccel; + p->parent->stash_hwaccel_context = avctx->hwaccel_context; + p->parent->stash_hwaccel_priv = avctx->internal->hwaccel_priv_data; + pthread_mutex_lock(&p->progress_mutex); if(atomic_load(&p->state) == STATE_SETUP_FINISHED){ av_log(avctx, AV_LOG_WARNING, "Multiple ff_thread_finish_setup() calls\n"); @@ -708,13 +731,6 @@ void ff_frame_thread_free(AVCodecContext *avctx, int thread_count) park_frame_worker_threads(fctx, thread_count); - if (fctx->prev_thread && avctx->internal->hwaccel_priv_data != - fctx->prev_thread->avctx->internal->hwaccel_priv_data) { - if (update_context_from_thread(avctx, fctx->prev_thread->avctx, 1) < 0) { - av_log(avctx, AV_LOG_ERROR, "Failed to update user thread.\n"); - } - } - for (i = 0; i < thread_count; i++) { PerThreadContext *p = &fctx->threads[i]; AVCodecContext *ctx = p->avctx; @@ -761,6 +777,13 @@ void ff_frame_thread_free(AVCodecContext *avctx, int thread_count) av_freep(&fctx->threads); ff_pthread_free(fctx, thread_ctx_offsets); + /* if we have stashed hwaccel state, move it to the user-facing context, + * so it will be freed in avcodec_close() */ + av_assert0(!avctx->hwaccel); + FFSWAP(const AVHWAccel*, avctx->hwaccel, fctx->stash_hwaccel); + FFSWAP(void*, avctx->hwaccel_context, fctx->stash_hwaccel_context); + FFSWAP(void*, avctx->internal->hwaccel_priv_data, fctx->stash_hwaccel_priv); + av_freep(&avctx->internal->thread_ctx); }