From patchwork Sat Mar 18 08:56:04 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: wm4 X-Patchwork-Id: 3003 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.103.50.79 with SMTP id y76csp609867vsy; Sat, 18 Mar 2017 01:57:26 -0700 (PDT) X-Received: by 10.28.183.4 with SMTP id h4mr1839278wmf.32.1489827446054; Sat, 18 Mar 2017 01:57:26 -0700 (PDT) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id m139si6536429wma.32.2017.03.18.01.57.25; Sat, 18 Mar 2017 01:57:26 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@googlemail.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=googlemail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id B59F468835F; Sat, 18 Mar 2017 10:56:16 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wr0-f195.google.com (mail-wr0-f195.google.com [209.85.128.195]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 6B997688396 for ; Sat, 18 Mar 2017 10:56:14 +0200 (EET) Received: by mail-wr0-f195.google.com with SMTP id u108so12007962wrb.2 for ; Sat, 18 Mar 2017 01:56:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=ewEwfhOI/t6PFvRLfGmq5V3uQkFip5wgUv1rKs0+iAk=; b=uGIyWpF8GkuYyRVEUsr/HBQKYRwrQA3IhYW4v8pdWVScBUz9eICnVB/GOWYe8w3WXi lBwtdVXAC5R/EdDm7JGYy4BNGAP5yljd8/ZoaCuz/wFqSVdDbxsJOj8wqbg5Z1Pi19xs KfrIKZYHQ7lqqdKxb9i6c2Nst48mEKh8oUmSAV0Fqxvh38RpQPgkJLMRvKurOGFrd5FL o6+gvAFRXSsqiuqPt7XRxXJKIdshsz6FiXO8fF5Qu6/HKQEuTGCjtYfR+eg7z06aco8+ GOSzzs46AN5n3+5xzWYWySrP3+jUjpX4fRyiiCSe7TOFwADz4QAOWHWJzzpR6iEFY/59 H6xg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=ewEwfhOI/t6PFvRLfGmq5V3uQkFip5wgUv1rKs0+iAk=; b=K3UqjM6vv/XeRLV7dKgSxf++glcf10KxxaZP9yVv2kE9AfDQVVJCB0BRwalH6KKj9O 7GZ4svpLDdy9SPTzKHyFCF0piYfoawYwlicOb51aLUInNJV5+KVhOLhQa5NANSKrE6n9 Z5ifU1qt22V5zZMe7b2R+fJiHmfdV9eAM0gNSfk8QpkW5+spGI+ff8W6pjxQYRsRLpAK 3CwY8et7PXq/5jHmCaj9CCvJO/6/4oFfgVX99fKLo9Z0/0CxcW9zcag8X5oCWjunZugC WJufyyUNIdtTqu+KxbbwCGOWeyexCS+T9KYPW3T5oUJxySP74uWTGOZfa1f1/0t0AsOM gTyg== X-Gm-Message-State: AFeK/H3bftC6m9bxDraGkfWKJ72P1tyZpuYBiTB+3OdFE+ys7V82QRVNENqfdbaxRHD10w== X-Received: by 10.223.135.163 with SMTP id b32mr16844238wrb.170.1489827390965; Sat, 18 Mar 2017 01:56:30 -0700 (PDT) Received: from localhost.localdomain (p4FF02CC6.dip0.t-ipconnect.de. [79.240.44.198]) by smtp.googlemail.com with ESMTPSA id p12sm12879637wrb.46.2017.03.18.01.56.27 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 18 Mar 2017 01:56:30 -0700 (PDT) From: wm4 To: ffmpeg-devel@ffmpeg.org Date: Sat, 18 Mar 2017 09:56:04 +0100 Message-Id: <20170318085606.26011-8-nfxjfg@googlemail.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20170318085606.26011-1-nfxjfg@googlemail.com> References: <20170318085606.26011-1-nfxjfg@googlemail.com> Subject: [FFmpeg-devel] [PATCH 7/9] pthread_frame: do not run hwaccel decoding asynchronously unless it's safe X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: wm4 MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" From: Anton Khirnov Certain hardware decoding APIs are not guaranteed to be thread-safe, so having the user access decoded hardware surfaces while the decoder is running in another thread can cause failures (this is mainly known to happen with DXVA2). For such hwaccels, only allow the decoding thread to run while the user is inside a lavc decode call (avcodec_send_packet/receive_frame). Merges Libav commit d4a91e65. Signed-off-by: wm4 --- libavcodec/avcodec.h | 5 +++++ libavcodec/hwaccel.h | 24 +++++++++++++++++++++ libavcodec/pthread_frame.c | 52 ++++++++++++++++++++++++++++++++++++++++------ libavcodec/vaapi_h264.c | 2 ++ libavcodec/vaapi_mpeg2.c | 2 ++ libavcodec/vaapi_mpeg4.c | 3 +++ libavcodec/vaapi_vc1.c | 3 +++ libavcodec/vdpau_h264.c | 2 ++ libavcodec/vdpau_hevc.c | 2 ++ libavcodec/vdpau_mpeg12.c | 3 +++ libavcodec/vdpau_mpeg4.c | 2 ++ libavcodec/vdpau_vc1.c | 3 +++ libavcodec/version.h | 2 +- 13 files changed, 98 insertions(+), 7 deletions(-) create mode 100644 libavcodec/hwaccel.h diff --git a/libavcodec/avcodec.h b/libavcodec/avcodec.h index 1923c9648d..dbbe4febcd 100644 --- a/libavcodec/avcodec.h +++ b/libavcodec/avcodec.h @@ -3904,6 +3904,11 @@ typedef struct AVHWAccel { * AVCodecInternal.hwaccel_priv_data. */ int priv_data_size; + + /** + * Internal hwaccel capabilities. + */ + int caps_internal; } AVHWAccel; /** diff --git a/libavcodec/hwaccel.h b/libavcodec/hwaccel.h new file mode 100644 index 0000000000..17af43707c --- /dev/null +++ b/libavcodec/hwaccel.h @@ -0,0 +1,24 @@ +/* + * This file is part of FFmpeg and was stolen from Libav. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#ifndef AVCODEC_HWACCEL_H +#define AVCODEC_HWACCEL_H + +#define HWACCEL_CAP_ASYNC_SAFE (1 << 0) + +#endif /* AVCODEC_HWACCEL_H */ diff --git a/libavcodec/pthread_frame.c b/libavcodec/pthread_frame.c index e3bfbab872..1b871c31d4 100644 --- a/libavcodec/pthread_frame.c +++ b/libavcodec/pthread_frame.c @@ -28,6 +28,7 @@ #include #include "avcodec.h" +#include "hwaccel.h" #include "internal.h" #include "pthread_internal.h" #include "thread.h" @@ -105,6 +106,7 @@ typedef struct PerThreadContext { int die; ///< Set when the thread should exit. int hwaccel_serializing; + int async_serializing; } PerThreadContext; /** @@ -120,6 +122,7 @@ typedef struct FrameThreadContext { * is used. */ pthread_mutex_t hwaccel_mutex; + pthread_mutex_t async_mutex; int next_decoding; ///< The next context to submit a packet to. int next_finished; ///< The next context to return output from. @@ -190,6 +193,11 @@ static attribute_align_arg void *frame_worker_thread(void *arg) pthread_mutex_unlock(&p->parent->hwaccel_mutex); } + if (p->async_serializing) { + p->async_serializing = 0; + pthread_mutex_unlock(&p->parent->async_mutex); + } + pthread_mutex_lock(&p->progress_mutex); #if 0 //BUFREF-FIXME for (i = 0; i < MAX_BUFFERS; i++) @@ -443,7 +451,11 @@ int ff_thread_decode_frame(AVCodecContext *avctx, FrameThreadContext *fctx = avctx->internal->thread_ctx; int finished = fctx->next_finished; PerThreadContext *p; - int err; + int err, ret; + + /* release the async lock, permitting blocked hwaccel threads to + * go forward while we are in this function */ + pthread_mutex_unlock(&fctx->async_mutex); /* * Submit a packet to the next decoding thread. @@ -451,9 +463,11 @@ int ff_thread_decode_frame(AVCodecContext *avctx, p = &fctx->threads[fctx->next_decoding]; err = update_context_from_user(p->avctx, avctx); - if (err) return err; + if (err) + goto finish; err = submit_packet(p, avpkt); - if (err) return err; + if (err) + goto finish; /* * If we're still receiving the initial packets, don't return a frame. @@ -464,8 +478,10 @@ int ff_thread_decode_frame(AVCodecContext *avctx, if (fctx->delaying) { *got_picture_ptr=0; - if (avpkt->size) - return avpkt->size; + if (avpkt->size) { + ret = avpkt->size; + goto finish; + } } /* @@ -518,7 +534,12 @@ int ff_thread_decode_frame(AVCodecContext *avctx, return err; /* return the size of the consumed packet if no error occurred */ - return (p->result >= 0) ? avpkt->size : p->result; + ret = (p->result >= 0) ? avpkt->size : p->result; +finish: + pthread_mutex_lock(&fctx->async_mutex); + if (err < 0) + return err; + return ret; } void ff_thread_report_progress(ThreadFrame *f, int n, int field) @@ -573,6 +594,13 @@ void ff_thread_finish_setup(AVCodecContext *avctx) { p->hwaccel_serializing = 1; } + /* this assumes that no hwaccel calls happen before ff_thread_finish_setup() */ + if (avctx->hwaccel && + !(avctx->hwaccel->caps_internal & HWACCEL_CAP_ASYNC_SAFE)) { + p->async_serializing = 1; + pthread_mutex_lock(&p->parent->async_mutex); + } + pthread_mutex_lock(&p->progress_mutex); if(atomic_load(&p->state) == STATE_SETUP_FINISHED){ av_log(avctx, AV_LOG_WARNING, "Multiple ff_thread_finish_setup() calls\n"); @@ -589,6 +617,8 @@ static void park_frame_worker_threads(FrameThreadContext *fctx, int thread_count { int i; + pthread_mutex_unlock(&fctx->async_mutex); + for (i = 0; i < thread_count; i++) { PerThreadContext *p = &fctx->threads[i]; @@ -600,6 +630,8 @@ static void park_frame_worker_threads(FrameThreadContext *fctx, int thread_count } p->got_frame = 0; } + + pthread_mutex_lock(&fctx->async_mutex); } void ff_frame_thread_free(AVCodecContext *avctx, int thread_count) @@ -663,6 +695,10 @@ void ff_frame_thread_free(AVCodecContext *avctx, int thread_count) av_freep(&fctx->threads); pthread_mutex_destroy(&fctx->buffer_mutex); pthread_mutex_destroy(&fctx->hwaccel_mutex); + + pthread_mutex_unlock(&fctx->async_mutex); + pthread_mutex_destroy(&fctx->async_mutex); + av_freep(&avctx->internal->thread_ctx); if (avctx->priv_data && avctx->codec && avctx->codec->priv_class) @@ -710,6 +746,10 @@ int ff_frame_thread_init(AVCodecContext *avctx) pthread_mutex_init(&fctx->buffer_mutex, NULL); pthread_mutex_init(&fctx->hwaccel_mutex, NULL); + + pthread_mutex_init(&fctx->async_mutex, NULL); + pthread_mutex_lock(&fctx->async_mutex); + fctx->delaying = 1; for (i = 0; i < thread_count; i++) { diff --git a/libavcodec/vaapi_h264.c b/libavcodec/vaapi_h264.c index 44e8462522..30e7026ccf 100644 --- a/libavcodec/vaapi_h264.c +++ b/libavcodec/vaapi_h264.c @@ -22,6 +22,7 @@ #include "h264dec.h" #include "h264_ps.h" +#include "hwaccel.h" #include "vaapi_decode.h" /** @@ -399,4 +400,5 @@ AVHWAccel ff_h264_vaapi_hwaccel = { .init = &ff_vaapi_decode_init, .uninit = &ff_vaapi_decode_uninit, .priv_data_size = sizeof(VAAPIDecodeContext), + .caps_internal = HWACCEL_CAP_ASYNC_SAFE, }; diff --git a/libavcodec/vaapi_mpeg2.c b/libavcodec/vaapi_mpeg2.c index b2417ee830..0d197c9692 100644 --- a/libavcodec/vaapi_mpeg2.c +++ b/libavcodec/vaapi_mpeg2.c @@ -20,6 +20,7 @@ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA */ +#include "hwaccel.h" #include "mpegutils.h" #include "mpegvideo.h" #include "internal.h" @@ -183,4 +184,5 @@ AVHWAccel ff_mpeg2_vaapi_hwaccel = { .init = &ff_vaapi_decode_init, .uninit = &ff_vaapi_decode_uninit, .priv_data_size = sizeof(VAAPIDecodeContext), + .caps_internal = HWACCEL_CAP_ASYNC_SAFE, }; diff --git a/libavcodec/vaapi_mpeg4.c b/libavcodec/vaapi_mpeg4.c index b00f73dce1..f8c5ddf209 100644 --- a/libavcodec/vaapi_mpeg4.c +++ b/libavcodec/vaapi_mpeg4.c @@ -21,6 +21,7 @@ */ #include "h263.h" +#include "hwaccel.h" #include "internal.h" #include "mpeg4video.h" #include "mpegvideo.h" @@ -189,6 +190,7 @@ AVHWAccel ff_mpeg4_vaapi_hwaccel = { .init = &ff_vaapi_decode_init, .uninit = &ff_vaapi_decode_uninit, .priv_data_size = sizeof(VAAPIDecodeContext), + .caps_internal = HWACCEL_CAP_ASYNC_SAFE, }; #endif @@ -205,5 +207,6 @@ AVHWAccel ff_h263_vaapi_hwaccel = { .init = &ff_vaapi_decode_init, .uninit = &ff_vaapi_decode_uninit, .priv_data_size = sizeof(VAAPIDecodeContext), + .caps_internal = HWACCEL_CAP_ASYNC_SAFE, }; #endif diff --git a/libavcodec/vaapi_vc1.c b/libavcodec/vaapi_vc1.c index a456149e6e..30c9ed3c8b 100644 --- a/libavcodec/vaapi_vc1.c +++ b/libavcodec/vaapi_vc1.c @@ -20,6 +20,7 @@ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA */ +#include "hwaccel.h" #include "internal.h" #include "vaapi_decode.h" #include "vc1.h" @@ -399,6 +400,7 @@ AVHWAccel ff_wmv3_vaapi_hwaccel = { .init = &ff_vaapi_decode_init, .uninit = &ff_vaapi_decode_uninit, .priv_data_size = sizeof(VAAPIDecodeContext), + .caps_internal = HWACCEL_CAP_ASYNC_SAFE, }; #endif @@ -414,4 +416,5 @@ AVHWAccel ff_vc1_vaapi_hwaccel = { .init = &ff_vaapi_decode_init, .uninit = &ff_vaapi_decode_uninit, .priv_data_size = sizeof(VAAPIDecodeContext), + .caps_internal = HWACCEL_CAP_ASYNC_SAFE, }; diff --git a/libavcodec/vdpau_h264.c b/libavcodec/vdpau_h264.c index 7265af2890..be6ba71433 100644 --- a/libavcodec/vdpau_h264.c +++ b/libavcodec/vdpau_h264.c @@ -27,6 +27,7 @@ #include "internal.h" #include "h264dec.h" #include "h264_ps.h" +#include "hwaccel.h" #include "mpegutils.h" #include "vdpau.h" #include "vdpau_internal.h" @@ -273,4 +274,5 @@ AVHWAccel ff_h264_vdpau_hwaccel = { .init = vdpau_h264_init, .uninit = ff_vdpau_common_uninit, .priv_data_size = sizeof(VDPAUContext), + .caps_internal = HWACCEL_CAP_ASYNC_SAFE, }; diff --git a/libavcodec/vdpau_hevc.c b/libavcodec/vdpau_hevc.c index ce2610f67f..ee93b3a5e8 100644 --- a/libavcodec/vdpau_hevc.c +++ b/libavcodec/vdpau_hevc.c @@ -25,6 +25,7 @@ #include "avcodec.h" #include "internal.h" #include "hevc.h" +#include "hwaccel.h" #include "vdpau.h" #include "vdpau_internal.h" @@ -423,4 +424,5 @@ AVHWAccel ff_hevc_vdpau_hwaccel = { .init = vdpau_hevc_init, .uninit = ff_vdpau_common_uninit, .priv_data_size = sizeof(VDPAUContext), + .caps_internal = HWACCEL_CAP_ASYNC_SAFE, }; diff --git a/libavcodec/vdpau_mpeg12.c b/libavcodec/vdpau_mpeg12.c index 3ac2cb827d..b657007ee7 100644 --- a/libavcodec/vdpau_mpeg12.c +++ b/libavcodec/vdpau_mpeg12.c @@ -24,6 +24,7 @@ #include #include "avcodec.h" +#include "hwaccel.h" #include "mpegvideo.h" #include "vdpau.h" #include "vdpau_internal.h" @@ -114,6 +115,7 @@ AVHWAccel ff_mpeg1_vdpau_hwaccel = { .init = vdpau_mpeg1_init, .uninit = ff_vdpau_common_uninit, .priv_data_size = sizeof(VDPAUContext), + .caps_internal = HWACCEL_CAP_ASYNC_SAFE, }; #endif @@ -148,5 +150,6 @@ AVHWAccel ff_mpeg2_vdpau_hwaccel = { .init = vdpau_mpeg2_init, .uninit = ff_vdpau_common_uninit, .priv_data_size = sizeof(VDPAUContext), + .caps_internal = HWACCEL_CAP_ASYNC_SAFE, }; #endif diff --git a/libavcodec/vdpau_mpeg4.c b/libavcodec/vdpau_mpeg4.c index 46a00cb27c..bbdd843a44 100644 --- a/libavcodec/vdpau_mpeg4.c +++ b/libavcodec/vdpau_mpeg4.c @@ -24,6 +24,7 @@ #include #include "avcodec.h" +#include "hwaccel.h" #include "mpeg4video.h" #include "vdpau.h" #include "vdpau_internal.h" @@ -121,4 +122,5 @@ AVHWAccel ff_mpeg4_vdpau_hwaccel = { .init = vdpau_mpeg4_init, .uninit = ff_vdpau_common_uninit, .priv_data_size = sizeof(VDPAUContext), + .caps_internal = HWACCEL_CAP_ASYNC_SAFE, }; diff --git a/libavcodec/vdpau_vc1.c b/libavcodec/vdpau_vc1.c index ffd6505d13..665a2333f4 100644 --- a/libavcodec/vdpau_vc1.c +++ b/libavcodec/vdpau_vc1.c @@ -24,6 +24,7 @@ #include #include "avcodec.h" +#include "hwaccel.h" #include "vc1.h" #include "vdpau.h" #include "vdpau_internal.h" @@ -147,6 +148,7 @@ AVHWAccel ff_wmv3_vdpau_hwaccel = { .init = vdpau_vc1_init, .uninit = ff_vdpau_common_uninit, .priv_data_size = sizeof(VDPAUContext), + .caps_internal = HWACCEL_CAP_ASYNC_SAFE, }; #endif @@ -162,4 +164,5 @@ AVHWAccel ff_vc1_vdpau_hwaccel = { .init = vdpau_vc1_init, .uninit = ff_vdpau_common_uninit, .priv_data_size = sizeof(VDPAUContext), + .caps_internal = HWACCEL_CAP_ASYNC_SAFE, }; diff --git a/libavcodec/version.h b/libavcodec/version.h index 3ed5a718d4..0378f1742f 100644 --- a/libavcodec/version.h +++ b/libavcodec/version.h @@ -29,7 +29,7 @@ #define LIBAVCODEC_VERSION_MAJOR 57 #define LIBAVCODEC_VERSION_MINOR 83 -#define LIBAVCODEC_VERSION_MICRO 100 +#define LIBAVCODEC_VERSION_MICRO 101 #define LIBAVCODEC_VERSION_INT AV_VERSION_INT(LIBAVCODEC_VERSION_MAJOR, \ LIBAVCODEC_VERSION_MINOR, \