From patchwork Sun Sep 4 16:47:29 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Philip Langdale X-Patchwork-Id: 415 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.103.140.134 with SMTP id o128csp2371617vsd; Sun, 4 Sep 2016 09:58:10 -0700 (PDT) X-Received: by 10.28.50.3 with SMTP id y3mr12281044wmy.23.1473008290672; Sun, 04 Sep 2016 09:58:10 -0700 (PDT) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id m2si21138020wjw.164.2016.09.04.09.58.08; Sun, 04 Sep 2016 09:58:10 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@overt.org; dkim=neutral (body hash did not verify) header.i=@overt.org; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id BA300689E31; Sun, 4 Sep 2016 19:57:51 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from rs224.mailgun.us (rs224.mailgun.us [209.61.151.224]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 9C08A689C4D for ; Sun, 4 Sep 2016 19:57:44 +0300 (EEST) DKIM-Signature: a=rsa-sha256; v=1; c=relaxed/relaxed; d=overt.org; q=dns/txt; s=k1; t=1473008271; h=References: In-Reply-To: Message-Id: Date: Subject: Cc: To: From: Sender; bh=RXFtH0pzvynPfZpAnAG2TQ0tj0WDPkmn0+8u8oubRIA=; b=IpZGHQFIvoAAdA/BB3j4zZHyH/5lnlPu0hhpiSosv1ZFvzDgXG7zT3S1qu4GWJznaoc2vCes NPM7eBC8NouVxABD4jTAo81LGAjjHC46qI0LlqLyYEdPHKgBjVUSLzdNB07AYXCdTZGnFTcp zJhRV8644s77bN33Pn5o8zsBicM= DomainKey-Signature: a=rsa-sha1; c=nofws; d=overt.org; s=k1; q=dns; h=Sender: From: To: Cc: Subject: Date: Message-Id: In-Reply-To: References; b=x7IPhbmlqUh5XQCrmH2VURg20R86EZCwODfWyQsxAOv7DjD+uQFNo+tzCCvCPmXs287Kc0 eaoVH1jLYOYfHVHQGA1AGDII3/VheA07EQguZ6JbSbZ1gTzPSKJlNLKWuHVzSq5hANvm0nvr WLkowHJuhfzUy07Kwc0FklO9kXPMY= X-Mailgun-Sending-Ip: 209.61.151.224 X-Mailgun-Sid: WyIyM2Q3MCIsICJmZm1wZWctZGV2ZWxAZmZtcGVnLm9yZyIsICI0YTg5NjEiXQ== Received: from mail.overt.org (155.208.178.107.bc.googleusercontent.com [107.178.208.155]) by mxa.mailgun.org with ESMTP id 57cc5029.7f5ec09be0a0-in02; Sun, 04 Sep 2016 16:47:37 -0000 (UTC) Received: from authenticated-user (mail.overt.org [107.178.208.155]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (No client certificate requested) by mail.overt.org (Postfix) with ESMTPSA id 0CBC16004C; Sun, 4 Sep 2016 16:47:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=overt.org; s=mail; t=1473007657; bh=W88DgqM58gy+0TdBXgEzIqEkm4S+KlKRbHrHECOGGLw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=CHe54LLMJp1f+z7hAQZR5UY3GDo8LEQHeL2tKYxxG5dXdr6eL4RDz+QjYUNIDdjyw vIb2/s4DayZ6vvIJyZZpduvgaITTqdrBkxuiz+pILhAfZ0z/07OEOmf53jBRluGIdD 0tHe7xfOFS+D0qfLEQFl1bVFrnidZhw0aFKUTPVDiga0hb5SRepvx+6Taa/QPKIs+A Snx66lEB/6xyfXt8BdIb0oBGwreu2F9T9so1Z5EwA21hplAQOD5ccYMRdCefD7iLu/ GUVOElEjeIJ5qsSurEDxyMAzUAxk5jNT1NJjZr6JvWuuGbSmJGv54cyvzDXj6DulOl Hd7tJkYGjAg1g== From: Philip Langdale To: ffmpeg-devel@ffmpeg.org Date: Sun, 4 Sep 2016 09:47:29 -0700 Message-Id: <1473007649-32429-3-git-send-email-philipl@overt.org> In-Reply-To: <1473007649-32429-1-git-send-email-philipl@overt.org> References: <1473007649-32429-1-git-send-email-philipl@overt.org> Subject: [FFmpeg-devel] [PATCH 2/2] cuvid: Implement flush to support seeking in media players X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Philip Langdale MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Right now, if we attempt to use cuvid in a media player and then try to seek, the decoder will happily pass out whatever frames were already in flight before the seek. There is both the output queue in our code and some number of frames within the cuvid decoder that need to be accounted for. cuvid doesn't support flush, so our only choice is to do a brute-force re-creation of the decoder, which also implies re-creating the parser, but this is fine. The only subtlty is that there is sanity check code in decoder initialisation that wants to make sure the HWContextFrame hasn't already been initialised. This is a fair check to do at the beginning but not after a flush, so it has to be made conditional. Signed-off-by: Philip Langdale --- libavcodec/cuvid.c | 130 +++++++++++++++++++++++++++++++++++++---------------- 1 file changed, 91 insertions(+), 39 deletions(-) diff --git a/libavcodec/cuvid.c b/libavcodec/cuvid.c index 7e03e30..7cc1331 100644 --- a/libavcodec/cuvid.c +++ b/libavcodec/cuvid.c @@ -46,9 +46,13 @@ typedef struct CuvidContext AVFifoBuffer *frame_queue; int internal_error; + int ever_flushed; cudaVideoCodec codec_type; cudaVideoChromaFormat chroma_format; + + CUVIDPARSERPARAMS cuparseinfo; + CUVIDEOFORMATEX cuparse_ext; } CuvidContext; static int check_cu(AVCodecContext *avctx, CUresult err, const char *func) @@ -127,7 +131,7 @@ static int CUDAAPI cuvid_handle_video_sequence(void *opaque, CUVIDEOFORMAT* form return 0; } - if (hwframe_ctx->pool) { + if (hwframe_ctx->pool && !ctx->ever_flushed) { av_log(avctx, AV_LOG_ERROR, "AVHWFramesContext is already initialized\n"); ctx->internal_error = AVERROR(EINVAL); return 0; @@ -164,14 +168,16 @@ static int CUDAAPI cuvid_handle_video_sequence(void *opaque, CUVIDEOFORMAT* form if (ctx->internal_error < 0) return 0; - hwframe_ctx->format = AV_PIX_FMT_CUDA; - hwframe_ctx->sw_format = AV_PIX_FMT_NV12; - hwframe_ctx->width = FFALIGN(avctx->coded_width, 32); - hwframe_ctx->height = FFALIGN(avctx->coded_height, 32); + if (!hwframe_ctx->pool) { + hwframe_ctx->format = AV_PIX_FMT_CUDA; + hwframe_ctx->sw_format = AV_PIX_FMT_NV12; + hwframe_ctx->width = FFALIGN(avctx->coded_width, 32); + hwframe_ctx->height = FFALIGN(avctx->coded_height, 32); - if ((ctx->internal_error = av_hwframe_ctx_init(ctx->hwframe)) < 0) { - av_log(avctx, AV_LOG_ERROR, "av_hwframe_ctx_init failed\n"); - return 0; + if ((ctx->internal_error = av_hwframe_ctx_init(ctx->hwframe)) < 0) { + av_log(avctx, AV_LOG_ERROR, "av_hwframe_ctx_init failed\n"); + return 0; + } } return 1; @@ -461,8 +467,6 @@ static av_cold int cuvid_decode_init(AVCodecContext *avctx) AVCUDADeviceContext *device_hwctx; AVHWDeviceContext *device_ctx; AVHWFramesContext *hwframe_ctx; - CUVIDPARSERPARAMS cuparseinfo; - CUVIDEOFORMATEX cuparse_ext; CUVIDSOURCEDATAPACKET seq_pkt; CUdevice device; CUcontext cuda_ctx = NULL; @@ -550,61 +554,61 @@ static av_cold int cuvid_decode_init(AVCodecContext *avctx) } } - memset(&cuparseinfo, 0, sizeof(cuparseinfo)); - memset(&cuparse_ext, 0, sizeof(cuparse_ext)); + memset(&ctx->cuparseinfo, 0, sizeof(ctx->cuparseinfo)); + memset(&ctx->cuparse_ext, 0, sizeof(ctx->cuparse_ext)); memset(&seq_pkt, 0, sizeof(seq_pkt)); - cuparseinfo.pExtVideoInfo = &cuparse_ext; + ctx->cuparseinfo.pExtVideoInfo = &ctx->cuparse_ext; switch (avctx->codec->id) { #if CONFIG_H263_CUVID_DECODER case AV_CODEC_ID_H263: - cuparseinfo.CodecType = cudaVideoCodec_MPEG4; + ctx->cuparseinfo.CodecType = cudaVideoCodec_MPEG4; break; #endif #if CONFIG_H264_CUVID_DECODER case AV_CODEC_ID_H264: - cuparseinfo.CodecType = cudaVideoCodec_H264; + ctx->cuparseinfo.CodecType = cudaVideoCodec_H264; break; #endif #if CONFIG_HEVC_CUVID_DECODER case AV_CODEC_ID_HEVC: - cuparseinfo.CodecType = cudaVideoCodec_HEVC; + ctx->cuparseinfo.CodecType = cudaVideoCodec_HEVC; break; #endif #if CONFIG_MJPEG_CUVID_DECODER case AV_CODEC_ID_MJPEG: - cuparseinfo.CodecType = cudaVideoCodec_JPEG; + ctx->cuparseinfo.CodecType = cudaVideoCodec_JPEG; break; #endif #if CONFIG_MPEG1_CUVID_DECODER case AV_CODEC_ID_MPEG1VIDEO: - cuparseinfo.CodecType = cudaVideoCodec_MPEG1; + ctx->cuparseinfo.CodecType = cudaVideoCodec_MPEG1; break; #endif #if CONFIG_MPEG2_CUVID_DECODER case AV_CODEC_ID_MPEG2VIDEO: - cuparseinfo.CodecType = cudaVideoCodec_MPEG2; + ctx->cuparseinfo.CodecType = cudaVideoCodec_MPEG2; break; #endif #if CONFIG_MPEG4_CUVID_DECODER case AV_CODEC_ID_MPEG4: - cuparseinfo.CodecType = cudaVideoCodec_MPEG4; + ctx->cuparseinfo.CodecType = cudaVideoCodec_MPEG4; break; #endif #if CONFIG_VP8_CUVID_DECODER case AV_CODEC_ID_VP8: - cuparseinfo.CodecType = cudaVideoCodec_VP8; + ctx->cuparseinfo.CodecType = cudaVideoCodec_VP8; break; #endif #if CONFIG_VP9_CUVID_DECODER case AV_CODEC_ID_VP9: - cuparseinfo.CodecType = cudaVideoCodec_VP9; + ctx->cuparseinfo.CodecType = cudaVideoCodec_VP9; break; #endif #if CONFIG_VC1_CUVID_DECODER case AV_CODEC_ID_VC1: - cuparseinfo.CodecType = cudaVideoCodec_VC1; + ctx->cuparseinfo.CodecType = cudaVideoCodec_VC1; break; #endif default: @@ -630,38 +634,38 @@ static av_cold int cuvid_decode_init(AVCodecContext *avctx) goto error; } - cuparse_ext.format.seqhdr_data_length = ctx->bsf->par_out->extradata_size; - memcpy(cuparse_ext.raw_seqhdr_data, + ctx->cuparse_ext.format.seqhdr_data_length = ctx->bsf->par_out->extradata_size; + memcpy(ctx->cuparse_ext.raw_seqhdr_data, ctx->bsf->par_out->extradata, - FFMIN(sizeof(cuparse_ext.raw_seqhdr_data), ctx->bsf->par_out->extradata_size)); + FFMIN(sizeof(ctx->cuparse_ext.raw_seqhdr_data), ctx->bsf->par_out->extradata_size)); } else if (avctx->extradata_size > 0) { - cuparse_ext.format.seqhdr_data_length = avctx->extradata_size; - memcpy(cuparse_ext.raw_seqhdr_data, + ctx->cuparse_ext.format.seqhdr_data_length = avctx->extradata_size; + memcpy(ctx->cuparse_ext.raw_seqhdr_data, avctx->extradata, - FFMIN(sizeof(cuparse_ext.raw_seqhdr_data), avctx->extradata_size)); + FFMIN(sizeof(ctx->cuparse_ext.raw_seqhdr_data), avctx->extradata_size)); } - cuparseinfo.ulMaxNumDecodeSurfaces = MAX_FRAME_COUNT; - cuparseinfo.ulMaxDisplayDelay = 4; - cuparseinfo.pUserData = avctx; - cuparseinfo.pfnSequenceCallback = cuvid_handle_video_sequence; - cuparseinfo.pfnDecodePicture = cuvid_handle_picture_decode; - cuparseinfo.pfnDisplayPicture = cuvid_handle_picture_display; + ctx->cuparseinfo.ulMaxNumDecodeSurfaces = MAX_FRAME_COUNT; + ctx->cuparseinfo.ulMaxDisplayDelay = 4; + ctx->cuparseinfo.pUserData = avctx; + ctx->cuparseinfo.pfnSequenceCallback = cuvid_handle_video_sequence; + ctx->cuparseinfo.pfnDecodePicture = cuvid_handle_picture_decode; + ctx->cuparseinfo.pfnDisplayPicture = cuvid_handle_picture_display; ret = CHECK_CU(cuCtxPushCurrent(cuda_ctx)); if (ret < 0) goto error; - ret = cuvid_test_dummy_decoder(avctx, &cuparseinfo); + ret = cuvid_test_dummy_decoder(avctx, &ctx->cuparseinfo); if (ret < 0) goto error; - ret = CHECK_CU(cuvidCreateVideoParser(&ctx->cuparser, &cuparseinfo)); + ret = CHECK_CU(cuvidCreateVideoParser(&ctx->cuparser, &ctx->cuparseinfo)); if (ret < 0) goto error; - seq_pkt.payload = cuparse_ext.raw_seqhdr_data; - seq_pkt.payload_size = cuparse_ext.format.seqhdr_data_length; + seq_pkt.payload = ctx->cuparse_ext.raw_seqhdr_data; + seq_pkt.payload_size = ctx->cuparse_ext.format.seqhdr_data_length; if (seq_pkt.payload && seq_pkt.payload_size) { ret = CHECK_CU(cuvidParseVideoData(ctx->cuparser, &seq_pkt)); @@ -673,6 +677,8 @@ static av_cold int cuvid_decode_init(AVCodecContext *avctx) if (ret < 0) goto error; + ctx->ever_flushed = 0; + return 0; error: @@ -680,6 +686,51 @@ error: return ret; } +static void cuvid_flush(AVCodecContext *avctx) +{ + CuvidContext *ctx = avctx->priv_data; + AVHWDeviceContext *device_ctx = (AVHWDeviceContext*)ctx->hwdevice->data; + AVCUDADeviceContext *device_hwctx = device_ctx->hwctx; + CUcontext dummy, cuda_ctx = device_hwctx->cuda_ctx; + int ret; + + ctx->ever_flushed = 1; + + ret = CHECK_CU(cuCtxPushCurrent(cuda_ctx)); + if (ret < 0) + goto error; + + av_fifo_freep(&ctx->frame_queue); + + ctx->frame_queue = av_fifo_alloc(MAX_FRAME_COUNT * sizeof(CUVIDPARSERDISPINFO)); + if (!ctx->frame_queue) { + av_log(avctx, AV_LOG_ERROR, "Failed to recreate frame queue on flush\n"); + return; + } + + if (ctx->cudecoder) { + cuvidDestroyDecoder(ctx->cudecoder); + ctx->cudecoder = NULL; + } + + if (ctx->cuparser) { + cuvidDestroyVideoParser(ctx->cuparser); + ctx->cuparser = NULL; + } + + ret = CHECK_CU(cuvidCreateVideoParser(&ctx->cuparser, &ctx->cuparseinfo)); + if (ret < 0) + goto error; + + ret = CHECK_CU(cuCtxPopCurrent(&dummy)); + if (ret < 0) + goto error; + + return; + error: + av_log(avctx, AV_LOG_ERROR, "CUDA reinit on flush failed\n"); +} + #define DEFINE_CUVID_CODEC(x, X) \ AVHWAccel ff_##x##_cuvid_hwaccel = { \ .name = #x "_cuvid", \ @@ -696,6 +747,7 @@ error: .init = cuvid_decode_init, \ .close = cuvid_decode_end, \ .decode = cuvid_decode_frame, \ + .flush = cuvid_flush, \ .capabilities = AV_CODEC_CAP_DELAY, \ .pix_fmts = (const enum AVPixelFormat[]){ AV_PIX_FMT_CUDA, \ AV_PIX_FMT_NV12, \