From patchwork Thu Feb 16 18:29:38 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aman Karmani X-Patchwork-Id: 2581 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.103.89.21 with SMTP id n21csp2613162vsb; Thu, 16 Feb 2017 10:37:41 -0800 (PST) X-Received: by 10.28.134.76 with SMTP id i73mr14050083wmd.51.1487270261480; Thu, 16 Feb 2017 10:37:41 -0800 (PST) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id e2si10288972wra.193.2017.02.16.10.37.40; Thu, 16 Feb 2017 10:37:41 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@tmm1-net.20150623.gappssmtp.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 64903689A70; Thu, 16 Feb 2017 20:37:23 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pg0-f68.google.com (mail-pg0-f68.google.com [74.125.83.68]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 333CB680754 for ; Thu, 16 Feb 2017 20:37:17 +0200 (EET) Received: by mail-pg0-f68.google.com with SMTP id y6so1539353pgy.2 for ; Thu, 16 Feb 2017 10:37:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=tmm1-net.20150623.gappssmtp.com; s=20150623; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references; bh=FMxuq8/pKGCCPEogJjSwCL6tjWSkQhdzJC88QHfKGNY=; b=bKyMlhaNp9878mdHWW/fY2z0j8YiypXGKdf3vxgSIw2ZRAjrBm/NL/uJcwPdguvqVs M7VirMlzQ8ygF/31FT+2ASBTH9+sWTsEVFpUnpRvvk0N5dL9Up7xG3QEJ6mYMTYAhQKq 875onSviPt7t4hc0Hdj7+KAhxXdJh4WlRCxoZ8DAo7F1niRP+fTF44R/jQ0WovT9FMyx +49C+OhekhzMjrJhcthUWyOpfIlfmUrWH+xsBMFk1jo5NjOUbYCINEReU1r9kyy/Jpxe t7w3ll1tlaG/qF2eoVlAQPcDrhtkg1BZmGhWDN2unEozNNGpHRMx9F2N3I2ECGfR1+KP IQUA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references; bh=FMxuq8/pKGCCPEogJjSwCL6tjWSkQhdzJC88QHfKGNY=; b=dl6G910N27UZxyL4B1tvhuI5ikYjvmtzd2bT340bHG84138Xm3GpO1xRozS24knXPE Ux1bOqPIENlIWrFuFkA6A5i4RydCkKx5edARxqTw8lrx83oryj96PHPYimWr6Hrw0irU AOEehqgP9Ziy4aahTpE19cQVnRU0fdoMsIxN2cXWT+b/8KlUennoV3aWD7X5rOMaRXfC TeE41kIpKa9W0noEpu2MqjRpAftbVFhQ3d5vu53ckWQ32aCRVXcRJ5kAhkW12USW6YNs /ocD2yo1CU/9p4lqPIo2tMsQeKVZ4Qdw/QI27Lq3f9E1SsguBC8dEpOlp6aZNHcMCn7n pJMw== X-Gm-Message-State: AMke39nFeUEkBpCc7KpbE0wFsKxZ8LPP4HgI1ubrdpQxvOiBYnaAOG1k3e5JQm0SICpiaQ== X-Received: by 10.84.193.101 with SMTP id e92mr134003pld.4.1487270242090; Thu, 16 Feb 2017 10:37:22 -0800 (PST) Received: from tmm1-macbook.local.net (c-73-252-174-83.hsd1.ca.comcast.net. [73.252.174.83]) by smtp.gmail.com with ESMTPSA id h4sm14939227pfk.96.2017.02.16.10.37.21 (version=TLS1 cipher=AES128-SHA bits=128/128); Thu, 16 Feb 2017 10:37:21 -0800 (PST) From: Aman Gupta To: ffmpeg-devel@ffmpeg.org Date: Thu, 16 Feb 2017 10:29:38 -0800 Message-Id: <20170216182938.55393-3-ffmpeg@tmm1.net> X-Mailer: git-send-email 2.10.1 (Apple Git-78) In-Reply-To: <20170216182938.55393-1-ffmpeg@tmm1.net> References: <20170216182938.55393-1-ffmpeg@tmm1.net> Subject: [FFmpeg-devel] [PATCH 3/3] avcodec/videotoolbox: restart decompression session on bad data errors X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Aman Gupta MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" From: Aman Gupta On some platforms (observed on macOS Sierra with 12" macbook), the VT decoder will start returning errors when encountering an SPS change in the h264 bitstream. With this patch, the kVTVideoDecoderBadDataErr response from the decoder is caught and the decompression session is recreated with a new avcC. The "bad data" is then fed into the new decompression session so that it can be decoded correctly. I discovered the underlying issue here by running ffmpeg with lldb, which causes macOS to display debug information from the VT hardware decoder on stderr. The following errors were shown, which indicated the need to restart the decoder session with a new SPS/avcC: ffmpeg[15127:4094995] GVA error: SPS mismatch ... ffmpeg[15127:4094995] GVA error: AVF_PushMetaData, first field kAVF_QT0_SPSPPSBoundaryMarker ffmpeg[15127:4094995] GVA error: pushMetaData, submitNewJobs ffmpeg[15127:4094995] GVA warning: OutputQueueReadyCallback status = 1, buffer == 0x0 Tested with the following sample, which contains an SPS change midstream: http://tmm1.s3.amazonaws.com/videotoolbox/spschange.ts --- libavcodec/videotoolbox.c | 28 ++++++++++++++++++++++++---- 1 file changed, 24 insertions(+), 4 deletions(-) diff --git a/libavcodec/videotoolbox.c b/libavcodec/videotoolbox.c index 9be7bee..159d98d 100644 --- a/libavcodec/videotoolbox.c +++ b/libavcodec/videotoolbox.c @@ -38,6 +38,9 @@ #define VIDEOTOOLBOX_ESDS_EXTRADATA_PADDING 12 +static void videotoolbox_stop(AVCodecContext *avctx); +static int videotoolbox_start(AVCodecContext *avctx); + static void videotoolbox_buffer_release(void *opaque, uint8_t *data) { CVPixelBufferRef cv_buffer = (CVImageBufferRef)data; @@ -350,13 +353,25 @@ static int videotoolbox_common_end_frame(AVCodecContext *avctx, AVFrame *frame) int status; AVVideotoolboxContext *videotoolbox = avctx->hwaccel_context; VTContext *vtctx = avctx->internal->hwaccel_priv_data; + int retry; av_buffer_unref(&frame->buf[0]); if (!videotoolbox->session || !vtctx->bitstream) return AVERROR_INVALIDDATA; - status = videotoolbox_session_decode_frame(avctx); + for (retry = 0; retry < 2; retry++) { + status = videotoolbox_session_decode_frame(avctx); + + if (status == kVTVideoDecoderBadDataErr) { + av_log(avctx, AV_LOG_DEBUG, "vt decoder got bad data error, restarting..\n"); + videotoolbox_stop(avctx); + videotoolbox_start(avctx); + continue; + } else { + break; + } + } if (status) { av_log(avctx, AV_LOG_ERROR, "Failed to decode frame (%d)\n", status); @@ -506,7 +521,7 @@ static CMVideoFormatDescriptionRef videotoolbox_format_desc_create(CMVideoCodecT return cm_fmt_desc; } -static int videotoolbox_default_init(AVCodecContext *avctx) +static int videotoolbox_start(AVCodecContext *avctx) { AVVideotoolboxContext *videotoolbox = avctx->hwaccel_context; OSStatus status; @@ -587,7 +602,12 @@ static int videotoolbox_default_init(AVCodecContext *avctx) } } -static void videotoolbox_default_free(AVCodecContext *avctx) +static int videotoolbox_default_init(AVCodecContext *avctx) +{ + return videotoolbox_start(avctx); +} + +static void videotoolbox_stop(AVCodecContext *avctx) { AVVideotoolboxContext *videotoolbox = avctx->hwaccel_context; if (!videotoolbox) @@ -696,7 +716,7 @@ int av_videotoolbox_default_init2(AVCodecContext *avctx, AVVideotoolboxContext * void av_videotoolbox_default_free(AVCodecContext *avctx) { - videotoolbox_default_free(avctx); + videotoolbox_stop(avctx); av_freep(&avctx->hwaccel_context); } #endif /* CONFIG_VIDEOTOOLBOX */