From patchwork Wed Feb 15 01:14:37 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aman Karmani X-Patchwork-Id: 2551 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.103.89.21 with SMTP id n21csp1732444vsb; Tue, 14 Feb 2017 17:22:26 -0800 (PST) X-Received: by 10.28.13.131 with SMTP id 125mr5582095wmn.122.1487121745946; Tue, 14 Feb 2017 17:22:25 -0800 (PST) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 89si2942166wre.175.2017.02.14.17.22.25; Tue, 14 Feb 2017 17:22:25 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@tmm1-net.20150623.gappssmtp.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 8EF196891FE; Wed, 15 Feb 2017 03:22:16 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-it0-f67.google.com (mail-it0-f67.google.com [209.85.214.67]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id AC78068799F for ; Wed, 15 Feb 2017 03:22:09 +0200 (EET) Received: by mail-it0-f67.google.com with SMTP id e137so8378185itc.0 for ; Tue, 14 Feb 2017 17:22:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=tmm1-net.20150623.gappssmtp.com; s=20150623; h=sender:from:to:cc:subject:date:message-id; bh=WxAvrpswbGyfMuHBeaVmHi8iLGKW8xSE2RoUBIpmFlk=; b=UNJhGipaU4Feh/EjdlqfG/c+74cEyaBNieIkemVrBCXxPl3liyMlmkkO/APycLdSet eh+TNHpfm4IEx5nReavEgo0KV6nHugBChedWMMD5JJeH/V68I5NNNTW+KZueaJVS3jMS 1def/ajgbKHBm0zyRjez00385ayO+tNeOlt0XTrzguE3pBX1a77ik+Z3KXhBXtYKNtSO Z6KoInTg7zj4oZ3x+4xumoF5I5OcGjThYJvJyMuFCDruCpE7/teemgIZA794r7HjqbCz Ee+dfqGd1ygO4Pyt1VAeQtFzFnzs57VEoe7685C6K4/WSnaYryY4WWlA4c9Yn0F2myuC tBSA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id; bh=WxAvrpswbGyfMuHBeaVmHi8iLGKW8xSE2RoUBIpmFlk=; b=E5iTa2JV1bZ2RI/PxkI6rC1Um5k/hs28Ukwae6aSx0vrETcgotpXGoM0lSbw3hruUR b12GQ2XztsJE29NECvFLMVC1TcHn7SWeEJdH3SVXgCnqS8RuEuc9SAnvjYwSThSfqocJ 3ZXXhWmxOEGd248HfWvvr1E1XQK03wwRzioizkFF4URnsxu9HFY0cscyNMP4K4YULKap LLb2WNuEsBEO4WtK3bcTQmrug23FyFHtpUzAxrmPoprhX5vAyo3wLkyLQUoiBA9idBtf NYKXvESQlF0JyVyFDS6840bUOJF/RxdbJOVcJdfpw6fZRHlBHar6LO+MjnzPIdwc7M7h e+bQ== X-Gm-Message-State: AMke39leFSwQi2RyyK5pVqYDNJ7ZPLJBtwxh9fdqcEAaHIuzEVfnHSBW0wJlvuS31A51lQ== X-Received: by 10.99.133.65 with SMTP id u62mr35726692pgd.70.1487121284808; Tue, 14 Feb 2017 17:14:44 -0800 (PST) Received: from tmm1-macbook.local.net (c-73-252-174-83.hsd1.ca.comcast.net. [73.252.174.83]) by smtp.gmail.com with ESMTPSA id r78sm3396784pfl.63.2017.02.14.17.14.43 (version=TLS1 cipher=AES128-SHA bits=128/128); Tue, 14 Feb 2017 17:14:43 -0800 (PST) From: Aman Gupta To: ffmpeg-devel@ffmpeg.org Date: Tue, 14 Feb 2017 17:14:37 -0800 Message-Id: <20170215011437.49702-1-ffmpeg@tmm1.net> X-Mailer: git-send-email 2.10.1 (Apple Git-78) Subject: [FFmpeg-devel] [PATCH] avcodec/h264, videotoolbox: handle streams with multiple/changing PPS X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Aman Gupta MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" From: Aman Gupta The videotoolbox hwaccel only receives SLICE and IDR_SLICE NALUs. This works fine most of the time, but some streams fail to decode because changes in PPS are not propagated to the VT decoder. The failures in this case are incredibly annoying, as VTDecodeFrame() still returns noErr. Simiarly the decoder callback is invoked with noErr, and a NULL imageBuffer. Even though all the VT apis indicate success, no frames are actually decoded. When running ffmpeg via lldb however, some internal VT errors and warnings show up all of a sudden. These suggest that the bitstream is failing some internal consistency checks. $ ffmpeg -y -loglevel error -threads 1 -hwaccel videotoolbox \ -i http://tmm1.s3.amazonaws.com/h264.ts -map v -f null /dev/null ... [h264 @ 0x7fdadc000000] hardware accelerator failed to decode picture Error while decoding stream #0:0: Unknown error occurred vt decoder cb: output image buffer is null ... $ lldb -- ffmpeg ... ... ffmpeg[49384:2009219] GVA error: AVC_RBSP::parseSliceHeader error ffmpeg[49384:2009219] GVA error: pushPicture parseSliceHeader vt decoder cb: output image buffer is null ffmpeg[49384:2009219] GVA warning: OutputQueueReadyCallback status = 1, buffer == 0x0 [h264 @ 0x10300a200] hardware accelerator failed to decode picture: Unknown error occurred Error while decoding stream #0:0: Unknown error occurred ... After this patch, there are no more errors and the sample decodes as expected. --- libavcodec/h264dec.c | 7 +++++++ libavcodec/videotoolbox.c | 8 +++++--- 2 files changed, 12 insertions(+), 3 deletions(-) diff --git a/libavcodec/h264dec.c b/libavcodec/h264dec.c index 41c0964..af8b256 100644 --- a/libavcodec/h264dec.c +++ b/libavcodec/h264dec.c @@ -755,6 +755,13 @@ FF_ENABLE_DEPRECATION_WARNINGS nal->size_bits); if (ret < 0 && (h->avctx->err_recognition & AV_EF_EXPLODE)) goto end; + if (avctx->hwaccel && avctx->hwaccel->pix_fmt == AV_PIX_FMT_VIDEOTOOLBOX) { + ret = avctx->hwaccel->decode_slice(avctx, + nal->raw_data, + nal->raw_size); + if (ret < 0) + goto end; + } break; case H264_NAL_AUD: case H264_NAL_END_SEQUENCE: diff --git a/libavcodec/videotoolbox.c b/libavcodec/videotoolbox.c index 1288aa5..1b5dffd 100644 --- a/libavcodec/videotoolbox.c +++ b/libavcodec/videotoolbox.c @@ -138,8 +138,6 @@ int ff_videotoolbox_h264_start_frame(AVCodecContext *avctx, VTContext *vtctx = avctx->internal->hwaccel_priv_data; H264Context *h = avctx->priv_data; - vtctx->bitstream_size = 0; - if (h->is_avc == 1) { return videotoolbox_buffer_copy(vtctx, buffer, size); } @@ -373,8 +371,12 @@ static int videotoolbox_h264_end_frame(AVCodecContext *avctx) { H264Context *h = avctx->priv_data; AVFrame *frame = h->cur_pic_ptr->f; + VTContext *vtctx = avctx->internal->hwaccel_priv_data; + int ret; - return videotoolbox_common_end_frame(avctx, frame); + ret = videotoolbox_common_end_frame(avctx, frame); + vtctx->bitstream_size = 0; + return ret; } static int videotoolbox_mpeg_start_frame(AVCodecContext *avctx,