From patchwork Thu Mar 22 18:14:50 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aman Karmani X-Patchwork-Id: 8105 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.2.1.70 with SMTP id c67csp1138281jad; Thu, 22 Mar 2018 11:15:08 -0700 (PDT) X-Google-Smtp-Source: AG47ELu7rQcL2DiRXArdCkbhmLigITB/TGWi0uEuQtAy/mgGku5B78L02zqgaUiPx+uH58qENgos X-Received: by 10.28.137.147 with SMTP id l141mr5961480wmd.141.1521742508008; Thu, 22 Mar 2018 11:15:08 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1521742507; cv=none; d=google.com; s=arc-20160816; b=V8N8kKfPbZl9NyYB8wsJ1ran9U5ojw+ecVZgx0mHS9mYT663x+5OOAvlska4PoXMK+ i+cSfENp+XTHNYGpsmROeWXK4QM9iUle6bhNE8kMue17WSedTJkNC6VThPut7UI4ulEC NfFjg/zsfSWgGZsa56c/M4CyWlujDobl8RdJ15YmExrl1nMU2yYd2ynm0VQpBrSv//3E ncTHMzEg06tjU0lArEpbaSmzhY0vtLmzz4sHQ6Je5PQ1p/8HCjW0mV4+3n57Lx7eDklJ SLPBE/XGWQuMGd0N75dVZ+6OwdMItsRXCoHnanXiuIZ0cHHEmnJIkY94/DGeq0dPrJMM aKXw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:message-id:date:to:from:dkim-signature :delivered-to:arc-authentication-results; bh=mLbV7rUkB5EA2zslzDzyZPGfW/EWKRFeWsCBiL/0LA0=; b=BohicvnQXzR2SrG/+3fPbYDsUkMTHyRcvxLjByBWFQrMwEpT3+VLZyKlHR7mZhgcg0 xuZ19bSALHgGWJNq3shXY4IJQDPlg5IclDwKubSJgmd+s3PSuTZP8pnXYwxYwrlSMGUq HVrsUF51QC9a+ufTMtnC5EZN5yIUbh7uSj55jNgXJExUcEChIr+kjydoR2kmAvHrXRMT FX+275ti0UQ6fo5+hmi0iIoIyOS6O/qrfeWFXLJ3QGajYho2/FoDOSltve1FsYVqrW4j 65htkiqTtRj/ac1Y8oZa4G6lvrrTG6N968AXnJFfmlUT9JBhiztHLmbwKeMAgVtyEQDg W01g== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@tmm1-net.20150623.gappssmtp.com header.s=20150623 header.b=yqefNTl7; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id k72si4863340wmd.226.2018.03.22.11.15.07; Thu, 22 Mar 2018 11:15:07 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@tmm1-net.20150623.gappssmtp.com header.s=20150623 header.b=yqefNTl7; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id D5A916807F0; Thu, 22 Mar 2018 20:14:49 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pg0-f49.google.com (mail-pg0-f49.google.com [74.125.83.49]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 7C7966807F0 for ; Thu, 22 Mar 2018 20:14:42 +0200 (EET) Received: by mail-pg0-f49.google.com with SMTP id i9so3576025pgq.10 for ; Thu, 22 Mar 2018 11:14:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=tmm1-net.20150623.gappssmtp.com; s=20150623; h=sender:from:to:cc:subject:date:message-id; bh=shylohBY54j0odkaKHradSAVVX0GKKoiSkWPMxk6uOk=; b=yqefNTl7J2vB4RAvBc25usKUWk+bP7Nl0rmT7iKMsVlV7CJ64rAKM186HzENt+bE0y gadZWbr6Vqzo4lb+9Fukp9UTgBXgp/+wha4DRybRszjVZK3N99Eynn6+8eSYxEUjL4RQ hdWA+srQr1EHiCFALF8HX2lZNqs0uhuaEYvaFkFPaklJpmDDBJiBWMcCYaIVNoJGB5Cm W2vib7uvy9YOQmXQk4CYbVI52qMDgVQ2Rm7ymNKbXBpFFvz8aEMn8yFKnSzClC0Z0gcH AVGBPzofHfgVyo8LXeauhOqbylPsritt32yArk4mLUTwnu65dA4BY8DL89P97ZMFKByI yGbg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id; bh=shylohBY54j0odkaKHradSAVVX0GKKoiSkWPMxk6uOk=; b=QiiKyd+ySQIribzciMJXWOz4ghBIBUOXwOly4YJveDlWi6rVAoe+JK8AxqSegvTBk+ uqF1n2otTSk3vGX7u5cQVhhy8bx+uEjnzeIM7srTvXgVuaXSQVrJZnlmOUG2J9p6Rwun vH3LajnPf2ZbL+Wzal7rotpPsumCrAe23/jbJ2j1MH9xy3mkCWB3ck4DXfRmuknznjbY 0KAQDgaGa+ps5caKNqwPtLSY9eODAlypOWrTHns0YuUhMZADJPQCyKtavALjX6FRGZr/ altQH6q1Y4uh6ecyQeFd1FDmsqfjHHlz+iVuywt9MSTKbsL0m4dh7llPmdv+G0waZTJR 6Zdw== X-Gm-Message-State: AElRT7F85O8Cc1rKMmmbsTZcHjnp0HvUHwHKS2UW1zXAbTR1I+th/yKe ZpxeG3z/M1ECs+lgAeeaT4sZWOIK X-Received: by 10.99.109.142 with SMTP id i136mr18591212pgc.306.1521742495697; Thu, 22 Mar 2018 11:14:55 -0700 (PDT) Received: from localhost.localdomain (c-69-181-54-242.hsd1.ca.comcast.net. [69.181.54.242]) by smtp.gmail.com with ESMTPSA id h6sm13175068pgn.65.2018.03.22.11.14.54 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 22 Mar 2018 11:14:54 -0700 (PDT) From: Aman Gupta To: ffmpeg-devel@ffmpeg.org Date: Thu, 22 Mar 2018 11:14:50 -0700 Message-Id: <20180322181450.1152-1-ffmpeg@tmm1.net> X-Mailer: git-send-email 2.14.2 Subject: [FFmpeg-devel] [PATCH] avcodec/mediacodecdec: work around for decoding h264 with coded fields X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: matthieu.bouron@gmail.com, Aman Gupta MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" From: Aman Gupta This is a hacky work-around for #7092, where the lavc h264 parser splits coded fields into separate video packets, only one of which has a PTS set. The MediaCodec#queueInputBuffer API expects a PTS along with incoming video packets, and breaks badly when the PTS is missing or incorrect (previously it would be passed in as AV_NOPTS_VALUE, but the same breakage happens if you pass in 0 instead). Since it seems there's no easy fix for #7092, this patch stores the previous PTS in the decoder context and re-uses it for the second packet. This emulates the behavior of other Android video players that don't split the coded fields, and pass them as a single buffer with the same timestamp. --- libavcodec/mediacodecdec_common.c | 26 +++++++++++++++----------- libavcodec/mediacodecdec_common.h | 2 ++ 2 files changed, 17 insertions(+), 11 deletions(-) diff --git a/libavcodec/mediacodecdec_common.c b/libavcodec/mediacodecdec_common.c index e31adb487c..16112b186d 100644 --- a/libavcodec/mediacodecdec_common.c +++ b/libavcodec/mediacodecdec_common.c @@ -450,6 +450,7 @@ static int mediacodec_dec_flush_codec(AVCodecContext *avctx, MediaCodecDecContex s->eos = 0; atomic_fetch_add(&s->serial, 1); atomic_init(&s->hw_buffer_count, 0); + s->last_pts = AV_NOPTS_VALUE; status = ff_AMediaCodec_flush(codec); if (status < 0) { @@ -477,6 +478,7 @@ int ff_mediacodec_dec_init(AVCodecContext *avctx, MediaCodecDecContext *s, atomic_init(&s->refcount, 1); atomic_init(&s->hw_buffer_count, 0); atomic_init(&s->serial, 1); + s->last_pts = AV_NOPTS_VALUE; pix_fmt = ff_get_format(avctx, pix_fmts); if (pix_fmt == AV_PIX_FMT_MEDIACODEC) { @@ -571,6 +573,7 @@ int ff_mediacodec_dec_send(AVCodecContext *avctx, MediaCodecDecContext *s, FFAMediaCodec *codec = s->codec; int status; int64_t input_dequeue_timeout_us = INPUT_DEQUEUE_TIMEOUT_US; + int64_t pts; if (s->flushing) { av_log(avctx, AV_LOG_ERROR, "Decoder is flushing and cannot accept new buffer " @@ -605,14 +608,21 @@ int ff_mediacodec_dec_send(AVCodecContext *avctx, MediaCodecDecContext *s, return AVERROR_EXTERNAL; } + pts = pkt->pts; + if (pts == AV_NOPTS_VALUE && s->last_pts != AV_NOPTS_VALUE) { + pts = s->last_pts; + } else if (pts == AV_NOPTS_VALUE) { + av_log(avctx, AV_LOG_WARNING, "Packet is missing PTS!\n"); + pts = 0; + } + s->last_pts = pkt->pts; + if (pts && avctx->pkt_timebase.num && avctx->pkt_timebase.den) { + pts = av_rescale_q(pts, avctx->pkt_timebase, av_make_q(1, 1000000)); + } + if (need_draining) { - int64_t pts = pkt->pts; uint32_t flags = ff_AMediaCodec_getBufferFlagEndOfStream(codec); - if (s->surface) { - pts = av_rescale_q(pts, avctx->pkt_timebase, av_make_q(1, 1000000)); - } - av_log(avctx, AV_LOG_DEBUG, "Sending End Of Stream signal\n"); status = ff_AMediaCodec_queueInputBuffer(codec, index, 0, 0, pts, flags); @@ -627,16 +637,10 @@ int ff_mediacodec_dec_send(AVCodecContext *avctx, MediaCodecDecContext *s, s->draining = 1; break; } else { - int64_t pts = pkt->pts; - size = FFMIN(pkt->size - offset, size); memcpy(data, pkt->data + offset, size); offset += size; - if (avctx->pkt_timebase.num && avctx->pkt_timebase.den) { - pts = av_rescale_q(pts, avctx->pkt_timebase, av_make_q(1, 1000000)); - } - status = ff_AMediaCodec_queueInputBuffer(codec, index, 0, size, pts, 0); if (status < 0) { av_log(avctx, AV_LOG_ERROR, "Failed to queue input buffer (status = %d)\n", status); diff --git a/libavcodec/mediacodecdec_common.h b/libavcodec/mediacodecdec_common.h index 023d4c5fa7..045b7aaac4 100644 --- a/libavcodec/mediacodecdec_common.h +++ b/libavcodec/mediacodecdec_common.h @@ -69,6 +69,8 @@ typedef struct MediaCodecDecContext { bool delay_flush; atomic_int serial; + int64_t last_pts; + } MediaCodecDecContext; int ff_mediacodec_dec_init(AVCodecContext *avctx,