From patchwork Thu May 3 02:24:58 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aman Karmani X-Patchwork-Id: 8754 Delivered-To: ffmpegpatchwork@gmail.com Received: by 2002:a02:155:0:0:0:0:0 with SMTP id c82-v6csp1297420jad; Wed, 2 May 2018 19:32:46 -0700 (PDT) X-Google-Smtp-Source: AB8JxZqgkqUe7vMaiEtNTypEn8J9J9d/jIbWky+PPJ9oxQnhSxTkLHkfIje6HCQ8rkkS3Fd78kPX X-Received: by 10.28.107.17 with SMTP id g17mr3925133wmc.65.1525314766021; Wed, 02 May 2018 19:32:46 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1525314765; cv=none; d=google.com; s=arc-20160816; b=DuTpu08j3fh3LanIExMaHA/TP0Ky3zxTqmvB5oChLa/z3EXFWMXZuFZGU0Z65xiuHi CtBGYV61T53odsRIAcQ03D9GQw4fmjKjOsRJTSqj6kcZBpqwfK8dPpjCP3b03xA9P4Ad 9sMAAFvvfI7xgGBAMoE9JtEpbE77vSmT/89CUIO/yidkws2RNUmmaychM5H0a4sXPyNw 7Mct6djCpC1O+ZNci2cup1fyjboo0J9x+jm/da62h3wGytAKF47TaAlTPEnSWJ7pq8xW 5EsHjigDDFWk8HeCLv8mYEgUi1HulFDGX7QehAFYqWxas3J56La3myTv/iIeXRxqy1yW o+wQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:dkim-signature:delivered-to:arc-authentication-results; bh=tnziu+AN4BN3+Bg6ZkoT8KV0ETnc8rqnsZwN2i+MCJs=; b=h0gJirsvbvWDKy3KLXLqOaq0Q6k3ydgH+8guP9PNYVXmcElKpxc42wuitegXrEUMcH Tlcsz2WAcn3U0WwYZs52H89LlBh8kyFC8J4DHhZ/SGoxEitp2ktd8TMaIV74GTCd16dC 0ky6uHeKi++/lZa/UtlstBRuo+hVgyvG9Mo1DqdSSsCEUDxuAcCOKweHx1CmgjF19vp2 uNwLAbdx16+5MTOvM7L7aZhSr58ACStRvg+dzVG60dVIjilffgAfB+BS0QflE9vqeyZK 6KfAE7DSY0nR5LEawQUQs4JaJbMoHVZ4xLB0uYKI7Sw3LLl+qQ6PxZON7d2SbJ7Nyn6b kWow== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@tmm1-net.20150623.gappssmtp.com header.s=20150623 header.b=vEr8SnF/; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id v11-v6si2460109wra.74.2018.05.02.19.32.45; Wed, 02 May 2018 19:32:45 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@tmm1-net.20150623.gappssmtp.com header.s=20150623 header.b=vEr8SnF/; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 8D09468A4A1; Thu, 3 May 2018 05:32:11 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pf0-f193.google.com (mail-pf0-f193.google.com [209.85.192.193]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 418EF68A51F for ; Thu, 3 May 2018 05:32:05 +0300 (EEST) Received: by mail-pf0-f193.google.com with SMTP id z9so13412653pfe.6 for ; Wed, 02 May 2018 19:32:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=tmm1-net.20150623.gappssmtp.com; s=20150623; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references; bh=Hs35I8m8Hp0SsPWQcV0FV5JfMM9/GVXx8xt+7aFXwZg=; b=vEr8SnF/NMD1D1B788fGpV55S2KR928BbPEsWlBkUN0L/SZs+24ShK1bVfIqhSvdIh Wn2itHxllUHJS4E3DcmEr4TatkBQmwoSNyOmj07PWhlcsIl5vtcu39hK6DxkRSUBoUpO UqHDKrFVrQIGl9tPbYW5e2zdA7SPWgLOyWL2rjHeSSxypoVOwp6qVvLfOV3rIYiLOYC2 UQM8xrH38/fOF4hBrFbwQAl9DBFQsek1aubdsYZlWnh6tktr0v6S/hjgbAAgtjR23dq/ sNIu+Jwl6jQ/ygeo+qdJRPfPQ7WNu5XvrlYouRm3aTksmlXpbStNQNeGF8OJpMoipowU xCEg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references; bh=Hs35I8m8Hp0SsPWQcV0FV5JfMM9/GVXx8xt+7aFXwZg=; b=ks1WjcOUba1ncuoqlkHUBLqDTdhSUTrWtqhRFveyNs8GSe6kbspeGvSt60SYg5CG9+ nYKkKpjYf1fyVDS2rqZhXl66IWpguOkd2hbI7H2iUTREQS48ozf+CChiCWhg6yGHlpU3 42DnK6t5XAZsOi4VPPg2wHczrxybh7JK/b+SlSKPORXDtjab77rl2HIoPK90rLsUK21a mSc7s/tzmW4Ge9MjC3TMRp4fvfsnLj3GIluPF1lzSb/vmNCUhasxS6jZ5laGtuzbb0Zj 74QY5hAt08n5Oy0PZ+4GmqoDKmawcsm+eUS0OiaUYN7voPN5+ozkih67X6j5HQcRvARM 2LRA== X-Gm-Message-State: ALQs6tB2gsHe8OUw+lZTWJNq53ap2xFsfjmHh/S7Bjct6nTuIMRbefNX sw2eNCYeRqX9pOLmAdaaMA69+3Ab X-Received: by 2002:a63:7c01:: with SMTP id x1-v6mr17858588pgc.286.1525314304057; Wed, 02 May 2018 19:25:04 -0700 (PDT) Received: from localhost.localdomain (c-69-181-54-242.hsd1.ca.comcast.net. [69.181.54.242]) by smtp.gmail.com with ESMTPSA id g26sm25171476pfh.76.2018.05.02.19.25.03 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Wed, 02 May 2018 19:25:03 -0700 (PDT) From: Aman Gupta To: ffmpeg-devel@ffmpeg.org Date: Wed, 2 May 2018 19:24:58 -0700 Message-Id: <20180503022500.76182-2-ffmpeg@tmm1.net> X-Mailer: git-send-email 2.14.2 In-Reply-To: <20180503022500.76182-1-ffmpeg@tmm1.net> References: <20180503022500.76182-1-ffmpeg@tmm1.net> Subject: [FFmpeg-devel] [PATCH v2 2/4] avcodec/mediacodecdec: restructure mediacodec_receive_frame X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: matthieu.bouron@gmail.com, Aman Gupta MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" From: Aman Gupta The new logic follows a recommendation by @rcombs to use dequeueInputBuffer with a timeout of 0 as a way to detect whether the codec wants more data. The dequeued buffer index is kept in MediaCodecDecContext until it can be used next. A similar technique is also used by the Google's official media player Exoplayer: see MediaCodecRenderer.feedInputBuffer(). Signed-off-by: Aman Gupta --- libavcodec/mediacodecdec.c | 80 ++++++++++++++++++++------------------- libavcodec/mediacodecdec_common.c | 28 ++++++++------ libavcodec/mediacodecdec_common.h | 4 +- 3 files changed, 61 insertions(+), 51 deletions(-) diff --git a/libavcodec/mediacodecdec.c b/libavcodec/mediacodecdec.c index 86ceee5a83..2ac22dd1f6 100644 --- a/libavcodec/mediacodecdec.c +++ b/libavcodec/mediacodecdec.c @@ -391,33 +391,11 @@ done: return ret; } -static int mediacodec_send_receive(AVCodecContext *avctx, - MediaCodecH264DecContext *s, - AVFrame *frame, bool wait) -{ - int ret; - - /* send any pending data from buffered packet */ - while (s->buffered_pkt.size) { - ret = ff_mediacodec_dec_send(avctx, s->ctx, &s->buffered_pkt); - if (ret == AVERROR(EAGAIN)) - break; - else if (ret < 0) - return ret; - s->buffered_pkt.size -= ret; - s->buffered_pkt.data += ret; - if (s->buffered_pkt.size <= 0) - av_packet_unref(&s->buffered_pkt); - } - - /* check for new frame */ - return ff_mediacodec_dec_receive(avctx, s->ctx, frame, wait); -} - static int mediacodec_receive_frame(AVCodecContext *avctx, AVFrame *frame) { MediaCodecH264DecContext *s = avctx->priv_data; int ret; + ssize_t index; /* In delay_flush mode, wait until the user has released or rendered all retained frames. */ @@ -427,28 +405,54 @@ static int mediacodec_receive_frame(AVCodecContext *avctx, AVFrame *frame) } } - /* flush buffered packet and check for new frame */ - ret = mediacodec_send_receive(avctx, s, frame, false); + /* poll for new frame */ + ret = ff_mediacodec_dec_receive(avctx, s->ctx, frame, false); if (ret != AVERROR(EAGAIN)) return ret; - /* skip fetching new packet if we still have one buffered */ - if (s->buffered_pkt.size > 0) - return mediacodec_send_receive(avctx, s, frame, true); + /* feed decoder */ + while (1) { + if (s->ctx->current_input_buffer < 0) { + /* poll for input space */ + index = ff_AMediaCodec_dequeueInputBuffer(s->ctx->codec, 0); + if (index < 0) { + /* no space, block for an output frame to appear */ + return ff_mediacodec_dec_receive(avctx, s->ctx, frame, true); + } + s->ctx->current_input_buffer = index; + } - /* fetch new packet or eof */ - ret = ff_decode_get_packet(avctx, &s->buffered_pkt); - if (ret == AVERROR_EOF) { - AVPacket null_pkt = { 0 }; - ret = ff_mediacodec_dec_send(avctx, s->ctx, &null_pkt); - if (ret < 0) + /* try to flush any buffered packet data */ + if (s->buffered_pkt.size > 0) { + ret = ff_mediacodec_dec_send(avctx, s->ctx, &s->buffered_pkt, false); + if (ret >= 0) { + s->buffered_pkt.size -= ret; + s->buffered_pkt.data += ret; + if (s->buffered_pkt.size <= 0) + av_packet_unref(&s->buffered_pkt); + } else if (ret < 0 && ret != AVERROR(EAGAIN)) { + return ret; + } + + /* poll for space again */ + continue; + } + + /* fetch new packet or eof */ + ret = ff_decode_get_packet(avctx, &s->buffered_pkt); + if (ret == AVERROR_EOF) { + AVPacket null_pkt = { 0 }; + ret = ff_mediacodec_dec_send(avctx, s->ctx, &null_pkt, true); + if (ret < 0) + return ret; + } else if (ret == AVERROR(EAGAIN) && s->ctx->current_input_buffer < 0) { + return ff_mediacodec_dec_receive(avctx, s->ctx, frame, true); + } else if (ret < 0) { return ret; + } } - else if (ret < 0) - return ret; - /* crank decoder with new packet */ - return mediacodec_send_receive(avctx, s, frame, true); + return AVERROR(EAGAIN); } static void mediacodec_decode_flush(AVCodecContext *avctx) diff --git a/libavcodec/mediacodecdec_common.c b/libavcodec/mediacodecdec_common.c index c0f0a6b983..887865a281 100644 --- a/libavcodec/mediacodecdec_common.c +++ b/libavcodec/mediacodecdec_common.c @@ -450,6 +450,7 @@ static int mediacodec_dec_flush_codec(AVCodecContext *avctx, MediaCodecDecContex s->eos = 0; atomic_fetch_add(&s->serial, 1); atomic_init(&s->hw_buffer_count, 0); + s->current_input_buffer = -1; status = ff_AMediaCodec_flush(codec); if (status < 0) { @@ -477,6 +478,7 @@ int ff_mediacodec_dec_init(AVCodecContext *avctx, MediaCodecDecContext *s, atomic_init(&s->refcount, 1); atomic_init(&s->hw_buffer_count, 0); atomic_init(&s->serial, 1); + s->current_input_buffer = -1; pix_fmt = ff_get_format(avctx, pix_fmts); if (pix_fmt == AV_PIX_FMT_MEDIACODEC) { @@ -561,16 +563,16 @@ fail: } int ff_mediacodec_dec_send(AVCodecContext *avctx, MediaCodecDecContext *s, - AVPacket *pkt) + AVPacket *pkt, bool wait) { int offset = 0; int need_draining = 0; uint8_t *data; - ssize_t index; + ssize_t index = s->current_input_buffer; size_t size; FFAMediaCodec *codec = s->codec; int status; - int64_t input_dequeue_timeout_us = INPUT_DEQUEUE_TIMEOUT_US; + int64_t input_dequeue_timeout_us = wait ? INPUT_DEQUEUE_TIMEOUT_US : 0; int64_t pts; if (s->flushing) { @@ -588,17 +590,19 @@ int ff_mediacodec_dec_send(AVCodecContext *avctx, MediaCodecDecContext *s, } while (offset < pkt->size || (need_draining && !s->draining)) { - - index = ff_AMediaCodec_dequeueInputBuffer(codec, input_dequeue_timeout_us); - if (ff_AMediaCodec_infoTryAgainLater(codec, index)) { - av_log(avctx, AV_LOG_TRACE, "No input buffer available, try again later\n"); - break; - } - if (index < 0) { - av_log(avctx, AV_LOG_ERROR, "Failed to dequeue input buffer (status=%zd)\n", index); - return AVERROR_EXTERNAL; + index = ff_AMediaCodec_dequeueInputBuffer(codec, input_dequeue_timeout_us); + if (ff_AMediaCodec_infoTryAgainLater(codec, index)) { + av_log(avctx, AV_LOG_TRACE, "No input buffer available, try again later\n"); + break; + } + + if (index < 0) { + av_log(avctx, AV_LOG_ERROR, "Failed to dequeue input buffer (status=%zd)\n", index); + return AVERROR_EXTERNAL; + } } + s->current_input_buffer = -1; data = ff_AMediaCodec_getInputBuffer(codec, index, &size); if (!data) { diff --git a/libavcodec/mediacodecdec_common.h b/libavcodec/mediacodecdec_common.h index 023d4c5fa7..0b21129fee 100644 --- a/libavcodec/mediacodecdec_common.h +++ b/libavcodec/mediacodecdec_common.h @@ -65,6 +65,7 @@ typedef struct MediaCodecDecContext { int display_height; uint64_t output_buffer_count; + ssize_t current_input_buffer; bool delay_flush; atomic_int serial; @@ -78,7 +79,8 @@ int ff_mediacodec_dec_init(AVCodecContext *avctx, int ff_mediacodec_dec_send(AVCodecContext *avctx, MediaCodecDecContext *s, - AVPacket *pkt); + AVPacket *pkt, + bool wait); int ff_mediacodec_dec_receive(AVCodecContext *avctx, MediaCodecDecContext *s,