From patchwork Tue Apr 24 20:59:26 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aman Karmani X-Patchwork-Id: 8640 Delivered-To: ffmpegpatchwork@gmail.com Received: by 2002:a02:155:0:0:0:0:0 with SMTP id c82-v6csp2282758jad; Tue, 24 Apr 2018 14:00:08 -0700 (PDT) X-Google-Smtp-Source: AIpwx49INH+bTZzuk/5T3yj4PhIpAqL54y76N7QFEm7mvBcxTFfmURVErfpwYZHdSs38zWdn1Ugy X-Received: by 10.28.132.203 with SMTP id g194mr11580675wmd.8.1524603608345; Tue, 24 Apr 2018 14:00:08 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1524603608; cv=none; d=google.com; s=arc-20160816; b=vzrtpWFqeYngogsgesvPEFTYPjiNSG+cevdaNtoiUvINd45LSVYy8mhAk5II3k3ZQg uJ/6t9uKUJqVnxLKkqTXj8ErhkMIOZ48BMSHzoxFqXgqHvM/xntk6BdspVu9PDUjfKnv CX6CrwvwzJTy7QTFytVEx+YvbYTs/QS04XxLGhomBHSjC3kbX+iclPpGAZPjt/RvJO/h DCOBTRa/LC7GmQHxDJnjnDPLB+8mKr6MhZo+9vhSI4kvuR9vaSV0kpGDMb7R4Ni/5g5k HN3CLofT42wzXLDKLd2X21IOjMFg6MBwUNQLWcJqBIuN9cSWYa78bHeLjRFr2rvr+VI3 NrtQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:dkim-signature:delivered-to:arc-authentication-results; bh=8tPAWTSQ5Zxs8bP1K0hAfimWaCtfm1s3Xunw5oEB/1U=; b=vVZhjLilbd2mYfP3dbdSy4jm1upu9WGz6EiJCfzU14vtureYMMpDAKsCoNhcZI/E1M Von1f9c56czQVTkmE4OyBwMm9AXiIDsybCAB6JS838uqctrYmoR+yCgzAt9/BB+2IGt4 6vceJaLc5M8beKsX8W1tKHZFqHWojkjabBBmBvAwB1tb8HNu9e1S3CzWn1lFwr1N/k2T Sp4YuL3DR3TkSYev2t8708qNGJ0UtiNKOZ8cqJ5PX181P+5/1lXr1k4netF5vAf2ckfD hGrroePI/YOCJWhfNp7OZpd9LoukqagC/g14CWSfmK35UpCfPl8ItHOmZfqmVjVU/9zp sK7A== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@tmm1-net.20150623.gappssmtp.com header.s=20150623 header.b=Eg3BQNVm; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id w107-v6si12036255wrc.177.2018.04.24.14.00.08; Tue, 24 Apr 2018 14:00:08 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@tmm1-net.20150623.gappssmtp.com header.s=20150623 header.b=Eg3BQNVm; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 898C5689FC7; Tue, 24 Apr 2018 23:59:12 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pg0-f51.google.com (mail-pg0-f51.google.com [74.125.83.51]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 83B66689EFF for ; Tue, 24 Apr 2018 23:59:05 +0300 (EEST) Received: by mail-pg0-f51.google.com with SMTP id i194so11739940pgd.0 for ; Tue, 24 Apr 2018 13:59:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=tmm1-net.20150623.gappssmtp.com; s=20150623; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references; bh=zl2As60cPO+JBJ4a7OWFIxzC78xOzZurJNTLkH88gFI=; b=Eg3BQNVmwsleep2fw7qvvjBXhWD4emIjF8MGhLs2qEb4PhbFaftb8l22YWrC9iZjT7 7liIE3jbJQfJtzDxlMljaoSu2HQHlTT9Qi2MUq7n3SAP3INWlDUeVKmqEnqU5kSexcjS iNU4boqYUrV22IlB29rJuHE61PudDOHwk8fIaOFuMiThA+4Qw9YvSknd6qq0bXoXszV0 HXKa4g/LyEJ6H0TCz8o/jQ1gfXnzRz/TkKhswGxmCcS+3/PHbZ1RjkEtYeoddKH8Klc1 709lmC7uys6eX1960lHEFGMFNoBh9TpSjztdnRrWOw59CFHj0kyA4t2rDUU9KfwymL0l U2rw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references; bh=zl2As60cPO+JBJ4a7OWFIxzC78xOzZurJNTLkH88gFI=; b=ooJM+o7HyL0kbZKOj3aOWJuIf9OpzW21cRYF39eu/vXuGi6BzfafJbA4qhJA3QdIal Jw9elZdXYynueuRGCc07sWbNHnZ2trjPh5bx/f99/wTS1TW/Wp0m8sdT6Iqa+QZa7FBK 34cOq4XOnoJBYyWo/3A9kvx8bZeELv34Q4baFycVkASPuCyFL4O3fPL8U5og+6IoRpCq oxUMpXR/KgQeopRwyTGIqlvH1du2G8+23rcA0FXWFc3PJW44/GYWo3CgdiM0fq/OpbCM 61xMBWPkTVwjiUVCspYhN7Q2PvVGZXmjtFUoyguQJxaDLS/clQFozgLYH2N5uMoa1ZWX cMkQ== X-Gm-Message-State: ALQs6tAfvV/aAIqsh6merp25a1HIXAFiqomiTsPQC/LlQihFbaMB//J5 a44j3Ot+1ZzL7dTv2vuOo4FHoZY/ X-Received: by 2002:a17:902:8ec4:: with SMTP id x4-v6mr26475720plo.391.1524603573618; Tue, 24 Apr 2018 13:59:33 -0700 (PDT) Received: from localhost.localdomain (c-69-181-54-242.hsd1.ca.comcast.net. [69.181.54.242]) by smtp.gmail.com with ESMTPSA id h6sm26184534pfi.35.2018.04.24.13.59.32 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 24 Apr 2018 13:59:33 -0700 (PDT) From: Aman Gupta To: ffmpeg-devel@ffmpeg.org Date: Tue, 24 Apr 2018 13:59:26 -0700 Message-Id: <20180424205926.42027-4-ffmpeg@tmm1.net> X-Mailer: git-send-email 2.14.2 In-Reply-To: <20180424205926.42027-1-ffmpeg@tmm1.net> References: <20180424205926.42027-1-ffmpeg@tmm1.net> Subject: [FFmpeg-devel] [PATCH 4/4] avcodec/mediacodecdec: restructure mediacodec_receive_frame X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: matthieu.bouron@gmail.com, Aman Gupta , jeebjp@gmail.com MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" From: Aman Gupta This fixes issues reported on some devices where both avcodec_send_packet and avcodec_receive_frame can return EAGAIN at the same time, instead of one or the other blocking. The new logic follows a recommendation by @rcombs to use dequeueInputBuffer with a timeout of 0 as a way to detect whether the codec wants more data. The dequeued buffer index is kept in MediaCodecDecContext until it can be used next. A similar technique is also used by the Google's official media player Exoplayer: see MediaCodecRenderer.feedInputBuffer(). --- libavcodec/mediacodecdec.c | 78 ++++++++++++++++++++------------------- libavcodec/mediacodecdec_common.c | 28 ++++++++------ libavcodec/mediacodecdec_common.h | 4 +- 3 files changed, 59 insertions(+), 51 deletions(-) diff --git a/libavcodec/mediacodecdec.c b/libavcodec/mediacodecdec.c index e5d3e6a0af..1a681d771e 100644 --- a/libavcodec/mediacodecdec.c +++ b/libavcodec/mediacodecdec.c @@ -391,33 +391,11 @@ done: return ret; } -static int mediacodec_send_receive(AVCodecContext *avctx, - MediaCodecH264DecContext *s, - AVFrame *frame, bool wait) -{ - int ret; - - /* send any pending data from buffered packet */ - while (s->buffered_pkt.size) { - ret = ff_mediacodec_dec_send(avctx, s->ctx, &s->buffered_pkt); - if (ret == AVERROR(EAGAIN)) - break; - else if (ret < 0) - return ret; - s->buffered_pkt.size -= ret; - s->buffered_pkt.data += ret; - if (s->buffered_pkt.size <= 0) - av_packet_unref(&s->buffered_pkt); - } - - /* check for new frame */ - return ff_mediacodec_dec_receive(avctx, s->ctx, frame, wait); -} - static int mediacodec_receive_frame(AVCodecContext *avctx, AVFrame *frame) { MediaCodecH264DecContext *s = avctx->priv_data; int ret; + ssize_t index; /* in delay_flush mode, wait until the user has released or rendered all retained frames. */ @@ -427,28 +405,52 @@ static int mediacodec_receive_frame(AVCodecContext *avctx, AVFrame *frame) } } - /* flush buffered packet and check for new frame */ - ret = mediacodec_send_receive(avctx, s, frame, false); + /* poll for new frame */ + ret = ff_mediacodec_dec_receive(avctx, s->ctx, frame, false); if (ret != AVERROR(EAGAIN)) return ret; - /* skip fetching new packet if we still have one buffered */ - if (s->buffered_pkt.size > 0) - return mediacodec_send_receive(avctx, s, frame, true); + /* feed decoder */ + while (1) { + if (s->ctx->current_input_buffer < 0) { + /* poll for input space */ + index = ff_AMediaCodec_dequeueInputBuffer(s->ctx->codec, 0); + if (index < 0) { + /* no space, wait a while for an output frame to appear */ + return ff_mediacodec_dec_receive(avctx, s->ctx, frame, true); + } + s->ctx->current_input_buffer = index; + } - /* fetch new packet or eof */ - ret = ff_decode_get_packet(avctx, &s->buffered_pkt); - if (ret == AVERROR_EOF) { - AVPacket null_pkt = { 0 }; - ret = ff_mediacodec_dec_send(avctx, s->ctx, &null_pkt); - if (ret < 0) + /* try to flush any buffered packet data */ + if (s->buffered_pkt.size > 0) { + ret = ff_mediacodec_dec_send(avctx, s->ctx, &s->buffered_pkt, false); + if (ret > 0) { + s->buffered_pkt.size -= ret; + s->buffered_pkt.data += ret; + if (s->buffered_pkt.size <= 0) + av_packet_unref(&s->buffered_pkt); + } else if (ret < 0 && ret == AVERROR(EAGAIN)) { + return ret; + } + continue; + } + + /* fetch new packet or eof */ + ret = ff_decode_get_packet(avctx, &s->buffered_pkt); + if (ret == AVERROR_EOF) { + AVPacket null_pkt = { 0 }; + ret = ff_mediacodec_dec_send(avctx, s->ctx, &null_pkt, true); + if (ret < 0) + return ret; + } + else if (ret == AVERROR(EAGAIN) && s->ctx->current_input_buffer < 0) + return ff_mediacodec_dec_receive(avctx, s->ctx, frame, true); + else if (ret < 0) return ret; } - else if (ret < 0) - return ret; - /* crank decoder with new packet */ - return mediacodec_send_receive(avctx, s, frame, true); + return AVERROR(EAGAIN); } static void mediacodec_decode_flush(AVCodecContext *avctx) diff --git a/libavcodec/mediacodecdec_common.c b/libavcodec/mediacodecdec_common.c index c0f0a6b983..887865a281 100644 --- a/libavcodec/mediacodecdec_common.c +++ b/libavcodec/mediacodecdec_common.c @@ -450,6 +450,7 @@ static int mediacodec_dec_flush_codec(AVCodecContext *avctx, MediaCodecDecContex s->eos = 0; atomic_fetch_add(&s->serial, 1); atomic_init(&s->hw_buffer_count, 0); + s->current_input_buffer = -1; status = ff_AMediaCodec_flush(codec); if (status < 0) { @@ -477,6 +478,7 @@ int ff_mediacodec_dec_init(AVCodecContext *avctx, MediaCodecDecContext *s, atomic_init(&s->refcount, 1); atomic_init(&s->hw_buffer_count, 0); atomic_init(&s->serial, 1); + s->current_input_buffer = -1; pix_fmt = ff_get_format(avctx, pix_fmts); if (pix_fmt == AV_PIX_FMT_MEDIACODEC) { @@ -561,16 +563,16 @@ fail: } int ff_mediacodec_dec_send(AVCodecContext *avctx, MediaCodecDecContext *s, - AVPacket *pkt) + AVPacket *pkt, bool wait) { int offset = 0; int need_draining = 0; uint8_t *data; - ssize_t index; + ssize_t index = s->current_input_buffer; size_t size; FFAMediaCodec *codec = s->codec; int status; - int64_t input_dequeue_timeout_us = INPUT_DEQUEUE_TIMEOUT_US; + int64_t input_dequeue_timeout_us = wait ? INPUT_DEQUEUE_TIMEOUT_US : 0; int64_t pts; if (s->flushing) { @@ -588,17 +590,19 @@ int ff_mediacodec_dec_send(AVCodecContext *avctx, MediaCodecDecContext *s, } while (offset < pkt->size || (need_draining && !s->draining)) { - - index = ff_AMediaCodec_dequeueInputBuffer(codec, input_dequeue_timeout_us); - if (ff_AMediaCodec_infoTryAgainLater(codec, index)) { - av_log(avctx, AV_LOG_TRACE, "No input buffer available, try again later\n"); - break; - } - if (index < 0) { - av_log(avctx, AV_LOG_ERROR, "Failed to dequeue input buffer (status=%zd)\n", index); - return AVERROR_EXTERNAL; + index = ff_AMediaCodec_dequeueInputBuffer(codec, input_dequeue_timeout_us); + if (ff_AMediaCodec_infoTryAgainLater(codec, index)) { + av_log(avctx, AV_LOG_TRACE, "No input buffer available, try again later\n"); + break; + } + + if (index < 0) { + av_log(avctx, AV_LOG_ERROR, "Failed to dequeue input buffer (status=%zd)\n", index); + return AVERROR_EXTERNAL; + } } + s->current_input_buffer = -1; data = ff_AMediaCodec_getInputBuffer(codec, index, &size); if (!data) { diff --git a/libavcodec/mediacodecdec_common.h b/libavcodec/mediacodecdec_common.h index 023d4c5fa7..0b21129fee 100644 --- a/libavcodec/mediacodecdec_common.h +++ b/libavcodec/mediacodecdec_common.h @@ -65,6 +65,7 @@ typedef struct MediaCodecDecContext { int display_height; uint64_t output_buffer_count; + ssize_t current_input_buffer; bool delay_flush; atomic_int serial; @@ -78,7 +79,8 @@ int ff_mediacodec_dec_init(AVCodecContext *avctx, int ff_mediacodec_dec_send(AVCodecContext *avctx, MediaCodecDecContext *s, - AVPacket *pkt); + AVPacket *pkt, + bool wait); int ff_mediacodec_dec_receive(AVCodecContext *avctx, MediaCodecDecContext *s,