From patchwork Tue Mar 6 21:15:55 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aman Karmani X-Patchwork-Id: 7827 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.2.181.170 with SMTP id m39csp4226441jaj; Tue, 6 Mar 2018 13:16:18 -0800 (PST) X-Google-Smtp-Source: AG47ELu4JZ0yAYXrmbcOImKVJvIAqdt/bkoIemnolB8ARaSkCqhtBH4iySFkolKGoGZV8bj2bkAk X-Received: by 10.223.198.17 with SMTP id n17mr9020915wrg.50.1520370978546; Tue, 06 Mar 2018 13:16:18 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1520370978; cv=none; d=google.com; s=arc-20160816; b=Pf6wDfe0i5KmmvYmVebPxALv6jDLeUMGK3yaDkIveY4nST5MAHN/gpOJqv47wPkmOi DmPX8Hyb7KT5D1nOpcy7/UlilTq31GebWH13fBJzaLcTzwrFAsY+oR4rMhiQUS5wcrjp j6zLlGjNkGlb9l4CCUdvjZHlXp4eBMIVVqtOfhowlW5Hw41xXCSJV5sF7NhXMqbClRV2 uXm5WXOZ/X67XR0o59vf29kL8swoYVWSWxU1sYPhS6kCbI/ypj1qKKEYd5oCYr7Vi+oX YPg4RvEWCvZpc9TEwfonRWHqztDzh4xuJH50CVYLZTb4MoNNGgqiJQqbH5tUx3YgNNLB kH1A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:message-id:date:to:from:dkim-signature :delivered-to:arc-authentication-results; bh=j0iIF/tp62vq4xhnEGYWWs6CVr+Eklpq1e4dZdw7eRM=; b=FRelwl82DkIfAJ1oc+1xrSA0BDI1vqwBg9sAgtPnViZcGv0LXQ3ZY1K/4QT7BBZ0Gk Q3Q6AjEPXJ+VVB5g0I4ySeZNBQwlqxNULlEm3iRAdedh5Jb+aiiKJzeQyZVq63Tk5+x0 rxkVq+W90F4+93b7cKsyiVSEeM0QjWLkqo5eCgMTcNwpDVFnotpqvQYVe7xDjgd7E/JS rBp99iA6XJWuTuJjYNEcP+BC0o0bpKZ6lIT0tmOdQ2QwQMr6hLZb5JdNxAmepDULCvn+ k59oZmyc7NRPEPt8Wh5FLTBXBaULXD2JVm4KMQI9PjfnRHbkmEWNDzOeX2eQlZ62DJv6 hTGw== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@tmm1-net.20150623.gappssmtp.com header.s=20150623 header.b=1pR5HnOS; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id u62si7099486wma.100.2018.03.06.13.16.16; Tue, 06 Mar 2018 13:16:18 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@tmm1-net.20150623.gappssmtp.com header.s=20150623 header.b=1pR5HnOS; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 07B5668A31D; Tue, 6 Mar 2018 23:16:07 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pl0-f47.google.com (mail-pl0-f47.google.com [209.85.160.47]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id C1100689C5E for ; Tue, 6 Mar 2018 23:15:59 +0200 (EET) Received: by mail-pl0-f47.google.com with SMTP id c11-v6so42787plo.0 for ; Tue, 06 Mar 2018 13:16:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=tmm1-net.20150623.gappssmtp.com; s=20150623; h=sender:from:to:cc:subject:date:message-id; bh=dVmJeB/+woAL6hjpGlu4ir6iAOhI5YWjn6kJx1KZt4w=; b=1pR5HnOSeS8hEazSsNnmEVsk9xYRaqssq0i9ph82RuQXN3LhlWFbOT50gQgpwtWdit Y8I8uN4wueY8vVLcxT03M1Tc2W5lUXEdaVlG4/P/gXvo9r9tGWdkB6yt6NLy8FnMMwF6 JbzGNHVu07YfZTHszM6YyjAaHU/4l6YOMs1qCLJgmldYXvEgOFU2vo9V7CFJR5t2g5GH fCA6uIuHEwfOgH0jEgAZ0s+tQc8HoGErLrcNAvyOyn+dQPbN1tYmbznqyN15fP71bFEA NpZ/5ZVtjSsRIzzLnb0a8RrJvT0pli2EQ4NzHwPjJyGmgmDslS8yGVErQEHGWHTo2Dvo F+OQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id; bh=dVmJeB/+woAL6hjpGlu4ir6iAOhI5YWjn6kJx1KZt4w=; b=AwsH+C7XftqjSYOxvqCgJjxPoy07yYJvnS6MaD5otUjYfgRX/bn1Qac2QnWVoEdgKG rS6cSnpNu/5TFqzIhYSS8MRcuZ34zhjaBcsRIwC1HYh9TGMBm5NPrwVC3OI5Os6l0MpI RXwjl4Ih8glyeXrBnkZWmoA4/GnENN/4gKDswh/wtXvjb7oatEr9JC5olFjaU8bCczVT vMzbaxy4FYgPw8iI4hwWpX5bgEPrOvVK9Qf0IOvfXEAoNuV7PD2rzxNwsXwhBy0pTJhy yQXOKHRA3AvbXJuc/6e6pe57vtSs74y851u48VKWSMUZfKfV7oE/pvzpRK6aRvkQwyem WCvg== X-Gm-Message-State: AElRT7HsxdYUR0MX+UL9775RjS/cm5cuTwHWsGV/bXFTcDMFgi21DVMj HU6OvBQmu/6V/Db9su09/tUkS+pQ X-Received: by 2002:a17:902:7e44:: with SMTP id a4-v6mr10000700pln.392.1520370965619; Tue, 06 Mar 2018 13:16:05 -0800 (PST) Received: from localhost.localdomain (c-69-181-54-242.hsd1.ca.comcast.net. [69.181.54.242]) by smtp.gmail.com with ESMTPSA id q67sm35242108pfg.180.2018.03.06.13.16.04 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 06 Mar 2018 13:16:04 -0800 (PST) From: Aman Gupta To: ffmpeg-devel@ffmpeg.org Date: Tue, 6 Mar 2018 13:15:55 -0800 Message-Id: <20180306211556.3083-1-ffmpeg@tmm1.net> X-Mailer: git-send-email 2.14.2 Subject: [FFmpeg-devel] [PATCH 1/2] avcodec/mediacodecdec: add delay_flush option X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: matthieu.bouron@gmail.com, Aman Gupta MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" From: Aman Gupta The default behavior of the mediacodec decoder before this commit was to delay flushes until all pending hardware frames were returned to the decoder. This was useful for certain types of applications, but was unexpected behavior for others. The new default behavior with this commit is now to execute flushes immediately to invalidate all pending frames. The old behavior can be enabled by setting delay_flush=1. With the new behavior, video players implementing seek can simply call flush on the decoder without having to worry about whether they have one or more mediacodec frames still buffered in their rendering pipeline. Previously, all these frames had to be explictly freed (or rendered) before the seek/flush would execute. The new behavior matches the behavior of all other lavc decoders, reducing the amount of special casing required when using the mediacodec decoder. --- libavcodec/mediacodec.c | 2 +- libavcodec/mediacodecdec.c | 23 +++++++++++++++++++++++ libavcodec/mediacodecdec_common.c | 11 ++++++++--- libavcodec/mediacodecdec_common.h | 4 ++++ 4 files changed, 36 insertions(+), 4 deletions(-) diff --git a/libavcodec/mediacodec.c b/libavcodec/mediacodec.c index d492eefe0b..bf1b7477f1 100644 --- a/libavcodec/mediacodec.c +++ b/libavcodec/mediacodec.c @@ -91,7 +91,7 @@ int av_mediacodec_release_buffer(AVMediaCodecBuffer *buffer, int render) MediaCodecDecContext *ctx = buffer->ctx; int released = atomic_fetch_add(&buffer->released, 1); - if (!released) { + if (!released && (ctx->delay_flush || buffer->serial == atomic_load(&ctx->serial))) { return ff_AMediaCodec_releaseOutputBuffer(ctx->codec, buffer->index, render); } diff --git a/libavcodec/mediacodecdec.c b/libavcodec/mediacodecdec.c index 0fe14846c3..3582a1377d 100644 --- a/libavcodec/mediacodecdec.c +++ b/libavcodec/mediacodecdec.c @@ -41,10 +41,14 @@ typedef struct MediaCodecH264DecContext { + AVClass *avclass; + MediaCodecDecContext *ctx; AVPacket buffered_pkt; + int delay_flush; + } MediaCodecH264DecContext; static av_cold int mediacodec_decode_close(AVCodecContext *avctx) @@ -366,6 +370,8 @@ static av_cold int mediacodec_decode_init(AVCodecContext *avctx) goto done; } + s->ctx->delay_flush = s->delay_flush; + if ((ret = ff_mediacodec_dec_init(avctx, s->ctx, codec_mime, format)) < 0) { s->ctx = NULL; goto done; @@ -485,7 +491,24 @@ static const AVCodecHWConfigInternal *mediacodec_hw_configs[] = { NULL }; +#define OFFSET(x) offsetof(MediaCodecH264DecContext, x) +#define VD AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_DECODING_PARAM +static const AVOption ff_mediacodec_vdec_options[] = { + { "delay_flush", "Delay flush until hw output buffers are returned to the decoder", + OFFSET(delay_flush), AV_OPT_TYPE_BOOL, {.i64 = 0}, 0, 1, VD }, + { NULL } +}; + +#define DECLARE_MEDIACODEC_VCLASS(short_name) \ +static const AVClass ff_##short_name##_mediacodec_dec_class = { \ + .class_name = #short_name "_mediacodec", \ + .item_name = av_default_item_name, \ + .option = ff_mediacodec_vdec_options, \ + .version = LIBAVUTIL_VERSION_INT, \ +}; + #define DECLARE_MEDIACODEC_VDEC(short_name, full_name, codec_id, bsf) \ +DECLARE_MEDIACODEC_VCLASS(short_name) \ AVCodec ff_##short_name##_mediacodec_decoder = { \ .name = #short_name "_mediacodec", \ .long_name = NULL_IF_CONFIG_SMALL(full_name " Android MediaCodec decoder"), \ diff --git a/libavcodec/mediacodecdec_common.c b/libavcodec/mediacodecdec_common.c index 929db78361..b4f1f6685b 100644 --- a/libavcodec/mediacodecdec_common.c +++ b/libavcodec/mediacodecdec_common.c @@ -178,11 +178,12 @@ static void mediacodec_buffer_release(void *opaque, uint8_t *data) MediaCodecDecContext *ctx = buffer->ctx; int released = atomic_load(&buffer->released); - if (!released) { + if (!released && (ctx->delay_flush || buffer->serial == atomic_load(&ctx->serial))) { ff_AMediaCodec_releaseOutputBuffer(ctx->codec, buffer->index, 0); } - ff_mediacodec_dec_unref(ctx); + if (ctx->delay_flush) + ff_mediacodec_dec_unref(ctx); av_freep(&buffer); } @@ -236,7 +237,9 @@ FF_ENABLE_DEPRECATION_WARNINGS } buffer->ctx = s; - ff_mediacodec_dec_ref(s); + buffer->serial = atomic_load(&s->serial); + if (s->delay_flush) + ff_mediacodec_dec_ref(s); buffer->index = index; buffer->pts = info->presentationTimeUs; @@ -425,6 +428,7 @@ static int mediacodec_dec_flush_codec(AVCodecContext *avctx, MediaCodecDecContex s->draining = 0; s->flushing = 0; s->eos = 0; + atomic_fetch_add(&s->serial, 1); status = ff_AMediaCodec_flush(codec); if (status < 0) { @@ -449,6 +453,7 @@ int ff_mediacodec_dec_init(AVCodecContext *avctx, MediaCodecDecContext *s, }; atomic_init(&s->refcount, 1); + atomic_init(&s->serial, 1); pix_fmt = ff_get_format(avctx, pix_fmts); if (pix_fmt == AV_PIX_FMT_MEDIACODEC) { diff --git a/libavcodec/mediacodecdec_common.h b/libavcodec/mediacodecdec_common.h index 85df507ffb..afb98ffed9 100644 --- a/libavcodec/mediacodecdec_common.h +++ b/libavcodec/mediacodecdec_common.h @@ -62,6 +62,9 @@ typedef struct MediaCodecDecContext { uint64_t output_buffer_count; + bool delay_flush; + atomic_int serial; + } MediaCodecDecContext; int ff_mediacodec_dec_init(AVCodecContext *avctx, @@ -93,6 +96,7 @@ typedef struct MediaCodecBuffer { ssize_t index; int64_t pts; atomic_int released; + int serial; } MediaCodecBuffer;