From patchwork Wed Aug 1 01:34:15 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aman Karmani X-Patchwork-Id: 9855 Delivered-To: ffmpegpatchwork@gmail.com Received: by 2002:a02:104:0:0:0:0:0 with SMTP id c4-v6csp266884jad; Tue, 31 Jul 2018 18:42:08 -0700 (PDT) X-Google-Smtp-Source: AAOMgpdVIvIVCNc/aRViWoGFsdmiLWTB0be4QRvTI/jKs7COlqkw1IS2txLvSdHu9HRyilPU4t6/ X-Received: by 2002:a1c:8b0d:: with SMTP id n13-v6mr1151822wmd.46.1533087728254; Tue, 31 Jul 2018 18:42:08 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1533087728; cv=none; d=google.com; s=arc-20160816; b=om1bbPy/WL+vK3qY34viByvq+xGhcOywCemY3YHMGG+gHqS0v9pI7oVTiKANH8xU7X +7DzWHlT5Q3vVa5k4WuOP1mgeOSQrB/PziBphTAUV8M0MOCs4bH5chnqkaEbOWQoF4ug rBpfBfp0PeulPvK7T/2Xl4bKQb76jFa+/YTq6vdmiUrHjChBtkxv9AkjjXbrmgcUc6MI QoXE4J8S7FWGZ7r3lgho7tCEXAjVjYEa93SZRGpccxJY9Wws+wrzQtnn0TbKgJr0RAcJ hNkmvv0x2scBqUe1AxyXygItJAMf8CSU0C/xoJnUc5U0JtFjuOcsTPf3TXFa/qPzf7Ie BeRA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:message-id:date:to:from:dkim-signature :delivered-to:arc-authentication-results; bh=FO5Hh9BUDVvtkZyzGTz6WEkfbgzRmEcNho1Ptu1xv9Y=; b=Qao5BnFxtXUkLrBxFr6Gws3QwBtvURsftIu50Q7vXa3BrgqTCzRHH2fvlIBLjUM/Cr Um/0mSATbfkRvsBwXPCW8xChN50H5OX3kjW1ERBV34KB09aK6M+yTE5O4qyQmy8DCi3a H4Ko/J/vOv1LdUxIgiIIpbCuuCBpT0h11zfhrTdM5XZq8qCbeLBo9V9eadVuKn39vOgY mgN8AZ6YpOrRPL3pZFAeA9UzEVD5a/J0KR1vfZ2aMZZjVC2bo3I6SphyJtJBxbfVWjcm Ifq4ramcwsZOTaT+hWmq6q1gG/DWnfX36PdExS/5BeoZ+vWH2lyYnImgX6nojLX5N/GM RIrw== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@tmm1-net.20150623.gappssmtp.com header.s=20150623 header.b=VXI7X6x6; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id p29-v6si2803460wmi.157.2018.07.31.18.42.07; Tue, 31 Jul 2018 18:42:08 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@tmm1-net.20150623.gappssmtp.com header.s=20150623 header.b=VXI7X6x6; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id F2D9568A4CB; Wed, 1 Aug 2018 04:41:47 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pl0-f65.google.com (mail-pl0-f65.google.com [209.85.160.65]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 9524668A254 for ; Wed, 1 Aug 2018 04:41:41 +0300 (EEST) Received: by mail-pl0-f65.google.com with SMTP id t17-v6so7966195ply.13 for ; Tue, 31 Jul 2018 18:41:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=tmm1-net.20150623.gappssmtp.com; s=20150623; h=sender:from:to:cc:subject:date:message-id; bh=7rn7+cgIgcF6zZLcnwnk2fEnLEp7JaV7hUEGBV3hHLs=; b=VXI7X6x6NB7woqNC0QZuBye2fFCshiaZ3UkPgCprYXz0EDdmTcr+uLYNlk4QX4tP/q qiZc/u2hStqHdXh3JNt8N5tOuOeenIQvv0MVShh+EH08eJs7BwpWwcQNIY5FJyjG/5Wn wGpc2gm3R503sAkOEff4ZHmHkLRju4mp/Kne0xQel1yVyCBx0cIzoybhoIfsGDX/3ZQG ZJYejabW6t0gnZR6fJ89u1yUO3t4ujuZfNbfKsyM2BNQEZSJWkUnOTzjdeyPhNlibBWF jpAfJKPrmg7mSLu9nnQd2Y8jX1htVkIwwFSdFcqrvXdjOmZT21KzV3QvU/k4d2YkA0ET 7x8A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id; bh=7rn7+cgIgcF6zZLcnwnk2fEnLEp7JaV7hUEGBV3hHLs=; b=ChhJ+HCkot1Dj73+0KxTNVzx5k0MBjWA4Hon3TqvxOkiXpGDlQpWspY7sApvC02cOs SY+uu3nb9VW6e+WpfFFDq4wqaJvYr9xGY9Z4MF7pUbYwK0ceeUAYCAQ9DZhRlvlTD7Sf z4f6OMYfX2FFiQolkUPfLyWkvqgmoR3aqcw6nZwhShBfyzkezwFOFR/OmLw3GohSK6g7 MloiqqkwUeWVusrFS9e/N9+HtNLumfyucpJWrp6t2o3c8ltxabTa1f0dq/M1bBUfw1/r zyqvyh6K3fmsJvGLypEClYrIA76exXd03gsS6z/3DQCCttziIr3m6uAiTLYfptvc4P2D xwzw== X-Gm-Message-State: AOUpUlGfbh48k0EbMrrTYpe+Dv7oXty5OnELyAyw944xs8US9y3xaLnO 80HfTXC1NOG5jG+MVPcG22tUzDttgtA= X-Received: by 2002:a17:902:e088:: with SMTP id cb8-v6mr22785127plb.189.1533087264310; Tue, 31 Jul 2018 18:34:24 -0700 (PDT) Received: from localhost.localdomain (c-69-181-54-242.hsd1.ca.comcast.net. [69.181.54.242]) by smtp.gmail.com with ESMTPSA id n9-v6sm26769504pfg.21.2018.07.31.18.34.22 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 31 Jul 2018 18:34:23 -0700 (PDT) From: Aman Gupta To: ffmpeg-devel@ffmpeg.org Date: Tue, 31 Jul 2018 18:34:15 -0700 Message-Id: <20180801013415.11040-1-ffmpeg@tmm1.net> X-Mailer: git-send-email 2.18.0 Subject: [FFmpeg-devel] [PATCH] avcodec/mediacodec: add av_mediacodec_render_buffer_at_time() X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: matthieu.bouron@gmail.com, Aman Gupta MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" From: Aman Gupta The existing av_mediacodec_release_buffer allows the user to render or discard the Surface-backed frame. This new method allows the user to control exactly when the frame will be rendered to it's SurfaceView. Available since Android API 21. Signed-off-by: Aman Gupta --- doc/APIchanges | 3 +++ libavcodec/mediacodec.c | 21 +++++++++++++++++++++ libavcodec/mediacodec.h | 13 +++++++++++++ libavcodec/mediacodec_wrapper.c | 2 +- libavcodec/version.h | 2 +- 5 files changed, 39 insertions(+), 2 deletions(-) diff --git a/doc/APIchanges b/doc/APIchanges index efe15ba4e0..62394fd837 100644 --- a/doc/APIchanges +++ b/doc/APIchanges @@ -15,6 +15,9 @@ libavutil: 2017-10-21 API changes, most recent first: +2018-08-01 - xxxxxxxxxx - lavc 58.23.100 - mediacodec.h + Add av_mediacodec_render_buffer_at_time(). + 2018-05-xx - xxxxxxxxxx - lavf 58.15.100 - avformat.h Add pmt_version field to AVProgram diff --git a/libavcodec/mediacodec.c b/libavcodec/mediacodec.c index b0aae43a87..aa14624fd0 100644 --- a/libavcodec/mediacodec.c +++ b/libavcodec/mediacodec.c @@ -102,6 +102,22 @@ int av_mediacodec_release_buffer(AVMediaCodecBuffer *buffer, int render) return 0; } +int av_mediacodec_render_buffer_at_time(AVMediaCodecBuffer *buffer, int64_t time) +{ + MediaCodecDecContext *ctx = buffer->ctx; + int released = atomic_fetch_add(&buffer->released, 1); + + if (!released && (ctx->delay_flush || buffer->serial == atomic_load(&ctx->serial))) { + atomic_fetch_sub(&ctx->hw_buffer_count, 1); + av_log(ctx->avctx, AV_LOG_DEBUG, + "Rendering output buffer %zd (%p) ts=%"PRId64" with time=%"PRId64" [%d pending]\n", + buffer->index, buffer, buffer->pts, time, atomic_load(&ctx->hw_buffer_count)); + return ff_AMediaCodec_releaseOutputBufferAtTime(ctx->codec, buffer->index, time); + } + + return 0; +} + #else #include @@ -125,4 +141,9 @@ int av_mediacodec_release_buffer(AVMediaCodecBuffer *buffer, int render) return AVERROR(ENOSYS); } +int av_mediacodec_render_buffer_at_time(AVMediaCodecBuffer *buffer, int64_t time) +{ + return AVERROR(ENOSYS); +} + #endif diff --git a/libavcodec/mediacodec.h b/libavcodec/mediacodec.h index 5606d24a1e..4c8545df03 100644 --- a/libavcodec/mediacodec.h +++ b/libavcodec/mediacodec.h @@ -85,4 +85,17 @@ typedef struct MediaCodecBuffer AVMediaCodecBuffer; */ int av_mediacodec_release_buffer(AVMediaCodecBuffer *buffer, int render); +/** + * Release a MediaCodec buffer and render it at the given time to the surface + * that is associated with the decoder. The timestamp must be within one second + * of the current java/lang/System#nanoTime() (which is implemented using + * CLOCK_MONOTONIC on Android). See the Android MediaCodec documentation + * of android/media/MediaCodec#releaseOutputBuffer(int,long) for more details. + * + * @param buffer the buffer to render + * @param time timestamp in nanoseconds of when to render the buffer + * @return 0 on success, < 0 otherwise + */ +int av_mediacodec_render_buffer_at_time(AVMediaCodecBuffer *buffer, int64_t time); + #endif /* AVCODEC_MEDIACODEC_H */ diff --git a/libavcodec/mediacodec_wrapper.c b/libavcodec/mediacodec_wrapper.c index c47c2c9a41..a024e3bdb1 100644 --- a/libavcodec/mediacodec_wrapper.c +++ b/libavcodec/mediacodec_wrapper.c @@ -1432,7 +1432,7 @@ int ff_AMediaCodec_releaseOutputBufferAtTime(FFAMediaCodec *codec, size_t idx, i JNI_GET_ENV_OR_RETURN(env, codec, AVERROR_EXTERNAL); - (*env)->CallVoidMethod(env, codec->object, codec->jfields.release_output_buffer_at_time_id, (jint)idx, timestampNs); + (*env)->CallVoidMethod(env, codec->object, codec->jfields.release_output_buffer_at_time_id, (jint)idx, (jlong)timestampNs); if (ff_jni_exception_check(env, 1, codec) < 0) { ret = AVERROR_EXTERNAL; goto fail; diff --git a/libavcodec/version.h b/libavcodec/version.h index 3f0d98efdf..a91e5f01e6 100644 --- a/libavcodec/version.h +++ b/libavcodec/version.h @@ -28,7 +28,7 @@ #include "libavutil/version.h" #define LIBAVCODEC_VERSION_MAJOR 58 -#define LIBAVCODEC_VERSION_MINOR 22 +#define LIBAVCODEC_VERSION_MINOR 23 #define LIBAVCODEC_VERSION_MICRO 100 #define LIBAVCODEC_VERSION_INT AV_VERSION_INT(LIBAVCODEC_VERSION_MAJOR, \