From patchwork Sun Nov 20 06:49:41 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhao Zhili X-Patchwork-Id: 39353 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a21:999a:b0:a4:2148:650a with SMTP id ve26csp1406783pzb; Sat, 19 Nov 2022 22:51:49 -0800 (PST) X-Google-Smtp-Source: AA0mqf4PtPldCodGKK4Bjg90EfZ/rAQxvORe8yu9VQ8YSRIl5PlYDR+yWPs4hsCcrYIu5JMtlZB7 X-Received: by 2002:a05:6402:4445:b0:468:eef6:a83c with SMTP id o5-20020a056402444500b00468eef6a83cmr11108843edb.191.1668927108996; Sat, 19 Nov 2022 22:51:48 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1668927108; cv=none; d=google.com; s=arc-20160816; b=OLRvAExjr/3FO9sJkIJS1Cltucz84CVXzk9XgWM5oGdUDzdkjgT/YzR1bczfHWiZD4 rRABCAE8klgDAPobt+6QtusN2313XHrKkxamu6Xt+cMbx1kAsHt7kvZXkkXZT+NUvMVc q/7QdquEl4tnN2NdvSL681aRNtwiaFvSzdwg1QPlekkOqbL3xrKpQiwUsZ3FnMXTZHXw qj1J6Yn93YOLoB5vF4E9/0E5ANaVZvViApur2Ez9FspilnvrwEOM3MUExcxW7+WepIwK 9bbMc7Kw5HWoSn1DgbIPL09gCDUeqwJA2IyLvwV3lo/9ZDmZ2hP2MCmJus6icYw1lyer OnnA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to:date :to:from:message-id:dkim-signature:delivered-to; bh=Jws0LnOcMyfm5OulcxX26/WAlmbpo1Q2MarbcOeIVeQ=; b=tdyL/lscKDA9AgkTqP9IqlZ7oOHQl4CMGEu64GSK66jDOjR94iVyCMbLIAViEw/0iU MU32wbjPJypXDMsvscoHrBEVXCF0MepFGCp160SUinaohqa1uJbPW9xJ7VsR+k5PsjJt ZaupWBNU5d1IQXUQVAH0M7Otpiy3eO2U2mdEa6vuhNqFB93kBJpEE+bFXROmrtzh1cnx yhxn94btkugOS1foU4L5ev2JA1Gg2G88JRNC2zFvneSiMfkeCE6prWI9OLcMq3BJ9qFu caqk9jE9JvEQs1ULmGcFREjY5bnb9MVTighTJh0KHtfo3p5cy1g2I84IarjN8+7hmsDD t/DQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@foxmail.com header.s=s201512 header.b=fpZ2HJk0; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=foxmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id cw4-20020a170906478400b007919c624eadsi7167627ejc.522.2022.11.19.22.51.48; Sat, 19 Nov 2022 22:51:48 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@foxmail.com header.s=s201512 header.b=fpZ2HJk0; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=foxmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id B531668BC19; Sun, 20 Nov 2022 08:50:52 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from out162-62-58-216.mail.qq.com (out162-62-58-216.mail.qq.com [162.62.58.216]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id A281F68BB78 for ; Sun, 20 Nov 2022 08:50:39 +0200 (EET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=foxmail.com; s=s201512; t=1668927037; bh=hEe0oeGZyPb5bShEwMZCChMpUyi8mUElNvypz4HHgtQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=fpZ2HJk0XeLeZeuK7VGHXptMomVMFG4YiRTKkcnJ/gmmGAXP/aXDRO8HrF9dNEsb7 O+xrnZn7ytsib8ZQy6bC/zNbiuyfidub3vf4mP1qCIykd5kJGbueipmVHIueSj22Cl XliZDVGyesN8RH2klZfEOGLdm/EPwJI+XCcGtQOo= Received: from localhost.localdomain ([14.153.77.200]) by newxmesmtplogicsvrszc2-1.qq.com (NewEsmtp) with SMTP id C8F0626E; Sun, 20 Nov 2022 14:50:15 +0800 X-QQ-mid: xmsmtpt1668927036t3ghjpr8q Message-ID: X-QQ-XMAILINFO: MNQ2FIpgVwZbXGHTJssymm8F/pofG6eUWVxDP5NBwMEbGYfUGT+eD+AobvTxZh 0yTPp8hLCW73iA1sjfyII0Lxc7SvCxP3VVm2vL79qgApM8uFBM8ufOGmJBfWkYmBFAl8Dz31fgsH X9wo0IWuzr/vr69QTdEHPDK+YsMSyB2LHNHFdmMUlfVyKzazSK0YVglxTr4KB82Xv1ZwFiUlA38D u2jZLfsGhzyOeCfhPbejtLRYz1Pz6eidDK0c8GvT4OZ2c0HfbLc35R7CsfW/gvc+Lhj+5MhjD01v 7vZV9lJ1REUo71Xp5cakS+6XeBbSbCFlMX9G796N+luf6kUf+95whZrqet0KFSyvt5WkoQ5ARIcL BzhOYE7rNjRYW0RKsqb8VcKQpSRafi/tMh8rlgWP41i7IYlodHuZ3IzZ5hPYvmdgfmGHPFpjTjTB EIcMtPmOfFmytg59yr6J6rc0DQQuUOErTOMd5HZQGjquqEavXggcIKUCWjGnhNQ8BBbEyljJc4fp 4QgDJ1l+vQ4PoTW/WWwfA+IuIAcfoY4m4q3V45wWA7ZQzyvcVN2QtmnyW1+3ptyXrDfovLW7OwHy 1uo7SC4rR+CpQYvaJDT82s7IKZ+iZkwfMyHjrc0uDuJ/8c3VCqJOsPGxI0rLUZ8hc7sULyhHbj1d Te8CoT63wltDWbVhiv/6VjNfm3NSgQYBblcB9RnmM74N2wFrfSwGCLtGi5lGxMpU8l5ttoNN68pl zjO20RB1zK9REtrAh14PVVs6+77x78TJHpU9fYjhY1EGB7IloKs78lTd4jvfarzoV7EDzOYWoZk2 shLQVMZgRb/qmGJjGhf9yyS7NayzKBknPELkP6m2Feg3Ty1kpPM0pEMAPIOehB8iGB8hqops+t03 R47dkUuz7AMmxQPfX9ktqGMHe2WrcEDpEkh5VKW3ly6Di4v5eTeJocn7qZ0fF0PIlD8UYtiqX+X4 kqkFE7hdG5o2AJBRFVNc/lrP/r/F3lgsHMgY6h6fYMoVWUK/EjcFxhGCJmaxCkMm2YY+0iJgk= From: Zhao Zhili To: ffmpeg-devel@ffmpeg.org Date: Sun, 20 Nov 2022 14:49:41 +0800 X-OQ-MSGID: <20221120064947.94540-7-quinkblack@foxmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221120064947.94540-1-quinkblack@foxmail.com> References: <20221120064947.94540-1-quinkblack@foxmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 06/12] avcodec/mediacodec: add NDK media codec wrapper X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: matthieu.bouron@stupeflix.com, Zhao Zhili , aman@tmm1.net Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: MgbD9ICovLnR From: Zhao Zhili Signed-off-by: Zhao Zhili --- configure | 2 + libavcodec/mediacodec_wrapper.c | 596 +++++++++++++++++++++++++++++- libavcodec/mediacodec_wrapper.h | 8 +- libavcodec/mediacodecdec.c | 2 +- libavcodec/mediacodecdec_common.c | 2 +- 5 files changed, 600 insertions(+), 10 deletions(-) diff --git a/configure b/configure index e54649fa48..9da5a36072 100755 --- a/configure +++ b/configure @@ -3181,6 +3181,7 @@ h264_crystalhd_decoder_select="crystalhd h264_mp4toannexb_bsf h264_parser" h264_cuvid_decoder_deps="cuvid" h264_cuvid_decoder_select="h264_mp4toannexb_bsf" h264_mediacodec_decoder_deps="mediacodec" +h264_mediacodec_decoder_extralibs="-landroid" h264_mediacodec_decoder_select="h264_mp4toannexb_bsf h264_parser" h264_mf_encoder_deps="mediafoundation" h264_mmal_decoder_deps="mmal" @@ -3199,6 +3200,7 @@ hevc_amf_encoder_deps="amf" hevc_cuvid_decoder_deps="cuvid" hevc_cuvid_decoder_select="hevc_mp4toannexb_bsf" hevc_mediacodec_decoder_deps="mediacodec" +hevc_mediacodec_decoder_extralibs="-landroid" hevc_mediacodec_decoder_select="hevc_mp4toannexb_bsf hevc_parser" hevc_mf_encoder_deps="mediafoundation" hevc_nvenc_encoder_deps="nvenc" diff --git a/libavcodec/mediacodec_wrapper.c b/libavcodec/mediacodec_wrapper.c index e0c614680e..b12aced711 100644 --- a/libavcodec/mediacodec_wrapper.c +++ b/libavcodec/mediacodec_wrapper.c @@ -20,7 +20,11 @@ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA */ +#include #include +#include +#include +#include #include "libavutil/avassert.h" #include "libavutil/mem.h" @@ -1794,23 +1798,607 @@ static const FFAMediaCodec media_codec_jni = { .cleanOutputBuffers = mediacodec_jni_cleanOutputBuffers, }; -FFAMediaFormat *ff_AMediaFormat_new(void) +typedef struct FFAMediaFormatNdk { + FFAMediaFormat api; + + void *libmedia; + AMediaFormat *impl; + + AMediaFormat *(*new)(void); + media_status_t (*delete)(AMediaFormat*); + + const char* (*toString)(AMediaFormat*); + + bool (*getInt32)(AMediaFormat*, const char *name, int32_t *out); + bool (*getInt64)(AMediaFormat*, const char *name, int64_t *out); + bool (*getFloat)(AMediaFormat*, const char *name, float *out); + bool (*getSize)(AMediaFormat*, const char *name, size_t *out); + bool (*getBuffer)(AMediaFormat*, const char *name, void** data, size_t *size); + bool (*getString)(AMediaFormat*, const char *name, const char **out); + + void (*setInt32)(AMediaFormat*, const char* name, int32_t value); + void (*setInt64)(AMediaFormat*, const char* name, int64_t value); + void (*setFloat)(AMediaFormat*, const char* name, float value); + void (*setString)(AMediaFormat*, const char* name, const char* value); + void (*setBuffer)(AMediaFormat*, const char* name, const void* data, size_t size); +} FFAMediaFormatNdk; + +typedef struct FFAMediaCodecNdk { + FFAMediaCodec api; + + void *libmedia; + AMediaCodec *impl; + ANativeWindow *window; + + AMediaCodec* (*createCodecByName)(const char *name); + AMediaCodec* (*createDecoderByType)(const char *mime_type); + AMediaCodec* (*createEncoderByType)(const char *mime_type); + media_status_t (*delete)(AMediaCodec*); + + media_status_t (*configure)(AMediaCodec *, + const AMediaFormat *format, + ANativeWindow *surface, + AMediaCrypto *crypto, + uint32_t flags); + media_status_t (*start)(AMediaCodec*); + media_status_t (*stop)(AMediaCodec*); + media_status_t (*flush)(AMediaCodec*); + + uint8_t* (*getInputBuffer)(AMediaCodec*, size_t idx, size_t *out_size); + uint8_t* (*getOutputBuffer)(AMediaCodec*, size_t idx, size_t *out_size); + + ssize_t (*dequeueInputBuffer)(AMediaCodec*, int64_t timeoutUs); + media_status_t (*queueInputBuffer)(AMediaCodec*, size_t idx, + long offset, size_t size, + uint64_t time, uint32_t flags); + + ssize_t (*dequeueOutputBuffer)(AMediaCodec*, AMediaCodecBufferInfo *info, int64_t timeoutUs); + AMediaFormat* (*getOutputFormat)(AMediaCodec*); + + media_status_t (*releaseOutputBuffer)(AMediaCodec*, size_t idx, bool render); + media_status_t (*releaseOutputBufferAtTime)(AMediaCodec *mData, size_t idx, int64_t timestampNs); + + // Available since API level 28. + media_status_t (*getName)(AMediaCodec*, char** out_name); + void (*releaseName)(AMediaCodec*, char* name); +} FFAMediaCodecNdk; + +static const FFAMediaFormat media_format_ndk; +static const FFAMediaCodec media_codec_ndk; + +static const AVClass amediaformat_ndk_class = { + .class_name = "amediaformat_ndk", + .item_name = av_default_item_name, + .version = LIBAVUTIL_VERSION_INT, +}; + +static const AVClass amediacodec_ndk_class = { + .class_name = "amediacodec_ndk", + .item_name = av_default_item_name, + .version = LIBAVUTIL_VERSION_INT, +}; + +static FFAMediaFormat *mediaformat_ndk_create(AMediaFormat *impl) +{ + FFAMediaFormatNdk *format = av_mallocz(sizeof(*format)); + if (!format) + return NULL; + + format->api = media_format_ndk; + + format->libmedia = dlopen("libmediandk.so", RTLD_NOW); + if (!format->libmedia) + goto error; + +#define GET_SYMBOL(sym) \ + format->sym = dlsym(format->libmedia, "AMediaFormat_" #sym); \ + if (!format->sym) \ + goto error; + + GET_SYMBOL(new) + GET_SYMBOL(delete) + + GET_SYMBOL(toString) + + GET_SYMBOL(getInt32) + GET_SYMBOL(getInt64) + GET_SYMBOL(getFloat) + GET_SYMBOL(getSize) + GET_SYMBOL(getBuffer) + GET_SYMBOL(getString) + + GET_SYMBOL(setInt32) + GET_SYMBOL(setInt64) + GET_SYMBOL(setFloat) + GET_SYMBOL(setString) + GET_SYMBOL(setBuffer) + +#undef GET_SYMBOL + + if (impl) { + format->impl = impl; + } else { + format->impl = format->new(); + if (!format->impl) + goto error; + } + + return (FFAMediaFormat *)format; + +error: + if (format->libmedia) + dlclose(format->libmedia); + av_freep(&format); + return NULL; +} + +static FFAMediaFormat *mediaformat_ndk_new(void) +{ + return mediaformat_ndk_create(NULL); +} + +static int mediaformat_ndk_delete(FFAMediaFormat* ctx) +{ + FFAMediaFormatNdk *format = (FFAMediaFormatNdk *)ctx; + int ret = 0; + if (!format) + return 0; + + av_assert0(format->api.class == &amediaformat_ndk_class); + + if (format->impl && (format->delete(format->impl) != AMEDIA_OK)) + ret = AVERROR_EXTERNAL; + if (format->libmedia) + dlclose(format->libmedia); + av_free(format); + + return ret; +} + +static char* mediaformat_ndk_toString(FFAMediaFormat* ctx) +{ + FFAMediaFormatNdk *format = (FFAMediaFormatNdk *)ctx; + const char *str = format->toString(format->impl); + return av_strdup(str); +} + +static int mediaformat_ndk_getInt32(FFAMediaFormat* ctx, const char *name, int32_t *out) +{ + FFAMediaFormatNdk *format = (FFAMediaFormatNdk *)ctx; + return format->getInt32(format->impl, name, out); +} + +static int mediaformat_ndk_getInt64(FFAMediaFormat* ctx, const char *name, int64_t *out) +{ + FFAMediaFormatNdk *format = (FFAMediaFormatNdk *)ctx; + return format->getInt64(format->impl, name, out); +} + +static int mediaformat_ndk_getFloat(FFAMediaFormat* ctx, const char *name, float *out) +{ + FFAMediaFormatNdk *format = (FFAMediaFormatNdk *)ctx; + return format->getFloat(format->impl, name, out); +} + +static int mediaformat_ndk_getBuffer(FFAMediaFormat* ctx, const char *name, void** data, size_t *size) +{ + FFAMediaFormatNdk *format = (FFAMediaFormatNdk *)ctx; + return format->getBuffer(format->impl, name, data, size); +} + +static int mediaformat_ndk_getString(FFAMediaFormat* ctx, const char *name, const char **out) +{ + FFAMediaFormatNdk *format = (FFAMediaFormatNdk *)ctx; + const char *tmp = NULL; + int ret = format->getString(format->impl, name, &tmp); + + if (tmp) + *out = av_strdup(tmp); + return ret; +} + +static void mediaformat_ndk_setInt32(FFAMediaFormat* ctx, const char* name, int32_t value) +{ + FFAMediaFormatNdk *format = (FFAMediaFormatNdk *)ctx; + format->setInt32(format->impl, name, value); +} + +static void mediaformat_ndk_setInt64(FFAMediaFormat* ctx, const char* name, int64_t value) +{ + FFAMediaFormatNdk *format = (FFAMediaFormatNdk *)ctx; + format->setInt64(format->impl, name, value); +} + +static void mediaformat_ndk_setFloat(FFAMediaFormat* ctx, const char* name, float value) +{ + FFAMediaFormatNdk *format = (FFAMediaFormatNdk *)ctx; + format->setFloat(format->impl, name, value); +} + +static void mediaformat_ndk_setString(FFAMediaFormat* ctx, const char* name, const char* value) +{ + FFAMediaFormatNdk *format = (FFAMediaFormatNdk *)ctx; + format->setString(format->impl, name, value); +} + +static void mediaformat_ndk_setBuffer(FFAMediaFormat* ctx, const char* name, void* data, size_t size) +{ + FFAMediaFormatNdk *format = (FFAMediaFormatNdk *)ctx; + format->setBuffer(format->impl, name, data, size); +} + +static char *mediacodec_ndk_getName(FFAMediaCodec *ctx) +{ + FFAMediaCodecNdk *codec = (FFAMediaCodecNdk *)ctx; + char *ret = NULL; + char *name = NULL; + + if (!codec->getName || !codec->releaseName) { + av_log(ctx, AV_LOG_DEBUG, "getName() unavailable\n"); + return ret; + } + + codec->getName(codec->impl, &name); + if (name) { + ret = av_strdup(name); + codec->releaseName(codec->impl, name); + } + + return ret; +} + +static inline FFAMediaCodec *ndk_codec_create(int method, const char *arg) { + FFAMediaCodecNdk *codec = av_mallocz(sizeof(*codec)); + const char *lib_name = "libmediandk.so"; + + if (!codec) + return NULL; + + codec->api = media_codec_ndk; + codec->libmedia = dlopen(lib_name, RTLD_NOW); + if (!codec->libmedia) + goto error; + +#define GET_SYMBOL(sym, required) \ + codec->sym = dlsym(codec->libmedia, "AMediaCodec_" #sym); \ + if (!codec->sym) { \ + av_log(codec, required ? AV_LOG_ERROR : AV_LOG_INFO, \ + #sym "() unavailable from %s\n", lib_name); \ + if (required) \ + goto error; \ + } + + GET_SYMBOL(createCodecByName, 1) + GET_SYMBOL(createDecoderByType, 1) + GET_SYMBOL(createEncoderByType, 1) + GET_SYMBOL(delete, 1) + + GET_SYMBOL(configure, 1) + GET_SYMBOL(start, 1) + GET_SYMBOL(stop, 1) + GET_SYMBOL(flush, 1) + + GET_SYMBOL(getInputBuffer, 1) + GET_SYMBOL(getOutputBuffer, 1) + + GET_SYMBOL(dequeueInputBuffer, 1) + GET_SYMBOL(queueInputBuffer, 1) + + GET_SYMBOL(dequeueOutputBuffer, 1) + GET_SYMBOL(getOutputFormat, 1) + + GET_SYMBOL(releaseOutputBuffer, 1) + GET_SYMBOL(releaseOutputBufferAtTime, 1) + + GET_SYMBOL(getName, 0) + GET_SYMBOL(releaseName, 0) + +#undef GET_SYMBOL + + switch (method) { + case CREATE_CODEC_BY_NAME: + codec->impl = codec->createCodecByName(arg); + break; + case CREATE_DECODER_BY_TYPE: + codec->impl = codec->createDecoderByType(arg); + break; + case CREATE_ENCODER_BY_TYPE: + codec->impl = codec->createEncoderByType(arg); + break; + default: + av_assert0(0); + } + if (!codec->impl) + goto error; + + return (FFAMediaCodec *)codec; + +error: + if (codec->libmedia) + dlclose(codec->libmedia); + av_freep(&codec); + return NULL; +} + +#define DECLARE_NDK_AMEDIACODEC_CREATE_FUNC(name, method) \ +static FFAMediaCodec *mediacodec_ndk_##name(const char *arg) \ +{ \ + return ndk_codec_create(method, arg); \ +} \ + +DECLARE_NDK_AMEDIACODEC_CREATE_FUNC(createCodecByName, CREATE_CODEC_BY_NAME) +DECLARE_NDK_AMEDIACODEC_CREATE_FUNC(createDecoderByType, CREATE_DECODER_BY_TYPE) +DECLARE_NDK_AMEDIACODEC_CREATE_FUNC(createEncoderByType, CREATE_ENCODER_BY_TYPE) + +static int mediacodec_ndk_delete(FFAMediaCodec* ctx) +{ + FFAMediaCodecNdk *codec = (FFAMediaCodecNdk *)ctx; + int ret = 0; + + if (!codec) + return 0; + + av_assert0(codec->api.class == &amediacodec_ndk_class); + + if (codec->impl && (codec->delete(codec->impl) != AMEDIA_OK)) + ret = AVERROR_EXTERNAL; + if (codec->window) + ANativeWindow_release(codec->window); + if (codec->libmedia) + dlclose(codec->libmedia); + av_free(codec); + + return ret; +} + +static int mediacodec_ndk_configure(FFAMediaCodec* ctx, const FFAMediaFormat* format_ctx, void* surface, void *crypto, uint32_t flags) +{ + FFAMediaCodecNdk *codec = (FFAMediaCodecNdk *)ctx; + FFAMediaFormatNdk *format = (FFAMediaFormatNdk *)format_ctx; + media_status_t status; + + if (surface) { + JNIEnv *env = NULL; + JNI_GET_ENV_OR_RETURN(env, ctx, -1); + codec->window = ANativeWindow_fromSurface(env, surface); + } + + if (format_ctx->class != &amediaformat_ndk_class) { + av_log(ctx, AV_LOG_ERROR, "invalid media format\n"); + return AVERROR(EINVAL); + } + + status = codec->configure(codec->impl, format->impl, codec->window, NULL, flags); + if (status != AMEDIA_OK) { + av_log(codec, AV_LOG_ERROR, "configure failed, %d\n", status); + return AVERROR_EXTERNAL; + } + + return 0; +} + +#define MEDIACODEC_NDK_WRAPPER(method) \ +static int mediacodec_ndk_ ## method(FFAMediaCodec* ctx) \ +{ \ + FFAMediaCodecNdk *codec = (FFAMediaCodecNdk *)ctx; \ + media_status_t status = codec->method(codec->impl); \ + \ + if (status != AMEDIA_OK) { \ + av_log(codec, AV_LOG_ERROR, #method " failed, %d\n", status); \ + return AVERROR_EXTERNAL; \ + } \ + \ + return 0; \ +} \ + +MEDIACODEC_NDK_WRAPPER(start) +MEDIACODEC_NDK_WRAPPER(stop) +MEDIACODEC_NDK_WRAPPER(flush) + +static uint8_t* mediacodec_ndk_getInputBuffer(FFAMediaCodec* ctx, size_t idx, size_t *out_size) +{ + FFAMediaCodecNdk *codec = (FFAMediaCodecNdk *)ctx; + return codec->getInputBuffer(codec->impl, idx, out_size); +} + +static uint8_t* mediacodec_ndk_getOutputBuffer(FFAMediaCodec* ctx, size_t idx, size_t *out_size) +{ + FFAMediaCodecNdk *codec = (FFAMediaCodecNdk *)ctx; + return codec->getOutputBuffer(codec->impl, idx, out_size); +} + +static ssize_t mediacodec_ndk_dequeueInputBuffer(FFAMediaCodec* ctx, int64_t timeoutUs) +{ + FFAMediaCodecNdk *codec = (FFAMediaCodecNdk *)ctx; + return codec->dequeueInputBuffer(codec->impl, timeoutUs); +} + +static int mediacodec_ndk_queueInputBuffer(FFAMediaCodec *ctx, size_t idx, + off_t offset, size_t size, + uint64_t time, uint32_t flags) +{ + FFAMediaCodecNdk *codec = (FFAMediaCodecNdk *)ctx; + return codec->queueInputBuffer(codec->impl, idx, offset, size, time, flags); +} + +static ssize_t mediacodec_ndk_dequeueOutputBuffer(FFAMediaCodec* ctx, FFAMediaCodecBufferInfo *info, int64_t timeoutUs) +{ + FFAMediaCodecNdk *codec = (FFAMediaCodecNdk *)ctx; + AMediaCodecBufferInfo buf_info = {0}; + ssize_t ret; + + ret = codec->dequeueOutputBuffer(codec->impl, &buf_info, timeoutUs); + info->offset = buf_info.offset; + info->size = buf_info.size; + info->presentationTimeUs = buf_info.presentationTimeUs; + info->flags = buf_info.flags; + + return ret; +} + +static FFAMediaFormat* mediacodec_ndk_getOutputFormat(FFAMediaCodec* ctx) +{ + FFAMediaCodecNdk *codec = (FFAMediaCodecNdk *)ctx; + AMediaFormat *format = codec->getOutputFormat(codec->impl); + + if (!format) + return NULL; + return mediaformat_ndk_create(format); +} + +static int mediacodec_ndk_releaseOutputBuffer(FFAMediaCodec* ctx, size_t idx, int render) +{ + FFAMediaCodecNdk *codec = (FFAMediaCodecNdk *)ctx; + media_status_t status; + + status = codec->releaseOutputBuffer(codec->impl, idx, render); + if (status != AMEDIA_OK) { + av_log(codec, AV_LOG_ERROR, "release output buffer failed, %d\n", status); + return AVERROR_EXTERNAL; + } + + return 0; +} + +static int mediacodec_ndk_releaseOutputBufferAtTime(FFAMediaCodec *ctx, size_t idx, int64_t timestampNs) +{ + FFAMediaCodecNdk *codec = (FFAMediaCodecNdk *)ctx; + media_status_t status; + + status = codec->releaseOutputBufferAtTime(codec->impl, idx, timestampNs); + if (status != AMEDIA_OK) { + av_log(codec, AV_LOG_ERROR, "releaseOutputBufferAtTime failed, %d\n", status); + return AVERROR_EXTERNAL; + } + + return 0; +} + +static int mediacodec_ndk_infoTryAgainLater(FFAMediaCodec *ctx, ssize_t idx) +{ + return idx == AMEDIACODEC_INFO_TRY_AGAIN_LATER; +} + +static int mediacodec_ndk_infoOutputBuffersChanged(FFAMediaCodec *ctx, ssize_t idx) +{ + return idx == AMEDIACODEC_INFO_OUTPUT_BUFFERS_CHANGED; +} + +static int mediacodec_ndk_infoOutputFormatChanged(FFAMediaCodec *ctx, ssize_t idx) +{ + return idx == AMEDIACODEC_INFO_OUTPUT_FORMAT_CHANGED; +} + +static int mediacodec_ndk_getBufferFlagCodecConfig(FFAMediaCodec *ctx) +{ + return AMEDIACODEC_BUFFER_FLAG_CODEC_CONFIG; +} + +static int mediacodec_ndk_getBufferFlagEndOfStream(FFAMediaCodec *ctx) +{ + return AMEDIACODEC_BUFFER_FLAG_END_OF_STREAM; +} + +static int mediacodec_ndk_getBufferFlagKeyFrame(FFAMediaCodec *ctx) +{ + return 1; +} + +static int mediacodec_ndk_getConfigureFlagEncode(FFAMediaCodec *ctx) +{ + return AMEDIACODEC_CONFIGURE_FLAG_ENCODE; +} + +static int mediacodec_ndk_cleanOutputBuffers(FFAMediaCodec *ctx) +{ + return 0; +} + +static const FFAMediaFormat media_format_ndk = { + .class = &amediaformat_ndk_class, + + .create = mediaformat_ndk_new, + .delete = mediaformat_ndk_delete, + + .toString = mediaformat_ndk_toString, + + .getInt32 = mediaformat_ndk_getInt32, + .getInt64 = mediaformat_ndk_getInt64, + .getFloat = mediaformat_ndk_getFloat, + .getBuffer = mediaformat_ndk_getBuffer, + .getString = mediaformat_ndk_getString, + + .setInt32 = mediaformat_ndk_setInt32, + .setInt64 = mediaformat_ndk_setInt64, + .setFloat = mediaformat_ndk_setFloat, + .setString = mediaformat_ndk_setString, + .setBuffer = mediaformat_ndk_setBuffer, +}; + +static const FFAMediaCodec media_codec_ndk = { + .class = &amediacodec_ndk_class, + + .getName = mediacodec_ndk_getName, + + .createCodecByName = mediacodec_ndk_createCodecByName, + .createDecoderByType = mediacodec_ndk_createDecoderByType, + .createEncoderByType = mediacodec_ndk_createEncoderByType, + .delete = mediacodec_ndk_delete, + + .configure = mediacodec_ndk_configure, + .start = mediacodec_ndk_start, + .stop = mediacodec_ndk_stop, + .flush = mediacodec_ndk_flush, + + .getInputBuffer = mediacodec_ndk_getInputBuffer, + .getOutputBuffer = mediacodec_ndk_getOutputBuffer, + + .dequeueInputBuffer = mediacodec_ndk_dequeueInputBuffer, + .queueInputBuffer = mediacodec_ndk_queueInputBuffer, + + .dequeueOutputBuffer = mediacodec_ndk_dequeueOutputBuffer, + .getOutputFormat = mediacodec_ndk_getOutputFormat, + + .releaseOutputBuffer = mediacodec_ndk_releaseOutputBuffer, + .releaseOutputBufferAtTime = mediacodec_ndk_releaseOutputBufferAtTime, + + .infoTryAgainLater = mediacodec_ndk_infoTryAgainLater, + .infoOutputBuffersChanged = mediacodec_ndk_infoOutputBuffersChanged, + .infoOutputFormatChanged = mediacodec_ndk_infoOutputFormatChanged, + + .getBufferFlagCodecConfig = mediacodec_ndk_getBufferFlagCodecConfig, + .getBufferFlagEndOfStream = mediacodec_ndk_getBufferFlagEndOfStream, + .getBufferFlagKeyFrame = mediacodec_ndk_getBufferFlagKeyFrame, + + .getConfigureFlagEncode = mediacodec_ndk_getConfigureFlagEncode, + .cleanOutputBuffers = mediacodec_ndk_cleanOutputBuffers, +}; + +FFAMediaFormat *ff_AMediaFormat_new(int ndk) { + if (ndk) + return media_format_ndk.create(); return media_format_jni.create(); } -FFAMediaCodec* ff_AMediaCodec_createCodecByName(const char *name) +FFAMediaCodec* ff_AMediaCodec_createCodecByName(const char *name, int ndk) { + if (ndk) + return media_codec_ndk.createCodecByName(name); return media_codec_jni.createCodecByName(name); } -FFAMediaCodec* ff_AMediaCodec_createDecoderByType(const char *mime_type) +FFAMediaCodec* ff_AMediaCodec_createDecoderByType(const char *mime_type, int ndk) { + if (ndk) + return media_codec_ndk.createDecoderByType(mime_type); return media_codec_jni.createDecoderByType(mime_type); } -FFAMediaCodec* ff_AMediaCodec_createEncoderByType(const char *mime_type) +FFAMediaCodec* ff_AMediaCodec_createEncoderByType(const char *mime_type, int ndk) { + if (ndk) + return media_codec_ndk.createEncoderByType(mime_type); return media_codec_jni.createEncoderByType(mime_type); } diff --git a/libavcodec/mediacodec_wrapper.h b/libavcodec/mediacodec_wrapper.h index 606fdbede5..b9b882f243 100644 --- a/libavcodec/mediacodec_wrapper.h +++ b/libavcodec/mediacodec_wrapper.h @@ -80,7 +80,7 @@ struct FFAMediaFormat { void (*setBuffer)(FFAMediaFormat* format, const char* name, void* data, size_t size); }; -FFAMediaFormat *ff_AMediaFormat_new(void); +FFAMediaFormat *ff_AMediaFormat_new(int ndk); static inline int ff_AMediaFormat_delete(FFAMediaFormat* format) { @@ -198,9 +198,9 @@ static inline char *ff_AMediaCodec_getName(FFAMediaCodec *codec) return codec->getName(codec); } -FFAMediaCodec* ff_AMediaCodec_createCodecByName(const char *name); -FFAMediaCodec* ff_AMediaCodec_createDecoderByType(const char *mime_type); -FFAMediaCodec* ff_AMediaCodec_createEncoderByType(const char *mime_type); +FFAMediaCodec* ff_AMediaCodec_createCodecByName(const char *name, int ndk); +FFAMediaCodec* ff_AMediaCodec_createDecoderByType(const char *mime_type, int ndk); +FFAMediaCodec* ff_AMediaCodec_createEncoderByType(const char *mime_type, int ndk); static inline int ff_AMediaCodec_configure(FFAMediaCodec* codec, const FFAMediaFormat* format, void* surface, void *crypto, uint32_t flags) { diff --git a/libavcodec/mediacodecdec.c b/libavcodec/mediacodecdec.c index 2c66f38541..2e07548b77 100644 --- a/libavcodec/mediacodecdec.c +++ b/libavcodec/mediacodecdec.c @@ -310,7 +310,7 @@ static av_cold int mediacodec_decode_init(AVCodecContext *avctx) FFAMediaFormat *format = NULL; MediaCodecH264DecContext *s = avctx->priv_data; - format = ff_AMediaFormat_new(); + format = ff_AMediaFormat_new(0); if (!format) { av_log(avctx, AV_LOG_ERROR, "Failed to create media format\n"); ret = AVERROR_EXTERNAL; diff --git a/libavcodec/mediacodecdec_common.c b/libavcodec/mediacodecdec_common.c index 69a462ec48..4a9e50b0df 100644 --- a/libavcodec/mediacodecdec_common.c +++ b/libavcodec/mediacodecdec_common.c @@ -613,7 +613,7 @@ int ff_mediacodec_dec_init(AVCodecContext *avctx, MediaCodecDecContext *s, } av_log(avctx, AV_LOG_DEBUG, "Found decoder %s\n", s->codec_name); - s->codec = ff_AMediaCodec_createCodecByName(s->codec_name); + s->codec = ff_AMediaCodec_createCodecByName(s->codec_name, 0); if (!s->codec) { av_log(avctx, AV_LOG_ERROR, "Failed to create media decoder for type %s and name %s\n", mime, s->codec_name); ret = AVERROR_EXTERNAL;