From patchwork Wed Nov 6 12:33:04 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhao Zhili X-Patchwork-Id: 52614 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:612c:2fe5:b0:4a6:1c7:11b7 with SMTP id kw5csp617812vqb; Wed, 6 Nov 2024 04:39:09 -0800 (PST) X-Forwarded-Encrypted: i=2; AJvYcCXt6U6BUPa/6JnNw3R9ZOLtMTjVmqoLYYq5So3D+DwdgOwcZm/NOdzEVzeBylIqHaa9Ku/UuGI8FOUqTAJGe0R4@gmail.com X-Google-Smtp-Source: AGHT+IGg8i3Z9thZC/EF5kVLoLf9+6PtSXr4UNzvToqJG8wU9xmw5OfWDzWPUq2r56zjrM+7tdqU X-Received: by 2002:a05:6402:2682:b0:5cb:67a8:cf0a with SMTP id 4fb4d7f45d1cf-5cbbf87519cmr12873177a12.2.1730896748709; Wed, 06 Nov 2024 04:39:08 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1730896748; cv=none; d=google.com; s=arc-20240605; b=lGcDG3uPG82oJI/WzKPKDJbGyaG6lXqVshdgZSAo4SE/iJCuC/6lRTZszrGvxn7i4k jysQ7IkV7XY2uTJ6TwkRvfGJ1TVyEpj9fv7e63S9DHte1Pbyd58aUGGOomaOs/nkCqa7 Req8LpP1WEkWg1VgN3uvcxD2iXRfGfMQlm4X14wkfTR7ePEcB8dWQ9PJrxY2QKsLUsaL TLZMUBvS1I0XEfezrKAF+hVKi1G7Srw2RGMwCsSJFtqkc9pZOSaIMyzqXxHw8GDEKXSN ir8xEsHtMYeclWlO5U8gdUnsMHJNgHA4oN8cZJkYqFJGx9lQZgm8zt0ynjFPj51dq6Bh nTJA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:date:to:from:message-id :dkim-signature:delivered-to; bh=/IF9Gkuz0rdrSVEjE/FeVxwBad8mGc1n6B0OVbWpkhM=; fh=HnHYuZ9XgUo86ZRXTLWWmQxhslYEI9B9taZ5X1DLFfc=; b=kUjwwiLCOn2P9agjNgRm5N5Q/a9io6iPRPfjuRwcYjbLU9DrvqAfE6ccQZbvcD/YLC kiCkCBWdjJqlR2hy1SrNyJIVmdQ+s9qw5lswIf3DpkrXsj/9pGdL9Q/ftuWvcQVL0Ojv V3P1RJxkg/wMpdcgRxSK6A8obERLKdskykouAxZVc3Iq1pt4xBkAAGjNKQjDsxLhNlIf cICrB1XtydUjpSxL7qNInpFmuLTIaFZeoAcP8j8cAOl8JdkFmv1lL2j4DVYYL6p3mKr1 OOqzSc+aE4rl/LWGrIgeRaoVZouyB3sS3VYaVC4LCBC9SG8LnUzd3fw0tO0Wyk4nHDV0 VsTQ==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@foxmail.com header.s=s201512 header.b=JDiND+5w; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=foxmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 4fb4d7f45d1cf-5cee6aff5efsi2748615a12.232.2024.11.06.04.39.08; Wed, 06 Nov 2024 04:39:08 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@foxmail.com header.s=s201512 header.b=JDiND+5w; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=foxmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id C42F968DDAE; Wed, 6 Nov 2024 14:33:21 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from out203-205-221-192.mail.qq.com (out203-205-221-192.mail.qq.com [203.205.221.192]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 9DDB168DDAE for ; Wed, 6 Nov 2024 14:33:14 +0200 (EET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=foxmail.com; s=s201512; t=1730896386; bh=pkXdjljJzHlnJrwhQ2bumvNneTsAWvIeMglWF1U3UuQ=; h=From:To:Cc:Subject:Date; b=JDiND+5wWn557Bt8IcVsiYhY1kv1EoSlaj+AmbZ1GP+cZS1DQeG+dt4XttHKTYNXX Z+NtsKLZJHnXziw9dQWsefdxed27gJS8gRxt/OL41d1yKcm9NUEB8eMqw36pafkoW3 +sXG1AQxtAvt/SZ9VCr4qebrByT0na8NVED8RxFc= Received: from ZHILIZHAO-MB1.tencent.com ([59.37.125.101]) by newxmesmtplogicsvrszb21-0.qq.com (NewEsmtp) with SMTP id 8458C695; Wed, 06 Nov 2024 20:33:05 +0800 X-QQ-mid: xmsmtpt1730896385tgad1reeo Message-ID: X-QQ-XMAILINFO: OATpkVjS499upLUs5s4W2f8569iLmDx9ka4hPmZAkTSS+BL+ADxaoRm2LkdaiQ H8C6jlVl51ImJE6a1acfulA8FQBhXmLCbtz+9iG/VPUn9WdpX2eKMCrK2Pmhkg+mmD/EI71S8/RA seGXjNV8RYPWg0qaJIZIqITYneZZuSHDzUn5p9jQIC7ONEUCug+g8hxPyrSP8hke1CRVpQWkVHOG RdPw1XMzNq2E1p6gTUbodcTZYqocucKb+s6x6IZD3TyM9/URD4pc+JtW9LU4LbNWLCKkZc7UcSih Pkvm4rKWWVK6ahRj/GvG9QnkiMrjgbsWoXer6pfLVh14K1gNmeG1fiUfHt5DpsGbgI840FNZZB69 eQMloe4OalBYcV9cOrs4mAj1xBV44HAde6jXPzboo/vgKzeK0Ne4Cfsw3bw930H3kehlJTolDImU dq4RBj9PBixkeb5ZMj3S7X7fvHZ+K3NMPCPUAdLGTpB/YQ8adEB+jCrhk2nbFyfRiKYmunAFfJmf edEpsyOO6HqCy7Fud6QpHby/ciWRq1wMHJpLFwkw2CCwgHYFDtKAs19d8lxuuH8N4QbdXOnkOfj7 Y428vckasEB/gJzhns7TF3412FsreVzWR1nkDpObH8Evho7qwDmIYpYlyXJrCLsehVeN40oA1fZc JqN39d8CkuU2k8RjiruekXPOb+NCR2BRRh0D8bRp0cN5qqCLvfG4OuBOGCI+OLMY22ThlowbxZjH 21e8P0uvGBvna4r8+drKHyEyyJSqzfdCzfY9LESrfBGvT4ZorlueEtf5fLBCObjyVDyNfXaKTmUB gYkvxAnwUudpIXHYDp/Sb0TrTJGIHmRvHZiUNlCZJfh5x8p9T4XQJW0r7WnJuSNTM18c8gHHAx9T EHGsw/wj3jWVEUDrUIFCd+xOKtJ+Oo6KX7F/qlhbP44PqgWtuo6vj63vbIXJpRDC3WR0KrJtaxR2 PdZ0BvhTJbgBWq5hEk1OJ6YTN+5L4I3t7WzDEC8crtLe+DG5ZJ300jL6UHXoq1W2lpTq2P29omSD 37CtXNa0HpVcpiaPJzGApE7Qx1DhQ24+TtwQt7Mw== X-QQ-XMRINFO: OD9hHCdaPRBwq3WW+NvGbIU= From: Zhao Zhili To: ffmpeg-devel@ffmpeg.org Date: Wed, 6 Nov 2024 20:33:04 +0800 X-OQ-MSGID: <20241106123304.17794-1-quinkblack@foxmail.com> X-Mailer: git-send-email 2.46.0 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 1/4] avcodec/mediacodecdec: Add operating_rate option X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Zhao Zhili Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: srxZCImznxoi From: Zhao Zhili The codec wants to know whether the usecase is realtime playback or full-speed transcoding, or playback at a higher speed. The codec runs faster when operating_rate higher than framerate. --- libavcodec/mediacodecdec.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/libavcodec/mediacodecdec.c b/libavcodec/mediacodecdec.c index cc55b306bd..4937828f21 100644 --- a/libavcodec/mediacodecdec.c +++ b/libavcodec/mediacodecdec.c @@ -57,6 +57,8 @@ typedef struct MediaCodecH264DecContext { int amlogic_mpeg2_api23_workaround; int use_ndk_codec; + // Ref. MediaFormat KEY_OPERATING_RATE + int operating_rate; } MediaCodecH264DecContext; static av_cold int mediacodec_decode_close(AVCodecContext *avctx) @@ -441,6 +443,8 @@ static av_cold int mediacodec_decode_init(AVCodecContext *avctx) ff_AMediaFormat_setInt32(format, "channel-count", avctx->ch_layout.nb_channels); ff_AMediaFormat_setInt32(format, "sample-rate", avctx->sample_rate); } + if (s->operating_rate > 0) + ff_AMediaFormat_setInt32(format, "operating-rate", s->operating_rate); s->ctx = av_mallocz(sizeof(*s->ctx)); if (!s->ctx) { @@ -599,6 +603,8 @@ static const AVOption ff_mediacodec_vdec_options[] = { OFFSET(delay_flush), AV_OPT_TYPE_BOOL, {.i64 = 0}, 0, 1, VD }, { "ndk_codec", "Use MediaCodec from NDK", OFFSET(use_ndk_codec), AV_OPT_TYPE_BOOL, {.i64 = -1}, -1, 1, VD }, + { "operating_rate", "The desired operating rate that the codec will need to operate at, zero for unspecified", + OFFSET(operating_rate), AV_OPT_TYPE_INT, {.i64 = 0}, 0, INT_MAX, VD }, { NULL } }; @@ -662,6 +668,8 @@ DECLARE_MEDIACODEC_VDEC(av1, "AV1", AV_CODEC_ID_AV1, NULL) static const AVOption ff_mediacodec_adec_options[] = { { "ndk_codec", "Use MediaCodec from NDK", OFFSET(use_ndk_codec), AV_OPT_TYPE_BOOL, {.i64 = -1}, -1, 1, AD }, + { "operating_rate", "The desired operating rate that the codec will need to operate at, zero for unspecified", + OFFSET(operating_rate), AV_OPT_TYPE_INT, {.i64 = 0}, 0, INT_MAX, AD }, { NULL } }; From patchwork Wed Nov 6 12:31:28 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhao Zhili X-Patchwork-Id: 52615 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:612c:2fe5:b0:4a6:1c7:11b7 with SMTP id kw5csp617819vqb; Wed, 6 Nov 2024 04:39:09 -0800 (PST) X-Forwarded-Encrypted: i=2; AJvYcCVvPr2FuP3k8Eme7/bgmc2Vh1ECbKRlvW9NQycI3fqzdCwUz41sJOmZKPe2TS7WS7+tDK7/FN6PpH1HoXSAUYmp@gmail.com X-Google-Smtp-Source: AGHT+IEKwppRZ6z6OCdMrhRGbQ86Y6OzzPzhy/yBaG8Unc0J1A23/XZ2HCUXstsjsxmEMFlhyiMv X-Received: by 2002:a05:6512:12c5:b0:539:e88f:23a1 with SMTP id 2adb3069b0e04-53b3491c80emr20968640e87.44.1730896749396; Wed, 06 Nov 2024 04:39:09 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1730896749; cv=none; d=google.com; s=arc-20240605; b=hbhUl/XlGC3a+1fbAIYnoVX7k6or1rhW0CA+Pk8BUSzoadhc01EvEZlIBnE/H0vhZk ihMWxR+Z4LkCYIDL5u86pSZcGOJRZ7K1/8WU1Qsk4+H/Vfo9DpTzWLKMkRfmzgOwWLD+ p9kGzGnyElhYynrJtQ8vJRzONPnnNZSTbCA1Z6zdy6IOFElNzVfXisF8XZS033Bw1PcP YfwYZqJm52Ly58UQQ4dC6WYswwwg/mqzfMOjn13Yg04ertwj2pg26QQfAq4UWqFrrUzO fH9iODJYUW9wR1U+vBKkaAIxvJSLvacP1gGwmnaLLb8QrkHPt+fvdjqJmKazJQYC130q NZbw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:date:to:from:message-id :dkim-signature:delivered-to; bh=bpWfqsoVFCs+V7f6kNAY5X3exiJx4IWX1wop4bmKNFA=; fh=HnHYuZ9XgUo86ZRXTLWWmQxhslYEI9B9taZ5X1DLFfc=; b=dr3Tj2CO6miDp36TD0pRMMu6T3YT2J3Zsfn084iO8wNZUko5Xpua5zr+UWF7rO5W+W TIeSKIJWb7wqy/U7oyJqWgdwJMmD4t40P9Ouo8a0YcHs0q7+fyW3o3+6EEMBLSWhbwaN AjJC2mTsba4Eoq2spBFfWK6Qry5khJ0P2x+D844yqp3P4NJv5OmKPUW0cQuBySZuJM4c xTem1euQ70Zwi9tU1OSd9zguno53yrJw69drWVOmZ0w3RBS/bKf+EgdolE14V5IuGl3G B/fjVi++oPTSZu3G4g4KcgNkUG5EyXl7LnQBn2tnDh98L97L0Od+6lctah9L88LONMCD 3AFw==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@foxmail.com header.s=s201512 header.b=w12tpI62; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=foxmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 2adb3069b0e04-53c7bdcb8d4si5156910e87.374.2024.11.06.04.39.09; Wed, 06 Nov 2024 04:39:09 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@foxmail.com header.s=s201512 header.b=w12tpI62; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=foxmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 46F3668DDF0; Wed, 6 Nov 2024 14:31:55 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from xmbghk7.mail.qq.com (xmbghk7.mail.qq.com [43.163.128.46]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 1561868DDAE for ; Wed, 6 Nov 2024 14:31:41 +0200 (EET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=foxmail.com; s=s201512; t=1730896291; bh=VVtXP6IVO3RbFdhKIPSsJqCzP3oH3vycc/RmQsR+r1U=; h=From:To:Cc:Subject:Date; b=w12tpI62CWmrLgED3vAirsh3TP57fAcUu0Nujp8ALqr0y4fJvrloysS0yCcQUV20F TRsB3Qhfl6dpDFAhksVGiUVmhL0WMRU42QDBxWRp669dc/cDnhMkTG+MhSqtD2OWOZ 7POuS3LNssyAMLX01+Z82nH+FIrXYcCsFEsp3WQ0= Received: from ZHILIZHAO-MB1.tencent.com ([59.37.125.101]) by newxmesmtplogicsvrsza36-0.qq.com (NewEsmtp) with SMTP id 7DDA0022; Wed, 06 Nov 2024 20:31:29 +0800 X-QQ-mid: xmsmtpt1730896289th3wuoq9s Message-ID: X-QQ-XMAILINFO: MmpliBmRb3iC3IwL82etZq2OnGLTd3cZddjL9C0zpWDII904DlV8XKkkgjj4VN GZpGZmui2Qmo7vMHqUgqPzVhdPg/LW7rpghUXjSfFspwOon+W4vIvLtazrpXB/HAdP2RsgBiSLOk g42VRIiPNqlrKUgui3kEX0hWxsYVyfny3cNfpxw1KflOUSmo/k9hMJWMKaEgmBJEtZFf08CJDx5S Dz382E0mdW7GDQvMTjHeXz0BOxKxFVG/g4HeuKdk8WvWMIhj/Tp+lXikDoL9qqDwqyPPLn5GrUrb n5IxEvIZMeBMTrG7GO5yJhluwCax4CY8WAgBEYpq1jiAF4DZkRn3GTTpdEdUcAc5fqVv0aaz84U0 pg+gld5iGj2f/ZCWzWBO+E3cnzYagObLYbs3RFBJmL80t14CskR+Ntowkd1/4GOX2uYHaFgYTXcC xRVszgjWoBE1hEPhdY+7IO6cUioUW3/nEfEq9XZcG1CVLPetm+tE3Ne0p1nQus3t5P3n5KSGfNS+ NksO8WBhA2jHkPRsIJHpdKimSrmlXtdYsj0dRk8nv6AfLzn3YuOXKQWlxbvF2KXFhYr8uhlh5VY9 TPy0IolLWQbTd740HyZHvX8d09OSWzzXjFM3dW9A0MIvlzyhoya5lxD1ClAYzI/wkCBPui94bg6T eCf9pfj9WFOKoGekg7U2NRxD6fp28jRyEyzoT6cghdR4Mas4NQRrAg5naJ9OsODLd8AOY8r0DrkM e3TtCs419gLRsoOTwNsM4n50MpVFr1l+a0YOa9hcv/ps+yRdbqkK6oUfhrFfnjXOlFQajcf+q9vY U53YxEXFHUUSqc/Mmyw+ixHefQ10zAHjFY1143KHJC103A1Pi8hQtzEEEHad1JDZoi0bep7/yn7y aT88iGoUCWBg15QY3UE4083I/S3urt7SZGT2LUh2BRQY3faeLb7gKBB6B+94/mgXcBF01uRFuv08 YdQyiLmNuzLgi/Owyuq3+cWOlT4OQVOoIzCRVE9l58f158TmMYVw== X-QQ-XMRINFO: NI4Ajvh11aEj8Xl/2s1/T8w= From: Zhao Zhili To: ffmpeg-devel@ffmpeg.org Date: Wed, 6 Nov 2024 20:31:28 +0800 X-OQ-MSGID: <20241106123128.17352-1-quinkblack@foxmail.com> X-Mailer: git-send-email 2.46.0 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 3/4] avcodec/mediacodec_wrapper: add async mode support X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Zhao Zhili Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: OkoOa5sOI0C8 From: Zhao Zhili --- libavcodec/mediacodec_wrapper.c | 130 ++++++++++++++++++++++++++++++++ libavcodec/mediacodec_wrapper.h | 28 +++++++ 2 files changed, 158 insertions(+) diff --git a/libavcodec/mediacodec_wrapper.c b/libavcodec/mediacodec_wrapper.c index 96c886666a..283bbe72d6 100644 --- a/libavcodec/mediacodec_wrapper.c +++ b/libavcodec/mediacodec_wrapper.c @@ -1762,6 +1762,14 @@ static int mediacodec_jni_signalEndOfInputStream(FFAMediaCodec *ctx) return 0; } +static int mediacodec_jni_setAsyncNotifyCallback(FFAMediaCodec *codec, + const FFAMediaCodecOnAsyncNotifyCallback *callback, + void *userdata) +{ + av_log(codec, AV_LOG_ERROR, "Doesn't support aync mode with JNI, please try ndk_codec=1\n"); + return AVERROR(ENOSYS); +} + static const FFAMediaFormat media_format_jni = { .class = &amediaformat_class, @@ -1821,6 +1829,7 @@ static const FFAMediaCodec media_codec_jni = { .getConfigureFlagEncode = mediacodec_jni_getConfigureFlagEncode, .cleanOutputBuffers = mediacodec_jni_cleanOutputBuffers, .signalEndOfInputStream = mediacodec_jni_signalEndOfInputStream, + .setAsyncNotifyCallback = mediacodec_jni_setAsyncNotifyCallback, }; typedef struct FFAMediaFormatNdk { @@ -1842,6 +1851,9 @@ typedef struct FFAMediaCodecNdk { AMediaCodec *impl; ANativeWindow *window; + FFAMediaCodecOnAsyncNotifyCallback async_cb; + void *async_userdata; + // Available since API level 28. media_status_t (*getName)(AMediaCodec*, char** out_name); void (*releaseName)(AMediaCodec*, char* name); @@ -1849,6 +1861,8 @@ typedef struct FFAMediaCodecNdk { // Available since API level 26. media_status_t (*setInputSurface)(AMediaCodec*, ANativeWindow *); media_status_t (*signalEndOfInputStream)(AMediaCodec *); + media_status_t (*setAsyncNotifyCallback)(AMediaCodec *, + struct AMediaCodecOnAsyncNotifyCallback callback, void *userdata); } FFAMediaCodecNdk; static const FFAMediaFormat media_format_ndk; @@ -1866,6 +1880,32 @@ static const AVClass amediacodec_ndk_class = { .version = LIBAVUTIL_VERSION_INT, }; +static int media_status_to_error(media_status_t status) +{ + switch (status) { + case AMEDIA_OK: + return 0; + case AMEDIACODEC_ERROR_INSUFFICIENT_RESOURCE: + return AVERROR(ENOMEM); + case AMEDIA_ERROR_MALFORMED: + return AVERROR_INVALIDDATA; + case AMEDIA_ERROR_UNSUPPORTED: + return AVERROR(ENOTSUP); + case AMEDIA_ERROR_INVALID_PARAMETER: + return AVERROR(EINVAL); + case AMEDIA_ERROR_INVALID_OPERATION: + return AVERROR(EOPNOTSUPP); + case AMEDIA_ERROR_END_OF_STREAM: + return AVERROR_EOF; + case AMEDIA_ERROR_IO: + return AVERROR(EIO); + case AMEDIA_ERROR_WOULD_BLOCK: + return AVERROR(EWOULDBLOCK); + default: + return AVERROR_EXTERNAL; + } +} + static FFAMediaFormat *mediaformat_ndk_create(AMediaFormat *impl) { FFAMediaFormatNdk *format = av_mallocz(sizeof(*format)); @@ -2060,6 +2100,7 @@ static inline FFAMediaCodec *ndk_codec_create(int method, const char *arg) { GET_SYMBOL(setInputSurface) GET_SYMBOL(signalEndOfInputStream) + GET_SYMBOL(setAsyncNotifyCallback) #undef GET_SYMBOL @@ -2335,6 +2376,94 @@ static int mediacodec_ndk_signalEndOfInputStream(FFAMediaCodec *ctx) return 0; } +static void mediacodec_ndk_onInputAvailable(AMediaCodec *impl, void *userdata, + int32_t index) +{ + FFAMediaCodecNdk *codec = userdata; + codec->async_cb.onAsyncInputAvailable((FFAMediaCodec *) codec, + codec->async_userdata, index); +} + +static void mediacodec_ndk_onOutputAvailable(AMediaCodec *impl, + void *userdata, + int32_t index, + AMediaCodecBufferInfo *buffer_info) +{ + FFAMediaCodecNdk *codec = userdata; + FFAMediaCodecBufferInfo info = { + .offset = buffer_info->offset, + .size = buffer_info->size, + .presentationTimeUs = buffer_info->presentationTimeUs, + .flags = buffer_info->flags, + }; + + codec->async_cb.onAsyncOutputAvailable(&codec->api, codec->async_userdata, + index, &info); +} + +static void mediacodec_ndk_onFormatChanged(AMediaCodec *impl, void *userdata, + AMediaFormat *format) +{ + FFAMediaCodecNdk *codec = userdata; + FFAMediaFormat *media_format = mediaformat_ndk_create(format); + if (!media_format) + return; + + codec->async_cb.onAsyncFormatChanged(&codec->api, codec->async_userdata, + media_format); + ff_AMediaFormat_delete(media_format); +} + +static void mediacodec_ndk_onError(AMediaCodec *impl, void *userdata, + media_status_t status, + int32_t actionCode, + const char *detail) +{ + FFAMediaCodecNdk *codec = userdata; + int error = media_status_to_error(status); + + codec->async_cb.onAsyncError(&codec->api, codec->async_userdata, error, + detail); +} + +static int mediacodec_ndk_setAsyncNotifyCallback(FFAMediaCodec *ctx, + const FFAMediaCodecOnAsyncNotifyCallback *callback, + void *userdata) +{ + FFAMediaCodecNdk *codec = (FFAMediaCodecNdk *)ctx; + struct AMediaCodecOnAsyncNotifyCallback cb = { + .onAsyncInputAvailable = mediacodec_ndk_onInputAvailable, + .onAsyncOutputAvailable = mediacodec_ndk_onOutputAvailable, + .onAsyncFormatChanged = mediacodec_ndk_onFormatChanged, + .onAsyncError = mediacodec_ndk_onError, + }; + media_status_t status; + + if (!codec->setAsyncNotifyCallback) { + av_log(codec, AV_LOG_ERROR, "setAsyncNotifyCallback unavailable\n"); + return AVERROR(ENOSYS); + } + + if (!callback || + !callback->onAsyncInputAvailable || + !callback->onAsyncOutputAvailable || + !callback->onAsyncFormatChanged || + !callback->onAsyncError) + return AVERROR(EINVAL); + + codec->async_cb = *callback; + codec->async_userdata = userdata; + + status = codec->setAsyncNotifyCallback(codec->impl, cb, codec); + if (status != AMEDIA_OK) { + av_log(codec, AV_LOG_ERROR, "setAsyncNotifyCallback failed, %d\n", + status); + return AVERROR_EXTERNAL; + } + + return 0; +} + static const FFAMediaFormat media_format_ndk = { .class = &amediaformat_ndk_class, @@ -2396,6 +2525,7 @@ static const FFAMediaCodec media_codec_ndk = { .getConfigureFlagEncode = mediacodec_ndk_getConfigureFlagEncode, .cleanOutputBuffers = mediacodec_ndk_cleanOutputBuffers, .signalEndOfInputStream = mediacodec_ndk_signalEndOfInputStream, + .setAsyncNotifyCallback = mediacodec_ndk_setAsyncNotifyCallback, }; FFAMediaFormat *ff_AMediaFormat_new(int ndk) diff --git a/libavcodec/mediacodec_wrapper.h b/libavcodec/mediacodec_wrapper.h index 11a4260497..18d0796445 100644 --- a/libavcodec/mediacodec_wrapper.h +++ b/libavcodec/mediacodec_wrapper.h @@ -178,6 +178,22 @@ struct FFAMediaCodecBufferInfo { typedef struct FFAMediaCodecBufferInfo FFAMediaCodecBufferInfo; typedef struct FFAMediaCodec FFAMediaCodec; + +typedef struct FFAMediaCodecOnAsyncNotifyCallback { + void (*onAsyncInputAvailable)(FFAMediaCodec *codec, void *userdata, + int32_t index); + + void (*onAsyncOutputAvailable)(FFAMediaCodec *codec, void *userdata, + int32_t index, + FFAMediaCodecBufferInfo *buffer_info); + + void (*onAsyncFormatChanged)(FFAMediaCodec *codec, void *userdata, + FFAMediaFormat *format); + + void (*onAsyncError)(FFAMediaCodec *codec, void *userdata, int error, + const char *detail); +} FFAMediaCodecOnAsyncNotifyCallback; + struct FFAMediaCodec { const AVClass *class; @@ -219,6 +235,11 @@ struct FFAMediaCodec { // For encoder with FFANativeWindow as input. int (*signalEndOfInputStream)(FFAMediaCodec *); + + // Introduced in Android API 28 + int (*setAsyncNotifyCallback)(FFAMediaCodec *codec, + const FFAMediaCodecOnAsyncNotifyCallback *callback, + void *userdata); }; static inline char *ff_AMediaCodec_getName(FFAMediaCodec *codec) @@ -343,6 +364,13 @@ static inline int ff_AMediaCodec_signalEndOfInputStream(FFAMediaCodec *codec) return codec->signalEndOfInputStream(codec); } +static inline int ff_AMediaCodec_setAsyncNotifyCallback(FFAMediaCodec *codec, + const FFAMediaCodecOnAsyncNotifyCallback *callback, + void *userdata) +{ + return codec->setAsyncNotifyCallback(codec, callback, userdata); +} + int ff_Build_SDK_INT(AVCodecContext *avctx); enum FFAMediaFormatColorRange {