From patchwork Mon Oct 24 03:16:47 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhao Zhili X-Patchwork-Id: 38939 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:4a86:b0:9d:28a3:170e with SMTP id fn6csp1997729pzb; Sun, 23 Oct 2022 20:17:55 -0700 (PDT) X-Google-Smtp-Source: AMsMyM5HuFgWGbdElfaoj1mngKiU70NI/2KeBh6QysiXgEX9D7znTFdzejv6ROXVKmQIEhTVuYY3 X-Received: by 2002:a05:6402:194d:b0:461:8982:52b8 with SMTP id f13-20020a056402194d00b00461898252b8mr7784400edz.49.1666581475663; Sun, 23 Oct 2022 20:17:55 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666581475; cv=none; d=google.com; s=arc-20160816; b=mGb5Kj82CbPitgNCcLEVfBc17GXk2aKMe6rLDXninhHlYBeEvvzo0Q9JR1G+AJ8FoD 58Z1WJEFPO45lm+auLA0n3tDA0nGuzjTaVbMf0iY5STdhq1wofzlrbT8bNsgqEOTPgAm 0Ou0ttE8GoiB9qIPpfFV/PKSaa/5AiGr3HcHRtwHkhe0QV4wEjnmSt6PolxCy5U8xcjA 2dxeKeSB84l5NuLukThpiW8eYBiAX7MlRKtEWH2mIPBQChUNPs9euNlQEEr10utha8YX sHN9MuRI4NsJtfV7Hek8MQ2hc7Okk9vcXPbC+H1t4XwLT83HEvi5qezYwmSvsdEIXkW6 //NQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to:date :to:from:message-id:dkim-signature:delivered-to; bh=9fqsUNTcGRYKSZ9USRpuhFM77tQOYVzDEGlSIUGc69Q=; b=FGEn0GQBfBqLvu+xwhyOk3lqhxdnU8fLM8W5Rb8vbjh9nJf26CTSyFYb2NE5avZMfw DdViVgo1bnBIP510jc3J3WwXbNxvBMl5D1Za1avOs4l5e9sL5fRsbiup9I7UtXuMjNI+ GaCcVqwLkiqBkNfNjcjuUU9uftmPim6lhsaXOd98/45eS2aNBughDSpk674T4kyGoKLP CKTqc9YYQOM1enq9MFQ8WlBwTrt3zkNGok1l/Zrfc+L3sb/0UlA6uwD03q4OliD9MIZa Y9voEwPXgzY0XM78ZZ9e0w9g05X66QaitWsgEmgtCgMSfV1csw5qxgSGIvjKLiNfa+K7 B4sA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@foxmail.com header.s=s201512 header.b=stcELmJG; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=foxmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id js4-20020a17090797c400b0078db70cc9b8si29038646ejc.606.2022.10.23.20.17.55; Sun, 23 Oct 2022 20:17:55 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@foxmail.com header.s=s201512 header.b=stcELmJG; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=foxmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 221DF68BBAF; Mon, 24 Oct 2022 06:17:29 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from out162-62-57-252.mail.qq.com (out162-62-57-252.mail.qq.com [162.62.57.252]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 4CB9F68BD0F for ; Mon, 24 Oct 2022 06:17:18 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=foxmail.com; s=s201512; t=1666581435; bh=Nk/ftF6Tg45G3EcdxHwzL07iX1oSf/CXMOpYtfIGhM8=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=stcELmJG190sxIxYhwXYvwyqIR3jhFo2shxLs3gXF5hvg01Nmv5yOI+T5pyqNgIoX KZ6K6+Z/OSzEGsdOT+Q1xS6CBLyDO0TMCjpz1Of82VCrXH47p/xE0NLFCSIRihVBFA EPzQQhCm9GQDQGgXN8ZZQL9daUn8olzBsBO+qeuM= Received: from localhost.localdomain ([59.37.125.102]) by newxmesmtplogicsvrsza31.qq.com (NewEsmtp) with SMTP id 44A14270; Mon, 24 Oct 2022 11:17:10 +0800 X-QQ-mid: xmsmtpt1666581431t1qki0mgf Message-ID: X-QQ-XMAILINFO: OZiGlmjmGvyhH3kI3GxFmDVD4v13C1Gsy2l8v4swa296wEr2I8P0vOEcXywBfZ PzvgoQ6ANYUs0JjXiMEqJTk90wW/NzGjJjCJ3xmfUbo9rjWtIMNHo+S35RYQ2QQj0VyLhNGblXAE M3vG9qcUb14+P3X9Nrrw34t00yhAprZE4keboNlZEGKBY9dJVKx4qmfEyaaoQ/LWDA60mThasCGH 2kzKO/43vq6ttvNbvWTzYr9APCATpuS7ioJv5IC/Rc5MuhjZI6qOQVkDzyHKTFAw+KW0x/NW2Mel e9i6oqdzbngoKEDc3F2iHb+/IYl7vNhNnClL6Ny65wElQ4WKgUDjFsrrRa6T6gVKKCuhYDfTXkJP IIgiQbZeYY0a52mP9hepR/CAxJaUSkkO/5D4bChxo0Z2M0LDk5Xs1J2EFURg3Px+4SFIXKAR1aHC DAZJ8PYQae8CcBgWh73mViJ0BkNV4O+OF1Y36Z01tc830sRB0KK0bY3Ti1cTi/z73KJsG0IneG+X r+PwtVP96S3pNm8KMBCWetJhyzZJyt8fYkHH6X/uk6xIkTzqDaz0A6ohwsg2ipqga7qkDXgGrZfw tom6Hwwhh2uiiYyZ5AqkkSluO1lkP9DWme1LxVIcEdqM+/VMzvm2IbJs5bCo95pREL8B64htql1i dxs4xILIaw07sjPut3PHBYjdBjMhmTbVVeaRZoWQOzy54bZ0aAIb4faXRGpaTqZEr0S78M463Cbv FLWwBUj5RIyxGZPJ1/WgqLseedRiwXaKTIOdk9E2mIB7EIuEf7ezdlnooTezuHJ/mvaXfJ8FJTUP G9zvbsMjRDtM/eDBSS2oM747OyPPXa2RdqxuMEg4mQKSRkaNGx45t5EWIto9AFE0LDo2ycpLjSXz Zh+/Kau1KO/TmB3oZSonfHMYik0zNYBfvyMVSwtMUdqhpqfNYBkYGjj+OSvTwr7g95vARYWbhoGj MBbSUA/EmUpnkLrSH0pl7gOJrsIx0r From: Zhao Zhili To: ffmpeg-devel@ffmpeg.org Date: Mon, 24 Oct 2022 11:16:47 +0800 X-OQ-MSGID: <20221024031658.1026118-2-quinkblack@foxmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221024031658.1026118-1-quinkblack@foxmail.com> References: <20221024031658.1026118-1-quinkblack@foxmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 01/12] avcodec/mediacodec: fix incorrect crop info X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: matthieu.bouron@stupeflix.com, Zhao Zhili , aman@tmm1.net Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: dALLiDi1Yeou From: Zhao Zhili The crop info is optional, but used unconditionally. Signed-off-by: Zhao Zhili --- libavcodec/mediacodecdec_common.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/libavcodec/mediacodecdec_common.c b/libavcodec/mediacodecdec_common.c index 2a605e7f5b..c1cbb28488 100644 --- a/libavcodec/mediacodecdec_common.c +++ b/libavcodec/mediacodecdec_common.c @@ -487,6 +487,11 @@ static int mediacodec_dec_parse_format(AVCodecContext *avctx, MediaCodecDecConte AMEDIAFORMAT_GET_INT32(s->crop_left, "crop-left", 0); AMEDIAFORMAT_GET_INT32(s->crop_right, "crop-right", 0); + if (s->crop_bottom == 0 || s->crop_right == 0) { + s->crop_top = s->crop_left = 0; + s->crop_right = s->width - 1; + s->crop_bottom = s->height - 1; + } width = s->crop_right + 1 - s->crop_left; height = s->crop_bottom + 1 - s->crop_top; From patchwork Mon Oct 24 03:16:48 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhao Zhili X-Patchwork-Id: 38937 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:4a86:b0:9d:28a3:170e with SMTP id fn6csp1997630pzb; Sun, 23 Oct 2022 20:17:38 -0700 (PDT) X-Google-Smtp-Source: AMsMyM5gxvi535TPo4L7UOohx3fOwwtc9efGMEHuO1CMdvuq0c3QABJAjxNn0nBhQFYTaxLoKDe2 X-Received: by 2002:a17:906:5d11:b0:78d:a41c:6a2d with SMTP id g17-20020a1709065d1100b0078da41c6a2dmr25366693ejt.215.1666581458658; Sun, 23 Oct 2022 20:17:38 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666581458; cv=none; d=google.com; s=arc-20160816; b=Rrs6A3smPB8u78th9fYArYU5DxQvX8XmIo1voP4R5DhWKmqLN45pjyhZxkd7OP7/1u sppRjlTpEMLjHY2oPLTPjxfxOdTSnFADxGLbg198qAHmLsFfJ0N8RNNNH+UlgH/lyQIw fVwQjsAQ7YuYahhnuxLXoLvNWB4xH6wfSI9iYbT3djRyErmqaa6nvAW5SS80xwwDmrl+ Y07viYK1xwuemCIskRLT6N/K5lL6eStO1cTKddFg+xm16owmufT1o0DROy/KqUAsJBeG Tn413EBdK/vHLcSmjUDBVfoNQ2tE8CjOu06Fzvm406Ol23T48O73t+gFyloKU4c8IFAa +pnQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to:date :to:from:message-id:dkim-signature:delivered-to; bh=D9pA4qF84TdVIbHxJaM9jTyL4eBlqxwpDkVuwY2Vibg=; b=mkOT1PIHH9uyFdIBPns9jrzy1A6lQBXAPS60pSAXbfFShGGZuD+h3onXWnVtNIfgN3 OmZ5HnC047d884csvFNa8bbiUXuNHSCPFx2KkzmN5sJa6K7TasRcKONYdt9c/wA2KF00 FRgG0Oy4eV2k3zHS2PLotf6v5KQ5cJIn3hmbulbzqrF+UL0lIbxdQ1ZjRsmSnONTtPZJ jxizJ9INghBlzsr2qPRL0pUueg9hzoCEXQYKPtzo8tzNio10uNeKNnKdIStA76OeLjN/ 0eYnlK39Q2DrGhzjiYj5tF/2mSU5JYmzGtdJ8NE3PJA1vsGaQ4VN+Ha/3/86Sr5SqYqt Tbvw== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@foxmail.com header.s=s201512 header.b=rxH+7EVy; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=foxmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id cw10-20020a170906478a00b007314b427e96si32305641ejc.633.2022.10.23.20.17.38; Sun, 23 Oct 2022 20:17:38 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@foxmail.com header.s=s201512 header.b=rxH+7EVy; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=foxmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 0617868BD0F; Mon, 24 Oct 2022 06:17:25 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from out162-62-57-210.mail.qq.com (out162-62-57-210.mail.qq.com [162.62.57.210]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 47E0868BB69 for ; Mon, 24 Oct 2022 06:17:18 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=foxmail.com; s=s201512; t=1666581435; bh=U+ZFDDSMx9NZbmIQBcSMvX6hrw2B6jNvxsIWP4P86hs=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=rxH+7EVy27a1S6KWw1IOlDKxuyaffGJ9T3IqG+nAkJXpH2KSUyCdSVQgKwWwCTeQ1 WGBlyZ+vcfMfsrCSOU/CrxzS86pMAPkwpA+mqY8jsO3CmAVqmNaaMWHLSZ4jtSlN4n 6qUoChI7CofltmPa30RQcgXtN4tZ2kHLu2si3h58= Received: from localhost.localdomain ([59.37.125.102]) by newxmesmtplogicsvrsza31.qq.com (NewEsmtp) with SMTP id 44A14270; Mon, 24 Oct 2022 11:17:10 +0800 X-QQ-mid: xmsmtpt1666581434ta25adzwi Message-ID: X-QQ-XMAILINFO: Mrv6PNPZjcp66JUX8FFyYklFuzJnOCED/BzKyPTWaVIKJbwC4hlJKoyo1siZ2v SfwokayrgMCkNK12JOmJHnCb1avz4F7AhCSS9g32zE2Y8oqUtA/HgzByQO98uXf70DfYMN4r+V5P luJzolbu1DAVdaVEr4Stre1T0uRoYvUcQAFjjyStb4A3NhBPzWJ/WUDCOMv1/+k1wsyZvFROoYIZ 7j4Bd1AZrcqzywFJ38AQq6UePT0JENsf27vq+d3eU0ASIIefzYhfCX8e5SVzhCIgt/1zOAejpHvb HfiErFMCj/ZwpSxdp/pESvWBvZKO7PsLySa8Q9zWHg9o/y7TFypZnZHvgE2e3xHchL03tSG3W0YU rhGc+Ud+0fOHLDDaIGtGZuzO6z+8M+TXdh23/dSjAV43d0c9Behib9f3jkXCi6ioFkotOOfv+aXt Uclmh1vDrughH2voXa6kEtMxliy0sSJrgfWWCw4lyWcVUj8padzJTCIQBfNSlXSfQ2DdrDFymPBv BdLIrncbOuKj7YGrd5whz78VdGJUhno3iZlGNL6SS7rxiYJc3D8Jq120r4+zTI5U49SQDfnN4HEx uFpzjqUu59/2Nh9fxg6nPDnViAzbesRqzp9sU9ZBQaHkBzxbY+hl5eAq7Rlh80iNAqy3C721CdG/ c1ng52kRfycnJqmYZoKZJSNGDXQ6Z8tGdMYAZpSH0nsPGg/7EML1F8BbVuv+TeqRJFBj0R9sSXZk BOJfFhqm0yLHdhbmTKApvIDyCtTt8uaeCdbNAniyTg3vWh5YLucVQewllk1U8+/v8O1FyIGnKpFv uYiQe20bwZLdEiYTJJF8FV9TDKr8eLXxq0AKWcnS4V2sJcoe/PK+eyBmiyTJBBVkXw4xIDEmu4M9 ZKyMZF+crjj1jPj+Mg+cMUJ2mosjq/RU/7FsMluY9Kjua6XmS5OeqPaFsj5lm/+cMcEmwON+SDo/ tjb+m6PHSNuejL+vOZHtpYDJxFfng4L4Y8v6HQ9ezadshSypcSlTyxywRna5JN From: Zhao Zhili To: ffmpeg-devel@ffmpeg.org Date: Mon, 24 Oct 2022 11:16:48 +0800 X-OQ-MSGID: <20221024031658.1026118-3-quinkblack@foxmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221024031658.1026118-1-quinkblack@foxmail.com> References: <20221024031658.1026118-1-quinkblack@foxmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 02/12] avcodec/mediacodecdec: don't break out if both input and output port return try again X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: matthieu.bouron@stupeflix.com, Zhao Zhili , aman@tmm1.net Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: wVZG8ADqAlWc From: Zhao Zhili At the beginning of decoding, if we feed mediacodec too fast, the input port will return try again. It takes some time for mediacodec to consume bitstream and output frame. So the output port also return try again. It possible that mediacodec_receive_frame doesn't consume any AVPacket and no AVFrame is output. Then both avcodec_send_packet() and avcodec_receive_frame() return EAGAIN, which shouldn't happen. This bug can be produced with decoding benchmark on Pixel 3. Signed-off-by: Zhao Zhili --- libavcodec/mediacodecdec.c | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/libavcodec/mediacodecdec.c b/libavcodec/mediacodecdec.c index 322b448d27..2c66f38541 100644 --- a/libavcodec/mediacodecdec.c +++ b/libavcodec/mediacodecdec.c @@ -444,7 +444,16 @@ static int mediacodec_receive_frame(AVCodecContext *avctx, AVFrame *frame) index = ff_AMediaCodec_dequeueInputBuffer(s->ctx->codec, 0); if (index < 0) { /* no space, block for an output frame to appear */ - return ff_mediacodec_dec_receive(avctx, s->ctx, frame, true); + ret = ff_mediacodec_dec_receive(avctx, s->ctx, frame, true); + /* Try again if both input port and output port return EAGAIN. + * If no data is consumed and no frame in output, it can make + * both avcodec_send_packet() and avcodec_receive_frame() + * return EAGAIN, which violate the design. + */ + if (ff_AMediaCodec_infoTryAgainLater(s->ctx->codec, index) && + ret == AVERROR(EAGAIN)) + continue; + return ret; } s->ctx->current_input_buffer = index; } From patchwork Mon Oct 24 03:16:49 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhao Zhili X-Patchwork-Id: 38938 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:4a86:b0:9d:28a3:170e with SMTP id fn6csp1997678pzb; Sun, 23 Oct 2022 20:17:47 -0700 (PDT) X-Google-Smtp-Source: AMsMyM7+61pdlKsRxRT9GoK+FHbCC+z7iSFPzPVEW02qgZYCAT3J4w0gEtN0hcu72beVKAqywvu1 X-Received: by 2002:a17:907:80d:b0:73d:a576:dfbd with SMTP id wv13-20020a170907080d00b0073da576dfbdmr25552752ejb.402.1666581466908; Sun, 23 Oct 2022 20:17:46 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666581466; cv=none; d=google.com; s=arc-20160816; b=iFqQjb5NebSwKwJmzyytRvxFYOF/8Sp8KNSnP7XsFbZVUfVe0rTeV+bdzifJ3UTq8B dt5jjsdir6PPMDQDKLUDkJj1FUX1sUp1wJQ8bm9iYSRhy20a0NIjq+Zv33kjuoOTejxZ iBx97dNy6u/or0Maco7oYB1nMcrfFcdSPnrhbBCLh6XTYtwyxRsZPMWQuTCvCohuoZi0 M+zMsnDRPUso20mQsf2fKymXzrl2hEveSC6XOgv+unKiEY/YeQTp7a3/oNji5DYiZ5u1 XTdq/6oGgp31SfTdZSWMU/GJe6Q7vD1NQ7TQ5NyKm1teHH8xqssrw3yJ5YouCeI7TjHV 4Gog== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to:date :to:from:message-id:dkim-signature:delivered-to; bh=BVcjrk34zxcTCuOKwqpfi2fg3TMTXILdwtQlS2E/xvE=; b=WwYrRfh0P6NFPSrVJ58InG0psoBxqWtQ2hXtBXBYacsptmfEORS2o24hkHuDHO+RC0 uO0nJVCUo2Wgiug7oZhbsMRyDEnFE1InTa/vYanmU6+sAcKgyRco07+fnguWxBKM7XoN mpiCKJlPRLFzTtPG9IxBY3qEJvmsgVrwArZyYJPeeKYeyC+2my/TGHwYZ42iNmXPWYW1 e+7f2YkMZwNyagwKlTvYl5dfW0myi2x4HpKsTtfo6Fp8H7ytl96EpYQmInAI/G7NsP+3 2csKT1zmKSzIs2TBPhVnxkNcUVCwnbWpmpFhKgWObwWsfnp0x6OZMOPc9x+OwCizvKRX cuWQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@foxmail.com header.s=s201512 header.b=dnij1S9Y; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=foxmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id v15-20020a056402174f00b004615ccd71a0si6593154edx.162.2022.10.23.20.17.46; Sun, 23 Oct 2022 20:17:46 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@foxmail.com header.s=s201512 header.b=dnij1S9Y; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=foxmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 1E7CD68BD2B; Mon, 24 Oct 2022 06:17:28 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from out162-62-58-211.mail.qq.com (out162-62-58-211.mail.qq.com [162.62.58.211]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 2772A68BB69 for ; Mon, 24 Oct 2022 06:17:18 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=foxmail.com; s=s201512; t=1666581436; bh=ZttxXjfD2IR15JRHIr2EYrd2X1ETDztr+V4dKhSVKGI=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=dnij1S9Y9gUQDE3wb43fQnK258RarGJjwKX6GexmLlYEQJObawgAzR8dSpa4zzmXo g2XACC8FNkdUve3VOscBmwmIq7TVN2aydNMuT/HzQuRtJxjZdFMtu7geuxg8R+x/vk DCBevjkby2Y0R2Arx6nx5UUUI3s9KGhEzdEWDOG0= Received: from localhost.localdomain ([59.37.125.102]) by newxmesmtplogicsvrsza31.qq.com (NewEsmtp) with SMTP id 44A14270; Mon, 24 Oct 2022 11:17:10 +0800 X-QQ-mid: xmsmtpt1666581435te65ziqpr Message-ID: X-QQ-XMAILINFO: NMGzQWUSIfvTP3OuA2zUMR1mjf3n+c3gi8rw5t97cHJJ1g/KflXpJrDy8ImKUS RlQZTomiS8WnMvPZ5p8p7tvAZxJ/PURqYUYhT6trcKpHyGncRJlj//bzpB6QlB9FKTe07t2D2qkV N7WP5IRQryy9WBaOypwcYAf70XoqFFUyYfTtjWGoW+iEaUU+uVrUgMnEedqwg45qG54qZ4uE6e+N Pc6RiTmPc28JcU2czQmGBZkUQCLKFewWAy93fGgH17iLD3UZ4Isk6JJshVLiv+/7ISwz/nK/3KHU AieeCsJ6DxhYiZx7A2w2PnCuJjsiDnCf/iuuH9SUuI1Hitkt+sfW17+iCPGIhIm7XljVuiaKH9/q lE4gGK/eG4Ehp8i2VSdcG4Rm54Eg5b1lY4q9ut4z0Wi2hQxbDiNmh59bjmkh9We22P7qImeaUUJb p818QEd7MUuowQ7M68XyRN44K2Sl0iyRVoZh4kTIK9dgva083tBQ1akb0Yc/sCJcuq9H035Yd6fx m7iReVl6bA5aA+8WZmw7AgGJsQQ1fzCKWShKxvKRHjvE9aPfJrVhq2mpRf1EnpcqCsUv+IU6UFb3 hSxeLe4zxO2FNdRXYZyLvptHQmeO0SUIA+87+xGNs6JgUuk42Sj8VruiP5YoaPg/2RvwguAbDXNu FmUj5CLkHqoFZFIHOUvZdbmBP95VfIOI2aG6quw+RW08c20Zilc6oITFjbZ7J36MgypGMx2z1m9X hFrFMW18aGJq3vdprO+a2Dy3zVVwqHOEwnbW9CzM45zRiwPhJDrNrQbybHqJk01Z1BDxBj4EJWTe SBjA1FDSkme7yM0CTPM5nit9fUK4CPbYQLguwqm633v3j4hcfJ+xJNeXw3n8zs4LP7e2xFQrI7Yi MdHRV0r25bs3k6zs7kQrcUtkbAoJXyV38Q1jIS7dYGR0XovSTYZyK4QpH7WJpM7sE2qHVDW8rgPV rsoDx6HceJOhsJL1XMQLO01iY+ehXxLfMLyYn5Y+J5kREHp0/jKGbiF+IgL4ez From: Zhao Zhili To: ffmpeg-devel@ffmpeg.org Date: Mon, 24 Oct 2022 11:16:49 +0800 X-OQ-MSGID: <20221024031658.1026118-4-quinkblack@foxmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221024031658.1026118-1-quinkblack@foxmail.com> References: <20221024031658.1026118-1-quinkblack@foxmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 03/12] avcodec/mediacodecdec_common: fix misuse av_free/av_freep X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: matthieu.bouron@stupeflix.com, Zhao Zhili , aman@tmm1.net Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: spPwkVtVr60+ From: Zhao Zhili Signed-off-by: Zhao Zhili --- libavcodec/mediacodecdec_common.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/libavcodec/mediacodecdec_common.c b/libavcodec/mediacodecdec_common.c index c1cbb28488..940b4e02d5 100644 --- a/libavcodec/mediacodecdec_common.c +++ b/libavcodec/mediacodecdec_common.c @@ -334,7 +334,7 @@ static int mediacodec_wrap_hw_buffer(AVCodecContext *avctx, return 0; fail: - av_freep(buffer); + av_freep(&buffer); av_buffer_unref(&frame->buf[0]); status = ff_AMediaCodec_releaseOutputBuffer(s->codec, index, 0); if (status < 0) { From patchwork Mon Oct 24 03:16:50 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhao Zhili X-Patchwork-Id: 38940 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:4a86:b0:9d:28a3:170e with SMTP id fn6csp1997786pzb; Sun, 23 Oct 2022 20:18:04 -0700 (PDT) X-Google-Smtp-Source: AMsMyM7pHrO5O4a7XF4dCfOfaYbUOrhW640YZjavB7Kr+DhibW9UsYn/d/YIIknc5+IhZlMXa/Py X-Received: by 2002:aa7:dc06:0:b0:461:b683:5fba with SMTP id b6-20020aa7dc06000000b00461b6835fbamr4165674edu.72.1666581484591; Sun, 23 Oct 2022 20:18:04 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666581484; cv=none; d=google.com; s=arc-20160816; b=oF9KJGt8xKV3s0xdZyYTxoIA5sspCsnKaW5dab89ZucjGuB7RcZHxHPSaSK8fImXK8 d706kTV2JkLVYUkwc5+aWAB0FPyzYh8ereLUoBwyoQkBFWsV+d6yNUmHMykjLxhoDb0o 79uDKjCGnvcVKDp3Had9qTCrrkOrQN+kSEE62DKMr0pWgxHCOc34NsMJLQnjboglTuSt SAiGHnFFzJMkJ57G3anGxEgUzL0ZyYhq5NSxKoZZEcNDyb+IePNyHRq9HW37zRFthnKq 9U7XuRZsijuw0qfz7wiIr1ucP7UPFaLXJvU3/Z381IH5nJaxh1SqO7Gk/1+vHjsr3d1D +GLQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to:date :to:from:message-id:dkim-signature:delivered-to; bh=ZCIsuZXAwpUxCBB8equlyeCOyasnD/zzNDSBWqqvlE4=; b=Ll9LDB2LbDoMgPrcYGeE4wY0jJIOkxdNG0AguDVAulxSHL9bAoNoY9adKBg4frL/pw 7HJEtGu3RST/IxVEm0Klg+zlbFZHN7C/HRtw67P8Uhz3YeRlM9VLuBtKA+6f00JS+N7E 0Os0T1/Hd8VUKUB5KAC9rT9PooTqd/a/13iAaUgFcvo7ABjnvGWkJmXGBlZMbxK9U6Xi NHQ8qn3B6EkyQwd6UqaEbc+BkwQtkzgzXQKuyAacMP1jDzX/PsjrTvbG+r/h5Ld6EnxD DL5MQBkgrtH4/Piw1zII5/yVeMe8kkDqhQVgeHK5OgCmYpgtu9twJljKkgZu1A3uQKUR 7lFQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@foxmail.com header.s=s201512 header.b="Lasr/X6t"; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=foxmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id n5-20020a170906700500b0077fc66b581esi21316788ejj.688.2022.10.23.20.18.03; Sun, 23 Oct 2022 20:18:04 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@foxmail.com header.s=s201512 header.b="Lasr/X6t"; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=foxmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 1511D68BB88; Mon, 24 Oct 2022 06:17:30 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from out162-62-57-49.mail.qq.com (out162-62-57-49.mail.qq.com [162.62.57.49]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 1CC3368BBAF for ; Mon, 24 Oct 2022 06:17:20 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=foxmail.com; s=s201512; t=1666581438; bh=zbi3i38HK7poIMQV1NZti2antgNBIWsM3d5p+oMlt10=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=Lasr/X6tEOplXZxeu/IMiyDY/ab7Z0olCLFMqnwOC5vs9o/n91n7dRSt3/0U6Ij7V htaQeAvuW4YhN8YoQmTHjMOOkwUTrcFZLNxCiXnabx603/LhWPPYFE5KeI0JOvUCd0 6KcUKN0Ntr+kxlAdlIBaAd2MyG1c5sywdfi0lxF8= Received: from localhost.localdomain ([59.37.125.102]) by newxmesmtplogicsvrsza31.qq.com (NewEsmtp) with SMTP id 44A14270; Mon, 24 Oct 2022 11:17:10 +0800 X-QQ-mid: xmsmtpt1666581436tli4h3e64 Message-ID: X-QQ-XMAILINFO: OKkKo7I1HxIeXAfR8K6M/P3vsbJUCrBqPiClHUaV/2n6XhVVkZ+chBj3hWMEcI BfwjBzSnyuoqLet56MnEyRV3uOnxHx1/lPeaa9bS1XnjWPoFNm5ELUgEzsV9Ccf5CTdpIFqV5JOW z76P7Qr342o88VuVsIB0sxHv1AYTqlOcSDfXOyaGZp1U1vcs8l30NbvX6YcoRXoBFXcx16Jxx+3f pW4dGNQaQa+XSKwhFQ5II7/b/e4G1niFQ/unmxdkTir4w1igtCc/QLJ8+Tz+tN1ogmBjs72XdXCQ 2FtkyNQWDApwQVQrdV+ajobngONyy8NqLRaxYMo4sPXiyXKWZwefx5Nw+LDjD8AIsWtJM2LI2xOA ldyLSiu4wqMyTsqcwD3qBtyADzcZuXh9FCqk3ylGU2hpmnpDv2MF4pB8UfL8nCm6YwTUr+JPSrZe p3SKJ4b8q97MqD2zogz1Xi0nu+MbzkoL+HeF6NNH7KGe71H7PNep/kF/YL4JiqJ6t7j4lpHWe/3O HQ7jGHrSwWId6O4dThr8NReUDanPTnz26anNFARA82rIdf9DJUaUYHuPiz/O526XaAvUqrPIY2f6 IPD4fS6JmLnNpDGJoUyEelxvy4mFYJdgb0Oliauc5Ej7mZshx7m8vkG+T8ojE5qJ2pJ/9NFym/z0 8Q3YVC+o0k8FoARxa6+qxIu2WAT0F+LC4yO/qMYZpCpF0+k81ALNIjeACx89TsdaAWQqW4/dE04N cTrcNVA6ccRg9CM/XnM6t0wkbGNKs9ovfCclRtT6+b6F+yCn3kPQIEI23Hfge8wpayi17eA2daxo zCTv0a4VJ4UGzrezeqlca1jvfu3DzNTScml4/Uospf5AXqCY5QIlp3IOcmtHMn5rpgKUZRwc5M4M KvVfakxgGt6+BIHQBJGaK4X3oY62z2K0/X9HwhcKi4reTLUtsFuT8EJ5Xz1+L66BKheCRPxkVHyL fVOwSy22y19Nrniqms7ITM5UiEf+vws28or5+nLng= From: Zhao Zhili To: ffmpeg-devel@ffmpeg.org Date: Mon, 24 Oct 2022 11:16:50 +0800 X-OQ-MSGID: <20221024031658.1026118-5-quinkblack@foxmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221024031658.1026118-1-quinkblack@foxmail.com> References: <20221024031658.1026118-1-quinkblack@foxmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 04/12] avcodec/mediacodecdec_common: fix useless av_buffer_unref X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: matthieu.bouron@stupeflix.com, Zhao Zhili , aman@tmm1.net Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: G/azf4Jr03FZ From: Zhao Zhili Since frame->buf[0] is always NULL in this case, av_buffer_unref has no effect. If it's not NULL, double-free will happen. Signed-off-by: Zhao Zhili --- libavcodec/mediacodecdec_common.c | 1 - 1 file changed, 1 deletion(-) diff --git a/libavcodec/mediacodecdec_common.c b/libavcodec/mediacodecdec_common.c index 940b4e02d5..d6ce709dd8 100644 --- a/libavcodec/mediacodecdec_common.c +++ b/libavcodec/mediacodecdec_common.c @@ -335,7 +335,6 @@ static int mediacodec_wrap_hw_buffer(AVCodecContext *avctx, return 0; fail: av_freep(&buffer); - av_buffer_unref(&frame->buf[0]); status = ff_AMediaCodec_releaseOutputBuffer(s->codec, index, 0); if (status < 0) { av_log(avctx, AV_LOG_ERROR, "Failed to release output buffer\n"); From patchwork Mon Oct 24 03:16:51 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhao Zhili X-Patchwork-Id: 38945 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:4a86:b0:9d:28a3:170e with SMTP id fn6csp1998374pzb; Sun, 23 Oct 2022 20:19:46 -0700 (PDT) X-Google-Smtp-Source: AMsMyM7zrY7gcimlSAb0XXNw7bIh0wWIRmwLBfLp3BJ40Vna3lIqlBa6FLJfO6e3C97kkby6zLX1 X-Received: by 2002:a17:906:cc0f:b0:7a0:b91c:855f with SMTP id ml15-20020a170906cc0f00b007a0b91c855fmr8872328ejb.26.1666581586604; Sun, 23 Oct 2022 20:19:46 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666581586; cv=none; d=google.com; s=arc-20160816; b=VDgNFo7I2cbby1VPQZcTksiqj1Dfm8lKIR+FDdPrtkYen/sfJkmUdGm8SqgBj4Bx4G 157OXSHzNLWtsHjxkVRb/PwaSfqB0RnQ3G4LLaE+Z8vY0rawEp5MzpdOUmSITVizjBV1 TEGAUPLCz3J1aprQnCQ4BOK+UwfTEs+TgC2Hm0md09PIZJYShpb7O4a/s4T0BHfw4+Cj uxSdv6YO4JxaSKFwEZiOcpRQITajXARapKLPCCjF/V2y16JZV5O8jBjc/l5ND/fbxuQZ lQCLboAjW8sAmdHJ9op3XGaQmDSbgjO9s1iV8TM75aCbaCO3JfdvP499yGzIB/Z4tiF0 3LBw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to:date :to:from:message-id:dkim-signature:delivered-to; bh=ifySuZb+cijYNaQBsrh6fWsJKongNm7crU+1pa0mqiE=; b=IlOCy4jEKbDW4VSlhVFuymRsDSHdCWE7Y1IBLIT+Y2g2DJjBjZ3c7Pz1hixopGPQ6k SnMOmcKAnuFmQuf28w6YDMisAF9oF+QtwqCF7HbjXWDWMxo9iCMhrRE3rsIg9Fe/BZs8 WGoQZpGAG2HdwjHnQy5sHMs/3+Z6g3yGOTHfGUXZb5AIwjrR8vf1WxIC3CQsvtDbnnww HLEUA3O6kuSkBv5J6t3hRsMlLz3+fXXB7l6kFpe4frx+afFtBupPxlHoH3llomQWyTVS VQFY+MG1s9kmvNFjGaIpLqzuyWbwmuwbx7sYcVJltiialTqymvW7vno8YqfNge33R1cb qTdA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@foxmail.com header.s=s201512 header.b="jN8K/88T"; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=foxmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id i8-20020a05640242c800b0045945eed10asi30634002edc.5.2022.10.23.20.19.46; Sun, 23 Oct 2022 20:19:46 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@foxmail.com header.s=s201512 header.b="jN8K/88T"; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=foxmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 38C1168BDA7; Mon, 24 Oct 2022 06:19:38 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from out203-205-221-205.mail.qq.com (out203-205-221-205.mail.qq.com [203.205.221.205]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 3BF9568BD2D for ; Mon, 24 Oct 2022 06:17:23 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=foxmail.com; s=s201512; t=1666581440; bh=5WT92i5iG9ElscGVw4V6WlitzhgIJkqZg0r2ou3Qick=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=jN8K/88Tcw5BWoiD7I87QQKWWR0Qvdh+WCTUUB6yCfDwkRqrOMgYj2nYFp35DPgc3 NQo8/RxCzjEroVZTEA1clFss0YUcEO8SrC5UR3AntY+cTR1EKuTPJA4tf+fg/IAN3L GKK63ueFS+BW3QrEE/6LHtvc+QUjeh5tKdUpDN00= Received: from localhost.localdomain ([59.37.125.102]) by newxmesmtplogicsvrsza31.qq.com (NewEsmtp) with SMTP id 44A14270; Mon, 24 Oct 2022 11:17:10 +0800 X-QQ-mid: xmsmtpt1666581437t4yhbnlc2 Message-ID: X-QQ-XMAILINFO: M/NR0wiIuy70K/iGIwS7Dp1Kbu7Oc6nEdvuVr/nl8LLE02e+YQFYj6/Tx2KJYQ UssFaMDxrCHKGnlYriKCkrxsmUw3lLF9vAZG9GH3XKC92IBAKMcrHaXEI+dmCFy9b3JP0XXiUfM/ 38WLBa1nyYXYIx3oTZWpE2KVkeTfxVOAXmE7vsPmT7J6cj91QLJ81lAbqXEi9kABHS/F954ARKmU 6no9QWOV8kKQLSXiOk9Kx3vd964BT/Scf8FmXDr0otZD8edU139AE7Qzl4XDM9y26wpq4R5dcjTD /TWcSivhfoKVolZFLyFW8GoF6caHI7DR4hFDqqZFLqcQdCSh200DqfNLE3Qojo5EwddsywC73crE Nax5VDoltrzUxDVnONx80eBnK/1AuwbGzK5wvGa6fSVI0pmt+23EYj0yCnXb02fsXnq1pIChfOzy yX/HO4Jj4Hk3+Nhg1Pa1tur91oijWHwTFa/2aDpHcOmzvPSjzecsAeceqIEVFc1tjra5ZVcx7hqQ t4SEZDK3e9DoB/8rTRkY7Tz5Mz8S2AncD//J+Can6fN2hov8x7Blm0PAg5EYKoREPnxEUuUNctt0 iyqFFBDg9bLCSxxNo0Qx6tCeLcpgdhm8048eyvg6J3ZlfJSY3ElYeFDHIOBzgwW4l5ZiHtC4oe5N Li4IYksXzRSCHk3w8ePAwTo30O8Tox/WwkIsguOSoCPjK68ARMw/Jph6TjLK/XEMDzgirG/4wQ56 N71r0ZSZ6yKzrCUfbtdp4MhHHGUWsA5xgjQm/hNXj+M/eKsT6+egSomQ7w6TVMCwOU0iGB+cGj2l tAMNrF/rSC2Xke14chmmizK6GY10cTCOmfqsuB2Bhg83ssfMYzBwdVLeqEwPs1Xj7UBeK/xvUSW7 4U5zQZCLPfNEP7iqsd+V/dNinIsV2RfytU6DBMK0PRxOPXFXTjkrjgAuypRNCSrGCFxkFc3Th8jD AtxcoOEcLhXUEDlaGusW97GN/K7iPQ From: Zhao Zhili To: ffmpeg-devel@ffmpeg.org Date: Mon, 24 Oct 2022 11:16:51 +0800 X-OQ-MSGID: <20221024031658.1026118-6-quinkblack@foxmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221024031658.1026118-1-quinkblack@foxmail.com> References: <20221024031658.1026118-1-quinkblack@foxmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 05/12] avcodec/mediacodec_wrapper: separate implementation from interface X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: matthieu.bouron@stupeflix.com, Zhao Zhili , aman@tmm1.net Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 75U44Yc1Wv1T From: Zhao Zhili This is in preparation for NDK media codec wrapper. Signed-off-by: Zhao Zhili --- libavcodec/mediacodec_wrapper.c | 244 +++++++++++++++++++++--------- libavcodec/mediacodec_wrapper.h | 255 +++++++++++++++++++++++++++----- 2 files changed, 394 insertions(+), 105 deletions(-) diff --git a/libavcodec/mediacodec_wrapper.c b/libavcodec/mediacodec_wrapper.c index 8ffc58e1d8..e0c614680e 100644 --- a/libavcodec/mediacodec_wrapper.c +++ b/libavcodec/mediacodec_wrapper.c @@ -160,12 +160,14 @@ static const AVClass amediaformat_class = { .version = LIBAVUTIL_VERSION_INT, }; -struct FFAMediaFormat { +typedef struct FFAMediaFormatJni { + FFAMediaFormat api; - const AVClass *class; struct JNIAMediaFormatFields jfields; jobject object; -}; +} FFAMediaFormatJni; + +static const FFAMediaFormat media_format_jni; struct JNIAMediaCodecFields { @@ -272,9 +274,8 @@ static const AVClass amediacodec_class = { .version = LIBAVUTIL_VERSION_INT, }; -struct FFAMediaCodec { - - const AVClass *class; +typedef struct FFAMediaCodecJni { + FFAMediaCodec api; struct JNIAMediaCodecFields jfields; @@ -295,7 +296,9 @@ struct FFAMediaCodec { int CONFIGURE_FLAG_ENCODE; int has_get_i_o_buffer; -}; +} FFAMediaCodecJni; + +static const FFAMediaCodec media_codec_jni; #define JNI_GET_ENV_OR_RETURN(env, log_ctx, ret) do { \ (env) = ff_jni_get_env(log_ctx); \ @@ -622,17 +625,17 @@ done: return name; } -FFAMediaFormat *ff_AMediaFormat_new(void) +static FFAMediaFormat *mediaformat_jni_new(void) { JNIEnv *env = NULL; - FFAMediaFormat *format = NULL; + FFAMediaFormatJni *format = NULL; jobject object = NULL; - format = av_mallocz(sizeof(FFAMediaFormat)); + format = av_mallocz(sizeof(*format)); if (!format) { return NULL; } - format->class = &amediaformat_class; + format->api = media_format_jni; env = ff_jni_get_env(format); if (!env) { @@ -664,19 +667,19 @@ fail: av_freep(&format); } - return format; + return (FFAMediaFormat *)format; } -static FFAMediaFormat *ff_AMediaFormat_newFromObject(void *object) +static FFAMediaFormat *mediaformat_jni_newFromObject(void *object) { JNIEnv *env = NULL; - FFAMediaFormat *format = NULL; + FFAMediaFormatJni *format = NULL; - format = av_mallocz(sizeof(FFAMediaFormat)); + format = av_mallocz(sizeof(*format)); if (!format) { return NULL; } - format->class = &amediaformat_class; + format->api = media_format_jni; env = ff_jni_get_env(format); if (!env) { @@ -693,7 +696,7 @@ static FFAMediaFormat *ff_AMediaFormat_newFromObject(void *object) goto fail; } - return format; + return (FFAMediaFormat *)format; fail: ff_jni_reset_jfields(env, &format->jfields, jni_amediaformat_mapping, 1, format); @@ -702,10 +705,10 @@ fail: return NULL; } -int ff_AMediaFormat_delete(FFAMediaFormat* format) +static int mediaformat_jni_delete(FFAMediaFormat* ctx) { int ret = 0; - + FFAMediaFormatJni *format = (FFAMediaFormatJni *)ctx; JNIEnv *env = NULL; if (!format) { @@ -724,10 +727,10 @@ int ff_AMediaFormat_delete(FFAMediaFormat* format) return ret; } -char* ff_AMediaFormat_toString(FFAMediaFormat* format) +static char* mediaformat_jni_toString(FFAMediaFormat* ctx) { char *ret = NULL; - + FFAMediaFormatJni *format = (FFAMediaFormatJni *)ctx; JNIEnv *env = NULL; jstring description = NULL; @@ -749,10 +752,10 @@ fail: return ret; } -int ff_AMediaFormat_getInt32(FFAMediaFormat* format, const char *name, int32_t *out) +static int mediaformat_jni_getInt32(FFAMediaFormat* ctx, const char *name, int32_t *out) { int ret = 1; - + FFAMediaFormatJni *format = (FFAMediaFormatJni *)ctx; JNIEnv *env = NULL; jstring key = NULL; jboolean contains_key; @@ -788,10 +791,10 @@ fail: return ret; } -int ff_AMediaFormat_getInt64(FFAMediaFormat* format, const char *name, int64_t *out) +static int mediaformat_jni_getInt64(FFAMediaFormat* ctx, const char *name, int64_t *out) { int ret = 1; - + FFAMediaFormatJni *format = (FFAMediaFormatJni *)ctx; JNIEnv *env = NULL; jstring key = NULL; jboolean contains_key; @@ -827,10 +830,10 @@ fail: return ret; } -int ff_AMediaFormat_getFloat(FFAMediaFormat* format, const char *name, float *out) +static int mediaformat_jni_getFloat(FFAMediaFormat* ctx, const char *name, float *out) { int ret = 1; - + FFAMediaFormatJni *format = (FFAMediaFormatJni *)ctx; JNIEnv *env = NULL; jstring key = NULL; jboolean contains_key; @@ -866,10 +869,10 @@ fail: return ret; } -int ff_AMediaFormat_getBuffer(FFAMediaFormat* format, const char *name, void** data, size_t *size) +static int mediaformat_jni_getBuffer(FFAMediaFormat* ctx, const char *name, void** data, size_t *size) { int ret = 1; - + FFAMediaFormatJni *format = (FFAMediaFormatJni *)ctx; JNIEnv *env = NULL; jstring key = NULL; jboolean contains_key; @@ -924,10 +927,10 @@ fail: return ret; } -int ff_AMediaFormat_getString(FFAMediaFormat* format, const char *name, const char **out) +static int mediaformat_jni_getString(FFAMediaFormat* ctx, const char *name, const char **out) { int ret = 1; - + FFAMediaFormatJni *format = (FFAMediaFormatJni *)ctx; JNIEnv *env = NULL; jstring key = NULL; jboolean contains_key; @@ -974,10 +977,11 @@ fail: return ret; } -void ff_AMediaFormat_setInt32(FFAMediaFormat* format, const char* name, int32_t value) +static void mediaformat_jni_setInt32(FFAMediaFormat* ctx, const char* name, int32_t value) { JNIEnv *env = NULL; jstring key = NULL; + FFAMediaFormatJni *format = (FFAMediaFormatJni *)ctx; av_assert0(format != NULL); @@ -999,10 +1003,11 @@ fail: } } -void ff_AMediaFormat_setInt64(FFAMediaFormat* format, const char* name, int64_t value) +static void mediaformat_jni_setInt64(FFAMediaFormat* ctx, const char* name, int64_t value) { JNIEnv *env = NULL; jstring key = NULL; + FFAMediaFormatJni *format = (FFAMediaFormatJni *)ctx; av_assert0(format != NULL); @@ -1024,10 +1029,11 @@ fail: } } -void ff_AMediaFormat_setFloat(FFAMediaFormat* format, const char* name, float value) +static void mediaformat_jni_setFloat(FFAMediaFormat* ctx, const char* name, float value) { JNIEnv *env = NULL; jstring key = NULL; + FFAMediaFormatJni *format = (FFAMediaFormatJni *)ctx; av_assert0(format != NULL); @@ -1049,11 +1055,12 @@ fail: } } -void ff_AMediaFormat_setString(FFAMediaFormat* format, const char* name, const char* value) +static void mediaformat_jni_setString(FFAMediaFormat* ctx, const char* name, const char* value) { JNIEnv *env = NULL; jstring key = NULL; jstring string = NULL; + FFAMediaFormatJni *format = (FFAMediaFormatJni *)ctx; av_assert0(format != NULL); @@ -1084,12 +1091,13 @@ fail: } } -void ff_AMediaFormat_setBuffer(FFAMediaFormat* format, const char* name, void* data, size_t size) +static void mediaformat_jni_setBuffer(FFAMediaFormat* ctx, const char* name, void* data, size_t size) { JNIEnv *env = NULL; jstring key = NULL; jobject buffer = NULL; void *buffer_data = NULL; + FFAMediaFormatJni *format = (FFAMediaFormatJni *)ctx; av_assert0(format != NULL); @@ -1131,7 +1139,7 @@ fail: } } -static int codec_init_static_fields(FFAMediaCodec *codec) +static int codec_init_static_fields(FFAMediaCodecJni *codec) { int ret = 0; JNIEnv *env = NULL; @@ -1193,17 +1201,17 @@ static inline FFAMediaCodec *codec_create(int method, const char *arg) { int ret = -1; JNIEnv *env = NULL; - FFAMediaCodec *codec = NULL; + FFAMediaCodecJni *codec = NULL; jstring jarg = NULL; jobject object = NULL; jobject buffer_info = NULL; jmethodID create_id = NULL; - codec = av_mallocz(sizeof(FFAMediaCodec)); + codec = av_mallocz(sizeof(*codec)); if (!codec) { return NULL; } - codec->class = &amediacodec_class; + codec->api = media_codec_jni; env = ff_jni_get_env(codec); if (!env) { @@ -1286,11 +1294,11 @@ fail: av_freep(&codec); } - return codec; + return (FFAMediaCodec *)codec; } #define DECLARE_FF_AMEDIACODEC_CREATE_FUNC(name, method) \ -FFAMediaCodec *ff_AMediaCodec_##name(const char *arg) \ +static FFAMediaCodec *mediacodec_jni_##name(const char *arg) \ { \ return codec_create(method, arg); \ } \ @@ -1299,10 +1307,10 @@ DECLARE_FF_AMEDIACODEC_CREATE_FUNC(createCodecByName, CREATE_CODEC_BY_NAME) DECLARE_FF_AMEDIACODEC_CREATE_FUNC(createDecoderByType, CREATE_DECODER_BY_TYPE) DECLARE_FF_AMEDIACODEC_CREATE_FUNC(createEncoderByType, CREATE_ENCODER_BY_TYPE) -int ff_AMediaCodec_delete(FFAMediaCodec* codec) +static int mediacodec_jni_delete(FFAMediaCodec* ctx) { int ret = 0; - + FFAMediaCodecJni *codec = (FFAMediaCodecJni *)ctx; JNIEnv *env = NULL; if (!codec) { @@ -1335,11 +1343,12 @@ int ff_AMediaCodec_delete(FFAMediaCodec* codec) return ret; } -char *ff_AMediaCodec_getName(FFAMediaCodec *codec) +static char *mediacodec_jni_getName(FFAMediaCodec *ctx) { char *ret = NULL; JNIEnv *env = NULL; jobject *name = NULL; + FFAMediaCodecJni *codec = (FFAMediaCodecJni *)ctx; JNI_GET_ENV_OR_RETURN(env, codec, NULL); @@ -1358,10 +1367,12 @@ fail: return ret; } -int ff_AMediaCodec_configure(FFAMediaCodec* codec, const FFAMediaFormat* format, void* surface, void *crypto, uint32_t flags) +static int mediacodec_jni_configure(FFAMediaCodec* ctx, const FFAMediaFormat* format_ctx, void* surface, void *crypto, uint32_t flags) { int ret = 0; JNIEnv *env = NULL; + FFAMediaCodecJni *codec = (FFAMediaCodecJni *)ctx; + const FFAMediaFormatJni *format = (FFAMediaFormatJni *)format_ctx; JNI_GET_ENV_OR_RETURN(env, codec, AVERROR_EXTERNAL); @@ -1375,10 +1386,11 @@ fail: return ret; } -int ff_AMediaCodec_start(FFAMediaCodec* codec) +static int mediacodec_jni_start(FFAMediaCodec* ctx) { int ret = 0; JNIEnv *env = NULL; + FFAMediaCodecJni *codec = (FFAMediaCodecJni *)ctx; JNI_GET_ENV_OR_RETURN(env, codec, AVERROR_EXTERNAL); @@ -1392,10 +1404,11 @@ fail: return ret; } -int ff_AMediaCodec_stop(FFAMediaCodec* codec) +static int mediacodec_jni_stop(FFAMediaCodec* ctx) { int ret = 0; JNIEnv *env = NULL; + FFAMediaCodecJni *codec = (FFAMediaCodecJni *)ctx; JNI_GET_ENV_OR_RETURN(env, codec, AVERROR_EXTERNAL); @@ -1409,10 +1422,11 @@ fail: return ret; } -int ff_AMediaCodec_flush(FFAMediaCodec* codec) +static int mediacodec_jni_flush(FFAMediaCodec* ctx) { int ret = 0; JNIEnv *env = NULL; + FFAMediaCodecJni *codec = (FFAMediaCodecJni *)ctx; JNI_GET_ENV_OR_RETURN(env, codec, AVERROR_EXTERNAL); @@ -1426,10 +1440,11 @@ fail: return ret; } -int ff_AMediaCodec_releaseOutputBuffer(FFAMediaCodec* codec, size_t idx, int render) +static int mediacodec_jni_releaseOutputBuffer(FFAMediaCodec* ctx, size_t idx, int render) { int ret = 0; JNIEnv *env = NULL; + FFAMediaCodecJni *codec = (FFAMediaCodecJni *)ctx; JNI_GET_ENV_OR_RETURN(env, codec, AVERROR_EXTERNAL); @@ -1443,10 +1458,11 @@ fail: return ret; } -int ff_AMediaCodec_releaseOutputBufferAtTime(FFAMediaCodec *codec, size_t idx, int64_t timestampNs) +static int mediacodec_jni_releaseOutputBufferAtTime(FFAMediaCodec *ctx, size_t idx, int64_t timestampNs) { int ret = 0; JNIEnv *env = NULL; + FFAMediaCodecJni *codec = (FFAMediaCodecJni *)ctx; JNI_GET_ENV_OR_RETURN(env, codec, AVERROR_EXTERNAL); @@ -1460,10 +1476,11 @@ fail: return ret; } -ssize_t ff_AMediaCodec_dequeueInputBuffer(FFAMediaCodec* codec, int64_t timeoutUs) +static ssize_t mediacodec_jni_dequeueInputBuffer(FFAMediaCodec* ctx, int64_t timeoutUs) { int ret = 0; JNIEnv *env = NULL; + FFAMediaCodecJni *codec = (FFAMediaCodecJni *)ctx; JNI_GET_ENV_OR_RETURN(env, codec, AVERROR_EXTERNAL); @@ -1477,10 +1494,11 @@ fail: return ret; } -int ff_AMediaCodec_queueInputBuffer(FFAMediaCodec* codec, size_t idx, off_t offset, size_t size, uint64_t time, uint32_t flags) +static int mediacodec_jni_queueInputBuffer(FFAMediaCodec* ctx, size_t idx, off_t offset, size_t size, uint64_t time, uint32_t flags) { int ret = 0; JNIEnv *env = NULL; + FFAMediaCodecJni *codec = (FFAMediaCodecJni *)ctx; JNI_GET_ENV_OR_RETURN(env, codec, AVERROR_EXTERNAL); @@ -1494,10 +1512,11 @@ fail: return ret; } -ssize_t ff_AMediaCodec_dequeueOutputBuffer(FFAMediaCodec* codec, FFAMediaCodecBufferInfo *info, int64_t timeoutUs) +static ssize_t mediacodec_jni_dequeueOutputBuffer(FFAMediaCodec* ctx, FFAMediaCodecBufferInfo *info, int64_t timeoutUs) { int ret = 0; JNIEnv *env = NULL; + FFAMediaCodecJni *codec = (FFAMediaCodecJni *)ctx; JNI_GET_ENV_OR_RETURN(env, codec, AVERROR_EXTERNAL); @@ -1529,11 +1548,11 @@ ssize_t ff_AMediaCodec_dequeueOutputBuffer(FFAMediaCodec* codec, FFAMediaCodecBu return ret; } -uint8_t* ff_AMediaCodec_getInputBuffer(FFAMediaCodec* codec, size_t idx, size_t *out_size) +static uint8_t* mediacodec_jni_getInputBuffer(FFAMediaCodec* ctx, size_t idx, size_t *out_size) { uint8_t *ret = NULL; JNIEnv *env = NULL; - + FFAMediaCodecJni *codec = (FFAMediaCodecJni *)ctx; jobject buffer = NULL; jobject input_buffers = NULL; @@ -1577,11 +1596,11 @@ fail: return ret; } -uint8_t* ff_AMediaCodec_getOutputBuffer(FFAMediaCodec* codec, size_t idx, size_t *out_size) +static uint8_t* mediacodec_jni_getOutputBuffer(FFAMediaCodec* ctx, size_t idx, size_t *out_size) { uint8_t *ret = NULL; JNIEnv *env = NULL; - + FFAMediaCodecJni *codec = (FFAMediaCodecJni *)ctx; jobject buffer = NULL; jobject output_buffers = NULL; @@ -1625,10 +1644,11 @@ fail: return ret; } -FFAMediaFormat* ff_AMediaCodec_getOutputFormat(FFAMediaCodec* codec) +static FFAMediaFormat* mediacodec_jni_getOutputFormat(FFAMediaCodec* ctx) { FFAMediaFormat *ret = NULL; JNIEnv *env = NULL; + FFAMediaCodecJni *codec = (FFAMediaCodecJni *)ctx; jobject mediaformat = NULL; @@ -1639,7 +1659,7 @@ FFAMediaFormat* ff_AMediaCodec_getOutputFormat(FFAMediaCodec* codec) goto fail; } - ret = ff_AMediaFormat_newFromObject(mediaformat); + ret = mediaformat_jni_newFromObject(mediaformat); fail: if (mediaformat) { (*env)->DeleteLocalRef(env, mediaformat); @@ -1648,44 +1668,52 @@ fail: return ret; } -int ff_AMediaCodec_infoTryAgainLater(FFAMediaCodec *codec, ssize_t idx) +static int mediacodec_jni_infoTryAgainLater(FFAMediaCodec *ctx, ssize_t idx) { + FFAMediaCodecJni *codec = (FFAMediaCodecJni *)ctx; return idx == codec->INFO_TRY_AGAIN_LATER; } -int ff_AMediaCodec_infoOutputBuffersChanged(FFAMediaCodec *codec, ssize_t idx) +static int mediacodec_jni_infoOutputBuffersChanged(FFAMediaCodec *ctx, ssize_t idx) { + FFAMediaCodecJni *codec = (FFAMediaCodecJni *)ctx; return idx == codec->INFO_OUTPUT_BUFFERS_CHANGED; } -int ff_AMediaCodec_infoOutputFormatChanged(FFAMediaCodec *codec, ssize_t idx) +static int mediacodec_jni_infoOutputFormatChanged(FFAMediaCodec *ctx, ssize_t idx) { + FFAMediaCodecJni *codec = (FFAMediaCodecJni *)ctx; return idx == codec->INFO_OUTPUT_FORMAT_CHANGED; } -int ff_AMediaCodec_getBufferFlagCodecConfig(FFAMediaCodec *codec) +static int mediacodec_jni_getBufferFlagCodecConfig(FFAMediaCodec *ctx) { + FFAMediaCodecJni *codec = (FFAMediaCodecJni *)ctx; return codec->BUFFER_FLAG_CODEC_CONFIG; } -int ff_AMediaCodec_getBufferFlagEndOfStream(FFAMediaCodec *codec) +static int mediacodec_jni_getBufferFlagEndOfStream(FFAMediaCodec *ctx) { + FFAMediaCodecJni *codec = (FFAMediaCodecJni *)ctx; return codec->BUFFER_FLAG_END_OF_STREAM; } -int ff_AMediaCodec_getBufferFlagKeyFrame(FFAMediaCodec *codec) +static int mediacodec_jni_getBufferFlagKeyFrame(FFAMediaCodec *ctx) { + FFAMediaCodecJni *codec = (FFAMediaCodecJni *)ctx; return codec->BUFFER_FLAG_KEY_FRAME; } -int ff_AMediaCodec_getConfigureFlagEncode(FFAMediaCodec *codec) +static int mediacodec_jni_getConfigureFlagEncode(FFAMediaCodec *ctx) { + FFAMediaCodecJni *codec = (FFAMediaCodecJni *)ctx; return codec->CONFIGURE_FLAG_ENCODE; } -int ff_AMediaCodec_cleanOutputBuffers(FFAMediaCodec *codec) +static int mediacodec_jni_cleanOutputBuffers(FFAMediaCodec *ctx) { int ret = 0; + FFAMediaCodecJni *codec = (FFAMediaCodecJni *)ctx; if (!codec->has_get_i_o_buffer) { if (codec->output_buffers) { @@ -1706,6 +1734,86 @@ fail: return ret; } +static const FFAMediaFormat media_format_jni = { + .class = &amediaformat_class, + + .create = mediaformat_jni_new, + .delete = mediaformat_jni_delete, + + .toString = mediaformat_jni_toString, + + .getInt32 = mediaformat_jni_getInt32, + .getInt64 = mediaformat_jni_getInt64, + .getFloat = mediaformat_jni_getFloat, + .getBuffer = mediaformat_jni_getBuffer, + .getString = mediaformat_jni_getString, + + .setInt32 = mediaformat_jni_setInt32, + .setInt64 = mediaformat_jni_setInt64, + .setFloat = mediaformat_jni_setFloat, + .setString = mediaformat_jni_setString, + .setBuffer = mediaformat_jni_setBuffer, +}; + +static const FFAMediaCodec media_codec_jni = { + .class = &amediacodec_class, + + .getName = mediacodec_jni_getName, + + .createCodecByName = mediacodec_jni_createCodecByName, + .createDecoderByType = mediacodec_jni_createDecoderByType, + .createEncoderByType = mediacodec_jni_createEncoderByType, + .delete = mediacodec_jni_delete, + + .configure = mediacodec_jni_configure, + .start = mediacodec_jni_start, + .stop = mediacodec_jni_stop, + .flush = mediacodec_jni_flush, + + .getInputBuffer = mediacodec_jni_getInputBuffer, + .getOutputBuffer = mediacodec_jni_getOutputBuffer, + + .dequeueInputBuffer = mediacodec_jni_dequeueInputBuffer, + .queueInputBuffer = mediacodec_jni_queueInputBuffer, + + .dequeueOutputBuffer = mediacodec_jni_dequeueOutputBuffer, + .getOutputFormat = mediacodec_jni_getOutputFormat, + + .releaseOutputBuffer = mediacodec_jni_releaseOutputBuffer, + .releaseOutputBufferAtTime = mediacodec_jni_releaseOutputBufferAtTime, + + .infoTryAgainLater = mediacodec_jni_infoTryAgainLater, + .infoOutputBuffersChanged = mediacodec_jni_infoOutputBuffersChanged, + .infoOutputFormatChanged = mediacodec_jni_infoOutputFormatChanged, + + .getBufferFlagCodecConfig = mediacodec_jni_getBufferFlagCodecConfig, + .getBufferFlagEndOfStream = mediacodec_jni_getBufferFlagEndOfStream, + .getBufferFlagKeyFrame = mediacodec_jni_getBufferFlagKeyFrame, + + .getConfigureFlagEncode = mediacodec_jni_getConfigureFlagEncode, + .cleanOutputBuffers = mediacodec_jni_cleanOutputBuffers, +}; + +FFAMediaFormat *ff_AMediaFormat_new(void) +{ + return media_format_jni.create(); +} + +FFAMediaCodec* ff_AMediaCodec_createCodecByName(const char *name) +{ + return media_codec_jni.createCodecByName(name); +} + +FFAMediaCodec* ff_AMediaCodec_createDecoderByType(const char *mime_type) +{ + return media_codec_jni.createDecoderByType(mime_type); +} + +FFAMediaCodec* ff_AMediaCodec_createEncoderByType(const char *mime_type) +{ + return media_codec_jni.createEncoderByType(mime_type); +} + int ff_Build_SDK_INT(AVCodecContext *avctx) { int ret = -1; diff --git a/libavcodec/mediacodec_wrapper.h b/libavcodec/mediacodec_wrapper.h index b106ff315a..606fdbede5 100644 --- a/libavcodec/mediacodec_wrapper.h +++ b/libavcodec/mediacodec_wrapper.h @@ -58,28 +58,90 @@ int ff_AMediaCodecProfile_getProfileFromAVCodecContext(AVCodecContext *avctx); char *ff_AMediaCodecList_getCodecNameByType(const char *mime, int profile, int encoder, void *log_ctx); -struct FFAMediaFormat; typedef struct FFAMediaFormat FFAMediaFormat; +struct FFAMediaFormat { + const AVClass *class; + + FFAMediaFormat *(*create)(void); + int (*delete)(FFAMediaFormat *); + + char* (*toString)(FFAMediaFormat* format); + + int (*getInt32)(FFAMediaFormat* format, const char *name, int32_t *out); + int (*getInt64)(FFAMediaFormat* format, const char *name, int64_t *out); + int (*getFloat)(FFAMediaFormat* format, const char *name, float *out); + int (*getBuffer)(FFAMediaFormat* format, const char *name, void** data, size_t *size); + int (*getString)(FFAMediaFormat* format, const char *name, const char **out); + + void (*setInt32)(FFAMediaFormat* format, const char* name, int32_t value); + void (*setInt64)(FFAMediaFormat* format, const char* name, int64_t value); + void (*setFloat)(FFAMediaFormat* format, const char* name, float value); + void (*setString)(FFAMediaFormat* format, const char* name, const char* value); + void (*setBuffer)(FFAMediaFormat* format, const char* name, void* data, size_t size); +}; FFAMediaFormat *ff_AMediaFormat_new(void); -int ff_AMediaFormat_delete(FFAMediaFormat* format); -char* ff_AMediaFormat_toString(FFAMediaFormat* format); +static inline int ff_AMediaFormat_delete(FFAMediaFormat* format) +{ + return format->delete(format); +} -int ff_AMediaFormat_getInt32(FFAMediaFormat* format, const char *name, int32_t *out); -int ff_AMediaFormat_getInt64(FFAMediaFormat* format, const char *name, int64_t *out); -int ff_AMediaFormat_getFloat(FFAMediaFormat* format, const char *name, float *out); -int ff_AMediaFormat_getBuffer(FFAMediaFormat* format, const char *name, void** data, size_t *size); -int ff_AMediaFormat_getString(FFAMediaFormat* format, const char *name, const char **out); +static inline char* ff_AMediaFormat_toString(FFAMediaFormat* format) +{ + return format->toString(format); +} -void ff_AMediaFormat_setInt32(FFAMediaFormat* format, const char* name, int32_t value); -void ff_AMediaFormat_setInt64(FFAMediaFormat* format, const char* name, int64_t value); -void ff_AMediaFormat_setFloat(FFAMediaFormat* format, const char* name, float value); -void ff_AMediaFormat_setString(FFAMediaFormat* format, const char* name, const char* value); -void ff_AMediaFormat_setBuffer(FFAMediaFormat* format, const char* name, void* data, size_t size); +static inline int ff_AMediaFormat_getInt32(FFAMediaFormat* format, const char *name, int32_t *out) +{ + return format->getInt32(format, name, out); +} + +static inline int ff_AMediaFormat_getInt64(FFAMediaFormat* format, const char *name, int64_t *out) +{ + return format->getInt64(format, name, out); +} + +static inline int ff_AMediaFormat_getFloat(FFAMediaFormat* format, const char *name, float *out) +{ + return format->getFloat(format, name, out); +} + +static inline int ff_AMediaFormat_getBuffer(FFAMediaFormat* format, const char *name, void** data, size_t *size) +{ + return format->getBuffer(format, name, data, size); +} + +static inline int ff_AMediaFormat_getString(FFAMediaFormat* format, const char *name, const char **out) +{ + return format->getString(format, name, out); +} + +static inline void ff_AMediaFormat_setInt32(FFAMediaFormat* format, const char* name, int32_t value) +{ + format->setInt32(format, name, value); +} + +static inline void ff_AMediaFormat_setInt64(FFAMediaFormat* format, const char* name, int64_t value) +{ + format->setInt64(format, name, value); +} + +static inline void ff_AMediaFormat_setFloat(FFAMediaFormat* format, const char* name, float value) +{ + format->setFloat(format, name, value); +} + +static inline void ff_AMediaFormat_setString(FFAMediaFormat* format, const char* name, const char* value) +{ + format->setString(format, name, value); +} + +static inline void ff_AMediaFormat_setBuffer(FFAMediaFormat* format, const char* name, void* data, size_t size) +{ + format->setBuffer(format, name, data, size); +} -struct FFAMediaCodec; -typedef struct FFAMediaCodec FFAMediaCodec; typedef struct FFAMediaCodecCryptoInfo FFAMediaCodecCryptoInfo; struct FFAMediaCodecBufferInfo { @@ -90,41 +152,160 @@ struct FFAMediaCodecBufferInfo { }; typedef struct FFAMediaCodecBufferInfo FFAMediaCodecBufferInfo; -char *ff_AMediaCodec_getName(FFAMediaCodec *codec); +typedef struct FFAMediaCodec FFAMediaCodec; +struct FFAMediaCodec { + const AVClass *class; + + char *(*getName)(FFAMediaCodec *codec); + + FFAMediaCodec* (*createCodecByName)(const char *name); + FFAMediaCodec* (*createDecoderByType)(const char *mime_type); + FFAMediaCodec* (*createEncoderByType)(const char *mime_type); + int (*delete)(FFAMediaCodec* codec); + + int (*configure)(FFAMediaCodec* codec, const FFAMediaFormat* format, void* surface, void *crypto, uint32_t flags); + int (*start)(FFAMediaCodec* codec); + int (*stop)(FFAMediaCodec* codec); + int (*flush)(FFAMediaCodec* codec); + + uint8_t* (*getInputBuffer)(FFAMediaCodec* codec, size_t idx, size_t *out_size); + uint8_t* (*getOutputBuffer)(FFAMediaCodec* codec, size_t idx, size_t *out_size); + + ssize_t (*dequeueInputBuffer)(FFAMediaCodec* codec, int64_t timeoutUs); + int (*queueInputBuffer)(FFAMediaCodec* codec, size_t idx, off_t offset, size_t size, uint64_t time, uint32_t flags); + + ssize_t (*dequeueOutputBuffer)(FFAMediaCodec* codec, FFAMediaCodecBufferInfo *info, int64_t timeoutUs); + FFAMediaFormat* (*getOutputFormat)(FFAMediaCodec* codec); + + int (*releaseOutputBuffer)(FFAMediaCodec* codec, size_t idx, int render); + int (*releaseOutputBufferAtTime)(FFAMediaCodec *codec, size_t idx, int64_t timestampNs); + + int (*infoTryAgainLater)(FFAMediaCodec *codec, ssize_t idx); + int (*infoOutputBuffersChanged)(FFAMediaCodec *codec, ssize_t idx); + int (*infoOutputFormatChanged)(FFAMediaCodec *codec, ssize_t indx); + + int (*getBufferFlagCodecConfig)(FFAMediaCodec *codec); + int (*getBufferFlagEndOfStream)(FFAMediaCodec *codec); + int (*getBufferFlagKeyFrame)(FFAMediaCodec *codec); + + int (*getConfigureFlagEncode)(FFAMediaCodec *codec); + + int (*cleanOutputBuffers)(FFAMediaCodec *codec); +}; + +static inline char *ff_AMediaCodec_getName(FFAMediaCodec *codec) +{ + return codec->getName(codec); +} FFAMediaCodec* ff_AMediaCodec_createCodecByName(const char *name); FFAMediaCodec* ff_AMediaCodec_createDecoderByType(const char *mime_type); FFAMediaCodec* ff_AMediaCodec_createEncoderByType(const char *mime_type); -int ff_AMediaCodec_configure(FFAMediaCodec* codec, const FFAMediaFormat* format, void* surface, void *crypto, uint32_t flags); -int ff_AMediaCodec_start(FFAMediaCodec* codec); -int ff_AMediaCodec_stop(FFAMediaCodec* codec); -int ff_AMediaCodec_flush(FFAMediaCodec* codec); -int ff_AMediaCodec_delete(FFAMediaCodec* codec); +static inline int ff_AMediaCodec_configure(FFAMediaCodec* codec, const FFAMediaFormat* format, void* surface, void *crypto, uint32_t flags) +{ + return codec->configure(codec, format, surface, crypto, flags); +} + +static inline int ff_AMediaCodec_start(FFAMediaCodec* codec) +{ + return codec->start(codec); +} + +static inline int ff_AMediaCodec_stop(FFAMediaCodec* codec) +{ + return codec->stop(codec); +} + +static inline int ff_AMediaCodec_flush(FFAMediaCodec* codec) +{ + return codec->flush(codec); +} + +static inline int ff_AMediaCodec_delete(FFAMediaCodec* codec) +{ + return codec->delete(codec); +} + +static inline uint8_t* ff_AMediaCodec_getInputBuffer(FFAMediaCodec* codec, size_t idx, size_t *out_size) +{ + return codec->getInputBuffer(codec, idx, out_size); +} + +static inline uint8_t* ff_AMediaCodec_getOutputBuffer(FFAMediaCodec* codec, size_t idx, size_t *out_size) +{ + return codec->getOutputBuffer(codec, idx, out_size); +} + +static inline ssize_t ff_AMediaCodec_dequeueInputBuffer(FFAMediaCodec* codec, int64_t timeoutUs) +{ + return codec->dequeueInputBuffer(codec, timeoutUs); +} + +static inline int ff_AMediaCodec_queueInputBuffer(FFAMediaCodec *codec, size_t idx, off_t offset, size_t size, uint64_t time, uint32_t flags) +{ + return codec->queueInputBuffer(codec, idx, offset, size, time, flags); +} + +static inline ssize_t ff_AMediaCodec_dequeueOutputBuffer(FFAMediaCodec* codec, FFAMediaCodecBufferInfo *info, int64_t timeoutUs) +{ + return codec->dequeueOutputBuffer(codec, info, timeoutUs); +} + +static inline FFAMediaFormat* ff_AMediaCodec_getOutputFormat(FFAMediaCodec* codec) +{ + return codec->getOutputFormat(codec); +} + +static inline int ff_AMediaCodec_releaseOutputBuffer(FFAMediaCodec* codec, size_t idx, int render) +{ + return codec->releaseOutputBuffer(codec, idx, render); +} + +static inline int ff_AMediaCodec_releaseOutputBufferAtTime(FFAMediaCodec *codec, size_t idx, int64_t timestampNs) +{ + return codec->releaseOutputBufferAtTime(codec, idx, timestampNs); +} -uint8_t* ff_AMediaCodec_getInputBuffer(FFAMediaCodec* codec, size_t idx, size_t *out_size); -uint8_t* ff_AMediaCodec_getOutputBuffer(FFAMediaCodec* codec, size_t idx, size_t *out_size); +static inline int ff_AMediaCodec_infoTryAgainLater(FFAMediaCodec *codec, ssize_t idx) +{ + return codec->infoTryAgainLater(codec, idx); +} -ssize_t ff_AMediaCodec_dequeueInputBuffer(FFAMediaCodec* codec, int64_t timeoutUs); -int ff_AMediaCodec_queueInputBuffer(FFAMediaCodec* codec, size_t idx, off_t offset, size_t size, uint64_t time, uint32_t flags); +static inline int ff_AMediaCodec_infoOutputBuffersChanged(FFAMediaCodec *codec, ssize_t idx) +{ + return codec->infoOutputBuffersChanged(codec, idx); +} -ssize_t ff_AMediaCodec_dequeueOutputBuffer(FFAMediaCodec* codec, FFAMediaCodecBufferInfo *info, int64_t timeoutUs); -FFAMediaFormat* ff_AMediaCodec_getOutputFormat(FFAMediaCodec* codec); +static inline int ff_AMediaCodec_infoOutputFormatChanged(FFAMediaCodec *codec, ssize_t idx) +{ + return codec->infoOutputFormatChanged(codec, idx); +} -int ff_AMediaCodec_releaseOutputBuffer(FFAMediaCodec* codec, size_t idx, int render); -int ff_AMediaCodec_releaseOutputBufferAtTime(FFAMediaCodec *codec, size_t idx, int64_t timestampNs); +static inline int ff_AMediaCodec_getBufferFlagCodecConfig(FFAMediaCodec *codec) +{ + return codec->getBufferFlagCodecConfig(codec); +} -int ff_AMediaCodec_infoTryAgainLater(FFAMediaCodec *codec, ssize_t idx); -int ff_AMediaCodec_infoOutputBuffersChanged(FFAMediaCodec *codec, ssize_t idx); -int ff_AMediaCodec_infoOutputFormatChanged(FFAMediaCodec *codec, ssize_t indx); +static inline int ff_AMediaCodec_getBufferFlagEndOfStream(FFAMediaCodec *codec) +{ + return codec->getBufferFlagEndOfStream(codec); +} -int ff_AMediaCodec_getBufferFlagCodecConfig (FFAMediaCodec *codec); -int ff_AMediaCodec_getBufferFlagEndOfStream(FFAMediaCodec *codec); -int ff_AMediaCodec_getBufferFlagKeyFrame(FFAMediaCodec *codec); +static inline int ff_AMediaCodec_getBufferFlagKeyFrame(FFAMediaCodec *codec) +{ + return codec->getBufferFlagKeyFrame(codec); +} -int ff_AMediaCodec_getConfigureFlagEncode(FFAMediaCodec *codec); +static inline int ff_AMediaCodec_getConfigureFlagEncode(FFAMediaCodec *codec) +{ + return codec->getConfigureFlagEncode(codec); +} -int ff_AMediaCodec_cleanOutputBuffers(FFAMediaCodec *codec); +static inline int ff_AMediaCodec_cleanOutputBuffers(FFAMediaCodec *codec) +{ + return codec->cleanOutputBuffers(codec); +} int ff_Build_SDK_INT(AVCodecContext *avctx); From patchwork Mon Oct 24 03:16:52 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhao Zhili X-Patchwork-Id: 38946 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:4a86:b0:9d:28a3:170e with SMTP id fn6csp1998030pzb; Sun, 23 Oct 2022 20:18:46 -0700 (PDT) X-Google-Smtp-Source: AMsMyM7xv4m8pneUE/C/VDadNTBUptXBmWSEwHXPOU8BqSRqsttpqqbB1u2rnK+Gnyx+Qgf/hxqd X-Received: by 2002:a17:907:9625:b0:78d:bb0c:4a40 with SMTP id gb37-20020a170907962500b0078dbb0c4a40mr24795305ejc.339.1666581526624; Sun, 23 Oct 2022 20:18:46 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666581526; cv=none; d=google.com; s=arc-20160816; b=w1rtR0GiigGvXbt1YETGCEkPt/gSNTRNEtIs5EfbKI3WPgDL184mq2Hl9L1cjFd0Ol 0cbehePO/DLbjpIsN48yynOVikudvlZflCKoADBHxrEcmWw2UsBprDIy9Y75EMtc9l3m m/hVp5nbu3iW87wSj8N5BhGJwgWq0uMfPFGAD7p5q2GF/+AVuXJ1izgI25Hh64vNwrjB qFaa8bIaaczu9OiTxfUXL5zIZnnDoUxjqsvZengD2K+WH6uhVhTRuPFNE0W0pT4BAxHe dp4g0LAWEsjAuTHRkiTPrirgLflUyhKeYcTcZHbxh9jzg7MkJWDTTZlhT0Yh2hf9ORK5 xM+A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to:date :to:from:message-id:dkim-signature:delivered-to; bh=EX0gBTT+E4QlNRxixBR8vL8eTRrwhxqfepIfqZpW6b8=; b=OShQsQa5+wYj4c6iCFWdbLc/lRwlEbb6MJPe5okoSFASObc7i79ZzqqzxdP8wilROX cqd/RB4D/7ZN3T1CTCSvehKsNn5w/NSYH55aSJ7WGW55MT4BQq9Gm2gexz7HXhV3+qzy H2OQvo/92fGG0lZAkYH6x3G0U6leArB6w/c61zAq/o1F74In3h5WkhCx+P5L+m0AT+Oe Upwuc3JbGw/HsfUE+zailgShyELWQcb9/l4wqn2kN7WclyzlisRSIsVlKr3njHpybMfD VJTh16ySxL5naNtsktyyeLqLEeKyBt1EceLmdR9VdQEVYuadkrNdHuS4/ZMrwspf53Cj cL7g== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@foxmail.com header.s=s201512 header.b="RK6uGa/7"; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=foxmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id ev18-20020a056402541200b00461a5271515si3263104edb.618.2022.10.23.20.18.46; Sun, 23 Oct 2022 20:18:46 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@foxmail.com header.s=s201512 header.b="RK6uGa/7"; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=foxmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 05D1D68BD6B; Mon, 24 Oct 2022 06:17:37 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from out203-205-221-164.mail.qq.com (out203-205-221-164.mail.qq.com [203.205.221.164]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 1460A68BD56 for ; Mon, 24 Oct 2022 06:17:27 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=foxmail.com; s=s201512; t=1666581444; bh=79SRpgkefsOJVdVZsZCqnQjTu7B0B1s70rmga1HT0Ac=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=RK6uGa/7gjmg4evnvKXH6K1Ivmh6T4f+Oqr1i/eWB9IVvisIDLvxkE57hTQUZkLx5 TQvzBp44ggRtfhxDaJj5u6XK82+VwxXC2vpQ+XozZoAwdc5sYiveyh/cgd0z2LNB/P trp35KnNDOaOj8t4UYd0+5eAu4RB3peC+h6UBVeQ= Received: from localhost.localdomain ([59.37.125.102]) by newxmesmtplogicsvrsza31.qq.com (NewEsmtp) with SMTP id 44A14270; Mon, 24 Oct 2022 11:17:10 +0800 X-QQ-mid: xmsmtpt1666581438tqzvu906m Message-ID: X-QQ-XMAILINFO: NDRJhlKLIrRLi7bcsknf0dSCTNo2QHo5MBYjFLJo2fVty3nEPgJ2RYYaF01zLR HfIxOXCUjvhv51alHbM/idWGQj2dIcgkP33Qx5P2vkLFGFfnrZsQxlCy52YnoLnKeVByVxXYSEsU 1QPFYEuI8Dk2QHx5RafrLJVjD4rIWV4WOY2C1Bp1J4L94wvIDjf2iF1j63SnlxrMPhNOVZKLIqau /8d+I/u9cfHfH1HnpxeNOoCiqi8jDAPz6CX+UbtUzUs7D4G7KY2MrU/Wj2J9RvoEx+OcZGXV2Qn0 Jdybd9TRv2gAmLZX+fedVDERo5txtZGrqQkANj67oLEAwo84w3/eYSRalSVpNNLL5kZtb6nhRv2C JCj3avmf3BRkM+xYcKWUgh0VeVRJfa1vTwanIpv2g4PjYpXJgUh6D2lYR7hs3q9ShszHtbXExkuo fae6WliCSg8DGc/AXKNTstYhWJyPwnCM+OqXoFtoh/9T7Bq3DlDh2sO7THzMdvykcPMTLEB8LvSs KCP19jHBaIuQpVmRa1SynMRXQZkDCN3J6AEglkSOzg1kjJWwXlLjfcVVyZwbFdbilogLbhGngyuF Qzfxj1Ip9Kd66RmZ+dK0MgXDejClXOL4Yr+WDsbJsDH85+oGNjbmH5jIDm6mWDin9HMzPe4n50aV GNZlPm2SAXpNvwQhnHAhaaWQSm6lXVcNFwIkAwOS4ej3uMb3qy2fmL8fb76gXMEu+MBUyvprZRuU hvuZ+xeQ3u+fd2OsrLNaOg5aVY745tJIuONvhmT8kxUf0RO+6bp17OPUG7mtVl7U8XO4F4H8rfNy d5kNQecOMbZM78XlYEfL6o7X9s4tB1Fyq2lljHVf29/0jSUM2rhRZwZxvetchjTmCqzI59gsqqiI ZTfn8PHqvyRKUDzbMs3UP+8q42GmzHzTqYlfYEcn2U2oeZFnq/dCEui1uo1I/KJYTELvydkNGX3C jjBDbsKK+oWk3LWGKdSP7ZQqcZ1/CJoeBeWbEodGB37Ny++ZIVkoHzboJtFPr7 From: Zhao Zhili To: ffmpeg-devel@ffmpeg.org Date: Mon, 24 Oct 2022 11:16:52 +0800 X-OQ-MSGID: <20221024031658.1026118-7-quinkblack@foxmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221024031658.1026118-1-quinkblack@foxmail.com> References: <20221024031658.1026118-1-quinkblack@foxmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 06/12] avcodec/mediacodec: add NDK media codec wrapper X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: matthieu.bouron@stupeflix.com, Zhao Zhili , aman@tmm1.net Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: VoCBPHYK79Hh From: Zhao Zhili Signed-off-by: Zhao Zhili --- configure | 2 + libavcodec/mediacodec_wrapper.c | 596 +++++++++++++++++++++++++++++- libavcodec/mediacodec_wrapper.h | 8 +- libavcodec/mediacodecdec.c | 2 +- libavcodec/mediacodecdec_common.c | 2 +- 5 files changed, 600 insertions(+), 10 deletions(-) diff --git a/configure b/configure index eefd103414..ee2e3ba6ac 100755 --- a/configure +++ b/configure @@ -3191,6 +3191,7 @@ h264_crystalhd_decoder_select="crystalhd h264_mp4toannexb_bsf h264_parser" h264_cuvid_decoder_deps="cuvid" h264_cuvid_decoder_select="h264_mp4toannexb_bsf" h264_mediacodec_decoder_deps="mediacodec" +h264_mediacodec_decoder_extralibs="-landroid" h264_mediacodec_decoder_select="h264_mp4toannexb_bsf h264_parser" h264_mf_encoder_deps="mediafoundation" h264_mmal_decoder_deps="mmal" @@ -3209,6 +3210,7 @@ hevc_amf_encoder_deps="amf" hevc_cuvid_decoder_deps="cuvid" hevc_cuvid_decoder_select="hevc_mp4toannexb_bsf" hevc_mediacodec_decoder_deps="mediacodec" +hevc_mediacodec_decoder_extralibs="-landroid" hevc_mediacodec_decoder_select="hevc_mp4toannexb_bsf hevc_parser" hevc_mf_encoder_deps="mediafoundation" hevc_nvenc_encoder_deps="nvenc" diff --git a/libavcodec/mediacodec_wrapper.c b/libavcodec/mediacodec_wrapper.c index e0c614680e..b12aced711 100644 --- a/libavcodec/mediacodec_wrapper.c +++ b/libavcodec/mediacodec_wrapper.c @@ -20,7 +20,11 @@ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA */ +#include #include +#include +#include +#include #include "libavutil/avassert.h" #include "libavutil/mem.h" @@ -1794,23 +1798,607 @@ static const FFAMediaCodec media_codec_jni = { .cleanOutputBuffers = mediacodec_jni_cleanOutputBuffers, }; -FFAMediaFormat *ff_AMediaFormat_new(void) +typedef struct FFAMediaFormatNdk { + FFAMediaFormat api; + + void *libmedia; + AMediaFormat *impl; + + AMediaFormat *(*new)(void); + media_status_t (*delete)(AMediaFormat*); + + const char* (*toString)(AMediaFormat*); + + bool (*getInt32)(AMediaFormat*, const char *name, int32_t *out); + bool (*getInt64)(AMediaFormat*, const char *name, int64_t *out); + bool (*getFloat)(AMediaFormat*, const char *name, float *out); + bool (*getSize)(AMediaFormat*, const char *name, size_t *out); + bool (*getBuffer)(AMediaFormat*, const char *name, void** data, size_t *size); + bool (*getString)(AMediaFormat*, const char *name, const char **out); + + void (*setInt32)(AMediaFormat*, const char* name, int32_t value); + void (*setInt64)(AMediaFormat*, const char* name, int64_t value); + void (*setFloat)(AMediaFormat*, const char* name, float value); + void (*setString)(AMediaFormat*, const char* name, const char* value); + void (*setBuffer)(AMediaFormat*, const char* name, const void* data, size_t size); +} FFAMediaFormatNdk; + +typedef struct FFAMediaCodecNdk { + FFAMediaCodec api; + + void *libmedia; + AMediaCodec *impl; + ANativeWindow *window; + + AMediaCodec* (*createCodecByName)(const char *name); + AMediaCodec* (*createDecoderByType)(const char *mime_type); + AMediaCodec* (*createEncoderByType)(const char *mime_type); + media_status_t (*delete)(AMediaCodec*); + + media_status_t (*configure)(AMediaCodec *, + const AMediaFormat *format, + ANativeWindow *surface, + AMediaCrypto *crypto, + uint32_t flags); + media_status_t (*start)(AMediaCodec*); + media_status_t (*stop)(AMediaCodec*); + media_status_t (*flush)(AMediaCodec*); + + uint8_t* (*getInputBuffer)(AMediaCodec*, size_t idx, size_t *out_size); + uint8_t* (*getOutputBuffer)(AMediaCodec*, size_t idx, size_t *out_size); + + ssize_t (*dequeueInputBuffer)(AMediaCodec*, int64_t timeoutUs); + media_status_t (*queueInputBuffer)(AMediaCodec*, size_t idx, + long offset, size_t size, + uint64_t time, uint32_t flags); + + ssize_t (*dequeueOutputBuffer)(AMediaCodec*, AMediaCodecBufferInfo *info, int64_t timeoutUs); + AMediaFormat* (*getOutputFormat)(AMediaCodec*); + + media_status_t (*releaseOutputBuffer)(AMediaCodec*, size_t idx, bool render); + media_status_t (*releaseOutputBufferAtTime)(AMediaCodec *mData, size_t idx, int64_t timestampNs); + + // Available since API level 28. + media_status_t (*getName)(AMediaCodec*, char** out_name); + void (*releaseName)(AMediaCodec*, char* name); +} FFAMediaCodecNdk; + +static const FFAMediaFormat media_format_ndk; +static const FFAMediaCodec media_codec_ndk; + +static const AVClass amediaformat_ndk_class = { + .class_name = "amediaformat_ndk", + .item_name = av_default_item_name, + .version = LIBAVUTIL_VERSION_INT, +}; + +static const AVClass amediacodec_ndk_class = { + .class_name = "amediacodec_ndk", + .item_name = av_default_item_name, + .version = LIBAVUTIL_VERSION_INT, +}; + +static FFAMediaFormat *mediaformat_ndk_create(AMediaFormat *impl) +{ + FFAMediaFormatNdk *format = av_mallocz(sizeof(*format)); + if (!format) + return NULL; + + format->api = media_format_ndk; + + format->libmedia = dlopen("libmediandk.so", RTLD_NOW); + if (!format->libmedia) + goto error; + +#define GET_SYMBOL(sym) \ + format->sym = dlsym(format->libmedia, "AMediaFormat_" #sym); \ + if (!format->sym) \ + goto error; + + GET_SYMBOL(new) + GET_SYMBOL(delete) + + GET_SYMBOL(toString) + + GET_SYMBOL(getInt32) + GET_SYMBOL(getInt64) + GET_SYMBOL(getFloat) + GET_SYMBOL(getSize) + GET_SYMBOL(getBuffer) + GET_SYMBOL(getString) + + GET_SYMBOL(setInt32) + GET_SYMBOL(setInt64) + GET_SYMBOL(setFloat) + GET_SYMBOL(setString) + GET_SYMBOL(setBuffer) + +#undef GET_SYMBOL + + if (impl) { + format->impl = impl; + } else { + format->impl = format->new(); + if (!format->impl) + goto error; + } + + return (FFAMediaFormat *)format; + +error: + if (format->libmedia) + dlclose(format->libmedia); + av_freep(&format); + return NULL; +} + +static FFAMediaFormat *mediaformat_ndk_new(void) +{ + return mediaformat_ndk_create(NULL); +} + +static int mediaformat_ndk_delete(FFAMediaFormat* ctx) +{ + FFAMediaFormatNdk *format = (FFAMediaFormatNdk *)ctx; + int ret = 0; + if (!format) + return 0; + + av_assert0(format->api.class == &amediaformat_ndk_class); + + if (format->impl && (format->delete(format->impl) != AMEDIA_OK)) + ret = AVERROR_EXTERNAL; + if (format->libmedia) + dlclose(format->libmedia); + av_free(format); + + return ret; +} + +static char* mediaformat_ndk_toString(FFAMediaFormat* ctx) +{ + FFAMediaFormatNdk *format = (FFAMediaFormatNdk *)ctx; + const char *str = format->toString(format->impl); + return av_strdup(str); +} + +static int mediaformat_ndk_getInt32(FFAMediaFormat* ctx, const char *name, int32_t *out) +{ + FFAMediaFormatNdk *format = (FFAMediaFormatNdk *)ctx; + return format->getInt32(format->impl, name, out); +} + +static int mediaformat_ndk_getInt64(FFAMediaFormat* ctx, const char *name, int64_t *out) +{ + FFAMediaFormatNdk *format = (FFAMediaFormatNdk *)ctx; + return format->getInt64(format->impl, name, out); +} + +static int mediaformat_ndk_getFloat(FFAMediaFormat* ctx, const char *name, float *out) +{ + FFAMediaFormatNdk *format = (FFAMediaFormatNdk *)ctx; + return format->getFloat(format->impl, name, out); +} + +static int mediaformat_ndk_getBuffer(FFAMediaFormat* ctx, const char *name, void** data, size_t *size) +{ + FFAMediaFormatNdk *format = (FFAMediaFormatNdk *)ctx; + return format->getBuffer(format->impl, name, data, size); +} + +static int mediaformat_ndk_getString(FFAMediaFormat* ctx, const char *name, const char **out) +{ + FFAMediaFormatNdk *format = (FFAMediaFormatNdk *)ctx; + const char *tmp = NULL; + int ret = format->getString(format->impl, name, &tmp); + + if (tmp) + *out = av_strdup(tmp); + return ret; +} + +static void mediaformat_ndk_setInt32(FFAMediaFormat* ctx, const char* name, int32_t value) +{ + FFAMediaFormatNdk *format = (FFAMediaFormatNdk *)ctx; + format->setInt32(format->impl, name, value); +} + +static void mediaformat_ndk_setInt64(FFAMediaFormat* ctx, const char* name, int64_t value) +{ + FFAMediaFormatNdk *format = (FFAMediaFormatNdk *)ctx; + format->setInt64(format->impl, name, value); +} + +static void mediaformat_ndk_setFloat(FFAMediaFormat* ctx, const char* name, float value) +{ + FFAMediaFormatNdk *format = (FFAMediaFormatNdk *)ctx; + format->setFloat(format->impl, name, value); +} + +static void mediaformat_ndk_setString(FFAMediaFormat* ctx, const char* name, const char* value) +{ + FFAMediaFormatNdk *format = (FFAMediaFormatNdk *)ctx; + format->setString(format->impl, name, value); +} + +static void mediaformat_ndk_setBuffer(FFAMediaFormat* ctx, const char* name, void* data, size_t size) +{ + FFAMediaFormatNdk *format = (FFAMediaFormatNdk *)ctx; + format->setBuffer(format->impl, name, data, size); +} + +static char *mediacodec_ndk_getName(FFAMediaCodec *ctx) +{ + FFAMediaCodecNdk *codec = (FFAMediaCodecNdk *)ctx; + char *ret = NULL; + char *name = NULL; + + if (!codec->getName || !codec->releaseName) { + av_log(ctx, AV_LOG_DEBUG, "getName() unavailable\n"); + return ret; + } + + codec->getName(codec->impl, &name); + if (name) { + ret = av_strdup(name); + codec->releaseName(codec->impl, name); + } + + return ret; +} + +static inline FFAMediaCodec *ndk_codec_create(int method, const char *arg) { + FFAMediaCodecNdk *codec = av_mallocz(sizeof(*codec)); + const char *lib_name = "libmediandk.so"; + + if (!codec) + return NULL; + + codec->api = media_codec_ndk; + codec->libmedia = dlopen(lib_name, RTLD_NOW); + if (!codec->libmedia) + goto error; + +#define GET_SYMBOL(sym, required) \ + codec->sym = dlsym(codec->libmedia, "AMediaCodec_" #sym); \ + if (!codec->sym) { \ + av_log(codec, required ? AV_LOG_ERROR : AV_LOG_INFO, \ + #sym "() unavailable from %s\n", lib_name); \ + if (required) \ + goto error; \ + } + + GET_SYMBOL(createCodecByName, 1) + GET_SYMBOL(createDecoderByType, 1) + GET_SYMBOL(createEncoderByType, 1) + GET_SYMBOL(delete, 1) + + GET_SYMBOL(configure, 1) + GET_SYMBOL(start, 1) + GET_SYMBOL(stop, 1) + GET_SYMBOL(flush, 1) + + GET_SYMBOL(getInputBuffer, 1) + GET_SYMBOL(getOutputBuffer, 1) + + GET_SYMBOL(dequeueInputBuffer, 1) + GET_SYMBOL(queueInputBuffer, 1) + + GET_SYMBOL(dequeueOutputBuffer, 1) + GET_SYMBOL(getOutputFormat, 1) + + GET_SYMBOL(releaseOutputBuffer, 1) + GET_SYMBOL(releaseOutputBufferAtTime, 1) + + GET_SYMBOL(getName, 0) + GET_SYMBOL(releaseName, 0) + +#undef GET_SYMBOL + + switch (method) { + case CREATE_CODEC_BY_NAME: + codec->impl = codec->createCodecByName(arg); + break; + case CREATE_DECODER_BY_TYPE: + codec->impl = codec->createDecoderByType(arg); + break; + case CREATE_ENCODER_BY_TYPE: + codec->impl = codec->createEncoderByType(arg); + break; + default: + av_assert0(0); + } + if (!codec->impl) + goto error; + + return (FFAMediaCodec *)codec; + +error: + if (codec->libmedia) + dlclose(codec->libmedia); + av_freep(&codec); + return NULL; +} + +#define DECLARE_NDK_AMEDIACODEC_CREATE_FUNC(name, method) \ +static FFAMediaCodec *mediacodec_ndk_##name(const char *arg) \ +{ \ + return ndk_codec_create(method, arg); \ +} \ + +DECLARE_NDK_AMEDIACODEC_CREATE_FUNC(createCodecByName, CREATE_CODEC_BY_NAME) +DECLARE_NDK_AMEDIACODEC_CREATE_FUNC(createDecoderByType, CREATE_DECODER_BY_TYPE) +DECLARE_NDK_AMEDIACODEC_CREATE_FUNC(createEncoderByType, CREATE_ENCODER_BY_TYPE) + +static int mediacodec_ndk_delete(FFAMediaCodec* ctx) +{ + FFAMediaCodecNdk *codec = (FFAMediaCodecNdk *)ctx; + int ret = 0; + + if (!codec) + return 0; + + av_assert0(codec->api.class == &amediacodec_ndk_class); + + if (codec->impl && (codec->delete(codec->impl) != AMEDIA_OK)) + ret = AVERROR_EXTERNAL; + if (codec->window) + ANativeWindow_release(codec->window); + if (codec->libmedia) + dlclose(codec->libmedia); + av_free(codec); + + return ret; +} + +static int mediacodec_ndk_configure(FFAMediaCodec* ctx, const FFAMediaFormat* format_ctx, void* surface, void *crypto, uint32_t flags) +{ + FFAMediaCodecNdk *codec = (FFAMediaCodecNdk *)ctx; + FFAMediaFormatNdk *format = (FFAMediaFormatNdk *)format_ctx; + media_status_t status; + + if (surface) { + JNIEnv *env = NULL; + JNI_GET_ENV_OR_RETURN(env, ctx, -1); + codec->window = ANativeWindow_fromSurface(env, surface); + } + + if (format_ctx->class != &amediaformat_ndk_class) { + av_log(ctx, AV_LOG_ERROR, "invalid media format\n"); + return AVERROR(EINVAL); + } + + status = codec->configure(codec->impl, format->impl, codec->window, NULL, flags); + if (status != AMEDIA_OK) { + av_log(codec, AV_LOG_ERROR, "configure failed, %d\n", status); + return AVERROR_EXTERNAL; + } + + return 0; +} + +#define MEDIACODEC_NDK_WRAPPER(method) \ +static int mediacodec_ndk_ ## method(FFAMediaCodec* ctx) \ +{ \ + FFAMediaCodecNdk *codec = (FFAMediaCodecNdk *)ctx; \ + media_status_t status = codec->method(codec->impl); \ + \ + if (status != AMEDIA_OK) { \ + av_log(codec, AV_LOG_ERROR, #method " failed, %d\n", status); \ + return AVERROR_EXTERNAL; \ + } \ + \ + return 0; \ +} \ + +MEDIACODEC_NDK_WRAPPER(start) +MEDIACODEC_NDK_WRAPPER(stop) +MEDIACODEC_NDK_WRAPPER(flush) + +static uint8_t* mediacodec_ndk_getInputBuffer(FFAMediaCodec* ctx, size_t idx, size_t *out_size) +{ + FFAMediaCodecNdk *codec = (FFAMediaCodecNdk *)ctx; + return codec->getInputBuffer(codec->impl, idx, out_size); +} + +static uint8_t* mediacodec_ndk_getOutputBuffer(FFAMediaCodec* ctx, size_t idx, size_t *out_size) +{ + FFAMediaCodecNdk *codec = (FFAMediaCodecNdk *)ctx; + return codec->getOutputBuffer(codec->impl, idx, out_size); +} + +static ssize_t mediacodec_ndk_dequeueInputBuffer(FFAMediaCodec* ctx, int64_t timeoutUs) +{ + FFAMediaCodecNdk *codec = (FFAMediaCodecNdk *)ctx; + return codec->dequeueInputBuffer(codec->impl, timeoutUs); +} + +static int mediacodec_ndk_queueInputBuffer(FFAMediaCodec *ctx, size_t idx, + off_t offset, size_t size, + uint64_t time, uint32_t flags) +{ + FFAMediaCodecNdk *codec = (FFAMediaCodecNdk *)ctx; + return codec->queueInputBuffer(codec->impl, idx, offset, size, time, flags); +} + +static ssize_t mediacodec_ndk_dequeueOutputBuffer(FFAMediaCodec* ctx, FFAMediaCodecBufferInfo *info, int64_t timeoutUs) +{ + FFAMediaCodecNdk *codec = (FFAMediaCodecNdk *)ctx; + AMediaCodecBufferInfo buf_info = {0}; + ssize_t ret; + + ret = codec->dequeueOutputBuffer(codec->impl, &buf_info, timeoutUs); + info->offset = buf_info.offset; + info->size = buf_info.size; + info->presentationTimeUs = buf_info.presentationTimeUs; + info->flags = buf_info.flags; + + return ret; +} + +static FFAMediaFormat* mediacodec_ndk_getOutputFormat(FFAMediaCodec* ctx) +{ + FFAMediaCodecNdk *codec = (FFAMediaCodecNdk *)ctx; + AMediaFormat *format = codec->getOutputFormat(codec->impl); + + if (!format) + return NULL; + return mediaformat_ndk_create(format); +} + +static int mediacodec_ndk_releaseOutputBuffer(FFAMediaCodec* ctx, size_t idx, int render) +{ + FFAMediaCodecNdk *codec = (FFAMediaCodecNdk *)ctx; + media_status_t status; + + status = codec->releaseOutputBuffer(codec->impl, idx, render); + if (status != AMEDIA_OK) { + av_log(codec, AV_LOG_ERROR, "release output buffer failed, %d\n", status); + return AVERROR_EXTERNAL; + } + + return 0; +} + +static int mediacodec_ndk_releaseOutputBufferAtTime(FFAMediaCodec *ctx, size_t idx, int64_t timestampNs) +{ + FFAMediaCodecNdk *codec = (FFAMediaCodecNdk *)ctx; + media_status_t status; + + status = codec->releaseOutputBufferAtTime(codec->impl, idx, timestampNs); + if (status != AMEDIA_OK) { + av_log(codec, AV_LOG_ERROR, "releaseOutputBufferAtTime failed, %d\n", status); + return AVERROR_EXTERNAL; + } + + return 0; +} + +static int mediacodec_ndk_infoTryAgainLater(FFAMediaCodec *ctx, ssize_t idx) +{ + return idx == AMEDIACODEC_INFO_TRY_AGAIN_LATER; +} + +static int mediacodec_ndk_infoOutputBuffersChanged(FFAMediaCodec *ctx, ssize_t idx) +{ + return idx == AMEDIACODEC_INFO_OUTPUT_BUFFERS_CHANGED; +} + +static int mediacodec_ndk_infoOutputFormatChanged(FFAMediaCodec *ctx, ssize_t idx) +{ + return idx == AMEDIACODEC_INFO_OUTPUT_FORMAT_CHANGED; +} + +static int mediacodec_ndk_getBufferFlagCodecConfig(FFAMediaCodec *ctx) +{ + return AMEDIACODEC_BUFFER_FLAG_CODEC_CONFIG; +} + +static int mediacodec_ndk_getBufferFlagEndOfStream(FFAMediaCodec *ctx) +{ + return AMEDIACODEC_BUFFER_FLAG_END_OF_STREAM; +} + +static int mediacodec_ndk_getBufferFlagKeyFrame(FFAMediaCodec *ctx) +{ + return 1; +} + +static int mediacodec_ndk_getConfigureFlagEncode(FFAMediaCodec *ctx) +{ + return AMEDIACODEC_CONFIGURE_FLAG_ENCODE; +} + +static int mediacodec_ndk_cleanOutputBuffers(FFAMediaCodec *ctx) +{ + return 0; +} + +static const FFAMediaFormat media_format_ndk = { + .class = &amediaformat_ndk_class, + + .create = mediaformat_ndk_new, + .delete = mediaformat_ndk_delete, + + .toString = mediaformat_ndk_toString, + + .getInt32 = mediaformat_ndk_getInt32, + .getInt64 = mediaformat_ndk_getInt64, + .getFloat = mediaformat_ndk_getFloat, + .getBuffer = mediaformat_ndk_getBuffer, + .getString = mediaformat_ndk_getString, + + .setInt32 = mediaformat_ndk_setInt32, + .setInt64 = mediaformat_ndk_setInt64, + .setFloat = mediaformat_ndk_setFloat, + .setString = mediaformat_ndk_setString, + .setBuffer = mediaformat_ndk_setBuffer, +}; + +static const FFAMediaCodec media_codec_ndk = { + .class = &amediacodec_ndk_class, + + .getName = mediacodec_ndk_getName, + + .createCodecByName = mediacodec_ndk_createCodecByName, + .createDecoderByType = mediacodec_ndk_createDecoderByType, + .createEncoderByType = mediacodec_ndk_createEncoderByType, + .delete = mediacodec_ndk_delete, + + .configure = mediacodec_ndk_configure, + .start = mediacodec_ndk_start, + .stop = mediacodec_ndk_stop, + .flush = mediacodec_ndk_flush, + + .getInputBuffer = mediacodec_ndk_getInputBuffer, + .getOutputBuffer = mediacodec_ndk_getOutputBuffer, + + .dequeueInputBuffer = mediacodec_ndk_dequeueInputBuffer, + .queueInputBuffer = mediacodec_ndk_queueInputBuffer, + + .dequeueOutputBuffer = mediacodec_ndk_dequeueOutputBuffer, + .getOutputFormat = mediacodec_ndk_getOutputFormat, + + .releaseOutputBuffer = mediacodec_ndk_releaseOutputBuffer, + .releaseOutputBufferAtTime = mediacodec_ndk_releaseOutputBufferAtTime, + + .infoTryAgainLater = mediacodec_ndk_infoTryAgainLater, + .infoOutputBuffersChanged = mediacodec_ndk_infoOutputBuffersChanged, + .infoOutputFormatChanged = mediacodec_ndk_infoOutputFormatChanged, + + .getBufferFlagCodecConfig = mediacodec_ndk_getBufferFlagCodecConfig, + .getBufferFlagEndOfStream = mediacodec_ndk_getBufferFlagEndOfStream, + .getBufferFlagKeyFrame = mediacodec_ndk_getBufferFlagKeyFrame, + + .getConfigureFlagEncode = mediacodec_ndk_getConfigureFlagEncode, + .cleanOutputBuffers = mediacodec_ndk_cleanOutputBuffers, +}; + +FFAMediaFormat *ff_AMediaFormat_new(int ndk) { + if (ndk) + return media_format_ndk.create(); return media_format_jni.create(); } -FFAMediaCodec* ff_AMediaCodec_createCodecByName(const char *name) +FFAMediaCodec* ff_AMediaCodec_createCodecByName(const char *name, int ndk) { + if (ndk) + return media_codec_ndk.createCodecByName(name); return media_codec_jni.createCodecByName(name); } -FFAMediaCodec* ff_AMediaCodec_createDecoderByType(const char *mime_type) +FFAMediaCodec* ff_AMediaCodec_createDecoderByType(const char *mime_type, int ndk) { + if (ndk) + return media_codec_ndk.createDecoderByType(mime_type); return media_codec_jni.createDecoderByType(mime_type); } -FFAMediaCodec* ff_AMediaCodec_createEncoderByType(const char *mime_type) +FFAMediaCodec* ff_AMediaCodec_createEncoderByType(const char *mime_type, int ndk) { + if (ndk) + return media_codec_ndk.createEncoderByType(mime_type); return media_codec_jni.createEncoderByType(mime_type); } diff --git a/libavcodec/mediacodec_wrapper.h b/libavcodec/mediacodec_wrapper.h index 606fdbede5..b9b882f243 100644 --- a/libavcodec/mediacodec_wrapper.h +++ b/libavcodec/mediacodec_wrapper.h @@ -80,7 +80,7 @@ struct FFAMediaFormat { void (*setBuffer)(FFAMediaFormat* format, const char* name, void* data, size_t size); }; -FFAMediaFormat *ff_AMediaFormat_new(void); +FFAMediaFormat *ff_AMediaFormat_new(int ndk); static inline int ff_AMediaFormat_delete(FFAMediaFormat* format) { @@ -198,9 +198,9 @@ static inline char *ff_AMediaCodec_getName(FFAMediaCodec *codec) return codec->getName(codec); } -FFAMediaCodec* ff_AMediaCodec_createCodecByName(const char *name); -FFAMediaCodec* ff_AMediaCodec_createDecoderByType(const char *mime_type); -FFAMediaCodec* ff_AMediaCodec_createEncoderByType(const char *mime_type); +FFAMediaCodec* ff_AMediaCodec_createCodecByName(const char *name, int ndk); +FFAMediaCodec* ff_AMediaCodec_createDecoderByType(const char *mime_type, int ndk); +FFAMediaCodec* ff_AMediaCodec_createEncoderByType(const char *mime_type, int ndk); static inline int ff_AMediaCodec_configure(FFAMediaCodec* codec, const FFAMediaFormat* format, void* surface, void *crypto, uint32_t flags) { diff --git a/libavcodec/mediacodecdec.c b/libavcodec/mediacodecdec.c index 2c66f38541..2e07548b77 100644 --- a/libavcodec/mediacodecdec.c +++ b/libavcodec/mediacodecdec.c @@ -310,7 +310,7 @@ static av_cold int mediacodec_decode_init(AVCodecContext *avctx) FFAMediaFormat *format = NULL; MediaCodecH264DecContext *s = avctx->priv_data; - format = ff_AMediaFormat_new(); + format = ff_AMediaFormat_new(0); if (!format) { av_log(avctx, AV_LOG_ERROR, "Failed to create media format\n"); ret = AVERROR_EXTERNAL; diff --git a/libavcodec/mediacodecdec_common.c b/libavcodec/mediacodecdec_common.c index d6ce709dd8..1905e686bc 100644 --- a/libavcodec/mediacodecdec_common.c +++ b/libavcodec/mediacodecdec_common.c @@ -606,7 +606,7 @@ int ff_mediacodec_dec_init(AVCodecContext *avctx, MediaCodecDecContext *s, } av_log(avctx, AV_LOG_DEBUG, "Found decoder %s\n", s->codec_name); - s->codec = ff_AMediaCodec_createCodecByName(s->codec_name); + s->codec = ff_AMediaCodec_createCodecByName(s->codec_name, 0); if (!s->codec) { av_log(avctx, AV_LOG_ERROR, "Failed to create media decoder for type %s and name %s\n", mime, s->codec_name); ret = AVERROR_EXTERNAL; From patchwork Mon Oct 24 03:16:53 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhao Zhili X-Patchwork-Id: 38948 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:4a86:b0:9d:28a3:170e with SMTP id fn6csp1998332pzb; Sun, 23 Oct 2022 20:19:40 -0700 (PDT) X-Google-Smtp-Source: AMsMyM4MYV87md8i3rhJKIopKcW59o/gqtrkCAiVeYkRPWhW7HPIE7/n0MCLkH0+JeR218/PbIn+ X-Received: by 2002:a05:6402:190a:b0:461:bd00:28c with SMTP id e10-20020a056402190a00b00461bd00028cmr3827370edz.296.1666581579905; Sun, 23 Oct 2022 20:19:39 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666581579; cv=none; d=google.com; s=arc-20160816; b=D+uSBp6GMNib4Evgtp1P75ctFvGGgycOuZr/c+V5xB9W3qHypSGSXK6TyPemKfltHL 2MKxEcIjp7imQBp7t1KG3El/cPVGsskOEULlRqKS0dBBCOBE9w3LCjD5skutbUQR6/ms G8q25kcawtgSgn3sVYL2RG84DGKm409h0rstknWFnW/vPYXxirNdqH1zHYICTSq+qhtA nY2jE5A1oK4jDo920yckMSwJrPqErEWRQ0KCGehKy6xt8xyn984/qkiwdbh5rqDbizZk vHWjh7Jkm1xq+Uusq72KV90Yvlf4bhmEeCzaSXohvJmCzTZyfBUtvxMXg9vSBbqsyYPn stkA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to:date :to:from:message-id:dkim-signature:delivered-to; bh=DOwyBiFIyxadnPHTWQyq5X7SsNZQqc2vLTJ2OCIXBQU=; b=huxK68mh9g0wIfS0FfIiAi0Mwzkw0j7AILh2arBmCgtb3fP8kADkD6pV2omE9mPAvS MbpMmow/78kn0tE0UnWJUbqTr5Y4z1R3ndWfwsQ0ljMms8Ifki81F2kpFVIjl3b3OuWn J1g+aQAsXEsAcHRinwHx6VNaXnYi8hz8MEGelUzul407fGvjXJid9trSBPCDkAs822nj RLx90TNLyASOJgq16vuBt82+63rINM9ScFvF81JjNgat1pEwKJs9hN/peL9v04FxaW9v zSPVfWjGsjSJSUqIYBSlgCbri9AA9Gg/JwMFXYVEbTXJtmjE73hFc9hFuV3aO6Da3WOE jtAw== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@foxmail.com header.s=s201512 header.b=RvDlfFY+; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=foxmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id qb36-20020a1709077ea400b0078d00203ab2si26729150ejc.41.2022.10.23.20.19.39; Sun, 23 Oct 2022 20:19:39 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@foxmail.com header.s=s201512 header.b=RvDlfFY+; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=foxmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 3C36568BD51; Mon, 24 Oct 2022 06:19:37 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from out203-205-221-155.mail.qq.com (out203-205-221-155.mail.qq.com [203.205.221.155]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 3817F68BD2C for ; Mon, 24 Oct 2022 06:17:22 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=foxmail.com; s=s201512; t=1666581440; bh=lAbR27IEHEb4F41Iz9dN7Dclol6/DVv7wDl4Mgl3zHk=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=RvDlfFY+gvP064rdSxLWeg5yrcNpvdaeLrAY7m4uvU+z+60O5skkoOvxxxvspA0Cq xPpLHmQQic8bViXRU9rQsa3c9MBTfba8zPXxMC8kktJoxBO6/oGN+ALuCvto+OzgGz LK3GPH7eorhLURnvVkpa76bPZR8U1NEAfnNtSEPs= Received: from localhost.localdomain ([59.37.125.102]) by newxmesmtplogicsvrsza31.qq.com (NewEsmtp) with SMTP id 44A14270; Mon, 24 Oct 2022 11:17:10 +0800 X-QQ-mid: xmsmtpt1666581439tq4faoaxe Message-ID: X-QQ-XMAILINFO: Mrv6PNPZjcp66JUX8FFyYklFuzJnOCED/CLC4fiptY8d70DPp0FjjCxGGeClq8 ekaqP5mNanjEM1I7uow85luroeBYGBPCzJHI5sd/91rpA/il9tV8Vf/NFBz/O07aG+92EY/K8TmP 9gHdAPgVfYcJ8aAhpwdBcAuvwr3qpdcTLnms+V8Ad0Js2Hrz9xn5nBcId1F51C/9w6n99SAlqXJv Hz/TDWeKXat++EyI2qkd92n18IwKbB/HtmQNJkAQYz1pIc822NNqBzxWII5mdGRKBDs80lwJi5AH VjbuuSizWt1f9xHC7qX0KPCaILZMiAYw5PhURvJhof/G4AeXmm4kRoQ9dGteh9Xo+XEIklCLhzGU o46oPEn4D0NUNnU3/v35jgW8B1SR4tuYOIfSaZ0oYReLOpsF7g1ISk73Rvk6ZjUpwPOZbOTLfDPV m1X+gRrsjMoPSZP8l4TOjOQHLVux7dnmHulAbbs812+FuTevVxCha7CAUf9Rnydv2g0CfpbojPh4 6zPNqImlto5gHNkAVNFDn4KWkcmvPyO5K2c79LEX9FStTJqiFb9GxqE+dCSSpaFWoFFtQZRGz12a Lp+nRkthVD/Ca0AeV3VhGOAhmGtNH6EcaSux8484EgOy3oWw0FX1kScAagUvXSZ7sslLAQUrCi5v 630qbexXIex+gYX4pNW/UapmEEpzhXaBXsIJbmEmsC76Lwq3dubjil7BmvcC5nBU+1fSVad3oiOl Mqes8w5hDRgrThcFeJHURpXHp8PPrMxPYI5v4t9wurMM6KAGsknoaUEudbVlbiYYhYYAd7wk0p12 nHqJ/xLDDSTHYUD9CAjo9NXrLSYHWqPdOGtZq4gLjmPJ2nIf+KVGN0cIHxft5JpBG3atHHyPUBRR ail9pX9f85IEVvp2CgMJNhZeBKqhGPBNF+eud7+NUgc76LcOpYhnMOVifk9yh1uPsYpt19vtdUoQ 85NCGQT975RXU7yVruMyKspNNUBBY8yYWIMiyFFFg/ezVhSEGIlg== From: Zhao Zhili To: ffmpeg-devel@ffmpeg.org Date: Mon, 24 Oct 2022 11:16:53 +0800 X-OQ-MSGID: <20221024031658.1026118-8-quinkblack@foxmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221024031658.1026118-1-quinkblack@foxmail.com> References: <20221024031658.1026118-1-quinkblack@foxmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 07/12] avcodec/mediacodecdec: enable NDK mediacodec X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: matthieu.bouron@stupeflix.com, Zhao Zhili , aman@tmm1.net Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: ypIZz2gxBb7R From: Zhao Zhili Signed-off-by: Zhao Zhili --- Changelog | 1 + libavcodec/mediacodecdec.c | 10 +++++++++- libavcodec/mediacodecdec_common.c | 23 +++++++++++++++++++---- libavcodec/mediacodecdec_common.h | 1 + libavcodec/version.h | 2 +- 5 files changed, 31 insertions(+), 6 deletions(-) diff --git a/Changelog b/Changelog index ec9de1bd85..9e203833aa 100644 --- a/Changelog +++ b/Changelog @@ -18,6 +18,7 @@ version : - Media 100i decoders - DTS to PTS reorder bsf - ViewQuest VQC decoder +- MediaCodec decoder via NDKMediaCodec version 5.1: diff --git a/libavcodec/mediacodecdec.c b/libavcodec/mediacodecdec.c index 2e07548b77..2c231d1a34 100644 --- a/libavcodec/mediacodecdec.c +++ b/libavcodec/mediacodecdec.c @@ -40,6 +40,7 @@ #include "hevc_parse.h" #include "hwconfig.h" #include "internal.h" +#include "jni.h" #include "mediacodec_wrapper.h" #include "mediacodecdec_common.h" @@ -54,6 +55,7 @@ typedef struct MediaCodecH264DecContext { int delay_flush; int amlogic_mpeg2_api23_workaround; + int use_ndk_codec; } MediaCodecH264DecContext; static av_cold int mediacodec_decode_close(AVCodecContext *avctx) @@ -310,7 +312,10 @@ static av_cold int mediacodec_decode_init(AVCodecContext *avctx) FFAMediaFormat *format = NULL; MediaCodecH264DecContext *s = avctx->priv_data; - format = ff_AMediaFormat_new(0); + if (s->use_ndk_codec < 0) + s->use_ndk_codec = !av_jni_get_java_vm(avctx); + + format = ff_AMediaFormat_new(s->use_ndk_codec); if (!format) { av_log(avctx, AV_LOG_ERROR, "Failed to create media format\n"); ret = AVERROR_EXTERNAL; @@ -388,6 +393,7 @@ static av_cold int mediacodec_decode_init(AVCodecContext *avctx) } s->ctx->delay_flush = s->delay_flush; + s->ctx->use_ndk_codec = s->use_ndk_codec; if ((ret = ff_mediacodec_dec_init(avctx, s->ctx, codec_mime, format)) < 0) { s->ctx = NULL; @@ -528,6 +534,8 @@ static const AVCodecHWConfigInternal *const mediacodec_hw_configs[] = { static const AVOption ff_mediacodec_vdec_options[] = { { "delay_flush", "Delay flush until hw output buffers are returned to the decoder", OFFSET(delay_flush), AV_OPT_TYPE_BOOL, {.i64 = 0}, 0, 1, VD }, + { "ndk_codec", "Use MediaCodec from NDK", + OFFSET(use_ndk_codec), AV_OPT_TYPE_BOOL, {.i64 = -1}, -1, 1, VD }, { NULL } }; diff --git a/libavcodec/mediacodecdec_common.c b/libavcodec/mediacodecdec_common.c index 1905e686bc..c3d5988063 100644 --- a/libavcodec/mediacodecdec_common.c +++ b/libavcodec/mediacodecdec_common.c @@ -601,12 +601,27 @@ int ff_mediacodec_dec_init(AVCodecContext *avctx, MediaCodecDecContext *s, s->codec_name = ff_AMediaCodecList_getCodecNameByType(mime, profile, 0, avctx); if (!s->codec_name) { - ret = AVERROR_EXTERNAL; - goto fail; + // getCodecNameByType() can fail due to missing JVM, while NDK + // mediacodec can be used without JVM. + if (!s->use_ndk_codec) { + ret = AVERROR_EXTERNAL; + goto fail; + } + av_log(avctx, AV_LOG_INFO, "Failed to getCodecNameByType\n"); + } else { + av_log(avctx, AV_LOG_DEBUG, "Found decoder %s\n", s->codec_name); } - av_log(avctx, AV_LOG_DEBUG, "Found decoder %s\n", s->codec_name); - s->codec = ff_AMediaCodec_createCodecByName(s->codec_name, 0); + if (s->codec_name) + s->codec = ff_AMediaCodec_createCodecByName(s->codec_name, s->use_ndk_codec); + else { + s->codec = ff_AMediaCodec_createDecoderByType(mime, s->use_ndk_codec); + if (s->codec) { + s->codec_name = ff_AMediaCodec_getName(s->codec); + if (!s->codec_name) + s->codec_name = av_strdup(mime); + } + } if (!s->codec) { av_log(avctx, AV_LOG_ERROR, "Failed to create media decoder for type %s and name %s\n", mime, s->codec_name); ret = AVERROR_EXTERNAL; diff --git a/libavcodec/mediacodecdec_common.h b/libavcodec/mediacodecdec_common.h index 0b21129fee..0ab29036bd 100644 --- a/libavcodec/mediacodecdec_common.h +++ b/libavcodec/mediacodecdec_common.h @@ -70,6 +70,7 @@ typedef struct MediaCodecDecContext { bool delay_flush; atomic_int serial; + bool use_ndk_codec; } MediaCodecDecContext; int ff_mediacodec_dec_init(AVCodecContext *avctx, diff --git a/libavcodec/version.h b/libavcodec/version.h index f8abc803b6..43d0d9a9fc 100644 --- a/libavcodec/version.h +++ b/libavcodec/version.h @@ -30,7 +30,7 @@ #include "version_major.h" #define LIBAVCODEC_VERSION_MINOR 51 -#define LIBAVCODEC_VERSION_MICRO 100 +#define LIBAVCODEC_VERSION_MICRO 101 #define LIBAVCODEC_VERSION_INT AV_VERSION_INT(LIBAVCODEC_VERSION_MAJOR, \ LIBAVCODEC_VERSION_MINOR, \ From patchwork Mon Oct 24 03:16:54 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhao Zhili X-Patchwork-Id: 38942 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:4a86:b0:9d:28a3:170e with SMTP id fn6csp1997872pzb; Sun, 23 Oct 2022 20:18:21 -0700 (PDT) X-Google-Smtp-Source: AMsMyM6Spoj5Uoypi1jIF1+rX+Zdy68MIJIpkgChqcM/H6UcW939PgUL6PlZUcu4ofahdybwJLGb X-Received: by 2002:a17:907:1dda:b0:7a6:8ffc:7dc with SMTP id og26-20020a1709071dda00b007a68ffc07dcmr3669017ejc.163.1666581501770; Sun, 23 Oct 2022 20:18:21 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666581501; cv=none; d=google.com; s=arc-20160816; b=Yq4Xtrm8BmENv4tfOM57v0apuhpcVNRQ8t0O43ppfSiQJ//tWUzt2oHUjEqNvsQbem EAKziU37NiVjFaxWKMFVuSPbiJ+75I1KMmcQwvthBYvE/i6VE+D5JGz6bTJBrYkhUTgW t+isHquylDGarvUT8EBiwDYHtzaLEJbw20g7IUnQJVRREJJMsUIwAos+UXwfaPYPY3Eg pvNSyFSKYtwRTIlu9fQCDQRSoL/O/ZOZo2HDaR8uRVcsfihb8OEbbOrBOrGhruXTS7VZ AUhwF09miRCQ8GLCfSIsItIn9x9zCvBA+YLJpzTYKdAOsTqN2IdRW/VxJPBZoLolI5Zk x9vQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to:date :to:from:message-id:dkim-signature:delivered-to; bh=x/uJdbBlCIACugD4XHgFfqxDSKrgDXImiyvCYtiFslQ=; b=txBdnfEntr2mz3OIXpFERF0gSF8jbh/ePXhkmLC+7spQu8e7sLjHly/Tx7p0Qr705I b2Jn2BiW+U9wBoXarJ+C41cswdnHB5WaRJDBYOFSawKdPOd8PLPCxoEfL1/jLDsfoAFt B7OPIwGejq7xsYG1okXSrMmoAxEltPIPEDZYgexCGyTEmDW6TJTmQJOmTqiqfgm1/esO w1/hAsdsEY3baPf5PPXVmTJpa6B4Rq4WjInzu4UQIU/hiQMxP9ZcLnXjke5iYrXcw0Ju GIRYTCnvQ12qec3bDMF8PcD0P+dUCv6dA82H9At1/paLdatK+vluRMsezlVQM8asYZkS VWbA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@foxmail.com header.s=s201512 header.b=gI41qgAu; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=foxmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id ne41-20020a1709077ba900b00780328a0868si27025588ejc.110.2022.10.23.20.18.21; Sun, 23 Oct 2022 20:18:21 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@foxmail.com header.s=s201512 header.b=gI41qgAu; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=foxmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 0A58468BD4C; Mon, 24 Oct 2022 06:17:34 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from out203-205-221-242.mail.qq.com (out203-205-221-242.mail.qq.com [203.205.221.242]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 59B6B68BD3D for ; Mon, 24 Oct 2022 06:17:26 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=foxmail.com; s=s201512; t=1666581444; bh=9BnfkiiBfux8cKi8l9K5MWN2BnVweN+WILGD7of8wTc=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=gI41qgAumHYQsBZpUYcFLRWO5pA3BnhCzxoC9t6pFd7DBRrTX1uZgfHzCh1fD29oR YFrSQ1aQNsmRzWSZUyVT546bMp7YFEuKuCuqtV3BWuoh78ZZuv3bhcDk/RobfPgKoY CqT893D9kxDuBEBFnVprzi9vspxi3peWmtLWSDsk= Received: from localhost.localdomain ([59.37.125.102]) by newxmesmtplogicsvrsza31.qq.com (NewEsmtp) with SMTP id 44A14270; Mon, 24 Oct 2022 11:17:10 +0800 X-QQ-mid: xmsmtpt1666581440t5phm0krn Message-ID: X-QQ-XMAILINFO: Mrv6PNPZjcp66JUX8FFyYklFuzJnOCED/BYloNalfjiv+K54aCldZWQEkK24ie vtKCnSxCRw9Iwl6/SMaFB/JJxVhiCgYo7APRIdpJcoKO09TT9THf8bd0MiOc7zrH+diPqbhDh4x4 zFF401BBsbQBk6VIm4sWiSJ03QluD4sZg4jteylLT4U9dML3kJdnjF3sLwG4gSxZkrcfyAnRwH0Q 2AjWdApmoQKSmLhMhrYrmZCUJyvBNZyemFOpSbpX87NK/Xz1dqSROEQVliaoZoVcmqwkSrBfwGIf NclgoKf+2c2rrHE/42K1T3YZB/BIU+oDSuGbxmRqks4MAnSEaCa/qjD9Zhw+hFrD/wGjBmhW0dEx 2GT8iRnHqpFxkayTC9C2o2sawPNyT7hB4HBTAMUtIj7a6T3Ms5UtwajKUjWkNAh7QpNHCjRLdCyZ BCGBBjur2jPHj+9wlaThvtpFxnV1F0Y5TPuNgwb3AWV84zYk0XBrPK58Bsuu062K12iOx0A33GJN xsJpdFVeHnGD1o80AIjWC4q6N1YvQzwxtOo5NTSvOKJG70geVx9EfzjdmY6+3LHO4NjBu6iKJtHq Yu3iFdBMbfGOHdsKf8QP0ugWRZPj4Qsr6KbiwqxGwNRFFEsZVQEuLR0eEu7ZIfXadDEMBzZgXrbM 7wxXGkdqL+TV4TGBVgOQO054bqxVI3l47396x/FTbw+mnCfeN5BOjtJ+4pXeV6dlot34eGHKwCUd lmRGGk39qMfuDhIxWuW5Fag0zsu8Je6Zq5lA410Cgr6qoOiIzCymCtQmVmgkNlDwC1aNQ1Y6CiGE 8vMJ0DBHDQsW/U+eIr5GfauDa6wecE41+lDYHY1mfpKf0/JR1YeAxzNgeZ22fYAtS4qNXMy46RFV 61uzXlzWBBm7l0uspqIBELi4V9ECwfZd9msru7ofXbSzQXy6SSq4m1QetEJFN9TTQxFisuKFekCq fvNGvXzjXM9whh5bypi7wmO/9PTvckfE+h1h54pd4= From: Zhao Zhili To: ffmpeg-devel@ffmpeg.org Date: Mon, 24 Oct 2022 11:16:54 +0800 X-OQ-MSGID: <20221024031658.1026118-9-quinkblack@foxmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221024031658.1026118-1-quinkblack@foxmail.com> References: <20221024031658.1026118-1-quinkblack@foxmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 08/12] avutil/hwcontext_mediacodec: add ANativeWindow support X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: matthieu.bouron@stupeflix.com, Zhao Zhili , aman@tmm1.net Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: hC9dRjporax1 From: Zhao Zhili Signed-off-by: Zhao Zhili --- libavutil/hwcontext_mediacodec.c | 56 +++++++++++++++++++++++++++++++- libavutil/hwcontext_mediacodec.h | 11 +++++++ libavutil/version.h | 4 +-- 3 files changed, 68 insertions(+), 3 deletions(-) diff --git a/libavutil/hwcontext_mediacodec.c b/libavutil/hwcontext_mediacodec.c index b0d8993e15..bb1779d34d 100644 --- a/libavutil/hwcontext_mediacodec.c +++ b/libavutil/hwcontext_mediacodec.c @@ -18,12 +18,24 @@ #include "config.h" +#include +#include +#include + #include "buffer.h" #include "common.h" #include "hwcontext.h" #include "hwcontext_internal.h" #include "hwcontext_mediacodec.h" +typedef struct MediaCodecDeviceContext { + AVMediaCodecDeviceContext ctx; + + void *libmedia; + media_status_t (*create_surface)(ANativeWindow **surface); +} MediaCodecDeviceContext; + + static int mc_device_create(AVHWDeviceContext *ctx, const char *device, AVDictionary *opts, int flags) { @@ -35,13 +47,55 @@ static int mc_device_create(AVHWDeviceContext *ctx, const char *device, return 0; } +static int mc_device_init(AVHWDeviceContext *ctx) +{ + MediaCodecDeviceContext *s = ctx->hwctx; + AVMediaCodecDeviceContext *dev = (AVMediaCodecDeviceContext *)s; + ANativeWindow *native_window = NULL; + + if (dev->surface) + return 0; + + if (dev->native_window) + return 0; + + s->libmedia = dlopen("libmediandk.so", RTLD_NOW); + if (!s->libmedia) + return AVERROR_UNKNOWN; + + s->create_surface = dlsym(s->libmedia, "AMediaCodec_createPersistentInputSurface"); + if (!s->create_surface) + return AVERROR_UNKNOWN; + + s->create_surface(&native_window); + dev->native_window = native_window; + return 0; +} + +static void mc_device_uninit(AVHWDeviceContext *ctx) +{ + MediaCodecDeviceContext *s = ctx->hwctx; + AVMediaCodecDeviceContext *dev = ctx->hwctx; + if (!s->libmedia) + return; + + if (dev->native_window) { + ANativeWindow_release(dev->native_window); + dev->native_window = NULL; + } + dlclose(s->libmedia); + s->libmedia = NULL; +} + const HWContextType ff_hwcontext_type_mediacodec = { .type = AV_HWDEVICE_TYPE_MEDIACODEC, .name = "mediacodec", - .device_hwctx_size = sizeof(AVMediaCodecDeviceContext), + .device_hwctx_size = sizeof(MediaCodecDeviceContext), .device_create = mc_device_create, + .device_init = mc_device_init, + .device_uninit = mc_device_uninit, .pix_fmts = (const enum AVPixelFormat[]){ AV_PIX_FMT_MEDIACODEC, diff --git a/libavutil/hwcontext_mediacodec.h b/libavutil/hwcontext_mediacodec.h index 101a9806d5..920e17764f 100644 --- a/libavutil/hwcontext_mediacodec.h +++ b/libavutil/hwcontext_mediacodec.h @@ -31,6 +31,17 @@ typedef struct AVMediaCodecDeviceContext { * This is the default surface used by decoders on this device. */ void *surface; + + /** + * Pointer to ANativeWindow. + * + * It both surface and native_window is NULL, try to create it + * automatically if OS support. + * + * It can be used as output surface for decoder and input surface for + * encoder. + */ + void *native_window; } AVMediaCodecDeviceContext; #endif /* AVUTIL_HWCONTEXT_MEDIACODEC_H */ diff --git a/libavutil/version.h b/libavutil/version.h index cb0c928bd0..2df788e529 100644 --- a/libavutil/version.h +++ b/libavutil/version.h @@ -79,8 +79,8 @@ */ #define LIBAVUTIL_VERSION_MAJOR 57 -#define LIBAVUTIL_VERSION_MINOR 39 -#define LIBAVUTIL_VERSION_MICRO 101 +#define LIBAVUTIL_VERSION_MINOR 40 +#define LIBAVUTIL_VERSION_MICRO 100 #define LIBAVUTIL_VERSION_INT AV_VERSION_INT(LIBAVUTIL_VERSION_MAJOR, \ LIBAVUTIL_VERSION_MINOR, \ From patchwork Mon Oct 24 03:16:55 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhao Zhili X-Patchwork-Id: 38947 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:4a86:b0:9d:28a3:170e with SMTP id fn6csp1998089pzb; Sun, 23 Oct 2022 20:18:55 -0700 (PDT) X-Google-Smtp-Source: AMsMyM7voeIXuwfdxkz7GCG1zAGQqKk/4lM1iNSh9I0Ic5pg//fgjJ/JsA6huYISmG2nW32mxHEI X-Received: by 2002:a05:6402:5107:b0:45d:409:1337 with SMTP id m7-20020a056402510700b0045d04091337mr29573511edd.185.1666581535235; Sun, 23 Oct 2022 20:18:55 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666581535; cv=none; d=google.com; s=arc-20160816; b=GLMQP+gr0tjKMV4IbuCScFEoH7NzWgff/cedjYqyVzrPYyC8XsOCeHa+tGqkUTv8e9 b2HyBe5t5aLl9elBIR8qfQgZeLBBoCwXQ6OylkS1yIRmIBibufDe7/AZtIlmdtLwsvs+ jXqnGeH98zWGZSUPeJ0sS/LUdM55bocSdDanoT8I9VEcK0kL0N5KSFKpvEktsUnWbfbZ rEUu4UF3hx9dxgb80CliRXwFi0jL6wN+Ls26J20IHrDRdMtxH6u6dobHFL2yq+YlLLDT +625DcDC3dMMF9fxWyDvMNVhcCYNmEthukCVnRVa7Y6m2o5wBb1+6CHEoXCx7KjtZBve 7tqg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to:date :to:from:message-id:dkim-signature:delivered-to; bh=C9JEYdJVz+lcQ7GjHNHB/wBO2ttb9BfcOKpvmT04GUQ=; b=OmfxcZJygAdCwt1Cu5tyEQk3nnMymiMarCb6VON01OtJ/ClYUFFfCWuppyydIpAc5y Y00h7UwBt7eL6Q5FZrbxLxdpSkkhR4EfejiDWGULn3QyRatbL3zuq+NJffSsjHEOzPnA fFTOFSAKPwgQtL6RzcwLcYMgrNWqfMKqwzqfO+HtolIIYmtUTTh1NE5DwiTAZowTtoDX kVhKzjJ4Su4EdeyLfkYgHOG63r9DLjAxJvgce+m0e5G6oVSQPFtGsuk0npZ/5XeTn2xZ CYHwsUxNH99EgStPkSthG2GSVBNLvg4Anl54v3WzFPq9Cr6ekKZsVsqaQnOH+PmSAeRr T+/w== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@foxmail.com header.s=s201512 header.b="TuDt0r/Z"; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=foxmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id hd43-20020a17090796ab00b0079fe11e92ddsi6945731ejc.906.2022.10.23.20.18.54; Sun, 23 Oct 2022 20:18:55 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@foxmail.com header.s=s201512 header.b="TuDt0r/Z"; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=foxmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 0AE8268BDAC; Mon, 24 Oct 2022 06:18:42 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from out162-62-58-216.mail.qq.com (out162-62-58-216.mail.qq.com [162.62.58.216]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id A8E1E68BDA4 for ; Mon, 24 Oct 2022 06:18:35 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=foxmail.com; s=s201512; t=1666581512; bh=kRsGIYX5HznuLWEGDPSIMmGJQO4i46DklQ/meCPAY2U=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=TuDt0r/Z4FSFLJ/cwxOgzDJHKCUohQx5BROnN+Za1NjSEtHZuLbPWwK9OhGFXDHRi ZhOPIWZqY2hpaPQw0RDCfX0Dc2ucfyIicmciaWOQt1ZsKoVps/Dhkc+UmbsOSk43vJ wFVy1Fp8rRucG8sXqyF72eNbbWnqlt0Iz8MTiS4Y= Received: from localhost.localdomain ([59.37.125.102]) by newxmesmtplogicsvrsza31.qq.com (NewEsmtp) with SMTP id 44A14270; Mon, 24 Oct 2022 11:17:10 +0800 X-QQ-mid: xmsmtpt1666581441tflbousg8 Message-ID: X-QQ-XMAILINFO: NQR8mRxMnur9/ThzoXtse/E97HdkXTsm+svZQwNHqrKg+NFuKa2O3LXgCstF0f RmKrwzZfgwB8oK0iYioKCdXNQSraEYTF5VzKPxpvS/vhqlu8vaNq1Oa9hMjA7iTkTR8OxLCiLcbp pr6qsCWExkauB2CDyNHb12n7xXeXdmW03q00BmkNF69XKpNuZ+2UGfqRwZmBggJu77ETv/Po90fx znO7s2ZqVa66zX8Gy02aSwQcKtyas18Z8apFrD2+crySfXUy+IFKqqxSp/OspydbyyuVy9ZfOURf yeVcqEX0i2nI65FacNzXLAHrRCYqvJEa5SQTFRLo7VkJDUJdU/YcCMJG0+XWPKlTsJHF1Bt93vhV YMzFguknd9QSOgnW85G8zD6ltQdz+ncCxLi3gLE2fdSuC92qAya2ipwhpYUrCoirpJmu/BOE6scA 1/HCFAxeHT5z21vcLObL5MPyXe0dO3Q8PyocJAUeqyGeybMkPuvo+MlCmVssxmcfU134mgnEQWir 7N7Divb0+iquDIFtA2OFsDXJtZVS+7/lY5rakzf2zjqxGnyuo1G5/BXavXH+fkgU2n5BnaZXWBEN J5Ga5Uj/8GWRO4HkFLGFh2buHtIZHEQHR5VFxZ0XaWAFhzktyzwAoQfobpJ05b7jDV+bO8ZkSyI/ fB4K8Wl1A9F4gVMkftybFEy09TZWm9eFGeZ/+I7+c7LDA6X47sPR0io3teFoaXY3fna4EaybpjkB BN6EMV0GcZijp+P89KCQaJPbG5UQ9bEUSYkqI1IlMu8032wF5yP7Z+U4vmS7U3lcprtfkyk9zC1T Tk5HI09grpaDozue2MBJ2fps6j8Hr5SeFDim1gw1KpqBQqI+bPo9FLuyMNL2payikGuNQ34Cutqp g6UGSG2QtsXAfx5N1fTVbheqMKaaq0RBnU/xrHjKOBXwT7C84IbrXDpb6R5snkCBIfd2OdGX+czr zsXq8GZYFo1mgpFf0T4ZSCMbo4DeIUAjFE5zredULqt4mzZGkrIAmAU66DPqoJ From: Zhao Zhili To: ffmpeg-devel@ffmpeg.org Date: Mon, 24 Oct 2022 11:16:55 +0800 X-OQ-MSGID: <20221024031658.1026118-10-quinkblack@foxmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221024031658.1026118-1-quinkblack@foxmail.com> References: <20221024031658.1026118-1-quinkblack@foxmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 09/12] avcodec/mediacodec: add ANativeWindow support X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: matthieu.bouron@stupeflix.com, Zhao Zhili , aman@tmm1.net Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: zbX9pxQXtONr From: Zhao Zhili ANativeWindow can be used without JVM. --- libavcodec/mediacodec_surface.c | 46 ++++++++++++++++++++++++------- libavcodec/mediacodec_surface.h | 8 ++++-- libavcodec/mediacodec_wrapper.c | 30 +++++++++++++++----- libavcodec/mediacodec_wrapper.h | 8 ++++-- libavcodec/mediacodecdec_common.c | 4 +-- 5 files changed, 72 insertions(+), 24 deletions(-) diff --git a/libavcodec/mediacodec_surface.c b/libavcodec/mediacodec_surface.c index 09a42295d2..ef41cdafa7 100644 --- a/libavcodec/mediacodec_surface.c +++ b/libavcodec/mediacodec_surface.c @@ -20,33 +20,59 @@ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA */ +#include #include +#include "libavutil/mem.h" #include "ffjni.h" #include "mediacodec_surface.h" -FFANativeWindow *ff_mediacodec_surface_ref(void *surface, void *log_ctx) +FFANativeWindow *ff_mediacodec_surface_ref(void *surface, void *native_window, void *log_ctx) { - JNIEnv *env = NULL; + FFANativeWindow *ret; - env = ff_jni_get_env(log_ctx); - if (!env) { + ret = av_mallocz(sizeof(*ret)); + if (!ret) return NULL; + + if (surface) { + JNIEnv *env = NULL; + + env = ff_jni_get_env(log_ctx); + if (env) + ret->surface = (*env)->NewGlobalRef(env, surface); + } + + if (native_window) { + ANativeWindow_acquire(native_window); + ret->native_window = native_window; } - return (*env)->NewGlobalRef(env, surface); + if (!ret->surface && !ret->native_window) { + av_log(log_ctx, AV_LOG_ERROR, "Both surface and native_window are NULL\n"); + av_freep(&ret); + } + + return ret; } int ff_mediacodec_surface_unref(FFANativeWindow *window, void *log_ctx) { - JNIEnv *env = NULL; + if (!window) + return 0; - env = ff_jni_get_env(log_ctx); - if (!env) { - return AVERROR_EXTERNAL; + if (window->surface) { + JNIEnv *env = NULL; + + env = ff_jni_get_env(log_ctx); + if (env) + (*env)->DeleteGlobalRef(env, window->surface); } - (*env)->DeleteGlobalRef(env, window); + if (window->native_window) + ANativeWindow_release(window->native_window); + + av_free(window); return 0; } diff --git a/libavcodec/mediacodec_surface.h b/libavcodec/mediacodec_surface.h index 933dc2bf51..e2ac1c9057 100644 --- a/libavcodec/mediacodec_surface.h +++ b/libavcodec/mediacodec_surface.h @@ -25,10 +25,12 @@ #include "libavcodec/avcodec.h" -struct FFANativeWindow; -typedef struct FFANativeWindow FFANativeWindow; +typedef struct FFANativeWindow { + void *surface; + void *native_window; +} FFANativeWindow; -FFANativeWindow *ff_mediacodec_surface_ref(void *surface, void *log_ctx); +FFANativeWindow *ff_mediacodec_surface_ref(void *surface, void *native_window, void *log_ctx); int ff_mediacodec_surface_unref(FFANativeWindow *window, void *log_ctx); #endif /* AVCODEC_MEDIACODEC_SURFACE_H */ diff --git a/libavcodec/mediacodec_wrapper.c b/libavcodec/mediacodec_wrapper.c index b12aced711..284d615980 100644 --- a/libavcodec/mediacodec_wrapper.c +++ b/libavcodec/mediacodec_wrapper.c @@ -1371,12 +1371,17 @@ fail: return ret; } -static int mediacodec_jni_configure(FFAMediaCodec* ctx, const FFAMediaFormat* format_ctx, void* surface, void *crypto, uint32_t flags) +static int mediacodec_jni_configure(FFAMediaCodec *ctx, + const FFAMediaFormat* format_ctx, + FFANativeWindow* window, + void *crypto, + uint32_t flags) { int ret = 0; JNIEnv *env = NULL; FFAMediaCodecJni *codec = (FFAMediaCodecJni *)ctx; const FFAMediaFormatJni *format = (FFAMediaFormatJni *)format_ctx; + jobject *surface = window ? window->surface : NULL; JNI_GET_ENV_OR_RETURN(env, codec, AVERROR_EXTERNAL); @@ -2151,16 +2156,27 @@ static int mediacodec_ndk_delete(FFAMediaCodec* ctx) return ret; } -static int mediacodec_ndk_configure(FFAMediaCodec* ctx, const FFAMediaFormat* format_ctx, void* surface, void *crypto, uint32_t flags) +static int mediacodec_ndk_configure(FFAMediaCodec* ctx, + const FFAMediaFormat* format_ctx, + FFANativeWindow* window, + void *crypto, + uint32_t flags) { FFAMediaCodecNdk *codec = (FFAMediaCodecNdk *)ctx; FFAMediaFormatNdk *format = (FFAMediaFormatNdk *)format_ctx; media_status_t status; + ANativeWindow *native_window = NULL; - if (surface) { - JNIEnv *env = NULL; - JNI_GET_ENV_OR_RETURN(env, ctx, -1); - codec->window = ANativeWindow_fromSurface(env, surface); + if (window) { + if (window->surface) { + JNIEnv *env = NULL; + JNI_GET_ENV_OR_RETURN(env, ctx, -1); + native_window = ANativeWindow_fromSurface(env, window->surface); + // Save for release + codec->window = native_window; + } else if (window->native_window) { + native_window = window->native_window; + } } if (format_ctx->class != &amediaformat_ndk_class) { @@ -2168,7 +2184,7 @@ static int mediacodec_ndk_configure(FFAMediaCodec* ctx, const FFAMediaFormat* fo return AVERROR(EINVAL); } - status = codec->configure(codec->impl, format->impl, codec->window, NULL, flags); + status = codec->configure(codec->impl, format->impl, native_window, NULL, flags); if (status != AMEDIA_OK) { av_log(codec, AV_LOG_ERROR, "configure failed, %d\n", status); return AVERROR_EXTERNAL; diff --git a/libavcodec/mediacodec_wrapper.h b/libavcodec/mediacodec_wrapper.h index b9b882f243..7cf3f4aecd 100644 --- a/libavcodec/mediacodec_wrapper.h +++ b/libavcodec/mediacodec_wrapper.h @@ -27,6 +27,7 @@ #include #include "avcodec.h" +#include "mediacodec_surface.h" /** * The following API around MediaCodec and MediaFormat is based on the @@ -163,7 +164,7 @@ struct FFAMediaCodec { FFAMediaCodec* (*createEncoderByType)(const char *mime_type); int (*delete)(FFAMediaCodec* codec); - int (*configure)(FFAMediaCodec* codec, const FFAMediaFormat* format, void* surface, void *crypto, uint32_t flags); + int (*configure)(FFAMediaCodec* codec, const FFAMediaFormat* format, FFANativeWindow* surface, void *crypto, uint32_t flags); int (*start)(FFAMediaCodec* codec); int (*stop)(FFAMediaCodec* codec); int (*flush)(FFAMediaCodec* codec); @@ -202,7 +203,10 @@ FFAMediaCodec* ff_AMediaCodec_createCodecByName(const char *name, int ndk); FFAMediaCodec* ff_AMediaCodec_createDecoderByType(const char *mime_type, int ndk); FFAMediaCodec* ff_AMediaCodec_createEncoderByType(const char *mime_type, int ndk); -static inline int ff_AMediaCodec_configure(FFAMediaCodec* codec, const FFAMediaFormat* format, void* surface, void *crypto, uint32_t flags) +static inline int ff_AMediaCodec_configure(FFAMediaCodec *codec, + const FFAMediaFormat *format, + FFANativeWindow *surface, + void *crypto, uint32_t flags) { return codec->configure(codec, format, surface, crypto, flags); } diff --git a/libavcodec/mediacodecdec_common.c b/libavcodec/mediacodecdec_common.c index c3d5988063..d56412aa6d 100644 --- a/libavcodec/mediacodecdec_common.c +++ b/libavcodec/mediacodecdec_common.c @@ -582,14 +582,14 @@ int ff_mediacodec_dec_init(AVCodecContext *avctx, MediaCodecDecContext *s, if (device_ctx->type == AV_HWDEVICE_TYPE_MEDIACODEC) { if (device_ctx->hwctx) { AVMediaCodecDeviceContext *mediacodec_ctx = (AVMediaCodecDeviceContext *)device_ctx->hwctx; - s->surface = ff_mediacodec_surface_ref(mediacodec_ctx->surface, avctx); + s->surface = ff_mediacodec_surface_ref(mediacodec_ctx->surface, mediacodec_ctx->native_window, avctx); av_log(avctx, AV_LOG_INFO, "Using surface %p\n", s->surface); } } } if (!s->surface && user_ctx && user_ctx->surface) { - s->surface = ff_mediacodec_surface_ref(user_ctx->surface, avctx); + s->surface = ff_mediacodec_surface_ref(user_ctx->surface, NULL, avctx); av_log(avctx, AV_LOG_INFO, "Using surface %p\n", s->surface); } } From patchwork Mon Oct 24 03:16:56 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhao Zhili X-Patchwork-Id: 38941 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:4a86:b0:9d:28a3:170e with SMTP id fn6csp1997825pzb; Sun, 23 Oct 2022 20:18:13 -0700 (PDT) X-Google-Smtp-Source: AMsMyM771XaspFEIqkCWMIGGw9gHBt1A2tMwVBNyBrAaQp/jD6lvxOu7Q1W4fJ32Fp//vNo1wiqV X-Received: by 2002:a17:907:60c8:b0:78d:bc56:8f04 with SMTP id hv8-20020a17090760c800b0078dbc568f04mr24612028ejc.560.1666581493085; Sun, 23 Oct 2022 20:18:13 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666581493; cv=none; d=google.com; s=arc-20160816; b=FI2P2QbuiBdxq1H5OBpP9DtiAiH1cDsppvZaGbmf1tuERR+WtlCNjbW5vhu7ZC5iBX y+OWVZZvT+VdrMLH3xbwZxOpI0dTtRBb0HjpSKlsBt5U/0eQMPFhJqMe42Pbb1QWNlx+ ARi5P9lh4HbX1PaxPbiaXC56wgNs270f+STUt8wVjQ0nPzLifJqJIQPJtvLiL8fKUR4b PrMunNZNrq780JVE+5nXjWXXoQKnJAe2t70iYA4zPyr/YlntIrmSFeq1/dvLXtawdwF/ 9LyHKZG3+1pG2e7YIGo/bEsxN5Az9OOqugfebwbcjBvabe9XnWDRDRSpLjI0m1wNPWIp oVYw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to:date :to:from:message-id:dkim-signature:delivered-to; bh=7jp99UjhDe0HyVrhewXjOYfFUdfE8P0VLg9Euh9G5qc=; b=Hie6jBCeokkD0aVsZLdjXm+znc6GVHKoguUjLwMmrtmX1EuouY22qzp/DvZRpM9rjq QMSzU8yz8zZa1uHHvVt7icxXy7HnF+aTU+JJqAU1tZuoCzEC3KskmWzmPrU+ywZK36lx MHzBX1BQ/pOW1j3g6d7ydxRW3zDoTmPNzw0nIB3wQFiE8hUr+SUnQ8ZV4Kngbp6d1CqE OktTcl5KL/WRvgO75VqL0BKB+yzf+LlPsPJj+UtnieuO0WSCtzCD5mx+lSPeKcHyhoBY tKJOei8oEbtY0DezuP1L4lgNHDdaD919uQFuid582wlwfrrjMZgK8ZxItd1a3SrgbOIc swAA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@foxmail.com header.s=s201512 header.b=a7QbAvO+; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=foxmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id q22-20020a170906771600b0078da3218b54si25762409ejm.171.2022.10.23.20.18.12; Sun, 23 Oct 2022 20:18:13 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@foxmail.com header.s=s201512 header.b=a7QbAvO+; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=foxmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 154AD68BD5A; Mon, 24 Oct 2022 06:17:33 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from out203-205-221-233.mail.qq.com (out203-205-221-233.mail.qq.com [203.205.221.233]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 1DBA368BD36 for ; Mon, 24 Oct 2022 06:17:25 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=foxmail.com; s=s201512; t=1666581443; bh=Tgk0cgXApfjEvmBYMJye+AF3qmZAo0wwCyz0ufLtWxM=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=a7QbAvO+h/wTcpJYi3PUlIGTV/CWIYw5HBSMjwyOdaDWg4e+65d+CUcaUBoLdFhLJ eJtYsXaXuctEvj7dZzpAFKnjh1/X/Vb3KrbB6ppLbYN7oEb3ttgj0/dFEfF2WgJMKe jOtasipUv77M98LvBcgKEPHbaT3k2UnCxXakTuqM= Received: from localhost.localdomain ([59.37.125.102]) by newxmesmtplogicsvrsza31.qq.com (NewEsmtp) with SMTP id 44A14270; Mon, 24 Oct 2022 11:17:10 +0800 X-QQ-mid: xmsmtpt1666581442tamh9nmkl Message-ID: X-QQ-XMAILINFO: N7h1OCCDntujXzJrSiYyeGwnj9V42ihEiOqdX6bcKI+/xIkaKSXu4dVg8puk8H k7OzC1XwnaGSW2yxf4FID7BF19+O+xqOS4q5vbQ0AOXJMr4wzcE6ojMDe3iHkqVkbsY0aEdU151K Ai85thjrWMYFBlZ1zt+TxigEn30aqjg5Tmg3JMiX+2XdISAHKXRHL8KKU+apFCMBjxVDyqNHi8PB VdfZdOnaZwkGo0IFwZTxhpIgra1RsfGY4Hb863Gle3OMetOLE38NiucYxI7+tPcKtb6PoUXYFj9S 2XEvx8bVfdLHV1/0sChuL89xKup8I9vO+y11FZ0VaqTZdfA3lsi9pTCfqm8x1CqUmtE0G63Qx6DZ 7q5pfu+O+IXfsTQ2ebjWj1uc8GbIvLIKLwWIMhLJKBC1c2z9HmAvKQgDovmhM+KOFfzGHdpywxfF T+VlvXw3ZMqyj8wbkiZWzg6dw827QfZREF0anGN+P2dsxnnlhntL+pndNZpdgPetjsNyNhqtgfha uhROOlFSv4iUWRLILD/YXGN35QDWjzxRaN0+buj/TL1iVwOYGeLJOkeKeyW1poX2AvG5Jd/+kzxl 0E17sDg9RDzIuq/B8f8Rgs6YiJgvRCthnVzTSJ0X6clY9S/8vlSAjiKtHAGJz8G35MfM/yjyf5Mt L250DcvSPooFE5au9+paYvq/QKoiNCJnPfw5bfMDaXvmAWD0IoanT0b3BjG720aFQafelMfDDE5A vEWvsF49oIb4fWxrpFXL2IGPZ0aRMAhn33qG/vKb8OmHS1AlXg1cl8+sl3DMmAwwLoSMH4BTQXKb p82ZHem2QzfuPenJ6LX7hVlPU+/I7sKBZwPqumr2qgqHEn7AKI7wncGOEuDQwC0wa6E3EYz9Hz56 l4/VgdAfO6cqUdvw8MoPBOvIa/0WhbKaS50T9adGeFcFTwNoVn0kpNn1YVyiDGXkl771LvDoLr3L tAwnTFuBsqU72LrJHNRLE8Uk55RPcq1EAsfchjl9afGCZF4RoaUw== From: Zhao Zhili To: ffmpeg-devel@ffmpeg.org Date: Mon, 24 Oct 2022 11:16:56 +0800 X-OQ-MSGID: <20221024031658.1026118-11-quinkblack@foxmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221024031658.1026118-1-quinkblack@foxmail.com> References: <20221024031658.1026118-1-quinkblack@foxmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 10/12] avcodec: add MediaCodec encoder X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: matthieu.bouron@stupeflix.com, Zhao Zhili , aman@tmm1.net Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: QTUVlC5/t/7X From: Zhao Zhili Signed-off-by: Zhao Zhili --- Changelog | 1 + configure | 4 + libavcodec/Makefile | 2 + libavcodec/allcodecs.c | 2 + libavcodec/mediacodec_wrapper.c | 102 ++++++- libavcodec/mediacodec_wrapper.h | 8 + libavcodec/mediacodecenc.c | 495 ++++++++++++++++++++++++++++++++ libavcodec/version.h | 4 +- 8 files changed, 611 insertions(+), 7 deletions(-) create mode 100644 libavcodec/mediacodecenc.c diff --git a/Changelog b/Changelog index 9e203833aa..9e39a35972 100644 --- a/Changelog +++ b/Changelog @@ -19,6 +19,7 @@ version : - DTS to PTS reorder bsf - ViewQuest VQC decoder - MediaCodec decoder via NDKMediaCodec +- MediaCodec encoder version 5.1: diff --git a/configure b/configure index ee2e3ba6ac..5114cda13f 100755 --- a/configure +++ b/configure @@ -3193,6 +3193,8 @@ h264_cuvid_decoder_select="h264_mp4toannexb_bsf" h264_mediacodec_decoder_deps="mediacodec" h264_mediacodec_decoder_extralibs="-landroid" h264_mediacodec_decoder_select="h264_mp4toannexb_bsf h264_parser" +h264_mediacodec_encoder_deps="mediacodec" +h264_mediacodec_encoder_extralibs="-landroid" h264_mf_encoder_deps="mediafoundation" h264_mmal_decoder_deps="mmal" h264_nvenc_encoder_deps="nvenc" @@ -3212,6 +3214,8 @@ hevc_cuvid_decoder_select="hevc_mp4toannexb_bsf" hevc_mediacodec_decoder_deps="mediacodec" hevc_mediacodec_decoder_extralibs="-landroid" hevc_mediacodec_decoder_select="hevc_mp4toannexb_bsf hevc_parser" +hevc_mediacodec_encoder_deps="mediacodec" +hevc_mediacodec_encoder_extralibs="-landroid" hevc_mf_encoder_deps="mediafoundation" hevc_nvenc_encoder_deps="nvenc" hevc_nvenc_encoder_select="atsc_a53" diff --git a/libavcodec/Makefile b/libavcodec/Makefile index 90c7f113a3..7d0b513eec 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -393,6 +393,7 @@ OBJS-$(CONFIG_H264_DECODER) += h264dec.o h264_cabac.o h264_cavlc.o \ OBJS-$(CONFIG_H264_AMF_ENCODER) += amfenc_h264.o OBJS-$(CONFIG_H264_CUVID_DECODER) += cuviddec.o OBJS-$(CONFIG_H264_MEDIACODEC_DECODER) += mediacodecdec.o +OBJS-$(CONFIG_H264_MEDIACODEC_ENCODER) += mediacodecenc.o OBJS-$(CONFIG_H264_MF_ENCODER) += mfenc.o mf_utils.o OBJS-$(CONFIG_H264_MMAL_DECODER) += mmaldec.o OBJS-$(CONFIG_H264_NVENC_ENCODER) += nvenc_h264.o nvenc.o @@ -417,6 +418,7 @@ OBJS-$(CONFIG_HEVC_DECODER) += hevcdec.o hevc_mvs.o \ OBJS-$(CONFIG_HEVC_AMF_ENCODER) += amfenc_hevc.o OBJS-$(CONFIG_HEVC_CUVID_DECODER) += cuviddec.o OBJS-$(CONFIG_HEVC_MEDIACODEC_DECODER) += mediacodecdec.o +OBJS-$(CONFIG_HEVC_MEDIACODEC_ENCODER) += mediacodecenc.o OBJS-$(CONFIG_HEVC_MF_ENCODER) += mfenc.o mf_utils.o OBJS-$(CONFIG_HEVC_NVENC_ENCODER) += nvenc_hevc.o nvenc.o OBJS-$(CONFIG_HEVC_QSV_DECODER) += qsvdec.o diff --git a/libavcodec/allcodecs.c b/libavcodec/allcodecs.c index 46ad3b5a25..4c33a9ec3c 100644 --- a/libavcodec/allcodecs.c +++ b/libavcodec/allcodecs.c @@ -154,6 +154,7 @@ extern const FFCodec ff_h264_decoder; extern const FFCodec ff_h264_crystalhd_decoder; extern const FFCodec ff_h264_v4l2m2m_decoder; extern const FFCodec ff_h264_mediacodec_decoder; +extern const FFCodec ff_h264_mediacodec_encoder; extern const FFCodec ff_h264_mmal_decoder; extern const FFCodec ff_h264_qsv_decoder; extern const FFCodec ff_h264_rkmpp_decoder; @@ -842,6 +843,7 @@ extern const FFCodec ff_h264_videotoolbox_encoder; extern const FFCodec ff_hevc_amf_encoder; extern const FFCodec ff_hevc_cuvid_decoder; extern const FFCodec ff_hevc_mediacodec_decoder; +extern const FFCodec ff_hevc_mediacodec_encoder; extern const FFCodec ff_hevc_mf_encoder; extern const FFCodec ff_hevc_nvenc_encoder; extern const FFCodec ff_hevc_qsv_encoder; diff --git a/libavcodec/mediacodec_wrapper.c b/libavcodec/mediacodec_wrapper.c index 284d615980..5d1a32031d 100644 --- a/libavcodec/mediacodec_wrapper.c +++ b/libavcodec/mediacodec_wrapper.c @@ -212,6 +212,9 @@ struct JNIAMediaCodecFields { jmethodID release_output_buffer_id; jmethodID release_output_buffer_at_time_id; + jmethodID set_input_surface_id; + jmethodID signal_end_of_input_stream_id; + jclass mediainfo_class; jmethodID init_id; @@ -261,6 +264,9 @@ static const struct FFJniField jni_amediacodec_mapping[] = { { "android/media/MediaCodec", "releaseOutputBuffer", "(IZ)V", FF_JNI_METHOD, offsetof(struct JNIAMediaCodecFields, release_output_buffer_id), 1 }, { "android/media/MediaCodec", "releaseOutputBuffer", "(IJ)V", FF_JNI_METHOD, offsetof(struct JNIAMediaCodecFields, release_output_buffer_at_time_id), 0 }, + { "android/media/MediaCodec", "setInputSurface", "(Landroid/view/Surface;)V", FF_JNI_METHOD, offsetof(struct JNIAMediaCodecFields, set_input_surface_id), 0 }, + { "android/media/MediaCodec", "signalEndOfInputStream", "()V", FF_JNI_METHOD, offsetof(struct JNIAMediaCodecFields, signal_end_of_input_stream_id), 0 }, + { "android/media/MediaCodec$BufferInfo", NULL, NULL, FF_JNI_CLASS, offsetof(struct JNIAMediaCodecFields, mediainfo_class), 1 }, { "android/media/MediaCodec.BufferInfo", "", "()V", FF_JNI_METHOD, offsetof(struct JNIAMediaCodecFields, init_id), 1 }, @@ -1385,7 +1391,26 @@ static int mediacodec_jni_configure(FFAMediaCodec *ctx, JNI_GET_ENV_OR_RETURN(env, codec, AVERROR_EXTERNAL); - (*env)->CallVoidMethod(env, codec->object, codec->jfields.configure_id, format->object, surface, NULL, flags); + if (flags & codec->CONFIGURE_FLAG_ENCODE) { + if (surface && !codec->jfields.set_input_surface_id) { + av_log(ctx, AV_LOG_ERROR, "System doesn't support setInputSurface\n"); + return AVERROR_EXTERNAL; + } + + (*env)->CallVoidMethod(env, codec->object, codec->jfields.configure_id, format->object, NULL, NULL, flags); + if (ff_jni_exception_check(env, 1, codec) < 0) + return AVERROR_EXTERNAL; + + if (!surface) + return 0; + + (*env)->CallVoidMethod(env, codec->object, codec->jfields.set_input_surface_id, surface); + if (ff_jni_exception_check(env, 1, codec) < 0) + return AVERROR_EXTERNAL; + return 0; + } else { + (*env)->CallVoidMethod(env, codec->object, codec->jfields.configure_id, format->object, surface, NULL, flags); + } if (ff_jni_exception_check(env, 1, codec) < 0) { ret = AVERROR_EXTERNAL; goto fail; @@ -1743,6 +1768,22 @@ fail: return ret; } +static int mediacodec_jni_signalEndOfInputStream(FFAMediaCodec *ctx) +{ + int ret = 0; + JNIEnv *env = NULL; + FFAMediaCodecJni *codec = (FFAMediaCodecJni *)ctx; + + JNI_GET_ENV_OR_RETURN(env, codec, AVERROR_EXTERNAL); + + (*env)->CallVoidMethod(env, codec->object, codec->jfields.signal_end_of_input_stream_id); + if (ff_jni_exception_check(env, 1, codec) < 0) { + return AVERROR_EXTERNAL; + } + + return 0; +} + static const FFAMediaFormat media_format_jni = { .class = &amediaformat_class, @@ -1801,6 +1842,7 @@ static const FFAMediaCodec media_codec_jni = { .getConfigureFlagEncode = mediacodec_jni_getConfigureFlagEncode, .cleanOutputBuffers = mediacodec_jni_cleanOutputBuffers, + .signalEndOfInputStream = mediacodec_jni_signalEndOfInputStream, }; typedef struct FFAMediaFormatNdk { @@ -1866,6 +1908,10 @@ typedef struct FFAMediaCodecNdk { // Available since API level 28. media_status_t (*getName)(AMediaCodec*, char** out_name); void (*releaseName)(AMediaCodec*, char* name); + + // Available since API level 26. + media_status_t (*setInputSurface)(AMediaCodec*, ANativeWindow *); + media_status_t (*signalEndOfInputStream)(AMediaCodec *); } FFAMediaCodecNdk; static const FFAMediaFormat media_format_ndk; @@ -2098,6 +2144,9 @@ static inline FFAMediaCodec *ndk_codec_create(int method, const char *arg) { GET_SYMBOL(getName, 0) GET_SYMBOL(releaseName, 0) + GET_SYMBOL(setInputSurface, 0) + GET_SYMBOL(signalEndOfInputStream, 0) + #undef GET_SYMBOL switch (method) { @@ -2184,10 +2233,32 @@ static int mediacodec_ndk_configure(FFAMediaCodec* ctx, return AVERROR(EINVAL); } - status = codec->configure(codec->impl, format->impl, native_window, NULL, flags); - if (status != AMEDIA_OK) { - av_log(codec, AV_LOG_ERROR, "configure failed, %d\n", status); - return AVERROR_EXTERNAL; + if (flags & AMEDIACODEC_CONFIGURE_FLAG_ENCODE) { + if (native_window && !codec->setInputSurface) { + av_log(ctx, AV_LOG_ERROR, "System doesn't support setInputSurface\n"); + return AVERROR_EXTERNAL; + } + + status = codec->configure(codec->impl, format->impl, NULL, NULL, flags); + if (status != AMEDIA_OK) { + av_log(codec, AV_LOG_ERROR, "Encoder configure failed, %d\n", status); + return AVERROR_EXTERNAL; + } + + if (!native_window) + return 0; + + status = codec->setInputSurface(codec->impl, native_window); + if (status != AMEDIA_OK) { + av_log(codec, AV_LOG_ERROR, "Encoder set input surface failed, %d\n", status); + return AVERROR_EXTERNAL; + } + } else { + status = codec->configure(codec->impl, format->impl, native_window, NULL, flags); + if (status != AMEDIA_OK) { + av_log(codec, AV_LOG_ERROR, "Decoder configure failed, %d\n", status); + return AVERROR_EXTERNAL; + } } return 0; @@ -2330,6 +2401,26 @@ static int mediacodec_ndk_cleanOutputBuffers(FFAMediaCodec *ctx) return 0; } +static int mediacodec_ndk_signalEndOfInputStream(FFAMediaCodec *ctx) +{ + FFAMediaCodecNdk *codec = (FFAMediaCodecNdk *)ctx; + media_status_t status; + + if (!codec->signalEndOfInputStream) { + av_log(codec, AV_LOG_ERROR, "signalEndOfInputStream unavailable\n"); + return AVERROR_EXTERNAL; + } + + status = codec->signalEndOfInputStream(codec->impl); + if (status != AMEDIA_OK) { + av_log(codec, AV_LOG_ERROR, "signalEndOfInputStream failed, %d\n", status); + return AVERROR_EXTERNAL; + } + av_log(codec, AV_LOG_DEBUG, "signalEndOfInputStream success\n"); + + return 0; +} + static const FFAMediaFormat media_format_ndk = { .class = &amediaformat_ndk_class, @@ -2388,6 +2479,7 @@ static const FFAMediaCodec media_codec_ndk = { .getConfigureFlagEncode = mediacodec_ndk_getConfigureFlagEncode, .cleanOutputBuffers = mediacodec_ndk_cleanOutputBuffers, + .signalEndOfInputStream = mediacodec_ndk_signalEndOfInputStream, }; FFAMediaFormat *ff_AMediaFormat_new(int ndk) diff --git a/libavcodec/mediacodec_wrapper.h b/libavcodec/mediacodec_wrapper.h index 7cf3f4aecd..f15ad66d83 100644 --- a/libavcodec/mediacodec_wrapper.h +++ b/libavcodec/mediacodec_wrapper.h @@ -192,6 +192,9 @@ struct FFAMediaCodec { int (*getConfigureFlagEncode)(FFAMediaCodec *codec); int (*cleanOutputBuffers)(FFAMediaCodec *codec); + + // For encoder with FFANativeWindow as input. + int (*signalEndOfInputStream)(FFAMediaCodec *); }; static inline char *ff_AMediaCodec_getName(FFAMediaCodec *codec) @@ -311,6 +314,11 @@ static inline int ff_AMediaCodec_cleanOutputBuffers(FFAMediaCodec *codec) return codec->cleanOutputBuffers(codec); } +static inline int ff_AMediaCodec_signalEndOfInputStream(FFAMediaCodec *codec) +{ + return codec->signalEndOfInputStream(codec); +} + int ff_Build_SDK_INT(AVCodecContext *avctx); #endif /* AVCODEC_MEDIACODEC_WRAPPER_H */ diff --git a/libavcodec/mediacodecenc.c b/libavcodec/mediacodecenc.c new file mode 100644 index 0000000000..c81050ec80 --- /dev/null +++ b/libavcodec/mediacodecenc.c @@ -0,0 +1,495 @@ +/* + * Android MediaCodec encoders + * + * Copyright (c) 2022 Zhao Zhili + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "config_components.h" + +#include "libavutil/avassert.h" +#include "libavutil/hwcontext_mediacodec.h" +#include "libavutil/imgutils.h" +#include "libavutil/opt.h" + +#include "avcodec.h" +#include "codec_internal.h" +#include "encode.h" +#include "hwconfig.h" +#include "jni.h" +#include "mediacodec.h" +#include "mediacodec_wrapper.h" +#include "mediacodecdec_common.h" + +#define INPUT_DEQUEUE_TIMEOUT_US 8000 +#define OUTPUT_DEQUEUE_TIMEOUT_US 8000 + +typedef struct MediaCodecEncContext { + AVClass *avclass; + FFAMediaCodec *codec; + int use_ndk_codec; + FFANativeWindow *window; + + int fps; + int width; + int height; + + uint8_t *extradata; + int extradata_size; + + // Since MediaCodec doesn't output DTS, use a timestamp queue to save pts + // of AVFrame and generate DTS for AVPacket. + // + // This doesn't work when use Surface as input, in that case frames can be + // sent to encoder without our notice. One exception is frames come from + // our MediaCodec decoder wrapper, since we can control it's render by + // av_mediacodec_release_buffer. + int64_t timestamps[32]; + int ts_head; + int ts_tail; + + int eof_sent; + + AVFrame *frame; +} MediaCodecEncContext; + +enum { + COLOR_FormatYUV420Planar = 0x13, + COLOR_FormatYUV420SemiPlanar = 0x15, + COLOR_FormatSurface = 0x7F000789, +}; + +static const struct { + int color_format; + enum AVPixelFormat pix_fmt; +} color_formats[] = { + { COLOR_FormatYUV420Planar, AV_PIX_FMT_YUV420P }, + { COLOR_FormatYUV420SemiPlanar, AV_PIX_FMT_NV12 }, + { COLOR_FormatSurface, AV_PIX_FMT_MEDIACODEC }, +}; + +static const enum AVPixelFormat avc_pix_fmts[] = { + AV_PIX_FMT_MEDIACODEC, + AV_PIX_FMT_YUV420P, + AV_PIX_FMT_NV12, + AV_PIX_FMT_NONE +}; + +static void mediacodec_output_format(AVCodecContext *avctx) +{ + MediaCodecEncContext *s = avctx->priv_data; + char *name = ff_AMediaCodec_getName(s->codec); + FFAMediaFormat *out_format = ff_AMediaCodec_getOutputFormat(s->codec); + char *str = ff_AMediaFormat_toString(out_format); + + av_log(avctx, AV_LOG_DEBUG, "MediaCodec encoder %s output format %s\n", + name ? name : "unknown", str); + av_free(name); + av_free(str); + ff_AMediaFormat_delete(out_format); +} + +static av_cold int mediacodec_init(AVCodecContext *avctx) +{ + const char *codec_mime = NULL; + MediaCodecEncContext *s = avctx->priv_data; + FFAMediaFormat *format = NULL; + int ret; + int gop; + + if (s->use_ndk_codec < 0) + s->use_ndk_codec = !av_jni_get_java_vm(avctx); + + switch (avctx->codec_id) { + case AV_CODEC_ID_H264: + codec_mime = "video/avc"; + break; + case AV_CODEC_ID_HEVC: + codec_mime = "video/hevc"; + break; + default: + av_assert0(0); + } + + s->codec = ff_AMediaCodec_createEncoderByType(codec_mime, s->use_ndk_codec); + if (!s->codec) { + av_log(avctx, AV_LOG_ERROR, "Failed to create encoder for type %s\n", + codec_mime); + return AVERROR_EXTERNAL; + } + + format = ff_AMediaFormat_new(s->use_ndk_codec); + if (!format) { + av_log(avctx, AV_LOG_ERROR, "Failed to create media format\n"); + return AVERROR_EXTERNAL; + } + + ff_AMediaFormat_setString(format, "mime", codec_mime); + s->width = FFALIGN(avctx->width, 16); + s->height = avctx->height; + ff_AMediaFormat_setInt32(format, "width", s->width); + ff_AMediaFormat_setInt32(format, "height", s->height); + + if (avctx->pix_fmt == AV_PIX_FMT_MEDIACODEC) { + AVMediaCodecContext *user_ctx = avctx->hwaccel_context; + if (avctx->hw_device_ctx) { + AVHWDeviceContext *device_ctx = (AVHWDeviceContext*)(avctx->hw_device_ctx->data); + AVMediaCodecDeviceContext *dev_ctx; + + if (device_ctx->type != AV_HWDEVICE_TYPE_MEDIACODEC || !device_ctx->hwctx) { + ret = AVERROR(EINVAL); + goto bailout; + } + dev_ctx = device_ctx->hwctx; + s->window = ff_mediacodec_surface_ref(dev_ctx->surface, dev_ctx->native_window, avctx); + } + + if (!s->window && user_ctx && user_ctx->surface) + s->window = ff_mediacodec_surface_ref(user_ctx->surface, NULL, avctx); + + if (!s->window) { + ret = AVERROR(EINVAL); + av_log(avctx, AV_LOG_ERROR, "Missing hw_device_ctx or hwaccel_context for AV_PIX_FMT_MEDIACODEC\n"); + goto bailout; + } + } + + for (int i = 0; i < FF_ARRAY_ELEMS(color_formats); i++) { + if (avctx->pix_fmt == color_formats[i].pix_fmt) { + ff_AMediaFormat_setInt32(format, "color-format", + color_formats[i].color_format); + break; + } + } + + if (avctx->bit_rate) + ff_AMediaFormat_setInt32(format, "bitrate", avctx->bit_rate); + // frame-rate and i-frame-interval are required to configure codec + if (avctx->framerate.num >= avctx->framerate.den && avctx->framerate.den > 0) + s->fps = avctx->framerate.num / avctx->framerate.den; + else + s->fps = 30; + gop = round(avctx->gop_size / s->fps); + if (gop == 0) + gop = 2; + ff_AMediaFormat_setInt32(format, "frame-rate", s->fps); + ff_AMediaFormat_setInt32(format, "i-frame-interval", gop); + + + ret = ff_AMediaCodec_getConfigureFlagEncode(s->codec); + ret = ff_AMediaCodec_configure(s->codec, format, s->window, NULL, ret); + if (ret) { + av_log(avctx, AV_LOG_ERROR, "MediaCodec configure failed, %s\n", av_err2str(ret)); + goto bailout; + } + + ret = ff_AMediaCodec_start(s->codec); + if (ret) { + av_log(avctx, AV_LOG_ERROR, "MediaCodec failed to start, %s\n", av_err2str(ret)); + goto bailout; + } + + mediacodec_output_format(avctx); + + s->frame = av_frame_alloc(); + if (!s->frame) + ret = AVERROR(ENOMEM); + +bailout: + if (format) + ff_AMediaFormat_delete(format); + return ret; +} + +static int mediacodec_receive(AVCodecContext *avctx, + AVPacket *pkt, + int *got_packet) +{ + MediaCodecEncContext *s = avctx->priv_data; + FFAMediaCodec *codec = s->codec; + FFAMediaCodecBufferInfo out_info = {0}; + uint8_t *out_buf; + size_t out_size = 0; + int ret; + int extradata_size = 0; + int64_t timeout_us = s->eof_sent ? OUTPUT_DEQUEUE_TIMEOUT_US : 0; + ssize_t index = ff_AMediaCodec_dequeueOutputBuffer(codec, &out_info, timeout_us); + + if (ff_AMediaCodec_infoTryAgainLater(codec, index)) + return AVERROR(EAGAIN); + + if (ff_AMediaCodec_infoOutputFormatChanged(codec, index)) { + mediacodec_output_format(avctx); + return AVERROR(EAGAIN); + } + + if (ff_AMediaCodec_infoOutputBuffersChanged(codec, index)) { + ff_AMediaCodec_cleanOutputBuffers(codec); + return AVERROR(EAGAIN); + } + + if (index < 0) + return AVERROR_EXTERNAL; + + if (out_info.flags & ff_AMediaCodec_getBufferFlagEndOfStream(codec)) + return AVERROR_EOF; + + out_buf = ff_AMediaCodec_getOutputBuffer(codec, index, &out_size); + if (!out_buf) { + ret = AVERROR_EXTERNAL; + goto bailout; + } + + if (out_info.flags & ff_AMediaCodec_getBufferFlagCodecConfig(codec)) { + ret = av_reallocp(&s->extradata, out_info.size); + if (ret) + goto bailout; + + s->extradata_size = out_info.size; + memcpy(s->extradata, out_buf + out_info.offset, out_info.size); + ff_AMediaCodec_releaseOutputBuffer(codec, index, false); + // try immediately + return mediacodec_receive(avctx, pkt, got_packet); + } + + ret = ff_get_encode_buffer(avctx, pkt, out_info.size + s->extradata_size, 0); + if (ret < 0) + goto bailout; + + if (s->extradata_size) { + extradata_size = s->extradata_size; + s->extradata_size = 0; + memcpy(pkt->data, s->extradata, extradata_size); + } + memcpy(pkt->data + extradata_size, out_buf + out_info.offset, out_info.size); + pkt->pts = av_rescale_q(out_info.presentationTimeUs, AV_TIME_BASE_Q, avctx->time_base); + if (s->ts_tail != s->ts_head) { + pkt->dts = s->timestamps[s->ts_tail]; + s->ts_tail = (s->ts_tail + 1) % FF_ARRAY_ELEMS(s->timestamps); + } + + if (out_info.flags & ff_AMediaCodec_getBufferFlagKeyFrame(codec)) + pkt->flags |= AV_PKT_FLAG_KEY; + ret = 0; + *got_packet = 1; + + av_log(avctx, AV_LOG_TRACE, "receive packet pts %" PRId64 " dts %" PRId64 + " flags %d extradata %d\n", + pkt->pts, pkt->dts, pkt->flags, extradata_size); + +bailout: + ff_AMediaCodec_releaseOutputBuffer(codec, index, false); + return ret; +} + +static void copy_frame_to_buffer(AVCodecContext *avctx, const AVFrame *frame, uint8_t *dst, size_t size) +{ + MediaCodecEncContext *s = avctx->priv_data; + uint8_t *dst_data[4] = {}; + int dst_linesize[4] = {}; + const uint8_t *src_data[4] = { + frame->data[0], frame->data[1], frame->data[2], frame->data[3] + }; + + if (avctx->pix_fmt == AV_PIX_FMT_YUV420P) { + dst_data[0] = dst; + dst_data[1] = dst + s->width * s->height; + dst_data[2] = dst_data[1] + s->width * s->height / 4; + + dst_linesize[0] = s->width; + dst_linesize[1] = dst_linesize[2] = s->width / 2; + } else if (avctx->pix_fmt == AV_PIX_FMT_NV12) { + dst_data[0] = dst; + dst_data[1] = dst + s->width * s->height; + + dst_linesize[0] = s->width; + dst_linesize[1] = s->width; + } else { + av_assert0(0); + } + + av_image_copy(dst_data, dst_linesize, src_data, frame->linesize, + avctx->pix_fmt, avctx->width, avctx->height); +} + +static int mediacodec_send(AVCodecContext *avctx, + const AVFrame *frame) { + MediaCodecEncContext *s = avctx->priv_data; + FFAMediaCodec *codec = s->codec; + ssize_t index; + uint8_t *input_buf = NULL; + size_t input_size = 0; + int64_t pts = 0; + uint32_t flags = 0; + int64_t timeout_us; + + if (s->eof_sent) + return 0; + + if (s->window) { + if (!frame) { + s->eof_sent = 1; + return ff_AMediaCodec_signalEndOfInputStream(codec); + } + + + if (frame->data[3]) { + pts = av_rescale_q(frame->pts, avctx->time_base, AV_TIME_BASE_Q); + s->timestamps[s->ts_head] = frame->pts; + s->ts_head = (s->ts_head + 1) % FF_ARRAY_ELEMS(s->timestamps); + + av_mediacodec_release_buffer((AVMediaCodecBuffer *)frame->data[3], 1); + } + return 0; + } + + timeout_us = INPUT_DEQUEUE_TIMEOUT_US; + index = ff_AMediaCodec_dequeueInputBuffer(codec, timeout_us); + if (ff_AMediaCodec_infoTryAgainLater(codec, index)) + return AVERROR(EAGAIN); + + if (index < 0) { + av_log(avctx, AV_LOG_ERROR, "dequeue input buffer failed, %zd", index); + return AVERROR_EXTERNAL; + } + + if (frame) { + input_buf = ff_AMediaCodec_getInputBuffer(codec, index, &input_size); + copy_frame_to_buffer(avctx, frame, input_buf, input_size); + + pts = av_rescale_q(frame->pts, avctx->time_base, AV_TIME_BASE_Q); + + s->timestamps[s->ts_head] = frame->pts; + s->ts_head = (s->ts_head + 1) % FF_ARRAY_ELEMS(s->timestamps); + } else { + flags |= ff_AMediaCodec_getBufferFlagEndOfStream(codec); + s->eof_sent = 1; + } + + ff_AMediaCodec_queueInputBuffer(codec, index, 0, input_size, pts, flags); + return 0; +} + +static int mediacodec_encode(AVCodecContext *avctx, AVPacket *pkt) +{ + MediaCodecEncContext *s = avctx->priv_data; + int ret; + int got_packet = 0; + + // Return on three case: + // 1. Serious error + // 2. Got a packet success + // 3. No AVFrame is available yet (don't return if get_frame return EOF) + while (1) { + ret = mediacodec_receive(avctx, pkt, &got_packet); + if (!ret) + return 0; + else if (ret != AVERROR(EAGAIN)) + return ret; + + if (!s->frame->buf[0]) { + ret = ff_encode_get_frame(avctx, s->frame); + if (ret && ret != AVERROR_EOF) + return ret; + } + + ret = mediacodec_send(avctx, s->frame->buf[0] ? s->frame : NULL); + if (!ret) + av_frame_unref(s->frame); + else if (ret != AVERROR(EAGAIN)) + return ret; + } + + return 0; +} + +static av_cold int mediacodec_close(AVCodecContext *avctx) +{ + MediaCodecEncContext *s = avctx->priv_data; + if (s->codec) { + ff_AMediaCodec_stop(s->codec); + ff_AMediaCodec_delete(s->codec); + s->codec = NULL; + } + + if (s->window) { + ff_mediacodec_surface_unref(s->window, avctx); + s->window = NULL; + } + + av_frame_free(&s->frame); + + return 0; +} + +static const AVCodecHWConfigInternal *const mediacodec_hw_configs[] = { + &(const AVCodecHWConfigInternal) { + .public = { + .pix_fmt = AV_PIX_FMT_MEDIACODEC, + .methods = AV_CODEC_HW_CONFIG_METHOD_AD_HOC | + AV_CODEC_HW_CONFIG_METHOD_HW_DEVICE_CTX, + .device_type = AV_HWDEVICE_TYPE_MEDIACODEC, + }, + .hwaccel = NULL, + }, + NULL +}; + +#define OFFSET(x) offsetof(MediaCodecEncContext, x) +#define VE AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_ENCODING_PARAM +static const AVOption common_options[] = { + { "ndk_codec", "Use MediaCodec from NDK", + OFFSET(use_ndk_codec), AV_OPT_TYPE_BOOL, {.i64 = -1}, -1, 1, VE }, + { NULL }, +}; + +#define MEDIACODEC_ENCODER_CLASS(name) \ +static const AVClass name ## _mediacodec_class = { \ + .class_name = #name "_mediacodec", \ + .item_name = av_default_item_name, \ + .option = common_options, \ + .version = LIBAVUTIL_VERSION_INT, \ +}; \ + +#define DECLARE_MEDIACODEC_ENCODER(short_name, long_name, codec_id) \ +MEDIACODEC_ENCODER_CLASS(short_name) \ +const FFCodec ff_ ## short_name ## _mediacodec_encoder = { \ + .p.name = #short_name "_mediacodec", \ + CODEC_LONG_NAME(long_name " Android MediaCodec encoder"), \ + .p.type = AVMEDIA_TYPE_VIDEO, \ + .p.id = codec_id, \ + .p.capabilities = AV_CODEC_CAP_DR1 | AV_CODEC_CAP_DELAY \ + | AV_CODEC_CAP_HARDWARE, \ + .priv_data_size = sizeof(MediaCodecEncContext), \ + .p.pix_fmts = avc_pix_fmts, \ + .init = mediacodec_init, \ + FF_CODEC_RECEIVE_PACKET_CB(mediacodec_encode), \ + .close = mediacodec_close, \ + .p.priv_class = &short_name ## _mediacodec_class, \ + .caps_internal = FF_CODEC_CAP_INIT_CLEANUP, \ + .p.wrapper_name = "mediacodec", \ + .hw_configs = mediacodec_hw_configs, \ +}; \ + +#if CONFIG_H264_MEDIACODEC_ENCODER +DECLARE_MEDIACODEC_ENCODER(h264, "H.264", AV_CODEC_ID_H264) +#endif + +#if CONFIG_HEVC_MEDIACODEC_ENCODER +DECLARE_MEDIACODEC_ENCODER(hevc, "H.265", AV_CODEC_ID_HEVC) +#endif diff --git a/libavcodec/version.h b/libavcodec/version.h index 43d0d9a9fc..86ac0f3871 100644 --- a/libavcodec/version.h +++ b/libavcodec/version.h @@ -29,8 +29,8 @@ #include "version_major.h" -#define LIBAVCODEC_VERSION_MINOR 51 -#define LIBAVCODEC_VERSION_MICRO 101 +#define LIBAVCODEC_VERSION_MINOR 52 +#define LIBAVCODEC_VERSION_MICRO 100 #define LIBAVCODEC_VERSION_INT AV_VERSION_INT(LIBAVCODEC_VERSION_MAJOR, \ LIBAVCODEC_VERSION_MINOR, \ From patchwork Mon Oct 24 03:16:57 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhao Zhili X-Patchwork-Id: 38943 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:4a86:b0:9d:28a3:170e with SMTP id fn6csp1997915pzb; Sun, 23 Oct 2022 20:18:30 -0700 (PDT) X-Google-Smtp-Source: AMsMyM4pq1DY5DG+JxrCVI6TrEnliNIcaJx0723+kkV8cE580CL1mPu1TCsYu0ieUdHvoB1Hkc2L X-Received: by 2002:a17:906:9b87:b0:733:1795:2855 with SMTP id dd7-20020a1709069b8700b0073317952855mr25757069ejc.156.1666581510361; Sun, 23 Oct 2022 20:18:30 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666581510; cv=none; d=google.com; s=arc-20160816; b=mB71Ag4QPGSiSUwDlasYy7swQTbfADzAudg6At/sZJIUzpR1HFk7dq2X2bl4zLZLOe HeHl+43BxeMvq9KlsyGscUAfqPsxD3/f8aWN55sJw606sDbcAXUrmvP9hZUJUUywnriK wEULMtonQHX4A2io/+VYJqDHt2Ur46ySje+KZES0dhagcXAbBb/uD54hvjmZoYOHbPO4 oyKr6Rlr4pPAQd9jFdNYzlCeMfSKQyBcmtXwGB4TKQCNWSeBxxRqqQWIcZReoPV4mgk+ FDxs0gjtNut0heIu44AY6+oJREYtbIzc9IRApZES1PP2bymsTay/wPXxpaIv5Eemj4lh bHsg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to:date :to:from:message-id:dkim-signature:delivered-to; bh=jKXNZqlvVUqqHS5QaCk3t3jxteeySVCZYQzf+vQ7zSU=; b=bK0MV0GijUTGwpx3hJku/HPiQvU+bsxYypCPUcHk+8A4bBKNqNWOyw0cdcwYIF3veq KFWkhwOKx0WkKkhylu0O2dPeVoAsZyeDTBTK1THYIF5gxSAlC8dMe0Clv4ohnOh29BJI wak1ztxI9kvfWjWhh6I/TaKgS8aPnVCwWC4XEPjbNUkPHDpbvgfFOOW9qyJvrxljc0IW eFmItozW5M2qyFF3yokJ5wchzdN07RNu2+S+NPgpyMWeoI3o7I3iGRI1DD5+xsY2opLe jcZGgqElnDr2b2rpw68NDJkyOLLTgNhMUerPgDnQsl5IDcIQNJfwzfbrmIT4WoiyVnUB qpsw== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@foxmail.com header.s=s201512 header.b=Nf2+xS5f; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=foxmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id rn25-20020a170906d93900b0076f061ffab4si21324850ejb.51.2022.10.23.20.18.30; Sun, 23 Oct 2022 20:18:30 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@foxmail.com header.s=s201512 header.b=Nf2+xS5f; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=foxmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id E9DBF68BD72; Mon, 24 Oct 2022 06:17:34 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from out203-205-221-210.mail.qq.com (out203-205-221-210.mail.qq.com [203.205.221.210]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 62E6A68BD3D for ; Mon, 24 Oct 2022 06:17:27 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=foxmail.com; s=s201512; t=1666581445; bh=aWQzvDj42aZ9vae0EUxLYZ2w2w2+N1oUXwzJ0MHmBM8=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=Nf2+xS5fXx6NO71OhueQw7DbBXRMmIgqOw8Mnfb2u/KhD9hMNenS7sXS4pLBA6FSG pXT9+QVMa/rts7esApT2blYsVKl54mfAqUBbh9TxDOJjeWwjZ4a9nZ880wyzU1H5Ru GtP0cLzYpdMnaI84fLxsEe2i/fGdgrldopBmVo74= Received: from localhost.localdomain ([59.37.125.102]) by newxmesmtplogicsvrsza31.qq.com (NewEsmtp) with SMTP id 44A14270; Mon, 24 Oct 2022 11:17:10 +0800 X-QQ-mid: xmsmtpt1666581443tj5b0980x Message-ID: X-QQ-XMAILINFO: NsLYjltVP0elw07cjmLNbburSJs6NkaJ+G0DiFnWbluwkuCgWNxcNI5UvFEbiL 8/wFLjb+hkfQ3PBrTEB8rJqojveDwynkKI1qmV9ropGswTHf1Y3MQVL5CU5HeuXhKAM4krPhh4m0 cafwiaAvhTWt+v41BxXld/DeCWSoX5WkXPMBhihuYdk12rF2cMzXnIs1CiuPs5OITSZGvBP0FZQR ft13GfD9Phd6W7ZVfNpk22lhH/6wA9OPWGqxZy6u0DghGagrI8hrj4Ldpv5AZH12A2+TMICOU0ai 3eP8BMWgWr5TeCL/SzXhTUyfVTIYyvDg6qHQ+Vaz3xmUYdLhnE6bzYxqzm9//CPYsb+rnfgHZ4n4 Tx7sWYKaDEsknEpHjvYoJboRDxpplTZag9lauzqK9Vp0I7G1SZdberAvaLj0s4us+/5jXPka9/sV bqY5ThnhpC1hJHUaL7o7Y5NwiLX8aeKKfOJ+M6qiz8pRTOyCG+Lc0Q11+8qIr7HmRwDyNbfzrlwy G6edkj5ACkG7dA/4d+ojuKBEO23e0xdTycpOW3OzZoIBEz3VymRMP5Qfc4n12Hf9zfhqWMSSMyZi OsN7sSXtpb+cY3UWOSLzlhYCuY8T7ZJsMI9F2fYwB491uXqLpitNoz2FUXbWL4FMVISA2UvJBhyN NrG24ntP5lGT7ckLiPMnmXMiUwL8PoTLEVeAus+Z3OYWfXHu4DagSaDKmTGzGeMMelCEqksrj0bd otrKgIESwjbNUHniqetRgfah6CdIvEz+71hwSJ2J4i+Gmw4Co4sJ3NzHR2f0WB+BFl925PKF67v/ i4M8qI1atkgRMLW9ynWjCG32IvFotXz6jcsoujBnYYtafdmfPC7UmlmTeHEHQ0TsZJbCEMHT/9LO 8YXPFXOCX8dvL1VA6NKgrkOhCN+RkfJKumwB3ln3QkK1xtxDfZhoOFeIIFzImrWAW8wROwAD/GWH 3kqQoTjwJcwe+BCydOW9AB0JY7BDDpvswimZi4LRkVHHWVhvvQxvZzNcb7CNMD From: Zhao Zhili To: ffmpeg-devel@ffmpeg.org Date: Mon, 24 Oct 2022 11:16:57 +0800 X-OQ-MSGID: <20221024031658.1026118-12-quinkblack@foxmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221024031658.1026118-1-quinkblack@foxmail.com> References: <20221024031658.1026118-1-quinkblack@foxmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 11/12] avutil/hwcontext: verify hw_frames_ctx in transfer_data_alloc X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: matthieu.bouron@stupeflix.com, Zhao Zhili , aman@tmm1.net Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: PWINadfMmDUE From: Zhao Zhili --- libavutil/hwcontext.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/libavutil/hwcontext.c b/libavutil/hwcontext.c index ab9ad3703e..3396598269 100644 --- a/libavutil/hwcontext.c +++ b/libavutil/hwcontext.c @@ -397,10 +397,14 @@ int av_hwframe_transfer_get_formats(AVBufferRef *hwframe_ref, static int transfer_data_alloc(AVFrame *dst, const AVFrame *src, int flags) { - AVHWFramesContext *ctx = (AVHWFramesContext*)src->hw_frames_ctx->data; + AVHWFramesContext *ctx; AVFrame *frame_tmp; int ret = 0; + if (!src->hw_frames_ctx) + return AVERROR(EINVAL); + ctx = (AVHWFramesContext*)src->hw_frames_ctx->data; + frame_tmp = av_frame_alloc(); if (!frame_tmp) return AVERROR(ENOMEM); From patchwork Mon Oct 24 03:16:58 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhao Zhili X-Patchwork-Id: 38944 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:4a86:b0:9d:28a3:170e with SMTP id fn6csp1997964pzb; Sun, 23 Oct 2022 20:18:38 -0700 (PDT) X-Google-Smtp-Source: AMsMyM5fx3zr4EJJXVY4cNaWMZA7WBqcJhw1JeWX4QwnJHiXJdoWp00pLaGD2u2NEQa9jRZG9II9 X-Received: by 2002:a05:6402:51c8:b0:461:ea0c:e12c with SMTP id r8-20020a05640251c800b00461ea0ce12cmr50259edd.47.1666581518127; Sun, 23 Oct 2022 20:18:38 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666581518; cv=none; d=google.com; s=arc-20160816; b=aZ0x0bQnVf/QRpBGn6+1BHKPSm7xRWCBAHxB4w9/SXwuavo9Qsl8ayfWu9DG+ejcmE WZNDaMlurtpebF/GCLC5xSkBwabXa//mNxVTXpoHXt8qD8rPHvoO5DSB+x6IadGTWSyS NptxHdlrq6h43FPfu/GFHRJFnHSDoAYDoUaqlLyXmGksVmeuAi9ctlI5Z7Ug6z6mZZjy 2AAXB093WEir17E1Br8WyeU0YEUiqCTwt7gbdQgR6w2bKCslPPXp3IJTtCgcF8d0OvW0 TBocgBOO73ufy8c+a/jVGHT8bt402tmArUbiuSnIh1B7TVhVAnT230J0YwH9x7vGgP7M 9z9Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to:date :to:from:message-id:dkim-signature:delivered-to; bh=cNeUmCgkI0yyphEAr6y11EJNYWGl1EUns8tgUtwNSzc=; b=wtHsyN0QIO4m+XLhJAw0PuEWfOslGAq0o7pBq3JEnggqbSvMR3NRGVyN+cuxKx/8qc Q54zoSP6YlaPrSJWbq/6dM9DSLX3IVLE3ZMv2TIaR9My4LKyuAt6LhzAvRAbIFzgWQR1 E4IssjxGnuPs+seG+1oesPWFWXE6KkKqeN0u/sIY3MbPUSgSc92Y6WJscARqDyq1eJCr U8t2SjU9iT1+zNPLslyCEqgKMD5uMvo1OOQ6mFfmqvGSLjxBKptKin9+ibZD6gbSmJeI 1LO/WrKlVexQdPR78WA7uW/ferPCxFUUor4jVHpDnCQOzGiTMojJt7KVuoXgg+ttUwdX EJ8Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@foxmail.com header.s=s201512 header.b=axaTv4dz; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=foxmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id sh37-20020a1709076ea500b0078d9d69ae0dsi28123612ejc.877.2022.10.23.20.18.37; Sun, 23 Oct 2022 20:18:38 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@foxmail.com header.s=s201512 header.b=axaTv4dz; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=foxmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id F410968BCD2; Mon, 24 Oct 2022 06:17:35 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from out162-62-57-210.mail.qq.com (out162-62-57-210.mail.qq.com [162.62.57.210]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 0341168BD54 for ; Mon, 24 Oct 2022 06:17:27 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=foxmail.com; s=s201512; t=1666581446; bh=GlKtidcZQNIVknl+knqa+XcLBwAxJR1nX9pKrZDru2A=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=axaTv4dzWaPr05LletiP43b8CNLpCm2b6mD6OfmyAF5aFQArK9xagHHlv+BqHr6Hp ekEeo0sboG0wpcuvnwB87DygjR9D2qmWixzl6UHhLJBNQabWC8cJ3YIL6v3F9JyALx VS/QgmCS9PX7kVYT1Ay8NZIEWXHpWrNjQ+FS/Jnk= Received: from localhost.localdomain ([59.37.125.102]) by newxmesmtplogicsvrsza31.qq.com (NewEsmtp) with SMTP id 44A14270; Mon, 24 Oct 2022 11:17:10 +0800 X-QQ-mid: xmsmtpt1666581444tp1cx24vr Message-ID: X-QQ-XMAILINFO: OGZxhFXqN7PJBshkBxfjIY8hHzju9WAbOH787IGpUz0slWvJk4rRMy4WGKX7Vc KdkovECyBER87w/gWDu+JBlr0tmVdaCwoISlIMDXpht1A+cISngAi4cIPzWOKqtiezOsyqcnWzM3 wArU1vfotG2X02w6BrM38kH7AbWAD6PNO6dovZjJLJ2Kuljj5uUWvwlMQ80HiJEo5yVJRF4WBSoN KDv1SaL8OaLhBduT0UzaJhwedu6LzItuBH/1oTCJpAS0xAyh2Q+Rd8Z3n8RD7enzM+029GGliiAG eyJ/HR/9e0e4dZinPFsbXTNEA79jL+tahJL5KAn/hf7q8AB8MartcCwH6Bzxj9IKZV2hhw5x7Lsx 5ifU6mskWuWG3WWj/uUdrC6JtAiGfYK5OGsh2of8RRch6Z0F6u+akUPcdHWmJqvGSx3zjquu67Vv buxD5dbXNhaBTI72Dl0LofuQrmwmzZeJc72leM+plj0PEIm6s/4M4fgCATBeV5rUal612Of1e96q ze4phDnHkRdoCW28hBIAslpOOqcTuOPbXuVHkJi4fJfyYxxvxBmi3qvvsNf3S9Fu8blYF9GigB4x UBGutWzQvBKWWEHk0iZNMJwbpbsP8xhMjPw6NuW4dm3GU3c630ofnTjI2PQG5TmUmoq9z27GA8fH 8W2KXjwTj/fxbd/uM9CcftcjM6rWkPq7PQbvktv1NAkqLx9LgBmXKytYBfYxYxk/gICr6tQ99/gn wIkzeDx8vjX9+LmOwCh0Ksz0z4pLVJfYwveJz/YM66fAv4tb1B6I5CGEnvdnytJ6fyFVP6jFWmWu /5cB/c2QAaj+xyJ0E+UQ8dXzCBx+1Eljs44G0Og0RVH5mGJRrrHk+q6x3lV6YcnUdnk1ISGiOgR1 4Wni2SgESXGw2xnHakZnG1KiOWRDqEVJxyb5DuxzW6d7h5erhIoyow0wisqBmq18IXTqQsrbbhKK 9OFXiLOREeiUblIzz9PrYKVSzPCqwUFySfayS4zSI= From: Zhao Zhili To: ffmpeg-devel@ffmpeg.org Date: Mon, 24 Oct 2022 11:16:58 +0800 X-OQ-MSGID: <20221024031658.1026118-13-quinkblack@foxmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221024031658.1026118-1-quinkblack@foxmail.com> References: <20221024031658.1026118-1-quinkblack@foxmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 12/12] fftools/ffmpeg_opt: set default hwaccel_output_format for mediacodec X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: matthieu.bouron@stupeflix.com, Zhao Zhili , aman@tmm1.net Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: Mdjh5DFj2xTN From: Zhao Zhili --- fftools/ffmpeg_opt.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/fftools/ffmpeg_opt.c b/fftools/ffmpeg_opt.c index 9245e02813..9c15858c05 100644 --- a/fftools/ffmpeg_opt.c +++ b/fftools/ffmpeg_opt.c @@ -785,6 +785,10 @@ static void add_input_streams(OptionsContext *o, AVFormatContext *ic) "with old commandlines. This behaviour is DEPRECATED and will be removed " "in the future. Please explicitly set \"-hwaccel_output_format qsv\".\n"); ist->hwaccel_output_format = AV_PIX_FMT_QSV; + } else if (!hwaccel_output_format && hwaccel && !strcmp(hwaccel, "mediacodec")) { + // There is no real AVHWFrameContext implementation. Set + // hwaccel_output_format to avoid av_hwframe_transfer_data error. + ist->hwaccel_output_format = AV_PIX_FMT_MEDIACODEC; } else if (hwaccel_output_format) { ist->hwaccel_output_format = av_get_pix_fmt(hwaccel_output_format); if (ist->hwaccel_output_format == AV_PIX_FMT_NONE) {