From patchwork Sun Nov 20 06:49:36 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhao Zhili X-Patchwork-Id: 39346 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a21:999a:b0:a4:2148:650a with SMTP id ve26csp1406452pzb; Sat, 19 Nov 2022 22:50:46 -0800 (PST) X-Google-Smtp-Source: AA0mqf48OTXXmXz51h2QljqKYvamTt4WEBwXUU/sfl0WEolhrEf9s1M5xSCYvV9Y/j7AWvaSd498 X-Received: by 2002:aa7:d556:0:b0:45c:6467:94e2 with SMTP id u22-20020aa7d556000000b0045c646794e2mr11577918edr.295.1668927046187; Sat, 19 Nov 2022 22:50:46 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1668927046; cv=none; d=google.com; s=arc-20160816; b=g9blqChY4PO81l1E2hbk8RvQ1xY/CwHONurJ6/hvElTc0NRkvtpv9OdLmnY5UGYCfc bhFhPqNQURw49PxSESvZFdvTrjOsM8qaluA6m9SirZm9HbVh+V48rcXUmU5CTAESHy4r VrZi19dje5QcSprrHnLR7gFQGweAZU2c3KBkB6TAWlmBOgW//FXtgcVqrqYia/IZXsKr SHGrvWTrhlABDHuKDcIU7R+U/Z1qWvCOs+u26fzAciAmdC39CqkNzrCE6vK06MHqJUlr 421Y1qtUN9hz616KO8+zzOvxIcKpyUiyS+YOM/vnsq3sPlLStFfZUN5zgtG9/I51MnI/ vdEw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to:date :to:from:message-id:dkim-signature:delivered-to; bh=RlzTN1GBLYKrXQp6xXfJ4QmCVmQuysnHPIfvX+grjZY=; b=os7hRnNa46P5aMsuJvwguk2vXDK9+Exb7eK+1m/3TzaydZsaXf9hKoWY1qKzGRmu6i XMUhRTWLogZyQHn8eTWCNGmq/Xa50zmYz2xowSumvoyqSYRipfl5LtMAEopDiPYrn3HK 4ArGdmdXqmA/vOjux5xkqZfZGQKjatcZ1kt7BHlq85o8NH0K7ZES12u5BMTJ/jUYb0Dk 2Bo6i4EHwbRum6NWaRyCwRNvkw7oP9QYzR0/ku1+4vHMMajj+9g4wKVghMsei2w3aLRq 0DpTZEotLhhNVCiZk7BqEea1ThcW95M1mqZlDwrRHitpU71US403ZmnVhIKtl0XkOfor rEXw== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@foxmail.com header.s=s201512 header.b=U9CRNTfT; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=foxmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id du16-20020a17090772d000b0078200e81b92si7605744ejc.758.2022.11.19.22.50.45; Sat, 19 Nov 2022 22:50:46 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@foxmail.com header.s=s201512 header.b=U9CRNTfT; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=foxmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id DB5DC68BB61; Sun, 20 Nov 2022 08:50:42 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from out162-62-58-216.mail.qq.com (out162-62-58-216.mail.qq.com [162.62.58.216]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 4D1D868BA72 for ; Sun, 20 Nov 2022 08:50:36 +0200 (EET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=foxmail.com; s=s201512; t=1668927033; bh=gMPJd4nShKvekB/U9PZ8WWZUq1hKF/yD4qApol1L39s=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=U9CRNTfT3KwK8TCTfjqtwk+V2pdXNFwkoMiEhU67cB/d7bTdUsc5YlRLCti2yi7Z8 qbcKwTLnz2Mejm05cbWtO3FPTgu/bN9M7YSH0Te4GiqxGJo6rbnAu+Go4dxP/xCR5P a9W4ncF94C8cNboYIqNtNO6W0yzTAhqdAtSLUiwI= Received: from localhost.localdomain ([14.153.77.200]) by newxmesmtplogicsvrszc2-1.qq.com (NewEsmtp) with SMTP id C8F0626E; Sun, 20 Nov 2022 14:50:15 +0800 X-QQ-mid: xmsmtpt1668927016thzo3ddli Message-ID: X-QQ-XMAILINFO: N4CzT1PKe+qFMaIHKUpjcVNbdC9VIGUctlFQx483VMxmftBd2640L/8/BqU7FD 7TZCt+Zn0V4p8QzvqTpq7mr5QTu790uZalMb4PLeIj3oq/w74iIQARBfeIDdXfH9Lt97e4p5bffm Vqzt+cnx+85ZKKywwI9ebNkcRpHXg+jULp6uAkGnerEPCYRUT4UdfAskEOLGk7rin1WS4fviFqXj j9Hm1ubFgcxhB/gr/AdmSticUxWAiQIIJYUpywLZMZwsF52Usl9lArcbNsf2mI6cK31Y1/6ec4hj rXQLWFAu93j8I+fkKyQg9c4YjmotThU4e9v0Wj+6o8uOohPD7qh6Y1Zw/8zG2kujIcWwnymgCVSI Chj2g58vKxUcMOzDe/TN+QLOZbPA1o+Se1pbyMrLWowaCeh++cY4/+HXE8BkyKV8Ajrg+BkSW9nc OJMyetkb8HiRl0Wsja4KEyW57jKJTbajCE+Gy+G79g1F0nMmrEGl7P8sH0iCGBWyrOIdRjdQDJ+l RYsUUEVmbGKHGz28dP2q0sDgZ3HR5cImmUFT7NjT/CiOpzeQ8rfuzhplS1d1PcfSD10Fsm67bp8O xSVtx1tMHMysSF9OXZCizrsktPcoTJRpv8QFzOp5bt4L5HrOWXGPnWUR8f66CgY/zwQOkxA/DzHd P/+kHpTC3DSJ+cBdK0rG4NevFT0wThZsD5+3x5pleyJ6yXUb8zGVoAbt1gXcEtWh2fJeSXcWpw1d kDoYPcDliVxY5epcmmqZxywtDjUpu/JRLi0pm1gRmJ3RcIc7SfEYu3syzkd5U1uX9/HaQImf84j4 iN5wnkcKrRexPQVfKwIJ47skFMhNNFiTU9A2AgMp5hn3kt8fz1QCsseDnGH9GFE/CEpNHw4hrvaT yvd4RJKa86KzK4b7efT7eYFoN3wm44h9r6uYfOuOjeiKX2qjL0V3Bw0+TeW2gx/P+a7mQ7NdRkGQ ZViKHlKe/INwnuKMiMJ5USIcZLOM1kHz3TIE/pnQs= From: Zhao Zhili To: ffmpeg-devel@ffmpeg.org Date: Sun, 20 Nov 2022 14:49:36 +0800 X-OQ-MSGID: <20221120064947.94540-2-quinkblack@foxmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221120064947.94540-1-quinkblack@foxmail.com> References: <20221120064947.94540-1-quinkblack@foxmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 01/12] avcodec/mediacodec: fix incorrect crop info X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: matthieu.bouron@stupeflix.com, Aman Karmani , Zhao Zhili , aman@tmm1.net Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: LNN2cCGi3MvJ From: Zhao Zhili The crop info is optional, but used unconditionally. Co-authored-by: Aman Karmani Signed-off-by: Zhao Zhili --- libavcodec/mediacodecdec_common.c | 16 ++++++++++++++-- 1 file changed, 14 insertions(+), 2 deletions(-) diff --git a/libavcodec/mediacodecdec_common.c b/libavcodec/mediacodecdec_common.c index 2a605e7f5b..f430cfed31 100644 --- a/libavcodec/mediacodecdec_common.c +++ b/libavcodec/mediacodecdec_common.c @@ -487,8 +487,20 @@ static int mediacodec_dec_parse_format(AVCodecContext *avctx, MediaCodecDecConte AMEDIAFORMAT_GET_INT32(s->crop_left, "crop-left", 0); AMEDIAFORMAT_GET_INT32(s->crop_right, "crop-right", 0); - width = s->crop_right + 1 - s->crop_left; - height = s->crop_bottom + 1 - s->crop_top; + if (s->crop_right && s->crop_bottom) { + width = s->crop_right + 1 - s->crop_left; + height = s->crop_bottom + 1 - s->crop_top; + } else { + /* TODO: NDK MediaFormat should try getRect() first. + * Try crop-width/crop-height, it works on NVIDIA Shield. + */ + AMEDIAFORMAT_GET_INT32(width, "crop-width", 0); + AMEDIAFORMAT_GET_INT32(height, "crop-height", 0); + } + if (!width || !height) { + width = s->width; + height = s->height; + } AMEDIAFORMAT_GET_INT32(s->display_width, "display-width", 0); AMEDIAFORMAT_GET_INT32(s->display_height, "display-height", 0); From patchwork Sun Nov 20 06:49:37 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhao Zhili X-Patchwork-Id: 39347 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a21:999a:b0:a4:2148:650a with SMTP id ve26csp1406502pzb; Sat, 19 Nov 2022 22:50:56 -0800 (PST) X-Google-Smtp-Source: AA0mqf5hr2GmPr0mF2pmzlvQ+XguYr2kVLiyZwEC5b968K+WrS5aYJF53TXMGaC4b4HVzh9/MuP+ X-Received: by 2002:a17:907:bd5:b0:7ad:9ada:f32e with SMTP id ez21-20020a1709070bd500b007ad9adaf32emr11252977ejc.591.1668927055869; Sat, 19 Nov 2022 22:50:55 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1668927055; cv=none; d=google.com; s=arc-20160816; b=jtAhsKFScbYNghPZIDk5/LkU+nrFtNBL+oSyPjOi44HGG45pAA/09ifK9B4dB2ndxE SwZH3Yk+WQhoHnpghwb3txm+ts9i6qa8O7pXROnXvff7x2CrZSvj4CUve/3eTg2W5IGU mn5fTPGxK+yv/T1KMWs1ULiyvrzq1dIfAuLZyve3YWcB3dJ64bYfPKRKSdiFlngJxLwy HCAtUQpJHhmEDBBQlsszhdZ4ZnichLTOM5rmpb7Tekk6YS8vHLop5nUvyg+PS7flkz8U stYu+MtKGPjkJoINRll1WcNlwKtCHdWPVTBZaL72TGM2cIKuP9eIYHtWzDtVkUACft3R GGZw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to:date :to:from:message-id:dkim-signature:delivered-to; bh=D9pA4qF84TdVIbHxJaM9jTyL4eBlqxwpDkVuwY2Vibg=; b=zM/ivENmJnH4eUNsAREIFXcXmVnhOFf0QRPZ/wy7vjExvQgBTKOq954zpy/i3dPu+R 7942Uf2t3uEa4Qny5x7eF8zZZ6wsT1nS6U6NdjRCNJeM87uZsRaM6ki1hDrKRXUEN9cY CYhxDeyDRG2iNMPtR4UNc4FKsuW6POMlF2rDmhwyOBkdjXFi5M8rh2ZkmegO2XshAaWW 7ZMKCvf3ILjPJ8Vafl9HB+AtDOyji63ZJF5K1m80Ir3CjQ/K/50M+bg+1RSPI9GDD9rF h61xG9mQuLeyMzQGvZNU1AqLxkfS8Sbf1/Td3nzpDnuuT+xqGqfoom65Rutq4vjzUoes OxGQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@foxmail.com header.s=s201512 header.b=Vo0XuFNm; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=foxmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id sa20-20020a1709076d1400b007aa6ce58d9asi7546677ejc.712.2022.11.19.22.50.55; Sat, 19 Nov 2022 22:50:55 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@foxmail.com header.s=s201512 header.b=Vo0XuFNm; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=foxmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id F188C68BB81; Sun, 20 Nov 2022 08:50:43 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from out162-62-58-211.mail.qq.com (out162-62-58-211.mail.qq.com [162.62.58.211]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 4CB5D68B7DE for ; Sun, 20 Nov 2022 08:50:37 +0200 (EET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=foxmail.com; s=s201512; t=1668927034; bh=U+ZFDDSMx9NZbmIQBcSMvX6hrw2B6jNvxsIWP4P86hs=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=Vo0XuFNmLXBIb0VYdsKcwpYec3niJfGFhUvN047jUpdCCx/hhy1Kucy9z6ZpG62mr A7+XRFldTT44OWEao6j0ysGO1YFwyoxg13fTZIApmN7WfNiwGwwQvUd/3Cz6HFY6WD lVMccNxzWbg98TCcMp6m7G8iWMwRXeH5MWeTSZpE= Received: from localhost.localdomain ([14.153.77.200]) by newxmesmtplogicsvrszc2-1.qq.com (NewEsmtp) with SMTP id C8F0626E; Sun, 20 Nov 2022 14:50:15 +0800 X-QQ-mid: xmsmtpt1668927033tkw3b0gfc Message-ID: X-QQ-XMAILINFO: Nq1uWKlIb9DM5TBK3iwpz0I3nqzOUPcdlimJI6yjA9BXyEA67SVrsO+gaVHRMe xgo7JwXxR9MnMuNPV85ant9Zyp1YvqubFySOV4kqmE08HGXO6cqhhjJexR/fNFlGq6gXaXGR/dPN V98XCUhRIvjg3dcN9faG/EcltzEb8qoK71bnAeiptBOkE91gT7ngNThobnZIoc3CHS2tY7eegKRm cjtHIKmrc0KpUB1Aqjq+zUOe8c4bDbA2TM2/1slL5chwMfVZhRCl0/8LMMlgxvLgTK0+ZwwEMSdo JjDepqmnqm4do0kUng9d2M61I/2LA2f9bpY5lFbNKPOfD74U1KyTehR7WGPhMRNdLF8B13kknVpH QYdO3Rfcfr1TncX47Ewm2Zw+uk8qA6+lA8Hh6PXv+N9M/eXKKw5cZcf9MJ4XLQWsIHpRm9WOS28i +wQJc40uHQ4K+UoSY5OAebI8oA1RLpLLWo9yA1AI0I1COope5ESemOmF85mDQ9RD07JcaZhAhLSC dxAvsVjvFykMg3DVGh+pTAnhh1qILotoJfgQGh595kGlwcfncGyMAFDUjJ9nP5d/xHPIQl0FzXh4 wvvICXk+5HDWpkhlltlDot6sAyJ6BQusbiqEUyoaTcay01vZ8Ds1KWSp0EV/Q06yxBzm8loYT1/6 RTjk0BTKW3SuspOjJH/qZiYBKZ60L9m3HHABW6y6hw/KnCuPI5BRFWvXYCoKfvE4vMrrgoEalNQh taQ43z+ZoGMjHdOwSzJlYoDywJTQ1cvHDI9hxRSgXzzd8WFCFWyQpgZm3AoewJo4Jd1UFzUEhXDI S36pwudgQbQzfQEgWduSx2/RswrWz3Npz5jtEpsM9NMFVsfRrzfBnVe7T+j0qHGGntLHg25b0Mec s1RDk85mL0GT+rRXMSQibNZJU8LE1N1YvEyAaEOJyv3na+Ig09Y9smhGUB5LzTZ5A4gcA90jvy11 MVD8QRgRRhmCB0qUhlXqFHUqKbXMHYDIm4Zvnb6sA= From: Zhao Zhili To: ffmpeg-devel@ffmpeg.org Date: Sun, 20 Nov 2022 14:49:37 +0800 X-OQ-MSGID: <20221120064947.94540-3-quinkblack@foxmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221120064947.94540-1-quinkblack@foxmail.com> References: <20221120064947.94540-1-quinkblack@foxmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 02/12] avcodec/mediacodecdec: don't break out if both input and output port return try again X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: matthieu.bouron@stupeflix.com, Zhao Zhili , aman@tmm1.net Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: s9DqUUn6FUUs From: Zhao Zhili At the beginning of decoding, if we feed mediacodec too fast, the input port will return try again. It takes some time for mediacodec to consume bitstream and output frame. So the output port also return try again. It possible that mediacodec_receive_frame doesn't consume any AVPacket and no AVFrame is output. Then both avcodec_send_packet() and avcodec_receive_frame() return EAGAIN, which shouldn't happen. This bug can be produced with decoding benchmark on Pixel 3. Signed-off-by: Zhao Zhili --- libavcodec/mediacodecdec.c | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/libavcodec/mediacodecdec.c b/libavcodec/mediacodecdec.c index 322b448d27..2c66f38541 100644 --- a/libavcodec/mediacodecdec.c +++ b/libavcodec/mediacodecdec.c @@ -444,7 +444,16 @@ static int mediacodec_receive_frame(AVCodecContext *avctx, AVFrame *frame) index = ff_AMediaCodec_dequeueInputBuffer(s->ctx->codec, 0); if (index < 0) { /* no space, block for an output frame to appear */ - return ff_mediacodec_dec_receive(avctx, s->ctx, frame, true); + ret = ff_mediacodec_dec_receive(avctx, s->ctx, frame, true); + /* Try again if both input port and output port return EAGAIN. + * If no data is consumed and no frame in output, it can make + * both avcodec_send_packet() and avcodec_receive_frame() + * return EAGAIN, which violate the design. + */ + if (ff_AMediaCodec_infoTryAgainLater(s->ctx->codec, index) && + ret == AVERROR(EAGAIN)) + continue; + return ret; } s->ctx->current_input_buffer = index; } From patchwork Sun Nov 20 06:49:38 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhao Zhili X-Patchwork-Id: 39348 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a21:999a:b0:a4:2148:650a with SMTP id ve26csp1406564pzb; Sat, 19 Nov 2022 22:51:04 -0800 (PST) X-Google-Smtp-Source: AA0mqf7odhIIjdLB7XyYTTv5J8ZEcJG8PWIlgEKAZBpJARUu1imVIwVW08XM+okH5djnL5piA91p X-Received: by 2002:a05:6402:5406:b0:467:4b3d:f2ed with SMTP id ev6-20020a056402540600b004674b3df2edmr11885681edb.101.1668927063925; Sat, 19 Nov 2022 22:51:03 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1668927063; cv=none; d=google.com; s=arc-20160816; b=QabVCeqesC+003iWhIH1j68q8xwd1igXd5X/S562ei45WzjTUxLOzjOib7OfyWgcHU t2P0xZIW+l0pxqLEj3vVPaeRTlZk4/Ie6Tt7v+JlytnDVL75Pub6Fg8r3oRkAbTHJM1l Szy9Dy6TNBPRM5v8K4+KzqQYIXpuiOeOd2FmR2fXbEq7I3xa/p6s+o6MSnsiTO4XEd1P ENyD8o1aOoqQ6I+kY9ZXKenwj2WEowmqv6fNLHRxc/LBaH3uZGsifVWBt3BEE+XC1eaQ K4nVwLbauXH6Azhrbz/b2u0SOxcECzBtjZ/6hyTtfxShNwxI7EXrQLf8CjWOzYhkCTm0 rD7Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to:date :to:from:message-id:dkim-signature:delivered-to; bh=QgqnqTMCh4xVtzvdPeiyxX57vvENxEXVu1rZ831lZGQ=; b=mv2Rz5vnR9YnvWUxoBI0+n955BDEIvuyGHjaoWNyl3NBX86KSh2Z0pBukjrFYIDt3R 2CAIbveXQzUgS7dfvZDmvBhOCxbTlDL6u9avMujtF0TtOZ+fEEmpaevttTUTnwlb4myE HH9DhM9reJgiYVnnxa3c1MHJA2BCKb4lUHqhLLW6L1drZNkZ4bceQf+1nuGCYDt3NJ+z O1VOD+eATxeE4dsuTGbuOE0ca73eO/DiMDvYfD2vb0D/YPbg4B/Ma4QoWW96x0Jz60xR Dumo3rXbTIR2NlrnVp4ChgPhphzdAnNt7jMDP+KHuQ3SQpltFHaijH7BnujTd4gD/75i 6K6A== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@foxmail.com header.s=s201512 header.b=Xfs3WT9m; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=foxmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id js21-20020a17090797d500b0078c6a12ff3dsi7521046ejc.215.2022.11.19.22.51.03; Sat, 19 Nov 2022 22:51:03 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@foxmail.com header.s=s201512 header.b=Xfs3WT9m; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=foxmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 31BAE68BBD2; Sun, 20 Nov 2022 08:50:47 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from out162-62-58-216.mail.qq.com (out162-62-58-216.mail.qq.com [162.62.58.216]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 740FA68BAC0 for ; Sun, 20 Nov 2022 08:50:37 +0200 (EET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=foxmail.com; s=s201512; t=1668927035; bh=fL7/hDXaULjMIYYIx88hrWUbmx9uw05eWzJ29BGW/xs=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=Xfs3WT9mv4s5AKVCxdc0LhbK9pgChAANSFms2R0KCh5aJOEhrJGBwrv3OZq1UWeXR 6wKK8vEQMdAs53zxG3ps3XcyIGhgWGJP2nJHDBrv2hnVgYTJAO4vqA4CpvtrMnqoNv woROpw76BtXEDImPkgRjBVtD4JjnFAKRSpvIXasc= Received: from localhost.localdomain ([14.153.77.200]) by newxmesmtplogicsvrszc2-1.qq.com (NewEsmtp) with SMTP id C8F0626E; Sun, 20 Nov 2022 14:50:15 +0800 X-QQ-mid: xmsmtpt1668927034thvcabc8v Message-ID: X-QQ-XMAILINFO: NxIimAsTiib4HcIu7mmMiR7M6RAgQT+WUWLoB9ctdA+XtnkrGwNv6uK/NPb3sN ToQrU/jH/ReDKe5R/1Ug7xbhJppYQJAcQkj2nCJH2o+JapZJA0MY2lW4F1j1mkRUNElnHrPyNVOw bfguAFIIJKYwinMlypty2uB17UdrTrUIjgMph366WvolOaD6nuR+Kpm/D8/z9UyMyH5qhCWwv/Xp /qtNriHlquVcLO05hm8c4A1OrHBeiodeLeH7FiJA/pf4f2EcLzK7uThagAMS+Cn45c45dl/ikz88 E8HTqLLyFr4ow6VPNC0NC5r8zk9pwljc5W8b2NFYGxVQaarqyOzEK3Z5mIg8/0UMfp/F5vTJKJfD TZGTtuXgDDFhX+SI2qeYpVvFuNH0OZ3kB9OCMTQKeLGbEQ9C2ivqL0KQSOWG7H2jOy7zyFxTzhxx vjm7nCOPUYasLQ3mY5zqQZeqeqr7+29Kc/jsUUWb8ZGtmyfu1XvMFKlatGut7JLRRTB5WrOklRHP FLEHMgi3wpa5h2Fi1RjQI286unon9Q8MBu3CjaMHf8zCL8PEza6ZTgQO2s9s4VTKe2SdH6Fd7V6E HFC85iQxkQiOzGltJi5oPqOUFYbtDnShhIzHNngVKjq4c7vyKNSiGOO+t/3jHMy3p4VHuNzcE/uK NEjuKxuL/+Gtx6fiB7fWbxy3dO7ycV1Zrrn8B7EAa/Pf0R2/tYCUhQdzZnK7R5fWGn8PzuRLCzzk ppkIw20jvysvmHXc5DQ1enbiv9+eqnl03oB/K48cA/loGVZATzqr0ltRA+rniOMuXOweKR26XV5b pyOmFmGWWvyGWFEIYekPNoT8eGch+Mf/5jX8nOWJnm0vl5tHQcF0uVPqA5MhyqNEC3payHAeN3ko z7UGOj1Q3G6IXBFdtN9kBDd4dESpNgCEo93/c+TrhaiXAu1Q5c4qdFpaeJ0awsMxlxTEVvWiEn3/ wph9GiFdKE+ddz+HlYsApN0oE+7RYd7AsrB0cdUi3/FVoYElfNwUXXwbSb3+9lO7R5xX0X0ic= From: Zhao Zhili To: ffmpeg-devel@ffmpeg.org Date: Sun, 20 Nov 2022 14:49:38 +0800 X-OQ-MSGID: <20221120064947.94540-4-quinkblack@foxmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221120064947.94540-1-quinkblack@foxmail.com> References: <20221120064947.94540-1-quinkblack@foxmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 03/12] avcodec/mediacodecdec_common: fix misuse av_free/av_freep X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: matthieu.bouron@stupeflix.com, Zhao Zhili , aman@tmm1.net Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 2AMP7JXjzbGv From: Zhao Zhili Signed-off-by: Zhao Zhili --- libavcodec/mediacodecdec_common.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/libavcodec/mediacodecdec_common.c b/libavcodec/mediacodecdec_common.c index f430cfed31..4c48b9142e 100644 --- a/libavcodec/mediacodecdec_common.c +++ b/libavcodec/mediacodecdec_common.c @@ -334,7 +334,7 @@ static int mediacodec_wrap_hw_buffer(AVCodecContext *avctx, return 0; fail: - av_freep(buffer); + av_freep(&buffer); av_buffer_unref(&frame->buf[0]); status = ff_AMediaCodec_releaseOutputBuffer(s->codec, index, 0); if (status < 0) { From patchwork Sun Nov 20 06:49:39 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhao Zhili X-Patchwork-Id: 39349 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a21:999a:b0:a4:2148:650a with SMTP id ve26csp1406602pzb; Sat, 19 Nov 2022 22:51:12 -0800 (PST) X-Google-Smtp-Source: AA0mqf6aAfW5t+MJi/mb+66ijQHkjFydujGAEojMhBM1M5iPOwRQ+jOG2aPTXDhBf+sGUtRc1O7Y X-Received: by 2002:a05:6402:170d:b0:467:69a4:afe1 with SMTP id y13-20020a056402170d00b0046769a4afe1mr12126001edu.332.1668927072415; Sat, 19 Nov 2022 22:51:12 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1668927072; cv=none; d=google.com; s=arc-20160816; b=Py+Q2jsqL4dKytza9QLSkwLWX/2MfL9Q6dolf9zYneqCqJ/FbbC9IF58hCimyckP/t V732pGpUDzZzP38Nusgx46lbXKU8Tu2p3Iu9wvoWWZ5ACV4+bk9EGOdSGFQfukvfO2Cb QvnXb4bU+Im4tH2lLxFQrI3ZqFbN6AC/qi1YciwBF8R1rIA08ydYeokTEHK3HZUTxLZl 9PmbK0lh18evA3dr6YLcEg6U/CtpRWWSAjfckjRNzSt9udUQDuOPF0YsGUJtX6VTHPRH xNfhB3zfJl45ykexyNtbQE06GSQ8itumHJKOhqGTa4oP9YOqVKuM0DKmUF3xmpw7M5I7 HKrg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to:date :to:from:message-id:dkim-signature:delivered-to; bh=92z76Ij5G63X7F0Tn0GhYUVvGzkrFKeC/6T+yhICzxo=; b=GjZF85KtWvfBc/jrB/iKpTMlSWQnJTa37PQJ+Lvwc/FI1dHFz/zcd5AUpjEAhcsrZq jbd1pG6gprFKpfQh5Pmw95/cfXPdlp12xoNRwFsW98hZ0LasQ+zfPNLa57B8H/cVLAki fuQ7AEp3vs989pYVoB6VLEK+8eDXE6rl5zTVowbILWi2W+1oSm4K9kHwsT3v62h6AMjA 9FrzBQKDIV7ik6lblULtdiPxYcgB87GL3ESenGDNNHiCoRJP7CJlKprk0FXFKifZDgJ6 NVciqJWBytwKg1PO2dhfrCTLBLlI5YsjCGzuSmX0sxZW2cReNPAINp4om9BI14CtAHM4 xKIg== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@foxmail.com header.s=s201512 header.b=tuHfhRxR; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=foxmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id nb11-20020a1709071c8b00b007a953d421bbsi6921032ejc.322.2022.11.19.22.51.12; Sat, 19 Nov 2022 22:51:12 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@foxmail.com header.s=s201512 header.b=tuHfhRxR; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=foxmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 4D34D68BBDE; Sun, 20 Nov 2022 08:50:48 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from out162-62-57-252.mail.qq.com (out162-62-57-252.mail.qq.com [162.62.57.252]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id D419E68BB60 for ; Sun, 20 Nov 2022 08:50:37 +0200 (EET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=foxmail.com; s=s201512; t=1668927035; bh=lWOd2ji+O5MZTTmhLV+r38FAccPUJAd3JtM8g+orwP8=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=tuHfhRxRgJv5z56TPdbODYvClevScOu56VRxxLo9bLxztgJm8I38jzv8c9dTs4x5K iXLZkD49VZxmBuTyrxmZBQGiGtEJWDeLrutePOVfJMJER1WlYrwMBghAxuk1MvIbPI abTTYoor8cY8iPAgwsgEbUboP7VL6XNDL6gnVXsU= Received: from localhost.localdomain ([14.153.77.200]) by newxmesmtplogicsvrszc2-1.qq.com (NewEsmtp) with SMTP id C8F0626E; Sun, 20 Nov 2022 14:50:15 +0800 X-QQ-mid: xmsmtpt1668927034t8jpa3uwu Message-ID: X-QQ-XMAILINFO: MB5+LsFw85NobaPc+fVyy6Fy/vd+qavIA951ygM6PDWqiYh/3s6zQF5KwMVCiA goKlG5bchq4VfYfj0MOmJ2mb4M5p5L/T+exhRxi4+DVKDMggzD3nY2JvW8QqOhCpo9ZWbj3ntE6d OcWbjNbXtW3L+HCZSxFyYhJyaw3doXzr3sEMMU0AuMnEOoxYOUbctLTjPH8of+2UpKF2gnC/DNdz DAREAzWeQUpw46uG60i85XucWDhH2zuUYh0eNOpC8Ssi2FaGKU80oWpucMbrx2ePGSpGFj7OIBCi fL85/Y0HW6lNFRwtXkqMSxPGQJMT5VjaC/psycNSuWeNpaRQTDYdEm6hMicu3lvjcAQJ5almCwGe kfPJonBiBBksEJXvxtQXMmmXGnjT0KmZFrtJSzHSn592nLaICjNgpGyZs8kIIWN1+xE/UT0o+BHz RDjrNB7DUqRUfu60CPJppPbgSQGSopMN0Ipfqbclqhaq27WiI0JmgebsmeBkuNWHq6xPq+7wTOJY xGmNI7duES3d2nlEWWkew2Ds6LdQO8saslXUMmf2HOKhktd7Zgy+X+Mo28DESmzGyKNWNt5q9oEd YWw7yqRE+W7DUU2TLSaIwY1MPAFnowqiCUOLu4gHoY/8ihN7HpL4tjcbvzsT2nKzyyKrKcFQ8Da0 kbK6hhcOrcf3HEJCBXwYM86Kz8C2YZzyowslK/0BHF8Hr63/5x6FvlM6FyBVllDCwUnOsR3OAfhh KZpdeOsNkAp3af8afEMlk/Vy9wHJvOnvmtj4zP4X3VlzhXUBnyv0gj1vJGeLSPncJRroD5BZmZQx aKzdiMGN9UtgUTPDhzWmZF6n5eg8KFReCtlrBOzd89cvFMmaqf3eeQREju93Ddh20RpWAwDatPjZ m8j4gQWHfUDh6ut+hQDHXQH4EpQdnRGFkEUqZaCu0LBTPyxdO8OF07bn0Uc2XEnNFzu7STalH0Vt kHMWVTDA10Az5LjRyFoE7esIFg7+iVe2KnLpz1EyNCtMP99qvNm7eOuTXztEgdRCcGWtWVuk4= From: Zhao Zhili To: ffmpeg-devel@ffmpeg.org Date: Sun, 20 Nov 2022 14:49:39 +0800 X-OQ-MSGID: <20221120064947.94540-5-quinkblack@foxmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221120064947.94540-1-quinkblack@foxmail.com> References: <20221120064947.94540-1-quinkblack@foxmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 04/12] avcodec/mediacodecdec_common: fix useless av_buffer_unref X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: matthieu.bouron@stupeflix.com, Zhao Zhili , aman@tmm1.net Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: VpOglDeljTIX From: Zhao Zhili Since frame->buf[0] is always NULL in this case, av_buffer_unref has no effect. If it's not NULL, double-free will happen. Signed-off-by: Zhao Zhili --- libavcodec/mediacodecdec_common.c | 1 - 1 file changed, 1 deletion(-) diff --git a/libavcodec/mediacodecdec_common.c b/libavcodec/mediacodecdec_common.c index 4c48b9142e..69a462ec48 100644 --- a/libavcodec/mediacodecdec_common.c +++ b/libavcodec/mediacodecdec_common.c @@ -335,7 +335,6 @@ static int mediacodec_wrap_hw_buffer(AVCodecContext *avctx, return 0; fail: av_freep(&buffer); - av_buffer_unref(&frame->buf[0]); status = ff_AMediaCodec_releaseOutputBuffer(s->codec, index, 0); if (status < 0) { av_log(avctx, AV_LOG_ERROR, "Failed to release output buffer\n"); From patchwork Sun Nov 20 06:49:40 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhao Zhili X-Patchwork-Id: 39351 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a21:999a:b0:a4:2148:650a with SMTP id ve26csp1406695pzb; Sat, 19 Nov 2022 22:51:30 -0800 (PST) X-Google-Smtp-Source: AA0mqf6DVJgH5S655nm0+LpRxSxd+VnBNjPCxZ4WZbRvKqkrreWrJKiwrAgO9AMhUhaXW/8ezUL0 X-Received: by 2002:a17:907:7e86:b0:7af:bc9:5e8d with SMTP id qb6-20020a1709077e8600b007af0bc95e8dmr11537716ejc.3.1668927090314; Sat, 19 Nov 2022 22:51:30 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1668927090; cv=none; d=google.com; s=arc-20160816; b=eZtUMxICJ95eU0QvOkeFMzevpG43wYVm3BNuBBVRusZRpwCgg24lywS7saJxzf9Ame LyAD5ZomopGOqlUWka7ZnFAUbhZQA8s3f5iU4s+uoZxyiIMLB4teeLuGdNhe9JgiXvAG fiqpDd9Qa3nGLNMlFfaYflqgkUYGzyrRzbPx9tx9Z6fvhL+RtqcBtOdVC3ekQr51wcMn giRpUD4yusznhg1GuVPQM3HplQQ6b5pchS1BknVPN5/DrrTXteepJWYqpFSz/An5U5eK LzGrsyizYkgqtxNG5yKEZ+deighxyij7gf9puzV9Fu0NTJCJiHx1GYydT0FsjhMdHcGX 475A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to:date :to:from:message-id:dkim-signature:delivered-to; bh=ifySuZb+cijYNaQBsrh6fWsJKongNm7crU+1pa0mqiE=; b=L4W6btI8c8CKfw3rM2kMr88djDz6LdMLQkd6WBRyYO7hQxF4o+P/1bcN7egU3LVG66 8qqpDC/wLJQJot0rgRR/r8OWZQFV29SSrFAY8rCrLoQMuUUaoAk8AjMDFqz6x5xcpzZq HrQDUJ3owsTwaJbPIHd40FAeuFHx60chK5nH5ZlikLylpDc/lT/tHTzbv78oDrrm4HmE oEgZA7E57sPzEzB2XlZelDGs+/4fFSy1i2CBZTr1FZYAwv3+L/mmycvQe6furUiyf9e/ VndmKlz9VwizTemU5ZsQ/l9LxL0dPKQLchG+MS+e+4JXQOOv9ugKPyE++fyiURuLnkVg X6Rw== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@foxmail.com header.s=s201512 header.b=ikAv+lEK; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=foxmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id qc24-20020a170906d8b800b007ae8a4b03d7si5505935ejb.973.2022.11.19.22.51.30; Sat, 19 Nov 2022 22:51:30 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@foxmail.com header.s=s201512 header.b=ikAv+lEK; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=foxmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 7A0FC68BBF4; Sun, 20 Nov 2022 08:50:50 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from out162-62-58-211.mail.qq.com (out162-62-58-211.mail.qq.com [162.62.58.211]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id D9DE368BB78 for ; Sun, 20 Nov 2022 08:50:38 +0200 (EET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=foxmail.com; s=s201512; t=1668927036; bh=5WT92i5iG9ElscGVw4V6WlitzhgIJkqZg0r2ou3Qick=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=ikAv+lEKJaVNVRAqB7llnRzxHhOWlId9tMoneAE1wv7Eh5vBG58gQg3yFoTIkSfno vkk9qih9G8qjno0eooTiy3mWTz6+yD3OIN0O/o4bUZt9+SYrNWqyt2OQueAHkk6EOt gF9ZB2ZLgmFikhu7/sG6K1mIIbhi4nS/s9ObEIlM= Received: from localhost.localdomain ([14.153.77.200]) by newxmesmtplogicsvrszc2-1.qq.com (NewEsmtp) with SMTP id C8F0626E; Sun, 20 Nov 2022 14:50:15 +0800 X-QQ-mid: xmsmtpt1668927035tuijmhfiy Message-ID: X-QQ-XMAILINFO: MR/iVh5QLeiewdwDs7p04NP3vTGqY7+8ZUgnYaDPP8P6eb9E4gFspZ3anacf6c bmrOMnrkH6uz25mPfveTeil7MJcAew54+athkl0CZp0izjPQrztOUnIuvYGzFSOqG841YbJTWTCG npz8+ag/oPMv7U2O7Zq1KudjTUFBROyHX68FS8rCoOSW8PCfB+lD6HSiSYzu0Cc9pkohuSESj8Iu QKLOVte2RmibYDqp9XwMJeyUo54VqLIeiC6ziPri1wtIw3cJn14cNLxQuw4brvldkTb5M5xQPW3T m5qg41qvrIyedGmU+W4qo+/6JaofQ0wDhogk0rQ1b4XIMWmgyAj9fYtfbkSY4OgsCcqqnlStE3AD XtdX11edRiejUeHul30EGD98bNXSS+o453hagmk5eIHX7HTo31OZBROfcbkDXrJYZIKZyh2dysar o2x7iJvRu05Ax8JlSHFi2D2q3bd82jzdFzOoQQqj6KqjCW8fVcyDnJqqhREE5+ekGf5fO3eZhOCV iA+DlwCesM7SJtk1TLk/rtC4X37HlssMRLJCW5nJNEbNze74u8vOG8zoL+zAuSLNlQK+ZNYxxN+i DS4uVBQkXm7PgZ8G1NIGy3sB2cgdSY/rZTzvk+vDaweqqfGcjIs/xq5KI4BytGA/xgk86N2CGmW9 cFLKsCx9OtWR9AUBUKmJOhr0D9hZmsqOBZYp6WNR6uRNsGGvf7j6eFyIlQkj8+37q8Om7PS/CT54 fFuMgzr7ZzNm1T4uTj4j+Lo1j4HIVym8ghEl2ybAnzo7i0SaLRObMgfSaJqBM8rvt0erEGp4Fe7o XHAMwUFEKBoVht0ekwe0WwD+Ci2EbcX+gqZDQ5C6orpBkjQQ7x5Xn29RxF7f09Ul/eMCrU7WTqcS cQnofBtsrgR3gidZzW8Yd/sIW7rMvC6fHYewTUQFM+rQZiDu920Z12OgL3rzMvs0mha0fGpzXFWS JgaFdunLMFydRnhgXBfrY7l0KGcay6zojDbpupVhM= From: Zhao Zhili To: ffmpeg-devel@ffmpeg.org Date: Sun, 20 Nov 2022 14:49:40 +0800 X-OQ-MSGID: <20221120064947.94540-6-quinkblack@foxmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221120064947.94540-1-quinkblack@foxmail.com> References: <20221120064947.94540-1-quinkblack@foxmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 05/12] avcodec/mediacodec_wrapper: separate implementation from interface X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: matthieu.bouron@stupeflix.com, Zhao Zhili , aman@tmm1.net Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: SDhGu5Tm4oCw From: Zhao Zhili This is in preparation for NDK media codec wrapper. Signed-off-by: Zhao Zhili --- libavcodec/mediacodec_wrapper.c | 244 +++++++++++++++++++++--------- libavcodec/mediacodec_wrapper.h | 255 +++++++++++++++++++++++++++----- 2 files changed, 394 insertions(+), 105 deletions(-) diff --git a/libavcodec/mediacodec_wrapper.c b/libavcodec/mediacodec_wrapper.c index 8ffc58e1d8..e0c614680e 100644 --- a/libavcodec/mediacodec_wrapper.c +++ b/libavcodec/mediacodec_wrapper.c @@ -160,12 +160,14 @@ static const AVClass amediaformat_class = { .version = LIBAVUTIL_VERSION_INT, }; -struct FFAMediaFormat { +typedef struct FFAMediaFormatJni { + FFAMediaFormat api; - const AVClass *class; struct JNIAMediaFormatFields jfields; jobject object; -}; +} FFAMediaFormatJni; + +static const FFAMediaFormat media_format_jni; struct JNIAMediaCodecFields { @@ -272,9 +274,8 @@ static const AVClass amediacodec_class = { .version = LIBAVUTIL_VERSION_INT, }; -struct FFAMediaCodec { - - const AVClass *class; +typedef struct FFAMediaCodecJni { + FFAMediaCodec api; struct JNIAMediaCodecFields jfields; @@ -295,7 +296,9 @@ struct FFAMediaCodec { int CONFIGURE_FLAG_ENCODE; int has_get_i_o_buffer; -}; +} FFAMediaCodecJni; + +static const FFAMediaCodec media_codec_jni; #define JNI_GET_ENV_OR_RETURN(env, log_ctx, ret) do { \ (env) = ff_jni_get_env(log_ctx); \ @@ -622,17 +625,17 @@ done: return name; } -FFAMediaFormat *ff_AMediaFormat_new(void) +static FFAMediaFormat *mediaformat_jni_new(void) { JNIEnv *env = NULL; - FFAMediaFormat *format = NULL; + FFAMediaFormatJni *format = NULL; jobject object = NULL; - format = av_mallocz(sizeof(FFAMediaFormat)); + format = av_mallocz(sizeof(*format)); if (!format) { return NULL; } - format->class = &amediaformat_class; + format->api = media_format_jni; env = ff_jni_get_env(format); if (!env) { @@ -664,19 +667,19 @@ fail: av_freep(&format); } - return format; + return (FFAMediaFormat *)format; } -static FFAMediaFormat *ff_AMediaFormat_newFromObject(void *object) +static FFAMediaFormat *mediaformat_jni_newFromObject(void *object) { JNIEnv *env = NULL; - FFAMediaFormat *format = NULL; + FFAMediaFormatJni *format = NULL; - format = av_mallocz(sizeof(FFAMediaFormat)); + format = av_mallocz(sizeof(*format)); if (!format) { return NULL; } - format->class = &amediaformat_class; + format->api = media_format_jni; env = ff_jni_get_env(format); if (!env) { @@ -693,7 +696,7 @@ static FFAMediaFormat *ff_AMediaFormat_newFromObject(void *object) goto fail; } - return format; + return (FFAMediaFormat *)format; fail: ff_jni_reset_jfields(env, &format->jfields, jni_amediaformat_mapping, 1, format); @@ -702,10 +705,10 @@ fail: return NULL; } -int ff_AMediaFormat_delete(FFAMediaFormat* format) +static int mediaformat_jni_delete(FFAMediaFormat* ctx) { int ret = 0; - + FFAMediaFormatJni *format = (FFAMediaFormatJni *)ctx; JNIEnv *env = NULL; if (!format) { @@ -724,10 +727,10 @@ int ff_AMediaFormat_delete(FFAMediaFormat* format) return ret; } -char* ff_AMediaFormat_toString(FFAMediaFormat* format) +static char* mediaformat_jni_toString(FFAMediaFormat* ctx) { char *ret = NULL; - + FFAMediaFormatJni *format = (FFAMediaFormatJni *)ctx; JNIEnv *env = NULL; jstring description = NULL; @@ -749,10 +752,10 @@ fail: return ret; } -int ff_AMediaFormat_getInt32(FFAMediaFormat* format, const char *name, int32_t *out) +static int mediaformat_jni_getInt32(FFAMediaFormat* ctx, const char *name, int32_t *out) { int ret = 1; - + FFAMediaFormatJni *format = (FFAMediaFormatJni *)ctx; JNIEnv *env = NULL; jstring key = NULL; jboolean contains_key; @@ -788,10 +791,10 @@ fail: return ret; } -int ff_AMediaFormat_getInt64(FFAMediaFormat* format, const char *name, int64_t *out) +static int mediaformat_jni_getInt64(FFAMediaFormat* ctx, const char *name, int64_t *out) { int ret = 1; - + FFAMediaFormatJni *format = (FFAMediaFormatJni *)ctx; JNIEnv *env = NULL; jstring key = NULL; jboolean contains_key; @@ -827,10 +830,10 @@ fail: return ret; } -int ff_AMediaFormat_getFloat(FFAMediaFormat* format, const char *name, float *out) +static int mediaformat_jni_getFloat(FFAMediaFormat* ctx, const char *name, float *out) { int ret = 1; - + FFAMediaFormatJni *format = (FFAMediaFormatJni *)ctx; JNIEnv *env = NULL; jstring key = NULL; jboolean contains_key; @@ -866,10 +869,10 @@ fail: return ret; } -int ff_AMediaFormat_getBuffer(FFAMediaFormat* format, const char *name, void** data, size_t *size) +static int mediaformat_jni_getBuffer(FFAMediaFormat* ctx, const char *name, void** data, size_t *size) { int ret = 1; - + FFAMediaFormatJni *format = (FFAMediaFormatJni *)ctx; JNIEnv *env = NULL; jstring key = NULL; jboolean contains_key; @@ -924,10 +927,10 @@ fail: return ret; } -int ff_AMediaFormat_getString(FFAMediaFormat* format, const char *name, const char **out) +static int mediaformat_jni_getString(FFAMediaFormat* ctx, const char *name, const char **out) { int ret = 1; - + FFAMediaFormatJni *format = (FFAMediaFormatJni *)ctx; JNIEnv *env = NULL; jstring key = NULL; jboolean contains_key; @@ -974,10 +977,11 @@ fail: return ret; } -void ff_AMediaFormat_setInt32(FFAMediaFormat* format, const char* name, int32_t value) +static void mediaformat_jni_setInt32(FFAMediaFormat* ctx, const char* name, int32_t value) { JNIEnv *env = NULL; jstring key = NULL; + FFAMediaFormatJni *format = (FFAMediaFormatJni *)ctx; av_assert0(format != NULL); @@ -999,10 +1003,11 @@ fail: } } -void ff_AMediaFormat_setInt64(FFAMediaFormat* format, const char* name, int64_t value) +static void mediaformat_jni_setInt64(FFAMediaFormat* ctx, const char* name, int64_t value) { JNIEnv *env = NULL; jstring key = NULL; + FFAMediaFormatJni *format = (FFAMediaFormatJni *)ctx; av_assert0(format != NULL); @@ -1024,10 +1029,11 @@ fail: } } -void ff_AMediaFormat_setFloat(FFAMediaFormat* format, const char* name, float value) +static void mediaformat_jni_setFloat(FFAMediaFormat* ctx, const char* name, float value) { JNIEnv *env = NULL; jstring key = NULL; + FFAMediaFormatJni *format = (FFAMediaFormatJni *)ctx; av_assert0(format != NULL); @@ -1049,11 +1055,12 @@ fail: } } -void ff_AMediaFormat_setString(FFAMediaFormat* format, const char* name, const char* value) +static void mediaformat_jni_setString(FFAMediaFormat* ctx, const char* name, const char* value) { JNIEnv *env = NULL; jstring key = NULL; jstring string = NULL; + FFAMediaFormatJni *format = (FFAMediaFormatJni *)ctx; av_assert0(format != NULL); @@ -1084,12 +1091,13 @@ fail: } } -void ff_AMediaFormat_setBuffer(FFAMediaFormat* format, const char* name, void* data, size_t size) +static void mediaformat_jni_setBuffer(FFAMediaFormat* ctx, const char* name, void* data, size_t size) { JNIEnv *env = NULL; jstring key = NULL; jobject buffer = NULL; void *buffer_data = NULL; + FFAMediaFormatJni *format = (FFAMediaFormatJni *)ctx; av_assert0(format != NULL); @@ -1131,7 +1139,7 @@ fail: } } -static int codec_init_static_fields(FFAMediaCodec *codec) +static int codec_init_static_fields(FFAMediaCodecJni *codec) { int ret = 0; JNIEnv *env = NULL; @@ -1193,17 +1201,17 @@ static inline FFAMediaCodec *codec_create(int method, const char *arg) { int ret = -1; JNIEnv *env = NULL; - FFAMediaCodec *codec = NULL; + FFAMediaCodecJni *codec = NULL; jstring jarg = NULL; jobject object = NULL; jobject buffer_info = NULL; jmethodID create_id = NULL; - codec = av_mallocz(sizeof(FFAMediaCodec)); + codec = av_mallocz(sizeof(*codec)); if (!codec) { return NULL; } - codec->class = &amediacodec_class; + codec->api = media_codec_jni; env = ff_jni_get_env(codec); if (!env) { @@ -1286,11 +1294,11 @@ fail: av_freep(&codec); } - return codec; + return (FFAMediaCodec *)codec; } #define DECLARE_FF_AMEDIACODEC_CREATE_FUNC(name, method) \ -FFAMediaCodec *ff_AMediaCodec_##name(const char *arg) \ +static FFAMediaCodec *mediacodec_jni_##name(const char *arg) \ { \ return codec_create(method, arg); \ } \ @@ -1299,10 +1307,10 @@ DECLARE_FF_AMEDIACODEC_CREATE_FUNC(createCodecByName, CREATE_CODEC_BY_NAME) DECLARE_FF_AMEDIACODEC_CREATE_FUNC(createDecoderByType, CREATE_DECODER_BY_TYPE) DECLARE_FF_AMEDIACODEC_CREATE_FUNC(createEncoderByType, CREATE_ENCODER_BY_TYPE) -int ff_AMediaCodec_delete(FFAMediaCodec* codec) +static int mediacodec_jni_delete(FFAMediaCodec* ctx) { int ret = 0; - + FFAMediaCodecJni *codec = (FFAMediaCodecJni *)ctx; JNIEnv *env = NULL; if (!codec) { @@ -1335,11 +1343,12 @@ int ff_AMediaCodec_delete(FFAMediaCodec* codec) return ret; } -char *ff_AMediaCodec_getName(FFAMediaCodec *codec) +static char *mediacodec_jni_getName(FFAMediaCodec *ctx) { char *ret = NULL; JNIEnv *env = NULL; jobject *name = NULL; + FFAMediaCodecJni *codec = (FFAMediaCodecJni *)ctx; JNI_GET_ENV_OR_RETURN(env, codec, NULL); @@ -1358,10 +1367,12 @@ fail: return ret; } -int ff_AMediaCodec_configure(FFAMediaCodec* codec, const FFAMediaFormat* format, void* surface, void *crypto, uint32_t flags) +static int mediacodec_jni_configure(FFAMediaCodec* ctx, const FFAMediaFormat* format_ctx, void* surface, void *crypto, uint32_t flags) { int ret = 0; JNIEnv *env = NULL; + FFAMediaCodecJni *codec = (FFAMediaCodecJni *)ctx; + const FFAMediaFormatJni *format = (FFAMediaFormatJni *)format_ctx; JNI_GET_ENV_OR_RETURN(env, codec, AVERROR_EXTERNAL); @@ -1375,10 +1386,11 @@ fail: return ret; } -int ff_AMediaCodec_start(FFAMediaCodec* codec) +static int mediacodec_jni_start(FFAMediaCodec* ctx) { int ret = 0; JNIEnv *env = NULL; + FFAMediaCodecJni *codec = (FFAMediaCodecJni *)ctx; JNI_GET_ENV_OR_RETURN(env, codec, AVERROR_EXTERNAL); @@ -1392,10 +1404,11 @@ fail: return ret; } -int ff_AMediaCodec_stop(FFAMediaCodec* codec) +static int mediacodec_jni_stop(FFAMediaCodec* ctx) { int ret = 0; JNIEnv *env = NULL; + FFAMediaCodecJni *codec = (FFAMediaCodecJni *)ctx; JNI_GET_ENV_OR_RETURN(env, codec, AVERROR_EXTERNAL); @@ -1409,10 +1422,11 @@ fail: return ret; } -int ff_AMediaCodec_flush(FFAMediaCodec* codec) +static int mediacodec_jni_flush(FFAMediaCodec* ctx) { int ret = 0; JNIEnv *env = NULL; + FFAMediaCodecJni *codec = (FFAMediaCodecJni *)ctx; JNI_GET_ENV_OR_RETURN(env, codec, AVERROR_EXTERNAL); @@ -1426,10 +1440,11 @@ fail: return ret; } -int ff_AMediaCodec_releaseOutputBuffer(FFAMediaCodec* codec, size_t idx, int render) +static int mediacodec_jni_releaseOutputBuffer(FFAMediaCodec* ctx, size_t idx, int render) { int ret = 0; JNIEnv *env = NULL; + FFAMediaCodecJni *codec = (FFAMediaCodecJni *)ctx; JNI_GET_ENV_OR_RETURN(env, codec, AVERROR_EXTERNAL); @@ -1443,10 +1458,11 @@ fail: return ret; } -int ff_AMediaCodec_releaseOutputBufferAtTime(FFAMediaCodec *codec, size_t idx, int64_t timestampNs) +static int mediacodec_jni_releaseOutputBufferAtTime(FFAMediaCodec *ctx, size_t idx, int64_t timestampNs) { int ret = 0; JNIEnv *env = NULL; + FFAMediaCodecJni *codec = (FFAMediaCodecJni *)ctx; JNI_GET_ENV_OR_RETURN(env, codec, AVERROR_EXTERNAL); @@ -1460,10 +1476,11 @@ fail: return ret; } -ssize_t ff_AMediaCodec_dequeueInputBuffer(FFAMediaCodec* codec, int64_t timeoutUs) +static ssize_t mediacodec_jni_dequeueInputBuffer(FFAMediaCodec* ctx, int64_t timeoutUs) { int ret = 0; JNIEnv *env = NULL; + FFAMediaCodecJni *codec = (FFAMediaCodecJni *)ctx; JNI_GET_ENV_OR_RETURN(env, codec, AVERROR_EXTERNAL); @@ -1477,10 +1494,11 @@ fail: return ret; } -int ff_AMediaCodec_queueInputBuffer(FFAMediaCodec* codec, size_t idx, off_t offset, size_t size, uint64_t time, uint32_t flags) +static int mediacodec_jni_queueInputBuffer(FFAMediaCodec* ctx, size_t idx, off_t offset, size_t size, uint64_t time, uint32_t flags) { int ret = 0; JNIEnv *env = NULL; + FFAMediaCodecJni *codec = (FFAMediaCodecJni *)ctx; JNI_GET_ENV_OR_RETURN(env, codec, AVERROR_EXTERNAL); @@ -1494,10 +1512,11 @@ fail: return ret; } -ssize_t ff_AMediaCodec_dequeueOutputBuffer(FFAMediaCodec* codec, FFAMediaCodecBufferInfo *info, int64_t timeoutUs) +static ssize_t mediacodec_jni_dequeueOutputBuffer(FFAMediaCodec* ctx, FFAMediaCodecBufferInfo *info, int64_t timeoutUs) { int ret = 0; JNIEnv *env = NULL; + FFAMediaCodecJni *codec = (FFAMediaCodecJni *)ctx; JNI_GET_ENV_OR_RETURN(env, codec, AVERROR_EXTERNAL); @@ -1529,11 +1548,11 @@ ssize_t ff_AMediaCodec_dequeueOutputBuffer(FFAMediaCodec* codec, FFAMediaCodecBu return ret; } -uint8_t* ff_AMediaCodec_getInputBuffer(FFAMediaCodec* codec, size_t idx, size_t *out_size) +static uint8_t* mediacodec_jni_getInputBuffer(FFAMediaCodec* ctx, size_t idx, size_t *out_size) { uint8_t *ret = NULL; JNIEnv *env = NULL; - + FFAMediaCodecJni *codec = (FFAMediaCodecJni *)ctx; jobject buffer = NULL; jobject input_buffers = NULL; @@ -1577,11 +1596,11 @@ fail: return ret; } -uint8_t* ff_AMediaCodec_getOutputBuffer(FFAMediaCodec* codec, size_t idx, size_t *out_size) +static uint8_t* mediacodec_jni_getOutputBuffer(FFAMediaCodec* ctx, size_t idx, size_t *out_size) { uint8_t *ret = NULL; JNIEnv *env = NULL; - + FFAMediaCodecJni *codec = (FFAMediaCodecJni *)ctx; jobject buffer = NULL; jobject output_buffers = NULL; @@ -1625,10 +1644,11 @@ fail: return ret; } -FFAMediaFormat* ff_AMediaCodec_getOutputFormat(FFAMediaCodec* codec) +static FFAMediaFormat* mediacodec_jni_getOutputFormat(FFAMediaCodec* ctx) { FFAMediaFormat *ret = NULL; JNIEnv *env = NULL; + FFAMediaCodecJni *codec = (FFAMediaCodecJni *)ctx; jobject mediaformat = NULL; @@ -1639,7 +1659,7 @@ FFAMediaFormat* ff_AMediaCodec_getOutputFormat(FFAMediaCodec* codec) goto fail; } - ret = ff_AMediaFormat_newFromObject(mediaformat); + ret = mediaformat_jni_newFromObject(mediaformat); fail: if (mediaformat) { (*env)->DeleteLocalRef(env, mediaformat); @@ -1648,44 +1668,52 @@ fail: return ret; } -int ff_AMediaCodec_infoTryAgainLater(FFAMediaCodec *codec, ssize_t idx) +static int mediacodec_jni_infoTryAgainLater(FFAMediaCodec *ctx, ssize_t idx) { + FFAMediaCodecJni *codec = (FFAMediaCodecJni *)ctx; return idx == codec->INFO_TRY_AGAIN_LATER; } -int ff_AMediaCodec_infoOutputBuffersChanged(FFAMediaCodec *codec, ssize_t idx) +static int mediacodec_jni_infoOutputBuffersChanged(FFAMediaCodec *ctx, ssize_t idx) { + FFAMediaCodecJni *codec = (FFAMediaCodecJni *)ctx; return idx == codec->INFO_OUTPUT_BUFFERS_CHANGED; } -int ff_AMediaCodec_infoOutputFormatChanged(FFAMediaCodec *codec, ssize_t idx) +static int mediacodec_jni_infoOutputFormatChanged(FFAMediaCodec *ctx, ssize_t idx) { + FFAMediaCodecJni *codec = (FFAMediaCodecJni *)ctx; return idx == codec->INFO_OUTPUT_FORMAT_CHANGED; } -int ff_AMediaCodec_getBufferFlagCodecConfig(FFAMediaCodec *codec) +static int mediacodec_jni_getBufferFlagCodecConfig(FFAMediaCodec *ctx) { + FFAMediaCodecJni *codec = (FFAMediaCodecJni *)ctx; return codec->BUFFER_FLAG_CODEC_CONFIG; } -int ff_AMediaCodec_getBufferFlagEndOfStream(FFAMediaCodec *codec) +static int mediacodec_jni_getBufferFlagEndOfStream(FFAMediaCodec *ctx) { + FFAMediaCodecJni *codec = (FFAMediaCodecJni *)ctx; return codec->BUFFER_FLAG_END_OF_STREAM; } -int ff_AMediaCodec_getBufferFlagKeyFrame(FFAMediaCodec *codec) +static int mediacodec_jni_getBufferFlagKeyFrame(FFAMediaCodec *ctx) { + FFAMediaCodecJni *codec = (FFAMediaCodecJni *)ctx; return codec->BUFFER_FLAG_KEY_FRAME; } -int ff_AMediaCodec_getConfigureFlagEncode(FFAMediaCodec *codec) +static int mediacodec_jni_getConfigureFlagEncode(FFAMediaCodec *ctx) { + FFAMediaCodecJni *codec = (FFAMediaCodecJni *)ctx; return codec->CONFIGURE_FLAG_ENCODE; } -int ff_AMediaCodec_cleanOutputBuffers(FFAMediaCodec *codec) +static int mediacodec_jni_cleanOutputBuffers(FFAMediaCodec *ctx) { int ret = 0; + FFAMediaCodecJni *codec = (FFAMediaCodecJni *)ctx; if (!codec->has_get_i_o_buffer) { if (codec->output_buffers) { @@ -1706,6 +1734,86 @@ fail: return ret; } +static const FFAMediaFormat media_format_jni = { + .class = &amediaformat_class, + + .create = mediaformat_jni_new, + .delete = mediaformat_jni_delete, + + .toString = mediaformat_jni_toString, + + .getInt32 = mediaformat_jni_getInt32, + .getInt64 = mediaformat_jni_getInt64, + .getFloat = mediaformat_jni_getFloat, + .getBuffer = mediaformat_jni_getBuffer, + .getString = mediaformat_jni_getString, + + .setInt32 = mediaformat_jni_setInt32, + .setInt64 = mediaformat_jni_setInt64, + .setFloat = mediaformat_jni_setFloat, + .setString = mediaformat_jni_setString, + .setBuffer = mediaformat_jni_setBuffer, +}; + +static const FFAMediaCodec media_codec_jni = { + .class = &amediacodec_class, + + .getName = mediacodec_jni_getName, + + .createCodecByName = mediacodec_jni_createCodecByName, + .createDecoderByType = mediacodec_jni_createDecoderByType, + .createEncoderByType = mediacodec_jni_createEncoderByType, + .delete = mediacodec_jni_delete, + + .configure = mediacodec_jni_configure, + .start = mediacodec_jni_start, + .stop = mediacodec_jni_stop, + .flush = mediacodec_jni_flush, + + .getInputBuffer = mediacodec_jni_getInputBuffer, + .getOutputBuffer = mediacodec_jni_getOutputBuffer, + + .dequeueInputBuffer = mediacodec_jni_dequeueInputBuffer, + .queueInputBuffer = mediacodec_jni_queueInputBuffer, + + .dequeueOutputBuffer = mediacodec_jni_dequeueOutputBuffer, + .getOutputFormat = mediacodec_jni_getOutputFormat, + + .releaseOutputBuffer = mediacodec_jni_releaseOutputBuffer, + .releaseOutputBufferAtTime = mediacodec_jni_releaseOutputBufferAtTime, + + .infoTryAgainLater = mediacodec_jni_infoTryAgainLater, + .infoOutputBuffersChanged = mediacodec_jni_infoOutputBuffersChanged, + .infoOutputFormatChanged = mediacodec_jni_infoOutputFormatChanged, + + .getBufferFlagCodecConfig = mediacodec_jni_getBufferFlagCodecConfig, + .getBufferFlagEndOfStream = mediacodec_jni_getBufferFlagEndOfStream, + .getBufferFlagKeyFrame = mediacodec_jni_getBufferFlagKeyFrame, + + .getConfigureFlagEncode = mediacodec_jni_getConfigureFlagEncode, + .cleanOutputBuffers = mediacodec_jni_cleanOutputBuffers, +}; + +FFAMediaFormat *ff_AMediaFormat_new(void) +{ + return media_format_jni.create(); +} + +FFAMediaCodec* ff_AMediaCodec_createCodecByName(const char *name) +{ + return media_codec_jni.createCodecByName(name); +} + +FFAMediaCodec* ff_AMediaCodec_createDecoderByType(const char *mime_type) +{ + return media_codec_jni.createDecoderByType(mime_type); +} + +FFAMediaCodec* ff_AMediaCodec_createEncoderByType(const char *mime_type) +{ + return media_codec_jni.createEncoderByType(mime_type); +} + int ff_Build_SDK_INT(AVCodecContext *avctx) { int ret = -1; diff --git a/libavcodec/mediacodec_wrapper.h b/libavcodec/mediacodec_wrapper.h index b106ff315a..606fdbede5 100644 --- a/libavcodec/mediacodec_wrapper.h +++ b/libavcodec/mediacodec_wrapper.h @@ -58,28 +58,90 @@ int ff_AMediaCodecProfile_getProfileFromAVCodecContext(AVCodecContext *avctx); char *ff_AMediaCodecList_getCodecNameByType(const char *mime, int profile, int encoder, void *log_ctx); -struct FFAMediaFormat; typedef struct FFAMediaFormat FFAMediaFormat; +struct FFAMediaFormat { + const AVClass *class; + + FFAMediaFormat *(*create)(void); + int (*delete)(FFAMediaFormat *); + + char* (*toString)(FFAMediaFormat* format); + + int (*getInt32)(FFAMediaFormat* format, const char *name, int32_t *out); + int (*getInt64)(FFAMediaFormat* format, const char *name, int64_t *out); + int (*getFloat)(FFAMediaFormat* format, const char *name, float *out); + int (*getBuffer)(FFAMediaFormat* format, const char *name, void** data, size_t *size); + int (*getString)(FFAMediaFormat* format, const char *name, const char **out); + + void (*setInt32)(FFAMediaFormat* format, const char* name, int32_t value); + void (*setInt64)(FFAMediaFormat* format, const char* name, int64_t value); + void (*setFloat)(FFAMediaFormat* format, const char* name, float value); + void (*setString)(FFAMediaFormat* format, const char* name, const char* value); + void (*setBuffer)(FFAMediaFormat* format, const char* name, void* data, size_t size); +}; FFAMediaFormat *ff_AMediaFormat_new(void); -int ff_AMediaFormat_delete(FFAMediaFormat* format); -char* ff_AMediaFormat_toString(FFAMediaFormat* format); +static inline int ff_AMediaFormat_delete(FFAMediaFormat* format) +{ + return format->delete(format); +} -int ff_AMediaFormat_getInt32(FFAMediaFormat* format, const char *name, int32_t *out); -int ff_AMediaFormat_getInt64(FFAMediaFormat* format, const char *name, int64_t *out); -int ff_AMediaFormat_getFloat(FFAMediaFormat* format, const char *name, float *out); -int ff_AMediaFormat_getBuffer(FFAMediaFormat* format, const char *name, void** data, size_t *size); -int ff_AMediaFormat_getString(FFAMediaFormat* format, const char *name, const char **out); +static inline char* ff_AMediaFormat_toString(FFAMediaFormat* format) +{ + return format->toString(format); +} -void ff_AMediaFormat_setInt32(FFAMediaFormat* format, const char* name, int32_t value); -void ff_AMediaFormat_setInt64(FFAMediaFormat* format, const char* name, int64_t value); -void ff_AMediaFormat_setFloat(FFAMediaFormat* format, const char* name, float value); -void ff_AMediaFormat_setString(FFAMediaFormat* format, const char* name, const char* value); -void ff_AMediaFormat_setBuffer(FFAMediaFormat* format, const char* name, void* data, size_t size); +static inline int ff_AMediaFormat_getInt32(FFAMediaFormat* format, const char *name, int32_t *out) +{ + return format->getInt32(format, name, out); +} + +static inline int ff_AMediaFormat_getInt64(FFAMediaFormat* format, const char *name, int64_t *out) +{ + return format->getInt64(format, name, out); +} + +static inline int ff_AMediaFormat_getFloat(FFAMediaFormat* format, const char *name, float *out) +{ + return format->getFloat(format, name, out); +} + +static inline int ff_AMediaFormat_getBuffer(FFAMediaFormat* format, const char *name, void** data, size_t *size) +{ + return format->getBuffer(format, name, data, size); +} + +static inline int ff_AMediaFormat_getString(FFAMediaFormat* format, const char *name, const char **out) +{ + return format->getString(format, name, out); +} + +static inline void ff_AMediaFormat_setInt32(FFAMediaFormat* format, const char* name, int32_t value) +{ + format->setInt32(format, name, value); +} + +static inline void ff_AMediaFormat_setInt64(FFAMediaFormat* format, const char* name, int64_t value) +{ + format->setInt64(format, name, value); +} + +static inline void ff_AMediaFormat_setFloat(FFAMediaFormat* format, const char* name, float value) +{ + format->setFloat(format, name, value); +} + +static inline void ff_AMediaFormat_setString(FFAMediaFormat* format, const char* name, const char* value) +{ + format->setString(format, name, value); +} + +static inline void ff_AMediaFormat_setBuffer(FFAMediaFormat* format, const char* name, void* data, size_t size) +{ + format->setBuffer(format, name, data, size); +} -struct FFAMediaCodec; -typedef struct FFAMediaCodec FFAMediaCodec; typedef struct FFAMediaCodecCryptoInfo FFAMediaCodecCryptoInfo; struct FFAMediaCodecBufferInfo { @@ -90,41 +152,160 @@ struct FFAMediaCodecBufferInfo { }; typedef struct FFAMediaCodecBufferInfo FFAMediaCodecBufferInfo; -char *ff_AMediaCodec_getName(FFAMediaCodec *codec); +typedef struct FFAMediaCodec FFAMediaCodec; +struct FFAMediaCodec { + const AVClass *class; + + char *(*getName)(FFAMediaCodec *codec); + + FFAMediaCodec* (*createCodecByName)(const char *name); + FFAMediaCodec* (*createDecoderByType)(const char *mime_type); + FFAMediaCodec* (*createEncoderByType)(const char *mime_type); + int (*delete)(FFAMediaCodec* codec); + + int (*configure)(FFAMediaCodec* codec, const FFAMediaFormat* format, void* surface, void *crypto, uint32_t flags); + int (*start)(FFAMediaCodec* codec); + int (*stop)(FFAMediaCodec* codec); + int (*flush)(FFAMediaCodec* codec); + + uint8_t* (*getInputBuffer)(FFAMediaCodec* codec, size_t idx, size_t *out_size); + uint8_t* (*getOutputBuffer)(FFAMediaCodec* codec, size_t idx, size_t *out_size); + + ssize_t (*dequeueInputBuffer)(FFAMediaCodec* codec, int64_t timeoutUs); + int (*queueInputBuffer)(FFAMediaCodec* codec, size_t idx, off_t offset, size_t size, uint64_t time, uint32_t flags); + + ssize_t (*dequeueOutputBuffer)(FFAMediaCodec* codec, FFAMediaCodecBufferInfo *info, int64_t timeoutUs); + FFAMediaFormat* (*getOutputFormat)(FFAMediaCodec* codec); + + int (*releaseOutputBuffer)(FFAMediaCodec* codec, size_t idx, int render); + int (*releaseOutputBufferAtTime)(FFAMediaCodec *codec, size_t idx, int64_t timestampNs); + + int (*infoTryAgainLater)(FFAMediaCodec *codec, ssize_t idx); + int (*infoOutputBuffersChanged)(FFAMediaCodec *codec, ssize_t idx); + int (*infoOutputFormatChanged)(FFAMediaCodec *codec, ssize_t indx); + + int (*getBufferFlagCodecConfig)(FFAMediaCodec *codec); + int (*getBufferFlagEndOfStream)(FFAMediaCodec *codec); + int (*getBufferFlagKeyFrame)(FFAMediaCodec *codec); + + int (*getConfigureFlagEncode)(FFAMediaCodec *codec); + + int (*cleanOutputBuffers)(FFAMediaCodec *codec); +}; + +static inline char *ff_AMediaCodec_getName(FFAMediaCodec *codec) +{ + return codec->getName(codec); +} FFAMediaCodec* ff_AMediaCodec_createCodecByName(const char *name); FFAMediaCodec* ff_AMediaCodec_createDecoderByType(const char *mime_type); FFAMediaCodec* ff_AMediaCodec_createEncoderByType(const char *mime_type); -int ff_AMediaCodec_configure(FFAMediaCodec* codec, const FFAMediaFormat* format, void* surface, void *crypto, uint32_t flags); -int ff_AMediaCodec_start(FFAMediaCodec* codec); -int ff_AMediaCodec_stop(FFAMediaCodec* codec); -int ff_AMediaCodec_flush(FFAMediaCodec* codec); -int ff_AMediaCodec_delete(FFAMediaCodec* codec); +static inline int ff_AMediaCodec_configure(FFAMediaCodec* codec, const FFAMediaFormat* format, void* surface, void *crypto, uint32_t flags) +{ + return codec->configure(codec, format, surface, crypto, flags); +} + +static inline int ff_AMediaCodec_start(FFAMediaCodec* codec) +{ + return codec->start(codec); +} + +static inline int ff_AMediaCodec_stop(FFAMediaCodec* codec) +{ + return codec->stop(codec); +} + +static inline int ff_AMediaCodec_flush(FFAMediaCodec* codec) +{ + return codec->flush(codec); +} + +static inline int ff_AMediaCodec_delete(FFAMediaCodec* codec) +{ + return codec->delete(codec); +} + +static inline uint8_t* ff_AMediaCodec_getInputBuffer(FFAMediaCodec* codec, size_t idx, size_t *out_size) +{ + return codec->getInputBuffer(codec, idx, out_size); +} + +static inline uint8_t* ff_AMediaCodec_getOutputBuffer(FFAMediaCodec* codec, size_t idx, size_t *out_size) +{ + return codec->getOutputBuffer(codec, idx, out_size); +} + +static inline ssize_t ff_AMediaCodec_dequeueInputBuffer(FFAMediaCodec* codec, int64_t timeoutUs) +{ + return codec->dequeueInputBuffer(codec, timeoutUs); +} + +static inline int ff_AMediaCodec_queueInputBuffer(FFAMediaCodec *codec, size_t idx, off_t offset, size_t size, uint64_t time, uint32_t flags) +{ + return codec->queueInputBuffer(codec, idx, offset, size, time, flags); +} + +static inline ssize_t ff_AMediaCodec_dequeueOutputBuffer(FFAMediaCodec* codec, FFAMediaCodecBufferInfo *info, int64_t timeoutUs) +{ + return codec->dequeueOutputBuffer(codec, info, timeoutUs); +} + +static inline FFAMediaFormat* ff_AMediaCodec_getOutputFormat(FFAMediaCodec* codec) +{ + return codec->getOutputFormat(codec); +} + +static inline int ff_AMediaCodec_releaseOutputBuffer(FFAMediaCodec* codec, size_t idx, int render) +{ + return codec->releaseOutputBuffer(codec, idx, render); +} + +static inline int ff_AMediaCodec_releaseOutputBufferAtTime(FFAMediaCodec *codec, size_t idx, int64_t timestampNs) +{ + return codec->releaseOutputBufferAtTime(codec, idx, timestampNs); +} -uint8_t* ff_AMediaCodec_getInputBuffer(FFAMediaCodec* codec, size_t idx, size_t *out_size); -uint8_t* ff_AMediaCodec_getOutputBuffer(FFAMediaCodec* codec, size_t idx, size_t *out_size); +static inline int ff_AMediaCodec_infoTryAgainLater(FFAMediaCodec *codec, ssize_t idx) +{ + return codec->infoTryAgainLater(codec, idx); +} -ssize_t ff_AMediaCodec_dequeueInputBuffer(FFAMediaCodec* codec, int64_t timeoutUs); -int ff_AMediaCodec_queueInputBuffer(FFAMediaCodec* codec, size_t idx, off_t offset, size_t size, uint64_t time, uint32_t flags); +static inline int ff_AMediaCodec_infoOutputBuffersChanged(FFAMediaCodec *codec, ssize_t idx) +{ + return codec->infoOutputBuffersChanged(codec, idx); +} -ssize_t ff_AMediaCodec_dequeueOutputBuffer(FFAMediaCodec* codec, FFAMediaCodecBufferInfo *info, int64_t timeoutUs); -FFAMediaFormat* ff_AMediaCodec_getOutputFormat(FFAMediaCodec* codec); +static inline int ff_AMediaCodec_infoOutputFormatChanged(FFAMediaCodec *codec, ssize_t idx) +{ + return codec->infoOutputFormatChanged(codec, idx); +} -int ff_AMediaCodec_releaseOutputBuffer(FFAMediaCodec* codec, size_t idx, int render); -int ff_AMediaCodec_releaseOutputBufferAtTime(FFAMediaCodec *codec, size_t idx, int64_t timestampNs); +static inline int ff_AMediaCodec_getBufferFlagCodecConfig(FFAMediaCodec *codec) +{ + return codec->getBufferFlagCodecConfig(codec); +} -int ff_AMediaCodec_infoTryAgainLater(FFAMediaCodec *codec, ssize_t idx); -int ff_AMediaCodec_infoOutputBuffersChanged(FFAMediaCodec *codec, ssize_t idx); -int ff_AMediaCodec_infoOutputFormatChanged(FFAMediaCodec *codec, ssize_t indx); +static inline int ff_AMediaCodec_getBufferFlagEndOfStream(FFAMediaCodec *codec) +{ + return codec->getBufferFlagEndOfStream(codec); +} -int ff_AMediaCodec_getBufferFlagCodecConfig (FFAMediaCodec *codec); -int ff_AMediaCodec_getBufferFlagEndOfStream(FFAMediaCodec *codec); -int ff_AMediaCodec_getBufferFlagKeyFrame(FFAMediaCodec *codec); +static inline int ff_AMediaCodec_getBufferFlagKeyFrame(FFAMediaCodec *codec) +{ + return codec->getBufferFlagKeyFrame(codec); +} -int ff_AMediaCodec_getConfigureFlagEncode(FFAMediaCodec *codec); +static inline int ff_AMediaCodec_getConfigureFlagEncode(FFAMediaCodec *codec) +{ + return codec->getConfigureFlagEncode(codec); +} -int ff_AMediaCodec_cleanOutputBuffers(FFAMediaCodec *codec); +static inline int ff_AMediaCodec_cleanOutputBuffers(FFAMediaCodec *codec) +{ + return codec->cleanOutputBuffers(codec); +} int ff_Build_SDK_INT(AVCodecContext *avctx); From patchwork Sun Nov 20 06:49:41 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhao Zhili X-Patchwork-Id: 39353 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a21:999a:b0:a4:2148:650a with SMTP id ve26csp1406783pzb; Sat, 19 Nov 2022 22:51:49 -0800 (PST) X-Google-Smtp-Source: AA0mqf4PtPldCodGKK4Bjg90EfZ/rAQxvORe8yu9VQ8YSRIl5PlYDR+yWPs4hsCcrYIu5JMtlZB7 X-Received: by 2002:a05:6402:4445:b0:468:eef6:a83c with SMTP id o5-20020a056402444500b00468eef6a83cmr11108843edb.191.1668927108996; Sat, 19 Nov 2022 22:51:48 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1668927108; cv=none; d=google.com; s=arc-20160816; b=OLRvAExjr/3FO9sJkIJS1Cltucz84CVXzk9XgWM5oGdUDzdkjgT/YzR1bczfHWiZD4 rRABCAE8klgDAPobt+6QtusN2313XHrKkxamu6Xt+cMbx1kAsHt7kvZXkkXZT+NUvMVc q/7QdquEl4tnN2NdvSL681aRNtwiaFvSzdwg1QPlekkOqbL3xrKpQiwUsZ3FnMXTZHXw qj1J6Yn93YOLoB5vF4E9/0E5ANaVZvViApur2Ez9FspilnvrwEOM3MUExcxW7+WepIwK 9bbMc7Kw5HWoSn1DgbIPL09gCDUeqwJA2IyLvwV3lo/9ZDmZ2hP2MCmJus6icYw1lyer OnnA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to:date :to:from:message-id:dkim-signature:delivered-to; bh=Jws0LnOcMyfm5OulcxX26/WAlmbpo1Q2MarbcOeIVeQ=; b=tdyL/lscKDA9AgkTqP9IqlZ7oOHQl4CMGEu64GSK66jDOjR94iVyCMbLIAViEw/0iU MU32wbjPJypXDMsvscoHrBEVXCF0MepFGCp160SUinaohqa1uJbPW9xJ7VsR+k5PsjJt ZaupWBNU5d1IQXUQVAH0M7Otpiy3eO2U2mdEa6vuhNqFB93kBJpEE+bFXROmrtzh1cnx yhxn94btkugOS1foU4L5ev2JA1Gg2G88JRNC2zFvneSiMfkeCE6prWI9OLcMq3BJ9qFu caqk9jE9JvEQs1ULmGcFREjY5bnb9MVTighTJh0KHtfo3p5cy1g2I84IarjN8+7hmsDD t/DQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@foxmail.com header.s=s201512 header.b=fpZ2HJk0; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=foxmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id cw4-20020a170906478400b007919c624eadsi7167627ejc.522.2022.11.19.22.51.48; Sat, 19 Nov 2022 22:51:48 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@foxmail.com header.s=s201512 header.b=fpZ2HJk0; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=foxmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id B531668BC19; Sun, 20 Nov 2022 08:50:52 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from out162-62-58-216.mail.qq.com (out162-62-58-216.mail.qq.com [162.62.58.216]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id A281F68BB78 for ; Sun, 20 Nov 2022 08:50:39 +0200 (EET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=foxmail.com; s=s201512; t=1668927037; bh=hEe0oeGZyPb5bShEwMZCChMpUyi8mUElNvypz4HHgtQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=fpZ2HJk0XeLeZeuK7VGHXptMomVMFG4YiRTKkcnJ/gmmGAXP/aXDRO8HrF9dNEsb7 O+xrnZn7ytsib8ZQy6bC/zNbiuyfidub3vf4mP1qCIykd5kJGbueipmVHIueSj22Cl XliZDVGyesN8RH2klZfEOGLdm/EPwJI+XCcGtQOo= Received: from localhost.localdomain ([14.153.77.200]) by newxmesmtplogicsvrszc2-1.qq.com (NewEsmtp) with SMTP id C8F0626E; Sun, 20 Nov 2022 14:50:15 +0800 X-QQ-mid: xmsmtpt1668927036t3ghjpr8q Message-ID: X-QQ-XMAILINFO: MNQ2FIpgVwZbXGHTJssymm8F/pofG6eUWVxDP5NBwMEbGYfUGT+eD+AobvTxZh 0yTPp8hLCW73iA1sjfyII0Lxc7SvCxP3VVm2vL79qgApM8uFBM8ufOGmJBfWkYmBFAl8Dz31fgsH X9wo0IWuzr/vr69QTdEHPDK+YsMSyB2LHNHFdmMUlfVyKzazSK0YVglxTr4KB82Xv1ZwFiUlA38D u2jZLfsGhzyOeCfhPbejtLRYz1Pz6eidDK0c8GvT4OZ2c0HfbLc35R7CsfW/gvc+Lhj+5MhjD01v 7vZV9lJ1REUo71Xp5cakS+6XeBbSbCFlMX9G796N+luf6kUf+95whZrqet0KFSyvt5WkoQ5ARIcL BzhOYE7rNjRYW0RKsqb8VcKQpSRafi/tMh8rlgWP41i7IYlodHuZ3IzZ5hPYvmdgfmGHPFpjTjTB EIcMtPmOfFmytg59yr6J6rc0DQQuUOErTOMd5HZQGjquqEavXggcIKUCWjGnhNQ8BBbEyljJc4fp 4QgDJ1l+vQ4PoTW/WWwfA+IuIAcfoY4m4q3V45wWA7ZQzyvcVN2QtmnyW1+3ptyXrDfovLW7OwHy 1uo7SC4rR+CpQYvaJDT82s7IKZ+iZkwfMyHjrc0uDuJ/8c3VCqJOsPGxI0rLUZ8hc7sULyhHbj1d Te8CoT63wltDWbVhiv/6VjNfm3NSgQYBblcB9RnmM74N2wFrfSwGCLtGi5lGxMpU8l5ttoNN68pl zjO20RB1zK9REtrAh14PVVs6+77x78TJHpU9fYjhY1EGB7IloKs78lTd4jvfarzoV7EDzOYWoZk2 shLQVMZgRb/qmGJjGhf9yyS7NayzKBknPELkP6m2Feg3Ty1kpPM0pEMAPIOehB8iGB8hqops+t03 R47dkUuz7AMmxQPfX9ktqGMHe2WrcEDpEkh5VKW3ly6Di4v5eTeJocn7qZ0fF0PIlD8UYtiqX+X4 kqkFE7hdG5o2AJBRFVNc/lrP/r/F3lgsHMgY6h6fYMoVWUK/EjcFxhGCJmaxCkMm2YY+0iJgk= From: Zhao Zhili To: ffmpeg-devel@ffmpeg.org Date: Sun, 20 Nov 2022 14:49:41 +0800 X-OQ-MSGID: <20221120064947.94540-7-quinkblack@foxmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221120064947.94540-1-quinkblack@foxmail.com> References: <20221120064947.94540-1-quinkblack@foxmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 06/12] avcodec/mediacodec: add NDK media codec wrapper X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: matthieu.bouron@stupeflix.com, Zhao Zhili , aman@tmm1.net Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: MgbD9ICovLnR From: Zhao Zhili Signed-off-by: Zhao Zhili --- configure | 2 + libavcodec/mediacodec_wrapper.c | 596 +++++++++++++++++++++++++++++- libavcodec/mediacodec_wrapper.h | 8 +- libavcodec/mediacodecdec.c | 2 +- libavcodec/mediacodecdec_common.c | 2 +- 5 files changed, 600 insertions(+), 10 deletions(-) diff --git a/configure b/configure index e54649fa48..9da5a36072 100755 --- a/configure +++ b/configure @@ -3181,6 +3181,7 @@ h264_crystalhd_decoder_select="crystalhd h264_mp4toannexb_bsf h264_parser" h264_cuvid_decoder_deps="cuvid" h264_cuvid_decoder_select="h264_mp4toannexb_bsf" h264_mediacodec_decoder_deps="mediacodec" +h264_mediacodec_decoder_extralibs="-landroid" h264_mediacodec_decoder_select="h264_mp4toannexb_bsf h264_parser" h264_mf_encoder_deps="mediafoundation" h264_mmal_decoder_deps="mmal" @@ -3199,6 +3200,7 @@ hevc_amf_encoder_deps="amf" hevc_cuvid_decoder_deps="cuvid" hevc_cuvid_decoder_select="hevc_mp4toannexb_bsf" hevc_mediacodec_decoder_deps="mediacodec" +hevc_mediacodec_decoder_extralibs="-landroid" hevc_mediacodec_decoder_select="hevc_mp4toannexb_bsf hevc_parser" hevc_mf_encoder_deps="mediafoundation" hevc_nvenc_encoder_deps="nvenc" diff --git a/libavcodec/mediacodec_wrapper.c b/libavcodec/mediacodec_wrapper.c index e0c614680e..b12aced711 100644 --- a/libavcodec/mediacodec_wrapper.c +++ b/libavcodec/mediacodec_wrapper.c @@ -20,7 +20,11 @@ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA */ +#include #include +#include +#include +#include #include "libavutil/avassert.h" #include "libavutil/mem.h" @@ -1794,23 +1798,607 @@ static const FFAMediaCodec media_codec_jni = { .cleanOutputBuffers = mediacodec_jni_cleanOutputBuffers, }; -FFAMediaFormat *ff_AMediaFormat_new(void) +typedef struct FFAMediaFormatNdk { + FFAMediaFormat api; + + void *libmedia; + AMediaFormat *impl; + + AMediaFormat *(*new)(void); + media_status_t (*delete)(AMediaFormat*); + + const char* (*toString)(AMediaFormat*); + + bool (*getInt32)(AMediaFormat*, const char *name, int32_t *out); + bool (*getInt64)(AMediaFormat*, const char *name, int64_t *out); + bool (*getFloat)(AMediaFormat*, const char *name, float *out); + bool (*getSize)(AMediaFormat*, const char *name, size_t *out); + bool (*getBuffer)(AMediaFormat*, const char *name, void** data, size_t *size); + bool (*getString)(AMediaFormat*, const char *name, const char **out); + + void (*setInt32)(AMediaFormat*, const char* name, int32_t value); + void (*setInt64)(AMediaFormat*, const char* name, int64_t value); + void (*setFloat)(AMediaFormat*, const char* name, float value); + void (*setString)(AMediaFormat*, const char* name, const char* value); + void (*setBuffer)(AMediaFormat*, const char* name, const void* data, size_t size); +} FFAMediaFormatNdk; + +typedef struct FFAMediaCodecNdk { + FFAMediaCodec api; + + void *libmedia; + AMediaCodec *impl; + ANativeWindow *window; + + AMediaCodec* (*createCodecByName)(const char *name); + AMediaCodec* (*createDecoderByType)(const char *mime_type); + AMediaCodec* (*createEncoderByType)(const char *mime_type); + media_status_t (*delete)(AMediaCodec*); + + media_status_t (*configure)(AMediaCodec *, + const AMediaFormat *format, + ANativeWindow *surface, + AMediaCrypto *crypto, + uint32_t flags); + media_status_t (*start)(AMediaCodec*); + media_status_t (*stop)(AMediaCodec*); + media_status_t (*flush)(AMediaCodec*); + + uint8_t* (*getInputBuffer)(AMediaCodec*, size_t idx, size_t *out_size); + uint8_t* (*getOutputBuffer)(AMediaCodec*, size_t idx, size_t *out_size); + + ssize_t (*dequeueInputBuffer)(AMediaCodec*, int64_t timeoutUs); + media_status_t (*queueInputBuffer)(AMediaCodec*, size_t idx, + long offset, size_t size, + uint64_t time, uint32_t flags); + + ssize_t (*dequeueOutputBuffer)(AMediaCodec*, AMediaCodecBufferInfo *info, int64_t timeoutUs); + AMediaFormat* (*getOutputFormat)(AMediaCodec*); + + media_status_t (*releaseOutputBuffer)(AMediaCodec*, size_t idx, bool render); + media_status_t (*releaseOutputBufferAtTime)(AMediaCodec *mData, size_t idx, int64_t timestampNs); + + // Available since API level 28. + media_status_t (*getName)(AMediaCodec*, char** out_name); + void (*releaseName)(AMediaCodec*, char* name); +} FFAMediaCodecNdk; + +static const FFAMediaFormat media_format_ndk; +static const FFAMediaCodec media_codec_ndk; + +static const AVClass amediaformat_ndk_class = { + .class_name = "amediaformat_ndk", + .item_name = av_default_item_name, + .version = LIBAVUTIL_VERSION_INT, +}; + +static const AVClass amediacodec_ndk_class = { + .class_name = "amediacodec_ndk", + .item_name = av_default_item_name, + .version = LIBAVUTIL_VERSION_INT, +}; + +static FFAMediaFormat *mediaformat_ndk_create(AMediaFormat *impl) +{ + FFAMediaFormatNdk *format = av_mallocz(sizeof(*format)); + if (!format) + return NULL; + + format->api = media_format_ndk; + + format->libmedia = dlopen("libmediandk.so", RTLD_NOW); + if (!format->libmedia) + goto error; + +#define GET_SYMBOL(sym) \ + format->sym = dlsym(format->libmedia, "AMediaFormat_" #sym); \ + if (!format->sym) \ + goto error; + + GET_SYMBOL(new) + GET_SYMBOL(delete) + + GET_SYMBOL(toString) + + GET_SYMBOL(getInt32) + GET_SYMBOL(getInt64) + GET_SYMBOL(getFloat) + GET_SYMBOL(getSize) + GET_SYMBOL(getBuffer) + GET_SYMBOL(getString) + + GET_SYMBOL(setInt32) + GET_SYMBOL(setInt64) + GET_SYMBOL(setFloat) + GET_SYMBOL(setString) + GET_SYMBOL(setBuffer) + +#undef GET_SYMBOL + + if (impl) { + format->impl = impl; + } else { + format->impl = format->new(); + if (!format->impl) + goto error; + } + + return (FFAMediaFormat *)format; + +error: + if (format->libmedia) + dlclose(format->libmedia); + av_freep(&format); + return NULL; +} + +static FFAMediaFormat *mediaformat_ndk_new(void) +{ + return mediaformat_ndk_create(NULL); +} + +static int mediaformat_ndk_delete(FFAMediaFormat* ctx) +{ + FFAMediaFormatNdk *format = (FFAMediaFormatNdk *)ctx; + int ret = 0; + if (!format) + return 0; + + av_assert0(format->api.class == &amediaformat_ndk_class); + + if (format->impl && (format->delete(format->impl) != AMEDIA_OK)) + ret = AVERROR_EXTERNAL; + if (format->libmedia) + dlclose(format->libmedia); + av_free(format); + + return ret; +} + +static char* mediaformat_ndk_toString(FFAMediaFormat* ctx) +{ + FFAMediaFormatNdk *format = (FFAMediaFormatNdk *)ctx; + const char *str = format->toString(format->impl); + return av_strdup(str); +} + +static int mediaformat_ndk_getInt32(FFAMediaFormat* ctx, const char *name, int32_t *out) +{ + FFAMediaFormatNdk *format = (FFAMediaFormatNdk *)ctx; + return format->getInt32(format->impl, name, out); +} + +static int mediaformat_ndk_getInt64(FFAMediaFormat* ctx, const char *name, int64_t *out) +{ + FFAMediaFormatNdk *format = (FFAMediaFormatNdk *)ctx; + return format->getInt64(format->impl, name, out); +} + +static int mediaformat_ndk_getFloat(FFAMediaFormat* ctx, const char *name, float *out) +{ + FFAMediaFormatNdk *format = (FFAMediaFormatNdk *)ctx; + return format->getFloat(format->impl, name, out); +} + +static int mediaformat_ndk_getBuffer(FFAMediaFormat* ctx, const char *name, void** data, size_t *size) +{ + FFAMediaFormatNdk *format = (FFAMediaFormatNdk *)ctx; + return format->getBuffer(format->impl, name, data, size); +} + +static int mediaformat_ndk_getString(FFAMediaFormat* ctx, const char *name, const char **out) +{ + FFAMediaFormatNdk *format = (FFAMediaFormatNdk *)ctx; + const char *tmp = NULL; + int ret = format->getString(format->impl, name, &tmp); + + if (tmp) + *out = av_strdup(tmp); + return ret; +} + +static void mediaformat_ndk_setInt32(FFAMediaFormat* ctx, const char* name, int32_t value) +{ + FFAMediaFormatNdk *format = (FFAMediaFormatNdk *)ctx; + format->setInt32(format->impl, name, value); +} + +static void mediaformat_ndk_setInt64(FFAMediaFormat* ctx, const char* name, int64_t value) +{ + FFAMediaFormatNdk *format = (FFAMediaFormatNdk *)ctx; + format->setInt64(format->impl, name, value); +} + +static void mediaformat_ndk_setFloat(FFAMediaFormat* ctx, const char* name, float value) +{ + FFAMediaFormatNdk *format = (FFAMediaFormatNdk *)ctx; + format->setFloat(format->impl, name, value); +} + +static void mediaformat_ndk_setString(FFAMediaFormat* ctx, const char* name, const char* value) +{ + FFAMediaFormatNdk *format = (FFAMediaFormatNdk *)ctx; + format->setString(format->impl, name, value); +} + +static void mediaformat_ndk_setBuffer(FFAMediaFormat* ctx, const char* name, void* data, size_t size) +{ + FFAMediaFormatNdk *format = (FFAMediaFormatNdk *)ctx; + format->setBuffer(format->impl, name, data, size); +} + +static char *mediacodec_ndk_getName(FFAMediaCodec *ctx) +{ + FFAMediaCodecNdk *codec = (FFAMediaCodecNdk *)ctx; + char *ret = NULL; + char *name = NULL; + + if (!codec->getName || !codec->releaseName) { + av_log(ctx, AV_LOG_DEBUG, "getName() unavailable\n"); + return ret; + } + + codec->getName(codec->impl, &name); + if (name) { + ret = av_strdup(name); + codec->releaseName(codec->impl, name); + } + + return ret; +} + +static inline FFAMediaCodec *ndk_codec_create(int method, const char *arg) { + FFAMediaCodecNdk *codec = av_mallocz(sizeof(*codec)); + const char *lib_name = "libmediandk.so"; + + if (!codec) + return NULL; + + codec->api = media_codec_ndk; + codec->libmedia = dlopen(lib_name, RTLD_NOW); + if (!codec->libmedia) + goto error; + +#define GET_SYMBOL(sym, required) \ + codec->sym = dlsym(codec->libmedia, "AMediaCodec_" #sym); \ + if (!codec->sym) { \ + av_log(codec, required ? AV_LOG_ERROR : AV_LOG_INFO, \ + #sym "() unavailable from %s\n", lib_name); \ + if (required) \ + goto error; \ + } + + GET_SYMBOL(createCodecByName, 1) + GET_SYMBOL(createDecoderByType, 1) + GET_SYMBOL(createEncoderByType, 1) + GET_SYMBOL(delete, 1) + + GET_SYMBOL(configure, 1) + GET_SYMBOL(start, 1) + GET_SYMBOL(stop, 1) + GET_SYMBOL(flush, 1) + + GET_SYMBOL(getInputBuffer, 1) + GET_SYMBOL(getOutputBuffer, 1) + + GET_SYMBOL(dequeueInputBuffer, 1) + GET_SYMBOL(queueInputBuffer, 1) + + GET_SYMBOL(dequeueOutputBuffer, 1) + GET_SYMBOL(getOutputFormat, 1) + + GET_SYMBOL(releaseOutputBuffer, 1) + GET_SYMBOL(releaseOutputBufferAtTime, 1) + + GET_SYMBOL(getName, 0) + GET_SYMBOL(releaseName, 0) + +#undef GET_SYMBOL + + switch (method) { + case CREATE_CODEC_BY_NAME: + codec->impl = codec->createCodecByName(arg); + break; + case CREATE_DECODER_BY_TYPE: + codec->impl = codec->createDecoderByType(arg); + break; + case CREATE_ENCODER_BY_TYPE: + codec->impl = codec->createEncoderByType(arg); + break; + default: + av_assert0(0); + } + if (!codec->impl) + goto error; + + return (FFAMediaCodec *)codec; + +error: + if (codec->libmedia) + dlclose(codec->libmedia); + av_freep(&codec); + return NULL; +} + +#define DECLARE_NDK_AMEDIACODEC_CREATE_FUNC(name, method) \ +static FFAMediaCodec *mediacodec_ndk_##name(const char *arg) \ +{ \ + return ndk_codec_create(method, arg); \ +} \ + +DECLARE_NDK_AMEDIACODEC_CREATE_FUNC(createCodecByName, CREATE_CODEC_BY_NAME) +DECLARE_NDK_AMEDIACODEC_CREATE_FUNC(createDecoderByType, CREATE_DECODER_BY_TYPE) +DECLARE_NDK_AMEDIACODEC_CREATE_FUNC(createEncoderByType, CREATE_ENCODER_BY_TYPE) + +static int mediacodec_ndk_delete(FFAMediaCodec* ctx) +{ + FFAMediaCodecNdk *codec = (FFAMediaCodecNdk *)ctx; + int ret = 0; + + if (!codec) + return 0; + + av_assert0(codec->api.class == &amediacodec_ndk_class); + + if (codec->impl && (codec->delete(codec->impl) != AMEDIA_OK)) + ret = AVERROR_EXTERNAL; + if (codec->window) + ANativeWindow_release(codec->window); + if (codec->libmedia) + dlclose(codec->libmedia); + av_free(codec); + + return ret; +} + +static int mediacodec_ndk_configure(FFAMediaCodec* ctx, const FFAMediaFormat* format_ctx, void* surface, void *crypto, uint32_t flags) +{ + FFAMediaCodecNdk *codec = (FFAMediaCodecNdk *)ctx; + FFAMediaFormatNdk *format = (FFAMediaFormatNdk *)format_ctx; + media_status_t status; + + if (surface) { + JNIEnv *env = NULL; + JNI_GET_ENV_OR_RETURN(env, ctx, -1); + codec->window = ANativeWindow_fromSurface(env, surface); + } + + if (format_ctx->class != &amediaformat_ndk_class) { + av_log(ctx, AV_LOG_ERROR, "invalid media format\n"); + return AVERROR(EINVAL); + } + + status = codec->configure(codec->impl, format->impl, codec->window, NULL, flags); + if (status != AMEDIA_OK) { + av_log(codec, AV_LOG_ERROR, "configure failed, %d\n", status); + return AVERROR_EXTERNAL; + } + + return 0; +} + +#define MEDIACODEC_NDK_WRAPPER(method) \ +static int mediacodec_ndk_ ## method(FFAMediaCodec* ctx) \ +{ \ + FFAMediaCodecNdk *codec = (FFAMediaCodecNdk *)ctx; \ + media_status_t status = codec->method(codec->impl); \ + \ + if (status != AMEDIA_OK) { \ + av_log(codec, AV_LOG_ERROR, #method " failed, %d\n", status); \ + return AVERROR_EXTERNAL; \ + } \ + \ + return 0; \ +} \ + +MEDIACODEC_NDK_WRAPPER(start) +MEDIACODEC_NDK_WRAPPER(stop) +MEDIACODEC_NDK_WRAPPER(flush) + +static uint8_t* mediacodec_ndk_getInputBuffer(FFAMediaCodec* ctx, size_t idx, size_t *out_size) +{ + FFAMediaCodecNdk *codec = (FFAMediaCodecNdk *)ctx; + return codec->getInputBuffer(codec->impl, idx, out_size); +} + +static uint8_t* mediacodec_ndk_getOutputBuffer(FFAMediaCodec* ctx, size_t idx, size_t *out_size) +{ + FFAMediaCodecNdk *codec = (FFAMediaCodecNdk *)ctx; + return codec->getOutputBuffer(codec->impl, idx, out_size); +} + +static ssize_t mediacodec_ndk_dequeueInputBuffer(FFAMediaCodec* ctx, int64_t timeoutUs) +{ + FFAMediaCodecNdk *codec = (FFAMediaCodecNdk *)ctx; + return codec->dequeueInputBuffer(codec->impl, timeoutUs); +} + +static int mediacodec_ndk_queueInputBuffer(FFAMediaCodec *ctx, size_t idx, + off_t offset, size_t size, + uint64_t time, uint32_t flags) +{ + FFAMediaCodecNdk *codec = (FFAMediaCodecNdk *)ctx; + return codec->queueInputBuffer(codec->impl, idx, offset, size, time, flags); +} + +static ssize_t mediacodec_ndk_dequeueOutputBuffer(FFAMediaCodec* ctx, FFAMediaCodecBufferInfo *info, int64_t timeoutUs) +{ + FFAMediaCodecNdk *codec = (FFAMediaCodecNdk *)ctx; + AMediaCodecBufferInfo buf_info = {0}; + ssize_t ret; + + ret = codec->dequeueOutputBuffer(codec->impl, &buf_info, timeoutUs); + info->offset = buf_info.offset; + info->size = buf_info.size; + info->presentationTimeUs = buf_info.presentationTimeUs; + info->flags = buf_info.flags; + + return ret; +} + +static FFAMediaFormat* mediacodec_ndk_getOutputFormat(FFAMediaCodec* ctx) +{ + FFAMediaCodecNdk *codec = (FFAMediaCodecNdk *)ctx; + AMediaFormat *format = codec->getOutputFormat(codec->impl); + + if (!format) + return NULL; + return mediaformat_ndk_create(format); +} + +static int mediacodec_ndk_releaseOutputBuffer(FFAMediaCodec* ctx, size_t idx, int render) +{ + FFAMediaCodecNdk *codec = (FFAMediaCodecNdk *)ctx; + media_status_t status; + + status = codec->releaseOutputBuffer(codec->impl, idx, render); + if (status != AMEDIA_OK) { + av_log(codec, AV_LOG_ERROR, "release output buffer failed, %d\n", status); + return AVERROR_EXTERNAL; + } + + return 0; +} + +static int mediacodec_ndk_releaseOutputBufferAtTime(FFAMediaCodec *ctx, size_t idx, int64_t timestampNs) +{ + FFAMediaCodecNdk *codec = (FFAMediaCodecNdk *)ctx; + media_status_t status; + + status = codec->releaseOutputBufferAtTime(codec->impl, idx, timestampNs); + if (status != AMEDIA_OK) { + av_log(codec, AV_LOG_ERROR, "releaseOutputBufferAtTime failed, %d\n", status); + return AVERROR_EXTERNAL; + } + + return 0; +} + +static int mediacodec_ndk_infoTryAgainLater(FFAMediaCodec *ctx, ssize_t idx) +{ + return idx == AMEDIACODEC_INFO_TRY_AGAIN_LATER; +} + +static int mediacodec_ndk_infoOutputBuffersChanged(FFAMediaCodec *ctx, ssize_t idx) +{ + return idx == AMEDIACODEC_INFO_OUTPUT_BUFFERS_CHANGED; +} + +static int mediacodec_ndk_infoOutputFormatChanged(FFAMediaCodec *ctx, ssize_t idx) +{ + return idx == AMEDIACODEC_INFO_OUTPUT_FORMAT_CHANGED; +} + +static int mediacodec_ndk_getBufferFlagCodecConfig(FFAMediaCodec *ctx) +{ + return AMEDIACODEC_BUFFER_FLAG_CODEC_CONFIG; +} + +static int mediacodec_ndk_getBufferFlagEndOfStream(FFAMediaCodec *ctx) +{ + return AMEDIACODEC_BUFFER_FLAG_END_OF_STREAM; +} + +static int mediacodec_ndk_getBufferFlagKeyFrame(FFAMediaCodec *ctx) +{ + return 1; +} + +static int mediacodec_ndk_getConfigureFlagEncode(FFAMediaCodec *ctx) +{ + return AMEDIACODEC_CONFIGURE_FLAG_ENCODE; +} + +static int mediacodec_ndk_cleanOutputBuffers(FFAMediaCodec *ctx) +{ + return 0; +} + +static const FFAMediaFormat media_format_ndk = { + .class = &amediaformat_ndk_class, + + .create = mediaformat_ndk_new, + .delete = mediaformat_ndk_delete, + + .toString = mediaformat_ndk_toString, + + .getInt32 = mediaformat_ndk_getInt32, + .getInt64 = mediaformat_ndk_getInt64, + .getFloat = mediaformat_ndk_getFloat, + .getBuffer = mediaformat_ndk_getBuffer, + .getString = mediaformat_ndk_getString, + + .setInt32 = mediaformat_ndk_setInt32, + .setInt64 = mediaformat_ndk_setInt64, + .setFloat = mediaformat_ndk_setFloat, + .setString = mediaformat_ndk_setString, + .setBuffer = mediaformat_ndk_setBuffer, +}; + +static const FFAMediaCodec media_codec_ndk = { + .class = &amediacodec_ndk_class, + + .getName = mediacodec_ndk_getName, + + .createCodecByName = mediacodec_ndk_createCodecByName, + .createDecoderByType = mediacodec_ndk_createDecoderByType, + .createEncoderByType = mediacodec_ndk_createEncoderByType, + .delete = mediacodec_ndk_delete, + + .configure = mediacodec_ndk_configure, + .start = mediacodec_ndk_start, + .stop = mediacodec_ndk_stop, + .flush = mediacodec_ndk_flush, + + .getInputBuffer = mediacodec_ndk_getInputBuffer, + .getOutputBuffer = mediacodec_ndk_getOutputBuffer, + + .dequeueInputBuffer = mediacodec_ndk_dequeueInputBuffer, + .queueInputBuffer = mediacodec_ndk_queueInputBuffer, + + .dequeueOutputBuffer = mediacodec_ndk_dequeueOutputBuffer, + .getOutputFormat = mediacodec_ndk_getOutputFormat, + + .releaseOutputBuffer = mediacodec_ndk_releaseOutputBuffer, + .releaseOutputBufferAtTime = mediacodec_ndk_releaseOutputBufferAtTime, + + .infoTryAgainLater = mediacodec_ndk_infoTryAgainLater, + .infoOutputBuffersChanged = mediacodec_ndk_infoOutputBuffersChanged, + .infoOutputFormatChanged = mediacodec_ndk_infoOutputFormatChanged, + + .getBufferFlagCodecConfig = mediacodec_ndk_getBufferFlagCodecConfig, + .getBufferFlagEndOfStream = mediacodec_ndk_getBufferFlagEndOfStream, + .getBufferFlagKeyFrame = mediacodec_ndk_getBufferFlagKeyFrame, + + .getConfigureFlagEncode = mediacodec_ndk_getConfigureFlagEncode, + .cleanOutputBuffers = mediacodec_ndk_cleanOutputBuffers, +}; + +FFAMediaFormat *ff_AMediaFormat_new(int ndk) { + if (ndk) + return media_format_ndk.create(); return media_format_jni.create(); } -FFAMediaCodec* ff_AMediaCodec_createCodecByName(const char *name) +FFAMediaCodec* ff_AMediaCodec_createCodecByName(const char *name, int ndk) { + if (ndk) + return media_codec_ndk.createCodecByName(name); return media_codec_jni.createCodecByName(name); } -FFAMediaCodec* ff_AMediaCodec_createDecoderByType(const char *mime_type) +FFAMediaCodec* ff_AMediaCodec_createDecoderByType(const char *mime_type, int ndk) { + if (ndk) + return media_codec_ndk.createDecoderByType(mime_type); return media_codec_jni.createDecoderByType(mime_type); } -FFAMediaCodec* ff_AMediaCodec_createEncoderByType(const char *mime_type) +FFAMediaCodec* ff_AMediaCodec_createEncoderByType(const char *mime_type, int ndk) { + if (ndk) + return media_codec_ndk.createEncoderByType(mime_type); return media_codec_jni.createEncoderByType(mime_type); } diff --git a/libavcodec/mediacodec_wrapper.h b/libavcodec/mediacodec_wrapper.h index 606fdbede5..b9b882f243 100644 --- a/libavcodec/mediacodec_wrapper.h +++ b/libavcodec/mediacodec_wrapper.h @@ -80,7 +80,7 @@ struct FFAMediaFormat { void (*setBuffer)(FFAMediaFormat* format, const char* name, void* data, size_t size); }; -FFAMediaFormat *ff_AMediaFormat_new(void); +FFAMediaFormat *ff_AMediaFormat_new(int ndk); static inline int ff_AMediaFormat_delete(FFAMediaFormat* format) { @@ -198,9 +198,9 @@ static inline char *ff_AMediaCodec_getName(FFAMediaCodec *codec) return codec->getName(codec); } -FFAMediaCodec* ff_AMediaCodec_createCodecByName(const char *name); -FFAMediaCodec* ff_AMediaCodec_createDecoderByType(const char *mime_type); -FFAMediaCodec* ff_AMediaCodec_createEncoderByType(const char *mime_type); +FFAMediaCodec* ff_AMediaCodec_createCodecByName(const char *name, int ndk); +FFAMediaCodec* ff_AMediaCodec_createDecoderByType(const char *mime_type, int ndk); +FFAMediaCodec* ff_AMediaCodec_createEncoderByType(const char *mime_type, int ndk); static inline int ff_AMediaCodec_configure(FFAMediaCodec* codec, const FFAMediaFormat* format, void* surface, void *crypto, uint32_t flags) { diff --git a/libavcodec/mediacodecdec.c b/libavcodec/mediacodecdec.c index 2c66f38541..2e07548b77 100644 --- a/libavcodec/mediacodecdec.c +++ b/libavcodec/mediacodecdec.c @@ -310,7 +310,7 @@ static av_cold int mediacodec_decode_init(AVCodecContext *avctx) FFAMediaFormat *format = NULL; MediaCodecH264DecContext *s = avctx->priv_data; - format = ff_AMediaFormat_new(); + format = ff_AMediaFormat_new(0); if (!format) { av_log(avctx, AV_LOG_ERROR, "Failed to create media format\n"); ret = AVERROR_EXTERNAL; diff --git a/libavcodec/mediacodecdec_common.c b/libavcodec/mediacodecdec_common.c index 69a462ec48..4a9e50b0df 100644 --- a/libavcodec/mediacodecdec_common.c +++ b/libavcodec/mediacodecdec_common.c @@ -613,7 +613,7 @@ int ff_mediacodec_dec_init(AVCodecContext *avctx, MediaCodecDecContext *s, } av_log(avctx, AV_LOG_DEBUG, "Found decoder %s\n", s->codec_name); - s->codec = ff_AMediaCodec_createCodecByName(s->codec_name); + s->codec = ff_AMediaCodec_createCodecByName(s->codec_name, 0); if (!s->codec) { av_log(avctx, AV_LOG_ERROR, "Failed to create media decoder for type %s and name %s\n", mime, s->codec_name); ret = AVERROR_EXTERNAL; From patchwork Sun Nov 20 06:49:45 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhao Zhili X-Patchwork-Id: 39350 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a21:999a:b0:a4:2148:650a with SMTP id ve26csp1406650pzb; Sat, 19 Nov 2022 22:51:21 -0800 (PST) X-Google-Smtp-Source: AA0mqf6o8X1DRJSCVvqF3wgZCsTTWzpzl2XKUx2uxLpdmc4d6gbijBiIBAmq80o0CM9NbyjWoo2s X-Received: by 2002:a17:906:2552:b0:7ad:917b:61ec with SMTP id j18-20020a170906255200b007ad917b61ecmr3053333ejb.513.1668927081354; Sat, 19 Nov 2022 22:51:21 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1668927081; cv=none; d=google.com; s=arc-20160816; b=NA8enKFwxsxs2GoEvL0+klfvYBmVAETZRnUXaKXkae2R0T+smPOQDvSSPsdky1+fzd ke9owV3L7lFX/bTxDcLDdVpDY0kEhKwD5nahSn4mWbh4OYBTeqC2/crtl9+qUGBbcWHI M6Bly2gLr7HYj3GWo993ldQv7hBNIR2nkQ7jI3agyhx/IJMHtC3tJhAafkubPc+PK7P7 iRzWajRZgsZ3uXLIGDBGCq9nUSn86d3qJPIbuMLDNsJzfUtGI8d0h+2BPdls1mc+RjSN 1uL2hXf/rAKo4XcqZx3hkJZBBafTISchCBWSqES0DkNrRZxQ6n1pMJIiXnfY6KjE0f/3 uNYg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to:date :to:from:message-id:dkim-signature:delivered-to; bh=TJcWAz20R91/Ye7bxqHtqhL49k5MwyMgkjVv19Fvx10=; b=MQyc1741bkQ0Ddp5oLk1skZzmE1zV5az8ZDTb/0aiW9EdOD3NcjA52AbDsQeo4Hz9Q P12kdLouVCTUqgjfoxtM2AN76fSjkGCToFmrJnaPQnJ8U/ac6vn5Zit/0RkjbGoXT3sB FcRNJ565nV9L8PGjTXTY4eF6vA4gHOZWxmXjar0REimcfHsUPOnyKqTBmc0jhhHYkd92 0+3m+8waEDjr5U7neWEGLqzQ091nYfob34uSIvEm7fuOIayt0ZprCXRpAeyGU7V/z+kp hAG5BlFX2snBxIWczkiYb9HiQJDWj3vkCWjtVf4HOO93NBmR6+imorFq+yXuNOmT0UXr rBLQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@foxmail.com header.s=s201512 header.b=cOwgRERR; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=foxmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id y7-20020a1709060a8700b0078e9ca562d8si5702043ejf.879.2022.11.19.22.51.21; Sat, 19 Nov 2022 22:51:21 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@foxmail.com header.s=s201512 header.b=cOwgRERR; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=foxmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 64D0968BBE2; Sun, 20 Nov 2022 08:50:49 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from out162-62-58-211.mail.qq.com (out162-62-58-211.mail.qq.com [162.62.58.211]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 9AC7068BB60 for ; Sun, 20 Nov 2022 08:50:43 +0200 (EET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=foxmail.com; s=s201512; t=1668927041; bh=Zqn00dhZlFE/PEKzsOiV2WxdjhnDLkkalqsTcYaJT4k=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=cOwgRERRrSxUvWdFxo/xoZMfLgv8+/9Ik1LtF4Fj7gytXFYlx8UOLwCg7uW40u9S8 pErBaIp3820SdQhTGEKu30na5EpgrjZb9kyE9Pm+nYxGD0IheIfD1r0fnCBj9FW1DZ CzqHWMLOLNfZA17soQQ/xT12Zn3nUA9K7gSExIk4= Received: from localhost.localdomain ([14.153.77.200]) by newxmesmtplogicsvrszc2-1.qq.com (NewEsmtp) with SMTP id C8F0626E; Sun, 20 Nov 2022 14:50:15 +0800 X-QQ-mid: xmsmtpt1668927040t1ov2xwbk Message-ID: X-QQ-XMAILINFO: OZZSS56D9fAjn5K/MzACCEwySgmxi1Cmh+40rV5R5654eOu+q550fjRK9qDcz+ NnTeSQAy8QwzP6DeoFJGVzifoVZcrn5W9/YXgz68d3sp8BSdV1KdWH/98H8ywT6+XsLPzEHd66Ty YkNyD38XA02ouiberhLVS7u+/2x7/LhV4xgHMVSgAi4xzBX/os0l7BjmdR+LpXrHI+njo/qmMQaa xpZfCBWHlqihRhgXqm7NpROwMLaYyyH6xQhZP1XVKSdeSgyqEpm7+z5kCtd+OALEnhneWLou2GRP xGSv3aySoJ8G/1bYfABz6t8jCow3uvyKBrOjKX3mCKhA/YjMQBCN4aYOI1qJSPaYPosTDnYXAyfG YdZ76BsbWDCrXf++o1VdnAOZq+9mCHjJMeLZr/moxMtb4xA41PwLcS7pk86PAlsbaEBGQ67P6kJs P6aYtWx8Z1gN1uno41Db6wvbc2OJQFtMeUkHjK27dAyeErizA3ScSxXBNZkyv2orYRIxQX4loxAz U1urRoOAybRdXoY3opUUAkv42WmUPRGeioxDdEsDZ+lxJlW72/BfgtDrYERVscE8gSymZMwh/cIl GsvFo/FfBI9BgqK7viyfEPrWVJ4Lw9VfVhC4XD/uxyp+gJ1UCTsbtiCK6U/3oSQZ/bnPJYKhJhez uPVh4LjFVdG1BE8crQJqG0nGI0TNWMFBCFq8mFPzyQP6cSa8sm8iq8nuZOqsyFRl5wQboo4Ie/EA pBPto4WZb1oXxbiCFz5HNl5CveEZOdA99+y6xi8r3cKA/2V8OQ2EyXTP7VAMdMVJb26g6KVkZuO8 86RyI9L8TCrtWle//wMxvVJ/A8/jcYJQ5c0thH3b82Nz48SAeUkwYZWHRAsQnsB/nO2mz8sPoJRf wj4ScoWa7DPgaiXoZkwlaRcdPZlo0097YPDmwlHV1vanYgDIbb3122nLzu8xFSLaZAIH5acOac9o Dy73g7dV53tgUXJ2cPPf21ep9e2g4nz1hNr2YVz0Gbv8A4vFubNyph9adw7iB6TGme7aY8Ie1Tmf sOM7VS91CBkLA/lW+bJuuxCncsasE9csiCX5T1Mw== From: Zhao Zhili To: ffmpeg-devel@ffmpeg.org Date: Sun, 20 Nov 2022 14:49:45 +0800 X-OQ-MSGID: <20221120064947.94540-11-quinkblack@foxmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221120064947.94540-1-quinkblack@foxmail.com> References: <20221120064947.94540-1-quinkblack@foxmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 10/12] avcodec: add MediaCodec encoder X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: matthieu.bouron@stupeflix.com, Zhao Zhili , aman@tmm1.net Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: bWkYktnx6u4U From: Zhao Zhili Signed-off-by: Zhao Zhili --- Changelog | 1 + configure | 4 + libavcodec/Makefile | 2 + libavcodec/allcodecs.c | 2 + libavcodec/mediacodec_wrapper.c | 101 ++++++- libavcodec/mediacodec_wrapper.h | 8 + libavcodec/mediacodecenc.c | 504 ++++++++++++++++++++++++++++++++ libavcodec/version.h | 2 +- 8 files changed, 618 insertions(+), 6 deletions(-) create mode 100644 libavcodec/mediacodecenc.c diff --git a/Changelog b/Changelog index eb8eb72127..9310706b72 100644 --- a/Changelog +++ b/Changelog @@ -21,6 +21,7 @@ version : - backgroundkey filter - nvenc AV1 encoding support - MediaCodec decoder via NDKMediaCodec +- MediaCodec encoder version 5.1: diff --git a/configure b/configure index 9da5a36072..03004826fa 100755 --- a/configure +++ b/configure @@ -3183,6 +3183,8 @@ h264_cuvid_decoder_select="h264_mp4toannexb_bsf" h264_mediacodec_decoder_deps="mediacodec" h264_mediacodec_decoder_extralibs="-landroid" h264_mediacodec_decoder_select="h264_mp4toannexb_bsf h264_parser" +h264_mediacodec_encoder_deps="mediacodec" +h264_mediacodec_encoder_extralibs="-landroid" h264_mf_encoder_deps="mediafoundation" h264_mmal_decoder_deps="mmal" h264_nvenc_encoder_deps="nvenc" @@ -3202,6 +3204,8 @@ hevc_cuvid_decoder_select="hevc_mp4toannexb_bsf" hevc_mediacodec_decoder_deps="mediacodec" hevc_mediacodec_decoder_extralibs="-landroid" hevc_mediacodec_decoder_select="hevc_mp4toannexb_bsf hevc_parser" +hevc_mediacodec_encoder_deps="mediacodec" +hevc_mediacodec_encoder_extralibs="-landroid" hevc_mf_encoder_deps="mediafoundation" hevc_nvenc_encoder_deps="nvenc" hevc_nvenc_encoder_select="atsc_a53" diff --git a/libavcodec/Makefile b/libavcodec/Makefile index 94dc75a1b2..0bc7943d16 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -396,6 +396,7 @@ OBJS-$(CONFIG_H264_DECODER) += h264dec.o h264_cabac.o h264_cavlc.o \ OBJS-$(CONFIG_H264_AMF_ENCODER) += amfenc_h264.o OBJS-$(CONFIG_H264_CUVID_DECODER) += cuviddec.o OBJS-$(CONFIG_H264_MEDIACODEC_DECODER) += mediacodecdec.o +OBJS-$(CONFIG_H264_MEDIACODEC_ENCODER) += mediacodecenc.o OBJS-$(CONFIG_H264_MF_ENCODER) += mfenc.o mf_utils.o OBJS-$(CONFIG_H264_MMAL_DECODER) += mmaldec.o OBJS-$(CONFIG_H264_NVENC_ENCODER) += nvenc_h264.o nvenc.o @@ -420,6 +421,7 @@ OBJS-$(CONFIG_HEVC_DECODER) += hevcdec.o hevc_mvs.o \ OBJS-$(CONFIG_HEVC_AMF_ENCODER) += amfenc_hevc.o OBJS-$(CONFIG_HEVC_CUVID_DECODER) += cuviddec.o OBJS-$(CONFIG_HEVC_MEDIACODEC_DECODER) += mediacodecdec.o +OBJS-$(CONFIG_HEVC_MEDIACODEC_ENCODER) += mediacodecenc.o OBJS-$(CONFIG_HEVC_MF_ENCODER) += mfenc.o mf_utils.o OBJS-$(CONFIG_HEVC_NVENC_ENCODER) += nvenc_hevc.o nvenc.o OBJS-$(CONFIG_HEVC_QSV_DECODER) += qsvdec.o diff --git a/libavcodec/allcodecs.c b/libavcodec/allcodecs.c index f5ec3bc6e1..d5a6c427e1 100644 --- a/libavcodec/allcodecs.c +++ b/libavcodec/allcodecs.c @@ -154,6 +154,7 @@ extern const FFCodec ff_h264_decoder; extern const FFCodec ff_h264_crystalhd_decoder; extern const FFCodec ff_h264_v4l2m2m_decoder; extern const FFCodec ff_h264_mediacodec_decoder; +extern const FFCodec ff_h264_mediacodec_encoder; extern const FFCodec ff_h264_mmal_decoder; extern const FFCodec ff_h264_qsv_decoder; extern const FFCodec ff_h264_rkmpp_decoder; @@ -844,6 +845,7 @@ extern const FFCodec ff_h264_videotoolbox_encoder; extern const FFCodec ff_hevc_amf_encoder; extern const FFCodec ff_hevc_cuvid_decoder; extern const FFCodec ff_hevc_mediacodec_decoder; +extern const FFCodec ff_hevc_mediacodec_encoder; extern const FFCodec ff_hevc_mf_encoder; extern const FFCodec ff_hevc_nvenc_encoder; extern const FFCodec ff_hevc_qsv_encoder; diff --git a/libavcodec/mediacodec_wrapper.c b/libavcodec/mediacodec_wrapper.c index 284d615980..555058e907 100644 --- a/libavcodec/mediacodec_wrapper.c +++ b/libavcodec/mediacodec_wrapper.c @@ -212,6 +212,9 @@ struct JNIAMediaCodecFields { jmethodID release_output_buffer_id; jmethodID release_output_buffer_at_time_id; + jmethodID set_input_surface_id; + jmethodID signal_end_of_input_stream_id; + jclass mediainfo_class; jmethodID init_id; @@ -261,6 +264,9 @@ static const struct FFJniField jni_amediacodec_mapping[] = { { "android/media/MediaCodec", "releaseOutputBuffer", "(IZ)V", FF_JNI_METHOD, offsetof(struct JNIAMediaCodecFields, release_output_buffer_id), 1 }, { "android/media/MediaCodec", "releaseOutputBuffer", "(IJ)V", FF_JNI_METHOD, offsetof(struct JNIAMediaCodecFields, release_output_buffer_at_time_id), 0 }, + { "android/media/MediaCodec", "setInputSurface", "(Landroid/view/Surface;)V", FF_JNI_METHOD, offsetof(struct JNIAMediaCodecFields, set_input_surface_id), 0 }, + { "android/media/MediaCodec", "signalEndOfInputStream", "()V", FF_JNI_METHOD, offsetof(struct JNIAMediaCodecFields, signal_end_of_input_stream_id), 0 }, + { "android/media/MediaCodec$BufferInfo", NULL, NULL, FF_JNI_CLASS, offsetof(struct JNIAMediaCodecFields, mediainfo_class), 1 }, { "android/media/MediaCodec.BufferInfo", "", "()V", FF_JNI_METHOD, offsetof(struct JNIAMediaCodecFields, init_id), 1 }, @@ -1385,7 +1391,26 @@ static int mediacodec_jni_configure(FFAMediaCodec *ctx, JNI_GET_ENV_OR_RETURN(env, codec, AVERROR_EXTERNAL); - (*env)->CallVoidMethod(env, codec->object, codec->jfields.configure_id, format->object, surface, NULL, flags); + if (flags & codec->CONFIGURE_FLAG_ENCODE) { + if (surface && !codec->jfields.set_input_surface_id) { + av_log(ctx, AV_LOG_ERROR, "System doesn't support setInputSurface\n"); + return AVERROR_EXTERNAL; + } + + (*env)->CallVoidMethod(env, codec->object, codec->jfields.configure_id, format->object, NULL, NULL, flags); + if (ff_jni_exception_check(env, 1, codec) < 0) + return AVERROR_EXTERNAL; + + if (!surface) + return 0; + + (*env)->CallVoidMethod(env, codec->object, codec->jfields.set_input_surface_id, surface); + if (ff_jni_exception_check(env, 1, codec) < 0) + return AVERROR_EXTERNAL; + return 0; + } else { + (*env)->CallVoidMethod(env, codec->object, codec->jfields.configure_id, format->object, surface, NULL, flags); + } if (ff_jni_exception_check(env, 1, codec) < 0) { ret = AVERROR_EXTERNAL; goto fail; @@ -1743,6 +1768,21 @@ fail: return ret; } +static int mediacodec_jni_signalEndOfInputStream(FFAMediaCodec *ctx) +{ + JNIEnv *env = NULL; + FFAMediaCodecJni *codec = (FFAMediaCodecJni *)ctx; + + JNI_GET_ENV_OR_RETURN(env, codec, AVERROR_EXTERNAL); + + (*env)->CallVoidMethod(env, codec->object, codec->jfields.signal_end_of_input_stream_id); + if (ff_jni_exception_check(env, 1, codec) < 0) { + return AVERROR_EXTERNAL; + } + + return 0; +} + static const FFAMediaFormat media_format_jni = { .class = &amediaformat_class, @@ -1801,6 +1841,7 @@ static const FFAMediaCodec media_codec_jni = { .getConfigureFlagEncode = mediacodec_jni_getConfigureFlagEncode, .cleanOutputBuffers = mediacodec_jni_cleanOutputBuffers, + .signalEndOfInputStream = mediacodec_jni_signalEndOfInputStream, }; typedef struct FFAMediaFormatNdk { @@ -1866,6 +1907,10 @@ typedef struct FFAMediaCodecNdk { // Available since API level 28. media_status_t (*getName)(AMediaCodec*, char** out_name); void (*releaseName)(AMediaCodec*, char* name); + + // Available since API level 26. + media_status_t (*setInputSurface)(AMediaCodec*, ANativeWindow *); + media_status_t (*signalEndOfInputStream)(AMediaCodec *); } FFAMediaCodecNdk; static const FFAMediaFormat media_format_ndk; @@ -2098,6 +2143,9 @@ static inline FFAMediaCodec *ndk_codec_create(int method, const char *arg) { GET_SYMBOL(getName, 0) GET_SYMBOL(releaseName, 0) + GET_SYMBOL(setInputSurface, 0) + GET_SYMBOL(signalEndOfInputStream, 0) + #undef GET_SYMBOL switch (method) { @@ -2184,10 +2232,32 @@ static int mediacodec_ndk_configure(FFAMediaCodec* ctx, return AVERROR(EINVAL); } - status = codec->configure(codec->impl, format->impl, native_window, NULL, flags); - if (status != AMEDIA_OK) { - av_log(codec, AV_LOG_ERROR, "configure failed, %d\n", status); - return AVERROR_EXTERNAL; + if (flags & AMEDIACODEC_CONFIGURE_FLAG_ENCODE) { + if (native_window && !codec->setInputSurface) { + av_log(ctx, AV_LOG_ERROR, "System doesn't support setInputSurface\n"); + return AVERROR_EXTERNAL; + } + + status = codec->configure(codec->impl, format->impl, NULL, NULL, flags); + if (status != AMEDIA_OK) { + av_log(codec, AV_LOG_ERROR, "Encoder configure failed, %d\n", status); + return AVERROR_EXTERNAL; + } + + if (!native_window) + return 0; + + status = codec->setInputSurface(codec->impl, native_window); + if (status != AMEDIA_OK) { + av_log(codec, AV_LOG_ERROR, "Encoder set input surface failed, %d\n", status); + return AVERROR_EXTERNAL; + } + } else { + status = codec->configure(codec->impl, format->impl, native_window, NULL, flags); + if (status != AMEDIA_OK) { + av_log(codec, AV_LOG_ERROR, "Decoder configure failed, %d\n", status); + return AVERROR_EXTERNAL; + } } return 0; @@ -2330,6 +2400,26 @@ static int mediacodec_ndk_cleanOutputBuffers(FFAMediaCodec *ctx) return 0; } +static int mediacodec_ndk_signalEndOfInputStream(FFAMediaCodec *ctx) +{ + FFAMediaCodecNdk *codec = (FFAMediaCodecNdk *)ctx; + media_status_t status; + + if (!codec->signalEndOfInputStream) { + av_log(codec, AV_LOG_ERROR, "signalEndOfInputStream unavailable\n"); + return AVERROR_EXTERNAL; + } + + status = codec->signalEndOfInputStream(codec->impl); + if (status != AMEDIA_OK) { + av_log(codec, AV_LOG_ERROR, "signalEndOfInputStream failed, %d\n", status); + return AVERROR_EXTERNAL; + } + av_log(codec, AV_LOG_DEBUG, "signalEndOfInputStream success\n"); + + return 0; +} + static const FFAMediaFormat media_format_ndk = { .class = &amediaformat_ndk_class, @@ -2388,6 +2478,7 @@ static const FFAMediaCodec media_codec_ndk = { .getConfigureFlagEncode = mediacodec_ndk_getConfigureFlagEncode, .cleanOutputBuffers = mediacodec_ndk_cleanOutputBuffers, + .signalEndOfInputStream = mediacodec_ndk_signalEndOfInputStream, }; FFAMediaFormat *ff_AMediaFormat_new(int ndk) diff --git a/libavcodec/mediacodec_wrapper.h b/libavcodec/mediacodec_wrapper.h index 7cf3f4aecd..f15ad66d83 100644 --- a/libavcodec/mediacodec_wrapper.h +++ b/libavcodec/mediacodec_wrapper.h @@ -192,6 +192,9 @@ struct FFAMediaCodec { int (*getConfigureFlagEncode)(FFAMediaCodec *codec); int (*cleanOutputBuffers)(FFAMediaCodec *codec); + + // For encoder with FFANativeWindow as input. + int (*signalEndOfInputStream)(FFAMediaCodec *); }; static inline char *ff_AMediaCodec_getName(FFAMediaCodec *codec) @@ -311,6 +314,11 @@ static inline int ff_AMediaCodec_cleanOutputBuffers(FFAMediaCodec *codec) return codec->cleanOutputBuffers(codec); } +static inline int ff_AMediaCodec_signalEndOfInputStream(FFAMediaCodec *codec) +{ + return codec->signalEndOfInputStream(codec); +} + int ff_Build_SDK_INT(AVCodecContext *avctx); #endif /* AVCODEC_MEDIACODEC_WRAPPER_H */ diff --git a/libavcodec/mediacodecenc.c b/libavcodec/mediacodecenc.c new file mode 100644 index 0000000000..7cdde59945 --- /dev/null +++ b/libavcodec/mediacodecenc.c @@ -0,0 +1,504 @@ +/* + * Android MediaCodec encoders + * + * Copyright (c) 2022 Zhao Zhili + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "config_components.h" + +#include "libavutil/avassert.h" +#include "libavutil/hwcontext_mediacodec.h" +#include "libavutil/imgutils.h" +#include "libavutil/opt.h" + +#include "avcodec.h" +#include "codec_internal.h" +#include "encode.h" +#include "hwconfig.h" +#include "jni.h" +#include "mediacodec.h" +#include "mediacodec_wrapper.h" +#include "mediacodecdec_common.h" + +#define INPUT_DEQUEUE_TIMEOUT_US 8000 +#define OUTPUT_DEQUEUE_TIMEOUT_US 8000 + +typedef struct MediaCodecEncContext { + AVClass *avclass; + FFAMediaCodec *codec; + int use_ndk_codec; + FFANativeWindow *window; + + int fps; + int width; + int height; + + uint8_t *extradata; + int extradata_size; + + // Since MediaCodec doesn't output DTS, use a timestamp queue to save pts + // of AVFrame and generate DTS for AVPacket. + // + // This doesn't work when use Surface as input, in that case frames can be + // sent to encoder without our notice. One exception is frames come from + // our MediaCodec decoder wrapper, since we can control it's render by + // av_mediacodec_release_buffer. + int64_t timestamps[32]; + int ts_head; + int ts_tail; + + int eof_sent; + + AVFrame *frame; +} MediaCodecEncContext; + +enum { + COLOR_FormatYUV420Planar = 0x13, + COLOR_FormatYUV420SemiPlanar = 0x15, + COLOR_FormatSurface = 0x7F000789, +}; + +static const struct { + int color_format; + enum AVPixelFormat pix_fmt; +} color_formats[] = { + { COLOR_FormatYUV420Planar, AV_PIX_FMT_YUV420P }, + { COLOR_FormatYUV420SemiPlanar, AV_PIX_FMT_NV12 }, + { COLOR_FormatSurface, AV_PIX_FMT_MEDIACODEC }, +}; + +static const enum AVPixelFormat avc_pix_fmts[] = { + AV_PIX_FMT_MEDIACODEC, + AV_PIX_FMT_YUV420P, + AV_PIX_FMT_NV12, + AV_PIX_FMT_NONE +}; + +static void mediacodec_output_format(AVCodecContext *avctx) +{ + MediaCodecEncContext *s = avctx->priv_data; + char *name = ff_AMediaCodec_getName(s->codec); + FFAMediaFormat *out_format = ff_AMediaCodec_getOutputFormat(s->codec); + char *str = ff_AMediaFormat_toString(out_format); + + av_log(avctx, AV_LOG_DEBUG, "MediaCodec encoder %s output format %s\n", + name ? name : "unknown", str); + av_free(name); + av_free(str); + ff_AMediaFormat_delete(out_format); +} + +static av_cold int mediacodec_init(AVCodecContext *avctx) +{ + const char *codec_mime = NULL; + MediaCodecEncContext *s = avctx->priv_data; + FFAMediaFormat *format = NULL; + int ret; + int gop; + + if (s->use_ndk_codec < 0) + s->use_ndk_codec = !av_jni_get_java_vm(avctx); + + switch (avctx->codec_id) { + case AV_CODEC_ID_H264: + codec_mime = "video/avc"; + break; + case AV_CODEC_ID_HEVC: + codec_mime = "video/hevc"; + break; + default: + av_assert0(0); + } + + s->codec = ff_AMediaCodec_createEncoderByType(codec_mime, s->use_ndk_codec); + if (!s->codec) { + av_log(avctx, AV_LOG_ERROR, "Failed to create encoder for type %s\n", + codec_mime); + return AVERROR_EXTERNAL; + } + + format = ff_AMediaFormat_new(s->use_ndk_codec); + if (!format) { + av_log(avctx, AV_LOG_ERROR, "Failed to create media format\n"); + return AVERROR_EXTERNAL; + } + + ff_AMediaFormat_setString(format, "mime", codec_mime); + s->width = FFALIGN(avctx->width, 16); + s->height = avctx->height; + ff_AMediaFormat_setInt32(format, "width", s->width); + ff_AMediaFormat_setInt32(format, "height", s->height); + + if (avctx->pix_fmt == AV_PIX_FMT_MEDIACODEC) { + AVMediaCodecContext *user_ctx = avctx->hwaccel_context; + if (avctx->hw_device_ctx) { + AVHWDeviceContext *device_ctx = (AVHWDeviceContext*)(avctx->hw_device_ctx->data); + AVMediaCodecDeviceContext *dev_ctx; + + if (device_ctx->type != AV_HWDEVICE_TYPE_MEDIACODEC || !device_ctx->hwctx) { + ret = AVERROR(EINVAL); + goto bailout; + } + dev_ctx = device_ctx->hwctx; + s->window = ff_mediacodec_surface_ref(dev_ctx->surface, dev_ctx->native_window, avctx); + } + + if (!s->window && user_ctx && user_ctx->surface) + s->window = ff_mediacodec_surface_ref(user_ctx->surface, NULL, avctx); + + if (!s->window) { + ret = AVERROR(EINVAL); + av_log(avctx, AV_LOG_ERROR, "Missing hw_device_ctx or hwaccel_context for AV_PIX_FMT_MEDIACODEC\n"); + goto bailout; + } + } + + for (int i = 0; i < FF_ARRAY_ELEMS(color_formats); i++) { + if (avctx->pix_fmt == color_formats[i].pix_fmt) { + ff_AMediaFormat_setInt32(format, "color-format", + color_formats[i].color_format); + break; + } + } + + if (avctx->bit_rate) + ff_AMediaFormat_setInt32(format, "bitrate", avctx->bit_rate); + // frame-rate and i-frame-interval are required to configure codec + if (avctx->framerate.num >= avctx->framerate.den && avctx->framerate.den > 0) { + s->fps = avctx->framerate.num / avctx->framerate.den; + } else { + s->fps = 30; + av_log(avctx, AV_LOG_INFO, "Use %d as the default MediaFormat frame-rate\n", s->fps); + } + gop = round(avctx->gop_size / s->fps); + if (gop == 0) { + gop = 1; + av_log(avctx, AV_LOG_INFO, + "Use %d as the default MediaFormat i-frame-interval, " + "please set gop_size properly (>= fps)\n", gop); + } else { + av_log(avctx, AV_LOG_DEBUG, "Set i-frame-interval to %d\n", gop); + } + + ff_AMediaFormat_setInt32(format, "frame-rate", s->fps); + ff_AMediaFormat_setInt32(format, "i-frame-interval", gop); + + + ret = ff_AMediaCodec_getConfigureFlagEncode(s->codec); + ret = ff_AMediaCodec_configure(s->codec, format, s->window, NULL, ret); + if (ret) { + av_log(avctx, AV_LOG_ERROR, "MediaCodec configure failed, %s\n", av_err2str(ret)); + goto bailout; + } + + ret = ff_AMediaCodec_start(s->codec); + if (ret) { + av_log(avctx, AV_LOG_ERROR, "MediaCodec failed to start, %s\n", av_err2str(ret)); + goto bailout; + } + + mediacodec_output_format(avctx); + + s->frame = av_frame_alloc(); + if (!s->frame) + ret = AVERROR(ENOMEM); + +bailout: + if (format) + ff_AMediaFormat_delete(format); + return ret; +} + +static int mediacodec_receive(AVCodecContext *avctx, + AVPacket *pkt, + int *got_packet) +{ + MediaCodecEncContext *s = avctx->priv_data; + FFAMediaCodec *codec = s->codec; + FFAMediaCodecBufferInfo out_info = {0}; + uint8_t *out_buf; + size_t out_size = 0; + int ret; + int extradata_size = 0; + int64_t timeout_us = s->eof_sent ? OUTPUT_DEQUEUE_TIMEOUT_US : 0; + ssize_t index = ff_AMediaCodec_dequeueOutputBuffer(codec, &out_info, timeout_us); + + if (ff_AMediaCodec_infoTryAgainLater(codec, index)) + return AVERROR(EAGAIN); + + if (ff_AMediaCodec_infoOutputFormatChanged(codec, index)) { + mediacodec_output_format(avctx); + return AVERROR(EAGAIN); + } + + if (ff_AMediaCodec_infoOutputBuffersChanged(codec, index)) { + ff_AMediaCodec_cleanOutputBuffers(codec); + return AVERROR(EAGAIN); + } + + if (index < 0) + return AVERROR_EXTERNAL; + + if (out_info.flags & ff_AMediaCodec_getBufferFlagEndOfStream(codec)) + return AVERROR_EOF; + + out_buf = ff_AMediaCodec_getOutputBuffer(codec, index, &out_size); + if (!out_buf) { + ret = AVERROR_EXTERNAL; + goto bailout; + } + + if (out_info.flags & ff_AMediaCodec_getBufferFlagCodecConfig(codec)) { + ret = av_reallocp(&s->extradata, out_info.size); + if (ret) + goto bailout; + + s->extradata_size = out_info.size; + memcpy(s->extradata, out_buf + out_info.offset, out_info.size); + ff_AMediaCodec_releaseOutputBuffer(codec, index, false); + // try immediately + return mediacodec_receive(avctx, pkt, got_packet); + } + + ret = ff_get_encode_buffer(avctx, pkt, out_info.size + s->extradata_size, 0); + if (ret < 0) + goto bailout; + + if (s->extradata_size) { + extradata_size = s->extradata_size; + s->extradata_size = 0; + memcpy(pkt->data, s->extradata, extradata_size); + } + memcpy(pkt->data + extradata_size, out_buf + out_info.offset, out_info.size); + pkt->pts = av_rescale_q(out_info.presentationTimeUs, AV_TIME_BASE_Q, avctx->time_base); + if (s->ts_tail != s->ts_head) { + pkt->dts = s->timestamps[s->ts_tail]; + s->ts_tail = (s->ts_tail + 1) % FF_ARRAY_ELEMS(s->timestamps); + } + + if (out_info.flags & ff_AMediaCodec_getBufferFlagKeyFrame(codec)) + pkt->flags |= AV_PKT_FLAG_KEY; + ret = 0; + *got_packet = 1; + + av_log(avctx, AV_LOG_TRACE, "receive packet pts %" PRId64 " dts %" PRId64 + " flags %d extradata %d\n", + pkt->pts, pkt->dts, pkt->flags, extradata_size); + +bailout: + ff_AMediaCodec_releaseOutputBuffer(codec, index, false); + return ret; +} + +static void copy_frame_to_buffer(AVCodecContext *avctx, const AVFrame *frame, uint8_t *dst, size_t size) +{ + MediaCodecEncContext *s = avctx->priv_data; + uint8_t *dst_data[4] = {}; + int dst_linesize[4] = {}; + const uint8_t *src_data[4] = { + frame->data[0], frame->data[1], frame->data[2], frame->data[3] + }; + + if (avctx->pix_fmt == AV_PIX_FMT_YUV420P) { + dst_data[0] = dst; + dst_data[1] = dst + s->width * s->height; + dst_data[2] = dst_data[1] + s->width * s->height / 4; + + dst_linesize[0] = s->width; + dst_linesize[1] = dst_linesize[2] = s->width / 2; + } else if (avctx->pix_fmt == AV_PIX_FMT_NV12) { + dst_data[0] = dst; + dst_data[1] = dst + s->width * s->height; + + dst_linesize[0] = s->width; + dst_linesize[1] = s->width; + } else { + av_assert0(0); + } + + av_image_copy(dst_data, dst_linesize, src_data, frame->linesize, + avctx->pix_fmt, avctx->width, avctx->height); +} + +static int mediacodec_send(AVCodecContext *avctx, + const AVFrame *frame) { + MediaCodecEncContext *s = avctx->priv_data; + FFAMediaCodec *codec = s->codec; + ssize_t index; + uint8_t *input_buf = NULL; + size_t input_size = 0; + int64_t pts = 0; + uint32_t flags = 0; + int64_t timeout_us; + + if (s->eof_sent) + return 0; + + if (s->window) { + if (!frame) { + s->eof_sent = 1; + return ff_AMediaCodec_signalEndOfInputStream(codec); + } + + + if (frame->data[3]) { + pts = av_rescale_q(frame->pts, avctx->time_base, AV_TIME_BASE_Q); + s->timestamps[s->ts_head] = frame->pts; + s->ts_head = (s->ts_head + 1) % FF_ARRAY_ELEMS(s->timestamps); + + av_mediacodec_release_buffer((AVMediaCodecBuffer *)frame->data[3], 1); + } + return 0; + } + + timeout_us = INPUT_DEQUEUE_TIMEOUT_US; + index = ff_AMediaCodec_dequeueInputBuffer(codec, timeout_us); + if (ff_AMediaCodec_infoTryAgainLater(codec, index)) + return AVERROR(EAGAIN); + + if (index < 0) { + av_log(avctx, AV_LOG_ERROR, "dequeue input buffer failed, %zd", index); + return AVERROR_EXTERNAL; + } + + if (frame) { + input_buf = ff_AMediaCodec_getInputBuffer(codec, index, &input_size); + copy_frame_to_buffer(avctx, frame, input_buf, input_size); + + pts = av_rescale_q(frame->pts, avctx->time_base, AV_TIME_BASE_Q); + + s->timestamps[s->ts_head] = frame->pts; + s->ts_head = (s->ts_head + 1) % FF_ARRAY_ELEMS(s->timestamps); + } else { + flags |= ff_AMediaCodec_getBufferFlagEndOfStream(codec); + s->eof_sent = 1; + } + + ff_AMediaCodec_queueInputBuffer(codec, index, 0, input_size, pts, flags); + return 0; +} + +static int mediacodec_encode(AVCodecContext *avctx, AVPacket *pkt) +{ + MediaCodecEncContext *s = avctx->priv_data; + int ret; + int got_packet = 0; + + // Return on three case: + // 1. Serious error + // 2. Got a packet success + // 3. No AVFrame is available yet (don't return if get_frame return EOF) + while (1) { + ret = mediacodec_receive(avctx, pkt, &got_packet); + if (!ret) + return 0; + else if (ret != AVERROR(EAGAIN)) + return ret; + + if (!s->frame->buf[0]) { + ret = ff_encode_get_frame(avctx, s->frame); + if (ret && ret != AVERROR_EOF) + return ret; + } + + ret = mediacodec_send(avctx, s->frame->buf[0] ? s->frame : NULL); + if (!ret) + av_frame_unref(s->frame); + else if (ret != AVERROR(EAGAIN)) + return ret; + } + + return 0; +} + +static av_cold int mediacodec_close(AVCodecContext *avctx) +{ + MediaCodecEncContext *s = avctx->priv_data; + if (s->codec) { + ff_AMediaCodec_stop(s->codec); + ff_AMediaCodec_delete(s->codec); + s->codec = NULL; + } + + if (s->window) { + ff_mediacodec_surface_unref(s->window, avctx); + s->window = NULL; + } + + av_frame_free(&s->frame); + + return 0; +} + +static const AVCodecHWConfigInternal *const mediacodec_hw_configs[] = { + &(const AVCodecHWConfigInternal) { + .public = { + .pix_fmt = AV_PIX_FMT_MEDIACODEC, + .methods = AV_CODEC_HW_CONFIG_METHOD_AD_HOC | + AV_CODEC_HW_CONFIG_METHOD_HW_DEVICE_CTX, + .device_type = AV_HWDEVICE_TYPE_MEDIACODEC, + }, + .hwaccel = NULL, + }, + NULL +}; + +#define OFFSET(x) offsetof(MediaCodecEncContext, x) +#define VE AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_ENCODING_PARAM +static const AVOption common_options[] = { + { "ndk_codec", "Use MediaCodec from NDK", + OFFSET(use_ndk_codec), AV_OPT_TYPE_BOOL, {.i64 = -1}, -1, 1, VE }, + { NULL }, +}; + +#define MEDIACODEC_ENCODER_CLASS(name) \ +static const AVClass name ## _mediacodec_class = { \ + .class_name = #name "_mediacodec", \ + .item_name = av_default_item_name, \ + .option = common_options, \ + .version = LIBAVUTIL_VERSION_INT, \ +}; \ + +#define DECLARE_MEDIACODEC_ENCODER(short_name, long_name, codec_id) \ +MEDIACODEC_ENCODER_CLASS(short_name) \ +const FFCodec ff_ ## short_name ## _mediacodec_encoder = { \ + .p.name = #short_name "_mediacodec", \ + CODEC_LONG_NAME(long_name " Android MediaCodec encoder"), \ + .p.type = AVMEDIA_TYPE_VIDEO, \ + .p.id = codec_id, \ + .p.capabilities = AV_CODEC_CAP_DR1 | AV_CODEC_CAP_DELAY \ + | AV_CODEC_CAP_HARDWARE, \ + .priv_data_size = sizeof(MediaCodecEncContext), \ + .p.pix_fmts = avc_pix_fmts, \ + .init = mediacodec_init, \ + FF_CODEC_RECEIVE_PACKET_CB(mediacodec_encode), \ + .close = mediacodec_close, \ + .p.priv_class = &short_name ## _mediacodec_class, \ + .caps_internal = FF_CODEC_CAP_INIT_CLEANUP, \ + .p.wrapper_name = "mediacodec", \ + .hw_configs = mediacodec_hw_configs, \ +}; \ + +#if CONFIG_H264_MEDIACODEC_ENCODER +DECLARE_MEDIACODEC_ENCODER(h264, "H.264", AV_CODEC_ID_H264) +#endif + +#if CONFIG_HEVC_MEDIACODEC_ENCODER +DECLARE_MEDIACODEC_ENCODER(hevc, "H.265", AV_CODEC_ID_HEVC) +#endif diff --git a/libavcodec/version.h b/libavcodec/version.h index fcf589f61f..d149bc6c46 100644 --- a/libavcodec/version.h +++ b/libavcodec/version.h @@ -29,7 +29,7 @@ #include "version_major.h" -#define LIBAVCODEC_VERSION_MINOR 53 +#define LIBAVCODEC_VERSION_MINOR 54 #define LIBAVCODEC_VERSION_MICRO 100 #define LIBAVCODEC_VERSION_INT AV_VERSION_INT(LIBAVCODEC_VERSION_MAJOR, \ From patchwork Sun Nov 20 06:49:46 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhao Zhili X-Patchwork-Id: 39352 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a21:999a:b0:a4:2148:650a with SMTP id ve26csp1406743pzb; Sat, 19 Nov 2022 22:51:39 -0800 (PST) X-Google-Smtp-Source: AA0mqf4UrWxKpK9vE8ZJkICqj53oOuyLhMHj7osb8A9NevzWkPwjwiURCdQ2op4szladJJL8bRpU X-Received: by 2002:a17:907:c314:b0:78c:2b55:59be with SMTP id tl20-20020a170907c31400b0078c2b5559bemr11528283ejc.2.1668927099582; Sat, 19 Nov 2022 22:51:39 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1668927099; cv=none; d=google.com; s=arc-20160816; b=DKNqZrv276lWiJK5W7jMITF1qFLAAejvDOt31fLYlVk9gfmSGU+lcS/1HaIqMoOWdS bOjrVLvu2+twqCXJvcLzgQB7a107OOl/HM5Gey+LIcEv9LYsa6sO5Q0x1fLjSooI2OKz fNKXMkDMJBHkIEJlIIACdS3i+B8lBf43jgDWgWLt5j2yjEmKMWuymtuJpWFKv/1uVtqT u5y0BUcXxNAR/l0wcBdiFgVhVG19WLoQHN86rZADZXps1bjP9dlnQyQKHBZgi/xzB7db AkR4fWLiDUV7P3qW4a9UlTKdiym1/iHFa+fgps5d95DFx8Nkqk2E5qC4EzPC6ulp1CWa LHEA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to:date :to:from:message-id:dkim-signature:delivered-to; bh=jKXNZqlvVUqqHS5QaCk3t3jxteeySVCZYQzf+vQ7zSU=; b=vsmDXLGtPRUU/BOEZK8qTJmPY+c3RXwGYyG4Mfy8Lc96L116fdVsNLGxvdRMqiQB2y 35c1P/q85dfro6Cndu+pyj2Z8gFi5v5Yc317ZRkQ2hguBVY1ZATZvyjeBEUB7pcYFgRB uFxHB1s0ZXxCKNdfqSto4WsMKLe1oGaYATR93iVdXxiqZhUYnimK6bl33lXZzx1Zls+T wFK9Op6L1nUlMvtrledWsE7ZJeyBiLNdaYmCFXxdVDo5/+uUsIoyzWheAMQ2UuPIzxmv WBG7QA7JCW2INHLpdjgwaiOFFr7NIg1t97RnDHzoLfq7uo7fckgB69bfPOy6ExnYD86Z N0dg== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@foxmail.com header.s=s201512 header.b=KfC4U7wN; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=foxmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id sg39-20020a170907a42700b00780a240cfd7si7203099ejc.493.2022.11.19.22.51.39; Sat, 19 Nov 2022 22:51:39 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@foxmail.com header.s=s201512 header.b=KfC4U7wN; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=foxmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 95F3A68BC05; Sun, 20 Nov 2022 08:50:51 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from out162-62-58-216.mail.qq.com (out162-62-58-216.mail.qq.com [162.62.58.216]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 4E76968AED6 for ; Sun, 20 Nov 2022 08:50:44 +0200 (EET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=foxmail.com; s=s201512; t=1668927042; bh=aWQzvDj42aZ9vae0EUxLYZ2w2w2+N1oUXwzJ0MHmBM8=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=KfC4U7wN9y3j0JGWNpKpyaLgFTjn/akJZ/IIFAGnwN6920ozmr8IlyyJzSe5JiN6e 3iLlJ1to31Q6R4+2B+3DZniJ7Tn1c+xkVJGiorgv2H9p0nzrEZzm3wWsE2CryzNoV+ zG/9Z9yXHnwXc9DQGfTiJwWTSNMDub7frEZiUqTk= Received: from localhost.localdomain ([14.153.77.200]) by newxmesmtplogicsvrszc2-1.qq.com (NewEsmtp) with SMTP id C8F0626E; Sun, 20 Nov 2022 14:50:15 +0800 X-QQ-mid: xmsmtpt1668927041t9y0sd72f Message-ID: X-QQ-XMAILINFO: NMmJpeSXIGQNdjYmKfPWH/onm9cICs75ftvlb+wTwj3CT4X8aflrfYce4+Rjg8 3XIorrrMD1YqMp058htGm65s8F+JzOFc9qzRbaaXtx4tzFiKcTNdi8eDisQgh6yA+NNIqXJ8sF9U LYVr2YYTWwhx7m4VC3I+GE1LhkFMirhfCx/Ir+7Wl+WYc2M7tQpitK7DrWXpqUMSMx6jLDvvPANl W7AKGvHenwotNTPJq6b5+P4Pqedg+l3EoKGJXA5Esxvb6ntP6tG66LWy9egJeyV8MoKlCvEBHQjY RE+mFmGX1CcVMw9NqLb8GpYDUv1YqEpLs2zwYk31KQboFZny41CH1U4mR7dsf2t9GSteZEfAcOTh 9mA799BTr7xXpXhqzBMssObJ29hF6dJtDa9IcTrQdQWVqIKCKA8Ewajpr73XqgFsP/nbgvidI+GQ oKx9p6Mbr/D+0QSoNoYI45P5Bp7tHYBeq5RZArI9bpjAbclyiur8qtrAUMgSDAHVj/VWFqufEyoT gWUNOOCyK98beWhHJWGexSRzQNXu4DStzr93gVYtyEsNx613+xF3OMVOOwqxbNBMTkq5xjlsH6rY LhjdIUvHR9C4sBdWO5g0Z7zYo2cC2SKFKf/K1Zrqbmzz+9m7eEI0k65qL7R46a7ERFOAGXBloLuW jWsNMPhXaWPrU0eH4QZLtK+rjPLH58FtqYj/lJIKxBNK+270bQvruirXTgUaaewROXMkWLk9WDYF FR881s52/nDNo0OuKGFQKxWhZcfdHo1Ka2FGgdcaNsNGSKpZ0Rd5B5q7MzVZ90Hvq+4MwieO6FiD JyUvAJx38v0PZvbkUizTSbSTCJ5iepuNVsqV/n7YJkjsyg60jbOEpV3DOFRu1fgIR6bsR8tmPBdG 0Z5p3qjA1hXnxJ76mvy19ebVwsbetdu/LPfQxIgvEC0sXWOTtYvVQEagPG5RmoPvb1/IW1OjToxu 1CzGPHiPwDiE/kbvzhhPC1jtz+s4N2mzs/vAzfvCH2mrqpb56G1pt9mGtkctBv4VeBpNP4kGs= From: Zhao Zhili To: ffmpeg-devel@ffmpeg.org Date: Sun, 20 Nov 2022 14:49:46 +0800 X-OQ-MSGID: <20221120064947.94540-12-quinkblack@foxmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221120064947.94540-1-quinkblack@foxmail.com> References: <20221120064947.94540-1-quinkblack@foxmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 11/12] avutil/hwcontext: verify hw_frames_ctx in transfer_data_alloc X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: matthieu.bouron@stupeflix.com, Zhao Zhili , aman@tmm1.net Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: V3d284R2+0Re From: Zhao Zhili --- libavutil/hwcontext.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/libavutil/hwcontext.c b/libavutil/hwcontext.c index ab9ad3703e..3396598269 100644 --- a/libavutil/hwcontext.c +++ b/libavutil/hwcontext.c @@ -397,10 +397,14 @@ int av_hwframe_transfer_get_formats(AVBufferRef *hwframe_ref, static int transfer_data_alloc(AVFrame *dst, const AVFrame *src, int flags) { - AVHWFramesContext *ctx = (AVHWFramesContext*)src->hw_frames_ctx->data; + AVHWFramesContext *ctx; AVFrame *frame_tmp; int ret = 0; + if (!src->hw_frames_ctx) + return AVERROR(EINVAL); + ctx = (AVHWFramesContext*)src->hw_frames_ctx->data; + frame_tmp = av_frame_alloc(); if (!frame_tmp) return AVERROR(ENOMEM);