From patchwork Tue Jun 13 06:21:51 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: "LongChair ." X-Patchwork-Id: 3961 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.103.22.4 with SMTP id 4csp198549vsw; Mon, 12 Jun 2017 23:22:04 -0700 (PDT) X-Received: by 10.28.26.18 with SMTP id a18mr9788400wma.34.1497334924514; Mon, 12 Jun 2017 23:22:04 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1497334924; cv=none; d=google.com; s=arc-20160816; b=s/RDPpEJXrDiSZa4Gr7MJZirTmeHEXg0VtO6EmYS/jP1IsYcEAMDdzSKqT0hcWAOiM No+F62iCKNRl0j/k2C22qsaIw6FCNil/VljGBklXlNGTW40r7weFYGi301SrIS5hzlMU HlFmTgOHR3fdL2rs4UC+58VDxLyJfoqz46xZKwKNFP42gMCM+MZUBo6RohZGBotwPMvd fjDVnJvTlrgBp3Mb0fe5992b7XowMeBTRWffwFuiJAu+M4ZQ2Q9wfSIWEr5B9GHuNJ75 kWqb0eblXUpmg8+fWkpVxDGyMa6Yp1YTZILevCHqzo9mQQjHGmD5VwC2MPc8tTkc1JX9 T15w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:content-id :spamdiagnosticmetadata:spamdiagnosticoutput:content-language :accept-language:in-reply-to:references:message-id:date:thread-index :thread-topic:to:from:dkim-signature:delivered-to :arc-authentication-results; bh=AZca1jbi3SA5XtvPuErMWxlTzmqGYmh4JmpT4/0zZZE=; b=Xa27LHaKgxl5egA6fYFMLXGztOG+UQ13knPPtOQvafKa04HNPEX4Yhxdj2sHRLv+8/ XiL1gqnvFqFaq9K9UqdI8XYxSvVxMZrAXH7wlRZ+Hi/mWxUhfx5+UlIN7FNB6LUyRKKo 731G0fXRucN7gYdEwdm8SZmxISGLaUsyAjdUZh9wr1E3FZKwMePlDK5UubjZJx2qgR9V F7ecnuZMpWoVr/wqpEiNnl7OghvDuxr7eaiRMW63L6LRYyb2Lc3DciwmaGO9E/wb7D8Z Hm9VLiCOAbHZN5ZznARrC0p2V+WWKHN/k/LBS2kszQFkJn9E6+ic4thnup7KiHBlAM3f hoQQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@hotmail.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=hotmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id u74si1732359wrb.292.2017.06.12.23.22.03; Mon, 12 Jun 2017 23:22:04 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@hotmail.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=hotmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 65A4B68A0F5; Tue, 13 Jun 2017 09:21:58 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from NAM02-CY1-obe.outbound.protection.outlook.com (mail-oln040092004065.outbound.protection.outlook.com [40.92.4.65]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id E81C1689FE6 for ; Tue, 13 Jun 2017 09:21:51 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hotmail.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=dGT2PQz+0XcRdTu8FF0D+ae6TuHg4djhfJHFCdzzT5s=; b=Y1UjlSMejr3L/7jElXED5NkqMF2U/MnokBfgYffwtsEqZ4KIUlzdFwrzEWGh+8TY6Cbvt4+FVFlaWDtsaFfriiLgCk1XY5FN+SWOFoceHgUMeOdhHuAEH24tit+v5UmuMc3bLG6KApnPM7ZlouHkZgg8um0APzMRRJSvDWNGglVwsNhAsJQebgHQ/KKiuGVy8Geyr4/FJjorfObOqImLzgHA9c1PllQltLWHf+OgD31N78JUoZK8MQsO9E/J0NhJiBkaUXdDuqKLkOEyhgNKvXKqrcz1iHMIAfrjPSRdLmJc1nmkfyPwU+znb/xgoFB+D23cgb3Y9M0GT0VeBqlXcA== Received: from BL2NAM02FT056.eop-nam02.prod.protection.outlook.com (10.152.76.55) by BL2NAM02HT201.eop-nam02.prod.protection.outlook.com (10.152.76.251) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.1143.11; Tue, 13 Jun 2017 06:21:51 +0000 Received: from CY4PR1701MB1782.namprd17.prod.outlook.com (10.152.76.55) by BL2NAM02FT056.mail.protection.outlook.com (10.152.77.221) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1143.11 via Frontend Transport; Tue, 13 Jun 2017 06:21:51 +0000 Received: from CY4PR1701MB1782.namprd17.prod.outlook.com ([10.171.211.141]) by CY4PR1701MB1782.namprd17.prod.outlook.com ([10.171.211.141]) with mapi id 15.01.1157.017; Tue, 13 Jun 2017 06:21:51 +0000 From: "LongChair ." To: "ffmpeg-devel@ffmpeg.org" Thread-Topic: [PATCH] Add support for RockChip Media Process Platform Thread-Index: AQHS5A1Y0q0vXe6ur0ixOniUeaZO1g== Date: Tue, 13 Jun 2017 06:21:51 +0000 Message-ID: References: In-Reply-To: Accept-Language: fr-FR, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: ffmpeg.org; dkim=none (message not signed) header.d=none; ffmpeg.org; dmarc=none action=none header.from=hotmail.com; x-incomingtopheadermarker: OriginalChecksum:4C27E341C19F24A8CD4B41B2F9D40D5EDDBC8523D8F7B98920547146F9DD55FD; UpperCasedChecksum:993898DEBA9489F21359612112C48EEE743170E8C2E45F50BB4BDE9BDB8EBA44; SizeAsReceived:7360; Count:46 x-ms-exchange-messagesentrepresentingtype: 1 x-tmn: [Gi+FR7gKUgSqDCOOdPX5UzNmsC/C3xqn] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; BL2NAM02HT201; 24:6A6bY+vz1zM+VFbWwA1hUXoKqFzDvHDYzb7STCrnjP7pgRm1/Dji9pPGnT1DcHNsnX2R4aEzmMHDFMo0vk72CAP29HVZnmFwFRjw3oep/1U=; 7:HBN7xHpFq3TLwEPN0CWdSpAUQAB+dFMapDcb9876ezU/mzrXiUfI7KoAmSDVzX+Y6RFabe2YV9xlSmvG7H3Zrj31KFm14C/oMvmAj3vqeuD5Pj75gODWrrhhEXblXF4YbuDpwwGC69lrbCIAPkBipuz0BzuV1tbCZIa/Pi92TIoFYDivfoO4D0WZkGSPWfSVm79MtqFuFIryXLnlaSnsDVMeq4rwt8/BEUU3GwRchr25+4qspkniU+CjPCSHgNIvzS0UM3h/VEVFQpudHBbZ/RbETpH0BQcYswUGYUn0WOAPHPqAgfHwUi4h5iLfuPnO x-incomingheadercount: 46 x-eopattributedmessage: 0 x-forefront-antispam-report: EFV:NLI; SFV:NSPM; SFS:(7070007)(98901004); DIR:OUT; SFP:1901; SCL:1; SRVR:BL2NAM02HT201; H:CY4PR1701MB1782.namprd17.prod.outlook.com; FPR:; SPF:None; LANG:en; x-ms-traffictypediagnostic: BL2NAM02HT201: x-ms-office365-filtering-correlation-id: 4f0c5e7d-5ae9-45dd-20a5-08d4b2247a4d x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001)(201702061074)(5061506573)(5061507331)(1603103135)(2017031320274)(2017031324274)(2017031323274)(201702181274)(2017031322274)(1601125374)(1603101448)(1701031045); SRVR:BL2NAM02HT201; x-ms-exchange-slblob-mailprops: pxu8rxgqUgjZkMFXurSOOejn5RLHgvGdjjmv3K2ERTiEyrE0B9MrJo7b3GJ8z2lnTWiqU3RNlwDviOkjRnq59Fk4Ha1Bgl0hQRK9GcGRMJIdXTDXldVtqhBFsveWvdqH9yzoln2K5cpRniGUAhU1uuJQBKuWag3Wl0Pgw3/wmYHs3IhGnerhLWt9ZyM2x5coDRKgRv5sw2uGtFCQ6fODZ18eJgjlbC25jpAF41fdNVCw2lUR1ltNlIW+Gj0enpxXbryuQ2wbIZpg7fRP2pvh1XralILTTxYC2OIpAep5JFAnSRHVLwqpQslgZuDzV6EPBLPbZcjkUwu9WvLeZJ4QMD16oLh6V2QpoP5Uw3zJkPzuD1gQLMbHMrDD4Ua+Vo8hQly0DkQ1NfQdHpKwt3DPoE/j1XjXMeE7YgrgjvPx95FMevSwgoEE0rC3waKAwZrAEU/WqG1Qp5MEZgDvg3OKYQ/YWEfcPESjO5ngnOtGvENam6OfPQO+rbb+i487zeCN9LMBZPz3YkVV0nA1gLh8bpxLpilMD6eFBVrrht0ZOQwZ7V7E4LfRozV7W9Gh7yjr14TFDF/mSbSqUZZQv5E1ctoupwok9ph7ZKfEw8PqwlAxjo2jDdNk/hM8DepMXG3JPfBMERH2x/+64cElmn8wJHeWUYrRvboScdMsvujQe72ABrK+9evT/oDfyoie9A6oCWHotP4+dPo1G24osRa1Og== x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(444000031); SRVR:BL2NAM02HT201; BCL:0; PCL:0; RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095); SRVR:BL2NAM02HT201; x-forefront-prvs: 0337AFFE9A spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-ID: MIME-Version: 1.0 X-OriginatorOrg: hotmail.com X-MS-Exchange-CrossTenant-originalarrivaltime: 13 Jun 2017 06:21:51.0899 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Internet X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL2NAM02HT201 Subject: [FFmpeg-devel] [PATCH] Add support for RockChip Media Process Platform X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: "LongChair ." Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" From: LongChair This adds hardware decoding for h264 / HEVC / VP8 using MPP Rockchip API. Will return frames holding a av_drmprime struct in buf[3} that allows drm / dmabuf usage. Was tested on RK3288 (TinkerBoard) and RK3328. Additions from patch v1 - Change AV_PIX_FMT_RKMPP to AV_PIX_FMT_DRMPRIME as any decoder able to output drmprime structures could use that pixel format. --- Changelog | 1 + configure | 12 ++ libavcodec/Makefile | 5 + libavcodec/allcodecs.c | 6 + libavcodec/drmprime.h | 17 ++ libavcodec/rkmppdec.c | 523 +++++++++++++++++++++++++++++++++++++++++++++++++ libavutil/pixdesc.c | 4 + libavutil/pixfmt.h | 5 + 8 files changed, 573 insertions(+) create mode 100644 libavcodec/drmprime.h create mode 100644 libavcodec/rkmppdec.c -- 2.7.4 diff --git a/Changelog b/Changelog index 3533bdc..498e433 100644 --- a/Changelog +++ b/Changelog @@ -52,6 +52,7 @@ version 3.3: - Removed asyncts filter (use af_aresample instead) - Intel QSV-accelerated VP8 video decoding - VAAPI-accelerated deinterlacing +- Addition of Rockchip MPP harware decoding version 3.2: diff --git a/configure b/configure index 4ec8f21..883fc84 100755 --- a/configure +++ b/configure @@ -304,6 +304,7 @@ External library support: --disable-nvenc disable Nvidia video encoding code [autodetect] --enable-omx enable OpenMAX IL code [no] --enable-omx-rpi enable OpenMAX IL code for Raspberry Pi [no] + --enable-rkmpp enable Rockchip Media Process Platform code [no] --disable-vaapi disable Video Acceleration API (mainly Unix/Intel) code [autodetect] --disable-vda disable Apple Video Decode Acceleration code [autodetect] --disable-vdpau disable Nvidia Video Decode and Presentation API for Unix code [autodetect] @@ -1607,6 +1608,7 @@ HWACCEL_LIBRARY_LIST=" libmfx mmal omx + rkmpp " DOCUMENT_LIST=" @@ -2616,6 +2618,7 @@ h264_dxva2_hwaccel_select="h264_decoder" h264_mediacodec_hwaccel_deps="mediacodec" h264_mmal_hwaccel_deps="mmal" h264_qsv_hwaccel_deps="libmfx" +h264_rkmpp_hwaccel_deps="rkmpp" h264_vaapi_hwaccel_deps="vaapi" h264_vaapi_hwaccel_select="h264_decoder" h264_vda_hwaccel_deps="vda" @@ -2634,6 +2637,7 @@ hevc_mediacodec_hwaccel_deps="mediacodec" hevc_dxva2_hwaccel_deps="dxva2 DXVA_PicParams_HEVC" hevc_dxva2_hwaccel_select="hevc_decoder" hevc_qsv_hwaccel_deps="libmfx" +hevc_rkmpp_hwaccel_deps="rkmpp" hevc_vaapi_hwaccel_deps="vaapi VAPictureParameterBufferHEVC" hevc_vaapi_hwaccel_select="hevc_decoder" hevc_vdpau_hwaccel_deps="vdpau VdpPictureInfoHEVC" @@ -2696,6 +2700,7 @@ vp9_cuvid_hwaccel_deps="cuda cuvid" vp9_cuvid_hwaccel_select="vp9_cuvid_decoder" vp8_mediacodec_hwaccel_deps="mediacodec" vp8_qsv_hwaccel_deps="libmfx" +vp8_rkmpp_hwaccel_deps="rkmpp" vp9_d3d11va_hwaccel_deps="d3d11va DXVA_PicParams_VP9" vp9_d3d11va_hwaccel_select="vp9_decoder" vp9_dxva2_hwaccel_deps="dxva2 DXVA_PicParams_VP9" @@ -2736,6 +2741,8 @@ h264_qsv_decoder_deps="libmfx" h264_qsv_decoder_select="h264_mp4toannexb_bsf h264_parser qsvdec h264_qsv_hwaccel" h264_qsv_encoder_deps="libmfx" h264_qsv_encoder_select="qsvenc" +h264_rkmpp_decoder_deps="rkmpp" +h264_rkmpp_decoder_select="h264_mp4toannexb_bsf rkmpp h264_rkmpp_hwaccel" h264_vaapi_encoder_deps="VAEncPictureParameterBufferH264" h264_vaapi_encoder_select="vaapi_encode golomb" h264_vda_decoder_deps="vda" @@ -2751,6 +2758,8 @@ hevc_qsv_decoder_deps="libmfx" hevc_qsv_decoder_select="hevc_mp4toannexb_bsf hevc_parser qsvdec hevc_qsv_hwaccel" hevc_qsv_encoder_deps="libmfx" hevc_qsv_encoder_select="hevcparse qsvenc" +hevc_rkmpp_decoder_deps="rkmpp" +hevc_rkmpp_decoder_select="hevc_mp4toannexb_bsf rkmpp hevc_rkmpp_hwaccel" hevc_vaapi_encoder_deps="VAEncPictureParameterBufferHEVC" hevc_vaapi_encoder_select="vaapi_encode golomb" mjpeg_cuvid_decoder_deps="cuda cuvid" @@ -2789,6 +2798,8 @@ vp8_cuvid_decoder_deps="cuda cuvid" vp8_mediacodec_decoder_deps="mediacodec" vp8_qsv_decoder_deps="libmfx" vp8_qsv_decoder_select="qsvdec vp8_qsv_hwaccel vp8_parser" +vp8_rkmpp_decoder_deps="rkmpp" +vp8_rkmpp_decoder_select="rkmpp vp8_rkmpp_hwaccel" vp8_vaapi_encoder_deps="VAEncPictureParameterBufferVP8" vp8_vaapi_encoder_select="vaapi_encode" vp9_cuvid_decoder_deps="cuda cuvid" @@ -5917,6 +5928,7 @@ enabled mmal && { check_lib mmal interface/mmal/mmal.h mmal_port_co check_lib mmal interface/mmal/mmal.h mmal_port_connect -lmmal_core -lmmal_util -lmmal_vc_client -lbcm_host; } || die "ERROR: mmal not found" && check_func_headers interface/mmal/mmal.h "MMAL_PARAMETER_VIDEO_MAX_NUM_CALLBACKS"; } +enabled rkmpp && { check_lib rkmpp rockchip/rk_mpi.h mpp_create "-lrockchip_mpp" || die "ERROR : Rockchip MPP was not found."; } enabled netcdf && require_pkg_config netcdf netcdf.h nc_inq_libvers enabled openal && { { for al_extralibs in "${OPENAL_LIBS}" "-lopenal" "-lOpenAL32"; do check_lib openal 'AL/al.h' alGetError "${al_extralibs}" && break; done } || diff --git a/libavcodec/Makefile b/libavcodec/Makefile index 0818950..0b1ebd0 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -6,6 +6,7 @@ HEADERS = avcodec.h \ avfft.h \ d3d11va.h \ dirac.h \ + drmprime.h \ dv_profile.h \ dxva2.h \ jni.h \ @@ -19,6 +20,7 @@ HEADERS = avcodec.h \ vorbis_parser.h \ xvmc.h \ + OBJS = allcodecs.o \ audioconvert.o \ avdct.o \ @@ -334,6 +336,7 @@ OBJS-$(CONFIG_H264_VDA_DECODER) += vda_h264_dec.o OBJS-$(CONFIG_H264_OMX_ENCODER) += omx.o OBJS-$(CONFIG_H264_QSV_DECODER) += qsvdec_h2645.o OBJS-$(CONFIG_H264_QSV_ENCODER) += qsvenc_h264.o +OBJS-$(CONFIG_H264_RKMPP_DECODER) += rkmppdec.o OBJS-$(CONFIG_H264_VAAPI_ENCODER) += vaapi_encode_h264.o vaapi_encode_h26x.o OBJS-$(CONFIG_H264_VIDEOTOOLBOX_ENCODER) += videotoolboxenc.o OBJS-$(CONFIG_HAP_DECODER) += hapdec.o hap.o @@ -348,6 +351,7 @@ OBJS-$(CONFIG_NVENC_HEVC_ENCODER) += nvenc_hevc.o OBJS-$(CONFIG_HEVC_QSV_DECODER) += qsvdec_h2645.o OBJS-$(CONFIG_HEVC_QSV_ENCODER) += qsvenc_hevc.o hevc_ps_enc.o \ hevc_data.o +OBJS-$(CONFIG_HEVC_RKMPP_DECODER) += rkmppdec.o OBJS-$(CONFIG_HEVC_VAAPI_ENCODER) += vaapi_encode_h265.o vaapi_encode_h26x.o OBJS-$(CONFIG_HNM4_VIDEO_DECODER) += hnm4video.o OBJS-$(CONFIG_HQ_HQA_DECODER) += hq_hqa.o hq_hqadata.o hq_hqadsp.o \ @@ -618,6 +622,7 @@ OBJS-$(CONFIG_VP8_DECODER) += vp8.o vp56rac.o OBJS-$(CONFIG_VP8_CUVID_DECODER) += cuvid.o OBJS-$(CONFIG_VP8_MEDIACODEC_DECODER) += mediacodecdec.o OBJS-$(CONFIG_VP8_QSV_DECODER) += qsvdec_other.o +OBJS-$(CONFIG_VP8_RKMPP_DECODER) += rkmppdec.o OBJS-$(CONFIG_VP8_VAAPI_ENCODER) += vaapi_encode_vp8.o OBJS-$(CONFIG_VP9_DECODER) += vp9.o vp9data.o vp9dsp.o vp9lpf.o vp9recon.o \ vp9block.o vp9prob.o vp9mvs.o vp56rac.o \ diff --git a/libavcodec/allcodecs.c b/libavcodec/allcodecs.c index 89fadcd..95d90fb 100644 --- a/libavcodec/allcodecs.c +++ b/libavcodec/allcodecs.c @@ -70,6 +70,7 @@ static void register_all(void) REGISTER_HWACCEL(H264_MEDIACODEC, h264_mediacodec); REGISTER_HWACCEL(H264_MMAL, h264_mmal); REGISTER_HWACCEL(H264_QSV, h264_qsv); + REGISTER_HWACCEL(H264_RKMPP, h264_rkmpp); REGISTER_HWACCEL(H264_VAAPI, h264_vaapi); REGISTER_HWACCEL(H264_VDA, h264_vda); REGISTER_HWACCEL(H264_VDA_OLD, h264_vda_old); @@ -80,6 +81,7 @@ static void register_all(void) REGISTER_HWACCEL(HEVC_DXVA2, hevc_dxva2); REGISTER_HWACCEL(HEVC_MEDIACODEC, hevc_mediacodec); REGISTER_HWACCEL(HEVC_QSV, hevc_qsv); + REGISTER_HWACCEL(HEVC_RKMPP, hevc_rkmpp); REGISTER_HWACCEL(HEVC_VAAPI, hevc_vaapi); REGISTER_HWACCEL(HEVC_VDPAU, hevc_vdpau); REGISTER_HWACCEL(MJPEG_CUVID, mjpeg_cuvid); @@ -112,6 +114,7 @@ static void register_all(void) REGISTER_HWACCEL(VP8_CUVID, vp8_cuvid); REGISTER_HWACCEL(VP8_MEDIACODEC, vp8_mediacodec); REGISTER_HWACCEL(VP8_QSV, vp8_qsv); + REGISTER_HWACCEL(VP8_RKMPP, vp8_rkmpp); REGISTER_HWACCEL(VP9_CUVID, vp9_cuvid); REGISTER_HWACCEL(VP9_D3D11VA, vp9_d3d11va); REGISTER_HWACCEL(VP9_DXVA2, vp9_dxva2); @@ -204,6 +207,7 @@ static void register_all(void) REGISTER_DECODER(H264_MEDIACODEC, h264_mediacodec); REGISTER_DECODER(H264_MMAL, h264_mmal); REGISTER_DECODER(H264_QSV, h264_qsv); + REGISTER_DECODER(H264_RKMPP, h264_rkmpp); REGISTER_DECODER(H264_VDA, h264_vda); #if FF_API_VDPAU REGISTER_DECODER(H264_VDPAU, h264_vdpau); @@ -211,6 +215,7 @@ static void register_all(void) REGISTER_ENCDEC (HAP, hap); REGISTER_DECODER(HEVC, hevc); REGISTER_DECODER(HEVC_QSV, hevc_qsv); + REGISTER_DECODER(HEVC_RKMPP, hevc_rkmpp); REGISTER_DECODER(HNM4_VIDEO, hnm4_video); REGISTER_DECODER(HQ_HQA, hq_hqa); REGISTER_DECODER(HQX, hqx); @@ -363,6 +368,7 @@ static void register_all(void) REGISTER_DECODER(VP6F, vp6f); REGISTER_DECODER(VP7, vp7); REGISTER_DECODER(VP8, vp8); + REGISTER_DECODER(VP8_RKMPP, vp8_rkmpp); REGISTER_DECODER(VP9, vp9); REGISTER_DECODER(VQA, vqa); REGISTER_DECODER(BITPACKED, bitpacked); diff --git a/libavcodec/drmprime.h b/libavcodec/drmprime.h new file mode 100644 index 0000000..44816db --- /dev/null +++ b/libavcodec/drmprime.h @@ -0,0 +1,17 @@ +#ifndef AVCODEC_DRMPRIME_H +#define AVCODEC_DRMPRIME_H + +#include + +#define AV_DRMPRIME_NUM_PLANES 4 // maximum number of planes + +typedef struct av_drmprime { + + int strides[AV_DRMPRIME_NUM_PLANES]; // stride in byte of the according plane + int offsets[AV_DRMPRIME_NUM_PLANES]; // offset from start in byte of the according plane + int fds[AV_DRMPRIME_NUM_PLANES]; // file descriptor for the plane, (0) if unused. + uint32_t format; // FOURC_CC drm format for the image + +} av_drmprime; + +#endif // AVCODEC_DRMPRIME_H diff --git a/libavcodec/rkmppdec.c b/libavcodec/rkmppdec.c new file mode 100644 index 0000000..5073b84 --- /dev/null +++ b/libavcodec/rkmppdec.c @@ -0,0 +1,523 @@ +/* + * RockChip MPP Video Decoder + * Copyright (c) 2017 Lionel CHAZALLON + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include +#include +#include +#include +#include +#include + +#include "avcodec.h" +#include "decode.h" +#include "drmprime.h" +#include "internal.h" +#include "libavutil/buffer.h" +#include "libavutil/common.h" +#include "libavutil/frame.h" +#include "libavutil/imgutils.h" +#include "libavutil/log.h" + +#define RECEIVE_FRAME_TIMEOUT 100 +#define FRAMEGROUP_MAX_FRAMES 16 + +typedef struct { + MppCtx ctx; + MppApi *mpi; + MppBufferGroup frame_group; + + char first_frame; + char first_packet; + char eos_reached; +} RKMPPDecoder; + +typedef struct { + AVClass *av_class; + AVBufferRef *decoder_ref; +} RKMPPDecodeContext; + +typedef struct { + MppFrame frame; + AVBufferRef *decoder_ref; +} RKMPPFrameContext; + +static MppCodingType ffrkmpp_get_codingtype(AVCodecContext *avctx) +{ + switch (avctx->codec_id) { + case AV_CODEC_ID_H264: return MPP_VIDEO_CodingAVC; + case AV_CODEC_ID_HEVC: return MPP_VIDEO_CodingHEVC; + case AV_CODEC_ID_VP8: return MPP_VIDEO_CodingVP8; + default: return MPP_VIDEO_CodingUnused; + } +} + +static int ffrkmpp_get_frameformat(MppFrameFormat mppformat) +{ + switch (mppformat) { + case MPP_FMT_YUV420SP: return DRM_FORMAT_NV12; +#ifdef DRM_FORMAT_NV12_10 + case MPP_FMT_YUV420SP_10BIT: return DRM_FORMAT_NV12_10; +#endif + default: return 0; + } +} + +static int ffrkmpp_write_data(AVCodecContext *avctx, uint8_t *buffer, int size, int64_t pts) +{ + RKMPPDecodeContext *rk_context = avctx->priv_data; + RKMPPDecoder *decoder = (RKMPPDecoder *)rk_context->decoder_ref->data; + MPP_RET ret = MPP_NOK; + MppPacket packet; + + // create the MPP packet + ret = mpp_packet_init(&packet, buffer, size); + if (ret != MPP_OK) { + av_log(avctx, AV_LOG_ERROR, "Failed to init MPP packet (code = %d)\n", ret); + return AVERROR_UNKNOWN; + } + + mpp_packet_set_pts(packet, pts); + + if (!buffer) + mpp_packet_set_eos(packet); + + ret = decoder->mpi->decode_put_packet(decoder->ctx, packet); + if (ret != MPP_OK) { + if (ret == MPP_ERR_BUFFER_FULL) { + av_log(avctx, AV_LOG_DEBUG, "Buffer full writing %d bytes to decoder\n", size); + ret = AVERROR(EAGAIN); + } else + ret = AVERROR_UNKNOWN; + } + else + av_log(avctx, AV_LOG_DEBUG, "Wrote %d bytes to decoder\n", size); + + mpp_packet_deinit(&packet); + + return ret; +} + +static int ffrkmpp_close_decoder(AVCodecContext *avctx) +{ + RKMPPDecodeContext *rk_context = avctx->priv_data; + av_buffer_unref(&rk_context->decoder_ref); + return 0; +} + +static void ffrkmpp_release_decoder(void *opaque, uint8_t *data) +{ + RKMPPDecoder *decoder = (RKMPPDecoder *)data; + + decoder->mpi->reset(decoder->ctx); + mpp_destroy(decoder->ctx); + decoder->ctx = NULL; + + if (decoder->frame_group) { + mpp_buffer_group_put(decoder->frame_group); + decoder->frame_group = NULL; + } + + av_free(decoder); +} + +static int ffrkmpp_init_decoder(AVCodecContext *avctx) +{ + RKMPPDecodeContext *rk_context = avctx->priv_data; + RKMPPDecoder *decoder = NULL; + MppCodingType codectype = MPP_VIDEO_CodingUnused; + MPP_RET ret = MPP_NOK; + RK_S64 paramS64; + RK_S32 paramS32; + + if ((ret = ff_get_format(avctx, avctx->codec->pix_fmts)) < 0) + return ret; + + avctx->pix_fmt = ret; + + // create a decoder and a ref to it + decoder = av_mallocz(sizeof(RKMPPDecoder)); + if (!decoder) { + ret = AVERROR(ENOMEM); + goto fail; + } + + rk_context->decoder_ref = av_buffer_create((uint8_t *)decoder, sizeof(*decoder), ffrkmpp_release_decoder, + NULL, AV_BUFFER_FLAG_READONLY); + if (!rk_context->decoder_ref) { + av_free(decoder); + ret = AVERROR(ENOMEM); + goto fail; + } + + av_log(avctx, AV_LOG_DEBUG, "Initializing RKMPP decoder.\n"); + + codectype = ffrkmpp_get_codingtype(avctx); + if (codectype == MPP_VIDEO_CodingUnused) { + av_log(avctx, AV_LOG_ERROR, "Unknown codec type (%d).\n", avctx->codec_id); + ret = AVERROR_UNKNOWN; + goto fail; + } + + // Create the MPP context + ret = mpp_create(&decoder->ctx, &decoder->mpi); + if (ret != MPP_OK) { + av_log(avctx, AV_LOG_ERROR, "Failed to create MPP context (code = %d).\n", ret); + ret = AVERROR_UNKNOWN; + goto fail; + } + + // initialize mpp + ret = mpp_init(decoder->ctx, MPP_CTX_DEC, codectype); + if (ret != MPP_OK) { + av_log(avctx, AV_LOG_ERROR, "Failed to initialize MPP context (code = %d).\n", ret); + ret = AVERROR_UNKNOWN; + goto fail; + } + + // make decode calls blocking with a timeout + paramS32 = MPP_POLL_BLOCK; + ret = decoder->mpi->control(decoder->ctx, MPP_SET_OUTPUT_BLOCK, ¶mS32); + if (ret != MPP_OK) { + av_log(avctx, AV_LOG_ERROR, "Failed to set blocking mode on MPI (code = %d).\n", ret); + ret = AVERROR_UNKNOWN; + goto fail; + } + + paramS64 = RECEIVE_FRAME_TIMEOUT; + ret = decoder->mpi->control(decoder->ctx, MPP_SET_OUTPUT_BLOCK_TIMEOUT, ¶mS64); + if (ret != MPP_OK) { + av_log(avctx, AV_LOG_ERROR, "Failed to set block timeout on MPI (code = %d).\n", ret); + ret = AVERROR_UNKNOWN; + goto fail; + } + + ret = mpp_buffer_group_get_internal(&decoder->frame_group, MPP_BUFFER_TYPE_ION); + if (ret) { + av_log(avctx, AV_LOG_ERROR, "Failed to retrieve buffer group (code = %d)\n", ret); + ret = AVERROR_UNKNOWN; + goto fail; + } + + ret = decoder->mpi->control(decoder->ctx, MPP_DEC_SET_EXT_BUF_GROUP, decoder->frame_group); + if (ret) { + av_log(avctx, AV_LOG_ERROR, "Failed to assign buffer group (code = %d)\n", ret); + ret = AVERROR_UNKNOWN; + goto fail; + } + + ret = mpp_buffer_group_limit_config(decoder->frame_group, 0, FRAMEGROUP_MAX_FRAMES); + if (ret) { + av_log(avctx, AV_LOG_ERROR, "Failed to set buffer group limit (code = %d)\n", ret); + ret = AVERROR_UNKNOWN; + goto fail; + } + + decoder->first_packet = 1; + + av_log(avctx, AV_LOG_DEBUG, "RKMPP decoder initialized successfully.\n"); + return 0; + +fail: + av_log(avctx, AV_LOG_ERROR, "Failed to initialize RKMPP decoder.\n"); + ffrkmpp_close_decoder(avctx); + return ret; +} + +static int ffrkmpp_send_packet(AVCodecContext *avctx, const AVPacket *avpkt) +{ + RKMPPDecodeContext *rk_context = avctx->priv_data; + RKMPPDecoder *decoder = (RKMPPDecoder *)rk_context->decoder_ref->data; + MPP_RET ret = MPP_NOK; + + // handle EOF + if (!avpkt->size) { + av_log(avctx, AV_LOG_DEBUG, "End of stream.\n"); + decoder->eos_reached = 1; + ret = ffrkmpp_write_data(avctx, NULL, 0, 0); + if (ret) + av_log(avctx, AV_LOG_ERROR, "Failed to send EOS to decoder (code = %d)\n", ret); + return ret; + } + + // on first packet, send extradata + if (decoder->first_packet) { + if (avctx->extradata_size) { + ret = ffrkmpp_write_data(avctx, avctx->extradata, + avctx->extradata_size, + avpkt->pts); + if (ret) { + av_log(avctx, AV_LOG_ERROR, "Failed to write extradata to decoder (code = %d)\n", ret); + return ret; + } + } + decoder->first_packet = 0; + } + + // now send packet + ret = ffrkmpp_write_data(avctx, avpkt->data, avpkt->size, avpkt->pts); + if (ret) + av_log(avctx, AV_LOG_ERROR, "Failed to write data to decoder (code = %d)\n", ret); + + return ret; +} + +static void ffrkmpp_release_frame(void *opaque, uint8_t *data) +{ + av_drmprime *primedata = (av_drmprime *)data; + AVBufferRef *framecontextref = (AVBufferRef *)opaque; + RKMPPFrameContext *framecontext = (RKMPPFrameContext *)framecontextref->data; + + mpp_frame_deinit(&framecontext->frame); + av_buffer_unref(&framecontext->decoder_ref); + av_buffer_unref(&framecontextref); + + av_free(primedata); +} + +static int ffrkmpp_retrieve_frame(AVCodecContext *avctx, AVFrame *frame) +{ + RKMPPDecodeContext *rk_context = avctx->priv_data; + RKMPPDecoder *decoder = (RKMPPDecoder *)rk_context->decoder_ref->data; + RKMPPFrameContext *framecontext = NULL; + AVBufferRef *framecontextref = NULL; + MPP_RET ret = MPP_NOK; + MppFrame mppframe = NULL; + MppBuffer buffer = NULL; + av_drmprime *primedata = NULL; + int retrycount = 0; + + // on start of decoding, MPP can return -1, which is supposed to be expected + // this is due to some internal MPP init which is not completed, that will + // only happen in the first few frames queries, but should not be interpreted + // as an error, Therefore we need to retry a couple times when we get -1 + // in order to let it time to complete it's init, then we sleep a bit between retries. +retry_get_frame: + ret = decoder->mpi->decode_get_frame(decoder->ctx, &mppframe); + if (ret != MPP_OK && ret != MPP_ERR_TIMEOUT) { + if (retrycount < 5) { + av_log(avctx, AV_LOG_DEBUG, "Failed to get a frame, retrying (code = %d, retrycount = %d)\n", ret, retrycount); + usleep(10000); + retrycount++; + goto retry_get_frame; + } else { + av_log(avctx, AV_LOG_ERROR, "Failed to get a frame from MPP (code = %d)\n", ret); + goto fail; + } + } + + if (mppframe) { + // Check wether we have a special frame or not + if (mpp_frame_get_info_change(mppframe)) { + av_log(avctx, AV_LOG_INFO, "Decoder noticed an info change (%dx%d), format=%d\n", + (int)mpp_frame_get_width(mppframe), (int)mpp_frame_get_height(mppframe), + (int)mpp_frame_get_fmt(mppframe)); + + avctx->width = mpp_frame_get_width(mppframe); + avctx->height = mpp_frame_get_height(mppframe); + + decoder->mpi->control(decoder->ctx, MPP_DEC_SET_INFO_CHANGE_READY, NULL); + decoder->first_frame = 1; + + // here decoder is fully initialized, we need to feed it again with data + ret = AVERROR(EAGAIN); + goto fail; + } else if (mpp_frame_get_eos(mppframe)) { + av_log(avctx, AV_LOG_DEBUG, "Received a EOS frame.\n"); + decoder->eos_reached = 1; + ret = AVERROR_EOF; + goto fail; + } else if (mpp_frame_get_discard(mppframe) || mpp_frame_get_errinfo(mppframe)) { + av_log(avctx, AV_LOG_DEBUG, "Received a discard/errinfo frame.\n"); + ret = AVERROR(EAGAIN); + goto fail; + } + + // here we should have a valid frame + av_log(avctx, AV_LOG_DEBUG, "Received a frame.\n"); + + // setup general frame fields + frame->format = AV_PIX_FMT_DRMPRIME; + frame->width = mpp_frame_get_width(mppframe); + frame->height = mpp_frame_get_height(mppframe); + frame->pts = mpp_frame_get_pts(mppframe); + + // now setup the frame buffer info + buffer = mpp_frame_get_buffer(mppframe); + if (buffer) { + primedata = av_mallocz(sizeof(av_drmprime)); + if (!primedata) { + ret = AVERROR(ENOMEM); + goto fail; + } + + primedata->strides[0] = mpp_frame_get_hor_stride(mppframe); + primedata->strides[1] = primedata->strides[0]; + primedata->offsets[0] = 0; + primedata->offsets[1] = primedata->strides[0] * mpp_frame_get_ver_stride(mppframe); + primedata->fds[0] = mpp_buffer_get_fd(buffer); + primedata->fds[1] = primedata->fds[0]; + primedata->format = ffrkmpp_get_frameformat(mpp_frame_get_fmt(mppframe)); + + // we also allocate a struct in buf[0] that will allow to hold additionnal information + // for releasing properly MPP frames and decoder + framecontextref = av_buffer_allocz(sizeof(*framecontext)); + if (!framecontextref) { + ret = AVERROR(ENOMEM); + goto fail; + } + + // MPP decoder needs to be closed only when all frames have been released. + framecontext = (RKMPPFrameContext *)framecontextref->data; + framecontext->decoder_ref = av_buffer_ref(rk_context->decoder_ref); + framecontext->frame = mppframe; + + frame->data[3] = (uint8_t *)primedata; + frame->buf[0] = av_buffer_create((uint8_t *)primedata, sizeof(*primedata), ffrkmpp_release_frame, + framecontextref, AV_BUFFER_FLAG_READONLY); + + if (!frame->buf[0]) { + ret = AVERROR(ENOMEM); + goto fail; + } + + decoder->first_frame = 0; + return 0; + } else { + av_log(avctx, AV_LOG_ERROR, "Failed to retrieve the frame buffer, frame is dropped (code = %d)\n", ret); + mpp_frame_deinit(&mppframe); + } + } else if (decoder->eos_reached) { + return AVERROR_EOF; + } else if (ret == MPP_ERR_TIMEOUT) { + av_log(avctx, AV_LOG_DEBUG, "Timeout when trying to get a frame from MPP\n"); + } + + return AVERROR(EAGAIN); + +fail: + if (mppframe) + mpp_frame_deinit(&mppframe); + + if (framecontext) + av_buffer_unref(&framecontext->decoder_ref); + + if (framecontextref) + av_buffer_unref(&framecontextref); + + if (primedata) + av_free(primedata); + + return ret; +} + +static int ffrkmpp_receive_frame(AVCodecContext *avctx, AVFrame *frame) +{ + RKMPPDecodeContext *rk_context = avctx->priv_data; + RKMPPDecoder *decoder = (RKMPPDecoder *)rk_context->decoder_ref->data; + MPP_RET ret = MPP_NOK; + AVPacket pkt = {0}; + RK_S32 freeslots; + + if (!decoder->eos_reached) { + // we get the available slots in decoder + ret = decoder->mpi->control(decoder->ctx, MPP_DEC_GET_FREE_PACKET_SLOT_COUNT, &freeslots); + if (ret != MPP_OK) { + av_log(avctx, AV_LOG_ERROR, "Failed to get decoder free slots (code = %d).\n", ret); + return ret; + } + + if (freeslots > 0) { + ret = ff_decode_get_packet(avctx, &pkt); + if (ret < 0 && ret != AVERROR_EOF) { + return ret; + } + + ret = ffrkmpp_send_packet(avctx, &pkt); + av_packet_unref(&pkt); + + if (ret < 0) { + av_log(avctx, AV_LOG_ERROR, "Failed to send packet to decoder (code = %d)\n", ret); + return ret; + } + } + + // make sure we keep decoder full + if (freeslots > 1 && decoder->first_frame) + return AVERROR(EAGAIN); + } + + return ffrkmpp_retrieve_frame(avctx, frame); +} + +static void ffrkmpp_flush(AVCodecContext *avctx) +{ + RKMPPDecodeContext *rk_context = avctx->priv_data; + RKMPPDecoder *decoder = (RKMPPDecoder *)rk_context->decoder_ref->data; + MPP_RET ret = MPP_NOK; + + av_log(avctx, AV_LOG_DEBUG, "Flush.\n"); + + ret = decoder->mpi->reset(decoder->ctx); + if (ret == MPP_OK) { + decoder->first_frame = 1; + decoder->first_packet = 1; + } else + av_log(avctx, AV_LOG_ERROR, "Failed to reset MPI (code = %d)\n", ret); +} + +#define FFRKMPP_DEC_HWACCEL(NAME, ID) \ + AVHWAccel ff_##NAME##_rkmpp_hwaccel = { \ + .name = #NAME "_rkmpp", \ + .type = AVMEDIA_TYPE_VIDEO,\ + .id = ID, \ + .pix_fmt = AV_PIX_FMT_DRMPRIME,\ + }; + +#define FFRKMPP_DEC_CLASS(NAME) \ + static const AVClass ffrkmpp_##NAME##_dec_class = { \ + .class_name = "rkmpp_" #NAME "_dec", \ + .version = LIBAVUTIL_VERSION_INT, \ + }; + +#define FFRKMPP_DEC(NAME, ID, BSFS) \ + FFRKMPP_DEC_CLASS(NAME) \ + FFRKMPP_DEC_HWACCEL(NAME, ID) \ + AVCodec ff_##NAME##_rkmpp_decoder = { \ + .name = #NAME "_rkmpp", \ + .long_name = NULL_IF_CONFIG_SMALL(#NAME " (rkmpp)"), \ + .type = AVMEDIA_TYPE_VIDEO, \ + .id = ID, \ + .priv_data_size = sizeof(RKMPPDecodeContext), \ + .init = ffrkmpp_init_decoder, \ + .close = ffrkmpp_close_decoder, \ + .receive_frame = ffrkmpp_receive_frame, \ + .flush = ffrkmpp_flush, \ + .priv_class = &ffrkmpp_##NAME##_dec_class, \ + .capabilities = AV_CODEC_CAP_DELAY, \ + .caps_internal = FF_CODEC_CAP_SETS_PKT_DTS, \ + .pix_fmts = (const enum AVPixelFormat[]) { AV_PIX_FMT_DRMPRIME, \ + AV_PIX_FMT_NONE}, \ + .bsfs = #BSFS, \ + }; + +FFRKMPP_DEC(h264, AV_CODEC_ID_H264, h264_mp4toannexb) +FFRKMPP_DEC(hevc, AV_CODEC_ID_HEVC, hevc_mp4toannexb) +FFRKMPP_DEC(vp8, AV_CODEC_ID_VP8, ) diff --git a/libavutil/pixdesc.c b/libavutil/pixdesc.c index d4a7a8b..1eed823 100644 --- a/libavutil/pixdesc.c +++ b/libavutil/pixdesc.c @@ -2028,6 +2028,10 @@ static const AVPixFmtDescriptor av_pix_fmt_descriptors[AV_PIX_FMT_NB] = { .name = "cuda", .flags = AV_PIX_FMT_FLAG_HWACCEL, }, + [AV_PIX_FMT_DRMPRIME] = { + .name = "drmprime", + .flags = AV_PIX_FMT_FLAG_HWACCEL, + }, [AV_PIX_FMT_AYUV64LE] = { .name = "ayuv64le", .nb_components = 4, diff --git a/libavutil/pixfmt.h b/libavutil/pixfmt.h index 6b7eea8..5c128a4 100644 --- a/libavutil/pixfmt.h +++ b/libavutil/pixfmt.h @@ -248,6 +248,11 @@ enum AVPixelFormat { */ AV_PIX_FMT_CUDA, + /** + * HW acceleration allowing to get drmprime information, data[3] contains a av_drmprime pointer + */ + AV_PIX_FMT_DRMPRIME, + AV_PIX_FMT_0RGB=0x123+4,///< packed RGB 8:8:8, 32bpp, XRGBXRGB... X=unused/undefined AV_PIX_FMT_RGB0, ///< packed RGB 8:8:8, 32bpp, RGBXRGBX... X=unused/undefined AV_PIX_FMT_0BGR, ///< packed BGR 8:8:8, 32bpp, XBGRXBGR... X=unused/undefined