From patchwork Sun Nov 19 19:52:28 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Philip Langdale X-Patchwork-Id: 6202 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.2.161.94 with SMTP id m30csp3112042jah; Sun, 19 Nov 2017 11:53:18 -0800 (PST) X-Google-Smtp-Source: AGs4zMZ8xWh1YpgaVL4H6giADHumPn4fAx/1LyR6MhHxFY37Lb3mJh0+U7hT4Qm2oGy3Tulkcfrf X-Received: by 10.28.139.144 with SMTP id n138mr8741233wmd.78.1511121198010; Sun, 19 Nov 2017 11:53:18 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1511121197; cv=none; d=google.com; s=arc-20160816; b=pDHKjfYDUEv5g0NKSDRdASF9oENMitN2SzHNNzlNh2BlIYph6IOsrTNUaXedgNphZq EROLkQiUER1lBmjNeH7oc5D4xPM+oBkyYYq37sw0P5fi0Q+C9IP7V2YafeV6eltvLi8r lf1zNF0su3eKuFRflW4bt+w7Uz6qmRKaXROycU9wa0TDJkE/3UzMdh+tkX00XVdZO4Pm 3QjknFJRgXm7bOYU0XA3YylyaLBXvmz6ZOrFdlcsG5xP0WjVGUs9hq2WX/EAWOBWpQaT Ay8y1rzqSxAfp/0TNjPkQ7vSH05D2fxVUbQdL9HTOfiE6dQAXjIcVc7ch0MaPj6n/qZL Zr8A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:dkim-signature:delivered-to:arc-authentication-results; bh=vR+U+36oXR6ZrNcyMUZRiUhfmJSxKAvE4z9bi40OPMI=; b=gdS+aIHfRcA4JtRMDN/98btGxGt+4xKNUcHSJQEPXKPMk6WxBCw6KExySxhbdUGsAE /aufRfzWfnXWMpXZplPwx40VpHP/FA3QIvIp7YWzRwKegyBznOJnKm4/deQ542JROyJp p0KJ9sbIkrPZ6e9uPjHnK1VhNvPRCcmrMSQcKvalt3vB8AesvvJk5zpvPBEVsqO5DvLP 9mr5huT7hUcDBsU8zEJ2b2Rhqqku0lXabRcBV9wDbrvnpBqpoF4CvfokKC3tLyIMYYFR zZHteI9z+mlIwUFhVeNribH+H//Z4gV/1JBQkvkTDzUVbIn2lHvXJUltR314wiVVlrFQ Djmw== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@overt.org header.s=mail header.b=gz8KNh5q; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id h25si7405683wrf.107.2017.11.19.11.53.17; Sun, 19 Nov 2017 11:53:17 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@overt.org header.s=mail header.b=gz8KNh5q; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id C9EF9689BFE; Sun, 19 Nov 2017 21:52:35 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-io0-f226.google.com (mail-io0-f226.google.com [209.85.223.226]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 8DB5A688323 for ; Sun, 19 Nov 2017 21:52:26 +0200 (EET) Received: by mail-io0-f226.google.com with SMTP id g73so13638576ioj.8 for ; Sun, 19 Nov 2017 11:52:43 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references; bh=lB+0KBT/yX2A6fy6lYX91TbCT2G52D6v+OHQOJ1Evw8=; b=JNLS9oWO6RcPDHiVAczElBSYqLH8tY+vPHaSeLpJPbzHhM9SyRyck5vi1bkNWVFBMR VzYZXdMBZJpcQO3SyC6mB694UgNo4n6Ypt/lAP1iTnGnqyT6jVd7XtFSNLkSHOlvRGgP ifx6hN13+A4SM9cQFyqKU/6/MPAhi3cEKM6agRiIqfIOPJfgFAgYZFGGsfiP1+hBhqZi 2/xa+zPAx8gQ7pKwc3N5BL+V3l1anU8oxq9U1/EZtVMFqbsFw6CDxLUl9plQhwEORvHs 0Al/+KYdKfig7hHUhwg2o2CetsHk4W4RwvH/OREIMwNnEwfCVOmFQWK6CiEnzqlZMHpc 2euQ== X-Gm-Message-State: AJaThX5H9XOqCpgl+GoimNc+hgTp8fcsPRtHxHm2k0QD+8OFylhjZf3H LP0gJkxWtlHiYbSpGYEqxOkZ7lguUeJDlJPKFNY5CWHKrRq6Gg== X-Received: by 10.107.8.32 with SMTP id 32mr11303610ioi.200.1511121162203; Sun, 19 Nov 2017 11:52:42 -0800 (PST) Received: from mail.overt.org (155.208.178.107.bc.googleusercontent.com. [107.178.208.155]) by smtp-relay.gmail.com with ESMTPS id t126sm2737820itb.7.2017.11.19.11.52.42 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 19 Nov 2017 11:52:42 -0800 (PST) X-Relaying-Domain: gapps.overt.org Received: from authenticated-user (mail.overt.org [107.178.208.155]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.overt.org (Postfix) with ESMTPSA id 956C26018E; Sun, 19 Nov 2017 19:52:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=overt.org; s=mail; t=1511121161; bh=EkPbju+j0dH48vQPToL6zMzqSZt0L3kwxwr5sdJP9Ko=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=gz8KNh5qbExDRYpeAqgIyssBZfa7vsLAbZljU099AgLp8yJMpgnFJjjLmFysGRtHY jWSf5wXyDNAlhPhNMS1bJNeASVLltx1t4FRMxC6I8tAWsrriQJTPz/2aGI5YJ4+yZ5 DR7mv0u0gLifdj3hxPM7w8HOF+MUCX+la0Rml1/VOKRbqRpG9bhX/O+6FEpkTyCFe5 G3sZ1Qcbus4AW7BtqXcQykOrUoK/kd/Qmmrb1ELBh/4Gjqi4dkR37a0v41Zoaz1p/u BA8y8i2cJZFK7UZP1crvbIhZXSxafPgD9lCt14IrCTbYDntLDD45oGac7km8KQ2TXH YgfVn70bjCt+w== From: Philip Langdale To: ffmpeg-devel@ffmpeg.org Date: Sun, 19 Nov 2017 11:52:28 -0800 Message-Id: <20171119195228.25719-4-philipl@overt.org> In-Reply-To: <20171119195228.25719-1-philipl@overt.org> References: <20171119195228.25719-1-philipl@overt.org> Subject: [FFmpeg-devel] [PATCH 3/3] avcodec: Implement mpeg4 nvdec hwaccel X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Philip Langdale MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" This was predictably nightmarish, given how ridiculous mpeg4 is. I had to stare at the cuvid parser output for a long time to work out what each field was supposed to be, and even then, I still don't fully understand some of them, particularly: vop_coded: I think this means whether the vop has a picture shape, and therefore a picture type. I have no samples where this is not the case. divx_flags: There's obviously no documentation on what the possible flags are. I simply observed that this is '0' for a normal bitstream and '5' for packed b-frames. gmc_enabled: This seems to map to mc_sel being non-zero, but I also have no samples where that is true. Also note that as with the vdpau hwaccel, the decoder needs to consume the entire frame and not the slice. Signed-off-by: Philip Langdale --- Changelog | 2 +- configure | 2 + libavcodec/Makefile | 1 + libavcodec/allcodecs.c | 1 + libavcodec/h263dec.c | 3 ++ libavcodec/nvdec.c | 1 + libavcodec/nvdec_mpeg4.c | 121 +++++++++++++++++++++++++++++++++++++++++++++++ libavcodec/version.h | 2 +- 8 files changed, 131 insertions(+), 2 deletions(-) create mode 100644 libavcodec/nvdec_mpeg4.c diff --git a/Changelog b/Changelog index 5a9d183aed..74ed35cfe6 100644 --- a/Changelog +++ b/Changelog @@ -13,7 +13,7 @@ version : - PCE support for extended channel layouts in the AAC encoder - native aptX encoder and decoder - Raw aptX muxer and demuxer -- NVIDIA NVDEC-accelerated H.264, HEVC, MPEG-1/2, VC1 and VP9 hwaccel decoding +- NVIDIA NVDEC-accelerated H.264, HEVC, MPEG-1/2/4, VC1 and VP9 hwaccel decoding - Intel QSV-accelerated overlay filter - mcompand audio filter - acontrast audio filter diff --git a/configure b/configure index 35713805fa..36ccf767dd 100755 --- a/configure +++ b/configure @@ -2731,6 +2731,8 @@ mpeg2_xvmc_hwaccel_select="mpeg2video_decoder" mpeg4_cuvid_hwaccel_select="mpeg4_cuvid_decoder" mpeg4_mediacodec_hwaccel_deps="mediacodec" mpeg4_mmal_hwaccel_deps="mmal" +mpeg4_nvdec_hwaccel_deps="nvdec" +mpeg4_nvdec_hwaccel_select="mpeg4_decoder" mpeg4_vaapi_hwaccel_deps="vaapi" mpeg4_vaapi_hwaccel_select="mpeg4_decoder" mpeg4_vdpau_hwaccel_deps="vdpau" diff --git a/libavcodec/Makefile b/libavcodec/Makefile index 0573454c7b..2af957ab72 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -861,6 +861,7 @@ OBJS-$(CONFIG_MPEG2_VAAPI_HWACCEL) += vaapi_mpeg2.o OBJS-$(CONFIG_MPEG2_VDPAU_HWACCEL) += vdpau_mpeg12.o OBJS-$(CONFIG_MPEG2_VIDEOTOOLBOX_HWACCEL) += videotoolbox.o OBJS-$(CONFIG_MPEG2_XVMC_HWACCEL) += mpegvideo_xvmc.o +OBJS-$(CONFIG_MPEG4_NVDEC_HWACCEL) += nvdec_mpeg4.o OBJS-$(CONFIG_MPEG4_VAAPI_HWACCEL) += vaapi_mpeg4.o OBJS-$(CONFIG_MPEG4_VDPAU_HWACCEL) += vdpau_mpeg4.o OBJS-$(CONFIG_MPEG4_VIDEOTOOLBOX_HWACCEL) += videotoolbox.o diff --git a/libavcodec/allcodecs.c b/libavcodec/allcodecs.c index e9df7049de..85c38c83aa 100644 --- a/libavcodec/allcodecs.c +++ b/libavcodec/allcodecs.c @@ -106,6 +106,7 @@ static void register_all(void) REGISTER_HWACCEL(MPEG4_CUVID, mpeg4_cuvid); REGISTER_HWACCEL(MPEG4_MEDIACODEC, mpeg4_mediacodec); REGISTER_HWACCEL(MPEG4_MMAL, mpeg4_mmal); + REGISTER_HWACCEL(MPEG4_NVDEC, mpeg4_nvdec); REGISTER_HWACCEL(MPEG4_VAAPI, mpeg4_vaapi); REGISTER_HWACCEL(MPEG4_VDPAU, mpeg4_vdpau); REGISTER_HWACCEL(MPEG4_VIDEOTOOLBOX, mpeg4_videotoolbox); diff --git a/libavcodec/h263dec.c b/libavcodec/h263dec.c index c7cf4bc0c2..b222de793b 100644 --- a/libavcodec/h263dec.c +++ b/libavcodec/h263dec.c @@ -714,6 +714,9 @@ const enum AVPixelFormat ff_h263_hwaccel_pixfmt_list_420[] = { #if CONFIG_H263_VAAPI_HWACCEL || CONFIG_MPEG4_VAAPI_HWACCEL AV_PIX_FMT_VAAPI, #endif +#if CONFIG_MPEG4_NVDEC_HWACCEL + AV_PIX_FMT_CUDA, +#endif #if CONFIG_MPEG4_VDPAU_HWACCEL AV_PIX_FMT_VDPAU, #endif diff --git a/libavcodec/nvdec.c b/libavcodec/nvdec.c index d5cf1058cb..efcd47a7f7 100644 --- a/libavcodec/nvdec.c +++ b/libavcodec/nvdec.c @@ -56,6 +56,7 @@ static int map_avcodec_id(enum AVCodecID id) case AV_CODEC_ID_HEVC: return cudaVideoCodec_HEVC; case AV_CODEC_ID_MPEG1VIDEO: return cudaVideoCodec_MPEG1; case AV_CODEC_ID_MPEG2VIDEO: return cudaVideoCodec_MPEG2; + case AV_CODEC_ID_MPEG4: return cudaVideoCodec_MPEG4; case AV_CODEC_ID_VC1: return cudaVideoCodec_VC1; case AV_CODEC_ID_VP9: return cudaVideoCodec_VP9; case AV_CODEC_ID_WMV3: return cudaVideoCodec_VC1; diff --git a/libavcodec/nvdec_mpeg4.c b/libavcodec/nvdec_mpeg4.c new file mode 100644 index 0000000000..be81bd958b --- /dev/null +++ b/libavcodec/nvdec_mpeg4.c @@ -0,0 +1,121 @@ +/* + * MPEG-4 Part 2 HW decode acceleration through NVDEC + * + * Copyright (c) 2017 Philip Langdale + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "avcodec.h" +#include "mpeg4video.h" +#include "nvdec.h" +#include "decode.h" + +static int nvdec_mpeg4_start_frame(AVCodecContext *avctx, const uint8_t *buffer, uint32_t size) +{ + Mpeg4DecContext *m = avctx->priv_data; + MpegEncContext *s = &m->m; + + NVDECContext *ctx = avctx->internal->hwaccel_priv_data; + CUVIDPICPARAMS *pp = &ctx->pic_params; + CUVIDPICPARAMS *picparams = &ctx->pic_params; + CUVIDMPEG4PICPARAMS *ppc = &pp->CodecSpecific.mpeg4; + FrameDecodeData *fdd; + NVDECFrame *cf; + AVFrame *cur_frame = s->current_picture.f; + + int ret, i; + + ret = ff_nvdec_start_frame(avctx, cur_frame); + if (ret < 0) + return ret; + + fdd = (FrameDecodeData*)cur_frame->private_ref->data; + cf = (NVDECFrame*)fdd->hwaccel_priv; + + *pp = (CUVIDPICPARAMS) { + .PicWidthInMbs = (cur_frame->width + 15) / 16, + .FrameHeightInMbs = (cur_frame->height + 15) / 16, + .CurrPicIdx = cf->idx, + + .intra_pic_flag = s->pict_type == AV_PICTURE_TYPE_I, + .ref_pic_flag = s->pict_type == AV_PICTURE_TYPE_I || + s->pict_type == AV_PICTURE_TYPE_P || + s->pict_type == AV_PICTURE_TYPE_S, + + .CodecSpecific.mpeg4 = { + .ForwardRefIdx = ff_nvdec_get_ref_idx(s->last_picture.f), + .BackwardRefIdx = ff_nvdec_get_ref_idx(s->next_picture.f), + + .video_object_layer_width = s->width, + .video_object_layer_height = s->height, + .vop_time_increment_bitcount = m->time_increment_bits, + .top_field_first = s->top_field_first, + .resync_marker_disable = !m->resync_marker, + .quant_type = s->mpeg_quant, + .quarter_sample = s->quarter_sample, + .short_video_header = avctx->codec->id == AV_CODEC_ID_H263, + .divx_flags = s->divx_packed ? 5 : 0, + + .vop_coding_type = s->pict_type - AV_PICTURE_TYPE_I, + .vop_coded = m->shape != RECT_SHAPE, + .vop_rounding_type = s->no_rounding, + .alternate_vertical_scan_flag = s->alternate_scan, + .interlaced = !s->progressive_sequence, + .vop_fcode_forward = s->f_code, + .vop_fcode_backward = s->b_code, + .trd = { s->pp_time, s->pp_field_time >> 1 }, + .trb = { s->pb_time, s->pb_field_time >> 1 }, + + .gmc_enabled = s->mcsel != 0, + } + }; + + for (i = 0; i < 64; ++i) { + ppc->QuantMatrixIntra[i] = s->intra_matrix[i]; + ppc->QuantMatrixInter[i] = s->inter_matrix[i]; + } + + // We need to pass the full frame buffer and not just the slice + return ff_nvdec_simple_decode_slice(avctx, buffer, size); +} + +static int nvdec_mpeg4_decode_slice(AVCodecContext *avctx, const uint8_t *buffer, uint32_t size) +{ + return 0; +} + +static int nvdec_mpeg4_frame_params(AVCodecContext *avctx, + AVBufferRef *hw_frames_ctx) +{ + // Each frame can at most have one P and one B reference + return ff_nvdec_frame_params(avctx, hw_frames_ctx, 2); +} + +AVHWAccel ff_mpeg4_nvdec_hwaccel = { + .name = "mpeg4_nvdec", + .type = AVMEDIA_TYPE_VIDEO, + .id = AV_CODEC_ID_MPEG4, + .pix_fmt = AV_PIX_FMT_CUDA, + .start_frame = nvdec_mpeg4_start_frame, + .end_frame = ff_nvdec_simple_end_frame, + .decode_slice = nvdec_mpeg4_decode_slice, + .frame_params = nvdec_mpeg4_frame_params, + .init = ff_nvdec_decode_init, + .uninit = ff_nvdec_decode_uninit, + .priv_data_size = sizeof(NVDECContext), +}; diff --git a/libavcodec/version.h b/libavcodec/version.h index ff54670ea9..c8550bca9a 100644 --- a/libavcodec/version.h +++ b/libavcodec/version.h @@ -29,7 +29,7 @@ #define LIBAVCODEC_VERSION_MAJOR 58 #define LIBAVCODEC_VERSION_MINOR 3 -#define LIBAVCODEC_VERSION_MICRO 104 +#define LIBAVCODEC_VERSION_MICRO 105 #define LIBAVCODEC_VERSION_INT AV_VERSION_INT(LIBAVCODEC_VERSION_MAJOR, \ LIBAVCODEC_VERSION_MINOR, \