From patchwork Fri Nov 24 00:51:32 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Thompson X-Patchwork-Id: 6314 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.2.161.94 with SMTP id m30csp1465332jah; Thu, 23 Nov 2017 16:53:50 -0800 (PST) X-Google-Smtp-Source: AGs4zMYw3ltVy0kPt83J1gNzXqJd9Orhf+uIvc87/iHoM4Oq+f+rNGuSdK/Dpi6TsKc3dhcfOVX9 X-Received: by 10.28.17.66 with SMTP id 63mr8011430wmr.66.1511484830347; Thu, 23 Nov 2017 16:53:50 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1511484830; cv=none; d=google.com; s=arc-20160816; b=efIm3c9f1eEZYeRafN5MZrKn2sbdw2qRPWW7PJPjCxBQWdCqYVRTmMoPByPGcRqzza I6jAofzc+4r0xN8GEMlqZiQdBu65NVInOfyexu3X9/MadlYBJ2psbZZ780/51tX3RG79 YQvufKePhYrCo7gyLue/JHeLON/eeNlNkPC1JKsTiDH6NV5hlFdlYVwX7nPVtxiAymQM 1Bimysz+zcw+ezqSKMf3DFBLtpolTXVUc5ul0X2y6vxinSBb3bqV5f8msuoIdqTbpgit /rDsOiXxUhP0L0cU3ylhOo6xfJI+il+tH06IRepk+ztHYsbjSSn7tkrIjD3YbSrX8ar7 I9pA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:dkim-signature:delivered-to:arc-authentication-results; bh=lYmwKlBeeznZNpNczTO+NzqF6Fm96yzZJGhuNS06/A8=; b=dynZ9P6nGs3C3YbG/iAizPVrzY4dFLjKo4nG1aMpIuK51xkS4cRrL7Hgf5cuZbzxRX mTF+kg6n85W1QxwLLLnFHbHcTT8lLZHhczTYduX9hKwFAa4MmrQMHOJUeM0CTTmeS0MZ 1JFrbR62/b0bLdeYmSOjZ9UZvsg0oGhc5gaNYJf4PMaTP7C+kY06vc6qsOpI2RTz5RqB 4n2WCCK4WGyOxCPjylzxWueiwldnmu/6U3kOKuo3eWZieVqbfvMyRW9b1CNFhKFvXLEJ cXGWwwyYC3ktX+Zo4nAjTrN2CwfONMXRPHKnJN4v6P8HDFvYf39jHg6vlktqqyw67ajJ tWOg== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@jkqxz-net.20150623.gappssmtp.com header.s=20150623 header.b=cr/FGj1x; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id s5si13510419wrb.484.2017.11.23.16.53.49; Thu, 23 Nov 2017 16:53:50 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@jkqxz-net.20150623.gappssmtp.com header.s=20150623 header.b=cr/FGj1x; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 9782D68A27D; Fri, 24 Nov 2017 02:52:01 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wr0-f180.google.com (mail-wr0-f180.google.com [209.85.128.180]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id B8B2868A251 for ; Fri, 24 Nov 2017 02:51:53 +0200 (EET) Received: by mail-wr0-f180.google.com with SMTP id k18so13992101wre.1 for ; Thu, 23 Nov 2017 16:51:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jkqxz-net.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references; bh=zYucTYZ7cOluAJOoIqjQbRV2thNcrdGALQdkGnB2gvE=; b=cr/FGj1xyOIOy3Uv146eu7DHyhvLQrOjYSMlO+ScygaRu0Y0cchUDwrS8lKI2EwLVg AkmZL9EwVeVEWsFmiSlPoYCoUYJryDx+Hc3hREEvtXHtFEiIzTTWnsIPIEiQpnFSsccC MGOtSqqpPT30JZWyHjY80OYnFreQNO4EfOBkatCd41n8iaY9GERCbsl91imVbO/J8FZp WNDMzE9QAk+LWrIgvn3hP+M4YUq2imB/7rqrXHljPqhM/3UAw3j7u2+MM5SjS9rLQ3F1 M0oNVRzO5ZCShIvAXTVXsR2v7n4qGMGn6J3UNmVvzbwwJp4jUQnPft6pwGm6a3iw8+2p 20Fg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=zYucTYZ7cOluAJOoIqjQbRV2thNcrdGALQdkGnB2gvE=; b=J1dsUM2rfuM2Bb8LpEsT1KPjAt4qpRHb7AjmvEIWh8SENrDvgCWgXHvLZe87EpGdZc Z61ocgXhbWUAmvNVpZPwkGfQNk/bIoxL57X8pLA0g3LyiR7g82KHi2pkgxJJqmBFc/LB BbuF0Texqoh5qFwaWVEAEHRvRkNcdv5sEQ3DIXiG7jphB26//xBT9gOFbLDWalOa/aHT R1yYHhgcaOwLfWCAaOvsmyXxWrrAZVO1zuBA+QDEEyWIzSSWM2I616u3xypxG0WO7kBI pFxJePSWIULbUu6v4Hlho3ot440q+Q7KI4dXczmaQ3BfD4gYp9Xhh047XotmOXQ7JdcU gqLw== X-Gm-Message-State: AJaThX7x7X76hw/Ggds83VCk5S882YdZNjrWvtc9NQiFqgABxP48UXCb 0rKU5/to88Ee4LlgZpS6OQ2t3PK4 X-Received: by 10.223.188.81 with SMTP id a17mr19133647wrh.146.1511484713519; Thu, 23 Nov 2017 16:51:53 -0800 (PST) Received: from rywe.jkqxz.net (cpc91242-cmbg18-2-0-cust650.5-4.cable.virginm.net. [82.8.130.139]) by smtp.gmail.com with ESMTPSA id h7sm4489655wrb.35.2017.11.23.16.51.52 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 23 Nov 2017 16:51:52 -0800 (PST) From: Mark Thompson To: ffmpeg-devel@ffmpeg.org Date: Fri, 24 Nov 2017 00:51:32 +0000 Message-Id: <20171124005134.5683-15-sw@jkqxz.net> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20171124005134.5683-1-sw@jkqxz.net> References: <20171124005134.5683-1-sw@jkqxz.net> Subject: [FFmpeg-devel] [PATCH 14/17] mjpegdec: Add hwaccel hooks X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" --- This works nicely on the Intel driver for YUV 4:2:2 (like you usually get from UVC webcams) with ffmpeg via hw_device_ctx, but YUV 4:2:0 requires trickier surface setup because it doesn't work on normal NV12 surfaces. It can be used via hw_frames_ctx by making the right surfaces externally, but doesn't work by hw_device_ctx without additional hackery. I will look into this further - it should be possible to propagate the layout requirements so that hw_device_ctx works for 4:2:0 as well. configure | 2 + libavcodec/Makefile | 1 + libavcodec/hwaccels.h | 1 + libavcodec/mjpegdec.c | 6 ++ libavcodec/vaapi_decode.c | 2 + libavcodec/vaapi_mjpeg.c | 160 ++++++++++++++++++++++++++++++++++++++++++++++ 6 files changed, 172 insertions(+) create mode 100644 libavcodec/vaapi_mjpeg.c diff --git a/configure b/configure index b95a91cbf0..0a103b3f67 100755 --- a/configure +++ b/configure @@ -2699,6 +2699,8 @@ hevc_vdpau_hwaccel_deps="vdpau VdpPictureInfoHEVC" hevc_vdpau_hwaccel_select="hevc_decoder" hevc_videotoolbox_hwaccel_deps="videotoolbox" hevc_videotoolbox_hwaccel_select="hevc_decoder" +mjpeg_vaapi_hwaccel_deps="vaapi" +mjpeg_vaapi_hwaccel_select="mjpeg_decoder" mpeg_xvmc_hwaccel_deps="xvmc" mpeg_xvmc_hwaccel_select="mpeg2video_decoder" mpeg1_nvdec_hwaccel_deps="nvdec" diff --git a/libavcodec/Makefile b/libavcodec/Makefile index 0ebd2820eb..640edfb590 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -849,6 +849,7 @@ OBJS-$(CONFIG_HEVC_NVDEC_HWACCEL) += nvdec_hevc.o OBJS-$(CONFIG_HEVC_QSV_HWACCEL) += qsvdec_h2645.o OBJS-$(CONFIG_HEVC_VAAPI_HWACCEL) += vaapi_hevc.o OBJS-$(CONFIG_HEVC_VDPAU_HWACCEL) += vdpau_hevc.o +OBJS-$(CONFIG_MJPEG_VAAPI_HWACCEL) += vaapi_mjpeg.o OBJS-$(CONFIG_MPEG1_NVDEC_HWACCEL) += nvdec_mpeg12.o OBJS-$(CONFIG_MPEG1_VDPAU_HWACCEL) += vdpau_mpeg12.o OBJS-$(CONFIG_MPEG1_VIDEOTOOLBOX_HWACCEL) += videotoolbox.o diff --git a/libavcodec/hwaccels.h b/libavcodec/hwaccels.h index afe7289341..cefd2b15be 100644 --- a/libavcodec/hwaccels.h +++ b/libavcodec/hwaccels.h @@ -37,6 +37,7 @@ extern const AVHWAccel ff_hevc_nvdec_hwaccel; extern const AVHWAccel ff_hevc_vaapi_hwaccel; extern const AVHWAccel ff_hevc_vdpau_hwaccel; extern const AVHWAccel ff_hevc_videotoolbox_hwaccel; +extern const AVHWAccel ff_mjpeg_vaapi_hwaccel; extern const AVHWAccel ff_mpeg1_nvdec_hwaccel; extern const AVHWAccel ff_mpeg1_vdpau_hwaccel; extern const AVHWAccel ff_mpeg1_videotoolbox_hwaccel; diff --git a/libavcodec/mjpegdec.c b/libavcodec/mjpegdec.c index 846dec2f42..2b79cd21bb 100644 --- a/libavcodec/mjpegdec.c +++ b/libavcodec/mjpegdec.c @@ -652,6 +652,9 @@ unk_pixfmt: s->avctx->pix_fmt = s->hwaccel_pix_fmt; } else { enum AVPixelFormat pix_fmts[] = { +#if CONFIG_MJPEG_VAAPI_HWACCEL + AV_PIX_FMT_VAAPI, +#endif s->avctx->pix_fmt, AV_PIX_FMT_NONE, }; @@ -2777,6 +2780,9 @@ AVCodec ff_mjpeg_decoder = { .caps_internal = FF_CODEC_CAP_INIT_THREADSAFE | FF_CODEC_CAP_SKIP_FRAME_FILL_PARAM, .hw_configs = (const AVCodecHWConfigInternal*[]) { +#if CONFIG_MJPEG_VAAPI_HWACCEL + HWACCEL_VAAPI(mjpeg), +#endif NULL }, }; diff --git a/libavcodec/vaapi_decode.c b/libavcodec/vaapi_decode.c index 572b3a40ac..b30d397113 100644 --- a/libavcodec/vaapi_decode.c +++ b/libavcodec/vaapi_decode.c @@ -253,6 +253,8 @@ static const struct { MAP(HEVC, HEVC_MAIN, HEVCMain ), MAP(HEVC, HEVC_MAIN_10, HEVCMain10 ), #endif + MAP(MJPEG, MJPEG_HUFFMAN_BASELINE_DCT, + JPEGBaseline), MAP(WMV3, VC1_SIMPLE, VC1Simple ), MAP(WMV3, VC1_MAIN, VC1Main ), MAP(WMV3, VC1_COMPLEX, VC1Advanced ), diff --git a/libavcodec/vaapi_mjpeg.c b/libavcodec/vaapi_mjpeg.c new file mode 100644 index 0000000000..0277cb04f0 --- /dev/null +++ b/libavcodec/vaapi_mjpeg.c @@ -0,0 +1,160 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include +#include + +#include "hwaccel.h" +#include "vaapi_decode.h" +#include "mjpegdec.h" + +static int vaapi_mjpeg_start_frame(AVCodecContext *avctx, + av_unused const uint8_t *buffer, + av_unused uint32_t size) +{ + const MJpegDecodeContext *s = avctx->priv_data; + VAAPIDecodePicture *pic = s->hwaccel_picture_private; + VAPictureParameterBufferJPEGBaseline pp; + int err, i; + + pic->output_surface = ff_vaapi_get_surface_id(s->picture_ptr); + + pp = (VAPictureParameterBufferJPEGBaseline) { + .picture_width = avctx->width, + .picture_height = avctx->height, + + .num_components = s->nb_components, + }; + + for (i = 0; i < s->nb_components; i++) { + pp.components[i].component_id = s->component_id[i]; + pp.components[i].h_sampling_factor = s->h_count[i]; + pp.components[i].v_sampling_factor = s->v_count[i]; + pp.components[i].quantiser_table_selector = s->quant_index[i]; + } + + err = ff_vaapi_decode_make_param_buffer(avctx, pic, + VAPictureParameterBufferType, + &pp, sizeof(pp)); + if (err < 0) + goto fail; + + return 0; + +fail: + ff_vaapi_decode_cancel(avctx, pic); + return err; +} + +static int vaapi_mjpeg_end_frame(AVCodecContext *avctx) +{ + const MJpegDecodeContext *s = avctx->priv_data; + VAAPIDecodePicture *pic = s->hwaccel_picture_private; + + return ff_vaapi_decode_issue(avctx, pic); +} + +static int vaapi_mjpeg_decode_slice(AVCodecContext *avctx, + const uint8_t *buffer, + uint32_t size) +{ + const MJpegDecodeContext *s = avctx->priv_data; + VAAPIDecodePicture *pic = s->hwaccel_picture_private; + VAHuffmanTableBufferJPEGBaseline huff; + VAIQMatrixBufferJPEGBaseline quant; + VASliceParameterBufferJPEGBaseline sp; + int err, i, j; + + memset(&huff, 0, sizeof(huff)); + for (i = 0; i < 2; i++) { + huff.load_huffman_table[i] = 1; + for (j = 0; j < 16; j++) + huff.huffman_table[i].num_dc_codes[j] = s->raw_huffman_lengths[0][i][j]; + for (j = 0; j < 12; j++) + huff.huffman_table[i].dc_values[j] = s->raw_huffman_values[0][i][j]; + for (j = 0; j < 16; j++) + huff.huffman_table[i].num_ac_codes[j] = s->raw_huffman_lengths[1][i][j]; + for (j = 0; j < 162; j++) + huff.huffman_table[i].ac_values[j] = s->raw_huffman_values[1][i][j]; + } + + err = ff_vaapi_decode_make_param_buffer(avctx, pic, + VAHuffmanTableBufferType, + &huff, sizeof(huff)); + if (err < 0) + goto fail; + + memset(&quant, 0, sizeof(quant)); + for (i = 0; i < 4; i++) { + quant.load_quantiser_table[i] = 1; + for (j = 0; j < 64; j++) + quant.quantiser_table[i][j] = s->quant_matrixes[i][j]; + } + + err = ff_vaapi_decode_make_param_buffer(avctx, pic, + VAIQMatrixBufferType, + &quant, sizeof(quant)); + if (err < 0) + goto fail; + + + sp = (VASliceParameterBufferJPEGBaseline) { + .slice_data_size = size, + .slice_data_offset = 0, + .slice_data_flag = VA_SLICE_DATA_FLAG_ALL, + + .slice_horizontal_position = 0, + .slice_vertical_position = 0, + + .restart_interval = s->restart_interval, + .num_mcus = s->mb_width * s->mb_height, + }; + + sp.num_components = s->nb_components; + for (i = 0; i < s->nb_components; i++) { + sp.components[i].component_selector = s->component_id[s->comp_index[i]]; + sp.components[i].dc_table_selector = s->dc_index[i]; + sp.components[i].ac_table_selector = s->ac_index[i]; + } + + err = ff_vaapi_decode_make_slice_buffer(avctx, pic, &sp, sizeof(sp), buffer, size); + if (err) + goto fail; + + return 0; + +fail: + ff_vaapi_decode_cancel(avctx, pic); + return err; +} + +const AVHWAccel ff_mjpeg_vaapi_hwaccel = { + .name = "mjpeg_vaapi", + .type = AVMEDIA_TYPE_VIDEO, + .id = AV_CODEC_ID_MJPEG, + .pix_fmt = AV_PIX_FMT_VAAPI, + .start_frame = &vaapi_mjpeg_start_frame, + .end_frame = &vaapi_mjpeg_end_frame, + .decode_slice = &vaapi_mjpeg_decode_slice, + .frame_priv_data_size = sizeof(VAAPIDecodePicture), + .init = &ff_vaapi_decode_init, + .uninit = &ff_vaapi_decode_uninit, + .frame_params = &ff_vaapi_common_frame_params, + .priv_data_size = sizeof(VAAPIDecodeContext), + .caps_internal = HWACCEL_CAP_ASYNC_SAFE, +};