From patchwork Sun Feb 18 23:53:31 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Philip Langdale X-Patchwork-Id: 7645 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.2.150.33 with SMTP id c30csp912972jai; Sun, 18 Feb 2018 15:59:15 -0800 (PST) X-Google-Smtp-Source: AH8x225Cyp4JNfoxX2CqBGUtWrhZs4SMFO31gdeGlcB5wChWWOGEu1qayDgtQNE+CjGOZ+RAXZZF X-Received: by 10.223.149.70 with SMTP id 64mr2197576wrs.76.1518998355441; Sun, 18 Feb 2018 15:59:15 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1518998355; cv=none; d=google.com; s=arc-20160816; b=biEr2WKgkjprzyt5Eixhzqq6qKlfD8qCpl0zy8p+LWIMz/UUyhYJq1nJV4vvkQNfXX d+NlqNOgl4SX0Lp24jeR2Z6Tk3R+LPSKT1CNK5XTIXSGXM+z7o/biGa2ndigJB7wrg0X jFlilmBLwfuIoxTOlTRxwPSsM1oivT4QHrtZz7T0WzJK3USzAAbz0cFB8cugVaIizwgm DPLrMRGoDzv8sUAbpBnhN9IOKzV0nE8Y4k0X4O64JYQi2/dGuuhcip+RXZYICAuuIFza Qp5c6ky/wiG5ky2NA5PeBG3wLzeDl8HsjGi969I38PWCgCsQEi6qcy+DcLtAw86T71n4 awYw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:dkim-signature:delivered-to:arc-authentication-results; bh=ySAjETFJs3+9C6nIH1n1xafo96d5Oqq3zrTHg7omWzY=; b=H9VNuJN+5SNA18Ti2nXbBF4wPkljUJmu4jcHVy+lgNf5si7k0O+zveu+YCBc0+ayWG IdowpEFJ1CPcmIzOzfg+6plOcx4BeNBSZk9gGjuuGSVLWHQDSY8Dd8IR1CtUqvHK9pmN wmM1Ke/sDPNt62zb7RFVKOPR/kXURdR8wMMlnyePnPMkQLbcb+4xVQ28NcLwCXAf3lpD /0CYVlgE49Vp/JgjVr71dCcVrh2usr2jzd027LVTAHjnlLny+neE+ubAf1q6HYDzC6N9 9fUBzsnLry4rbfbibVjV5jJkTetFLUmtRTfNBzqHL9UzL+FzTTTQz8gBfE24ParRbGUM M5pg== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@overt.org header.s=mail header.b=mynQkNI2; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id v66si13338197wmd.262.2018.02.18.15.59.15; Sun, 18 Feb 2018 15:59:15 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@overt.org header.s=mail header.b=mynQkNI2; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 57001689C2A; Mon, 19 Feb 2018 01:59:13 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-io0-f227.google.com (mail-io0-f227.google.com [209.85.223.227]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id AD97A689936 for ; Mon, 19 Feb 2018 01:59:06 +0200 (EET) Received: by mail-io0-f227.google.com with SMTP id p188so9558948ioe.12 for ; Sun, 18 Feb 2018 15:59:07 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references; bh=iC9ZTPXlYUXxl/WLHAmp/aVqadqm2QL+y4VQJzbTMZA=; b=kWs+6b7ZmlggrJHCD6ncx8jQhNY5ZRV1CXWA642f0vsiEiWwUPxOpBAVsp6oKKy2Sq zOMJ4HtChkX4KyRB1h07kfKl6iKKHeO1Lu5iiaquOmFmQUOV+1o00t826xFllP325Hso bEig93HDaAUlH8qHk/6HQb4ZRPlTpGvWlBny4pJMhAJRP37XatPy9QIJdFHg97wfYgqc maEIdJ1tSzoMiJ/Eggl0cCPeRx40K1jxI+shZ3fqUo9Er1W3bJdhrFlAbELbsMnn0HTf liHVmYlUbvJdtExZbane7J8ssN7C8pn8/F4riYsBN77TkHpQ8Mx+R/Gj7w5POwiHtQN0 wBAg== X-Gm-Message-State: APf1xPCX3iAH5RISOe2Mxru0ZC5AgkykjJaYeYSs0TJRBKEeM5tZtYpk AKpo+3NBkfc77BdlVgo7RHYt+cKaASgSWKSkd99jGCsD/8QUrg== X-Received: by 10.107.182.139 with SMTP id g133mr18013650iof.77.1518998033164; Sun, 18 Feb 2018 15:53:53 -0800 (PST) Received: from mail.overt.org (155.208.178.107.bc.googleusercontent.com. [107.178.208.155]) by smtp-relay.gmail.com with ESMTPS id e201sm5697736ite.10.2018.02.18.15.53.53 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 18 Feb 2018 15:53:53 -0800 (PST) X-Relaying-Domain: gapps.overt.org Received: from authenticated-user (mail.overt.org [107.178.208.155]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.overt.org (Postfix) with ESMTPSA id 9411A601B2; Sun, 18 Feb 2018 23:53:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=overt.org; s=mail; t=1518998032; bh=oKJh+uzAWRWb7XvnTv0S2pnjK/iadY+zD+YfmT1/TNQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=mynQkNI2BMtgDydXjNrl1Ezs/gbgGIh/EkPMGa2aGCG9HdyGjgm/8r0gjQFsSU2dC VaP3JIx5JEX+m1fX7jrHmLBkXXIFDVSaqnpC26yrGlxhE9txIpGfr1H/e6sdR/alZI Ro1a/dll8lWZnAYTa2E5UqYQedxDhGUomjmeX4naWu6sdZ+aEcsHCAMOo5EQfJVlWf YB1OT7yFLYRMifLb1LeIsRUxAy/mx/otf8x8fo4l/7fvC+MK9hWnwkRsMNLkbudiTe L7kw4xDVsD1R/GmgWCFZ+8ydw+gQkOTUI9OAd2FuYFYYbbJ0nM5MbQ2Jte0XFP9cp1 p+Y3E4YscQbZw== From: Philip Langdale To: ffmpeg-devel@ffmpeg.org Date: Sun, 18 Feb 2018 15:53:31 -0800 Message-Id: <20180218235333.6965-2-philipl@overt.org> In-Reply-To: <20180218235333.6965-1-philipl@overt.org> References: <20180218235333.6965-1-philipl@overt.org> Subject: [FFmpeg-devel] [PATCH 1/3] mjpegdec: Add hwaccel hooks X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Mark Thompson MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" From: Mark Thompson Also adds some extra fields to the main context structure that may be needed by a hwaccel decoder. (Modified by philipl to remove a YUVJ mapping hack that isn't required by nvdec and would otherwise block merging the rest of the change.) --- libavcodec/mjpegdec.c | 68 +++++++++++++++++++++++++++++++++++++++++++++++++-- libavcodec/mjpegdec.h | 11 +++++++++ 2 files changed, 77 insertions(+), 2 deletions(-) diff --git a/libavcodec/mjpegdec.c b/libavcodec/mjpegdec.c index 5055ee2826..9a7a329b19 100644 --- a/libavcodec/mjpegdec.c +++ b/libavcodec/mjpegdec.c @@ -36,6 +36,7 @@ #include "avcodec.h" #include "blockdsp.h" #include "copy_block.h" +#include "hwaccel.h" #include "idctdsp.h" #include "internal.h" #include "jpegtables.h" @@ -147,6 +148,7 @@ av_cold int ff_mjpeg_decode_init(AVCodecContext *avctx) s->org_height = avctx->coded_height; avctx->chroma_sample_location = AVCHROMA_LOC_CENTER; avctx->colorspace = AVCOL_SPC_BT470BG; + s->hwaccel_pix_fmt = s->hwaccel_sw_pix_fmt = AV_PIX_FMT_NONE; if ((ret = build_basic_mjpeg_vlc(s)) < 0) return ret; @@ -279,6 +281,11 @@ int ff_mjpeg_decode_dht(MJpegDecodeContext *s) code_max + 1, 0, 0)) < 0) return ret; } + + for (i = 0; i < 16; i++) + s->raw_huffman_lengths[class][index][i] = bits_table[i + 1]; + for (i = 0; i < 256; i++) + s->raw_huffman_values[class][index][i] = val_table[i]; } return 0; } @@ -636,6 +643,24 @@ unk_pixfmt: return AVERROR_BUG; } + // TODO: Handle the fact that YUVJ pix_fmts are not recognised by hwaccels and + // might impact the ability to allocate hw frames. + + if (s->avctx->pix_fmt == s->hwaccel_sw_pix_fmt) { + s->avctx->pix_fmt = s->hwaccel_pix_fmt; + } else { + enum AVPixelFormat pix_fmts[] = { + s->avctx->pix_fmt, + AV_PIX_FMT_NONE, + }; + s->hwaccel_pix_fmt = ff_get_format(s->avctx, pix_fmts); + if (s->hwaccel_pix_fmt < 0) + return AVERROR(EINVAL); + + s->hwaccel_sw_pix_fmt = s->avctx->pix_fmt; + s->avctx->pix_fmt = s->hwaccel_pix_fmt; + } + if (s->avctx->skip_frame == AVDISCARD_ALL) { s->picture_ptr->pict_type = AV_PICTURE_TYPE_I; s->picture_ptr->key_frame = 1; @@ -683,6 +708,19 @@ unk_pixfmt: } memset(s->coefs_finished, 0, sizeof(s->coefs_finished)); } + + if (s->avctx->hwaccel) { + s->hwaccel_picture_private = + av_mallocz(s->avctx->hwaccel->frame_priv_data_size); + if (!s->hwaccel_picture_private) + return AVERROR(ENOMEM); + + ret = s->avctx->hwaccel->start_frame(s->avctx, s->raw_buffer, + s->raw_buffer_size); + if (ret < 0) + return ret; + } + return 0; } @@ -1510,7 +1548,7 @@ int ff_mjpeg_decode_sos(MJpegDecodeContext *s, const uint8_t *mb_bitmask, } } - av_assert0(s->picture_ptr->data[0]); + //av_assert0(s->picture_ptr->data[0]); /* XXX: verify len field validity */ len = get_bits(&s->gb, 16); nb_components = get_bits(&s->gb, 8); @@ -1600,7 +1638,18 @@ next_field: for (i = 0; i < nb_components; i++) s->last_dc[i] = (4 << s->bits); - if (s->lossless) { + if (s->avctx->hwaccel) { + int bytes_to_start = get_bits_count(&s->gb) / 8; + av_assert0(bytes_to_start >= 0 && + s->raw_buffer_size >= bytes_to_start); + + ret = s->avctx->hwaccel->decode_slice(s->avctx, + s->raw_buffer + bytes_to_start, + s->raw_buffer_size - bytes_to_start); + if (ret < 0) + return ret; + + } else if (s->lossless) { av_assert0(s->picture_ptr == s->picture); if (CONFIG_JPEGLS_DECODER && s->ls) { // for () { @@ -2226,6 +2275,9 @@ int ff_mjpeg_decode_frame(AVCodecContext *avctx, void *data, int *got_frame, goto fail; } + s->raw_buffer = buf_ptr; + s->raw_buffer_size = buf_end - buf_ptr; + s->start_code = start_code; if (s->avctx->debug & FF_DEBUG_STARTCODE) av_log(avctx, AV_LOG_DEBUG, "startcode: %X\n", start_code); @@ -2342,6 +2394,13 @@ eoi_parser: s->got_picture = 0; goto the_end_no_picture; } + if (s->avctx->hwaccel) { + ret = s->avctx->hwaccel->end_frame(s->avctx); + if (ret < 0) + return ret; + + av_freep(&s->hwaccel_picture_private); + } if ((ret = av_frame_ref(frame, s->picture_ptr)) < 0) return ret; *got_frame = 1; @@ -2666,6 +2725,8 @@ av_cold int ff_mjpeg_decode_end(AVCodecContext *avctx) reset_icc_profile(s); + av_freep(&s->hwaccel_picture_private); + return 0; } @@ -2706,6 +2767,9 @@ AVCodec ff_mjpeg_decoder = { .priv_class = &mjpegdec_class, .caps_internal = FF_CODEC_CAP_INIT_THREADSAFE | FF_CODEC_CAP_SKIP_FRAME_FILL_PARAM, + .hw_configs = (const AVCodecHWConfigInternal*[]) { + NULL + }, }; #endif #if CONFIG_THP_DECODER diff --git a/libavcodec/mjpegdec.h b/libavcodec/mjpegdec.h index c36fba5f22..11dfdbf62b 100644 --- a/libavcodec/mjpegdec.h +++ b/libavcodec/mjpegdec.h @@ -137,6 +137,17 @@ typedef struct MJpegDecodeContext { int *iccdatalens; int iccnum; int iccread; + + // Raw stream data for hwaccel use. + const uint8_t *raw_buffer; + size_t raw_buffer_size; + + uint8_t raw_huffman_lengths[2][4][16]; + uint8_t raw_huffman_values[2][4][256]; + + enum AVPixelFormat hwaccel_sw_pix_fmt; + enum AVPixelFormat hwaccel_pix_fmt; + void *hwaccel_picture_private; } MJpegDecodeContext; int ff_mjpeg_decode_init(AVCodecContext *avctx);