From patchwork Sat Nov 13 21:09:16 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: rcombs X-Patchwork-Id: 31399 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a6b:d206:0:0:0:0:0 with SMTP id q6csp3679681iob; Sat, 13 Nov 2021 13:11:51 -0800 (PST) X-Google-Smtp-Source: ABdhPJwPegIxRunO16A6HoD+crr94TaFY6oKquY60+nInSD6ccgAdyVMUrCQRK9SSLttWg56mN2h X-Received: by 2002:a05:6402:2552:: with SMTP id l18mr35562107edb.90.1636837911398; Sat, 13 Nov 2021 13:11:51 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1636837911; cv=none; d=google.com; s=arc-20160816; b=QuwJT/xew+ompqmpo1lZmNLX0Wi/JhOjA0Ei6HGTCGEdS8gr8lmwKk8ZnX60tjlxjL fXFqhEfXAyHUuBBlU9Cbev6ruk6rT/HbicZ7pJGXk0kSoID2BK5H8KpJKkU0helh6EGj 337fVGmATLBvXVT2IlkLzYm+5H3skCxFacVAailhv2oLP6kBUqyWY0rnwpEWcKJLl19T EA6xUh6TUfw6i6KUAXEJmOEw4JtraFVRv0i2aplu7GXGsy1JSzXX7EDXEOSFlcX5UrpQ Y7hUs2rnuIm6vlhFZhzPdlMNyNvELXWra4KoLPeRFteW/2/0x31SZ3uCsp1OmBTaT8MH 6X8g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:dkim-signature:delivered-to; bh=SJt6gsYiecM4ab+WrUMbUGgkjNbYZhdpcZDBT3fkMvI=; b=snOLgUPVnVDAxin1jEQx0Ktl6NbqndL98PU843Fr74zMdL0KpDhMjumKBQGMcUrsAu 1RdAtWKPUYnk36DwifxEyh75hT+TPGmfnmeKBwyKUjnscJcJDsO91Ft8vC2loLaNfjVL mN/D8oTJGQ6xr3STjcd/67Tn2KPI3ZMiEOCJacGUlnVgVxRU+hi9jjXxzrcJrlODGk+w wyVCL4caWJ6orpUZ4VPlxVTqJCEDS8SYpV3OUq7ytJZrXZ10K0+cwom+vKLT0bji28mu aEfoKTzFH1PuWURZNpYfHESVPbF9rRlttCv+s7fMD+UVGXORXF6cXnwGUK/IF+85eevR GD7w== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@rcombs.me header.s=google header.b=HIqKL7k7; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=rcombs.me Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id g4si24963858ejt.338.2021.11.13.13.11.51; Sat, 13 Nov 2021 13:11:51 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@rcombs.me header.s=google header.b=HIqKL7k7; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=rcombs.me Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 26DC168ADE7; Sat, 13 Nov 2021 23:09:43 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-io1-f46.google.com (mail-io1-f46.google.com [209.85.166.46]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 072F668AD50 for ; Sat, 13 Nov 2021 23:09:34 +0200 (EET) Received: by mail-io1-f46.google.com with SMTP id k21so15850210ioh.4 for ; Sat, 13 Nov 2021 13:09:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rcombs.me; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=lQXUUSfpwLrgBpK8qtkkRZmHcoNryYVcHUT3X55DdMA=; b=HIqKL7k7louaX5M6NHQvGZ/kzKYM/qYga7bkPvjLWH14zG9hc+Ney2wC3YKF528VkQ tvWqCHk5xa18MoeQS0bGfJKCHw5jPbLubw5BY8gNIh8GVM1U+bMa+2CMm0lKrWb2o/5Q hH7k7fKeA7r4/fSygSUDVGdj4HrLAQoS7JACf1lbzoy+FvixSU9Ht99u/PgSm0K68lt5 1OCLu4rMX2E4h73MNDQmSHw7LflclLPG3mKIP6NaPAI+QRMzef0AoAG/F7XLdpOUL3DP 8C0w5usmoPdwgywrYJHKIXQn4oCKqA18/QB5qwOk0WVCHuRRZvPCAgfXcBcg1YsqvuuH vsNg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=lQXUUSfpwLrgBpK8qtkkRZmHcoNryYVcHUT3X55DdMA=; b=sJ5RrXZtUErhoM0BvQIoYHaTKvljgug7svUF8cJLRrG3MIbSX8n9HVW6PXUubFjGDC WMbgp1qBvY3QdVbRlK3JQlWkD396pR4fyuiwvY/ffgizOmFFx1rMB5EjVmCmZSaEF05H 4akvz2pHBXMLwyrCrGlteXtMb2kE/AZjw1IbxTElJinmz84EjIo4epvFYEBvU7bNZuAW LVSdme9toxehVvfcD8FRnvUI81pd+NdgFovcGV/K/Xc8yE47a7FpOMkCckdjCPMgFcDS IsceDdIXAIXBJjnuLJEc3XLHT6VzFxyDkpmlD1OkJgwr3431B228V/36B6lhOkpl4ZDJ 7h9g== X-Gm-Message-State: AOAM531uHXhDy3gHv6dn3yD74hcp1DClxfQOZGzQjcX1UmI0hH/S47N5 eneIVWLvYMJCJuURGSmmJmoIuCpOCSZlr7g= X-Received: by 2002:a05:6638:2656:: with SMTP id n22mr18454535jat.35.1636837773412; Sat, 13 Nov 2021 13:09:33 -0800 (PST) Received: from rcombs-mbp.localdomain ([2601:243:2000:5ac:516:f322:7db9:ae0a]) by smtp.gmail.com with ESMTPSA id j23sm5243896iog.53.2021.11.13.13.09.32 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sat, 13 Nov 2021 13:09:32 -0800 (PST) From: rcombs To: ffmpeg-devel@ffmpeg.org Date: Sat, 13 Nov 2021 15:09:16 -0600 Message-Id: <20211113210916.49167-15-rcombs@rcombs.me> X-Mailer: git-send-email 2.33.1 In-Reply-To: <20211113210916.49167-1-rcombs@rcombs.me> References: <20211113210916.49167-1-rcombs@rcombs.me> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 15/15] lavc/proresdec: add videotoolbox hwaccel X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: AWf2Wr0LKW7c --- configure | 2 + libavcodec/hwaccels.h | 1 + libavcodec/proresdec2.c | 12 +++++- libavcodec/videotoolbox.c | 77 ++++++++++++++++++++++++++++++++++++++- 4 files changed, 89 insertions(+), 3 deletions(-) diff --git a/configure b/configure index 2c6dd59e62..6983b8af7a 100755 --- a/configure +++ b/configure @@ -3054,6 +3054,8 @@ mpeg4_vdpau_hwaccel_deps="vdpau" mpeg4_vdpau_hwaccel_select="mpeg4_decoder" mpeg4_videotoolbox_hwaccel_deps="videotoolbox" mpeg4_videotoolbox_hwaccel_select="mpeg4_decoder" +prores_videotoolbox_hwaccel_deps="videotoolbox" +prores_videotoolbox_hwaccel_select="prores_decoder" vc1_d3d11va_hwaccel_deps="d3d11va" vc1_d3d11va_hwaccel_select="vc1_decoder" vc1_d3d11va2_hwaccel_deps="d3d11va" diff --git a/libavcodec/hwaccels.h b/libavcodec/hwaccels.h index 65e778f3e4..1e7b464950 100644 --- a/libavcodec/hwaccels.h +++ b/libavcodec/hwaccels.h @@ -60,6 +60,7 @@ extern const AVHWAccel ff_mpeg4_nvdec_hwaccel; extern const AVHWAccel ff_mpeg4_vaapi_hwaccel; extern const AVHWAccel ff_mpeg4_vdpau_hwaccel; extern const AVHWAccel ff_mpeg4_videotoolbox_hwaccel; +extern const AVHWAccel ff_prores_videotoolbox_hwaccel; extern const AVHWAccel ff_vc1_d3d11va_hwaccel; extern const AVHWAccel ff_vc1_d3d11va2_hwaccel; extern const AVHWAccel ff_vc1_dxva2_hwaccel; diff --git a/libavcodec/proresdec2.c b/libavcodec/proresdec2.c index 5b1d7da693..719194f21e 100644 --- a/libavcodec/proresdec2.c +++ b/libavcodec/proresdec2.c @@ -33,6 +33,7 @@ #include "avcodec.h" #include "get_bits.h" +#include "hwconfig.h" #include "idctdsp.h" #include "internal.h" #include "profiles.h" @@ -268,12 +269,15 @@ static int decode_frame_header(ProresContext *ctx, const uint8_t *buf, } if (pix_fmt != ctx->pix_fmt) { -#define HWACCEL_MAX 0 +#define HWACCEL_MAX (CONFIG_PRORES_VIDEOTOOLBOX_HWACCEL) enum AVPixelFormat pix_fmts[HWACCEL_MAX + 2], *fmtp = pix_fmts; int ret; ctx->pix_fmt = pix_fmt; +#if CONFIG_PRORES_VIDEOTOOLBOX_HWACCEL + *fmtp++ = AV_PIX_FMT_VIDEOTOOLBOX; +#endif *fmtp++ = ctx->pix_fmt; *fmtp = AV_PIX_FMT_NONE; @@ -864,4 +868,10 @@ const AVCodec ff_prores_decoder = { .capabilities = AV_CODEC_CAP_DR1 | AV_CODEC_CAP_SLICE_THREADS | AV_CODEC_CAP_FRAME_THREADS, .profiles = NULL_IF_CONFIG_SMALL(ff_prores_profiles), .caps_internal = FF_CODEC_CAP_INIT_THREADSAFE, + .hw_configs = (const AVCodecHWConfigInternal *const []) { +#if CONFIG_PRORES_VIDEOTOOLBOX_HWACCEL + HWACCEL_VIDEOTOOLBOX(prores), +#endif + NULL + }, }; diff --git a/libavcodec/videotoolbox.c b/libavcodec/videotoolbox.c index 9ca84e873d..029c1c09c7 100644 --- a/libavcodec/videotoolbox.c +++ b/libavcodec/videotoolbox.c @@ -32,6 +32,7 @@ #include "h264dec.h" #include "hevcdec.h" #include "mpegvideo.h" +#include "proresdec.h" #include #include @@ -183,7 +184,6 @@ CFDataRef ff_videotoolbox_avcc_extradata_create(AVCodecContext *avctx) int pps_size = escape_ps(NULL, h->ps.pps->data, h->ps.pps->data_size); int vt_extradata_size; uint8_t *vt_extradata; - int i; vt_extradata_size = 6 + 2 + sps_size + 3 + pps_size; vt_extradata = av_malloc(vt_extradata_size); @@ -870,6 +870,31 @@ static int videotoolbox_start(AVCodecContext *avctx) case AV_CODEC_ID_MPEG4 : videotoolbox->cm_codec_type = kCMVideoCodecType_MPEG4Video; break; + case AV_CODEC_ID_PRORES : + switch (avctx->codec_tag) { + case MKTAG('a','p','c','o'): + videotoolbox->cm_codec_type = kCMVideoCodecType_AppleProRes422Proxy; + break; + case MKTAG('a','p','c','s'): + videotoolbox->cm_codec_type = kCMVideoCodecType_AppleProRes422LT; + break; + case MKTAG('a','p','c','n'): + videotoolbox->cm_codec_type = kCMVideoCodecType_AppleProRes422; + break; + case MKTAG('a','p','c','h'): + videotoolbox->cm_codec_type = kCMVideoCodecType_AppleProRes422HQ; + break; + case MKTAG('a','p','4','h'): + videotoolbox->cm_codec_type = kCMVideoCodecType_AppleProRes4444; + break; + case MKTAG('a','p','4','x'): + videotoolbox->cm_codec_type = kCMVideoCodecType_AppleProRes4444XQ; + break; + default: + videotoolbox->cm_codec_type = avctx->codec_tag; + av_log(avctx, AV_LOG_WARNING, "Unknown prores profile %d\n", avctx->codec_tag); + } + break; case AV_CODEC_ID_VP9 : videotoolbox->cm_codec_type = kCMVideoCodecType_VP9; break; @@ -877,6 +902,14 @@ static int videotoolbox_start(AVCodecContext *avctx) break; } +#ifdef __MAC_10_9 + if (avctx->codec_id == AV_CODEC_ID_PRORES) { + if (__builtin_available(macOS 10.9, *)) { + VTRegisterProfessionalVideoWorkflowVideoDecoders(); + } + } +#endif + #ifdef __MAC_10_11 if (__builtin_available(macOS 10.11, *)) { VTRegisterSupplementalVideoDecoderIfAvailable(videotoolbox->cm_codec_type); @@ -1065,6 +1098,30 @@ static int videotoolbox_mpeg_end_frame(AVCodecContext *avctx) return ff_videotoolbox_common_end_frame(avctx, frame); } +static int videotoolbox_prores_start_frame(AVCodecContext *avctx, + const uint8_t *buffer, + uint32_t size) +{ + return 0; +} + +static int videotoolbox_prores_decode_slice(AVCodecContext *avctx, + const uint8_t *buffer, + uint32_t size) +{ + VTContext *vtctx = avctx->internal->hwaccel_priv_data; + + return ff_videotoolbox_buffer_copy(vtctx, buffer, size); +} + +static int videotoolbox_prores_end_frame(AVCodecContext *avctx) +{ + ProresContext *ctx = avctx->priv_data; + AVFrame *frame = ctx->frame; + + return ff_videotoolbox_common_end_frame(avctx, frame); +} + static enum AVPixelFormat videotoolbox_best_pixel_format(AVCodecContext *avctx) { const AVPixFmtDescriptor *descriptor = av_pix_fmt_desc_get(avctx->sw_pix_fmt); if (!descriptor) @@ -1076,7 +1133,8 @@ static enum AVPixelFormat videotoolbox_best_pixel_format(AVCodecContext *avctx) return AV_PIX_FMT_AYUV64; #if HAVE_KCVPIXELFORMATTYPE_444YPCBCR16BIPLANARVIDEORANGE - if (depth > 10) + // ProRes has issues when decoding to 10-bit, so always use 16 + if (depth > 10 || avctx->codec_id == AV_CODEC_ID_PRORES) return descriptor->log2_chroma_w == 0 ? AV_PIX_FMT_P416 : AV_PIX_FMT_P216; #endif @@ -1288,6 +1346,21 @@ const AVHWAccel ff_mpeg4_videotoolbox_hwaccel = { .priv_data_size = sizeof(VTContext), }; +const AVHWAccel ff_prores_videotoolbox_hwaccel = { + .name = "prores_videotoolbox", + .type = AVMEDIA_TYPE_VIDEO, + .id = AV_CODEC_ID_PRORES, + .pix_fmt = AV_PIX_FMT_VIDEOTOOLBOX, + .alloc_frame = ff_videotoolbox_alloc_frame, + .start_frame = videotoolbox_prores_start_frame, + .decode_slice = videotoolbox_prores_decode_slice, + .end_frame = videotoolbox_prores_end_frame, + .frame_params = ff_videotoolbox_frame_params, + .init = ff_videotoolbox_common_init, + .uninit = ff_videotoolbox_uninit, + .priv_data_size = sizeof(VTContext), +}; + static AVVideotoolboxContext *av_videotoolbox_alloc_context_with_pix_fmt(enum AVPixelFormat pix_fmt, bool full_range) {