From patchwork Mon Nov 22 21:58:20 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: rcombs X-Patchwork-Id: 31546 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a6b:d206:0:0:0:0:0 with SMTP id q6csp7668066iob; Mon, 22 Nov 2021 14:00:50 -0800 (PST) X-Google-Smtp-Source: ABdhPJw4ySd4LbMIOAD4jlCeVd4V6g8jXc54qhyF88HWS0nyN/vxuQkCoz/yo1GioAmM5C8Lt77K X-Received: by 2002:a17:907:dab:: with SMTP id go43mr645411ejc.537.1637618450815; Mon, 22 Nov 2021 14:00:50 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1637618450; cv=none; d=google.com; s=arc-20160816; b=inRZU0yTR1tsrh6MPSiSW5YYSNcAgIuJ8w4rjGQmPT29URdcYO8V/firNbV/noiX1d qQcP4OGo+TaVArWrugOBn/OLuvXbUjDe8eKYU0kHsQ5+0sGFGCsVAl0aoO+jn39SWzoK y+zKg6t1XGbLQ5FK95bfoy2N/1E5YxLkGJvZyD9O+jhcZGHHHPKPoABrmDy0kuCHEtnM zaEXAc/MaLHNXbYYsSy6EddpoXIP0nGtayzYg9hTPeZc869JbEKd/HlZ77uOcqaXvKhu n+MrRjxeQH9tSwcPrZAUW1HaTPzuVe1P1KwmZQwDxci8qwi71tcnn2bf13Vwd7efRS6y r0xw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:dkim-signature:delivered-to; bh=ovQWuGDSrk3bZIlBe0GvCfYgY03bZmq8V+NbuWomrhQ=; b=EZevc3kaluS0/oUrE8HROQZq+p746ReJ4cyzQlugeBRp5A0LL2XBWS1McVafvnZGJX mI1MHbDezQ7cIt/KDORY2MYzKRxj6icLKrAXXqXJnbvuZ38B4A7uEYrCHlKVorV1/9Zm pk1fDsFieKx/+h9teIRtAtv5Nj+jTU4vLwLdOqQtaxvc0o1xJkm7vwXYqwhg5OlViFjv Bc/lA1AY4xCj0gnuJrKgRqsLJZqUoU+rdI4yIA68BGyKhUJJQqtDwSHjyYtoqx1GfjEW 3KLcc3o+NruVOX8GDy30RON9xyFG/HL8mZGEfIpO8HPzeOe3K514fjNCteJbAevoxKa1 HY1Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@rcombs.me header.s=google header.b=T+EYoSVg; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=rcombs.me Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id q23si21926658edw.289.2021.11.22.14.00.48; Mon, 22 Nov 2021 14:00:50 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@rcombs.me header.s=google header.b=T+EYoSVg; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=rcombs.me Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 7BF4C68AC4D; Mon, 22 Nov 2021 23:58:51 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-io1-f49.google.com (mail-io1-f49.google.com [209.85.166.49]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 3A0E368AE68 for ; Mon, 22 Nov 2021 23:58:45 +0200 (EET) Received: by mail-io1-f49.google.com with SMTP id c3so25391349iob.6 for ; Mon, 22 Nov 2021 13:58:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rcombs.me; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=fu55E76kt4YRwjp34oQVtylB7Mnd5wNBLpEZClGcfpU=; b=T+EYoSVgXrPqbTeqMWxepAgJdJBnoHQTDEM69r/ZXSsfeDWbv9fv28Iyoh3Nle8KrI Jm4OtUIKnRHxV7vl2e0EcV7mH2HFchHf0cPj5gD7g38/VbqV5cOEfcjNVsB33FpSfj3W Yy9dyM8Ol4aokt4BXmHJUL5Wr03oCA3RzI1iQWHff7XnQYVlTznQdf8POosWRHJvnbIW AwC+PowH9by+o0FeffP94C4uN+jcz6PDYMK5g8OuTeOivQCX3oYsJbznGiVHjBgqKCIn W2x+E74MFzfusds8dwRWl1JSXw/xKHbpGW0i9Tgr84apcKsUQK33z2/WJZT6tcRj4JCb bCaw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=fu55E76kt4YRwjp34oQVtylB7Mnd5wNBLpEZClGcfpU=; b=zhogp6Cu3/rZkzl3cC82D10H5UHbM7cB7r6kuZN8TUoBv9lAUxn7O+d+ez0jKTWmyM b4M/sFtKgr8RqdgwTGGb8rg1Lg/MZsG6cN2z/bRl62B1AuFg/hikIb3Hnejn/njlX5ar NDF/MRZcjtWJRBf3h8WIM4cmU3lP0U6yZqJPsN6VoM+jo0h/gdIDT7/2T9nmZI0HCQjk CvD/RbV2ry9riUgpT4caO4xFaGEupLU02ChcXeXHPc7lFq6vgG56A1l0dUSJfY6+1eb+ 0XkUoH11Zj56sH7qH54skrHe5zo1531BYWBV3L2M/EAni5YF74Va20D6F406uBqLzKLS 6LuA== X-Gm-Message-State: AOAM532Mphtm0G1URjuPrqGtsgZRYAStvoT+MzmXdkIPulQth7EJRcJW DefxvUuLtbi36I+Qme5yr6sYpELZnS9vh/c= X-Received: by 2002:a02:ab8f:: with SMTP id t15mr251437jan.147.1637618323586; Mon, 22 Nov 2021 13:58:43 -0800 (PST) Received: from rcombs-mbp.localdomain ([2601:243:2000:5ac:edf5:499:2479:42a8]) by smtp.gmail.com with ESMTPSA id o1sm5668802ilj.41.2021.11.22.13.58.42 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 22 Nov 2021 13:58:43 -0800 (PST) From: rcombs To: ffmpeg-devel@ffmpeg.org Date: Mon, 22 Nov 2021 15:58:20 -0600 Message-Id: <20211122215821.9849-15-rcombs@rcombs.me> X-Mailer: git-send-email 2.33.1 In-Reply-To: <20211122215821.9849-1-rcombs@rcombs.me> References: <20211122215821.9849-1-rcombs@rcombs.me> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 15/16] lavc/proresdec: add videotoolbox hwaccel X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 7MPJZdmURGah --- Changelog | 1 + configure | 2 ++ libavcodec/hwaccels.h | 1 + libavcodec/proresdec2.c | 12 ++++++- libavcodec/videotoolbox.c | 74 ++++++++++++++++++++++++++++++++++++++- 5 files changed, 88 insertions(+), 2 deletions(-) diff --git a/Changelog b/Changelog index d6be6bbe4f..e80b3fd82d 100644 --- a/Changelog +++ b/Changelog @@ -32,6 +32,7 @@ version : - huesaturation video filter - colorspectrum source video filter - VideoToolbox VP9 hwaccel +- VideoToolbox ProRes hwaccel version 4.4: diff --git a/configure b/configure index 262261b544..599ced3df6 100755 --- a/configure +++ b/configure @@ -3056,6 +3056,8 @@ mpeg4_vdpau_hwaccel_deps="vdpau" mpeg4_vdpau_hwaccel_select="mpeg4_decoder" mpeg4_videotoolbox_hwaccel_deps="videotoolbox" mpeg4_videotoolbox_hwaccel_select="mpeg4_decoder" +prores_videotoolbox_hwaccel_deps="videotoolbox" +prores_videotoolbox_hwaccel_select="prores_decoder" vc1_d3d11va_hwaccel_deps="d3d11va" vc1_d3d11va_hwaccel_select="vc1_decoder" vc1_d3d11va2_hwaccel_deps="d3d11va" diff --git a/libavcodec/hwaccels.h b/libavcodec/hwaccels.h index 65e778f3e4..1e7b464950 100644 --- a/libavcodec/hwaccels.h +++ b/libavcodec/hwaccels.h @@ -60,6 +60,7 @@ extern const AVHWAccel ff_mpeg4_nvdec_hwaccel; extern const AVHWAccel ff_mpeg4_vaapi_hwaccel; extern const AVHWAccel ff_mpeg4_vdpau_hwaccel; extern const AVHWAccel ff_mpeg4_videotoolbox_hwaccel; +extern const AVHWAccel ff_prores_videotoolbox_hwaccel; extern const AVHWAccel ff_vc1_d3d11va_hwaccel; extern const AVHWAccel ff_vc1_d3d11va2_hwaccel; extern const AVHWAccel ff_vc1_dxva2_hwaccel; diff --git a/libavcodec/proresdec2.c b/libavcodec/proresdec2.c index 5b1d7da693..719194f21e 100644 --- a/libavcodec/proresdec2.c +++ b/libavcodec/proresdec2.c @@ -33,6 +33,7 @@ #include "avcodec.h" #include "get_bits.h" +#include "hwconfig.h" #include "idctdsp.h" #include "internal.h" #include "profiles.h" @@ -268,12 +269,15 @@ static int decode_frame_header(ProresContext *ctx, const uint8_t *buf, } if (pix_fmt != ctx->pix_fmt) { -#define HWACCEL_MAX 0 +#define HWACCEL_MAX (CONFIG_PRORES_VIDEOTOOLBOX_HWACCEL) enum AVPixelFormat pix_fmts[HWACCEL_MAX + 2], *fmtp = pix_fmts; int ret; ctx->pix_fmt = pix_fmt; +#if CONFIG_PRORES_VIDEOTOOLBOX_HWACCEL + *fmtp++ = AV_PIX_FMT_VIDEOTOOLBOX; +#endif *fmtp++ = ctx->pix_fmt; *fmtp = AV_PIX_FMT_NONE; @@ -864,4 +868,10 @@ const AVCodec ff_prores_decoder = { .capabilities = AV_CODEC_CAP_DR1 | AV_CODEC_CAP_SLICE_THREADS | AV_CODEC_CAP_FRAME_THREADS, .profiles = NULL_IF_CONFIG_SMALL(ff_prores_profiles), .caps_internal = FF_CODEC_CAP_INIT_THREADSAFE, + .hw_configs = (const AVCodecHWConfigInternal *const []) { +#if CONFIG_PRORES_VIDEOTOOLBOX_HWACCEL + HWACCEL_VIDEOTOOLBOX(prores), +#endif + NULL + }, }; diff --git a/libavcodec/videotoolbox.c b/libavcodec/videotoolbox.c index 542fe9316a..40d231acc1 100644 --- a/libavcodec/videotoolbox.c +++ b/libavcodec/videotoolbox.c @@ -32,6 +32,7 @@ #include "h264dec.h" #include "hevcdec.h" #include "mpegvideo.h" +#include "proresdec.h" #include #include #include @@ -186,7 +187,6 @@ CFDataRef ff_videotoolbox_avcc_extradata_create(AVCodecContext *avctx) int pps_size = escape_ps(NULL, h->ps.pps->data, h->ps.pps->data_size); int vt_extradata_size; uint8_t *vt_extradata; - int i; vt_extradata_size = 6 + 2 + sps_size + 3 + pps_size; vt_extradata = av_malloc(vt_extradata_size); @@ -873,6 +873,31 @@ static int videotoolbox_start(AVCodecContext *avctx) case AV_CODEC_ID_MPEG4 : videotoolbox->cm_codec_type = kCMVideoCodecType_MPEG4Video; break; + case AV_CODEC_ID_PRORES : + switch (avctx->codec_tag) { + case MKTAG('a','p','c','o'): + videotoolbox->cm_codec_type = kCMVideoCodecType_AppleProRes422Proxy; + break; + case MKTAG('a','p','c','s'): + videotoolbox->cm_codec_type = kCMVideoCodecType_AppleProRes422LT; + break; + case MKTAG('a','p','c','n'): + videotoolbox->cm_codec_type = kCMVideoCodecType_AppleProRes422; + break; + case MKTAG('a','p','c','h'): + videotoolbox->cm_codec_type = kCMVideoCodecType_AppleProRes422HQ; + break; + case MKTAG('a','p','4','h'): + videotoolbox->cm_codec_type = kCMVideoCodecType_AppleProRes4444; + break; + case MKTAG('a','p','4','x'): + videotoolbox->cm_codec_type = kCMVideoCodecType_AppleProRes4444XQ; + break; + default: + videotoolbox->cm_codec_type = avctx->codec_tag; + av_log(avctx, AV_LOG_WARNING, "Unknown prores profile %d\n", avctx->codec_tag); + } + break; case AV_CODEC_ID_VP9 : videotoolbox->cm_codec_type = kCMVideoCodecType_VP9; break; @@ -880,6 +905,14 @@ static int videotoolbox_start(AVCodecContext *avctx) break; } +#if defined(MAC_OS_X_VERSION_10_9) && !TARGET_OS_IPHONE && (MAC_OS_X_VERSION_MAX_ALLOWED >= MAC_OS_X_VERSION_10_9) + if (avctx->codec_id == AV_CODEC_ID_PRORES) { + if (__builtin_available(macOS 10.9, *)) { + VTRegisterProfessionalVideoWorkflowVideoDecoders(); + } + } +#endif + #if defined(MAC_OS_VERSION_11_0) && !TARGET_OS_IPHONE && (MAC_OS_X_VERSION_MAX_ALLOWED >= MAC_OS_VERSION_11_0) if (__builtin_available(macOS 11.0, *)) { VTRegisterSupplementalVideoDecoderIfAvailable(videotoolbox->cm_codec_type); @@ -1068,6 +1101,30 @@ static int videotoolbox_mpeg_end_frame(AVCodecContext *avctx) return ff_videotoolbox_common_end_frame(avctx, frame); } +static int videotoolbox_prores_start_frame(AVCodecContext *avctx, + const uint8_t *buffer, + uint32_t size) +{ + return 0; +} + +static int videotoolbox_prores_decode_slice(AVCodecContext *avctx, + const uint8_t *buffer, + uint32_t size) +{ + VTContext *vtctx = avctx->internal->hwaccel_priv_data; + + return ff_videotoolbox_buffer_copy(vtctx, buffer, size); +} + +static int videotoolbox_prores_end_frame(AVCodecContext *avctx) +{ + ProresContext *ctx = avctx->priv_data; + AVFrame *frame = ctx->frame; + + return ff_videotoolbox_common_end_frame(avctx, frame); +} + static enum AVPixelFormat videotoolbox_best_pixel_format(AVCodecContext *avctx) { const AVPixFmtDescriptor *descriptor = av_pix_fmt_desc_get(avctx->sw_pix_fmt); if (!descriptor) @@ -1291,6 +1348,21 @@ const AVHWAccel ff_mpeg4_videotoolbox_hwaccel = { .priv_data_size = sizeof(VTContext), }; +const AVHWAccel ff_prores_videotoolbox_hwaccel = { + .name = "prores_videotoolbox", + .type = AVMEDIA_TYPE_VIDEO, + .id = AV_CODEC_ID_PRORES, + .pix_fmt = AV_PIX_FMT_VIDEOTOOLBOX, + .alloc_frame = ff_videotoolbox_alloc_frame, + .start_frame = videotoolbox_prores_start_frame, + .decode_slice = videotoolbox_prores_decode_slice, + .end_frame = videotoolbox_prores_end_frame, + .frame_params = ff_videotoolbox_frame_params, + .init = ff_videotoolbox_common_init, + .uninit = ff_videotoolbox_uninit, + .priv_data_size = sizeof(VTContext), +}; + static AVVideotoolboxContext *av_videotoolbox_alloc_context_with_pix_fmt(enum AVPixelFormat pix_fmt, bool full_range) {