From patchwork Mon Nov 22 20:53:29 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ridley Combs X-Patchwork-Id: 31531 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a6b:d206:0:0:0:0:0 with SMTP id q6csp7585319iob; Mon, 22 Nov 2021 12:56:28 -0800 (PST) X-Google-Smtp-Source: ABdhPJx8h1SxChZ6uzSFB/BPUOptoINfmQjQ5Tg1fsrvNrIANl9DzKSLWiqi5Zm9anwpTWYtABR3 X-Received: by 2002:a17:906:4e42:: with SMTP id g2mr182298ejw.230.1637614588024; Mon, 22 Nov 2021 12:56:28 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1637614588; cv=none; d=google.com; s=arc-20160816; b=FyV/jQM9LEsMC/rLVm1CJDQKKOMSbU8Y5PzFWz+C7edq0Dhtu08RGnxP5h5m6ijhAC rE03VFMNGrrQDAYaK6iaUZEoz+wN6paiHlkurAQCk0D9qh5wCKwCED/ZUpq7rkmBRt9J K5OMSdoU8783o+zqNjCYzenRDCUIdurMRFSB5UXIQw++KJrPep79L3T+3W+llrxVXwBk 5wGC1778fu1pi9tD7wD4apP+KLTmnxdgfd96V22vwl4BZeoMSUjYL4zk+Af03naxuaOg HMmzUrkO8CNTUzigPasgZsP+zK2UtwdJfdht+kOJ/cvJQYluSkxuTYGfZ6t8fsZmjRxi aBwQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:dkim-signature:delivered-to; bh=ovQWuGDSrk3bZIlBe0GvCfYgY03bZmq8V+NbuWomrhQ=; b=oQUeH9AHhzYKPUslnsM/1xFJcgNHL/UbYnhP9BiGCuBrvYWDGB0K2BUdhcFpG1bwW0 CJj4tvcrqvjMs2/6yOdBxzwyDbs5wHpmuv49Q30ul1k58P0LIT19FyKoJ/ZuKmBhaTFi nEcQF77LT5iuXb9K8ewcFyTExgwAFHBoAjTdpveKerYqlMLNsHFM5cqQa4X4euL8QynV Ju5mrPSoSmAm4LcaHwJzxp9k071nD9b4OcZnmSGrEWLTaCjpynS/eIJugAesG1G9HEZ2 sOKWO1Kk9KWnclKvYjFwyWP56xxZogu+i9ZcfugFcLPWFMNh4WMu5nOMtKSEg2m8o7Qd NqHA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@rcombs.me header.s=google header.b=gnSRxcGK; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=rcombs.me Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id p14si23519344edi.622.2021.11.22.12.56.27; Mon, 22 Nov 2021 12:56:28 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@rcombs.me header.s=google header.b=gnSRxcGK; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=rcombs.me Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 1AA7868AE42; Mon, 22 Nov 2021 22:53:58 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-il1-f172.google.com (mail-il1-f172.google.com [209.85.166.172]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 771E568A755 for ; Mon, 22 Nov 2021 22:53:48 +0200 (EET) Received: by mail-il1-f172.google.com with SMTP id t8so9584056ilu.8 for ; Mon, 22 Nov 2021 12:53:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rcombs.me; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=fu55E76kt4YRwjp34oQVtylB7Mnd5wNBLpEZClGcfpU=; b=gnSRxcGKF52/il0t9R+QXq/ly9TPcXDieEoOAJAzTI+QOam9PjbIVnJK+aypiX43aL k6jdBkOZWAznrnZxBQxa0pstL9rCSmLrN/Vi2UqqRQ5IzwQMtkaB8wV2iJGJ6CEqiebf f7v0G9+G8SPQiD0DiLQViH6QeIu/S7O0Gf7tEpgsq/WcZDAPav5GG0SKwjWKJRcJyedw 4hNsj8ZHc0UnD/CfZFTSuGJ2oHvv7B7z7h8jrC5zeH8uH4ZDS9rx92nuVqc+rpAPBi+4 7SbfNQeYj9FtTl9vBr93OsV2udydQKjZkR8mVUcCy82WgxmIpfovvtDjM+fT2+TYXz3s IaWw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=fu55E76kt4YRwjp34oQVtylB7Mnd5wNBLpEZClGcfpU=; b=hs2o+YZ5qD3kOFQD4nOxRbRH8cgqt4NUD+iHBKMeqRAVECVvI6J+Z97dtw39nr04TE OLTjgOtr/8DwaQuhHor5pLZzHJD7L+ijBnLzUvPBphqOzGwCfknW7hspvO7E1v/hB91e jL4/+KclrSAaFNn4WZv+3P68NTqh93LLL+qRniv/d3RdbxCZGONl6Hb0NToFsxgFkQo8 ESrKfdEZ9VNoejxrUQECaZBRrrVnPx2rvB+fjpUmxzKROZsP6Sbl243KOC+k7BQ+iJut 7FCwWS+KWmy6gg79m9Zgww1piiu8+Iyvgcjzm/gQ3JGKLnECf8bkeFCVld3yyX/+rly6 ZOgg== X-Gm-Message-State: AOAM533USr1vnNtk0rAmn/n6Tr9pem5yi1LKm/Cr1z0I2d8xzySXFSDQ QaBl0yiLkLfRUbWwIjn4hlh9YWh5kt+Og0w= X-Received: by 2002:a05:6e02:1e02:: with SMTP id g2mr22782534ila.290.1637614426733; Mon, 22 Nov 2021 12:53:46 -0800 (PST) Received: from rcombs-mbp.localdomain ([2601:243:2000:5ac:edf5:499:2479:42a8]) by smtp.gmail.com with ESMTPSA id b3sm3571580ile.26.2021.11.22.12.53.45 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 22 Nov 2021 12:53:46 -0800 (PST) From: rcombs To: ffmpeg-devel@ffmpeg.org Date: Mon, 22 Nov 2021 14:53:29 -0600 Message-Id: <20211122205330.43546-15-rcombs@rcombs.me> X-Mailer: git-send-email 2.33.1 In-Reply-To: <20211122205330.43546-1-rcombs@rcombs.me> References: <20211122205330.43546-1-rcombs@rcombs.me> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 15/16] lavc/proresdec: add videotoolbox hwaccel X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: WtS+mOAE9qoQ --- Changelog | 1 + configure | 2 ++ libavcodec/hwaccels.h | 1 + libavcodec/proresdec2.c | 12 ++++++- libavcodec/videotoolbox.c | 74 ++++++++++++++++++++++++++++++++++++++- 5 files changed, 88 insertions(+), 2 deletions(-) diff --git a/Changelog b/Changelog index d6be6bbe4f..e80b3fd82d 100644 --- a/Changelog +++ b/Changelog @@ -32,6 +32,7 @@ version : - huesaturation video filter - colorspectrum source video filter - VideoToolbox VP9 hwaccel +- VideoToolbox ProRes hwaccel version 4.4: diff --git a/configure b/configure index 262261b544..599ced3df6 100755 --- a/configure +++ b/configure @@ -3056,6 +3056,8 @@ mpeg4_vdpau_hwaccel_deps="vdpau" mpeg4_vdpau_hwaccel_select="mpeg4_decoder" mpeg4_videotoolbox_hwaccel_deps="videotoolbox" mpeg4_videotoolbox_hwaccel_select="mpeg4_decoder" +prores_videotoolbox_hwaccel_deps="videotoolbox" +prores_videotoolbox_hwaccel_select="prores_decoder" vc1_d3d11va_hwaccel_deps="d3d11va" vc1_d3d11va_hwaccel_select="vc1_decoder" vc1_d3d11va2_hwaccel_deps="d3d11va" diff --git a/libavcodec/hwaccels.h b/libavcodec/hwaccels.h index 65e778f3e4..1e7b464950 100644 --- a/libavcodec/hwaccels.h +++ b/libavcodec/hwaccels.h @@ -60,6 +60,7 @@ extern const AVHWAccel ff_mpeg4_nvdec_hwaccel; extern const AVHWAccel ff_mpeg4_vaapi_hwaccel; extern const AVHWAccel ff_mpeg4_vdpau_hwaccel; extern const AVHWAccel ff_mpeg4_videotoolbox_hwaccel; +extern const AVHWAccel ff_prores_videotoolbox_hwaccel; extern const AVHWAccel ff_vc1_d3d11va_hwaccel; extern const AVHWAccel ff_vc1_d3d11va2_hwaccel; extern const AVHWAccel ff_vc1_dxva2_hwaccel; diff --git a/libavcodec/proresdec2.c b/libavcodec/proresdec2.c index 5b1d7da693..719194f21e 100644 --- a/libavcodec/proresdec2.c +++ b/libavcodec/proresdec2.c @@ -33,6 +33,7 @@ #include "avcodec.h" #include "get_bits.h" +#include "hwconfig.h" #include "idctdsp.h" #include "internal.h" #include "profiles.h" @@ -268,12 +269,15 @@ static int decode_frame_header(ProresContext *ctx, const uint8_t *buf, } if (pix_fmt != ctx->pix_fmt) { -#define HWACCEL_MAX 0 +#define HWACCEL_MAX (CONFIG_PRORES_VIDEOTOOLBOX_HWACCEL) enum AVPixelFormat pix_fmts[HWACCEL_MAX + 2], *fmtp = pix_fmts; int ret; ctx->pix_fmt = pix_fmt; +#if CONFIG_PRORES_VIDEOTOOLBOX_HWACCEL + *fmtp++ = AV_PIX_FMT_VIDEOTOOLBOX; +#endif *fmtp++ = ctx->pix_fmt; *fmtp = AV_PIX_FMT_NONE; @@ -864,4 +868,10 @@ const AVCodec ff_prores_decoder = { .capabilities = AV_CODEC_CAP_DR1 | AV_CODEC_CAP_SLICE_THREADS | AV_CODEC_CAP_FRAME_THREADS, .profiles = NULL_IF_CONFIG_SMALL(ff_prores_profiles), .caps_internal = FF_CODEC_CAP_INIT_THREADSAFE, + .hw_configs = (const AVCodecHWConfigInternal *const []) { +#if CONFIG_PRORES_VIDEOTOOLBOX_HWACCEL + HWACCEL_VIDEOTOOLBOX(prores), +#endif + NULL + }, }; diff --git a/libavcodec/videotoolbox.c b/libavcodec/videotoolbox.c index 542fe9316a..40d231acc1 100644 --- a/libavcodec/videotoolbox.c +++ b/libavcodec/videotoolbox.c @@ -32,6 +32,7 @@ #include "h264dec.h" #include "hevcdec.h" #include "mpegvideo.h" +#include "proresdec.h" #include #include #include @@ -186,7 +187,6 @@ CFDataRef ff_videotoolbox_avcc_extradata_create(AVCodecContext *avctx) int pps_size = escape_ps(NULL, h->ps.pps->data, h->ps.pps->data_size); int vt_extradata_size; uint8_t *vt_extradata; - int i; vt_extradata_size = 6 + 2 + sps_size + 3 + pps_size; vt_extradata = av_malloc(vt_extradata_size); @@ -873,6 +873,31 @@ static int videotoolbox_start(AVCodecContext *avctx) case AV_CODEC_ID_MPEG4 : videotoolbox->cm_codec_type = kCMVideoCodecType_MPEG4Video; break; + case AV_CODEC_ID_PRORES : + switch (avctx->codec_tag) { + case MKTAG('a','p','c','o'): + videotoolbox->cm_codec_type = kCMVideoCodecType_AppleProRes422Proxy; + break; + case MKTAG('a','p','c','s'): + videotoolbox->cm_codec_type = kCMVideoCodecType_AppleProRes422LT; + break; + case MKTAG('a','p','c','n'): + videotoolbox->cm_codec_type = kCMVideoCodecType_AppleProRes422; + break; + case MKTAG('a','p','c','h'): + videotoolbox->cm_codec_type = kCMVideoCodecType_AppleProRes422HQ; + break; + case MKTAG('a','p','4','h'): + videotoolbox->cm_codec_type = kCMVideoCodecType_AppleProRes4444; + break; + case MKTAG('a','p','4','x'): + videotoolbox->cm_codec_type = kCMVideoCodecType_AppleProRes4444XQ; + break; + default: + videotoolbox->cm_codec_type = avctx->codec_tag; + av_log(avctx, AV_LOG_WARNING, "Unknown prores profile %d\n", avctx->codec_tag); + } + break; case AV_CODEC_ID_VP9 : videotoolbox->cm_codec_type = kCMVideoCodecType_VP9; break; @@ -880,6 +905,14 @@ static int videotoolbox_start(AVCodecContext *avctx) break; } +#if defined(MAC_OS_X_VERSION_10_9) && !TARGET_OS_IPHONE && (MAC_OS_X_VERSION_MAX_ALLOWED >= MAC_OS_X_VERSION_10_9) + if (avctx->codec_id == AV_CODEC_ID_PRORES) { + if (__builtin_available(macOS 10.9, *)) { + VTRegisterProfessionalVideoWorkflowVideoDecoders(); + } + } +#endif + #if defined(MAC_OS_VERSION_11_0) && !TARGET_OS_IPHONE && (MAC_OS_X_VERSION_MAX_ALLOWED >= MAC_OS_VERSION_11_0) if (__builtin_available(macOS 11.0, *)) { VTRegisterSupplementalVideoDecoderIfAvailable(videotoolbox->cm_codec_type); @@ -1068,6 +1101,30 @@ static int videotoolbox_mpeg_end_frame(AVCodecContext *avctx) return ff_videotoolbox_common_end_frame(avctx, frame); } +static int videotoolbox_prores_start_frame(AVCodecContext *avctx, + const uint8_t *buffer, + uint32_t size) +{ + return 0; +} + +static int videotoolbox_prores_decode_slice(AVCodecContext *avctx, + const uint8_t *buffer, + uint32_t size) +{ + VTContext *vtctx = avctx->internal->hwaccel_priv_data; + + return ff_videotoolbox_buffer_copy(vtctx, buffer, size); +} + +static int videotoolbox_prores_end_frame(AVCodecContext *avctx) +{ + ProresContext *ctx = avctx->priv_data; + AVFrame *frame = ctx->frame; + + return ff_videotoolbox_common_end_frame(avctx, frame); +} + static enum AVPixelFormat videotoolbox_best_pixel_format(AVCodecContext *avctx) { const AVPixFmtDescriptor *descriptor = av_pix_fmt_desc_get(avctx->sw_pix_fmt); if (!descriptor) @@ -1291,6 +1348,21 @@ const AVHWAccel ff_mpeg4_videotoolbox_hwaccel = { .priv_data_size = sizeof(VTContext), }; +const AVHWAccel ff_prores_videotoolbox_hwaccel = { + .name = "prores_videotoolbox", + .type = AVMEDIA_TYPE_VIDEO, + .id = AV_CODEC_ID_PRORES, + .pix_fmt = AV_PIX_FMT_VIDEOTOOLBOX, + .alloc_frame = ff_videotoolbox_alloc_frame, + .start_frame = videotoolbox_prores_start_frame, + .decode_slice = videotoolbox_prores_decode_slice, + .end_frame = videotoolbox_prores_end_frame, + .frame_params = ff_videotoolbox_frame_params, + .init = ff_videotoolbox_common_init, + .uninit = ff_videotoolbox_uninit, + .priv_data_size = sizeof(VTContext), +}; + static AVVideotoolboxContext *av_videotoolbox_alloc_context_with_pix_fmt(enum AVPixelFormat pix_fmt, bool full_range) {