From patchwork Fri Dec 17 00:12:15 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: rcombs X-Patchwork-Id: 32678 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a6b:cd86:0:0:0:0:0 with SMTP id d128csp1002774iog; Thu, 16 Dec 2021 16:14:12 -0800 (PST) X-Google-Smtp-Source: ABdhPJwQHf2+p7bhUj8vqeXYL7yk5jtRwaxFh+3bHNP/ZPr7FBV8k647MkoJjB52LYh85F8p4LWr X-Received: by 2002:a05:6402:1768:: with SMTP id da8mr409468edb.252.1639700052695; Thu, 16 Dec 2021 16:14:12 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1639700052; cv=none; d=google.com; s=arc-20160816; b=g7kwqPgC+i89/iVCTg4s+gfUsj6JKcLowNnm4bz1H/0cAsm0o9UFFGR78qefmuI/bZ 1kS+Y6mcq7yhCPrSCljRck08+wJPscELGWqkZqB0a2SpiSBHDZjE6pKqhpBPLXHjR5jI x6hhv0nkqwYesBt/x0GH54xb2/Dz49qO3HR4H40dlrt4+953FW5xuncVrhzrQAV9Mgqg duB5VWnINf1ZjU9zTUgMD4Usft5ha0+7hhmx7/XkFla6jFLyZIyy3uwkmylzNIhuGbob FYOORWcSRuiRBKOfzDoJT26iZUgh2hDdwGG9RKUobd3xvr3eoQO5GfpoA//+l0qKa2OE pPRQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:dkim-signature:delivered-to; bh=hKKu0rzgDluejoyir8Pb/iI9R+CrivXKu3TdPL6Wzwk=; b=ohh3dRAHsr8yw3CSGc7EKvcmmCnwYx6lz3Q+FetEdfq/Ljq11J6LRWtqp8hiaOLJ5C IptlGqpkyy6q8/fYTF4af2VrN0prXllgSxK0kHIHA701lJV2aGoo8HfxWjc1Zr/GWj5V ZfJXhudezuwYZF7VRgqZ9W2vwRqka5UQ+xnxlRDPKYAulLFy7VWpHmLYFm5TYCQSABNy vWseMUz/9Ry7ZxL8KSxxuEl+AGdAkJiXmgec61IOA4oCQwK11OLooWRzPZ1Pw/fs0IkS JQA42qg1lXCAdkIJJK07Hv/o/8MlepOYWh8YfU+ZRUeTyn5lRS8Ja7B4f3nF5hniNFCA 9V0g== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@rcombs.me header.s=google header.b=LVUlaqZY; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=rcombs.me Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id gk1si5405899ejc.516.2021.12.16.16.14.12; Thu, 16 Dec 2021 16:14:12 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@rcombs.me header.s=google header.b=LVUlaqZY; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=rcombs.me Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 3561D68B01B; Fri, 17 Dec 2021 02:12:39 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-io1-f54.google.com (mail-io1-f54.google.com [209.85.166.54]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 95B4A68AFEF for ; Fri, 17 Dec 2021 02:12:28 +0200 (EET) Received: by mail-io1-f54.google.com with SMTP id p65so748739iof.3 for ; Thu, 16 Dec 2021 16:12:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rcombs.me; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=zogxGJq7PjzXjQNjZ21iIbEL2KNXdJULIORrvJZSSUg=; b=LVUlaqZYestEtt3kl6hHLSyP6UKSMaNnRv/mzL8w4tD6Akq9OBUxb2JZ4x6ZLKoxhT FJPsUUdA78twsWAKx+gfyFXDkjSRDDiZIRyNT4NDVYT6XxRM5jm4SPVJJ1dIQeZaI2bZ g3JbYudRI8snVNOKVoJN26bKqeKvWxwiwG8sGWVHwohUv4yGTHjxkydObwumwqJrmeDY bYNtLZ9bqvxbzztuPfRfuxUNV9ssMDNUUSsaWTlUu8kEfvAjRR+fEd6MTPtEAaEUGmaZ NXW92uNpHskhwaMPaZm7w/9hU5bhrQ5zhZfS5yDOzeCDhmT8uh0aKMEnjRRGFKM+lKPM xEaQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=zogxGJq7PjzXjQNjZ21iIbEL2KNXdJULIORrvJZSSUg=; b=244vvyaIQAW16Mku3ND8IjaIhHR5fwVliKJVc3NAorc34jFkexOYkBP1Kcxi3ZIfBT UdDOJlKF42jRw020KTLv2Y2kOrM7obQHusV+jyjnomar4zbYLUUDIIerTPkulyculkuc 86Fxjq+v1kfFfN/OqvSquIltBIxx1Cs8SVnk+qzKQD43MbU7PBpFX0UW2YWY+Rq4qIJe iP+ByNzS9gU6FCBnpUulm8EvVc7QA7dKddrFgvUPdi/WSLPNH/Mns39buJlzmjFb+O7T 8y04fbWhIrHikuZB1Hlax2gHUYxJHut9GSNUWPoHDfoY2YqlSsQItoyHws+HXeeOQgp2 98ZQ== X-Gm-Message-State: AOAM530aj4H1HcskiI0feFyj1QHzuJX+5weVy47+2aHHZQ2L1P1918ao l8sOWGRj2MUYYat2i3Fo3WzVihAtXcY1H68= X-Received: by 2002:a05:6638:3001:: with SMTP id r1mr223883jak.263.1639699946742; Thu, 16 Dec 2021 16:12:26 -0800 (PST) Received: from rcombs-mbp.localdomain ([2601:243:2000:5ac:1163:bda5:7473:fab3]) by smtp.gmail.com with ESMTPSA id z13sm1571257iln.43.2021.12.16.16.12.26 for (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Thu, 16 Dec 2021 16:12:26 -0800 (PST) From: rcombs To: ffmpeg-devel@ffmpeg.org Date: Thu, 16 Dec 2021 18:12:15 -0600 Message-Id: <20211217001215.75135-11-rcombs@rcombs.me> X-Mailer: git-send-email 2.33.1 In-Reply-To: <20211217001215.75135-1-rcombs@rcombs.me> References: <20211217001215.75135-1-rcombs@rcombs.me> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 10/10] lavc/videotoolboxenc: add ProRes support X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: DWJ3gBO7HANY --- Changelog | 1 + configure | 2 + libavcodec/Makefile | 1 + libavcodec/allcodecs.c | 1 + libavcodec/version.h | 2 +- libavcodec/videotoolboxenc.c | 119 +++++++++++++++++++++++++++++++++-- 6 files changed, 119 insertions(+), 7 deletions(-) diff --git a/Changelog b/Changelog index 5c6adc0fa7..ebbfed14a0 100644 --- a/Changelog +++ b/Changelog @@ -41,6 +41,7 @@ version : - libplacebo filter - vflip_vulkan, hflip_vulkan and flip_vulkan filters - adynamicequalizer audio filter +- VideoToolbox ProRes encoder version 4.4: diff --git a/configure b/configure index 5fffcb8afe..0146b28ff1 100755 --- a/configure +++ b/configure @@ -3300,6 +3300,8 @@ h264_videotoolbox_encoder_deps="pthreads" h264_videotoolbox_encoder_select="atsc_a53 videotoolbox_encoder" hevc_videotoolbox_encoder_deps="pthreads" hevc_videotoolbox_encoder_select="atsc_a53 videotoolbox_encoder" +prores_videotoolbox_encoder_deps="pthreads" +prores_videotoolbox_encoder_select="videotoolbox_encoder" libaom_av1_decoder_deps="libaom" libaom_av1_encoder_deps="libaom" libaom_av1_encoder_select="extract_extradata_bsf" diff --git a/libavcodec/Makefile b/libavcodec/Makefile index beb91e8c12..f15a56eabd 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -565,6 +565,7 @@ OBJS-$(CONFIG_PRORES_DECODER) += proresdec2.o proresdsp.o proresdata.o OBJS-$(CONFIG_PRORES_ENCODER) += proresenc_anatoliy.o proresdata.o OBJS-$(CONFIG_PRORES_AW_ENCODER) += proresenc_anatoliy.o proresdata.o OBJS-$(CONFIG_PRORES_KS_ENCODER) += proresenc_kostya.o proresdata.o +OBJS-$(CONFIG_PRORES_VIDEOTOOLBOX_ENCODER) += videotoolboxenc.o OBJS-$(CONFIG_PROSUMER_DECODER) += prosumer.o OBJS-$(CONFIG_PSD_DECODER) += psd.o OBJS-$(CONFIG_PTX_DECODER) += ptx.o diff --git a/libavcodec/allcodecs.c b/libavcodec/allcodecs.c index 89d65266c8..d1e10197de 100644 --- a/libavcodec/allcodecs.c +++ b/libavcodec/allcodecs.c @@ -837,6 +837,7 @@ extern const AVCodec ff_mpeg4_cuvid_decoder; extern const AVCodec ff_mpeg4_mediacodec_decoder; extern const AVCodec ff_mpeg4_omx_encoder; extern const AVCodec ff_mpeg4_v4l2m2m_encoder; +extern const AVCodec ff_prores_videotoolbox_encoder; extern const AVCodec ff_vc1_cuvid_decoder; extern const AVCodec ff_vp8_cuvid_decoder; extern const AVCodec ff_vp8_mediacodec_decoder; diff --git a/libavcodec/version.h b/libavcodec/version.h index 8a0b94f5aa..db71fb1130 100644 --- a/libavcodec/version.h +++ b/libavcodec/version.h @@ -28,7 +28,7 @@ #include "libavutil/version.h" #define LIBAVCODEC_VERSION_MAJOR 59 -#define LIBAVCODEC_VERSION_MINOR 14 +#define LIBAVCODEC_VERSION_MINOR 15 #define LIBAVCODEC_VERSION_MICRO 100 #define LIBAVCODEC_VERSION_INT AV_VERSION_INT(LIBAVCODEC_VERSION_MAJOR, \ diff --git a/libavcodec/videotoolboxenc.c b/libavcodec/videotoolboxenc.c index 67bb563639..8453da7bc4 100644 --- a/libavcodec/videotoolboxenc.c +++ b/libavcodec/videotoolboxenc.c @@ -407,6 +407,7 @@ static int count_nalus(size_t length_code_size, } static CMVideoCodecType get_cm_codec_type(AVCodecContext *avctx, + int64_t profile, double alpha_quality) { const AVPixFmtDescriptor *desc = av_pix_fmt_desc_get(avctx->pix_fmt == AV_PIX_FMT_VIDEOTOOLBOX ? avctx->sw_pix_fmt : avctx->pix_fmt); @@ -417,6 +418,31 @@ static CMVideoCodecType get_cm_codec_type(AVCodecContext *avctx, return kCMVideoCodecType_HEVCWithAlpha; } return kCMVideoCodecType_HEVC; + case AV_CODEC_ID_PRORES: + switch (profile) { + case FF_PROFILE_PRORES_PROXY: + return MKBETAG('a','p','c','o'); // kCMVideoCodecType_AppleProRes422Proxy + case FF_PROFILE_PRORES_LT: + return MKBETAG('a','p','c','s'); // kCMVideoCodecType_AppleProRes422LT + case FF_PROFILE_PRORES_STANDARD: + return MKBETAG('a','p','c','n'); // kCMVideoCodecType_AppleProRes422 + case FF_PROFILE_PRORES_HQ: + return MKBETAG('a','p','c','h'); // kCMVideoCodecType_AppleProRes422HQ + case FF_PROFILE_PRORES_4444: + return MKBETAG('a','p','4','h'); // kCMVideoCodecType_AppleProRes4444 + case FF_PROFILE_PRORES_XQ: + return MKBETAG('a','p','4','x'); // kCMVideoCodecType_AppleProRes4444XQ + + default: + av_log(avctx, AV_LOG_ERROR, "Unknown profile ID: %"PRId64", using auto\n", profile); + case FF_PROFILE_UNKNOWN: + if (desc && + ((desc->flags & AV_PIX_FMT_FLAG_ALPHA) || + desc->log2_chroma_w == 0)) + return MKBETAG('a','p','4','h'); // kCMVideoCodecType_AppleProRes4444 + else + return MKBETAG('a','p','c','n'); // kCMVideoCodecType_AppleProRes422 + } default: return 0; } } @@ -1104,7 +1130,7 @@ static int vtenc_create_encoder(AVCodecContext *avctx, kVTCompressionPropertyKey_Quality, quality_num); CFRelease(quality_num); - } else { + } else if (avctx->codec_id != AV_CODEC_ID_PRORES) { bit_rate_num = CFNumberCreate(kCFAllocatorDefault, kCFNumberSInt32Type, &bit_rate); @@ -1191,7 +1217,7 @@ static int vtenc_create_encoder(AVCodecContext *avctx, } } - if (avctx->gop_size > 0) { + if (avctx->gop_size > 0 && avctx->codec_id != AV_CODEC_ID_PRORES) { CFNumberRef interval = CFNumberCreate(kCFAllocatorDefault, kCFNumberIntType, &avctx->gop_size); @@ -1340,7 +1366,7 @@ static int vtenc_create_encoder(AVCodecContext *avctx, } } - if (!vtctx->has_b_frames) { + if (!vtctx->has_b_frames && avctx->codec_id != AV_CODEC_ID_PRORES) { status = VTSessionSetProperty(vtctx->session, kVTCompressionPropertyKey_AllowFrameReordering, kCFBooleanFalse); @@ -1390,16 +1416,24 @@ static int vtenc_configure_encoder(AVCodecContext *avctx) CFMutableDictionaryRef pixel_buffer_info; CMVideoCodecType codec_type; VTEncContext *vtctx = avctx->priv_data; - CFStringRef profile_level; + CFStringRef profile_level = NULL; CFNumberRef gamma_level = NULL; int status; - codec_type = get_cm_codec_type(avctx, vtctx->alpha_quality); + codec_type = get_cm_codec_type(avctx, vtctx->profile, vtctx->alpha_quality); if (!codec_type) { av_log(avctx, AV_LOG_ERROR, "Error: no mapping for AVCodecID %d\n", avctx->codec_id); return AVERROR(EINVAL); } +#if defined(MAC_OS_X_VERSION_10_9) && !TARGET_OS_IPHONE && (MAC_OS_X_VERSION_MAX_ALLOWED >= MAC_OS_X_VERSION_10_9) + if (avctx->codec_id == AV_CODEC_ID_PRORES) { + if (__builtin_available(macOS 10.9, *)) { + VTRegisterProfessionalVideoWorkflowVideoEncoders(); + } + } +#endif + vtctx->codec_id = avctx->codec_id; if (vtctx->codec_id == AV_CODEC_ID_H264) { @@ -1417,12 +1451,14 @@ static int vtenc_configure_encoder(AVCodecContext *avctx) } if (!get_vt_h264_profile_level(avctx, &profile_level)) return AVERROR(EINVAL); - } else { + } else if (vtctx->codec_id == AV_CODEC_ID_HEVC) { vtctx->get_param_set_func = compat_keys.CMVideoFormatDescriptionGetHEVCParameterSetAtIndex; if (!vtctx->get_param_set_func) return AVERROR(EINVAL); if (!get_vt_hevc_profile_level(avctx, &profile_level)) return AVERROR(EINVAL); // HEVC has b-byramid vtctx->has_b_frames = avctx->max_b_frames > 0 ? 2 : 0; + } else if (vtctx->codec_id == AV_CODEC_ID_PRORES) { + avctx->codec_tag = av_bswap32(codec_type); } enc_info = CFDictionaryCreateMutable( @@ -2600,6 +2636,38 @@ static const enum AVPixelFormat hevc_pix_fmts[] = { AV_PIX_FMT_NONE }; +static const enum AVPixelFormat prores_pix_fmts[] = { + AV_PIX_FMT_YUV420P, +#ifdef kCFCoreFoundationVersionNumber10_7 + AV_PIX_FMT_NV12, + AV_PIX_FMT_AYUV64, +#endif + AV_PIX_FMT_UYVY422, +#if HAVE_KCVPIXELFORMATTYPE_420YPCBCR10BIPLANARVIDEORANGE + AV_PIX_FMT_P010, +#endif +#if HAVE_KCVPIXELFORMATTYPE_422YPCBCR8BIPLANARVIDEORANGE + AV_PIX_FMT_NV16, +#endif +#if HAVE_KCVPIXELFORMATTYPE_422YPCBCR10BIPLANARVIDEORANGE + AV_PIX_FMT_P210, +#endif +#if HAVE_KCVPIXELFORMATTYPE_422YPCBCR16BIPLANARVIDEORANGE + AV_PIX_FMT_P216, +#endif +#if HAVE_KCVPIXELFORMATTYPE_444YPCBCR8BIPLANARVIDEORANGE + AV_PIX_FMT_NV24, +#endif +#if HAVE_KCVPIXELFORMATTYPE_444YPCBCR10BIPLANARVIDEORANGE + AV_PIX_FMT_P410, +#endif +#if HAVE_KCVPIXELFORMATTYPE_444YPCBCR16BIPLANARVIDEORANGE + AV_PIX_FMT_P416, +#endif + AV_PIX_FMT_BGRA, + AV_PIX_FMT_NONE +}; + #define VE AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_ENCODING_PARAM #define COMMON_OPTIONS \ { "allow_sw", "Allow software encoding", OFFSET(allow_sw), AV_OPT_TYPE_BOOL, \ @@ -2703,3 +2771,42 @@ const AVCodec ff_hevc_videotoolbox_encoder = { FF_CODEC_CAP_INIT_CLEANUP, .wrapper_name = "videotoolbox", }; + +static const AVOption prores_options[] = { + { "profile", "Profile", OFFSET(profile), AV_OPT_TYPE_INT64, { .i64 = FF_PROFILE_UNKNOWN }, FF_PROFILE_UNKNOWN, FF_PROFILE_PRORES_XQ, VE, "profile" }, + { "auto", "Automatically determine based on input format", 0, AV_OPT_TYPE_CONST, { .i64 = FF_PROFILE_UNKNOWN }, INT_MIN, INT_MAX, VE, "profile" }, + { "proxy", "ProRes 422 Proxy", 0, AV_OPT_TYPE_CONST, { .i64 = FF_PROFILE_PRORES_PROXY }, INT_MIN, INT_MAX, VE, "profile" }, + { "lt", "ProRes 422 LT", 0, AV_OPT_TYPE_CONST, { .i64 = FF_PROFILE_PRORES_LT }, INT_MIN, INT_MAX, VE, "profile" }, + { "standard", "ProRes 422", 0, AV_OPT_TYPE_CONST, { .i64 = FF_PROFILE_PRORES_STANDARD }, INT_MIN, INT_MAX, VE, "profile" }, + { "hq", "ProRes 422 HQ", 0, AV_OPT_TYPE_CONST, { .i64 = FF_PROFILE_PRORES_HQ }, INT_MIN, INT_MAX, VE, "profile" }, + { "4444", "ProRes 4444", 0, AV_OPT_TYPE_CONST, { .i64 = FF_PROFILE_PRORES_4444 }, INT_MIN, INT_MAX, VE, "profile" }, + { "xq", "ProRes 4444 XQ", 0, AV_OPT_TYPE_CONST, { .i64 = FF_PROFILE_PRORES_XQ }, INT_MIN, INT_MAX, VE, "profile" }, + + COMMON_OPTIONS + { NULL }, +}; + +static const AVClass prores_videotoolbox_class = { + .class_name = "prores_videotoolbox", + .item_name = av_default_item_name, + .option = prores_options, + .version = LIBAVUTIL_VERSION_INT, +}; + +const AVCodec ff_prores_videotoolbox_encoder = { + .name = "prores_videotoolbox", + .long_name = NULL_IF_CONFIG_SMALL("VideoToolbox ProRes Encoder"), + .type = AVMEDIA_TYPE_VIDEO, + .id = AV_CODEC_ID_PRORES, + .capabilities = AV_CODEC_CAP_DR1 | AV_CODEC_CAP_DELAY | + AV_CODEC_CAP_HARDWARE, + .priv_data_size = sizeof(VTEncContext), + .pix_fmts = prores_pix_fmts, + .init = vtenc_init, + .encode2 = vtenc_frame, + .close = vtenc_close, + .priv_class = &prores_videotoolbox_class, + .caps_internal = FF_CODEC_CAP_INIT_THREADSAFE | + FF_CODEC_CAP_INIT_CLEANUP, + .wrapper_name = "videotoolbox", +};