From patchwork Sat May 20 22:24:22 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rick Kern X-Patchwork-Id: 41751 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:ba91:b0:105:feb:71f2 with SMTP id fb17csp780549pzb; Sat, 20 May 2023 15:24:38 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7u54unVcnMNQX2x9Cz9G3kAryJefrjH7gTdXmnAXV+L00RsRFvchrkKY0zPKnckUEapwe5 X-Received: by 2002:a17:907:1b24:b0:96a:7196:6e2b with SMTP id mp36-20020a1709071b2400b0096a71966e2bmr5914902ejc.14.1684621477909; Sat, 20 May 2023 15:24:37 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684621477; cv=none; d=google.com; s=arc-20160816; b=NLXTjEWGt/B1Ejl6HAJTuu/9KdnbYsUMu4CtFKAMhPA4SWGeBRLQhQibvyETTx3Q6Z /48V4T+5HL41opCrTHzh/Jr0xHVEZnnEsMLuHh2BpgytuTU+R8THNS86TbjdUBqwzhGo wU9qqPB5VgNeAW6XNhwunJBUqYLUjDz2IXMaxgvTNY8xBd20CrEZpGvIoUN4PJQFN6bt Xz/SU9n/cSo52j+o3bjgt0ZooH5S4Rt4I3tEkl+U6l46hwPT6Dm2gEca6f2uQJ0bEgk1 a3ORNk9Wb72EDDRHDEOTJD/O0YtBENVlOKovvAcTzmRpbpWMDUr1IShuoepe2PBoepf0 wM7w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:message-id:date:to:from :dkim-signature:delivered-to; bh=brles8FygjxJQqJgym53VB7bgI41BDxHniDfZZC+bV8=; b=zdgZYqDrLehpZJIojauJXBSLbvn8ao1xByA2miVKfe6kRtiSAg0oOau1Ky1t0rWSNc ubTLDUDn9kJDtQ/oUiDWjxeamhyrY4QNnYnG7oQQI0WoHRhNinZnUdltCsPj8TeUJQ2V T4nbsY9Wv1As4i92axGuYgpPcusmMn1DGo4hyDxjYxxNeFc4aIRZqHX6Uk3+WPjAwA6H xZg1d1Xudu/jo5DPP9mzZnW2I7E8MxQLTAfUfDJ28cKu9OVA83ZUMIUoCAzjcsKS1isD oDIhaaapczvKKouunjI/hloFlZQQbUFedqxae9AqNd2lgouVJwyzaPfG9yBGLuskCcHr mEhA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20221208 header.b=SOLubtkI; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id vj3-20020a170907d48300b0096f512456ffsi901806ejc.937.2023.05.20.15.24.37; Sat, 20 May 2023 15:24:37 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20221208 header.b=SOLubtkI; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 53FDA68C109; Sun, 21 May 2023 01:24:33 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-qt1-f176.google.com (mail-qt1-f176.google.com [209.85.160.176]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 2ECD068B649 for ; Sun, 21 May 2023 01:24:27 +0300 (EEST) Received: by mail-qt1-f176.google.com with SMTP id d75a77b69052e-3f69032f8a1so12473351cf.1 for ; Sat, 20 May 2023 15:24:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1684621465; x=1687213465; h=content-transfer-encoding:mime-version:message-id:date:subject:to :from:from:to:cc:subject:date:message-id:reply-to; bh=NnQHBOWLE3nieTLMOdn7kBy3N9TMFN1hIX5mBJ1j7kA=; b=SOLubtkI+hz2M7tCFQpWx6+W0kSOvo0hfqEonLZ5PeJ4z0rvJI7GDgaoopDv4FbnXG FSzlhpRDOAl12nKncxsXq0/K2FAVc3KpKTN2TAtwZuBHjeE+lU79lB8IZuuRPx2eEM0X E7pWDotxrQNbxY6Yv+pepVTeV0v/APLE7M9RIwiuVBFuAWUM2D1qKho1boQR08R23KrH sf1tRYv/8i0s3Y7TzTD0pP0+9ehDxeoDphYUqiOgJzZld0gXPCCSxbMQ83On+K3fMJHk l2gJsVj5mTq1eg5ruZXPdwZvPpASUM6ryonlsawEpQUJY/BuzTwycU2QSln5upXs1+xu v/Pg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684621465; x=1687213465; h=content-transfer-encoding:mime-version:message-id:date:subject:to :from:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=NnQHBOWLE3nieTLMOdn7kBy3N9TMFN1hIX5mBJ1j7kA=; b=UUZODzfskz7jNB51XFhT9vOmkLRPxX3UX0kUwB9o21vCWxkvJDi11UdfqZOew2GUmH 8RgXgdJO6usCD/8w/f1caDgEJS2aCQJEf0HQAD5Kxkc9X6kXeiEz416SWSQMLa451YnK 2xzkK4U7dkoBiuKYMMTIrACm1zB6pKEpm7GtZRpzowsIF2vIy8v+fyPTbzCQ7Jy/cEaD jQpMiYjX7tUPOR2y0lp/26l+cGW+xE4JdTyY50A2zG/YyV0bkPsbPouVEk6GW9zdbiNV wRlCLpnAmXbIZZ9OXqiac0fcZoX0w0dWbn3vv5+p2YUivpIVch2ay9W3LWjFlyPcbgxB Ti3w== X-Gm-Message-State: AC+VfDxEFyLQiwFvQh7xIhuOsVQ106u3qDt4caibbT6xA0eXN+bHSKCr dcmyRSuEm+s28R0XvByvI8WBEbHSq08= X-Received: by 2002:a05:622a:178a:b0:3ed:a811:23d0 with SMTP id s10-20020a05622a178a00b003eda81123d0mr10623922qtk.41.1684621464653; Sat, 20 May 2023 15:24:24 -0700 (PDT) Received: from localhost.localdomain ([2600:1002:b02d:bcfd:bc14:d473:8190:cac8]) by smtp.gmail.com with ESMTPSA id q16-20020a05620a039000b0075939692d18sm706645qkm.83.2023.05.20.15.24.23 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 20 May 2023 15:24:24 -0700 (PDT) From: Rick Kern To: ffmpeg-devel@ffmpeg.org Date: Sat, 20 May 2023 18:24:22 -0400 Message-Id: <20230520222422.68633-1-kernrj@gmail.com> X-Mailer: git-send-email 2.38.1 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH] lavc/videotoolboxenc: support additional options X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: jeUx560LRRDG Added support for more VideoToolbox encoder options: - qmin and qmax options are now used - max_slice_bytes: Max number of bytes per H.264 slice - max_ref_frames: Limit the number of reference frames - Disable open GOP when the cgop flag is set - power_efficient: Enable power-efficient mode --- libavcodec/videotoolboxenc.c | 177 ++++++++++++++++++++++++++++++++++- 1 file changed, 176 insertions(+), 1 deletion(-) diff --git a/libavcodec/videotoolboxenc.c b/libavcodec/videotoolboxenc.c index b017c90c36..be29d13629 100644 --- a/libavcodec/videotoolboxenc.c +++ b/libavcodec/videotoolboxenc.c @@ -120,6 +120,11 @@ static struct{ CFStringRef kVTVideoEncoderSpecification_EnableHardwareAcceleratedVideoEncoder; CFStringRef kVTVideoEncoderSpecification_RequireHardwareAcceleratedVideoEncoder; CFStringRef kVTVideoEncoderSpecification_EnableLowLatencyRateControl; + CFStringRef kVTCompressionPropertyKey_AllowOpenGOP; + CFStringRef kVTCompressionPropertyKey_MaximizePowerEfficiency; + CFStringRef kVTCompressionPropertyKey_ReferenceBufferCount; + CFStringRef kVTCompressionPropertyKey_MaxAllowedFrameQP; + CFStringRef kVTCompressionPropertyKey_MinAllowedFrameQP; getParameterSetAtIndex CMVideoFormatDescriptionGetHEVCParameterSetAtIndex; } compat_keys; @@ -188,6 +193,13 @@ static void loadVTEncSymbols(void){ "RequireHardwareAcceleratedVideoEncoder"); GET_SYM(kVTVideoEncoderSpecification_EnableLowLatencyRateControl, "EnableLowLatencyRateControl"); + GET_SYM(kVTCompressionPropertyKey_AllowOpenGOP, "AllowOpenGOP"); + GET_SYM(kVTCompressionPropertyKey_MaximizePowerEfficiency, + "MaximizePowerEfficiency"); + GET_SYM(kVTCompressionPropertyKey_ReferenceBufferCount, + "ReferenceBufferCount"); + GET_SYM(kVTCompressionPropertyKey_MaxAllowedFrameQP, "MaxAllowedFrameQP"); + GET_SYM(kVTCompressionPropertyKey_MinAllowedFrameQP, "MinAllowedFrameQP"); } typedef enum VT_H264Profile { @@ -268,6 +280,10 @@ typedef struct VTEncContext { /* can't be bool type since AVOption will access it as int */ int a53_cc; + + int max_slice_bytes; + int power_efficient; + int max_ref_frames; } VTEncContext; static int vtenc_populate_extradata(AVCodecContext *avctx, @@ -1106,6 +1122,78 @@ static bool vtenc_qscale_enabled(void) return !TARGET_OS_IPHONE && TARGET_CPU_ARM64; } +static int set_encoder_int_property_or_log( + AVCodecContext* avctx, + CFStringRef key, + const char* print_option_name, + int value) { + int status; + VTEncContext *vtctx = avctx->priv_data; + CFNumberRef value_cfnum = CFNumberCreate( + kCFAllocatorDefault, + kCFNumberIntType, + &value); + + if (value_cfnum == NULL) { + return AVERROR(ENOMEM); + } + + status = VTSessionSetProperty(vtctx->session, key, value_cfnum); + if (status == kVTPropertyNotSupportedErr) { + av_log( + avctx, + AV_LOG_INFO, + "This device does not support the %s option. Value %d ignored.\n", + print_option_name, + value); + } else if (status != 0) { + av_log( + avctx, + AV_LOG_ERROR, + "Error setting %s=%d: Error %d\n", + print_option_name, + value, + status); + } + + CFRelease (value_cfnum); + + return 0; +} + +static int set_encoder_bool_property_or_log( + AVCodecContext* avctx, + CFStringRef key, + const char* print_option_name, + int value) { + int status; + VTEncContext *vtctx = avctx->priv_data; + + status = VTSessionSetProperty( + vtctx->session, + key, + value ? kCFBooleanTrue : kCFBooleanFalse); + + if (status == kVTPropertyNotSupportedErr) { + av_log( + avctx, + AV_LOG_INFO, + "This device does not support the %s option. Value %d ignored.\n", + print_option_name, + value); + } else if (status != 0) { + av_log( + avctx, + AV_LOG_ERROR, + "Error setting %s=%d: Error %d\n", + print_option_name, + value, + status); + } + + return 0; +} + static int vtenc_create_encoder(AVCodecContext *avctx, CMVideoCodecType codec_type, CFStringRef profile_level, @@ -1476,6 +1564,81 @@ static int vtenc_create_encoder(AVCodecContext *avctx, } } + if ((avctx->flags & AV_CODEC_FLAG_CLOSED_GOP) != 0) { + set_encoder_bool_property_or_log( + avctx, + compat_keys.kVTCompressionPropertyKey_AllowOpenGOP, + "AllowOpenGop", + 0); + + if (status) { + return status; + } + } + + if (avctx->qmin >= 0) { + status = set_encoder_int_property_or_log( + avctx, + compat_keys.kVTCompressionPropertyKey_MinAllowedFrameQP, + "qmin", + avctx->qmin); + + if (status != 0) { + return status; + } + } + + if (avctx->qmax >= 0) { + status = set_encoder_int_property_or_log( + avctx, + compat_keys.kVTCompressionPropertyKey_MaxAllowedFrameQP, + "qmax", + avctx->qmax); + + if (status != 0) { + return status; + } + } + + if (vtctx->max_slice_bytes >= 0 && avctx->codec_id == AV_CODEC_ID_H264) { + status = set_encoder_int_property_or_log( + avctx, + kVTCompressionPropertyKey_MaxH264SliceBytes, + "max_slice_bytes", + vtctx->max_slice_bytes); + + if (status != 0) { + return status; + } + } + + if (vtctx->power_efficient >= 0) { + status = VTSessionSetProperty( + vtctx->session, + compat_keys.kVTCompressionPropertyKey_MaximizePowerEfficiency, + vtctx->power_efficient ? kCFBooleanTrue : kCFBooleanFalse); + + if (status) { + av_log( + avctx, + AV_LOG_ERROR, + "Error setting power_efficient property: %d\n", + status); + } + } + + if (vtctx->max_ref_frames > 0) { + status = set_encoder_int_property_or_log( + avctx, + compat_keys.kVTCompressionPropertyKey_ReferenceBufferCount, + "max_ref_frames", + vtctx->max_ref_frames); + + if (status != 0) { + return status; + } + } + status = VTCompressionSessionPrepareToEncodeFrames(vtctx->session); if (status) { av_log(avctx, AV_LOG_ERROR, "Error: cannot prepare encoder: %d\n", status); @@ -2753,6 +2916,11 @@ static const enum AVPixelFormat prores_pix_fmts[] = { OFFSET(frames_after), AV_OPT_TYPE_BOOL, { .i64 = 0 }, 0, 1, VE }, \ { "prio_speed", "prioritize encoding speed", OFFSET(prio_speed), AV_OPT_TYPE_BOOL, \ { .i64 = -1 }, -1, 1, VE }, \ + { "power_efficient", "Set to 1 to enable more power-efficient encoding if supported.", \ + OFFSET(power_efficient), AV_OPT_TYPE_INT, { .i64 = -1 }, -1, 1, VE }, \ + { "max_ref_frames", \ + "Sets the maximum number of reference frames. This only has an effect when the value is less than the maximum allowed by the profile/level.", \ + OFFSET(max_ref_frames), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, INT_MAX, VE }, #define OFFSET(x) offsetof(VTEncContext, x) static const AVOption h264_options[] = { @@ -2783,7 +2951,14 @@ static const AVOption h264_options[] = { { "a53cc", "Use A53 Closed Captions (if available)", OFFSET(a53_cc), AV_OPT_TYPE_BOOL, {.i64 = 1}, 0, 1, VE }, { "constant_bit_rate", "Require constant bit rate (macOS 13 or newer)", OFFSET(constant_bit_rate), AV_OPT_TYPE_BOOL, { .i64 = 0 }, 0, 1, VE }, - + { "max_slice_bytes", + "Set the maximum number of bytes in an H.264 slice.", + OFFSET(max_slice_bytes), + AV_OPT_TYPE_INT, + { .i64 = -1 }, + -1, + INT_MAX, + VE }, COMMON_OPTIONS { NULL }, };