From patchwork Tue Feb 23 05:02:27 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: bouno@rouge.plala.or.jp X-Patchwork-Id: 25926 Delivered-To: andriy.gelman@gmail.com Received: by 2002:a25:6d42:0:0:0:0:0 with SMTP id i63csp2190031ybc; Mon, 22 Feb 2021 21:03:32 -0800 (PST) X-Google-Smtp-Source: ABdhPJyVCAJ43LFp3UoOa2IKlMtSWJ9zB5Q3SGdbECdCkuRCeiwRkUqE1T6AdR7KhmhEH/eGkc7G X-Received: by 2002:aa7:d38b:: with SMTP id x11mr25857057edq.23.1614056612222; Mon, 22 Feb 2021 21:03:32 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1614056612; cv=none; d=google.com; s=arc-20160816; b=own1g/+qMWvH3d8Y60WEZCnXo7mrGaIMr1EWAj8mxZC/yV9YvsbUANrLykZB3qItTD PlIr7OhxcW+SGhPK3pt3IyYtCBCuyLjbDzUfFhGznAktfmIntbYFYQw24UmEqXTN6FGQ oF8cEupXmyvEQRfL2gitlCg34XuGyKV99Mc7XbceXaAzinq9SFuan16eF3/PDiStdqu5 +LVSZK9tUu5vgPTuaMuS7Xkty1y1g60GHbteGkxHJmXAavKY/U1dwnCr8kRKe+fklgq+ kD8XnVc8/sXW1F37r9/etWlXdOWqDBiKjQPthqTgk4vrDwqT8OEez8EeC/XjEbJE3hYE /JgQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:message-id:date:to:from :delivered-to; bh=aiuTmOnztXfdCyduHIMAFCTP4TyVG5qHKJrygc5HXTs=; b=bYjoDEue6aZAe0KiBbmIskx7sR3Mv2MXX9yQH9/FtKGKc3B6ByTXqU+KrgeIaRcPcm nvV6owUtUI6oWPDUwaraLlWyLZM0hoGix5eHCaZpyaXsKrC2vWto/MB866jB7LLdsA4K 9KUaQId3il1F/YPqTKSfmJX9mQka5Y/4CUrGioE388BWKphAQsCs0XT1XzeAIhllAbkw RuM7MPgL0uXrBXVyxoYKMmD8UdZwrbdVUA46B6K5U84UhAAXp02qtRS+nbzESWZmwPuz IION4206TZJdQYVMJNNuasztNMhXGSIc/Gp73yTu0gum02tDsT2YdfwTYBLFLFWJ0iR3 tm6Q== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id v21si9837852ejg.151.2021.02.22.21.03.31; Mon, 22 Feb 2021 21:03:32 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 7FFD168A326; Tue, 23 Feb 2021 07:03:28 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from msc14.plala.or.jp (msc14.plala.or.jp [60.36.166.24]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 7BD6568A326 for ; Tue, 23 Feb 2021 07:03:16 +0200 (EET) Received: from localhost.localdomain ([125.202.53.66]) by msc14.plala.or.jp with ESMTP id <20210223050313.JLCW400.msc14.plala.or.jp@localhost.localdomain>; Tue, 23 Feb 2021 14:03:13 +0900 From: bouno@rouge.plala.or.jp To: ffmpeg-devel@ffmpeg.org Date: Tue, 23 Feb 2021 14:02:27 +0900 Message-Id: <20210223050227.65698-1-bouno@rouge.plala.or.jp> X-Mailer: git-send-email 2.24.3 (Apple Git-128) MIME-Version: 1.0 X-VirusScan: Outbound; mvir-ac14; Tue, 23 Feb 2021 14:03:13 +0900 Subject: [FFmpeg-devel] [PATCH] lavc/videotoolboxenc: Add support for HEVC with Alpha. X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Hironori Bono Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: WW6XTSBH0Oui Content-Length: 10395 From: Hironori Bono This change supports the "HEVC Video with Alpha" profile introduced in WWDC 2019 . (This change is a partial fix for Ticket #7965.) For example, the following command converts an animation PNG file to an HEVC with Alpha video: ./ffmpeg -i fate-suite/apng/clock.png -c:v hevc_videotoolbox -allow_sw 1 -alpha_quality 0.75 -vtag hvc1 clock.mov (This change uses the "HEVC Video with Alpha" profile only when the '-alpha_quality' value is not 0 for backward compatibility.) Signed-off-by: Hironori Bono Signed-off-by: Hironori Bono --- configure | 2 ++ libavcodec/videotoolboxenc.c | 47 +++++++++++++++++++++++++++++++++--- 2 files changed, 46 insertions(+), 3 deletions(-) diff --git a/configure b/configure index 336301cb40..63adf131b9 100755 --- a/configure +++ b/configure @@ -2288,6 +2288,7 @@ TOOLCHAIN_FEATURES=" TYPES_LIST=" kCMVideoCodecType_HEVC + kCMVideoCodecType_HEVCWithAlpha kCVPixelFormatType_420YpCbCr10BiPlanarVideoRange kCVImageBufferTransferFunction_SMPTE_ST_2084_PQ kCVImageBufferTransferFunction_ITU_R_2100_HLG @@ -6211,6 +6212,7 @@ enabled avfoundation && { enabled videotoolbox && { check_lib coreservices CoreServices/CoreServices.h UTGetOSTypeFromString "-framework CoreServices" check_func_headers CoreMedia/CMFormatDescription.h kCMVideoCodecType_HEVC "-framework CoreMedia" + check_func_headers CoreMedia/CMFormatDescription.h kCMVideoCodecType_HEVCWithAlpha "-framework CoreMedia" check_func_headers CoreVideo/CVPixelBuffer.h kCVPixelFormatType_420YpCbCr10BiPlanarVideoRange "-framework CoreVideo" check_func_headers CoreVideo/CVImageBuffer.h kCVImageBufferTransferFunction_SMPTE_ST_2084_PQ "-framework CoreVideo" check_func_headers CoreVideo/CVImageBuffer.h kCVImageBufferTransferFunction_ITU_R_2100_HLG "-framework CoreVideo" diff --git a/libavcodec/videotoolboxenc.c b/libavcodec/videotoolboxenc.c index c487d2dc60..9d3c7e29dc 100644 --- a/libavcodec/videotoolboxenc.c +++ b/libavcodec/videotoolboxenc.c @@ -40,6 +40,10 @@ enum { kCMVideoCodecType_HEVC = 'hvc1' }; #endif +#if !HAVE_KCMVIDEOCODECTYPE_HEVCWITHALPHA +enum { kCMVideoCodecType_HEVCWithAlpha = 'muxa' }; +#endif + #if !HAVE_KCVPIXELFORMATTYPE_420YPCBCR10BIPLANARVIDEORANGE enum { kCVPixelFormatType_420YpCbCr10BiPlanarFullRange = 'xf20' }; enum { kCVPixelFormatType_420YpCbCr10BiPlanarVideoRange = 'x420' }; @@ -88,6 +92,7 @@ static struct{ CFStringRef kVTProfileLevel_HEVC_Main10_AutoLevel; CFStringRef kVTCompressionPropertyKey_RealTime; + CFStringRef kVTCompressionPropertyKey_TargetQualityForAlpha; CFStringRef kVTVideoEncoderSpecification_EnableHardwareAcceleratedVideoEncoder; CFStringRef kVTVideoEncoderSpecification_RequireHardwareAcceleratedVideoEncoder; @@ -147,6 +152,8 @@ static void loadVTEncSymbols(){ GET_SYM(kVTProfileLevel_HEVC_Main10_AutoLevel, "HEVC_Main10_AutoLevel"); GET_SYM(kVTCompressionPropertyKey_RealTime, "RealTime"); + GET_SYM(kVTCompressionPropertyKey_TargetQualityForAlpha, + "TargetQualityForAlpha"); GET_SYM(kVTVideoEncoderSpecification_EnableHardwareAcceleratedVideoEncoder, "EnableHardwareAcceleratedVideoEncoder"); @@ -222,6 +229,7 @@ typedef struct VTEncContext { int64_t allow_sw; int64_t require_sw; + double alpha_quality; bool flushing; bool has_b_frames; @@ -392,11 +400,17 @@ static int count_nalus(size_t length_code_size, return 0; } -static CMVideoCodecType get_cm_codec_type(enum AVCodecID id) +static CMVideoCodecType get_cm_codec_type(enum AVCodecID id, + enum AVPixelFormat fmt, + double alpha_quality) { switch (id) { case AV_CODEC_ID_H264: return kCMVideoCodecType_H264; - case AV_CODEC_ID_HEVC: return kCMVideoCodecType_HEVC; + case AV_CODEC_ID_HEVC: + if (fmt == AV_PIX_FMT_BGRA && alpha_quality > 0.0) { + return kCMVideoCodecType_HEVCWithAlpha; + } + return kCMVideoCodecType_HEVC; default: return 0; } } @@ -786,6 +800,8 @@ static int get_cv_pixel_format(AVCodecContext* avctx, *av_pixel_format = range == AVCOL_RANGE_JPEG ? kCVPixelFormatType_420YpCbCr8PlanarFullRange : kCVPixelFormatType_420YpCbCr8Planar; + } else if (fmt == AV_PIX_FMT_BGRA) { + *av_pixel_format = kCVPixelFormatType_32BGRA; } else if (fmt == AV_PIX_FMT_P010LE) { *av_pixel_format = range == AVCOL_RANGE_JPEG ? kCVPixelFormatType_420YpCbCr10BiPlanarFullRange : @@ -1114,6 +1130,20 @@ static int vtenc_create_encoder(AVCodecContext *avctx, } } + if (vtctx->codec_id == AV_CODEC_ID_HEVC) { + if (avctx->pix_fmt == AV_PIX_FMT_BGRA && vtctx->alpha_quality > 0.0) { + CFNumberRef alpha_quality_num = CFNumberCreate(kCFAllocatorDefault, + kCFNumberDoubleType, + &vtctx->alpha_quality); + if (!alpha_quality_num) return AVERROR(ENOMEM); + + status = VTSessionSetProperty(vtctx->session, + compat_keys.kVTCompressionPropertyKey_TargetQualityForAlpha, + alpha_quality_num); + CFRelease(alpha_quality_num); + } + } + if (profile_level) { status = VTSessionSetProperty(vtctx->session, kVTCompressionPropertyKey_ProfileLevel, @@ -1326,7 +1356,7 @@ static int vtenc_configure_encoder(AVCodecContext *avctx) CFNumberRef gamma_level = NULL; int status; - codec_type = get_cm_codec_type(avctx->codec_id); + codec_type = get_cm_codec_type(avctx->codec_id, avctx->pix_fmt, vtctx->alpha_quality); if (!codec_type) { av_log(avctx, AV_LOG_ERROR, "Error: no mapping for AVCodecID %d\n", avctx->codec_id); return AVERROR(EINVAL); @@ -2036,6 +2066,14 @@ static int get_cv_pixel_info( strides[2] = frame ? frame->linesize[2] : (avctx->width + 1) / 2; break; + case AV_PIX_FMT_BGRA: + *plane_count = 1; + + widths [0] = avctx->width; + heights[0] = avctx->height; + strides[0] = frame ? frame->linesize[0] : avctx->width * 4; + break; + case AV_PIX_FMT_P010LE: *plane_count = 2; widths[0] = avctx->width; @@ -2534,6 +2572,7 @@ static const enum AVPixelFormat hevc_pix_fmts[] = { AV_PIX_FMT_VIDEOTOOLBOX, AV_PIX_FMT_NV12, AV_PIX_FMT_YUV420P, + AV_PIX_FMT_BGRA, AV_PIX_FMT_P010LE, AV_PIX_FMT_NONE }; @@ -2611,6 +2650,8 @@ static const AVOption hevc_options[] = { { "main", "Main Profile", 0, AV_OPT_TYPE_CONST, { .i64 = HEVC_PROF_MAIN }, INT_MIN, INT_MAX, VE, "profile" }, { "main10", "Main10 Profile", 0, AV_OPT_TYPE_CONST, { .i64 = HEVC_PROF_MAIN10 }, INT_MIN, INT_MAX, VE, "profile" }, + { "alpha_quality", "Compression quality for the alpha channel", OFFSET(alpha_quality), AV_OPT_TYPE_DOUBLE, { .dbl = 0.0 }, 0.0, 1.0, VE }, + COMMON_OPTIONS { NULL }, };