From patchwork Mon Jan 9 18:40:24 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aman Karmani X-Patchwork-Id: 39946 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:bc95:b0:ad:ade2:bfd2 with SMTP id fx21csp3327805pzb; Mon, 9 Jan 2023 10:40:44 -0800 (PST) X-Google-Smtp-Source: AMrXdXtni+mvZuYnVkJ9qhKsaI00jEDTdPerAPW3l688CCJ4cgOKasAE5FeEgTp/ivCpFHKEXBnu X-Received: by 2002:a50:fe94:0:b0:48e:a9a1:c584 with SMTP id d20-20020a50fe94000000b0048ea9a1c584mr21021074edt.35.1673289644158; Mon, 09 Jan 2023 10:40:44 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1673289644; cv=none; d=google.com; s=arc-20160816; b=TiCZlNadyQRLT5rFd8Bfj+nPjjMCW+IokhZ8sG4flFTGuyH7QhmEx54wz9AickLyzX c+X2kdqFQdWoRHuiOCqpr7PRAz255XSJR+80axdSLNZYFGM5DFixtxWdQY/zSOB4N8oJ 1AVmlpGhtc4MJwnvO9I98fFfZxN5ORpyVvPdmrvqrqe4CJ9NAz5LrW++cD3ntlMfRlj+ Ch8x2BAbxZVgOZYupXSPols2nRg6y658Y2eawCqETtSADHgB5SFqJUr3096YYa5ews1M f2EnvsydRotuaTqDe3bbqEo6AivjPETIXcFU5i6kq6K09KQJuQW5o+bSD+IjURnPuhpX np/A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:to:mime-version:fcc:date:references :in-reply-to:message-id:from:dkim-signature:delivered-to; bh=o0MkN0bAd5LFEP1oD70aLyvo2zVLsO2YXgdRxdnn0tk=; b=qk+mNvDjVpeHeVN3MxlH6sLcVB4UtrZiWtQSbergzZFFNOjkuV9uk3VmsY/cD7f7q0 i3fp1nZiouv6Z+AXA4kDC6zyG60l1BhmniJCm6trtg8zu1XeR2CCwwW/PMDyaF0QVsIl +Wcf3jlRz8oFFB3z4kRKwwDc0BqRfp+g+iobNVjfIyzA/BqyFBcaJmYbMYFQr8cQEg3P e2h/LKt7hsQxFRBVKilI9RI19rkTDQ+GCn+Z/F/0J/A77ndEj1uCXptM05mgB/Sz3fq9 fnMi5f22eU3iZcBzWmAIVvk9MSga5qHZmhz93U+qRPDIJn4S10xlQmUVIyPWi2wE9Oaq T/xA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=o78Yml+V; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id l24-20020aa7cad8000000b0048772a84504si9048938edt.591.2023.01.09.10.40.43; Mon, 09 Jan 2023 10:40:44 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=o78Yml+V; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 793F668BC4A; Mon, 9 Jan 2023 20:40:39 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pj1-f45.google.com (mail-pj1-f45.google.com [209.85.216.45]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id E365968A215 for ; Mon, 9 Jan 2023 20:40:27 +0200 (EET) Received: by mail-pj1-f45.google.com with SMTP id n12so9702777pjp.1; Mon, 09 Jan 2023 10:40:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date :references:in-reply-to:message-id:from:from:to:cc:subject:date :message-id:reply-to; bh=xVdxBNWw+z7VJL7TKXrq0n9tozX4uOoYrYBB6JQuVhw=; b=o78Yml+VWxHW5RePmZ7BPgJqyA5bINBLLe4EL8kf2mptv1PA1xfGSa3Ys3q4IWIKa+ f6iG6r3HJEI/VPJDf+e0QRYKtL96/E6V1bE5acD13mWRWjUHVZ7XgNb2Gk20sz6pq/HZ ipUV63vYYOrUVzjzapWfUL3WocG/AZBJ/2bhbNifZhyaK4C/EL90fjxMr2Ot4BavPGdk ZANXyK65qNa7qzIFW1+9gjt1mpf7VG7Y70dtQgmtCJvxLvPjFTnzQlRqsCpMnPsxXVxy SMX8cW376c57TO/0eU/77GHkz3CJSrrqiC5bWbluGhh9UipGUtZpmTVV3Uu+39X9bru0 iTnw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date :references:in-reply-to:message-id:from:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=xVdxBNWw+z7VJL7TKXrq0n9tozX4uOoYrYBB6JQuVhw=; b=HyhrnG8qOAuCueM4tR0kkYTSKWqY4XFMWeWvamIgyetJ1Ll4FXvh14KPNR6SubMKos wnhcUWdH069X5kbscf0mNkd8IgMLm40pdbU8O21ZR/SjeemsX8Uj3YW0tmWrJ4pjlMhs yGPMQyeMaInrl3+DfUGvIzTHLJ5y7HSm4Om/5OGImNAmV5EK9wJl8yOnwx+8wYfuqg3W N84zDdNqWhYBiY5NND75WrAgiHIZ53DmpgAQ4dXBZeGloUj3WMBtt1YcDAzzmZUp4RAK b3sZkpfADGM/kGMivmPmAzI65Fa12+F+GH0VfjbjO8TFBskK9b3PjNVioZRQNni3/WN1 uuEQ== X-Gm-Message-State: AFqh2kqBA/KlKRWC4vIegcB/yvnrWVxAAk6kr+kUcTrLENHD1cETiQmt SoK2DBw1j0kzRFFOUtSEIWJWyuTmz4A= X-Received: by 2002:a17:903:32c6:b0:189:df3c:1ba1 with SMTP id i6-20020a17090332c600b00189df3c1ba1mr90801274plr.38.1673289625741; Mon, 09 Jan 2023 10:40:25 -0800 (PST) Received: from [127.0.0.1] (master.gitmailbox.com. [34.83.118.50]) by smtp.gmail.com with ESMTPSA id n7-20020a170902e54700b0019269969d14sm6345536plf.282.2023.01.09.10.40.25 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 09 Jan 2023 10:40:25 -0800 (PST) From: Aman Karmani X-Google-Original-From: Aman Karmani Message-Id: In-Reply-To: References: Date: Mon, 09 Jan 2023 18:40:24 +0000 Fcc: Sent MIME-Version: 1.0 To: ffmpeg-devel@ffmpeg.org Subject: [FFmpeg-devel] [PATCH v4] avcodec/vaapi_encode_h26x: passthrough A53 CC data as H264/HEVC SEI X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Aman Karmani , "Xiang, Haihao" , Andreas Rheinhardt Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 9jUPu3UDjUcZ From: Aman Karmani Signed-off-by: Aman Karmani --- avcodec/vaapi_encode_h26x: passthrough A53 CC data as H264/HEVC SEI v4: updated deps in configure v3: fix build failure v2: add control via sei parameter Published-As: https://github.com/ffstaging/FFmpeg/releases/tag/pr-ffstaging-46%2Ftmm1%2Fvaapi-a53cc-v4 Fetch-It-Via: git fetch https://github.com/ffstaging/FFmpeg pr-ffstaging-46/tmm1/vaapi-a53cc-v4 Pull-Request: https://github.com/ffstaging/FFmpeg/pull/46 Range-diff vs v3: 1: 149fa8e61c ! 1: 367db524fb avcodec/vaapi_encode_h26x: passthrough A53 CC data as H264/HEVC SEI @@ Commit message Signed-off-by: Aman Karmani + ## configure ## +@@ configure: h264_qsv_decoder_select="h264_mp4toannexb_bsf qsvdec" + h264_qsv_encoder_select="atsc_a53 qsvenc" + h264_rkmpp_decoder_deps="rkmpp" + h264_rkmpp_decoder_select="h264_mp4toannexb_bsf" +-h264_vaapi_encoder_select="cbs_h264 vaapi_encode" ++h264_vaapi_encoder_select="atsc_a53 cbs_h264 vaapi_encode" + h264_v4l2m2m_decoder_deps="v4l2_m2m h264_v4l2_m2m" + h264_v4l2m2m_decoder_select="h264_mp4toannexb_bsf" + h264_v4l2m2m_encoder_deps="v4l2_m2m h264_v4l2_m2m" +@@ configure: hevc_qsv_encoder_select="hevcparse qsvenc" + hevc_rkmpp_decoder_deps="rkmpp" + hevc_rkmpp_decoder_select="hevc_mp4toannexb_bsf" + hevc_vaapi_encoder_deps="VAEncPictureParameterBufferHEVC" +-hevc_vaapi_encoder_select="cbs_h265 vaapi_encode" ++hevc_vaapi_encoder_select="atsc_a53 cbs_h265 vaapi_encode" + hevc_v4l2m2m_decoder_deps="v4l2_m2m hevc_v4l2_m2m" + hevc_v4l2m2m_decoder_select="hevc_mp4toannexb_bsf" + hevc_v4l2m2m_encoder_deps="v4l2_m2m hevc_v4l2_m2m" + ## libavcodec/vaapi_encode_h264.c ## @@ #include "libavutil/internal.h" configure | 4 ++-- libavcodec/vaapi_encode_h264.c | 33 +++++++++++++++++++++++++++++++- libavcodec/vaapi_encode_h265.c | 35 +++++++++++++++++++++++++++++++++- 3 files changed, 68 insertions(+), 4 deletions(-) base-commit: 3bcec58535d395945a390bdc7af4048a7abc60eb diff --git a/configure b/configure index e54649fa48..5ba1395bea 100755 --- a/configure +++ b/configure @@ -3191,7 +3191,7 @@ h264_qsv_decoder_select="h264_mp4toannexb_bsf qsvdec" h264_qsv_encoder_select="atsc_a53 qsvenc" h264_rkmpp_decoder_deps="rkmpp" h264_rkmpp_decoder_select="h264_mp4toannexb_bsf" -h264_vaapi_encoder_select="cbs_h264 vaapi_encode" +h264_vaapi_encoder_select="atsc_a53 cbs_h264 vaapi_encode" h264_v4l2m2m_decoder_deps="v4l2_m2m h264_v4l2_m2m" h264_v4l2m2m_decoder_select="h264_mp4toannexb_bsf" h264_v4l2m2m_encoder_deps="v4l2_m2m h264_v4l2_m2m" @@ -3208,7 +3208,7 @@ hevc_qsv_encoder_select="hevcparse qsvenc" hevc_rkmpp_decoder_deps="rkmpp" hevc_rkmpp_decoder_select="hevc_mp4toannexb_bsf" hevc_vaapi_encoder_deps="VAEncPictureParameterBufferHEVC" -hevc_vaapi_encoder_select="cbs_h265 vaapi_encode" +hevc_vaapi_encoder_select="atsc_a53 cbs_h265 vaapi_encode" hevc_v4l2m2m_decoder_deps="v4l2_m2m hevc_v4l2_m2m" hevc_v4l2m2m_decoder_select="hevc_mp4toannexb_bsf" hevc_v4l2m2m_encoder_deps="v4l2_m2m hevc_v4l2_m2m" diff --git a/libavcodec/vaapi_encode_h264.c b/libavcodec/vaapi_encode_h264.c index b1b503b2a6..9f9d77a0e9 100644 --- a/libavcodec/vaapi_encode_h264.c +++ b/libavcodec/vaapi_encode_h264.c @@ -26,6 +26,7 @@ #include "libavutil/internal.h" #include "libavutil/opt.h" +#include "atsc_a53.h" #include "avcodec.h" #include "cbs.h" #include "cbs_h264.h" @@ -40,6 +41,7 @@ enum { SEI_TIMING = 0x01, SEI_IDENTIFIER = 0x02, SEI_RECOVERY_POINT = 0x04, + SEI_A53_CC = 0x08, }; // Random (version 4) ISO 11578 UUID. @@ -98,6 +100,8 @@ typedef struct VAAPIEncodeH264Context { H264RawSEIRecoveryPoint sei_recovery_point; SEIRawUserDataUnregistered sei_identifier; char *sei_identifier_string; + SEIRawUserDataRegistered sei_a53cc; + void *sei_a53cc_data; int aud_needed; int sei_needed; @@ -248,6 +252,13 @@ static int vaapi_encode_h264_write_extra_header(AVCodecContext *avctx, if (err < 0) goto fail; } + if (priv->sei_needed & SEI_A53_CC) { + err = ff_cbs_sei_add_message(priv->cbc, au, 1, + SEI_TYPE_USER_DATA_REGISTERED_ITU_T_T35, + &priv->sei_a53cc, NULL); + if (err < 0) + goto fail; + } priv->sei_needed = 0; @@ -681,6 +692,22 @@ static int vaapi_encode_h264_init_picture_params(AVCodecContext *avctx, priv->sei_needed |= SEI_RECOVERY_POINT; } + if (priv->sei & SEI_A53_CC) { + int err; + size_t sei_a53cc_len; + av_freep(&priv->sei_a53cc_data); + err = ff_alloc_a53_sei(pic->input_image, 0, &priv->sei_a53cc_data, &sei_a53cc_len); + if (err < 0) + return err; + if (priv->sei_a53cc_data != NULL) { + priv->sei_a53cc.itu_t_t35_country_code = 181; + priv->sei_a53cc.data = (uint8_t *)priv->sei_a53cc_data + 1; + priv->sei_a53cc.data_length = sei_a53cc_len - 1; + + priv->sei_needed |= SEI_A53_CC; + } + } + vpic->CurrPic = (VAPictureH264) { .picture_id = pic->recon_surface, .frame_idx = hpic->frame_num, @@ -1226,6 +1253,7 @@ static av_cold int vaapi_encode_h264_close(AVCodecContext *avctx) ff_cbs_fragment_free(&priv->current_access_unit); ff_cbs_close(&priv->cbc); av_freep(&priv->sei_identifier_string); + av_freep(&priv->sei_a53cc_data); return ff_vaapi_encode_close(avctx); } @@ -1252,7 +1280,7 @@ static const AVOption vaapi_encode_h264_options[] = { { "sei", "Set SEI to include", OFFSET(sei), AV_OPT_TYPE_FLAGS, - { .i64 = SEI_IDENTIFIER | SEI_TIMING | SEI_RECOVERY_POINT }, + { .i64 = SEI_IDENTIFIER | SEI_TIMING | SEI_RECOVERY_POINT | SEI_A53_CC }, 0, INT_MAX, FLAGS, "sei" }, { "identifier", "Include encoder version identifier", 0, AV_OPT_TYPE_CONST, { .i64 = SEI_IDENTIFIER }, @@ -1263,6 +1291,9 @@ static const AVOption vaapi_encode_h264_options[] = { { "recovery_point", "Include recovery points where appropriate", 0, AV_OPT_TYPE_CONST, { .i64 = SEI_RECOVERY_POINT }, INT_MIN, INT_MAX, FLAGS, "sei" }, + { "a53_cc", "Include A/53 caption data", + 0, AV_OPT_TYPE_CONST, { .i64 = SEI_A53_CC }, + INT_MIN, INT_MAX, FLAGS, "sei" }, { "profile", "Set profile (profile_idc and constraint_set*_flag)", OFFSET(profile), AV_OPT_TYPE_INT, diff --git a/libavcodec/vaapi_encode_h265.c b/libavcodec/vaapi_encode_h265.c index 94b56c6578..f5b85cef6f 100644 --- a/libavcodec/vaapi_encode_h265.c +++ b/libavcodec/vaapi_encode_h265.c @@ -27,6 +27,7 @@ #include "libavutil/opt.h" #include "libavutil/mastering_display_metadata.h" +#include "atsc_a53.h" #include "avcodec.h" #include "cbs.h" #include "cbs_h265.h" @@ -40,6 +41,7 @@ enum { SEI_MASTERING_DISPLAY = 0x08, SEI_CONTENT_LIGHT_LEVEL = 0x10, + SEI_A53_CC = 0x20, }; typedef struct VAAPIEncodeH265Picture { @@ -84,6 +86,8 @@ typedef struct VAAPIEncodeH265Context { SEIRawMasteringDisplayColourVolume sei_mastering_display; SEIRawContentLightLevelInfo sei_content_light_level; + SEIRawUserDataRegistered sei_a53cc; + void *sei_a53cc_data; CodedBitstreamContext *cbc; CodedBitstreamFragment current_access_unit; @@ -226,6 +230,13 @@ static int vaapi_encode_h265_write_extra_header(AVCodecContext *avctx, if (err < 0) goto fail; } + if (priv->sei_needed & SEI_A53_CC) { + err = ff_cbs_sei_add_message(priv->cbc, au, 1, + SEI_TYPE_USER_DATA_REGISTERED_ITU_T_T35, + &priv->sei_a53cc, NULL); + if (err < 0) + goto fail; + } priv->sei_needed = 0; @@ -888,6 +899,22 @@ static int vaapi_encode_h265_init_picture_params(AVCodecContext *avctx, } } + if (priv->sei & SEI_A53_CC) { + int err; + size_t sei_a53cc_len; + av_freep(&priv->sei_a53cc_data); + err = ff_alloc_a53_sei(pic->input_image, 0, &priv->sei_a53cc_data, &sei_a53cc_len); + if (err < 0) + return err; + if (priv->sei_a53cc_data != NULL) { + priv->sei_a53cc.itu_t_t35_country_code = 181; + priv->sei_a53cc.data = (uint8_t *)priv->sei_a53cc_data + 1; + priv->sei_a53cc.data_length = sei_a53cc_len - 1; + + priv->sei_needed |= SEI_A53_CC; + } + } + vpic->decoded_curr_pic = (VAPictureHEVC) { .picture_id = pic->recon_surface, .pic_order_cnt = hpic->pic_order_cnt, @@ -1355,6 +1382,7 @@ static av_cold int vaapi_encode_h265_close(AVCodecContext *avctx) ff_cbs_fragment_free(&priv->current_access_unit); ff_cbs_close(&priv->cbc); + av_freep(&priv->sei_a53cc_data); return ff_vaapi_encode_close(avctx); } @@ -1413,7 +1441,7 @@ static const AVOption vaapi_encode_h265_options[] = { { "sei", "Set SEI to include", OFFSET(sei), AV_OPT_TYPE_FLAGS, - { .i64 = SEI_MASTERING_DISPLAY | SEI_CONTENT_LIGHT_LEVEL }, + { .i64 = SEI_MASTERING_DISPLAY | SEI_CONTENT_LIGHT_LEVEL | SEI_A53_CC }, 0, INT_MAX, FLAGS, "sei" }, { "hdr", "Include HDR metadata for mastering display colour volume " @@ -1421,6 +1449,11 @@ static const AVOption vaapi_encode_h265_options[] = { 0, AV_OPT_TYPE_CONST, { .i64 = SEI_MASTERING_DISPLAY | SEI_CONTENT_LIGHT_LEVEL }, INT_MIN, INT_MAX, FLAGS, "sei" }, + { "a53_cc", + "Include A/53 caption data", + 0, AV_OPT_TYPE_CONST, + { .i64 = SEI_A53_CC }, + INT_MIN, INT_MAX, FLAGS, "sei" }, { "tiles", "Tile columns x rows", OFFSET(common.tile_cols), AV_OPT_TYPE_IMAGE_SIZE,