From patchwork Fri Jan 6 17:41:24 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aman Karmani X-Patchwork-Id: 39905 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:bc95:b0:ad:ade2:bfd2 with SMTP id fx21csp1316062pzb; Fri, 6 Jan 2023 09:41:43 -0800 (PST) X-Google-Smtp-Source: AMrXdXuV1roNZ6Q/sRr8GBKzh7sIOLLMDcG6FP5a1USDaqHF5hjmtd9CtAOhXNXkCc2ovpRm95mm X-Received: by 2002:a17:907:8a24:b0:846:95f9:5b4d with SMTP id sc36-20020a1709078a2400b0084695f95b4dmr62775029ejc.42.1673026903493; Fri, 06 Jan 2023 09:41:43 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1673026903; cv=none; d=google.com; s=arc-20160816; b=FYNbqSzrtfAQMZkTz1A6O32COn2EJWa9aDpjf687VVutWZY+re43W0FcUv2V8VGYRZ DDSith1JCJx5VEO2/FJNIkK4hshqDmjXe4I/7H+ZXZrRjdVaBwfr+uIqmIC9zi5rxLPg BAVwFaauh9rsyeBxU6MegxnXWVCyiX8l+pZHq1zU6+XJlDR77x4A0O3JEHcEdGqOvQg2 3MTU9QC65/DgX7vDRmgdR1vc1YCh0XQvXuW/uibbxSJotxiSolTF/85J19ljpo1XvM/D zSSdnAUMqc4Ag9sjrtx1OimvOn8vUrmJOYkX1IAhFV+NOi7Adv0UxcMSHGiWQVLfnQm/ 322g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:to:mime-version:fcc:date:references :in-reply-to:message-id:from:dkim-signature:delivered-to; bh=x/qWB6Lqkdfm3q14v7vp5jVk1cfb0YUOulAWtnjCwL8=; b=PdPOlH5B1YguDOCEd8GMp+pEsFAPNWKo1iUtmy2LKmty/7ny9KTMPggrbA0Cm1Ct+V tgLmsPRsRigHPRNdDmdhror8lTMR8GeAVBt1VaNWkKzLYoADlnzjCcdMI/7Jg2duamVD I6X2DZj0XgsayIVMECr/KJCQZloWTBewOaalwu4suiadiwGWfQsbJcYF6zLOwFu9Z6/J WHdK/cxVJnZyifGyJ0vbOth4azD+lpmy7hi/LsEO81RQ+52XKtZJaWZ/Rt+A1nFfBmyi NmtguubtWTP2MYLarIZJdgA9T2b9r1hl3J1cCehE/1RFa/UWMrlMqQA9WWeXmlbITg8n 1Gvg== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=d3mXu1V2; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id s19-20020a170906455300b0084d200455aesi1342446ejq.224.2023.01.06.09.41.42; Fri, 06 Jan 2023 09:41:43 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=d3mXu1V2; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 0E07668BCC3; Fri, 6 Jan 2023 19:41:40 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pj1-f47.google.com (mail-pj1-f47.google.com [209.85.216.47]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 7433468BA61 for ; Fri, 6 Jan 2023 19:41:28 +0200 (EET) Received: by mail-pj1-f47.google.com with SMTP id q64so2147219pjq.4; Fri, 06 Jan 2023 09:41:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date :references:in-reply-to:message-id:from:from:to:cc:subject:date :message-id:reply-to; bh=dk0UBIetwQOS5mpEq2QjuvNGoW0FqOfm/OKHt/PH5ro=; b=d3mXu1V2Cs8dsIlUKrDDjQPY8Vj3JOay3WrGU73eCQYfwgByX61hr7lCDikGGi6+CG VREVgeIYkAURjP65oVuH/ezl4/R5n/3hxc/qL4WJVZi1XOlhnUazbu8j0gTddusUO7D8 3P+RfTvnb/UjN2vWhJsQbqQ5HtJTlXw2fnOa8WMKYgKCO+4r8Ajg2CQJBeQK14LLeeIj 2B4Xn7fxDNFDh2+lq1ID8jc+ci6z/tNMDtZbr8kpTcVV3B+LKYsIOZ6TibUAA27McyjE K9twJpjTCs2BSsasvzAoM73tv7gaLFDWrBB5X1PDlMxAZ0Q4IeuGncX3l4adyRqDUX5V KGiw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date :references:in-reply-to:message-id:from:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=dk0UBIetwQOS5mpEq2QjuvNGoW0FqOfm/OKHt/PH5ro=; b=KdUD/wAeGjLd35nHc8GJORT6YCleYViXAmdTjYdYfqUeneZ/jlb9J+/UilUczVJJz5 jENTI4gvgwKg4y+NCm1K//DPFkb1HV2aFdEiIQSAxs1OP4gHHg00MlMy3qlTGpXskKjO 3wbSXFNVEdYIyJFGtW+dyweT+C6Wai5jlnEGcGF9kdexv0EEd+wuAA5P+AuO7ZJYp68l XH8XHqqPEmsEBWRV/3+Po2d8kwyjTyw/USClXqqd9JbuzlVNFAQeqy2nVG8HGyFKKc8j LScdOV6OaYNTJXGCoMkvwQl6QkdRL//VvG3TEVH74yEkETyAynN0Mzt4/TwDfir1TuLG iz3A== X-Gm-Message-State: AFqh2kqzHXbwLoABFgbJCSywNJb/EF5/lxv7t1qxS+GX7/oUFm90mS1b l6shUBsC60MlxOJqL6pHffPhVOs48YA= X-Received: by 2002:a05:6a20:b044:b0:ac:ab16:442 with SMTP id dx4-20020a056a20b04400b000acab160442mr54765888pzb.4.1673026886366; Fri, 06 Jan 2023 09:41:26 -0800 (PST) Received: from [127.0.0.1] (master.gitmailbox.com. [34.83.118.50]) by smtp.gmail.com with ESMTPSA id o9-20020a170903210900b0017fe9b038fdsm1280480ple.14.2023.01.06.09.41.24 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 06 Jan 2023 09:41:25 -0800 (PST) From: Aman Karmani X-Google-Original-From: Aman Karmani Message-Id: In-Reply-To: References: Date: Fri, 06 Jan 2023 17:41:24 +0000 Fcc: Sent MIME-Version: 1.0 To: ffmpeg-devel@ffmpeg.org Subject: [FFmpeg-devel] [PATCH v3] avcodec/vaapi_encode_h26x: passthrough A53 CC data as H264/HEVC SEI X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Aman Karmani , "Xiang, Haihao" Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: dn8akKILjdQp From: Aman Karmani Signed-off-by: Aman Karmani --- avcodec/vaapi_encode_h26x: passthrough A53 CC data as H264/HEVC SEI v3: fix build failure v2: add control via sei parameter Published-As: https://github.com/ffstaging/FFmpeg/releases/tag/pr-ffstaging-46%2Ftmm1%2Fvaapi-a53cc-v3 Fetch-It-Via: git fetch https://github.com/ffstaging/FFmpeg pr-ffstaging-46/tmm1/vaapi-a53cc-v3 Pull-Request: https://github.com/ffstaging/FFmpeg/pull/46 Range-diff vs v2: 1: 9a2fee07a9 ! 1: 149fa8e61c avcodec/vaapi_encode_h26x: passthrough A53 CC data as H264/HEVC SEI @@ libavcodec/vaapi_encode_h264.c: static const AVOption vaapi_encode_h264_options[ { "recovery_point", "Include recovery points where appropriate", 0, AV_OPT_TYPE_CONST, { .i64 = SEI_RECOVERY_POINT }, INT_MIN, INT_MAX, FLAGS, "sei" }, -+ { "a53_cc", "Include A/53 caption data" ++ { "a53_cc", "Include A/53 caption data", + 0, AV_OPT_TYPE_CONST, { .i64 = SEI_A53_CC }, + INT_MIN, INT_MAX, FLAGS, "sei" }, @@ libavcodec/vaapi_encode_h265.c: static const AVOption vaapi_encode_h265_options[ { .i64 = SEI_MASTERING_DISPLAY | SEI_CONTENT_LIGHT_LEVEL }, INT_MIN, INT_MAX, FLAGS, "sei" }, + { "a53_cc", -+ "Include A/53 caption data" ++ "Include A/53 caption data", + 0, AV_OPT_TYPE_CONST, + { .i64 = SEI_A53_CC }, + INT_MIN, INT_MAX, FLAGS, "sei" }, libavcodec/vaapi_encode_h264.c | 33 +++++++++++++++++++++++++++++++- libavcodec/vaapi_encode_h265.c | 35 +++++++++++++++++++++++++++++++++- 2 files changed, 66 insertions(+), 2 deletions(-) base-commit: 3bcec58535d395945a390bdc7af4048a7abc60eb diff --git a/libavcodec/vaapi_encode_h264.c b/libavcodec/vaapi_encode_h264.c index b1b503b2a6..9f9d77a0e9 100644 --- a/libavcodec/vaapi_encode_h264.c +++ b/libavcodec/vaapi_encode_h264.c @@ -26,6 +26,7 @@ #include "libavutil/internal.h" #include "libavutil/opt.h" +#include "atsc_a53.h" #include "avcodec.h" #include "cbs.h" #include "cbs_h264.h" @@ -40,6 +41,7 @@ enum { SEI_TIMING = 0x01, SEI_IDENTIFIER = 0x02, SEI_RECOVERY_POINT = 0x04, + SEI_A53_CC = 0x08, }; // Random (version 4) ISO 11578 UUID. @@ -98,6 +100,8 @@ typedef struct VAAPIEncodeH264Context { H264RawSEIRecoveryPoint sei_recovery_point; SEIRawUserDataUnregistered sei_identifier; char *sei_identifier_string; + SEIRawUserDataRegistered sei_a53cc; + void *sei_a53cc_data; int aud_needed; int sei_needed; @@ -248,6 +252,13 @@ static int vaapi_encode_h264_write_extra_header(AVCodecContext *avctx, if (err < 0) goto fail; } + if (priv->sei_needed & SEI_A53_CC) { + err = ff_cbs_sei_add_message(priv->cbc, au, 1, + SEI_TYPE_USER_DATA_REGISTERED_ITU_T_T35, + &priv->sei_a53cc, NULL); + if (err < 0) + goto fail; + } priv->sei_needed = 0; @@ -681,6 +692,22 @@ static int vaapi_encode_h264_init_picture_params(AVCodecContext *avctx, priv->sei_needed |= SEI_RECOVERY_POINT; } + if (priv->sei & SEI_A53_CC) { + int err; + size_t sei_a53cc_len; + av_freep(&priv->sei_a53cc_data); + err = ff_alloc_a53_sei(pic->input_image, 0, &priv->sei_a53cc_data, &sei_a53cc_len); + if (err < 0) + return err; + if (priv->sei_a53cc_data != NULL) { + priv->sei_a53cc.itu_t_t35_country_code = 181; + priv->sei_a53cc.data = (uint8_t *)priv->sei_a53cc_data + 1; + priv->sei_a53cc.data_length = sei_a53cc_len - 1; + + priv->sei_needed |= SEI_A53_CC; + } + } + vpic->CurrPic = (VAPictureH264) { .picture_id = pic->recon_surface, .frame_idx = hpic->frame_num, @@ -1226,6 +1253,7 @@ static av_cold int vaapi_encode_h264_close(AVCodecContext *avctx) ff_cbs_fragment_free(&priv->current_access_unit); ff_cbs_close(&priv->cbc); av_freep(&priv->sei_identifier_string); + av_freep(&priv->sei_a53cc_data); return ff_vaapi_encode_close(avctx); } @@ -1252,7 +1280,7 @@ static const AVOption vaapi_encode_h264_options[] = { { "sei", "Set SEI to include", OFFSET(sei), AV_OPT_TYPE_FLAGS, - { .i64 = SEI_IDENTIFIER | SEI_TIMING | SEI_RECOVERY_POINT }, + { .i64 = SEI_IDENTIFIER | SEI_TIMING | SEI_RECOVERY_POINT | SEI_A53_CC }, 0, INT_MAX, FLAGS, "sei" }, { "identifier", "Include encoder version identifier", 0, AV_OPT_TYPE_CONST, { .i64 = SEI_IDENTIFIER }, @@ -1263,6 +1291,9 @@ static const AVOption vaapi_encode_h264_options[] = { { "recovery_point", "Include recovery points where appropriate", 0, AV_OPT_TYPE_CONST, { .i64 = SEI_RECOVERY_POINT }, INT_MIN, INT_MAX, FLAGS, "sei" }, + { "a53_cc", "Include A/53 caption data", + 0, AV_OPT_TYPE_CONST, { .i64 = SEI_A53_CC }, + INT_MIN, INT_MAX, FLAGS, "sei" }, { "profile", "Set profile (profile_idc and constraint_set*_flag)", OFFSET(profile), AV_OPT_TYPE_INT, diff --git a/libavcodec/vaapi_encode_h265.c b/libavcodec/vaapi_encode_h265.c index 94b56c6578..f5b85cef6f 100644 --- a/libavcodec/vaapi_encode_h265.c +++ b/libavcodec/vaapi_encode_h265.c @@ -27,6 +27,7 @@ #include "libavutil/opt.h" #include "libavutil/mastering_display_metadata.h" +#include "atsc_a53.h" #include "avcodec.h" #include "cbs.h" #include "cbs_h265.h" @@ -40,6 +41,7 @@ enum { SEI_MASTERING_DISPLAY = 0x08, SEI_CONTENT_LIGHT_LEVEL = 0x10, + SEI_A53_CC = 0x20, }; typedef struct VAAPIEncodeH265Picture { @@ -84,6 +86,8 @@ typedef struct VAAPIEncodeH265Context { SEIRawMasteringDisplayColourVolume sei_mastering_display; SEIRawContentLightLevelInfo sei_content_light_level; + SEIRawUserDataRegistered sei_a53cc; + void *sei_a53cc_data; CodedBitstreamContext *cbc; CodedBitstreamFragment current_access_unit; @@ -226,6 +230,13 @@ static int vaapi_encode_h265_write_extra_header(AVCodecContext *avctx, if (err < 0) goto fail; } + if (priv->sei_needed & SEI_A53_CC) { + err = ff_cbs_sei_add_message(priv->cbc, au, 1, + SEI_TYPE_USER_DATA_REGISTERED_ITU_T_T35, + &priv->sei_a53cc, NULL); + if (err < 0) + goto fail; + } priv->sei_needed = 0; @@ -888,6 +899,22 @@ static int vaapi_encode_h265_init_picture_params(AVCodecContext *avctx, } } + if (priv->sei & SEI_A53_CC) { + int err; + size_t sei_a53cc_len; + av_freep(&priv->sei_a53cc_data); + err = ff_alloc_a53_sei(pic->input_image, 0, &priv->sei_a53cc_data, &sei_a53cc_len); + if (err < 0) + return err; + if (priv->sei_a53cc_data != NULL) { + priv->sei_a53cc.itu_t_t35_country_code = 181; + priv->sei_a53cc.data = (uint8_t *)priv->sei_a53cc_data + 1; + priv->sei_a53cc.data_length = sei_a53cc_len - 1; + + priv->sei_needed |= SEI_A53_CC; + } + } + vpic->decoded_curr_pic = (VAPictureHEVC) { .picture_id = pic->recon_surface, .pic_order_cnt = hpic->pic_order_cnt, @@ -1355,6 +1382,7 @@ static av_cold int vaapi_encode_h265_close(AVCodecContext *avctx) ff_cbs_fragment_free(&priv->current_access_unit); ff_cbs_close(&priv->cbc); + av_freep(&priv->sei_a53cc_data); return ff_vaapi_encode_close(avctx); } @@ -1413,7 +1441,7 @@ static const AVOption vaapi_encode_h265_options[] = { { "sei", "Set SEI to include", OFFSET(sei), AV_OPT_TYPE_FLAGS, - { .i64 = SEI_MASTERING_DISPLAY | SEI_CONTENT_LIGHT_LEVEL }, + { .i64 = SEI_MASTERING_DISPLAY | SEI_CONTENT_LIGHT_LEVEL | SEI_A53_CC }, 0, INT_MAX, FLAGS, "sei" }, { "hdr", "Include HDR metadata for mastering display colour volume " @@ -1421,6 +1449,11 @@ static const AVOption vaapi_encode_h265_options[] = { 0, AV_OPT_TYPE_CONST, { .i64 = SEI_MASTERING_DISPLAY | SEI_CONTENT_LIGHT_LEVEL }, INT_MIN, INT_MAX, FLAGS, "sei" }, + { "a53_cc", + "Include A/53 caption data", + 0, AV_OPT_TYPE_CONST, + { .i64 = SEI_A53_CC }, + INT_MIN, INT_MAX, FLAGS, "sei" }, { "tiles", "Tile columns x rows", OFFSET(common.tile_cols), AV_OPT_TYPE_IMAGE_SIZE,