From patchwork Fri Jul 21 21:30:55 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Devin Heitmueller X-Patchwork-Id: 42895 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:be1d:b0:130:ccc6:6c4b with SMTP id ge29csp504322pzb; Fri, 21 Jul 2023 14:31:23 -0700 (PDT) X-Google-Smtp-Source: APBJJlEXzI+q0N5/Hk/4CN/J+n+In9vZPkn+fSzp0qv35oNhb/nQRPxv6kuM7Pf/oZ39jR9fjKTk X-Received: by 2002:a17:906:2212:b0:98d:63c5:d132 with SMTP id s18-20020a170906221200b0098d63c5d132mr2646580ejs.72.1689975082817; Fri, 21 Jul 2023 14:31:22 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1689975082; cv=none; d=google.com; s=arc-20160816; b=HxrzCjtaXkzZGxJNfjwVu8I9FmvbZvpBf5rMYFmrFLlp5zl9pxJSalOyqt/j9Ae0Dv fZLiRY67HbLUY+Tgs4GRFLG24OTTQPex81AtGUbR357gT2xSe15vFh8i3FgJVcEv5fv5 sndc+nSU+DQ4Pt3+N0eT6GWJV8F2WsJANt7GUhFJ9cp5MrZTsjVA3CAa9455mNP7AD0H CpV2rmT5MgjTtHCZDFYwNJkcrao/Bb8S3EdZE+gTOS3RSpCJK2NSNpj/vt//cQhjBcZy Cwo6Is2U8I74EhNpKDfjD1J4++IcOh/448/rSgnSJyhRewx5AP/S+ESBhKntrW5p88/m tkmw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:dkim-signature:delivered-to; bh=J407YStqrm1tmPqYY6p2T+WO8Ci0U9MTHHKBlDKxYWs=; fh=DKzdJpyueDOox6ShoNYUi750QZs6R2naOS9+hcsrM90=; b=xSIB1wJ4SUoUjWRYqmZDcLi+9Cyh4Qrm3Sb3ZaJBUQf3jNFp/v2C+7Z5HbI3/3tF6O j0lZOHjp6WjB1bTv6CwmO7qJ+THmIVx4rHdB/BixcgzLk/4I8HuIDSHHQAYwa/EFjjIc khN91wx7R0uYTTLVWVB9T64XGBGYZGz+i/tvvkQLvTTMsaDqukV82tMrqFq0+RJ0z1u6 UgtzWAEW7B2aQdInXgr/wq6Ioe+bAcUDyqr2fo9gA794lcPknD6xSA5anjOaj0ZHDUYo 7RnPaM+XhdRImSNzRajwVQJT3B6fpZAQnFan9E/fCFcoHXM2LEr4zyQf1joICQSA5HeS 5eJQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20221208.gappssmtp.com header.s=20221208 header.b=lEmDiNdG; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id qp25-20020a170907207900b00988698d7c35si2614320ejb.944.2023.07.21.14.31.22; Fri, 21 Jul 2023 14:31:22 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20221208.gappssmtp.com header.s=20221208 header.b=lEmDiNdG; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id B268468C50A; Sat, 22 Jul 2023 00:31:09 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-qk1-f181.google.com (mail-qk1-f181.google.com [209.85.222.181]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 4E3D868BA8D for ; Sat, 22 Jul 2023 00:31:02 +0300 (EEST) Received: by mail-qk1-f181.google.com with SMTP id af79cd13be357-76ad8892d49so171780185a.1 for ; Fri, 21 Jul 2023 14:31:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ltnglobal-com.20221208.gappssmtp.com; s=20221208; t=1689975061; x=1690579861; h=references:in-reply-to:message-id:date:subject:cc:to:from:from:to :cc:subject:date:message-id:reply-to; bh=cgwJo8lrgybHdg/86SG6hXfodKe0QgRAPYHHpn84UKk=; b=lEmDiNdGt7dawZZvSC3eMOby68iE0L7zsTg59Q+H/zDpJ63U/cu8dhDKs257TLJalv Pjb0darNISK/BGSEI8pqoDgStUFUmxmf1NKNkmzqXWISyyewbMgy0l4tigMLUIjC8+3A AR5UNZbkB05OevZRhJLDk4PBml4y09MeSuBJ23uubYErCRxEag33y+dgNsc0TIrzhNJ9 yfNPmdrTJM/lsvWhZmjKcYSb1xyhf4Z+Tn+mAH8jYBcmrfw3dGxCcWcxXV7pQ4P+vN3f YvsJSm82IqQJSsyO9uOuvPsFz3sopMDqe+/snq97gbtkZDgztrwniyZdLrzaLgN9VYLH 6ikg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689975061; x=1690579861; h=references:in-reply-to:message-id:date:subject:cc:to:from :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=cgwJo8lrgybHdg/86SG6hXfodKe0QgRAPYHHpn84UKk=; b=cGwGAj2KFtLMFJrHfBZR8dUVEoOjC6n95R7WuLk5/eB50EkIvfsAuhRZKtRPYNH4P7 E4Hq6lb43A8lTzCKID0gPjK9OZQctcfRcYZkiip2RgFcmr/COO2tNggvPrqYZuBytt6q mHjqLlp9Mxy/mEsT9Ltfi56sRwM1MY5ypoinrSNDMGC+1eXzfQBICIx4NMFA0J7TrS0Y BusLFK81Chc5ZhpJ/g/Rc7btonYWZAyzRfS1+rqq56fjdtDzvLvCnihNHmoiU5y/16Ds aaAOFrFtx7rml8O1gbYlZt5lW2xlN5TZ6WS/KKT1pEQmXZ6nofDb2YQ31Ixdbj0eTSJI Cxcw== X-Gm-Message-State: ABy/qLYT+zEbAffLDwecW9b8wWfVne71iOfgT9qqBeftGZ3ycv/W56UU ERFefc7Ow0hLwlXfx/5objaOxeg2J/DcTWLxvJs= X-Received: by 2002:a05:620a:44d4:b0:767:f1e8:d38c with SMTP id y20-20020a05620a44d400b00767f1e8d38cmr1630549qkp.15.1689975060929; Fri, 21 Jul 2023 14:31:00 -0700 (PDT) Received: from ltnt-nyc-580testdevin.livetimenet.com (pool-71-105-132-214.nycmny.fios.verizon.net. [71.105.132.214]) by smtp.gmail.com with ESMTPSA id z21-20020ae9c115000000b0076738337cd1sm1406333qki.1.2023.07.21.14.31.00 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 21 Jul 2023 14:31:00 -0700 (PDT) From: Devin Heitmueller X-Google-Original-From: Devin Heitmueller To: ffmpeg-devel@ffmpeg.org Date: Fri, 21 Jul 2023 17:30:55 -0400 Message-Id: <1689975057-22226-2-git-send-email-dheitmueller@ltnglobal.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1689975057-22226-1-git-send-email-dheitmueller@ltnglobal.com> References: <1689975057-22226-1-git-send-email-dheitmueller@ltnglobal.com> Subject: [FFmpeg-devel] [RFC][PATCH 1/3] v210enc: Refactor side data passthrough X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Devin Heitmueller MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: URa5Qv0HQJop As the number of side-data items we pass through increases, refactor the values into a table in the same manner as done in decode.c. In the future it might make sense to make this a shared function which can be reused by other encoders. But for now keep it local. Signed-off-by: Devin Heitmueller --- libavcodec/v210enc.c | 29 +++++++++++++++-------------- 1 file changed, 15 insertions(+), 14 deletions(-) diff --git a/libavcodec/v210enc.c b/libavcodec/v210enc.c index 2a30ed7..110d7eb 100644 --- a/libavcodec/v210enc.c +++ b/libavcodec/v210enc.c @@ -74,6 +74,13 @@ static int encode_frame(AVCodecContext *avctx, AVPacket *pkt, AVFrameSideData *side_data; int ret; uint8_t *dst; + static const struct { + enum AVPacketSideDataType packet; + enum AVFrameSideDataType frame; + } sd[] = { + { AV_PKT_DATA_A53_CC, AV_FRAME_DATA_A53_CC }, + { AV_PKT_DATA_AFD, AV_FRAME_DATA_AFD }, + }; ret = ff_get_encode_buffer(avctx, pkt, avctx->height * stride, 0); if (ret < 0) { @@ -87,20 +94,14 @@ static int encode_frame(AVCodecContext *avctx, AVPacket *pkt, else if(pic->format == AV_PIX_FMT_YUV422P) v210_enc_8(avctx, dst, pic); - side_data = av_frame_get_side_data(pic, AV_FRAME_DATA_A53_CC); - if (side_data && side_data->size) { - uint8_t *buf = av_packet_new_side_data(pkt, AV_PKT_DATA_A53_CC, side_data->size); - if (!buf) - return AVERROR(ENOMEM); - memcpy(buf, side_data->data, side_data->size); - } - - side_data = av_frame_get_side_data(pic, AV_FRAME_DATA_AFD); - if (side_data && side_data->size) { - uint8_t *buf = av_packet_new_side_data(pkt, AV_PKT_DATA_AFD, side_data->size); - if (!buf) - return AVERROR(ENOMEM); - memcpy(buf, side_data->data, side_data->size); + for (int i = 0; i < FF_ARRAY_ELEMS(sd); i++) { + side_data = av_frame_get_side_data(pic, sd[i].frame); + if (side_data && side_data->size) { + uint8_t *buf = av_packet_new_side_data(pkt, sd[i].packet, side_data->size); + if (!buf) + return AVERROR(ENOMEM); + memcpy(buf, side_data->data, side_data->size); + } } *got_packet = 1; From patchwork Fri Jul 21 21:30:56 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Devin Heitmueller X-Patchwork-Id: 42896 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:be1d:b0:130:ccc6:6c4b with SMTP id ge29csp504380pzb; Fri, 21 Jul 2023 14:31:31 -0700 (PDT) X-Google-Smtp-Source: APBJJlHoE26Af0Gp41cBmJap3Q9Dc0S+s/J+Ru5PYhhwlYa4WcvgxpX9ACvIdXbcNtWqzdgqgnHW X-Received: by 2002:a2e:3e02:0:b0:2b5:7dd9:74f5 with SMTP id l2-20020a2e3e02000000b002b57dd974f5mr2334051lja.21.1689975091179; Fri, 21 Jul 2023 14:31:31 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1689975091; cv=none; d=google.com; s=arc-20160816; b=ng6rD7pUu1g0NwW+krzr6NcU7kugCQeQ1+dgV0DtlrdQxMuul58+Rod8wVLwh9NALM ZgCa8cVt0hsYhfRT71qJSgmO+XplZLnYJHvwQtdQg6swr8wRh6CRRpCMzj4t4P65R3R8 O2H+txIfSaSLjbSH5+hD2+gSOhEZkCOrrgDXdnRrJyzsZPnpgckB/81cx6w0JF3f2ynD tzo8vvND7CnXxwKXzodvR7XlErmwyZ/0TDDvMuQx5EaX03xUDuHAeAG0P8kEYlIhq/2C IQ5BU3sn9t5sIqSnnozHQpM+0XZFutnnlEdKuNK1SEEH6NQxb1fYc56fWoAWV3scMVZ0 ocGg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:dkim-signature:delivered-to; bh=bojrZr0CYZ424OYvrnW52pzfCGJHSOeR6fMH4203EqI=; fh=DKzdJpyueDOox6ShoNYUi750QZs6R2naOS9+hcsrM90=; b=g3fumy2V1wV5/wlS87ocZEqDiLM9J6MQqcEv/YAOYJsiK03utROX++qRZjH+sdQBgb 9AvMGGETf6fQwzzkAJiNH8N2sTqLuEEOoJBXhCzp0Yr2mljjV7WFKv+2Jz9CnX5JYqoz g2XM5mGOJPJCSc/Pm/AZ3ctxTU7EcjOZrzpg10wTssz6RgRZdZMKsvRls+iT5aDWeb2x 5KE0yzpc3OnI2nCQsv+ZXbBdBOiqSZROkb+fCyq5NPcWQxotojrnJIB9sYaGF1lUwjfp +yP409euFhlc/nUZzzYMwG9/q6pe9B6tpWkx2Cu6nntCLZg35IpzkG1ABSwpA6BVh2uB tgLQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20221208.gappssmtp.com header.s=20221208 header.b=CEELUhc8; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id lc19-20020a170906f91300b00988c051a9edsi2610037ejb.651.2023.07.21.14.31.30; Fri, 21 Jul 2023 14:31:31 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20221208.gappssmtp.com header.s=20221208 header.b=CEELUhc8; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id B27E668C646; Sat, 22 Jul 2023 00:31:10 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-qv1-f43.google.com (mail-qv1-f43.google.com [209.85.219.43]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 6C51C68C60E for ; Sat, 22 Jul 2023 00:31:03 +0300 (EEST) Received: by mail-qv1-f43.google.com with SMTP id 6a1803df08f44-63cebd0a7c5so64886d6.3 for ; Fri, 21 Jul 2023 14:31:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ltnglobal-com.20221208.gappssmtp.com; s=20221208; t=1689975062; x=1690579862; h=references:in-reply-to:message-id:date:subject:cc:to:from:from:to :cc:subject:date:message-id:reply-to; bh=ClvG9Aac0TnLT89w6raOuGYuE2McKOkQbPi3zhCUd/E=; b=CEELUhc8Gb0d1E9MuMTTD/TyNjltE+nhK2ipV0jWRx+3EI7h9RLOfXm7eieuNRPqDR vjRoHUos8OpSl/QRpxa0EFgDN0wONIGhVCm0sJzIRFNloSmrS2vmyVD2CETCfehZItpP Cnt5JUxAXtGBv5ZAA3y/74+lPSciwGqjwAs/KFi1HQ8EFnZInz1tf7prPfPSfPgiXv7N m7Uoe+jc1PGK2bJQV5OfVYywDTr63fVVfZMxgZLlvWMS4XJ14UdnypXEDClMtn4sJKKh UZLVAmHD5R+IAvei8HmDCrKwD6fQSk7LE43oZU9q5WFt15r0cnkPPHJwLheHHfa7S/Cg IUdw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689975062; x=1690579862; h=references:in-reply-to:message-id:date:subject:cc:to:from :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=ClvG9Aac0TnLT89w6raOuGYuE2McKOkQbPi3zhCUd/E=; b=S3W+gcLkREnGlOAKl6y/rgsILhPR75ElKFJoVx5pHAHkEqA21639fQnzibskbDlZIX s/TUBtZ5sZzi5tBOVHJyAc0FNXiXoV5gdnIc6GRwB+4ByTGQUdKw7738zIlj0Kk+NVn5 2o+iWzrS+UBNoK1gWIpOh6hLZTi/oAP1+e/tdERsnqCWh4kyA+1L/7u69LVbd565nf6T vl7UPeKgF2SAOIUL2m+fLYD68iJIndJ7qx+Kn28EzcrGyj6ZFWaDISvfrM6waZ2a6r+/ OSwy6fWEl7hXkuhti8/ioSfPEm7OQH0yPhVj9ocL2O4kU9YmVDtYIw2BYNaUAMeLFxg5 JviA== X-Gm-Message-State: ABy/qLay4/P9MRbCxrhk3jD1D2nIXfsK2TjajaCVS6gIjCQNtkNPMWH3 3CpsBOLVwNmTwRewRFnrxcQExCc8h9G72hCGWEA= X-Received: by 2002:a0c:dc08:0:b0:63c:dcd5:6998 with SMTP id s8-20020a0cdc08000000b0063cdcd56998mr1226482qvk.60.1689975061857; Fri, 21 Jul 2023 14:31:01 -0700 (PDT) Received: from ltnt-nyc-580testdevin.livetimenet.com (pool-71-105-132-214.nycmny.fios.verizon.net. [71.105.132.214]) by smtp.gmail.com with ESMTPSA id z21-20020ae9c115000000b0076738337cd1sm1406333qki.1.2023.07.21.14.31.01 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 21 Jul 2023 14:31:01 -0700 (PDT) From: Devin Heitmueller X-Google-Original-From: Devin Heitmueller To: ffmpeg-devel@ffmpeg.org Date: Fri, 21 Jul 2023 17:30:56 -0400 Message-Id: <1689975057-22226-3-git-send-email-dheitmueller@ltnglobal.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1689975057-22226-1-git-send-email-dheitmueller@ltnglobal.com> References: <1689975057-22226-1-git-send-email-dheitmueller@ltnglobal.com> Subject: [FFmpeg-devel] [RFC][PATCH 2/3] v210enc: Add HDR metadata passthrough X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Devin Heitmueller MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: m5VUwbDhikIT In order to implement HDR output for decklink we need to expose the side data containing the mastering and light level info. Add the two additional types of side table to those which are passed through. Signed-off-by: Devin Heitmueller --- libavcodec/v210enc.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/libavcodec/v210enc.c b/libavcodec/v210enc.c index 110d7eb..25b1332 100644 --- a/libavcodec/v210enc.c +++ b/libavcodec/v210enc.c @@ -80,6 +80,8 @@ static int encode_frame(AVCodecContext *avctx, AVPacket *pkt, } sd[] = { { AV_PKT_DATA_A53_CC, AV_FRAME_DATA_A53_CC }, { AV_PKT_DATA_AFD, AV_FRAME_DATA_AFD }, + { AV_PKT_DATA_MASTERING_DISPLAY_METADATA, AV_FRAME_DATA_MASTERING_DISPLAY_METADATA }, + { AV_PKT_DATA_CONTENT_LIGHT_LEVEL, AV_FRAME_DATA_CONTENT_LIGHT_LEVEL }, }; ret = ff_get_encode_buffer(avctx, pkt, avctx->height * stride, 0); From patchwork Fri Jul 21 21:30:57 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Devin Heitmueller X-Patchwork-Id: 42897 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:be1d:b0:130:ccc6:6c4b with SMTP id ge29csp504454pzb; Fri, 21 Jul 2023 14:31:40 -0700 (PDT) X-Google-Smtp-Source: APBJJlEW+0rtmA7whnwY6a3pH+7kR1wvPh6x6RTfY3dt9tPTyj0XBU25G56BpVKsiiMsM+autF2G X-Received: by 2002:a17:906:11a:b0:991:fef4:bb9 with SMTP id 26-20020a170906011a00b00991fef40bb9mr2520162eje.58.1689975100353; Fri, 21 Jul 2023 14:31:40 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1689975100; cv=none; d=google.com; s=arc-20160816; b=PA10X1TON2CAQuuBf+zJXDJU+mHt1athpDuPLmNght8zFeDDxrUIgvXl5M2CVmSTri s5qyzU0T8iU6KD93vUvmsYFjFMostwxK2OHxrR15wluCGb57R1RyJLBvS7hOhZ6KqSia D0X6wEMfs8V93gSAprjSHz7THK8jD3Cu+Vpuc8wal53UVOnoDQfZFkF+5axxUjRIuWjK RZQ9tG6icVQyaPxS+zOq2ptKOBSX+jzJ/PjWz9NKvjuduCMWK/DzfJY97ndILYxQiUE1 PVfX6kXbHcO+XH0Uelmqnokr6UtQSYnaVUDBu7zt2W7tj/LvDAQBduYaHYSrGa6wBXKt FQ9w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:dkim-signature:delivered-to; bh=0fBnlJPNOQDzSwEZYSw0e4voIc5GbGgj6DtkYKY7o3w=; fh=DKzdJpyueDOox6ShoNYUi750QZs6R2naOS9+hcsrM90=; b=ndqr8UwyzQDbjAdg5q1MAAmqWlW9/p2h5CyVI26LdEmqeUBFlvczl0Kws3c96kW254 PbG96cyqxyHFp7GM72oerLca6Qmy/FvnoN7tJ/snzAMJHAjSXliSSlhWg/UCIrFtXRRg FTzSIR4u5m+tl+L3KjNXz0whQyHTjh9/NDrVYB1anBoTtfQ/p07PG9TPwgsQSiVMVnhi xwFYQoQToRz8IGBhfFFRsKoolHtty3qYRjOIdb9pKLwxi7qVStwEdBQcYrIFXzJILDX+ T5I6OEcvzkZmkB7vbBmNRk7N0dfV3zG02J61y9961fWs4jEMWKrMcEBCJh/nbivHuy/y fSCA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20221208.gappssmtp.com header.s=20221208 header.b=YXUS9B+f; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id mc21-20020a170906eb5500b00982a59bf247si2625140ejb.1027.2023.07.21.14.31.39; Fri, 21 Jul 2023 14:31:40 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@ltnglobal-com.20221208.gappssmtp.com header.s=20221208 header.b=YXUS9B+f; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id E6D9868C621; Sat, 22 Jul 2023 00:31:11 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-qk1-f180.google.com (mail-qk1-f180.google.com [209.85.222.180]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 2B57268BA8D for ; Sat, 22 Jul 2023 00:31:04 +0300 (EEST) Received: by mail-qk1-f180.google.com with SMTP id af79cd13be357-76728ae3162so199661085a.3 for ; Fri, 21 Jul 2023 14:31:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ltnglobal-com.20221208.gappssmtp.com; s=20221208; t=1689975063; x=1690579863; h=references:in-reply-to:message-id:date:subject:cc:to:from:from:to :cc:subject:date:message-id:reply-to; bh=k27HggGtyh/7SQlalRdyM0KZT2OoJTKokQsrvPp4dcE=; b=YXUS9B+fw33LEH61elljofnFozKcX5u1Xfgt7ORzxd2RoNxZtHrfpVIjpeSVMNkMqL 6QedwBYXM5HPkFg8uONEzhD39EAGd7Ww1OxgQtzS+HnPc1jlXNKOCwktmIxvhCTC8FXu paWrhZvpnMCBSnty2wY0l3Hgjtmw7Y3jXX+2LUg7fjrVue6ZbAWp39xxdp32VBQXXVqY e2OB81C74cPptH6b1yf7kT4p4uR1bmqk3XSDLqC/tSu9voH+f1TpF+a0H0amVKe+IXI/ kDKDGrEIOSnDs+kEyAOUogJpywZfvovY/FA1SUCVoelPlN+DxkTWvgT7VjuQHtFBpSC0 jZKA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689975063; x=1690579863; h=references:in-reply-to:message-id:date:subject:cc:to:from :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=k27HggGtyh/7SQlalRdyM0KZT2OoJTKokQsrvPp4dcE=; b=DOO5ukWW2VWyqKbhjE/3JdgnvcbC3gt9MnVoJEfPuJE3fEYvKYVubO/yfgv3IMxEB3 mib+wBPAwY4NSDvTGGhXXJo+U1RxleA5Zx0BrDyxNSxsbv+l2pBj8TsqsA+Cr4PP00cz +g63U243Y9aXaE+NxuK25lxv15rioCPm1Xm8BwWKiulWYxn56yysU/6Ylvcxga3oOHRH WcbMADPeAuun6NqdVJZAgPxlUn8SUEUEs1f2WSnSB9LZObWNAMjQyU6fz03ks/myF1LM 2hN6OCK145SqMcaO4lT9nHeHRQvmWDTCzJhpZurfwlQEeMe/IrTuqx6DuaZ3OjDWP70s Et2Q== X-Gm-Message-State: ABy/qLatIxypzjxHSQ1zwwxhHktEmLqei1FQnOGtm2J6deIXYCqnHRYl EIYw6Db1DzOBRSUixf5hUqxul9amCcdDTiQXNxw= X-Received: by 2002:a05:620a:4501:b0:767:d6f9:452b with SMTP id t1-20020a05620a450100b00767d6f9452bmr1523390qkp.12.1689975062775; Fri, 21 Jul 2023 14:31:02 -0700 (PDT) Received: from ltnt-nyc-580testdevin.livetimenet.com (pool-71-105-132-214.nycmny.fios.verizon.net. [71.105.132.214]) by smtp.gmail.com with ESMTPSA id z21-20020ae9c115000000b0076738337cd1sm1406333qki.1.2023.07.21.14.31.02 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 21 Jul 2023 14:31:02 -0700 (PDT) From: Devin Heitmueller X-Google-Original-From: Devin Heitmueller To: ffmpeg-devel@ffmpeg.org Date: Fri, 21 Jul 2023 17:30:57 -0400 Message-Id: <1689975057-22226-4-git-send-email-dheitmueller@ltnglobal.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1689975057-22226-1-git-send-email-dheitmueller@ltnglobal.com> References: <1689975057-22226-1-git-send-email-dheitmueller@ltnglobal.com> Subject: [FFmpeg-devel] [RFC][PATCH 3/3] decklink: Add support for output of HDR metadata X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Devin Heitmueller MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: fEmrWxfILCep Add HDR support to the decklink output for cards that support such functionality. This includes setting the EOTF, the colorspace, the mastering info, and the content light level info. Both the Payload Identification HANC data as well as the SMPTE ST 2108-1 VANC data are being set. Tested with in-house content as well as samples from 4kmedia.org. Testing was done with the Decklink 8K Pro and the Duo2 with 12.5.1 firmware, as well as with the Duo2 with 10.11.2 (before it supported HDR) to ensure there are no regressions. Signed-off-by: Devin Heitmueller --- libavdevice/decklink_common.cpp | 12 +++ libavdevice/decklink_common.h | 2 + libavdevice/decklink_enc.cpp | 204 ++++++++++++++++++++++++++++++++++++++-- 3 files changed, 211 insertions(+), 7 deletions(-) diff --git a/libavdevice/decklink_common.cpp b/libavdevice/decklink_common.cpp index 47de7ef..c1bcb82 100644 --- a/libavdevice/decklink_common.cpp +++ b/libavdevice/decklink_common.cpp @@ -251,6 +251,18 @@ int ff_decklink_set_configs(AVFormatContext *avctx, } } + DECKLINK_BOOL hdr_supported; + if (ctx->attr->GetFlag(BMDDeckLinkSupportsHDRMetadata, &hdr_supported) == S_OK) { + if (hdr_supported) + ctx->supports_hdr = 1; + } + + DECKLINK_BOOL colorspace_supported; + if (ctx->attr->GetFlag(BMDDeckLinkSupportsColorspaceMetadata, &colorspace_supported) == S_OK) { + if (colorspace_supported) + ctx->supports_colorspace = 1; + } + return 0; } diff --git a/libavdevice/decklink_common.h b/libavdevice/decklink_common.h index 34ab1b9..d50007f 100644 --- a/libavdevice/decklink_common.h +++ b/libavdevice/decklink_common.h @@ -109,6 +109,8 @@ struct decklink_ctx { int bmd_height; int bmd_field_dominance; int supports_vanc; + int supports_hdr; + int supports_colorspace; /* Capture buffer queue */ DecklinkPacketQueue queue; diff --git a/libavdevice/decklink_enc.cpp b/libavdevice/decklink_enc.cpp index ffd0ad9..92d6bbe 100644 --- a/libavdevice/decklink_enc.cpp +++ b/libavdevice/decklink_enc.cpp @@ -35,6 +35,7 @@ extern "C" { #include "libavcodec/bytestream.h" #include "libavutil/internal.h" #include "libavutil/imgutils.h" +#include "libavutil/mastering_display_metadata.h" #include "avdevice.h" } @@ -47,13 +48,13 @@ extern "C" { #endif /* DeckLink callback class declaration */ -class decklink_frame : public IDeckLinkVideoFrame +class decklink_frame : public IDeckLinkVideoFrame, public IDeckLinkVideoFrameMetadataExtensions { public: decklink_frame(struct decklink_ctx *ctx, AVFrame *avframe, AVCodecID codec_id, int height, int width) : _ctx(ctx), _avframe(avframe), _avpacket(NULL), _codec_id(codec_id), _ancillary(NULL), _height(height), _width(width), _refs(1) { } decklink_frame(struct decklink_ctx *ctx, AVPacket *avpacket, AVCodecID codec_id, int height, int width) : - _ctx(ctx), _avframe(NULL), _avpacket(avpacket), _codec_id(codec_id), _ancillary(NULL), _height(height), _width(width), _refs(1) { } + _ctx(ctx), _avframe(NULL), _avpacket(avpacket), _codec_id(codec_id), _ancillary(NULL), _height(height), _width(width), _colorspace(AVCOL_SPC_BT709), _eotf(AVCOL_TRC_BT709), hdr(NULL), lighting(NULL), _refs(1) { } virtual long STDMETHODCALLTYPE GetWidth (void) { return _width; } virtual long STDMETHODCALLTYPE GetHeight (void) { return _height; } virtual long STDMETHODCALLTYPE GetRowBytes (void) @@ -72,10 +73,14 @@ public: } virtual BMDFrameFlags STDMETHODCALLTYPE GetFlags (void) { - if (_codec_id == AV_CODEC_ID_WRAPPED_AVFRAME) - return _avframe->linesize[0] < 0 ? bmdFrameFlagFlipVertical : bmdFrameFlagDefault; - else - return bmdFrameFlagDefault; + if (_codec_id == AV_CODEC_ID_WRAPPED_AVFRAME) { + return _avframe->linesize[0] < 0 ? bmdFrameFlagFlipVertical : bmdFrameFlagDefault; + } else { + if (_ctx->supports_hdr && (hdr || lighting)) + return bmdFrameFlagDefault | bmdFrameContainsHDRMetadata; + else + return bmdFrameFlagDefault; + } } virtual HRESULT STDMETHODCALLTYPE GetBytes (void **buffer) @@ -110,7 +115,176 @@ public: _ancillary->AddRef(); return S_OK; } - virtual HRESULT STDMETHODCALLTYPE QueryInterface(REFIID iid, LPVOID *ppv) { return E_NOINTERFACE; } + + virtual HRESULT STDMETHODCALLTYPE SetMetadata(enum AVColorSpace colorspace, enum AVColorTransferCharacteristic eotf) + { + _colorspace = colorspace; + _eotf = eotf; + return S_OK; + } + + // IDeckLinkVideoFrameMetadataExtensions interface + virtual HRESULT GetInt(BMDDeckLinkFrameMetadataID metadataID, int64_t* value) + { + HRESULT result = S_OK; + + switch (metadataID) { + case bmdDeckLinkFrameMetadataHDRElectroOpticalTransferFunc: + /* See CTA-861-G Sec 6.9 Dynamic Range and Mastering */ + + switch(_eotf) { + case AVCOL_TRC_SMPTEST2084: + /* PQ */ + *value = 2; + break; + case AVCOL_TRC_ARIB_STD_B67: + /* Also known as "HLG" */ + *value = 3; + break; + case AVCOL_TRC_SMPTE170M: + case AVCOL_TRC_SMPTE240M: + case AVCOL_TRC_BT709: + default: + /* SDR */ + *value = 0; + break; + } + break; + + case bmdDeckLinkFrameMetadataColorspace: + if (!_ctx->supports_colorspace) { + result = E_NOTIMPL; + break; + } + switch(_colorspace) { + case AVCOL_SPC_BT470BG: + case AVCOL_SPC_SMPTE170M: + case AVCOL_SPC_SMPTE240M: + *value = bmdColorspaceRec601; + break; + case AVCOL_SPC_BT2020_CL: + case AVCOL_SPC_BT2020_NCL: + *value = bmdColorspaceRec2020; + break; + case AVCOL_SPC_BT709: + default: + *value = bmdColorspaceRec709; + break; + } + break; + default: + result = E_INVALIDARG; + } + + return result; + } + virtual HRESULT GetFloat(BMDDeckLinkFrameMetadataID metadataID, double* value) + { + *value = 0; + + switch (metadataID) { + case bmdDeckLinkFrameMetadataHDRDisplayPrimariesRedX: + if (hdr && hdr->has_primaries) + *value = av_q2d(hdr->display_primaries[0][0]); + break; + case bmdDeckLinkFrameMetadataHDRDisplayPrimariesRedY: + if (hdr && hdr->has_primaries) + *value = av_q2d(hdr->display_primaries[0][1]); + break; + case bmdDeckLinkFrameMetadataHDRDisplayPrimariesGreenX: + if (hdr && hdr->has_primaries) + *value = av_q2d(hdr->display_primaries[1][0]); + break; + case bmdDeckLinkFrameMetadataHDRDisplayPrimariesGreenY: + if (hdr && hdr->has_primaries) + *value = av_q2d(hdr->display_primaries[1][1]); + break; + case bmdDeckLinkFrameMetadataHDRDisplayPrimariesBlueX: + if (hdr && hdr->has_primaries) + *value = av_q2d(hdr->display_primaries[2][0]); + break; + case bmdDeckLinkFrameMetadataHDRDisplayPrimariesBlueY: + if (hdr && hdr->has_primaries) + *value = av_q2d(hdr->display_primaries[2][1]); + break; + case bmdDeckLinkFrameMetadataHDRWhitePointX: + if (hdr && hdr->has_primaries) + *value = av_q2d(hdr->white_point[0]); + break; + case bmdDeckLinkFrameMetadataHDRWhitePointY: + if (hdr && hdr->has_primaries) + *value = av_q2d(hdr->white_point[1]); + break; + case bmdDeckLinkFrameMetadataHDRMaxDisplayMasteringLuminance: + if (hdr && hdr->has_luminance) + *value = av_q2d(hdr->max_luminance); + break; + case bmdDeckLinkFrameMetadataHDRMinDisplayMasteringLuminance: + if (hdr && hdr->has_luminance) + *value = av_q2d(hdr->min_luminance); + break; + case bmdDeckLinkFrameMetadataHDRMaximumContentLightLevel: + if (lighting) + *value = (float) lighting->MaxCLL; + else + *value = 0; + break; + case bmdDeckLinkFrameMetadataHDRMaximumFrameAverageLightLevel: + if (lighting) + *value = (float) lighting->MaxFALL; + else + *value = 0; + break; + default: + return E_INVALIDARG; + } + + return S_OK; + } + + virtual HRESULT GetFlag(BMDDeckLinkFrameMetadataID metadataID, bool* value) + { + *value = false; + return E_INVALIDARG; + } + virtual HRESULT GetString(BMDDeckLinkFrameMetadataID metadataID, const char** value) + { + *value = nullptr; + return E_INVALIDARG; + } + virtual HRESULT GetBytes(BMDDeckLinkFrameMetadataID metadataID, void* buffer, uint32_t* bufferSize) + { + *bufferSize = 0; + return E_INVALIDARG; + } + + virtual HRESULT STDMETHODCALLTYPE QueryInterface(REFIID iid, LPVOID *ppv) + { + CFUUIDBytes iunknown; + HRESULT result = S_OK; + + if (!ppv) + return E_INVALIDARG; + + *ppv = NULL; + + iunknown = CFUUIDGetUUIDBytes(IUnknownUUID); + if (memcmp(&iid, &iunknown, sizeof(REFIID)) == 0) { + *ppv = this; + AddRef(); + } else if (memcmp(&iid, &IID_IDeckLinkVideoFrame, sizeof(REFIID)) == 0) { + *ppv = static_cast(this); + AddRef(); + } else if (memcmp(&iid, &IID_IDeckLinkVideoFrameMetadataExtensions, sizeof(REFIID)) == 0) { + *ppv = static_cast(this); + AddRef(); + } else { + result = E_NOINTERFACE; + } + + return result; + } + virtual ULONG STDMETHODCALLTYPE AddRef(void) { return ++_refs; } virtual ULONG STDMETHODCALLTYPE Release(void) { @@ -132,6 +306,10 @@ public: IDeckLinkVideoFrameAncillary *_ancillary; int _height; int _width; + enum AVColorSpace _colorspace; + enum AVColorTransferCharacteristic _eotf; + const AVMasteringDisplayMetadata *hdr; + const AVContentLightMetadata *lighting; private: std::atomic _refs; @@ -726,6 +904,18 @@ static int decklink_write_video_packet(AVFormatContext *avctx, AVPacket *pkt) return AVERROR(EIO); } + /* Set frame metadata properties */ + size_t size; + const AVMasteringDisplayMetadata *hdr = (const AVMasteringDisplayMetadata *) av_packet_get_side_data(pkt, AV_PKT_DATA_MASTERING_DISPLAY_METADATA, &size); + if (hdr && size > 0) + frame->hdr = hdr; + + const AVContentLightMetadata *lighting = (const AVContentLightMetadata *) av_packet_get_side_data(pkt, AV_PKT_DATA_CONTENT_LIGHT_LEVEL, &size); + if (hdr && size > 0) + frame->lighting = lighting; + + frame->SetMetadata(st->codecpar->color_space, st->codecpar->color_trc); + /* Always keep at most one second of frames buffered. */ pthread_mutex_lock(&ctx->mutex); while (ctx->frames_buffer_available_spots == 0) {