From patchwork Sun Feb 21 19:51:08 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Thompson X-Patchwork-Id: 25863 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id DB6EB449DB5 for ; Sun, 21 Feb 2021 21:53:36 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id B004B689246; Sun, 21 Feb 2021 21:53:36 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wr1-f41.google.com (mail-wr1-f41.google.com [209.85.221.41]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id B0126688050 for ; Sun, 21 Feb 2021 21:53:30 +0200 (EET) Received: by mail-wr1-f41.google.com with SMTP id h98so12093030wrh.11 for ; Sun, 21 Feb 2021 11:53:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jkqxz-net.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:mime-version :content-transfer-encoding; bh=51jsgz9/gqXvtrugxWgaEf56PaUtND7XaaDIJ13z28g=; b=tpLnLyvlEg5iYQA17u67beKJ0+QxYEhLQRjLQ7Dn1GyBg/VqoZak7LYzmDZKO9nxCe acwoEytOOS8UghtKSK7rFRyy3i1fflHfzwmqVPwXf55s2rpkUPYEgQG9WzK3Atjd44uS 0Fcyv8Q4iLnByBq67WXW3Y5Sg9VpJ5OmUzB6xG8B2pJ3JphInA1AJlVHfLtLB/qepP5I Ww+BEcbeZAT25CcyeojwKLip+qRF3th9FVreaFZuqna9a6x2i2dccubY6sfK+nT7F941 gwsj7On0s0hFQMQ3iv/wske2sUTrPrrmNzvimAsSpDlA7F9zA0Y5gLS4OFo+ba4RcSdj 0WTA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:mime-version :content-transfer-encoding; bh=51jsgz9/gqXvtrugxWgaEf56PaUtND7XaaDIJ13z28g=; b=BRs1jq1eA3mW1NUYlz8QX1lMFRtk8olVC8Ihmft8uBcSpTSrvQbJaqkpH+r9m2vV3m hGRhnreprZqKXYvKueqcqaaymX3vW2DLrDgjAiNIFt9WfyZkrYsbnS24dWH8S9/Zk1pM fm9acu8xtEPPFLdptpdgIJCtKZWhaQ1Dl6Y8lVkwDAkP424YieQJfgrxvv8U44j+7d5Q zJfPJgHLnnoGFcty6Gg/rZQ+qAp8OZlkmZhWsv/Pv18mUHsJMxgccMqetkR6/zciMRdm dEDNO+5gAkLGFWd5SnhdNWrxR5X79sxcAG3kJ/CCNCixcGcZoIDrAuXwWB4EQQSKLDKG S1YQ== X-Gm-Message-State: AOAM5336qSSocxDDGSS9IzLFCB2yqfNG4LQw7I3/8Dv43tkDcHE3+DIx w2jHzNKQ8wtTp5T6IE4OnMLgsr39lfTYOQ== X-Google-Smtp-Source: ABdhPJxHZ9ub2q3/wAArlJvh4dTfq2pprru2OOnWa0IJFGn7AmO8MIYuIHsfPsL/EgSVKCOfbLPx4A== X-Received: by 2002:adf:fa4c:: with SMTP id y12mr6375783wrr.427.1613937210279; Sun, 21 Feb 2021 11:53:30 -0800 (PST) Received: from localhost.localdomain (cpc91226-cmbg18-2-0-cust7.5-4.cable.virginm.net. [82.0.29.8]) by smtp.gmail.com with ESMTPSA id z18sm1780372wrs.19.2021.02.21.11.53.29 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 21 Feb 2021 11:53:29 -0800 (PST) From: Mark Thompson To: ffmpeg-devel@ffmpeg.org Date: Sun, 21 Feb 2021 19:51:08 +0000 Message-Id: <20210221195125.1901683-1-sw@jkqxz.net> X-Mailer: git-send-email 2.30.0 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 01/18] cbs_sei: Delete SEI NAL units containing no messages X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" When we remove the last SEI message from a NAL unit, the unit itself should also be deleted. --- libavcodec/cbs_sei.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/libavcodec/cbs_sei.c b/libavcodec/cbs_sei.c index c49830ad77..14f1cae506 100644 --- a/libavcodec/cbs_sei.c +++ b/libavcodec/cbs_sei.c @@ -353,7 +353,7 @@ void ff_cbs_sei_delete_message_type(CodedBitstreamContext *ctx, { int err, i, j; - for (i = 0; i < au->nb_units; i++) { + for (i = au->nb_units - 1; i >= 0; i--) { CodedBitstreamUnit *unit = &au->units[i]; SEIRawMessageList *list; @@ -365,5 +365,10 @@ void ff_cbs_sei_delete_message_type(CodedBitstreamContext *ctx, if (list->messages[j].payload_type == payload_type) cbs_sei_delete_message(list, j); } + + if (list->nb_messages == 0) { + // The SEI NAL unit is now empty, so get rid of it. + ff_cbs_delete_unit(au, i); + } } } From patchwork Sun Feb 21 19:51:09 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Thompson X-Patchwork-Id: 25864 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id E400B449DB5 for ; Sun, 21 Feb 2021 21:53:39 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id C8F7068A558; Sun, 21 Feb 2021 21:53:39 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wm1-f43.google.com (mail-wm1-f43.google.com [209.85.128.43]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 7ABD3689246 for ; Sun, 21 Feb 2021 21:53:31 +0200 (EET) Received: by mail-wm1-f43.google.com with SMTP id m1so12509369wml.2 for ; Sun, 21 Feb 2021 11:53:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jkqxz-net.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=djO7NnHXIWlj2S5dVzckVj/xE1cyOrM0VJLt1K3O+90=; b=kkU+ce4Gks55LlH0bNkVNOpmrAKIrySpx0miJ+B6MWDA39bCOkux5ZwoLTC1WHu8KQ pQobwoJk0UFHELAdFkN70cSJiD7USEeewDWfZIhKDNYwYByLtRaU7oNzHD+XAwsUyYd2 e9pmnTl6A48RxMgcemErUuF3N1JRqeORuNIwCeYU2vkal8cIA/mkvfvDQWPDPj91shQ0 wepd3bMBqWO0TzfhSnwt4s1pFpteusQm9ZWt2hpIEBcs/0T0Xr6N1Yn3NNb/XrU34IF6 ALx46NuKCzt35CKIh/FVQCJIgkDE91WQ3lT/FJahVs59mNByBH7MI1HWkcsLsdpFuqWu f5ag== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=djO7NnHXIWlj2S5dVzckVj/xE1cyOrM0VJLt1K3O+90=; b=OWndJwW00WEk3DZi+taqaPC3Vt4KynoJwmRuLWUZr9kdmx1Ku3vJ0N9YKf45dfAmei 2Nt1qecCqLgWSyONOGmVNsQe0OiAkNVqw6wOTgygGHbQ8c2BdnkIm76GTffw/AlnONui lFyBN1hB7kfC1MlmNxIt7GYCetCeEVErgUJisuYifAl7g9ZiuQe5DvEUGrp3vVilhTFH 5RETfY+CyBKssYPThFl6aIXyAVeKVFIQrAruBE0FqLNBX1Q9jfqRGGqiAusWALA4oXjS 7XNqFqLWKsz4xxom7dTfix81VpYVZzj+cTT/PaWZlD6nrW0qoXzR9mv5t/xXyk7SaOgZ OmDg== X-Gm-Message-State: AOAM532UNglBgZ9QLCWEkeJVBRYnuZ3rl8dyPI23IynJEStpuAfKLRh6 8jV74kWh0MndiKIvO/ZtE2Vb3Ur62f1bGg== X-Google-Smtp-Source: ABdhPJxCGQwnNa0h1oNfdplvXmexQjuL5LDzUCZV83LD9yn2kpYRvRamx/dw5RiR2Y6jKcPsSGYdyw== X-Received: by 2002:a7b:c842:: with SMTP id c2mr17248371wml.100.1613937210941; Sun, 21 Feb 2021 11:53:30 -0800 (PST) Received: from localhost.localdomain (cpc91226-cmbg18-2-0-cust7.5-4.cable.virginm.net. [82.0.29.8]) by smtp.gmail.com with ESMTPSA id z18sm1780372wrs.19.2021.02.21.11.53.30 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 21 Feb 2021 11:53:30 -0800 (PST) From: Mark Thompson To: ffmpeg-devel@ffmpeg.org Date: Sun, 21 Feb 2021 19:51:09 +0000 Message-Id: <20210221195125.1901683-2-sw@jkqxz.net> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20210221195125.1901683-1-sw@jkqxz.net> References: <20210221195125.1901683-1-sw@jkqxz.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 02/18] cbs_h264: Add support for frame packing arrangement SEI messages X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" --- libavcodec/cbs_h264.h | 23 ++++++++++++++++ libavcodec/cbs_h2645.c | 6 +++++ libavcodec/cbs_h264_syntax_template.c | 39 +++++++++++++++++++++++++++ 3 files changed, 68 insertions(+) diff --git a/libavcodec/cbs_h264.h b/libavcodec/cbs_h264.h index 9eb97eae24..1466ed12fa 100644 --- a/libavcodec/cbs_h264.h +++ b/libavcodec/cbs_h264.h @@ -282,6 +282,29 @@ typedef struct H264RawSEIRecoveryPoint { uint8_t changing_slice_group_idc; } H264RawSEIRecoveryPoint; +typedef struct H264RawSEIFramePackingArrangement { + uint32_t frame_packing_arrangement_id; + uint8_t frame_packing_arrangement_cancel_flag; + + uint8_t frame_packing_arrangement_type; + uint8_t quincunx_sampling_flag; + uint8_t content_interpretation_type; + uint8_t spatial_flipping_flag; + uint8_t frame0_flipped_flag; + uint8_t field_views_flag; + uint8_t current_frame_is_frame0_flag; + uint8_t frame0_self_contained_flag; + uint8_t frame1_self_contained_flag; + uint8_t frame0_grid_position_x; + uint8_t frame0_grid_position_y; + uint8_t frame1_grid_position_x; + uint8_t frame1_grid_position_y; + uint8_t frame_packing_arrangement_reserved_byte; + uint16_t frame_packing_arrangement_repetition_period; + + uint8_t frame_packing_arrangement_extension_flag; +} H264RawSEIFramePackingArrangement; + typedef struct H264RawSEIDisplayOrientation { uint8_t display_orientation_cancel_flag; uint8_t hor_flip; diff --git a/libavcodec/cbs_h2645.c b/libavcodec/cbs_h2645.c index 6005d46e0d..0c591871d4 100644 --- a/libavcodec/cbs_h2645.c +++ b/libavcodec/cbs_h2645.c @@ -1570,6 +1570,12 @@ static const SEIMessageTypeDescriptor cbs_sei_h264_types[] = { sizeof(H264RawSEIRecoveryPoint), SEI_MESSAGE_RW(h264, sei_recovery_point), }, + { + SEI_TYPE_FRAME_PACKING_ARRANGEMENT, + 1, 0, + sizeof(H264RawSEIFramePackingArrangement), + SEI_MESSAGE_RW(h264, sei_frame_packing_arrangement), + }, { SEI_TYPE_DISPLAY_ORIENTATION, 1, 0, diff --git a/libavcodec/cbs_h264_syntax_template.c b/libavcodec/cbs_h264_syntax_template.c index 9587f33985..e03d41e47a 100644 --- a/libavcodec/cbs_h264_syntax_template.c +++ b/libavcodec/cbs_h264_syntax_template.c @@ -719,6 +719,45 @@ static int FUNC(sei_recovery_point)(CodedBitstreamContext *ctx, RWContext *rw, return 0; } + + +static int FUNC(sei_frame_packing_arrangement) + (CodedBitstreamContext *ctx, RWContext *rw, + H264RawSEIFramePackingArrangement *current, SEIMessageState *sei) +{ + int err; + + HEADER("Frame Packing Arrangement"); + + ue(frame_packing_arrangement_id, 0, UINT32_MAX - 1); + flag(frame_packing_arrangement_cancel_flag); + + if (!current->frame_packing_arrangement_cancel_flag) { + u(7, frame_packing_arrangement_type, 0, 7); + flag(quincunx_sampling_flag); + u(6, content_interpretation_type, 0, 2); + flag(spatial_flipping_flag); + flag(frame0_flipped_flag); + flag(field_views_flag); + flag(current_frame_is_frame0_flag); + flag(frame0_self_contained_flag); + flag(frame1_self_contained_flag); + if (!current->quincunx_sampling_flag && + current->frame_packing_arrangement_type != 5) { + ub(4, frame0_grid_position_x); + ub(4, frame0_grid_position_y); + ub(4, frame1_grid_position_x); + ub(4, frame1_grid_position_y); + } + u(8, frame_packing_arrangement_reserved_byte, 0, 0); + ue(frame_packing_arrangement_repetition_period, 0, 16384); + } + + flag(frame_packing_arrangement_extension_flag); + + return 0; +} + static int FUNC(sei_display_orientation)(CodedBitstreamContext *ctx, RWContext *rw, H264RawSEIDisplayOrientation *current, SEIMessageState *sei) From patchwork Sun Feb 21 19:51:10 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Thompson X-Patchwork-Id: 25865 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id CB478449DB5 for ; Sun, 21 Feb 2021 21:53:40 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id B504468A77B; Sun, 21 Feb 2021 21:53:40 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wr1-f45.google.com (mail-wr1-f45.google.com [209.85.221.45]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 4516668A497 for ; Sun, 21 Feb 2021 21:53:32 +0200 (EET) Received: by mail-wr1-f45.google.com with SMTP id v14so16892530wro.7 for ; Sun, 21 Feb 2021 11:53:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jkqxz-net.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=bshsMVXlpk0qGWu/nTlPn+RaKqs7gJpCDb8QeYN4IRE=; b=xEVHD4g8mTFFA774bhIkmD6jLZlDIbQfPNl+VafgjuTKL/Mx2mKuVKFBBo/Ve+zYqC jYPQ4WqeH4nC5j429APQprorgNjObMQtSGlLn22YuXzm2PuUD/7LV6icPZcgkau/z7lh 1JNjgG2X7Qtvugw8RXFRQDeSys8V6hGM8PAwf5V9z4XmRfYvi7hj8kVJq+GP5NSx4ROD 5P26MzE7Cx2H20Rf40yJMRieG9Iv2caIwAbx1qTLhRcuh4XCqkP2XlPfOKJv6e+gtABx +YfqrRT62gbeMfxbmanj8R32mULR7tbMjIMQR1hh/rMPr/z/JPKui96612JB7CDYwZ0z RrXQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=bshsMVXlpk0qGWu/nTlPn+RaKqs7gJpCDb8QeYN4IRE=; b=hr5+A4OFX7AfeBEX4RvjvwnphpMNuWHvrFILwEjb+w/gnt9XqOba1//ZCMf0lu4gTI 304695pd2GygxepoTCe/p+ig/BkB5s6Rl8immCGmAZUuvzn0qNKtZxK0/1j59fcbOUij uRNoUOgyGeNYQ8kJmeuYAKpYZWaFO+afI/581qdJLrZIlW+rbPCtVuuxqQxqLFS4LUuj cJMHdlXfT8SUA3RbhZpMkWEx3vpfuVZlVHFG40KyJg0CENcxyjUdOY5G57C9vv/sDyR0 UA0rkHJog0beL/TsBhSfOVnA/J+q5pvEk59X30Oqn34rLgeZRGlfy60FprpT3kc2m2QQ Ph5g== X-Gm-Message-State: AOAM533Y83bPZoB3uRGXUg4f79ASn5+qWC0dAjdtKFmeeYwLQuJoknod iwYKvCzqSgoVjbpZlMhI6rag7t/PHp6fdQ== X-Google-Smtp-Source: ABdhPJwNDNDXD+5XsD/DtEtML/tYW+EwNHHGwmogx+bUO3fbjEFxUdk/PKWrFhxMLfuHuNNfGTKrbg== X-Received: by 2002:a5d:63ce:: with SMTP id c14mr5096454wrw.15.1613937211708; Sun, 21 Feb 2021 11:53:31 -0800 (PST) Received: from localhost.localdomain (cpc91226-cmbg18-2-0-cust7.5-4.cable.virginm.net. [82.0.29.8]) by smtp.gmail.com with ESMTPSA id z18sm1780372wrs.19.2021.02.21.11.53.31 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 21 Feb 2021 11:53:31 -0800 (PST) From: Mark Thompson To: ffmpeg-devel@ffmpeg.org Date: Sun, 21 Feb 2021 19:51:10 +0000 Message-Id: <20210221195125.1901683-3-sw@jkqxz.net> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20210221195125.1901683-1-sw@jkqxz.net> References: <20210221195125.1901683-1-sw@jkqxz.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 03/18] cbs: Add interface for manipulating metadata in fragments X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" --- libavcodec/Makefile | 2 +- libavcodec/cbs_internal.h | 17 ++++++ libavcodec/cbs_metadata.c | 110 ++++++++++++++++++++++++++++++++++++++ libavcodec/cbs_metadata.h | 94 ++++++++++++++++++++++++++++++++ 4 files changed, 222 insertions(+), 1 deletion(-) create mode 100644 libavcodec/cbs_metadata.c create mode 100644 libavcodec/cbs_metadata.h diff --git a/libavcodec/Makefile b/libavcodec/Makefile index 6ec1d360ac..43a54caba8 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -69,7 +69,7 @@ OBJS-$(CONFIG_AUDIODSP) += audiodsp.o OBJS-$(CONFIG_BLOCKDSP) += blockdsp.o OBJS-$(CONFIG_BSWAPDSP) += bswapdsp.o OBJS-$(CONFIG_CABAC) += cabac.o -OBJS-$(CONFIG_CBS) += cbs.o cbs_bsf.o +OBJS-$(CONFIG_CBS) += cbs.o cbs_bsf.o cbs_metadata.o OBJS-$(CONFIG_CBS_AV1) += cbs_av1.o OBJS-$(CONFIG_CBS_H264) += cbs_h2645.o cbs_sei.o h2645_parse.o OBJS-$(CONFIG_CBS_H265) += cbs_h2645.o cbs_sei.o h2645_parse.o diff --git a/libavcodec/cbs_internal.h b/libavcodec/cbs_internal.h index a392880036..6aa2c62bc2 100644 --- a/libavcodec/cbs_internal.h +++ b/libavcodec/cbs_internal.h @@ -83,6 +83,8 @@ typedef const struct CodedBitstreamUnitTypeDescriptor { int (*content_clone)(AVBufferRef **ref, CodedBitstreamUnit *unit); } CodedBitstreamUnitTypeDescriptor; +enum CBSMetadataType; + typedef struct CodedBitstreamType { enum AVCodecID codec_id; @@ -128,6 +130,21 @@ typedef struct CodedBitstreamType { // Free the codec internal state. void (*close)(CodedBitstreamContext *ctx); + + // Insert metadata to fragment. + int (*insert_metadata)(CodedBitstreamContext *ctx, + CodedBitstreamFragment *frag, + enum CBSMetadataType type, const void *data); + + // Remove metadata from fragment. + int (*remove_metadata)(CodedBitstreamContext *ctx, + CodedBitstreamFragment *frag, + enum CBSMetadataType type); + + // Extract metadata from fragment. + int (*extract_metadata)(CodedBitstreamContext *ctx, + CodedBitstreamFragment *frag, + enum CBSMetadataType type, void *data); } CodedBitstreamType; diff --git a/libavcodec/cbs_metadata.c b/libavcodec/cbs_metadata.c new file mode 100644 index 0000000000..4dc0243998 --- /dev/null +++ b/libavcodec/cbs_metadata.c @@ -0,0 +1,110 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "libavutil/film_grain_params.h" +#include "libavutil/mastering_display_metadata.h" +#include "libavutil/stereo3d.h" + +#include "cbs.h" +#include "cbs_internal.h" +#include "cbs_metadata.h" + +static const CBSMetadataTypeDescriptor cbs_metadata_types[] = { + { + CBS_METADATA_MASTERING_DISPLAY, + "Mastering Display", + 1, AV_FRAME_DATA_MASTERING_DISPLAY_METADATA, + 1, AV_PKT_DATA_MASTERING_DISPLAY_METADATA, + }, + { + CBS_METADATA_CONTENT_LIGHT_LEVEL, + "Content Light Level", + 1, AV_FRAME_DATA_CONTENT_LIGHT_LEVEL, + 1, AV_PKT_DATA_CONTENT_LIGHT_LEVEL, + }, + { + CBS_METADATA_DISPLAY_MATRIX, + "Display Matrix", + 1, AV_FRAME_DATA_DISPLAYMATRIX, + 1, AV_PKT_DATA_DISPLAYMATRIX, + }, + { + CBS_METADATA_STEREO3D, + "Stereo 3D", + 1, AV_FRAME_DATA_STEREO3D, + 1, AV_PKT_DATA_STEREO3D, + }, +}; + +const CBSMetadataTypeDescriptor *ff_cbs_metadata_find_type(enum CBSMetadataType type) +{ + for (int i = 0; i < FF_ARRAY_ELEMS(cbs_metadata_types); i++) { + if (cbs_metadata_types[i].type == type) + return &cbs_metadata_types[i]; + } + return NULL; +} + +void *ff_cbs_alloc_metadata(enum CBSMetadataType type, size_t *size) +{ + switch (type) { + case CBS_METADATA_MASTERING_DISPLAY: + if (size) + *size = sizeof(AVMasteringDisplayMetadata); + return av_mastering_display_metadata_alloc(); + case CBS_METADATA_CONTENT_LIGHT_LEVEL: + return av_content_light_metadata_alloc(size); + case CBS_METADATA_DISPLAY_MATRIX: + if (size) + *size = 9 * sizeof(int32_t); + return av_mallocz(9 * sizeof(int32_t)); + case CBS_METADATA_STEREO3D: + if (size) + *size = sizeof(AVStereo3D); + return av_stereo3d_alloc(); + default: + return NULL; + } +} + +int ff_cbs_insert_metadata(CodedBitstreamContext *ctx, + CodedBitstreamFragment *frag, + enum CBSMetadataType type, const void *data) +{ + if (!ctx->codec->insert_metadata) + return AVERROR(ENOSYS); + return ctx->codec->insert_metadata(ctx, frag, type, data); +} + +int ff_cbs_remove_metadata(CodedBitstreamContext *ctx, + CodedBitstreamFragment *frag, + enum CBSMetadataType type) +{ + if (!ctx->codec->remove_metadata) + return AVERROR(ENOSYS); + return ctx->codec->remove_metadata(ctx, frag, type); +} + +int ff_cbs_extract_metadata(CodedBitstreamContext *ctx, + CodedBitstreamFragment *frag, + enum CBSMetadataType type, void *data) +{ + if (!ctx->codec->extract_metadata) + return AVERROR(ENOSYS); + return ctx->codec->extract_metadata(ctx, frag, type, data); +} diff --git a/libavcodec/cbs_metadata.h b/libavcodec/cbs_metadata.h new file mode 100644 index 0000000000..79ba6f4e26 --- /dev/null +++ b/libavcodec/cbs_metadata.h @@ -0,0 +1,94 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#ifndef AVCODEC_CBS_METADATA_H +#define AVCODEC_CBS_METADATA_H + +#include "libavutil/frame.h" + +#include "cbs.h" +#include "packet.h" + + +enum CBSMetadataType { + // List sentinel element. + CBS_METADATA_NONE, + // AVMasteringDisplayMetadata. + CBS_METADATA_MASTERING_DISPLAY, + // AVContentLightMetadata. + CBS_METADATA_CONTENT_LIGHT_LEVEL, + // int32_t matrix[9]. + CBS_METADATA_DISPLAY_MATRIX, + // AVStereo3D. + CBS_METADATA_STEREO3D, +}; + +typedef struct CBSMetadataTypeDescriptor { + enum CBSMetadataType type; + const char *name; + + // Matching frame and packet side data types, if available. + int has_frame_side_data; + enum AVFrameSideDataType frame_side_data_type; + int has_packet_side_data; + enum AVPacketSideDataType packet_side_data_type; +} CBSMetadataTypeDescriptor; + +/** + * Retrieve the descriptor for the given metadata type. + */ +const CBSMetadataTypeDescriptor *ff_cbs_metadata_find_type(enum CBSMetadataType type); + +/** + * Allocates an object for the given metadata type. + * + * If not null, size is written with the size of the object returned. + */ +void *ff_cbs_alloc_metadata(enum CBSMetadataType type, size_t *size); + +/** + * Insert metadata into a fragment. + * + * type is a CBS_METADATA_* value. data must be a pointer to a filled + * structure matching the specified type. + */ +int ff_cbs_insert_metadata(CodedBitstreamContext *ctx, + CodedBitstreamFragment *frag, + enum CBSMetadataType type, const void *data); + +/** + * Remove metadata from a fragment. + * + * Remove all metadata of the given CBS_METADATA_* type. + */ +int ff_cbs_remove_metadata(CodedBitstreamContext *ctx, + CodedBitstreamFragment *frag, + enum CBSMetadataType type); + +/** + * Extract metadata from a fragment. + * + * type is a CBS_METADATA_* value. data must be a pointer to an empty + * structure matching the specified type, which will be filled on + * success. + */ +int ff_cbs_extract_metadata(CodedBitstreamContext *ctx, + CodedBitstreamFragment *frag, + enum CBSMetadataType type, void *data); + +#endif /* AVCODEC_CBS_METADATA_H */ From patchwork Sun Feb 21 19:51:11 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Thompson X-Patchwork-Id: 25866 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id AF726449DB5 for ; Sun, 21 Feb 2021 21:53:42 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 9899968A87F; Sun, 21 Feb 2021 21:53:42 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wr1-f45.google.com (mail-wr1-f45.google.com [209.85.221.45]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 20A01688050 for ; Sun, 21 Feb 2021 21:53:33 +0200 (EET) Received: by mail-wr1-f45.google.com with SMTP id v15so16911762wrx.4 for ; Sun, 21 Feb 2021 11:53:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jkqxz-net.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=XIaGDBhNrSWCctxGM2T0020/FVJGHMEV90lsGnxNKwM=; b=dkeqzDUNH5sabX4fZqYp81wLMw0dsD2bnQuNNqw+nn4JVn5TECzJfb0vlhbBKqFAL4 7uXejzquVLLpMbfglBjQzFL7nyteeYT2JhoG+9/U9Xsz0NL/aDRrU1FBAIWPB+YKgiiF TdL4UDNX7lUPTKoPSHzCYtByJCvNXIqg+ccd9b9Gy+yAsWSHGdjcMJHO5Lvw8NySzUik sG7n9mFRpXLk+7fCcbLJylsPWR5HVrsYpwOFR3/Q5zbAVlFLFnfU+N9i1Jb/wgw2kq9L RSJJhRHjVSqzN1zxV4cogzXkICcQ50yF6gsfkhK6gps2aIti09fOGO5kwUVmgjuGJMf6 lDlw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=XIaGDBhNrSWCctxGM2T0020/FVJGHMEV90lsGnxNKwM=; b=r+2CrJfaZSQIEo4dBd2wuxj/TQuniLH7ZoXB7cjTLwp7tKuT1xkhYaTQjZtMwzcIiu IKOVF4468MQWkyq9Xl4Rs5oM1iruHNsHuSiiOlpT5SbeHjczEUuktMqEC2v0/WMJf9rP bDm1QFr6ubszxk6kpCfsz6XRr4jo2AUHdrpjgU12jBXvx964B8otBHPU42xumXpkssMW HjWRJKTzgMnJNH36DRLk371BsyR6kMpUrPiu7krVMyd0bBJPbQ2+Q9G2ly6KZzjvxf67 Ahp29IlDjp+9t4fC0i1wsUJ7evqh/ybEfDo6ZUUq0vpkYKW1lBTB6TnLPh+muTu27J7G b5fQ== X-Gm-Message-State: AOAM533D0N61ivAiU+5xyqVZiSYok2QbYLI12OCAxjLK8BkZzUUOrq9D TGp2KwY9nkfcvFAKk5CrGxYH9dfYrq7VqQ== X-Google-Smtp-Source: ABdhPJxV+Rykw3vw0k3HuynNEeD5xkfJ7CsSj2yVPVpKdW6Y076xD2qcWMPASTHyTDpN1mj60WfE1Q== X-Received: by 2002:a5d:4842:: with SMTP id n2mr6715966wrs.181.1613937212454; Sun, 21 Feb 2021 11:53:32 -0800 (PST) Received: from localhost.localdomain (cpc91226-cmbg18-2-0-cust7.5-4.cable.virginm.net. [82.0.29.8]) by smtp.gmail.com with ESMTPSA id z18sm1780372wrs.19.2021.02.21.11.53.31 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 21 Feb 2021 11:53:32 -0800 (PST) From: Mark Thompson To: ffmpeg-devel@ffmpeg.org Date: Sun, 21 Feb 2021 19:51:11 +0000 Message-Id: <20210221195125.1901683-4-sw@jkqxz.net> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20210221195125.1901683-1-sw@jkqxz.net> References: <20210221195125.1901683-1-sw@jkqxz.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 04/18] cbs_sei: Implement metadata manipulation for SEI codecs X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" --- libavcodec/cbs_h2645.c | 8 ++++ libavcodec/cbs_sei.c | 94 ++++++++++++++++++++++++++++++++++++++++++ libavcodec/cbs_sei.h | 35 ++++++++++++++++ 3 files changed, 137 insertions(+) diff --git a/libavcodec/cbs_h2645.c b/libavcodec/cbs_h2645.c index 0c591871d4..cb0005cc1b 100644 --- a/libavcodec/cbs_h2645.c +++ b/libavcodec/cbs_h2645.c @@ -1487,6 +1487,10 @@ const CodedBitstreamType ff_cbs_type_h264 = { .flush = &cbs_h264_flush, .close = &cbs_h264_close, + + .insert_metadata = &ff_cbs_sei_insert_metadata, + .remove_metadata = &ff_cbs_sei_remove_metadata, + .extract_metadata = &ff_cbs_sei_extract_metadata, }; const CodedBitstreamType ff_cbs_type_h265 = { @@ -1503,6 +1507,10 @@ const CodedBitstreamType ff_cbs_type_h265 = { .flush = &cbs_h265_flush, .close = &cbs_h265_close, + + .insert_metadata = &ff_cbs_sei_insert_metadata, + .remove_metadata = &ff_cbs_sei_remove_metadata, + .extract_metadata = &ff_cbs_sei_extract_metadata, }; static const SEIMessageTypeDescriptor cbs_sei_common_types[] = { diff --git a/libavcodec/cbs_sei.c b/libavcodec/cbs_sei.c index 14f1cae506..83bb5cca55 100644 --- a/libavcodec/cbs_sei.c +++ b/libavcodec/cbs_sei.c @@ -21,6 +21,7 @@ #include "cbs_h264.h" #include "cbs_h265.h" #include "cbs_sei.h" +#include "cbs_metadata.h" static void cbs_free_user_data_registered(void *opaque, uint8_t *data) { @@ -372,3 +373,96 @@ void ff_cbs_sei_delete_message_type(CodedBitstreamContext *ctx, } } } + +typedef struct SEIMetadata { + enum CBSMetadataType cbs_type; + int sei_type; +} SEIMetadata; + +static const SEIMetadata cbs_sei_metadata[] = { +}; + +static const SEIMessageTypeDescriptor *cbs_sei_find_type_from_metadata + (CodedBitstreamContext *ctx, enum CBSMetadataType metadata_type) +{ + const SEIMetadata *metadata; + + metadata = NULL; + for (int i = 0; i < FF_ARRAY_ELEMS(cbs_sei_metadata); i++) { + if (metadata_type == cbs_sei_metadata[i].cbs_type) { + metadata = &cbs_sei_metadata[i]; + break; + } + } + if (metadata) + return ff_cbs_sei_find_type(ctx, metadata->sei_type); + else + return NULL; +} + +int ff_cbs_sei_insert_metadata(CodedBitstreamContext *ctx, + CodedBitstreamFragment *au, + enum CBSMetadataType metadata_type, + const void *data) +{ + const SEIMessageTypeDescriptor *desc; + AVBufferRef *payload_buf; + int err; + + desc = cbs_sei_find_type_from_metadata(ctx, metadata_type); + if (!desc || !desc->fill) + return AVERROR(EINVAL); + + payload_buf = av_buffer_alloc(desc->size); + if (!payload_buf) + return AVERROR(ENOMEM); + + desc->fill(payload_buf->data, data); + + // All the metadata SEI messages must be unique in an access unit, + // so delete any existing ones of the given type. + ff_cbs_sei_delete_message_type(ctx, au, desc->type); + + err = ff_cbs_sei_add_message(ctx, au, desc->prefix, desc->type, + payload_buf->data, payload_buf); + av_buffer_unref(&payload_buf); + return err; +} + +int ff_cbs_sei_remove_metadata(CodedBitstreamContext *ctx, + CodedBitstreamFragment *au, + enum CBSMetadataType metadata_type) +{ + const SEIMessageTypeDescriptor *desc; + + desc = cbs_sei_find_type_from_metadata(ctx, metadata_type); + if (!desc) + return AVERROR(EINVAL); + + ff_cbs_sei_delete_message_type(ctx, au, desc->type); + return 0; +} + +int ff_cbs_sei_extract_metadata(CodedBitstreamContext *ctx, + CodedBitstreamFragment *au, + enum CBSMetadataType metadata_type, + void *data) +{ + const SEIMessageTypeDescriptor *desc; + SEIRawMessage *message; + int err; + + desc = cbs_sei_find_type_from_metadata(ctx, metadata_type); + if (!desc || !desc->extract) + return AVERROR(EINVAL); + + message = NULL; + err = ff_cbs_sei_find_message(ctx, au, desc->type, &message); + if (err < 0) { + // No message of the given type found. + return err; + } + + desc->extract(data, message->payload); + return 0; +} diff --git a/libavcodec/cbs_sei.h b/libavcodec/cbs_sei.h index c7a7a95be0..4fbcb6bfc0 100644 --- a/libavcodec/cbs_sei.h +++ b/libavcodec/cbs_sei.h @@ -110,6 +110,8 @@ typedef int (*SEIMessageWriteFunction)(CodedBitstreamContext *ctx, void *current, SEIMessageState *sei); +typedef void (*SEIConvertFunction)(void *dst, const void *src); + typedef struct SEIMessageTypeDescriptor { // Payload type for the message. (-1 in this field ends a list.) int type; @@ -123,6 +125,10 @@ typedef struct SEIMessageTypeDescriptor { SEIMessageReadFunction read; // Write bitstream from SEI message. SEIMessageWriteFunction write; + // Fill SEI structure from AV metadata. + SEIConvertFunction fill; + // Extract SEI metadata to AV structure. + SEIConvertFunction extract; } SEIMessageTypeDescriptor; // Macro for the read/write pair. The clumsy cast is needed because the @@ -132,6 +138,10 @@ typedef struct SEIMessageTypeDescriptor { .read = (SEIMessageReadFunction) cbs_ ## codec ## _read_ ## name, \ .write = (SEIMessageWriteFunction)cbs_ ## codec ## _write_ ## name +#define SEI_MESSAGE_FE(codec, name) \ + .fill = (SEIConvertFunction)cbs_ ## codec ## _fill_ ## name, \ + .extract = (SEIConvertFunction)cbs_ ## codec ## _extract_ ## name + // End-of-list sentinel element. #define SEI_MESSAGE_TYPE_END { .type = -1 } @@ -189,6 +199,8 @@ int ff_cbs_sei_find_message(CodedBitstreamContext *ctx, uint32_t payload_type, SEIRawMessage **message); +enum CBSMetadataType; + /** * Delete all messages with the given payload type from an access unit. */ @@ -196,4 +208,27 @@ void ff_cbs_sei_delete_message_type(CodedBitstreamContext *ctx, CodedBitstreamFragment *au, uint32_t payload_type); +/** + * Insert metadata into an access unit. + */ +int ff_cbs_sei_insert_metadata(CodedBitstreamContext *ctx, + CodedBitstreamFragment *au, + enum CBSMetadataType type, + const void *data); + +/** + * Remove metadata from an access unit. + */ +int ff_cbs_sei_remove_metadata(CodedBitstreamContext *ctx, + CodedBitstreamFragment *au, + enum CBSMetadataType type); + +/** + * Extract metadata from an access unit. + */ +int ff_cbs_sei_extract_metadata(CodedBitstreamContext *ctx, + CodedBitstreamFragment *au, + enum CBSMetadataType type, + void *data); + #endif /* AVCODEC_CBS_SEI_H */ From patchwork Sun Feb 21 19:51:12 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Thompson X-Patchwork-Id: 25867 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id BBA49449DB5 for ; Sun, 21 Feb 2021 21:53:43 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id A4C0F68A8A5; Sun, 21 Feb 2021 21:53:43 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wr1-f46.google.com (mail-wr1-f46.google.com [209.85.221.46]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id EDE9F689805 for ; Sun, 21 Feb 2021 21:53:33 +0200 (EET) Received: by mail-wr1-f46.google.com with SMTP id v1so16894618wrd.6 for ; Sun, 21 Feb 2021 11:53:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jkqxz-net.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=dO27ShCrhaFJMYDCGI5mQrfGS5JRIX3xoGS/wzX8gFQ=; b=wXSPhNkehFVFDRBzGQ+4OtJkdhzrlML12hi3VF01x9CIE3nlmfBLMSYebXznQmV+KC zxsTqCogYdhyaNrVgOfVDlOmFUvzYtveLBobOEx7pOVN7LmrUalWUYhBdzwhSOn6tQcD rtahk8M/FgEAk5VSugiiwtifAhly0ALg4a8tHzQ8vOQYJdbIJIBsbFqHPPABuXC36O9X AsmwOw9JfP4NeOCHorfl75Wfl8Z3BmD1wprK/lMmQwT1Nf7SKBJDyoXmJ6U6KfSqDD5Y Y45mOXucxjSJdI8s4IHACGWcq4zmht9aYZSSV2Rci6m1wMm8WPk/JPpxuCe8+w6kYUGt xigw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=dO27ShCrhaFJMYDCGI5mQrfGS5JRIX3xoGS/wzX8gFQ=; b=lm5PlgFYqVI0tZ73wQ6ahXuj7K93E8OJISm7Pvqb4kv/jTQnOe8jPKVWb9JCVR/BfC mVXQMBNiv2SM/DB0AqvO9fqSEvsyobqm3rXdYjoUFocmpMHMOTehHxPMgiUERDNNC6rI n+9nVFZAvb3FhRsv/gDq13SOZVSgW1bRa623cqalTyGVBRR0i0LH4rfy1S5lMT1pRKE6 1YkPSZnEMEpLlcV6O/F5dXpgiv6ndICk2CbYvtq08vgoDdt08dT4S/qE96I8IERKiEeq 5LPQ3uB3j7cgd6EHJ+PmQSdQ8glQppwO118XficvvCN3QM8VQae7nuTxh7rToCdniUWA i+KQ== X-Gm-Message-State: AOAM5336QMZVACfspOPZ8U/uPmNoH2Ot1OMT73c9cnv7r1rjiNqzqD87 MWUFy/33cn5baltXTmQhDfBOlCnaxWmO4A== X-Google-Smtp-Source: ABdhPJwKPeLUSdy553Tloun8TvHpaN42QAgOd28ZyCliZSC5fuPO9IM97CaRwVh5H8FrErE9nHIVaA== X-Received: by 2002:adf:d20c:: with SMTP id j12mr18477415wrh.76.1613937213449; Sun, 21 Feb 2021 11:53:33 -0800 (PST) Received: from localhost.localdomain (cpc91226-cmbg18-2-0-cust7.5-4.cable.virginm.net. [82.0.29.8]) by smtp.gmail.com with ESMTPSA id z18sm1780372wrs.19.2021.02.21.11.53.32 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 21 Feb 2021 11:53:32 -0800 (PST) From: Mark Thompson To: ffmpeg-devel@ffmpeg.org Date: Sun, 21 Feb 2021 19:51:12 +0000 Message-Id: <20210221195125.1901683-5-sw@jkqxz.net> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20210221195125.1901683-1-sw@jkqxz.net> References: <20210221195125.1901683-1-sw@jkqxz.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 05/18] cbs_sei: Implement fill and extract for HDR SEI messages X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Fill and extract both mastering display colour volume and content light level info messages. --- libavcodec/cbs_h2645.c | 130 +++++++++++++++++++++++++++++++++++++++++ libavcodec/cbs_sei.c | 4 ++ 2 files changed, 134 insertions(+) diff --git a/libavcodec/cbs_h2645.c b/libavcodec/cbs_h2645.c index cb0005cc1b..ecd2001816 100644 --- a/libavcodec/cbs_h2645.c +++ b/libavcodec/cbs_h2645.c @@ -18,6 +18,7 @@ #include "libavutil/attributes.h" #include "libavutil/avassert.h" +#include "libavutil/mastering_display_metadata.h" #include "bytestream.h" #include "cbs.h" @@ -1513,6 +1514,133 @@ const CodedBitstreamType ff_cbs_type_h265 = { .extract_metadata = &ff_cbs_sei_extract_metadata, }; +static uint32_t rescale_clip(AVRational value, uint32_t scale, + uint32_t min, uint32_t max) +{ + int64_t scaled = av_rescale(scale, value.num, value.den); + return av_clip64(scaled, min, max); +} + +static void cbs_sei_fill_mastering_display_colour_volume + (SEIRawMasteringDisplayColourVolume *mdcv, + const AVMasteringDisplayMetadata *mdm) +{ + memset(mdcv, 0, sizeof(*mdcv)); + + if (mdm->has_primaries) { + // The values in the metadata structure are fractions between 0 and 1, + // while the SEI message contains fixed-point values with an increment + // of 0.00002. So, scale up by 50000 to convert between them and clip + // to the allowed range ([5, 37000] for x, [5, 42000] for y). + + for (int a = 0; a < 3; a++) { + // The metadata structure stores this in RGB order, but the SEI + // wants it in GBR order. + static const uint8_t mapping[] = { 1, 2, 0 }; + int b = mapping[a]; + mdcv->display_primaries_x[a] = + rescale_clip(mdm->display_primaries[b][0], 50000, 5, 37000); + mdcv->display_primaries_y[a] = + rescale_clip(mdm->display_primaries[b][1], 50000, 5, 42000); + } + + mdcv->white_point_x = + rescale_clip(mdm->white_point[0], 50000, 5, 37000); + mdcv->white_point_y = + rescale_clip(mdm->white_point[1], 50000, 5, 42000); + } + + if (mdm->has_luminance) { + // Metadata are rational values in candelas per square metre, SEI + // contains fixed point in units of 0.0001 candelas per square + // metre. So scale up by 10000 to convert between them, and clip to + // the allowed ranges. + + mdcv->max_display_mastering_luminance = + rescale_clip(mdm->max_luminance, 10000, 50000, 100000000); + mdcv->min_display_mastering_luminance = + rescale_clip(mdm->min_luminance, 10000, 1, 50000); + + // The spec requires that they are not equal when in the normal + // range. + if (mdcv->min_display_mastering_luminance >= + mdcv->max_display_mastering_luminance) { + mdcv->min_display_mastering_luminance = + mdcv->max_display_mastering_luminance - 1; + } + } else { + mdcv->max_display_mastering_luminance = 0; + mdcv->min_display_mastering_luminance = 0; + } +} + +static void cbs_sei_extract_mastering_display_colour_volume + (AVMasteringDisplayMetadata *mdm, + const SEIRawMasteringDisplayColourVolume *mdcv) +{ +#define IN_RANGE(v, min, max) ((v) >= (min) && (v) <= (max)) +#define IS_VALID_COORD(x, y) (IN_RANGE(x, 5, 37000) && IN_RANGE(y, 5, 42000)) + int valid_chromaticity = 1; + for (int a = 0; a < 3; a++) { + if (!IS_VALID_COORD(mdcv->display_primaries_x[a], + mdcv->display_primaries_y[a])) + valid_chromaticity = 0; + } + if (!IS_VALID_COORD(mdcv->white_point_x, mdcv->white_point_y)) + valid_chromaticity = 0; + + memset(mdm, 0, sizeof(*mdm)); + + if (valid_chromaticity) { + for (int a = 0; a < 3; a++) { + // SEI message in GBR order, but metadata structure in RGB order. + static const uint8_t mapping[] = { 2, 0, 1 }; + int b = mapping[a]; + + mdm->display_primaries[a][0] = + av_make_q(mdcv->display_primaries_x[b], 50000); + mdm->display_primaries[a][1] = + av_make_q(mdcv->display_primaries_y[b], 50000); + } + + mdm->white_point[0] = av_make_q(mdcv->white_point_x, 50000); + mdm->white_point[1] = av_make_q(mdcv->white_point_y, 50000); + + mdm->has_primaries = 1; + } + + if (IN_RANGE(mdcv->min_display_mastering_luminance, 1, 50000) && + IN_RANGE(mdcv->max_display_mastering_luminance, 50000, 100000000)) { + mdm->min_luminance = av_make_q(mdcv->min_display_mastering_luminance, 10000); + mdm->max_luminance = av_make_q(mdcv->max_display_mastering_luminance, 10000); + + mdm->has_luminance = 1; + } +#undef IN_RANGE +#undef IS_VALID_COORD +} + +static void cbs_sei_fill_content_light_level_info + (SEIRawContentLightLevelInfo *cll, const AVContentLightMetadata *clm) +{ + memset(cll, 0, sizeof(*cll)); + + // Both the metadata and the SEI are in units of candelas per square + // metre, so we only need to clip to ensure that they are in the valid + // range. + + cll->max_content_light_level = av_clip_uintp2(clm->MaxCLL, 16); + cll->max_pic_average_light_level = av_clip_uintp2(clm->MaxFALL, 16); +} + + +static void cbs_sei_extract_content_light_level_info + (AVContentLightMetadata *clm, const SEIRawContentLightLevelInfo *cll) +{ + clm->MaxCLL = cll->max_content_light_level; + clm->MaxFALL = cll->max_pic_average_light_level; +} + static const SEIMessageTypeDescriptor cbs_sei_common_types[] = { { SEI_TYPE_FILLER_PAYLOAD, @@ -1537,12 +1665,14 @@ static const SEIMessageTypeDescriptor cbs_sei_common_types[] = { 1, 0, sizeof(SEIRawMasteringDisplayColourVolume), SEI_MESSAGE_RW(sei, mastering_display_colour_volume), + SEI_MESSAGE_FE(sei, mastering_display_colour_volume), }, { SEI_TYPE_CONTENT_LIGHT_LEVEL_INFO, 1, 0, sizeof(SEIRawContentLightLevelInfo), SEI_MESSAGE_RW(sei, content_light_level_info), + SEI_MESSAGE_FE(sei, content_light_level_info), }, { SEI_TYPE_ALTERNATIVE_TRANSFER_CHARACTERISTICS, diff --git a/libavcodec/cbs_sei.c b/libavcodec/cbs_sei.c index 83bb5cca55..a5990205d0 100644 --- a/libavcodec/cbs_sei.c +++ b/libavcodec/cbs_sei.c @@ -380,6 +380,10 @@ typedef struct SEIMetadata { } SEIMetadata; static const SEIMetadata cbs_sei_metadata[] = { + { CBS_METADATA_MASTERING_DISPLAY, + SEI_TYPE_MASTERING_DISPLAY_COLOUR_VOLUME }, + { CBS_METADATA_CONTENT_LIGHT_LEVEL, + SEI_TYPE_CONTENT_LIGHT_LEVEL_INFO }, }; static const SEIMessageTypeDescriptor *cbs_sei_find_type_from_metadata From patchwork Sun Feb 21 19:51:13 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Thompson X-Patchwork-Id: 25868 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id B8A4D449DB5 for ; Sun, 21 Feb 2021 21:53:44 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 9CFFD68A8F0; Sun, 21 Feb 2021 21:53:44 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wr1-f54.google.com (mail-wr1-f54.google.com [209.85.221.54]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 956C6689805 for ; Sun, 21 Feb 2021 21:53:34 +0200 (EET) Received: by mail-wr1-f54.google.com with SMTP id n8so16853823wrm.10 for ; Sun, 21 Feb 2021 11:53:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jkqxz-net.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=FcscOkfabZE42eht3O6YMCf42g2uK2gOUnDoqoWNNuk=; b=YYLdoAf1SHV4jNx0djFGHmCBcffPPbtvT4LESDUOKZCVnwCbrP5J+B1mPsOmfSjlYs 48QLk/trgGNQJLL94hu6piTRiRA5dw5mVdud2Bz3RWNzzXXvDwNqpdFpG4dZPCQt1Ki5 1k8q0P5EQouDZ1sr/rIef3CPCVl240Xv9Mk/iKLpxvcS/pD/BbCmgZEHPVqcyzQao9nQ J6lqZ+lEkCND+5uKV9zOxf99wi/9Fd7tnBUOLsqjlMU2rdsvHOKVkhWkCRAvPL5TbFmH mPurnosTOILdpws3s0x2HMmWcRNJCrkd/L7gg7eCJpBO8ogbxMfIjBSg+jMBkXPPXDoU Xc2w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=FcscOkfabZE42eht3O6YMCf42g2uK2gOUnDoqoWNNuk=; b=V8xjUiKjyirnEk864/lSjqNCPoUT5iZ8y7cI1VGk5v3nFkdpErhXyfcLlrCvUNpMQ5 pFzT0qOzrLoTMFQtovdW3dTfrCdUgsusCbs47/xip4rZagKIErIgts4YA3nnOrnsDbrH AdwuoTm2pH9PTeW3RGHw7zT+bibbEvjTBMJnmbC8z0h93B/0cdmw8IYAuziU0qki7pgD d3TNhgmdUEJbUNnCsULsGDtplRv8vlqJO/p+ZoKxNAnbXB3aL7n7VsJTbKgFzyWcTzol zeyIn7MmPqmhAnl88PEnNEAVUxqGvx9f7M6sjHBCsavKu+KG5H+K/yDieqb/LPRFps2S lMrg== X-Gm-Message-State: AOAM5318apbMG9gmpE6TrYDPWYarLqILztcrQTkiJnWVyGej2m/cDhjs E6pWChysHO9HZVXRrOczbo5UsDdejsqPvA== X-Google-Smtp-Source: ABdhPJxCBY6kpIH/ZzCpSaVPtz7zAJo+L02MJbkn8L5DIpTKOdQQ+md4GQc8GBe3IkbHtAlggssMfQ== X-Received: by 2002:a05:6000:1806:: with SMTP id m6mr18355948wrh.320.1613937214167; Sun, 21 Feb 2021 11:53:34 -0800 (PST) Received: from localhost.localdomain (cpc91226-cmbg18-2-0-cust7.5-4.cable.virginm.net. [82.0.29.8]) by smtp.gmail.com with ESMTPSA id z18sm1780372wrs.19.2021.02.21.11.53.33 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 21 Feb 2021 11:53:33 -0800 (PST) From: Mark Thompson To: ffmpeg-devel@ffmpeg.org Date: Sun, 21 Feb 2021 19:51:13 +0000 Message-Id: <20210221195125.1901683-6-sw@jkqxz.net> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20210221195125.1901683-1-sw@jkqxz.net> References: <20210221195125.1901683-1-sw@jkqxz.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 06/18] cbs_h264: Implement fill and extract for display orientation SEI message X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" --- libavcodec/cbs_h2645.c | 79 ++++++++++++++++++++++++++++++++++++++++++ libavcodec/cbs_sei.c | 2 ++ 2 files changed, 81 insertions(+) diff --git a/libavcodec/cbs_h2645.c b/libavcodec/cbs_h2645.c index ecd2001816..c01d243455 100644 --- a/libavcodec/cbs_h2645.c +++ b/libavcodec/cbs_h2645.c @@ -18,6 +18,7 @@ #include "libavutil/attributes.h" #include "libavutil/avassert.h" +#include "libavutil/display.h" #include "libavutil/mastering_display_metadata.h" #include "bytestream.h" @@ -1683,6 +1684,83 @@ static const SEIMessageTypeDescriptor cbs_sei_common_types[] = { SEI_MESSAGE_TYPE_END, }; +static void cbs_h264_fill_sei_display_orientation + (H264RawSEIDisplayOrientation *disp, const int32_t matrix[9]) +{ + double dmatrix[9]; + int hflip, vflip, i; + double scale_x, scale_y, angle; + uint16_t rotation; + + if (matrix[2] != 0 || matrix[5] != 0 || + matrix[6] != 0 || matrix[7] != 0) { + // Not representable. + goto cancel; + } + + for (i = 0; i < 9; i++) + dmatrix[i] = matrix[i] / 65536.0; + + // Extract scale factors. + scale_x = hypot(dmatrix[0], dmatrix[3]); + scale_y = hypot(dmatrix[1], dmatrix[4]); + + // Select flips to make the main diagonal positive. + hflip = dmatrix[0] < 0.0; + vflip = dmatrix[4] < 0.0; + if (hflip) + scale_x = -scale_x; + if (vflip) + scale_y = -scale_y; + + // Rescale. + for (i = 0; i < 9; i += 3) { + dmatrix[i] /= scale_x; + dmatrix[i + 1] /= scale_y; + } + + // Extract rotation. + angle = -atan2(dmatrix[3], dmatrix[0]); + if (!(angle >= -M_PI && angle <= M_PI)) { + // Not representable. + goto cancel; + } + if (angle < 0.0) + angle += 2 * M_PI; + rotation = (uint16_t)lrint(angle * 32768.0 / M_PI); + + if (!hflip && !vflip && rotation == 0) { + // This is the identity transformation, so rather than setting + // values for a new display orientation we should instead cancel + // any previous message. + cancel: + *disp = (H264RawSEIDisplayOrientation) { + .display_orientation_cancel_flag = 1, + }; + } else { + *disp = (H264RawSEIDisplayOrientation) { + .hor_flip = hflip, + .ver_flip = vflip, + .anticlockwise_rotation = rotation, + .display_orientation_repetition_period = 1, + }; + } +} + +static void cbs_h264_extract_sei_display_orientation + (int32_t matrix[9], const H264RawSEIDisplayOrientation *disp) +{ + if (disp->display_orientation_cancel_flag) { + // Set identity matrix. + av_display_rotation_set(matrix, 0.0); + } else { + av_display_rotation_set(matrix, + disp->anticlockwise_rotation * + 360.0 / 65536.0); + av_display_matrix_flip(matrix, disp->hor_flip, disp->ver_flip); + } +} + static const SEIMessageTypeDescriptor cbs_sei_h264_types[] = { { SEI_TYPE_BUFFERING_PERIOD, @@ -1719,6 +1797,7 @@ static const SEIMessageTypeDescriptor cbs_sei_h264_types[] = { 1, 0, sizeof(H264RawSEIDisplayOrientation), SEI_MESSAGE_RW(h264, sei_display_orientation), + SEI_MESSAGE_FE(h264, sei_display_orientation), }, SEI_MESSAGE_TYPE_END }; diff --git a/libavcodec/cbs_sei.c b/libavcodec/cbs_sei.c index a5990205d0..27b5997e47 100644 --- a/libavcodec/cbs_sei.c +++ b/libavcodec/cbs_sei.c @@ -384,6 +384,8 @@ static const SEIMetadata cbs_sei_metadata[] = { SEI_TYPE_MASTERING_DISPLAY_COLOUR_VOLUME }, { CBS_METADATA_CONTENT_LIGHT_LEVEL, SEI_TYPE_CONTENT_LIGHT_LEVEL_INFO }, + { CBS_METADATA_DISPLAY_MATRIX, + SEI_TYPE_DISPLAY_ORIENTATION }, }; static const SEIMessageTypeDescriptor *cbs_sei_find_type_from_metadata From patchwork Sun Feb 21 19:51:14 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Thompson X-Patchwork-Id: 25870 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id DA7D3449DB5 for ; Sun, 21 Feb 2021 21:53:46 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id C32DF68A912; Sun, 21 Feb 2021 21:53:46 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wr1-f41.google.com (mail-wr1-f41.google.com [209.85.221.41]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 37C9268A59A for ; Sun, 21 Feb 2021 21:53:35 +0200 (EET) Received: by mail-wr1-f41.google.com with SMTP id 7so16925759wrz.0 for ; Sun, 21 Feb 2021 11:53:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jkqxz-net.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=l28x5CP7JRAA4A0zUj/Ih4P2eWUErAmkuFVAZSSLzrA=; b=yuQ/rMHvT43MkhL6pVDnriHIqK7BlusVZZoUR5t1pIC5rVjYzcmlCs/Ejjtj7kBWId gSivHMdO2Kggf/apRGpg0g5TDtyEj8e0Pjp+y3vaGft4OenKQLE8T/LBXjXvBpml82T8 FobLSN1dLKLFRNyxsKGKrAtl8Gttmg77imqh2TspWy6WMsriGOA05oS1pFgLNgcaDsnj 4GjgJmAK6EFrZSsRxpoBX5jZHQEa4HGhbSLFWFk0yAosbgKuheN22OAJn1xn6wo4iMP/ UxFI70uE/UUS+2lwA59S1r1ntvLoq2TB9X3t/LuogyvcYzD+1B9FxWxVQ/41mvoLSnHA cisQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=l28x5CP7JRAA4A0zUj/Ih4P2eWUErAmkuFVAZSSLzrA=; b=OURg1mVoM9v30Gfc4SLYJNKpT2xIqoG2LOR5MFn3wu0c8WEOM5Ab0cPSKU9B8Y6p2h OyfLzlfPJZWyFAaBy26d79ZvIXiwZBv/1PeAtB8zMtSVlpUc5PjXY7sZSb94+f8fd8tv POsqDNIhhkbxvOXY7Zp+aF6pq0KSy0doU1hNOLIS4cYMd7OIF1+uFk8XmnfUmmGJnTVS VjL9ZwUXeNOf78ZPo6tq5eYA0P24LsTy+wYS6ib3UT5Xx+d2s6pE9EhR/b7Eh49lC4E6 oVtBNijk6P8HDf+IU/6ghw1/5G96VR8mc7fvc5ev4OwYNPTo5DJL/rAiq1xbmshQ6NNB u1rA== X-Gm-Message-State: AOAM530kgxNssawL+BlHRXD3S8Ou6XjPhfsKvD52jLjXYXob3M7TLgAs xBsOl++/mjv3rST0TFOgdB1kfb0dT4tn6Q== X-Google-Smtp-Source: ABdhPJwAvMl3ezyLsgDcJTbU/r1aiRoDFuIWsJosXMUa1BiGka01Pd0Jx83ptxBW9oGPnQpshdFWaQ== X-Received: by 2002:adf:f38a:: with SMTP id m10mr18869102wro.218.1613937214817; Sun, 21 Feb 2021 11:53:34 -0800 (PST) Received: from localhost.localdomain (cpc91226-cmbg18-2-0-cust7.5-4.cable.virginm.net. [82.0.29.8]) by smtp.gmail.com with ESMTPSA id z18sm1780372wrs.19.2021.02.21.11.53.34 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 21 Feb 2021 11:53:34 -0800 (PST) From: Mark Thompson To: ffmpeg-devel@ffmpeg.org Date: Sun, 21 Feb 2021 19:51:14 +0000 Message-Id: <20210221195125.1901683-7-sw@jkqxz.net> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20210221195125.1901683-1-sw@jkqxz.net> References: <20210221195125.1901683-1-sw@jkqxz.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 07/18] cbs_h264: Implement fill and extract for frame packing arrangement message X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" This maps from/to stereo 3D metadata. --- libavcodec/cbs_h2645.c | 83 ++++++++++++++++++++++++++++++++++++++++++ libavcodec/cbs_sei.c | 2 + 2 files changed, 85 insertions(+) diff --git a/libavcodec/cbs_h2645.c b/libavcodec/cbs_h2645.c index c01d243455..8efcd87156 100644 --- a/libavcodec/cbs_h2645.c +++ b/libavcodec/cbs_h2645.c @@ -20,6 +20,7 @@ #include "libavutil/avassert.h" #include "libavutil/display.h" #include "libavutil/mastering_display_metadata.h" +#include "libavutil/stereo3d.h" #include "bytestream.h" #include "cbs.h" @@ -1684,6 +1685,87 @@ static const SEIMessageTypeDescriptor cbs_sei_common_types[] = { SEI_MESSAGE_TYPE_END, }; +static struct { + enum AVStereo3DType st_type; + uint8_t fpa_type; +} cbs_sei_fpa_type_map[] = { + { AV_STEREO3D_2D, 6 }, + { AV_STEREO3D_SIDEBYSIDE, 3 }, + { AV_STEREO3D_TOPBOTTOM, 4 }, + { AV_STEREO3D_FRAMESEQUENCE, 5 }, + { AV_STEREO3D_CHECKERBOARD, 0 }, + { AV_STEREO3D_SIDEBYSIDE_QUINCUNX, 3 }, + { AV_STEREO3D_LINES, 2 }, + { AV_STEREO3D_COLUMNS, 1 }, +}; + +static void cbs_h264_fill_sei_frame_packing_arrangement + (H264RawSEIFramePackingArrangement *fpa, const AVStereo3D *st) +{ + memset(fpa, 0, sizeof(*fpa)); + + for (int i = 0; i < FF_ARRAY_ELEMS(cbs_sei_fpa_type_map); i++) { + if (cbs_sei_fpa_type_map[i].st_type == st->type) { + fpa->frame_packing_arrangement_type = + cbs_sei_fpa_type_map[i].fpa_type; + break; + } + } + + fpa->quincunx_sampling_flag = + st->type == AV_STEREO3D_CHECKERBOARD || + st->type == AV_STEREO3D_SIDEBYSIDE_QUINCUNX; + + if (st->type == AV_STEREO3D_2D) + fpa->content_interpretation_type = 0; + else if (st->flags & AV_STEREO3D_FLAG_INVERT) + fpa->content_interpretation_type = 2; + else + fpa->content_interpretation_type = 1; + + if (st->type == AV_STEREO3D_FRAMESEQUENCE) { + if (st->flags & AV_STEREO3D_FLAG_INVERT) + fpa->current_frame_is_frame0_flag = + st->view == AV_STEREO3D_VIEW_RIGHT; + else + fpa->current_frame_is_frame0_flag = + st->view == AV_STEREO3D_VIEW_LEFT; + } + + fpa->frame_packing_arrangement_repetition_period = + st->type != AV_STEREO3D_FRAMESEQUENCE; +} + +static void cbs_h264_extract_sei_frame_packing_arrangement + (AVStereo3D *st, const H264RawSEIFramePackingArrangement *fpa) +{ + memset(st, 0, sizeof(*st)); + + if (fpa->frame_packing_arrangement_cancel_flag) { + st->type = AV_STEREO3D_2D; + return; + } + + for (int i = 0; i < FF_ARRAY_ELEMS(cbs_sei_fpa_type_map); i++) { + if (cbs_sei_fpa_type_map[i].fpa_type == + fpa->frame_packing_arrangement_type) { + st->type = cbs_sei_fpa_type_map[i].st_type; + break; + } + } + + if (st->type == AV_STEREO3D_SIDEBYSIDE && + fpa->quincunx_sampling_flag) + st->type = AV_STEREO3D_SIDEBYSIDE_QUINCUNX; + + if (st->type == AV_STEREO3D_2D) { + if (fpa->current_frame_is_frame0_flag) + st->view = AV_STEREO3D_VIEW_LEFT; + else + st->view = AV_STEREO3D_VIEW_RIGHT; + } +} + static void cbs_h264_fill_sei_display_orientation (H264RawSEIDisplayOrientation *disp, const int32_t matrix[9]) { @@ -1791,6 +1873,7 @@ static const SEIMessageTypeDescriptor cbs_sei_h264_types[] = { 1, 0, sizeof(H264RawSEIFramePackingArrangement), SEI_MESSAGE_RW(h264, sei_frame_packing_arrangement), + SEI_MESSAGE_FE(h264, sei_frame_packing_arrangement), }, { SEI_TYPE_DISPLAY_ORIENTATION, diff --git a/libavcodec/cbs_sei.c b/libavcodec/cbs_sei.c index 27b5997e47..3fbea264aa 100644 --- a/libavcodec/cbs_sei.c +++ b/libavcodec/cbs_sei.c @@ -386,6 +386,8 @@ static const SEIMetadata cbs_sei_metadata[] = { SEI_TYPE_CONTENT_LIGHT_LEVEL_INFO }, { CBS_METADATA_DISPLAY_MATRIX, SEI_TYPE_DISPLAY_ORIENTATION }, + { CBS_METADATA_STEREO3D, + SEI_TYPE_FRAME_PACKING_ARRANGEMENT }, }; static const SEIMessageTypeDescriptor *cbs_sei_find_type_from_metadata From patchwork Sun Feb 21 19:51:15 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Thompson X-Patchwork-Id: 25872 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 80F73449DB5 for ; Sun, 21 Feb 2021 21:53:48 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 6B77568A943; Sun, 21 Feb 2021 21:53:48 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wr1-f43.google.com (mail-wr1-f43.google.com [209.85.221.43]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 1A84D68A7D8 for ; Sun, 21 Feb 2021 21:53:36 +0200 (EET) Received: by mail-wr1-f43.google.com with SMTP id c7so147304wru.8 for ; Sun, 21 Feb 2021 11:53:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jkqxz-net.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=Ny90rHshpebG9RGSsS0q3RoLqrZDau5o5xeJK+AfEBs=; b=CKs8IqQ/q2xkN/98pukA6oZZCVOKaW8k8hKp/HFdMXRycTJx7dNqjWCszSMBzV+JMk 0Z2Tg2LQq9jm2ng/k9g0hgyXtfL6DUiMO4GOdsU6fpii0wypZtxjgKnRyPKeRc846wmI BJ4WyLg3ivlZCJEhd1GlhfgS8NjE9eGNXjz/UV+kYhVdm7SyaBc6aDrt/r2cG7dzt1Od Faa1xqjCl8WfI3HLXMw2gfCyDpj4Rma3d4KFJI7rZ80mDJhg7LomBVfcmpaaly729BP0 eWfb1lrLj7EIzKb/Rwc2B2GUHfu3+Z2PMksq2KTF2mo3PcoKOImeH/aiDAVGhDnOJRIy cXmg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Ny90rHshpebG9RGSsS0q3RoLqrZDau5o5xeJK+AfEBs=; b=VTKLbes88EYt8TJag5JP+kjNpFLYOYQ8OY+25tFeVjmvP8EMLhZ4tLuz4PlWYC9dW3 4zT4ISQX0bwvdMaDPcfkmA/xyiuwncy29Ld6AkoIjGw8LXoBe75UCx924SIyzTprpG2h RGZUXmsebjBy/Z/5dWPH9WC1COgmjG1ljJNx3ZPS5SeUDkGM5VZ+Ruf6Dbk4HVGoWVk1 U43XKn0/NlWWBY+/nR40QQJQVrE3i/anL1HFztsRc6Z1FO2szOVzm7ZGZhhoippm/tk+ Vx4t8YXiQZXeD+NTOXMm9TjqCkvV+gveC961+6hVWP4MElxkne5pdStGv6WLLsZagFrX JwXQ== X-Gm-Message-State: AOAM531P6NPV+t47T7JvtS5xH7AE0Oj2RMsuu0WUMNoy8sVei5EZ5OWI 6BSobqcwdFSYmzJYeOsJxhJZPEXvhOtBqw== X-Google-Smtp-Source: ABdhPJwVYIAIQtiml+T+dPMJ/brZmhFdNwVuG1Bw91f+SsatqRhXh88mXwioJXQYgl2Q1OngOOsudg== X-Received: by 2002:adf:f608:: with SMTP id t8mr16851913wrp.196.1613937215443; Sun, 21 Feb 2021 11:53:35 -0800 (PST) Received: from localhost.localdomain (cpc91226-cmbg18-2-0-cust7.5-4.cable.virginm.net. [82.0.29.8]) by smtp.gmail.com with ESMTPSA id z18sm1780372wrs.19.2021.02.21.11.53.34 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 21 Feb 2021 11:53:35 -0800 (PST) From: Mark Thompson To: ffmpeg-devel@ffmpeg.org Date: Sun, 21 Feb 2021 19:51:15 +0000 Message-Id: <20210221195125.1901683-8-sw@jkqxz.net> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20210221195125.1901683-1-sw@jkqxz.net> References: <20210221195125.1901683-1-sw@jkqxz.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 08/18] cbs_av1: Implement metadata manipulation X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" --- libavcodec/cbs_av1.c | 142 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 142 insertions(+) diff --git a/libavcodec/cbs_av1.c b/libavcodec/cbs_av1.c index 302e1f38f5..0c958c81c2 100644 --- a/libavcodec/cbs_av1.c +++ b/libavcodec/cbs_av1.c @@ -22,6 +22,7 @@ #include "cbs.h" #include "cbs_internal.h" +#include "cbs_metadata.h" #include "cbs_av1.h" #include "internal.h" @@ -1267,6 +1268,143 @@ static void cbs_av1_close(CodedBitstreamContext *ctx) av_buffer_unref(&priv->frame_header_ref); } +static int cbs_av1_find_metadata_unit(CodedBitstreamFragment *tu, + uint64_t metadata_type, + CodedBitstreamUnit **iter) +{ + int i, found; + + found = 0; + for (i = 0; i < tu->nb_units; i++) { + CodedBitstreamUnit *unit = &tu->units[i]; + AV1RawOBU *obu = unit->content; + + if (!obu) + continue; + + if (obu->header.obu_type == AV1_OBU_METADATA) { + if (!*iter || found) { + *iter = unit; + return 0; + } + if (unit == *iter) + found = 1; + } + } + + return AVERROR(ENOENT); +} + +static uint64_t cbs_av1_find_metadata_type(enum CBSMetadataType type) +{ + struct { + enum CBSMetadataType cbs_type; + uint64_t av1_type; + } map[] = { + }; + + for (int i = 0; i < FF_ARRAY_ELEMS(map); i++) { + if (map[i].cbs_type == type) + return map[i].av1_type; + } + return 0; +} + +static void cbs_av1_delete_metadata_type(CodedBitstreamFragment *tu, + uint64_t metadata_type) +{ + int err; + while (1) { + CodedBitstreamUnit *unit = NULL; + err = cbs_av1_find_metadata_unit(tu, metadata_type, &unit); + if (err < 0) + break; + ff_cbs_delete_unit(tu, unit - tu->units); + } +} + +static int cbs_av1_remove_metadata(CodedBitstreamContext *ctx, + CodedBitstreamFragment *tu, + enum CBSMetadataType type) +{ + uint64_t metadata_type; + + metadata_type = cbs_av1_find_metadata_type(type); + if (metadata_type == 0) + return AVERROR(EINVAL); + + cbs_av1_delete_metadata_type(tu, metadata_type); + return 0; +} + +static int cbs_av1_insert_metadata(CodedBitstreamContext *ctx, + CodedBitstreamFragment *tu, + enum CBSMetadataType type, + const void *data) +{ + AVBufferRef *md_buf; + AV1RawOBU *obu; + uint64_t metadata_type; + int err; + + metadata_type = cbs_av1_find_metadata_type(type); + if (metadata_type == 0) + return AVERROR(EINVAL); + + md_buf = av_buffer_allocz(sizeof(AV1RawOBU)); + if (!md_buf) + return AVERROR(ENOMEM); + + obu = (AV1RawOBU*)md_buf->data; + + obu->header = (AV1RawOBUHeader) { + .obu_type = AV1_OBU_METADATA, + }; + obu->obu.metadata.metadata_type = metadata_type; + + switch (metadata_type) { + default: + av_assert0(0); + } + + cbs_av1_delete_metadata_type(tu, metadata_type); + + err = ff_cbs_insert_unit_content(tu, -1, AV1_OBU_METADATA, + obu, md_buf); + av_buffer_unref(&md_buf); + return err; +} + +static int cbs_av1_extract_metadata(CodedBitstreamContext *ctx, + CodedBitstreamFragment *tu, + enum CBSMetadataType type, + void *data) +{ + CodedBitstreamUnit *unit; + AV1RawOBU *obu; + uint64_t metadata_type; + int err; + + metadata_type = cbs_av1_find_metadata_type(type); + if (metadata_type == 0) + return AVERROR(EINVAL); + + unit = NULL; + err = cbs_av1_find_metadata_unit(tu, metadata_type, &unit); + if (err < 0) { + // No metadata with that type found. + return err; + } + obu = unit->content; + + switch (metadata_type) { + default: + av_assert0(0); + } + + return 0; +} + static void cbs_av1_free_metadata(void *unit, uint8_t *content) { AV1RawOBU *obu = (AV1RawOBU*)content; @@ -1333,4 +1471,8 @@ const CodedBitstreamType ff_cbs_type_av1 = { .flush = &cbs_av1_flush, .close = &cbs_av1_close, + + .insert_metadata = &cbs_av1_insert_metadata, + .remove_metadata = &cbs_av1_remove_metadata, + .extract_metadata = &cbs_av1_extract_metadata, }; From patchwork Sun Feb 21 19:51:16 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Thompson X-Patchwork-Id: 25873 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 64223449DB5 for ; Sun, 21 Feb 2021 21:53:49 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 51D0B68A96E; Sun, 21 Feb 2021 21:53:49 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wr1-f49.google.com (mail-wr1-f49.google.com [209.85.221.49]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 9F69668A83C for ; Sun, 21 Feb 2021 21:53:36 +0200 (EET) Received: by mail-wr1-f49.google.com with SMTP id n4so16936392wrx.1 for ; Sun, 21 Feb 2021 11:53:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jkqxz-net.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=3IK5HH85ktEL6gdLkCtW2mB6/DjmC2FQMFOKUNbVr0k=; b=vnGFlcZ5GvVYT0coZP8o0d2+1SbFXBLYhEshutVHotVYm0rgkXgRTEL12VJ4fwOxS7 x9J94zgqDB/qfmiIRaNXnD9C40MT1LnFxhmOaMHUyqeaQle/S0tE86vjC7ejDT2ZM+Le ZDmu2WKE/S6ZULNsUyghi/ZVgNdiS61k/5CUoYh5URST8fSAHUpH1bofWkw3NRcLDIhh vrMyrhr3ZzqL1JugqTosveH7OW+qp7sPf+LurUYpTQTQKl/S5km/xsp2LLr1/Cha2hV1 rKj3wIyctl7ykROVYiYZ+XMt2I6bKOEUqjm+v6Q5mb1wSCxhKvGgej6mH6poyC/UOS1D ckgQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=3IK5HH85ktEL6gdLkCtW2mB6/DjmC2FQMFOKUNbVr0k=; b=lvedbA8tDENdG4Y4a/ubVfFkBIhj8X7thHpaxA35Dl+R5AaTWjXwdeWwiVlzDA1qOa 4YuSlQ7ySakJdRj6yF0f6wOU7We1YeVaEmb13LOVawpaN3Ue3npkb1721mfYjwrp3CzO Kiqfx3oBt7cJ7kWyjQ0t7ar5ofrvxIXHQhtjqBg4Dg7t+RHKIq9ncJU3S8RQmuUOjvDr z/4n57apsm128D43RKslXCbOybMLDSlS21YgeP4zU6zv+PnLBGq3v/bAyj5PR6tRsFIS ppYo9ES+nMpDHAlGssMmQGS93GEZAXCbR8MyedA6ef+6Hq+pTbrSnYFqcROlKZ9kMli5 OkLQ== X-Gm-Message-State: AOAM531LT18OWDhNJG1nc41hh7l60hnQw+TrMnFnVFikEKXQd8YJRlN9 G67VsXwLzXcBxNaRV5rRB+QaytK182EzCw== X-Google-Smtp-Source: ABdhPJx45NOVFtObIwOFYPOAU3aID28yCuNY0jvaKUd0Sd55x9H00Zz79M+ic776Hq/1Cici2gSfrQ== X-Received: by 2002:a5d:6404:: with SMTP id z4mr18125114wru.103.1613937216158; Sun, 21 Feb 2021 11:53:36 -0800 (PST) Received: from localhost.localdomain (cpc91226-cmbg18-2-0-cust7.5-4.cable.virginm.net. [82.0.29.8]) by smtp.gmail.com with ESMTPSA id z18sm1780372wrs.19.2021.02.21.11.53.35 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 21 Feb 2021 11:53:35 -0800 (PST) From: Mark Thompson To: ffmpeg-devel@ffmpeg.org Date: Sun, 21 Feb 2021 19:51:16 +0000 Message-Id: <20210221195125.1901683-9-sw@jkqxz.net> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20210221195125.1901683-1-sw@jkqxz.net> References: <20210221195125.1901683-1-sw@jkqxz.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 09/18] cbs_av1: Support manipulating HDR CLL and MDCV metadata X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" --- libavcodec/cbs_av1.c | 91 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 91 insertions(+) diff --git a/libavcodec/cbs_av1.c b/libavcodec/cbs_av1.c index 0c958c81c2..281382c792 100644 --- a/libavcodec/cbs_av1.c +++ b/libavcodec/cbs_av1.c @@ -17,6 +17,7 @@ */ #include "libavutil/avassert.h" +#include "libavutil/mastering_display_metadata.h" #include "libavutil/opt.h" #include "libavutil/pixfmt.h" @@ -1301,6 +1302,10 @@ static uint64_t cbs_av1_find_metadata_type(enum CBSMetadataType type) enum CBSMetadataType cbs_type; uint64_t av1_type; } map[] = { + { CBS_METADATA_CONTENT_LIGHT_LEVEL, + AV1_METADATA_TYPE_HDR_CLL }, + { CBS_METADATA_MASTERING_DISPLAY, + AV1_METADATA_TYPE_HDR_MDCV }, }; for (int i = 0; i < FF_ARRAY_ELEMS(map); i++) { @@ -1337,6 +1342,13 @@ static int cbs_av1_remove_metadata(CodedBitstreamContext *ctx, return 0; } +static uint32_t rescale_clip(AVRational value, uint32_t scale, + uint32_t bits) +{ + int64_t scaled = av_rescale(scale, value.num, value.den); + return av_clip64(scaled, 0, INT64_C(1) << bits); +} + static int cbs_av1_insert_metadata(CodedBitstreamContext *ctx, CodedBitstreamFragment *tu, enum CBSMetadataType type, @@ -1363,6 +1375,47 @@ static int cbs_av1_insert_metadata(CodedBitstreamContext *ctx, obu->obu.metadata.metadata_type = metadata_type; switch (metadata_type) { + case AV1_METADATA_TYPE_HDR_CLL: + { + const AVContentLightMetadata *clm = data; + AV1RawMetadataHDRCLL *cll = + &obu->obu.metadata.metadata.hdr_cll; + + cll->max_cll = av_clip_uintp2(clm->MaxCLL, 16); + cll->max_fall = av_clip_uintp2(clm->MaxFALL, 16); + } + break; + case AV1_METADATA_TYPE_HDR_MDCV: + { + const AVMasteringDisplayMetadata *mdm = data; + AV1RawMetadataHDRMDCV *mdcv = + &obu->obu.metadata.metadata.hdr_mdcv; + + if (mdm->has_primaries) { + for (int i = 0; i < 3; i++) { + mdcv->primary_chromaticity_x[i] = + rescale_clip(mdm->display_primaries[i][0], + 1 << 16, 16); + mdcv->primary_chromaticity_y[i] = + rescale_clip(mdm->display_primaries[i][1], + 1 << 16, 16); + } + + mdcv->white_point_chromaticity_x = + rescale_clip(mdm->white_point[0], 1 << 16, 16); + mdcv->white_point_chromaticity_y = + rescale_clip(mdm->white_point[1], 1 << 16, 16); + } + + if (mdm->has_luminance) { + uint32_t value; + value = rescale_clip(mdm->max_luminance, 1 << 8, 32); + mdcv->luminance_max = value; + mdcv->luminance_min = + rescale_clip(mdm->min_luminance, 1 << 14, 32); + } + } + break; default: av_assert0(0); } @@ -1398,6 +1451,44 @@ static int cbs_av1_extract_metadata(CodedBitstreamContext *ctx, obu = unit->content; switch (metadata_type) { + case AV1_METADATA_TYPE_HDR_CLL: + { + AVContentLightMetadata *clm = data; + const AV1RawMetadataHDRCLL *cll = + &obu->obu.metadata.metadata.hdr_cll; + + clm->MaxCLL = cll->max_cll; + clm->MaxFALL = cll->max_fall; + } + break; + case AV1_METADATA_TYPE_HDR_MDCV: + { + AVMasteringDisplayMetadata *mdm = data; + const AV1RawMetadataHDRMDCV *mdcv = + &obu->obu.metadata.metadata.hdr_mdcv; + + for (int i = 0; i < 3; i++) { + mdm->display_primaries[i][0] = + av_make_q(mdcv->primary_chromaticity_x[i], 1 << 16); + mdm->display_primaries[i][1] = + av_make_q(mdcv->primary_chromaticity_y[i], 1 << 16); + } + + mdm->white_point[0] = + av_make_q(mdcv->white_point_chromaticity_x, 1 << 16); + mdm->white_point[1] = + av_make_q(mdcv->white_point_chromaticity_y, 1 << 16); + + mdm->max_luminance = + av_make_q(mdcv->luminance_max, 1 << 8); + mdm->min_luminance = + av_make_q(mdcv->luminance_min, 1 << 14); + + // The standard doesn't allow partially-filled MDCV blocks, so + // if the metadata appears it must be complete. + mdm->has_primaries = mdm->has_luminance = 1; + } + break; default: av_assert0(0); } From patchwork Sun Feb 21 19:51:17 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Thompson X-Patchwork-Id: 25875 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 01683449DB5 for ; Sun, 21 Feb 2021 21:53:51 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id E5DAA68A996; Sun, 21 Feb 2021 21:53:50 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wr1-f42.google.com (mail-wr1-f42.google.com [209.85.221.42]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 549CE68A72F for ; Sun, 21 Feb 2021 21:53:37 +0200 (EET) Received: by mail-wr1-f42.google.com with SMTP id r3so5259144wro.9 for ; Sun, 21 Feb 2021 11:53:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jkqxz-net.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=kILhC3CjQHSrXJEnhX7QUWeqkpRgH2f5lci5SGSBSIk=; b=iKf9OT7t3RVlbn1OWyiCS83EpAIkPg7vmDLpEyCn+OalWA92Ue3do1KOE/VsM6+9jy A8ZdryKRQLiX+KgEp3nobqh/K42raqxmN64qg+lWEA1ZQWwz7wQmbvr4ijxtfzZTfdYM a0SG/NhRg9z5kjoB5YEqXzjQDIMFAtPoGTFxxka5dSPDs2rnkR5fYceZ60RkwlAYeuKi EC4/9uJBoEBOocHAr45xbjyQc3yrSMQ2S3lgDwdsc+jkeEtP3Vg7MIzUHzr2laGTuON/ RYpOzh+QxI8pqqt4NtLR7WnXT+NLKGfiRm0e2i95GxfykhPyfAVWQwg4qWSl10nFwuY0 GsKg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=kILhC3CjQHSrXJEnhX7QUWeqkpRgH2f5lci5SGSBSIk=; b=oCdgcYS1bF2fkk4xULnVXGbNarRkQLhheAaNqcuKEgsSTwHpROCGcwBUNa5hDspcJv +1g33VdrmmD9C83ogVrv9udkc9D8gUdIgRZaXPt+TUFMWo/t1+XHJg3BmdJRUGx7mAwU 5CoIbmjjUGUSubcpDnskryuoGonXCjDUnJ3vVxWBOwrs9HxrHo/yNahw53Mf8s57PNVC knHsnDBsBNRWLdUZxRqFNEetsx6FMNkYGahDGFnGrRTWHfflyQkSxmjyFu8mj0aMsNBd Yyvo6LCf8AGfoeqrAyOnAVlsGa0RIDJTdOZlE7OnIhx3A/d67ZZwyTzRCidjr2O8gdVW xu3A== X-Gm-Message-State: AOAM531orRIFe2WKcsk+cNtavjKD00STTreWNVCq0wyuvuL2u7IXzLnK W/UtFDdVK5ZoOiB7rricVVqHz5Ra6LASKA== X-Google-Smtp-Source: ABdhPJzBcG3LYAC96xCaKUIWyTSI2gnRf0UxxdrdpL8dHKLHAuds0hP92ZpXYHME2U9xeP5pC/N6YA== X-Received: by 2002:adf:f890:: with SMTP id u16mr4621605wrp.67.1613937216834; Sun, 21 Feb 2021 11:53:36 -0800 (PST) Received: from localhost.localdomain (cpc91226-cmbg18-2-0-cust7.5-4.cable.virginm.net. [82.0.29.8]) by smtp.gmail.com with ESMTPSA id z18sm1780372wrs.19.2021.02.21.11.53.36 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 21 Feb 2021 11:53:36 -0800 (PST) From: Mark Thompson To: ffmpeg-devel@ffmpeg.org Date: Sun, 21 Feb 2021 19:51:17 +0000 Message-Id: <20210221195125.1901683-10-sw@jkqxz.net> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20210221195125.1901683-1-sw@jkqxz.net> References: <20210221195125.1901683-1-sw@jkqxz.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 10/18] cbs: Add interface to allow BSFs easy access to metadata manipulation X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" --- libavcodec/cbs_bsf.c | 73 ++++++++++++++++++++++++++++++++++++++++++++ libavcodec/cbs_bsf.h | 6 ++++ 2 files changed, 79 insertions(+) diff --git a/libavcodec/cbs_bsf.c b/libavcodec/cbs_bsf.c index 9b521cf111..4f8f7f499d 100644 --- a/libavcodec/cbs_bsf.c +++ b/libavcodec/cbs_bsf.c @@ -18,6 +18,7 @@ #include "bsf_internal.h" #include "cbs_bsf.h" +#include "cbs_metadata.h" static int cbs_bsf_update_side_data(AVBSFContext *bsf, AVPacket *pkt) { @@ -157,3 +158,75 @@ void ff_cbs_bsf_generic_close(AVBSFContext *bsf) ff_cbs_close(&ctx->input); ff_cbs_close(&ctx->output); } + +int ff_cbs_bsf_apply_metadata(AVBSFContext *bsf, + AVPacket *pkt, + CodedBitstreamFragment *frag, + enum CBSMetadataType type, int action) +{ + CBSBSFContext *ctx = bsf->priv_data; + const CBSMetadataTypeDescriptor *desc; + uint8_t *side_data; + size_t size; + int err, ignored; + + desc = ff_cbs_metadata_find_type(type); + if (!desc) + return AVERROR(EINVAL); + if (!desc->has_packet_side_data) + return AVERROR(EINVAL); + + switch (action) { + case BSF_ELEMENT_PASS: + // Nothing to do. + return 0; + + case BSF_ELEMENT_REMOVE: + // Remove all existing instances of the metadata. + err = ff_cbs_remove_metadata(ctx->output, frag, type); + if (err < 0) { + av_log(bsf, AV_LOG_ERROR, "Failed to remove %s metadata " + "from %s.\n", desc->name, ctx->type->fragment_name); + } + return err; + + case BSF_ELEMENT_INSERT: + // Insert matching side-data to replace existing metadata. + side_data = av_packet_get_side_data(pkt, + desc->packet_side_data_type, + &ignored); + if (!side_data) { + // No side data of this type. + return 0; + } + err = ff_cbs_insert_metadata(ctx->output, frag, type, side_data); + if (err < 0) { + av_log(bsf, AV_LOG_ERROR, "Failed to insert %s metadata " + "into %s.\n", desc->name, ctx->type->fragment_name); + } + + case BSF_ELEMENT_EXTRACT: + // Extract the metadata and attach it to the packet as side-data. + side_data = ff_cbs_alloc_metadata(type, &size); + if (!side_data) + return AVERROR(ENOMEM); + err = ff_cbs_extract_metadata(ctx->output, frag, type, side_data); + if (err == AVERROR(ENOENT)) { + // No metadata of that type. + return 0; + } + if (err < 0) { + av_log(bsf, AV_LOG_ERROR, "Failed to extract %s metadata " + "from %s.\n", desc->name, ctx->type->fragment_name); + return err; + } + err = av_packet_add_side_data(pkt, desc->packet_side_data_type, + side_data, size); + if (err < 0) + av_free(side_data); + return err; + + default: + return AVERROR(EINVAL); + } +} diff --git a/libavcodec/cbs_bsf.h b/libavcodec/cbs_bsf.h index 6be95abc3f..7027c20746 100644 --- a/libavcodec/cbs_bsf.h +++ b/libavcodec/cbs_bsf.h @@ -86,6 +86,12 @@ void ff_cbs_bsf_generic_close(AVBSFContext *bsf); */ int ff_cbs_bsf_generic_filter(AVBSFContext *bsf, AVPacket *pkt); +enum CBSMetadataType; + +int ff_cbs_bsf_apply_metadata(AVBSFContext *bsf, + AVPacket *pkt, + CodedBitstreamFragment *frag, + enum CBSMetadataType type, int action); // Options for element manipulation. enum { From patchwork Sun Feb 21 19:51:18 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Thompson X-Patchwork-Id: 25878 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id C008544A325 for ; Sun, 21 Feb 2021 21:56:15 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 924A468A9ED; Sun, 21 Feb 2021 21:53:54 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wr1-f51.google.com (mail-wr1-f51.google.com [209.85.221.51]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 1B4B168A86D for ; Sun, 21 Feb 2021 21:53:38 +0200 (EET) Received: by mail-wr1-f51.google.com with SMTP id v15so16911886wrx.4 for ; Sun, 21 Feb 2021 11:53:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jkqxz-net.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=ULoIM3i8FJmOZJt2W0pvKe2ehywLkIzBpPORkpkKQGg=; b=TRR6mdOFtL91UhZpDqh6mdHr+RKsZ5n1DWJO4I4IkBx7hRtkoPgJRhkDN2arJE3fJU oY4nLa2mct8stI4s5Q08He63J7krYwftDwa4gQq/fyJOu1uq2kBPbpGcfKS1tKnZAZiE 9xQQWRKZfCT/iEqkbeDKnRXurfpd58Q4CAT3A4HapnKVzc7ZD7TIsoqfizwRwWZjGtX+ FQy4xZ0JjBCRPp5dtYH/MKI4pVc1aq/z9i16gSWvOq5L34lLY0XWbFIYb85wPo6vt3j3 VY22UCTMHTN2AJfkv7wQz4r8rErLJghSYdrK0OjxQcuvkK8+U30EZvacdR+sdrYobT+p oVsw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ULoIM3i8FJmOZJt2W0pvKe2ehywLkIzBpPORkpkKQGg=; b=C5MHQM0R2rUuv9LREr/ZqSy2rww/ElPwNZbT5xPxBTyH00fB56aJATCD1hm8IpvJOP RrwYbh24XSrm74DFtE60MQ299Kih6G4108zuxRFTjORI6HRoW4NyDkp5T6f2uVRnVCHN ZuI6lHbBW3brWWtgs7DuNQLX2Clrf33fR6tD/nEYHRfrDE2MOhM0Z13qr2efK0TlCCVR KK/aRDLoaj6JNE7Ay4ZtVJ02m07vof/ysby2ii4TMdWLifLCVQobphCHFC0bmUTOd7ea B25GNIi7QC8wi8EbUqzgYPU0CAlif03MnBCbCQOrrSWqddFHt/Eem5wCOpL5rJKt6XWa CceQ== X-Gm-Message-State: AOAM533PCjVwHCa/gx8rCcwx2e2WhByX/NZl/jYHVTYPJnZ4fv/H9Apn +Y6S0vDjyhact6A58COMIiE1ir5vH5XYJQ== X-Google-Smtp-Source: ABdhPJwgfrVkoP5ovblK0SkyIiGuDJWye5+H40/wpXVBop9lclys7+C/gexPw+WUFHjddwmB4CLyag== X-Received: by 2002:adf:fb52:: with SMTP id c18mr5864699wrs.4.1613937217595; Sun, 21 Feb 2021 11:53:37 -0800 (PST) Received: from localhost.localdomain (cpc91226-cmbg18-2-0-cust7.5-4.cable.virginm.net. [82.0.29.8]) by smtp.gmail.com with ESMTPSA id z18sm1780372wrs.19.2021.02.21.11.53.36 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 21 Feb 2021 11:53:37 -0800 (PST) From: Mark Thompson To: ffmpeg-devel@ffmpeg.org Date: Sun, 21 Feb 2021 19:51:18 +0000 Message-Id: <20210221195125.1901683-11-sw@jkqxz.net> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20210221195125.1901683-1-sw@jkqxz.net> References: <20210221195125.1901683-1-sw@jkqxz.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 11/18] h264_metadata_bsf: Simplify display orientation handling X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" The case where the user supplies new value directly as options still requires a bit of special handling. --- libavcodec/h264_metadata_bsf.c | 118 ++++++--------------------------- 1 file changed, 21 insertions(+), 97 deletions(-) diff --git a/libavcodec/h264_metadata_bsf.c b/libavcodec/h264_metadata_bsf.c index 6673a32601..6511c4b776 100644 --- a/libavcodec/h264_metadata_bsf.c +++ b/libavcodec/h264_metadata_bsf.c @@ -25,6 +25,7 @@ #include "cbs.h" #include "cbs_bsf.h" #include "cbs_h264.h" +#include "cbs_metadata.h" #include "h264.h" #include "h264_levels.h" #include "h264_sei.h" @@ -75,7 +76,6 @@ typedef struct H264MetadataContext { int display_orientation; double rotate; int flip; - H264RawSEIDisplayOrientation display_orientation_payload; int level; } H264MetadataContext; @@ -328,122 +328,46 @@ static int h264_metadata_handle_display_orientation(AVBSFContext *bsf, int seek_point) { H264MetadataContext *ctx = bsf->priv_data; - SEIRawMessage *message; int err; - message = NULL; - while (ff_cbs_sei_find_message(ctx->common.output, au, - SEI_TYPE_DISPLAY_ORIENTATION, - &message) == 0) { - H264RawSEIDisplayOrientation *disp = message->payload; - int32_t *matrix; - - matrix = av_malloc(9 * sizeof(int32_t)); - if (!matrix) - return AVERROR(ENOMEM); - - av_display_rotation_set(matrix, - disp->anticlockwise_rotation * - 180.0 / 65536.0); - av_display_matrix_flip(matrix, disp->hor_flip, disp->ver_flip); - - // If there are multiple display orientation messages in an - // access unit, then the last one added to the packet (i.e. - // the first one in the access unit) will prevail. - err = av_packet_add_side_data(pkt, AV_PKT_DATA_DISPLAYMATRIX, - (uint8_t*)matrix, - 9 * sizeof(int32_t)); - if (err < 0) { - av_log(bsf, AV_LOG_ERROR, "Failed to attach extracted " - "displaymatrix side data to packet.\n"); - av_free(matrix); - return AVERROR(ENOMEM); - } - } - - if (ctx->display_orientation == BSF_ELEMENT_REMOVE || - ctx->display_orientation == BSF_ELEMENT_INSERT) { - ff_cbs_sei_delete_message_type(ctx->common.output, au, - SEI_TYPE_DISPLAY_ORIENTATION); - } - - if (ctx->display_orientation == BSF_ELEMENT_INSERT) { - H264RawSEIDisplayOrientation *disp = - &ctx->display_orientation_payload; + if (ctx->display_orientation != BSF_ELEMENT_INSERT) { + err = ff_cbs_bsf_apply_metadata(bsf, pkt, au, + CBS_METADATA_DISPLAY_MATRIX, + ctx->display_orientation); + if (err < 0) + return err; + } else { + int32_t matrix[9]; uint8_t *data; int size; int write = 0; + ff_cbs_sei_delete_message_type(ctx->common.output, au, + SEI_TYPE_DISPLAY_ORIENTATION); + data = av_packet_get_side_data(pkt, AV_PKT_DATA_DISPLAYMATRIX, &size); if (data && size >= 9 * sizeof(int32_t)) { - int32_t matrix[9]; - double dmatrix[9]; - int hflip, vflip, i; - double scale_x, scale_y, angle; - - memcpy(matrix, data, sizeof(matrix)); - - for (i = 0; i < 9; i++) - dmatrix[i] = matrix[i] / 65536.0; - - // Extract scale factors. - scale_x = hypot(dmatrix[0], dmatrix[3]); - scale_y = hypot(dmatrix[1], dmatrix[4]); - - // Select flips to make the main diagonal positive. - hflip = dmatrix[0] < 0.0; - vflip = dmatrix[4] < 0.0; - if (hflip) - scale_x = -scale_x; - if (vflip) - scale_y = -scale_y; - - // Rescale. - for (i = 0; i < 9; i += 3) { - dmatrix[i] /= scale_x; - dmatrix[i + 1] /= scale_y; - } - - // Extract rotation. - angle = atan2(dmatrix[3], dmatrix[0]); - - if (!(angle >= -M_PI && angle <= M_PI) || - matrix[2] != 0.0 || matrix[5] != 0.0 || - matrix[6] != 0.0 || matrix[7] != 0.0) { - av_log(bsf, AV_LOG_WARNING, "Input display matrix is not " - "representable in H.264 parameters.\n"); - } else { - disp->hor_flip = hflip; - disp->ver_flip = vflip; - disp->anticlockwise_rotation = - (uint16_t)rint((angle >= 0.0 ? angle - : angle + 2 * M_PI) * - 32768.0 / M_PI); - write = 1; - } + memcpy(matrix, data, 9 * sizeof(int32_t)); + write = 1; } if (seek_point) { if (!isnan(ctx->rotate)) { - disp->anticlockwise_rotation = - (uint16_t)rint((ctx->rotate >= 0.0 ? ctx->rotate - : ctx->rotate + 360.0) * - 65536.0 / 360.0); + av_display_rotation_set(matrix, ctx->rotate); write = 1; } if (ctx->flip) { - disp->hor_flip = !!(ctx->flip & FLIP_HORIZONTAL); - disp->ver_flip = !!(ctx->flip & FLIP_VERTICAL); + av_display_matrix_flip(matrix, + !!(ctx->flip & FLIP_HORIZONTAL), + !!(ctx->flip & FLIP_VERTICAL)); write = 1; } } if (write) { - disp->display_orientation_repetition_period = 1; - - err = ff_cbs_sei_add_message(ctx->common.output, au, 1, - SEI_TYPE_DISPLAY_ORIENTATION, - disp, NULL); + err = ff_cbs_sei_insert_metadata(ctx->common.output, au, + CBS_METADATA_DISPLAY_MATRIX, + matrix); if (err < 0) { av_log(bsf, AV_LOG_ERROR, "Failed to add display orientation " "SEI message to access unit.\n"); From patchwork Sun Feb 21 19:51:19 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Thompson X-Patchwork-Id: 25879 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id B658A44A325 for ; Sun, 21 Feb 2021 21:56:27 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 795C468A846; Sun, 21 Feb 2021 21:53:55 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wr1-f45.google.com (mail-wr1-f45.google.com [209.85.221.45]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id BFF9068A8E2 for ; Sun, 21 Feb 2021 21:53:38 +0200 (EET) Received: by mail-wr1-f45.google.com with SMTP id v14so16892706wro.7 for ; Sun, 21 Feb 2021 11:53:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jkqxz-net.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=W7ndWcExdP5tgseH7GKoA2hosE9QcgGDABUWcwJ4fRQ=; b=srEI8/R/C/LM3MuoDZvtdramwzTmhc8Yx0MnsjWLdt99MBmCKEnpz6X5Jw9+Ns7YcM Xe7AaTd6asAqQwNOEacnrpK/wRw/ga3c7e/HPsGwXhRsejsd1WV/DbZG1I1CoWjSknFN yVCcyvhN3YBjfZQffuhmFEGgwbdMPOh2wZHmcO9xgdmanLPbWXotfKMIkpgR1poDnNeA StLychrkj1VbLxRsnTTiRnBDtFVxucrYBqho5oLtKAno3cJiDhxEj4OchIEWypSGHdiB MW/sirDEF6jKNpI3j6+sm/pqIvLIH0X+HFTEso83QznptEYxeucQrHJGpGiS0O5UZ8qn yCtQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=W7ndWcExdP5tgseH7GKoA2hosE9QcgGDABUWcwJ4fRQ=; b=n7Z3xgLiP/nDg7E4Bm8jzSgj6MPfVDMknBil44jyB9/S0ipKKoK+BKY5IdkLYIhRCn +cMtSG4z9XonOXKN0+RmvIjT+oK6mNnsW3Zx4cwHa7oR99C5tWb0A8xZSO1z8uBKjMM+ SXL7mk8fzh2ACy027cRPrtTVDBTGJzwvvAeC05MAC5vVYyRrLbqVUT7b+tmms8CIHz1J 0ICaJgfEnKnxN4DRPk0I2D+0aUcMLhl5gnzknJwgMFd+mjtVdF0tBsorJqZzQQ/yln6Z vIC+WGa9+yKQ0FHU/Qwp2hyyDVzyJ46jw8nhSPrdAZx/VZeZO6cyUoJ/NRQnu/PsOl4m YQAA== X-Gm-Message-State: AOAM5305+jMY20nH5grALg3Z+vWBRIyZbIC3R5tFrCgaCdEgomyCh6x9 9m9csiimwYMzQtSsoPk3ykDT8By0+vjH5g== X-Google-Smtp-Source: ABdhPJzPu6LEYqtLE76wsjWdbq+FXnxXqNzDsBePeAwAgN7s8HJWqU5RFdk0QPAWTTHLsbn9pZ1bEA== X-Received: by 2002:a5d:6b47:: with SMTP id x7mr18859431wrw.170.1613937218327; Sun, 21 Feb 2021 11:53:38 -0800 (PST) Received: from localhost.localdomain (cpc91226-cmbg18-2-0-cust7.5-4.cable.virginm.net. [82.0.29.8]) by smtp.gmail.com with ESMTPSA id z18sm1780372wrs.19.2021.02.21.11.53.37 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 21 Feb 2021 11:53:37 -0800 (PST) From: Mark Thompson To: ffmpeg-devel@ffmpeg.org Date: Sun, 21 Feb 2021 19:51:19 +0000 Message-Id: <20210221195125.1901683-12-sw@jkqxz.net> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20210221195125.1901683-1-sw@jkqxz.net> References: <20210221195125.1901683-1-sw@jkqxz.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 12/18] h264_metadata_bsf: Support HDR metadata X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" --- doc/bitstream_filters.texi | 17 +++++++++++++++++ libavcodec/h264_metadata_bsf.c | 25 +++++++++++++++++++++++++ 2 files changed, 42 insertions(+) diff --git a/doc/bitstream_filters.texi b/doc/bitstream_filters.texi index 60e729484d..0ff76b86f0 100644 --- a/doc/bitstream_filters.texi +++ b/doc/bitstream_filters.texi @@ -285,6 +285,23 @@ level_idc value (for example, @samp{42}), or the special name @samp{auto} indicating that the filter should attempt to guess the level from the input stream properties. +@item mastering_display +@item content_light_level +Manipulate mastering display colour volume (see H.264 section D.2.29) +and content light level info (see H.264 section D.2.31) SEI messages in +the stream. + +Possible actions: +@table @samp +@item insert +Insert this type of message, taking the values from packet side-data. +@item remove +Remove all instances of this message. +@item extract +Extract the content of this type of message, attaching it to the packets +as side-data. +@end table + @end table @section h264_mp4toannexb diff --git a/libavcodec/h264_metadata_bsf.c b/libavcodec/h264_metadata_bsf.c index 6511c4b776..069d65432e 100644 --- a/libavcodec/h264_metadata_bsf.c +++ b/libavcodec/h264_metadata_bsf.c @@ -78,6 +78,9 @@ typedef struct H264MetadataContext { int flip; int level; + + int mastering_display; + int content_light_level; } H264MetadataContext; @@ -449,6 +452,20 @@ static int h264_metadata_update_fragment(AVBSFContext *bsf, AVPacket *pkt, return err; } + if (pkt) { + err = ff_cbs_bsf_apply_metadata(bsf, pkt, au, + CBS_METADATA_MASTERING_DISPLAY, + ctx->mastering_display); + if (err < 0) + return err; + + err = ff_cbs_bsf_apply_metadata(bsf, pkt, au, + CBS_METADATA_CONTENT_LIGHT_LEVEL, + ctx->content_light_level); + if (err < 0) + return err; + } + if (pkt) ctx->done_first_au = 1; @@ -609,6 +626,14 @@ static const AVOption h264_metadata_options[] = { { LEVEL("6.2", 62) }, #undef LEVEL + BSF_ELEMENT_OPTIONS_PIRE("mastering_display", + "Mastering display colour volume SEI", + mastering_display, FLAGS), + + BSF_ELEMENT_OPTIONS_PIRE("content_light_level", + "Content light level information SEI", + content_light_level, FLAGS), + { NULL } }; From patchwork Sun Feb 21 19:51:20 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Thompson X-Patchwork-Id: 25869 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id B1E80449DB5 for ; Sun, 21 Feb 2021 21:53:45 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 9C8A468A8FA; Sun, 21 Feb 2021 21:53:45 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wr1-f41.google.com (mail-wr1-f41.google.com [209.85.221.41]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 9C65D68A558 for ; Sun, 21 Feb 2021 21:53:39 +0200 (EET) Received: by mail-wr1-f41.google.com with SMTP id b3so16895431wrj.5 for ; Sun, 21 Feb 2021 11:53:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jkqxz-net.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=n0i/EZqyhI8NIKAxEmXUcTtl8cV6Mrju9aMqTUk8Xm0=; b=1pKvEHVu6gQTMDPCYOKnuReoiyrsNt0AlU7ybo/2pb8Xd3WLxhvSvzs9f3wO9STFWZ 1LWYAIHiSNf6+HbTVu+vcss9bC9/ZL8T4FcXIztcGXBckMk7O9K183HEvXJAFGXvYtz+ w4rcsQAk9Ab5Un8Av+aSj+34UmYBagiYLae+CuvgaRtgK6SQQe+1MkU60u03DZHQbeg1 St1DoPq+C2vEyvukicSqkjYZdGph7lHC+BU0RwmextOtjDvNzZvXzv223VwI6yKWDY+n b95gwLR1bov71r2Sxt8WryMDEC42bOHDR4StQKPeouMjAq3ztdHGYhCAd59KUgDgN5RK GuHA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=n0i/EZqyhI8NIKAxEmXUcTtl8cV6Mrju9aMqTUk8Xm0=; b=Clb99KSX4A4UIDLu6aA2HrKNpB/RmFy7vLjQGVOtVwRRlBGY0EbcFPg4oJ9fHDLloA 4rbgrZET+wk4uia5dCZ8+YfS6Eoomj/LLumqvgwYR6MS6Arzq/yrypv/7SF1jNKZS3Pc +9mL9ZFsJGTlWLdhzb9yD7rBsAkRVNjugngBZCuTWOag/nPeZvTUu4q43NO8e8rsvpzY 1qi64wyyuE5yeVdPNK+yzAycSbvVdOPexcX6MwmT1HNQAL00uoT3Nfxac774lkFEnwKy fXR1GIVQVuMDUEzttyPF9znTa7HS7y2SNGElAUfxTtVsDpSccuYr2bWQmOqo6/u0P3eL ZZyA== X-Gm-Message-State: AOAM530rvqJPDyV8yIJoa4dMxbL1BMHXO8Iyt4G8lY5Ix7ZOZpqYDH2z ce75AJkVKc/4d/Uczz5ggX/ip+3fALY9ig== X-Google-Smtp-Source: ABdhPJwq0mdfPt+wP6hPzo+dyOU5oEk7RpFMQVZqND5JbaMNS+ieGjitrSRm3Y10XtubbW7Ncmz97A== X-Received: by 2002:a5d:43c2:: with SMTP id v2mr18065440wrr.81.1613937219053; Sun, 21 Feb 2021 11:53:39 -0800 (PST) Received: from localhost.localdomain (cpc91226-cmbg18-2-0-cust7.5-4.cable.virginm.net. [82.0.29.8]) by smtp.gmail.com with ESMTPSA id z18sm1780372wrs.19.2021.02.21.11.53.38 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 21 Feb 2021 11:53:38 -0800 (PST) From: Mark Thompson To: ffmpeg-devel@ffmpeg.org Date: Sun, 21 Feb 2021 19:51:20 +0000 Message-Id: <20210221195125.1901683-13-sw@jkqxz.net> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20210221195125.1901683-1-sw@jkqxz.net> References: <20210221195125.1901683-1-sw@jkqxz.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 13/18] h265_metadata_bsf: Support HDR metadata X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" --- doc/bitstream_filters.texi | 17 +++++++++++++++++ libavcodec/h265_metadata_bsf.c | 26 ++++++++++++++++++++++++++ 2 files changed, 43 insertions(+) diff --git a/doc/bitstream_filters.texi b/doc/bitstream_filters.texi index 0ff76b86f0..e739832665 100644 --- a/doc/bitstream_filters.texi +++ b/doc/bitstream_filters.texi @@ -393,6 +393,23 @@ The argument must be the name of a level (for example, @samp{5.1}), a or the special name @samp{auto} indicating that the filter should attempt to guess the level from the input stream properties. +@item mastering_display +@item content_light_level +Manipulate mastering display colour volume (see H.265 section D.3.28) +and content light level info (see H.265 section D.3.35) SEI messages in +the stream. + +Possible actions: +@table @samp +@item insert +Insert this type of message, taking the values from packet side-data. +@item remove +Remove all instances of this message. +@item extract +Extract the content of this type of message, attaching it to the packets +as side-data. +@end table + @end table @section hevc_mp4toannexb diff --git a/libavcodec/h265_metadata_bsf.c b/libavcodec/h265_metadata_bsf.c index 59325c0471..0986564237 100644 --- a/libavcodec/h265_metadata_bsf.c +++ b/libavcodec/h265_metadata_bsf.c @@ -23,6 +23,7 @@ #include "cbs.h" #include "cbs_bsf.h" #include "cbs_h265.h" +#include "cbs_metadata.h" #include "hevc.h" #include "h265_profile_level.h" @@ -60,6 +61,9 @@ typedef struct H265MetadataContext { int level; int level_guess; int level_warned; + + int mastering_display; + int content_light_level; } H265MetadataContext; @@ -393,6 +397,20 @@ static int h265_metadata_update_fragment(AVBSFContext *bsf, AVPacket *pkt, } } + if (pkt) { + err = ff_cbs_bsf_apply_metadata(bsf, pkt, au, + CBS_METADATA_MASTERING_DISPLAY, + ctx->mastering_display); + if (err < 0) + return err; + + err = ff_cbs_bsf_apply_metadata(bsf, pkt, au, + CBS_METADATA_CONTENT_LIGHT_LEVEL, + ctx->content_light_level); + if (err < 0) + return err; + } + return 0; } @@ -484,6 +502,14 @@ static const AVOption h265_metadata_options[] = { { LEVEL("8.5", 255) }, #undef LEVEL + BSF_ELEMENT_OPTIONS_PIRE("mastering_display", + "Mastering display colour volume SEI", + mastering_display, FLAGS), + + BSF_ELEMENT_OPTIONS_PIRE("content_light_level", + "Content light level information SEI", + content_light_level, FLAGS), + { NULL } }; From patchwork Sun Feb 21 19:51:21 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Thompson X-Patchwork-Id: 25871 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id A5250449DB5 for ; Sun, 21 Feb 2021 21:53:47 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 8CA9068A8F5; Sun, 21 Feb 2021 21:53:47 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wr1-f46.google.com (mail-wr1-f46.google.com [209.85.221.46]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 127DA68A626 for ; Sun, 21 Feb 2021 21:53:40 +0200 (EET) Received: by mail-wr1-f46.google.com with SMTP id v1so16894775wrd.6 for ; Sun, 21 Feb 2021 11:53:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jkqxz-net.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=+lWwVoLtWITqmPTARsZA/Cm2R5QuqTD61g/LAacmBms=; b=D/9lrjhRL6qLu01fX/B22UO84hghg6W6cOkcatuWHcJkfvBpS7mjXnsJoBkVMBouBw cy+9dBYN5ui904A92asUWY1UOz9SKsQ6jcitR9xg9KmGv/oRt8ZI6l7hONE5M4uTISMi qtPidoyRaTjDh0Psvl9iJ1Lz/UpBg3C4E40gl7EYmp3m5fDAZuO1apa/g6xdLUiw05f8 +M+nZrhoB883zEqDApA/ry5XRHGUmvQ7GUxOFwHp062cpOu1xB30tnopemNxnY3pebG4 r1Xiuafp4ZKM+5OoIzEqDA+C5ijeo/oYwZOeex6GBtZrmJ6+sGatgFRcDN5Z1mRI6q6f 9BwA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=+lWwVoLtWITqmPTARsZA/Cm2R5QuqTD61g/LAacmBms=; b=SMNXuehUpoXSBRyFK6ucGpdL3K5aPhWyRz9CNJVGPMH5KsbxoN7nZN+ory5k0EaZH3 0JkbHesJnIzYHIzVQO2vdZrTpVZug8ur39m4AibVR6Zfw5pH5niELV68mYpTKrp1GqUU UWiJeNoZlkAy7YJHfP50mjcgmxEXLVK2A1S2jA+Lb6eAqUYJMs+jP/6IqArb5cXlK1eO LaGxEwBaCLEyZEFff4Y6NdIXNcmgZOMIzlqRAzHm5cyQbNxba5zv4vryM30mcnHy952U DNIS6R5e2XryvPfHedGji8/1RErvx3RLh6Gr9sUVwyc4BBU3T6OV5a777cczKISTXOTk r61A== X-Gm-Message-State: AOAM531xRib1xenC7sJcSquvcJZaAyJqnFKIDC/nUrTuPVFtn3WpQLsy 4tUCj4/awLTepvUQ62rBp9ti9Jtk8AmWBg== X-Google-Smtp-Source: ABdhPJxFEWmxbDCz3Yh5kKHsgXTbUqbS5YZEK7nUzmtXD7N2U4jpV4GImYf9FgLCshgS3WdDJWzkRg== X-Received: by 2002:adf:ee0e:: with SMTP id y14mr5133904wrn.380.1613937219680; Sun, 21 Feb 2021 11:53:39 -0800 (PST) Received: from localhost.localdomain (cpc91226-cmbg18-2-0-cust7.5-4.cable.virginm.net. [82.0.29.8]) by smtp.gmail.com with ESMTPSA id z18sm1780372wrs.19.2021.02.21.11.53.39 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 21 Feb 2021 11:53:39 -0800 (PST) From: Mark Thompson To: ffmpeg-devel@ffmpeg.org Date: Sun, 21 Feb 2021 19:51:21 +0000 Message-Id: <20210221195125.1901683-14-sw@jkqxz.net> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20210221195125.1901683-1-sw@jkqxz.net> References: <20210221195125.1901683-1-sw@jkqxz.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 14/18] av1_metadata_bsf: Support HDR metadata X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" --- doc/bitstream_filters.texi | 17 +++++++++++++++++ libavcodec/av1_metadata_bsf.c | 26 ++++++++++++++++++++++++++ 2 files changed, 43 insertions(+) diff --git a/doc/bitstream_filters.texi b/doc/bitstream_filters.texi index e739832665..6a4c0cfe5e 100644 --- a/doc/bitstream_filters.texi +++ b/doc/bitstream_filters.texi @@ -90,6 +90,23 @@ has a fixed framerate. Ignored if @option{tick_rate} is not also set. @item delete_padding Deletes Padding OBUs. +@item hdr_cll +@item hdr_mdcv +Manipulate content light level (see AV1 section 6.7.3) and mastering +display colour volume (see AV1 section 6.7.4) metadata messages in the +stream. + +Possible actions: +@table @samp +@item insert +Insert this type of message, taking the values from packet side-data. +@item remove +Remove all instances of this message. +@item extract +Extract the content of this type of message, attaching it to the packets +as side-data. +@end table + @end table @section chomp diff --git a/libavcodec/av1_metadata_bsf.c b/libavcodec/av1_metadata_bsf.c index 8cafe461d1..77062e0400 100644 --- a/libavcodec/av1_metadata_bsf.c +++ b/libavcodec/av1_metadata_bsf.c @@ -23,6 +23,7 @@ #include "cbs.h" #include "cbs_bsf.h" #include "cbs_av1.h" +#include "cbs_metadata.h" typedef struct AV1MetadataContext { CBSBSFContext common; @@ -40,6 +41,9 @@ typedef struct AV1MetadataContext { int num_ticks_per_picture; int delete_padding; + + int hdr_cll; + int hdr_mdcv; } AV1MetadataContext; @@ -143,6 +147,20 @@ static int av1_metadata_update_fragment(AVBSFContext *bsf, AVPacket *pkt, } } + if (pkt) { + err = ff_cbs_bsf_apply_metadata(bsf, pkt, frag, + CBS_METADATA_CONTENT_LIGHT_LEVEL, + ctx->hdr_mdcv); + if (err < 0) + return err; + + err = ff_cbs_bsf_apply_metadata(bsf, pkt, frag, + CBS_METADATA_MASTERING_DISPLAY, + ctx->hdr_mdcv); + if (err < 0) + return err; + } + return 0; } @@ -203,6 +221,14 @@ static const AVOption av1_metadata_options[] = { OFFSET(delete_padding), AV_OPT_TYPE_BOOL, { .i64 = 0 }, 0, 1, FLAGS}, + BSF_ELEMENT_OPTIONS_PIRE("hdr_cll", + "HDR content light level metadata", + hdr_cll, FLAGS), + + BSF_ELEMENT_OPTIONS_PIRE("hdr_mdcv", + "HDR mastering display colour volume metadata", + hdr_mdcv, FLAGS), + { NULL } }; From patchwork Sun Feb 21 19:51:22 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Thompson X-Patchwork-Id: 25874 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 35903449DB5 for ; Sun, 21 Feb 2021 21:53:50 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 23CBB68A98D; Sun, 21 Feb 2021 21:53:50 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wr1-f44.google.com (mail-wr1-f44.google.com [209.85.221.44]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id B2E7E68A68F for ; Sun, 21 Feb 2021 21:53:40 +0200 (EET) Received: by mail-wr1-f44.google.com with SMTP id v1so16894793wrd.6 for ; Sun, 21 Feb 2021 11:53:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jkqxz-net.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=KbbINlyfAMKTgEYn7pJIg7XAblWOPeY9On6VPnqJMEo=; b=Vf/PsRjG1vqUXWe9R2eqqCzpjMatOlwuwqu0KDPrlqQdHveWuMSzghIqdPs5LdjQEm KkmnuRCcP0AU8AY168Q82LfFMGPRfx2pJmn9Gnez0uuwoYvsgn18OJvYO96YrNVhNX2A y4tDjmT1z3+tUvTuAHD/esZ3saAlVidN6JYEzQtBUJVNd1pJ+RAhINjMVefdDJTodLAs ow88fuydY8NC+m43L3yFc2WmCkMRZ5WIoXcmsXUqGTnrLtRjaRQXjh8j8rN4ZDm/6IwD AXGRf/nrTPm/Rk2UXl+ipM4sJyFp4s+c83TVdX5/hnEuKhJE+pADO4Mi/pWnOzL31ErJ B2Fg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=KbbINlyfAMKTgEYn7pJIg7XAblWOPeY9On6VPnqJMEo=; b=sAfYuFgnr3Qui4cXvgGRzza5tVFqkVp468jr0BHtCnDnLj10y7MRaBOpqcYUmIqJWX BfoUmTAlCnh5mi/kxGL5lwk/DnamJih9xetQWRGCL65SgJHF+uOHYbWfcssBy0N6CEnh wyKnMecplrdmJ9YqIkAY6GGOF+7h2Ytd0B/PFhAKIYBY4UUbrsM0ruoFxaHlRM/N9NC3 ozFcQZpvcHw1TGpJZmlWi0G2m2T5xyqmo/IZTtlO/9x4kRp9YmsWELa3Waw6BSU5b6Fs p8lyS0euQ2D5SDeLGabCGW/hFRDkRGB4ZPK0GIe9gyotlVpHY6OFTwV67WX8OXvBN0w/ 2gAQ== X-Gm-Message-State: AOAM532t3z0blzrbb1w5LWA1L1KXsjbMedMZO05bfQZ9x5Ykn1dUt/o2 gEVOoC/YINn83Orecmtw4B0+cXpgfwIhhA== X-Google-Smtp-Source: ABdhPJyipF4iOnEsnd7cbRXVAlee6iwlWzajkM/hsz7WYQSn080etpy20XBSuQrAHVkP3WA1XcZTHg== X-Received: by 2002:adf:fb49:: with SMTP id c9mr18779706wrs.72.1613937220358; Sun, 21 Feb 2021 11:53:40 -0800 (PST) Received: from localhost.localdomain (cpc91226-cmbg18-2-0-cust7.5-4.cable.virginm.net. [82.0.29.8]) by smtp.gmail.com with ESMTPSA id z18sm1780372wrs.19.2021.02.21.11.53.39 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 21 Feb 2021 11:53:40 -0800 (PST) From: Mark Thompson To: ffmpeg-devel@ffmpeg.org Date: Sun, 21 Feb 2021 19:51:22 +0000 Message-Id: <20210221195125.1901683-15-sw@jkqxz.net> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20210221195125.1901683-1-sw@jkqxz.net> References: <20210221195125.1901683-1-sw@jkqxz.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 15/18] vaapi_encode: Unify SEI option mask enum X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" This was in two disjoint parts in the H.264 and H.265 files. Unify them in the header to avoid any confusion, because they are really the same enum. --- libavcodec/vaapi_encode.h | 8 ++++++++ libavcodec/vaapi_encode_h264.c | 6 ------ libavcodec/vaapi_encode_h265.c | 5 ----- 3 files changed, 8 insertions(+), 11 deletions(-) diff --git a/libavcodec/vaapi_encode.h b/libavcodec/vaapi_encode.h index b41604a883..980bbb1b7a 100644 --- a/libavcodec/vaapi_encode.h +++ b/libavcodec/vaapi_encode.h @@ -474,5 +474,13 @@ int ff_vaapi_encode_close(AVCodecContext *avctx); VAAPI_ENCODE_RC_MODE(QVBR, "Quality-defined variable-bitrate"), \ VAAPI_ENCODE_RC_MODE(AVBR, "Average variable-bitrate") +// SEI type mask used in options for H.264 and H.265. +enum { + SEI_TIMING = 0x01, + SEI_IDENTIFIER = 0x02, + SEI_RECOVERY_POINT = 0x04, + SEI_MASTERING_DISPLAY = 0x08, + SEI_CONTENT_LIGHT_LEVEL = 0x10, +}; #endif /* AVCODEC_VAAPI_ENCODE_H */ diff --git a/libavcodec/vaapi_encode_h264.c b/libavcodec/vaapi_encode_h264.c index ce73e269f9..b546ddec08 100644 --- a/libavcodec/vaapi_encode_h264.c +++ b/libavcodec/vaapi_encode_h264.c @@ -35,12 +35,6 @@ #include "internal.h" #include "vaapi_encode.h" -enum { - SEI_TIMING = 0x01, - SEI_IDENTIFIER = 0x02, - SEI_RECOVERY_POINT = 0x04, -}; - // Random (version 4) ISO 11578 UUID. static const uint8_t vaapi_encode_h264_sei_identifier_uuid[16] = { 0x59, 0x94, 0x8b, 0x28, 0x11, 0xec, 0x45, 0xaf, diff --git a/libavcodec/vaapi_encode_h265.c b/libavcodec/vaapi_encode_h265.c index 2c356fbd38..33acbf23f2 100644 --- a/libavcodec/vaapi_encode_h265.c +++ b/libavcodec/vaapi_encode_h265.c @@ -37,11 +37,6 @@ #include "put_bits.h" #include "vaapi_encode.h" -enum { - SEI_MASTERING_DISPLAY = 0x08, - SEI_CONTENT_LIGHT_LEVEL = 0x10, -}; - typedef struct VAAPIEncodeH265Picture { int pic_order_cnt; From patchwork Sun Feb 21 19:51:23 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Thompson X-Patchwork-Id: 25876 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 03491449DB5 for ; Sun, 21 Feb 2021 21:53:52 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id DC4DE68A9BE; Sun, 21 Feb 2021 21:53:51 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wr1-f53.google.com (mail-wr1-f53.google.com [209.85.221.53]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 62FD068A857 for ; Sun, 21 Feb 2021 21:53:41 +0200 (EET) Received: by mail-wr1-f53.google.com with SMTP id 7so16925892wrz.0 for ; Sun, 21 Feb 2021 11:53:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jkqxz-net.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=nffE39tD0t8oYzOgkDi/f3pMhV/FkjvYbdYEM2ppLR4=; b=sbbYOHkoClnR9XVKqTHCd9yXiEig4zWWlw/PhKlEzdrPjt3WN5EzGSBmzsyI/EQ0Vi 0/Tr3YGu5uLfixvBrNRnHqs5QKDQSdEHA5LOdqEpBCTccDvX3oh/CNXXRFBywP2YDNUj qVkO4qYrFa0ZyPmad0RGfayRo9QnwthEqczK1VCDECNpsfaBicwKA8P/D7yA92+zXF6g oIP86lMBNmGAM22eYUPbSMZF4LrapTL8rwoU7CYvCtXFh+KX5+KLKpYmrmS1tFuOB5XO cVdWfFYtJAGMQTxGMQ+pynQ4353VqHsti+pwWQySO1mbG8NCBL1FyiIddxstBBzpLVMM yzxA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=nffE39tD0t8oYzOgkDi/f3pMhV/FkjvYbdYEM2ppLR4=; b=qoXV337Go/Dz2w0a/7SCXu8coSzDsGmNwSW7ufAaDDJgn5UZ/FLt8Wm5PBUWc1EyaI EiocMYgYJfhJdzMmb1cRXITTOP95Bt7+Ud7GHPlTN/hQbXWG/JNGOr5VHimD2i3OZyzN 2lgv3r3+m3thm+be5EHYKVbWq8yG8EIMCrq/LVO05d5lgDQiHCE8gvnh4LP041nbpBGS ot/QGQ63jUnGhTLS9n0Mkx1cUbOz3jP3FoJZAXvcCu+th2Iq41XY0KzBdZlWD72BQFne qgTnXllP2JMqJ3BEAlwQppX/+Jnsw6KhbPyu1PG67c/Ss2zCS/zEEOJuwMY/gpj+d+y9 1OSg== X-Gm-Message-State: AOAM533J6mrgwY3WSDnKroQmX32MhiOUDbhgQTQx/JHnRkiqB0Hj6tap wxJ84J77nQrYnM2HMsdwI0IXnIHpWttvwA== X-Google-Smtp-Source: ABdhPJyuvYlUDqgcIXkncbzly88yR448rq4SXOJIvG6XLJfTSnnSJ0/p45wJuCXhMPuD5FVgVwKNOw== X-Received: by 2002:adf:e444:: with SMTP id t4mr7326214wrm.97.1613937221094; Sun, 21 Feb 2021 11:53:41 -0800 (PST) Received: from localhost.localdomain (cpc91226-cmbg18-2-0-cust7.5-4.cable.virginm.net. [82.0.29.8]) by smtp.gmail.com with ESMTPSA id z18sm1780372wrs.19.2021.02.21.11.53.40 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 21 Feb 2021 11:53:40 -0800 (PST) From: Mark Thompson To: ffmpeg-devel@ffmpeg.org Date: Sun, 21 Feb 2021 19:51:23 +0000 Message-Id: <20210221195125.1901683-16-sw@jkqxz.net> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20210221195125.1901683-1-sw@jkqxz.net> References: <20210221195125.1901683-1-sw@jkqxz.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 16/18] vaapi_encode_h265: Ensure that HDR metadata is included with seek points X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Save the most recent HDR side data structures and use them when generating an IDR or CRA frame. --- libavcodec/vaapi_encode_h265.c | 93 ++++++++++++---------------------- 1 file changed, 32 insertions(+), 61 deletions(-) diff --git a/libavcodec/vaapi_encode_h265.c b/libavcodec/vaapi_encode_h265.c index 33acbf23f2..75ebb63e74 100644 --- a/libavcodec/vaapi_encode_h265.c +++ b/libavcodec/vaapi_encode_h265.c @@ -30,6 +30,7 @@ #include "avcodec.h" #include "cbs.h" #include "cbs_h265.h" +#include "cbs_metadata.h" #include "h265_profile_level.h" #include "hevc.h" #include "hevc_sei.h" @@ -70,13 +71,15 @@ typedef struct VAAPIEncodeH265Context { H265RawPPS raw_pps; H265RawSlice raw_slice; - SEIRawMasteringDisplayColourVolume sei_mastering_display; - SEIRawContentLightLevelInfo sei_content_light_level; - CodedBitstreamContext *cbc; CodedBitstreamFragment current_access_unit; int aud_needed; int sei_needed; + + int have_mastering_display; + AVMasteringDisplayMetadata mastering_display; + int have_content_light_level; + AVContentLightMetadata content_light_level; } VAAPIEncodeH265Context; @@ -200,17 +203,17 @@ static int vaapi_encode_h265_write_extra_header(AVCodecContext *avctx, } if (priv->sei_needed & SEI_MASTERING_DISPLAY) { - err = ff_cbs_sei_add_message(priv->cbc, au, 1, - SEI_TYPE_MASTERING_DISPLAY_COLOUR_VOLUME, - &priv->sei_mastering_display, NULL); + err = ff_cbs_insert_metadata(priv->cbc, au, + CBS_METADATA_MASTERING_DISPLAY, + &priv->mastering_display); if (err < 0) goto fail; } if (priv->sei_needed & SEI_CONTENT_LIGHT_LEVEL) { - err = ff_cbs_sei_add_message(priv->cbc, au, 1, - SEI_TYPE_CONTENT_LIGHT_LEVEL_INFO, - &priv->sei_content_light_level, NULL); + err = ff_cbs_insert_metadata(priv->cbc, au, + CBS_METADATA_CONTENT_LIGHT_LEVEL, + &priv->content_light_level); if (err < 0) goto fail; } @@ -746,72 +749,40 @@ static int vaapi_encode_h265_init_picture_params(AVCodecContext *avctx, priv->sei_needed = 0; - // Only look for the metadata on I/IDR frame on the output. We - // may force an IDR frame on the output where the medadata gets - // changed on the input frame. - if ((priv->sei & SEI_MASTERING_DISPLAY) && - (pic->type == PICTURE_TYPE_I || pic->type == PICTURE_TYPE_IDR)) { + // We check for new metadata on every frame, but only include the + // SEI with IRAP frames. + if (priv->sei & SEI_MASTERING_DISPLAY) { AVFrameSideData *sd = av_frame_get_side_data(pic->input_image, AV_FRAME_DATA_MASTERING_DISPLAY_METADATA); if (sd) { - AVMasteringDisplayMetadata *mdm = - (AVMasteringDisplayMetadata *)sd->data; - - // SEI is needed when both the primaries and luminance are set - if (mdm->has_primaries && mdm->has_luminance) { - SEIRawMasteringDisplayColourVolume *mdcv = - &priv->sei_mastering_display; - const int mapping[3] = {1, 2, 0}; - const int chroma_den = 50000; - const int luma_den = 10000; - - for (i = 0; i < 3; i++) { - const int j = mapping[i]; - mdcv->display_primaries_x[i] = - FFMIN(lrint(chroma_den * - av_q2d(mdm->display_primaries[j][0])), - chroma_den); - mdcv->display_primaries_y[i] = - FFMIN(lrint(chroma_den * - av_q2d(mdm->display_primaries[j][1])), - chroma_den); - } - - mdcv->white_point_x = - FFMIN(lrint(chroma_den * av_q2d(mdm->white_point[0])), - chroma_den); - mdcv->white_point_y = - FFMIN(lrint(chroma_den * av_q2d(mdm->white_point[1])), - chroma_den); - - mdcv->max_display_mastering_luminance = - lrint(luma_den * av_q2d(mdm->max_luminance)); - mdcv->min_display_mastering_luminance = - FFMIN(lrint(luma_den * av_q2d(mdm->min_luminance)), - mdcv->max_display_mastering_luminance); - - priv->sei_needed |= SEI_MASTERING_DISPLAY; - } + av_assert0(sd->size >= sizeof(priv->mastering_display)); + priv->have_mastering_display = 1; + memcpy(&priv->mastering_display, sd->data, + sizeof(priv->mastering_display)); + } + + if (priv->have_mastering_display && + (pic->type == PICTURE_TYPE_IDR || pic->type == PICTURE_TYPE_I)) { + priv->sei_needed |= SEI_MASTERING_DISPLAY; } } - if ((priv->sei & SEI_CONTENT_LIGHT_LEVEL) && - (pic->type == PICTURE_TYPE_I || pic->type == PICTURE_TYPE_IDR)) { + if (priv->sei & SEI_CONTENT_LIGHT_LEVEL) { AVFrameSideData *sd = av_frame_get_side_data(pic->input_image, AV_FRAME_DATA_CONTENT_LIGHT_LEVEL); if (sd) { - AVContentLightMetadata *clm = - (AVContentLightMetadata *)sd->data; - SEIRawContentLightLevelInfo *clli = - &priv->sei_content_light_level; - - clli->max_content_light_level = FFMIN(clm->MaxCLL, 65535); - clli->max_pic_average_light_level = FFMIN(clm->MaxFALL, 65535); + av_assert0(sd->size >= sizeof(priv->content_light_level)); + priv->have_mastering_display = 1; + memcpy(&priv->content_light_level, sd->data, + sizeof(priv->content_light_level)); + } + if (priv->have_content_light_level && + (pic->type == PICTURE_TYPE_IDR || pic->type == PICTURE_TYPE_I)) { priv->sei_needed |= SEI_CONTENT_LIGHT_LEVEL; } } From patchwork Sun Feb 21 19:51:24 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Thompson X-Patchwork-Id: 25880 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 872EE44A325 for ; Sun, 21 Feb 2021 21:56:39 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 58D4F68A8BD; Sun, 21 Feb 2021 21:53:56 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wm1-f41.google.com (mail-wm1-f41.google.com [209.85.128.41]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 539B768A8F5 for ; Sun, 21 Feb 2021 21:53:42 +0200 (EET) Received: by mail-wm1-f41.google.com with SMTP id i9so1692360wml.5 for ; Sun, 21 Feb 2021 11:53:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jkqxz-net.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=hJlnmK7zf8lnNs9QOEPH58O5wS14BH0Fy3gG75fpKEo=; b=tFTF0ZLJusJJ4aIp6p0Qq6bh7cPG50AWvQx/MY06+zTmQUXnse8iFmHgHeog7sIY4M 37phD7KpKgxoMI5J05k7zyq96w8FULmN1fjHQ6ea2AlQAu6NPSrBuNZgxPgRr5kaEG0x XbBOMVgss8Ert1A2ZbAMKe+Y1qMaXFiK5xGg6gH44W2lcfU3mSv8+X2SsJIbZEVKs45O gwMOZv2gn6feMs6bUa8JU4IvRKRKogNMX/3l0akUNhJcYlwVBCMP6VD3pVXFyr+cbRsR JDO3UtE8UFVN6vnIWVDV4Eeqh65kFO/uy80cDTFf7ml1sMW+H0RQkexO0r/SwFusAgZe zlrA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=hJlnmK7zf8lnNs9QOEPH58O5wS14BH0Fy3gG75fpKEo=; b=mBnGbmPqlNwI94mI6x29s7WUJFongKwplajocKS/dOZmHqO7FMXGPEnmVpju2SK4eN Dq2xgN1brBs+MTOuQPJiJfE6aPmcR3jK48bEJf4HGzi/7vsDOCtJGxpZ64ddDqGa9df5 ih65cdtZoxDvW19SJFucj25QVSnyp0Iwj7oup6Olo0yRXZrMkEAsy0Z+y8tiaUztjJwc EYKmcOxEQKy7eq1uByOkg+PbH/eod5dlr3vdGOCF2/wM709rjWn5p3Pd3sHtpwpuuH4Q LCE5yEE2IZ8oIds6bNYGSDnIp7ga5cSnY4WQPm/up+2AYNyv5MSZUIOWo727le6WoRBe H4Kw== X-Gm-Message-State: AOAM531kIPVbTlzKaT+84oosnPNjS5FxRWAOjNtZFjD4hSe2xLs5XRDz 4G2CLYQRpOuBT4kZJvOwbrMXSAhQ+Ok+YA== X-Google-Smtp-Source: ABdhPJyaA/VHKie/1NFR8ZI4aasjhlXRRvR/+C2DlXItmTqsheN+UPCiGJ2N9jfi0oJrVZC4Hhs8mw== X-Received: by 2002:a1c:a985:: with SMTP id s127mr11368743wme.158.1613937221828; Sun, 21 Feb 2021 11:53:41 -0800 (PST) Received: from localhost.localdomain (cpc91226-cmbg18-2-0-cust7.5-4.cable.virginm.net. [82.0.29.8]) by smtp.gmail.com with ESMTPSA id z18sm1780372wrs.19.2021.02.21.11.53.41 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 21 Feb 2021 11:53:41 -0800 (PST) From: Mark Thompson To: ffmpeg-devel@ffmpeg.org Date: Sun, 21 Feb 2021 19:51:24 +0000 Message-Id: <20210221195125.1901683-17-sw@jkqxz.net> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20210221195125.1901683-1-sw@jkqxz.net> References: <20210221195125.1901683-1-sw@jkqxz.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 17/18] vaapi_encode_h264: Support HDR metadata X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" This can now be done in exactly the same way as it is for H.265. --- libavcodec/vaapi_encode_h264.c | 66 +++++++++++++++++++++++++++++++++- 1 file changed, 65 insertions(+), 1 deletion(-) diff --git a/libavcodec/vaapi_encode_h264.c b/libavcodec/vaapi_encode_h264.c index b546ddec08..582a855586 100644 --- a/libavcodec/vaapi_encode_h264.c +++ b/libavcodec/vaapi_encode_h264.c @@ -24,11 +24,13 @@ #include "libavutil/avassert.h" #include "libavutil/common.h" #include "libavutil/internal.h" +#include "libavutil/mastering_display_metadata.h" #include "libavutil/opt.h" #include "avcodec.h" #include "cbs.h" #include "cbs_h264.h" +#include "cbs_metadata.h" #include "h264.h" #include "h264_levels.h" #include "h264_sei.h" @@ -95,6 +97,11 @@ typedef struct VAAPIEncodeH264Context { int aud_needed; int sei_needed; int sei_cbr_workaround_needed; + + int have_mastering_display; + AVMasteringDisplayMetadata mastering_display; + int have_content_light_level; + AVContentLightMetadata content_light_level; } VAAPIEncodeH264Context; @@ -241,6 +248,20 @@ static int vaapi_encode_h264_write_extra_header(AVCodecContext *avctx, if (err < 0) goto fail; } + if (priv->sei_needed & SEI_MASTERING_DISPLAY) { + err = ff_cbs_insert_metadata(priv->cbc, au, + CBS_METADATA_MASTERING_DISPLAY, + &priv->mastering_display); + if (err < 0) + goto fail; + } + if (priv->sei_needed & SEI_CONTENT_LIGHT_LEVEL) { + err = ff_cbs_insert_metadata(priv->cbc, au, + CBS_METADATA_CONTENT_LIGHT_LEVEL, + &priv->content_light_level); + if (err < 0) + goto fail; + } priv->sei_needed = 0; @@ -674,6 +695,42 @@ static int vaapi_encode_h264_init_picture_params(AVCodecContext *avctx, priv->sei_needed |= SEI_RECOVERY_POINT; } + if (priv->sei & SEI_MASTERING_DISPLAY) { + AVFrameSideData *sd = + av_frame_get_side_data(pic->input_image, + AV_FRAME_DATA_MASTERING_DISPLAY_METADATA); + + if (sd) { + av_assert0(sd->size >= sizeof(priv->mastering_display)); + priv->have_mastering_display = 1; + memcpy(&priv->mastering_display, sd->data, + sizeof(priv->mastering_display)); + } + + if (priv->have_mastering_display && + (pic->type == PICTURE_TYPE_IDR || pic->type == PICTURE_TYPE_I)) { + priv->sei_needed |= SEI_MASTERING_DISPLAY; + } + } + + if (priv->sei & SEI_CONTENT_LIGHT_LEVEL) { + AVFrameSideData *sd = + av_frame_get_side_data(pic->input_image, + AV_FRAME_DATA_CONTENT_LIGHT_LEVEL); + + if (sd) { + av_assert0(sd->size >= sizeof(priv->content_light_level)); + priv->have_mastering_display = 1; + memcpy(&priv->content_light_level, sd->data, + sizeof(priv->content_light_level)); + } + + if (priv->have_content_light_level && + (pic->type == PICTURE_TYPE_IDR || pic->type == PICTURE_TYPE_I)) { + priv->sei_needed |= SEI_CONTENT_LIGHT_LEVEL; + } + } + vpic->CurrPic = (VAPictureH264) { .picture_id = pic->recon_surface, .frame_idx = hpic->frame_num, @@ -1245,7 +1302,8 @@ static const AVOption vaapi_encode_h264_options[] = { { "sei", "Set SEI to include", OFFSET(sei), AV_OPT_TYPE_FLAGS, - { .i64 = SEI_IDENTIFIER | SEI_TIMING | SEI_RECOVERY_POINT }, + { .i64 = SEI_IDENTIFIER | SEI_TIMING | SEI_RECOVERY_POINT | + SEI_MASTERING_DISPLAY | SEI_CONTENT_LIGHT_LEVEL }, 0, INT_MAX, FLAGS, "sei" }, { "identifier", "Include encoder version identifier", 0, AV_OPT_TYPE_CONST, { .i64 = SEI_IDENTIFIER }, @@ -1256,6 +1314,12 @@ static const AVOption vaapi_encode_h264_options[] = { { "recovery_point", "Include recovery points where appropriate", 0, AV_OPT_TYPE_CONST, { .i64 = SEI_RECOVERY_POINT }, INT_MIN, INT_MAX, FLAGS, "sei" }, + { "hdr", + "Include HDR metadata (mastering display colour volume " + "and content light level information)", + 0, AV_OPT_TYPE_CONST, + { .i64 = SEI_MASTERING_DISPLAY | SEI_CONTENT_LIGHT_LEVEL }, + INT_MIN, INT_MAX, FLAGS, "sei" }, { "profile", "Set profile (profile_idc and constraint_set*_flag)", OFFSET(profile), AV_OPT_TYPE_INT, From patchwork Sun Feb 21 19:51:25 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Thompson X-Patchwork-Id: 25877 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id C6E9A449DB5 for ; Sun, 21 Feb 2021 21:53:53 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id B26F568A9E1; Sun, 21 Feb 2021 21:53:53 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wr1-f43.google.com (mail-wr1-f43.google.com [209.85.221.43]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 1946468A8A5 for ; Sun, 21 Feb 2021 21:53:43 +0200 (EET) Received: by mail-wr1-f43.google.com with SMTP id v14so16892805wro.7 for ; Sun, 21 Feb 2021 11:53:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jkqxz-net.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=4yrxjQQ1rGzsgdtKtUvOcH/RVKvdTT0/G5dh5P6NCAQ=; b=vwTKeqad+PagEBOJwgygPfN+h3mHvthC2nShWxPFlLCBH6CcRM50DFjGeOB0/r8ro5 et7DvHCPAfkoEYTBlOiIPUY+xhE1gVXDf/YIlFxYKyfZ8lr9xu59xOu7Ia7cqckOCIVz eF5fW6hCu7Dkk9Dcaq4GNQjmnNwO/bLoQp6aHT9pWGDvO7BEoYcgckiviYgA/MIUuDg/ mE2FAaQ1bT4x1YVGh2RzNRRaZBcgzzJqtKCrMgMtKRDSYTgcIep/VHpYW2YJ5wtObSif suxWW5pj/cG4rOfLXMsH9Sj71Dq8QvjivDUQzDP479VhdT4Q5MLPvXviJlNA/4txd6GR J7Xw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=4yrxjQQ1rGzsgdtKtUvOcH/RVKvdTT0/G5dh5P6NCAQ=; b=r2ZkVi9qXtwiEIkCD8d7I0pNieWWTDzGo00g0OBbc7TcMbIqJdaFoEWkADpFh6UXIT BpS0/kfJDXr04+Ybppqu3QYshwCtwlqztwEcySH3soLPZaSmR0Fq3ahawNX9PnT/C/+W XL1E9StuuCldq2mD2DeE57BjmptACywNt/+UNSjoNIstuPiiKh0MAX58VcuI0valIXzA ekHcIwtbumKh2yhvmzkOHSHx4rb62Plf0KUJkbwPEPTXHnubccD1diL5O6IdrMn5ADcA 4rI8mf6+5BMbALSXLFOuGrL97SExP4kqtv5cN7Z2eRuG/HQ8oBNEHAminmyNxWEEHTz7 pK8A== X-Gm-Message-State: AOAM532kF1uXqdWXh6uDJJ05uNnvLimpNhJwzeJyBo2vTZDKFW18DPkO cAHfaUMLQRWf4JvytzAIeZtvrBk5d9t9GA== X-Google-Smtp-Source: ABdhPJwp9IslA8EPSfzcPEW+WWcfXJUUmL7nZdG8laymHz+CJ15TWdNvYiHQb/eN+GisSdXQy7Q7ew== X-Received: by 2002:a5d:63ce:: with SMTP id c14mr5096814wrw.15.1613937222547; Sun, 21 Feb 2021 11:53:42 -0800 (PST) Received: from localhost.localdomain (cpc91226-cmbg18-2-0-cust7.5-4.cable.virginm.net. [82.0.29.8]) by smtp.gmail.com with ESMTPSA id z18sm1780372wrs.19.2021.02.21.11.53.41 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 21 Feb 2021 11:53:42 -0800 (PST) From: Mark Thompson To: ffmpeg-devel@ffmpeg.org Date: Sun, 21 Feb 2021 19:51:25 +0000 Message-Id: <20210221195125.1901683-18-sw@jkqxz.net> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20210221195125.1901683-1-sw@jkqxz.net> References: <20210221195125.1901683-1-sw@jkqxz.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 18/18] cbs: Add metadata test X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" This tests insert/extract/remove for each supported codec/type combination. --- libavcodec/Makefile | 1 + libavcodec/tests/cbs_metadata.c | 390 ++++++++++++++++++++++++++++++++ tests/fate/libavcodec.mak | 5 + 3 files changed, 396 insertions(+) create mode 100644 libavcodec/tests/cbs_metadata.c diff --git a/libavcodec/Makefile b/libavcodec/Makefile index 43a54caba8..9133afcd0e 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -1225,6 +1225,7 @@ TESTPROGS = avpacket \ utils \ TESTPROGS-$(CONFIG_CABAC) += cabac +TESTPROGS-$(CONFIG_CBS) += cbs_metadata TESTPROGS-$(CONFIG_DCT) += avfft TESTPROGS-$(CONFIG_FFT) += fft fft-fixed32 TESTPROGS-$(CONFIG_GOLOMB) += golomb diff --git a/libavcodec/tests/cbs_metadata.c b/libavcodec/tests/cbs_metadata.c new file mode 100644 index 0000000000..260a32ce7a --- /dev/null +++ b/libavcodec/tests/cbs_metadata.c @@ -0,0 +1,390 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "config.h" + +#include "libavutil/mastering_display_metadata.h" +#include "libavutil/stereo3d.h" + +#include "libavcodec/cbs.h" +#include "libavcodec/cbs_metadata.h" + +typedef struct MetadataTestCodec { + enum AVCodecID codec_id; + enum CBSMetadataType *types; + const uint8_t *input; + size_t input_size; +} MetadataTestCodec; + +typedef struct MetadataTestType { + enum CBSMetadataType type; + const void *input; + int (*compare)(const void *input, const void *output); +} MetadataTestType; + +// These test inputs are a single frame of 64x64 black. + +static uint8_t h264_test_stream[] = { + 0x00, 0x00, 0x00, 0x01, 0x67, 0x64, 0x00, 0x0a, 0xac, 0xd9, 0x44, 0x26, + 0xc0, 0x44, 0x00, 0x00, 0x03, 0x00, 0x04, 0x00, 0x00, 0x03, 0x00, 0xc8, + 0x3c, 0x48, 0x96, 0x58, 0x00, 0x00, 0x00, 0x01, 0x68, 0xeb, 0xe3, 0xcb, + 0x22, 0xc0, 0x00, 0x00, 0x01, 0x65, 0x88, 0x84, 0x00, 0x2b, 0xff, 0xfe, + 0xd8, 0xe7, 0xf3, 0x2c, 0xa7, 0xf4, 0xda, 0xbb, 0xf0, 0xac, 0xc4, 0x99, + 0x8a, 0xa5, 0xc3, 0xab, 0x2f, 0x6b, 0xe0, 0x64, 0x26, 0xdd, 0x1d, 0x09, + 0x60, 0xb8, 0x0d, 0x60, 0xf6, 0x89 +}; + +static uint8_t h265_test_stream[] = { + 0x00, 0x00, 0x00, 0x01, 0x40, 0x01, 0x0c, 0x01, 0xff, 0xff, 0x01, 0x60, + 0x00, 0x00, 0x03, 0x00, 0x90, 0x00, 0x00, 0x03, 0x00, 0x00, 0x03, 0x00, + 0x1e, 0x95, 0x98, 0x09, 0x00, 0x00, 0x00, 0x01, 0x42, 0x01, 0x01, 0x01, + 0x60, 0x00, 0x00, 0x03, 0x00, 0x90, 0x00, 0x00, 0x03, 0x00, 0x00, 0x03, + 0x00, 0x1e, 0xa0, 0x20, 0x81, 0x05, 0x96, 0x56, 0x69, 0x24, 0xca, 0xf0, + 0x16, 0x80, 0x80, 0x00, 0x00, 0x03, 0x00, 0x80, 0x00, 0x00, 0x0c, 0x84, + 0x00, 0x00, 0x00, 0x01, 0x44, 0x01, 0xc1, 0x72, 0xb4, 0x22, 0x40, 0x00, + 0x00, 0x01, 0x28, 0x01, 0xaf, 0x1d, 0x80, 0xf7, 0xcf, 0x80, 0xff, 0xf8, + 0x90, 0xfa, 0x3b, 0x77, 0x87, 0x96, 0x96, 0xbc, 0x7c +}; + +static uint8_t av1_test_stream[] = { + 0x12, 0x00, 0x0a, 0x0a, + 0x00, 0x00, 0x00, 0x02, 0xaf, 0xff, 0x89, 0x5f, 0x20, 0x08, 0x32, 0x16, + 0x10, 0x00, 0xa6, 0x80, 0x10, 0x40, 0x82, 0x08, 0x00, 0x00, 0x4b, 0xb7, + 0x9a, 0x9b, 0x57, 0x72, 0xea, 0xe9, 0x28, 0xb8, 0x15, 0x30 +}; + +static const MetadataTestCodec test_codecs[] = { +#if CONFIG_CBS_H264 + { + AV_CODEC_ID_H264, + (enum CBSMetadataType[]) { + CBS_METADATA_MASTERING_DISPLAY, + CBS_METADATA_CONTENT_LIGHT_LEVEL, + CBS_METADATA_DISPLAY_MATRIX, + CBS_METADATA_STEREO3D, + CBS_METADATA_NONE, + }, + h264_test_stream, + sizeof(h264_test_stream), + }, +#endif +#if CONFIG_CBS_H265 + { + AV_CODEC_ID_H265, + (enum CBSMetadataType[]) { + CBS_METADATA_MASTERING_DISPLAY, + CBS_METADATA_CONTENT_LIGHT_LEVEL, + CBS_METADATA_NONE, + }, + h265_test_stream, + sizeof(h265_test_stream), + }, +#endif +#if CONFIG_CBS_AV1 + { + AV_CODEC_ID_AV1, + (enum CBSMetadataType[]) { + CBS_METADATA_MASTERING_DISPLAY, + CBS_METADATA_CONTENT_LIGHT_LEVEL, + CBS_METADATA_NONE, + }, + av1_test_stream, + sizeof(av1_test_stream), + }, +#endif +}; + +#define VALUE_RATIONAL(name) do { \ + AVRational diff = av_sub_q(a->name, b->name); \ + diff.num = FFABS(diff.num); \ + if (av_cmp_q(diff, av_make_q(1, 25000)) >= 0) { \ + av_log(NULL, AV_LOG_ERROR, \ + "%s differs by %d/%d after reading back.\n", \ + #name, diff.num, diff.den); \ + return 1; \ + } \ + } while (0) + +#define VALUE_ENUM(name) do { \ + if (a->name != b->name) { \ + av_log(NULL, AV_LOG_ERROR, \ + "%s differs after reading back.\n", #name); \ + return 1; \ + } \ + } while (0) + +#define VALUE_UINT(name) do { \ + unsigned int diff; \ + if (a->name > b->name) \ + diff = a->name - b->name; \ + else \ + diff = b->name - a->name; \ + if (diff > 1) { \ + av_log(NULL, AV_LOG_ERROR, \ + "%s differs by %d after reading back.\n", \ + #name, diff); \ + return 1; \ + } \ + } while (0) + +static int compare_mastering_display_metadata(const void *va, + const void *vb) +{ + const AVMasteringDisplayMetadata *a = va, *b = vb; + VALUE_RATIONAL(display_primaries[0][0]); + VALUE_RATIONAL(display_primaries[0][1]); + VALUE_RATIONAL(display_primaries[1][0]); + VALUE_RATIONAL(display_primaries[1][1]); + VALUE_RATIONAL(display_primaries[2][0]); + VALUE_RATIONAL(display_primaries[2][1]); + VALUE_RATIONAL(white_point[0]); + VALUE_RATIONAL(white_point[1]); + VALUE_RATIONAL(min_luminance); + VALUE_RATIONAL(max_luminance); + return 0; +} + +static int compare_content_light_level_metadata(const void *va, + const void *vb) +{ + const AVContentLightMetadata *a = va, *b = vb; + VALUE_UINT(MaxCLL); + VALUE_UINT(MaxFALL); + return 0; +} + +static int compare_display_matrix(const void *va, + const void *vb) +{ + const int32_t *a = va, *b = vb; + for (int i = 0; i < 9; i++) { + int diff = FFABS(a[i] - b[i]); + if (diff > 1) { + av_log(NULL, AV_LOG_ERROR, + "Matrix %d differs by %d after reading back.\n", + i, diff); + return 1; + } + } + return 0; +} + +static int compare_stereo3d(const void *va, + const void *vb) +{ + const AVStereo3D *a = va, *b = vb; + VALUE_ENUM(type); + VALUE_ENUM(view); + return 0; +} + +static const MetadataTestType test_types[] = { + { + CBS_METADATA_MASTERING_DISPLAY, + &(AVMasteringDisplayMetadata) { + // This test colour space is BT.2020. + .display_primaries = { + { { 708, 1000 }, { 292, 1000 }, }, + { { 170, 1000 }, { 797, 1000 }, }, + { { 131, 1000 }, { 46, 1000 }, }, + }, + .white_point = { + { 3127, 10000 }, { 3290, 10000 }, + }, + // These are arbitrary numbers in a plausible range. + .min_luminance = { 56, 100 }, + .max_luminance = { 1234, 1 }, + .has_primaries = 1, + .has_luminance = 1, + }, + &compare_mastering_display_metadata, + }, + { + CBS_METADATA_CONTENT_LIGHT_LEVEL, + &(AVContentLightMetadata) { + .MaxCLL = 1234, + .MaxFALL = 567, + }, + &compare_content_light_level_metadata, + }, + { + CBS_METADATA_DISPLAY_MATRIX, + &(int32_t[9]) { + // This is a quarter-turn clockwise. + 0, -1 * (1 << 16), 0, + 1 * (1 << 16), 0, 0, + 0, 0, 1 << 30, + }, + &compare_display_matrix, + }, + { + CBS_METADATA_STEREO3D, + &(AVStereo3D) { + .type = AV_STEREO3D_SIDEBYSIDE, + .flags = 0, + .view = AV_STEREO3D_VIEW_PACKED, + }, + &compare_stereo3d, + }, +}; + +#define CHECK(op) do { \ + if ((op) < 0) { \ + av_log(NULL, AV_LOG_ERROR, "%s failed.\n", #op); \ + return 1; \ + } \ + } while (0) + +int main(void) +{ + // None of these tests care about any persistent header information, + // so we use the same context for all reading/writing. + CodedBitstreamContext *cbc; + CodedBitstreamFragment frag, frag2; + AVPacket *pkt; + int c, t, i; + + memset(&frag, 0, sizeof(frag)); + memset(&frag2, 0, sizeof(frag2)); + + pkt = av_packet_alloc(); + if (!pkt) + return 1; + + for (c = 0; c < FF_ARRAY_ELEMS(test_codecs); c++) { + const MetadataTestCodec *test_codec; + const AVCodecDescriptor *codec_desc; + + test_codec = &test_codecs[c]; + codec_desc = avcodec_descriptor_get(test_codec->codec_id); + if (!codec_desc) { + av_log(NULL, AV_LOG_ERROR, "Invalid codec %d used in test.\n", + test_codec->codec_id); + return 1; + } + + av_log(NULL, AV_LOG_INFO, + "Testing codec %d (%s):\n", + test_codec->codec_id, codec_desc->name); + + CHECK(ff_cbs_init(&cbc, test_codec->codec_id, NULL)); + + // Having tracing enabled on the CBS read/write operations may be + // helpful when debug problems found during this test. + if (0) { + cbc->trace_enable = 1; + cbc->trace_level = AV_LOG_INFO; + } + + CHECK(ff_cbs_read(cbc, &frag, + test_codec->input, test_codec->input_size)); + + // Verify no change. + CHECK(ff_cbs_write_packet(cbc, pkt, &frag)); + if (memcmp(pkt->data, test_codec->input, test_codec->input_size)) { + av_log(NULL, AV_LOG_ERROR, + "Passthrough output does not match.\n"); + return 1; + } + + for (t = 0; t < FF_ARRAY_ELEMS(test_types); t++) { + const MetadataTestType *test_type; + const CBSMetadataTypeDescriptor *type_desc; + void *output; + + test_type = &test_types[t]; + type_desc = ff_cbs_metadata_find_type(test_type->type); + if (!type_desc) { + av_log(NULL, AV_LOG_ERROR, "Invalid metadata type %d " + "used in test.\n", test_type->type); + return 1; + } + + for (i = 0; test_codec->types[i] != CBS_METADATA_NONE; i++) { + if (test_codec->types[i] == test_type->type) + break; + } + if (test_codec->types[i] != test_type->type) { + // Type not supported by this codec. + continue; + } + + av_log(NULL, AV_LOG_INFO, + "Testing metadata type %d (%s) with codec %d (%s).\n", + test_type->type, type_desc->name, + test_codec->codec_id, codec_desc->name); + + // Insert the metadata into the packet. + CHECK(ff_cbs_insert_metadata(cbc, &frag, + test_type->type, + test_type->input)); + + // Write packet containing the new metadata. + CHECK(ff_cbs_write_packet(cbc, pkt, &frag)); + + // Ensure that that actually did something. + if (pkt->size == test_codec->input_size && + memcmp(pkt->data, test_codec->input, + test_codec->input_size) == 0) { + av_log(NULL, AV_LOG_ERROR, + "Output not changed by metadata addition.\n"); + return 1; + } + + // Read packet back into a different fragment. + CHECK(ff_cbs_read_packet(cbc, &frag2, pkt)); + + // Make a new instance of the same structure to extract to. + output = ff_cbs_alloc_metadata(test_type->type, NULL); + if (!output) + return 1; + + // Extract the metadata from the new fragment. + CHECK(ff_cbs_extract_metadata(cbc, &frag2, + test_type->type, output)); + + // Make sure the result is sensible. + if (test_type->compare(test_type->input, output)) { + av_log(NULL, AV_LOG_ERROR, + "Output metadata does not match input.\n"); + return 1; + } + + av_free(output); + ff_cbs_fragment_free(&frag2); + + // Remove the metadata from the fragment again. + CHECK(ff_cbs_remove_metadata(cbc, &frag, test_type->type)); + + // Write it back to make sure it's been removed. + CHECK(ff_cbs_write_packet(cbc, pkt, &frag)); + if (pkt->size != test_codec->input_size || + memcmp(pkt->data, test_codec->input, + test_codec->input_size)) { + av_log(NULL, AV_LOG_ERROR, + "Output changed after adding/removing metadata.\n"); + return 1; + } + } + + ff_cbs_fragment_free(&frag); + ff_cbs_close(&cbc); + } + + return 0; +} diff --git a/tests/fate/libavcodec.mak b/tests/fate/libavcodec.mak index 747dae3704..e303e46b48 100644 --- a/tests/fate/libavcodec.mak +++ b/tests/fate/libavcodec.mak @@ -8,6 +8,11 @@ fate-cabac: libavcodec/tests/cabac$(EXESUF) fate-cabac: CMD = run libavcodec/tests/cabac$(EXESUF) fate-cabac: CMP = null +FATE_LIBAVCODEC-$(CONFIG_CBS) += fate-cbs-metadata +fate-cbs-metadata: libavcodec/tests/cbs_metadata$(EXESUF) +fate-cbs-metadata: CMD = run libavcodec/tests/cbs_metadata$(EXESUF) +fate-cbs-metadata: CMP = null + FATE_LIBAVCODEC-yes += fate-celp_math fate-celp_math: libavcodec/tests/celp_math$(EXESUF) fate-celp_math: CMD = run libavcodec/tests/celp_math$(EXESUF)