From patchwork Sun Feb 23 23:41:04 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Thompson X-Patchwork-Id: 17881 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id ECA7C448772 for ; Mon, 24 Feb 2020 01:46:55 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id C819468B4D6; Mon, 24 Feb 2020 01:46:55 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wr1-f66.google.com (mail-wr1-f66.google.com [209.85.221.66]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id EA2D968B063 for ; Mon, 24 Feb 2020 01:46:48 +0200 (EET) Received: by mail-wr1-f66.google.com with SMTP id g3so8237728wrs.12 for ; Sun, 23 Feb 2020 15:46:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jkqxz-net.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:mime-version :content-transfer-encoding; bh=ShtsuA9rA/mXfe+ihpMAvBV6uzuw3Q+IpcWIL5fCQeE=; b=kMhq+EN0G7OIEhGqoXEf4d1WHDOefx/pfP8YdEs9lvSBbFzTWpkidMzoZxiub/QqV+ m7LrDYdGNShdKpaI5WTwt/YAV2Wgm6hrXxoSZ+Srf2bHssuioHE6akoxRLOp4yM6WhXR EiXsIQDMoLZiX3lplSfmV10VDPomJg39ZT5z8LCgOJOAwhRRzw+g7EPqitPW0+GzLOjZ 7gPhdffdpmYlgPAy4YmUe08sdUPe517+bmcXhUtESa69QI1H8+ENWU9ZzZcCZHWEG9/L J026SxXW3f0AwYir/WBsmG3YKtT8ZobYKpysRNWGIFG3RaTgvFz3mttkoHhFVH+9tVwR pqRg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:mime-version :content-transfer-encoding; bh=ShtsuA9rA/mXfe+ihpMAvBV6uzuw3Q+IpcWIL5fCQeE=; b=n3L0LdHyLsnui1Z1+L+5Yj2D1hEOW1PlSkTF9Xlkq2z7AiuHc4XfdtkcDEkI8NYah/ FYkx+nxMqyCqp/NVF23eUMxGt573MpH49TGKHJzh/WJXLqXTdDmek8zxID+mgceSjGD3 Bt+S8NoTGJybyjAgWzJq6f4cIXiOcm/Iokb9RJ+JJbC/1T5K36yb97/5SxXmj0Ur2E5U KBCJVFEUTyoGcBq3uwI/MpMn6ljQo6vjCIeeQ/Pplf20ErCkim2+dvNZvfZytADqxtRI J92mRnmd/p+AbZiRwPTdlHvcGIgyr+IhME3AbWO8W5rPLQt4KhfLOmXi5lOZyKTmaMs9 26WQ== X-Gm-Message-State: APjAAAWKX1DSPaJZxkyrbFkIzI88MpEufynS8sOJ1WGyqYdsgZeTtTE1 vHI1VYWiHdHpApRW9/kMOXal9vdlPMk= X-Google-Smtp-Source: APXvYqxruVHWKL3So7qiVec8Fpqhf6T0BkBhv7wytTpg63YbOhHQwT/uFMAhUL/ucolMmcpL7J9ofA== X-Received: by 2002:a5d:5044:: with SMTP id h4mr59540620wrt.4.1582501294501; Sun, 23 Feb 2020 15:41:34 -0800 (PST) Received: from rywe.jkqxz.net (cpc91242-cmbg18-2-0-cust650.5-4.cable.virginm.net. [82.8.130.139]) by smtp.gmail.com with ESMTPSA id y131sm16358070wmc.13.2020.02.23.15.41.33 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 23 Feb 2020 15:41:33 -0800 (PST) From: Mark Thompson To: ffmpeg-devel@ffmpeg.org Date: Sun, 23 Feb 2020 23:41:04 +0000 Message-Id: <20200223234124.17689-1-sw@jkqxz.net> X-Mailer: git-send-email 2.25.0 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v4 01/21] cbs: Mention all codecs in unit type comment X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" --- libavcodec/cbs.h | 2 ++ 1 file changed, 2 insertions(+) diff --git a/libavcodec/cbs.h b/libavcodec/cbs.h index 9ca1fbd609..cb3081e2c6 100644 --- a/libavcodec/cbs.h +++ b/libavcodec/cbs.h @@ -45,8 +45,10 @@ struct CodedBitstreamType; /** * The codec-specific type of a bitstream unit. * + * AV1: obu_type * H.264 / AVC: nal_unit_type * H.265 / HEVC: nal_unit_type + * JPEG: marker value (without 0xff prefix) * MPEG-2: start code value (without prefix) * VP9: unused, set to zero (every unit is a frame) */ From patchwork Sun Feb 23 23:41:05 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Thompson X-Patchwork-Id: 17895 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 2738A44A08D for ; Mon, 24 Feb 2020 01:49:53 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 0D3D468B55B; Mon, 24 Feb 2020 01:49:53 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wm1-f49.google.com (mail-wm1-f49.google.com [209.85.128.49]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id A2D9168B549 for ; Mon, 24 Feb 2020 01:49:50 +0200 (EET) Received: by mail-wm1-f49.google.com with SMTP id q9so7213233wmj.5 for ; Sun, 23 Feb 2020 15:49:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jkqxz-net.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=Ov3NY4ZfG7aUysRX/D/uc8edoHZOgxV2tVuYEZBl+Xo=; b=sX67560R9VREk9nDw25XCFhSUn7/A1ECAMq6c9AUsXQLXCxHOmqvjXk/FhkrRhYUlY mcHrQoDRZIsvF+lSMZITdOpXU7PUsmy4YuL8h6nbMRNnDXeweTsJgq0/llSEVyqVqjDT U+cMtwwH0vGDAnuHbHWmakKCMbt4H+HATzK+96zVOYyVko7wYytPlrVUSZDoIpcuzPgy IwV+3niRXCN26TFOEMGuGajvuMXKIEb47mvqbycWVxfMAOyGfd2cxlcDOlcl4Sc9PAG+ hH0lClgdtYGOAUUDgGFOiDHVRSAjNIypaHc1qQe8Kk+riT9P21s6vbVUuafOftf9WwoA qwgg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Ov3NY4ZfG7aUysRX/D/uc8edoHZOgxV2tVuYEZBl+Xo=; b=jaG+9F7nPrprraLERNgd+lDY2+FbhlHr4DO1jqu/O3mG2ATSc18chLFdpu8r4gFRCA 5gH7bGEkrJX3uQnmlakhGFV0qhLRdkq5ZOxjHT2kU35VsqJ4kiJB4ecGgL5S6WdobWKB iGFxHO1JCRjgGWyPQGmNPldsgbwTlSHtP7dasDykTRWQi/bQb51pphXolDMRbeNUfxyp VL2JOP+VUhOKx2OgeMdRfW8oOJUzZn8Tt4EoadX2cHjfvOP050IrDpAorHjx+humasAY 7lnu6BCEhn/pWzBUryRLHQCZllVq0rYihYOer3LjeglZ+qZpx54571MAtHwQL/YeLN3R y4AA== X-Gm-Message-State: APjAAAUTBqyQnJDraNj/I/famg9y12nhfUzqZh753lzHhQ077ahosrEH DVhFd/+Ki/9SdEeLmAqR7QizyTQUCGQ= X-Google-Smtp-Source: APXvYqwT33WuTQQKY7sVE3F5gsuf/XA0uyimpN3ayaInUryz4SRqbvM1DwbS+eG9VZdx8Ko1cbrz9Q== X-Received: by 2002:a1c:6588:: with SMTP id z130mr17977602wmb.0.1582501295411; Sun, 23 Feb 2020 15:41:35 -0800 (PST) Received: from rywe.jkqxz.net (cpc91242-cmbg18-2-0-cust650.5-4.cable.virginm.net. [82.8.130.139]) by smtp.gmail.com with ESMTPSA id y131sm16358070wmc.13.2020.02.23.15.41.34 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 23 Feb 2020 15:41:34 -0800 (PST) From: Mark Thompson To: ffmpeg-devel@ffmpeg.org Date: Sun, 23 Feb 2020 23:41:05 +0000 Message-Id: <20200223234124.17689-2-sw@jkqxz.net> X-Mailer: git-send-email 2.25.0 In-Reply-To: <20200223234124.17689-1-sw@jkqxz.net> References: <20200223234124.17689-1-sw@jkqxz.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v4 02/21] cbs: Ensure that reference fields always follow the associated pointer X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Hvaing these together allows us to find both pointers given the address of only one of them. --- libavcodec/cbs_av1.h | 6 +++--- libavcodec/cbs_h264.h | 18 +++++++++--------- libavcodec/cbs_h265.h | 16 ++++++++-------- libavcodec/cbs_jpeg.h | 2 +- libavcodec/cbs_mpeg2.h | 10 +++++----- libavcodec/cbs_vp9.h | 2 +- 6 files changed, 27 insertions(+), 27 deletions(-) diff --git a/libavcodec/cbs_av1.h b/libavcodec/cbs_av1.h index fdc629b00c..8925c45772 100644 --- a/libavcodec/cbs_av1.h +++ b/libavcodec/cbs_av1.h @@ -284,8 +284,8 @@ typedef struct AV1RawFrameHeader { typedef struct AV1RawTileData { uint8_t *data; - size_t data_size; AVBufferRef *data_ref; + size_t data_size; } AV1RawTileData; typedef struct AV1RawTileGroup { @@ -346,8 +346,8 @@ typedef struct AV1RawMetadataITUTT35 { uint8_t itu_t_t35_country_code_extension_byte; uint8_t *payload; - size_t payload_size; AVBufferRef *payload_ref; + size_t payload_size; } AV1RawMetadataITUTT35; typedef struct AV1RawMetadataTimecode { @@ -379,8 +379,8 @@ typedef struct AV1RawMetadata { typedef struct AV1RawPadding { uint8_t *payload; - size_t payload_size; AVBufferRef *payload_ref; + size_t payload_size; } AV1RawPadding; diff --git a/libavcodec/cbs_h264.h b/libavcodec/cbs_h264.h index 9f7c2a0d30..65659ae52c 100644 --- a/libavcodec/cbs_h264.h +++ b/libavcodec/cbs_h264.h @@ -277,16 +277,16 @@ typedef struct H264RawSEIPanScanRect { typedef struct H264RawSEIUserDataRegistered { uint8_t itu_t_t35_country_code; uint8_t itu_t_t35_country_code_extension_byte; - uint8_t *data; - size_t data_length; + uint8_t *data; AVBufferRef *data_ref; + size_t data_length; } H264RawSEIUserDataRegistered; typedef struct H264RawSEIUserDataUnregistered { uint8_t uuid_iso_iec_11578[16]; - uint8_t *data; - size_t data_length; + uint8_t *data; AVBufferRef *data_ref; + size_t data_length; } H264RawSEIUserDataUnregistered; typedef struct H264RawSEIRecoveryPoint { @@ -334,9 +334,9 @@ typedef struct H264RawSEIPayload { H264RawSEIAlternativeTransferCharacteristics alternative_transfer_characteristics; struct { - uint8_t *data; - size_t data_length; + uint8_t *data; AVBufferRef *data_ref; + size_t data_length; } other; } payload; } H264RawSEIPayload; @@ -429,10 +429,10 @@ typedef struct H264RawSliceHeader { typedef struct H264RawSlice { H264RawSliceHeader header; - uint8_t *data; - size_t data_size; - int data_bit_start; + uint8_t *data; AVBufferRef *data_ref; + size_t data_size; + int data_bit_start; } H264RawSlice; typedef struct H264RawFiller { diff --git a/libavcodec/cbs_h265.h b/libavcodec/cbs_h265.h index ad746bf35f..f5eb5af5b2 100644 --- a/libavcodec/cbs_h265.h +++ b/libavcodec/cbs_h265.h @@ -184,8 +184,8 @@ typedef struct H265RawVUI { typedef struct H265RawPSExtensionData { uint8_t *data; - size_t bit_length; AVBufferRef *data_ref; + size_t bit_length; } H265RawPSExtensionData; typedef struct H265RawVPS { @@ -541,10 +541,10 @@ typedef struct H265RawSliceHeader { typedef struct H265RawSlice { H265RawSliceHeader header; - uint8_t *data; - size_t data_size; - int data_bit_start; + uint8_t *data; AVBufferRef *data_ref; + size_t data_size; + int data_bit_start; } H265RawSlice; @@ -600,15 +600,15 @@ typedef struct H265RawSEIUserDataRegistered { uint8_t itu_t_t35_country_code; uint8_t itu_t_t35_country_code_extension_byte; uint8_t *data; - size_t data_length; AVBufferRef *data_ref; + size_t data_length; } H265RawSEIUserDataRegistered; typedef struct H265RawSEIUserDataUnregistered { uint8_t uuid_iso_iec_11578[16]; uint8_t *data; - size_t data_length; AVBufferRef *data_ref; + size_t data_length; } H265RawSEIUserDataUnregistered; typedef struct H265RawSEIRecoveryPoint { @@ -710,9 +710,9 @@ typedef struct H265RawSEIPayload { alternative_transfer_characteristics; H265RawSEIAlphaChannelInfo alpha_channel_info; struct { - uint8_t *data; - size_t data_length; + uint8_t *data; AVBufferRef *data_ref; + size_t data_length; } other; } payload; } H265RawSEIPayload; diff --git a/libavcodec/cbs_jpeg.h b/libavcodec/cbs_jpeg.h index ff1961106f..6305f0ee86 100644 --- a/libavcodec/cbs_jpeg.h +++ b/libavcodec/cbs_jpeg.h @@ -80,8 +80,8 @@ typedef struct JPEGRawScanHeader { typedef struct JPEGRawScan { JPEGRawScanHeader header; uint8_t *data; - size_t data_size; AVBufferRef *data_ref; + size_t data_size; } JPEGRawScan; typedef struct JPEGRawQuantisationTable { diff --git a/libavcodec/cbs_mpeg2.h b/libavcodec/cbs_mpeg2.h index 5bcafd09f0..f7075a460d 100644 --- a/libavcodec/cbs_mpeg2.h +++ b/libavcodec/cbs_mpeg2.h @@ -76,9 +76,9 @@ typedef struct MPEG2RawSequenceHeader { typedef struct MPEG2RawUserData { uint8_t user_data_start_code; - uint8_t *user_data; - size_t user_data_length; + uint8_t *user_data; AVBufferRef *user_data_ref; + size_t user_data_length; } MPEG2RawUserData; typedef struct MPEG2RawSequenceExtension { @@ -206,10 +206,10 @@ typedef struct MPEG2RawSliceHeader { typedef struct MPEG2RawSlice { MPEG2RawSliceHeader header; - uint8_t *data; - size_t data_size; - int data_bit_start; + uint8_t *data; AVBufferRef *data_ref; + size_t data_size; + int data_bit_start; } MPEG2RawSlice; typedef struct MPEG2RawSequenceEnd { diff --git a/libavcodec/cbs_vp9.h b/libavcodec/cbs_vp9.h index 40e62476ed..af15eb4bac 100644 --- a/libavcodec/cbs_vp9.h +++ b/libavcodec/cbs_vp9.h @@ -165,8 +165,8 @@ typedef struct VP9RawFrame { VP9RawFrameHeader header; uint8_t *data; - size_t data_size; AVBufferRef *data_ref; + size_t data_size; } VP9RawFrame; typedef struct VP9RawSuperframeIndex { From patchwork Sun Feb 23 23:41:06 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Thompson X-Patchwork-Id: 17896 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id C451044AD26 for ; Mon, 24 Feb 2020 02:06:26 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 92F7D68B54D; Mon, 24 Feb 2020 02:06:26 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wr1-f50.google.com (mail-wr1-f50.google.com [209.85.221.50]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 46AA568AE45 for ; Mon, 24 Feb 2020 02:06:20 +0200 (EET) Received: by mail-wr1-f50.google.com with SMTP id w12so8315084wrt.2 for ; Sun, 23 Feb 2020 16:06:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jkqxz-net.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=eZJeyHXhmu5SBZBk7tF9TTp9o2xqq5ayLYmHZawSjzA=; b=tMn8vebIsFoxYggzS1cjKgTWXpvGmTFWD+yhFDaI8EhPV42jP8K+nLWVwcYKTmr84S c985a/ebooElqmLDIHo9tHENMAbc2DaVd5EBiXnUH5F/5x3HgpnY8Pc/e0cTywxAomDk kkh/rfVGHNXyj8Gf4vjQoEP7FPdrYxhHia3I7LVwIQ+Yf0Tm5+mQV9oHV3ED0O+4+HBW zXMmHpys5KJH/8bJphirHaLY4TXBZJiWH/SU02CM6p3w2On5x44PoLqLK/tgjY+EFkgq PhABU9JABLcUGyXtIRE+ug2H30QQdd5Z1Gjg7MofR5NlmJ2uoj3iLtGOLksUfTSMVaSw aZTw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=eZJeyHXhmu5SBZBk7tF9TTp9o2xqq5ayLYmHZawSjzA=; b=Rw/zQzXnsfb2AZL4DKv2WcuF9QY5OeSWzVYN1GggV46Cb2jnJdB3XedATh2xoM0Hzk QTZCN9vZDrB84sCbB8287LTndGTF9hQG2TuIzVdxu+6KZXja/gjOI3/Ta1t3itRMrHNr mW1u/iG+2Ou1CyYbLqtuqe/1ePSgAd1g2pW2E4ouT6Ct9UnFOs8mn+jsLjOj/JOOWyUZ hanveS9lTrD4aTCVf4vdus5ZoxzQz89YfTX42x1uDo5fOE+Q+ZrKXtw18XD27OpQOXSO nRyxmplDAx494sR7S5pg49VxwFoxpjfp+qgHe+nwkY8UVcY2oHh/XKlbnhjeJ3PI58fE AsIg== X-Gm-Message-State: APjAAAXNWJx40aRPKaQxL4gQsh4s6xzG2n/DstU88l0F9PTW0SuDfqfz HBnPRB+ts3ACU3QCfd7FMBpOcBfj3tA= X-Google-Smtp-Source: APXvYqw3q/QeqqecNbpqGm2Hpn3FRF0Unx1YnBS7Gk3m5LshZctL/OO6+ssALeX7nNgDcEnlrLiiaw== X-Received: by 2002:adf:d847:: with SMTP id k7mr10241254wrl.161.1582501296395; Sun, 23 Feb 2020 15:41:36 -0800 (PST) Received: from rywe.jkqxz.net (cpc91242-cmbg18-2-0-cust650.5-4.cable.virginm.net. [82.8.130.139]) by smtp.gmail.com with ESMTPSA id y131sm16358070wmc.13.2020.02.23.15.41.35 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 23 Feb 2020 15:41:35 -0800 (PST) From: Mark Thompson To: ffmpeg-devel@ffmpeg.org Date: Sun, 23 Feb 2020 23:41:06 +0000 Message-Id: <20200223234124.17689-3-sw@jkqxz.net> X-Mailer: git-send-email 2.25.0 In-Reply-To: <20200223234124.17689-1-sw@jkqxz.net> References: <20200223234124.17689-1-sw@jkqxz.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v4 03/21] cbs: Describe allocate/free methods in tabular form X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Unit types are split into three categories, depending on how their content is managed: * POD structure - these require no special treatment. * Structure containing references to refcounted buffers - these can use a common free function when the offsets of all the internal references are known. * More complex structures - these still require ad-hoc treatment. For each codec we can then maintain a table of descriptors for each set of equivalent unit types, defining the mechanism needed to allocate/free that unit content. This is not required to be used immediately - a new alloc function supports this, but does not replace the old one which works without referring to these tables. --- libavcodec/cbs.c | 69 +++++++++++++++++++++++++++++++++++++++ libavcodec/cbs.h | 9 +++++ libavcodec/cbs_internal.h | 60 ++++++++++++++++++++++++++++++++++ 3 files changed, 138 insertions(+) diff --git a/libavcodec/cbs.c b/libavcodec/cbs.c index 0bd5e1ac5d..6cc559e545 100644 --- a/libavcodec/cbs.c +++ b/libavcodec/cbs.c @@ -812,3 +812,72 @@ void ff_cbs_delete_unit(CodedBitstreamContext *ctx, frag->units + position + 1, (frag->nb_units - position) * sizeof(*frag->units)); } + +static void cbs_default_free_unit_content(void *opaque, uint8_t *data) +{ + const CodedBitstreamUnitTypeDescriptor *desc = opaque; + if (desc->content_type == CBS_CONTENT_TYPE_INTERNAL_REFS) { + int i; + for (i = 0; i < desc->nb_ref_offsets; i++) { + void **ptr = (void**)(data + desc->ref_offsets[i]); + av_buffer_unref((AVBufferRef**)(ptr + 1)); + } + } + av_free(data); +} + +static const CodedBitstreamUnitTypeDescriptor + *cbs_find_unit_type_desc(CodedBitstreamContext *ctx, + CodedBitstreamUnit *unit) +{ + const CodedBitstreamUnitTypeDescriptor *desc; + int i, j; + + if (!ctx->codec->unit_types) + return NULL; + + for (i = 0;; i++) { + desc = &ctx->codec->unit_types[i]; + if (desc->nb_unit_types == 0) + break; + if (desc->nb_unit_types == CBS_UNIT_TYPE_RANGE) { + if (unit->type >= desc->unit_type_range_start && + unit->type <= desc->unit_type_range_end) + return desc; + } else { + for (j = 0; j < desc->nb_unit_types; j++) { + if (desc->unit_types[j] == unit->type) + return desc; + } + } + } + return NULL; +} + +int ff_cbs_alloc_unit_content2(CodedBitstreamContext *ctx, + CodedBitstreamUnit *unit) +{ + const CodedBitstreamUnitTypeDescriptor *desc; + + av_assert0(!unit->content && !unit->content_ref); + + desc = cbs_find_unit_type_desc(ctx, unit); + if (!desc) + return AVERROR(ENOSYS); + + unit->content = av_mallocz(desc->content_size); + if (!unit->content) + return AVERROR(ENOMEM); + + unit->content_ref = + av_buffer_create(unit->content, desc->content_size, + desc->content_free ? desc->content_free + : cbs_default_free_unit_content, + (void*)desc, 0); + if (!unit->content_ref) { + av_freep(&unit->content); + return AVERROR(ENOMEM); + } + + return 0; +} diff --git a/libavcodec/cbs.h b/libavcodec/cbs.h index cb3081e2c6..2a5959a2b0 100644 --- a/libavcodec/cbs.h +++ b/libavcodec/cbs.h @@ -352,6 +352,15 @@ int ff_cbs_alloc_unit_content(CodedBitstreamContext *ctx, size_t size, void (*free)(void *opaque, uint8_t *content)); +/** + * Allocate a new internal content buffer matching the type of the unit. + * + * The content will be zeroed. + */ +int ff_cbs_alloc_unit_content2(CodedBitstreamContext *ctx, + CodedBitstreamUnit *unit); + + /** * Allocate a new internal data buffer of the given size in the unit. * diff --git a/libavcodec/cbs_internal.h b/libavcodec/cbs_internal.h index 4c5a535ca6..615f514a85 100644 --- a/libavcodec/cbs_internal.h +++ b/libavcodec/cbs_internal.h @@ -25,11 +25,71 @@ #include "put_bits.h" +enum { + // Unit content is a simple structure. + CBS_CONTENT_TYPE_POD, + // Unit content contains some references to other structures, but all + // managed via buffer reference counting. The descriptor defines the + // structure offsets of every buffer reference. + CBS_CONTENT_TYPE_INTERNAL_REFS, + // Unit content is something more complex. The descriptor defines + // special functions to manage the content. + CBS_CONTENT_TYPE_COMPLEX, +}; + +enum { + // Maximum number of unit types described by the same unit type + // descriptor. + CBS_MAX_UNIT_TYPES = 16, + // Maximum number of reference buffer offsets in any one unit. + CBS_MAX_REF_OFFSETS = 2, + // Special value used in a unit type descriptor to indicate that it + // applies to a large range of types rather than a set of discrete + // values. + CBS_UNIT_TYPE_RANGE = -1, +}; + +typedef struct CodedBitstreamUnitTypeDescriptor { + // Number of entries in the unit_types array, or the special value + // CBS_UNIT_TYPE_RANGE to indicate that the range fields should be + // used instead. + int nb_unit_types; + + // Array of unit types that this entry describes. + const CodedBitstreamUnitType unit_types[CBS_MAX_UNIT_TYPES]; + + // Start and end of unit type range, used if nb_unit_types == 0. + const CodedBitstreamUnitType unit_type_range_start; + const CodedBitstreamUnitType unit_type_range_end; + + // The type of content described, from CBS_CONTENT_TYPE_*. + int content_type; + // The size of the structure which should be allocated to contain + // the decomposed content of this type of unit. + size_t content_size; + + // Number of entries in the ref_offsets array. Only used if the + // content_type is CBS_CONTENT_TYPE_INTERNAL_REFS. + int nb_ref_offsets; + // The structure must contain two adjacent elements: + // type *field; + // AVBufferRef *field_ref; + // where field points to something in the buffer referred to by + // field_ref. This offset is then set to offsetof(struct, field). + size_t ref_offsets[CBS_MAX_REF_OFFSETS]; + + void (*content_free)(void *opaque, uint8_t *data); +} CodedBitstreamUnitTypeDescriptor; + typedef struct CodedBitstreamType { enum AVCodecID codec_id; size_t priv_data_size; + // List of unit type descriptors for this codec. + // Terminated by a descriptor with nb_unit_types equal to zero. + const CodedBitstreamUnitTypeDescriptor *unit_types; + // Split frag->data into coded bitstream units, creating the // frag->units array. Fill data but not content on each unit. // The header argument should be set if the fragment came from From patchwork Sun Feb 23 23:41:07 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Thompson X-Patchwork-Id: 17887 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 8BD4C44A08D for ; Mon, 24 Feb 2020 01:49:17 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 6F1F268B502; Mon, 24 Feb 2020 01:49:17 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-lj1-f193.google.com (mail-lj1-f193.google.com [209.85.208.193]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id E27AF68B042 for ; Mon, 24 Feb 2020 01:49:15 +0200 (EET) Received: by mail-lj1-f193.google.com with SMTP id w1so8051732ljh.5 for ; Sun, 23 Feb 2020 15:49:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jkqxz-net.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=pq1n5+MfpfEnWYr+rMfOSLVlSHIPep+1WlGuvA/Iwbw=; b=KH1bsDXCgCzZzsBklV7wZHODY8t1D9hNsfswuvk0ExuHvWc252AsryMhZKT9rZbAGh Gp4hziBbcUpk5xtyHZ9Tigz+btduHJdPnSfl4rjvnSokL8Kt4y9PP2Uk+ssWPeCEXQ8b ujsV42ZHjJM1SOshHljJlBUuG1414QNinz83SHLbIfiQv+zGO0ib9gWEu6wHFZ4lG9Au HRqCXKk0G3aWAqk3mTU/ZTiPNwWv1VQASS3AXhdkaFb0n2miAsYUusx9rdA46yWGaqVw lBX/BmWNl7hfQCv2S3wtZB+xRVmFmbClmvPlW4onbKKHvu+nwbqM/ecxl3tGEgjX+M+Y tlIw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=pq1n5+MfpfEnWYr+rMfOSLVlSHIPep+1WlGuvA/Iwbw=; b=QDulphWH76/u+nXHCVY+UGksuEeYXmXCfmrZLnRwQehV4T6XL/3HdFPjb2QE/SDo6I mhxna/cfGj2J1V155US3BiABTlbO9AvLilXno9uMk/+xHq57goSVkqUcHIdf+mDZCTpY bbkYKk+gM5GoZLcA2z+8Si6j3lQyzuC4FZsJMqUhc4erhJIwJH0Z/iO2fnXsbl/Pgfv8 NiL9vyRIQnAKGcTjV8i4axWVNsw8K21b/MkKKgWNXj+WzH+bN+eloCf7IT281NI/qLLh AYg0nDqzlvOx7qC8pbTfNu76vmIAaG61fAHGuKwtWWn1HiuGgmAklI6mx0t93qm9QOnL gKbQ== X-Gm-Message-State: APjAAAV258OzFEQcCC3Ql60GC88PSC/RK3mudUqirAKMvmrcb8JS/0sa k+Z1DnzL4OfsItavahaYVAMXYZTAKio= X-Google-Smtp-Source: APXvYqzTG+Q3dD1rXJGrBEn+B1GAZI9sJ4cWICpTbm3ZDyiOpO60OllSPqwoU8JQ2XtD2Qg3Tf7e9A== X-Received: by 2002:a5d:5221:: with SMTP id i1mr58552337wra.44.1582501297173; Sun, 23 Feb 2020 15:41:37 -0800 (PST) Received: from rywe.jkqxz.net (cpc91242-cmbg18-2-0-cust650.5-4.cable.virginm.net. [82.8.130.139]) by smtp.gmail.com with ESMTPSA id y131sm16358070wmc.13.2020.02.23.15.41.36 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 23 Feb 2020 15:41:36 -0800 (PST) From: Mark Thompson To: ffmpeg-devel@ffmpeg.org Date: Sun, 23 Feb 2020 23:41:07 +0000 Message-Id: <20200223234124.17689-4-sw@jkqxz.net> X-Mailer: git-send-email 2.25.0 In-Reply-To: <20200223234124.17689-1-sw@jkqxz.net> References: <20200223234124.17689-1-sw@jkqxz.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v4 04/21] cbs: Add macros to support defining unit type tables X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" --- libavcodec/cbs_internal.h | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) diff --git a/libavcodec/cbs_internal.h b/libavcodec/cbs_internal.h index 615f514a85..2922878ed0 100644 --- a/libavcodec/cbs_internal.h +++ b/libavcodec/cbs_internal.h @@ -166,6 +166,30 @@ int ff_cbs_write_signed(CodedBitstreamContext *ctx, PutBitContext *pbc, #define MIN_INT_BITS(length) (-(INT64_C(1) << ((length) - 1))) +#define CBS_UNIT_TYPE_POD(type, structure) { \ + .nb_unit_types = 1, \ + .unit_types = { type }, \ + .content_type = CBS_CONTENT_TYPE_POD, \ + .content_size = sizeof(structure), \ + } +#define CBS_UNIT_TYPE_INTERNAL_REF(type, structure, ref_field) { \ + .nb_unit_types = 1, \ + .unit_types = { type }, \ + .content_type = CBS_CONTENT_TYPE_INTERNAL_REFS, \ + .content_size = sizeof(structure), \ + .nb_ref_offsets = 1, \ + .ref_offsets = { offsetof(structure, ref_field) }, \ + } +#define CBS_UNIT_TYPE_COMPLEX(type, structure, free_func) { \ + .nb_unit_types = 1, \ + .unit_types = { type }, \ + .content_type = CBS_CONTENT_TYPE_COMPLEX, \ + .content_size = sizeof(structure), \ + .content_free = free_func, \ + } +#define CBS_UNIT_TYPE_END_OF_LIST { .nb_unit_types = 0 } + + extern const CodedBitstreamType ff_cbs_type_av1; extern const CodedBitstreamType ff_cbs_type_h264; extern const CodedBitstreamType ff_cbs_type_h265; From patchwork Sun Feb 23 23:41:08 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Thompson X-Patchwork-Id: 17884 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id B82504488C6 for ; Mon, 24 Feb 2020 01:48:23 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 9F84E68B502; Mon, 24 Feb 2020 01:48:23 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-ed1-f67.google.com (mail-ed1-f67.google.com [209.85.208.67]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id DE45968B4F0 for ; Mon, 24 Feb 2020 01:48:22 +0200 (EET) Received: by mail-ed1-f67.google.com with SMTP id p23so9822560edr.5 for ; Sun, 23 Feb 2020 15:48:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jkqxz-net.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=BpgiPhRaNZJU7bhBtKqC6frMuY+Vb6DG7V/snBf6aHM=; b=vcL2AJlv8FUhtndmzJBooA8NyfJdZXKG3hltYIUJ2ORIjE6p4A619qdCqNLuNbvYpS KDjGDWzwV8kY2urF8yoNaryGb3cnEtdM4uIUJGMOixJNSQWovaxWRIAtjesvIQGauob6 v/bELKMAiZGj/9SIlbggIEe1d9EIMapY5DckBpt2abG2Li6t6NCO16tYDx1M9136zW8I BLjZi0vUGJadN4OTF3g+Kw3DcyjzwN6J+qvKR+xzmiIuJkDPiUIBuy6sY0vG/uKqILma Vw/AW8x9aub/COXekNvT4WdMVFkfxVR5ChgkTrAAo2cuhIRlZcJEWA5C8i9ZMXNE9S2t QP2w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=BpgiPhRaNZJU7bhBtKqC6frMuY+Vb6DG7V/snBf6aHM=; b=DNll7vTicn+KY/oXniO8rUV3DVF35Wnjxg2HQdhcVYsctv1EurppDDEbOdNhWaN7R6 bzhTMGmTAt6lM5bFnUd+opTznAY8iK5iS1jM1JnWtg27zX20FrBKaayQ7XmyKcD3gyM1 yFtDm6Ml7fnpsuM1LMB3cL4X33FqCjpowartiC9m0Rvc/3BnZasfGZUZpcPv4SBgwoMr CMoJw4gSNijgotelmgbuB/74wWLAPX4fu1Ql23iMdMVG4wrEl2SlRg7xQaKEe+VXqOX6 Db66uUsLEW1EbBJHq+STNT5H0+Y/bEV0ABdeP9uKYpPv2RbJ2nQV51T0PDvVXNlhPhGO JRnQ== X-Gm-Message-State: APjAAAUfwl4Yubmk+msk+ObTbLSFyGtMyFQFs5b+En6/cVqW0gAKrTPY 8HUbiUZQDGSH2D8e1xO2nbaE4ngCL1g= X-Google-Smtp-Source: APXvYqySGdixU1ABrLVEbcvVAT4lL8zLhBU+uPi95Sw4x9WHWyguAbtGVTg2Q7MtF72tx4m02qiUHw== X-Received: by 2002:adf:e943:: with SMTP id m3mr62087058wrn.397.1582501298019; Sun, 23 Feb 2020 15:41:38 -0800 (PST) Received: from rywe.jkqxz.net (cpc91242-cmbg18-2-0-cust650.5-4.cable.virginm.net. [82.8.130.139]) by smtp.gmail.com with ESMTPSA id y131sm16358070wmc.13.2020.02.23.15.41.37 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 23 Feb 2020 15:41:37 -0800 (PST) From: Mark Thompson To: ffmpeg-devel@ffmpeg.org Date: Sun, 23 Feb 2020 23:41:08 +0000 Message-Id: <20200223234124.17689-5-sw@jkqxz.net> X-Mailer: git-send-email 2.25.0 In-Reply-To: <20200223234124.17689-1-sw@jkqxz.net> References: <20200223234124.17689-1-sw@jkqxz.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v4 05/21] cbs_h264: Use table-based alloc/free X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" --- libavcodec/cbs_h2645.c | 163 ++++++++++++++++++----------------------- 1 file changed, 70 insertions(+), 93 deletions(-) diff --git a/libavcodec/cbs_h2645.c b/libavcodec/cbs_h2645.c index c8347ba5fa..7e2cd4b00c 100644 --- a/libavcodec/cbs_h2645.c +++ b/libavcodec/cbs_h2645.c @@ -443,52 +443,6 @@ static int cbs_h2645_read_more_rbsp_data(GetBitContext *gbc) #undef allocate -static void cbs_h264_free_pps(void *opaque, uint8_t *content) -{ - H264RawPPS *pps = (H264RawPPS*)content; - av_buffer_unref(&pps->slice_group_id_ref); - av_freep(&content); -} - -static void cbs_h264_free_sei_payload(H264RawSEIPayload *payload) -{ - switch (payload->payload_type) { - case H264_SEI_TYPE_BUFFERING_PERIOD: - case H264_SEI_TYPE_PIC_TIMING: - case H264_SEI_TYPE_PAN_SCAN_RECT: - case H264_SEI_TYPE_RECOVERY_POINT: - case H264_SEI_TYPE_DISPLAY_ORIENTATION: - case H264_SEI_TYPE_MASTERING_DISPLAY_COLOUR_VOLUME: - case H264_SEI_TYPE_ALTERNATIVE_TRANSFER: - break; - case H264_SEI_TYPE_USER_DATA_REGISTERED: - av_buffer_unref(&payload->payload.user_data_registered.data_ref); - break; - case H264_SEI_TYPE_USER_DATA_UNREGISTERED: - av_buffer_unref(&payload->payload.user_data_unregistered.data_ref); - break; - default: - av_buffer_unref(&payload->payload.other.data_ref); - break; - } -} - -static void cbs_h264_free_sei(void *opaque, uint8_t *content) -{ - H264RawSEI *sei = (H264RawSEI*)content; - int i; - for (i = 0; i < sei->payload_count; i++) - cbs_h264_free_sei_payload(&sei->payload[i]); - av_freep(&content); -} - -static void cbs_h264_free_slice(void *opaque, uint8_t *content) -{ - H264RawSlice *slice = (H264RawSlice*)content; - av_buffer_unref(&slice->data_ref); - av_freep(&content); -} - static void cbs_h265_free_vps(void *opaque, uint8_t *content) { H265RawVPS *vps = (H265RawVPS*)content; @@ -790,15 +744,14 @@ static int cbs_h264_read_nal_unit(CodedBitstreamContext *ctx, if (err < 0) return err; + err = ff_cbs_alloc_unit_content2(ctx, unit); + if (err < 0) + return err; + switch (unit->type) { case H264_NAL_SPS: { - H264RawSPS *sps; - - err = ff_cbs_alloc_unit_content(ctx, unit, sizeof(*sps), NULL); - if (err < 0) - return err; - sps = unit->content; + H264RawSPS *sps = unit->content; err = cbs_h264_read_sps(ctx, &gbc, sps); if (err < 0) @@ -812,12 +765,6 @@ static int cbs_h264_read_nal_unit(CodedBitstreamContext *ctx, case H264_NAL_SPS_EXT: { - err = ff_cbs_alloc_unit_content(ctx, unit, - sizeof(H264RawSPSExtension), - NULL); - if (err < 0) - return err; - err = cbs_h264_read_sps_extension(ctx, &gbc, unit->content); if (err < 0) return err; @@ -826,13 +773,7 @@ static int cbs_h264_read_nal_unit(CodedBitstreamContext *ctx, case H264_NAL_PPS: { - H264RawPPS *pps; - - err = ff_cbs_alloc_unit_content(ctx, unit, sizeof(*pps), - &cbs_h264_free_pps); - if (err < 0) - return err; - pps = unit->content; + H264RawPPS *pps = unit->content; err = cbs_h264_read_pps(ctx, &gbc, pps); if (err < 0) @@ -848,15 +789,9 @@ static int cbs_h264_read_nal_unit(CodedBitstreamContext *ctx, case H264_NAL_IDR_SLICE: case H264_NAL_AUXILIARY_SLICE: { - H264RawSlice *slice; + H264RawSlice *slice = unit->content; int pos, len; - err = ff_cbs_alloc_unit_content(ctx, unit, sizeof(*slice), - &cbs_h264_free_slice); - if (err < 0) - return err; - slice = unit->content; - err = cbs_h264_read_slice_header(ctx, &gbc, &slice->header); if (err < 0) return err; @@ -882,11 +817,6 @@ static int cbs_h264_read_nal_unit(CodedBitstreamContext *ctx, case H264_NAL_AUD: { - err = ff_cbs_alloc_unit_content(ctx, unit, - sizeof(H264RawAUD), NULL); - if (err < 0) - return err; - err = cbs_h264_read_aud(ctx, &gbc, unit->content); if (err < 0) return err; @@ -895,11 +825,6 @@ static int cbs_h264_read_nal_unit(CodedBitstreamContext *ctx, case H264_NAL_SEI: { - err = ff_cbs_alloc_unit_content(ctx, unit, sizeof(H264RawSEI), - &cbs_h264_free_sei); - if (err < 0) - return err; - err = cbs_h264_read_sei(ctx, &gbc, unit->content); if (err < 0) return err; @@ -908,11 +833,6 @@ static int cbs_h264_read_nal_unit(CodedBitstreamContext *ctx, case H264_NAL_FILLER_DATA: { - err = ff_cbs_alloc_unit_content(ctx, unit, - sizeof(H264RawFiller), NULL); - if (err < 0) - return err; - err = cbs_h264_read_filler(ctx, &gbc, unit->content); if (err < 0) return err; @@ -922,12 +842,6 @@ static int cbs_h264_read_nal_unit(CodedBitstreamContext *ctx, case H264_NAL_END_SEQUENCE: case H264_NAL_END_STREAM: { - err = ff_cbs_alloc_unit_content(ctx, unit, - sizeof(H264RawNALUnitHeader), - NULL); - if (err < 0) - return err; - err = (unit->type == H264_NAL_END_SEQUENCE ? cbs_h264_read_end_of_sequence : cbs_h264_read_end_of_stream)(ctx, &gbc, unit->content); @@ -1501,11 +1415,74 @@ static void cbs_h265_close(CodedBitstreamContext *ctx) av_buffer_unref(&h265->pps_ref[i]); } +static void cbs_h264_free_sei_payload(H264RawSEIPayload *payload) +{ + switch (payload->payload_type) { + case H264_SEI_TYPE_BUFFERING_PERIOD: + case H264_SEI_TYPE_PIC_TIMING: + case H264_SEI_TYPE_PAN_SCAN_RECT: + case H264_SEI_TYPE_RECOVERY_POINT: + case H264_SEI_TYPE_DISPLAY_ORIENTATION: + case H264_SEI_TYPE_MASTERING_DISPLAY_COLOUR_VOLUME: + case H264_SEI_TYPE_ALTERNATIVE_TRANSFER: + break; + case H264_SEI_TYPE_USER_DATA_REGISTERED: + av_buffer_unref(&payload->payload.user_data_registered.data_ref); + break; + case H264_SEI_TYPE_USER_DATA_UNREGISTERED: + av_buffer_unref(&payload->payload.user_data_unregistered.data_ref); + break; + default: + av_buffer_unref(&payload->payload.other.data_ref); + break; + } +} + +static void cbs_h264_free_sei(void *opaque, uint8_t *content) +{ + H264RawSEI *sei = (H264RawSEI*)content; + int i; + for (i = 0; i < sei->payload_count; i++) + cbs_h264_free_sei_payload(&sei->payload[i]); + av_freep(&content); +} + +static const CodedBitstreamUnitTypeDescriptor cbs_h264_unit_types[] = { + CBS_UNIT_TYPE_POD(H264_NAL_SPS, H264RawSPS), + CBS_UNIT_TYPE_POD(H264_NAL_SPS_EXT, H264RawSPSExtension), + + CBS_UNIT_TYPE_INTERNAL_REF(H264_NAL_PPS, H264RawPPS, slice_group_id), + + { + .nb_unit_types = 3, + .unit_types = { + H264_NAL_IDR_SLICE, + H264_NAL_SLICE, + H264_NAL_AUXILIARY_SLICE, + }, + .content_type = CBS_CONTENT_TYPE_INTERNAL_REFS, + .content_size = sizeof(H264RawSlice), + .nb_ref_offsets = 1, + .ref_offsets = { offsetof(H264RawSlice, data) }, + }, + + CBS_UNIT_TYPE_POD(H264_NAL_AUD, H264RawAUD), + CBS_UNIT_TYPE_POD(H264_NAL_FILLER_DATA, H264RawFiller), + CBS_UNIT_TYPE_POD(H264_NAL_END_SEQUENCE, H264RawNALUnitHeader), + CBS_UNIT_TYPE_POD(H264_NAL_END_STREAM, H264RawNALUnitHeader), + + CBS_UNIT_TYPE_COMPLEX(H264_NAL_SEI, H264RawSEI, &cbs_h264_free_sei), + + CBS_UNIT_TYPE_END_OF_LIST +}; + const CodedBitstreamType ff_cbs_type_h264 = { .codec_id = AV_CODEC_ID_H264, .priv_data_size = sizeof(CodedBitstreamH264Context), + .unit_types = cbs_h264_unit_types, + .split_fragment = &cbs_h2645_split_fragment, .read_unit = &cbs_h264_read_nal_unit, .write_unit = &cbs_h264_write_nal_unit, From patchwork Sun Feb 23 23:41:09 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Thompson X-Patchwork-Id: 17891 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 2B82F44A08D for ; Mon, 24 Feb 2020 01:49:44 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 16D1D68B522; Mon, 24 Feb 2020 01:49:44 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-ed1-f65.google.com (mail-ed1-f65.google.com [209.85.208.65]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id A0E1868B501 for ; Mon, 24 Feb 2020 01:49:42 +0200 (EET) Received: by mail-ed1-f65.google.com with SMTP id p23so9824687edr.5 for ; Sun, 23 Feb 2020 15:49:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jkqxz-net.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=XeXZBFcDdRMLZwBGw4+b3CTcExYjFYyEhJHooMC7UE8=; b=j8MI5jP3K9s8FokbSzGry8H1R7UHr3JOUOO6ops6jb2rJvVkOl4kvNEdbFkb7aOffP 452Uz1LVgJhBY5m8vIhpbT5t0NC0BtA4dRAUVoXeg8/EdcjQEd8yBW2QiroO2+c2vtLU qJX0vuHsDgms35z9US+G9u5FnvP2wmoPr9V8iCErM3IQMlBEfSwyqDbjUzuJ5secHq/3 Jd48Rp3BachIutClAdW5FA0nLxxYb9xxBqmsByMZsPS+1GS3BUYSBJl+DIS25xUaGnA6 v9ty0yIitTlPzjNEWynvi6ewnZmo9KCJvCsLJe4OXjR4Js+0umBPS4iug25zPZNWfFGY 53Cg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=XeXZBFcDdRMLZwBGw4+b3CTcExYjFYyEhJHooMC7UE8=; b=NT9rrJGRdKE3v2WjF/CLVEHrFfEpUl9AUxHx0xhGbCxQ4Odqx004mLNUxyllQdbBjY qp6X3g3+W/GDj1eCWQDIRVsv9UShzUJblGLf86b9DXQvtpRKymE4CW7FVFJ8GZg7am1M I3uoRxp6q9g1UReQRZct/3GnsuKcsP48K8Ao3wJIs80T+ZQohETNvKnLJL2ENPoMmHIM PtXQkEUwPlwqXDIfHMIApyWkM6NsdrI4TmoDpc88bbZTDMJ1gO1dMpRjBw4huISIMVvB SzXtgORyIaTo0X3zXZM7qLZNyQwmhzoYdHlIjFbJRQ/kSXz3ycRaBL2g/OqdpEMPPhOF QNiQ== X-Gm-Message-State: APjAAAXY5TGwFmPycW3CYd6xTl/7WpIVNAIyAjSzKXnNgvWihuzm4IwF BDlNR9AZ4CmlAYFbVhHLvBXPWFjuzGk= X-Google-Smtp-Source: APXvYqzYPkNrjtSTMnz3i8QvS961m67MUyzTnd2CKiJKmmbTWMfxoUcA+qv30gyWXeH6bnGTj7Q2fQ== X-Received: by 2002:a5d:410e:: with SMTP id l14mr59939087wrp.238.1582501298950; Sun, 23 Feb 2020 15:41:38 -0800 (PST) Received: from rywe.jkqxz.net (cpc91242-cmbg18-2-0-cust650.5-4.cable.virginm.net. [82.8.130.139]) by smtp.gmail.com with ESMTPSA id y131sm16358070wmc.13.2020.02.23.15.41.38 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 23 Feb 2020 15:41:38 -0800 (PST) From: Mark Thompson To: ffmpeg-devel@ffmpeg.org Date: Sun, 23 Feb 2020 23:41:09 +0000 Message-Id: <20200223234124.17689-6-sw@jkqxz.net> X-Mailer: git-send-email 2.25.0 In-Reply-To: <20200223234124.17689-1-sw@jkqxz.net> References: <20200223234124.17689-1-sw@jkqxz.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v4 06/21] cbs: Add support functions for handling unit content references X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Use the unit type table to determine what we need to do to clone the internals of the unit content when making copies for refcounting or writeability. (This will still fail for units with complex content if they do not have a defined clone function.) Setup and naming from a patch by Andreas Rheinhardt , but with the implementation changed to use the unit type information if possible rather than requiring a codec-specific function. --- libavcodec/cbs.c | 172 ++++++++++++++++++++++++++++++++++++++ libavcodec/cbs.h | 29 +++++++ libavcodec/cbs_internal.h | 1 + 3 files changed, 202 insertions(+) diff --git a/libavcodec/cbs.c b/libavcodec/cbs.c index 6cc559e545..91788f6dfb 100644 --- a/libavcodec/cbs.c +++ b/libavcodec/cbs.c @@ -881,3 +881,175 @@ int ff_cbs_alloc_unit_content2(CodedBitstreamContext *ctx, return 0; } + +static int cbs_clone_unit_content(AVBufferRef **clone_ref, + CodedBitstreamUnit *unit, + const CodedBitstreamUnitTypeDescriptor *desc) +{ + uint8_t *src, *copy; + uint8_t **src_ptr, **copy_ptr; + AVBufferRef **src_buf, **copy_buf; + int err, i; + + av_assert0(unit->content); + src = unit->content; + + copy = av_malloc(desc->content_size); + if (!copy) + return AVERROR(ENOMEM); + + memcpy(copy, src, desc->content_size); + + for (i = 0; i < desc->nb_ref_offsets; i++) { + src_ptr = (uint8_t**)(src + desc->ref_offsets[i]); + src_buf = (AVBufferRef**)(src_ptr + 1); + copy_ptr = (uint8_t**)(copy + desc->ref_offsets[i]); + copy_buf = (AVBufferRef**)(copy_ptr + 1); + + if (!*src_ptr) { + av_assert0(!*src_buf); + continue; + } + if (!*src_buf) { + // We can't handle a non-refcounted pointer here - we don't + // have enough information to handle whatever structure lies + // at the other end of it. + err = AVERROR(EINVAL); + goto fail; + } + + // src_ptr is required to point somewhere inside src_buf. If it + // doesn't, there is a bug somewhere. + av_assert0(*src_ptr >= (*src_buf)->data && + *src_ptr < (*src_buf)->data + (*src_buf)->size); + + *copy_buf = av_buffer_ref(*src_buf); + if (!*copy_buf) { + err = AVERROR(ENOMEM); + goto fail; + } + + err = av_buffer_make_writable(copy_buf); + if (err < 0) { + av_buffer_unref(copy_buf); + goto fail; + } + + *copy_ptr = (*copy_buf)->data + (*src_ptr - (*src_buf)->data); + } + + *clone_ref = av_buffer_create(copy, desc->content_size, + desc->content_free ? desc->content_free : + cbs_default_free_unit_content, + (void*)desc, 0); + if (!*clone_ref) { + err = AVERROR(ENOMEM); + goto fail; + } + + return 0; + +fail: + for (--i; i >= 0; i--) + av_buffer_unref((AVBufferRef**)(copy + desc->ref_offsets[i])); + av_freep(©); + *clone_ref = NULL; + return err; +} + +int ff_cbs_make_unit_refcounted(CodedBitstreamContext *ctx, + CodedBitstreamUnit *unit) +{ + const CodedBitstreamUnitTypeDescriptor *desc; + AVBufferRef *ref; + int err; + + av_assert0(unit->content); + if (unit->content_ref) { + // Already refcounted, nothing to do. + return 0; + } + + desc = cbs_find_unit_type_desc(ctx, unit); + if (!desc) + return AVERROR(ENOSYS); + + switch (desc->content_type) { + case CBS_CONTENT_TYPE_POD: + ref = av_buffer_alloc(desc->content_size); + if (!ref) + return AVERROR(ENOMEM); + memcpy(ref->data, unit->content, desc->content_size); + err = 0; + break; + + case CBS_CONTENT_TYPE_INTERNAL_REFS: + err = cbs_clone_unit_content(&ref, unit, desc); + break; + + case CBS_CONTENT_TYPE_COMPLEX: + if (!desc->content_clone) + return AVERROR_PATCHWELCOME; + err = desc->content_clone(&ref, unit); + break; + + default: + av_assert0(0 && "Invalid content type."); + } + + if (err < 0) + return err; + + unit->content_ref = ref; + unit->content = ref->data; + return 0; +} + +int ff_cbs_make_unit_writable(CodedBitstreamContext *ctx, + CodedBitstreamUnit *unit) +{ + const CodedBitstreamUnitTypeDescriptor *desc; + AVBufferRef *ref; + int err; + + // This can only be applied to refcounted units. + err = ff_cbs_make_unit_refcounted(ctx, unit); + if (err < 0) + return err; + av_assert0(unit->content && unit->content_ref); + + if (av_buffer_is_writable(unit->content_ref)) + return 0; + + desc = cbs_find_unit_type_desc(ctx, unit); + if (!desc) + return AVERROR(ENOSYS); + + switch (desc->content_type) { + case CBS_CONTENT_TYPE_POD: + err = av_buffer_make_writable(&unit->content_ref); + break; + + case CBS_CONTENT_TYPE_INTERNAL_REFS: + err = cbs_clone_unit_content(&ref, unit, desc); + break; + + case CBS_CONTENT_TYPE_COMPLEX: + if (!desc->content_clone) + return AVERROR_PATCHWELCOME; + err = desc->content_clone(&ref, unit); + break; + + default: + av_assert0(0 && "Invalid content type."); + } + if (err < 0) + return err; + + if (desc->content_type != CBS_CONTENT_TYPE_POD) { + av_buffer_unref(&unit->content_ref); + unit->content_ref = ref; + } + unit->content = unit->content_ref->data; + return 0; +} diff --git a/libavcodec/cbs.h b/libavcodec/cbs.h index 2a5959a2b0..eb7f4b878b 100644 --- a/libavcodec/cbs.h +++ b/libavcodec/cbs.h @@ -407,4 +407,33 @@ void ff_cbs_delete_unit(CodedBitstreamContext *ctx, int position); +/** + * Make the content of a unit refcounted. + * + * If the unit is not refcounted, this will do a deep copy of the unit + * content to new refcounted buffers. + * + * It is not valid to call this function on a unit which does not have + * decomposed content. + */ +int ff_cbs_make_unit_refcounted(CodedBitstreamContext *ctx, + CodedBitstreamUnit *unit); + +/** + * Make the content of a unit writable so that internal fields can be + * modified. + * + * If it is known that there are no other references to the content of + * the unit, does nothing and returns success. Otherwise (including the + * case where the unit content is not refcounted), it does a full clone + * of the content (including any internal buffers) to make a new copy, + * and replaces the existing references inside the unit with that. + * + * It is not valid to call this function on a unit which does not have + * decomposed content. + */ +int ff_cbs_make_unit_writable(CodedBitstreamContext *ctx, + CodedBitstreamUnit *unit); + + #endif /* AVCODEC_CBS_H */ diff --git a/libavcodec/cbs_internal.h b/libavcodec/cbs_internal.h index 2922878ed0..6fee4e90a5 100644 --- a/libavcodec/cbs_internal.h +++ b/libavcodec/cbs_internal.h @@ -79,6 +79,7 @@ typedef struct CodedBitstreamUnitTypeDescriptor { size_t ref_offsets[CBS_MAX_REF_OFFSETS]; void (*content_free)(void *opaque, uint8_t *data); + int (*content_clone)(AVBufferRef **ref, CodedBitstreamUnit *unit); } CodedBitstreamUnitTypeDescriptor; typedef struct CodedBitstreamType { From patchwork Sun Feb 23 23:41:10 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Thompson X-Patchwork-Id: 17885 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id A29954488C6 for ; Mon, 24 Feb 2020 01:48:27 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 8FBA768B514; Mon, 24 Feb 2020 01:48:27 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wm1-f65.google.com (mail-wm1-f65.google.com [209.85.128.65]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id C3DE768B509 for ; Mon, 24 Feb 2020 01:48:25 +0200 (EET) Received: by mail-wm1-f65.google.com with SMTP id s10so7222782wmh.3 for ; Sun, 23 Feb 2020 15:48:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jkqxz-net.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=BaCTuicKdV2L//4yapJ2vZsVqsOYD+JVHZmCGisTc0k=; b=X13j/Plm8aYxNkxUWCvDNqnyunHej7/5Ae4QAyhvO/LandIdScruDE7LxcpzVYSSm3 uw/aIlWg5Dor2wFgnd1XR7N4DjeFC02AHgR3LKNUbqygMZxvPCJoC8lSSNSuzyNDaDM0 fMtTOfbi+s1V4OfYjp4GR6jf8bt9fScTNg0lxJpgWGf/o8AOogYIdok/expQosDfq8LN 9qVzi3Iwk53xATbzFapTKM7SDpYzS9s4sDpI7+olJEJAAxSkDpEIv61GPkeCh5YreWJI MUAJFwwEYSz3bVKcphDKXq7ye3nplbPMPEfFxep1Fu2PDFK8Yf8xOng4OmuVCi2cDfzC gXbw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=BaCTuicKdV2L//4yapJ2vZsVqsOYD+JVHZmCGisTc0k=; b=s5LgDIpYclkpzyqTObFnevWrmbHNnl5iEvkPZurizn53KGXJPlmSJeLL7Eq1YHB8RX 9EnMGxvC+iTicJClF/68Rk98148kqPTYsJPtwIJXe45U+sKwppJhJ+1sdg6qvz8naoWx UuF8cs4L8chZFHFN0xBcvjlySwtJxD2hPJITZ30h45jm6QS/m2RahU3wMurzSun+iCwg t8WNbaFncyCVMLUU88Y6ENaiJOFZQYNbcY165XsAzR+C0g9ujGoepBNVL6eNLc2/QNyr qnxR7skMyZ2vIUoYQjXdPyANeLA88ue2NGw3Kx7BeR/W2AYfqz9g8s6Jo1PR4u68ziej XdUQ== X-Gm-Message-State: APjAAAW+YVXzXy9+8zjcxXkmf5IMA2u/i4ukol3nUpkVdDsr8tCQj1yr YpMcRPpjdDB9fAKjj8pguS0VHUuRsXk= X-Google-Smtp-Source: APXvYqx7oC6A5YnwnD0c1pFu2AIkhYtK9+1dlZtstMW66G/nqcGUtv2zN95YXQqGZjmg2F/BS7VMtw== X-Received: by 2002:a1c:a752:: with SMTP id q79mr17313990wme.64.1582501299815; Sun, 23 Feb 2020 15:41:39 -0800 (PST) Received: from rywe.jkqxz.net (cpc91242-cmbg18-2-0-cust650.5-4.cable.virginm.net. [82.8.130.139]) by smtp.gmail.com with ESMTPSA id y131sm16358070wmc.13.2020.02.23.15.41.39 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 23 Feb 2020 15:41:39 -0800 (PST) From: Mark Thompson To: ffmpeg-devel@ffmpeg.org Date: Sun, 23 Feb 2020 23:41:10 +0000 Message-Id: <20200223234124.17689-7-sw@jkqxz.net> X-Mailer: git-send-email 2.25.0 In-Reply-To: <20200223234124.17689-1-sw@jkqxz.net> References: <20200223234124.17689-1-sw@jkqxz.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v4 07/21] cbs_h2645: Ensure that non-refcounted parameter sets are fully copied X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Only copying the main structure is not necessarily sufficient - there could be references to substructures. --- libavcodec/cbs_h2645.c | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/libavcodec/cbs_h2645.c b/libavcodec/cbs_h2645.c index 7e2cd4b00c..84cb70b863 100644 --- a/libavcodec/cbs_h2645.c +++ b/libavcodec/cbs_h2645.c @@ -708,18 +708,20 @@ static int cbs_h26 ## h26n ## _replace_ ## ps_var(CodedBitstreamContext *ctx, \ CodedBitstreamH26 ## h26n ## Context *priv = ctx->priv_data; \ H26 ## h26n ## Raw ## ps_name *ps_var = unit->content; \ unsigned int id = ps_var->id_element; \ + int err; \ if (id >= FF_ARRAY_ELEMS(priv->ps_var)) { \ av_log(ctx->log_ctx, AV_LOG_ERROR, "Invalid " #ps_name \ " id : %d.\n", id); \ return AVERROR_INVALIDDATA; \ } \ + err = ff_cbs_make_unit_refcounted(ctx, unit); \ + if (err < 0) \ + return err; \ if (priv->ps_var[id] == priv->active_ ## ps_var) \ priv->active_ ## ps_var = NULL ; \ av_buffer_unref(&priv->ps_var ## _ref[id]); \ - if (unit->content_ref) \ - priv->ps_var ## _ref[id] = av_buffer_ref(unit->content_ref); \ - else \ - priv->ps_var ## _ref[id] = av_buffer_alloc(sizeof(*ps_var)); \ + av_assert0(unit->content_ref); \ + priv->ps_var ## _ref[id] = av_buffer_ref(unit->content_ref); \ if (!priv->ps_var ## _ref[id]) \ return AVERROR(ENOMEM); \ priv->ps_var[id] = (H26 ## h26n ## Raw ## ps_name *)priv->ps_var ## _ref[id]->data; \ From patchwork Sun Feb 23 23:41:11 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Thompson X-Patchwork-Id: 17890 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 528E544A08D for ; Mon, 24 Feb 2020 01:49:29 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 3D9BC68B52C; Mon, 24 Feb 2020 01:49:29 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wm1-f65.google.com (mail-wm1-f65.google.com [209.85.128.65]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 4D45468B502 for ; Mon, 24 Feb 2020 01:49:23 +0200 (EET) Received: by mail-wm1-f65.google.com with SMTP id s10so7223823wmh.3 for ; Sun, 23 Feb 2020 15:49:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jkqxz-net.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=+iadSbpoe7rWMQHRauxkCwBQ6SYyQU9zNDNe/7/NSUU=; b=oA6bvOgj38QFGRH694vSyZksmugDTejuwzlgcaCL/PoQMjYjcbVAfy4WgNVFePVeDd KHnap/GCXJvN9uJrnv5c5jBiVyUzscCmlXbVcfjVYIMh+YI2htKuZ3KRiEEFh2IMj8Q0 FiKCDaZnp2u5+qmZg0Nb+D2ZUATx4DKOpTB6rGoDohjRVggUVr/Y4F7aF2PhpQSTpWsQ tllZM7Upk8gCZm9G/WPXktmz252rkNzZJ6WLxxZUpuScILDwhL/Z17pW9FnvCkUQUgHQ FN3BUmkJTIPp/kY4shVTk7TufKXjob2Wc/OW52d99o3rWFlf4KK860/ofXVj+Zqpt2VY KumA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=+iadSbpoe7rWMQHRauxkCwBQ6SYyQU9zNDNe/7/NSUU=; b=BbrGr5j4TCJNfuYzHB+HGd4AlZMA4c01VZWKRUvITLL8d3AL05mejPnHW3c1xuqy/H Vg/ahBrFtB2jAjmhdlBSYpSrpOnLDd9QcoDjK6YgbxCCzfjy99iES7T4tGAffR52SY/z PLsm5eIVa/hi+vbxK539x+ezOd3QOVvFTUo/YWZyiTkLkns9mg/kA0pDw8DUi2kTnGUV b9h0AI2r3z6jign0I948Z+vV63g2YrxIMfjpFbIZUS300CIi1rbyhdj25zcSKc1MyIOm njkPXaawi9PmDO/O1Zyabr5ef94erPtkYK4WsOzYDOJjg8J0tPvqAn4AEeINBUS5TIA5 Wzqw== X-Gm-Message-State: APjAAAVQ36rZ6xDJsuaT6QAQ+0yod8M4wZqihN3FrwCGv3WF2POPB6T1 CSWcp9bXEhnjkvFHybJSOs08BPuZFPM= X-Google-Smtp-Source: APXvYqwyXsN833kpzxJhYTK9R5VDmuWQ/yPDspKd5BKq1CKp7i/J2V/WY7qDfk6wUsLBC+JI0GGb1A== X-Received: by 2002:a1c:6408:: with SMTP id y8mr18094238wmb.130.1582501300583; Sun, 23 Feb 2020 15:41:40 -0800 (PST) Received: from rywe.jkqxz.net (cpc91242-cmbg18-2-0-cust650.5-4.cable.virginm.net. [82.8.130.139]) by smtp.gmail.com with ESMTPSA id y131sm16358070wmc.13.2020.02.23.15.41.39 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 23 Feb 2020 15:41:40 -0800 (PST) From: Mark Thompson To: ffmpeg-devel@ffmpeg.org Date: Sun, 23 Feb 2020 23:41:11 +0000 Message-Id: <20200223234124.17689-8-sw@jkqxz.net> X-Mailer: git-send-email 2.25.0 In-Reply-To: <20200223234124.17689-1-sw@jkqxz.net> References: <20200223234124.17689-1-sw@jkqxz.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v4 08/21] h264_redundant_pps: Make it reference-compatible X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" From: Andreas Rheinhardt Since c6a63e11092c975b89d824f08682fe31948d3686, the parameter sets modified as content of PPS units were references shared with the CodedBitstreamH264Context, so modifying them alters the parsing process of future access units which meant that frames often got discarded because invalid values were parsed. This patch makes h264_redundant_pps compatible with the reality of reference-counted parameter sets. Signed-off-by: Andreas Rheinhardt Signed-off-by: Mark Thompson --- libavcodec/h264_redundant_pps_bsf.c | 17 ++++++++++++++--- 1 file changed, 14 insertions(+), 3 deletions(-) diff --git a/libavcodec/h264_redundant_pps_bsf.c b/libavcodec/h264_redundant_pps_bsf.c index 8405738c4b..4d5dd9a90f 100644 --- a/libavcodec/h264_redundant_pps_bsf.c +++ b/libavcodec/h264_redundant_pps_bsf.c @@ -40,8 +40,19 @@ typedef struct H264RedundantPPSContext { static int h264_redundant_pps_fixup_pps(H264RedundantPPSContext *ctx, - H264RawPPS *pps) + CodedBitstreamUnit *unit) { + H264RawPPS *pps; + int err; + + // The changes we are about to perform affect the parsing process, + // so we must make sure that the PPS is writable, otherwise the + // parsing of future slices will be incorrect and even raise errors. + err = ff_cbs_make_unit_writable(ctx->input, unit); + if (err < 0) + return err; + pps = unit->content; + // Record the current value of pic_init_qp in order to fix up // following slices, then overwrite with the global value. ctx->current_pic_init_qp = pps->pic_init_qp_minus26 + 26; @@ -88,7 +99,7 @@ static int h264_redundant_pps_filter(AVBSFContext *bsf, AVPacket *pkt) if (nal->type == H264_NAL_SPS) au_has_sps = 1; if (nal->type == H264_NAL_PPS) { - err = h264_redundant_pps_fixup_pps(ctx, nal->content); + err = h264_redundant_pps_fixup_pps(ctx, nal); if (err < 0) goto fail; if (!au_has_sps) { @@ -144,7 +155,7 @@ static int h264_redundant_pps_init(AVBSFContext *bsf) for (i = 0; i < au->nb_units; i++) { if (au->units[i].type == H264_NAL_PPS) { - err = h264_redundant_pps_fixup_pps(ctx, au->units[i].content); + err = h264_redundant_pps_fixup_pps(ctx, &au->units[i]); if (err < 0) goto fail; } From patchwork Sun Feb 23 23:41:12 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Thompson X-Patchwork-Id: 17882 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 3E9554488C6 for ; Mon, 24 Feb 2020 01:48:05 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 277C668B4E6; Mon, 24 Feb 2020 01:48:05 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-lj1-f173.google.com (mail-lj1-f173.google.com [209.85.208.173]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id AD86F68B4DC for ; Mon, 24 Feb 2020 01:47:58 +0200 (EET) Received: by mail-lj1-f173.google.com with SMTP id x14so8008173ljd.13 for ; Sun, 23 Feb 2020 15:47:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jkqxz-net.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=fJxZKbUNLRcPAKQFEVwqo4NVOAGFiiuqDSNgRlHhKj0=; b=1rQ60xANJ7ktOWuoMaIzcOyqJHhpgws06HEY9fl0aspPmRhgbUdBxJUAEs8Sy6VFcU gn72rzNPrSI4PbwsHbWX0AWfX1iUxmy+XMJgvxYEtuqqWEBcMhvxuVD0RL4H2fxHeQdt nhYVaVtZlDP3SWUJp8sMaJ+shJzPLTVM2mBoCLowaBf+GVmOvVYxSeeY9fKiEzUSlgC1 9lCLM2hidSF2DKp4EP1C5MTFL6rfY0lMTUrMbzgHAp7//f/Bn4LnjSOz2I+MniUPd0pG SZOBMrP+cvUmUSA6JwH72T9OL5etZunE7tQG8VMWVvP1WohvZrWhsbw9ICQDd+CDY9Pf 5Teg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=fJxZKbUNLRcPAKQFEVwqo4NVOAGFiiuqDSNgRlHhKj0=; b=okC5npG/6j0A6fcxTa2KBLVyAOn7q/FarFL/yyR0RoKSEsIoCGZtOWasrdpbTm8Z4Q r5W2n20MVnLb+wSWr8geBJnbMD8r2xJ3eRG82KQnQArc74NutrfVBVH9Bp7+r1Gd8SYb gTM760GVE+/ZX/jIRQt6kCDF89NaKvTHXE2bVqhtC7OYLhcUDPm3Zebi+dh8DQbc9AX9 qYnQu68V2ZkMrRhutBWREIxM0oJNdUUUeB1LZ335eP3+YtnIoChL9nRcmbo7szRthmCU w/qug1w7W+xjKE2z68Ab+Qa795eaeLFF1RNyGKTW8Ko7tC87Yk/oapSyDCjf/eRWfB7l kolA== X-Gm-Message-State: APjAAAV/KrBG0f7XB5s9+HpE7L08l3S8AHCHn5mirwPQVKqILFjBXq07 TlQNnWZiV5x8GWHUKTF9BzKfU6OrWZc= X-Google-Smtp-Source: APXvYqw+SuwBaCWFYQrXhd4ibt+nXz9Zos/ev58cb+H5q27/4iSka5i7AOAJKkLH6YwOlcmTf1dvPA== X-Received: by 2002:adf:edd0:: with SMTP id v16mr61624318wro.310.1582501301612; Sun, 23 Feb 2020 15:41:41 -0800 (PST) Received: from rywe.jkqxz.net (cpc91242-cmbg18-2-0-cust650.5-4.cable.virginm.net. [82.8.130.139]) by smtp.gmail.com with ESMTPSA id y131sm16358070wmc.13.2020.02.23.15.41.40 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 23 Feb 2020 15:41:41 -0800 (PST) From: Mark Thompson To: ffmpeg-devel@ffmpeg.org Date: Sun, 23 Feb 2020 23:41:12 +0000 Message-Id: <20200223234124.17689-9-sw@jkqxz.net> X-Mailer: git-send-email 2.25.0 In-Reply-To: <20200223234124.17689-1-sw@jkqxz.net> References: <20200223234124.17689-1-sw@jkqxz.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v4 09/21] cbs_h265: Use table-based alloc/free X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" --- libavcodec/cbs_h2645.c | 198 +++++++++++++++++++---------------------- 1 file changed, 94 insertions(+), 104 deletions(-) diff --git a/libavcodec/cbs_h2645.c b/libavcodec/cbs_h2645.c index 84cb70b863..62d253d042 100644 --- a/libavcodec/cbs_h2645.c +++ b/libavcodec/cbs_h2645.c @@ -443,71 +443,6 @@ static int cbs_h2645_read_more_rbsp_data(GetBitContext *gbc) #undef allocate -static void cbs_h265_free_vps(void *opaque, uint8_t *content) -{ - H265RawVPS *vps = (H265RawVPS*)content; - av_buffer_unref(&vps->extension_data.data_ref); - av_freep(&content); -} - -static void cbs_h265_free_sps(void *opaque, uint8_t *content) -{ - H265RawSPS *sps = (H265RawSPS*)content; - av_buffer_unref(&sps->extension_data.data_ref); - av_freep(&content); -} - -static void cbs_h265_free_pps(void *opaque, uint8_t *content) -{ - H265RawPPS *pps = (H265RawPPS*)content; - av_buffer_unref(&pps->extension_data.data_ref); - av_freep(&content); -} - -static void cbs_h265_free_slice(void *opaque, uint8_t *content) -{ - H265RawSlice *slice = (H265RawSlice*)content; - av_buffer_unref(&slice->data_ref); - av_freep(&content); -} - -static void cbs_h265_free_sei_payload(H265RawSEIPayload *payload) -{ - switch (payload->payload_type) { - case HEVC_SEI_TYPE_BUFFERING_PERIOD: - case HEVC_SEI_TYPE_PICTURE_TIMING: - case HEVC_SEI_TYPE_PAN_SCAN_RECT: - case HEVC_SEI_TYPE_RECOVERY_POINT: - case HEVC_SEI_TYPE_DISPLAY_ORIENTATION: - case HEVC_SEI_TYPE_ACTIVE_PARAMETER_SETS: - case HEVC_SEI_TYPE_DECODED_PICTURE_HASH: - case HEVC_SEI_TYPE_TIME_CODE: - case HEVC_SEI_TYPE_MASTERING_DISPLAY_INFO: - case HEVC_SEI_TYPE_CONTENT_LIGHT_LEVEL_INFO: - case HEVC_SEI_TYPE_ALTERNATIVE_TRANSFER_CHARACTERISTICS: - case HEVC_SEI_TYPE_ALPHA_CHANNEL_INFO: - break; - case HEVC_SEI_TYPE_USER_DATA_REGISTERED_ITU_T_T35: - av_buffer_unref(&payload->payload.user_data_registered.data_ref); - break; - case HEVC_SEI_TYPE_USER_DATA_UNREGISTERED: - av_buffer_unref(&payload->payload.user_data_unregistered.data_ref); - break; - default: - av_buffer_unref(&payload->payload.other.data_ref); - break; - } -} - -static void cbs_h265_free_sei(void *opaque, uint8_t *content) -{ - H265RawSEI *sei = (H265RawSEI*)content; - int i; - for (i = 0; i < sei->payload_count; i++) - cbs_h265_free_sei_payload(&sei->payload[i]); - av_freep(&content); -} - static int cbs_h2645_fragment_add_nals(CodedBitstreamContext *ctx, CodedBitstreamFragment *frag, const H2645Packet *packet) @@ -869,16 +804,14 @@ static int cbs_h265_read_nal_unit(CodedBitstreamContext *ctx, if (err < 0) return err; + err = ff_cbs_alloc_unit_content2(ctx, unit); + if (err < 0) + return err; + switch (unit->type) { case HEVC_NAL_VPS: { - H265RawVPS *vps; - - err = ff_cbs_alloc_unit_content(ctx, unit, sizeof(*vps), - &cbs_h265_free_vps); - if (err < 0) - return err; - vps = unit->content; + H265RawVPS *vps = unit->content; err = cbs_h265_read_vps(ctx, &gbc, vps); if (err < 0) @@ -891,13 +824,7 @@ static int cbs_h265_read_nal_unit(CodedBitstreamContext *ctx, break; case HEVC_NAL_SPS: { - H265RawSPS *sps; - - err = ff_cbs_alloc_unit_content(ctx, unit, sizeof(*sps), - &cbs_h265_free_sps); - if (err < 0) - return err; - sps = unit->content; + H265RawSPS *sps = unit->content; err = cbs_h265_read_sps(ctx, &gbc, sps); if (err < 0) @@ -911,13 +838,7 @@ static int cbs_h265_read_nal_unit(CodedBitstreamContext *ctx, case HEVC_NAL_PPS: { - H265RawPPS *pps; - - err = ff_cbs_alloc_unit_content(ctx, unit, sizeof(*pps), - &cbs_h265_free_pps); - if (err < 0) - return err; - pps = unit->content; + H265RawPPS *pps = unit->content; err = cbs_h265_read_pps(ctx, &gbc, pps); if (err < 0) @@ -946,15 +867,9 @@ static int cbs_h265_read_nal_unit(CodedBitstreamContext *ctx, case HEVC_NAL_IDR_N_LP: case HEVC_NAL_CRA_NUT: { - H265RawSlice *slice; + H265RawSlice *slice = unit->content; int pos, len; - err = ff_cbs_alloc_unit_content(ctx, unit, sizeof(*slice), - &cbs_h265_free_slice); - if (err < 0) - return err; - slice = unit->content; - err = cbs_h265_read_slice_segment_header(ctx, &gbc, &slice->header); if (err < 0) return err; @@ -980,11 +895,6 @@ static int cbs_h265_read_nal_unit(CodedBitstreamContext *ctx, case HEVC_NAL_AUD: { - err = ff_cbs_alloc_unit_content(ctx, unit, - sizeof(H265RawAUD), NULL); - if (err < 0) - return err; - err = cbs_h265_read_aud(ctx, &gbc, unit->content); if (err < 0) return err; @@ -994,12 +904,6 @@ static int cbs_h265_read_nal_unit(CodedBitstreamContext *ctx, case HEVC_NAL_SEI_PREFIX: case HEVC_NAL_SEI_SUFFIX: { - err = ff_cbs_alloc_unit_content(ctx, unit, sizeof(H265RawSEI), - &cbs_h265_free_sei); - - if (err < 0) - return err; - err = cbs_h265_read_sei(ctx, &gbc, unit->content, unit->type == HEVC_NAL_SEI_PREFIX); @@ -1478,6 +1382,90 @@ static const CodedBitstreamUnitTypeDescriptor cbs_h264_unit_types[] = { CBS_UNIT_TYPE_END_OF_LIST }; +static void cbs_h265_free_sei_payload(H265RawSEIPayload *payload) +{ + switch (payload->payload_type) { + case HEVC_SEI_TYPE_BUFFERING_PERIOD: + case HEVC_SEI_TYPE_PICTURE_TIMING: + case HEVC_SEI_TYPE_PAN_SCAN_RECT: + case HEVC_SEI_TYPE_RECOVERY_POINT: + case HEVC_SEI_TYPE_DISPLAY_ORIENTATION: + case HEVC_SEI_TYPE_ACTIVE_PARAMETER_SETS: + case HEVC_SEI_TYPE_DECODED_PICTURE_HASH: + case HEVC_SEI_TYPE_TIME_CODE: + case HEVC_SEI_TYPE_MASTERING_DISPLAY_INFO: + case HEVC_SEI_TYPE_CONTENT_LIGHT_LEVEL_INFO: + case HEVC_SEI_TYPE_ALTERNATIVE_TRANSFER_CHARACTERISTICS: + case HEVC_SEI_TYPE_ALPHA_CHANNEL_INFO: + break; + case HEVC_SEI_TYPE_USER_DATA_REGISTERED_ITU_T_T35: + av_buffer_unref(&payload->payload.user_data_registered.data_ref); + break; + case HEVC_SEI_TYPE_USER_DATA_UNREGISTERED: + av_buffer_unref(&payload->payload.user_data_unregistered.data_ref); + break; + default: + av_buffer_unref(&payload->payload.other.data_ref); + break; + } +} + +static void cbs_h265_free_sei(void *opaque, uint8_t *content) +{ + H265RawSEI *sei = (H265RawSEI*)content; + int i; + for (i = 0; i < sei->payload_count; i++) + cbs_h265_free_sei_payload(&sei->payload[i]); + av_freep(&content); +} + +static const CodedBitstreamUnitTypeDescriptor cbs_h265_unit_types[] = { + CBS_UNIT_TYPE_INTERNAL_REF(HEVC_NAL_VPS, H265RawVPS, extension_data.data), + CBS_UNIT_TYPE_INTERNAL_REF(HEVC_NAL_SPS, H265RawSPS, extension_data.data), + CBS_UNIT_TYPE_INTERNAL_REF(HEVC_NAL_PPS, H265RawPPS, extension_data.data), + + CBS_UNIT_TYPE_POD(HEVC_NAL_AUD, H265RawAUD), + + { + .nb_unit_types = 16, + .unit_types = { + HEVC_NAL_TRAIL_N, + HEVC_NAL_TRAIL_R, + HEVC_NAL_TSA_N, + HEVC_NAL_TSA_R, + HEVC_NAL_STSA_N, + HEVC_NAL_STSA_R, + HEVC_NAL_RADL_N, + HEVC_NAL_RADL_R, + HEVC_NAL_RASL_N, + HEVC_NAL_RASL_R, + HEVC_NAL_BLA_W_LP, + HEVC_NAL_BLA_W_RADL, + HEVC_NAL_BLA_N_LP, + HEVC_NAL_IDR_W_RADL, + HEVC_NAL_IDR_N_LP, + HEVC_NAL_CRA_NUT, + }, + .content_type = CBS_CONTENT_TYPE_INTERNAL_REFS, + .content_size = sizeof(H265RawSlice), + .nb_ref_offsets = 1, + .ref_offsets = { offsetof(H265RawSlice, data) }, + }, + + { + .nb_unit_types = 2, + .unit_types = { + HEVC_NAL_SEI_PREFIX, + HEVC_NAL_SEI_SUFFIX + }, + .content_type = CBS_CONTENT_TYPE_COMPLEX, + .content_size = sizeof(H265RawSEI), + .content_free = &cbs_h265_free_sei, + }, + + CBS_UNIT_TYPE_END_OF_LIST +}; + const CodedBitstreamType ff_cbs_type_h264 = { .codec_id = AV_CODEC_ID_H264, @@ -1498,6 +1486,8 @@ const CodedBitstreamType ff_cbs_type_h265 = { .priv_data_size = sizeof(CodedBitstreamH265Context), + .unit_types = cbs_h265_unit_types, + .split_fragment = &cbs_h2645_split_fragment, .read_unit = &cbs_h265_read_nal_unit, .write_unit = &cbs_h265_write_nal_unit, From patchwork Sun Feb 23 23:41:13 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Thompson X-Patchwork-Id: 17889 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 80A7644A08D for ; Mon, 24 Feb 2020 01:49:28 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 6F0C668B4CF; Mon, 24 Feb 2020 01:49:28 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wm1-f67.google.com (mail-wm1-f67.google.com [209.85.128.67]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 54ADA68B490 for ; Mon, 24 Feb 2020 01:49:22 +0200 (EET) Received: by mail-wm1-f67.google.com with SMTP id a5so7218217wmb.0 for ; Sun, 23 Feb 2020 15:49:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jkqxz-net.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=8OHbcLoG7hKdisBCfFz6HODqXCg7EZyQpQZB4zabBxs=; b=WsdNj54GeqyE3HkGgCcqkuPjaRRqS789cevZEPxifWVNs4QOrCsr/qHbQ9iKz4j5WQ 4Ong15BeK97XfjdFDzPhl2ee7wId7m6QSz3kgPhvtFXWww/KIs8SxAT6FE62YCUFWWoK 4Mjw6VZ2q6mtu7kn6R5xgXtaoPyXDoQXew3ogkMG42URB5jOrjCmrlSnmlCwXVaTxUzO P5K+IHoXNHWv9gi0KdCH0H0SUx9ZolVmZEHx/Th/0Dio6Z4/aaVBYe3JA8koX4KE3Ccf ASJp2GqouIGEL2hjELdd8JVt5do9CWDmow7eJgZgLn1Z2s3PzGRkyGGWzqhNWhFKDJwh Qhqw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=8OHbcLoG7hKdisBCfFz6HODqXCg7EZyQpQZB4zabBxs=; b=eA/WAQ/d1gXjuVVobxW9T1cfLJk7iAhYvFQjWRzSC4ho/Nm9DkIw4tHNXdYT+z3TvB 9goLxTOGBt21J6hvMGpBUQCZyFjy3SqwZSOQZ4t42l+rbdguykP501lhqFEsUcwdLoEg 4YposlSoeWjALAsM7Ksiuo4aKbPZmnvXHUCeLpi7OoiMab34DiAqq6nWrPvLz9fLAkGV /wiy2t2FQTT4rozSiRfb2xaG3hFJyS/lJZQtJejy7eR+Rm6/g8d+UN+Kq+Cyo2q+HdwT xW/whTcRTmQyyp1VL0mV2Ow2i3gg9vY9vV5ExkOJ5SetV7mILGwDu50WxQxSORdqis3B 7X8A== X-Gm-Message-State: APjAAAU6HqSP9sU7ygvy1aiLXkd6WP7rS6jYn3d0E4Ioqs/bllxGk1jr r9ePbYdh38l+sWG4NvgWm2uRT10gjJ8= X-Google-Smtp-Source: APXvYqxyL1dX31C91H54Qk7EW0wPmhPB+dMPvHWtsiQnC9fWa4oKkjORNSLGFPyW2CSHsOMgI5NTJQ== X-Received: by 2002:a05:600c:20e:: with SMTP id 14mr18191359wmi.104.1582501302577; Sun, 23 Feb 2020 15:41:42 -0800 (PST) Received: from rywe.jkqxz.net (cpc91242-cmbg18-2-0-cust650.5-4.cable.virginm.net. [82.8.130.139]) by smtp.gmail.com with ESMTPSA id y131sm16358070wmc.13.2020.02.23.15.41.41 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 23 Feb 2020 15:41:41 -0800 (PST) From: Mark Thompson To: ffmpeg-devel@ffmpeg.org Date: Sun, 23 Feb 2020 23:41:13 +0000 Message-Id: <20200223234124.17689-10-sw@jkqxz.net> X-Mailer: git-send-email 2.25.0 In-Reply-To: <20200223234124.17689-1-sw@jkqxz.net> References: <20200223234124.17689-1-sw@jkqxz.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v4 10/21] cbs_vp9: Use table-based alloc/free X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" --- libavcodec/cbs_vp9.c | 17 ++++++++--------- 1 file changed, 8 insertions(+), 9 deletions(-) diff --git a/libavcodec/cbs_vp9.c b/libavcodec/cbs_vp9.c index ec82f11c76..343bf51275 100644 --- a/libavcodec/cbs_vp9.c +++ b/libavcodec/cbs_vp9.c @@ -480,13 +480,6 @@ static int cbs_vp9_split_fragment(CodedBitstreamContext *ctx, return 0; } -static void cbs_vp9_free_frame(void *opaque, uint8_t *content) -{ - VP9RawFrame *frame = (VP9RawFrame*)content; - av_buffer_unref(&frame->data_ref); - av_freep(&frame); -} - static int cbs_vp9_read_unit(CodedBitstreamContext *ctx, CodedBitstreamUnit *unit) { @@ -498,8 +491,7 @@ static int cbs_vp9_read_unit(CodedBitstreamContext *ctx, if (err < 0) return err; - err = ff_cbs_alloc_unit_content(ctx, unit, sizeof(*frame), - &cbs_vp9_free_frame); + err = ff_cbs_alloc_unit_content2(ctx, unit); if (err < 0) return err; frame = unit->content; @@ -643,11 +635,18 @@ static int cbs_vp9_assemble_fragment(CodedBitstreamContext *ctx, return 0; } +static const CodedBitstreamUnitTypeDescriptor cbs_vp9_unit_types[] = { + CBS_UNIT_TYPE_INTERNAL_REF(0, VP9RawFrame, data), + CBS_UNIT_TYPE_END_OF_LIST +}; + const CodedBitstreamType ff_cbs_type_vp9 = { .codec_id = AV_CODEC_ID_VP9, .priv_data_size = sizeof(CodedBitstreamVP9Context), + .unit_types = cbs_vp9_unit_types, + .split_fragment = &cbs_vp9_split_fragment, .read_unit = &cbs_vp9_read_unit, .write_unit = &cbs_vp9_write_unit, From patchwork Sun Feb 23 23:41:14 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Thompson X-Patchwork-Id: 17886 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id A1E464488C6 for ; Mon, 24 Feb 2020 01:48:28 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 8841868B51C; Mon, 24 Feb 2020 01:48:28 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-ed1-f67.google.com (mail-ed1-f67.google.com [209.85.208.67]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id E1DDE68B50D for ; Mon, 24 Feb 2020 01:48:25 +0200 (EET) Received: by mail-ed1-f67.google.com with SMTP id j17so9839808edp.3 for ; Sun, 23 Feb 2020 15:48:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jkqxz-net.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=SKlODlXr4o6WwAfflOG+CR+Ig1o7mZEMPF72fH8a7QI=; b=Lh19TGZnjsYR+ily6271gaxjNKtEd8+lppLmy++1WfF4oSUhq9VtHxCbKOWBX9xeSB xWW13sVoYNETW3jDoLItI0LLtq7InRrbx+QoQiLUp7aYCZUoLVHgYYDXdKh+49bCwWst 8AC8c2tDh1mvtU5qezXcVHtPxFrIHxiON1nvbjMFFnXBBfBmcaGZ8Mj2ADvEsQb8xBsz qx37znCRMwxrh8HVIV8pjlyKJjhcgInRW3RXfZldqYQf73g4UsEZv7dP0fZLL70F8RMY QCKFPaOGu2wmdT13rqcjp78Srv6dup2XJICYXWnYCRv53AMwQSMxKEJIUE/M8usuTeHP 8TdA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=SKlODlXr4o6WwAfflOG+CR+Ig1o7mZEMPF72fH8a7QI=; b=MsPVcc9VvcWzNlgJw3kQszrCZBXtGqDIfr8L9AJ1Oa/cjk3LBVFSbSeyNCJskKSZRP 6/6+oL3W1gm+isp6LpUAo9xYzxS/MttYquGxCSOHiuJf+FjIuPQf1qHipZ+BZunx9JrE LKc/8+lXu30IZ6P4ZrLzU8UrtzYiFoNNsh9Aegx9Td2Orr97cmaEwsXtIdJBbxVBmvo4 A9hamtFt0Fvi6UxHfjZnZFn9VC4MNkxBkielpO3bzLNZN7keo9LZ0mR3AJGBLbev+BhS wLzdIEd96cjbnhEqJ8aagikusEJtLxu7KZJiFkntxOFQm+4RE6KxqbA7RXB391/VDsia Z3Dg== X-Gm-Message-State: APjAAAVeoVHzAphjxJTOlY2LPMtjMX1LywhIAoiNnemqf2JqTTJrzXhG kq8szXyLwHpucGz4zCAwrEHIBCFhejY= X-Google-Smtp-Source: APXvYqzCJ8CmKbRFEvvhZoFM1Mpkwbpvf5djSLdR1Wilz1l91axRgDC2XYJMAJHiFsm/s4HdggyvQA== X-Received: by 2002:adf:ec08:: with SMTP id x8mr58809191wrn.138.1582501303358; Sun, 23 Feb 2020 15:41:43 -0800 (PST) Received: from rywe.jkqxz.net (cpc91242-cmbg18-2-0-cust650.5-4.cable.virginm.net. [82.8.130.139]) by smtp.gmail.com with ESMTPSA id y131sm16358070wmc.13.2020.02.23.15.41.42 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 23 Feb 2020 15:41:42 -0800 (PST) From: Mark Thompson To: ffmpeg-devel@ffmpeg.org Date: Sun, 23 Feb 2020 23:41:14 +0000 Message-Id: <20200223234124.17689-11-sw@jkqxz.net> X-Mailer: git-send-email 2.25.0 In-Reply-To: <20200223234124.17689-1-sw@jkqxz.net> References: <20200223234124.17689-1-sw@jkqxz.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v4 11/21] cbs_av1: Use table-based alloc/free X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" --- libavcodec/cbs_av1.c | 85 ++++++++++++++++++++------------------------ 1 file changed, 39 insertions(+), 46 deletions(-) diff --git a/libavcodec/cbs_av1.c b/libavcodec/cbs_av1.c index 16eb7143b6..34b2634ece 100644 --- a/libavcodec/cbs_av1.c +++ b/libavcodec/cbs_av1.c @@ -812,50 +812,6 @@ fail: return err; } -static void cbs_av1_free_tile_data(AV1RawTileData *td) -{ - av_buffer_unref(&td->data_ref); -} - -static void cbs_av1_free_padding(AV1RawPadding *pd) -{ - av_buffer_unref(&pd->payload_ref); -} - -static void cbs_av1_free_metadata(AV1RawMetadata *md) -{ - switch (md->metadata_type) { - case AV1_METADATA_TYPE_ITUT_T35: - av_buffer_unref(&md->metadata.itut_t35.payload_ref); - break; - } -} - -static void cbs_av1_free_obu(void *opaque, uint8_t *content) -{ - AV1RawOBU *obu = (AV1RawOBU*)content; - - switch (obu->header.obu_type) { - case AV1_OBU_TILE_GROUP: - cbs_av1_free_tile_data(&obu->obu.tile_group.tile_data); - break; - case AV1_OBU_FRAME: - cbs_av1_free_tile_data(&obu->obu.frame.tile_group.tile_data); - break; - case AV1_OBU_TILE_LIST: - cbs_av1_free_tile_data(&obu->obu.tile_list.tile_data); - break; - case AV1_OBU_METADATA: - cbs_av1_free_metadata(&obu->obu.metadata); - break; - case AV1_OBU_PADDING: - cbs_av1_free_padding(&obu->obu.padding); - break; - } - - av_freep(&obu); -} - static int cbs_av1_ref_tile_data(CodedBitstreamContext *ctx, CodedBitstreamUnit *unit, GetBitContext *gbc, @@ -890,8 +846,7 @@ static int cbs_av1_read_unit(CodedBitstreamContext *ctx, GetBitContext gbc; int err, start_pos, end_pos; - err = ff_cbs_alloc_unit_content(ctx, unit, sizeof(*obu), - &cbs_av1_free_obu); + err = ff_cbs_alloc_unit_content2(ctx, unit); if (err < 0) return err; obu = unit->content; @@ -1260,11 +1215,49 @@ static void cbs_av1_close(CodedBitstreamContext *ctx) av_buffer_unref(&priv->frame_header_ref); } +static void cbs_av1_free_metadata(void *unit, uint8_t *content) +{ + AV1RawOBU *obu = (AV1RawOBU*)content; + AV1RawMetadata *md; + + av_assert0(obu->header.obu_type == AV1_OBU_METADATA); + md = &obu->obu.metadata; + + switch (md->metadata_type) { + case AV1_METADATA_TYPE_ITUT_T35: + av_buffer_unref(&md->metadata.itut_t35.payload_ref); + break; + } +} + +static const CodedBitstreamUnitTypeDescriptor cbs_av1_unit_types[] = { + CBS_UNIT_TYPE_POD(AV1_OBU_SEQUENCE_HEADER, AV1RawOBU), + CBS_UNIT_TYPE_POD(AV1_OBU_TEMPORAL_DELIMITER, AV1RawOBU), + CBS_UNIT_TYPE_POD(AV1_OBU_FRAME_HEADER, AV1RawOBU), + CBS_UNIT_TYPE_POD(AV1_OBU_REDUNDANT_FRAME_HEADER, AV1RawOBU), + + CBS_UNIT_TYPE_INTERNAL_REF(AV1_OBU_TILE_GROUP, AV1RawOBU, + obu.tile_group.tile_data.data), + CBS_UNIT_TYPE_INTERNAL_REF(AV1_OBU_FRAME, AV1RawOBU, + obu.frame.tile_group.tile_data.data), + CBS_UNIT_TYPE_INTERNAL_REF(AV1_OBU_TILE_LIST, AV1RawOBU, + obu.tile_list.tile_data.data), + CBS_UNIT_TYPE_INTERNAL_REF(AV1_OBU_PADDING, AV1RawOBU, + obu.padding.payload), + + CBS_UNIT_TYPE_COMPLEX(AV1_OBU_METADATA, AV1RawOBU, + &cbs_av1_free_metadata), + + CBS_UNIT_TYPE_END_OF_LIST +}; + const CodedBitstreamType ff_cbs_type_av1 = { .codec_id = AV_CODEC_ID_AV1, .priv_data_size = sizeof(CodedBitstreamAV1Context), + .unit_types = cbs_av1_unit_types, + .split_fragment = &cbs_av1_split_fragment, .read_unit = &cbs_av1_read_unit, .write_unit = &cbs_av1_write_obu, From patchwork Sun Feb 23 23:41:15 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Thompson X-Patchwork-Id: 17893 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 0AA3B44A08D for ; Mon, 24 Feb 2020 01:49:50 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id E5D2C68B547; Mon, 24 Feb 2020 01:49:49 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wr1-f65.google.com (mail-wr1-f65.google.com [209.85.221.65]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 5990568B519 for ; Mon, 24 Feb 2020 01:49:48 +0200 (EET) Received: by mail-wr1-f65.google.com with SMTP id p18so4644342wre.9 for ; Sun, 23 Feb 2020 15:49:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jkqxz-net.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=pT5E494IFLHCODtDQ0rr1fFO84XHLrJg/9HFf4O9/94=; b=OI0zEPfbF7PzeK+yuix5c70z+WW6etklRXhulmIOsFAxbzjdVOCxDDIRsB7g5Ok+St lbgvFnV5KRK9DprQCLHQl9UtuTtwdncc+ND0lKEBGKAncX/7DHM7MwQMaFhYNlhSols2 x4ZZfN14X9NQffxqRkALvlzenG3+jmBxqfWVIS7F36F6nSe80WhvDAKC8Hvz3gtsp6iu aAgPLfBjOT1SUxzjsV42xS9Qv0nS6+2MWYa0Bs/q1igfteudSgnhlEK8GfR78/eQT9Io lAKsmF98nqBHEylkOkE9Kfo1SXkc32OQ5VtVSOqNpmzOmxycQmuxNok/cr/+ol8jzNvC 23zA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=pT5E494IFLHCODtDQ0rr1fFO84XHLrJg/9HFf4O9/94=; b=IFBfYuybTFR978QaiZm6zNxkx0YHrRuxiBaqK7yVCVhh02fm/EaCiMYnWyL39rZxGc vuYjbHUvd4I9E+CUAIYhTYoQC3SFOur/E9+w4FzOdorlA5wzi3dJENThXzZ8zKpeVeiY 1TK4CzNTOTwI2NXfWKQSXDHL5C+9FR64n3lNQ8dU/+CBQImLqfZS5l8L0oIQiTzlRrwC l7rMkWNF5x5zGu21rNlXZtGMLFa2KMkXYsW9Ogvqt7oxVFgIqiSt5N4UJACtL83sGnxp 8TKo9jde1xr/bucbxe8Elt/LZAFC34NBUy3Dx+staFwtlyovubmRUKunH2bbH96OHPqc UAxQ== X-Gm-Message-State: APjAAAUGaHqnGKs8GbVkEqRLvOHiblEi39wxtnSXU0iy6TGBaTyOElp6 0pzwpACQdRUCZouooxo+fwE3Zgk5M84= X-Google-Smtp-Source: APXvYqxddor2LHK8IdkH38cH7o3v7DaxSj+aAIYv51lN91Akm2sApJ/4kYWgMlTMrsbI3/j463wlQw== X-Received: by 2002:adf:eec3:: with SMTP id a3mr59296254wrp.337.1582501304239; Sun, 23 Feb 2020 15:41:44 -0800 (PST) Received: from rywe.jkqxz.net (cpc91242-cmbg18-2-0-cust650.5-4.cable.virginm.net. [82.8.130.139]) by smtp.gmail.com with ESMTPSA id y131sm16358070wmc.13.2020.02.23.15.41.43 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 23 Feb 2020 15:41:43 -0800 (PST) From: Mark Thompson To: ffmpeg-devel@ffmpeg.org Date: Sun, 23 Feb 2020 23:41:15 +0000 Message-Id: <20200223234124.17689-12-sw@jkqxz.net> X-Mailer: git-send-email 2.25.0 In-Reply-To: <20200223234124.17689-1-sw@jkqxz.net> References: <20200223234124.17689-1-sw@jkqxz.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v4 12/21] cbs_mpeg2: Use table-based alloc/free X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" --- libavcodec/cbs_mpeg2.c | 70 +++++++++++++++++++++--------------------- 1 file changed, 35 insertions(+), 35 deletions(-) diff --git a/libavcodec/cbs_mpeg2.c b/libavcodec/cbs_mpeg2.c index 0e5d08ecbf..fabf8f7412 100644 --- a/libavcodec/cbs_mpeg2.c +++ b/libavcodec/cbs_mpeg2.c @@ -140,28 +140,6 @@ #undef infer -static void cbs_mpeg2_free_picture_header(void *opaque, uint8_t *content) -{ - MPEG2RawPictureHeader *picture = (MPEG2RawPictureHeader*)content; - av_buffer_unref(&picture->extra_information_picture.extra_information_ref); - av_freep(&content); -} - -static void cbs_mpeg2_free_user_data(void *opaque, uint8_t *content) -{ - MPEG2RawUserData *user = (MPEG2RawUserData*)content; - av_buffer_unref(&user->user_data_ref); - av_freep(&content); -} - -static void cbs_mpeg2_free_slice(void *opaque, uint8_t *content) -{ - MPEG2RawSlice *slice = (MPEG2RawSlice*)content; - av_buffer_unref(&slice->header.extra_information_slice.extra_information_ref); - av_buffer_unref(&slice->data_ref); - av_freep(&content); -} - static int cbs_mpeg2_split_fragment(CodedBitstreamContext *ctx, CodedBitstreamFragment *frag, int header) @@ -231,16 +209,14 @@ static int cbs_mpeg2_read_unit(CodedBitstreamContext *ctx, if (err < 0) return err; + err = ff_cbs_alloc_unit_content2(ctx, unit); + if (err < 0) + return err; + if (MPEG2_START_IS_SLICE(unit->type)) { - MPEG2RawSlice *slice; + MPEG2RawSlice *slice = unit->content; int pos, len; - err = ff_cbs_alloc_unit_content(ctx, unit, sizeof(*slice), - &cbs_mpeg2_free_slice); - if (err < 0) - return err; - slice = unit->content; - err = cbs_mpeg2_read_slice_header(ctx, &gbc, &slice->header); if (err < 0) return err; @@ -264,12 +240,7 @@ static int cbs_mpeg2_read_unit(CodedBitstreamContext *ctx, #define START(start_code, type, read_func, free_func) \ case start_code: \ { \ - type *header; \ - err = ff_cbs_alloc_unit_content(ctx, unit, \ - sizeof(*header), free_func); \ - if (err < 0) \ - return err; \ - header = unit->content; \ + type *header = unit->content; \ err = cbs_mpeg2_read_ ## read_func(ctx, &gbc, header); \ if (err < 0) \ return err; \ @@ -420,11 +391,40 @@ static int cbs_mpeg2_assemble_fragment(CodedBitstreamContext *ctx, return 0; } +static const CodedBitstreamUnitTypeDescriptor cbs_mpeg2_unit_types[] = { + CBS_UNIT_TYPE_INTERNAL_REF(MPEG2_START_PICTURE, MPEG2RawPictureHeader, + extra_information_picture.extra_information), + + { + .nb_unit_types = CBS_UNIT_TYPE_RANGE, + .unit_type_range_start = 0x01, + .unit_type_range_end = 0xaf, + + .content_type = CBS_CONTENT_TYPE_INTERNAL_REFS, + .content_size = sizeof(MPEG2RawSlice), + .nb_ref_offsets = 2, + .ref_offsets = { offsetof(MPEG2RawSlice, header.extra_information_slice.extra_information), + offsetof(MPEG2RawSlice, data) }, + }, + + CBS_UNIT_TYPE_INTERNAL_REF(MPEG2_START_USER_DATA, MPEG2RawUserData, + user_data), + + CBS_UNIT_TYPE_POD(MPEG2_START_SEQUENCE_HEADER, MPEG2RawSequenceHeader), + CBS_UNIT_TYPE_POD(MPEG2_START_EXTENSION, MPEG2RawExtensionData), + CBS_UNIT_TYPE_POD(MPEG2_START_SEQUENCE_END, MPEG2RawSequenceEnd), + CBS_UNIT_TYPE_POD(MPEG2_START_GROUP, MPEG2RawGroupOfPicturesHeader), + + CBS_UNIT_TYPE_END_OF_LIST +}; + const CodedBitstreamType ff_cbs_type_mpeg2 = { .codec_id = AV_CODEC_ID_MPEG2VIDEO, .priv_data_size = sizeof(CodedBitstreamMPEG2Context), + .unit_types = cbs_mpeg2_unit_types, + .split_fragment = &cbs_mpeg2_split_fragment, .read_unit = &cbs_mpeg2_read_unit, .write_unit = &cbs_mpeg2_write_unit, From patchwork Sun Feb 23 23:41:16 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Thompson X-Patchwork-Id: 17888 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id B506544A08D for ; Mon, 24 Feb 2020 01:49:24 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 99A9368B519; Mon, 24 Feb 2020 01:49:24 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-ed1-f67.google.com (mail-ed1-f67.google.com [209.85.208.67]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 9031D68B490 for ; Mon, 24 Feb 2020 01:49:18 +0200 (EET) Received: by mail-ed1-f67.google.com with SMTP id v28so9785992edw.12 for ; Sun, 23 Feb 2020 15:49:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jkqxz-net.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=5XqoqRIwZLdaPWIG0YwU48zasXFqIG/AnlSMBawDreU=; b=gucTORP9EdcrSbPEoz09zmIs85e3u3CwhfVsOXNvcg1+MgHntvemf+fP9IB6yCzCp/ cLo8sCr9sUvypHDFV6nZWLmdlPn+dr6hVwHTFzYpuWIeHAhHwM7AnhceThSXtlw9YUt+ nWALXmnEcdZ3CEuzp4kL3HCQdN7za4KrPbIL5yOgNCMBnQl6HdXqIW6XjGQgrR7EAuVB hkY9793KuxGzDj0H+iE0x47U7+PBFjqQW2S5zoSbt31as/8qygr4ipg/YYP+9oVhl+No s9Ndl5OKVhyGyOJDyh1RvgyFlFi7llTJqskjXXxuuViqzrPWq36hfuuGFYTUBAbTEs1S HmzQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=5XqoqRIwZLdaPWIG0YwU48zasXFqIG/AnlSMBawDreU=; b=keVymKA2ljuHIPuNQZwttAlhOgOT4K7dXDhcRcKL2x6K8AYYIb5iGgWXcUVG0XxQCM WKQUSJxYg8GIfBYAemgzCRFy/rKXuKNuwi4M3yTkQEv+DSWLoN/FyNSqTbBAQf6vHJIV iatCDJCioD2mLpwN0C+6MSmi2D7CLMsQh8qNLtW0vDVGRJPJ9fZPYvtZ5eUN/Enmm8O4 tV1FPL/i0iwsK0qhzygQ9yD8SGIs9ObHqvnUd0ath0vyKnkt3aKoLzcf/HBCXwh2ACYt K276obHNAaM5QWrRzlIDnZIwdWx+OkGs1kgQviPdu4qfS6BOM2888ywjWLGjwyBx6lqC js4g== X-Gm-Message-State: APjAAAVDnEFCbTI6T0EO1FisBSUsaDRuDHCyCjLW24Xg6Rg1I+5Dvt7p ghlVSbSifXspmVFFyIA5AOVOuS0DNBE= X-Google-Smtp-Source: APXvYqzZHxacUUmANj+0nQW+EBuU3MqNwfJpV5QKW7yyFSK5+JU7dJIZl9x71xeEJFScDtqF1t+kdA== X-Received: by 2002:adf:e543:: with SMTP id z3mr61052474wrm.369.1582501305038; Sun, 23 Feb 2020 15:41:45 -0800 (PST) Received: from rywe.jkqxz.net (cpc91242-cmbg18-2-0-cust650.5-4.cable.virginm.net. [82.8.130.139]) by smtp.gmail.com with ESMTPSA id y131sm16358070wmc.13.2020.02.23.15.41.44 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 23 Feb 2020 15:41:44 -0800 (PST) From: Mark Thompson To: ffmpeg-devel@ffmpeg.org Date: Sun, 23 Feb 2020 23:41:16 +0000 Message-Id: <20200223234124.17689-13-sw@jkqxz.net> X-Mailer: git-send-email 2.25.0 In-Reply-To: <20200223234124.17689-1-sw@jkqxz.net> References: <20200223234124.17689-1-sw@jkqxz.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v4 13/21] cbs: Expose the function to insert a new empty unit into a fragment X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" This will be helpful when adding new SEI to an existing access unit. --- libavcodec/cbs.c | 10 +++++----- libavcodec/cbs.h | 7 +++++++ 2 files changed, 12 insertions(+), 5 deletions(-) diff --git a/libavcodec/cbs.c b/libavcodec/cbs.c index 91788f6dfb..12a742b40a 100644 --- a/libavcodec/cbs.c +++ b/libavcodec/cbs.c @@ -680,9 +680,9 @@ int ff_cbs_alloc_unit_data(CodedBitstreamContext *ctx, return 0; } -static int cbs_insert_unit(CodedBitstreamContext *ctx, - CodedBitstreamFragment *frag, - int position) +int ff_cbs_insert_unit(CodedBitstreamContext *ctx, + CodedBitstreamFragment *frag, + int position) { CodedBitstreamUnit *units; @@ -742,7 +742,7 @@ int ff_cbs_insert_unit_content(CodedBitstreamContext *ctx, content_ref = NULL; } - err = cbs_insert_unit(ctx, frag, position); + err = ff_cbs_insert_unit(ctx, frag, position); if (err < 0) { av_buffer_unref(&content_ref); return err; @@ -781,7 +781,7 @@ int ff_cbs_insert_unit_data(CodedBitstreamContext *ctx, return AVERROR(ENOMEM); } - err = cbs_insert_unit(ctx, frag, position); + err = ff_cbs_insert_unit(ctx, frag, position); if (err < 0) { av_buffer_unref(&data_ref); return err; diff --git a/libavcodec/cbs.h b/libavcodec/cbs.h index eb7f4b878b..09b06a047d 100644 --- a/libavcodec/cbs.h +++ b/libavcodec/cbs.h @@ -370,6 +370,13 @@ int ff_cbs_alloc_unit_data(CodedBitstreamContext *ctx, CodedBitstreamUnit *unit, size_t size); +/** + * Insert a new empty unit into a fragment. + */ +int ff_cbs_insert_unit(CodedBitstreamContext *ctx, + CodedBitstreamFragment *frag, + int position); + /** * Insert a new unit into a fragment with the given content. * From patchwork Sun Feb 23 23:41:17 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Thompson X-Patchwork-Id: 17894 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 17DD544A08D for ; Mon, 24 Feb 2020 01:49:52 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 06E9A68B549; Mon, 24 Feb 2020 01:49:52 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wm1-f45.google.com (mail-wm1-f45.google.com [209.85.128.45]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 56AEC68B549 for ; Mon, 24 Feb 2020 01:49:50 +0200 (EET) Received: by mail-wm1-f45.google.com with SMTP id q9so7213223wmj.5 for ; Sun, 23 Feb 2020 15:49:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jkqxz-net.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=0Irl4GrMiP6ooqMRKdbFcPNVLIsQSFByRR/9hfHK4+4=; b=EAAwIRvu1Xr2gyO8LUih3MQmljldpyNTHNGkVV9w7hesqCC+OcbMnQFFgIqR3PGBmT oR6G00lCvVpqVeEgnIV0rapxtP4ImeNpsvB5fVDNKGwe0wZpRyCM6Vtkri9GZDVbK8Qw btvZNYg/auvEkkSvzuWxtpnYol6fHuiITNbA6N6KO+0ir/ZEuc4ut+EYDZioqP7u2pME HoTWpEsLUt4mHA+1JDwzfn6USBP9ykrOdSShwybatqHvyiLXWSwCADbZYnj7Cb7/UtCE k55tjid0A81OmFCMajCipsJDlD/ACqIHRHNj7EtxzvoJuqMQ++f3drxa2Z1MRaC+mYnR utLA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=0Irl4GrMiP6ooqMRKdbFcPNVLIsQSFByRR/9hfHK4+4=; b=A9guiJSHK/t13buNoHU5cA3J10RTe1dHmeg3V1Mfyif23XA1PpjgSJswONx21E4AuO rv3gCA7DecfrMY+TCrOp/EqSmXngYMebgQjTyyVTVam0XqawwXTFYHsgvk0EXMu/Ji5N ONRC0r51QN+mxeXp0ZbTRgjM2fGFfhIC5h/opjJQvDt1jVSyNM2yPwMS8n0JHh+a9s/v C7DYcWySXgevlP5Bh6pBY0lgn+oW/UbuSVdYgfKkx8CBR+qMJVijim/NHJ7DicxIQ9cl yCpDnJChY5+cL9Dm5UnfyGsvq//jzEMz33JagqWnfz3g7ZacjX333H3i/2SczNTsaaCv +Yuw== X-Gm-Message-State: APjAAAVcjKHrcw4fU3RtSDRJ2RgqX5agfzDQGjTeHk9dAl0N8gskI6xF mirZSy/E3gNQ1ViNFCF6GXmZNEPC9fw= X-Google-Smtp-Source: APXvYqyzaEc70/4JgHhrRsDFAl4DdubO74aBFWHcs5cuYjQzQhkoOXlFpYdLzI0IeQIb89hxOpz6BQ== X-Received: by 2002:a1c:a381:: with SMTP id m123mr18100334wme.158.1582501305853; Sun, 23 Feb 2020 15:41:45 -0800 (PST) Received: from rywe.jkqxz.net (cpc91242-cmbg18-2-0-cust650.5-4.cable.virginm.net. [82.8.130.139]) by smtp.gmail.com with ESMTPSA id y131sm16358070wmc.13.2020.02.23.15.41.45 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 23 Feb 2020 15:41:45 -0800 (PST) From: Mark Thompson To: ffmpeg-devel@ffmpeg.org Date: Sun, 23 Feb 2020 23:41:17 +0000 Message-Id: <20200223234124.17689-14-sw@jkqxz.net> X-Mailer: git-send-email 2.25.0 In-Reply-To: <20200223234124.17689-1-sw@jkqxz.net> References: <20200223234124.17689-1-sw@jkqxz.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v4 14/21] cbs_h264: Simplify SEI addition X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" At the same time, move the H.264 SEI functions to a new file - the combined H.26[45] CBS file is already very large, and these functions do not require any of the common read/write elements. --- libavcodec/Makefile | 2 +- libavcodec/cbs_h264.c | 106 +++++++++++++++++++++++++++++++++++++++++ libavcodec/cbs_h264.h | 11 +++++ libavcodec/cbs_h2645.c | 96 +------------------------------------ 4 files changed, 120 insertions(+), 95 deletions(-) create mode 100644 libavcodec/cbs_h264.c diff --git a/libavcodec/Makefile b/libavcodec/Makefile index 12704af6bd..0c4547f3a1 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -64,7 +64,7 @@ OBJS-$(CONFIG_BSWAPDSP) += bswapdsp.o OBJS-$(CONFIG_CABAC) += cabac.o OBJS-$(CONFIG_CBS) += cbs.o OBJS-$(CONFIG_CBS_AV1) += cbs_av1.o -OBJS-$(CONFIG_CBS_H264) += cbs_h2645.o h2645_parse.o +OBJS-$(CONFIG_CBS_H264) += cbs_h2645.o cbs_h264.o h2645_parse.o OBJS-$(CONFIG_CBS_H265) += cbs_h2645.o h2645_parse.o OBJS-$(CONFIG_CBS_JPEG) += cbs_jpeg.o OBJS-$(CONFIG_CBS_MPEG2) += cbs_mpeg2.o diff --git a/libavcodec/cbs_h264.c b/libavcodec/cbs_h264.c new file mode 100644 index 0000000000..75759c7f25 --- /dev/null +++ b/libavcodec/cbs_h264.c @@ -0,0 +1,106 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "cbs_h264.h" + +int ff_cbs_h264_add_sei_message(CodedBitstreamContext *ctx, + CodedBitstreamFragment *au, + H264RawSEIPayload *payload) +{ + H264RawSEI *sei = NULL; + int err, i; + + // Find an existing SEI NAL unit to add to. + for (i = 0; i < au->nb_units; i++) { + if (au->units[i].type == H264_NAL_SEI) { + sei = au->units[i].content; + if (sei->payload_count < H264_MAX_SEI_PAYLOADS) + break; + + sei = NULL; + } + } + + if (!sei) { + // Need to make a new SEI NAL unit. Insert it before the first + // slice data NAL unit; if no slice data, add at the end. + CodedBitstreamUnit *unit; + + for (i = 0; i < au->nb_units; i++) { + if (au->units[i].type == H264_NAL_SLICE || + au->units[i].type == H264_NAL_IDR_SLICE) + break; + } + + err = ff_cbs_insert_unit(ctx, au, i); + if (err < 0) + goto fail; + unit = &au->units[i]; + + err = ff_cbs_alloc_unit_content2(ctx, unit); + if (err < 0) { + ff_cbs_delete_unit(ctx, au, i); + goto fail; + } + sei = unit->content; + + *sei = (H264RawSEI) { + .nal_unit_header = { + .nal_unit_type = H264_NAL_SEI, + }, + }; + } + + memcpy(&sei->payload[sei->payload_count], payload, sizeof(*payload)); + ++sei->payload_count; + + return 0; +fail: + ff_cbs_h264_free_sei_payload(payload); + return err; +} + +void ff_cbs_h264_delete_sei_message(CodedBitstreamContext *ctx, + CodedBitstreamFragment *au, + CodedBitstreamUnit *nal, + int position) +{ + H264RawSEI *sei = nal->content; + + av_assert0(nal->type == H264_NAL_SEI); + av_assert0(position >= 0 && position < sei->payload_count); + + if (position == 0 && sei->payload_count == 1) { + // Deleting NAL unit entirely. + int i; + + for (i = 0; i < au->nb_units; i++) { + if (&au->units[i] == nal) + break; + } + + ff_cbs_delete_unit(ctx, au, i); + } else { + ff_cbs_h264_free_sei_payload(&sei->payload[position]); + + --sei->payload_count; + memmove(sei->payload + position, + sei->payload + position + 1, + (sei->payload_count - position) * sizeof(*sei->payload)); + } +} diff --git a/libavcodec/cbs_h264.h b/libavcodec/cbs_h264.h index 65659ae52c..abc0c1b732 100644 --- a/libavcodec/cbs_h264.h +++ b/libavcodec/cbs_h264.h @@ -466,11 +466,22 @@ typedef struct CodedBitstreamH264Context { } CodedBitstreamH264Context; +/** + * Free an SEI payload structure. + * + * This is useful to have standalone when manipulating SEI messages inside + * an access unit. + */ +void ff_cbs_h264_free_sei_payload(H264RawSEIPayload *payload); + /** * Add an SEI message to an access unit. * * On success, the payload will be owned by a unit in access_unit; * on failure, the content of the payload will be freed. + * + * Adds at the end of an existing SEI NAL unit if one is present, otherwise + * creates a new SEI NAL unit to contain the message. */ int ff_cbs_h264_add_sei_message(CodedBitstreamContext *ctx, CodedBitstreamFragment *access_unit, diff --git a/libavcodec/cbs_h2645.c b/libavcodec/cbs_h2645.c index 62d253d042..f0bd5cacc8 100644 --- a/libavcodec/cbs_h2645.c +++ b/libavcodec/cbs_h2645.c @@ -1321,7 +1321,7 @@ static void cbs_h265_close(CodedBitstreamContext *ctx) av_buffer_unref(&h265->pps_ref[i]); } -static void cbs_h264_free_sei_payload(H264RawSEIPayload *payload) +void ff_cbs_h264_free_sei_payload(H264RawSEIPayload *payload) { switch (payload->payload_type) { case H264_SEI_TYPE_BUFFERING_PERIOD: @@ -1349,7 +1349,7 @@ static void cbs_h264_free_sei(void *opaque, uint8_t *content) H264RawSEI *sei = (H264RawSEI*)content; int i; for (i = 0; i < sei->payload_count; i++) - cbs_h264_free_sei_payload(&sei->payload[i]); + ff_cbs_h264_free_sei_payload(&sei->payload[i]); av_freep(&content); } @@ -1495,95 +1495,3 @@ const CodedBitstreamType ff_cbs_type_h265 = { .close = &cbs_h265_close, }; - -int ff_cbs_h264_add_sei_message(CodedBitstreamContext *ctx, - CodedBitstreamFragment *au, - H264RawSEIPayload *payload) -{ - H264RawSEI *sei = NULL; - int err, i; - - // Find an existing SEI NAL unit to add to. - for (i = 0; i < au->nb_units; i++) { - if (au->units[i].type == H264_NAL_SEI) { - sei = au->units[i].content; - if (sei->payload_count < H264_MAX_SEI_PAYLOADS) - break; - - sei = NULL; - } - } - - if (!sei) { - // Need to make a new SEI NAL unit. Insert it before the first - // slice data NAL unit; if no slice data, add at the end. - AVBufferRef *sei_ref; - - sei = av_mallocz(sizeof(*sei)); - if (!sei) { - err = AVERROR(ENOMEM); - goto fail; - } - - sei->nal_unit_header.nal_unit_type = H264_NAL_SEI; - sei->nal_unit_header.nal_ref_idc = 0; - - sei_ref = av_buffer_create((uint8_t*)sei, sizeof(*sei), - &cbs_h264_free_sei, NULL, 0); - if (!sei_ref) { - av_freep(&sei); - err = AVERROR(ENOMEM); - goto fail; - } - - for (i = 0; i < au->nb_units; i++) { - if (au->units[i].type == H264_NAL_SLICE || - au->units[i].type == H264_NAL_IDR_SLICE) - break; - } - - err = ff_cbs_insert_unit_content(ctx, au, i, H264_NAL_SEI, - sei, sei_ref); - av_buffer_unref(&sei_ref); - if (err < 0) - goto fail; - } - - memcpy(&sei->payload[sei->payload_count], payload, sizeof(*payload)); - ++sei->payload_count; - - return 0; -fail: - cbs_h264_free_sei_payload(payload); - return err; -} - -void ff_cbs_h264_delete_sei_message(CodedBitstreamContext *ctx, - CodedBitstreamFragment *au, - CodedBitstreamUnit *nal, - int position) -{ - H264RawSEI *sei = nal->content; - - av_assert0(nal->type == H264_NAL_SEI); - av_assert0(position >= 0 && position < sei->payload_count); - - if (position == 0 && sei->payload_count == 1) { - // Deleting NAL unit entirely. - int i; - - for (i = 0; i < au->nb_units; i++) { - if (&au->units[i] == nal) - break; - } - - ff_cbs_delete_unit(ctx, au, i); - } else { - cbs_h264_free_sei_payload(&sei->payload[position]); - - --sei->payload_count; - memmove(sei->payload + position, - sei->payload + position + 1, - (sei->payload_count - position) * sizeof(*sei->payload)); - } -} From patchwork Sun Feb 23 23:41:18 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Thompson X-Patchwork-Id: 17892 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 2B5C344A08D for ; Mon, 24 Feb 2020 01:49:49 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 098D168B519; Mon, 24 Feb 2020 01:49:49 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-ed1-f65.google.com (mail-ed1-f65.google.com [209.85.208.65]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 4010168B53B for ; Mon, 24 Feb 2020 01:49:47 +0200 (EET) Received: by mail-ed1-f65.google.com with SMTP id c7so9825671edu.2 for ; Sun, 23 Feb 2020 15:49:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jkqxz-net.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=VWw/CoYpC5wU844xIATbxL7bCaRyGvbY1EWkJyAZiwU=; b=0vHpbUjHZ0n6UzP7qD7iqzqvcv5di+ab5SnBHB1fMuSswFuz86Rboi4dsPnX4aXenw vWtrUPpt5MhxrmdUIqFYmscBCfACX6ai/t5c8p1k6awwxZGIoQzmeoiEmoSgOW9zQoU6 T6Lq60xZSZSVFcV2dlERfh5rph8EReYhZT92hBTZaXyZ7IHvw1LI4db1dlxhbg6xjqR5 UaiEd0ONNnWspBZ3r41rxeFZ/2AsugdzyLWnkloRHDIT4W3ydjqHDuvt1hA+g2kMw8Qe twz8TXLDNZg6dcJug/c6xZXU8ZTdKm42GaKq+CnnS7JlRCirFXSnB9cjUkK4B4z22Zts Yr5A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=VWw/CoYpC5wU844xIATbxL7bCaRyGvbY1EWkJyAZiwU=; b=LsePJTD6xFNgrjte9lr6sCHilhp2RwjAz6p2E7+14/C3TW2G6KSF/KoO9g75WSoyVI sYHGmxii1/IRauPq6XO/Z3sFEtzlbRbAqCuFer3uh91tyOLaXotWF1cue+JaK/oxk9jm 5P+dv+ZLniDZyVHA36A22u/UPlhu9tn7Z2lkEQHY4+9VFc2n+FOyxAo+1GpxiJ5YAlrX 2Tt4VuZ93JTzknpMAi7RwMUZHfcMYYT+o+TtnnSN/SIiBsKszkrTV3rJ8zxSAIHdz9KK +9S74ypo43/RshLcRt3rVTDdsI+cSELWIbUdN/xgEOoqzCu0hpdxVpSQl5nY3CnEVg9m x24A== X-Gm-Message-State: APjAAAU+Ctz4lDaJNFdnhKzX5QqEsNBRxBV5tH388tbSGqflNosvV4B+ Wss5l5W5HJ7Q/XFjB/FsptXlvaOuCso= X-Google-Smtp-Source: APXvYqzyyODNpZAwIIP27WHjgW4XATp3DgKhM8g6esTIsP+XCDe2LJgwxnWX/CmI8+HQBAr62sqzIw== X-Received: by 2002:adf:9dc9:: with SMTP id q9mr60085833wre.286.1582501307095; Sun, 23 Feb 2020 15:41:47 -0800 (PST) Received: from rywe.jkqxz.net (cpc91242-cmbg18-2-0-cust650.5-4.cable.virginm.net. [82.8.130.139]) by smtp.gmail.com with ESMTPSA id y131sm16358070wmc.13.2020.02.23.15.41.45 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 23 Feb 2020 15:41:46 -0800 (PST) From: Mark Thompson To: ffmpeg-devel@ffmpeg.org Date: Sun, 23 Feb 2020 23:41:18 +0000 Message-Id: <20200223234124.17689-15-sw@jkqxz.net> X-Mailer: git-send-email 2.25.0 In-Reply-To: <20200223234124.17689-1-sw@jkqxz.net> References: <20200223234124.17689-1-sw@jkqxz.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v4 15/21] cbs_h264: Add support for frame packing arrangement SEI messages X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" --- libavcodec/cbs_h264.h | 24 ++++++++++++++++ libavcodec/cbs_h2645.c | 1 + libavcodec/cbs_h264_syntax_template.c | 40 +++++++++++++++++++++++++++ 3 files changed, 65 insertions(+) diff --git a/libavcodec/cbs_h264.h b/libavcodec/cbs_h264.h index abc0c1b732..512674ec07 100644 --- a/libavcodec/cbs_h264.h +++ b/libavcodec/cbs_h264.h @@ -296,6 +296,29 @@ typedef struct H264RawSEIRecoveryPoint { uint8_t changing_slice_group_idc; } H264RawSEIRecoveryPoint; +typedef struct H264RawSEIFramePackingArrangement { + uint32_t frame_packing_arrangement_id; + uint8_t frame_packing_arrangement_cancel_flag; + + uint8_t frame_packing_arrangement_type; + uint8_t quincunx_sampling_flag; + uint8_t content_interpretation_type; + uint8_t spatial_flipping_flag; + uint8_t frame0_flipped_flag; + uint8_t field_views_flag; + uint8_t current_frame_is_frame0_flag; + uint8_t frame0_self_contained_flag; + uint8_t frame1_self_contained_flag; + uint8_t frame0_grid_position_x; + uint8_t frame0_grid_position_y; + uint8_t frame1_grid_position_x; + uint8_t frame1_grid_position_y; + uint8_t frame_packing_arrangement_reserved_byte; + uint16_t frame_packing_arrangement_repetition_period; + + uint8_t frame_packing_arrangement_extension_flag; +} H264RawSEIFramePackingArrangement; + typedef struct H264RawSEIDisplayOrientation { uint8_t display_orientation_cancel_flag; uint8_t hor_flip; @@ -329,6 +352,7 @@ typedef struct H264RawSEIPayload { H264RawSEIUserDataRegistered user_data_registered; H264RawSEIUserDataUnregistered user_data_unregistered; H264RawSEIRecoveryPoint recovery_point; + H264RawSEIFramePackingArrangement frame_packing_arrangement; H264RawSEIDisplayOrientation display_orientation; H264RawSEIMasteringDisplayColourVolume mastering_display_colour_volume; H264RawSEIAlternativeTransferCharacteristics diff --git a/libavcodec/cbs_h2645.c b/libavcodec/cbs_h2645.c index f0bd5cacc8..8cea55509b 100644 --- a/libavcodec/cbs_h2645.c +++ b/libavcodec/cbs_h2645.c @@ -1328,6 +1328,7 @@ void ff_cbs_h264_free_sei_payload(H264RawSEIPayload *payload) case H264_SEI_TYPE_PIC_TIMING: case H264_SEI_TYPE_PAN_SCAN_RECT: case H264_SEI_TYPE_RECOVERY_POINT: + case H264_SEI_TYPE_FRAME_PACKING: case H264_SEI_TYPE_DISPLAY_ORIENTATION: case H264_SEI_TYPE_MASTERING_DISPLAY_COLOUR_VOLUME: case H264_SEI_TYPE_ALTERNATIVE_TRANSFER: diff --git a/libavcodec/cbs_h264_syntax_template.c b/libavcodec/cbs_h264_syntax_template.c index 878d348b94..e68944d3b8 100644 --- a/libavcodec/cbs_h264_syntax_template.c +++ b/libavcodec/cbs_h264_syntax_template.c @@ -779,6 +779,42 @@ static int FUNC(sei_recovery_point)(CodedBitstreamContext *ctx, RWContext *rw, return 0; } +static int FUNC(sei_frame_packing_arrangement)(CodedBitstreamContext *ctx, RWContext *rw, + H264RawSEIFramePackingArrangement *current) +{ + int err; + + HEADER("Frame Packing Arrangement"); + + ue(frame_packing_arrangement_id, 0, UINT32_MAX - 1); + flag(frame_packing_arrangement_cancel_flag); + + if (!current->frame_packing_arrangement_cancel_flag) { + u(7, frame_packing_arrangement_type, 0, 7); + flag(quincunx_sampling_flag); + u(6, content_interpretation_type, 0, 2); + flag(spatial_flipping_flag); + flag(frame0_flipped_flag); + flag(field_views_flag); + flag(current_frame_is_frame0_flag); + flag(frame0_self_contained_flag); + flag(frame1_self_contained_flag); + if (!current->quincunx_sampling_flag && + current->frame_packing_arrangement_type != 5) { + ub(4, frame0_grid_position_x); + ub(4, frame0_grid_position_y); + ub(4, frame1_grid_position_x); + ub(4, frame1_grid_position_y); + } + u(8, frame_packing_arrangement_reserved_byte, 0, 0); + ue(frame_packing_arrangement_repetition_period, 0, 16384); + } + + flag(frame_packing_arrangement_extension_flag); + + return 0; +} + static int FUNC(sei_display_orientation)(CodedBitstreamContext *ctx, RWContext *rw, H264RawSEIDisplayOrientation *current) { @@ -879,6 +915,10 @@ static int FUNC(sei_payload)(CodedBitstreamContext *ctx, RWContext *rw, CHECK(FUNC(sei_display_orientation) (ctx, rw, ¤t->payload.display_orientation)); break; + case H264_SEI_TYPE_FRAME_PACKING: + CHECK(FUNC(sei_frame_packing_arrangement) + (ctx, rw, ¤t->payload.frame_packing_arrangement)); + break; case H264_SEI_TYPE_MASTERING_DISPLAY_COLOUR_VOLUME: CHECK(FUNC(sei_mastering_display_colour_volume) (ctx, rw, ¤t->payload.mastering_display_colour_volume)); From patchwork Sun Feb 23 23:41:19 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Thompson X-Patchwork-Id: 17883 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id F20984488C6 for ; Mon, 24 Feb 2020 01:48:08 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id E118F68B503; Mon, 24 Feb 2020 01:48:08 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-lj1-f196.google.com (mail-lj1-f196.google.com [209.85.208.196]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 3FDD868B4DC for ; Mon, 24 Feb 2020 01:48:02 +0200 (EET) Received: by mail-lj1-f196.google.com with SMTP id a13so8019064ljm.10 for ; Sun, 23 Feb 2020 15:48:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jkqxz-net.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=JSJ7SCnI7T5B6/8Pk/NJkfuGDf2itZVndRFST2S0tG0=; b=CV0TkCt4Kn5jQCUmWD4By5V/2rEBsQx/ZCdo804smDmMVc//Bovbp6VhxUulXWitZH UoEJ2yPdOiy8/Kmvhben0T0Cy29Fzc8NsjyRWB4OraRnYu1aZzla1nVjAqhPAPVKY5cS OTEZX8akD7g53T8fSMh6ZNeXrql6+jE6jBa5Z0UwolJyhTp2gbusibQAbMz4QEbNXEIC BhGL3MJiQILh8GJP3i1Be2g27r3XVw3yb1SWjKr/vUOsT8tfbPhGWQDtwG4GYFQClhmJ MNp2vlSvU/9uQb8ZKhXd2K3RNwK4H2PS6GbIgW6/ki5IWX8UERSvYwdshGC8TjYV92fE tfhQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=JSJ7SCnI7T5B6/8Pk/NJkfuGDf2itZVndRFST2S0tG0=; b=H7zpe4146dVQv5JsK7ogWjk+1GQ0tEFDscjymlA09Z0+ajEBcwxsGR/sXuh5O02YIb WcSCIClGTvp5faFqQ8g/Ip81sSoqDep04TRVrIwdWnEp+E42vFc2vgQTT9oQZheK3GtC b7Fr+HkzoMK+HWEi9KuGibIbIdAQC1n9z9a9xxdRc+B+Iry/b1q3SFmj1+yeEz8Qsnr4 heVEexnYKOjauH7B+X2Lw90svjhrOYq/pn9qNY9rLofDlOlkE1nFdFnoEL24Unx1e+FH zxRR8gVsVO9Bj2l/OyapM2VRNxmGdedMk4FHlmCHtQsUfLR52Q2Aadr9NhKBCaCQYaZd KdLg== X-Gm-Message-State: APjAAAUogNz7AyPPDrxdbFWTB6sNd65L9RRJubuyaRJ/v5dGaQOlR0nN XbamvO4f3aJg5x2lWmPnIprJ9sZOfaA= X-Google-Smtp-Source: APXvYqzxZ9oBj8a4qhrbm61ataLT+4nxovHeKYWBVw1GhWcWYwDbteqvCOu0kJPleRalHgZHFV+O3w== X-Received: by 2002:a5d:610c:: with SMTP id v12mr59593912wrt.88.1582501308218; Sun, 23 Feb 2020 15:41:48 -0800 (PST) Received: from rywe.jkqxz.net (cpc91242-cmbg18-2-0-cust650.5-4.cable.virginm.net. [82.8.130.139]) by smtp.gmail.com with ESMTPSA id y131sm16358070wmc.13.2020.02.23.15.41.47 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 23 Feb 2020 15:41:47 -0800 (PST) From: Mark Thompson To: ffmpeg-devel@ffmpeg.org Date: Sun, 23 Feb 2020 23:41:19 +0000 Message-Id: <20200223234124.17689-16-sw@jkqxz.net> X-Mailer: git-send-email 2.25.0 In-Reply-To: <20200223234124.17689-1-sw@jkqxz.net> References: <20200223234124.17689-1-sw@jkqxz.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v4 16/21] cbs_h264: Add a function to turn stereo 3D metadata into SEI X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" --- libavcodec/cbs_h264.c | 47 +++++++++++++++++++++++++++++++++++++++++++ libavcodec/cbs_h264.h | 8 ++++++++ 2 files changed, 55 insertions(+) diff --git a/libavcodec/cbs_h264.c b/libavcodec/cbs_h264.c index 75759c7f25..cc52f68550 100644 --- a/libavcodec/cbs_h264.c +++ b/libavcodec/cbs_h264.c @@ -16,6 +16,8 @@ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA */ +#include "libavutil/stereo3d.h" + #include "cbs_h264.h" int ff_cbs_h264_add_sei_message(CodedBitstreamContext *ctx, @@ -104,3 +106,48 @@ void ff_cbs_h264_delete_sei_message(CodedBitstreamContext *ctx, (sei->payload_count - position) * sizeof(*sei->payload)); } } + +void ff_cbs_h264_fill_sei_frame_packing_arrangement(H264RawSEIFramePackingArrangement *fp, + const AVStereo3D *st) +{ + static const int type_map[] = { + [AV_STEREO3D_2D] = 6, + [AV_STEREO3D_SIDEBYSIDE] = 3, + [AV_STEREO3D_TOPBOTTOM] = 4, + [AV_STEREO3D_FRAMESEQUENCE] = 5, + [AV_STEREO3D_CHECKERBOARD] = 0, + [AV_STEREO3D_SIDEBYSIDE_QUINCUNX] = 3, + [AV_STEREO3D_LINES] = 2, + [AV_STEREO3D_COLUMNS] = 1, + }; + + memset(fp, 0, sizeof(*fp)); + + if (st->type >= FF_ARRAY_ELEMS(type_map)) + return; + + fp->frame_packing_arrangement_type = type_map[st->type]; + + fp->quincunx_sampling_flag = + st->type == AV_STEREO3D_CHECKERBOARD || + st->type == AV_STEREO3D_SIDEBYSIDE_QUINCUNX; + + if (st->type == AV_STEREO3D_2D) + fp->content_interpretation_type = 0; + else if (st->flags & AV_STEREO3D_FLAG_INVERT) + fp->content_interpretation_type = 2; + else + fp->content_interpretation_type = 1; + + if (st->type == AV_STEREO3D_FRAMESEQUENCE) { + if (st->flags & AV_STEREO3D_FLAG_INVERT) + fp->current_frame_is_frame0_flag = + st->view == AV_STEREO3D_VIEW_RIGHT; + else + fp->current_frame_is_frame0_flag = + st->view == AV_STEREO3D_VIEW_LEFT; + } + + fp->frame_packing_arrangement_repetition_period = + st->type != AV_STEREO3D_FRAMESEQUENCE; +} diff --git a/libavcodec/cbs_h264.h b/libavcodec/cbs_h264.h index 512674ec07..76211c976b 100644 --- a/libavcodec/cbs_h264.h +++ b/libavcodec/cbs_h264.h @@ -525,4 +525,12 @@ void ff_cbs_h264_delete_sei_message(CodedBitstreamContext *ctx, CodedBitstreamUnit *nal_unit, int position); +struct AVStereo3D; +/** + * Fill an SEI Frame Packing Arrangement structure with values derived from + * the AVStereo3D side-data structure. + */ +void ff_cbs_h264_fill_sei_frame_packing_arrangement(H264RawSEIFramePackingArrangement *fp, + const struct AVStereo3D *st); + #endif /* AVCODEC_CBS_H264_H */ From patchwork Sun Feb 23 23:41:20 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Thompson X-Patchwork-Id: 17897 Delivered-To: andriy.gelman@gmail.com Received: by 2002:a05:6214:1865:0:0:0:0 with SMTP id eh5csp2175056qvb; Sun, 23 Feb 2020 15:50:35 -0800 (PST) X-Google-Smtp-Source: APXvYqxHdZ6aQOtOIHd3IEZM9JuFyZwp6Je1c6JHoo6rM1Wf07N+TBf0dpiXr7Kc+5yH3RX1pgsi X-Received: by 2002:a05:6402:744:: with SMTP id p4mr27296863edy.104.1582501835231; Sun, 23 Feb 2020 15:50:35 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1582501835; cv=none; d=google.com; s=arc-20160816; b=KMEA48X+djE2kczLJDxA3SGJYczIABqt1ikd4HWWXYE4p27cFWKzCADK4poNf1+/Mm weEKJTat+J3a2bY7Q29axheb195HxIK49kDz1tqIYjW5O67pIgFMvYtdiqZNLcOAudtm miCuUu4eoCJeOW9c9XbCHES49kSeMROm3kSA+6+H7//50JCDoyqPN8tHRJ9zPaenO3w8 ezR9rcPfeiP4ghPRClvWUbywq1IyCSvxsKwhYNXwPZNG62M3puIuXyRP9c/2vmSYnv+h aWOW6AwuuNVP4zkeDrwgQSxfxcLsp7J31QYoW54GYuxHH/KFjI46sw6mrs+YOo3AF95r rIiQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:dkim-signature:delivered-to; bh=r1+o0PjHu11fS1abkOu6reQY3gcfkTHIdJPlDjvOX0o=; b=LJ95uHeFSfVWArZsAocclI8v/jMYwjMgnkCUa4EMxs1Zp2c7aQZI3QpTeOqAhNywUR DRdasBzuQCQ9S89FtApcNVUPl66zT/CO7wsp0hDjjs6vgHCVUy06NU4fA28vuGCl+K1l RAu2RdriA1ipKp4UcFyjWXsyVoc6Zw/hctDnQT4LvyEU9c6Hp08yvjvPgVFGzcyhI8OS pAC5nsru5COpkSBq86+RMEUW80dxx/CoqcofQsmQX25PqhTTBuMnLhSO/ra2d/TcccpW MrHOM+Ui89lfFYwAH0JdWx0mHRIvGZ5l/2Abh9TQudB9HsHBN4YLl+xHL1QwgthTLVMA ZluA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@jkqxz-net.20150623.gappssmtp.com header.s=20150623 header.b=DIbQgaYz; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 10si5885816ejo.50.2020.02.23.15.50.34; Sun, 23 Feb 2020 15:50:35 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@jkqxz-net.20150623.gappssmtp.com header.s=20150623 header.b=DIbQgaYz; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 537A668B569; Mon, 24 Feb 2020 01:50:04 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wm1-f68.google.com (mail-wm1-f68.google.com [209.85.128.68]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id D7B4768B53B for ; Mon, 24 Feb 2020 01:50:02 +0200 (EET) Received: by mail-wm1-f68.google.com with SMTP id m10so9485082wmc.0 for ; Sun, 23 Feb 2020 15:50:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jkqxz-net.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=HF4AAmE9RVcIyYnBhZKqR5HQct8iO0B+N2656ELW8cU=; b=DIbQgaYzokpY6Hf2ERdS69nFT/jv12MUOLeQXkbGM4THlWwYWJtOQvfFslEVyJaEKA +GpwFnrI8PWDOosMcUppT4BA9chFsTPo59NpeVZ8NbdLHYNRSk554/x5xTpaotcsvqKs XOgdv8blw6KsE3TbyoqYTr39K6ZmrskscRLe+Hw0DmF7czIKHQn7CPCw7rca9hAF+s8e d9W4FY+KzBHe+N5TJRqjIW3vQsSoq2mSK7AOjVlwHpDKFG19KEPPV7rPF0RpvLW0P9Xz klx8OgFdy2S5x1Rj869PCCGOgfwzg0fOKbiqcaI3VBUvOuTH/knhdMF0E9Ng8uUHmQ3f g1aw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=HF4AAmE9RVcIyYnBhZKqR5HQct8iO0B+N2656ELW8cU=; b=ZlvCfiqVbWtpYKCtmUGSWauXpbGlCSWP8FB0P9cDdMRsy2+jasQLPan5l6HVo24T76 P+fERAZCkf6c16T306Z5ETK3bjfjZZVduHE+Y6+WfD31Iv9zIevPnmt2yxgmrgc4fnWp k5HPzAPrV8mzFDc+jo4kTvycGpBGCG4FI4XuBtvqc/NN7CiaQJb7OrBITwqEoylGwmrB QkFxpoXdOXSiEC0nuIMcNBZSJQVG9qM1v4SNpAtl5MAaQH/o7r6ppDhsbG936mwCecgJ UCzzZq9uLa3/AHhzzQPj+giexlyFWhgeZ6cLEIecajzCZjteddA0UCek8+4MaOyPMBBr UVwg== X-Gm-Message-State: APjAAAVpg9LUSGev7R/wLmG0uHqiajE14o/J62mDYDkLZuvE/tWwzbSg 8SOy2pphKQCyzhyDhIiNsSsOwoc7iKw= X-Received: by 2002:a1c:1dc7:: with SMTP id d190mr18544183wmd.48.1582501308965; Sun, 23 Feb 2020 15:41:48 -0800 (PST) Received: from rywe.jkqxz.net (cpc91242-cmbg18-2-0-cust650.5-4.cable.virginm.net. [82.8.130.139]) by smtp.gmail.com with ESMTPSA id y131sm16358070wmc.13.2020.02.23.15.41.48 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 23 Feb 2020 15:41:48 -0800 (PST) From: Mark Thompson To: ffmpeg-devel@ffmpeg.org Date: Sun, 23 Feb 2020 23:41:20 +0000 Message-Id: <20200223234124.17689-17-sw@jkqxz.net> X-Mailer: git-send-email 2.25.0 In-Reply-To: <20200223234124.17689-1-sw@jkqxz.net> References: <20200223234124.17689-1-sw@jkqxz.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v4 17/21] vaapi_encode_h264: Support stereo 3D metadata X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: hHoApfPiwWwQ Content-Length: 5735 Insert frame packing arrangement messages into the stream when input frames have associated stereo 3D side-data. --- doc/encoders.texi | 3 +++ libavcodec/vaapi_encode_h264.c | 25 ++++++++++++++++++++++++- 2 files changed, 27 insertions(+), 1 deletion(-) diff --git a/doc/encoders.texi b/doc/encoders.texi index e23b6b32fe..62b6902197 100644 --- a/doc/encoders.texi +++ b/doc/encoders.texi @@ -3065,6 +3065,9 @@ Include picture timing parameters (@emph{buffering_period} and @emph{pic_timing} messages). @item recovery_point Include recovery points where appropriate (@emph{recovery_point} messages). +@item frame_packing +Include stereo 3D metadata if the input frames have it +(@emph{frame_packing_arrangement} messages). @end table @end table diff --git a/libavcodec/vaapi_encode_h264.c b/libavcodec/vaapi_encode_h264.c index f4965d8b09..58eae613c4 100644 --- a/libavcodec/vaapi_encode_h264.c +++ b/libavcodec/vaapi_encode_h264.c @@ -25,6 +25,7 @@ #include "libavutil/common.h" #include "libavutil/internal.h" #include "libavutil/opt.h" +#include "libavutil/stereo3d.h" #include "avcodec.h" #include "cbs.h" @@ -39,6 +40,7 @@ enum { SEI_TIMING = 0x01, SEI_IDENTIFIER = 0x02, SEI_RECOVERY_POINT = 0x04, + SEI_FRAME_PACKING = 0x20, }; // Random (version 4) ISO 11578 UUID. @@ -96,6 +98,7 @@ typedef struct VAAPIEncodeH264Context { H264RawSEIBufferingPeriod sei_buffering_period; H264RawSEIPicTiming sei_pic_timing; H264RawSEIRecoveryPoint sei_recovery_point; + H264RawSEIFramePackingArrangement sei_frame_packing; H264RawSEIUserDataUnregistered sei_identifier; char *sei_identifier_string; @@ -251,6 +254,12 @@ static int vaapi_encode_h264_write_extra_header(AVCodecContext *avctx, sei->payload[i].payload.recovery_point = priv->sei_recovery_point; ++i; } + if (priv->sei_needed & SEI_FRAME_PACKING) { + sei->payload[i].payload_type = H264_SEI_TYPE_FRAME_PACKING; + sei->payload[i].payload.frame_packing_arrangement = + priv->sei_frame_packing; + ++i; + } sei->payload_count = i; av_assert0(sei->payload_count > 0); @@ -700,6 +709,17 @@ static int vaapi_encode_h264_init_picture_params(AVCodecContext *avctx, priv->sei_needed |= SEI_RECOVERY_POINT; } + if (priv->sei & SEI_FRAME_PACKING) { + AVFrameSideData *sd = av_frame_get_side_data(pic->input_image, + AV_FRAME_DATA_STEREO3D); + if (sd) { + ff_cbs_h264_fill_sei_frame_packing_arrangement( + &priv->sei_frame_packing, (const AVStereo3D*)sd->data); + } + + priv->sei_needed |= SEI_FRAME_PACKING; + } + vpic->CurrPic = (VAPictureH264) { .picture_id = pic->recon_surface, .frame_idx = hpic->frame_num, @@ -1271,7 +1291,7 @@ static const AVOption vaapi_encode_h264_options[] = { { "sei", "Set SEI to include", OFFSET(sei), AV_OPT_TYPE_FLAGS, - { .i64 = SEI_IDENTIFIER | SEI_TIMING | SEI_RECOVERY_POINT }, + { .i64 = SEI_IDENTIFIER | SEI_TIMING | SEI_RECOVERY_POINT | SEI_FRAME_PACKING }, 0, INT_MAX, FLAGS, "sei" }, { "identifier", "Include encoder version identifier", 0, AV_OPT_TYPE_CONST, { .i64 = SEI_IDENTIFIER }, @@ -1282,6 +1302,9 @@ static const AVOption vaapi_encode_h264_options[] = { { "recovery_point", "Include recovery points where appropriate", 0, AV_OPT_TYPE_CONST, { .i64 = SEI_RECOVERY_POINT }, INT_MIN, INT_MAX, FLAGS, "sei" }, + { "frame_packing", "Include frame packing arrangement for stereo 3D", + 0, AV_OPT_TYPE_CONST, { .i64 = SEI_FRAME_PACKING }, + INT_MIN, INT_MAX, FLAGS, "sei" }, { "profile", "Set profile (profile_idc and constraint_set*_flag)", OFFSET(profile), AV_OPT_TYPE_INT, From patchwork Sun Feb 23 23:41:21 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Thompson X-Patchwork-Id: 17877 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id DA05D449F1B for ; Mon, 24 Feb 2020 01:41:51 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id BA72468B4BE; Mon, 24 Feb 2020 01:41:51 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wm1-f51.google.com (mail-wm1-f51.google.com [209.85.128.51]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id E9199687F1F for ; Mon, 24 Feb 2020 01:41:50 +0200 (EET) Received: by mail-wm1-f51.google.com with SMTP id a5so7209851wmb.0 for ; Sun, 23 Feb 2020 15:41:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jkqxz-net.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=TnQk8VtuBLouVH3sg27FCCbyn3m9EI8ADSlsRhD1JzU=; b=CVP8G8Ob5uH1U4fGilY399ODWmkZKXkm08RMEg0Wpaocnq/SKewVM8vUdeJmEbcFvc HKVmx6oTydbtH8QOv2qX0jIB/VlHd5uyZnTXXhGQ5f4V1ICeC1Xjm/D0WYr//mTifIzR 8pJ2nEdiMKYiCQj/DhsLmRIRjoDzqG24Wl/D5+ThLvizTXX7RYe9ANnvltGq948/ZrQT QkbrKtp+KpLwNoc7VFa9SxFJwT+shmzsII3K3VA+ffgwqDJUcGO2CkDjM4Ud+/SxIQv/ uNXwshG3vVbm5Yoh3Kl2PjTTuROtmPZa9bo8QY/94Ce/D1IMP2iCSNwWk+MOVlkUbGO+ mgBg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=TnQk8VtuBLouVH3sg27FCCbyn3m9EI8ADSlsRhD1JzU=; b=Yq84d97LSh+0nJ0PpFognQgu89RucRxf5DuosNexxz/72q6n89J8AsE6MNiOJaQuxV 6EdfIFFGffP8qLLns8JOieda2eZbnULMRe6SrtqGcRsbt/GwXw6FoxCJCgplIyZRubLU dIU6a3jNU9h0793DqKDyHzVeV6Ks4t4riOsls4YYtTk1kOn0mY1yV1HJK/Q/+PmmDt18 HA8SZbciC1k8ING1Ii2A4twORBf+unVtsbRc/quwtQBalUFtsWXMyobBQTP63lw/sRiy rnmolSK3/2G2qiIIO+xr77T478jAJ4+gi65/bXId8ZC0fF5ah3nR78+pgjWXnIOBfowb BzsA== X-Gm-Message-State: APjAAAUDJ7DX3ZKsuTe7bZ9xI3zrGuA+NTj86wlS4iFcGNC2J9rRzkzR ZmBZkxf/NeXyDNSu559E8updWXZbz9c= X-Google-Smtp-Source: APXvYqyKTD55AcG8915QBWVo8Vl8AweurZP9OGxwLAIGq0LU8lXU9JHUB+tyj9JyE6l/klLsJoPdZQ== X-Received: by 2002:a1c:4144:: with SMTP id o65mr17655307wma.81.1582501309899; Sun, 23 Feb 2020 15:41:49 -0800 (PST) Received: from rywe.jkqxz.net (cpc91242-cmbg18-2-0-cust650.5-4.cable.virginm.net. [82.8.130.139]) by smtp.gmail.com with ESMTPSA id y131sm16358070wmc.13.2020.02.23.15.41.49 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 23 Feb 2020 15:41:49 -0800 (PST) From: Mark Thompson To: ffmpeg-devel@ffmpeg.org Date: Sun, 23 Feb 2020 23:41:21 +0000 Message-Id: <20200223234124.17689-18-sw@jkqxz.net> X-Mailer: git-send-email 2.25.0 In-Reply-To: <20200223234124.17689-1-sw@jkqxz.net> References: <20200223234124.17689-1-sw@jkqxz.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v4 18/21] cbs_h265: Add functions to turn HDR metadata into SEI X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" --- libavcodec/Makefile | 2 +- libavcodec/cbs_h265.c | 99 +++++++++++++++++++++++++++++++++++++++++++ libavcodec/cbs_h265.h | 18 ++++++++ 3 files changed, 118 insertions(+), 1 deletion(-) create mode 100644 libavcodec/cbs_h265.c diff --git a/libavcodec/Makefile b/libavcodec/Makefile index 0c4547f3a1..1ce079687b 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -65,7 +65,7 @@ OBJS-$(CONFIG_CABAC) += cabac.o OBJS-$(CONFIG_CBS) += cbs.o OBJS-$(CONFIG_CBS_AV1) += cbs_av1.o OBJS-$(CONFIG_CBS_H264) += cbs_h2645.o cbs_h264.o h2645_parse.o -OBJS-$(CONFIG_CBS_H265) += cbs_h2645.o h2645_parse.o +OBJS-$(CONFIG_CBS_H265) += cbs_h2645.o cbs_h265.o h2645_parse.o OBJS-$(CONFIG_CBS_JPEG) += cbs_jpeg.o OBJS-$(CONFIG_CBS_MPEG2) += cbs_mpeg2.o OBJS-$(CONFIG_CBS_VP9) += cbs_vp9.o diff --git a/libavcodec/cbs_h265.c b/libavcodec/cbs_h265.c new file mode 100644 index 0000000000..590977cf00 --- /dev/null +++ b/libavcodec/cbs_h265.c @@ -0,0 +1,99 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "libavutil/mathematics.h" +#include "libavutil/mastering_display_metadata.h" + +#include "cbs_h265.h" + + +static uint32_t rescale_clip(AVRational value, uint32_t scale, uint32_t max) +{ + int64_t scaled = av_rescale(scale, value.num, value.den); + return av_clip64(scaled, 0, max); +} + +static uint32_t rescale_fraction(AVRational value, uint32_t max) +{ + return rescale_clip(value, max, max); +} + +void ff_cbs_h265_fill_sei_mastering_display(H265RawSEIMasteringDisplayColourVolume *mdcv, + const AVMasteringDisplayMetadata *mdm) +{ + memset(mdcv, 0, sizeof(*mdcv)); + + if (mdm->has_primaries) { + // The values in the metadata structure are fractions between 0 and 1, + // while the SEI message contains fixed-point values with an increment + // of 0.00002. So, scale up by 50000 to convert between them. + + for (int a = 0; a < 3; a++) { + // The metadata structure stores this in RGB order, but the SEI + // wants it in GBR order. + int b = (a + 1) % 3; + mdcv->display_primaries_x[a] = + rescale_fraction(mdm->display_primaries[b][0], 50000); + mdcv->display_primaries_y[a] = + rescale_fraction(mdm->display_primaries[b][1], 50000); + } + + mdcv->white_point_x = rescale_fraction(mdm->white_point[0], 50000); + mdcv->white_point_y = rescale_fraction(mdm->white_point[1], 50000); + } + + if (mdm->has_luminance) { + // Metadata are rational values in candelas per square metre, SEI + // contains fixed point in units of 0.0001 candelas per square + // metre. So scale up by 10000 to convert between them, and clip to + // ensure that we don't overflow. + + mdcv->max_display_mastering_luminance = + rescale_clip(mdm->max_luminance, 10000, UINT32_MAX); + mdcv->min_display_mastering_luminance = + rescale_clip(mdm->min_luminance, 10000, UINT32_MAX); + + // The spec has a hard requirement that min is less than the max, + // and the SEI-writing code enforces that. + if (!(mdcv->min_display_mastering_luminance < + mdcv->max_display_mastering_luminance)) { + if (mdcv->max_display_mastering_luminance == UINT32_MAX) + mdcv->min_display_mastering_luminance = + mdcv->max_display_mastering_luminance - 1; + else + mdcv->max_display_mastering_luminance = + mdcv->min_display_mastering_luminance + 1; + } + } else { + mdcv->max_display_mastering_luminance = 1; + mdcv->min_display_mastering_luminance = 0; + } +} + +void ff_cbs_h265_fill_sei_content_light_level(H265RawSEIContentLightLevelInfo *cll, + const AVContentLightMetadata *clm) +{ + memset(cll, 0, sizeof(*cll)); + + // Both the metadata and the SEI are in units of candelas per square + // metre, so we only need to clip to ensure that they are in the valid + // range. + + cll->max_content_light_level = av_clip_uintp2(clm->MaxCLL, 16); + cll->max_pic_average_light_level = av_clip_uintp2(clm->MaxFALL, 16); +} diff --git a/libavcodec/cbs_h265.h b/libavcodec/cbs_h265.h index f5eb5af5b2..793675ad41 100644 --- a/libavcodec/cbs_h265.h +++ b/libavcodec/cbs_h265.h @@ -746,4 +746,22 @@ typedef struct CodedBitstreamH265Context { } CodedBitstreamH265Context; +struct AVMasteringDisplayMetadata; +struct AVContentLightMetadata; + +/** + * Fill an SEI Mastering Display Colour Volume structure with values derived + * from the AVMasteringDisplayMetadata side-data structure. + */ +void ff_cbs_h265_fill_sei_mastering_display(H265RawSEIMasteringDisplayColourVolume *mdcv, + const struct AVMasteringDisplayMetadata *mdm); + +/** + * Fill an SEI Content Light Level Info structure with values derived from + * the AVContentLightMetadata side-data structure. + */ +void ff_cbs_h265_fill_sei_content_light_level(H265RawSEIContentLightLevelInfo *cll, + const struct AVContentLightMetadata *clm); + + #endif /* AVCODEC_CBS_H265_H */ From patchwork Sun Feb 23 23:41:22 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Thompson X-Patchwork-Id: 17878 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id D4362449F1B for ; Mon, 24 Feb 2020 01:41:54 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id BE5A668B4D5; Mon, 24 Feb 2020 01:41:54 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wm1-f68.google.com (mail-wm1-f68.google.com [209.85.128.68]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id B948A68B4BC for ; Mon, 24 Feb 2020 01:41:51 +0200 (EET) Received: by mail-wm1-f68.google.com with SMTP id a9so7485663wmj.3 for ; Sun, 23 Feb 2020 15:41:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jkqxz-net.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=4KGhCdz/52Pa1viFYhkRU2vailMXPxJ/tpQS/t+piu8=; b=lupn7hZWPEBIRLtCANtZiXW/3jHb7B9laBjFMBmtSbNYw7CaORcxZxZql4Jdxijbhy FVkWNZJr6vTwQ8/eEKo8OrF1j3gYH6c7b2yRg29eUPPLm8L3s9T4PDUOCO5Kz+PYBysx CPhfkxR8ZcB+HfF2KiQrFOCvZeybB1saCxPCLUG9F9K9idC/F0ftFjQSJG+OOgXJNmrK yMJYCYLq6LohsxG4ntf4GPg69KvmD8HSz9mrkLXEUuBfuJCiEcef/HfnVtuXGt2SE79E C5cN/9X04OokVGbdk81l3+E4Ua2AEMNwXAxItrlTejd/7rQc3BZfGrzLe7+UUuHyEKrj S2NQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=4KGhCdz/52Pa1viFYhkRU2vailMXPxJ/tpQS/t+piu8=; b=FiAT0TYnnCRzmu9ZV6O3pcq/mnx5eMNqiFZdy1rZsvP8u+b8T5nFjKWASr4BHFD5Gh Cux1Nbs3EvN/qbJ/qtA/9XBF8QhcHw9qGIkqRZiIkWEorf+fgQ6WyGg/KLehkv1bgxWq mI0UCaUVkU1c+fApuCOcofejWGd0tllWn16EW9sNiCUBNK0htQsTSqAWOwmmFsxmRI0l AlHpPEL6gbA0Y1ita0uj5sfi8fOReYRciKS92fkKsfb1L+wJxU/QYJyih5/Nce41LnEa zop1SbMiSlXsd4vl2ESgoAQAdCJERcrZKCzXqdBE7ZPuRzValHpzFak7StZEpYJjvNHQ LaPQ== X-Gm-Message-State: APjAAAUJgP1uvmpYejv5hSjQCYpjW5yNVYY2wrcUI1gB7UlQmAQhZVMi Sv7x8ABt+HXPRcExOzyBZVZHTQyb8VY= X-Google-Smtp-Source: APXvYqyI7Kneg/3t87UE+LXOQB9xq5aloXopQyyHcz97qfl910q58oWEBg1e02MlgJG3u/T9DkoHjw== X-Received: by 2002:a1c:4d03:: with SMTP id o3mr17542212wmh.164.1582501310918; Sun, 23 Feb 2020 15:41:50 -0800 (PST) Received: from rywe.jkqxz.net (cpc91242-cmbg18-2-0-cust650.5-4.cable.virginm.net. [82.8.130.139]) by smtp.gmail.com with ESMTPSA id y131sm16358070wmc.13.2020.02.23.15.41.49 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 23 Feb 2020 15:41:50 -0800 (PST) From: Mark Thompson To: ffmpeg-devel@ffmpeg.org Date: Sun, 23 Feb 2020 23:41:22 +0000 Message-Id: <20200223234124.17689-19-sw@jkqxz.net> X-Mailer: git-send-email 2.25.0 In-Reply-To: <20200223234124.17689-1-sw@jkqxz.net> References: <20200223234124.17689-1-sw@jkqxz.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v4 19/21] vaapi_encode_h265: Use common handling for HDR metadata X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" --- libavcodec/vaapi_encode_h265.c | 45 +++++----------------------------- 1 file changed, 6 insertions(+), 39 deletions(-) diff --git a/libavcodec/vaapi_encode_h265.c b/libavcodec/vaapi_encode_h265.c index 538862a9d5..443139dfdb 100644 --- a/libavcodec/vaapi_encode_h265.c +++ b/libavcodec/vaapi_encode_h265.c @@ -750,39 +750,10 @@ static int vaapi_encode_h265_init_picture_params(AVCodecContext *avctx, AVMasteringDisplayMetadata *mdm = (AVMasteringDisplayMetadata *)sd->data; - // SEI is needed when both the primaries and luminance are set + // Only needed when both primaries and luminance are set. if (mdm->has_primaries && mdm->has_luminance) { - H265RawSEIMasteringDisplayColourVolume *mdcv = - &priv->sei_mastering_display; - const int mapping[3] = {1, 2, 0}; - const int chroma_den = 50000; - const int luma_den = 10000; - - for (i = 0; i < 3; i++) { - const int j = mapping[i]; - mdcv->display_primaries_x[i] = - FFMIN(lrint(chroma_den * - av_q2d(mdm->display_primaries[j][0])), - chroma_den); - mdcv->display_primaries_y[i] = - FFMIN(lrint(chroma_den * - av_q2d(mdm->display_primaries[j][1])), - chroma_den); - } - - mdcv->white_point_x = - FFMIN(lrint(chroma_den * av_q2d(mdm->white_point[0])), - chroma_den); - mdcv->white_point_y = - FFMIN(lrint(chroma_den * av_q2d(mdm->white_point[1])), - chroma_den); - - mdcv->max_display_mastering_luminance = - lrint(luma_den * av_q2d(mdm->max_luminance)); - mdcv->min_display_mastering_luminance = - FFMIN(lrint(luma_den * av_q2d(mdm->min_luminance)), - mdcv->max_display_mastering_luminance); - + ff_cbs_h265_fill_sei_mastering_display( + &priv->sei_mastering_display, mdm); priv->sei_needed |= SEI_MASTERING_DISPLAY; } } @@ -795,13 +766,9 @@ static int vaapi_encode_h265_init_picture_params(AVCodecContext *avctx, AV_FRAME_DATA_CONTENT_LIGHT_LEVEL); if (sd) { - AVContentLightMetadata *clm = - (AVContentLightMetadata *)sd->data; - H265RawSEIContentLightLevelInfo *clli = - &priv->sei_content_light_level; - - clli->max_content_light_level = FFMIN(clm->MaxCLL, 65535); - clli->max_pic_average_light_level = FFMIN(clm->MaxFALL, 65535); + ff_cbs_h265_fill_sei_content_light_level( + &priv->sei_content_light_level, + (const AVContentLightMetadata*)sd->data); priv->sei_needed |= SEI_CONTENT_LIGHT_LEVEL; } From patchwork Sun Feb 23 23:41:23 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Thompson X-Patchwork-Id: 17879 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id A4603449F1B for ; Mon, 24 Feb 2020 01:41:55 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 8EFCA68B4E3; Mon, 24 Feb 2020 01:41:55 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wr1-f66.google.com (mail-wr1-f66.google.com [209.85.221.66]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 2C9F8687F1F for ; Mon, 24 Feb 2020 01:41:53 +0200 (EET) Received: by mail-wr1-f66.google.com with SMTP id w12so8280632wrt.2 for ; Sun, 23 Feb 2020 15:41:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jkqxz-net.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=mkYnY1CJSA77AaEfwSOCv4aZQLMTjNue6al2uXWWjb0=; b=LXVI/0boIWKpWKsmoA9esov0+A88yvHCOyCl64MuGQAPDDT4U+QncqLzZW6jJv87px 15fSgQkWZYTgvmwgUP4Km3cs5V1/0FgL1LSmGCqiZgIQk4CNErylk4HcGMMfHAHLqSoz sr8pmB3rkEpVNqRXEMoB+iXELiwbeMHsdaJSSb1g7B8B8wJ+liXwfpR5zf32DTSDB+Yf l4O3OefUqnec7rUSf1NWR/m9XFJjomNw2H3Vi9NaoPTvqCw1HOcSP8iD70d8OniAbgvo MbIwHeCwc7/PeWx9ttChi7NtvORt+6moflfh6v3Sl5mnvPUA68LxZxyXQfTzSfdR6Bbg aCrw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=mkYnY1CJSA77AaEfwSOCv4aZQLMTjNue6al2uXWWjb0=; b=s2R0nJv371O4X6sQvMFmK5w+Ebkwnh4XO7kDt2hx2MSkSA5pUNFDO9A62AhdNzOmHo BQ0q1upkeRh8z0I4S9eGdF5E+roBHhxmIM7PmCPyOQT3Nw7UDhHfv/9JKnCsJlQsp8Ju t/fQJ2NFaPxgR1zNh7Bl+12Qyrb8TxLSXPd6tqpAwJoF2vyqz05kaq+vUDuydcT4zMXE 1XtlqOBc4E3TAFruYmY5YGGuNEBydVUf0hLqIcc0Gxqv050t0poDQbFM9kKxAMLzBZCO WwX5lXiR2OeHmdLInmbmXu5gsQDcUUQchb7lIVoxhd5ershph1ELYyF8e52QxgAulsf2 XQ8w== X-Gm-Message-State: APjAAAVuA5pnUEjEKDuYTUmwBWeXlyhOIcKImXKeeYbMTqB0pNvgIZ26 KNtjaAii2V2ozkQky0XSPPJLwoHmV1A= X-Google-Smtp-Source: APXvYqybZsoeZFjYeCPgRI9XZjf7G1HevN0gR5uTKAShmSR8t/Qo17PF+2ZrBVgLHeoLFt5vEChlmA== X-Received: by 2002:adf:9c8c:: with SMTP id d12mr61489951wre.404.1582501311951; Sun, 23 Feb 2020 15:41:51 -0800 (PST) Received: from rywe.jkqxz.net (cpc91242-cmbg18-2-0-cust650.5-4.cable.virginm.net. [82.8.130.139]) by smtp.gmail.com with ESMTPSA id y131sm16358070wmc.13.2020.02.23.15.41.50 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 23 Feb 2020 15:41:51 -0800 (PST) From: Mark Thompson To: ffmpeg-devel@ffmpeg.org Date: Sun, 23 Feb 2020 23:41:23 +0000 Message-Id: <20200223234124.17689-20-sw@jkqxz.net> X-Mailer: git-send-email 2.25.0 In-Reply-To: <20200223234124.17689-1-sw@jkqxz.net> References: <20200223234124.17689-1-sw@jkqxz.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v4 20/21] h264_metadata_bsf: Refactor the filter function into smaller parts X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" --- libavcodec/h264_metadata_bsf.c | 436 ++++++++++++++++++--------------- 1 file changed, 237 insertions(+), 199 deletions(-) diff --git a/libavcodec/h264_metadata_bsf.c b/libavcodec/h264_metadata_bsf.c index d96a50dbf7..b2304373bf 100644 --- a/libavcodec/h264_metadata_bsf.c +++ b/libavcodec/h264_metadata_bsf.c @@ -54,6 +54,7 @@ typedef struct H264MetadataContext { int done_first_au; int aud; + H264RawAUD aud_nal; AVRational sample_aspect_ratio; @@ -76,6 +77,7 @@ typedef struct H264MetadataContext { int crop_bottom; const char *sei_user_data; + H264RawSEIPayload sei_user_data_payload; int delete_filler; @@ -87,6 +89,59 @@ typedef struct H264MetadataContext { } H264MetadataContext; +static int h264_metadata_insert_aud(AVBSFContext *bsf, + CodedBitstreamFragment *au) +{ + H264MetadataContext *ctx = bsf->priv_data; + int primary_pic_type_mask = 0xff; + int err, i, j; + + static const int primary_pic_type_table[] = { + 0x084, // 2, 7 + 0x0a5, // 0, 2, 5, 7 + 0x0e7, // 0, 1, 2, 5, 6, 7 + 0x210, // 4, 9 + 0x318, // 3, 4, 8, 9 + 0x294, // 2, 4, 7, 9 + 0x3bd, // 0, 2, 3, 4, 5, 7, 8, 9 + 0x3ff, // 0, 1, 2, 3, 4, 5, 6, 7, 8, 9 + }; + + for (i = 0; i < au->nb_units; i++) { + if (au->units[i].type == H264_NAL_SLICE || + au->units[i].type == H264_NAL_IDR_SLICE) { + H264RawSlice *slice = au->units[i].content; + for (j = 0; j < FF_ARRAY_ELEMS(primary_pic_type_table); j++) { + if (!(primary_pic_type_table[j] & + (1 << slice->header.slice_type))) + primary_pic_type_mask &= ~(1 << j); + } + } + } + for (j = 0; j < FF_ARRAY_ELEMS(primary_pic_type_table); j++) + if (primary_pic_type_mask & (1 << j)) + break; + if (j >= FF_ARRAY_ELEMS(primary_pic_type_table)) { + av_log(bsf, AV_LOG_ERROR, "No usable primary_pic_type: " + "invalid slice types?\n"); + return AVERROR_INVALIDDATA; + } + + ctx->aud_nal = (H264RawAUD) { + .nal_unit_header.nal_unit_type = H264_NAL_AUD, + .primary_pic_type = j, + }; + + err = ff_cbs_insert_unit_content(ctx->cbc, au, + 0, H264_NAL_AUD, &ctx->aud_nal, NULL); + if (err < 0) { + av_log(bsf, AV_LOG_ERROR, "Failed to insert AUD.\n"); + return err; + } + + return 0; +} + static int h264_metadata_update_sps(AVBSFContext *bsf, H264RawSPS *sps) { @@ -275,217 +330,60 @@ static int h264_metadata_update_sps(AVBSFContext *bsf, return 0; } -static int h264_metadata_filter(AVBSFContext *bsf, AVPacket *pkt) +static int h264_metadata_handle_display_orientation(AVBSFContext *bsf, + AVPacket *pkt, + CodedBitstreamFragment *au, + int seek_point) { H264MetadataContext *ctx = bsf->priv_data; - CodedBitstreamFragment *au = &ctx->access_unit; - int err, i, j, has_sps; - H264RawAUD aud; + int err, i, j; - err = ff_bsf_get_packet_ref(bsf, pkt); - if (err < 0) - return err; + for (i = au->nb_units - 1; i >= 0; i--) { + H264RawSEI *sei; + if (au->units[i].type != H264_NAL_SEI) + continue; + sei = au->units[i].content; - err = ff_cbs_read_packet(ctx->cbc, au, pkt); - if (err < 0) { - av_log(bsf, AV_LOG_ERROR, "Failed to read packet.\n"); - goto fail; - } + for (j = sei->payload_count - 1; j >= 0; j--) { + H264RawSEIDisplayOrientation *disp; + int32_t *matrix; - if (au->nb_units == 0) { - av_log(bsf, AV_LOG_ERROR, "No NAL units in packet.\n"); - err = AVERROR_INVALIDDATA; - goto fail; - } - - // If an AUD is present, it must be the first NAL unit. - if (au->units[0].type == H264_NAL_AUD) { - if (ctx->aud == REMOVE) - ff_cbs_delete_unit(ctx->cbc, au, 0); - } else { - if (ctx->aud == INSERT) { - static const int primary_pic_type_table[] = { - 0x084, // 2, 7 - 0x0a5, // 0, 2, 5, 7 - 0x0e7, // 0, 1, 2, 5, 6, 7 - 0x210, // 4, 9 - 0x318, // 3, 4, 8, 9 - 0x294, // 2, 4, 7, 9 - 0x3bd, // 0, 2, 3, 4, 5, 7, 8, 9 - 0x3ff, // 0, 1, 2, 3, 4, 5, 6, 7, 8, 9 - }; - int primary_pic_type_mask = 0xff; - - for (i = 0; i < au->nb_units; i++) { - if (au->units[i].type == H264_NAL_SLICE || - au->units[i].type == H264_NAL_IDR_SLICE) { - H264RawSlice *slice = au->units[i].content; - for (j = 0; j < FF_ARRAY_ELEMS(primary_pic_type_table); j++) { - if (!(primary_pic_type_table[j] & - (1 << slice->header.slice_type))) - primary_pic_type_mask &= ~(1 << j); - } - } - } - for (j = 0; j < FF_ARRAY_ELEMS(primary_pic_type_table); j++) - if (primary_pic_type_mask & (1 << j)) - break; - if (j >= FF_ARRAY_ELEMS(primary_pic_type_table)) { - av_log(bsf, AV_LOG_ERROR, "No usable primary_pic_type: " - "invalid slice types?\n"); - err = AVERROR_INVALIDDATA; - goto fail; - } - - aud = (H264RawAUD) { - .nal_unit_header.nal_unit_type = H264_NAL_AUD, - .primary_pic_type = j, - }; - - err = ff_cbs_insert_unit_content(ctx->cbc, au, - 0, H264_NAL_AUD, &aud, NULL); - if (err < 0) { - av_log(bsf, AV_LOG_ERROR, "Failed to insert AUD.\n"); - goto fail; - } - } - } - - has_sps = 0; - for (i = 0; i < au->nb_units; i++) { - if (au->units[i].type == H264_NAL_SPS) { - err = h264_metadata_update_sps(bsf, au->units[i].content); - if (err < 0) - goto fail; - has_sps = 1; - } - } - - // Only insert the SEI in access units containing SPSs, and also - // unconditionally in the first access unit we ever see. - if (ctx->sei_user_data && (has_sps || !ctx->done_first_au)) { - H264RawSEIPayload payload = { - .payload_type = H264_SEI_TYPE_USER_DATA_UNREGISTERED, - }; - H264RawSEIUserDataUnregistered *udu = - &payload.payload.user_data_unregistered; - - for (i = j = 0; j < 32 && ctx->sei_user_data[i]; i++) { - int c, v; - c = ctx->sei_user_data[i]; - if (c == '-') { + if (sei->payload[j].payload_type != + H264_SEI_TYPE_DISPLAY_ORIENTATION) continue; - } else if (av_isxdigit(c)) { - c = av_tolower(c); - v = (c <= '9' ? c - '0' : c - 'a' + 10); - } else { - goto invalid_user_data; - } - if (j & 1) - udu->uuid_iso_iec_11578[j / 2] |= v; - else - udu->uuid_iso_iec_11578[j / 2] = v << 4; - ++j; - } - if (j == 32 && ctx->sei_user_data[i] == '+') { - size_t len = strlen(ctx->sei_user_data + i + 1); - - udu->data_ref = av_buffer_alloc(len + 1); - if (!udu->data_ref) { - err = AVERROR(ENOMEM); - goto fail; - } - - udu->data = udu->data_ref->data; - udu->data_length = len + 1; - memcpy(udu->data, ctx->sei_user_data + i + 1, len + 1); + disp = &sei->payload[j].payload.display_orientation; - err = ff_cbs_h264_add_sei_message(ctx->cbc, au, &payload); - if (err < 0) { - av_log(bsf, AV_LOG_ERROR, "Failed to add user data SEI " - "message to access unit.\n"); - goto fail; - } - - } else { - invalid_user_data: - av_log(bsf, AV_LOG_ERROR, "Invalid user data: " - "must be \"UUID+string\".\n"); - err = AVERROR(EINVAL); - goto fail; - } - } - - if (ctx->delete_filler) { - for (i = au->nb_units - 1; i >= 0; i--) { - if (au->units[i].type == H264_NAL_FILLER_DATA) { - ff_cbs_delete_unit(ctx->cbc, au, i); + if (ctx->display_orientation == REMOVE || + ctx->display_orientation == INSERT) { + ff_cbs_h264_delete_sei_message(ctx->cbc, au, + &au->units[i], j); continue; } - if (au->units[i].type == H264_NAL_SEI) { - // Filler SEI messages. - H264RawSEI *sei = au->units[i].content; - - for (j = sei->payload_count - 1; j >= 0; j--) { - if (sei->payload[j].payload_type == - H264_SEI_TYPE_FILLER_PAYLOAD) - ff_cbs_h264_delete_sei_message(ctx->cbc, au, - &au->units[i], j); - } + matrix = av_malloc(9 * sizeof(int32_t)); + if (!matrix) + return AVERROR(ENOMEM); + + av_display_rotation_set(matrix, + disp->anticlockwise_rotation * + 180.0 / 65536.0); + av_display_matrix_flip(matrix, disp->hor_flip, disp->ver_flip); + + // If there are multiple display orientation messages in an + // access unit, then the last one added to the packet (i.e. + // the first one in the access unit) will prevail. + err = av_packet_add_side_data(pkt, AV_PKT_DATA_DISPLAYMATRIX, + (uint8_t*)matrix, + 9 * sizeof(int32_t)); + if (err < 0) { + av_log(bsf, AV_LOG_ERROR, "Failed to attach extracted " + "displaymatrix side data to packet.\n"); + av_freep(matrix); + return AVERROR(ENOMEM); } } } - if (ctx->display_orientation != PASS) { - for (i = au->nb_units - 1; i >= 0; i--) { - H264RawSEI *sei; - if (au->units[i].type != H264_NAL_SEI) - continue; - sei = au->units[i].content; - - for (j = sei->payload_count - 1; j >= 0; j--) { - H264RawSEIDisplayOrientation *disp; - int32_t *matrix; - - if (sei->payload[j].payload_type != - H264_SEI_TYPE_DISPLAY_ORIENTATION) - continue; - disp = &sei->payload[j].payload.display_orientation; - - if (ctx->display_orientation == REMOVE || - ctx->display_orientation == INSERT) { - ff_cbs_h264_delete_sei_message(ctx->cbc, au, - &au->units[i], j); - continue; - } - - matrix = av_malloc(9 * sizeof(int32_t)); - if (!matrix) { - err = AVERROR(ENOMEM); - goto fail; - } - - av_display_rotation_set(matrix, - disp->anticlockwise_rotation * - 180.0 / 65536.0); - av_display_matrix_flip(matrix, disp->hor_flip, disp->ver_flip); - - // If there are multiple display orientation messages in an - // access unit, then the last one added to the packet (i.e. - // the first one in the access unit) will prevail. - err = av_packet_add_side_data(pkt, AV_PKT_DATA_DISPLAYMATRIX, - (uint8_t*)matrix, - 9 * sizeof(int32_t)); - if (err < 0) { - av_log(bsf, AV_LOG_ERROR, "Failed to attach extracted " - "displaymatrix side data to packet.\n"); - av_freep(matrix); - goto fail; - } - } - } - } if (ctx->display_orientation == INSERT) { H264RawSEIPayload payload = { .payload_type = H264_SEI_TYPE_DISPLAY_ORIENTATION, @@ -529,7 +427,7 @@ static int h264_metadata_filter(AVBSFContext *bsf, AVPacket *pkt) } } - if (has_sps || !ctx->done_first_au) { + if (seek_point) { if (!isnan(ctx->rotate)) { disp->anticlockwise_rotation = (uint16_t)rint((ctx->rotate >= 0.0 ? ctx->rotate @@ -551,11 +449,101 @@ static int h264_metadata_filter(AVBSFContext *bsf, AVPacket *pkt) if (err < 0) { av_log(bsf, AV_LOG_ERROR, "Failed to add display orientation " "SEI message to access unit.\n"); + return err; + } + } + } + + return 0; +} + +static int h264_metadata_filter(AVBSFContext *bsf, AVPacket *pkt) +{ + H264MetadataContext *ctx = bsf->priv_data; + CodedBitstreamFragment *au = &ctx->access_unit; + int err, i, j, has_sps, seek_point; + + err = ff_bsf_get_packet_ref(bsf, pkt); + if (err < 0) + return err; + + err = ff_cbs_read_packet(ctx->cbc, au, pkt); + if (err < 0) { + av_log(bsf, AV_LOG_ERROR, "Failed to read packet.\n"); + goto fail; + } + + if (au->nb_units == 0) { + av_log(bsf, AV_LOG_ERROR, "No NAL units in packet.\n"); + err = AVERROR_INVALIDDATA; + goto fail; + } + + // If an AUD is present, it must be the first NAL unit. + if (au->units[0].type == H264_NAL_AUD) { + if (ctx->aud == REMOVE) + ff_cbs_delete_unit(ctx->cbc, au, 0); + } else { + if (ctx->aud == INSERT) + h264_metadata_insert_aud(bsf, au); + } + + has_sps = 0; + for (i = 0; i < au->nb_units; i++) { + if (au->units[i].type == H264_NAL_SPS) { + err = h264_metadata_update_sps(bsf, au->units[i].content); + if (err < 0) goto fail; + has_sps = 1; + } + } + + // The current packet should be treated as a seek point for metadata + // insertion if any of: + // - It is the first packet in the stream. + // - It contains an SPS, indicating that a sequence might start here. + // - It is marked as containing a key frame. + seek_point = !ctx->done_first_au || has_sps || + (pkt->flags & AV_PKT_FLAG_KEY); + + if (ctx->sei_user_data && seek_point) { + err = ff_cbs_h264_add_sei_message(ctx->cbc, au, + &ctx->sei_user_data_payload); + if (err < 0) { + av_log(bsf, AV_LOG_ERROR, "Failed to add user data SEI " + "message to access unit.\n"); + goto fail; + } + } + + if (ctx->delete_filler) { + for (i = au->nb_units - 1; i >= 0; i--) { + if (au->units[i].type == H264_NAL_FILLER_DATA) { + ff_cbs_delete_unit(ctx->cbc, au, i); + continue; + } + + if (au->units[i].type == H264_NAL_SEI) { + // Filler SEI messages. + H264RawSEI *sei = au->units[i].content; + + for (j = sei->payload_count - 1; j >= 0; j--) { + if (sei->payload[j].payload_type == + H264_SEI_TYPE_FILLER_PAYLOAD) + ff_cbs_h264_delete_sei_message(ctx->cbc, au, + &au->units[i], j); + } } } } + if (ctx->display_orientation != PASS) { + err = h264_metadata_handle_display_orientation(bsf, pkt, au, + seek_point); + if (err < 0) + goto fail; + } + err = ff_cbs_write_packet(ctx->cbc, pkt, au); if (err < 0) { av_log(bsf, AV_LOG_ERROR, "Failed to write packet.\n"); @@ -578,7 +566,57 @@ static int h264_metadata_init(AVBSFContext *bsf) { H264MetadataContext *ctx = bsf->priv_data; CodedBitstreamFragment *au = &ctx->access_unit; - int err, i; + int err, i, j; + + if (ctx->sei_user_data) { + H264RawSEIUserDataUnregistered *udu; + int c, v; + size_t len; + + ctx->sei_user_data_payload = (H264RawSEIPayload) { + .payload_type = H264_SEI_TYPE_USER_DATA_UNREGISTERED, + }; + udu = &ctx->sei_user_data_payload.payload.user_data_unregistered; + + // Parse UUID. It must be a hex string of length 32, possibly + // containing '-'s which we ignore. + for (i = j = 0; j < 32 && ctx->sei_user_data[i]; i++) { + c = ctx->sei_user_data[i]; + if (c == '-') { + continue; + } else if (av_isxdigit(c)) { + c = av_tolower(c); + v = (c <= '9' ? c - '0' : c - 'a' + 10); + } else { + break; + } + if (j & 1) + udu->uuid_iso_iec_11578[j / 2] |= v; + else + udu->uuid_iso_iec_11578[j / 2] = v << 4; + ++j; + } + if (j < 32 || ctx->sei_user_data[i] != '+') { + av_log(bsf, AV_LOG_ERROR, "Invalid user data: " + "must be of the form \"UUID+string\".\n"); + return AVERROR(EINVAL); + } else { + // Skip over the '+'. + ++i; + + // Length of the actual data to insert (could be zero). + len = strlen(ctx->sei_user_data + i); + + udu->data_ref = av_buffer_alloc(len + 1); + if (!udu->data_ref) + return AVERROR(ENOMEM); + + udu->data_length = len + 1; + udu->data = udu->data_ref->data; + memcpy(udu->data, ctx->sei_user_data + 1, len); + udu->data[len] = 0; + } + } err = ff_cbs_init(&ctx->cbc, AV_CODEC_ID_H264, bsf); if (err < 0) From patchwork Sun Feb 23 23:41:24 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Thompson X-Patchwork-Id: 17880 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id B5912449F1B for ; Mon, 24 Feb 2020 01:41:56 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id A1D3968B4EA; Mon, 24 Feb 2020 01:41:56 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-wr1-f66.google.com (mail-wr1-f66.google.com [209.85.221.66]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 9FCCE687F1F for ; Mon, 24 Feb 2020 01:41:53 +0200 (EET) Received: by mail-wr1-f66.google.com with SMTP id u6so8296615wrt.0 for ; Sun, 23 Feb 2020 15:41:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jkqxz-net.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=BDxOag9W3yj8SDavhDhYMJeGpd38vVFhnceMlGSxJFA=; b=X+EItI0I5F5EvvGrUzB02dVsZvPuHGDz99mPy24e3Z+8KV0ooKHG0gj0+iNKUFxfRR VkZwNgKrreO0kp7aAXGX4o0qe/ag5hblDo2VcQfZ6zQIDl72ti3Y4EJ8A5helIbB2Knh KzLZFHMcGgul/tYoWhSGdOeckp0GMi3OqqagivBZ38/uaHwxNoEj7SebNshkRGc3Krsm 4FTuh/IkbOE1MDcyQN/KP0zW4zIQAX2mfSgSkn3tFrYtfhQFuDdTb88ePQ8ex2ryIQUY +gXpP6P337AOA72Aoq1vBxHlTHA6UZWXiuVSDDpB7TxrpJorPOMq1fyv3BoV9XUcci/Z Vh8w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=BDxOag9W3yj8SDavhDhYMJeGpd38vVFhnceMlGSxJFA=; b=WDfUujfjB2ucGHJbpapYZ1cXZ00G5/vMpI2BZS5tqMGIVlO34hYwnXEaA85R94OIZ1 aE/LW1LFnRdXXWGe29rQ0UkAEj+ZN7EG/IpG69QuRgG8X2VM/7kJw0r7QD2SCzhaAmqE QBjm6/V3gxJcsMGTIt7/CoLuSzZFTAriqQApyJGl9LxRfkzjY/G1t+UjOX7Or+d71t9v fTV/mUtgFdSOUUMzb0DmkmVwCPyx51E1clYD5sA1fexldsd3su++VyMPfOT0piDPoICk fegW6AgP0Wkig4FosB+8Kj5HEPnCSFoaDyzTvkQd3aZQ5ouyPCJz9SHpvYPLTElAW49e Ca5g== X-Gm-Message-State: APjAAAWKdaGZ6a0FewO0HxEw8SJ9YrJRe/cM8tgbAVatvwgiMWqaFRQO otwv4LTgpdrrOs+mLMa98ACqAATfVh8= X-Google-Smtp-Source: APXvYqzz/sgx5CjC83I1JX4TRH5BEonF+AurxjTkxiU93JQu9etP2dO3+59YxrfR/gXDFA8ksfo2Jw== X-Received: by 2002:a5d:6445:: with SMTP id d5mr61980158wrw.244.1582501312902; Sun, 23 Feb 2020 15:41:52 -0800 (PST) Received: from rywe.jkqxz.net (cpc91242-cmbg18-2-0-cust650.5-4.cable.virginm.net. [82.8.130.139]) by smtp.gmail.com with ESMTPSA id y131sm16358070wmc.13.2020.02.23.15.41.52 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 23 Feb 2020 15:41:52 -0800 (PST) From: Mark Thompson To: ffmpeg-devel@ffmpeg.org Date: Sun, 23 Feb 2020 23:41:24 +0000 Message-Id: <20200223234124.17689-21-sw@jkqxz.net> X-Mailer: git-send-email 2.25.0 In-Reply-To: <20200223234124.17689-1-sw@jkqxz.net> References: <20200223234124.17689-1-sw@jkqxz.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v4 21/21] h264_metadata_bsf: Improve interpretation of input display matrices X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" The previous code here only worked in more limited cases. --- libavcodec/h264_metadata_bsf.c | 42 +++++++++++++++++++++++----------- 1 file changed, 29 insertions(+), 13 deletions(-) diff --git a/libavcodec/h264_metadata_bsf.c b/libavcodec/h264_metadata_bsf.c index b2304373bf..8fc02c5f41 100644 --- a/libavcodec/h264_metadata_bsf.c +++ b/libavcodec/h264_metadata_bsf.c @@ -397,23 +397,39 @@ static int h264_metadata_handle_display_orientation(AVBSFContext *bsf, data = av_packet_get_side_data(pkt, AV_PKT_DATA_DISPLAYMATRIX, &size); if (data && size >= 9 * sizeof(int32_t)) { int32_t matrix[9]; + double dmatrix[9]; int hflip, vflip; - double angle; + double scale_x, scale_y, angle; memcpy(matrix, data, sizeof(matrix)); - hflip = vflip = 0; - if (matrix[0] < 0 && matrix[4] > 0) - hflip = 1; - else if (matrix[0] > 0 && matrix[4] < 0) - vflip = 1; - av_display_matrix_flip(matrix, hflip, vflip); + for (i = 0; i < 9; i++) + dmatrix[i] = matrix[i] / 65536.0; + + // Extract scale factors. + scale_x = hypot(dmatrix[0], dmatrix[3]); + scale_y = hypot(dmatrix[1], dmatrix[4]); + + // Select flips to make the main diagonal positive. + hflip = dmatrix[0] < 0.0; + vflip = dmatrix[4] < 0.0; + if (hflip) + scale_x = -scale_x; + if (vflip) + scale_y = -scale_y; + + // Rescale. + for (i = 0; i < 9; i += 3) { + dmatrix[i] /= scale_x; + dmatrix[i + 1] /= scale_y; + } - angle = av_display_rotation_get(matrix); + // Extract rotation. + angle = atan2(dmatrix[3], dmatrix[0]); - if (!(angle >= -180.0 && angle <= 180.0 /* also excludes NaN */) || - matrix[2] != 0 || matrix[5] != 0 || - matrix[6] != 0 || matrix[7] != 0) { + if (!(angle >= -M_PI && angle <= M_PI) || + matrix[2] != 0.0 || matrix[5] != 0.0 || + matrix[6] != 0.0 || matrix[7] != 0.0) { av_log(bsf, AV_LOG_WARNING, "Input display matrix is not " "representable in H.264 parameters.\n"); } else { @@ -421,8 +437,8 @@ static int h264_metadata_handle_display_orientation(AVBSFContext *bsf, disp->ver_flip = vflip; disp->anticlockwise_rotation = (uint16_t)rint((angle >= 0.0 ? angle - : angle + 360.0) * - 65536.0 / 360.0); + : angle + 2 * M_PI) * + 32768.0 / M_PI); write = 1; } }