From patchwork Sun Jul 3 10:26:34 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Khirnov X-Patchwork-Id: 36627 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:8b27:b0:88:1bbf:7fd2 with SMTP id l39csp2138897pzh; Sun, 3 Jul 2022 03:28:23 -0700 (PDT) X-Google-Smtp-Source: AGRyM1uu2TP8wuEGVVs+KgnCKNRDK935OZessrxN0bHEDOh1w2XDo3uuTmOUc3T0db4B7W8+gTSO X-Received: by 2002:a05:6402:4302:b0:437:7990:992e with SMTP id m2-20020a056402430200b004377990992emr31011052edc.194.1656844102082; Sun, 03 Jul 2022 03:28:22 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1656844102; cv=none; d=google.com; s=arc-20160816; b=KzbMUdJOw3ojSSB8HqogAWjT9487FSmpU7x8X5cFbwAqOTftGbLJ6tlHGRUPll3GZX vmI8ACbJ1lzI1GCU9ckDHaiba4fbwarReADiRSx51hdbrnft6+6ycV2zXwNGv7Jex2yq CSNhnn7BZ/wiiVc03885wwthtlYn2+P5gaPaibhEE9oyt2OmTZzHzQf5hEyybP0KkOwf 8skli1hP88OJM9V+COk+2uqdacgBA39CuuacFcvUXmd6WRErrib2sznP2iIu/qGGsqSI S0GDKTVsZgnYD4SnJp8QOfLSPG8zkKwNgadm/Oqma3jX33pBGziE7alHyAxZFPscVClr cKvw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:message-id:date:to:from :delivered-to; bh=f/5NF4U4qf5OWy/tFK7gqVPNGTLum1Nd/hSJhuv9jcw=; b=QAWDQMNaRJEOG6PHIWAWydAMgT3e9phfEbBcy4L/jC6i3D+9O2hhCxQo7eEU5nytQd DqzHZWf9DCnRoMosnG1x48iFIT9p+9hzwHHUHZYeDbNKdDOXVeyNsirLRnf0NVeXCz0J 4LDqQLWfkr+brikyUN+uvNhUGmw4o0x+klPLA/CUkb11HjdLFuVTJzf7ic85UcjgXDOP UdJWlrw8mSwcZL7On/o4bsat0I0ZaoxtgPNliEJrvCOCWGVdhZTawrXkzKumyOaFTlsQ HYm4ARgDoVA2uYyAlFEtML4yTLR6etajU05p1O7DaoPCL7hVLrueFOxo9No55X9aIZvG XduQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id q20-20020a170906b29400b0072a60823affsi11485711ejz.49.2022.07.03.03.28.21; Sun, 03 Jul 2022 03:28:22 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 9EDD668B9C1; Sun, 3 Jul 2022 13:28:19 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail0.khirnov.net (red.khirnov.net [176.97.15.12]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 1766268B987 for ; Sun, 3 Jul 2022 13:28:13 +0300 (EEST) Received: from localhost (localhost [IPv6:::1]) by mail0.khirnov.net (Postfix) with ESMTP id 0B71824017E for ; Sun, 3 Jul 2022 12:28:12 +0200 (CEST) Received: from mail0.khirnov.net ([IPv6:::1]) by localhost (mail0.khirnov.net [IPv6:::1]) (amavisd-new, port 10024) with ESMTP id PspM6gE9q5sl for ; Sun, 3 Jul 2022 12:28:10 +0200 (CEST) Received: from libav.khirnov.net (libav.khirnov.net [IPv6:2a00:c500:561:201::7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "libav.khirnov.net", Issuer "smtp.khirnov.net SMTP CA" (verified OK)) by mail0.khirnov.net (Postfix) with ESMTPS id 84CC92400F5 for ; Sun, 3 Jul 2022 12:28:10 +0200 (CEST) Received: by libav.khirnov.net (Postfix, from userid 1000) id C50DA3A061D; Sun, 3 Jul 2022 12:28:10 +0200 (CEST) From: Anton Khirnov To: ffmpeg-devel@ffmpeg.org Date: Sun, 3 Jul 2022 12:26:34 +0200 Message-Id: <20220703102636.32523-1-anton@khirnov.net> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 1/3] get_bits: move check_marker() to mpegvideodec.h X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: EbF+IAJUerI9 It is only used by mpegvideo-based decoders - specifically mpeg12, intelh263, ituh263, mpeg4video. --- libavcodec/get_bits.h | 11 ----------- libavcodec/intelh263dec.c | 1 + libavcodec/ituh263dec.c | 1 + libavcodec/mpegvideodec.h | 10 ++++++++++ 4 files changed, 12 insertions(+), 11 deletions(-) diff --git a/libavcodec/get_bits.h b/libavcodec/get_bits.h index 16f8af5107..992765dc92 100644 --- a/libavcodec/get_bits.h +++ b/libavcodec/get_bits.h @@ -31,7 +31,6 @@ #include "libavutil/common.h" #include "libavutil/intreadwrite.h" -#include "libavutil/log.h" #include "libavutil/avassert.h" #include "defs.h" @@ -610,16 +609,6 @@ static inline unsigned int show_bits_long(GetBitContext *s, int n) } } -static inline int check_marker(void *logctx, GetBitContext *s, const char *msg) -{ - int bit = get_bits1(s); - if (!bit) - av_log(logctx, AV_LOG_INFO, "Marker bit missing at %d of %d %s\n", - get_bits_count(s) - 1, s->size_in_bits, msg); - - return bit; -} - static inline int init_get_bits_xe(GetBitContext *s, const uint8_t *buffer, int bit_size, int is_le) { diff --git a/libavcodec/intelh263dec.c b/libavcodec/intelh263dec.c index e7e821d3b3..ded0a7f618 100644 --- a/libavcodec/intelh263dec.c +++ b/libavcodec/intelh263dec.c @@ -21,6 +21,7 @@ #include "codec_internal.h" #include "mpegutils.h" #include "mpegvideo.h" +#include "mpegvideodec.h" #include "h263data.h" #include "h263dec.h" #include "mpegvideodata.h" diff --git a/libavcodec/ituh263dec.c b/libavcodec/ituh263dec.c index f01c942f04..af054360d8 100644 --- a/libavcodec/ituh263dec.c +++ b/libavcodec/ituh263dec.c @@ -48,6 +48,7 @@ #include "rv10dec.h" #include "mpeg4video.h" #include "mpegvideodata.h" +#include "mpegvideodec.h" #include "mpeg4videodec.h" // The defines below define the number of bits that are read at once for diff --git a/libavcodec/mpegvideodec.h b/libavcodec/mpegvideodec.h index 1af8ebac36..10394a616c 100644 --- a/libavcodec/mpegvideodec.h +++ b/libavcodec/mpegvideodec.h @@ -67,4 +67,14 @@ static inline int mpeg_get_qscale(MpegEncContext *s) return qscale << 1; } +static inline int check_marker(void *logctx, GetBitContext *s, const char *msg) +{ + int bit = get_bits1(s); + if (!bit) + av_log(logctx, AV_LOG_INFO, "Marker bit missing at %d of %d %s\n", + get_bits_count(s) - 1, s->size_in_bits, msg); + + return bit; +} + #endif /* AVCODEC_MPEGVIDEODEC_H */ From patchwork Sun Jul 3 10:26:35 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Anton Khirnov X-Patchwork-Id: 36628 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:8b27:b0:88:1bbf:7fd2 with SMTP id l39csp2138919pzh; Sun, 3 Jul 2022 03:28:29 -0700 (PDT) X-Google-Smtp-Source: AGRyM1ugQybm3HPjFLdStsha9GXaDLjf4LRvjMqROpsGeZePv2seTj35pyxyozd88X58/jcIZRZ2 X-Received: by 2002:a17:907:94d1:b0:726:efcd:ca4 with SMTP id dn17-20020a17090794d100b00726efcd0ca4mr23434929ejc.148.1656844109764; Sun, 03 Jul 2022 03:28:29 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1656844109; cv=none; d=google.com; s=arc-20160816; b=jBkyhQy1TB9nCvQaDdiAQjXCBJWNDVGT1EbgYOELALNdCNyEUJQ/lDTZbO7n0JBZ0N PU3N38aVIQgmqrH71eIGiNfPDh0R8tKbg4XLXRG+Q+RFToi2kgjwXswE0gKrN8PlHrQX 1ImlHIryaI1RDfLs06JdId2O4TURKfhuSQzTdH4Cp7Ku2a0LBhEXj0JQWWg8INlstvrO EiMnjuwMqbgV/24Qy2dHfWAqB14g54J5iMD06d4ydZ3Ww/os+cqR8gBTSfkgVzl1PhXS C9elvi2riGrzdLpS+EIMV8CMTBSuK6E7i/MJwbYuna7rB4VHDC8S0GsZYZq2AI1vPG94 DqFw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:delivered-to; bh=mZc8rFAoeeqYyLM38tRQ3Z2DkUbgVEjl593LcNTf+fQ=; b=Zjr+Fq8rhq6S5fNqC9Lbd23mfde08ZUQebREULD27EIxajXSb1BxqzgwBIyfhxjOy+ HriWdOMiANlzw4xOfejc8YfExIvQtONc887m/6SHuEbGCwu82FUj6fA9cHzp30caXbq8 iur/IEJw6v3kgrxPnQOtQ9n3uKmcR0OC4aRq3z9agQ78R7qrlUDZg/3rVjjn2X2Xw8Jv uwLr1KZC06scX470cY3Qg3T8uTAi6WBJtTY6RuQiSOipQYmqMIUpsFIIuvIsnFrgcjj4 4rwN1Db8tyeZxSSLXhvXZxupj2XYfIw5lWgbU4AfnAry+YsptrLO+89pSBpIpFVkJHic Y9ig== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id cq4-20020a056402220400b0043a41583a03si593509edb.71.2022.07.03.03.28.29; Sun, 03 Jul 2022 03:28:29 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 8EEA368B9C5; Sun, 3 Jul 2022 13:28:20 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail0.khirnov.net (red.khirnov.net [176.97.15.12]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 3949768B98E for ; Sun, 3 Jul 2022 13:28:13 +0300 (EEST) Received: from localhost (localhost [IPv6:::1]) by mail0.khirnov.net (Postfix) with ESMTP id 85FC42400F5 for ; Sun, 3 Jul 2022 12:28:12 +0200 (CEST) Received: from mail0.khirnov.net ([IPv6:::1]) by localhost (mail0.khirnov.net [IPv6:::1]) (amavisd-new, port 10024) with ESMTP id vITsvEreZgOL for ; Sun, 3 Jul 2022 12:28:10 +0200 (CEST) Received: from libav.khirnov.net (libav.khirnov.net [IPv6:2a00:c500:561:201::7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "libav.khirnov.net", Issuer "smtp.khirnov.net SMTP CA" (verified OK)) by mail0.khirnov.net (Postfix) with ESMTPS id 9A143240179 for ; Sun, 3 Jul 2022 12:28:10 +0200 (CEST) Received: by libav.khirnov.net (Postfix, from userid 1000) id C95583A0909; Sun, 3 Jul 2022 12:28:10 +0200 (CEST) From: Anton Khirnov To: ffmpeg-devel@ffmpeg.org Date: Sun, 3 Jul 2022 12:26:35 +0200 Message-Id: <20220703102636.32523-2-anton@khirnov.net> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220703102636.32523-1-anton@khirnov.net> References: <20220703102636.32523-1-anton@khirnov.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 2/3] lavc: add standalone cached bitstream reader X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 94K6HkisM2VI From: Alexandra Hájková The cached bitstream reader was originally written by Alexandra Hájková for Libav, with significant input from Kostya Shishkov and Luca Barbato. It was then committed to FFmpeg in ca079b09549, by merging it with the implementation of the current bitstream reader. This merge makes the code of get_bits.h significantly harder to read, since it now contains two different bitstream readers interleaved with #ifdefs. Additionally, the code was committed without proper authorship attribution. This commit re-adds the cached bitstream reader as a standalone header, as it was originally developed. It will be made useful in following commits. Integration by Anton Khirnov. Signed-off-by: Anton Khirnov --- Addressed all review comments by Andreas. Resending only the bistream reader for now, since any significant changes to it mean that the templatization patch needs to be basically redone from scratch. Will send the rest after this is approved. --- libavcodec/bitstream.h | 540 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 540 insertions(+) create mode 100644 libavcodec/bitstream.h diff --git a/libavcodec/bitstream.h b/libavcodec/bitstream.h new file mode 100644 index 0000000000..53bcdf4643 --- /dev/null +++ b/libavcodec/bitstream.h @@ -0,0 +1,540 @@ +/* + * Copyright (c) 2016 Alexandra Hájková + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * bitstream reader API header. + */ + +#ifndef AVCODEC_BITSTREAM_H +#define AVCODEC_BITSTREAM_H + +#include + +#include "config.h" + +#include "libavutil/avassert.h" +#include "libavutil/common.h" +#include "libavutil/intreadwrite.h" +#include "libavutil/log.h" + +#include "mathops.h" +#include "vlc.h" + +#ifndef UNCHECKED_BITSTREAM_READER +#define UNCHECKED_BITSTREAM_READER !CONFIG_SAFE_BITSTREAM_READER +#endif + +typedef struct BitstreamContext { + uint64_t bits; // stores bits read from the buffer + const uint8_t *buffer, *buffer_end; + const uint8_t *ptr; // pointer to the position inside a buffer + unsigned bits_left; // number of bits left in bits field + unsigned size_in_bits; +} BitstreamContext; + +/** + * @return + * - 0 on successful refill + * - a negative number when bitstream end is hit + * + * Always succeeds when UNCHECKED_BITSTREAM_READER is enabled. + */ +static inline int bits_priv_refill_64(BitstreamContext *bc) +{ +#if !UNCHECKED_BITSTREAM_READER + if (bc->ptr >= bc->buffer_end) + return -1; +#endif + +#ifdef BITSTREAM_READER_LE + bc->bits = AV_RL64(bc->ptr); +#else + bc->bits = AV_RB64(bc->ptr); +#endif + bc->ptr += 8; + bc->bits_left = 64; + + return 0; +} + +/** + * @return + * - 0 on successful refill + * - a negative number when bitstream end is hit + * + * Always succeeds when UNCHECKED_BITSTREAM_READER is enabled. + */ +static inline int bits_priv_refill_32(BitstreamContext *bc) +{ +#if !UNCHECKED_BITSTREAM_READER + if (bc->ptr >= bc->buffer_end) + return -1; +#endif + +#ifdef BITSTREAM_READER_LE + bc->bits = (uint64_t)AV_RL32(bc->ptr) << bc->bits_left | bc->bits; +#else + bc->bits = bc->bits | (uint64_t)AV_RB32(bc->ptr) << (32 - bc->bits_left); +#endif + bc->ptr += 4; + bc->bits_left += 32; + + return 0; +} + +/** + * Initialize BitstreamContext. + * @param buffer bitstream buffer, must be AV_INPUT_BUFFER_PADDING_SIZE bytes + * larger than the actual read bits because some optimized bitstream + * readers read 32 or 64 bits at once and could read over the end + * @param bit_size the size of the buffer in bits + * @return 0 on success, AVERROR_INVALIDDATA if the buffer_size would overflow. + */ +static inline int bits_init(BitstreamContext *bc, const uint8_t *buffer, + unsigned int bit_size) +{ + unsigned int buffer_size; + + if (bit_size > INT_MAX - 7 || !buffer) { + bc->buffer = NULL; + bc->ptr = NULL; + bc->bits_left = 0; + return AVERROR_INVALIDDATA; + } + + buffer_size = (bit_size + 7) >> 3; + + bc->buffer = buffer; + bc->buffer_end = buffer + buffer_size; + bc->ptr = bc->buffer; + bc->size_in_bits = bit_size; + bc->bits_left = 0; + bc->bits = 0; + + bits_priv_refill_64(bc); + + return 0; +} + +/** + * Initialize BitstreamContext. + * @param buffer bitstream buffer, must be AV_INPUT_BUFFER_PADDING_SIZE bytes + * larger than the actual read bits because some optimized bitstream + * readers read 32 or 64 bits at once and could read over the end + * @param byte_size the size of the buffer in bytes + * @return 0 on success, AVERROR_INVALIDDATA if the buffer_size would overflow + */ +static inline int bits_init8(BitstreamContext *bc, const uint8_t *buffer, + unsigned int byte_size) +{ + if (byte_size > INT_MAX / 8) + return AVERROR_INVALIDDATA; + return bits_init(bc, buffer, byte_size * 8); +} + +/** + * Return number of bits already read. + */ +static inline int bits_tell(const BitstreamContext *bc) +{ + return (bc->ptr - bc->buffer) * 8 - bc->bits_left; +} + +/** + * Return buffer size in bits. + */ +static inline int bits_size(const BitstreamContext *bc) +{ + return bc->size_in_bits; +} + +/** + * Return the number of the bits left in a buffer. + */ +static inline int bits_left(const BitstreamContext *bc) +{ + return (bc->buffer - bc->ptr) * 8 + bc->size_in_bits + bc->bits_left; +} + +static inline uint64_t bits_priv_val_get(BitstreamContext *bc, unsigned int n) +{ + uint64_t ret; + + av_assert2(n > 0 && n < 64); + +#ifdef BITSTREAM_READER_LE + ret = bc->bits & ((UINT64_C(1) << n) - 1); + bc->bits >>= n; +#else + ret = bc->bits >> (64 - n); + bc->bits <<= n; +#endif + bc->bits_left -= n; + + return ret; +} + +/** + * Return one bit from the buffer. + */ +static inline unsigned int bits_read_bit(BitstreamContext *bc) +{ + if (!bc->bits_left) + bits_priv_refill_64(bc); + + return bits_priv_val_get(bc, 1); +} + +/** + * Return n bits from the buffer, n has to be in the 1-32 range. + * May be faster than bits_read() when n is not a compile-time constant and is + * known to be non-zero; + */ +static inline uint32_t bits_read_nz(BitstreamContext *bc, unsigned int n) +{ + av_assert2(n > 0 && n <= 32); + + if (n > bc->bits_left) { + if (bits_priv_refill_32(bc) < 0) + bc->bits_left = n; + } + + return bits_priv_val_get(bc, n); +} + +/** + * Return n bits from the buffer, n has to be in the 0-32 range. + */ +static inline uint32_t bits_read(BitstreamContext *bc, unsigned int n) +{ + av_assert2(n <= 32); + + if (!n) + return 0; + + return bits_read_nz(bc, n); +} + +/** + * Return n bits from the buffer, n has to be in the 0-63 range. + */ +static inline uint64_t bits_read_63(BitstreamContext *bc, unsigned int n) +{ + uint64_t ret = 0; + unsigned left = 0; + + av_assert2(n <= 63); + + if (!n) + return 0; + + if (n > bc->bits_left) { + left = bc->bits_left; + n -= left; + + if (left) + ret = bits_priv_val_get(bc, left); + + if (bits_priv_refill_64(bc) < 0) + bc->bits_left = n; + + } + +#ifdef BITSTREAM_READER_LE + ret = bits_priv_val_get(bc, n) << left | ret; +#else + ret = bits_priv_val_get(bc, n) | ret << n; +#endif + + return ret; +} + +/** + * Return n bits from the buffer, n has to be in the 0-64 range. + */ +static inline uint64_t bits_read_64(BitstreamContext *bc, unsigned int n) +{ + av_assert2(n <= 64); + + if (n == 64) { + uint64_t ret = bits_read(bc, 32); +#ifdef BITSTREAM_READER_LE + return ret | ((uint64_t)bits_read(bc, 32) << 32); +#else + return (ret << 32) | bits_read(bc, 32); +#endif + } + return bits_read_63(bc, n); +} + +/** + * Return n bits from the buffer as a signed integer. + * n has to be in the 0-32 range. + */ +static inline int32_t bits_read_signed(BitstreamContext *bc, unsigned int n) +{ + return sign_extend(bits_read(bc, n), n); +} + +static inline unsigned int bits_priv_val_show(BitstreamContext *bc, unsigned int n) +{ +#ifdef BITSTREAM_READER_LE + return bc->bits & ((UINT64_C(1) << n) - 1); +#else + return bc->bits >> (64 - n); +#endif +} + +/** + * Return n bits from the buffer but do not change the buffer state. + * n has to be in the 0-32 range. + */ +static inline unsigned int bits_peek(BitstreamContext *bc, unsigned int n) +{ + av_assert2(n <= 32); + + if (n > bc->bits_left) + bits_priv_refill_32(bc); + + return bits_priv_val_show(bc, n); +} + +/** + * Return n bits from the buffer as a signed integer, + * do not change the buffer state. + * n has to be in the 0-32 range. + */ +static inline int bits_peek_signed(BitstreamContext *bc, unsigned int n) +{ + return sign_extend(bits_peek(bc, n), n); +} + +static inline void bits_priv_skip_remaining(BitstreamContext *bc, unsigned int n) +{ +#ifdef BITSTREAM_READER_LE + bc->bits >>= n; +#else + bc->bits <<= n; +#endif + bc->bits_left -= n; +} + +/** + * Skip n bits in the buffer. + */ +static inline void bits_skip(BitstreamContext *bc, unsigned int n) +{ + if (n < bc->bits_left) + bits_priv_skip_remaining(bc, n); + else { + n -= bc->bits_left; + bc->bits = 0; + bc->bits_left = 0; + + if (n >= 64) { + unsigned int skip = n / 8; + + n -= skip * 8; + bc->ptr += skip; + } + bits_priv_refill_64(bc); + if (n) + bits_priv_skip_remaining(bc, n); + } +} + +/** + * Seek to the given bit position. + */ +static inline void bits_seek(BitstreamContext *bc, unsigned pos) +{ + bc->ptr = bc->buffer; + bc->bits = 0; + bc->bits_left = 0; + + bits_skip(bc, pos); +} + +/** + * Skip bits to a byte boundary. + */ +static inline const uint8_t *bits_align(BitstreamContext *bc) +{ + unsigned int n = -bits_tell(bc) & 7; + if (n) + bits_skip(bc, n); + return bc->buffer + (bits_tell(bc) >> 3); +} + +/** + * Read MPEG-1 dc-style VLC (sign bit + mantissa with no MSB). + * If MSB not set it is negative. + * @param n length in bits + */ +static inline int bits_read_xbits(BitstreamContext *bc, unsigned int n) +{ + int32_t cache = bits_peek(bc, 32); + int sign = ~cache >> 31; + bits_priv_skip_remaining(bc, n); + + return ((((uint32_t)(sign ^ cache)) >> (32 - n)) ^ sign) - sign; +} + +/** + * Return decoded truncated unary code for the values 0, 1, 2. + */ +static inline int bits_decode012(BitstreamContext *bc) +{ + if (!bits_read_bit(bc)) + return 0; + else + return bits_read_bit(bc) + 1; +} + +/** + * Return decoded truncated unary code for the values 2, 1, 0. + */ +static inline int bits_decode210(BitstreamContext *bc) +{ + if (bits_read_bit(bc)) + return 0; + else + return 2 - bits_read_bit(bc); +} + +/* Read sign bit and flip the sign of the provided value accordingly. */ +static inline int bits_apply_sign(BitstreamContext *bc, int val) +{ + int sign = bits_read_signed(bc, 1); + return (val ^ sign) - sign; +} + +static inline int bits_skip_1stop_8data(BitstreamContext *s) +{ + if (bits_left(s) <= 0) + return AVERROR_INVALIDDATA; + + while (bits_read(s, 1)) { + bits_skip(s, 8); + if (bits_left(s) <= 0) + return AVERROR_INVALIDDATA; + } + + return 0; +} + +/* Unget up to 32 bits. */ +static inline void bits_unget(BitstreamContext *bc, uint64_t value, + unsigned amount) +{ + unsigned cache_size = sizeof(bc->bits) * 8; + + av_assert2(amount <= 32); + + if (bc->bits_left + amount > cache_size) { + bc->bits_left -= 4 * 8; + bc->ptr -= 4; + } + +#ifdef BITSTREAM_READER_LE + bc->bits = (bc->bits << amount) | value; +#else + bc->bits = (bc->bits >> amount) | (value << (cache_size - amount)); +#endif + bc->bits_left += amount; +} + +/** + * Return the LUT element for the given bitstream configuration. + */ +static inline int bits_priv_set_idx(BitstreamContext *bc, int code, int *n, int *nb_bits, + const VLCElem *table) +{ + unsigned idx; + + *nb_bits = -*n; + idx = bits_peek(bc, *nb_bits) + code; + *n = table[idx].len; + + return table[idx].sym; +} + +/** + * Parse a vlc code. + * @param bits is the number of bits which will be read at once, must be + * identical to nb_bits in init_vlc() + * @param max_depth is the number of times bits bits must be read to completely + * read the longest vlc code + * = (max_vlc_length + bits - 1) / bits + * If the vlc code is invalid and max_depth=1, then no bits will be removed. + * If the vlc code is invalid and max_depth>1, then the number of bits removed + * is undefined. + */ +static inline int bits_read_vlc(BitstreamContext *bc, const VLCElem *table, + int bits, int max_depth) +{ + int nb_bits; + unsigned idx = bits_peek(bc, bits); + int code = table[idx].sym; + int n = table[idx].len; + + if (max_depth > 1 && n < 0) { + bits_priv_skip_remaining(bc, bits); + code = bits_priv_set_idx(bc, code, &n, &nb_bits, table); + if (max_depth > 2 && n < 0) { + bits_priv_skip_remaining(bc, nb_bits); + code = bits_priv_set_idx(bc, code, &n, &nb_bits, table); + } + } + bits_priv_skip_remaining(bc, n); + + return code; +} + +#define BITS_RL_VLC(level, run, bc, table, bits, max_depth) \ + do { \ + int n, nb_bits; \ + unsigned int index = bits_peek(bc, bits); \ + level = table[index].level; \ + n = table[index].len; \ + \ + if (max_depth > 1 && n < 0) { \ + bits_skip(bc, bits); \ + \ + nb_bits = -n; \ + \ + index = bits_peek(bc, nb_bits) + level; \ + level = table[index].level; \ + n = table[index].len; \ + if (max_depth > 2 && n < 0) { \ + bits_skip(bc, nb_bits); \ + nb_bits = -n; \ + \ + index = bits_peek(bc, nb_bits) + level; \ + level = table[index].level; \ + n = table[index].len; \ + } \ + } \ + run = table[index].run; \ + bits_skip(bc, n); \ + } while (0) + +#endif /* AVCODEC_BITSTREAM_H */ From patchwork Sun Jul 3 10:26:36 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Khirnov X-Patchwork-Id: 36629 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:8b27:b0:88:1bbf:7fd2 with SMTP id l39csp2138960pzh; Sun, 3 Jul 2022 03:28:38 -0700 (PDT) X-Google-Smtp-Source: AGRyM1s8nzIRG+4j8CtQbRnKKvWTD1MapyPmnPV5LqCtjuKmwDk/uStIQghWp3uyk+/rZVeNwW7K X-Received: by 2002:a17:907:ea5:b0:726:2c1c:312f with SMTP id ho37-20020a1709070ea500b007262c1c312fmr22691833ejc.248.1656844117907; Sun, 03 Jul 2022 03:28:37 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1656844117; cv=none; d=google.com; s=arc-20160816; b=uCVq+iwLiA36MhMwpzaUDSgyu816S9iEC6S3KrYGCPPCWiTA5bK+aBodS5hCQDzu6B T79QVz3V/VksoP7XTooiwLNkxlzcU25Edr10oBsaZ6eck3cK5CBkMZ1+448CNBQ646HH vzFd/DbaF47kDY3KSxXnnWy86i/L/oZV7wc+NGu/NN2PnvrE/pmRmE/K6KzolJ0H/Ebp 4U9wb0/nZV3mLlXLTEfHKbv4W2z3eWC4G7iCqTW40qilT4wj/xNELEdMvHD1y7e7Sp2o rvMTfE6eJYTcqUcTKFRZQz6mz2V003qLjg9qiD9SQdlaWMtS0mN75cSffngzsn25L0qB JCJg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:delivered-to; bh=kin57EYFyD1vXWcBZe+1SsBQn+wz+FJffcpIvsEj9Lk=; b=BZNaMtbCAqdWfmg7f/TGw+oyE8SaXPn3N5STyeGyoMj7e4ozDeeCJ+5LZCSqTRzD9s s7nscZkZjLTkbTV2zhXLs1i7niDkL6B5ZZw8Y1hyQjEWoPnR4Pm6bDQVsHL5ml9rREgV R/5dWcKbr3/QlHGIEdV8qKr0o5drzK5PXaaEMFxVFK4d8pHAYfUfVXe/NynsY3Sk2PGa mLacJTll3gO+TuPWluwWlImwH5S04oPDFJGNcZidTaWh3xLlDLU5CX/Bvz8hM+buy9X9 r8ptDwdL8rCUVLO4uF5pF2/Hq6EK8BfU7jVBJkoCi+DECPCL6S2VofC/LE+JPgXYzEgU ORNg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id cy20-20020a0564021c9400b0043a2cd05f6csi944332edb.178.2022.07.03.03.28.37; Sun, 03 Jul 2022 03:28:37 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 97DC668B987; Sun, 3 Jul 2022 13:28:21 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail0.khirnov.net (red.khirnov.net [176.97.15.12]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 823A068B98E for ; Sun, 3 Jul 2022 13:28:13 +0300 (EEST) Received: from localhost (localhost [IPv6:::1]) by mail0.khirnov.net (Postfix) with ESMTP id D54DC240179 for ; Sun, 3 Jul 2022 12:28:12 +0200 (CEST) Received: from mail0.khirnov.net ([IPv6:::1]) by localhost (mail0.khirnov.net [IPv6:::1]) (amavisd-new, port 10024) with ESMTP id LBWJ9PylkATR for ; Sun, 3 Jul 2022 12:28:12 +0200 (CEST) Received: from libav.khirnov.net (libav.khirnov.net [IPv6:2a00:c500:561:201::7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "libav.khirnov.net", Issuer "smtp.khirnov.net SMTP CA" (verified OK)) by mail0.khirnov.net (Postfix) with ESMTPS id A673C24017C for ; Sun, 3 Jul 2022 12:28:10 +0200 (CEST) Received: by libav.khirnov.net (Postfix, from userid 1000) id CE34E3A09CF; Sun, 3 Jul 2022 12:28:10 +0200 (CEST) From: Anton Khirnov To: ffmpeg-devel@ffmpeg.org Date: Sun, 3 Jul 2022 12:26:36 +0200 Message-Id: <20220703102636.32523-3-anton@khirnov.net> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220703102636.32523-1-anton@khirnov.net> References: <20220703102636.32523-1-anton@khirnov.net> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 3/3] lavc/tests: add a cached bitstream reader test X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 2V+0SUcA7tQp --- libavcodec/Makefile | 2 + libavcodec/tests/bitstream_be.c | 19 ++++ libavcodec/tests/bitstream_le.c | 20 ++++ libavcodec/tests/bitstream_template.c | 153 ++++++++++++++++++++++++++ tests/fate/libavcodec.mak | 10 ++ 5 files changed, 204 insertions(+) create mode 100644 libavcodec/tests/bitstream_be.c create mode 100644 libavcodec/tests/bitstream_le.c create mode 100644 libavcodec/tests/bitstream_template.c diff --git a/libavcodec/Makefile b/libavcodec/Makefile index 050934101c..0457f6d0fd 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -1252,6 +1252,8 @@ SKIPHEADERS-$(CONFIG_V4L2_M2M) += v4l2_buffers.h v4l2_context.h v4l2_m2m TESTPROGS = avcodec \ avpacket \ + bitstream_be \ + bitstream_le \ celp_math \ codec_desc \ htmlsubtitles \ diff --git a/libavcodec/tests/bitstream_be.c b/libavcodec/tests/bitstream_be.c new file mode 100644 index 0000000000..bc562ed3b1 --- /dev/null +++ b/libavcodec/tests/bitstream_be.c @@ -0,0 +1,19 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "bitstream_template.c" diff --git a/libavcodec/tests/bitstream_le.c b/libavcodec/tests/bitstream_le.c new file mode 100644 index 0000000000..a907676438 --- /dev/null +++ b/libavcodec/tests/bitstream_le.c @@ -0,0 +1,20 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#define BITSTREAM_READER_LE +#include "bitstream_template.c" diff --git a/libavcodec/tests/bitstream_template.c b/libavcodec/tests/bitstream_template.c new file mode 100644 index 0000000000..8685f27b0c --- /dev/null +++ b/libavcodec/tests/bitstream_template.c @@ -0,0 +1,153 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#define ASSERT_LEVEL 2 + +#include "libavutil/avassert.h" +#include "libavutil/lfg.h" +#include "libavutil/random_seed.h" + +#include "libavcodec/bitstream.h" +#include "libavcodec/defs.h" + +#ifdef BITSTREAM_READER_LE +#define BITSTREAM_WRITER_LE +#endif +#include "libavcodec/put_bits.h" + +#define SIZE 157 + +enum Op { + OP_READ, + OP_READ_NZ, + OP_READ_BIT, + OP_READ_63, + OP_READ_64, + OP_READ_SIGNED, + OP_UNGET, + OP_NB, +}; + +int main(int argc, char **argv) +{ + BitstreamContext bc; + PutBitContext pb; + AVLFG lfg; + + uint8_t buf[SIZE + AV_INPUT_BUFFER_PADDING_SIZE]; + uint8_t dst[SIZE + AV_INPUT_BUFFER_PADDING_SIZE]; + + uint32_t random_seed; + uint64_t val; + int32_t sval; + unsigned count; + + /* generate random input, using a given or random seed */ + if (argc > 1) + random_seed = strtoul(argv[1], NULL, 0); + else + random_seed = av_get_random_seed(); + + fprintf(stderr, "Testing with LFG seed: %"PRIu32"\n", random_seed); + av_lfg_init(&lfg, random_seed); + + for (unsigned i = 0; i < SIZE; i++) + buf[i] = av_lfg_get(&lfg); + + bits_init8 (&bc, buf, SIZE); + init_put_bits(&pb, dst, SIZE); + + /* use a random sequence of bitreading operations to transfer data + * from BitstreamContext to PutBitContext */ + while (bits_left(&bc) > 0) { + enum Op op = av_lfg_get(&lfg) % OP_NB; + + switch (op) { + case OP_READ: + count = av_lfg_get(&lfg) % FFMIN(33, bits_left(&bc) + 1); + val = bits_read(&bc, count); + + fprintf(stderr, "%d read %u: %"PRIu64"\n", bits_tell(&bc) - count, count, val); + + put_bits64(&pb, count, val); + break; + case OP_READ_NZ: + count = av_lfg_get(&lfg) % FFMIN(33, bits_left(&bc) + 1); + count = FFMAX(count, 1); + val = bits_read_nz(&bc, count); + + fprintf(stderr, "%d read_nz %u: %"PRIu64"\n", bits_tell(&bc) - count, count, val); + + put_bits64(&pb, count, val); + break; + case OP_READ_BIT: + val = bits_read_bit(&bc); + + fprintf(stderr, "%d read_bit: %"PRIu64"\n", bits_tell(&bc) - 1, val); + + put_bits(&pb, 1, val); + break; + case OP_READ_63: + count = av_lfg_get(&lfg) % FFMIN(64, bits_left(&bc) + 1); + val = bits_read_63(&bc, count); + + fprintf(stderr, "%d read_63 %u: %"PRIu64"\n", bits_tell(&bc) - count, count, val); + + put_bits64(&pb, count, val); + break; + case OP_READ_64: + count = av_lfg_get(&lfg) % FFMIN(65, bits_left(&bc) + 1); + val = bits_read_64(&bc, count); + + fprintf(stderr, "%d read_64 %u: %"PRIu64"\n", bits_tell(&bc) - count, count, val); + + put_bits64(&pb, count, val); + break; + case OP_READ_SIGNED: + count = av_lfg_get(&lfg) % FFMIN(33, bits_left(&bc) + 1); + sval = bits_read_signed(&bc, count); + + fprintf(stderr, "%d read_signed %u: %"PRId32"\n", bits_tell(&bc) - count, count, sval); + + if (count == 32) put_bits32(&pb, sval); + else put_sbits(&pb, count, sval); + break; + case OP_UNGET: + count = av_lfg_get(&lfg) % FFMIN(33, bits_left(&bc) + 1); + val = bits_read(&bc, count); + bits_unget(&bc, val, count); + + fprintf(stderr, "%d unget %u: %"PRIu64"\n", bits_tell(&bc), count, val); + + break; + default: + av_assert0(0); + } + } + + flush_put_bits(&pb); + + for (unsigned i = 0; i < SIZE; i++) + if (buf[i] != dst[i]) { + fprintf(stderr, "Mismatch at byte %u: %hhu %hhu; seed %"PRIu32"\n", + i, buf[i], dst[i], random_seed); + return 1; + } + + return 0; +} diff --git a/tests/fate/libavcodec.mak b/tests/fate/libavcodec.mak index aa199e0308..8f56fae3a8 100644 --- a/tests/fate/libavcodec.mak +++ b/tests/fate/libavcodec.mak @@ -3,6 +3,16 @@ fate-avpacket: libavcodec/tests/avpacket$(EXESUF) fate-avpacket: CMD = run libavcodec/tests/avpacket$(EXESUF) fate-avpacket: CMP = null +FATE_LIBAVCODEC-yes += fate-bitstream-be +fate-bitstream-be: libavcodec/tests/bitstream_be$(EXESUF) +fate-bitstream-be: CMD = run libavcodec/tests/bitstream_be$(EXESUF) +fate-bitstream-be: CMP = null + +FATE_LIBAVCODEC-yes += fate-bitstream-le +fate-bitstream-le: libavcodec/tests/bitstream_le$(EXESUF) +fate-bitstream-le: CMD = run libavcodec/tests/bitstream_le$(EXESUF) +fate-bitstream-le: CMP = null + FATE_LIBAVCODEC-$(CONFIG_CABAC) += fate-cabac fate-cabac: libavcodec/tests/cabac$(EXESUF) fate-cabac: CMD = run libavcodec/tests/cabac$(EXESUF)