From patchwork Mon Jun 20 20:14:35 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Martijn van Beurden X-Patchwork-Id: 36357 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:1a22:b0:84:42e0:ad30 with SMTP id cj34csp1849056pzb; Mon, 20 Jun 2022 13:15:13 -0700 (PDT) X-Google-Smtp-Source: AGRyM1smgj8nPHOhoN1sOs3mhtt7jjWZg0pxA1Cbw5fMoY0vAvzEXMepxNverKq1yPZB3Z4U0Iuw X-Received: by 2002:a05:6402:299a:b0:435:c75:64e6 with SMTP id eq26-20020a056402299a00b004350c7564e6mr31240238edb.134.1655756112936; Mon, 20 Jun 2022 13:15:12 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1655756112; cv=none; d=google.com; s=arc-20160816; b=IJL/WgGRkwyqTRrF4wi5oUBD4DVdyseT93SndAs8FLr8FjsA3P8GGlu9Cwtz1NV95/ UJs3Jlcy4Gf0GsYOYVqkiVXr0kIM93OPEaCr+B38g6H4EHZwoaboYhJyDiYHVn4X1D7n TfDsKGh0HrF3lYgjTA0AxCiust7aRc1ENv9BFNWhlBgbsmMH8gF+GDdRSIsWZBN+3cDJ ujZ02ANzAQrpz465QAMbSoyEyeEu8I2HNwFXPrE/83J7yltYaIGB143pDNPpLE3uG+ma kHQAU0IsCV/UJiqZyLI1pWW0K5zs3Fqv/uNnZiZkLMS5jj6/O072CRNjPR9pqU4ygWN5 GhSw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:message-id:date:to:from :dkim-signature:delivered-to; bh=3xouj/n/1O3JMzT7LTvzbGo/La4YeeuIZjmXtLDXXIU=; b=UVSMu6iXHy+f9zEkcNTBNfHg+nosdzcQZGOwF++7WRsqGEOEflTAvsJlo7w/GKJYPd F0ZR/64xP2TJvKIMUBKrASxVK9JhB6b/rIh5FviF3DAjraHQ5e2JrnzrzptuIBFqDKUG zKds8WQFE6F4X21MQgy7Ker/f/p9+eMDrCUBqnyrRziphFJPKb0m4UJV8OSnKQxBd+B4 DR5ZCO7PcVcGzEH9tgVuLLWmnYxFh62ZVJNYgyo7TCnu3QJH1yg4TlOJb92iV4uXFoTX LWGqP5vMKQKOZBlj93xPubbzWb2Bo9rGgEsqLRZRzhADYnynKe00iGHd95eYkEpK0HS0 J2Iw== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b="a4s/Fy3d"; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id cw12-20020a170906478c00b006f3c6ceb0efsi12913116ejc.51.2022.06.20.13.15.10; Mon, 20 Jun 2022 13:15:12 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b="a4s/Fy3d"; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id DD39068B616; Mon, 20 Jun 2022 23:15:06 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-ed1-f50.google.com (mail-ed1-f50.google.com [209.85.208.50]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id E256C68B289 for ; Mon, 20 Jun 2022 23:15:00 +0300 (EEST) Received: by mail-ed1-f50.google.com with SMTP id o9so7372641edt.12 for ; Mon, 20 Jun 2022 13:15:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=xtDfmStMIJHijFxKMNtgjvBBCiC0ZRh87BqZk/TP94g=; b=a4s/Fy3d2kEeGPLOYcqhL+EO84udCr9W1J+BbvvcJZCtEO8dzh3/PyHIy4M1mxwGaf K6F/i5WKpcyY7vGlT/gwI9rHdSQxZzt/nKDrPTQRi1yhEV4KkuUqKtM0hQk1QG/5mppa ifIDwtWgrp1Pr4tV1oMl9iDgrsXLpYeO6ZLdx9HqXD2zaX6P40LazQLKkn2ggK1A4w4L By5BITmjytb0OiKv4m9TBbjlNa81quaKlqNOT+hdGIW+QahGdS6G6JGpll3X46zNRMm+ 7m2sidw2I01LpHbykEgS/0IDTFQCbiQA5aG/JT1qjmYO/31TsCGxqcddzE30PirFw2Xz 6idA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=xtDfmStMIJHijFxKMNtgjvBBCiC0ZRh87BqZk/TP94g=; b=ex8m4Y8KDDEho1sNU1gAiQ8CKLq2/WDkb8qO2TordEgPo8NBMLWgKGCi7NMXIDbKfR wcNTImIwk1khzcopKIRhpVapOHPgyO6FbceOAA+nC9YimmLqumq2X+TobPcjCI+0pce7 U2dTvSwsYkpEH4poI3BYcGr7OGS5gJ34re6o5PWhQohjAqVZfyBZL+0hV3YHWe90kM9l RgRPVVptN2iba/NaOpgDf7F4I4NMjxet95Dp36IwatN5C5buWhB2nhVF6PQm5kGg5VMZ MlSoC6HSmw1CSnmztEKCHfEVvSBxjOR8eib+h86f7KLBfI3k1+PhV2bWU+OvvaVAyL3+ Wffw== X-Gm-Message-State: AJIora/JMNSjoIO5k6b9paiDBrF8ojiIVY56fxlzreBeMx5TlrIbLBFg wgSBi5K85Yn0FW9hB/hgnvMooFsp8PV8+w== X-Received: by 2002:a05:6402:2789:b0:42d:ce10:1d6 with SMTP id b9-20020a056402278900b0042dce1001d6mr31463960ede.188.1655756100069; Mon, 20 Jun 2022 13:15:00 -0700 (PDT) Received: from localhost.localdomain (92-64-99-37.biz.kpn.net. [92.64.99.37]) by smtp.googlemail.com with ESMTPSA id q20-20020a17090676d400b006f3ef214db4sm6448296ejn.26.2022.06.20.13.14.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Jun 2022 13:14:59 -0700 (PDT) From: Martijn van Beurden To: ffmpeg-devel@ffmpeg.org Date: Mon, 20 Jun 2022 22:14:35 +0200 Message-Id: <20220620201435.2083423-1-mvanb1@gmail.com> X-Mailer: git-send-email 2.30.2 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH] avformat/flacdec: Implement decoding of 32-bit PCM X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Martijn van Beurden Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: JcFae8Ixiblx Recently libFLAC gained the ability (not in any released version yet though) to create FLAC files containing 32-bit int PCM samples. To keep complexity reasonable, the choice was made to limit residuals to 32-bit integers, which the encoder must make sure of. In case the encoder cannot find any predictor of which the residuals fit this limit, it must default to using a verbatim subframe. Tests have shown that this does not happen often (<0.1% of subframes on a music corpus of various styles). See also discussion here: https://github.com/ietf-wg-cellar/flac-specification/pull/148 This patch adds decoding of these files to ffmpeg. --- libavcodec/flac.c | 4 +- libavcodec/flacdec.c | 248 ++++++++++++++++++++++++++++++++++++++---- libavcodec/get_bits.h | 12 ++ libavcodec/mathops.h | 9 ++ 4 files changed, 250 insertions(+), 23 deletions(-) diff --git a/libavcodec/flac.c b/libavcodec/flac.c index dd68830622..f326d8fa5c 100644 --- a/libavcodec/flac.c +++ b/libavcodec/flac.c @@ -27,7 +27,7 @@ #include "flac.h" #include "flacdata.h" -static const int8_t sample_size_table[] = { 0, 8, 12, 0, 16, 20, 24, 0 }; +static const int8_t sample_size_table[] = { 0, 8, 12, 0, 16, 20, 24, 32 }; static const AVChannelLayout flac_channel_layouts[8] = { AV_CHANNEL_LAYOUT_MONO, @@ -81,7 +81,7 @@ int ff_flac_decode_frame_header(AVCodecContext *avctx, GetBitContext *gb, /* bits per sample */ bps_code = get_bits(gb, 3); - if (bps_code == 3 || bps_code == 7) { + if (bps_code == 3) { av_log(avctx, AV_LOG_ERROR + log_level_offset, "invalid sample size code (%d)\n", bps_code); diff --git a/libavcodec/flacdec.c b/libavcodec/flacdec.c index 87f20c7425..49952ce120 100644 --- a/libavcodec/flacdec.c +++ b/libavcodec/flacdec.c @@ -63,6 +63,9 @@ typedef struct FLACContext { int32_t *decoded[FLAC_MAX_CHANNELS]; ///< decoded samples uint8_t *decoded_buffer; unsigned int decoded_buffer_size; + int64_t *decoded_33bps; ///< decoded samples for a 33 bps subframe + uint8_t *decoded_buffer_33bps; + unsigned int decoded_buffer_size_33bps; int buggy_lpc; ///< use workaround for old lavc encoded files FLACDSPContext dsp; @@ -154,6 +157,24 @@ static int allocate_buffers(FLACContext *s) s->flac_stream_info.channels, s->flac_stream_info.max_blocksize, AV_SAMPLE_FMT_S32P, 0); + if (ret >= 0 && s->flac_stream_info.bps == 32 && s->flac_stream_info.channels == 2) { + buf_size = av_samples_get_buffer_size(NULL, 1, + s->flac_stream_info.max_blocksize, + AV_SAMPLE_FMT_S64P, 0); + if (buf_size < 0) + return buf_size; + + av_fast_malloc(&s->decoded_buffer_33bps, &s->decoded_buffer_size_33bps, buf_size); + if (!s->decoded_buffer) + return AVERROR(ENOMEM); + + ret = av_samples_fill_arrays((uint8_t **)&s->decoded_33bps, NULL, + s->decoded_buffer_33bps, + 1, + s->flac_stream_info.max_blocksize, + AV_SAMPLE_FMT_S64P, 0); + + } return ret < 0 ? ret : 0; } @@ -331,6 +352,94 @@ static int decode_subframe_fixed(FLACContext *s, int32_t *decoded, return 0; } +static int decode_subframe_fixed_wide(FLACContext *s, int32_t *decoded, + int pred_order, int bps) +{ + const int blocksize = s->blocksize; + int i; + int ret; + + /* warm up samples */ + for (i = 0; i < pred_order; i++) { + decoded[i] = get_sbits_long(&s->gb, bps); + } + + if ((ret = decode_residuals(s, decoded, pred_order)) < 0) + return ret; + + switch (pred_order) { + case 0: + break; + case 1: + for (i = pred_order; i < blocksize; i++) + decoded[i] += decoded[i-1]; + break; + case 2: + for (i = pred_order; i < blocksize; i++) + decoded[i] = (int64_t)decoded[i] + 2*(int64_t)decoded[i-1] - (int64_t)decoded[i-2]; + break; + case 3: + for (i = pred_order; i < blocksize; i++) + decoded[i] = (int64_t)decoded[i] + 3*(int64_t)decoded[i-1] - 3*(int64_t)decoded[i-2] + (int64_t)decoded[i-3]; + break; + case 4: + for (i = pred_order; i < blocksize; i++) + decoded[i] = (int64_t)decoded[i] + 4*(int64_t)decoded[i-1] - 6*(int64_t)decoded[i-2] + 4*(int64_t)decoded[i-3] - (int64_t)decoded[i-4]; + break; + default: + av_log(s->avctx, AV_LOG_ERROR, "illegal pred order %d\n", pred_order); + return AVERROR_INVALIDDATA; + } + + return 0; +} + + +static int decode_subframe_fixed_33bps(FLACContext *s, int64_t *decoded, + int32_t *residual, int pred_order) +{ + const int blocksize = s->blocksize; + int i; + int ret; + + /* warm up samples */ + for (i = 0; i < pred_order; i++) { + decoded[i] = get_sbits64(&s->gb, 33); + av_log(s->avctx, AV_LOG_DEBUG, "warm-up %d = %" PRId64 "\n", i, decoded[i]); + } + + if ((ret = decode_residuals(s, residual, pred_order)) < 0) + return ret; + + switch (pred_order) { + case 0: + for (i = pred_order; i < blocksize; i++) + decoded[i] = residual[i]; + break; + case 1: + for (i = pred_order; i < blocksize; i++) + decoded[i] = residual[i] + decoded[i-1]; + break; + case 2: + for (i = pred_order; i < blocksize; i++) + decoded[i] = residual[i] + 2*decoded[i-1] - decoded[i-2]; + break; + case 3: + for (i = pred_order; i < blocksize; i++) + decoded[i] = residual[i] + 3*decoded[i-1] - 3*decoded[i-2] + decoded[i-3]; + break; + case 4: + for (i = pred_order; i < blocksize; i++) + decoded[i] = residual[i] + 4*decoded[i-1] - 6*decoded[i-2] + 4*decoded[i-3] - decoded[i-4]; + break; + default: + av_log(s->avctx, AV_LOG_ERROR, "illegal pred order %d\n", pred_order); + return AVERROR_INVALIDDATA; + } + + return 0; +} + static void lpc_analyze_remodulate(SUINT32 *decoded, const int coeffs[32], int order, int qlevel, int len, int bps) { @@ -402,12 +511,53 @@ static int decode_subframe_lpc(FLACContext *s, int32_t *decoded, int pred_order, return 0; } +static int decode_subframe_lpc_33bps(FLACContext *s, int64_t *decoded, + int32_t *residual, int pred_order) +{ + int i, j, ret; + int coeff_prec, qlevel; + int coeffs[32]; + + /* warm up samples */ + for (i = 0; i < pred_order; i++) { + decoded[i] = get_sbits64(&s->gb, 33); + } + + coeff_prec = get_bits(&s->gb, 4) + 1; + if (coeff_prec == 16) { + av_log(s->avctx, AV_LOG_ERROR, "invalid coeff precision\n"); + return AVERROR_INVALIDDATA; + } + qlevel = get_sbits(&s->gb, 5); + if (qlevel < 0) { + av_log(s->avctx, AV_LOG_ERROR, "qlevel %d not supported, maybe buggy stream\n", + qlevel); + return AVERROR_INVALIDDATA; + } + + for (i = 0; i < pred_order; i++) { + coeffs[pred_order - i - 1] = get_sbits(&s->gb, coeff_prec); + } + + if ((ret = decode_residuals(s, residual, pred_order)) < 0) + return ret; + + for (i = pred_order; i < s->blocksize; i++, decoded++) { + int64_t sum = 0; + for (j = 0; j < pred_order; j++) + sum += (int64_t)coeffs[j] * decoded[j]; + decoded[j] = residual[i] + (sum >> qlevel); + } + + return 0; +} + static inline int decode_subframe(FLACContext *s, int channel) { int32_t *decoded = s->decoded[channel]; int type, wasted = 0; int bps = s->flac_stream_info.bps; - int i, tmp, ret; + int i, ret; if (channel == 0) { if (s->ch_mode == FLAC_CHMODE_RIGHT_SIDE) @@ -436,34 +586,63 @@ static inline int decode_subframe(FLACContext *s, int channel) wasted = 1 + get_unary(&s->gb, 1, get_bits_left(&s->gb)); bps -= wasted; } - if (bps > 32) { - avpriv_report_missing_feature(s->avctx, "Decorrelated bit depth > 32"); - return AVERROR_PATCHWELCOME; - } //FIXME use av_log2 for types if (type == 0) { - tmp = get_sbits_long(&s->gb, bps); - for (i = 0; i < s->blocksize; i++) - decoded[i] = tmp; + if (bps < 33) { + int32_t tmp = get_sbits_long(&s->gb, bps); + for (i = 0; i < s->blocksize; i++) + decoded[i] = tmp; + } else { + int64_t tmp = get_sbits64(&s->gb, 33); + for (i = 0; i < s->blocksize; i++) + s->decoded_33bps[i] = tmp; + } } else if (type == 1) { - for (i = 0; i < s->blocksize; i++) - decoded[i] = get_sbits_long(&s->gb, bps); + if (bps < 33) { + for (i = 0; i < s->blocksize; i++) + decoded[i] = get_sbits_long(&s->gb, bps); + } else { + for (i = 0; i < s->blocksize; i++) + s->decoded_33bps[i] = get_sbits64(&s->gb, 33); + } } else if ((type >= 8) && (type <= 12)) { - if ((ret = decode_subframe_fixed(s, decoded, type & ~0x8, bps)) < 0) - return ret; + int order = type & ~0x8; + if (bps < 33) { + if (bps + order <= 32) { + if ((ret = decode_subframe_fixed(s, decoded, order, bps)) < 0) + return ret; + } else { + if ((ret = decode_subframe_fixed_wide(s, decoded, order, bps)) < 0) + return ret; + } + } else { + if ((ret = decode_subframe_fixed_33bps(s, s->decoded_33bps, decoded, order)) < 0) + return ret; + } } else if (type >= 32) { - if ((ret = decode_subframe_lpc(s, decoded, (type & ~0x20)+1, bps)) < 0) - return ret; + if (bps < 33) { + if ((ret = decode_subframe_lpc(s, decoded, (type & ~0x20)+1, bps)) < 0) + return ret; + } else { + if ((ret = decode_subframe_lpc_33bps(s, s->decoded_33bps, decoded, (type & ~0x20)+1)) < 0) + return ret; + } } else { av_log(s->avctx, AV_LOG_ERROR, "invalid coding type\n"); return AVERROR_INVALIDDATA; } - if (wasted && wasted < 32) { - int i; - for (i = 0; i < s->blocksize; i++) - decoded[i] = (unsigned)decoded[i] << wasted; + if (wasted) { + if (wasted+bps == 33) { + int i; + for (i = 0; i < s->blocksize; i++) + s->decoded_33bps[i] = (uint64_t)decoded[i] << wasted; + } else if (wasted < 32) { + int i; + for (i = 0; i < s->blocksize; i++) + decoded[i] = (unsigned)decoded[i] << wasted; + } } return 0; @@ -554,6 +733,26 @@ static int decode_frame(FLACContext *s) return 0; } +static void decorrelate_33bps(int ch_mode, int32_t **decoded, int64_t *decoded_33bps, int len) +{ + int i; + if (ch_mode == FLAC_CHMODE_LEFT_SIDE ) { + for (i = 0; i < len; i++) + decoded[1][i] = decoded[0][i] - decoded_33bps[i]; + } else if (ch_mode == FLAC_CHMODE_RIGHT_SIDE ) { + for (i = 0; i < len; i++) + decoded[0][i] = decoded[1][i] + decoded_33bps[i]; + } else if (ch_mode == FLAC_CHMODE_MID_SIDE ) { + for (i = 0; i < len; i++) { + uint64_t a = decoded[0][i]; + int64_t b = decoded_33bps[i]; + a -= b >> 1; + decoded[0][i] = (a + b); + decoded[1][i] = a; + } + } +} + static int flac_decode_frame(AVCodecContext *avctx, AVFrame *frame, int *got_frame_ptr, AVPacket *avpkt) { @@ -618,9 +817,15 @@ static int flac_decode_frame(AVCodecContext *avctx, AVFrame *frame, if ((ret = ff_thread_get_buffer(avctx, frame, 0)) < 0) return ret; - s->dsp.decorrelate[s->ch_mode](frame->data, s->decoded, - s->flac_stream_info.channels, - s->blocksize, s->sample_shift); + if (s->flac_stream_info.bps == 32 && s->ch_mode > 0) { + decorrelate_33bps(s->ch_mode, s->decoded, s->decoded_33bps, s->blocksize); + s->dsp.decorrelate[0](frame->data, s->decoded, s->flac_stream_info.channels, + s->blocksize, s->sample_shift); + } else { + s->dsp.decorrelate[s->ch_mode](frame->data, s->decoded, + s->flac_stream_info.channels, + s->blocksize, s->sample_shift); + } if (bytes_read > buf_size) { av_log(s->avctx, AV_LOG_ERROR, "overread: %d\n", bytes_read - buf_size); @@ -641,6 +846,7 @@ static av_cold int flac_decode_close(AVCodecContext *avctx) FLACContext *s = avctx->priv_data; av_freep(&s->decoded_buffer); + av_freep(&s->decoded_buffer_33bps); return 0; } diff --git a/libavcodec/get_bits.h b/libavcodec/get_bits.h index 16f8af5107..c12b62d4be 100644 --- a/libavcodec/get_bits.h +++ b/libavcodec/get_bits.h @@ -597,6 +597,18 @@ static inline int get_sbits_long(GetBitContext *s, int n) return sign_extend(get_bits_long(s, n), n); } +/** + * Read 0-64 bits as a signed integer. + */ +static inline int64_t get_sbits64(GetBitContext *s, int n) +{ + // sign_extend(x, 0) is undefined + if (!n) + return 0; + + return sign_extend64(get_bits64(s, n), n); +} + /** * Show 0-32 bits. */ diff --git a/libavcodec/mathops.h b/libavcodec/mathops.h index f81d21f9c4..8a82d9d086 100644 --- a/libavcodec/mathops.h +++ b/libavcodec/mathops.h @@ -137,6 +137,15 @@ static inline av_const int sign_extend(int val, unsigned bits) } #endif +#ifndef sign_extend64 +static inline av_const int64_t sign_extend64(int64_t val, unsigned bits) +{ + unsigned shift = 8 * sizeof(int64_t) - bits; + union { uint64_t u; int64_t s; } v = { (uint64_t) val << shift }; + return v.s >> shift; +} +#endif + #ifndef zero_extend static inline av_const unsigned zero_extend(unsigned val, unsigned bits) {