From patchwork Mon Jan 3 20:26:08 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Martijn van Beurden X-Patchwork-Id: 33038 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a6b:cd86:0:0:0:0:0 with SMTP id d128csp16414895iog; Mon, 3 Jan 2022 12:26:35 -0800 (PST) X-Google-Smtp-Source: ABdhPJyKwkdIprl7+DIAdFO1+k+RXmJCgBf/CL/J7hl0NNMNqRoiKI3+H1Mt6FPE7Xm51ow7su61 X-Received: by 2002:a17:907:3e0f:: with SMTP id hp15mr37608601ejc.432.1641241594733; Mon, 03 Jan 2022 12:26:34 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1641241594; cv=none; d=google.com; s=arc-20160816; b=m0shleH/CON6D7jySB6a7a7MVRqNz0veU0OUux6hJQO35SpJlyaAIu2Ivgg5Urx6Zs rSPyqwdmA6k7moWoOoQjQzyhS/R14NQH6DI6Nfp0KHN3XpMnWbd4DJGPZH2eMBK1xQ0e pYBLvvzj1MxNAbLiAG0jeIgbt9AULkRC6H93+2g5qdc+VJyHD2kw4n5mn4EVvLAxHsVK I8AlRHwtMXWYF8h8M9f8BI+gSyZQ9ZdcIliq2wT2cwYNPOtv1FZjX+9Q0nvVfqW60SN6 SLFdRwoIOY7X/gNRz+VxXsk7+YKsdrkzXqFeQHy+rv+gt8krt9YRNvBM9tI2KufqgJpb RHkw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=TcJla6vHBrMnbnVdBdwnihBmtX/jeQXFuP/WhSxe7wA=; b=eexAZSArm8QDErb9BLBOiyj+yJHoY1jJ+KKi19VNQr8K7DjPGyWyMW6S6Tbx+8ut3I VOuH2sxe2SX/nBwwN3NEzkuF8l8j6XqaIHNs0wLAP4RXUPBP/MUbfMWxwboNvD6ZFrFN ME1LdslmFt8KrTC7GZRVuekOmB5qr7Fw2cffjw3ToFcOUPQCJmizp3LnYMAa8daz2yVY 7VPmL+lgyp/wyY5R71SEdrZvH4UuEsEt+e2c2vF78m0Eq85LR0XHRSmqTEdOxgyffxDq i+E0FrGuno7Xtv52AbTDKen4aZ13s3y7Lmqb5SG5mnWeauIeRfZHzaVcmmHNyl6ZiDza xpmA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=o1N9cfoW; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id bg18si1445982ejb.676.2022.01.03.12.26.33; Mon, 03 Jan 2022 12:26:34 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=o1N9cfoW; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 25B7B68A569; Mon, 3 Jan 2022 22:26:30 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-ed1-f45.google.com (mail-ed1-f45.google.com [209.85.208.45]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id CF98568A569 for ; Mon, 3 Jan 2022 22:26:23 +0200 (EET) Received: by mail-ed1-f45.google.com with SMTP id m21so141288698edc.0 for ; Mon, 03 Jan 2022 12:26:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=DGNALI0ypkpshwi8HNhzry9Er0Wvgu32ur/DL9qBHQk=; b=o1N9cfoWsPwaBBr7vW4La/y+Lx4/I6B3zgJ2yt02Mg9jJUi+tilu0jIrUpmWcGhHNq GgCkOEnuq/nNh1xuRaw43yD71PP8J7OqR/6Fw9z1ofGA0lxrWVEK41nfE7fa0GtwO8WU bCjoCvGw372BVzwRPFu/N5A1/QoM9y/cSyp9mlW9hYR94O5yqRR9ukLkh/fOLR52RTxM UJhZnYbD1itSZmglOnlMO5pwN7V1erbdRlnjENjBfY9ucFQdBMlXfkFRFvFR3KOMYN67 wwpEHFhcuTwxqU3bxu6+iYKGsJPiDF8CLPV1YmTUXcOxYJL3mkvUY8MZFLAkeISVN++q oIFw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=DGNALI0ypkpshwi8HNhzry9Er0Wvgu32ur/DL9qBHQk=; b=QbrL2sSiioInC+nC9yUhMl5T7SlH76wn4NkNW/0UuaxCiPI9dD78D4y4uBdtD74xap fWPpxCu7hCD9GpO95D77j75bILaoIYDBgNw6jLSQswaYr+++E3sBOYayYX9agSLBtv3K 4q+OfZ/VrGjJQBrOEr6V+tLh8/uJzPmUXDaCXT9exdAToUGU94oyNGDV8kXDsMFJfN/p XFkIsBBqTzG9gfFtsM4D2LSfl/OFi9BYIVwt6ynNihN+WMrcU7ih4WmOl/gWE0kyWFtB 7WW5E1+5/EIGWFwYlaGiDqJpHJEWuPrD0Woh9efeqOisSI1/sAKz86FITT7Dq5pnHNhi 3yYw== X-Gm-Message-State: AOAM5327aClMVX8us51Olbh9uOFMeSu0PZ8p8kkTJqCcfua6Pmwa2ayD 9lBugE5aUaAh+xdsnFdaOTELrIOuFwU= X-Received: by 2002:a17:906:c156:: with SMTP id dp22mr38557682ejc.109.1641241582628; Mon, 03 Jan 2022 12:26:22 -0800 (PST) Received: from localhost.localdomain (92-64-99-37.biz.kpn.net. [92.64.99.37]) by smtp.googlemail.com with ESMTPSA id e19sm13835915edu.47.2022.01.03.12.26.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 03 Jan 2022 12:26:22 -0800 (PST) From: Martijn van Beurden To: ffmpeg-devel@ffmpeg.org Date: Mon, 3 Jan 2022 21:26:08 +0100 Message-Id: <20220103202608.56364-1-mvanb1@gmail.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: References: MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v3] libavcodec/flacenc: add backward-compatible 32 bit-per-sample capability X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Martijn van Beurden Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: sYLnkOtjax3i Enables creation of FLAC files with up to 32 bits-per-sample, up from the previous limit of 24 bit. This is a feature requested for RAWcooked, the archiving community has a need for storing files with 32-bit integer audio samples. See https://github.com/MediaArea/RAWcooked/issues/356 Restrictions to the encoder are added so created files are compatible with existing decoders. Stereo decorrelation is disabled on 32 bit-per-sample, because a side channel would need 33 bit-per-sample, causing problems in existing 32 bit datapaths. Also only LPC encoding is enabled, because decoders capable of processing 24-bit files already use 64-bit processing for LPC, but not for fixed subframes. Furthermore, predictions and residuals are checked for causing integer overflow, reverting to a verbatim (store) subframe in case no LPC coeffs can be found that do not cause overflow. ffmpeg's FLAC decoder has been forward-compatible with this change since commit c720b9ce98 (May 2015). libFLAC is forward-compatible since release 1.2.1 (September 2007), the flac command line tool however blocks 32-bit files out of caution, it having been untested until now. To create 32 bit files, -bits_per_raw_sample 32 must be specified because of trac ticket 9563 --- libavcodec/flacdsp.c | 45 +++++++++++++++++++++ libavcodec/flacdsp.h | 3 ++ libavcodec/flacenc.c | 94 +++++++++++++++++++++++++++++++++++++------- 3 files changed, 127 insertions(+), 15 deletions(-) diff --git a/libavcodec/flacdsp.c b/libavcodec/flacdsp.c index bc9a5dbed9..b6c163981e 100644 --- a/libavcodec/flacdsp.c +++ b/libavcodec/flacdsp.c @@ -43,6 +43,51 @@ #define PLANAR 1 #include "flacdsp_template.c" +#define ZIGZAG_32BIT_MAX 0x3FFFFFFF +#define ZIGZAG_32BIT_MIN -0x3FFFFFFF + +int ff_flacdsp_lpc_encode_c_32_overflow_detect(int32_t *res, const int32_t *smp, int len, + int order, int32_t *coefs, int shift) +{ + /* This function checks for every prediction and every residual + * whether they cause integer overflow in existing decoders. In + * case the prediction exceeds int32_t limits, prediction + * coefficients are lowered accordingly. If the residual exceeds + * ZIGZAG_32BIT_MAX and _MIN or coefficients have been lowered + * twice but the prediction still overflows, give up */ + int lpc_reduction_tries = 0; + int64_t pmax; + for (int i = 0; i < order; i++) + res[i] = smp[i]; + do { + pmax = 0; + for (int i = order; i < len; i++) { + int64_t p = 0, tmp; + for (int j = 0; j < order; j++) + p += (int64_t)coefs[j]*smp[(i-1)-j]; + p >>= shift; + tmp = smp[i] - p; + if (p > INT32_MAX && p > pmax) + pmax = p; + else if (p < INT32_MIN && (p * -1) > pmax) + pmax = p * -1; + if (tmp > ZIGZAG_32BIT_MAX || tmp < ZIGZAG_32BIT_MIN) + return 0; + res[i] = tmp; + } + + if (pmax > 0) { + if (lpc_reduction_tries >= 2) + return 0; + lpc_reduction_tries++; + for (int i = 0; i < order; i++) + coefs[i] = ((int64_t)coefs[i] * INT32_MAX) / pmax; + } + } while (pmax > 0); + return 1; +} + + static void flac_lpc_16_c(int32_t *decoded, const int coeffs[32], int pred_order, int qlevel, int len) { diff --git a/libavcodec/flacdsp.h b/libavcodec/flacdsp.h index 7bb0dd0e9a..5978a4722a 100644 --- a/libavcodec/flacdsp.h +++ b/libavcodec/flacdsp.h @@ -40,4 +40,7 @@ void ff_flacdsp_init(FLACDSPContext *c, enum AVSampleFormat fmt, int channels, i void ff_flacdsp_init_arm(FLACDSPContext *c, enum AVSampleFormat fmt, int channels, int bps); void ff_flacdsp_init_x86(FLACDSPContext *c, enum AVSampleFormat fmt, int channels, int bps); +int ff_flacdsp_lpc_encode_c_32_overflow_detect(int32_t *res, const int32_t *smp, int len, + int order, int32_t *coefs, int shift); + #endif /* AVCODEC_FLACDSP_H */ diff --git a/libavcodec/flacenc.c b/libavcodec/flacenc.c index 595928927d..bb7138acb3 100644 --- a/libavcodec/flacenc.c +++ b/libavcodec/flacenc.c @@ -254,10 +254,30 @@ static av_cold int flac_encode_init(AVCodecContext *avctx) s->bps_code = 4; break; case AV_SAMPLE_FMT_S32: - if (avctx->bits_per_raw_sample != 24) - av_log(avctx, AV_LOG_WARNING, "encoding as 24 bits-per-sample\n"); - avctx->bits_per_raw_sample = 24; - s->bps_code = 6; + if (avctx->bits_per_raw_sample <= 24) { + if (avctx->bits_per_raw_sample < 24) + av_log(avctx, AV_LOG_WARNING, "encoding as 24 bits-per-sample\n"); + avctx->bits_per_raw_sample = 24; + s->bps_code = 6; + } else { + av_log(avctx, AV_LOG_WARNING, "non-streamable bits-per-sample\n"); + s->bps_code = 0; + if (avctx->bits_per_raw_sample == 0) + avctx->bits_per_raw_sample = 32; + if (s->options.lpc_type != FF_LPC_TYPE_LEVINSON && + s->options.lpc_type != FF_LPC_TYPE_CHOLESKY) { + av_log(avctx, AV_LOG_WARNING, "forcing lpc_type levinson, lpc_type fixed or none not supported with >24 bits-per-sample FLAC\n"); + s->options.lpc_type = FF_LPC_TYPE_LEVINSON; + } + if (avctx->bits_per_raw_sample == 32) { + /* Because stereo decorrelation can raise the bitdepth of + * a subframe to 33 bits, we disable it */ + if (s->options.ch_mode != FLAC_CHMODE_INDEPENDENT) { + av_log(avctx, AV_LOG_WARNING, "disabling stereo decorrelation, not supported with 32 bits-per-sample FLAC\n"); + s->options.ch_mode = FLAC_CHMODE_INDEPENDENT; + } + } + } break; } @@ -686,7 +706,7 @@ static uint64_t calc_rice_params(RiceContext *rc, tmp_rc.coding_mode = rc->coding_mode; - for (i = 0; i < n; i++) + for (i = pred_order; i < n; i++) udata[i] = (2 * data[i]) ^ (data[i] >> 31); calc_sum_top(pmax, exact ? kmax : 0, udata, n, pred_order, sums); @@ -868,7 +888,11 @@ static int encode_residual_ch(FlacEncodeContext *s, int ch) order = av_clip(order, min_order - 1, max_order - 1); if (order == last_order) continue; - if (s->bps_code * 4 + s->options.lpc_coeff_precision + av_log2(order) <= 32) { + if (s->avctx->bits_per_raw_sample > 24) { + if (!ff_flacdsp_lpc_encode_c_32_overflow_detect(res, smp, n, order+1, + coefs[order], shift[order])) + continue; + } else if (s->bps_code * 4 + s->options.lpc_coeff_precision + av_log2(order) <= 32) { s->flac_dsp.lpc16_encode(res, smp, n, order+1, coefs[order], shift[order]); } else { @@ -888,7 +912,11 @@ static int encode_residual_ch(FlacEncodeContext *s, int ch) opt_order = 0; bits[0] = UINT32_MAX; for (i = min_order-1; i < max_order; i++) { - if (s->bps_code * 4 + s->options.lpc_coeff_precision + av_log2(i) <= 32) { + if (s->avctx->bits_per_raw_sample > 24) { + if (!ff_flacdsp_lpc_encode_c_32_overflow_detect(res, smp, n, i+1, + coefs[i], shift[i])) + continue; + } else if (s->bps_code * 4 + s->options.lpc_coeff_precision + av_log2(i) <= 32) { s->flac_dsp.lpc16_encode(res, smp, n, i+1, coefs[i], shift[i]); } else { s->flac_dsp.lpc32_encode(res, smp, n, i+1, coefs[i], shift[i]); @@ -910,7 +938,11 @@ static int encode_residual_ch(FlacEncodeContext *s, int ch) for (i = last-step; i <= last+step; i += step) { if (i < min_order-1 || i >= max_order || bits[i] < UINT32_MAX) continue; - if (s->bps_code * 4 + s->options.lpc_coeff_precision + av_log2(i) <= 32) { + if (s->avctx->bits_per_raw_sample > 24) { + if (!ff_flacdsp_lpc_encode_c_32_overflow_detect(res, smp, n, i+1, + coefs[i], shift[i])) + continue; + } else if (s->bps_code * 4 + s->options.lpc_coeff_precision + av_log2(i) <= 32) { s->flac_dsp.lpc32_encode(res, smp, n, i+1, coefs[i], shift[i]); } else { s->flac_dsp.lpc16_encode(res, smp, n, i+1, coefs[i], shift[i]); @@ -951,7 +983,11 @@ static int encode_residual_ch(FlacEncodeContext *s, int ch) if (diffsum >8) continue; - if (s->bps_code * 4 + s->options.lpc_coeff_precision + av_log2(opt_order - 1) <= 32) { + if (s->avctx->bits_per_raw_sample > 24) { + if (!ff_flacdsp_lpc_encode_c_32_overflow_detect(res, smp, n, opt_order, + lpc_try, shift[opt_order-1])) + continue; + } else if (s->bps_code * 4 + s->options.lpc_coeff_precision + av_log2(opt_order-1) <= 32) { s->flac_dsp.lpc16_encode(res, smp, n, opt_order, lpc_try, shift[opt_order-1]); } else { s->flac_dsp.lpc32_encode(res, smp, n, opt_order, lpc_try, shift[opt_order-1]); @@ -972,7 +1008,16 @@ static int encode_residual_ch(FlacEncodeContext *s, int ch) for (i = 0; i < sub->order; i++) sub->coefs[i] = coefs[sub->order-1][i]; - if (s->bps_code * 4 + s->options.lpc_coeff_precision + av_log2(opt_order) <= 32) { + if (s->avctx->bits_per_raw_sample > 24) { + if (!ff_flacdsp_lpc_encode_c_32_overflow_detect(res, smp, n, sub->order, + sub->coefs, sub->shift)) { + /* No coefs found that do not cause integer overflow, + * so return a verbatim subframe instead */ + sub->type = sub->type_code = FLAC_SUBFRAME_VERBATIM; + memcpy(res, smp, n * sizeof(int32_t)); + return subframe_count_exact(s, sub, 0); + } + } else if (s->bps_code * 4 + s->options.lpc_coeff_precision + av_log2(sub->order) <= 32) { s->flac_dsp.lpc16_encode(res, smp, n, sub->order, sub->coefs, sub->shift); } else { s->flac_dsp.lpc32_encode(res, smp, n, sub->order, sub->coefs, sub->shift); @@ -1227,12 +1272,22 @@ static void write_subframes(FlacEncodeContext *s) if (sub->type == FLAC_SUBFRAME_CONSTANT) { put_sbits(&s->pb, sub->obits, res[0]); } else if (sub->type == FLAC_SUBFRAME_VERBATIM) { - while (res < frame_end) - put_sbits(&s->pb, sub->obits, *res++); + if (sub->obits < 32) { + while (res < frame_end) + put_sbits(&s->pb, sub->obits, *res++); + } else { + while (res < frame_end) + put_bits32(&s->pb, *res++); + } } else { /* warm-up samples */ - for (i = 0; i < sub->order; i++) - put_sbits(&s->pb, sub->obits, *res++); + if (sub->obits < 32) { + for (i = 0; i < sub->order; i++) + put_sbits(&s->pb, sub->obits, *res++); + }else{ + for (i = 0; i < sub->order; i++) + put_bits32(&s->pb, *res++); + } /* LPC coefficients */ if (sub->type == FLAC_SUBFRAME_LPC) { @@ -1305,7 +1360,7 @@ static int update_md5_sum(FlacEncodeContext *s, const void *samples) (const uint16_t *) samples, buf_size / 2); buf = s->md5_buffer; #endif - } else { + } else if (s->avctx->bits_per_raw_sample <= 24) { int i; const int32_t *samples0 = samples; uint8_t *tmp = s->md5_buffer; @@ -1315,6 +1370,15 @@ static int update_md5_sum(FlacEncodeContext *s, const void *samples) AV_WL24(tmp + 3*i, v); } buf = s->md5_buffer; + } else { + /* s->avctx->bits_per_raw_sample <= 32 */ + int i; + const int32_t *samples0 = samples; + uint8_t *tmp = s->md5_buffer; + + for (i = 0; i < s->frame.blocksize * s->channels; i++) + AV_WL32(tmp + 4*i, samples0[i]); + buf = s->md5_buffer; } av_md5_update(s->md5ctx, buf, buf_size);