diff mbox series

[FFmpeg-devel] Add 32 bit-per-sample capability to FLAC encoder

Message ID 20211216194321.18669-1-mvanb1@gmail.com
State Superseded
Headers show
Series [FFmpeg-devel] Add 32 bit-per-sample capability to FLAC encoder | expand

Checks

Context Check Description
andriy/commit_msg_x86 warning The first line of the commit message must start with a context terminated by a colon and a space, for example "lavu/opt: " or "doc: ".
andriy/make_x86 success Make finished
andriy/make_fate_x86 success Make fate finished
andriy/commit_msg_ppc warning The first line of the commit message must start with a context terminated by a colon and a space, for example "lavu/opt: " or "doc: ".
andriy/make_ppc success Make finished
andriy/make_fate_ppc success Make fate finished

Commit Message

Martijn van Beurden Dec. 16, 2021, 7:43 p.m. UTC 
This commit makes ffmpeg create FLAC files with up to 32 bits-per-sample,
up from the previous limit of 24 bit. This is because of a feature request
to RAWcooked, apparently the archiving community has a need for storing
files with 32-bit integer audio samples.

See https://github.com/MediaArea/RAWcooked/issues/356

Care has been taken to create files that are compatible with existing
decoders, which were never tested with such files. Stereo decorrelation
is disabled on 32 bit-per-sample, because a side channel would need
33 bit-per-sample, causing problems in existing decoders using 32 bit
integers. Also, only LPC encoding is enabled, because decoders capable
of processing 24-bit files already use 64-bit processing for LPC, but
not for fixed subframes.

Additionally, on encoding checks are added making certain that no
prediction overflows 32 bit signed integer and no residual is larger
than 2^30 or smaller than -1*2^30, because most decoders (including ffmpeg)
convert the unsigned folded representation to a signed one before dividing
by two (part of the unfolding).

Testing has been done with CDDA upsampled to 32 bit-per-sample files with
SoX, using a hilbert filter to fill the added 16 bits. Also a
32 bit-per-sample file was generated with SoX synth filter.

ffmpeg has been unintentionally forward-compatible with this change since
commit c720b9ce98, committed in May 2015. ffmpeg n2.7 was the first release
containing this commit. libFLAC is forward-compatible since release 1.2.1
(September 2007), the flac command line tool however blocks 32-bit files
out of caution, it having been untested until now.
---
 libavcodec/flacdsp.c | 25 +++++++++++
 libavcodec/flacdsp.h |  3 ++
 libavcodec/flacenc.c | 99 +++++++++++++++++++++++++++++++++++++-------
 3 files changed, 113 insertions(+), 14 deletions(-)

Comments

Paul B Mahol Dec. 16, 2021, 9:09 p.m. UTC | #1 
use wavpack, this is bad patch.
Martijn van Beurden Dec. 17, 2021, 6:13 a.m. UTC | #2 
Could you please explain what makes this a bad patch, so I can try to
improve it?

Wavpack was already suggested, but using FLAC for archival is
preferred because the FLAC standard is currently in the process of
becoming an IETF RFC.

Op do 16 dec. 2021 om 22:09 schreef Paul B Mahol <onemda@gmail.com>:
>
> use wavpack, this is bad patch.
Michael Niedermayer Dec. 17, 2021, 11:34 a.m. UTC | #3 
On Thu, Dec 16, 2021 at 10:09:13PM +0100, Paul B Mahol wrote:
> use wavpack, this is bad patch.

the flac specification supports 32bit
"FLAC supports from 4 to 32 bits per sample"
so theres nothing fundamental wrong with adding 32bit support

thx

[...]
Michael Niedermayer Dec. 17, 2021, 11:43 a.m. UTC | #4 
On Thu, Dec 16, 2021 at 08:43:21PM +0100, Martijn van Beurden wrote:

[...]

> @@ -972,7 +1007,25 @@ static int encode_residual_ch(FlacEncodeContext *s, int ch)
>      for (i = 0; i < sub->order; i++)
>          sub->coefs[i] = coefs[sub->order-1][i];
>  
> -    if (s->bps_code * 4 + s->options.lpc_coeff_precision + av_log2(opt_order) <= 32) {
> +    if (s->avctx->bits_per_raw_sample > 24) {
> +        if (!ff_flacdsp_lpc_encode_c_32_overflow_detect(res, smp, n, sub->order,
> +                                                       sub->coefs, sub->shift)) {
> +            /* The found LPC coefficients produce predictions that overflow
> +             * 32-bit signed integer or produce residuals that do not fall
> +             * between -2^30 and 2^30. First try again with slightly smaller
> +             * coefficients so that the prediction undershoots, if that
> +             * doesn't help return a verbatim subframe instead */
> +            for (i = 0; i < sub->order; i++) {
> +                sub->coefs[i] = sub->coefs[i]*0.98;
                                                 ^^^^
This is ugly, the amount of actual overflow should be known at this point
so no arbitrary downscale should be needed here
                                                 
                                                 

> +                if (!ff_flacdsp_lpc_encode_c_32_overflow_detect(res, smp, n, sub->order,
> +                                                                sub->coefs, sub->shift)) {
> +                    sub->type = sub->type_code = FLAC_SUBFRAME_VERBATIM;
> +                    memcpy(res, smp, n * sizeof(int32_t));
> +                    return subframe_count_exact(s, sub, 0);

How often does this occur ?

thx

[...]
Martijn van Beurden Dec. 17, 2021, 8:08 p.m. UTC | #5 
Op vr 17 dec. 2021 om 12:43 schreef Michael Niedermayer
<michael@niedermayer.cc>:
> > +                sub->coefs[i] = sub->coefs[i]*0.98;
>                                                  ^^^^
> This is ugly, the amount of actual overflow should be known at this point
> so no arbitrary downscale should be needed here


Many thanks for the suggestion, I didn't think of using a more
intelligent approach. I'll work something out.


> > +                if (!ff_flacdsp_lpc_encode_c_32_overflow_detect(res, smp, n, sub->order,
> > +                                                                sub->coefs, sub->shift)) {
> > +                    sub->type = sub->type_code = FLAC_SUBFRAME_VERBATIM;
> > +                    memcpy(res, smp, n * sizeof(int32_t));
> > +                    return subframe_count_exact(s, sub, 0);
>
> How often does this occur ?


Depends on the content. On low dynamic range, distorted, overdriven
sounds like certain kinds of metal, this happens on about half of the
subframes. On most popular music, up to about 5%. On classical music
never. For the intended use (archiving of tape recordings with ample
headroom) probably never as well. A more intelligent approach as
suggested could bring these numbers down.
diff mbox series

Patch

diff --git a/libavcodec/flacdsp.c b/libavcodec/flacdsp.c
index bc9a5dbed9..84d8b9571a 100644
--- a/libavcodec/flacdsp.c
+++ b/libavcodec/flacdsp.c
@@ -43,6 +43,31 @@ 
 #define PLANAR 1
 #include "flacdsp_template.c"
 
+#define ZIGZAG_32BIT_MAX  0x3FFFFFFF
+#define ZIGZAG_32BIT_MIN -0x3FFFFFFF
+
+int ff_flacdsp_lpc_encode_c_32_overflow_detect(int32_t *res, const int32_t *smp, int len,
+                                    int order, const int32_t *coefs, int shift)
+{
+    int i;
+    for (i = 0; i < order; i++)
+        res[i] = smp[i];
+    for (int i = order; i < len; i++) {
+        int64_t p = 0, tmp = 0;
+        for (int j = 0; j < order; j++) {
+            p    += (int64_t)coefs[j]*smp[(i-1)-j];
+        }
+        p >>= shift;
+        tmp = smp[i] - p;
+        if(p > INT32_MAX || p < INT32_MIN ||
+           tmp > ZIGZAG_32BIT_MAX || tmp < ZIGZAG_32BIT_MIN)
+            return 0;
+        res[i] = tmp;
+   }
+   return 1;
+}
+
+
 static void flac_lpc_16_c(int32_t *decoded, const int coeffs[32],
                           int pred_order, int qlevel, int len)
 {
diff --git a/libavcodec/flacdsp.h b/libavcodec/flacdsp.h
index 7bb0dd0e9a..7441e4ca62 100644
--- a/libavcodec/flacdsp.h
+++ b/libavcodec/flacdsp.h
@@ -40,4 +40,7 @@  void ff_flacdsp_init(FLACDSPContext *c, enum AVSampleFormat fmt, int channels, i
 void ff_flacdsp_init_arm(FLACDSPContext *c, enum AVSampleFormat fmt, int channels, int bps);
 void ff_flacdsp_init_x86(FLACDSPContext *c, enum AVSampleFormat fmt, int channels, int bps);
 
+int ff_flacdsp_lpc_encode_c_32_overflow_detect(int32_t *res, const int32_t *smp, int len,
+                                    int order, const int32_t *coefs, int shift);
+
 #endif /* AVCODEC_FLACDSP_H */
diff --git a/libavcodec/flacenc.c b/libavcodec/flacenc.c
index 595928927d..f9c1451771 100644
--- a/libavcodec/flacenc.c
+++ b/libavcodec/flacenc.c
@@ -254,10 +254,29 @@  static av_cold int flac_encode_init(AVCodecContext *avctx)
         s->bps_code                = 4;
         break;
     case AV_SAMPLE_FMT_S32:
-        if (avctx->bits_per_raw_sample != 24)
-            av_log(avctx, AV_LOG_WARNING, "encoding as 24 bits-per-sample\n");
-        avctx->bits_per_raw_sample = 24;
-        s->bps_code                = 6;
+        if (avctx->bits_per_raw_sample > 0 && avctx->bits_per_raw_sample <= 24){
+            if(avctx->bits_per_raw_sample < 24)
+                av_log(avctx, AV_LOG_WARNING, "encoding as 24 bits-per-sample\n");
+            avctx->bits_per_raw_sample = 24;
+            s->bps_code                = 6;
+        } else {
+            av_log(avctx, AV_LOG_WARNING, "non-streamable bits-per-sample\n");
+            s->bps_code = 0;
+            if (avctx->bits_per_raw_sample == 0)
+                avctx->bits_per_raw_sample = 32;
+            if(s->options.lpc_type != FF_LPC_TYPE_LEVINSON){
+                av_log(avctx, AV_LOG_WARNING, "forcing lpc_type levinson, others not supported with >24 bits-per-sample FLAC\n");
+                s->options.lpc_type = FF_LPC_TYPE_LEVINSON;
+            }
+            if (avctx->bits_per_raw_sample == 32){
+                /* Because stereo decorrelation can raise the bitdepth of
+                 * a subframe to 33 bits, we disable it */
+                if(s->options.ch_mode != FLAC_CHMODE_INDEPENDENT){
+                    av_log(avctx, AV_LOG_WARNING, "disabling stereo decorrelation, not supported with 32 bits-per-sample FLAC\n");
+                    s->options.ch_mode = FLAC_CHMODE_INDEPENDENT;
+                }
+            }
+        }
         break;
     }
 
@@ -686,7 +705,7 @@  static uint64_t calc_rice_params(RiceContext *rc,
 
     tmp_rc.coding_mode = rc->coding_mode;
 
-    for (i = 0; i < n; i++)
+    for (i = pred_order; i < n; i++)
         udata[i] = (2 * data[i]) ^ (data[i] >> 31);
 
     calc_sum_top(pmax, exact ? kmax : 0, udata, n, pred_order, sums);
@@ -868,7 +887,11 @@  static int encode_residual_ch(FlacEncodeContext *s, int ch)
             order = av_clip(order, min_order - 1, max_order - 1);
             if (order == last_order)
                 continue;
-            if (s->bps_code * 4 + s->options.lpc_coeff_precision + av_log2(order) <= 32) {
+            if (s->avctx->bits_per_raw_sample > 24) {
+                if(!ff_flacdsp_lpc_encode_c_32_overflow_detect(res, smp, n, order+1,
+                                                               coefs[order], shift[order]))
+                    continue;
+            } else if (s->bps_code * 4 + s->options.lpc_coeff_precision + av_log2(order) <= 32) {
                 s->flac_dsp.lpc16_encode(res, smp, n, order+1, coefs[order],
                                          shift[order]);
             } else {
@@ -888,7 +911,11 @@  static int encode_residual_ch(FlacEncodeContext *s, int ch)
         opt_order = 0;
         bits[0]   = UINT32_MAX;
         for (i = min_order-1; i < max_order; i++) {
-            if (s->bps_code * 4 + s->options.lpc_coeff_precision + av_log2(i) <= 32) {
+            if (s->avctx->bits_per_raw_sample > 24) {
+                if(!ff_flacdsp_lpc_encode_c_32_overflow_detect(res, smp, n, i+1,
+                                                               coefs[i], shift[i]))
+                    continue;
+            } else if (s->bps_code * 4 + s->options.lpc_coeff_precision + av_log2(i) <= 32) {
                 s->flac_dsp.lpc16_encode(res, smp, n, i+1, coefs[i], shift[i]);
             } else {
                 s->flac_dsp.lpc32_encode(res, smp, n, i+1, coefs[i], shift[i]);
@@ -910,7 +937,11 @@  static int encode_residual_ch(FlacEncodeContext *s, int ch)
             for (i = last-step; i <= last+step; i += step) {
                 if (i < min_order-1 || i >= max_order || bits[i] < UINT32_MAX)
                     continue;
-                if (s->bps_code * 4 + s->options.lpc_coeff_precision + av_log2(i) <= 32) {
+                if (s->avctx->bits_per_raw_sample > 24) {
+                    if(!ff_flacdsp_lpc_encode_c_32_overflow_detect(res, smp, n, i+1,
+                                                                   coefs[i], shift[i]))
+                        continue;
+                } else if (s->bps_code * 4 + s->options.lpc_coeff_precision + av_log2(i) <= 32) {
                     s->flac_dsp.lpc32_encode(res, smp, n, i+1, coefs[i], shift[i]);
                 } else {
                     s->flac_dsp.lpc16_encode(res, smp, n, i+1, coefs[i], shift[i]);
@@ -951,7 +982,11 @@  static int encode_residual_ch(FlacEncodeContext *s, int ch)
                 if (diffsum >8)
                     continue;
 
-                if (s->bps_code * 4 + s->options.lpc_coeff_precision + av_log2(opt_order - 1) <= 32) {
+                if (s->avctx->bits_per_raw_sample > 24) {
+                    if(!ff_flacdsp_lpc_encode_c_32_overflow_detect(res, smp, n, opt_order,
+                                                                   lpc_try, shift[opt_order-1]))
+                        continue;
+                } else if (s->bps_code * 4 + s->options.lpc_coeff_precision + av_log2(opt_order-1) <= 32) {
                     s->flac_dsp.lpc16_encode(res, smp, n, opt_order, lpc_try, shift[opt_order-1]);
                 } else {
                     s->flac_dsp.lpc32_encode(res, smp, n, opt_order, lpc_try, shift[opt_order-1]);
@@ -972,7 +1007,25 @@  static int encode_residual_ch(FlacEncodeContext *s, int ch)
     for (i = 0; i < sub->order; i++)
         sub->coefs[i] = coefs[sub->order-1][i];
 
-    if (s->bps_code * 4 + s->options.lpc_coeff_precision + av_log2(opt_order) <= 32) {
+    if (s->avctx->bits_per_raw_sample > 24) {
+        if (!ff_flacdsp_lpc_encode_c_32_overflow_detect(res, smp, n, sub->order,
+                                                       sub->coefs, sub->shift)) {
+            /* The found LPC coefficients produce predictions that overflow
+             * 32-bit signed integer or produce residuals that do not fall
+             * between -2^30 and 2^30. First try again with slightly smaller
+             * coefficients so that the prediction undershoots, if that
+             * doesn't help return a verbatim subframe instead */
+            for (i = 0; i < sub->order; i++) {
+                sub->coefs[i] = sub->coefs[i]*0.98;
+                if (!ff_flacdsp_lpc_encode_c_32_overflow_detect(res, smp, n, sub->order,
+                                                                sub->coefs, sub->shift)) {
+                    sub->type = sub->type_code = FLAC_SUBFRAME_VERBATIM;
+                    memcpy(res, smp, n * sizeof(int32_t));
+                    return subframe_count_exact(s, sub, 0);
+                }
+            }
+        }
+    } else if (s->bps_code * 4 + s->options.lpc_coeff_precision + av_log2(sub->order) <= 32) {
         s->flac_dsp.lpc16_encode(res, smp, n, sub->order, sub->coefs, sub->shift);
     } else {
         s->flac_dsp.lpc32_encode(res, smp, n, sub->order, sub->coefs, sub->shift);
@@ -1226,13 +1279,21 @@  static void write_subframes(FlacEncodeContext *s)
         /* subframe */
         if (sub->type == FLAC_SUBFRAME_CONSTANT) {
             put_sbits(&s->pb, sub->obits, res[0]);
-        } else if (sub->type == FLAC_SUBFRAME_VERBATIM) {
+        } else if (sub->type == FLAC_SUBFRAME_VERBATIM && sub->obits < 32) {
             while (res < frame_end)
                 put_sbits(&s->pb, sub->obits, *res++);
+        } else if (sub->type == FLAC_SUBFRAME_VERBATIM) {
+            while (res < frame_end)
+                put_bits32(&s->pb, *res++);
         } else {
             /* warm-up samples */
-            for (i = 0; i < sub->order; i++)
-                put_sbits(&s->pb, sub->obits, *res++);
+            if(sub->obits < 32){
+                for (i = 0; i < sub->order; i++)
+                    put_sbits(&s->pb, sub->obits, *res++);
+            }else{
+                for (i = 0; i < sub->order; i++)
+                    put_bits32(&s->pb, *res++);
+            }
 
             /* LPC coefficients */
             if (sub->type == FLAC_SUBFRAME_LPC) {
@@ -1305,7 +1366,7 @@  static int update_md5_sum(FlacEncodeContext *s, const void *samples)
                             (const uint16_t *) samples, buf_size / 2);
         buf = s->md5_buffer;
 #endif
-    } else {
+    } else if (s->avctx->bits_per_raw_sample <= 24) {
         int i;
         const int32_t *samples0 = samples;
         uint8_t *tmp            = s->md5_buffer;
@@ -1315,6 +1376,16 @@  static int update_md5_sum(FlacEncodeContext *s, const void *samples)
             AV_WL24(tmp + 3*i, v);
         }
         buf = s->md5_buffer;
+    } else {
+        /* s->avctx->bits_per_raw_sample <= 32 */
+        int i;
+        const int32_t *samples0 = samples;
+        uint8_t *tmp            = s->md5_buffer;
+
+        for (i = 0; i < s->frame.blocksize * s->channels; i++) {
+            AV_WL32(tmp + 4*i, samples0[i]);
+        }
+        buf = s->md5_buffer;
     }
     av_md5_update(s->md5ctx, buf, buf_size);