| Message ID | db9dd520-b6b8-28cb-f0ab-ab881000a526@gmail.com |
|---|---|
| State | New |
| Headers | show |
| Series | [FFmpeg-devel,v5,1/2] libavcodec: Added DFPWM1a codec | expand |
| Context | Check | Description |
|---|---|---|
| yinshiyou/make_loongarch64 | success | Make finished |
| yinshiyou/make_fate_loongarch64 | success | Make fate finished |
On 2/27/22, Jack Bruienne <jackbruienne@gmail.com> wrote: > > From the wiki page (https://wiki.vexatos.com/dfpwm): >> DFPWM (Dynamic Filter Pulse Width Modulation) is an audio codec >> created by Ben “GreaseMonkey” Russell in 2012, originally to be used >> as a voice codec for asiekierka's pixmess, a C remake of 64pixels. >> It is a 1-bit-per-sample codec which uses a dynamic-strength one-pole >> low-pass filter as a predictor. Due to the fact that a raw DPFWM decoding >> creates a high-pitched whine, it is often followed by some >> post-processing >> filters to make the stream more listenable. > > It has recently gained popularity through the ComputerCraft mod for > Minecraft, which added support for audio through this codec, as well as > the Computronics expansion which preceeded the official support. These > both implement the slightly adjusted 1a version of the codec, which is > the version I have chosen for this patch. > > This patch adds a new codec (with encoding and decoding) for DFPWM1a. > The codec sources are pretty simple: they use the reference codec with > a basic wrapper to connect it to the FFmpeg AVCodec system. > > To clarify, the codec does not have a specific sample rate - it is > provided by the container (or user), which is typically 48000, but has > also been known to be 32768. The codec does not specify channel info > either, and it's pretty much always used with one mono channel. > However, since it appears that libavcodec expects both sample rate and > channel count to be handled by either the codec or container, I have > made the decision to allow multiple channels interleaved, which as far > as I know has never been used, but it works fine here nevertheless. The > accompanying raw format has a channels option to set this. (I expect > most users of this will not use multiple channels, but it remains an > option just in case.) > > This patch will be highly useful to ComputerCraft developers who are > working with audio, as it is the standard format for audio, and there > are few user-friendly encoders out there, and even fewer decoders. It > will streamline the process for importing and listening to audio, > replacing the need to write code or use tools that require very > specific input formats. > > You may use the CraftOS-PC program (https://www.craftos-pc.cc) to test > out DFPWM playback. To use it, run the program and type this command: > "attach left speaker" Then run "speaker play <file.dfpwm>" for each file. > The app runs in a sandbox, so files have to be transferred in first; > the easiest way to do this is to simply drag the file on the window. > (Or copy files to the folder at https://www.craftos-pc.cc/docs/saves.) > > Sample DFPWM files can be generated with an online tool at > https://music.madefor.cc. This is the current best way to encode DFPWM > files. Simply drag an audio file onto the page, and it will encode it, > giving a download link on the page. > > I've made sure to update all of the docs as per Developer§7, and I've > tested it as per section 8. Test files encoded to DFPWM play correctly > in ComputerCraft, and other files that work in CC are correctly decoded. > I have also verified that corrupt files do not crash the decoder - this > should theoretically not be an issue as the result size is constant with > respect to the input size. > > Changes since v4: > Fixed missing channel check in decoder. > > Changes since v3: > Added support for multiple interleaved channels, and cleaned up the > code a bunch. > > Changes since v2: > I've found that the reference encoder has a few errors, and sounds > worse than the Java-based implementation that is used most often. I got > in contact with someone who knows DFPWM much better than I do, and I > worked with them to make a few adjustments that should improve the > audio quality. I also made sure that the output matches the Java > codec exactly, so it should have the exact same quality as other codecs. > > Signed-off-by: Jack Bruienne <jackbruienne@gmail.com> > --- > Changelog | 1 + > MAINTAINERS | 1 + > doc/general_contents.texi | 1 + > libavcodec/Makefile | 2 + > libavcodec/allcodecs.c | 2 + > libavcodec/codec_desc.c | 7 ++ > libavcodec/codec_id.h | 1 + > libavcodec/dfpwmdec.c | 133 ++++++++++++++++++++++++++++++++++++++ > libavcodec/dfpwmenc.c | 121 ++++++++++++++++++++++++++++++++++ > libavcodec/utils.c | 2 + > libavcodec/version.h | 2 +- > 11 files changed, 272 insertions(+), 1 deletion(-) > create mode 100644 libavcodec/dfpwmdec.c > create mode 100644 libavcodec/dfpwmenc.c > > Please move channel check to init.
On 3/3/22, Paul B Mahol <onemda@gmail.com> wrote: > On 2/27/22, Jack Bruienne <jackbruienne@gmail.com> wrote: >> >> From the wiki page (https://wiki.vexatos.com/dfpwm): >>> DFPWM (Dynamic Filter Pulse Width Modulation) is an audio codec >>> created by Ben “GreaseMonkey” Russell in 2012, originally to be used >>> as a voice codec for asiekierka's pixmess, a C remake of 64pixels. >>> It is a 1-bit-per-sample codec which uses a dynamic-strength one-pole >>> low-pass filter as a predictor. Due to the fact that a raw DPFWM >>> decoding >>> creates a high-pitched whine, it is often followed by some >>> post-processing >>> filters to make the stream more listenable. >> >> It has recently gained popularity through the ComputerCraft mod for >> Minecraft, which added support for audio through this codec, as well as >> the Computronics expansion which preceeded the official support. These >> both implement the slightly adjusted 1a version of the codec, which is >> the version I have chosen for this patch. >> >> This patch adds a new codec (with encoding and decoding) for DFPWM1a. >> The codec sources are pretty simple: they use the reference codec with >> a basic wrapper to connect it to the FFmpeg AVCodec system. >> >> To clarify, the codec does not have a specific sample rate - it is >> provided by the container (or user), which is typically 48000, but has >> also been known to be 32768. The codec does not specify channel info >> either, and it's pretty much always used with one mono channel. >> However, since it appears that libavcodec expects both sample rate and >> channel count to be handled by either the codec or container, I have >> made the decision to allow multiple channels interleaved, which as far >> as I know has never been used, but it works fine here nevertheless. The >> accompanying raw format has a channels option to set this. (I expect >> most users of this will not use multiple channels, but it remains an >> option just in case.) >> >> This patch will be highly useful to ComputerCraft developers who are >> working with audio, as it is the standard format for audio, and there >> are few user-friendly encoders out there, and even fewer decoders. It >> will streamline the process for importing and listening to audio, >> replacing the need to write code or use tools that require very >> specific input formats. >> >> You may use the CraftOS-PC program (https://www.craftos-pc.cc) to test >> out DFPWM playback. To use it, run the program and type this command: >> "attach left speaker" Then run "speaker play <file.dfpwm>" for each file. >> The app runs in a sandbox, so files have to be transferred in first; >> the easiest way to do this is to simply drag the file on the window. >> (Or copy files to the folder at https://www.craftos-pc.cc/docs/saves.) >> >> Sample DFPWM files can be generated with an online tool at >> https://music.madefor.cc. This is the current best way to encode DFPWM >> files. Simply drag an audio file onto the page, and it will encode it, >> giving a download link on the page. >> >> I've made sure to update all of the docs as per Developer§7, and I've >> tested it as per section 8. Test files encoded to DFPWM play correctly >> in ComputerCraft, and other files that work in CC are correctly decoded. >> I have also verified that corrupt files do not crash the decoder - this >> should theoretically not be an issue as the result size is constant with >> respect to the input size. >> >> Changes since v4: >> Fixed missing channel check in decoder. >> >> Changes since v3: >> Added support for multiple interleaved channels, and cleaned up the >> code a bunch. >> >> Changes since v2: >> I've found that the reference encoder has a few errors, and sounds >> worse than the Java-based implementation that is used most often. I got >> in contact with someone who knows DFPWM much better than I do, and I >> worked with them to make a few adjustments that should improve the >> audio quality. I also made sure that the output matches the Java >> codec exactly, so it should have the exact same quality as other codecs. >> >> Signed-off-by: Jack Bruienne <jackbruienne@gmail.com> >> --- >> Changelog | 1 + >> MAINTAINERS | 1 + >> doc/general_contents.texi | 1 + >> libavcodec/Makefile | 2 + >> libavcodec/allcodecs.c | 2 + >> libavcodec/codec_desc.c | 7 ++ >> libavcodec/codec_id.h | 1 + >> libavcodec/dfpwmdec.c | 133 ++++++++++++++++++++++++++++++++++++++ >> libavcodec/dfpwmenc.c | 121 ++++++++++++++++++++++++++++++++++ >> libavcodec/utils.c | 2 + >> libavcodec/version.h | 2 +- >> 11 files changed, 272 insertions(+), 1 deletion(-) >> create mode 100644 libavcodec/dfpwmdec.c >> create mode 100644 libavcodec/dfpwmenc.c >> >> > > > Please move channel check to init. > Also check for possible overflows when multiplying with 8 the packet->size.
diff --git a/Changelog b/Changelog index 5ad2cef..5170a6a 100644 --- a/Changelog +++ b/Changelog @@ -4,6 +4,7 @@ releases are sorted from youngest to oldest. version 5.1: - dialogue enhance audio filter - dropped obsolete XvMC hwaccel +- DFPWM audio encoder/decoder version 5.0: diff --git a/MAINTAINERS b/MAINTAINERS index f33ccbd..57b6f33 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -161,6 +161,7 @@ Codecs: cscd.c Reimar Doeffinger cuviddec.c Timo Rothenpieler dca* foo86 + dfpwm* Jack Bruienne dirac* Rostislav Pehlivanov dnxhd* Baptiste Coudurier dolby_e* foo86 diff --git a/doc/general_contents.texi b/doc/general_contents.texi index df1692c..14aeaed 100644 --- a/doc/general_contents.texi +++ b/doc/general_contents.texi @@ -1194,6 +1194,7 @@ following image formats are supported: @item CRI HCA @tab @tab X @item Delphine Software International CIN audio @tab @tab X @tab Codec used in Delphine Software International games. +@item DFPWM @tab X @tab X @item Digital Speech Standard - Standard Play mode (DSS SP) @tab @tab X @item Discworld II BMV Audio @tab @tab X @item COOK @tab @tab X diff --git a/libavcodec/Makefile b/libavcodec/Makefile index 6076b4a..7474220 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -289,6 +289,8 @@ OBJS-$(CONFIG_DERF_DPCM_DECODER) += dpcm.o OBJS-$(CONFIG_DIRAC_DECODER) += diracdec.o dirac.o diracdsp.o diractab.o \ dirac_arith.o dirac_dwt.o dirac_vlc.o OBJS-$(CONFIG_DFA_DECODER) += dfa.o +OBJS-$(CONFIG_DFPWM_DECODER) += dfpwmdec.o +OBJS-$(CONFIG_DFPWM_ENCODER) += dfpwmenc.o OBJS-$(CONFIG_DNXHD_DECODER) += dnxhddec.o dnxhddata.o OBJS-$(CONFIG_DNXHD_ENCODER) += dnxhdenc.o dnxhddata.o OBJS-$(CONFIG_DOLBY_E_DECODER) += dolby_e.o dolby_e_parse.o kbdwin.o diff --git a/libavcodec/allcodecs.c b/libavcodec/allcodecs.c index d1e1019..c3a0c26 100644 --- a/libavcodec/allcodecs.c +++ b/libavcodec/allcodecs.c @@ -437,6 +437,8 @@ extern const AVCodec ff_bmv_audio_decoder; extern const AVCodec ff_cook_decoder; extern const AVCodec ff_dca_encoder; extern const AVCodec ff_dca_decoder; +extern const AVCodec ff_dfpwm_encoder; +extern const AVCodec ff_dfpwm_decoder; extern const AVCodec ff_dolby_e_decoder; extern const AVCodec ff_dsd_lsbf_decoder; extern const AVCodec ff_dsd_msbf_decoder; diff --git a/libavcodec/codec_desc.c b/libavcodec/codec_desc.c index 725c687..81f3b3c 100644 --- a/libavcodec/codec_desc.c +++ b/libavcodec/codec_desc.c @@ -3237,6 +3237,13 @@ static const AVCodecDescriptor codec_descriptors[] = { .long_name = NULL_IF_CONFIG_SMALL("MSN Siren"), .props = AV_CODEC_PROP_INTRA_ONLY | AV_CODEC_PROP_LOSSY, }, + { + .id = AV_CODEC_ID_DFPWM, + .type = AVMEDIA_TYPE_AUDIO, + .name = "dfpwm", + .long_name = NULL_IF_CONFIG_SMALL("DFPWM (Dynamic Filter Pulse Width Modulation)"), + .props = AV_CODEC_PROP_LOSSY, + }, /* subtitle codecs */ { diff --git a/libavcodec/codec_id.h b/libavcodec/codec_id.h index ab265ec..3ffb9bd 100644 --- a/libavcodec/codec_id.h +++ b/libavcodec/codec_id.h @@ -516,6 +516,7 @@ enum AVCodecID { AV_CODEC_ID_HCA, AV_CODEC_ID_FASTAUDIO, AV_CODEC_ID_MSNSIREN, + AV_CODEC_ID_DFPWM, /* subtitle codecs */ AV_CODEC_ID_FIRST_SUBTITLE = 0x17000, ///< A dummy ID pointing at the start of subtitle codecs. diff --git a/libavcodec/dfpwmdec.c b/libavcodec/dfpwmdec.c new file mode 100644 index 0000000..7946428 --- /dev/null +++ b/libavcodec/dfpwmdec.c @@ -0,0 +1,133 @@ +/* + * DFPWM decoder + * Copyright (c) 2022 Jack Bruienne + * Copyright (c) 2012, 2016 Ben "GreaseMonkey" Russell + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * DFPWM1a decoder + */ + +#include "libavutil/internal.h" +#include "avcodec.h" +#include "codec_id.h" +#include "internal.h" + +typedef struct { + int fq, q, s, lt; +} DFPWMState; + +// DFPWM codec from https://github.com/ChenThread/dfpwm/blob/master/1a/ +// Licensed in the public domain + +static void au_decompress(DFPWMState *state, int fs, int len, uint8_t *outbuf, uint8_t *inbuf) +{ + unsigned d; + for (int i = 0; i < len; i++) { + // get bits + d = *(inbuf++); + for (int j = 0; j < 8; j++) { + int nq, lq, st, ns, ov; + // set target + int t = ((d&1) ? 127 : -128); + d >>= 1; + + // adjust charge + nq = state->q + ((state->s * (t-state->q) + 512)>>10); + if(nq == state->q && nq != t) + nq += (t == 127 ? 1 : -1); + lq = state->q; + state->q = nq; + + // adjust strength + st = (t != state->lt ? 0 : 1023); + ns = state->s; + if(ns != st) + ns += (st != 0 ? 1 : -1); + if(ns < 8) ns = 8; + state->s = ns; + + // FILTER: perform antijerk + ov = (t != state->lt ? (nq+lq+1)>>1 : nq); + + // FILTER: perform LPF + state->fq += ((fs*(ov-state->fq) + 0x80)>>8); + ov = state->fq; + + // output sample + *(outbuf++) = ov + 128; + + state->lt = t; + } + } +} + +static av_cold int dfpwm_dec_init(struct AVCodecContext *ctx) +{ + DFPWMState *state = ctx->priv_data; + + state->fq = 0; + state->q = 0; + state->s = 0; + state->lt = -128; + + ctx->sample_fmt = AV_SAMPLE_FMT_U8; + ctx->bits_per_raw_sample = 8; + + return 0; +} + +static int dfpwm_dec_frame(struct AVCodecContext *ctx, void *data, + int *got_frame, struct AVPacket *packet) +{ + DFPWMState *state = ctx->priv_data; + AVFrame *frame = data; + int ret; + + if (ctx->channels <= 0) { + av_log(ctx, AV_LOG_ERROR, "Invalid number of channels\n"); + *got_frame = 0; + return AVERROR(EINVAL); + } + + frame->nb_samples = packet->size * 8 / ctx->channels; + ret = ff_get_buffer(ctx, frame, 0); + if (ret) { + *got_frame = 0; + return ret; + } + + au_decompress(state, 140, packet->size, frame->data[0], packet->data); + + *got_frame = 1; + return packet->size; +} + +const AVCodec ff_dfpwm_decoder = { + .name = "dfpwm", + .long_name = NULL_IF_CONFIG_SMALL("DFPWM1a audio"), + .type = AVMEDIA_TYPE_AUDIO, + .id = AV_CODEC_ID_DFPWM, + .priv_data_size = sizeof(DFPWMState), + .init = dfpwm_dec_init, + .decode = dfpwm_dec_frame, + .capabilities = AV_CODEC_CAP_DR1, + .caps_internal = FF_CODEC_CAP_INIT_THREADSAFE, +}; diff --git a/libavcodec/dfpwmenc.c b/libavcodec/dfpwmenc.c new file mode 100644 index 0000000..02f2e64 --- /dev/null +++ b/libavcodec/dfpwmenc.c @@ -0,0 +1,121 @@ +/* + * DFPWM encoder + * Copyright (c) 2022 Jack Bruienne + * Copyright (c) 2012, 2016 Ben "GreaseMonkey" Russell + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * DFPWM1a encoder + */ + +#include "libavutil/internal.h" +#include "avcodec.h" +#include "codec_id.h" +#include "encode.h" +#include "internal.h" + +typedef struct { + int fq, q, s, lt; +} DFPWMState; + +// DFPWM codec from https://github.com/ChenThread/dfpwm/blob/master/1a/ +// Licensed in the public domain + +// note, len denotes how many compressed bytes there are (uncompressed bytes / 8). +static void au_compress(DFPWMState *state, int len, uint8_t *outbuf, uint8_t *inbuf) +{ + unsigned d = 0; + for (int i = 0; i < len; i++) { + for (int j = 0; j < 8; j++) { + int nq, st, ns; + // get sample + int v = *(inbuf++) - 128; + // set bit / target + int t = (v > state->q || (v == state->q && v == 127) ? 127 : -128); + d >>= 1; + if(t > 0) + d |= 0x80; + + // adjust charge + nq = state->q + ((state->s * (t-state->q) + 512)>>10); + if(nq == state->q && nq != t) + nq += (t == 127 ? 1 : -1); + state->q = nq; + + // adjust strength + st = (t != state->lt ? 0 : 1023); + ns = state->s; + if(ns != st) + ns += (st != 0 ? 1 : -1); + if(ns < 8) ns = 8; + state->s = ns; + + state->lt = t; + } + + // output bits + *(outbuf++) = d; + } +} + +static av_cold int dfpwm_enc_init(struct AVCodecContext *ctx) +{ + DFPWMState *state = ctx->priv_data; + + state->fq = 0; + state->q = 0; + state->s = 0; + state->lt = -128; + + ctx->bits_per_coded_sample = 1; + + return 0; +} + +static int dfpwm_enc_frame(struct AVCodecContext *ctx, struct AVPacket *packet, + const struct AVFrame *frame, int *got_packet) +{ + DFPWMState *state = ctx->priv_data; + int size = frame->nb_samples * frame->channels / 8 + (frame->nb_samples % 8 > 0 ? 1 : 0); + int ret = ff_get_encode_buffer(ctx, packet, size, 0); + + if (ret) { + *got_packet = 0; + return ret; + } + + au_compress(state, size, packet->data, frame->data[0]); + + *got_packet = 1; + return 0; +} + +const AVCodec ff_dfpwm_encoder = { + .name = "dfpwm", + .long_name = NULL_IF_CONFIG_SMALL("DFPWM1a audio"), + .type = AVMEDIA_TYPE_AUDIO, + .id = AV_CODEC_ID_DFPWM, + .priv_data_size = sizeof(DFPWMState), + .init = dfpwm_enc_init, + .encode2 = dfpwm_enc_frame, + .sample_fmts = (const enum AVSampleFormat[]){AV_SAMPLE_FMT_U8, AV_SAMPLE_FMT_NONE}, + .capabilities = AV_CODEC_CAP_DR1 | AV_CODEC_CAP_VARIABLE_FRAME_SIZE, + .caps_internal = FF_CODEC_CAP_INIT_THREADSAFE, +}; diff --git a/libavcodec/utils.c b/libavcodec/utils.c index 6f9d90a..066da76 100644 --- a/libavcodec/utils.c +++ b/libavcodec/utils.c @@ -577,6 +577,8 @@ enum AVCodecID av_get_pcm_codec(enum AVSampleFormat fmt, int be) int av_get_bits_per_sample(enum AVCodecID codec_id) { switch (codec_id) { + case AV_CODEC_ID_DFPWM: + return 1; case AV_CODEC_ID_ADPCM_SBPRO_2: return 2; case AV_CODEC_ID_ADPCM_SBPRO_3: diff --git a/libavcodec/version.h b/libavcodec/version.h index d900503..84f3979 100644 --- a/libavcodec/version.h +++ b/libavcodec/version.h @@ -28,7 +28,7 @@ #include "libavutil/version.h" #define LIBAVCODEC_VERSION_MAJOR 59 -#define LIBAVCODEC_VERSION_MINOR 21 +#define LIBAVCODEC_VERSION_MINOR 22 #define LIBAVCODEC_VERSION_MICRO 100 #define LIBAVCODEC_VERSION_INT AV_VERSION_INT(LIBAVCODEC_VERSION_MAJOR, \