| Message ID | 0692ba87-361f-498a-dc08-85d771f6bdaf@gmail.com |
|---|---|
| State | New |
| Headers | show |
| Series | [FFmpeg-devel,v6,1/2] libavcodec: Added DFPWM1a codec | expand |
| Context | Check | Description |
|---|---|---|
| yinshiyou/make_loongarch64 | success | Make finished |
| yinshiyou/make_fate_loongarch64 | success | Make fate finished |
tor 2022-03-03 klockan 10:44 -0500 skrev Jack Bruienne: > > From the wiki page (https://wiki.vexatos.com/dfpwm): > > DFPWM (Dynamic Filter Pulse Width Modulation) is an audio codec > > created by Ben “GreaseMonkey” Russell in 2012, originally to be > > used > > as a voice codec for asiekierka's pixmess, a C remake of 64pixels. > > It is a 1-bit-per-sample codec which uses a dynamic-strength one- > > pole > > low-pass filter as a predictor. Due to the fact that a raw DPFWM > > decoding > > creates a high-pitched whine, it is often followed by some post- > > processing > > filters to make the stream more listenable. This sounds similar to something I wrote for the Atari 2600 a number of years ago ( https://www.pouet.net/prod.php?which=59283 ) I found an encoder online for DFPWM, and it seems to suffer from the same "beeping" that my debeeping hack fixes (suppressing 0xAA and 0x55 bytes). Perhaps a similar hack could be useful in dfpwmenc.c > > It has recently gained popularity through the ComputerCraft mod for > Minecraft, which added support for audio through this codec, as well > as > the Computronics expansion which preceeded the official support. > These > both implement the slightly adjusted 1a version of the codec, which > is > the version I have chosen for this patch. > > This patch adds a new codec (with encoding and decoding) for DFPWM1a. > The codec sources are pretty simple: they use the reference codec > with > a basic wrapper to connect it to the FFmpeg AVCodec system. > > To clarify, the codec does not have a specific sample rate - it is > provided by the container (or user), which is typically 48000, but > has > also been known to be 32768. The codec does not specify channel info > either, and it's pretty much always used with one mono channel. > However, since it appears that libavcodec expects both sample rate > and > channel count to be handled by either the codec or container, I have > made the decision to allow multiple channels interleaved, which as > far > as I know has never been used, but it works fine here nevertheless. From experience it's usually better to be strict when it comes to stuff like this. The ComputerCraft people should work out a standard for this, preferably a container. We've had a similar problem in FreeDV where which codec was being used was implicit most of the time, which has been resolved with the .c2 format. > --- > Changelog | 1 + > MAINTAINERS | 1 + > doc/general_contents.texi | 1 + > libavcodec/Makefile | 2 + > libavcodec/allcodecs.c | 2 + > libavcodec/codec_desc.c | 7 ++ > libavcodec/codec_id.h | 1 + > libavcodec/dfpwmdec.c | 134 > ++++++++++++++++++++++++++++++++++++++ > libavcodec/dfpwmenc.c | 121 ++++++++++++++++++++++++++++++++++ > libavcodec/utils.c | 2 + > libavcodec/version.h | 4 +- > 11 files changed, 274 insertions(+), 2 deletions(-) > create mode 100644 libavcodec/dfpwmdec.c > create mode 100644 libavcodec/dfpwmenc.c Patch doesn't apply on current master (e645a1d) /Tomas
On 3/7/22 06:03, Tomas Härdin wrote: > tor 2022-03-03 klockan 10:44 -0500 skrev Jack Bruienne: >> From the wiki page (https://wiki.vexatos.com/dfpwm): >>> DFPWM (Dynamic Filter Pulse Width Modulation) is an audio codec >>> created by Ben “GreaseMonkey” Russell in 2012, originally to be >>> used >>> as a voice codec for asiekierka's pixmess, a C remake of 64pixels. >>> It is a 1-bit-per-sample codec which uses a dynamic-strength one- >>> pole >>> low-pass filter as a predictor. Due to the fact that a raw DPFWM >>> decoding >>> creates a high-pitched whine, it is often followed by some post- >>> processing >>> filters to make the stream more listenable. > This sounds similar to something I wrote for the Atari 2600 a number of > years ago (https://www.pouet.net/prod.php?which=59283 ) > > I found an encoder online for DFPWM, and it seems to suffer from the > same "beeping" that my debeeping hack fixes (suppressing 0xAA and 0x55 > bytes). Perhaps a similar hack could be useful in dfpwmenc.c I'm curious how this works. Do you just cut out those bytes from the encoder output, or is it modified in some way? Wouldn't removing the data entirely eventually cause much of the audio to be lost? >> It has recently gained popularity through the ComputerCraft mod for >> Minecraft, which added support for audio through this codec, as well >> as >> the Computronics expansion which preceeded the official support. >> These >> both implement the slightly adjusted 1a version of the codec, which >> is >> the version I have chosen for this patch. >> >> This patch adds a new codec (with encoding and decoding) for DFPWM1a. >> The codec sources are pretty simple: they use the reference codec >> with >> a basic wrapper to connect it to the FFmpeg AVCodec system. >> >> To clarify, the codec does not have a specific sample rate - it is >> provided by the container (or user), which is typically 48000, but >> has >> also been known to be 32768. The codec does not specify channel info >> either, and it's pretty much always used with one mono channel. >> However, since it appears that libavcodec expects both sample rate >> and >> channel count to be handled by either the codec or container, I have >> made the decision to allow multiple channels interleaved, which as >> far >> as I know has never been used, but it works fine here nevertheless. > From experience it's usually better to be strict when it comes to stuff > like this. The ComputerCraft people should work out a standard for > this, preferably a container. We've had a similar problem in FreeDV > where which codec was being used was implicit most of the time, which > has been resolved with the .c2 format. I think the best standardized container for DFPWM will be WAV. I'll have to figure out the best place to properly standardize this, but one first step may be to implement this in CC directly. Unfortunately, ComputerCraft is currently without a maintainer - the previous one stepped down in December, and no changes are being made outside of critical bugfixes. I do hope to pick up development in the future, and I already run a reimplementation of the mod outside Minecraft, but as it stands today, getting this to be picked up by users will be quite difficult. However, I do also maintain my own audio processing library AUKit, which now includes support for DFPWM in WAV. Since CC only has a basic raw DFPWM player built-in (which strictly requires 48kHz mono audio), many users turn to it to play more complex formats like WAV with a different sample rate, FLAC, etc. This may be a decent start, but it's far from having full support built-in. I'll see if I can get in contact with the owner of the DFPWM wiki page. This will be especially important if this patch gets merged, as people likely won't see that you can use FFmpeg on those pages. I am able to submit pull requests to the ComputerCraft documentation site, however, so I'm able to get changes made there much easier. In the meantime, I've made a sort-of RFC for the format:https://gist.github.com/MCJack123/90c24b64c8e626c7f130b57e9800962c >> --- >> Changelog | 1 + >> MAINTAINERS | 1 + >> doc/general_contents.texi | 1 + >> libavcodec/Makefile | 2 + >> libavcodec/allcodecs.c | 2 + >> libavcodec/codec_desc.c | 7 ++ >> libavcodec/codec_id.h | 1 + >> libavcodec/dfpwmdec.c | 134 >> ++++++++++++++++++++++++++++++++++++++ >> libavcodec/dfpwmenc.c | 121 ++++++++++++++++++++++++++++++++++ >> libavcodec/utils.c | 2 + >> libavcodec/version.h | 4 +- >> 11 files changed, 274 insertions(+), 2 deletions(-) >> create mode 100644 libavcodec/dfpwmdec.c >> create mode 100644 libavcodec/dfpwmenc.c > Patch doesn't apply on current master (e645a1d) I'll update the patch to apply to the current master, but I feel like by the time this next gets reviewed master may break again. Should I just keep updating the patch to master and resubmitting whenever I notice a new commit breaks it? > /Tomas > > _______________________________________________ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".
mån 2022-03-07 klockan 22:04 -0500 skrev Jack Bruienne: > On 3/7/22 06:03, Tomas Härdin wrote: > > > tor 2022-03-03 klockan 10:44 -0500 skrev Jack Bruienne: > > > From the wiki page (https://wiki.vexatos.com/dfpwm): > > > > DFPWM (Dynamic Filter Pulse Width Modulation) is an audio codec > > > > created by Ben “GreaseMonkey” Russell in 2012, originally to be > > > > used > > > > as a voice codec for asiekierka's pixmess, a C remake of > > > > 64pixels. > > > > It is a 1-bit-per-sample codec which uses a dynamic-strength > > > > one- > > > > pole > > > > low-pass filter as a predictor. Due to the fact that a raw > > > > DPFWM > > > > decoding > > > > creates a high-pitched whine, it is often followed by some > > > > post- > > > > processing > > > > filters to make the stream more listenable. > > This sounds similar to something I wrote for the Atari 2600 a > > number of > > years ago (https://www.pouet.net/prod.php?which=59283 ) > > > > I found an encoder online for DFPWM, and it seems to suffer from > > the > > same "beeping" that my debeeping hack fixes (suppressing 0xAA and > > 0x55 > > bytes). Perhaps a similar hack could be useful in dfpwmenc.c > > I'm curious how this works. Do you just cut out those bytes from the > encoder output, or is it modified in some way? Wouldn't removing the > data entirely eventually cause much of the audio to be lost? The source code is included in the release. Look at audioquant.cpp. I've attached it for convenience. The codec is based on a state machine where each state is a 5-bit PCM value that can go to either of two states, also 5-bit PCM values. Hence 1 bit per sample. I also have a low-pass filter in the decoder. I penalize state machines which result in 0x55 and 0xAA being overly represented. This is done via computing a histogram of the output bytes and scaling the RMS error according to how many of those bytes are in the (tentative) output. Another approach could be to detect and blank excessive runs of 0x55 and 0xAA bytes. > > From experience it's usually better to be strict when it comes to > > stuff > > like this. The ComputerCraft people should work out a standard for > > this, preferably a container. We've had a similar problem in FreeDV > > where which codec was being used was implicit most of the time, > > which > > has been resolved with the .c2 format. > > I think the best standardized container for DFPWM will be WAV. I agree, and I see this was already pushed. /Tomas
Could we please have tests for all new decoders/demuxers?
diff --git a/Changelog b/Changelog index 5ad2cef..5170a6a 100644 --- a/Changelog +++ b/Changelog @@ -4,6 +4,7 @@ releases are sorted from youngest to oldest. version 5.1: - dialogue enhance audio filter - dropped obsolete XvMC hwaccel +- DFPWM audio encoder/decoder version 5.0: diff --git a/MAINTAINERS b/MAINTAINERS index f33ccbd..57b6f33 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -161,6 +161,7 @@ Codecs: cscd.c Reimar Doeffinger cuviddec.c Timo Rothenpieler dca* foo86 + dfpwm* Jack Bruienne dirac* Rostislav Pehlivanov dnxhd* Baptiste Coudurier dolby_e* foo86 diff --git a/doc/general_contents.texi b/doc/general_contents.texi index df1692c..14aeaed 100644 --- a/doc/general_contents.texi +++ b/doc/general_contents.texi @@ -1194,6 +1194,7 @@ following image formats are supported: @item CRI HCA @tab @tab X @item Delphine Software International CIN audio @tab @tab X @tab Codec used in Delphine Software International games. +@item DFPWM @tab X @tab X @item Digital Speech Standard - Standard Play mode (DSS SP) @tab @tab X @item Discworld II BMV Audio @tab @tab X @item COOK @tab @tab X diff --git a/libavcodec/Makefile b/libavcodec/Makefile index 24effa8..01cb4b2 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -289,6 +289,8 @@ OBJS-$(CONFIG_DERF_DPCM_DECODER) += dpcm.o OBJS-$(CONFIG_DIRAC_DECODER) += diracdec.o dirac.o diracdsp.o diractab.o \ dirac_arith.o dirac_dwt.o dirac_vlc.o OBJS-$(CONFIG_DFA_DECODER) += dfa.o +OBJS-$(CONFIG_DFPWM_DECODER) += dfpwmdec.o +OBJS-$(CONFIG_DFPWM_ENCODER) += dfpwmenc.o OBJS-$(CONFIG_DNXHD_DECODER) += dnxhddec.o dnxhddata.o OBJS-$(CONFIG_DNXHD_ENCODER) += dnxhdenc.o dnxhddata.o OBJS-$(CONFIG_DOLBY_E_DECODER) += dolby_e.o dolby_e_parse.o kbdwin.o diff --git a/libavcodec/allcodecs.c b/libavcodec/allcodecs.c index d1e1019..c3a0c26 100644 --- a/libavcodec/allcodecs.c +++ b/libavcodec/allcodecs.c @@ -437,6 +437,8 @@ extern const AVCodec ff_bmv_audio_decoder; extern const AVCodec ff_cook_decoder; extern const AVCodec ff_dca_encoder; extern const AVCodec ff_dca_decoder; +extern const AVCodec ff_dfpwm_encoder; +extern const AVCodec ff_dfpwm_decoder; extern const AVCodec ff_dolby_e_decoder; extern const AVCodec ff_dsd_lsbf_decoder; extern const AVCodec ff_dsd_msbf_decoder; diff --git a/libavcodec/codec_desc.c b/libavcodec/codec_desc.c index 725c687..81f3b3c 100644 --- a/libavcodec/codec_desc.c +++ b/libavcodec/codec_desc.c @@ -3237,6 +3237,13 @@ static const AVCodecDescriptor codec_descriptors[] = { .long_name = NULL_IF_CONFIG_SMALL("MSN Siren"), .props = AV_CODEC_PROP_INTRA_ONLY | AV_CODEC_PROP_LOSSY, }, + { + .id = AV_CODEC_ID_DFPWM, + .type = AVMEDIA_TYPE_AUDIO, + .name = "dfpwm", + .long_name = NULL_IF_CONFIG_SMALL("DFPWM (Dynamic Filter Pulse Width Modulation)"), + .props = AV_CODEC_PROP_LOSSY, + }, /* subtitle codecs */ { diff --git a/libavcodec/codec_id.h b/libavcodec/codec_id.h index ab265ec..3ffb9bd 100644 --- a/libavcodec/codec_id.h +++ b/libavcodec/codec_id.h @@ -516,6 +516,7 @@ enum AVCodecID { AV_CODEC_ID_HCA, AV_CODEC_ID_FASTAUDIO, AV_CODEC_ID_MSNSIREN, + AV_CODEC_ID_DFPWM, /* subtitle codecs */ AV_CODEC_ID_FIRST_SUBTITLE = 0x17000, ///< A dummy ID pointing at the start of subtitle codecs. diff --git a/libavcodec/dfpwmdec.c b/libavcodec/dfpwmdec.c new file mode 100644 index 0000000..05f7944 --- /dev/null +++ b/libavcodec/dfpwmdec.c @@ -0,0 +1,134 @@ +/* + * DFPWM decoder + * Copyright (c) 2022 Jack Bruienne + * Copyright (c) 2012, 2016 Ben "GreaseMonkey" Russell + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * DFPWM1a decoder + */ + +#include "libavutil/internal.h" +#include "avcodec.h" +#include "codec_id.h" +#include "internal.h" + +typedef struct { + int fq, q, s, lt; +} DFPWMState; + +// DFPWM codec from https://github.com/ChenThread/dfpwm/blob/master/1a/ +// Licensed in the public domain + +static void au_decompress(DFPWMState *state, int fs, int len, uint8_t *outbuf, uint8_t *inbuf) +{ + unsigned d; + for (int i = 0; i < len; i++) { + // get bits + d = *(inbuf++); + for (int j = 0; j < 8; j++) { + int nq, lq, st, ns, ov; + // set target + int t = ((d&1) ? 127 : -128); + d >>= 1; + + // adjust charge + nq = state->q + ((state->s * (t-state->q) + 512)>>10); + if(nq == state->q && nq != t) + nq += (t == 127 ? 1 : -1); + lq = state->q; + state->q = nq; + + // adjust strength + st = (t != state->lt ? 0 : 1023); + ns = state->s; + if(ns != st) + ns += (st != 0 ? 1 : -1); + if(ns < 8) ns = 8; + state->s = ns; + + // FILTER: perform antijerk + ov = (t != state->lt ? (nq+lq+1)>>1 : nq); + + // FILTER: perform LPF + state->fq += ((fs*(ov-state->fq) + 0x80)>>8); + ov = state->fq; + + // output sample + *(outbuf++) = ov + 128; + + state->lt = t; + } + } +} + +static av_cold int dfpwm_dec_init(struct AVCodecContext *ctx) +{ + DFPWMState *state = ctx->priv_data; + + if (ctx->channels <= 0) { + av_log(ctx, AV_LOG_ERROR, "Invalid number of channels\n"); + return AVERROR(EINVAL); + } + + state->fq = 0; + state->q = 0; + state->s = 0; + state->lt = -128; + + ctx->sample_fmt = AV_SAMPLE_FMT_U8; + ctx->bits_per_raw_sample = 8; + + return 0; +} + +static int dfpwm_dec_frame(struct AVCodecContext *ctx, void *data, + int *got_frame, struct AVPacket *packet) +{ + DFPWMState *state = ctx->priv_data; + AVFrame *frame = data; + int ret; + + frame->nb_samples = packet->size * 8 / ctx->channels; + if (frame->nb_samples <= 0) { + av_log(ctx, AV_LOG_ERROR, "invalid number of samples in packet\n"); + return AVERROR_INVALIDDATA; + } + + if ((ret = ff_get_buffer(ctx, frame, 0)) < 0) + return ret; + + au_decompress(state, 140, packet->size, frame->data[0], packet->data); + + *got_frame = 1; + return packet->size; +} + +const AVCodec ff_dfpwm_decoder = { + .name = "dfpwm", + .long_name = NULL_IF_CONFIG_SMALL("DFPWM1a audio"), + .type = AVMEDIA_TYPE_AUDIO, + .id = AV_CODEC_ID_DFPWM, + .priv_data_size = sizeof(DFPWMState), + .init = dfpwm_dec_init, + .decode = dfpwm_dec_frame, + .capabilities = AV_CODEC_CAP_DR1, + .caps_internal = FF_CODEC_CAP_INIT_THREADSAFE, +}; diff --git a/libavcodec/dfpwmenc.c b/libavcodec/dfpwmenc.c new file mode 100644 index 0000000..02f2e64 --- /dev/null +++ b/libavcodec/dfpwmenc.c @@ -0,0 +1,121 @@ +/* + * DFPWM encoder + * Copyright (c) 2022 Jack Bruienne + * Copyright (c) 2012, 2016 Ben "GreaseMonkey" Russell + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * DFPWM1a encoder + */ + +#include "libavutil/internal.h" +#include "avcodec.h" +#include "codec_id.h" +#include "encode.h" +#include "internal.h" + +typedef struct { + int fq, q, s, lt; +} DFPWMState; + +// DFPWM codec from https://github.com/ChenThread/dfpwm/blob/master/1a/ +// Licensed in the public domain + +// note, len denotes how many compressed bytes there are (uncompressed bytes / 8). +static void au_compress(DFPWMState *state, int len, uint8_t *outbuf, uint8_t *inbuf) +{ + unsigned d = 0; + for (int i = 0; i < len; i++) { + for (int j = 0; j < 8; j++) { + int nq, st, ns; + // get sample + int v = *(inbuf++) - 128; + // set bit / target + int t = (v > state->q || (v == state->q && v == 127) ? 127 : -128); + d >>= 1; + if(t > 0) + d |= 0x80; + + // adjust charge + nq = state->q + ((state->s * (t-state->q) + 512)>>10); + if(nq == state->q && nq != t) + nq += (t == 127 ? 1 : -1); + state->q = nq; + + // adjust strength + st = (t != state->lt ? 0 : 1023); + ns = state->s; + if(ns != st) + ns += (st != 0 ? 1 : -1); + if(ns < 8) ns = 8; + state->s = ns; + + state->lt = t; + } + + // output bits + *(outbuf++) = d; + } +} + +static av_cold int dfpwm_enc_init(struct AVCodecContext *ctx) +{ + DFPWMState *state = ctx->priv_data; + + state->fq = 0; + state->q = 0; + state->s = 0; + state->lt = -128; + + ctx->bits_per_coded_sample = 1; + + return 0; +} + +static int dfpwm_enc_frame(struct AVCodecContext *ctx, struct AVPacket *packet, + const struct AVFrame *frame, int *got_packet) +{ + DFPWMState *state = ctx->priv_data; + int size = frame->nb_samples * frame->channels / 8 + (frame->nb_samples % 8 > 0 ? 1 : 0); + int ret = ff_get_encode_buffer(ctx, packet, size, 0); + + if (ret) { + *got_packet = 0; + return ret; + } + + au_compress(state, size, packet->data, frame->data[0]); + + *got_packet = 1; + return 0; +} + +const AVCodec ff_dfpwm_encoder = { + .name = "dfpwm", + .long_name = NULL_IF_CONFIG_SMALL("DFPWM1a audio"), + .type = AVMEDIA_TYPE_AUDIO, + .id = AV_CODEC_ID_DFPWM, + .priv_data_size = sizeof(DFPWMState), + .init = dfpwm_enc_init, + .encode2 = dfpwm_enc_frame, + .sample_fmts = (const enum AVSampleFormat[]){AV_SAMPLE_FMT_U8, AV_SAMPLE_FMT_NONE}, + .capabilities = AV_CODEC_CAP_DR1 | AV_CODEC_CAP_VARIABLE_FRAME_SIZE, + .caps_internal = FF_CODEC_CAP_INIT_THREADSAFE, +}; diff --git a/libavcodec/utils.c b/libavcodec/utils.c index 6f9d90a..066da76 100644 --- a/libavcodec/utils.c +++ b/libavcodec/utils.c @@ -577,6 +577,8 @@ enum AVCodecID av_get_pcm_codec(enum AVSampleFormat fmt, int be) int av_get_bits_per_sample(enum AVCodecID codec_id) { switch (codec_id) { + case AV_CODEC_ID_DFPWM: + return 1; case AV_CODEC_ID_ADPCM_SBPRO_2: return 2; case AV_CODEC_ID_ADPCM_SBPRO_3: diff --git a/libavcodec/version.h b/libavcodec/version.h index 063d61e..84f3979 100644 --- a/libavcodec/version.h +++ b/libavcodec/version.h @@ -28,8 +28,8 @@ #include "libavutil/version.h" #define LIBAVCODEC_VERSION_MAJOR 59 -#define LIBAVCODEC_VERSION_MINOR 21 -#define LIBAVCODEC_VERSION_MICRO 103 +#define LIBAVCODEC_VERSION_MINOR 22 +#define LIBAVCODEC_VERSION_MICRO 100 #define LIBAVCODEC_VERSION_INT AV_VERSION_INT(LIBAVCODEC_VERSION_MAJOR, \ LIBAVCODEC_VERSION_MINOR, \