From patchwork Sun May 1 23:16:16 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Fletcher X-Patchwork-Id: 35535 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:a885:b0:7f:4be2:bd17 with SMTP id ca5csp1537268pzb; Sun, 1 May 2022 16:17:31 -0700 (PDT) X-Google-Smtp-Source: ABdhPJw2TsYceQWCevFJxYPp2mbT0Mwed5w+ybkLWbrkZc9kEFWhufhJMa6TI64w+Gs+8KLNuet+ X-Received: by 2002:a05:6402:84e:b0:422:b76c:bef8 with SMTP id b14-20020a056402084e00b00422b76cbef8mr11158496edz.238.1651447051651; Sun, 01 May 2022 16:17:31 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1651447051; cv=none; d=google.com; s=arc-20160816; b=vicYf4QkZ5+FqWLfBGO6vqeY+j2/0nfVVRxmVVmE62b0avsHoleuCWBZ+WPQu0rR/q pEAuQlqu3FaMET1nNWuXBaZFRviRTpOkMPyFLi2pCpRhjdGpIMbaunKdSksoQq3sQmuN pSMkbdvCvsGJTdsX6rextgLp5h2ewZnwr+9NDAQabFWpTebQaRwvY+ER6lehJY/KQIOM ccjCPH6tRXPhpt4cXwa3IKJstIeMbK1eiwzuC//OUzzOcxUyfJPy0OmC4ppvSRLWv+8q HamZkI+esPD/9fi7X7fmQhrgYrLlkEvGX9tW6FyhP3CZMEJUxHyU1H2SjZfcGPbWpNbg VRYw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:reply-to:list-subscribe:list-help:list-post :list-archive:list-unsubscribe:list-id:precedence:subject :mime-version:bounce-to:from:message-id:date:to:delivered-to; bh=COkbn8OnGYKbZG4cYxBm8XnyVXwfR9aaEQl7z7ARqUM=; b=M4FBnWMpKXsqgy9CaMGgIwlgqbWVvKCjuQHZYeaZs8rfIiwHg9c7IJWBjAz7HRYwK0 IaOxaH+A+/eeggjQB1V3obvxyycJkcTPFPcCO9p+V3Fh+Z3gL8LgOXY0/4bticWSg94E mIaiZhU1rb4K6sfXREWAZ8aOiF54HGgXVrWVTFeMhZQCyako7YdVMSu1/sW5BZSAZfsb dScmnUqbaYpmVV6b5i2n7Zoo4NHHDpzGhOpX6u0U2QMWZ2OCwRhS9IZq9dTOKDXI6uCM UpgEs/X3zZOlCRkf8sk6XzR6JnnGhhAQVTwVKmTiR2uB59S3j5o4SOXg+Voxytx7TkNU nRmg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id gi10-20020a1709070c8a00b006f3a306a1e7si9708415ejc.555.2022.05.01.16.17.31; Sun, 01 May 2022 16:17:31 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 185CE68B284; Mon, 2 May 2022 02:17:28 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from avasout-ptp-002.plus.net (avasout-ptp-002.plus.net [84.93.230.235]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 8313368AE7E for ; Mon, 2 May 2022 02:17:20 +0300 (EEST) Received: from sparrow.collectfair.co.uk ([80.229.167.224]) by smtp with SMTP id lIoYn2PS3O2rilIoZnVSlX; Mon, 02 May 2022 00:17:20 +0100 X-Clacks-Overhead: "GNU Terry Pratchett" X-CM-Score: 0.00 X-CNFS-Analysis: v=2.4 cv=Zs/+lv3G c=1 sm=1 tr=0 ts=626f1500 a=kGmXhoCkr3jB6KbzR35ddQ==:117 a=kGmXhoCkr3jB6KbzR35ddQ==:17 a=oZkIemNP1mAA:10 a=dUtWRZAB1rDCoh9MlMwA:9 a=JeY4nRuWAAAA:8 a=wcOQnT_eH2YsDj3zZLgA:9 a=wPNLvfGTeEIA:10 a=fHJYd22vZ_cA:10 a=oXpFOLvHeOhgG0KdW9kA:9 a=B2y7HmGcmWMA:10 a=3mYD0m367-DnXB0HWSkl:22 Received: (qmail 15579 invoked from network); 1 May 2022 23:16:16 -0000 Received: from localhost (HELO webmail.megapico.co.uk) (127.0.0.1) by sparrow.collectfair.co.uk with SMTP; 1 May 2022 23:16:16 -0000 Received: from 192.168.0.1 (auth. user dif@localhost) by webmail.megapico.co.uk with HTTP; Sun, 01 May 2022 23:16:16 +0000 To: ffmpeg-devel@ffmpeg.org Date: Sun, 01 May 2022 23:16:16 +0000 X-Mailer: IlohaMail/0.8.14 Message-ID: <5HQI8uvv.1651446976.2498880.dif@localhost> From: "David Fletcher" Bounce-To: "David Fletcher" MIME-Version: 1.0 X-CMAE-Envelope: MS4xfBbxOMxMsb4PQn4VJeoA3XQ2/+vJ5PUVovo6tDPN1DxJBXTIbN82V1kVqFO+DXqrqF12BlOiU+Jr4kFt40GH6H2B+O95TpTWMfN6KhcUfWl2WGSDNlmo PWQpNhzxE3uqnPUyjivNU+ezcnWVLtgZc9iWsvw6RE5YjiNLfov987c1Y8TK/yzrj5Mm+sy9x3gqn1YNH2UAqx/Hol0vihrHqLM= Subject: [FFmpeg-devel] PATCH - libmad MP3 decoding support X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 7O9S4gs+kLFy Please find attached a patch adding support for MP3 decoding using libmad. This is against release ffmpeg-5.0.1, and works for libmad 0.15.1b (the most recent available). I hope that this is useful. Some context - I found the FFmpeg inbuild fixed point MP3 decoder produced very distorted audio for low bitrate streams (56k/s and lower). This was on an ARMv4 CPU for which I needed to make some adaptation to the fixed point maths routines which currently target more recent CPUs. I was not able to resolve the distortion issue and it was easier to integrate libmad decoding which is already optimised for very fast decoding on the ARMv4 CPU. More info about this application here: http://www.megapico.co.uk/sharpfin/mediaserver.html. Best regards, David. diff -Nur ./ffmpeg-5.0.1/configure ./ffmpeg-5.0.1-mad/configure --- ./ffmpeg-5.0.1/configure 2022-04-04 15:40:22.000000000 +0100 +++ ./ffmpeg-5.0.1-mad/configure 2022-05-01 22:50:01.435432431 +0100 @@ -244,6 +244,7 @@ --enable-libklvanc enable Kernel Labs VANC processing [no] --enable-libkvazaar enable HEVC encoding via libkvazaar [no] --enable-liblensfun enable lensfun lens correction [no] + --enable-libmad enable MP3 decoding via libmad [no] --enable-libmodplug enable ModPlug via libmodplug [no] --enable-libmp3lame enable MP3 encoding via libmp3lame [no] --enable-libopencore-amrnb enable AMR-NB de/encoding via libopencore-amrnb [no] @@ -1772,6 +1773,7 @@ frei0r libcdio libdavs2 + libmad librubberband libvidstab libx264 @@ -3334,6 +3336,7 @@ libilbc_encoder_deps="libilbc" libkvazaar_encoder_deps="libkvazaar" libmodplug_demuxer_deps="libmodplug" +libmad_decoder_deps="libmad" libmp3lame_encoder_deps="libmp3lame" libmp3lame_encoder_select="audio_frame_queue mpegaudioheader" libopencore_amrnb_decoder_deps="libopencore_amrnb" @@ -6569,6 +6572,7 @@ fi enabled libmodplug && require_pkg_config libmodplug libmodplug libmodplug/modplug.h ModPlug_Load +enabled libmad && require libmad "mad.h" mad_decoder_init -lmad enabled libmp3lame && require "libmp3lame >= 3.98.3" lame/lame.h lame_set_VBR_quality -lmp3lame $libm_extralibs enabled libmysofa && { check_pkg_config libmysofa libmysofa mysofa.h mysofa_neighborhood_init_withstepdefine || require libmysofa mysofa.h mysofa_neighborhood_init_withstepdefine -lmysofa $zlib_extralibs; } diff -Nur ./ffmpeg-5.0.1/libavcodec/Makefile ./ffmpeg-5.0.1-mad/libavcodec/Makefile --- ./ffmpeg-5.0.1/libavcodec/Makefile 2022-01-14 18:45:39.000000000 +0000 +++ ./ffmpeg-5.0.1-mad/libavcodec/Makefile 2022-04-06 22:56:06.000000000 +0100 @@ -1054,6 +1054,7 @@ OBJS-$(CONFIG_LIBILBC_DECODER) += libilbc.o OBJS-$(CONFIG_LIBILBC_ENCODER) += libilbc.o OBJS-$(CONFIG_LIBKVAZAAR_ENCODER) += libkvazaar.o +OBJS-$(CONFIG_LIBMAD_DECODER) += libmaddec.o OBJS-$(CONFIG_LIBMP3LAME_ENCODER) += libmp3lame.o OBJS-$(CONFIG_LIBOPENCORE_AMRNB_DECODER) += libopencore-amr.o OBJS-$(CONFIG_LIBOPENCORE_AMRNB_ENCODER) += libopencore-amr.o diff -Nur ./ffmpeg-5.0.1/libavcodec/allcodecs.c ./ffmpeg-5.0.1-mad/libavcodec/allcodecs.c --- ./ffmpeg-5.0.1/libavcodec/allcodecs.c 2022-01-14 18:45:39.000000000 +0000 +++ ./ffmpeg-5.0.1-mad/libavcodec/allcodecs.c 2022-04-06 22:54:45.000000000 +0100 @@ -744,6 +744,7 @@ extern const AVCodec ff_libgsm_ms_decoder; extern const AVCodec ff_libilbc_encoder; extern const AVCodec ff_libilbc_decoder; +extern const AVCodec ff_libmad_decoder; extern const AVCodec ff_libmp3lame_encoder; extern const AVCodec ff_libopencore_amrnb_encoder; extern const AVCodec ff_libopencore_amrnb_decoder; diff -Nur ./ffmpeg-5.0.1/libavcodec/codec_id.h ./ffmpeg-5.0.1-mad/libavcodec/codec_id.h --- ./ffmpeg-5.0.1/libavcodec/codec_id.h 2022-01-14 18:45:39.000000000 +0000 +++ ./ffmpeg-5.0.1-mad/libavcodec/codec_id.h 2022-04-06 22:52:18.000000000 +0100 @@ -516,6 +516,7 @@ AV_CODEC_ID_HCA, AV_CODEC_ID_FASTAUDIO, AV_CODEC_ID_MSNSIREN, + AV_CODEC_ID_LIBMAD, /* subtitle codecs */ AV_CODEC_ID_FIRST_SUBTITLE = 0x17000, ///< A dummy ID pointing at the start of subtitle codecs. diff -Nur ./ffmpeg-5.0.1/libavcodec/libmaddec.c ./ffmpeg-5.0.1-mad/libavcodec/libmaddec.c --- ./ffmpeg-5.0.1/libavcodec/libmaddec.c 1970-01-01 01:00:00.000000000 +0100 +++ ./ffmpeg-5.0.1-mad/libavcodec/libmaddec.c 2022-04-12 19:00:21.000000000 +0100 @@ -0,0 +1,181 @@ +/* + * MP3 decoder using libmad + * Copyright (c) 2022 David Fletcher + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include + +#include "libavutil/channel_layout.h" +#include "libavutil/common.h" +#include "avcodec.h" +#include "internal.h" +#include "decode.h" + +#define MAD_BUFSIZE (32 * 1024) +#define MIN(a, b) ((a) < (b) ? (a) : (b)) + +typedef struct libmad_context { + uint8_t input_buffer[MAD_BUFSIZE+MAD_BUFFER_GUARD]; + struct mad_synth synth; + struct mad_stream stream; + struct mad_frame frame; + struct mad_header header; + int got_header; +}libmad_context; + +/* utility to scale and round samples to 16 bits */ +static inline signed int mad_scale(mad_fixed_t sample) +{ + /* round */ + sample += (1L << (MAD_F_FRACBITS - 16)); + + /* clip */ + if (sample >= MAD_F_ONE) + sample = MAD_F_ONE - 1; + else if (sample < -MAD_F_ONE) + sample = -MAD_F_ONE; + + /* quantize */ + return sample >> (MAD_F_FRACBITS + 1 - 16); +} + +static av_cold int libmad_decode_init(AVCodecContext *avc) +{ + libmad_context *mad = avc->priv_data; + + mad_synth_init (&mad->synth); + mad_stream_init (&mad->stream); + mad_frame_init (&mad->frame); + mad->got_header = 0; + + return 0; +} + +static av_cold int libmad_decode_close(AVCodecContext *avc) +{ + libmad_context *mad = avc->priv_data; + + mad_synth_finish(&mad->synth); + mad_frame_finish(&mad->frame); + mad_stream_finish(&mad->stream); + + mad = NULL; + + return 0; +} + +static int libmad_decode_frame(AVCodecContext *avc, void *data, + int *got_frame_ptr, AVPacket *pkt) +{ + AVFrame *frame = data; + libmad_context *mad = avc->priv_data; + struct mad_pcm *pcm; + mad_fixed_t const *left_ch; + mad_fixed_t const *right_ch; + int16_t *output; + int nsamples; + int nchannels; + size_t bytes_read = 0; + size_t remaining = 0; + + if (!avc) + return 0; + + if (!mad) + return 0; + + remaining = mad->stream.bufend - mad->stream.next_frame; + memmove(mad->input_buffer, mad->stream.next_frame, remaining); + bytes_read = MIN(pkt->size, MAD_BUFSIZE - remaining); + memcpy(mad->input_buffer+remaining, pkt->data, bytes_read); + + if (bytes_read == 0){ + *got_frame_ptr = 0; + return 0; + } + + mad_stream_buffer(&mad->stream, mad->input_buffer, remaining + bytes_read); + mad->stream.error = 0; + + if(!mad->got_header){ + mad_header_decode(&mad->header, &mad->stream); + mad->got_header = 1; + avc->frame_size = 32 * (mad->header.layer == MAD_LAYER_I ? 12 : \ + ((mad->header.layer == MAD_LAYER_III && \ + (mad->header.flags & MAD_FLAG_LSF_EXT)) ? 18 : 36)); + avc->sample_fmt = AV_SAMPLE_FMT_S16; + if(mad->header.mode == MAD_MODE_SINGLE_CHANNEL){ + avc->channel_layout = AV_CH_LAYOUT_MONO; + avc->channels = 1; + }else{ + avc->channel_layout = AV_CH_LAYOUT_STEREO; + avc->channels = 2; + } + } + + frame->channel_layout = avc->channel_layout; + frame->format = avc->sample_fmt; + frame->channels = avc->channels; + frame->nb_samples = avc->frame_size; + + if ((ff_get_buffer(avc, frame, 0)) < 0) + return 0; + + if (mad_frame_decode(&mad->frame, &mad->stream) == -1) { + *got_frame_ptr = 0; + return mad->stream.bufend - mad->stream.next_frame; + } + + mad_synth_frame (&mad->synth, &mad->frame); + + pcm = &mad->synth.pcm; + output = (int16_t *)frame->data[0]; + nsamples = pcm->length; + nchannels = pcm->channels; + left_ch = pcm->samples[0]; + right_ch = pcm->samples[1]; + while (nsamples--) { + *output++ = mad_scale(*(left_ch++)); + if (nchannels == 2) { + *output++ = mad_scale(*(right_ch++)); + } + //Players should recognise mono and play through both channels + //Writing the same thing to both left and right channels here causes + //memory issues as it creates double the number of samples allocated. + } + + *got_frame_ptr = 1; + + return mad->stream.bufend - mad->stream.next_frame; +} + +AVCodec ff_libmad_decoder = { + .name = "libmad", + .long_name = NULL_IF_CONFIG_SMALL("libmad MP3 decoder"), + .wrapper_name = "libmad", + .type = AVMEDIA_TYPE_AUDIO, + .id = AV_CODEC_ID_MP3, + .sample_fmts = (const enum AVSampleFormat[]) { AV_SAMPLE_FMT_S16, AV_SAMPLE_FMT_NONE }, + .capabilities = AV_CODEC_CAP_DR1 | AV_CODEC_CAP_CHANNEL_CONF, + .priv_data_size = sizeof(libmad_context), + .init = libmad_decode_init, + .close = libmad_decode_close, + .decode = libmad_decode_frame +}; +