From patchwork Thu Nov 5 13:48:27 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zane van Iperen X-Patchwork-Id: 23405 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 82E61448C17 for ; Thu, 5 Nov 2020 15:48:59 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 6C27A68B873; Thu, 5 Nov 2020 15:48:59 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from out1.migadu.com (out1.migadu.com [91.121.223.63]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 274F368B86E for ; Thu, 5 Nov 2020 15:48:52 +0200 (EET) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=zanevaniperen.com; s=key1; t=1604584131; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fLBkQzAP0+VhwzZybVdAmAuNcMIDzEDSvOcNg8eoNHA=; b=B/0RJ+WfS5fYIG7i6An8cF7sa+qrxN8JFIa2E4/AFL1KbdAiX/TEZan0HyTXkUVcR8JJMT qR+KP3rYjnjt2LlV4JtfsB0CH97UP7sE1uL/B0V79GTkAqHqX6jqBB8s4DhZUO4pG772P7 VbrNvGOWwhb+cXoXj4fXSFPw85pzZq7pJsWU51UsRe81xhqt/LE/B8FYbrtaaohInVKYCm REab7F2kA6sfowv69ilWGNYHntOnS5UDXlS50N6+6BI1a0RE0pBLcz/SgVHwateubMzKbU 0A86S/YdTIfJdypV+yqVoeca7+0AFPsMvQcXcBd/5urna/47Km7voBP5MuL6EA== From: Zane van Iperen To: ffmpeg-devel@ffmpeg.org Date: Thu, 5 Nov 2020 23:48:27 +1000 Message-Id: <20201105134827.25564-6-zane@zanevaniperen.com> In-Reply-To: <20201105134827.25564-1-zane@zanevaniperen.com> References: <20201105134827.25564-1-zane@zanevaniperen.com> MIME-Version: 1.0 X-Authenticated-User: git-morningstar@zanevaniperen.com X-Spam-Score: 0.00 Subject: [FFmpeg-devel] [PATCH v4 5/5] avformat: add amv muxer X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Zane van Iperen Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" AMV is a hard-coded (and broken) subset of AVI. It's not worth sullying the existing AVI muxer with its filth. Fixes ticket #747. Signed-off-by: Zane van Iperen --- MAINTAINERS | 1 + libavformat/Makefile | 1 + libavformat/allformats.c | 1 + libavformat/amvenc.c | 413 +++++++++++++++++++++++++++++++++++++++ 4 files changed, 416 insertions(+) create mode 100644 libavformat/amvenc.c diff --git a/MAINTAINERS b/MAINTAINERS index 7a4161e2e5..2464f82be3 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -394,6 +394,7 @@ Muxers/Demuxers: aiffdec.c Baptiste Coudurier, Matthieu Bouron aiffenc.c Baptiste Coudurier, Matthieu Bouron alp.c Zane van Iperen + amvenc.c Zane van Iperen apm.c Zane van Iperen apngdec.c Benoit Fouet argo_asf.c Zane van Iperen diff --git a/libavformat/Makefile b/libavformat/Makefile index 25bb8b2c69..be5a482b01 100644 --- a/libavformat/Makefile +++ b/libavformat/Makefile @@ -93,6 +93,7 @@ OBJS-$(CONFIG_AMR_DEMUXER) += amr.o OBJS-$(CONFIG_AMR_MUXER) += amr.o rawenc.o OBJS-$(CONFIG_AMRNB_DEMUXER) += amr.o OBJS-$(CONFIG_AMRWB_DEMUXER) += amr.o +OBJS-$(CONFIG_AMV_MUXER) += amvenc.o OBJS-$(CONFIG_ANM_DEMUXER) += anm.o OBJS-$(CONFIG_APC_DEMUXER) += apc.o OBJS-$(CONFIG_APE_DEMUXER) += ape.o apetag.o img2.o diff --git a/libavformat/allformats.c b/libavformat/allformats.c index 9b1b611a49..53e5374255 100644 --- a/libavformat/allformats.c +++ b/libavformat/allformats.c @@ -54,6 +54,7 @@ extern AVInputFormat ff_amr_demuxer; extern AVOutputFormat ff_amr_muxer; extern AVInputFormat ff_amrnb_demuxer; extern AVInputFormat ff_amrwb_demuxer; +extern AVOutputFormat ff_amv_muxer; extern AVInputFormat ff_anm_demuxer; extern AVInputFormat ff_apc_demuxer; extern AVInputFormat ff_ape_demuxer; diff --git a/libavformat/amvenc.c b/libavformat/amvenc.c new file mode 100644 index 0000000000..4d4ec7a2b1 --- /dev/null +++ b/libavformat/amvenc.c @@ -0,0 +1,413 @@ +/* + * AMV muxer + * + * Copyright (C) 2020 Zane van Iperen (zane@zanevaniperen.com) + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ +#include "avformat.h" +#include "riff.h" +#include "internal.h" +#include "avio_internal.h" +#include "libavutil/intreadwrite.h" +#include "libavutil/avassert.h" + +/* + * Things to note: + * - AMV is a hard-coded (and broken) subset of AVI. It's not worth sullying the + * existing AVI muxer with its filth. + * - No separate demuxer as the existing AVI demuxer can handle these. + * - The sizes of certain tags are deliberately set to 0 as some players break + * when they're set correctly. Ditto with some header fields. + * - There is no index. + * - Players are **very** sensitive to the frame order and sizes. + * - Frames must be strictly interleaved as V-A, any V-V or A-A will + * cause crashes. + * - Variable video frame sizes seem to be handled fine. + * - Variable audio frame sizes cause crashes. + * - If audio is shorter than video, it's padded with silence. + * - If video is shorter than audio, the most recent frame is repeated. + */ + +#define AMV_STREAM_COUNT 2 +#define AMV_STREAM_VIDEO 0 +#define AMV_STREAM_AUDIO 1 +#define AMV_VIDEO_STRH_SIZE 56 +#define AMV_VIDEO_STRF_SIZE 36 +#define AMV_AUDIO_STRH_SIZE 48 +#define AMV_AUDIO_STRF_SIZE 20 /* sizeof(WAVEFORMATEX) + 2 */ + +typedef struct AMVContext +{ + int64_t riff_start; + int64_t movi_list; + int64_t offset_duration; + int last_stream; + + int32_t us_per_frame; /* Microseconds per frame. */ + + int32_t aframe_size; /* Expected audio frame size. */ + int32_t ablock_align; /* Expected audio block align. */ + AVPacket apad; /* Dummy audio packet for padding. */ + AVPacket vpad; /* Most recent video frame, for padding. */ + + /* + * Cumulative PTS values for each stream, used for the final + * duration calculcation. + */ + int64_t lastpts[AMV_STREAM_COUNT]; +} AMVContext; + +/* ff_{start,end}_tag(), but sets the size to 0. */ +static int64_t amv_start_tag(AVIOContext *pb, const char *tag) +{ + ffio_wfourcc(pb, tag); + avio_wl32(pb, 0); + return avio_tell(pb); +} + +static void amv_end_tag(AVIOContext *pb, int64_t start) +{ + int64_t pos; + av_assert0((start&1) == 0); + + pos = avio_tell(pb); + if (pos & 1) + avio_w8(pb, 0); +} + +static av_cold int amv_init(AVFormatContext *s) +{ + AMVContext *amv = s->priv_data; + AVStream *vst, *ast; + int ret; + + amv->last_stream = -1; + + if (s->nb_streams != AMV_STREAM_COUNT) { + av_log(s, AV_LOG_ERROR, "AMV files only support 2 streams\n"); + return AVERROR(EINVAL); + } + + vst = s->streams[AMV_STREAM_VIDEO]; + ast = s->streams[AMV_STREAM_AUDIO]; + + if (vst->codecpar->codec_id != AV_CODEC_ID_AMV) { + av_log(s, AV_LOG_ERROR, "First AMV stream must be %s\n", + avcodec_get_name(AV_CODEC_ID_AMV)); + return AVERROR(EINVAL); + } + + if (ast->codecpar->codec_id != AV_CODEC_ID_ADPCM_IMA_AMV) { + av_log(s, AV_LOG_ERROR, "Second AMV stream must be %s\n", + avcodec_get_name(AV_CODEC_ID_ADPCM_IMA_AMV)); + return AVERROR(EINVAL); + } + + /* These files are broken-enough as they are. They shouldn't be streamed. */ + if (!(s->pb->seekable & AVIO_SEEKABLE_NORMAL)) { + av_log(s, AV_LOG_ERROR, "Stream not seekable, unable to write output file\n"); + return AVERROR(EINVAL); + } + + amv->us_per_frame = av_rescale(AV_TIME_BASE, vst->time_base.num, vst->time_base.den); + amv->aframe_size = av_rescale(ast->codecpar->sample_rate, amv->us_per_frame, AV_TIME_BASE); + amv->ablock_align = 8 + (FFALIGN(amv->aframe_size, 2) / 2); + + av_log(s, AV_LOG_TRACE, "us_per_frame = %d\n", amv->us_per_frame); + av_log(s, AV_LOG_TRACE, "aframe_size = %d\n", amv->aframe_size); + av_log(s, AV_LOG_TRACE, "ablock_align = %d\n", amv->ablock_align); + + /* + * Bail if the framerate's too high. Prevents the audio frame size from + * getting too small. 63fps is the closest value to 60fps that divides + * cleanly, so cap it there. + */ + if (amv->us_per_frame < 15873) { + av_log(s, AV_LOG_ERROR, "Refusing to mux >63fps video\n"); + return AVERROR(EINVAL); + } + + /* + * frame_size will be set if coming from the encoder. + * Make sure the its been configured correctly. The audio frame duration + * needs to match that of the video. + */ + if (ast->codecpar->frame_size) { + AVCodecParameters *par = ast->codecpar; + int bad = 0; + + if (par->frame_size != amv->aframe_size) { + av_log(s, AV_LOG_ERROR, "Invalid audio frame size. Got %d, wanted %d\n", + par->frame_size, amv->aframe_size); + bad = 1; + } + + if (par->block_align != amv->ablock_align) { + av_log(s, AV_LOG_ERROR, "Invalid audio block align. Got %d, wanted %d\n", + par->block_align, amv->ablock_align); + bad = 1; + } + + if (bad) { + av_log(s, AV_LOG_ERROR, "Try -block_size %d\n", amv->aframe_size); + return AVERROR(EINVAL); + } + + if (ast->codecpar->sample_rate % amv->aframe_size) { + av_log(s, AV_LOG_ERROR, "Audio sample rate not a multiple of the frame size.\n" + "Please change video frame rate. Suggested rates: 10,14,15,18,21,25,30\n"); + return AVERROR(EINVAL); + } + } else { + /* If remuxing from the same source, then this will match the video. */ + int32_t aus = av_rescale(AV_TIME_BASE, ast->time_base.num, ast->time_base.den); + if (aus != amv->us_per_frame) { + av_log(s, AV_LOG_ERROR, "Cannot remux streams with a different time base\n"); + return AVERROR(EINVAL); + } + } + + /* Allocate and fill dummy packet so we can pad the audio. */ + if ((ret = av_new_packet(&amv->apad, amv->ablock_align)) < 0) + return ret; + + amv->apad.stream_index = AMV_STREAM_AUDIO; + memset(amv->apad.data, 0, amv->ablock_align); + AV_WL32(amv->apad.data + 4, amv->aframe_size); + + av_init_packet(&amv->vpad); + amv->vpad.stream_index = AMV_STREAM_VIDEO; + amv->vpad.duration = 1; + return 0; +} + +static void amv_deinit(AVFormatContext *s) +{ + AMVContext *amv = s->priv_data; + + av_packet_unref(&amv->apad); + av_packet_unref(&amv->vpad); +} + +static void amv_write_vlist(AVFormatContext *s, AVCodecParameters *par) +{ + int64_t tag_list, tag_str; + + av_assert0(par->codec_id == AV_CODEC_ID_AMV); + + tag_list = amv_start_tag(s->pb, "LIST"); + ffio_wfourcc(s->pb, "strl"); + tag_str = ff_start_tag(s->pb, "strh"); + ffio_fill(s->pb, 0, AMV_VIDEO_STRH_SIZE); + ff_end_tag(s->pb, tag_str); + + tag_str = ff_start_tag(s->pb, "strf"); + ffio_fill(s->pb, 0, AMV_VIDEO_STRF_SIZE); + ff_end_tag(s->pb, tag_str); + + amv_end_tag(s->pb, tag_list); +} + +static void amv_write_alist(AVFormatContext *s, AVCodecParameters *par) +{ + uint8_t buf[AMV_AUDIO_STRF_SIZE]; + AVIOContext *pb = s->pb; + int64_t tag_list, tag_str; + + av_assert0(par->codec_id == AV_CODEC_ID_ADPCM_IMA_AMV); + + tag_list = amv_start_tag(pb, "LIST"); + ffio_wfourcc(pb, "strl"); + tag_str = ff_start_tag(pb, "strh"); + ffio_fill(s->pb, 0, AMV_AUDIO_STRH_SIZE); + ff_end_tag(pb, tag_str); + + /* Bodge an (incorrect) WAVEFORMATEX (+2 pad bytes) */ + tag_str = ff_start_tag(pb, "strf"); + AV_WL16(buf + 0, 1); + AV_WL16(buf + 2, par->channels); + AV_WL32(buf + 4, par->sample_rate); + AV_WL32(buf + 8, par->sample_rate * par->channels * 2); + AV_WL16(buf + 12, 2); + AV_WL16(buf + 14, 16); + AV_WL16(buf + 16, 0); + AV_WL16(buf + 18, 0); + avio_write(pb, buf, AMV_AUDIO_STRF_SIZE); + ff_end_tag(pb, tag_str); + + amv_end_tag(pb, tag_list); +} + +static int amv_write_header(AVFormatContext *s) +{ + AMVContext *amv = s->priv_data; + AVIOContext *pb = s->pb; + AVStream *vst = s->streams[AMV_STREAM_VIDEO]; + AVStream *ast = s->streams[AMV_STREAM_AUDIO]; + uint8_t amvh[56] = {0}; + int64_t list1; + + amv->riff_start = amv_start_tag(pb, "RIFF"); + ffio_wfourcc(pb, "AMV "); + list1 = amv_start_tag(pb, "LIST"); + ffio_wfourcc(pb, "hdrl"); + + ffio_wfourcc(pb, "amvh"); + avio_wl32(pb, 56); + + AV_WL32(amvh + 0, amv->us_per_frame); + AV_WL32(amvh + 32, vst->codecpar->width); + AV_WL32(amvh + 36, vst->codecpar->height); + AV_WL32(amvh + 40, vst->time_base.den); + AV_WL32(amvh + 44, vst->time_base.num); + AV_WL32(amvh + 48, 0); + AV_WL32(amvh + 52, 0); /* duration, filled in later. */ + + avio_write(pb, amvh, sizeof(amvh)); + amv->offset_duration = avio_tell(pb) - 4; + + amv_write_vlist(s, vst->codecpar); + amv_write_alist(s, ast->codecpar); + amv_end_tag(pb, list1); + + amv->movi_list = amv_start_tag(pb, "LIST"); + ffio_wfourcc(pb, "movi"); + return 0; +} + +static int amv_write_packet_internal(AVFormatContext *s, AVPacket *pkt) +{ + AMVContext *amv = s->priv_data; + + if (pkt->stream_index == AMV_STREAM_VIDEO) + ffio_wfourcc(s->pb, "00dc"); + else if (pkt->stream_index == AMV_STREAM_AUDIO) + ffio_wfourcc(s->pb, "01wb"); + else + av_assert0(0); + + if (pkt->stream_index == AMV_STREAM_AUDIO && pkt->size != amv->ablock_align) { + /* Can happen when remuxing files produced by another encoder. */ + av_log(s, AV_LOG_WARNING, "Invalid audio packet size (%d != %d)\n", + pkt->size, amv->ablock_align); + } + + avio_wl32(s->pb, pkt->size); + avio_write(s->pb, pkt->data, pkt->size); + + amv->lastpts[pkt->stream_index] += pkt->duration; + amv->last_stream = pkt->stream_index; + return 0; +} + +static int amv_pad(AVFormatContext *s, AVPacket *pkt) +{ + AMVContext *amv = s->priv_data; + int stream_index = pkt->stream_index; + + if (stream_index != amv->last_stream) + return 0; + + stream_index = (stream_index + 1) % s->nb_streams; + if (stream_index == AMV_STREAM_VIDEO) + return amv_write_packet_internal(s, &amv->vpad); + else if (stream_index == AMV_STREAM_AUDIO) + return amv_write_packet_internal(s, &amv->apad); + else + av_assert0(0); + + return AVERROR(EINVAL); +} + +static int amv_write_packet(AVFormatContext *s, AVPacket *pkt) +{ + AMVContext *amv = s->priv_data; + int ret; + + /* Add a dummy frame if we've received two of the same index. */ + if ((ret = amv_pad(s, pkt)) < 0) + return ret; + + if ((ret = amv_write_packet_internal(s, pkt)) < 0) + return ret; + + if (pkt->stream_index == AMV_STREAM_VIDEO) { + /* Save the last packet for padding. */ + av_packet_unref(&amv->vpad); + if ((ret = av_packet_ref(&amv->vpad, pkt)) < 0) + return ret; + } + + return 0; +} + +static int amv_write_trailer(AVFormatContext *s) +{ + AMVContext *amv = s->priv_data; + AVStream *vst = s->streams[AMV_STREAM_VIDEO]; + AVStream *ast = s->streams[AMV_STREAM_AUDIO]; + int64_t maxpts, ret; + int hh, mm, ss; + + /* Pad-out one last audio frame if needed. */ + if (amv->last_stream == AMV_STREAM_VIDEO) { + if ((ret = amv_write_packet_internal(s, &amv->apad)) < 0) + return ret; + } + + amv_end_tag(s->pb, amv->movi_list); + amv_end_tag(s->pb, amv->riff_start); + + ffio_wfourcc(s->pb, "AMV_"); + ffio_wfourcc(s->pb, "END_"); + + if ((ret = avio_seek(s->pb, amv->offset_duration, SEEK_SET)) < 0) + return ret; + + /* Go back and write the duration. */ + maxpts = FFMAX( + av_rescale_q(amv->lastpts[AMV_STREAM_VIDEO], vst->time_base, AV_TIME_BASE_Q), + av_rescale_q(amv->lastpts[AMV_STREAM_AUDIO], ast->time_base, AV_TIME_BASE_Q) + ); + + ss = maxpts / AV_TIME_BASE; + mm = ss / 60; + hh = mm / 60; + ss %= 60; + mm %= 60; + + avio_w8(s->pb, ss); + avio_w8(s->pb, mm); + avio_wl16(s->pb, hh); + return 0; +} + +AVOutputFormat ff_amv_muxer = { + .name = "amv", + .long_name = NULL_IF_CONFIG_SMALL("AMV"), + .mime_type = "video/amv", + .extensions = "amv", + .priv_data_size = sizeof(AMVContext), + .audio_codec = AV_CODEC_ID_ADPCM_IMA_AMV, + .video_codec = AV_CODEC_ID_AMV, + .init = amv_init, + .deinit = amv_deinit, + .write_header = amv_write_header, + .write_packet = amv_write_packet, + .write_trailer = amv_write_trailer, +};