From patchwork Mon Aug 30 11:14:06 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Jan_Ekstr=C3=B6m?= X-Patchwork-Id: 29878 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6602:2a4a:0:0:0:0 with SMTP id k10csp3852788iov; Mon, 30 Aug 2021 04:14:42 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwfhv7SCZ+KbFXuU0yru7LXEtcSu+R53Ia5xcZiq/Ktco97lKwDIbTbNu1+4twN5P1xZqfe X-Received: by 2002:a05:6402:180f:: with SMTP id g15mr17882086edy.258.1630322082709; Mon, 30 Aug 2021 04:14:42 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1630322082; cv=none; d=google.com; s=arc-20160816; b=001Bw18aKhlKO7aEq9BFigXBD3cRr6xxcjVNSPsJkB9AMFrR7eGIqnOs33m9n9MWyh fTyQr5KIDrqd0yC8RZB3EKNl998aE+11o0236es8NvTvaLcs4DyIiYAAbywg5dAxoNh9 Hs8dBu0hlXcPt2oA3PFxZqXgdAp4sRLpdZ/V7xIHSLkBCTMESYsiXpQIzSBImaENjfLY ySXnqXcY8jat65MOE9B4HxGilDS9PDpwgrqSFGpc9kucJRi1sOgGa0aZKdp7sOqs4/H5 GNPnr2Qtm9yOcRuMJYCAEyF+KJWyyVyHxUZI5G2gxVb9rNIzwXoGUMYo6kfOrR6X6eT2 FFIQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:dkim-signature:delivered-to; bh=hkqN7/+uzZsCktpGj2EsvDIsk06tk3tD8L1CtPYbtKk=; b=nmNHca/tvYbKvJmo+jCcthjnQm2bYsrRE1jrr8BeUo6PCGwJgyDroSa7kkrk5MIkcP OrvDLn8FPiIEXFiYAsEDH9J9q7FxpLDcQBoQ234JdjPls0nnW5B7JMlTATt0zROzdcyr Rk/AXEI8bMe7Sae8klq0bVn03/w1VQI+o0xJmSs+pTfPbVPq/NbM8r5NOIbryM+1aZNP 8j882GPPCtEwky6CQJA21xoMFhjJofUGCCbxIgA21mpx+e+gsd24EXHHypZvjih9BWLy h/b/lZYFUOqu8pZY97n2EC8g8Kq1b86JIo9rTHhEcmCt5x16tOTY+B2Y4j4ZlIVNlBRl 9G0w== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b=KCEQqvjf; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id di3si12962457ejc.728.2021.08.30.04.14.37; Mon, 30 Aug 2021 04:14:42 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b=KCEQqvjf; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id C52A8689FB2; Mon, 30 Aug 2021 14:14:22 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-lf1-f54.google.com (mail-lf1-f54.google.com [209.85.167.54]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id CC6C7680701 for ; Mon, 30 Aug 2021 14:14:14 +0300 (EEST) Received: by mail-lf1-f54.google.com with SMTP id k5so30490673lfu.4 for ; Mon, 30 Aug 2021 04:14:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=XKgmxc6g/DRYRjA5QTbG7icC8bSm/WHkm3BhuLtM900=; b=KCEQqvjftodOA9wEYxRfuLNzEW93lqhYeXyLlKKRiyU/NxVLXZJtZ1T+teBDlGlau9 WX+WOn5w4twxvKKh+RQX+WD1rrgWDD8Oin6ZEUrI5snGIbrs6zREIFAIqmuh1zSAD3lP 6q2JRGaynKwp+99QQLAX2j2ZpKDfXIiS8BTMAPwBuC6qc5vqgatUj1Ay4yPuje9v6h0O wJ6tHPEox0htmIT0Yfkh5mV3iVRvJZn7a1+Im1o1aez91rpSqmN8Fd4aQLL4JNLWL8dt m0s77LQsZPso/ydICLsFcGN/Uaw8sPKlt/PJ6y/9HO4NAVnAwRQoWXbWFJeMt8hcKpjd GmlQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=XKgmxc6g/DRYRjA5QTbG7icC8bSm/WHkm3BhuLtM900=; b=PXQt0CatW/fqkXWwIXXm6y9Y61QCyJ6UblKPfy3Zw5MXfh+K1UKiP7dBcNlPBLvHBR IbzZpBY2IFeb6thIiF+6BulEiKajuIOi3hLAq9SY76wprKYZUBpVtxtqwFzadsKyoxpi Jrhe3cvfvmb4dQQlzpm0QMPaVew6q9G2/l4xI/r0uvuC2O/X8WhwfNIgvz1nvFX7OctZ 6XLOjzYtoCN08KQTlrCR5cRLkXRn+CM73i2njYUH44tuO0KYOGCVNGU1Rl2IVDEFaZ9C m1dXVCV8H2tRhIpC6plx3+D4MXFQ0sRYewHkLUALG9wGGstTlArauQgKgukQnopMVmxz mHmg== X-Gm-Message-State: AOAM532NbcQg6Z6dmooeWwuhK8zKa+/+C40BIDvK2b+VfN7+bxD+35bw rKOJaw9XSwH/K3Z0MTparGgV3rqIFEY= X-Received: by 2002:a05:6512:158e:: with SMTP id bp14mr17349414lfb.509.1630322053768; Mon, 30 Aug 2021 04:14:13 -0700 (PDT) Received: from localhost.localdomain (91-159-194-103.elisa-laajakaista.fi. [91.159.194.103]) by smtp.gmail.com with ESMTPSA id bt29sm1366286lfb.4.2021.08.30.04.14.13 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 30 Aug 2021 04:14:13 -0700 (PDT) From: =?utf-8?q?Jan_Ekstr=C3=B6m?= To: ffmpeg-devel@ffmpeg.org Date: Mon, 30 Aug 2021 14:14:06 +0300 Message-Id: <20210830111407.5655-2-jeebjp@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210830111407.5655-1-jeebjp@gmail.com> References: <20210830111407.5655-1-jeebjp@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 1/2] avformat/{isom, mov, movenc}: add support for CMAF DASH roles X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 2hBCcle/3iST From: Jan Ekström This information is coded in a standard MP4 KindBox and utilizes the scheme and values as per the DASH role scheme defined in MPEG-DASH. Other schemes are technically allowed, but where multiple schemes define the same concepts, the DASH scheme should be utilized. Such flagging is additionally utilized by the DASH-IF CMAF ingest specification, enabling an encoder to inform the following component of the roles of the incoming media streams. A test is added for this functionality in a similar manner to the matroska test. Signed-off-by: Jan Ekström --- libavformat/isom.c | 19 +++++ libavformat/isom.h | 12 +++ libavformat/mov.c | 67 +++++++++++++++ libavformat/movenc.c | 51 ++++++++++++ tests/fate/mov.mak | 9 +++ .../ref/fate/mov-mp4-disposition-mpegts-remux | 81 +++++++++++++++++++ 6 files changed, 239 insertions(+) create mode 100644 tests/ref/fate/mov-mp4-disposition-mpegts-remux diff --git a/libavformat/isom.c b/libavformat/isom.c index 4df5440023..300ba927c2 100644 --- a/libavformat/isom.c +++ b/libavformat/isom.c @@ -430,3 +430,22 @@ void ff_mov_write_chan(AVIOContext *pb, int64_t channel_layout) } avio_wb32(pb, 0); // mNumberChannelDescriptions } + +static const struct MP4TrackKindValueMapping dash_role_map[] = { + { AV_DISPOSITION_HEARING_IMPAIRED|AV_DISPOSITION_CAPTIONS, + "caption" }, + { AV_DISPOSITION_COMMENT, + "commentary" }, + { AV_DISPOSITION_VISUAL_IMPAIRED|AV_DISPOSITION_DESCRIPTIONS, + "description" }, + { AV_DISPOSITION_DUB, + "dub" }, + { AV_DISPOSITION_FORCED, + "forced-subtitle" }, + { 0, NULL } +}; + +const struct MP4TrackKindMapping ff_mov_track_kind_table[] = { + { "urn:mpeg:dash:role:2011", dash_role_map }, + { 0, NULL } +}; diff --git a/libavformat/isom.h b/libavformat/isom.h index 34a58c79b7..c62fcf2bfe 100644 --- a/libavformat/isom.h +++ b/libavformat/isom.h @@ -390,4 +390,16 @@ static inline enum AVCodecID ff_mov_get_lpcm_codec_id(int bps, int flags) #define MOV_ISMV_TTML_TAG MKTAG('d', 'f', 'x', 'p') #define MOV_MP4_TTML_TAG MKTAG('s', 't', 'p', 'p') +struct MP4TrackKindValueMapping { + int disposition; + const char *value; +}; + +struct MP4TrackKindMapping { + const char *scheme_uri; + const struct MP4TrackKindValueMapping *value_maps; +}; + +extern const struct MP4TrackKindMapping ff_mov_track_kind_table[]; + #endif /* AVFORMAT_ISOM_H */ diff --git a/libavformat/mov.c b/libavformat/mov.c index c5583e07c7..4330736fa3 100644 --- a/libavformat/mov.c +++ b/libavformat/mov.c @@ -28,6 +28,7 @@ #include #include "libavutil/attributes.h" +#include "libavutil/avstring.h" #include "libavutil/channel_layout.h" #include "libavutil/internal.h" #include "libavutil/intreadwrite.h" @@ -6835,6 +6836,71 @@ static int mov_read_dvcc_dvvc(MOVContext *c, AVIOContext *pb, MOVAtom atom) return 0; } +static int mov_read_kind(MOVContext *c, AVIOContext *pb, MOVAtom atom) +{ + AVFormatContext *ctx = c->fc; + AVStream *st = NULL; + char scheme_str[1024] = { 0 }, value_str[1024] = { 0 }; + int scheme_str_len = 0, value_str_len = 0; + int version, flags; + int64_t size = atom.size; + + if (atom.size < 6) + // 4 bytes for version + flags, 2x 1 byte for null + return AVERROR_INVALIDDATA; + + if (c->fc->nb_streams < 1) + return 0; + st = c->fc->streams[c->fc->nb_streams-1]; + + version = avio_r8(pb); + flags = avio_rb24(pb); + size -= 4; + + if (version != 0 || flags != 0) { + av_log(ctx, AV_LOG_ERROR, + "Unsupported 'kind' box with version %d, flags: %x", + version, flags); + return AVERROR_INVALIDDATA; + } + + scheme_str_len = avio_get_str(pb, size, scheme_str, sizeof(scheme_str)); + if (scheme_str_len < 0) + return AVERROR_INVALIDDATA; + + if (scheme_str_len == size) + // we need to have another string, even if nullptr + return AVERROR_INVALIDDATA; + + size -= scheme_str_len; + + value_str_len = avio_get_str(pb, size, value_str, sizeof(value_str)); + if (value_str_len < 0) + return AVERROR_INVALIDDATA; + + av_log(ctx, AV_LOG_TRACE, + "%s stream %d KindBox(scheme: %s, value: %s)\n", + av_get_media_type_string(st->codecpar->codec_type), + st->index, + scheme_str, value_str); + + for (int i = 0; ff_mov_track_kind_table[i].scheme_uri; i++) { + const struct MP4TrackKindMapping map = ff_mov_track_kind_table[i]; + if (!av_strstart(scheme_str, map.scheme_uri, NULL)) + continue; + + for (int j = 0; map.value_maps[j].disposition; j++) { + const struct MP4TrackKindValueMapping value_map = map.value_maps[j]; + if (!av_strstart(value_str, value_map.value, NULL)) + continue; + + st->disposition |= value_map.disposition; + } + } + + return 0; +} + static const MOVParseTableEntry mov_default_parse_table[] = { { MKTAG('A','C','L','R'), mov_read_aclr }, { MKTAG('A','P','R','G'), mov_read_avid }, @@ -6932,6 +6998,7 @@ static const MOVParseTableEntry mov_default_parse_table[] = { { MKTAG('c','l','l','i'), mov_read_clli }, { MKTAG('d','v','c','C'), mov_read_dvcc_dvvc }, { MKTAG('d','v','v','C'), mov_read_dvcc_dvvc }, +{ MKTAG('k','i','n','d'), mov_read_kind }, { 0, NULL } }; diff --git a/libavformat/movenc.c b/libavformat/movenc.c index d5929e6395..4070fc9ef7 100644 --- a/libavformat/movenc.c +++ b/libavformat/movenc.c @@ -3329,6 +3329,52 @@ static int mov_write_track_metadata(AVIOContext *pb, AVStream *st, return update_size(pb, pos); } +static int mov_write_track_kind(AVIOContext *pb, const char *scheme_uri, + const char *value) +{ + int64_t pos = avio_tell(pb); + + /* Box|FullBox basics */ + avio_wb32(pb, 0); /* size placeholder */ + ffio_wfourcc(pb, (const unsigned char *)"kind"); + avio_w8(pb, 0); /* version = 0 */ + avio_wb24(pb, 0); /* flags = 0 */ + + /* Required null-terminated scheme URI */ + avio_write(pb, (const unsigned char *)scheme_uri, + strlen(scheme_uri)); + avio_w8(pb, 0); + + /* Optional value string */ + if (value && value[0]) + avio_write(pb, (const unsigned char *)value, + strlen(value)); + + avio_w8(pb, 0); + + return update_size(pb, pos); +} + +static int mov_write_track_kinds(AVIOContext *pb, AVStream *st) +{ + int ret = AVERROR_BUG; + + for (int i = 0; ff_mov_track_kind_table[i].scheme_uri; i++) { + const struct MP4TrackKindMapping map = ff_mov_track_kind_table[i]; + + for (int j = 0; map.value_maps[j].disposition; j++) { + const struct MP4TrackKindValueMapping value_map = map.value_maps[j]; + if (!(st->disposition & value_map.disposition)) + continue; + + if ((ret = mov_write_track_kind(pb, map.scheme_uri, value_map.value)) < 0) + return ret; + } + } + + return 0; +} + static int mov_write_track_udta_tag(AVIOContext *pb, MOVMuxContext *mov, AVStream *st) { @@ -3346,6 +3392,11 @@ static int mov_write_track_udta_tag(AVIOContext *pb, MOVMuxContext *mov, if (mov->mode & (MODE_MP4|MODE_MOV)) mov_write_track_metadata(pb_buf, st, "name", "title"); + if (mov->mode & MODE_MP4) { + if ((ret = mov_write_track_kinds(pb_buf, st)) < 0) + return ret; + } + if ((size = avio_get_dyn_buf(pb_buf, &buf)) > 0) { avio_wb32(pb, size + 8); ffio_wfourcc(pb, "udta"); diff --git a/tests/fate/mov.mak b/tests/fate/mov.mak index 0c9177aa81..5ca992e181 100644 --- a/tests/fate/mov.mak +++ b/tests/fate/mov.mak @@ -136,6 +136,15 @@ FATE_MOV_FFMPEG_FFPROBE-$(call ALLYES, FILE_PROTOCOL SRT_DEMUXER MOV_DEMUXER SUB fate-mov-mp4-ttml-stpp: CMD = transcode srt $(TARGET_SAMPLES)/sub/SubRip_capability_tester.srt mp4 "-map 0:s -c:s ttml -time_base:s 1:1000" "-map 0 -c copy" "" "-of json -show_entries packet:stream=index,codec_type,codec_tag_string,codec_tag,codec_name,time_base,start_time,duration_ts,duration,nb_frames,nb_read_packets:stream_tags" fate-mov-mp4-ttml-dfxp: CMD = transcode srt $(TARGET_SAMPLES)/sub/SubRip_capability_tester.srt mp4 "-map 0:s -c:s ttml -time_base:s 1:1000 -tag:s dfxp -strict unofficial" "-map 0 -c copy" "" "-of json -show_entries packet:stream=index,codec_type,codec_tag_string,codec_tag,codec_name,time_base,start_time,duration_ts,duration,nb_frames,nb_read_packets:stream_tags" +# Resulting remux should have: +# 1. first audio stream with AV_DISPOSITION_HEARING_IMPAIRED +# 2. second audio stream with AV_DISPOSITION_VISUAL_IMPAIRED | DESCRIPTIONS +FATE_MOV_FFMPEG_FFPROBE-$(call ALLYES, FILE_PROTOCOL PIPE_PROTOCOL \ + MPEGTS_DEMUXER MOV_DEMUXER AC3_DECODER \ + MP4_MUXER FRAMECRC_MUXER ) \ + += fate-mov-mp4-disposition-mpegts-remux +fate-mov-mp4-disposition-mpegts-remux: CMD = transcode mpegts $(TARGET_SAMPLES)/mpegts/pmtchange.ts mp4 "-map 0:1 -map 0:2 -c copy -disposition:a:0 +hearing_impaired" "-map 0 -c copy" "" "-of json -show_entries stream_disposition:stream=index" + FATE_SAMPLES_FFMPEG_FFPROBE += $(FATE_MOV_FFMPEG_FFPROBE-yes) fate-mov: $(FATE_MOV) $(FATE_MOV_FFPROBE) $(FATE_MOV_FASTSTART) $(FATE_MOV_FFMPEG_FFPROBE-yes) diff --git a/tests/ref/fate/mov-mp4-disposition-mpegts-remux b/tests/ref/fate/mov-mp4-disposition-mpegts-remux new file mode 100644 index 0000000000..2df48a0b0d --- /dev/null +++ b/tests/ref/fate/mov-mp4-disposition-mpegts-remux @@ -0,0 +1,81 @@ +e9a12470bc7645cebca985a9f3762c6c *tests/data/fate/mov-mp4-disposition-mpegts-remux.mp4 +5705 tests/data/fate/mov-mp4-disposition-mpegts-remux.mp4 +#tb 0: 1/48000 +#media_type 0: audio +#codec_id 0: ac3 +#sample_rate 0: 48000 +#channel_layout 0: 3 +#channel_layout_name 0: stereo +#tb 1: 1/48000 +#media_type 1: audio +#codec_id 1: ac3 +#sample_rate 1: 48000 +#channel_layout 1: 3 +#channel_layout_name 1: stereo +1, 0, 0, 1536, 768, 0xa63778d4, S=1, 4 +1, 1536, 1536, 1536, 768, 0x7d577f3f +0, 3072, 3072, 1536, 768, 0xc2867884, S=1, 4 +1, 3072, 3072, 1536, 768, 0xd86b7c8f +0, 4608, 4608, 1536, 690, 0xa2714bf3 +1, 4608, 4608, 1536, 626, 0x09f4382f +{ + "programs": [ + + ], + "streams": [ + { + "index": 0, + "disposition": { + "default": 1, + "dub": 0, + "original": 0, + "comment": 0, + "lyrics": 0, + "karaoke": 0, + "forced": 0, + "hearing_impaired": 1, + "visual_impaired": 0, + "clean_effects": 0, + "attached_pic": 0, + "timed_thumbnails": 0, + "captions": 1, + "descriptions": 0, + "metadata": 0, + "dependent": 0, + "still_image": 0 + }, + "side_data_list": [ + { + + } + ] + }, + { + "index": 1, + "disposition": { + "default": 0, + "dub": 0, + "original": 0, + "comment": 0, + "lyrics": 0, + "karaoke": 0, + "forced": 0, + "hearing_impaired": 0, + "visual_impaired": 1, + "clean_effects": 0, + "attached_pic": 0, + "timed_thumbnails": 0, + "captions": 0, + "descriptions": 1, + "metadata": 0, + "dependent": 0, + "still_image": 0 + }, + "side_data_list": [ + { + + } + ] + } + ] +} From patchwork Mon Aug 30 11:14:07 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Jan_Ekstr=C3=B6m?= X-Patchwork-Id: 29876 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6602:2a4a:0:0:0:0 with SMTP id k10csp3852913iov; Mon, 30 Aug 2021 04:14:53 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxIH7swcIwbIji6er4Q/k06i+TYRVHOnBOQT+1TgKDS2vvVqaVmCJKN7kfyz8Zg4/tBUy53 X-Received: by 2002:a05:6402:2751:: with SMTP id z17mr23276300edd.290.1630322093427; Mon, 30 Aug 2021 04:14:53 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1630322093; cv=none; d=google.com; s=arc-20160816; b=tdBMPn1fmY+bjwumSEfEDoFOLaVcV+l+JKET4bub06tQcWFKV0uqPcYRt+hX4QMlfS CYKu/gd3FcmM0WailTpAWXq/ngaphiDfi+kBaJw+U72rWborKgXDlcrKJXzoUcSrAbvG 1uS5ynIseo1U6cG2Ur+EuYaZ36SNKOFHXpxZUfeHDmZ+sZIWLYhaHkN3q6mJDKGwAlgu YWwKkLKWCXfZzJorDsZ4vzIcqr0XkeAeAC67SXQ4cCB4hgkp7urZ5BAnOeBCZw+HTr/7 eO/3pwE6Ho5euQKTNDh+Yc8d8REVhtvEtKUu20/oyRcqwlc/rcSxnWWnTRjLRVWcHGgv 1FVw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:dkim-signature:delivered-to; bh=WKQ+mcP/RmzHC/4GHCA8kIgBidPOXB7lM/Dr7Gm2e0M=; b=q7VrwncOZo7tm5UM9g5wbIcEsThOUhSXAAFVBa/1h5W57LqsT1frjsDgnPWAMwBTdq pMwvNxAy4gALDkdAAPmi6sYL2+61trrug90IjryglT4u7BVTWYBRGLJnyjyMScb7iiEa GvzAP/I/JQ1hpgsLq4PbdjWmTkyxpv/j4/sUxL9JZSEwUSZbvyJS/S66sQM1jcrusrlo tnAnQPHnjzsV3VEJGmxpy4iAg2Sg2ljjeGLRMyYoCzSFtsLqtlepAhYJVCKnau9XAt2M c5NvaQ73p1SrkPP+hAtG7GtkXvhkvWwZwi1gL3Oi6lrI2qbM6tz0EcvFYqzhUjyXAx+F TJHQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b=kZ6N5Cog; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id jy1si8488557ejc.592.2021.08.30.04.14.48; Mon, 30 Aug 2021 04:14:53 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b=kZ6N5Cog; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id D2E27689F08; Mon, 30 Aug 2021 14:14:23 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-lj1-f179.google.com (mail-lj1-f179.google.com [209.85.208.179]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 32C21680701 for ; Mon, 30 Aug 2021 14:14:15 +0300 (EEST) Received: by mail-lj1-f179.google.com with SMTP id p15so25214627ljn.3 for ; Mon, 30 Aug 2021 04:14:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=WVUpJIfTlnaFIzqin84UFAtevkWJZ5d/dHxAHQ3QATQ=; b=kZ6N5Cog83fCjoi0cJPukvICFYW4+uwGzuLAsgRf+cIruisRxDMSxnEg25sF4tbznJ ySinNyTzaZOCD6lmZM9guVDabqr1kqdMAcG0Q3Q62vsfO4FeZ0rfRNzzhIwpOjpEz3Bk Ol9iMTS9Ut2bDraHOw4gHho7r8OBTLHJDdStXRZ3UJOxj2WUR3i5GWg/hcL18xCI6ZTX LgebpwSrQqaPuA+JGVHpGayNY6O0IFqpiwpuA8YxBq3/9Pya24+VzBwo5GwhXiaePSeF fOAu+r1os/5B3l3FWg/bv9F8vNygRKNc4pqR8lshB0KtMITKewJ9bxrcUZHs11VRbFXv wTFg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=WVUpJIfTlnaFIzqin84UFAtevkWJZ5d/dHxAHQ3QATQ=; b=pORHM6S0LqfICky33/R3GD3NgjuA7XZ3USTArhhfQ7PsR3oDUcLDBheR5/1PYD55Pa JD1RbEov0tjSWNR6/7z2CoTNtkasAKh5jKQyvhPt4O3FDj7vB7kjcqpzmduHiXgjORiN XKsnqdu3xF0E+oFnxOeOMQqAv+CoV+/j5cPuwxuOfX9TG4BwzM0egQYD6tHKi5U9FiAF RSiJDpA0pEApugzWxgTQGdz9BzKPZAvSNzBzDGjbxXBPJOxwZ1wlDw9DeqqDQeinTCVg tYtMnXhG2myIH4zN4sVFdtxYjfND3iTmnyaCKAfFbRs7kNrG7HQ2t4nTM9ryfRsM4aQD y9CA== X-Gm-Message-State: AOAM533DbPJev1Wlon/nRUli/UX4OiVee5VSJ0MKtObLEigo707G3RhX Y/hvGQWg9sa4Y/AUVv+Tagn8V/nvDTw= X-Received: by 2002:a2e:b53a:: with SMTP id z26mr19714623ljm.95.1630322054451; Mon, 30 Aug 2021 04:14:14 -0700 (PDT) Received: from localhost.localdomain (91-159-194-103.elisa-laajakaista.fi. [91.159.194.103]) by smtp.gmail.com with ESMTPSA id bt29sm1366286lfb.4.2021.08.30.04.14.13 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 30 Aug 2021 04:14:14 -0700 (PDT) From: =?utf-8?q?Jan_Ekstr=C3=B6m?= To: ffmpeg-devel@ffmpeg.org Date: Mon, 30 Aug 2021 14:14:07 +0300 Message-Id: <20210830111407.5655-3-jeebjp@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210830111407.5655-1-jeebjp@gmail.com> References: <20210830111407.5655-1-jeebjp@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 2/2] avformat/{isom, movenc}: add kind box compatibility mode for Unified Origin X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: A24INo08LKBz From: Jan Ekström Unfortunately the current production versions of this software do not 100% adhere to the CMAF specification, and have decided to utilize the HTML5 media track identifier for audio descriptions. This way the default mode of operation is according to the CMAF specification, but it is also possible to output streams with which this piece of software is capable of interoperating with. Signed-off-by: Jan Ekström --- libavformat/isom.c | 23 ++++-- libavformat/isom.h | 6 ++ libavformat/movenc.c | 12 ++- libavformat/movenc.h | 2 + tests/fate/mov.mak | 8 ++ ...p4-disposition-unified-origin-mpegts-remux | 81 +++++++++++++++++++ 6 files changed, 124 insertions(+), 8 deletions(-) create mode 100644 tests/ref/fate/mov-mp4-disposition-unified-origin-mpegts-remux diff --git a/libavformat/isom.c b/libavformat/isom.c index 300ba927c2..fb8ad3d824 100644 --- a/libavformat/isom.c +++ b/libavformat/isom.c @@ -433,19 +433,32 @@ void ff_mov_write_chan(AVIOContext *pb, int64_t channel_layout) static const struct MP4TrackKindValueMapping dash_role_map[] = { { AV_DISPOSITION_HEARING_IMPAIRED|AV_DISPOSITION_CAPTIONS, - "caption" }, + "caption", + KindWritingModeCMAF|KindWritingModeUnifiedOrigin }, { AV_DISPOSITION_COMMENT, - "commentary" }, + "commentary", + KindWritingModeCMAF|KindWritingModeUnifiedOrigin }, { AV_DISPOSITION_VISUAL_IMPAIRED|AV_DISPOSITION_DESCRIPTIONS, - "description" }, + "description", + KindWritingModeCMAF }, { AV_DISPOSITION_DUB, - "dub" }, + "dub", + KindWritingModeCMAF|KindWritingModeUnifiedOrigin }, { AV_DISPOSITION_FORCED, - "forced-subtitle" }, + "forced-subtitle", + KindWritingModeCMAF|KindWritingModeUnifiedOrigin }, + { 0, NULL } +}; + +static const struct MP4TrackKindValueMapping html_kind_map[] = { + { AV_DISPOSITION_VISUAL_IMPAIRED|AV_DISPOSITION_DESCRIPTIONS, + "main-desc", + KindWritingModeUnifiedOrigin }, { 0, NULL } }; const struct MP4TrackKindMapping ff_mov_track_kind_table[] = { { "urn:mpeg:dash:role:2011", dash_role_map }, + { "about:html-kind", html_kind_map }, { 0, NULL } }; diff --git a/libavformat/isom.h b/libavformat/isom.h index c62fcf2bfe..1252fc6603 100644 --- a/libavformat/isom.h +++ b/libavformat/isom.h @@ -390,9 +390,15 @@ static inline enum AVCodecID ff_mov_get_lpcm_codec_id(int bps, int flags) #define MOV_ISMV_TTML_TAG MKTAG('d', 'f', 'x', 'p') #define MOV_MP4_TTML_TAG MKTAG('s', 't', 'p', 'p') +enum MP4TrackKindWritingMode { + KindWritingModeCMAF = (1 << 0), + KindWritingModeUnifiedOrigin = (1 << 1), +}; + struct MP4TrackKindValueMapping { int disposition; const char *value; + uint32_t writing_modes; }; struct MP4TrackKindMapping { diff --git a/libavformat/movenc.c b/libavformat/movenc.c index 4070fc9ef7..baaae7d3ad 100644 --- a/libavformat/movenc.c +++ b/libavformat/movenc.c @@ -111,6 +111,9 @@ static const AVOption options[] = { { "pts", NULL, 0, AV_OPT_TYPE_CONST, {.i64 = MOV_PRFT_SRC_PTS}, 0, 0, AV_OPT_FLAG_ENCODING_PARAM, "prft"}, { "empty_hdlr_name", "write zero-length name string in hdlr atoms within mdia and minf atoms", offsetof(MOVMuxContext, empty_hdlr_name), AV_OPT_TYPE_BOOL, {.i64 = 0}, 0, 1, AV_OPT_FLAG_ENCODING_PARAM}, { "movie_timescale", "set movie timescale", offsetof(MOVMuxContext, movie_timescale), AV_OPT_TYPE_INT, {.i64 = MOV_TIMESCALE}, 1, INT_MAX, AV_OPT_FLAG_ENCODING_PARAM}, + { "kind_writing_mode", "set kind box writing mode", offsetof(MOVMuxContext, kind_writing_mode), AV_OPT_TYPE_INT, {.i64 = KindWritingModeCMAF}, KindWritingModeCMAF, KindWritingModeUnifiedOrigin, AV_OPT_FLAG_ENCODING_PARAM, "kind_writing_mode"}, + { "cmaf", "CMAF writing mode", 0, AV_OPT_TYPE_CONST, {.i64 = KindWritingModeCMAF}, INT_MIN, INT_MAX, AV_OPT_FLAG_ENCODING_PARAM, "kind_writing_mode"}, + { "unified_origin", "Compatibility mode for Unified Origin (all DASH except for audio description)", 0, AV_OPT_TYPE_CONST, {.i64 = KindWritingModeUnifiedOrigin}, INT_MIN, INT_MAX, AV_OPT_FLAG_ENCODING_PARAM, "kind_writing_mode"}, { NULL }, }; @@ -3355,7 +3358,8 @@ static int mov_write_track_kind(AVIOContext *pb, const char *scheme_uri, return update_size(pb, pos); } -static int mov_write_track_kinds(AVIOContext *pb, AVStream *st) +static int mov_write_track_kinds(AVIOContext *pb, AVStream *st, + enum MP4TrackKindWritingMode mode) { int ret = AVERROR_BUG; @@ -3364,7 +3368,8 @@ static int mov_write_track_kinds(AVIOContext *pb, AVStream *st) for (int j = 0; map.value_maps[j].disposition; j++) { const struct MP4TrackKindValueMapping value_map = map.value_maps[j]; - if (!(st->disposition & value_map.disposition)) + if (!(st->disposition & value_map.disposition) || + !(value_map.writing_modes & mode)) continue; if ((ret = mov_write_track_kind(pb, map.scheme_uri, value_map.value)) < 0) @@ -3393,7 +3398,8 @@ static int mov_write_track_udta_tag(AVIOContext *pb, MOVMuxContext *mov, mov_write_track_metadata(pb_buf, st, "name", "title"); if (mov->mode & MODE_MP4) { - if ((ret = mov_write_track_kinds(pb_buf, st)) < 0) + if ((ret = mov_write_track_kinds(pb_buf, st, + mov->kind_writing_mode)) < 0) return ret; } diff --git a/libavformat/movenc.h b/libavformat/movenc.h index 40077b1afe..e02a086b1f 100644 --- a/libavformat/movenc.h +++ b/libavformat/movenc.h @@ -25,6 +25,7 @@ #define AVFORMAT_MOVENC_H #include "avformat.h" +#include "isom.h" #include "movenccenc.h" #include "libavcodec/packet_internal.h" @@ -242,6 +243,7 @@ typedef struct MOVMuxContext { MOVPrftBox write_prft; int empty_hdlr_name; int movie_timescale; + enum MP4TrackKindWritingMode kind_writing_mode; } MOVMuxContext; #define FF_MOV_FLAG_RTP_HINT (1 << 0) diff --git a/tests/fate/mov.mak b/tests/fate/mov.mak index 5ca992e181..20f085803b 100644 --- a/tests/fate/mov.mak +++ b/tests/fate/mov.mak @@ -145,6 +145,14 @@ FATE_MOV_FFMPEG_FFPROBE-$(call ALLYES, FILE_PROTOCOL PIPE_PROTOCOL \ += fate-mov-mp4-disposition-mpegts-remux fate-mov-mp4-disposition-mpegts-remux: CMD = transcode mpegts $(TARGET_SAMPLES)/mpegts/pmtchange.ts mp4 "-map 0:1 -map 0:2 -c copy -disposition:a:0 +hearing_impaired" "-map 0 -c copy" "" "-of json -show_entries stream_disposition:stream=index" +# Same as the previous test, but the audio disposition should now be tagged +# with the HTML5 media track identifier as opposed to the DASH identifier. +FATE_MOV_FFMPEG_FFPROBE-$(call ALLYES, FILE_PROTOCOL PIPE_PROTOCOL \ + MPEGTS_DEMUXER MOV_DEMUXER AC3_DECODER \ + MP4_MUXER FRAMECRC_MUXER ) \ + += fate-mov-mp4-disposition-unified-origin-mpegts-remux +fate-mov-mp4-disposition-unified-origin-mpegts-remux: CMD = transcode mpegts $(TARGET_SAMPLES)/mpegts/pmtchange.ts mp4 "-map 0:1 -map 0:2 -c copy -disposition:a:0 +hearing_impaired -kind_writing_mode unified_origin" "-map 0 -c copy" "" "-of json -show_entries stream_disposition:stream=index" + FATE_SAMPLES_FFMPEG_FFPROBE += $(FATE_MOV_FFMPEG_FFPROBE-yes) fate-mov: $(FATE_MOV) $(FATE_MOV_FFPROBE) $(FATE_MOV_FASTSTART) $(FATE_MOV_FFMPEG_FFPROBE-yes) diff --git a/tests/ref/fate/mov-mp4-disposition-unified-origin-mpegts-remux b/tests/ref/fate/mov-mp4-disposition-unified-origin-mpegts-remux new file mode 100644 index 0000000000..0242cffb9b --- /dev/null +++ b/tests/ref/fate/mov-mp4-disposition-unified-origin-mpegts-remux @@ -0,0 +1,81 @@ +99f1d34f8028c674cb10d2604a7f0117 *tests/data/fate/mov-mp4-disposition-unified-origin-mpegts-remux.mp4 +5695 tests/data/fate/mov-mp4-disposition-unified-origin-mpegts-remux.mp4 +#tb 0: 1/48000 +#media_type 0: audio +#codec_id 0: ac3 +#sample_rate 0: 48000 +#channel_layout 0: 3 +#channel_layout_name 0: stereo +#tb 1: 1/48000 +#media_type 1: audio +#codec_id 1: ac3 +#sample_rate 1: 48000 +#channel_layout 1: 3 +#channel_layout_name 1: stereo +1, 0, 0, 1536, 768, 0xa63778d4, S=1, 4 +1, 1536, 1536, 1536, 768, 0x7d577f3f +0, 3072, 3072, 1536, 768, 0xc2867884, S=1, 4 +1, 3072, 3072, 1536, 768, 0xd86b7c8f +0, 4608, 4608, 1536, 690, 0xa2714bf3 +1, 4608, 4608, 1536, 626, 0x09f4382f +{ + "programs": [ + + ], + "streams": [ + { + "index": 0, + "disposition": { + "default": 1, + "dub": 0, + "original": 0, + "comment": 0, + "lyrics": 0, + "karaoke": 0, + "forced": 0, + "hearing_impaired": 1, + "visual_impaired": 0, + "clean_effects": 0, + "attached_pic": 0, + "timed_thumbnails": 0, + "captions": 1, + "descriptions": 0, + "metadata": 0, + "dependent": 0, + "still_image": 0 + }, + "side_data_list": [ + { + + } + ] + }, + { + "index": 1, + "disposition": { + "default": 0, + "dub": 0, + "original": 0, + "comment": 0, + "lyrics": 0, + "karaoke": 0, + "forced": 0, + "hearing_impaired": 0, + "visual_impaired": 1, + "clean_effects": 0, + "attached_pic": 0, + "timed_thumbnails": 0, + "captions": 0, + "descriptions": 1, + "metadata": 0, + "dependent": 0, + "still_image": 0 + }, + "side_data_list": [ + { + + } + ] + } + ] +}