From patchwork Mon Sep 20 15:00:48 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Jan_Ekstr=C3=B6m?= X-Patchwork-Id: 30381 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a6b:6506:0:0:0:0:0 with SMTP id z6csp1532486iob; Mon, 20 Sep 2021 08:02:43 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyo1qt08eq/d2FYIMPv4DpP3WcJeUXrp+80ZzlLx3rA7/wJ5aYVI2DZdC9ZFa3amzwlW6ty X-Received: by 2002:a05:600c:4f55:: with SMTP id m21mr22464625wmq.149.1632150162770; Mon, 20 Sep 2021 08:02:42 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1632150162; cv=none; d=google.com; s=arc-20160816; b=KzKCEQIKjPazgi8AJ3l35hTfuVCsak37blAHOl6AzvozH40e3trLHx2TO9PIFjq6GY zBTttPA0ipv7+QyDQVZNlLSdwB5k2aS76nSF/uZbzrvQndiBIhNXfrAcTEpEo02dfcnN 0wq2rKvFEpfUtB+ysOHIVBAyyo3hIqDDQQyP85NFnyG7FqWVFDK0d+7Isn6LFRBgcB3P VcMBJw5eO5rwhyff3ZFPyOyNKMYzmdjXSoBCj5clJ5Ws4BJ4Mh1ESu3pxzAAAEZQv5s7 8tvkmbCPvwDOtL5BPOq4fZeBFfJZN9n/ieADKjZ/dfyERaJH89lnUytZK1FLOuDOqS68 DWVQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:dkim-signature:delivered-to; bh=zJD+RGTiGzOjBI4Nbwv5zmM56LzxJ9NJ8G3dHVlozm0=; b=LqGc8cVn0lPXsMNaD7zi5f9mc1Zzg44HL0kGOxdq3YBTjM1jRKFef8SI4Hp1u0FHFQ Hytg4aNVhIOBOy715ZNcIF4VsCo4dQAFbZE9k3/poGvr6Txjy83xeq2xPY1C8c+YMIjL Isku5BOfzbUx3gXv9B8MK2yvDUU8F/jdMOdxOb0mKEJE6pTLN5+ie1AIBm1WdmOH6cMy TAsS9qsAc4pFl+0h5u6LWDHFoli9PrrUP4IggObPzAbWT2A+wX+ovg1FT59EXNtGwiUp 8qtuuHz0hTlVF/lfe+Hfnj09M1DBpTy1kQ9/tIMzO5rjmWPvSQpWlqrPWVv2Zy8dxvtP Hx/A== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=S6UjFlOv; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id r6si16867173edq.273.2021.09.20.08.02.37; Mon, 20 Sep 2021 08:02:42 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=S6UjFlOv; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 4A1A968AED9; Mon, 20 Sep 2021 18:02:24 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-lf1-f53.google.com (mail-lf1-f53.google.com [209.85.167.53]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 4655A68AECE for ; Mon, 20 Sep 2021 18:02:16 +0300 (EEST) Received: by mail-lf1-f53.google.com with SMTP id g41so36261881lfv.1 for ; Mon, 20 Sep 2021 08:02:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=6wmVhyCgmMhbf8vSCUWJY2M3I4xRZQwtXo2/e3ddpl4=; b=S6UjFlOvKaxLKX5qGlol2ReFD9j+p3NFKavO91EH89Dv0DwfOU1/eGUIx5Vd1nVHHT b8DiVXo4uz4c0tG/nES2FokCyZj2Ae66e7CbFFP8tn9/xLB4I/X4n2sPBSPAydI2IKw0 8mCaevl5xO0Won3PL0mN9B0tWEwZ/tTC8ynYYA4GN4lMvyLE0yO/VFiILg8RU30/+EAD IcllKPUY/sZ30uy5RsgrJOsOMmIzjrKiiG6c3RSTMkzZjP87tXdGB0wE+QliPf9qV1k2 xRFYZkZOAV4t9+tSIVyqjzLQM50X/MDRrK3UkrIQNvr5uA7P8cYzpLA5+SZGUE+tS8xu KfqQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=6wmVhyCgmMhbf8vSCUWJY2M3I4xRZQwtXo2/e3ddpl4=; b=QBQ4UsLOraflQy5cdJd2X51KF/yonkld80lrjfeGPsJwlg8SR29rht/SP2D8Phpe3O y5Qg7ChEQ86f0r6iwZTqnCcxo9bbUJgmW4W8QiFThG6N06fGIE3bNmPxSLKivAahxnlZ rYxyZ+fSR2nvAo8MW59/L2dyNRbYb5PmJPuU7BFOzqksUoFPHOaM/hjkckFMFKt5AOEt dzf8j4qfok2UREo80NudrrMIGI2uwc+ExbsShLcKJwyc0LAzyVe70Jj4Cjw3KsONefPF 1QUYWzpyN10htChR3tagdy4zFBzUPqeWFV5YrFf+lv6fl61de2QWxV24vUkanLDbE74u Ud7Q== X-Gm-Message-State: AOAM532Tz2fqAswZlun6RrlKzwmatSAkN27dDZWTxzUn7mHCGgcHIeZo gUOpxgIW+6GFkIhjo0rJCHw5U4sC3P8= X-Received: by 2002:ac2:5387:: with SMTP id g7mr19068224lfh.541.1632150059322; Mon, 20 Sep 2021 08:00:59 -0700 (PDT) Received: from localhost.localdomain (91-159-194-103.elisa-laajakaista.fi. [91.159.194.103]) by smtp.gmail.com with ESMTPSA id v26sm1318300ljv.77.2021.09.20.08.00.58 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Sep 2021 08:00:58 -0700 (PDT) From: =?utf-8?q?Jan_Ekstr=C3=B6m?= To: ffmpeg-devel@ffmpeg.org Date: Mon, 20 Sep 2021 18:00:48 +0300 Message-Id: <20210920150048.8790-6-jeebjp@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210920150048.8790-1-jeebjp@gmail.com> References: <20210920150048.8790-1-jeebjp@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v3 5/5] avformat/{isom, movenc}: add kind box compatibility mode for Unified Origin X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: FxNq09M/K9iM From: Jan Ekström Unfortunately the current production versions of this software do not 100% adhere to the CMAF specification, and have decided to utilize the HTML5 media track identifier for audio descriptions. This way the default mode of operation is according to the CMAF specification, but it is also possible to output streams with which this piece of software is capable of interoperating with. Signed-off-by: Jan Ekström --- libavformat/isom.c | 23 ++++-- libavformat/isom.h | 7 ++ libavformat/movenc.c | 12 ++- libavformat/movenc.h | 2 + tests/fate/mov.mak | 8 ++ ...p4-disposition-unified-origin-mpegts-remux | 81 +++++++++++++++++++ 6 files changed, 125 insertions(+), 8 deletions(-) create mode 100644 tests/ref/fate/mov-mp4-disposition-unified-origin-mpegts-remux diff --git a/libavformat/isom.c b/libavformat/isom.c index 300ba927c2..512ffba651 100644 --- a/libavformat/isom.c +++ b/libavformat/isom.c @@ -433,19 +433,32 @@ void ff_mov_write_chan(AVIOContext *pb, int64_t channel_layout) static const struct MP4TrackKindValueMapping dash_role_map[] = { { AV_DISPOSITION_HEARING_IMPAIRED|AV_DISPOSITION_CAPTIONS, - "caption" }, + "caption", + KindWritingModeCMAF | KindWritingModeUnifiedOrigin }, { AV_DISPOSITION_COMMENT, - "commentary" }, + "commentary", + KindWritingModeCMAF | KindWritingModeUnifiedOrigin }, { AV_DISPOSITION_VISUAL_IMPAIRED|AV_DISPOSITION_DESCRIPTIONS, - "description" }, + "description", + KindWritingModeCMAF }, { AV_DISPOSITION_DUB, - "dub" }, + "dub", + KindWritingModeCMAF | KindWritingModeUnifiedOrigin }, { AV_DISPOSITION_FORCED, - "forced-subtitle" }, + "forced-subtitle", + KindWritingModeCMAF | KindWritingModeUnifiedOrigin }, + { 0, NULL } +}; + +static const struct MP4TrackKindValueMapping html_kind_map[] = { + { AV_DISPOSITION_VISUAL_IMPAIRED|AV_DISPOSITION_DESCRIPTIONS, + "main-desc", + KindWritingModeUnifiedOrigin }, { 0, NULL } }; const struct MP4TrackKindMapping ff_mov_track_kind_table[] = { { "urn:mpeg:dash:role:2011", dash_role_map }, + { "about:html-kind", html_kind_map }, { 0, NULL } }; diff --git a/libavformat/isom.h b/libavformat/isom.h index c62fcf2bfe..f0381e9a8e 100644 --- a/libavformat/isom.h +++ b/libavformat/isom.h @@ -390,9 +390,16 @@ static inline enum AVCodecID ff_mov_get_lpcm_codec_id(int bps, int flags) #define MOV_ISMV_TTML_TAG MKTAG('d', 'f', 'x', 'p') #define MOV_MP4_TTML_TAG MKTAG('s', 't', 'p', 'p') +enum MP4TrackKindWritingMode { + KindWritingModeCMAF = (1 << 0), + KindWritingModeUnifiedOrigin = (1 << 1), + KindWritingModeNB, +}; + struct MP4TrackKindValueMapping { int disposition; const char *value; + uint32_t writing_modes; }; struct MP4TrackKindMapping { diff --git a/libavformat/movenc.c b/libavformat/movenc.c index 0c4a24c313..1550bdd16f 100644 --- a/libavformat/movenc.c +++ b/libavformat/movenc.c @@ -111,6 +111,9 @@ static const AVOption options[] = { { "pts", NULL, 0, AV_OPT_TYPE_CONST, {.i64 = MOV_PRFT_SRC_PTS}, 0, 0, AV_OPT_FLAG_ENCODING_PARAM, "prft"}, { "empty_hdlr_name", "write zero-length name string in hdlr atoms within mdia and minf atoms", offsetof(MOVMuxContext, empty_hdlr_name), AV_OPT_TYPE_BOOL, {.i64 = 0}, 0, 1, AV_OPT_FLAG_ENCODING_PARAM}, { "movie_timescale", "set movie timescale", offsetof(MOVMuxContext, movie_timescale), AV_OPT_TYPE_INT, {.i64 = MOV_TIMESCALE}, 1, INT_MAX, AV_OPT_FLAG_ENCODING_PARAM}, + { "kind_writing_mode", "set kind box writing mode", offsetof(MOVMuxContext, kind_writing_mode), AV_OPT_TYPE_INT, {.i64 = KindWritingModeCMAF}, KindWritingModeCMAF, KindWritingModeNB - 1, AV_OPT_FLAG_ENCODING_PARAM, "kind_writing_mode"}, + { "cmaf", "CMAF writing mode", 0, AV_OPT_TYPE_CONST, {.i64 = KindWritingModeCMAF}, INT_MIN, INT_MAX, AV_OPT_FLAG_ENCODING_PARAM, "kind_writing_mode"}, + { "unified_origin", "Compatibility mode for Unified Origin (all DASH except for audio description)", 0, AV_OPT_TYPE_CONST, {.i64 = KindWritingModeUnifiedOrigin}, INT_MIN, INT_MAX, AV_OPT_FLAG_ENCODING_PARAM, "kind_writing_mode"}, { NULL }, }; @@ -3355,7 +3358,8 @@ static int mov_write_track_kind(AVIOContext *pb, const char *scheme_uri, return update_size(pb, pos); } -static int mov_write_track_kinds(AVIOContext *pb, AVStream *st) +static int mov_write_track_kinds(AVIOContext *pb, AVStream *st, + enum MP4TrackKindWritingMode mode) { int ret = AVERROR_BUG; @@ -3364,7 +3368,8 @@ static int mov_write_track_kinds(AVIOContext *pb, AVStream *st) for (int j = 0; map.value_maps[j].disposition; j++) { const struct MP4TrackKindValueMapping value_map = map.value_maps[j]; - if (!(st->disposition & value_map.disposition)) + if (!(st->disposition & value_map.disposition) || + !(value_map.writing_modes & mode)) continue; if ((ret = mov_write_track_kind(pb, map.scheme_uri, value_map.value)) < 0) @@ -3393,7 +3398,8 @@ static int mov_write_track_udta_tag(AVIOContext *pb, MOVMuxContext *mov, mov_write_track_metadata(pb_buf, st, "name", "title"); if (mov->mode & MODE_MP4) { - if ((ret = mov_write_track_kinds(pb_buf, st)) < 0) + if ((ret = mov_write_track_kinds(pb_buf, st, + mov->kind_writing_mode)) < 0) return ret; } diff --git a/libavformat/movenc.h b/libavformat/movenc.h index 40077b1afe..e02a086b1f 100644 --- a/libavformat/movenc.h +++ b/libavformat/movenc.h @@ -25,6 +25,7 @@ #define AVFORMAT_MOVENC_H #include "avformat.h" +#include "isom.h" #include "movenccenc.h" #include "libavcodec/packet_internal.h" @@ -242,6 +243,7 @@ typedef struct MOVMuxContext { MOVPrftBox write_prft; int empty_hdlr_name; int movie_timescale; + enum MP4TrackKindWritingMode kind_writing_mode; } MOVMuxContext; #define FF_MOV_FLAG_RTP_HINT (1 << 0) diff --git a/tests/fate/mov.mak b/tests/fate/mov.mak index 5ca992e181..20f085803b 100644 --- a/tests/fate/mov.mak +++ b/tests/fate/mov.mak @@ -145,6 +145,14 @@ FATE_MOV_FFMPEG_FFPROBE-$(call ALLYES, FILE_PROTOCOL PIPE_PROTOCOL \ += fate-mov-mp4-disposition-mpegts-remux fate-mov-mp4-disposition-mpegts-remux: CMD = transcode mpegts $(TARGET_SAMPLES)/mpegts/pmtchange.ts mp4 "-map 0:1 -map 0:2 -c copy -disposition:a:0 +hearing_impaired" "-map 0 -c copy" "" "-of json -show_entries stream_disposition:stream=index" +# Same as the previous test, but the audio disposition should now be tagged +# with the HTML5 media track identifier as opposed to the DASH identifier. +FATE_MOV_FFMPEG_FFPROBE-$(call ALLYES, FILE_PROTOCOL PIPE_PROTOCOL \ + MPEGTS_DEMUXER MOV_DEMUXER AC3_DECODER \ + MP4_MUXER FRAMECRC_MUXER ) \ + += fate-mov-mp4-disposition-unified-origin-mpegts-remux +fate-mov-mp4-disposition-unified-origin-mpegts-remux: CMD = transcode mpegts $(TARGET_SAMPLES)/mpegts/pmtchange.ts mp4 "-map 0:1 -map 0:2 -c copy -disposition:a:0 +hearing_impaired -kind_writing_mode unified_origin" "-map 0 -c copy" "" "-of json -show_entries stream_disposition:stream=index" + FATE_SAMPLES_FFMPEG_FFPROBE += $(FATE_MOV_FFMPEG_FFPROBE-yes) fate-mov: $(FATE_MOV) $(FATE_MOV_FFPROBE) $(FATE_MOV_FASTSTART) $(FATE_MOV_FFMPEG_FFPROBE-yes) diff --git a/tests/ref/fate/mov-mp4-disposition-unified-origin-mpegts-remux b/tests/ref/fate/mov-mp4-disposition-unified-origin-mpegts-remux new file mode 100644 index 0000000000..0242cffb9b --- /dev/null +++ b/tests/ref/fate/mov-mp4-disposition-unified-origin-mpegts-remux @@ -0,0 +1,81 @@ +99f1d34f8028c674cb10d2604a7f0117 *tests/data/fate/mov-mp4-disposition-unified-origin-mpegts-remux.mp4 +5695 tests/data/fate/mov-mp4-disposition-unified-origin-mpegts-remux.mp4 +#tb 0: 1/48000 +#media_type 0: audio +#codec_id 0: ac3 +#sample_rate 0: 48000 +#channel_layout 0: 3 +#channel_layout_name 0: stereo +#tb 1: 1/48000 +#media_type 1: audio +#codec_id 1: ac3 +#sample_rate 1: 48000 +#channel_layout 1: 3 +#channel_layout_name 1: stereo +1, 0, 0, 1536, 768, 0xa63778d4, S=1, 4 +1, 1536, 1536, 1536, 768, 0x7d577f3f +0, 3072, 3072, 1536, 768, 0xc2867884, S=1, 4 +1, 3072, 3072, 1536, 768, 0xd86b7c8f +0, 4608, 4608, 1536, 690, 0xa2714bf3 +1, 4608, 4608, 1536, 626, 0x09f4382f +{ + "programs": [ + + ], + "streams": [ + { + "index": 0, + "disposition": { + "default": 1, + "dub": 0, + "original": 0, + "comment": 0, + "lyrics": 0, + "karaoke": 0, + "forced": 0, + "hearing_impaired": 1, + "visual_impaired": 0, + "clean_effects": 0, + "attached_pic": 0, + "timed_thumbnails": 0, + "captions": 1, + "descriptions": 0, + "metadata": 0, + "dependent": 0, + "still_image": 0 + }, + "side_data_list": [ + { + + } + ] + }, + { + "index": 1, + "disposition": { + "default": 0, + "dub": 0, + "original": 0, + "comment": 0, + "lyrics": 0, + "karaoke": 0, + "forced": 0, + "hearing_impaired": 0, + "visual_impaired": 1, + "clean_effects": 0, + "attached_pic": 0, + "timed_thumbnails": 0, + "captions": 0, + "descriptions": 1, + "metadata": 0, + "dependent": 0, + "still_image": 0 + }, + "side_data_list": [ + { + + } + ] + } + ] +}