From patchwork Mon Jul 26 13:41:02 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Jan_Ekstr=C3=B6m?= X-Patchwork-Id: 29056 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a5d:965a:0:0:0:0:0 with SMTP id d26csp3802349ios; Mon, 26 Jul 2021 06:47:40 -0700 (PDT) X-Google-Smtp-Source: ABdhPJx7V+uKeUqT7b4VJbGe5QF8rMGeZgoz0R6JO+4UbTBZXKGafQtXwEMmsr3uMTjpkXdhlEqp X-Received: by 2002:a05:6402:35ca:: with SMTP id z10mr21950690edc.159.1627307260486; Mon, 26 Jul 2021 06:47:40 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1627307260; cv=none; d=google.com; s=arc-20160816; b=T9akhK5rgQZWZIlCIOVhLxCFgiBDpBcTzaUR9VK+8V21lX/ytdNlKSkSET0p8gFfB0 xe4j7rnsGRdxJAsRGHqM/rd6i70qt/LnL2Tu3QfNTcEsPFopp32mi2pnsp2m4x6Pi+uy /1HFUyHIQm1BF/sY2x80pIYy/nlpS2+CPDXvgKDmHAkcSzjLx1I7Eo7DJniDrnMxome6 r4QR28dch6GlM5KthwJBRCxr/2X3lRgbZJDZTWejLtLzAwsP30aI8+RsO/0fILijcu73 djLrP1iyCNqYEUaffZaLPhlBuo9EYO2N27T0HRjU+2tTdmPYYgPkNXRc+sAewLo2H9Zy upOg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:message-id:date:to:from :dkim-signature:delivered-to; bh=tspFIcel8AAzZVn2gxN9fyUJdDQhk58xCzdIWMwqRNM=; b=iEikP6ytXfzXqz1wYuZoOXJTgSQMMlJ+clekMzSxLEJ2N8gfdVbU2W3RJMo4FJoxPv MBZYEu0dDkiUShHKydpwi7vDY6WpJnwU0smKcBqku5smZMf47NIBqEyK+o+mD6U5lSAN 6woI09/nfqyyiAhbHCtV5kMNT3pvFGkRHM0ytzSb5v4pTYf4aujQ+RyhYU1jPJwfgCyt HQK6+WhpA9SPeOuZPXYXFfBgsPXoxIKSIhWbKR8Pr+cV2JIywiWxsw/TMR9gW776VccH F6wDJnm+yuvV8PdNYBssYC1ItLbOp7LJa0xJ7UvM5yXrLeGEIpAPN+CSi6hw/s68w+ZR wIIg== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b=pHwX+Orj; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id c90si18362edf.274.2021.07.26.06.47.39; Mon, 26 Jul 2021 06:47:40 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b=pHwX+Orj; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 220D2689954; Mon, 26 Jul 2021 16:47:37 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-lf1-f43.google.com (mail-lf1-f43.google.com [209.85.167.43]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id D2420688051 for ; Mon, 26 Jul 2021 16:47:35 +0300 (EEST) Received: by mail-lf1-f43.google.com with SMTP id bp1so15607759lfb.3 for ; Mon, 26 Jul 2021 06:47:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id:mime-version :content-transfer-encoding; bh=o7EuBv+cvMbv67DG7ZWqG8SfB8OrW1sIs7Uybhpo56E=; b=pHwX+Orj+dvu39NRNwq0tdi5SW5ScmuaPvKkkOriFxE6T99hkc4DrT2ctXSKng7x4e RUWP96G8GiYy9b+IzmJW75OOfMYXTyD5gLLBZJj3bpi+YIeO201Kthwb9GOPAWLlTASl Iuza5fTPVMRlf21lSg5Y6N1iZpVvyqFIZAqmod9AF/KcXel/x6NX99esL9zVEZuQSkZ3 C/3oq1FQZaj23hNhk4mqZrMwcPThwK+0ZDAtpYCTsrmSJBG8qY6xbIkrUdCjMKpYjh9p YQEWj7FWLOuy946V7Z2m/nopNLQcgN8+gwuGs7qhTj1SrJJFEMAN2KSV0EecEUuphtfV Ncsw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:mime-version :content-transfer-encoding; bh=o7EuBv+cvMbv67DG7ZWqG8SfB8OrW1sIs7Uybhpo56E=; b=m5+Gc4t+bUMltLYYvptf+H3fJLYqCMuOti0JHVQpGoK2NfJ0yjC5EJNK6WO6H1Kd7s ZwWijY528kM9Gz/QCdHP3xSK6tjgjnc9HEkQgQ0av3yy4lHw/2sYntCSvsfyPDYrynnA 4V1Lz+NGGNyp8sts8Ef2RXzp6Dzj17kuDcv6Z7ryU6rM2zfyWQRjtGkd5zDVYiO29NOZ bkRiiPU9v3SU/RjlrZyq3htqnb3osudgw09YTnLT+H70XgvMLJjkWxuuWwK+RWDHrtYV w7xR7a5l3Ty9AUdG1/k2kFonEmZvfLgkYk+Vy6I66LQ55gupfc9kb0ZJwtA593bozttV t5Ug== X-Gm-Message-State: AOAM530V1b2SsnR9gjReqLhgZTdPiTx/tgY6atPB0jtjfffVCYG4bgBp BDlGstT+PP7SoJbjBYoOvHB8JuVbFEZ6vQ== X-Received: by 2002:a2e:a307:: with SMTP id l7mr12227318lje.318.1627306870843; Mon, 26 Jul 2021 06:41:10 -0700 (PDT) Received: from localhost.localdomain (91-159-194-103.elisa-laajakaista.fi. [91.159.194.103]) by smtp.gmail.com with ESMTPSA id a18sm1714649ljq.14.2021.07.26.06.41.10 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 26 Jul 2021 06:41:10 -0700 (PDT) From: =?utf-8?q?Jan_Ekstr=C3=B6m?= To: ffmpeg-devel@ffmpeg.org Date: Mon, 26 Jul 2021 16:41:02 +0300 Message-Id: <20210726134104.42737-1-jeebjp@gmail.com> X-Mailer: git-send-email 2.31.1 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v3 0/2] TTML in MP4, part 1 X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: uB/YpqGN+vf3 To keep it simpler this first part includes only non-fragmented use cases if the built-in sample squashing is utilized. Contains the basic mapping code as well as the function that writes out the squashed documents. Works with both subtitle-only output as well as with other streams. In a separate patch set I will add additional complexity on top by introducing fragmentation matching based on the other streams in the mux (since most things that ingest fragmented MP4 with subtitles want the fragment time ranges to match between the video/audio streams and subtitles in the mux). Changes from v2: * Switched to the specified handler type (`subt`, which is the ISOBMFF subtitle handler type and should be utilized for XMLSubtitleSampleEntry) and SubtitleMediaHeaderBox (`sthd`, an empty all-zeroes FullBox) with MP4 style TTML. I did not notice this until now since most implementations seem to be fine with just the sample entry being correct. ISMV style TTML was getting the correct boxes written, `text` and `nmhd`. * Switched the `dfxp` tag check from track->mode == MODE_MP4 to != MODE_ISM, as that matches better; After all, `dfxp` is only supposed to be utilized in that specific context. * A trailing semicolon was removed in mov_write_ttml_document_from_queue. * Applied changes based on Andreas' review: ** Using AVERROR_PATCHWELCOME when erroring out that fragmentation is not yet supported. ** movenc_ttml::mov_init_ttml_writer now returns error ASAP if the lavf context allocation fails making the context always a valid pointer after that in the function. As mov_init_ttml_writer handles its own clean-up, the error is then propagated right up in ff_mov_generate_squashed_ttml_packet as well. ** ffio_free_dyn_buf is now being utilized to free the muxer's dynamic buffer in movenc_ttml. ** movenc_ttml::mov_write_ttml_document_from_queue now utilizes a passed AVPacket instead of allocating its own. Jan Jan Ekström (2): avformat/ttml: split TTML paragraph based or not check into header avformat/movenc: add support for TTML muxing libavformat/Makefile | 2 +- libavformat/isom.h | 3 + libavformat/movenc.c | 179 ++++++++++++++++++++++++++++++- libavformat/movenc.h | 5 + libavformat/movenc_ttml.c | 178 ++++++++++++++++++++++++++++++ libavformat/movenc_ttml.h | 31 ++++++ libavformat/ttmlenc.c | 9 +- libavformat/ttmlenc.h | 39 +++++++ tests/fate/subtitles.mak | 4 + tests/ref/fate/sub-ttml-mp4-dfxp | 44 ++++++++ tests/ref/fate/sub-ttml-mp4-stpp | 44 ++++++++ 11 files changed, 528 insertions(+), 10 deletions(-) create mode 100644 libavformat/movenc_ttml.c create mode 100644 libavformat/movenc_ttml.h create mode 100644 libavformat/ttmlenc.h create mode 100644 tests/ref/fate/sub-ttml-mp4-dfxp create mode 100644 tests/ref/fate/sub-ttml-mp4-stpp