[FFmpeg-devel,v10,1/2] avformat/imf: Demuxer

From: Pierre-Anthony Lemieux <pal@palemieux.com>

Signed-off-by: Pierre-Anthony Lemieux <pal@palemieux.com>
---

Notes:
    The IMF demuxer accepts as input an IMF CPL. The assets referenced by the CPL can be
    contained in multiple deliveries, each defined by an ASSETMAP file:
    
    ffmpeg -assetmaps <path of ASSETMAP1>,<path of ASSETMAP>,... -i <path of CPL>
    
    If -assetmaps is not specified, FFMPEG looks for a file called ASSETMAP.xml in the same directory as the CPL.
    
    EXAMPLE:
        ffmpeg -i http://ffmpeg-imf-samples-public.s3-website-us-west-1.amazonaws.com/countdown/CPL_f5095caa-f204-4e1c-8a84-7af48c7ae16b.xml out.mp4
    
    The Interoperable Master Format (IMF) is a file-based media format for the
    delivery and storage of professional audio-visual masters.
    An IMF Composition consists of an XML playlist (the Composition Playlist)
    and a collection of MXF files (the Track Files). The Composition Playlist (CPL)
    assembles the Track Files onto a timeline, which consists of multiple tracks.
    The location of the Track Files referenced by the Composition Playlist is stored
    in one or more XML documents called Asset Maps. More details at https://www.imfug.com/explainer.
    The IMF standard was first introduced in 2013 and is managed by the SMPTE.
    
    CHANGE NOTES:
    
    - add imf_probe
    - improve imf-specific function names

 MAINTAINERS              |   1 +
 configure                |   3 +-
 doc/demuxers.texi        |   6 +
 libavformat/Makefile     |   1 +
 libavformat/allformats.c |   1 +
 libavformat/imf.h        | 207 +++++++++
 libavformat/imf_cpl.c    | 782 +++++++++++++++++++++++++++++++++
 libavformat/imfdec.c     | 905 +++++++++++++++++++++++++++++++++++++++
 8 files changed, 1905 insertions(+), 1 deletion(-)
 create mode 100644 libavformat/imf.h
 create mode 100644 libavformat/imf_cpl.c
 create mode 100644 libavformat/imfdec.c

Does anyone have any particularly strong feelings against this?

I see nothing obviously wrong with it. The UUID and XML discussion can be finished later,
so I see no reason why this shouldn't make it in time for the 5.0 release.

I'll apply this weekend if no objections.

On 13/12/21 15:43, pal@sandflow.com wrote:
> From: Pierre-Anthony Lemieux <pal@palemieux.com>
> 
> Signed-off-by: Pierre-Anthony Lemieux <pal@palemieux.com>
> ---
> 
> Notes:
>      The IMF demuxer accepts as input an IMF CPL. The assets referenced by the CPL can be
>      contained in multiple deliveries, each defined by an ASSETMAP file:
>      
>      ffmpeg -assetmaps <path of ASSETMAP1>,<path of ASSETMAP>,... -i <path of CPL>
>      
>      If -assetmaps is not specified, FFMPEG looks for a file called ASSETMAP.xml in the same directory as the CPL.
>      
>      EXAMPLE:
>          ffmpeg -i http://ffmpeg-imf-samples-public.s3-website-us-west-1.amazonaws.com/countdown/CPL_f5095caa-f204-4e1c-8a84-7af48c7ae16b.xml out.mp4
>      
>      The Interoperable Master Format (IMF) is a file-based media format for the
>      delivery and storage of professional audio-visual masters.
>      An IMF Composition consists of an XML playlist (the Composition Playlist)
>      and a collection of MXF files (the Track Files). The Composition Playlist (CPL)
>      assembles the Track Files onto a timeline, which consists of multiple tracks.
>      The location of the Track Files referenced by the Composition Playlist is stored
>      in one or more XML documents called Asset Maps. More details at https://www.imfug.com/explainer.
>      The IMF standard was first introduced in 2013 and is managed by the SMPTE.
>      
>      CHANGE NOTES:
>      
>      - add imf_probe
>      - improve imf-specific function names
> 
>   MAINTAINERS              |   1 +
>   configure                |   3 +-
>   doc/demuxers.texi        |   6 +
>   libavformat/Makefile     |   1 +
>   libavformat/allformats.c |   1 +
>   libavformat/imf.h        | 207 +++++++++
>   libavformat/imf_cpl.c    | 782 +++++++++++++++++++++++++++++++++
>   libavformat/imfdec.c     | 905 +++++++++++++++++++++++++++++++++++++++
>   8 files changed, 1905 insertions(+), 1 deletion(-)
>   create mode 100644 libavformat/imf.h
>   create mode 100644 libavformat/imf_cpl.c
>   create mode 100644 libavformat/imfdec.c
> 
> diff --git a/MAINTAINERS b/MAINTAINERS
> index dcac46003e..7a6972fe1a 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -433,6 +433,7 @@ Muxers/Demuxers:
>     idroqdec.c                            Mike Melanson
>     iff.c                                 Jaikrishnan Menon
>     img2*.c                               Michael Niedermayer
> +  imf*.c                                Marc-Antoine Arnaud, Pierre-Anthony Lemieux, Valentin Noël
>     ipmovie.c                             Mike Melanson
>     ircam*                                Paul B Mahol
>     iss.c                                 Stefan Gehrer
> diff --git a/configure b/configure
> index a7593ec2db..aa8bae4d62 100755
> --- a/configure
> +++ b/configure
> @@ -298,7 +298,7 @@ External library support:
>     --enable-libxvid         enable Xvid encoding via xvidcore,
>                              native MPEG-4/Xvid encoder exists [no]
>     --enable-libxml2         enable XML parsing using the C library libxml2, needed
> -                           for dash demuxing support [no]
> +                           for dash and imf demuxing support [no]
>     --enable-libzimg         enable z.lib, needed for zscale filter [no]
>     --enable-libzmq          enable message passing via libzmq [no]
>     --enable-libzvbi         enable teletext support via libzvbi [no]
> @@ -3400,6 +3400,7 @@ hls_muxer_select="mpegts_muxer"
>   hls_muxer_suggest="gcrypt openssl"
>   image2_alias_pix_demuxer_select="image2_demuxer"
>   image2_brender_pix_demuxer_select="image2_demuxer"
> +imf_demuxer_deps="libxml2"
>   ipod_muxer_select="mov_muxer"
>   ismv_muxer_select="mov_muxer"
>   ivf_muxer_select="av1_metadata_bsf vp9_superframe_bsf"
> diff --git a/doc/demuxers.texi b/doc/demuxers.texi
> index cab8a7072c..655704d2c4 100644
> --- a/doc/demuxers.texi
> +++ b/doc/demuxers.texi
> @@ -267,6 +267,12 @@ which streams to actually receive.
>   Each stream mirrors the @code{id} and @code{bandwidth} properties from the
>   @code{<Representation>} as metadata keys named "id" and "variant_bitrate" respectively.
>   
> +@section imf
> +
> +Interoperable Master Format demuxer.
> +
> +This demuxer presents audio and video streams found in an IMF Composition.
> +
>   @section flv, live_flv, kux
>   
>   Adobe Flash Video Format demuxer.
> diff --git a/libavformat/Makefile b/libavformat/Makefile
> index 2b5caf9d33..7f058f3ea0 100644
> --- a/libavformat/Makefile
> +++ b/libavformat/Makefile
> @@ -285,6 +285,7 @@ OBJS-$(CONFIG_IMAGE_WEBP_PIPE_DEMUXER)    += img2dec.o img2.o
>   OBJS-$(CONFIG_IMAGE_XBM_PIPE_DEMUXER)     += img2dec.o img2.o
>   OBJS-$(CONFIG_IMAGE_XPM_PIPE_DEMUXER)     += img2dec.o img2.o
>   OBJS-$(CONFIG_IMAGE_XWD_PIPE_DEMUXER)     += img2dec.o img2.o
> +OBJS-$(CONFIG_IMF_DEMUXER)               += imfdec.o imf_cpl.o
>   OBJS-$(CONFIG_INGENIENT_DEMUXER)         += ingenientdec.o rawdec.o
>   OBJS-$(CONFIG_IPMOVIE_DEMUXER)           += ipmovie.o
>   OBJS-$(CONFIG_IPU_DEMUXER)               += ipudec.o rawdec.o
> diff --git a/libavformat/allformats.c b/libavformat/allformats.c
> index 1054ac9667..f680616cdd 100644
> --- a/libavformat/allformats.c
> +++ b/libavformat/allformats.c
> @@ -212,6 +212,7 @@ extern const AVInputFormat  ff_image2pipe_demuxer;
>   extern const AVOutputFormat ff_image2pipe_muxer;
>   extern const AVInputFormat  ff_image2_alias_pix_demuxer;
>   extern const AVInputFormat  ff_image2_brender_pix_demuxer;
> +extern const AVInputFormat  ff_imf_demuxer;
>   extern const AVInputFormat  ff_ingenient_demuxer;
>   extern const AVInputFormat  ff_ipmovie_demuxer;
>   extern const AVOutputFormat ff_ipod_muxer;
> diff --git a/libavformat/imf.h b/libavformat/imf.h
> new file mode 100644
> index 0000000000..65575f6c3a
> --- /dev/null
> +++ b/libavformat/imf.h
> @@ -0,0 +1,207 @@
> +/*
> + * This file is part of FFmpeg.
> + *
> + * FFmpeg is free software; you can redistribute it and/or
> + * modify it under the terms of the GNU Lesser General Public
> + * License as published by the Free Software Foundation; either
> + * version 2.1 of the License, or (at your option) any later version.
> + *
> + * FFmpeg is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> + * Lesser General Public License for more details.
> + *
> + * You should have received a copy of the GNU Lesser General Public
> + * License along with FFmpeg; if not, write to the Free Software
> + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
> + */
> +
> +/*
> + *
> + * Copyright (c) Sandflow Consulting LLC
> + *
> + * Redistribution and use in source and binary forms, with or without
> + * modification, are permitted provided that the following conditions are met:
> + *
> + * * Redistributions of source code must retain the above copyright notice, this
> + *   list of conditions and the following disclaimer.
> + * * Redistributions in binary form must reproduce the above copyright notice,
> + *   this list of conditions and the following disclaimer in the documentation
> + *   and/or other materials provided with the distribution.
> + *
> + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
> + * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
> + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
> + * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE
> + * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
> + * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
> + * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
> + * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
> + * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
> + * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
> + * POSSIBILITY OF SUCH DAMAGE.
> + */
> +
> +/**
> + * Public header file for the processing of Interoperable Master Format (IMF)
> + * packages.
> + *
> + * @author Pierre-Anthony Lemieux
> + * @author Valentin Noel
> + * @file
> + * @ingroup lavu_imf
> + */
> +
> +#ifndef AVFORMAT_IMF_H
> +#define AVFORMAT_IMF_H
> +
> +#include "avformat.h"
> +#include "libavformat/avio.h"
> +#include "libavutil/rational.h"
> +#include <libxml/tree.h>
> +
> +#define FF_IMF_UUID_FORMAT                            \
> +    "urn:uuid:%02hhx%02hhx%02hhx%02hhx-%02hhx%02hhx-" \
> +    "%02hhx%02hhx-%02hhx%02hhx-%02hhx%02hhx%02hhx%02hhx%02hhx%02hhx"
> +
> +/**
> + * UUID as defined in IETF RFC 422
> + */
> +typedef uint8_t FFIMFUUID[16];
> +
> +/**
> + * IMF Composition Playlist Base Resource
> + */
> +typedef struct FFIMFBaseResource {
> +    AVRational edit_rate;  /**< BaseResourceType/EditRate */
> +    uint32_t entry_point;  /**< BaseResourceType/EntryPoint */
> +    uint32_t duration;     /**< BaseResourceType/Duration */
> +    uint32_t repeat_count; /**< BaseResourceType/RepeatCount */
> +} FFIMFBaseResource;
> +
> +/**
> + * IMF Composition Playlist Track File Resource
> + */
> +typedef struct FFIMFTrackFileResource {
> +    FFIMFBaseResource base;
> +    FFIMFUUID track_file_uuid; /**< TrackFileResourceType/TrackFileId */
> +} FFIMFTrackFileResource;
> +
> +/**
> + * IMF Marker
> + */
> +typedef struct FFIMFMarker {
> +    xmlChar *label_utf8; /**< Marker/Label */
> +    xmlChar *scope_utf8; /**< Marker/Label/\@scope */
> +    uint32_t offset;     /**< Marker/Offset */
> +} FFIMFMarker;
> +
> +/**
> + * IMF Composition Playlist Marker Resource
> + */
> +typedef struct FFIMFMarkerResource {
> +    FFIMFBaseResource base;
> +    uint32_t marker_count; /**< Number of Marker elements */
> +    FFIMFMarker *markers;  /**< Marker elements */
> +} FFIMFMarkerResource;
> +
> +/**
> + * IMF Composition Playlist Virtual Track
> + */
> +typedef struct FFIMFBaseVirtualTrack {
> +    FFIMFUUID id_uuid; /**< TrackId associated with the Virtual Track */
> +} FFIMFBaseVirtualTrack;
> +
> +/**
> + * IMF Composition Playlist Virtual Track that consists of Track File Resources
> + */
> +typedef struct FFIMFTrackFileVirtualTrack {
> +    FFIMFBaseVirtualTrack base;
> +    uint32_t resource_count;           /**< Number of Resource elements present in the Virtual Track */
> +    FFIMFTrackFileResource *resources; /**< Resource elements of the Virtual Track */
> +    uint32_t resources_alloc_sz;       /**< Size of the resources buffer */
> +} FFIMFTrackFileVirtualTrack;
> +
> +/**
> + * IMF Composition Playlist Virtual Track that consists of Marker Resources
> + */
> +typedef struct FFIMFMarkerVirtualTrack {
> +    FFIMFBaseVirtualTrack base;
> +    uint32_t resource_count;        /**< Number of Resource elements present in the Virtual Track */
> +    FFIMFMarkerResource *resources; /**< Resource elements of the Virtual Track */
> +} FFIMFMarkerVirtualTrack;
> +
> +/**
> + * IMF Composition Playlist
> + */
> +typedef struct FFIMFCPL {
> +    FFIMFUUID id_uuid;                               /**< CompositionPlaylist/Id element */
> +    xmlChar *content_title_utf8;                     /**< CompositionPlaylist/ContentTitle element */
> +    AVRational edit_rate;                            /**< CompositionPlaylist/EditRate element */
> +    FFIMFMarkerVirtualTrack *main_markers_track;     /**< Main Marker Virtual Track */
> +    FFIMFTrackFileVirtualTrack *main_image_2d_track; /**< Main Image Virtual Track */
> +    uint32_t main_audio_track_count;                 /**< Number of Main Audio Virtual Tracks */
> +    FFIMFTrackFileVirtualTrack *main_audio_tracks;   /**< Main Audio Virtual Tracks */
> +} FFIMFCPL;
> +
> +/**
> + * Parse an IMF CompositionPlaylist element into the FFIMFCPL data structure.
> + * @param[in] doc An XML document from which the CPL is read.
> + * @param[out] cpl Pointer to a memory area (allocated by the client), where the
> + *  function writes a pointer to the newly constructed FFIMFCPL structure (or
> + *  NULL if the CPL could not be parsed). The client is responsible for freeing
> + *  the FFIMFCPL structure using ff_imf_cpl_free().
> + * @return A non-zero value in case of an error.
> + */
> +int ff_imf_parse_cpl_from_xml_dom(xmlDocPtr doc, FFIMFCPL **cpl);
> +
> +/**
> + * Parse an IMF Composition Playlist document into the FFIMFCPL data structure.
> + * @param[in] in The context from which the CPL is read.
> + * @param[out] cpl Pointer to a memory area (allocated by the client), where the
> + * function writes a pointer to the newly constructed FFIMFCPL structure (or
> + * NULL if the CPL could not be parsed). The client is responsible for freeing
> + * the FFIMFCPL structure using ff_imf_cpl_free().
> + * @return A non-zero value in case of an error.
> + */
> +int ff_imf_parse_cpl(AVIOContext *in, FFIMFCPL **cpl);
> +
> +/**
> + * Allocates and initializes an FFIMFCPL data structure.
> + * @return A pointer to the newly constructed FFIMFCPL structure (or NULL if the
> + * structure could not be constructed). The client is responsible for freeing
> + * the FFIMFCPL structure using ff_imf_cpl_free().
> + */
> +FFIMFCPL *ff_imf_cpl_alloc(void);
> +
> +/**
> + * Deletes an FFIMFCPL data structure previously instantiated with ff_imf_cpl_alloc().
> + * @param[in] cpl The FFIMFCPL structure to delete.
> + */
> +void ff_imf_cpl_free(FFIMFCPL *cpl);
> +
> +/**
> + * Reads an unsigned 32-bit integer from an XML element
> + * @return 0 on success, < 0 AVERROR code on error.
> + */
> +int ff_imf_xml_read_uint32(xmlNodePtr element, uint32_t *number);
> +
> +/**
> + * Reads an AVRational from an XML element
> + * @return 0 on success, < 0 AVERROR code on error.
> + */
> +int ff_imf_xml_read_rational(xmlNodePtr element, AVRational *rational);
> +
> +/**
> + * Reads a UUID from an XML element
> + * @return 0 on success, < 0 AVERROR code on error.
> + */
> +int ff_imf_xml_read_uuid(xmlNodePtr element, uint8_t uuid[16]);
> +
> +/**
> + * Returns the first child element with the specified local name
> + * @return A pointer to the child element, or NULL if no such child element exists.
> + */
> +xmlNodePtr ff_imf_xml_get_child_element_by_name(xmlNodePtr parent, const char *name_utf8);
> +
> +#endif
> diff --git a/libavformat/imf_cpl.c b/libavformat/imf_cpl.c
> new file mode 100644
> index 0000000000..048c5fd59d
> --- /dev/null
> +++ b/libavformat/imf_cpl.c
> @@ -0,0 +1,782 @@
> +/*
> + * This file is part of FFmpeg.
> + *
> + * FFmpeg is free software; you can redistribute it and/or
> + * modify it under the terms of the GNU Lesser General Public
> + * License as published by the Free Software Foundation; either
> + * version 2.1 of the License, or (at your option) any later version.
> + *
> + * FFmpeg is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> + * Lesser General Public License for more details.
> + *
> + * You should have received a copy of the GNU Lesser General Public
> + * License along with FFmpeg; if not, write to the Free Software
> + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
> + */
> +
> +/*
> + *
> + * Copyright (c) Sandflow Consulting LLC
> + *
> + * Redistribution and use in source and binary forms, with or without
> + * modification, are permitted provided that the following conditions are met:
> + *
> + * * Redistributions of source code must retain the above copyright notice, this
> + *   list of conditions and the following disclaimer.
> + * * Redistributions in binary form must reproduce the above copyright notice,
> + *   this list of conditions and the following disclaimer in the documentation
> + *   and/or other materials provided with the distribution.
> + *
> + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
> + * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
> + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
> + * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE
> + * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
> + * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
> + * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
> + * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
> + * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
> + * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
> + * POSSIBILITY OF SUCH DAMAGE.
> + */
> +
> +/**
> + * Implements IMP CPL processing
> + *
> + * @author Pierre-Anthony Lemieux
> + * @file
> + * @ingroup lavu_imf
> + */
> +
> +#include "imf.h"
> +#include "libavformat/mxf.h"
> +#include "libavutil/bprint.h"
> +#include "libavutil/error.h"
> +#include <libxml/parser.h>
> +
> +xmlNodePtr ff_imf_xml_get_child_element_by_name(xmlNodePtr parent, const char *name_utf8)
> +{
> +    xmlNodePtr cur_element;
> +
> +    cur_element = xmlFirstElementChild(parent);
> +    while (cur_element) {
> +        if (xmlStrcmp(cur_element->name, name_utf8) == 0)
> +            return cur_element;
> +        cur_element = xmlNextElementSibling(cur_element);
> +    }
> +    return NULL;
> +}
> +
> +int ff_imf_xml_read_uuid(xmlNodePtr element, uint8_t uuid[16])
> +{
> +    xmlChar *element_text = NULL;
> +    int scanf_ret;
> +    int ret = 0;
> +
> +    element_text = xmlNodeListGetString(element->doc, element->xmlChildrenNode, 1);
> +    scanf_ret = sscanf(element_text,
> +        FF_IMF_UUID_FORMAT,
> +        &uuid[0],
> +        &uuid[1],
> +        &uuid[2],
> +        &uuid[3],
> +        &uuid[4],
> +        &uuid[5],
> +        &uuid[6],
> +        &uuid[7],
> +        &uuid[8],
> +        &uuid[9],
> +        &uuid[10],
> +        &uuid[11],
> +        &uuid[12],
> +        &uuid[13],
> +        &uuid[14],
> +        &uuid[15]);
> +    if (scanf_ret != 16) {
> +        av_log(NULL, AV_LOG_ERROR, "Invalid UUID\n");
> +        ret = AVERROR_INVALIDDATA;
> +    }
> +    xmlFree(element_text);
> +
> +    return ret;
> +}
> +
> +int ff_imf_xml_read_rational(xmlNodePtr element, AVRational *rational)
> +{
> +    xmlChar *element_text = NULL;
> +    int ret = 0;
> +
> +    element_text = xmlNodeListGetString(element->doc, element->xmlChildrenNode, 1);
> +    if (sscanf(element_text, "%i %i", &rational->num, &rational->den) != 2) {
> +        av_log(NULL, AV_LOG_ERROR, "Invalid rational number\n");
> +        ret = AVERROR_INVALIDDATA;
> +    }
> +    xmlFree(element_text);
> +
> +    return ret;
> +}
> +
> +int ff_imf_xml_read_uint32(xmlNodePtr element, uint32_t *number)
> +{
> +    xmlChar *element_text = NULL;
> +    int ret = 0;
> +
> +    element_text = xmlNodeListGetString(element->doc, element->xmlChildrenNode, 1);
> +    if (sscanf(element_text, "%" PRIu32, number) != 1) {
> +        av_log(NULL, AV_LOG_ERROR, "Invalid unsigned 32-bit integer");
> +        ret = AVERROR_INVALIDDATA;
> +    }
> +    xmlFree(element_text);
> +
> +    return ret;
> +}
> +
> +static void imf_base_virtual_track_init(FFIMFBaseVirtualTrack *track)
> +{
> +    memset(track->id_uuid, 0, sizeof(track->id_uuid));
> +}
> +
> +static void imf_marker_virtual_track_init(FFIMFMarkerVirtualTrack *track)
> +{
> +    imf_base_virtual_track_init((FFIMFBaseVirtualTrack *)track);
> +    track->resource_count = 0;
> +    track->resources = NULL;
> +}
> +
> +static void imf_trackfile_virtual_track_init(FFIMFTrackFileVirtualTrack *track)
> +{
> +    imf_base_virtual_track_init((FFIMFBaseVirtualTrack *)track);
> +    track->resource_count = 0;
> +    track->resources_alloc_sz = 0;
> +    track->resources = NULL;
> +}
> +
> +static void imf_base_resource_init(FFIMFBaseResource *rsrc)
> +{
> +    rsrc->duration = 0;
> +    rsrc->edit_rate = av_make_q(0, 1);
> +    rsrc->entry_point = 0;
> +    rsrc->repeat_count = 1;
> +}
> +
> +static void imf_marker_resource_init(FFIMFMarkerResource *rsrc)
> +{
> +    imf_base_resource_init((FFIMFBaseResource *)rsrc);
> +    rsrc->marker_count = 0;
> +    rsrc->markers = NULL;
> +}
> +
> +static void imf_marker_init(FFIMFMarker *marker)
> +{
> +    marker->label_utf8 = NULL;
> +    marker->offset = 0;
> +    marker->scope_utf8 = NULL;
> +}
> +
> +static void imf_trackfile_resource_init(FFIMFTrackFileResource *rsrc)
> +{
> +    imf_base_resource_init((FFIMFBaseResource *)rsrc);
> +    memset(rsrc->track_file_uuid, 0, sizeof(rsrc->track_file_uuid));
> +}
> +
> +static int fill_content_title(xmlNodePtr cpl_element, FFIMFCPL *cpl)
> +{
> +    xmlNodePtr element = NULL;
> +
> +    if (!(element = ff_imf_xml_get_child_element_by_name(cpl_element, "ContentTitle"))) {
> +        av_log(NULL, AV_LOG_ERROR, "ContentTitle element not found in the IMF CPL\n");
> +        return AVERROR_INVALIDDATA;
> +    }
> +    cpl->content_title_utf8 = xmlNodeListGetString(cpl_element->doc,
> +        element->xmlChildrenNode,
> +        1);
> +
> +    return 0;
> +}
> +
> +static int fill_edit_rate(xmlNodePtr cpl_element, FFIMFCPL *cpl)
> +{
> +    xmlNodePtr element = NULL;
> +
> +    if (!(element = ff_imf_xml_get_child_element_by_name(cpl_element, "EditRate"))) {
> +        av_log(NULL, AV_LOG_ERROR, "EditRate element not found in the IMF CPL\n");
> +        return AVERROR_INVALIDDATA;
> +    }
> +
> +    return ff_imf_xml_read_rational(element, &cpl->edit_rate);
> +}
> +
> +static int fill_id(xmlNodePtr cpl_element, FFIMFCPL *cpl)
> +{
> +    xmlNodePtr element = NULL;
> +
> +    if (!(element = ff_imf_xml_get_child_element_by_name(cpl_element, "Id"))) {
> +        av_log(NULL, AV_LOG_ERROR, "Id element not found in the IMF CPL\n");
> +        return AVERROR_INVALIDDATA;
> +    }
> +
> +    return ff_imf_xml_read_uuid(element, cpl->id_uuid);
> +}
> +
> +static int fill_marker(xmlNodePtr marker_elem, FFIMFMarker *marker)
> +{
> +    xmlNodePtr element = NULL;
> +    int ret = 0;
> +
> +    /* read Offset */
> +    if (!(element = ff_imf_xml_get_child_element_by_name(marker_elem, "Offset"))) {
> +        av_log(NULL, AV_LOG_ERROR, "Offset element not found in a Marker\n");
> +        return AVERROR_INVALIDDATA;
> +    }
> +    if ((ret = ff_imf_xml_read_uint32(element, &marker->offset)))
> +        return ret;
> +
> +    /* read Label and Scope */
> +    if (!(element = ff_imf_xml_get_child_element_by_name(marker_elem, "Label"))) {
> +        av_log(NULL, AV_LOG_ERROR, "Label element not found in a Marker\n");
> +        return AVERROR_INVALIDDATA;
> +    }
> +    if (!(marker->label_utf8 = xmlNodeListGetString(element->doc, element->xmlChildrenNode, 1))) {
> +        av_log(NULL, AV_LOG_ERROR, "Empty Label element found in a Marker\n");
> +        return AVERROR_INVALIDDATA;
> +    }
> +    if (!(marker->scope_utf8 = xmlGetNoNsProp(element, "scope"))) {
> +        marker->scope_utf8 = xmlCharStrdup("http://www.smpte-ra.org/schemas/2067-3/2013#standard-markers");
> +        if (!marker->scope_utf8) {
> +            xmlFree(marker->label_utf8);
> +            return AVERROR(ENOMEM);
> +        }
> +    }
> +
> +    return ret;
> +}
> +
> +static int fill_base_resource(xmlNodePtr resource_elem, FFIMFBaseResource *resource, FFIMFCPL *cpl)
> +{
> +    xmlNodePtr element = NULL;
> +    int ret = 0;
> +
> +    /* read EditRate */
> +    if (!(element = ff_imf_xml_get_child_element_by_name(resource_elem, "EditRate"))) {
> +        resource->edit_rate = cpl->edit_rate;
> +    } else if (ret = ff_imf_xml_read_rational(element, &resource->edit_rate)) {
> +        av_log(NULL, AV_LOG_ERROR, "Invalid EditRate element found in a Resource\n");
> +        return ret;
> +    }
> +
> +    /* read EntryPoint */
> +    if (element = ff_imf_xml_get_child_element_by_name(resource_elem, "EntryPoint")) {
> +        if (ret = ff_imf_xml_read_uint32(element, &resource->entry_point)) {
> +            av_log(NULL, AV_LOG_ERROR, "Invalid EntryPoint element found in a Resource\n");
> +            return ret;
> +        }
> +    } else {
> +        resource->entry_point = 0;
> +    }
> +
> +    /* read IntrinsicDuration */
> +    if (!(element = ff_imf_xml_get_child_element_by_name(resource_elem, "IntrinsicDuration"))) {
> +        av_log(NULL, AV_LOG_ERROR, "IntrinsicDuration element missing from Resource\n");
> +        return AVERROR_INVALIDDATA;
> +    }
> +    if (ret = ff_imf_xml_read_uint32(element, &resource->duration)) {
> +        av_log(NULL, AV_LOG_ERROR, "Invalid IntrinsicDuration element found in a Resource\n");
> +        return ret;
> +    }
> +    resource->duration -= resource->entry_point;
> +
> +    /* read SourceDuration */
> +    if (element = ff_imf_xml_get_child_element_by_name(resource_elem, "SourceDuration"))
> +        if (ret = ff_imf_xml_read_uint32(element, &resource->duration)) {
> +            av_log(NULL, AV_LOG_ERROR, "SourceDuration element missing from Resource\n");
> +            return ret;
> +        }
> +
> +    /* read RepeatCount */
> +    if (element = ff_imf_xml_get_child_element_by_name(resource_elem, "RepeatCount"))
> +        ret = ff_imf_xml_read_uint32(element, &resource->repeat_count);
> +
> +    return ret;
> +}
> +
> +static int fill_trackfile_resource(xmlNodePtr tf_resource_elem,
> +    FFIMFTrackFileResource *tf_resource,
> +    FFIMFCPL *cpl)
> +{
> +    xmlNodePtr element = NULL;
> +    int ret = 0;
> +
> +    if (ret = fill_base_resource(tf_resource_elem, (FFIMFBaseResource *)tf_resource, cpl))
> +        return ret;
> +
> +    /* read TrackFileId */
> +    if (element = ff_imf_xml_get_child_element_by_name(tf_resource_elem, "TrackFileId")) {
> +        if (ret = ff_imf_xml_read_uuid(element, tf_resource->track_file_uuid)) {
> +            av_log(NULL, AV_LOG_ERROR, "Invalid TrackFileId element found in Resource\n");
> +            return ret;
> +        }
> +    } else {
> +        av_log(NULL, AV_LOG_ERROR, "TrackFileId element missing from Resource\n");
> +        return AVERROR_INVALIDDATA;
> +    }
> +
> +    return ret;
> +}
> +
> +static int fill_marker_resource(xmlNodePtr marker_resource_elem,
> +    FFIMFMarkerResource *marker_resource,
> +    FFIMFCPL *cpl)
> +{
> +    xmlNodePtr element = NULL;
> +    void *tmp;
> +    int ret = 0;
> +
> +    if (ret = fill_base_resource(marker_resource_elem, (FFIMFBaseResource *)marker_resource, cpl))
> +        return ret;
> +
> +    /* read markers */
> +    element = xmlFirstElementChild(marker_resource_elem);
> +    while (element) {
> +        if (xmlStrcmp(element->name, "Marker") == 0) {
> +            tmp = av_realloc(marker_resource->markers,
> +                (marker_resource->marker_count + 1) * sizeof(FFIMFMarker));
> +            if (!tmp)
> +                return AVERROR(ENOMEM);
> +            marker_resource->markers = tmp;
> +            imf_marker_init(&marker_resource->markers[marker_resource->marker_count]);
> +            ret = fill_marker(element,
> +                &marker_resource->markers[marker_resource->marker_count]);
> +            marker_resource->marker_count++;
> +            if (ret)
> +                return ret;
> +        }
> +        element = xmlNextElementSibling(element);
> +    }
> +
> +    return ret;
> +}
> +
> +static int push_marker_sequence(xmlNodePtr marker_sequence_elem, FFIMFCPL *cpl)
> +{
> +    int ret = 0;
> +    uint8_t uuid[16];
> +    xmlNodePtr resource_list_elem = NULL;
> +    xmlNodePtr resource_elem = NULL;
> +    xmlNodePtr track_id_elem = NULL;
> +    unsigned long resource_elem_count;
> +    void *tmp;
> +
> +    /* read TrackID element */
> +    if (!(track_id_elem = ff_imf_xml_get_child_element_by_name(marker_sequence_elem, "TrackId"))) {
> +        av_log(NULL, AV_LOG_ERROR, "TrackId element missing from Sequence\n");
> +        return AVERROR_INVALIDDATA;
> +    }
> +    if (ret = ff_imf_xml_read_uuid(track_id_elem, uuid)) {
> +        av_log(NULL, AV_LOG_ERROR, "Invalid TrackId element found in Sequence\n");
> +        return AVERROR_INVALIDDATA;
> +    }
> +    av_log(NULL,
> +        AV_LOG_DEBUG,
> +        "Processing IMF CPL Marker Sequence for Virtual Track " FF_IMF_UUID_FORMAT "\n",
> +        UID_ARG(uuid));
> +
> +    /* create main marker virtual track if it does not exist */
> +    if (!cpl->main_markers_track) {
> +        cpl->main_markers_track = av_malloc(sizeof(FFIMFMarkerVirtualTrack));
> +        if (!cpl->main_markers_track)
> +            return AVERROR(ENOMEM);
> +        imf_marker_virtual_track_init(cpl->main_markers_track);
> +        memcpy(cpl->main_markers_track->base.id_uuid, uuid, sizeof(uuid));
> +    } else if (memcmp(cpl->main_markers_track->base.id_uuid, uuid, sizeof(uuid)) != 0) {
> +        av_log(NULL, AV_LOG_ERROR, "Multiple marker virtual tracks were found\n");
> +        return AVERROR_INVALIDDATA;
> +    }
> +
> +    /* process resources */
> +    resource_list_elem = ff_imf_xml_get_child_element_by_name(marker_sequence_elem, "ResourceList");
> +    if (!resource_list_elem)
> +        return 0;
> +    resource_elem_count = xmlChildElementCount(resource_list_elem);
> +    tmp = av_realloc(cpl->main_markers_track->resources,
> +        (cpl->main_markers_track->resource_count + resource_elem_count)
> +            * sizeof(FFIMFMarkerResource));
> +    if (!tmp) {
> +        av_log(NULL, AV_LOG_ERROR, "Cannot allocate Marker Resources\n");
> +        return AVERROR(ENOMEM);
> +    }
> +    cpl->main_markers_track->resources = tmp;
> +
> +    resource_elem = xmlFirstElementChild(resource_list_elem);
> +    while (resource_elem) {
> +        imf_marker_resource_init(
> +            &cpl->main_markers_track->resources[cpl->main_markers_track->resource_count]);
> +        ret = fill_marker_resource(resource_elem,
> +            &cpl->main_markers_track->resources[cpl->main_markers_track->resource_count],
> +            cpl);
> +        cpl->main_markers_track->resource_count++;
> +        if (ret)
> +            return ret;
> +        resource_elem = xmlNextElementSibling(resource_elem);
> +    }
> +
> +    return ret;
> +}
> +
> +static int has_stereo_resources(xmlNodePtr element)
> +{
> +    if (xmlStrcmp(element->name, "Left") == 0 || xmlStrcmp(element->name, "Right") == 0)
> +        return 1;
> +    element = xmlFirstElementChild(element);
> +    while (element) {
> +        if (has_stereo_resources(element))
> +            return 1;
> +        element = xmlNextElementSibling(element);
> +    }
> +    return 0;
> +}
> +
> +static int push_main_audio_sequence(xmlNodePtr audio_sequence_elem, FFIMFCPL *cpl)
> +{
> +    int ret = 0;
> +    uint8_t uuid[16];
> +    xmlNodePtr resource_list_elem = NULL;
> +    xmlNodePtr resource_elem = NULL;
> +    xmlNodePtr track_id_elem = NULL;
> +    unsigned long resource_elem_count;
> +    FFIMFTrackFileVirtualTrack *vt = NULL;
> +    void *tmp;
> +
> +    /* read TrackID element */
> +    if (!(track_id_elem = ff_imf_xml_get_child_element_by_name(audio_sequence_elem, "TrackId"))) {
> +        av_log(NULL, AV_LOG_ERROR, "TrackId element missing from audio sequence\n");
> +        return AVERROR_INVALIDDATA;
> +    }
> +    if (ret = ff_imf_xml_read_uuid(track_id_elem, uuid)) {
> +        av_log(NULL, AV_LOG_ERROR, "Invalid TrackId element found in audio sequence\n");
> +        return ret;
> +    }
> +    av_log(NULL,
> +        AV_LOG_DEBUG,
> +        "Processing IMF CPL Audio Sequence for Virtual Track " FF_IMF_UUID_FORMAT "\n",
> +        UID_ARG(uuid));
> +
> +    /* get the main audio virtual track corresponding to the sequence */
> +    for (uint32_t i = 0; i < cpl->main_audio_track_count; i++)
> +        if (memcmp(cpl->main_audio_tracks[i].base.id_uuid, uuid, sizeof(uuid)) == 0) {
> +            vt = &cpl->main_audio_tracks[i];
> +            break;
> +        }
> +
> +    /* create a main audio virtual track if none exists for the sequence */
> +    if (!vt) {
> +        tmp = av_realloc(cpl->main_audio_tracks,
> +            (cpl->main_audio_track_count + 1) * sizeof(FFIMFTrackFileVirtualTrack));
> +        if (!tmp)
> +            return AVERROR(ENOMEM);
> +        cpl->main_audio_tracks = tmp;
> +        vt = &cpl->main_audio_tracks[cpl->main_audio_track_count];
> +        imf_trackfile_virtual_track_init(vt);
> +        cpl->main_audio_track_count++;
> +        memcpy(vt->base.id_uuid, uuid, sizeof(uuid));
> +    }
> +
> +    /* process resources */
> +    resource_list_elem = ff_imf_xml_get_child_element_by_name(audio_sequence_elem, "ResourceList");
> +    if (!resource_list_elem)
> +        return 0;
> +    resource_elem_count = xmlChildElementCount(resource_list_elem);
> +    tmp = av_fast_realloc(vt->resources,
> +        &vt->resources_alloc_sz,
> +        (vt->resource_count + resource_elem_count) * sizeof(FFIMFTrackFileResource));
> +    if (!tmp) {
> +        av_log(NULL, AV_LOG_ERROR, "Cannot allocate Main Audio Resources\n");
> +        return AVERROR(ENOMEM);
> +    }
> +    vt->resources = tmp;
> +
> +    resource_elem = xmlFirstElementChild(resource_list_elem);
> +    while (resource_elem) {
> +        imf_trackfile_resource_init(&vt->resources[vt->resource_count]);
> +        ret = fill_trackfile_resource(resource_elem,
> +            &vt->resources[vt->resource_count],
> +            cpl);
> +        vt->resource_count++;
> +        if (ret) {
> +            av_log(NULL, AV_LOG_ERROR, "Invalid Resource\n");
> +            continue;
> +        }
> +        resource_elem = xmlNextElementSibling(resource_elem);
> +    }
> +
> +    return ret;
> +}
> +
> +static int push_main_image_2d_sequence(xmlNodePtr image_sequence_elem, FFIMFCPL *cpl)
> +{
> +    int ret = 0;
> +    uint8_t uuid[16];
> +    xmlNodePtr resource_list_elem = NULL;
> +    xmlNodePtr resource_elem = NULL;
> +    xmlNodePtr track_id_elem = NULL;
> +    void *tmp;
> +    unsigned long resource_elem_count;
> +
> +    /* skip stereoscopic resources */
> +    if (has_stereo_resources(image_sequence_elem)) {
> +        av_log(NULL, AV_LOG_ERROR, "Stereoscopic 3D image virtual tracks not supported\n");
> +        return AVERROR_PATCHWELCOME;
> +    }
> +
> +    /* read TrackId element*/
> +    if (!(track_id_elem = ff_imf_xml_get_child_element_by_name(image_sequence_elem, "TrackId"))) {
> +        av_log(NULL, AV_LOG_ERROR, "TrackId element missing from audio sequence\n");
> +        return AVERROR_INVALIDDATA;
> +    }
> +    if (ret = ff_imf_xml_read_uuid(track_id_elem, uuid)) {
> +        av_log(NULL, AV_LOG_ERROR, "Invalid TrackId element found in audio sequence\n");
> +        return ret;
> +    }
> +
> +    /* create main image virtual track if one does not exist */
> +    if (!cpl->main_image_2d_track) {
> +        cpl->main_image_2d_track = av_malloc(sizeof(FFIMFTrackFileVirtualTrack));
> +        if (!cpl->main_image_2d_track)
> +            return AVERROR(ENOMEM);
> +        imf_trackfile_virtual_track_init(cpl->main_image_2d_track);
> +        memcpy(cpl->main_image_2d_track->base.id_uuid, uuid, sizeof(uuid));
> +    } else if (memcmp(cpl->main_image_2d_track->base.id_uuid, uuid, sizeof(uuid)) != 0) {
> +        av_log(NULL, AV_LOG_ERROR, "Multiple MainImage virtual tracks found\n");
> +        return AVERROR_INVALIDDATA;
> +    }
> +    av_log(NULL,
> +        AV_LOG_DEBUG,
> +        "Processing IMF CPL Main Image Sequence for Virtual Track " FF_IMF_UUID_FORMAT "\n",
> +        UID_ARG(uuid));
> +
> +    /* process resources */
> +    resource_list_elem = ff_imf_xml_get_child_element_by_name(image_sequence_elem, "ResourceList");
> +    if (!resource_list_elem)
> +        return 0;
> +    resource_elem_count = xmlChildElementCount(resource_list_elem);
> +    tmp = av_fast_realloc(cpl->main_image_2d_track->resources,
> +        &cpl->main_image_2d_track->resources_alloc_sz,
> +        (cpl->main_image_2d_track->resource_count + resource_elem_count) * sizeof(FFIMFTrackFileResource));
> +    if (!tmp) {
> +        av_log(NULL, AV_LOG_ERROR, "Cannot allocate Main Image Resources\n");
> +        return AVERROR(ENOMEM);
> +    }
> +    cpl->main_image_2d_track->resources = tmp;
> +
> +    resource_elem = xmlFirstElementChild(resource_list_elem);
> +    while (resource_elem) {
> +        imf_trackfile_resource_init(
> +            &cpl->main_image_2d_track->resources[cpl->main_image_2d_track->resource_count]);
> +        ret = fill_trackfile_resource(resource_elem,
> +            &cpl->main_image_2d_track->resources[cpl->main_image_2d_track->resource_count],
> +            cpl);
> +        cpl->main_image_2d_track->resource_count++;
> +        if (ret) {
> +            av_log(NULL, AV_LOG_ERROR, "Invalid Resource\n");
> +            continue;
> +        }
> +        resource_elem = xmlNextElementSibling(resource_elem);
> +    }
> +
> +    return 0;
> +}
> +
> +static int fill_virtual_tracks(xmlNodePtr cpl_element, FFIMFCPL *cpl)
> +{
> +    int ret = 0;
> +    xmlNodePtr segment_list_elem = NULL;
> +    xmlNodePtr segment_elem = NULL;
> +    xmlNodePtr sequence_list_elem = NULL;
> +    xmlNodePtr sequence_elem = NULL;
> +
> +    if (!(segment_list_elem = ff_imf_xml_get_child_element_by_name(cpl_element, "SegmentList"))) {
> +        av_log(NULL, AV_LOG_ERROR, "SegmentList element missing\n");
> +        return AVERROR_INVALIDDATA;
> +    }
> +
> +    /* process sequences */
> +    segment_elem = xmlFirstElementChild(segment_list_elem);
> +    while (segment_elem) {
> +        av_log(NULL, AV_LOG_DEBUG, "Processing IMF CPL Segment\n");
> +        sequence_list_elem = ff_imf_xml_get_child_element_by_name(segment_elem, "SequenceList");
> +        if (!segment_list_elem)
> +            continue;
> +        sequence_elem = xmlFirstElementChild(sequence_list_elem);
> +        while (sequence_elem) {
> +            if (xmlStrcmp(sequence_elem->name, "MarkerSequence") == 0)
> +                ret = push_marker_sequence(sequence_elem, cpl);
> +            else if (xmlStrcmp(sequence_elem->name, "MainImageSequence") == 0)
> +                ret = push_main_image_2d_sequence(sequence_elem, cpl);
> +            else if (xmlStrcmp(sequence_elem->name, "MainAudioSequence") == 0)
> +                ret = push_main_audio_sequence(sequence_elem, cpl);
> +            else
> +                av_log(NULL,
> +                    AV_LOG_INFO,
> +                    "The following Sequence is not supported and is ignored: %s\n",
> +                    sequence_elem->name);
> +            if (ret == AVERROR(ENOMEM))
> +                /* abort parsing only if memory error occurred */
> +                return ret;
> +            sequence_elem = xmlNextElementSibling(sequence_elem);
> +        }
> +        segment_elem = xmlNextElementSibling(segment_elem);
> +    }
> +
> +    return ret;
> +}
> +
> +int ff_imf_parse_cpl_from_xml_dom(xmlDocPtr doc, FFIMFCPL **cpl)
> +{
> +    int ret = 0;
> +    xmlNodePtr cpl_element = NULL;
> +
> +    *cpl = ff_imf_cpl_alloc();
> +    if (!*cpl) {
> +        ret = AVERROR(ENOMEM);
> +        goto cleanup;
> +    }
> +    cpl_element = xmlDocGetRootElement(doc);
> +    if (xmlStrcmp(cpl_element->name, "CompositionPlaylist")) {
> +        av_log(NULL, AV_LOG_ERROR, "The root element of the CPL is not CompositionPlaylist\n");
> +        ret = AVERROR_INVALIDDATA;
> +        goto cleanup;
> +    }
> +    if (ret = fill_content_title(cpl_element, *cpl))
> +        goto cleanup;
> +    if (ret = fill_id(cpl_element, *cpl))
> +        goto cleanup;
> +    if (ret = fill_edit_rate(cpl_element, *cpl))
> +        goto cleanup;
> +    if (ret = fill_virtual_tracks(cpl_element, *cpl))
> +        goto cleanup;
> +
> +cleanup:
> +    if (*cpl && ret) {
> +        ff_imf_cpl_free(*cpl);
> +        *cpl = NULL;
> +    }
> +    return ret;
> +}
> +
> +static void imf_marker_free(FFIMFMarker *marker)
> +{
> +    if (!marker)
> +        return;
> +    xmlFree(marker->label_utf8);
> +    xmlFree(marker->scope_utf8);
> +}
> +
> +static void imf_marker_resource_free(FFIMFMarkerResource *rsrc)
> +{
> +    if (!rsrc)
> +        return;
> +    for (uint32_t i = 0; i < rsrc->marker_count; i++)
> +        imf_marker_free(&rsrc->markers[i]);
> +    av_freep(&rsrc->markers);
> +}
> +
> +static void imf_marker_virtual_track_free(FFIMFMarkerVirtualTrack *vt)
> +{
> +    if (!vt)
> +        return;
> +    for (uint32_t i = 0; i < vt->resource_count; i++)
> +        imf_marker_resource_free(&vt->resources[i]);
> +    av_freep(&vt->resources);
> +}
> +
> +static void imf_trackfile_virtual_track_free(FFIMFTrackFileVirtualTrack *vt)
> +{
> +    if (!vt)
> +        return;
> +    av_freep(&vt->resources);
> +}
> +
> +static void imf_cpl_init(FFIMFCPL *cpl)
> +{
> +    memset(cpl->id_uuid, 0, sizeof(cpl->id_uuid));
> +    cpl->content_title_utf8 = NULL;
> +    cpl->edit_rate = av_make_q(0, 1);
> +    cpl->main_markers_track = NULL;
> +    cpl->main_image_2d_track = NULL;
> +    cpl->main_audio_track_count = 0;
> +    cpl->main_audio_tracks = NULL;
> +}
> +
> +FFIMFCPL *ff_imf_cpl_alloc(void)
> +{
> +    FFIMFCPL *cpl;
> +
> +    cpl = av_malloc(sizeof(FFIMFCPL));
> +    if (!cpl)
> +        return NULL;
> +    imf_cpl_init(cpl);
> +    return cpl;
> +}
> +
> +void ff_imf_cpl_free(FFIMFCPL *cpl)
> +{
> +    if (!cpl)
> +        return;
> +    xmlFree(cpl->content_title_utf8);
> +    imf_marker_virtual_track_free(cpl->main_markers_track);
> +    if (cpl->main_markers_track)
> +        av_freep(&cpl->main_markers_track);
> +    imf_trackfile_virtual_track_free(cpl->main_image_2d_track);
> +    if (cpl->main_image_2d_track)
> +        av_freep(&cpl->main_image_2d_track);
> +    for (uint32_t i = 0; i < cpl->main_audio_track_count; i++)
> +        imf_trackfile_virtual_track_free(&cpl->main_audio_tracks[i]);
> +    if (cpl->main_audio_tracks)
> +        av_freep(&cpl->main_audio_tracks);
> +    av_freep(&cpl);
> +}
> +
> +int ff_imf_parse_cpl(AVIOContext *in, FFIMFCPL **cpl)
> +{
> +    AVBPrint buf;
> +    xmlDoc *doc = NULL;
> +    int ret = 0;
> +    int64_t filesize = 0;
> +
> +    filesize = avio_size(in);
> +    filesize = filesize > 0 ? filesize : 8192;
> +    av_bprint_init(&buf, filesize + 1, AV_BPRINT_SIZE_UNLIMITED);
> +    ret = avio_read_to_bprint(in, &buf, UINT_MAX - 1);
> +    if (ret < 0 || !avio_feof(in) || (filesize = buf.len) == 0) {
> +        if (ret == 0) {
> +            av_log(NULL, AV_LOG_ERROR, "Cannot read IMF CPL\n");
> +            return AVERROR_INVALIDDATA;
> +        }
> +    } else {
> +        LIBXML_TEST_VERSION
> +        doc = xmlReadMemory(buf.str, filesize, NULL, NULL, 0);
> +        if (!doc) {
> +            av_log(NULL,
> +                AV_LOG_ERROR,
> +                "XML parsing failed when reading the IMF CPL\n");
> +            ret = AVERROR_INVALIDDATA;
> +        }
> +        if (ret = ff_imf_parse_cpl_from_xml_dom(doc, cpl)) {
> +            av_log(NULL, AV_LOG_ERROR, "Cannot parse IMF CPL\n");
> +        } else {
> +            av_log(NULL,
> +                AV_LOG_INFO,
> +                "IMF CPL ContentTitle: %s\n",
> +                (*cpl)->content_title_utf8);
> +            av_log(NULL,
> +                AV_LOG_INFO,
> +                "IMF CPL Id: " FF_IMF_UUID_FORMAT "\n",
> +                UID_ARG((*cpl)->id_uuid));
> +        }
> +        xmlFreeDoc(doc);
> +    }
> +    av_bprint_finalize(&buf, NULL);
> +
> +    return ret;
> +}
> diff --git a/libavformat/imfdec.c b/libavformat/imfdec.c
> new file mode 100644
> index 0000000000..b2081536ed
> --- /dev/null
> +++ b/libavformat/imfdec.c
> @@ -0,0 +1,905 @@
> +/*
> + * This file is part of FFmpeg.
> + *
> + * FFmpeg is free software; you can redistribute it and/or
> + * modify it under the terms of the GNU Lesser General Public
> + * License as published by the Free Software Foundation; either
> + * version 2.1 of the License, or (at your option) any later version.
> + *
> + * FFmpeg is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> + * Lesser General Public License for more details.
> + *
> + * You should have received a copy of the GNU Lesser General Public
> + * License along with FFmpeg; if not, write to the Free Software
> + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
> + */
> +
> +/*
> + *
> + * Copyright (c) Sandflow Consulting LLC
> + *
> + * Redistribution and use in source and binary forms, with or without
> + * modification, are permitted provided that the following conditions are met:
> + *
> + * * Redistributions of source code must retain the above copyright notice, this
> + *   list of conditions and the following disclaimer.
> + * * Redistributions in binary form must reproduce the above copyright notice,
> + *   this list of conditions and the following disclaimer in the documentation
> + *   and/or other materials provided with the distribution.
> + *
> + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
> + * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
> + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
> + * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE
> + * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
> + * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
> + * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
> + * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
> + * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
> + * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
> + * POSSIBILITY OF SUCH DAMAGE.
> + */
> +
> +/**
> + * Demuxes an IMF Composition
> + *
> + * References
> + * OV 2067-0:2018 - SMPTE Overview Document - Interoperable Master Format
> + * ST 2067-2:2020 - SMPTE Standard - Interoperable Master Format — Core Constraints
> + * ST 2067-3:2020 - SMPTE Standard - Interoperable Master Format — Composition Playlist
> + * ST 2067-5:2020 - SMPTE Standard - Interoperable Master Format — Essence Component
> + * ST 2067-20:2016 - SMPTE Standard - Interoperable Master Format — Application #2
> + * ST 2067-21:2020 - SMPTE Standard - Interoperable Master Format — Application #2 Extended
> + * ST 2067-102:2017 - SMPTE Standard - Interoperable Master Format — Common Image Pixel Color Schemes
> + * ST 429-9:2007 - SMPTE Standard - D-Cinema Packaging — Asset Mapping and File Segmentation
> + *
> + * @author Marc-Antoine Arnaud
> + * @author Valentin Noel
> + * @author Nicholas Vanderzwet
> + * @file
> + * @ingroup lavu_imf
> + */
> +
> +#include "imf.h"
> +#include "internal.h"
> +#include "libavutil/avstring.h"
> +#include "libavutil/bprint.h"
> +#include "libavutil/opt.h"
> +#include "mxf.h"
> +#include "url.h"
> +#include <inttypes.h>
> +#include <libxml/parser.h>
> +
> +#define MAX_BPRINT_READ_SIZE (UINT_MAX - 1)
> +#define DEFAULT_ASSETMAP_SIZE 8 * 1024
> +#define AVRATIONAL_FORMAT "%d/%d"
> +#define AVRATIONAL_ARG(rational) rational.num, rational.den
> +
> +/**
> + * IMF Asset locator
> + */
> +typedef struct IMFAssetLocator {
> +    FFIMFUUID uuid;
> +    char *absolute_uri;
> +} IMFAssetLocator;
> +
> +/**
> + * IMF Asset locator map
> + * Results from the parsing of one or more ASSETMAP XML files
> + */
> +typedef struct IMFAssetLocatorMap {
> +    uint32_t asset_count;
> +    IMFAssetLocator *assets;
> +} IMFAssetLocatorMap;
> +
> +typedef struct IMFVirtualTrackResourcePlaybackCtx {
> +    IMFAssetLocator *locator;
> +    FFIMFTrackFileResource *resource;
> +    AVFormatContext *ctx;
> +} IMFVirtualTrackResourcePlaybackCtx;
> +
> +typedef struct IMFVirtualTrackPlaybackCtx {
> +    // Track index in playlist
> +    int32_t index;
> +    // Time counters
> +    AVRational current_timestamp;
> +    AVRational duration;
> +    // Resources
> +    uint32_t resource_count;
> +    uint32_t resources_alloc_sz;
> +    IMFVirtualTrackResourcePlaybackCtx *resources;
> +    // Decoding cursors
> +    uint32_t current_resource_index;
> +    int64_t last_pts;
> +} IMFVirtualTrackPlaybackCtx;
> +
> +typedef struct IMFContext {
> +    const AVClass *class;
> +    const char *base_url;
> +    char *asset_map_paths;
> +    AVIOInterruptCB *interrupt_callback;
> +    AVDictionary *avio_opts;
> +    FFIMFCPL *cpl;
> +    IMFAssetLocatorMap asset_locator_map;
> +    uint32_t track_count;
> +    IMFVirtualTrackPlaybackCtx **tracks;
> +} IMFContext;
> +
> +static int imf_uri_is_url(const char *string)
> +{
> +    return strstr(string, "://") != NULL;
> +}
> +
> +static int imf_uri_is_unix_abs_path(const char *string)
> +{
> +    return string[0] == '/';
> +}
> +
> +static int imf_uri_is_dos_abs_path(const char *string)
> +{
> +    /* Absolute path case: `C:\path\to\somwhere` */
> +    if (string[1] == ':' && string[2] == '\\')
> +        return 1;
> +
> +    /* Absolute path case: `C:/path/to/somwhere` */
> +    if (string[1] == ':' && string[2] == '/')
> +        return 1;
> +
> +    /* Network path case: `\\path\to\somwhere` */
> +    if (string[0] == '\\' && string[1] == '\\')
> +        return 1;
> +
> +    return 0;
> +}
> +
> +/**
> + * Parse a ASSETMAP XML file to extract the UUID-URI mapping of assets.
> + * @param s the current format context, if any (can be NULL).
> + * @param doc the XML document to be parsed.
> + * @param asset_map pointer on the IMFAssetLocatorMap to fill.
> + * @param base_url the url of the asset map XML file, if any (can be NULL).
> + * @return a negative value in case of error, 0 otherwise.
> + */
> +static int parse_imf_asset_map_from_xml_dom(AVFormatContext *s,
> +    xmlDocPtr doc,
> +    IMFAssetLocatorMap *asset_map,
> +    const char *base_url)
> +{
> +    xmlNodePtr asset_map_element = NULL;
> +    xmlNodePtr node = NULL;
> +    xmlNodePtr asset_element = NULL;
> +    char *uri;
> +    int ret = 0;
> +    IMFAssetLocator *asset = NULL;
> +    void *tmp;
> +
> +    asset_map_element = xmlDocGetRootElement(doc);
> +
> +    if (!asset_map_element) {
> +        av_log(s, AV_LOG_ERROR, "Unable to parse asset map XML - missing root node\n");
> +        return AVERROR_INVALIDDATA;
> +    }
> +
> +    if (asset_map_element->type != XML_ELEMENT_NODE || av_strcasecmp(asset_map_element->name, "AssetMap")) {
> +        av_log(s,
> +            AV_LOG_ERROR,
> +            "Unable to parse asset map XML - wrong root node name[%s] type[%d]\n",
> +            asset_map_element->name,
> +            (int)asset_map_element->type);
> +        return AVERROR_INVALIDDATA;
> +    }
> +
> +    /* parse asset locators */
> +    if (!(node = ff_imf_xml_get_child_element_by_name(asset_map_element, "AssetList"))) {
> +        av_log(s, AV_LOG_ERROR, "Unable to parse asset map XML - missing AssetList node\n");
> +        return AVERROR_INVALIDDATA;
> +    }
> +    tmp = av_realloc(asset_map->assets,
> +        (xmlChildElementCount(node) + asset_map->asset_count)
> +            * sizeof(IMFAssetLocator));
> +    if (!tmp) {
> +        av_log(NULL, AV_LOG_ERROR, "Cannot allocate IMF asset locators\n");
> +        return AVERROR(ENOMEM);
> +    }
> +    asset_map->assets = tmp;
> +
> +    asset_element = xmlFirstElementChild(node);
> +    while (asset_element) {
> +        if (av_strcasecmp(asset_element->name, "Asset") != 0)
> +            continue;
> +
> +        asset = &(asset_map->assets[asset_map->asset_count]);
> +
> +        if (ff_imf_xml_read_uuid(ff_imf_xml_get_child_element_by_name(asset_element, "Id"), asset->uuid)) {
> +            av_log(s, AV_LOG_ERROR, "Could not parse UUID from asset in asset map.\n");
> +            return AVERROR_INVALIDDATA;
> +        }
> +
> +        av_log(s, AV_LOG_DEBUG, "Found asset id: " FF_IMF_UUID_FORMAT "\n", UID_ARG(asset->uuid));
> +
> +        if (!(node = ff_imf_xml_get_child_element_by_name(asset_element, "ChunkList"))) {
> +            av_log(s, AV_LOG_ERROR, "Unable to parse asset map XML - missing ChunkList node\n");
> +            return AVERROR_INVALIDDATA;
> +        }
> +
> +        if (!(node = ff_imf_xml_get_child_element_by_name(node, "Chunk"))) {
> +            av_log(s, AV_LOG_ERROR, "Unable to parse asset map XML - missing Chunk node\n");
> +            return AVERROR_INVALIDDATA;
> +        }
> +
> +        uri = xmlNodeGetContent(ff_imf_xml_get_child_element_by_name(node, "Path"));
> +        if (!imf_uri_is_url(uri) && !imf_uri_is_unix_abs_path(uri) && !imf_uri_is_dos_abs_path(uri))
> +            asset->absolute_uri = av_append_path_component(base_url, uri);
> +        else
> +            asset->absolute_uri = av_strdup(uri);
> +        xmlFree(uri);
> +        if (!asset->absolute_uri) {
> +            return AVERROR(ENOMEM);
> +        }
> +
> +        av_log(s, AV_LOG_DEBUG, "Found asset absolute URI: %s\n", asset->absolute_uri);
> +
> +        asset_map->asset_count++;
> +        asset_element = xmlNextElementSibling(asset_element);
> +    }
> +
> +    return ret;
> +}
> +
> +/**
> + * Initializes an IMFAssetLocatorMap structure.
> + */
> +static void imf_asset_locator_map_init(IMFAssetLocatorMap *asset_map)
> +{
> +    asset_map->assets = NULL;
> +    asset_map->asset_count = 0;
> +}
> +
> +/**
> + * Free a IMFAssetLocatorMap pointer.
> + */
> +static void imf_asset_locator_map_deinit(IMFAssetLocatorMap *asset_map)
> +{
> +    for (uint32_t i = 0; i < asset_map->asset_count; ++i)
> +        av_freep(&asset_map->assets[i].absolute_uri);
> +    av_freep(&asset_map->assets);
> +}
> +
> +static int parse_assetmap(AVFormatContext *s, const char *url, AVIOContext *in)
> +{
> +    IMFContext *c = s->priv_data;
> +    struct AVBPrint buf;
> +    AVDictionary *opts = NULL;
> +    xmlDoc *doc = NULL;
> +    const char *base_url;
> +    char *tmp_str = NULL;
> +    int close_in = 0;
> +    int ret;
> +    int64_t filesize;
> +
> +    av_log(s, AV_LOG_DEBUG, "Asset Map URL: %s\n", url);
> +
> +    if (!in) {
> +        close_in = 1;
> +
> +        av_dict_copy(&opts, c->avio_opts, 0);
> +        ret = s->io_open(s, &in, url, AVIO_FLAG_READ, &opts);
> +        av_dict_free(&opts);
> +        if (ret < 0)
> +            return ret;
> +    }
> +
> +    filesize = avio_size(in);
> +    filesize = filesize > 0 ? filesize : DEFAULT_ASSETMAP_SIZE;
> +
> +    av_bprint_init(&buf, filesize + 1, AV_BPRINT_SIZE_UNLIMITED);
> +
> +    ret = avio_read_to_bprint(in, &buf, MAX_BPRINT_READ_SIZE);
> +    if (ret < 0 || !avio_feof(in) || (filesize = buf.len) == 0) {
> +        av_log(s, AV_LOG_ERROR, "Unable to read to asset map '%s'\n", url);
> +        if (ret == 0)
> +            ret = AVERROR_INVALIDDATA;
> +        goto clean_up;
> +    }
> +
> +    LIBXML_TEST_VERSION
> +
> +    tmp_str = av_strdup(url);
> +    if (!tmp_str) {
> +        ret = AVERROR(ENOMEM);
> +        goto clean_up;
> +    }
> +    base_url = av_dirname(tmp_str);
> +
> +    doc = xmlReadMemory(buf.str, filesize, url, NULL, 0);
> +
> +    ret = parse_imf_asset_map_from_xml_dom(s, doc, &c->asset_locator_map, base_url);
> +    if (!ret)
> +        av_log(s,
> +            AV_LOG_DEBUG,
> +            "Found %d assets from %s\n",
> +            c->asset_locator_map.asset_count,
> +            url);
> +
> +    xmlFreeDoc(doc);
> +
> +clean_up:
> +    if (tmp_str)
> +        av_freep(&tmp_str);
> +    if (close_in)
> +        avio_close(in);
> +    av_bprint_finalize(&buf, NULL);
> +
> +    return ret;
> +}
> +
> +static IMFAssetLocator *find_asset_map_locator(IMFAssetLocatorMap *asset_map, FFIMFUUID uuid)
> +{
> +    for (uint32_t i = 0; i < asset_map->asset_count; ++i)
> +        if (memcmp(asset_map->assets[i].uuid, uuid, 16) == 0)
> +            return &(asset_map->assets[i]);
> +    return NULL;
> +}
> +
> +static int open_track_resource_context(AVFormatContext *s,
> +    IMFVirtualTrackResourcePlaybackCtx *track_resource)
> +{
> +    IMFContext *c = s->priv_data;
> +    int ret = 0;
> +    int64_t entry_point;
> +    AVDictionary *opts = NULL;
> +
> +    if (!track_resource->ctx) {
> +        track_resource->ctx = avformat_alloc_context();
> +        if (!track_resource->ctx)
> +            return AVERROR(ENOMEM);
> +    }
> +
> +    if (track_resource->ctx->iformat) {
> +        av_log(s,
> +            AV_LOG_DEBUG,
> +            "Input context already opened for %s.\n",
> +            track_resource->locator->absolute_uri);
> +        return 0;
> +    }
> +
> +    track_resource->ctx->io_open = s->io_open;
> +    track_resource->ctx->io_close = s->io_close;
> +    track_resource->ctx->flags |= s->flags & ~AVFMT_FLAG_CUSTOM_IO;
> +
> +    if ((ret = ff_copy_whiteblacklists(track_resource->ctx, s)) < 0)
> +        goto cleanup;
> +
> +    av_dict_copy(&opts, c->avio_opts, 0);
> +    ret = avformat_open_input(&track_resource->ctx,
> +        track_resource->locator->absolute_uri,
> +        NULL,
> +        &opts);
> +    av_dict_free(&opts);
> +    if (ret < 0) {
> +        av_log(s,
> +            AV_LOG_ERROR,
> +            "Could not open %s input context: %s\n",
> +            track_resource->locator->absolute_uri,
> +            av_err2str(ret));
> +        return ret;
> +    }
> +
> +    ret = avformat_find_stream_info(track_resource->ctx, NULL);
> +    if (ret < 0) {
> +        av_log(s,
> +            AV_LOG_ERROR,
> +            "Could not find %s stream information: %s\n",
> +            track_resource->locator->absolute_uri,
> +            av_err2str(ret));
> +        goto cleanup;
> +    }
> +
> +    /* Compare the source timebase to the resource edit rate, considering the first stream of the source file */
> +    if (av_cmp_q(track_resource->ctx->streams[0]->time_base, av_inv_q(track_resource->resource->base.edit_rate)))
> +        av_log(s,
> +            AV_LOG_WARNING,
> +            "Incoherent source stream timebase %d/%d regarding resource edit rate: %d/%d",
> +            track_resource->ctx->streams[0]->time_base.num,
> +            track_resource->ctx->streams[0]->time_base.den,
> +            track_resource->resource->base.edit_rate.den,
> +            track_resource->resource->base.edit_rate.num);
> +
> +    entry_point = (int64_t)track_resource->resource->base.entry_point
> +        * track_resource->resource->base.edit_rate.den
> +        * AV_TIME_BASE
> +        / track_resource->resource->base.edit_rate.num;
> +
> +    if (entry_point) {
> +        av_log(s,
> +            AV_LOG_DEBUG,
> +            "Seek at resource %s entry point: %" PRIu32 "\n",
> +            track_resource->locator->absolute_uri,
> +            track_resource->resource->base.entry_point);
> +        ret = avformat_seek_file(track_resource->ctx, -1, entry_point, entry_point, entry_point, 0);
> +        if (ret < 0) {
> +            av_log(s,
> +                AV_LOG_ERROR,
> +                "Could not seek at %" PRId64 "on %s: %s\n",
> +                entry_point,
> +                track_resource->locator->absolute_uri,
> +                av_err2str(ret));
> +            goto cleanup;
> +        }
> +    }
> +
> +    return ret;
> +cleanup:
> +    avformat_free_context(track_resource->ctx);
> +    track_resource->ctx = NULL;
> +    return ret;
> +}
> +
> +static int open_track_file_resource(AVFormatContext *s,
> +    FFIMFTrackFileResource *track_file_resource,
> +    IMFVirtualTrackPlaybackCtx *track)
> +{
> +    IMFContext *c = s->priv_data;
> +    IMFAssetLocator *asset_locator;
> +    IMFVirtualTrackResourcePlaybackCtx vt_ctx;
> +    void *tmp;
> +    int ret;
> +
> +    asset_locator = find_asset_map_locator(&c->asset_locator_map, track_file_resource->track_file_uuid);
> +    if (!asset_locator) {
> +        av_log(s,
> +            AV_LOG_ERROR,
> +            "Could not find asset locator for UUID: " FF_IMF_UUID_FORMAT "\n",
> +            UID_ARG(track_file_resource->track_file_uuid));
> +        return AVERROR_INVALIDDATA;
> +    }
> +
> +    av_log(s,
> +        AV_LOG_DEBUG,
> +        "Found locator for " FF_IMF_UUID_FORMAT ": %s\n",
> +        UID_ARG(asset_locator->uuid),
> +        asset_locator->absolute_uri);
> +    tmp = av_fast_realloc(track->resources,
> +        &track->resources_alloc_sz,
> +        (track->resource_count + track_file_resource->base.repeat_count)
> +            * sizeof(IMFVirtualTrackResourcePlaybackCtx));
> +    if (!tmp)
> +        return AVERROR(ENOMEM);
> +    track->resources = tmp;
> +
> +    for (uint32_t i = 0; i < track_file_resource->base.repeat_count; ++i) {
> +        vt_ctx.locator = asset_locator;
> +        vt_ctx.resource = track_file_resource;
> +        vt_ctx.ctx = NULL;
> +        if ((ret = open_track_resource_context(s, &vt_ctx)) != 0)
> +            return ret;
> +        track->resources[track->resource_count++] = vt_ctx;
> +        track->duration = av_add_q(track->duration,
> +            av_make_q((int)track_file_resource->base.duration * track_file_resource->base.edit_rate.den,
> +                track_file_resource->base.edit_rate.num));
> +    }
> +
> +    return ret;
> +}
> +
> +static void imf_virtual_track_playback_context_deinit(IMFVirtualTrackPlaybackCtx *track)
> +{
> +    for (uint32_t i = 0; i < track->resource_count; ++i)
> +        if (track->resources[i].ctx && track->resources[i].ctx->iformat)
> +            avformat_close_input(&track->resources[i].ctx);
> +
> +    av_freep(&track->resources);
> +}
> +
> +static int open_virtual_track(AVFormatContext *s,
> +    FFIMFTrackFileVirtualTrack *virtual_track,
> +    int32_t track_index)
> +{
> +    IMFContext *c = s->priv_data;
> +    IMFVirtualTrackPlaybackCtx *track = NULL;
> +    void *tmp;
> +    int ret = 0;
> +
> +    if (!(track = av_mallocz(sizeof(IMFVirtualTrackPlaybackCtx))))
> +        return AVERROR(ENOMEM);
> +    track->index = track_index;
> +    track->duration = av_make_q(0, 1);
> +
> +    for (uint32_t i = 0; i < virtual_track->resource_count; i++) {
> +        av_log(s,
> +            AV_LOG_DEBUG,
> +            "Open stream from file " FF_IMF_UUID_FORMAT ", stream %d\n",
> +            UID_ARG(virtual_track->resources[i].track_file_uuid),
> +            i);
> +        if ((ret = open_track_file_resource(s, &virtual_track->resources[i], track)) != 0) {
> +            av_log(s,
> +                AV_LOG_ERROR,
> +                "Could not open image track resource " FF_IMF_UUID_FORMAT "\n",
> +                UID_ARG(virtual_track->resources[i].track_file_uuid));
> +            goto clean_up;
> +        }
> +    }
> +
> +    track->current_timestamp = av_make_q(0, track->duration.den);
> +
> +    tmp = av_realloc(c->tracks, (c->track_count + 1) * sizeof(IMFVirtualTrackPlaybackCtx *));
> +    if (!tmp) {
> +        ret = AVERROR(ENOMEM);
> +        goto clean_up;
> +    }
> +    c->tracks = tmp;
> +    c->tracks[c->track_count++] = track;
> +
> +    return 0;
> +
> +clean_up:
> +    imf_virtual_track_playback_context_deinit(track);
> +    av_free(track);
> +    return ret;
> +}
> +
> +static int set_context_streams_from_tracks(AVFormatContext *s)
> +{
> +    IMFContext *c = s->priv_data;
> +
> +    AVStream *asset_stream;
> +    AVStream *first_resource_stream;
> +
> +    int ret = 0;
> +
> +    for (uint32_t i = 0; i < c->track_count; ++i) {
> +        /* Open the first resource of the track to get stream information */
> +        first_resource_stream = c->tracks[i]->resources[0].ctx->streams[0];
> +        av_log(s, AV_LOG_DEBUG, "Open the first resource of track %d\n", c->tracks[i]->index);
> +
> +        /* Copy stream information */
> +        asset_stream = avformat_new_stream(s, NULL);
> +        if (!asset_stream) {
> +            av_log(s, AV_LOG_ERROR, "Could not create stream\n");
> +            break;
> +        }
> +        asset_stream->id = i;
> +        ret = avcodec_parameters_copy(asset_stream->codecpar, first_resource_stream->codecpar);
> +        if (ret < 0) {
> +            av_log(s, AV_LOG_ERROR, "Could not copy stream parameters\n");
> +            break;
> +        }
> +        avpriv_set_pts_info(asset_stream,
> +            first_resource_stream->pts_wrap_bits,
> +            first_resource_stream->time_base.num,
> +            first_resource_stream->time_base.den);
> +        asset_stream->duration = (int64_t)av_q2d(av_mul_q(c->tracks[i]->duration,
> +            av_inv_q(asset_stream->time_base)));
> +    }
> +
> +    return ret;
> +}
> +
> +static int save_avio_options(AVFormatContext *s)
> +{
> +    IMFContext *c = s->priv_data;
> +    static const char *const opts[] = {
> +        "headers", "http_proxy", "user_agent", "cookies", "referer", "rw_timeout", "icy", NULL};
> +    const char *const *opt = opts;
> +    uint8_t *buf;
> +    int ret = 0;
> +
> +    while (*opt) {
> +        if (av_opt_get(s->pb, *opt, AV_OPT_SEARCH_CHILDREN | AV_OPT_ALLOW_NULL, &buf) >= 0) {
> +            ret = av_dict_set(&c->avio_opts, *opt, buf, AV_DICT_DONT_STRDUP_VAL);
> +            if (ret < 0)
> +                return ret;
> +        }
> +        opt++;
> +    }
> +
> +    return ret;
> +}
> +
> +static int open_cpl_tracks(AVFormatContext *s)
> +{
> +    IMFContext *c = s->priv_data;
> +    int32_t track_index = 0;
> +    int ret;
> +
> +    if (c->cpl->main_image_2d_track)
> +        if ((ret = open_virtual_track(s, c->cpl->main_image_2d_track, track_index++)) != 0) {
> +            av_log(s,
> +                AV_LOG_ERROR,
> +                "Could not open image track " FF_IMF_UUID_FORMAT "\n",
> +                UID_ARG(c->cpl->main_image_2d_track->base.id_uuid));
> +            return ret;
> +        }
> +
> +    for (uint32_t i = 0; i < c->cpl->main_audio_track_count; ++i)
> +        if ((ret = open_virtual_track(s, &c->cpl->main_audio_tracks[i], track_index++)) != 0) {
> +            av_log(s,
> +                AV_LOG_ERROR,
> +                "Could not open audio track " FF_IMF_UUID_FORMAT "\n",
> +                UID_ARG(c->cpl->main_audio_tracks[i].base.id_uuid));
> +            return ret;
> +        }
> +
> +    return set_context_streams_from_tracks(s);
> +}
> +
> +static int imf_read_header(AVFormatContext *s)
> +{
> +    IMFContext *c = s->priv_data;
> +    char *asset_map_path;
> +    char *tmp_str;
> +    int ret = 0;
> +
> +    c->interrupt_callback = &s->interrupt_callback;
> +    tmp_str = av_strdup(s->url);
> +    if (!tmp_str) {
> +        ret = AVERROR(ENOMEM);
> +        return ret;
> +    }
> +    c->base_url = av_dirname(tmp_str);
> +    if ((ret = save_avio_options(s)) < 0)
> +        return ret;
> +
> +    av_log(s, AV_LOG_DEBUG, "start parsing IMF CPL: %s\n", s->url);
> +
> +    if ((ret = ff_imf_parse_cpl(s->pb, &c->cpl)) < 0)
> +        return ret;
> +
> +    av_log(s,
> +        AV_LOG_DEBUG,
> +        "parsed IMF CPL: " FF_IMF_UUID_FORMAT "\n",
> +        UID_ARG(c->cpl->id_uuid));
> +
> +    if (!c->asset_map_paths) {
> +        c->asset_map_paths = av_append_path_component(c->base_url, "ASSETMAP.xml");
> +        if (!c->asset_map_paths) {
> +            ret = AVERROR(ENOMEM);
> +            return ret;
> +        }
> +        av_log(s, AV_LOG_DEBUG, "No asset maps provided, using the default ASSETMAP.xml\n");
> +    }
> +
> +    /* Parse each asset map XML file */
> +    imf_asset_locator_map_init(&c->asset_locator_map);
> +    asset_map_path = av_strtok(c->asset_map_paths, ",", &tmp_str);
> +    while (asset_map_path != NULL) {
> +        av_log(s, AV_LOG_DEBUG, "start parsing IMF Asset Map: %s\n", asset_map_path);
> +
> +        if (ret = parse_assetmap(s, asset_map_path, NULL))
> +            return ret;
> +
> +        asset_map_path = av_strtok(NULL, ",", &tmp_str);
> +    }
> +
> +    av_log(s, AV_LOG_DEBUG, "parsed IMF Asset Maps\n");
> +
> +    if (ret = open_cpl_tracks(s))
> +        return ret;
> +
> +    av_log(s, AV_LOG_DEBUG, "parsed IMF package\n");
> +
> +    return 0;
> +}
> +
> +static IMFVirtualTrackPlaybackCtx *get_next_track_with_minimum_timestamp(AVFormatContext *s)
> +{
> +    IMFContext *c = s->priv_data;
> +    IMFVirtualTrackPlaybackCtx *track;
> +
> +    AVRational minimum_timestamp = av_make_q(INT32_MAX, 1);
> +    for (uint32_t i = 0; i < c->track_count; ++i) {
> +        av_log(
> +            s,
> +            AV_LOG_DEBUG,
> +            "Compare track %d timestamp " AVRATIONAL_FORMAT
> +            " to minimum " AVRATIONAL_FORMAT
> +            " (over duration: " AVRATIONAL_FORMAT
> +            ")\n",
> +            i,
> +            AVRATIONAL_ARG(c->tracks[i]->current_timestamp),
> +            AVRATIONAL_ARG(minimum_timestamp),
> +            AVRATIONAL_ARG(c->tracks[i]->duration));
> +        if (av_cmp_q(c->tracks[i]->current_timestamp, minimum_timestamp) < 0) {
> +            track = c->tracks[i];
> +            minimum_timestamp = track->current_timestamp;
> +        }
> +    }
> +
> +    av_log(s,
> +        AV_LOG_DEBUG,
> +        "Found next track to read: %d (timestamp: %lf / %lf)\n",
> +        track->index,
> +        av_q2d(track->current_timestamp),
> +        av_q2d(minimum_timestamp));
> +    return track;
> +}
> +
> +static IMFVirtualTrackResourcePlaybackCtx *get_resource_context_for_timestamp(AVFormatContext *s,
> +    IMFVirtualTrackPlaybackCtx *track)
> +{
> +    AVRational edit_unit_duration = av_inv_q(track->resources[0].resource->base.edit_rate);
> +    AVRational cumulated_duration = av_make_q(0, edit_unit_duration.den);
> +
> +    av_log(s,
> +        AV_LOG_DEBUG,
> +        "Looking for track %d resource for timestamp = %lf / %lf\n",
> +        track->index,
> +        av_q2d(track->current_timestamp),
> +        av_q2d(track->duration));
> +    for (uint32_t i = 0; i < track->resource_count; ++i) {
> +        cumulated_duration = av_add_q(cumulated_duration,
> +            av_make_q((int)track->resources[i].resource->base.duration * edit_unit_duration.num,
> +                edit_unit_duration.den));
> +
> +        if (av_cmp_q(av_add_q(track->current_timestamp, edit_unit_duration), cumulated_duration) <= 0) {
> +            av_log(s,
> +                AV_LOG_DEBUG,
> +                "Found resource %d in track %d to read for timestamp %lf "
> +                "(on cumulated=%lf): entry=%" PRIu32
> +                ", duration=%" PRIu32
> +                ", editrate=" AVRATIONAL_FORMAT
> +                " | edit_unit_duration=%lf\n",
> +                i,
> +                track->index,
> +                av_q2d(track->current_timestamp),
> +                av_q2d(cumulated_duration),
> +                track->resources[i].resource->base.entry_point,
> +                track->resources[i].resource->base.duration,
> +                AVRATIONAL_ARG(track->resources[i].resource->base.edit_rate),
> +                av_q2d(edit_unit_duration));
> +
> +            if (track->current_resource_index != i) {
> +                av_log(s,
> +                    AV_LOG_DEBUG,
> +                    "Switch resource on track %d: re-open context\n",
> +                    track->index);
> +                avformat_close_input(&(track->resources[track->current_resource_index].ctx));
> +                if (open_track_resource_context(s, &(track->resources[i])) != 0)
> +                    return NULL;
> +                track->current_resource_index = i;
> +            }
> +            return &(track->resources[track->current_resource_index]);
> +        }
> +    }
> +    return NULL;
> +}
> +
> +static int imf_read_packet(AVFormatContext *s, AVPacket *pkt)
> +{
> +    IMFContext *c = s->priv_data;
> +    IMFVirtualTrackResourcePlaybackCtx *resource_to_read = NULL;
> +    AVRational edit_unit_duration;
> +    int ret = 0;
> +    IMFVirtualTrackPlaybackCtx *track;
> +    FFStream *track_stream;
> +
> +    track = get_next_track_with_minimum_timestamp(s);
> +
> +    if (av_cmp_q(track->current_timestamp, track->duration) == 0)
> +        return AVERROR_EOF;
> +
> +    resource_to_read = get_resource_context_for_timestamp(s, track);
> +
> +    if (!resource_to_read) {
> +        edit_unit_duration = av_inv_q(track->resources[track->current_resource_index].resource->base.edit_rate);
> +        if (av_cmp_q(av_add_q(track->current_timestamp, edit_unit_duration),
> +                track->duration)
> +            > 0)
> +            return AVERROR_EOF;
> +
> +        av_log(s, AV_LOG_ERROR, "Could not find IMF track resource to read\n");
> +        return AVERROR_STREAM_NOT_FOUND;
> +    }
> +
> +    while (!ff_check_interrupt(c->interrupt_callback) && !ret) {
> +        ret = av_read_frame(resource_to_read->ctx, pkt);
> +        av_log(s,
> +            AV_LOG_DEBUG,
> +            "Got packet: pts=%" PRId64
> +            ", dts=%" PRId64
> +            ", duration=%" PRId64
> +            ", stream_index=%d, pos=%" PRId64
> +            "\n",
> +            pkt->pts,
> +            pkt->dts,
> +            pkt->duration,
> +            pkt->stream_index,
> +            pkt->pos);
> +        track_stream = ffstream(s->streams[track->index]);
> +        if (ret >= 0) {
> +            /* Update packet info from track */
> +            if (pkt->dts < track_stream->cur_dts && track->last_pts > 0)
> +                pkt->dts = track_stream->cur_dts;
> +
> +            pkt->pts = track->last_pts;
> +            pkt->dts = pkt->dts
> +                - (int64_t)track->resources[track->current_resource_index].resource->base.entry_point;
> +            pkt->stream_index = track->index;
> +
> +            /* Update track cursors */
> +            track->current_timestamp = av_add_q(track->current_timestamp,
> +                av_make_q((int)pkt->duration * resource_to_read->ctx->streams[0]->time_base.num,
> +                    resource_to_read->ctx->streams[0]->time_base.den));
> +            track->last_pts += pkt->duration;
> +
> +            return 0;
> +        } else if (ret != AVERROR_EOF) {
> +            av_log(s,
> +                AV_LOG_ERROR,
> +                "Could not get packet from track %d: %s\n",
> +                track->index,
> +                av_err2str(ret));
> +            return ret;
> +        }
> +    }
> +
> +    return AVERROR_EOF;
> +}
> +
> +static int imf_close(AVFormatContext *s)
> +{
> +    IMFContext *c = s->priv_data;
> +
> +    av_log(s, AV_LOG_DEBUG, "Close IMF package\n");
> +    av_dict_free(&c->avio_opts);
> +    av_freep(&c->base_url);
> +    imf_asset_locator_map_deinit(&c->asset_locator_map);
> +    ff_imf_cpl_free(c->cpl);
> +
> +    for (uint32_t i = 0; i < c->track_count; ++i) {
> +        imf_virtual_track_playback_context_deinit(c->tracks[i]);
> +        av_freep(&c->tracks[i]);
> +    }
> +
> +    av_freep(&c->tracks);
> +
> +    return 0;
> +}
> +
> +static int imf_probe(const AVProbeData *p)
> +{
> +    if (!strstr(p->buf, "<CompositionPlaylist"))
> +        return 0;
> +
> +    /* check for a ContentTitle element without including ContentTitleText,
> +     * which is used by the D-Cinema CPL.
> +     */
> +    if (!strstr(p->buf, "ContentTitle>"))
> +        return 0;
> +
> +    return AVPROBE_SCORE_MAX;
> +}
> +
> +static const AVOption imf_options[] = {
> +    {
> +        .name        = "assetmaps",
> +        .help        = "Comma-separated paths to ASSETMAP files."
> +                       "If not specified, the `ASSETMAP.xml` file in the same directory as the CPL is used.",
> +        .offset      = offsetof(IMFContext, asset_map_paths),
> +        .type        = AV_OPT_TYPE_STRING,
> +        .default_val = {.str = NULL},
> +        .flags       = AV_OPT_FLAG_DECODING_PARAM,
> +    },
> +    {NULL},
> +};
> +
> +static const AVClass imf_class = {
> +    .class_name = "imf",
> +    .item_name  = av_default_item_name,
> +    .option     = imf_options,
> +    .version    = LIBAVUTIL_VERSION_INT,
> +};
> +
> +const AVInputFormat ff_imf_demuxer = {
> +    .name           = "imf",
> +    .long_name      = NULL_IF_CONFIG_SMALL("IMF (Interoperable Master Format)"),
> +    .flags_internal = FF_FMT_INIT_CLEANUP,
> +    .priv_class     = &imf_class,
> +    .priv_data_size = sizeof(IMFContext),
> +    .read_probe     = imf_probe,
> +    .read_header    = imf_read_header,
> +    .read_packet    = imf_read_packet,
> +    .read_close     = imf_close
> +};
>

Quoting pal@sandflow.com (2021-12-13 06:43:35)
> From: Pierre-Anthony Lemieux <pal@palemieux.com>
> 
> Signed-off-by: Pierre-Anthony Lemieux <pal@palemieux.com>
> ---
> 
> Notes:
>     The IMF demuxer accepts as input an IMF CPL. The assets referenced by the CPL can be
>     contained in multiple deliveries, each defined by an ASSETMAP file:
>     
>     ffmpeg -assetmaps <path of ASSETMAP1>,<path of ASSETMAP>,... -i <path of CPL>
>     
>     If -assetmaps is not specified, FFMPEG looks for a file called ASSETMAP.xml in the same directory as the CPL.
>     
>     EXAMPLE:
>         ffmpeg -i http://ffmpeg-imf-samples-public.s3-website-us-west-1.amazonaws.com/countdown/CPL_f5095caa-f204-4e1c-8a84-7af48c7ae16b.xml out.mp4
>     
>     The Interoperable Master Format (IMF) is a file-based media format for the
>     delivery and storage of professional audio-visual masters.
>     An IMF Composition consists of an XML playlist (the Composition Playlist)
>     and a collection of MXF files (the Track Files). The Composition Playlist (CPL)

As far as I can tell, nothing enforces that the files opened are
actually MXF. Perhaps that should be done. Otherwise I can imagine at
least the danger of recursion.

More generally, I am somewhat concerned about the security implications
of all this. From a brief glance at the patch, the demuxer just opens
whatever arbitrary URLs it finds in the asset maps. Have you considered
what undesirable effects (like information leaks) this might have?

Quoting pal@sandflow.com (2021-12-13 06:43:35)
> +static int parse_assetmap(AVFormatContext *s, const char *url, AVIOContext *in)

in is always NULL, might as well make it a local variable

> +{
> +    IMFContext *c = s->priv_data;
> +    struct AVBPrint buf;
> +    AVDictionary *opts = NULL;
> +    xmlDoc *doc = NULL;
> +    const char *base_url;
> +    char *tmp_str = NULL;
> +    int close_in = 0;
> +    int ret;
> +    int64_t filesize;
> +
> +    av_log(s, AV_LOG_DEBUG, "Asset Map URL: %s\n", url);
> +
> +    if (!in) {
> +        close_in = 1;
> +
> +        av_dict_copy(&opts, c->avio_opts, 0);
> +        ret = s->io_open(s, &in, url, AVIO_FLAG_READ, &opts);
> +        av_dict_free(&opts);
> +        if (ret < 0)
> +            return ret;
> +    }
> +
> +    filesize = avio_size(in);
> +    filesize = filesize > 0 ? filesize : DEFAULT_ASSETMAP_SIZE;
> +
> +    av_bprint_init(&buf, filesize + 1, AV_BPRINT_SIZE_UNLIMITED);
> +
> +    ret = avio_read_to_bprint(in, &buf, MAX_BPRINT_READ_SIZE);
> +    if (ret < 0 || !avio_feof(in) || (filesize = buf.len) == 0) {

Stealth assignments in if () clauses are sinful and a recipe for bugs.
Especially when a part of several statements. Please don't.

> +static int open_track_resource_context(AVFormatContext *s,
> +    IMFVirtualTrackResourcePlaybackCtx *track_resource)
> +{
> +    IMFContext *c = s->priv_data;
> +    int ret = 0;
> +    int64_t entry_point;
> +    AVDictionary *opts = NULL;
> +
> +    if (!track_resource->ctx) {
> +        track_resource->ctx = avformat_alloc_context();
> +        if (!track_resource->ctx)
> +            return AVERROR(ENOMEM);
> +    }
> +
> +    if (track_resource->ctx->iformat) {
> +        av_log(s,
> +            AV_LOG_DEBUG,
> +            "Input context already opened for %s.\n",
> +            track_resource->locator->absolute_uri);
> +        return 0;
> +    }
> +
> +    track_resource->ctx->io_open = s->io_open;
> +    track_resource->ctx->io_close = s->io_close;
> +    track_resource->ctx->flags |= s->flags & ~AVFMT_FLAG_CUSTOM_IO;
> +
> +    if ((ret = ff_copy_whiteblacklists(track_resource->ctx, s)) < 0)
> +        goto cleanup;
> +
> +    av_dict_copy(&opts, c->avio_opts, 0);
> +    ret = avformat_open_input(&track_resource->ctx,
> +        track_resource->locator->absolute_uri,
> +        NULL,
> +        &opts);
> +    av_dict_free(&opts);
> +    if (ret < 0) {
> +        av_log(s,
> +            AV_LOG_ERROR,
> +            "Could not open %s input context: %s\n",
> +            track_resource->locator->absolute_uri,
> +            av_err2str(ret));
> +        return ret;
> +    }
> +
> +    ret = avformat_find_stream_info(track_resource->ctx, NULL);
> +    if (ret < 0) {
> +        av_log(s,
> +            AV_LOG_ERROR,
> +            "Could not find %s stream information: %s\n",
> +            track_resource->locator->absolute_uri,
> +            av_err2str(ret));
> +        goto cleanup;
> +    }

Is this really necessary? This might potentially take a long time and
invoke decoders and so should be avoided unless you really need it.

On Tue, Dec 14, 2021 at 2:33 AM Anton Khirnov <anton@khirnov.net> wrote:
>
> Quoting pal@sandflow.com (2021-12-13 06:43:35)
> > +static int parse_assetmap(AVFormatContext *s, const char *url, AVIOContext *in)
>
> in is always NULL, might as well make it a local variable

Addressed by v11 of the patchset.

>
> > +{
> > +    IMFContext *c = s->priv_data;
> > +    struct AVBPrint buf;
> > +    AVDictionary *opts = NULL;
> > +    xmlDoc *doc = NULL;
> > +    const char *base_url;
> > +    char *tmp_str = NULL;
> > +    int close_in = 0;
> > +    int ret;
> > +    int64_t filesize;
> > +
> > +    av_log(s, AV_LOG_DEBUG, "Asset Map URL: %s\n", url);
> > +
> > +    if (!in) {
> > +        close_in = 1;
> > +
> > +        av_dict_copy(&opts, c->avio_opts, 0);
> > +        ret = s->io_open(s, &in, url, AVIO_FLAG_READ, &opts);
> > +        av_dict_free(&opts);
> > +        if (ret < 0)
> > +            return ret;
> > +    }
> > +
> > +    filesize = avio_size(in);
> > +    filesize = filesize > 0 ? filesize : DEFAULT_ASSETMAP_SIZE;
> > +
> > +    av_bprint_init(&buf, filesize + 1, AV_BPRINT_SIZE_UNLIMITED);
> > +
> > +    ret = avio_read_to_bprint(in, &buf, MAX_BPRINT_READ_SIZE);
> > +    if (ret < 0 || !avio_feof(in) || (filesize = buf.len) == 0) {
>
> Stealth assignments in if () clauses are sinful and a recipe for bugs.
> Especially when a part of several statements. Please don't.

Addressed by v11 of the patchset.

>
> > +static int open_track_resource_context(AVFormatContext *s,
> > +    IMFVirtualTrackResourcePlaybackCtx *track_resource)
> > +{
> > +    IMFContext *c = s->priv_data;
> > +    int ret = 0;
> > +    int64_t entry_point;
> > +    AVDictionary *opts = NULL;
> > +
> > +    if (!track_resource->ctx) {
> > +        track_resource->ctx = avformat_alloc_context();
> > +        if (!track_resource->ctx)
> > +            return AVERROR(ENOMEM);
> > +    }
> > +
> > +    if (track_resource->ctx->iformat) {
> > +        av_log(s,
> > +            AV_LOG_DEBUG,
> > +            "Input context already opened for %s.\n",
> > +            track_resource->locator->absolute_uri);
> > +        return 0;
> > +    }
> > +
> > +    track_resource->ctx->io_open = s->io_open;
> > +    track_resource->ctx->io_close = s->io_close;
> > +    track_resource->ctx->flags |= s->flags & ~AVFMT_FLAG_CUSTOM_IO;
> > +
> > +    if ((ret = ff_copy_whiteblacklists(track_resource->ctx, s)) < 0)
> > +        goto cleanup;
> > +
> > +    av_dict_copy(&opts, c->avio_opts, 0);
> > +    ret = avformat_open_input(&track_resource->ctx,
> > +        track_resource->locator->absolute_uri,
> > +        NULL,
> > +        &opts);
> > +    av_dict_free(&opts);
> > +    if (ret < 0) {
> > +        av_log(s,
> > +            AV_LOG_ERROR,
> > +            "Could not open %s input context: %s\n",
> > +            track_resource->locator->absolute_uri,
> > +            av_err2str(ret));
> > +        return ret;
> > +    }
> > +
> > +    ret = avformat_find_stream_info(track_resource->ctx, NULL);
> > +    if (ret < 0) {
> > +        av_log(s,
> > +            AV_LOG_ERROR,
> > +            "Could not find %s stream information: %s\n",
> > +            track_resource->locator->absolute_uri,
> > +            av_err2str(ret));
> > +        goto cleanup;
> > +    }
>
> Is this really necessary? This might potentially take a long time and
> invoke decoders and so should be avoided unless you really need it.

Addressed by v11 of the patchset.

By restricting Track Files to MXF, I think we can avoid
avformat_find_stream_info() since the MXF demuxer fills in
streams->time_base on opening.

>
> --
> Anton Khirnov

On Tue, Dec 14, 2021 at 2:31 AM Anton Khirnov <anton@khirnov.net> wrote:
>
> Quoting pal@sandflow.com (2021-12-13 06:43:35)
> > From: Pierre-Anthony Lemieux <pal@palemieux.com>
> >
> > Signed-off-by: Pierre-Anthony Lemieux <pal@palemieux.com>
> > ---
> >
> > Notes:
> >     The IMF demuxer accepts as input an IMF CPL. The assets referenced by the CPL can be
> >     contained in multiple deliveries, each defined by an ASSETMAP file:
> >
> >     ffmpeg -assetmaps <path of ASSETMAP1>,<path of ASSETMAP>,... -i <path of CPL>
> >
> >     If -assetmaps is not specified, FFMPEG looks for a file called ASSETMAP.xml in the same directory as the CPL.
> >
> >     EXAMPLE:
> >         ffmpeg -i http://ffmpeg-imf-samples-public.s3-website-us-west-1.amazonaws.com/countdown/CPL_f5095caa-f204-4e1c-8a84-7af48c7ae16b.xml out.mp4
> >
> >     The Interoperable Master Format (IMF) is a file-based media format for the
> >     delivery and storage of professional audio-visual masters.
> >     An IMF Composition consists of an XML playlist (the Composition Playlist)
> >     and a collection of MXF files (the Track Files). The Composition Playlist (CPL)
>
> As far as I can tell, nothing enforces that the files opened are
> actually MXF. Perhaps that should be done. Otherwise I can imagine at
> least the danger of recursion.

Addressed by v11 of the patchset.

Track Files are now constrained to "mxf".

>
> More generally, I am somewhat concerned about the security implications
> of all this. From a brief glance at the patch, the demuxer just opens
> whatever arbitrary URLs it finds in the asset maps. Have you considered
> what undesirable effects (like information leaks) this might have?

The ASSETMAP file, where the demuxer finds the URLs of the Track
Files, is an integral part of an IMF delivery and is assumed to be
trusted. This is typically achieved by retrieving the ASSETMAP over a
trusted channel, e.g. HTTPS to a trusted server or on a trusted local
drive, etc.

>
> --
> Anton Khirnov

Quoting Pierre-Anthony Lemieux (2021-12-14 17:52:48)
> On Tue, Dec 14, 2021 at 2:31 AM Anton Khirnov <anton@khirnov.net> wrote:
> >
> > Quoting pal@sandflow.com (2021-12-13 06:43:35)
> > > From: Pierre-Anthony Lemieux <pal@palemieux.com>
> > >
> > > Signed-off-by: Pierre-Anthony Lemieux <pal@palemieux.com>
> > > ---
> > >
> > > Notes:
> > >     The IMF demuxer accepts as input an IMF CPL. The assets referenced by the CPL can be
> > >     contained in multiple deliveries, each defined by an ASSETMAP file:
> > >
> > >     ffmpeg -assetmaps <path of ASSETMAP1>,<path of ASSETMAP>,... -i <path of CPL>
> > >
> > >     If -assetmaps is not specified, FFMPEG looks for a file called ASSETMAP.xml in the same directory as the CPL.
> > >
> > >     EXAMPLE:
> > >         ffmpeg -i http://ffmpeg-imf-samples-public.s3-website-us-west-1.amazonaws.com/countdown/CPL_f5095caa-f204-4e1c-8a84-7af48c7ae16b.xml out.mp4
> > >
> > >     The Interoperable Master Format (IMF) is a file-based media format for the
> > >     delivery and storage of professional audio-visual masters.
> > >     An IMF Composition consists of an XML playlist (the Composition Playlist)
> > >     and a collection of MXF files (the Track Files). The Composition Playlist (CPL)
> >
> > As far as I can tell, nothing enforces that the files opened are
> > actually MXF. Perhaps that should be done. Otherwise I can imagine at
> > least the danger of recursion.
> 
> Addressed by v11 of the patchset.
> 
> Track Files are now constrained to "mxf".
> 
> >
> > More generally, I am somewhat concerned about the security implications
> > of all this. From a brief glance at the patch, the demuxer just opens
> > whatever arbitrary URLs it finds in the asset maps. Have you considered
> > what undesirable effects (like information leaks) this might have?
> 
> The ASSETMAP file, where the demuxer finds the URLs of the Track
> Files, is an integral part of an IMF delivery and is assumed to be
> trusted. This is typically achieved by retrieving the ASSETMAP over a
> trusted channel, e.g. HTTPS to a trusted server or on a trusted local
> drive, etc.

I don't think you can just assume that much, since libavformat is
commonly used with untrusted input. It's good that at least the assetmap
file path is not specified in the main CPL file, but I can still imagine
a situation where the user somehow receives a directory with a
CPL+ASSETMAP.xml from an untrusted source and tries to open it e.g. in
VLC.

Now the question is whether a malicious attacker can craft those two
files to get access to anything they shouldn't. I suppose at the very
least the attacker can get information that the user opened the file (by
adding an asset on an attacker's server) but that will be a danger with
any playlists allowing network resources and can be controlled with
io_open(). Can you think of any other possible issues?

>
> Now the question is whether a malicious attacker can craft those two
> files to get access to anything they shouldn't. I suppose at the very
> least the attacker can get information that the user opened the file (by
> adding an asset on an attacker's server) but that will be a danger with
> any playlists allowing network resources and can be controlled with
> io_open(). Can you think of any other possible issues?
>

Some security considerations:

- a DDoS can conceivably occur if a malicious CPL+ASSETMAP is widely
distributed. Both an ASSETMAP and a CPL are required since (a) the CPL
does not contain paths/hyperlinks and (b) only those resources
referenced by the CPL are fetched using the ASSETMAP.
- the CPL uses XML, which has its own security considerations. For
example, XML parsing can result in entities being fetched over the
network, but this is disabled by default in libxml AFAIK.
- several elements/attributes of the IMF CPL use URIs as unique
identifiers. These URIs could conceivably be dereferenced.
Dereferencing these URIs is however not a requirement and the IMF
demuxer does not do so.
- IMF only uses MXF to wrap essence but supports various kinds of
essence, e.g. Prores and J2K, each with its own security
considerations
- IMF has a mechanism to associate arbitrary files with a CPL. This is
not required for processing of the CPL and is not implemented by the
IMF demuxer.
- IMF includes an (optional) XML digital signature mechanism that
allows a user to confirm the origin and authenticity of the CPL,
preventing malicious insertion of hyperlinks. XML digital signature
has its own security considerations.

Many of these security considerations are shared with DASH and HLS.

Quoting Pierre-Anthony Lemieux (2021-12-15 01:17:26)
> >
> > Now the question is whether a malicious attacker can craft those two
> > files to get access to anything they shouldn't. I suppose at the very
> > least the attacker can get information that the user opened the file (by
> > adding an asset on an attacker's server) but that will be a danger with
> > any playlists allowing network resources and can be controlled with
> > io_open(). Can you think of any other possible issues?
> >
> 
> Some security considerations:
> 
> - a DDoS can conceivably occur if a malicious CPL+ASSETMAP is widely
> distributed. Both an ASSETMAP and a CPL are required since (a) the CPL
> does not contain paths/hyperlinks and (b) only those resources
> referenced by the CPL are fetched using the ASSETMAP.
> - the CPL uses XML, which has its own security considerations. For
> example, XML parsing can result in entities being fetched over the
> network, but this is disabled by default in libxml AFAIK.

This is concerning. From a brief glance at libxml2, it seems that you
need to pass XML_PARSE_NONET as the last parameter to xmlReadMemory() to
actually disabling network fetching.
But it is possible I'm misreading the code, so if you or anyone else
understands this better then clarifications are welcome.

> - several elements/attributes of the IMF CPL use URIs as unique
> identifiers. These URIs could conceivably be dereferenced.
> Dereferencing these URIs is however not a requirement and the IMF
> demuxer does not do so.
> - IMF only uses MXF to wrap essence but supports various kinds of
> essence, e.g. Prores and J2K, each with its own security
> considerations

The demuxer should not be concerned about what the user does with the
returned data. Only the behavior of the demuxer itself (and whatever
code it calls) is the question here.

On Wed, Dec 15, 2021 at 12:20 PM Anton Khirnov <anton@khirnov.net> wrote:
>
> Quoting Pierre-Anthony Lemieux (2021-12-15 01:17:26)
> > >
> > > Now the question is whether a malicious attacker can craft those two
> > > files to get access to anything they shouldn't. I suppose at the very
> > > least the attacker can get information that the user opened the file (by
> > > adding an asset on an attacker's server) but that will be a danger with
> > > any playlists allowing network resources and can be controlled with
> > > io_open(). Can you think of any other possible issues?
> > >
> >
> > Some security considerations:
> >
> > - a DDoS can conceivably occur if a malicious CPL+ASSETMAP is widely
> > distributed. Both an ASSETMAP and a CPL are required since (a) the CPL
> > does not contain paths/hyperlinks and (b) only those resources
> > referenced by the CPL are fetched using the ASSETMAP.
> > - the CPL uses XML, which has its own security considerations. For
> > example, XML parsing can result in entities being fetched over the
> > network, but this is disabled by default in libxml AFAIK.
>
> This is concerning. From a brief glance at libxml2, it seems that you
> need to pass XML_PARSE_NONET as the last parameter to xmlReadMemory() to
> actually disabling network fetching.
> But it is possible I'm misreading the code, so if you or anyone else
> understands this better then clarifications are welcome.

I was referring to entity expansion and the loading of DTDs being
disabled by default -- see XML_PARSE_NOENT and XML_PARSE_DTDLOAD at
[1-2].

[1] http://xmlsoft.org/html/libxml-parser.html
[2] https://cheatsheetseries.owasp.org/cheatsheets/XML_External_Entity_Prevention_Cheat_Sheet.html

>
> > - several elements/attributes of the IMF CPL use URIs as unique
> > identifiers. These URIs could conceivably be dereferenced.
> > Dereferencing these URIs is however not a requirement and the IMF
> > demuxer does not do so.
> > - IMF only uses MXF to wrap essence but supports various kinds of
> > essence, e.g. Prores and J2K, each with its own security
> > considerations
>
> The demuxer should not be concerned about what the user does with the
> returned data. Only the behavior of the demuxer itself (and whatever
> code it calls) is the question here.
>
> --
> Anton Khirnov
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Quoting Pierre-Anthony Lemieux (2021-12-15 21:41:25)
> On Wed, Dec 15, 2021 at 12:20 PM Anton Khirnov <anton@khirnov.net> wrote:
> >
> > Quoting Pierre-Anthony Lemieux (2021-12-15 01:17:26)
> > > >
> > > > Now the question is whether a malicious attacker can craft those two
> > > > files to get access to anything they shouldn't. I suppose at the very
> > > > least the attacker can get information that the user opened the file (by
> > > > adding an asset on an attacker's server) but that will be a danger with
> > > > any playlists allowing network resources and can be controlled with
> > > > io_open(). Can you think of any other possible issues?
> > > >
> > >
> > > Some security considerations:
> > >
> > > - a DDoS can conceivably occur if a malicious CPL+ASSETMAP is widely
> > > distributed. Both an ASSETMAP and a CPL are required since (a) the CPL
> > > does not contain paths/hyperlinks and (b) only those resources
> > > referenced by the CPL are fetched using the ASSETMAP.
> > > - the CPL uses XML, which has its own security considerations. For
> > > example, XML parsing can result in entities being fetched over the
> > > network, but this is disabled by default in libxml AFAIK.
> >
> > This is concerning. From a brief glance at libxml2, it seems that you
> > need to pass XML_PARSE_NONET as the last parameter to xmlReadMemory() to
> > actually disabling network fetching.
> > But it is possible I'm misreading the code, so if you or anyone else
> > understands this better then clarifications are welcome.
> 
> I was referring to entity expansion and the loading of DTDs being
> disabled by default -- see XML_PARSE_NOENT and XML_PARSE_DTDLOAD at
> [1-2].

Okay then. If nobody has further comments, I will push your latest patch
in a few days.

Dec 17, 2021, 3:25 PM by anton@khirnov.net:

> Quoting Pierre-Anthony Lemieux (2021-12-15 21:41:25)
>
>> On Wed, Dec 15, 2021 at 12:20 PM Anton Khirnov <anton@khirnov.net> wrote:
>> >
>> > Quoting Pierre-Anthony Lemieux (2021-12-15 01:17:26)
>> > > >
>> > > > Now the question is whether a malicious attacker can craft those two
>> > > > files to get access to anything they shouldn't. I suppose at the very
>> > > > least the attacker can get information that the user opened the file (by
>> > > > adding an asset on an attacker's server) but that will be a danger with
>> > > > any playlists allowing network resources and can be controlled with
>> > > > io_open(). Can you think of any other possible issues?
>> > > >
>> > >
>> > > Some security considerations:
>> > >
>> > > - a DDoS can conceivably occur if a malicious CPL+ASSETMAP is widely
>> > > distributed. Both an ASSETMAP and a CPL are required since (a) the CPL
>> > > does not contain paths/hyperlinks and (b) only those resources
>> > > referenced by the CPL are fetched using the ASSETMAP.
>> > > - the CPL uses XML, which has its own security considerations. For
>> > > example, XML parsing can result in entities being fetched over the
>> > > network, but this is disabled by default in libxml AFAIK.
>> >
>> > This is concerning. From a brief glance at libxml2, it seems that you
>> > need to pass XML_PARSE_NONET as the last parameter to xmlReadMemory() to
>> > actually disabling network fetching.
>> > But it is possible I'm misreading the code, so if you or anyone else
>> > understands this better then clarifications are welcome.
>>
>> I was referring to entity expansion and the loading of DTDs being
>> disabled by default -- see XML_PARSE_NOENT and XML_PARSE_DTDLOAD at
>> [1-2].
>>
>
> Okay then. If nobody has further comments, I will push your latest patch
> in a few days.
>

I think this shouldn't get merged into 5.0. It would get minimal amount
of fuzzing if it does now, so let's leave it for a later release?
I'd still like to see libuuid being used, we have several uses for it already.

On Fri, Dec 17, 2021 at 12:54 PM Lynne <dev@lynne.ee> wrote:
>
> Dec 17, 2021, 3:25 PM by anton@khirnov.net:
>
> > Quoting Pierre-Anthony Lemieux (2021-12-15 21:41:25)
> >
> >> On Wed, Dec 15, 2021 at 12:20 PM Anton Khirnov <anton@khirnov.net> wrote:
> >> >
> >> > Quoting Pierre-Anthony Lemieux (2021-12-15 01:17:26)
> >> > > >
> >> > > > Now the question is whether a malicious attacker can craft those two
> >> > > > files to get access to anything they shouldn't. I suppose at the very
> >> > > > least the attacker can get information that the user opened the file (by
> >> > > > adding an asset on an attacker's server) but that will be a danger with
> >> > > > any playlists allowing network resources and can be controlled with
> >> > > > io_open(). Can you think of any other possible issues?
> >> > > >
> >> > >
> >> > > Some security considerations:
> >> > >
> >> > > - a DDoS can conceivably occur if a malicious CPL+ASSETMAP is widely
> >> > > distributed. Both an ASSETMAP and a CPL are required since (a) the CPL
> >> > > does not contain paths/hyperlinks and (b) only those resources
> >> > > referenced by the CPL are fetched using the ASSETMAP.
> >> > > - the CPL uses XML, which has its own security considerations. For
> >> > > example, XML parsing can result in entities being fetched over the
> >> > > network, but this is disabled by default in libxml AFAIK.
> >> >
> >> > This is concerning. From a brief glance at libxml2, it seems that you
> >> > need to pass XML_PARSE_NONET as the last parameter to xmlReadMemory() to
> >> > actually disabling network fetching.
> >> > But it is possible I'm misreading the code, so if you or anyone else
> >> > understands this better then clarifications are welcome.
> >>
> >> I was referring to entity expansion and the loading of DTDs being
> >> disabled by default -- see XML_PARSE_NOENT and XML_PARSE_DTDLOAD at
> >> [1-2].
> >>
> >
> > Okay then. If nobody has further comments, I will push your latest patch
> > in a few days.
> >
>
> I think this shouldn't get merged into 5.0. It would get minimal amount
> of fuzzing if it does now, so let's leave it for a later release?

What amount of fuzzing are you looking for? Is there an HLS and/or
DASH equivalent? Happy to put in the work to make it happen.

> I'd still like to see libuuid being used, we have several uses for it already.

libuuid is not an obvious match with the IMF demuxer -- see details in
previous thread. I am happy to work on a UUID refactoring across
libavformat, but this is a separable task with impact on several other
modules.

>
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request@ffmpeg.org with subject "unsubscribe".

Quoting Lynne (2021-12-17 21:54:14)
> Dec 17, 2021, 3:25 PM by anton@khirnov.net:
> 
> > Quoting Pierre-Anthony Lemieux (2021-12-15 21:41:25)
> >
> >> On Wed, Dec 15, 2021 at 12:20 PM Anton Khirnov <anton@khirnov.net> wrote:
> >> >
> >> > Quoting Pierre-Anthony Lemieux (2021-12-15 01:17:26)
> >> > > >
> >> > > > Now the question is whether a malicious attacker can craft those two
> >> > > > files to get access to anything they shouldn't. I suppose at the very
> >> > > > least the attacker can get information that the user opened the file (by
> >> > > > adding an asset on an attacker's server) but that will be a danger with
> >> > > > any playlists allowing network resources and can be controlled with
> >> > > > io_open(). Can you think of any other possible issues?
> >> > > >
> >> > >
> >> > > Some security considerations:
> >> > >
> >> > > - a DDoS can conceivably occur if a malicious CPL+ASSETMAP is widely
> >> > > distributed. Both an ASSETMAP and a CPL are required since (a) the CPL
> >> > > does not contain paths/hyperlinks and (b) only those resources
> >> > > referenced by the CPL are fetched using the ASSETMAP.
> >> > > - the CPL uses XML, which has its own security considerations. For
> >> > > example, XML parsing can result in entities being fetched over the
> >> > > network, but this is disabled by default in libxml AFAIK.
> >> >
> >> > This is concerning. From a brief glance at libxml2, it seems that you
> >> > need to pass XML_PARSE_NONET as the last parameter to xmlReadMemory() to
> >> > actually disabling network fetching.
> >> > But it is possible I'm misreading the code, so if you or anyone else
> >> > understands this better then clarifications are welcome.
> >>
> >> I was referring to entity expansion and the loading of DTDs being
> >> disabled by default -- see XML_PARSE_NOENT and XML_PARSE_DTDLOAD at
> >> [1-2].
> >>
> >
> > Okay then. If nobody has further comments, I will push your latest patch
> > in a few days.
> >
> 
> I think this shouldn't get merged into 5.0. It would get minimal amount
> of fuzzing if it does now, so let's leave it for a later release?
> I'd still like to see libuuid being used, we have several uses for it already.

I don't like this kind of reasoning. Plenty of things get no fuzzing at
all, because they have no tests, yet they go in without complaint.