From patchwork Wed Jul 13 16:11:52 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vignesh Venkat X-Patchwork-Id: 36777 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a21:6da0:b0:8b:e47:9dbf with SMTP id wl32csp462536pzb; Wed, 13 Jul 2022 09:12:07 -0700 (PDT) X-Google-Smtp-Source: AGRyM1t6MEcVirQGTMNTiJK+BTpDoPq9Nw/Dwqlhtv4SWlHP41hiFP4MUWFglxsZAXYav+okeK82 X-Received: by 2002:a05:6402:3591:b0:43a:9076:e2be with SMTP id y17-20020a056402359100b0043a9076e2bemr6041945edc.171.1657728727226; Wed, 13 Jul 2022 09:12:07 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1657728727; cv=none; d=google.com; s=arc-20160816; b=to6d5qk0vvkjsFQ8Gvr0gMyyEFCOPRbYY7Hl8GvRkM8dhYTtHH959RvggdlSF9m6O8 OdnZvLT1SVZX3hfUJ3K7qSxQioTZZ4C7FRjb5X0lqG0Gm0Fn2lswmMW+DiLT77qo9200 o22VEeJK4FiWjZqAEZxL91rRavRyP0rZxh8ADS0mIYYtmYJuIVaogQ9f1ZuUoWez2nJ9 CavGL8kpN1EfNsGrx1DhV5tJMKTcVn+r2McbcASUgfWzdu8RjjrOaoxEUJk0Qy5nZe4s zelSMugB7maI3L8k5VENP6GpdIPHrkBZp76J+FfY4UNT/TuxJ6hOuP2t2BjTG8XMfLlP bu/g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:to:from:references:mime-version :message-id:in-reply-to:date:dkim-signature:delivered-to; bh=eweqkUKUCl40zR/cjLV1lg5TYzI6JruV82OZJK60uls=; b=StLrd50jRqAWC2rXxDcO7oqL9h2Tqm/xZ1VyXfu7SQUW/cTO5pAFo5Za6CaG+i72ul ibX/WZ3f+/QWZzhd6usdzERriHXJ1rQym+N3Qw2ipuNpKxJ6pBMaRoYELy29Fsr1ur46 Am5t3d2XddZ1CTkwiBhN5uubo+0fGSefE9WfpJ6G9BbPTrUVaUFZxqsRXREA7SDe/xyH uq/c2A9MqOPIGyx3n6lLYCJyjtPhDYSYr4AJ5cktyYEe1AGtwhqMeH7ALBtfruw1ZV7W pe2PB2Ja+yww/jM1lxu62bGLHC2I/90Sd8zvtsxqllvveigv++sRICrqC03F6sG2lze8 j3nw== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@google.com header.s=20210112 header.b=kqo6Mckt; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id m18-20020a170906721200b00722e7756f3dsi189891ejk.462.2022.07.13.09.12.06; Wed, 13 Jul 2022 09:12:07 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@google.com header.s=20210112 header.b=kqo6Mckt; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 0218B68B9E4; Wed, 13 Jul 2022 19:12:04 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-yw1-f202.google.com (mail-yw1-f202.google.com [209.85.128.202]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 621FA68B82B for ; Wed, 13 Jul 2022 19:11:57 +0300 (EEST) Received: by mail-yw1-f202.google.com with SMTP id 00721157ae682-31c9f68d48cso98052097b3.0 for ; Wed, 13 Jul 2022 09:11:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=+AYmKYUuDsKIY/g4W64oxV6oS8SIhxjANrBm9QYR4uw=; b=kqo6McktYso88TyNUfNqIjOG/OVVtEJd8+b2AFWvTLJaal1OOArHSiPXlrUEa6nrUQ VbaoDIZ+Abk6m+Ao5lsdZhBizlg2/EFNPzs6zLGUSu0OIoMdiE71B27ItKJrBL7PBSz2 J8UAYy+uxA4uCkx545kUPMtqfi98NJ6pVzcABcmQig+Q43gkvjDkbaUXVpDIlqKdRVVU REPa84FVcBc2f3agF5q6t+ukoCLywPxLdqHb5AVSc0nTubidPcNbESElbryfq2L9Hq8J 4y3PrtFf01psWCLYUTKwk1ID69G7nDovIGL5pFZI9ilTm4BsZLhHMRCRhK4Wuw+1wQp/ cfJg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=+AYmKYUuDsKIY/g4W64oxV6oS8SIhxjANrBm9QYR4uw=; b=grfhyot+9Li2i33Y7Om0rdk0z4pwLEEUqeVZzZrdyYiYAg4INekbuYVsZrdCDoVQ5Q 5EqaywSX5ih2Sx8IFYxVXC+OsBzGQ69O39EdgDUsJGNg/ugtWQjqaYkhw20SblfWEB2F KixjZq8kJI13lDaNRata9W3ju4P0urNWsUsGIkM6ylLvUgVkplZwWzFOIyhTQjZU2qbK 15q5QDpyCW8l2NnHuC8RMbuuqkvwga7rzazXryu+i0BmDu1Y/jb0NC3dgsDuEv6fnfRb WNaWJ735mmampc6VQNTrkaUly0/dYov1HDbAlzkOLA3C1IQl8m8iva10nGbYAksvR+Pu WFIQ== X-Gm-Message-State: AJIora9ndgck80Jfc0vo3Nq5e8y5ZWqdIsu1lgHgoUy3ss99Bmee8mQ8 759sT9BOiZJ4iOcOvCSb/zcwoK2j1VjRaDX1ZDKAsfEyTyU0k6SMCMtjFkAP7o4Wx2dBwi/mQL1 JljgJkaTdNvq+krHKyKCNK8OE9HzfHhOywvrZvuZlgQbmUPOpKtfJOBrlXNfjE68LdUvb X-Received: from vigneshv3.mtv.corp.google.com ([2620:0:1000:2511:dd57:3f80:fd6f:4d2e]) (user=vigneshv job=sendgmr) by 2002:a25:aaae:0:b0:66e:48de:9272 with SMTP id t43-20020a25aaae000000b0066e48de9272mr4286629ybi.523.1657728715993; Wed, 13 Jul 2022 09:11:55 -0700 (PDT) Date: Wed, 13 Jul 2022 09:11:52 -0700 In-Reply-To: Message-Id: <20220713161153.2337146-1-vigneshv@google.com> Mime-Version: 1.0 References: X-Mailer: git-send-email 2.37.0.144.g8ac04bfd2-goog From: Vignesh Venkatasubramanian To: ffmpeg-devel@ffmpeg.org Subject: [FFmpeg-devel] [PATCH 1/2] avformat/mov: Rework the AVIF parser to handle multiple items X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Vignesh Venkatasubramanian Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: yZkFhkXjc0hw Stores the item ids of all the items found in the file and processes the primary item at the end of the meta box. This patch does not change any behavior. It sets up the code for parsing alpha channel (and possibly images with 'grid') in follow up patches. Signed-off-by: Vignesh Venkatasubramanian --- libavformat/isom.h | 4 ++ libavformat/mov.c | 146 ++++++++++++++++++++++++++++----------------- 2 files changed, 95 insertions(+), 55 deletions(-) diff --git a/libavformat/isom.h b/libavformat/isom.h index f05c2d9c28..d8b262e915 100644 --- a/libavformat/isom.h +++ b/libavformat/isom.h @@ -318,6 +318,10 @@ typedef struct MOVContext { uint32_t max_stts_delta; int is_still_picture_avif; int primary_item_id; + int *avif_item_ids; + int avif_item_ids_size; + int *avif_extent_lengths; + int64_t *avif_extent_offsets; } MOVContext; int ff_mp4_read_descr_len(AVIOContext *pb); diff --git a/libavformat/mov.c b/libavformat/mov.c index 88669faa70..cd87088f3e 100644 --- a/libavformat/mov.c +++ b/libavformat/mov.c @@ -4683,6 +4683,69 @@ static int mov_read_custom(MOVContext *c, AVIOContext *pb, MOVAtom atom) return ret; } +static int avif_add_stream(MOVContext *c, int item_id) +{ + MOVStreamContext *sc; + AVStream *st; + int item_index = -1; + for (int i = 0; i < c->avif_item_ids_size; i++) + if (c->avif_item_ids[i] == item_id) { + item_index = i; + break; + } + if (item_index < 0) + return AVERROR_INVALIDDATA; + st = avformat_new_stream(c->fc, NULL); + if (!st) + return AVERROR(ENOMEM); + st->id = c->fc->nb_streams; + sc = av_mallocz(sizeof(MOVStreamContext)); + if (!sc) + return AVERROR(ENOMEM); + + st->priv_data = sc; + st->codecpar->codec_type = AVMEDIA_TYPE_VIDEO; + st->codecpar->codec_id = AV_CODEC_ID_AV1; + sc->ffindex = st->index; + c->trak_index = st->index; + st->avg_frame_rate.num = st->avg_frame_rate.den = 1; + st->time_base.num = st->time_base.den = 1; + st->nb_frames = 1; + sc->time_scale = 1; + sc = st->priv_data; + sc->pb = c->fc->pb; + sc->pb_is_copied = 1; + + // Populate the necessary fields used by mov_build_index. + sc->stsc_count = 1; + sc->stsc_data = av_malloc_array(1, sizeof(*sc->stsc_data)); + if (!sc->stsc_data) + return AVERROR(ENOMEM); + sc->stsc_data[0].first = 1; + sc->stsc_data[0].count = 1; + sc->stsc_data[0].id = 1; + sc->chunk_count = 1; + sc->chunk_offsets = av_malloc_array(1, sizeof(*sc->chunk_offsets)); + if (!sc->chunk_offsets) + return AVERROR(ENOMEM); + sc->sample_count = 1; + sc->sample_sizes = av_malloc_array(1, sizeof(*sc->sample_sizes)); + if (!sc->sample_sizes) + return AVERROR(ENOMEM); + sc->stts_count = 1; + sc->stts_data = av_malloc_array(1, sizeof(*sc->stts_data)); + if (!sc->stts_data) + return AVERROR(ENOMEM); + sc->stts_data[0].count = 1; + // Not used for still images. But needed by mov_build_index. + sc->stts_data[0].duration = 0; + sc->sample_sizes[0] = c->avif_extent_lengths[item_index]; + sc->chunk_offsets[0] = c->avif_extent_offsets[item_index]; + + mov_build_index(c, st); + return 0; +} + static int mov_read_meta(MOVContext *c, AVIOContext *pb, MOVAtom atom) { while (atom.size > 8) { @@ -4692,9 +4755,23 @@ static int mov_read_meta(MOVContext *c, AVIOContext *pb, MOVAtom atom) tag = avio_rl32(pb); atom.size -= 4; if (tag == MKTAG('h','d','l','r')) { + int ret; avio_seek(pb, -8, SEEK_CUR); atom.size += 8; - return mov_read_default(c, pb, atom); + if ((ret = mov_read_default(c, pb, atom)) < 0) + return ret; + if (c->is_still_picture_avif) { + int ret; + // Add a stream for the YUV planes (primary item). + if ((ret = avif_add_stream(c, c->primary_item_id)) < 0) + return ret; + // For still AVIF images, the meta box contains all the + // necessary information that would generally be provided by the + // moov box. So simply mark that we have found the moov box so + // that parsing can continue. + c->found_moov = 1; + } + return ret; } } return 0; @@ -7483,8 +7560,6 @@ static int mov_read_iloc(MOVContext *c, AVIOContext *pb, MOVAtom atom) int item_count, extent_count; uint64_t base_offset, extent_offset, extent_length; uint8_t value; - AVStream *st; - MOVStreamContext *sc; if (!c->is_still_picture_avif) { // * For non-avif, we simply ignore the iloc box. @@ -7498,27 +7573,6 @@ static int mov_read_iloc(MOVContext *c, AVIOContext *pb, MOVAtom atom) return 0; } - st = avformat_new_stream(c->fc, NULL); - if (!st) - return AVERROR(ENOMEM); - st->id = c->fc->nb_streams; - sc = av_mallocz(sizeof(MOVStreamContext)); - if (!sc) - return AVERROR(ENOMEM); - - st->priv_data = sc; - st->codecpar->codec_type = AVMEDIA_TYPE_VIDEO; - st->codecpar->codec_id = AV_CODEC_ID_AV1; - sc->ffindex = st->index; - c->trak_index = st->index; - st->avg_frame_rate.num = st->avg_frame_rate.den = 1; - st->time_base.num = st->time_base.den = 1; - st->nb_frames = 1; - sc->time_scale = 1; - sc = st->priv_data; - sc->pb = c->fc->pb; - sc->pb_is_copied = 1; - version = avio_r8(pb); avio_rb24(pb); // flags. @@ -7534,32 +7588,20 @@ static int mov_read_iloc(MOVContext *c, AVIOContext *pb, MOVAtom atom) } item_count = (version < 2) ? avio_rb16(pb) : avio_rb32(pb); - // Populate the necessary fields used by mov_build_index. - sc->stsc_count = 1; - sc->stsc_data = av_malloc_array(1, sizeof(*sc->stsc_data)); - if (!sc->stsc_data) + c->avif_item_ids = av_malloc_array(item_count, sizeof(*c->avif_item_ids)); + if (!c->avif_item_ids) return AVERROR(ENOMEM); - sc->stsc_data[0].first = 1; - sc->stsc_data[0].count = 1; - sc->stsc_data[0].id = 1; - sc->chunk_count = 1; - sc->chunk_offsets = av_malloc_array(1, sizeof(*sc->chunk_offsets)); - if (!sc->chunk_offsets) + c->avif_item_ids_size = item_count; + c->avif_extent_lengths = av_malloc_array(item_count, sizeof(*c->avif_extent_lengths)); + if (!c->avif_extent_lengths) return AVERROR(ENOMEM); - sc->sample_count = 1; - sc->sample_sizes = av_malloc_array(1, sizeof(*sc->sample_sizes)); - if (!sc->sample_sizes) + c->avif_extent_offsets = av_malloc_array(item_count, sizeof(*c->avif_extent_offsets)); + if (!c->avif_extent_offsets) return AVERROR(ENOMEM); - sc->stts_count = 1; - sc->stts_data = av_malloc_array(1, sizeof(*sc->stts_data)); - if (!sc->stts_data) - return AVERROR(ENOMEM); - sc->stts_data[0].count = 1; - // Not used for still images. But needed by mov_build_index. - sc->stts_data[0].duration = 0; for (int i = 0; i < item_count; i++) { int item_id = (version < 2) ? avio_rb16(pb) : avio_rb32(pb); + c->avif_item_ids[i] = item_id; if (version > 0) avio_rb16(pb); // construction_method. avio_rb16(pb); // data_reference_index. @@ -7575,20 +7617,11 @@ static int mov_read_iloc(MOVContext *c, AVIOContext *pb, MOVAtom atom) if (rb_size(pb, &extent_offset, offset_size) < 0 || rb_size(pb, &extent_length, length_size) < 0) return AVERROR_INVALIDDATA; - if (item_id == c->primary_item_id) { - sc->sample_sizes[0] = extent_length; - sc->chunk_offsets[0] = base_offset + extent_offset; - } + c->avif_extent_lengths[i] = extent_length; + c->avif_extent_offsets[i] = base_offset + extent_offset; } } - mov_build_index(c, st); - - // For still AVIF images, the iloc box contains all the necessary - // information that would generally be provided by the moov box. So simply - // mark that we have found the moov box so that parsing can continue. - c->found_moov = 1; - return atom.size; } @@ -8192,6 +8225,9 @@ static int mov_read_close(AVFormatContext *s) av_freep(&mov->aes_decrypt); av_freep(&mov->chapter_tracks); + av_freep(&mov->avif_item_ids); + av_freep(&mov->avif_extent_lengths); + av_freep(&mov->avif_extent_offsets); return 0; }