From patchwork Wed Jul 27 16:12:02 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vignesh Venkat X-Patchwork-Id: 37010 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a21:1649:b0:8b:613a:194d with SMTP id no9csp453931pzb; Wed, 27 Jul 2022 09:12:20 -0700 (PDT) X-Google-Smtp-Source: AGRyM1vkEqMtbkM7PlJlCfVPrmkFOUMRHnLaPqDpWpqA7EVZe1ZXLPlcmBlMIuAlWlI/G7cVK05i X-Received: by 2002:a17:907:1c0c:b0:72e:dd1f:a230 with SMTP id nc12-20020a1709071c0c00b0072edd1fa230mr18498108ejc.8.1658938339823; Wed, 27 Jul 2022 09:12:19 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1658938339; cv=none; d=google.com; s=arc-20160816; b=D8Upf5/9dd0mkZ9vWXFVs7bYv6UzT6b4evdcD39JcWwd0VFiXCvgMnvXAj1j9neC/r SJyIt3Qu/LxopedtEskC332+WlQ/HEprsDSRJUQWg4Tl4x+aPPRmmJIwB71DtDZ4/PQk GncYGB/qrXokG02M9TALZ7ASeT8gFiY6ybVSuyifzpxrynUyo7EgXHMjLHOCxxugW3R8 9vNIwY9NPO5nmZ+dEvLkkErqVHth/9ps6ob1BotfZLEctA/ZJxoIFLXDDTaywa8NiyJo cWs1k5TZIb4EnYS1PwhtW6fyKTDAj+fu5Pm7DPABDEFEedMgprB3AdIA4ja46Uc+h6t7 X8xw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:to:from:references:mime-version :message-id:in-reply-to:date:dkim-signature:delivered-to; bh=csKT+a+teiCoGkOzemcer1Wzp8sbF1hElZSi0NqRRFs=; b=DKTe+1jXIKiYVDStDef+xIe97uA9jYN5sRWyhOoGQOMeBliuaJjaO/Ju+xJjWxTxZC BFre9ILf8vCvjWGdC3JQGhMWLyG2TsFMPG0PQwpr3ALGqxcqF3JOcY9ICOe7lTst1SkG av3xgosbNY+C/WK6iK48crPK1hh202W0v9LlB4/+E3GKfbImiLrVpWXCmAyHzfhKGFwO 0+LIw0JW9sAyPOLXXUpYFzzpgqBtxjyFs9oa1UiudPnaoc+THMYCDfefqCjG0zF0btv9 j8vdt5aU6TmEoWxa3BpDFexJ2dIyXEllBY8051vlzaJItyFc5XTyrZcg0lbz9nVYa6R8 iTTA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@google.com header.s=20210112 header.b=o2Jt6BcK; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id nc41-20020a1709071c2900b007263709af8fsi13891129ejc.995.2022.07.27.09.12.18; Wed, 27 Jul 2022 09:12:19 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@google.com header.s=20210112 header.b=o2Jt6BcK; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id E392268B8A9; Wed, 27 Jul 2022 19:12:15 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pf1-f201.google.com (mail-pf1-f201.google.com [209.85.210.201]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 45F7968B501 for ; Wed, 27 Jul 2022 19:12:09 +0300 (EEST) Received: by mail-pf1-f201.google.com with SMTP id a19-20020aa780d3000000b0052bccd363f8so4725778pfn.22 for ; Wed, 27 Jul 2022 09:12:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=cc:to:from:subject:references:mime-version:message-id:in-reply-to :date:from:to:cc; bh=J4G9E7lzJU9zZJ1khxz/xLizXXdINranC+suSlX4gAc=; b=o2Jt6BcK+iuVCuGp3FAIpdSiHpPIbcBdn0yFbDWmYp8En1YjQEC9N9+dCXzfQ1aFVq Uvmhwhv3x6FV6maUqrU7kN7w3FGBRXt0R3bGlNEI7IifZlgGaIx2b+v/h0uOSs6QZOf8 5LxrWdq4XXUEut7filZ24Rde3zw9In8KCg+JKKj2D45LT+edjLFb6zXIabsrzi4/PpPc oAPIAgSRFc/Kmpj20WFP8IPulEiOOwNXQA8IGFp0wQg+QOugHMad4aivACGoGEfsn4dc xnRV8AHigtfm0rU5TzPx4sW6cUgInPq7QtvLRBwlFJKKHYgBGer8WJuomSNvdqN9tJ1Y C/xw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:from:subject:references:mime-version:message-id:in-reply-to :date:x-gm-message-state:from:to:cc; bh=J4G9E7lzJU9zZJ1khxz/xLizXXdINranC+suSlX4gAc=; b=VtKodotLjMwAfCs9BMzOMB/W2n9wy1TYz6lUiMHJbjNDtJ5r1LdzE8OFN18ebsJwlA L8B8dZHKTS2NjlPK7eYNCQHCtUz+i4g1zE+GeOcXD+kxONiWJMSrDT7dQzNNwCcitiII Xo+woRhqQu5wA6QXvMpi47efMpm6kJea5NTXI1htDl0kw2WmSh90Fwkpz/qmiEe/4cJy 5ED/Pzn7Z3skVSBZfMTrfIa+4xBE5KjKiDtmMmp8W+rtAc6iQSn6ZP//Dh8zlenp/scM eClQoavf05uq8XUG093icvyPN8jRCgnNUrFcf7oWFDr9Tj7ZjAcCSBUL5+8RyR9rEAkY tg3Q== X-Gm-Message-State: AJIora+qPoWFNdurPAjMJCY0aP7oUYLz83YFsGPeDmz5k4MIYhd/X5rH 2Lrycq9imsMn97vNOkGwLrOrpdU/3aF/mDHwH2Aj3Od0HM2gZgty2hKqaSyWHhz6kRnIBYCL5Tk NYLVklRjW6sK5uUU6tFYT+7cJO5v7bOT+H+XzU2mq112Ev/v4rgnoUfYYwKlg5A24Grcm X-Received: from vigneshv3.mtv.corp.google.com ([2620:0:1000:2511:f837:3580:fc09:a5a3]) (user=vigneshv job=sendgmr) by 2002:a17:902:e54d:b0:16d:6bd8:58cd with SMTP id n13-20020a170902e54d00b0016d6bd858cdmr16336890plf.66.1658938327039; Wed, 27 Jul 2022 09:12:07 -0700 (PDT) Date: Wed, 27 Jul 2022 09:12:02 -0700 In-Reply-To: Message-Id: <20220727161202.2764806-1-vigneshv@google.com> Mime-Version: 1.0 References: X-Mailer: git-send-email 2.37.1.359.gd136c6c3e2-goog From: Vignesh Venkatasubramanian To: ffmpeg-devel@ffmpeg.org Subject: [FFmpeg-devel] [PATCH 1/2] avformat/mov: Rework the AVIF parser to handle multiple items X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Vignesh Venkatasubramanian Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: OoySBblXSl1+ Stores the item ids of all the items found in the file and processes the primary item at the end of the meta box. This patch does not change any behavior. It sets up the code for parsing alpha channel (and possibly images with 'grid') in follow up patches. Signed-off-by: Vignesh Venkatasubramanian --- libavformat/isom.h | 4 ++ libavformat/mov.c | 146 ++++++++++++++++++++++++++++----------------- 2 files changed, 95 insertions(+), 55 deletions(-) diff --git a/libavformat/isom.h b/libavformat/isom.h index f05c2d9c28..d8b262e915 100644 --- a/libavformat/isom.h +++ b/libavformat/isom.h @@ -318,6 +318,10 @@ typedef struct MOVContext { uint32_t max_stts_delta; int is_still_picture_avif; int primary_item_id; + int *avif_item_ids; + int avif_item_ids_size; + int *avif_extent_lengths; + int64_t *avif_extent_offsets; } MOVContext; int ff_mp4_read_descr_len(AVIOContext *pb); diff --git a/libavformat/mov.c b/libavformat/mov.c index a09a762d91..fc6a691da4 100644 --- a/libavformat/mov.c +++ b/libavformat/mov.c @@ -4698,6 +4698,69 @@ static int mov_read_custom(MOVContext *c, AVIOContext *pb, MOVAtom atom) return ret; } +static int avif_add_stream(MOVContext *c, int item_id) +{ + MOVStreamContext *sc; + AVStream *st; + int item_index = -1; + for (int i = 0; i < c->avif_item_ids_size; i++) + if (c->avif_item_ids[i] == item_id) { + item_index = i; + break; + } + if (item_index < 0) + return AVERROR_INVALIDDATA; + st = avformat_new_stream(c->fc, NULL); + if (!st) + return AVERROR(ENOMEM); + st->id = c->fc->nb_streams; + sc = av_mallocz(sizeof(MOVStreamContext)); + if (!sc) + return AVERROR(ENOMEM); + + st->priv_data = sc; + st->codecpar->codec_type = AVMEDIA_TYPE_VIDEO; + st->codecpar->codec_id = AV_CODEC_ID_AV1; + sc->ffindex = st->index; + c->trak_index = st->index; + st->avg_frame_rate.num = st->avg_frame_rate.den = 1; + st->time_base.num = st->time_base.den = 1; + st->nb_frames = 1; + sc->time_scale = 1; + sc = st->priv_data; + sc->pb = c->fc->pb; + sc->pb_is_copied = 1; + + // Populate the necessary fields used by mov_build_index. + sc->stsc_count = 1; + sc->stsc_data = av_malloc_array(1, sizeof(*sc->stsc_data)); + if (!sc->stsc_data) + return AVERROR(ENOMEM); + sc->stsc_data[0].first = 1; + sc->stsc_data[0].count = 1; + sc->stsc_data[0].id = 1; + sc->chunk_count = 1; + sc->chunk_offsets = av_malloc_array(1, sizeof(*sc->chunk_offsets)); + if (!sc->chunk_offsets) + return AVERROR(ENOMEM); + sc->sample_count = 1; + sc->sample_sizes = av_malloc_array(1, sizeof(*sc->sample_sizes)); + if (!sc->sample_sizes) + return AVERROR(ENOMEM); + sc->stts_count = 1; + sc->stts_data = av_malloc_array(1, sizeof(*sc->stts_data)); + if (!sc->stts_data) + return AVERROR(ENOMEM); + sc->stts_data[0].count = 1; + // Not used for still images. But needed by mov_build_index. + sc->stts_data[0].duration = 0; + sc->sample_sizes[0] = c->avif_extent_lengths[item_index]; + sc->chunk_offsets[0] = c->avif_extent_offsets[item_index]; + + mov_build_index(c, st); + return 0; +} + static int mov_read_meta(MOVContext *c, AVIOContext *pb, MOVAtom atom) { while (atom.size > 8) { @@ -4707,9 +4770,23 @@ static int mov_read_meta(MOVContext *c, AVIOContext *pb, MOVAtom atom) tag = avio_rl32(pb); atom.size -= 4; if (tag == MKTAG('h','d','l','r')) { + int ret; avio_seek(pb, -8, SEEK_CUR); atom.size += 8; - return mov_read_default(c, pb, atom); + if ((ret = mov_read_default(c, pb, atom)) < 0) + return ret; + if (c->is_still_picture_avif) { + int ret; + // Add a stream for the YUV planes (primary item). + if ((ret = avif_add_stream(c, c->primary_item_id)) < 0) + return ret; + // For still AVIF images, the meta box contains all the + // necessary information that would generally be provided by the + // moov box. So simply mark that we have found the moov box so + // that parsing can continue. + c->found_moov = 1; + } + return ret; } } return 0; @@ -7478,8 +7555,6 @@ static int mov_read_iloc(MOVContext *c, AVIOContext *pb, MOVAtom atom) int item_count, extent_count; uint64_t base_offset, extent_offset, extent_length; uint8_t value; - AVStream *st; - MOVStreamContext *sc; if (!c->is_still_picture_avif) { // * For non-avif, we simply ignore the iloc box. @@ -7493,27 +7568,6 @@ static int mov_read_iloc(MOVContext *c, AVIOContext *pb, MOVAtom atom) return 0; } - st = avformat_new_stream(c->fc, NULL); - if (!st) - return AVERROR(ENOMEM); - st->id = c->fc->nb_streams; - sc = av_mallocz(sizeof(MOVStreamContext)); - if (!sc) - return AVERROR(ENOMEM); - - st->priv_data = sc; - st->codecpar->codec_type = AVMEDIA_TYPE_VIDEO; - st->codecpar->codec_id = AV_CODEC_ID_AV1; - sc->ffindex = st->index; - c->trak_index = st->index; - st->avg_frame_rate.num = st->avg_frame_rate.den = 1; - st->time_base.num = st->time_base.den = 1; - st->nb_frames = 1; - sc->time_scale = 1; - sc = st->priv_data; - sc->pb = c->fc->pb; - sc->pb_is_copied = 1; - version = avio_r8(pb); avio_rb24(pb); // flags. @@ -7529,34 +7583,22 @@ static int mov_read_iloc(MOVContext *c, AVIOContext *pb, MOVAtom atom) } item_count = (version < 2) ? avio_rb16(pb) : avio_rb32(pb); - // Populate the necessary fields used by mov_build_index. - sc->stsc_count = 1; - sc->stsc_data = av_malloc_array(1, sizeof(*sc->stsc_data)); - if (!sc->stsc_data) + c->avif_item_ids = av_malloc_array(item_count, sizeof(*c->avif_item_ids)); + if (!c->avif_item_ids) return AVERROR(ENOMEM); - sc->stsc_data[0].first = 1; - sc->stsc_data[0].count = 1; - sc->stsc_data[0].id = 1; - sc->chunk_count = 1; - sc->chunk_offsets = av_malloc_array(1, sizeof(*sc->chunk_offsets)); - if (!sc->chunk_offsets) + c->avif_item_ids_size = item_count; + c->avif_extent_lengths = av_malloc_array(item_count, sizeof(*c->avif_extent_lengths)); + if (!c->avif_extent_lengths) return AVERROR(ENOMEM); - sc->sample_count = 1; - sc->sample_sizes = av_malloc_array(1, sizeof(*sc->sample_sizes)); - if (!sc->sample_sizes) + c->avif_extent_offsets = av_malloc_array(item_count, sizeof(*c->avif_extent_offsets)); + if (!c->avif_extent_offsets) return AVERROR(ENOMEM); - sc->stts_count = 1; - sc->stts_data = av_malloc_array(1, sizeof(*sc->stts_data)); - if (!sc->stts_data) - return AVERROR(ENOMEM); - sc->stts_data[0].count = 1; - // Not used for still images. But needed by mov_build_index. - sc->stts_data[0].duration = 0; for (int i = 0; i < item_count; i++) { int item_id = (version < 2) ? avio_rb16(pb) : avio_rb32(pb); if (avio_feof(pb)) return AVERROR_INVALIDDATA; + c->avif_item_ids[i] = item_id; if (version > 0) avio_rb16(pb); // construction_method. avio_rb16(pb); // data_reference_index. @@ -7572,20 +7614,11 @@ static int mov_read_iloc(MOVContext *c, AVIOContext *pb, MOVAtom atom) if (rb_size(pb, &extent_offset, offset_size) < 0 || rb_size(pb, &extent_length, length_size) < 0) return AVERROR_INVALIDDATA; - if (item_id == c->primary_item_id) { - sc->sample_sizes[0] = extent_length; - sc->chunk_offsets[0] = base_offset + extent_offset; - } + c->avif_extent_lengths[i] = extent_length; + c->avif_extent_offsets[i] = base_offset + extent_offset; } } - mov_build_index(c, st); - - // For still AVIF images, the iloc box contains all the necessary - // information that would generally be provided by the moov box. So simply - // mark that we have found the moov box so that parsing can continue. - c->found_moov = 1; - return atom.size; } @@ -8189,6 +8222,9 @@ static int mov_read_close(AVFormatContext *s) av_freep(&mov->aes_decrypt); av_freep(&mov->chapter_tracks); + av_freep(&mov->avif_item_ids); + av_freep(&mov->avif_extent_lengths); + av_freep(&mov->avif_extent_offsets); return 0; }