From patchwork Thu Jul 28 18:25:16 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vignesh Venkat X-Patchwork-Id: 37028 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a21:1649:b0:8b:613a:194d with SMTP id no9csp438120pzb; Thu, 28 Jul 2022 11:25:35 -0700 (PDT) X-Google-Smtp-Source: AGRyM1vw0nFqhk7nElulM02COdnkZtNUOceiqk04w8A7DAYih4P59gX3B2ZM3jU/GJIU7VnbdJgY X-Received: by 2002:a17:906:5d04:b0:722:f46c:b891 with SMTP id g4-20020a1709065d0400b00722f46cb891mr176392ejt.4.1659032735418; Thu, 28 Jul 2022 11:25:35 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1659032735; cv=none; d=google.com; s=arc-20160816; b=mZaYkr3sclj4l0avCOK42jdyKhAi3hXXED6tZtz41CmA4pmuWt+l/WsokoTZENgsDb oiU3lzJD4xhP9VmEDJnhWpx2gZxq2Bx62ec3xrkqAX1zEnf5nR7Vg/NUDl1I+0t8bsDK NOZ9qESVeT4n5MvDYNSV2chWbP+nQYdqAJXd5SOQPD9G/2BNPB04CSVxESSneGKlw2Lu HPrxlpBcDYvrM+Ys6uAQToaZA26a9AU6jLQj5WcFC2cZRBZemfVuikKMkYai+oB5+4Fy hnyha6zOwSXsydRdsRo/qYCvLpFjcFNoIkBoAog6EYUSWDeLenRWC6zifpiM86ruWzyu 9WFQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:to:from:references:mime-version :message-id:in-reply-to:date:dkim-signature:delivered-to; bh=csb3MQt2cCYXmSUCyTrKWnG94ILt0KhomxR+40NPhsc=; b=d0PFxvcIaW+W90JEYSXq4lN777ZbapNuqXEBClqp6p9T+KZI/GAd0VZI3HkPcrRmMb DDrhwy5HE5RGkQijMVUwLntAsimZUeFcbP+Xd9fm7j0Lr1AdcIB/Ne0McZ8eljwBkPOb ZTHmsfEx2/99NBQNvW4KiztHLIvx0RJ0t3lQK/LqFdcq0V91sckLTpIoYBp6egJvK8dw yV/BvAg5H99s6DQ4aB9FgTVirMkRX86Xtwn0QX8lG0LPMOMs9ihH1GcmVtxisM2T+BqE NLWeZ1oNUCQ+WEgpRtCzJSncb6+jBMs2trguSvZ8IJ1RXi0I8m6xjf9+/YbzXmFVE5Pb THjQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@google.com header.s=20210112 header.b=cQQhBJmW; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id fp6-20020a1709069e0600b0072af3c1acf5si1021227ejc.657.2022.07.28.11.25.35; Thu, 28 Jul 2022 11:25:35 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@google.com header.s=20210112 header.b=cQQhBJmW; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 525D868B9B5; Thu, 28 Jul 2022 21:25:27 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-yw1-f202.google.com (mail-yw1-f202.google.com [209.85.128.202]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id ECF2F68B8C4 for ; Thu, 28 Jul 2022 21:25:20 +0300 (EEST) Received: by mail-yw1-f202.google.com with SMTP id 00721157ae682-31f49eea05fso22867227b3.15 for ; Thu, 28 Jul 2022 11:25:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=cc:to:from:subject:references:mime-version:message-id:in-reply-to :date:from:to:cc; bh=FMaT6dl7gTpa0aUGRc0fORkbcMk6fX7mvooSKA7kCnw=; b=cQQhBJmW9yFsdR3EjWKZ59P68avAThi/aRuh9t/C72YMcUay+8w98O7EdoyW+hIHqA WS1TdQpev/BolKZS+6cwcibZCMeHUeTvBuF/3RYJNPBY52OQLhTKqdyo2XglKOqlq6Ni ebAyPys4GuHXdmcyDUbeIrPJ6UKqh2Ww35Tfv96HHVXkkbPktbXOUjZm85g04l9iCehz iJTOcrR57jvCxPaFs64lE/xkuGNO9wE4gnimRryOpThwwIXtYHFzceJGS+/nz6IDzHak 9obcJx9oSSgWL/AsSxbnEKmQlzK2jA7poOa0y7CfOLqyBo/iZ/UauGjdcAoMlbxSwGkj E6ZQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:from:subject:references:mime-version:message-id:in-reply-to :date:x-gm-message-state:from:to:cc; bh=FMaT6dl7gTpa0aUGRc0fORkbcMk6fX7mvooSKA7kCnw=; b=m7qw0K91oitbsuBFErzm9vV7odo1VUeNKepRXNjNzaDk+eSkP5+s/bWafUcn/H7JME yB7DcxBp1lrzOgtjhAOEQp5AOv3DgT84BE4LN/deCUEm8cNgMLbqfRAaqlCdJ6cRj0Ia wg/6WK0mUtmLe/w1dOJEmVQOggDDgAA68jr467GvZfBtnq649lTSaBu0pqmooYpjGoPG Xd8YKjfLLZxcLs0z0Z7+Ra9z9RdDNLgVS0tzJnCn1YAPbS/MC+9aU3yWd6LVi2szOb6j ReILaQu9IR3bmj5g04wogxsHbsJYIXggVf+RJ+bP2zgA4Nk2aZGCEY0PR/8wxoygXij4 DOnQ== X-Gm-Message-State: AJIora9pQ5G0c1sHD2TyrCSEBrt/Q81C/KA/bFmw9DmlddR2M/WEJf+I zKkJS0zQryzHmOXqQRvnhn2vEtCdGlZjAMG3JOT6Aaa5cKRaoJ+rDLuKUwjtqnvDBDk4bv0OrP8 LYJ8zYGR4L/xb6ePX0Uply/osHSwBgL7l/5texCLaUq6MZrLggV1uB+b/TWroB2cD27kw X-Received: from vigneshv3.mtv.corp.google.com ([2620:0:1000:2511:eeb1:1213:bdf1:9e9f]) (user=vigneshv job=sendgmr) by 2002:a0d:dc42:0:b0:31c:7b70:1f31 with SMTP id f63-20020a0ddc42000000b0031c7b701f31mr24214643ywe.513.1659032719628; Thu, 28 Jul 2022 11:25:19 -0700 (PDT) Date: Thu, 28 Jul 2022 11:25:16 -0700 In-Reply-To: Message-Id: <20220728182516.233366-1-vigneshv@google.com> Mime-Version: 1.0 References: X-Mailer: git-send-email 2.37.1.455.g008518b4e5-goog From: Vignesh Venkatasubramanian To: ffmpeg-devel@ffmpeg.org Subject: [FFmpeg-devel] [PATCH 1/2] avformat/mov: Rework the AVIF parser to handle multiple items X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Vignesh Venkatasubramanian Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: b5TFHuCMdfE7 Stores the item ids of all the items found in the file and processes the primary item at the end of the meta box. This patch does not change any behavior. It sets up the code for parsing alpha channel (and possibly images with 'grid') in follow up patches. Signed-off-by: Vignesh Venkatasubramanian --- libavformat/isom.h | 6 ++ libavformat/mov.c | 143 +++++++++++++++++++++++++++------------------ 2 files changed, 92 insertions(+), 57 deletions(-) diff --git a/libavformat/isom.h b/libavformat/isom.h index f05c2d9c28..9d8646d2ea 100644 --- a/libavformat/isom.h +++ b/libavformat/isom.h @@ -318,6 +318,12 @@ typedef struct MOVContext { uint32_t max_stts_delta; int is_still_picture_avif; int primary_item_id; + struct { + int item_id; + int extent_length; + int64_t extent_offset; + } *avif_info; + int avif_info_size; } MOVContext; int ff_mp4_read_descr_len(AVIOContext *pb); diff --git a/libavformat/mov.c b/libavformat/mov.c index a09a762d91..6ee6ed0950 100644 --- a/libavformat/mov.c +++ b/libavformat/mov.c @@ -4698,6 +4698,69 @@ static int mov_read_custom(MOVContext *c, AVIOContext *pb, MOVAtom atom) return ret; } +static int avif_add_stream(MOVContext *c, int item_id) +{ + MOVStreamContext *sc; + AVStream *st; + int item_index = -1; + for (int i = 0; i < c->avif_info_size; i++) + if (c->avif_info[i].item_id == item_id) { + item_index = i; + break; + } + if (item_index < 0) + return AVERROR_INVALIDDATA; + st = avformat_new_stream(c->fc, NULL); + if (!st) + return AVERROR(ENOMEM); + st->id = c->fc->nb_streams; + sc = av_mallocz(sizeof(MOVStreamContext)); + if (!sc) + return AVERROR(ENOMEM); + + st->priv_data = sc; + st->codecpar->codec_type = AVMEDIA_TYPE_VIDEO; + st->codecpar->codec_id = AV_CODEC_ID_AV1; + sc->ffindex = st->index; + c->trak_index = st->index; + st->avg_frame_rate.num = st->avg_frame_rate.den = 1; + st->time_base.num = st->time_base.den = 1; + st->nb_frames = 1; + sc->time_scale = 1; + sc = st->priv_data; + sc->pb = c->fc->pb; + sc->pb_is_copied = 1; + + // Populate the necessary fields used by mov_build_index. + sc->stsc_count = 1; + sc->stsc_data = av_malloc_array(1, sizeof(*sc->stsc_data)); + if (!sc->stsc_data) + return AVERROR(ENOMEM); + sc->stsc_data[0].first = 1; + sc->stsc_data[0].count = 1; + sc->stsc_data[0].id = 1; + sc->chunk_count = 1; + sc->chunk_offsets = av_malloc_array(1, sizeof(*sc->chunk_offsets)); + if (!sc->chunk_offsets) + return AVERROR(ENOMEM); + sc->sample_count = 1; + sc->sample_sizes = av_malloc_array(1, sizeof(*sc->sample_sizes)); + if (!sc->sample_sizes) + return AVERROR(ENOMEM); + sc->stts_count = 1; + sc->stts_data = av_malloc_array(1, sizeof(*sc->stts_data)); + if (!sc->stts_data) + return AVERROR(ENOMEM); + sc->stts_data[0].count = 1; + // Not used for still images. But needed by mov_build_index. + sc->stts_data[0].duration = 0; + sc->sample_sizes[0] = c->avif_info[item_index].extent_length; + sc->chunk_offsets[0] = c->avif_info[item_index].extent_offset; + + mov_build_index(c, st); + return 0; +} + static int mov_read_meta(MOVContext *c, AVIOContext *pb, MOVAtom atom) { while (atom.size > 8) { @@ -4707,9 +4770,23 @@ static int mov_read_meta(MOVContext *c, AVIOContext *pb, MOVAtom atom) tag = avio_rl32(pb); atom.size -= 4; if (tag == MKTAG('h','d','l','r')) { + int ret; avio_seek(pb, -8, SEEK_CUR); atom.size += 8; - return mov_read_default(c, pb, atom); + if ((ret = mov_read_default(c, pb, atom)) < 0) + return ret; + if (c->is_still_picture_avif) { + int ret; + // Add a stream for the YUV planes (primary item). + if ((ret = avif_add_stream(c, c->primary_item_id)) < 0) + return ret; + // For still AVIF images, the meta box contains all the + // necessary information that would generally be provided by the + // moov box. So simply mark that we have found the moov box so + // that parsing can continue. + c->found_moov = 1; + } + return ret; } } return 0; @@ -7478,8 +7555,6 @@ static int mov_read_iloc(MOVContext *c, AVIOContext *pb, MOVAtom atom) int item_count, extent_count; uint64_t base_offset, extent_offset, extent_length; uint8_t value; - AVStream *st; - MOVStreamContext *sc; if (!c->is_still_picture_avif) { // * For non-avif, we simply ignore the iloc box. @@ -7493,27 +7568,6 @@ static int mov_read_iloc(MOVContext *c, AVIOContext *pb, MOVAtom atom) return 0; } - st = avformat_new_stream(c->fc, NULL); - if (!st) - return AVERROR(ENOMEM); - st->id = c->fc->nb_streams; - sc = av_mallocz(sizeof(MOVStreamContext)); - if (!sc) - return AVERROR(ENOMEM); - - st->priv_data = sc; - st->codecpar->codec_type = AVMEDIA_TYPE_VIDEO; - st->codecpar->codec_id = AV_CODEC_ID_AV1; - sc->ffindex = st->index; - c->trak_index = st->index; - st->avg_frame_rate.num = st->avg_frame_rate.den = 1; - st->time_base.num = st->time_base.den = 1; - st->nb_frames = 1; - sc->time_scale = 1; - sc = st->priv_data; - sc->pb = c->fc->pb; - sc->pb_is_copied = 1; - version = avio_r8(pb); avio_rb24(pb); // flags. @@ -7529,34 +7583,17 @@ static int mov_read_iloc(MOVContext *c, AVIOContext *pb, MOVAtom atom) } item_count = (version < 2) ? avio_rb16(pb) : avio_rb32(pb); - // Populate the necessary fields used by mov_build_index. - sc->stsc_count = 1; - sc->stsc_data = av_malloc_array(1, sizeof(*sc->stsc_data)); - if (!sc->stsc_data) - return AVERROR(ENOMEM); - sc->stsc_data[0].first = 1; - sc->stsc_data[0].count = 1; - sc->stsc_data[0].id = 1; - sc->chunk_count = 1; - sc->chunk_offsets = av_malloc_array(1, sizeof(*sc->chunk_offsets)); - if (!sc->chunk_offsets) + c->avif_info = av_malloc_array(item_count, sizeof(*c->avif_info)); + if (!c->avif_info) return AVERROR(ENOMEM); - sc->sample_count = 1; - sc->sample_sizes = av_malloc_array(1, sizeof(*sc->sample_sizes)); - if (!sc->sample_sizes) - return AVERROR(ENOMEM); - sc->stts_count = 1; - sc->stts_data = av_malloc_array(1, sizeof(*sc->stts_data)); - if (!sc->stts_data) - return AVERROR(ENOMEM); - sc->stts_data[0].count = 1; - // Not used for still images. But needed by mov_build_index. - sc->stts_data[0].duration = 0; + c->avif_info_size = item_count; for (int i = 0; i < item_count; i++) { int item_id = (version < 2) ? avio_rb16(pb) : avio_rb32(pb); if (avio_feof(pb)) return AVERROR_INVALIDDATA; + c->avif_info[i].item_id = item_id; + if (version > 0) avio_rb16(pb); // construction_method. avio_rb16(pb); // data_reference_index. @@ -7572,20 +7609,11 @@ static int mov_read_iloc(MOVContext *c, AVIOContext *pb, MOVAtom atom) if (rb_size(pb, &extent_offset, offset_size) < 0 || rb_size(pb, &extent_length, length_size) < 0) return AVERROR_INVALIDDATA; - if (item_id == c->primary_item_id) { - sc->sample_sizes[0] = extent_length; - sc->chunk_offsets[0] = base_offset + extent_offset; - } + c->avif_info[i].extent_length = extent_length; + c->avif_info[i].extent_offset = base_offset + extent_offset; } } - mov_build_index(c, st); - - // For still AVIF images, the iloc box contains all the necessary - // information that would generally be provided by the moov box. So simply - // mark that we have found the moov box so that parsing can continue. - c->found_moov = 1; - return atom.size; } @@ -8189,6 +8217,7 @@ static int mov_read_close(AVFormatContext *s) av_freep(&mov->aes_decrypt); av_freep(&mov->chapter_tracks); + av_freep(&mov->avif_info); return 0; }