From patchwork Thu Jun 30 21:04:33 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vignesh Venkat X-Patchwork-Id: 36556 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:8b27:b0:88:1bbf:7fd2 with SMTP id l39csp579923pzh; Thu, 30 Jun 2022 14:04:57 -0700 (PDT) X-Google-Smtp-Source: AGRyM1tXT/nrbKeUSSS84CmefWZKrtsZ2iUX11vCslLRGLn8h8FLXMReBF5LTPKkoqCH0HA3bxz7 X-Received: by 2002:a17:906:8461:b0:72a:8262:b066 with SMTP id hx1-20020a170906846100b0072a8262b066mr1506882ejc.706.1656623097106; Thu, 30 Jun 2022 14:04:57 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1656623097; cv=none; d=google.com; s=arc-20160816; b=dxB7h4tv4y3nJYZvVOnywVdK8SkdMexbPEboYv5rkW8XSu8pjiZ4hC4CjJCXjYqaEQ tbnwWm2JPSK8Ts750m7zD415FIL+mUX+KmNlBauNczV98giB8dZ7lWMNrXp5WPYUFB2P Jf4v0IPr2U+5ww+UfMxOFxjho44e2qdcEhqGoWBDpvvoXgqytN+AP8DJWFwuhLSdmIqT fFJYEmAuVEJ5jwqjT/0qzaDb5MRPQr5FOp3nzgWUl1tgYMxl41taMaX5xJfgu4ga2z66 3e4ax12qUbS1N6azJwSd223wR+DjgXCIjwVeZG0oErZAunA2kHzRzDLBrG7c0zwRQa7a sVlg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:to:from:mime-version:message-id:date :dkim-signature:delivered-to; bh=YuZxILgtfgzJ4CbviCrnffn2NRJHWU9WCdziIQP6tGk=; b=hTcgAkIZ+r1PxX35k1o93j3qiqX96ie41BFkwsJk86hiGgYhPh4WX1lvrO0CFffeaR GTyWa2bPcXQiQueY5rqB5IIh7sfgZS93BVopCzuplhvbyYW6aNpMSgcTD5Q94Q/Lk9ht BT+d74qt+4xZ4FYuCfCeAX5WxnVpnZGNyM1iHj3zW1LZ13jBkY4g/VJY1w4pc3Y+qkJc WmiXgP5Q4BgkcasxHEc9vXfr0WuNgfUhC7z0WL2TxPsHhxZZ46pmwC5fDDF3o7UlQPWe kGIu7j/bLGbz41coRlq9yyFkOChLWWW7ivm2DvmWs5xSYFOMjUjMUaIjzqWJEYum1KYv TmFQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@google.com header.s=20210112 header.b=scgChz8c; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id md27-20020a170906ae9b00b0071bbadaaf42si21245227ejb.425.2022.06.30.14.04.54; Thu, 30 Jun 2022 14:04:57 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@google.com header.s=20210112 header.b=scgChz8c; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 60B9568B4E6; Fri, 1 Jul 2022 00:04:51 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-yb1-f201.google.com (mail-yb1-f201.google.com [209.85.219.201]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 0A50B68B4E6 for ; Fri, 1 Jul 2022 00:04:43 +0300 (EEST) Received: by mail-yb1-f201.google.com with SMTP id b11-20020a5b008b000000b00624ea481d55so261459ybp.19 for ; Thu, 30 Jun 2022 14:04:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:message-id:mime-version:subject:from:to:cc; bh=1Nv8RBNmjf7OuC9tB4jAf74yhMW5q1FdfbRr8FFgod4=; b=scgChz8cnV7eaiP0QKaRx3rzoEGSIhAYXj08boe2ynv37nwIet7XG9ZQZZmp30dCya IqCR4vwyB2z7bAX79hzF5ILPdx50wN0NQBTecSJihWJhRugpQHGHXffU4zclWqp/AQRR UUu3e9ueoqweEo1gaEQxENpmYH5Xs9Pxkj995c2Cw128NnNZaSis23O0ILOOoqcL1OCX 5bWNvOYBrAg2uP2eFYVhM3XLNRs0NxzeSBUB5wdS61mhpdlIvcueSqAflDObPrE0lVFn QxXdM9fx6MUC6/OjIzEWcuVXYVw08nQK+SKJLALvmDI7F7IpICUX9GSPTovfooOLOxuk 6Dog== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:message-id:mime-version:subject:from:to:cc; bh=1Nv8RBNmjf7OuC9tB4jAf74yhMW5q1FdfbRr8FFgod4=; b=qxzRIgMQ/GAXoPFlvfdwp1LdmAp1I+A/7Qymhpn2RfFrqlBVSlLSTclqX0n9aJH/aG rHB1jpfPHwXQpt6C0ZKpXd7aVnXPZczLOdHF30rzA9D48HFPPJhcfN3NSDAH5TBPRYFI Xh0QxTptyWrkxToriGPooJN5svg0D9RU9+CzkyvZ4gsL2q/wR6LgzBq6sed7ER+ldkZN +C6i4LOqRevVADiXNf58ott9Ywn12dX1vbdHCcOCzgnURRWgQqpnbyyOObwp5xA9KPTQ Y3pspw74nZDpnPRLs8w66gNtHVhq3x729UzisJ6UpvhOGIxo6k6v592+nfJVoUql3J/6 0eMA== X-Gm-Message-State: AJIora/y6mHTKDaS8i3T+gxoVBA8esjSV965AB2SGC5sWfSnVzHmno4v NBa5ct5ymaXpWJq/IFztFW3sXprrDGb19SFQwvkjubO5n7UD4+5gaLrYbNgza3EIfI7YPzCtnDx Q3DKgYF4gSRr+H893SzyNkU2B4gdJyoP7q7o3bMUrzUO1usrP5IhA9Npih51FbZFVwTX+ X-Received: from vigneshv3.mtv.corp.google.com ([2620:0:1000:2511:f6b5:b6ce:7578:e219]) (user=vigneshv job=sendgmr) by 2002:a81:85c5:0:b0:31c:1f50:1bbb with SMTP id v188-20020a8185c5000000b0031c1f501bbbmr11737222ywf.3.1656623082264; Thu, 30 Jun 2022 14:04:42 -0700 (PDT) Date: Thu, 30 Jun 2022 14:04:33 -0700 Message-Id: <20220630210434.1551769-1-vigneshv@google.com> Mime-Version: 1.0 X-Mailer: git-send-email 2.37.0.rc0.161.g10f37bed90-goog From: Vignesh Venkatasubramanian To: ffmpeg-devel@ffmpeg.org Subject: [FFmpeg-devel] [PATCH 1/2] avformat/mov: Rework the AVIF parser to handle multiple items X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Vignesh Venkatasubramanian Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: C0GQTHMr7nHv Stores the item ids of all the items found in the file and processes the primary item at the end of the meta box. This patch does not change any behavior. It sets up the code for parsing alpha channel (and possibly images with 'grid') in follow up patches. Signed-off-by: Vignesh Venkatasubramanian --- libavformat/isom.h | 4 ++ libavformat/mov.c | 148 ++++++++++++++++++++++++++++----------------- 2 files changed, 97 insertions(+), 55 deletions(-) diff --git a/libavformat/isom.h b/libavformat/isom.h index f05c2d9c28..d8b262e915 100644 --- a/libavformat/isom.h +++ b/libavformat/isom.h @@ -318,6 +318,10 @@ typedef struct MOVContext { uint32_t max_stts_delta; int is_still_picture_avif; int primary_item_id; + int *avif_item_ids; + int avif_item_ids_size; + int *avif_extent_lengths; + int64_t *avif_extent_offsets; } MOVContext; int ff_mp4_read_descr_len(AVIOContext *pb); diff --git a/libavformat/mov.c b/libavformat/mov.c index 88669faa70..9df5055d4e 100644 --- a/libavformat/mov.c +++ b/libavformat/mov.c @@ -4683,6 +4683,69 @@ static int mov_read_custom(MOVContext *c, AVIOContext *pb, MOVAtom atom) return ret; } +static int avif_add_stream(MOVContext *c, int item_id) +{ + MOVStreamContext *sc; + AVStream *st; + int item_index = -1; + for (int i = 0; i < c->avif_item_ids_size; i++) + if (c->avif_item_ids[i] == item_id) { + item_index = i; + break; + } + if (item_index < 0) + return AVERROR_INVALIDDATA; + st = avformat_new_stream(c->fc, NULL); + if (!st) + return AVERROR(ENOMEM); + st->id = c->fc->nb_streams; + sc = av_mallocz(sizeof(MOVStreamContext)); + if (!sc) + return AVERROR(ENOMEM); + + st->priv_data = sc; + st->codecpar->codec_type = AVMEDIA_TYPE_VIDEO; + st->codecpar->codec_id = AV_CODEC_ID_AV1; + sc->ffindex = st->index; + c->trak_index = st->index; + st->avg_frame_rate.num = st->avg_frame_rate.den = 1; + st->time_base.num = st->time_base.den = 1; + st->nb_frames = 1; + sc->time_scale = 1; + sc = st->priv_data; + sc->pb = c->fc->pb; + sc->pb_is_copied = 1; + + // Populate the necessary fields used by mov_build_index. + sc->stsc_count = 1; + sc->stsc_data = av_malloc_array(1, sizeof(*sc->stsc_data)); + if (!sc->stsc_data) + return AVERROR(ENOMEM); + sc->stsc_data[0].first = 1; + sc->stsc_data[0].count = 1; + sc->stsc_data[0].id = 1; + sc->chunk_count = 1; + sc->chunk_offsets = av_malloc_array(1, sizeof(*sc->chunk_offsets)); + if (!sc->chunk_offsets) + return AVERROR(ENOMEM); + sc->sample_count = 1; + sc->sample_sizes = av_malloc_array(1, sizeof(*sc->sample_sizes)); + if (!sc->sample_sizes) + return AVERROR(ENOMEM); + sc->stts_count = 1; + sc->stts_data = av_malloc_array(1, sizeof(*sc->stts_data)); + if (!sc->stts_data) + return AVERROR(ENOMEM); + sc->stts_data[0].count = 1; + // Not used for still images. But needed by mov_build_index. + sc->stts_data[0].duration = 0; + sc->sample_sizes[0] = c->avif_extent_lengths[item_index]; + sc->chunk_offsets[0] = c->avif_extent_offsets[item_index]; + + mov_build_index(c, st); + return 0; +} + static int mov_read_meta(MOVContext *c, AVIOContext *pb, MOVAtom atom) { while (atom.size > 8) { @@ -4692,9 +4755,25 @@ static int mov_read_meta(MOVContext *c, AVIOContext *pb, MOVAtom atom) tag = avio_rl32(pb); atom.size -= 4; if (tag == MKTAG('h','d','l','r')) { + int ret; avio_seek(pb, -8, SEEK_CUR); atom.size += 8; - return mov_read_default(c, pb, atom); + ret = mov_read_default(c, pb, atom); + if (ret < 0) + return ret; + if (c->is_still_picture_avif) { + int ret; + // Add a stream for the YUV planes (primary item). + ret = avif_add_stream(c, c->primary_item_id); + if (ret) + return ret; + // For still AVIF images, the meta box contains all the + // necessary information that would generally be provided by the + // moov box. So simply mark that we have found the moov box so + // that parsing can continue. + c->found_moov = 1; + } + return ret; } } return 0; @@ -7483,8 +7562,6 @@ static int mov_read_iloc(MOVContext *c, AVIOContext *pb, MOVAtom atom) int item_count, extent_count; uint64_t base_offset, extent_offset, extent_length; uint8_t value; - AVStream *st; - MOVStreamContext *sc; if (!c->is_still_picture_avif) { // * For non-avif, we simply ignore the iloc box. @@ -7498,27 +7575,6 @@ static int mov_read_iloc(MOVContext *c, AVIOContext *pb, MOVAtom atom) return 0; } - st = avformat_new_stream(c->fc, NULL); - if (!st) - return AVERROR(ENOMEM); - st->id = c->fc->nb_streams; - sc = av_mallocz(sizeof(MOVStreamContext)); - if (!sc) - return AVERROR(ENOMEM); - - st->priv_data = sc; - st->codecpar->codec_type = AVMEDIA_TYPE_VIDEO; - st->codecpar->codec_id = AV_CODEC_ID_AV1; - sc->ffindex = st->index; - c->trak_index = st->index; - st->avg_frame_rate.num = st->avg_frame_rate.den = 1; - st->time_base.num = st->time_base.den = 1; - st->nb_frames = 1; - sc->time_scale = 1; - sc = st->priv_data; - sc->pb = c->fc->pb; - sc->pb_is_copied = 1; - version = avio_r8(pb); avio_rb24(pb); // flags. @@ -7534,32 +7590,20 @@ static int mov_read_iloc(MOVContext *c, AVIOContext *pb, MOVAtom atom) } item_count = (version < 2) ? avio_rb16(pb) : avio_rb32(pb); - // Populate the necessary fields used by mov_build_index. - sc->stsc_count = 1; - sc->stsc_data = av_malloc_array(1, sizeof(*sc->stsc_data)); - if (!sc->stsc_data) + c->avif_item_ids = av_malloc_array(item_count, sizeof(*c->avif_item_ids)); + if (!c->avif_item_ids) return AVERROR(ENOMEM); - sc->stsc_data[0].first = 1; - sc->stsc_data[0].count = 1; - sc->stsc_data[0].id = 1; - sc->chunk_count = 1; - sc->chunk_offsets = av_malloc_array(1, sizeof(*sc->chunk_offsets)); - if (!sc->chunk_offsets) + c->avif_item_ids_size = item_count; + c->avif_extent_lengths = av_malloc_array(item_count, sizeof(*c->avif_extent_lengths)); + if (!c->avif_extent_lengths) return AVERROR(ENOMEM); - sc->sample_count = 1; - sc->sample_sizes = av_malloc_array(1, sizeof(*sc->sample_sizes)); - if (!sc->sample_sizes) + c->avif_extent_offsets = av_malloc_array(item_count, sizeof(*c->avif_extent_offsets)); + if (!c->avif_extent_offsets) return AVERROR(ENOMEM); - sc->stts_count = 1; - sc->stts_data = av_malloc_array(1, sizeof(*sc->stts_data)); - if (!sc->stts_data) - return AVERROR(ENOMEM); - sc->stts_data[0].count = 1; - // Not used for still images. But needed by mov_build_index. - sc->stts_data[0].duration = 0; for (int i = 0; i < item_count; i++) { int item_id = (version < 2) ? avio_rb16(pb) : avio_rb32(pb); + c->avif_item_ids[i] = item_id; if (version > 0) avio_rb16(pb); // construction_method. avio_rb16(pb); // data_reference_index. @@ -7575,20 +7619,11 @@ static int mov_read_iloc(MOVContext *c, AVIOContext *pb, MOVAtom atom) if (rb_size(pb, &extent_offset, offset_size) < 0 || rb_size(pb, &extent_length, length_size) < 0) return AVERROR_INVALIDDATA; - if (item_id == c->primary_item_id) { - sc->sample_sizes[0] = extent_length; - sc->chunk_offsets[0] = base_offset + extent_offset; - } + c->avif_extent_lengths[i] = extent_length; + c->avif_extent_offsets[i] = base_offset + extent_offset; } } - mov_build_index(c, st); - - // For still AVIF images, the iloc box contains all the necessary - // information that would generally be provided by the moov box. So simply - // mark that we have found the moov box so that parsing can continue. - c->found_moov = 1; - return atom.size; } @@ -8192,6 +8227,9 @@ static int mov_read_close(AVFormatContext *s) av_freep(&mov->aes_decrypt); av_freep(&mov->chapter_tracks); + av_freep(&mov->avif_item_ids); + av_freep(&mov->avif_extent_lengths); + av_freep(&mov->avif_extent_offsets); return 0; }