From patchwork Thu Mar 3 23:57:55 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vignesh Venkat X-Patchwork-Id: 34596 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6838:d078:0:0:0:0 with SMTP id x24csp1077383nkx; Thu, 3 Mar 2022 15:58:18 -0800 (PST) X-Google-Smtp-Source: ABdhPJyIQLp+KlQ+zESKU+BcLXf3Ep6h+mSO0MIefcIHyDKKBZKArLmVDHJk3NxBCuAb5TJ/66Rc X-Received: by 2002:a17:906:b893:b0:6da:ab5e:ea34 with SMTP id hb19-20020a170906b89300b006daab5eea34mr1706016ejb.657.1646351898651; Thu, 03 Mar 2022 15:58:18 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1646351898; cv=none; d=google.com; s=arc-20160816; b=BFi7btMUDCRmhxbxAHppXMEyWRr5A9WmMtPp40d5LTecYvuh/abJTwCUWLkI8BwOaQ fqDpN3LcCArKT7YFKJ9OscMtPMicACG0yitfeyXHDFO9T4h81GhVie3cNreGifVhSaNL pKA9LiBZRsA6hHg5AKRJQCTdJfPe79BmFYcPVu4qM04dEffQLWe59RjRf8SjtaWz98zJ OFNeXSikuw22fzW0e2z0w3jgs9r3VJNsSsh1qO0DseiFvPWwgl9pacMOp83eU0bP5CMy N+hXc+JPkgPVhO2q20jmm8G/pzn997dbiTayK4PRufJnYeFs45Be6pOlP3mWdo9yZZOv G9ow== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:to:from:mime-version:message-id:date :dkim-signature:delivered-to; bh=0yjWIyuDKV3l38Gbi4I6JTSx7GquhZUh9ru2vbMGTho=; b=nvl28RowOyS0rNdgafZtC5zBWTaBiFT51U2qoTybOTDdZClnmrW5tdT7NGPx2Ln3la BDKhNFmMm1Nb3A+ducDmzfpTtWrBpLr4ncAi1u+AJxG2WrmHV7l+GlqufEfXG+bwUNbl tcjfhWOYmVaavr9A44mU53tFxG1BcjTfYuo3SygQLm/DgCeJe2v47m7hoqF27rLhrj8O bL3NazfGIiVJVbxIAjIB/dv2ZHX5Us8nttwpCyjabojctcHLWI1gcGgJTN0tM5zZ2pbg 8D4NTFxboVPSuFRK6Ke4V0KFG1d6gVmo+BCinyFgoRenTspAU8iYqsM7p98d2g+ZnSVx GAxg== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@google.com header.s=20210112 header.b=ExXZBMPe; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id ec16-20020a170906b6d000b006b0d08d0dbasi2146069ejb.846.2022.03.03.15.58.17; Thu, 03 Mar 2022 15:58:18 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@google.com header.s=20210112 header.b=ExXZBMPe; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 03D6068B00E; Fri, 4 Mar 2022 01:58:14 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pf1-f202.google.com (mail-pf1-f202.google.com [209.85.210.202]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 0642E68AF62 for ; Fri, 4 Mar 2022 01:58:06 +0200 (EET) Received: by mail-pf1-f202.google.com with SMTP id t184-20020a6281c1000000b004e103c5f726so4009834pfd.8 for ; Thu, 03 Mar 2022 15:58:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:message-id:mime-version:subject:from:to:cc; bh=GQEPji0HORDz2WumNx/3SpcR1R8lmzTv0Tbq/8nB+vA=; b=ExXZBMPeLuZIl/cqU/H2spZi07CIBFWHQhhdC9MCBRV5Iy3UdoVBcmlyfRZh+JBoTU 60Zq63xZFru3BgbXr6xs1DFMUiz5GsQMR3HB/cqmDyXTsIEuYNo4sIZKUnxjzfrqUGkv pdibdQ+tdmEDRRK09NNN5gDBc3gFhiGo3U2wlGaonWQt5Q6Tnm4HBO4rbO6Tdqs/V8yD UlmUm6fAHxaAcQmFd1E1Py1ykSpEqBiU2U4u8F2u4Yol80FOn5XBDnIKB8BaEATe1FaO h9eawYpNaV3dYxZ43VRM59JJcsOuNK+XxU+K61VV9t1lNrgzigHAhabA71vj4Z12T0Wu z08g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:message-id:mime-version:subject:from:to:cc; bh=GQEPji0HORDz2WumNx/3SpcR1R8lmzTv0Tbq/8nB+vA=; b=DUvpxzykc4wKfwYhoB1Ua9vMUgo7lT/pmo3U5jBD+D7Y9GGKvMrWO9Tq1XQY0PYchq ZDt/ihQoPz8oGN4gMQLf1mtah3WQGsLzoFH5EYksrYlZAm0iS68Ue45kpuKYxMAGpGpY NXWT6DUYThrCCeHTx2xeeb1zwCNGjYnLuroFGMcXwPAQAUqRSTqXbARj4B7tjZUX4XDp XjxuTmPFQ0+5Lk8d53+RN1UpQ1qtETezm+tLNIxgLqc222nXjW++FqN0XGcKdj12BkK6 EW4f2pGvaKdiNh2bxhbPLk+emJSaHCyyqZLA8VtfK7Ne4aDOHQRL85VAw5QPvY4RAWY+ uyYQ== X-Gm-Message-State: AOAM531BLVghicsL4G0Y6VsGpeNHr6m72D5O8tzn+GOEp4EeOy0e7whN n52N/WwROi3eokN0nKYfMRTPWm63PWsQjn8ze/itdjobSZjEolvbrzkA6QUqNNexyLTwPRQ20d7 iYoLZ8BxLhUtRN18IzTf85TnBd9aJ9/aruEBzIo86IkjzkYp2q0ccxUWPMPddbM0c743S X-Received: from vigneshv3.mtv.corp.google.com ([2620:0:1000:2511:737d:b35f:e1e6:5da6]) (user=vigneshv job=sendgmr) by 2002:a17:90b:10b:b0:1bc:6f86:b209 with SMTP id p11-20020a17090b010b00b001bc6f86b209mr7889118pjz.33.1646351884859; Thu, 03 Mar 2022 15:58:04 -0800 (PST) Date: Thu, 3 Mar 2022 15:57:55 -0800 Message-Id: <20220303235755.3947932-1-vigneshv@google.com> Mime-Version: 1.0 X-Mailer: git-send-email 2.35.1.616.g0bdcbb4464-goog From: Vignesh Venkatasubramanian To: ffmpeg-devel@ffmpeg.org Subject: [FFmpeg-devel] [PATCH] avformat/mov: Add support for still image AVIF parsing X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Vignesh Venkatasubramanian Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: oZt9FIq67PAl Add support for parsing AVIF still images. This patches supports AVIF still images that have exactly 1 item (i.e.) no alpha channel. Essentially, we will have to parse the "iloc" box and populate the mov index. With this patch, we can decode still AVIF images like so: ffmpeg -i image.avif image.png Partially fixes trac ticket #7621 Signed-off-by: Vignesh Venkatasubramanian --- libavformat/isom.h | 1 + libavformat/mov.c | 142 +++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 143 insertions(+) diff --git a/libavformat/isom.h b/libavformat/isom.h index 625dea8421..cc0a8e1ca9 100644 --- a/libavformat/isom.h +++ b/libavformat/isom.h @@ -306,6 +306,7 @@ typedef struct MOVContext { int have_read_mfra_size; uint32_t mfra_size; uint32_t max_stts_delta; + int is_still_picture_avif; } MOVContext; int ff_mp4_read_descr_len(AVIOContext *pb); diff --git a/libavformat/mov.c b/libavformat/mov.c index 68b6d7f075..d260024e47 100644 --- a/libavformat/mov.c +++ b/libavformat/mov.c @@ -1126,6 +1126,7 @@ static int mov_read_ftyp(MOVContext *c, AVIOContext *pb, MOVAtom atom) c->isom = 1; av_log(c->fc, AV_LOG_DEBUG, "ISO: File Type Major Brand: %.4s\n",(char *)&type); av_dict_set(&c->fc->metadata, "major_brand", type, 0); + c->is_still_picture_avif = !strncmp(type, "avif", 4); minor_ver = avio_rb32(pb); /* minor version */ av_dict_set_int(&c->fc->metadata, "minor_version", minor_ver, 0); @@ -7173,6 +7174,146 @@ cleanup: return ret; } +static int rb_size(AVIOContext *pb, uint64_t* value, int size) +{ + if (size == 0) { + *value = 0; + } else if (size == 1) { + *value = avio_r8(pb); + } else if (size == 2) { + *value = avio_rb16(pb); + } else if (size == 4) { + *value = avio_rb32(pb); + } else if (size == 8) { + *value = avio_rb64(pb); + } else { + return -1; + } + return size; +} + +static int mov_read_iloc(MOVContext *c, AVIOContext *pb, MOVAtom atom) +{ + int version, offset_size, length_size, base_offset_size, index_size; + int item_count, extent_count; + uint64_t base_offset, extent_offset, extent_length; + int i, j; + uint8_t value; + AVStream *st; + MOVStreamContext *sc; + + if (!c->is_still_picture_avif) { + // * For non-avif, we simply ignore the iloc box. + // * For animated avif, we don't care about the iloc box as all the + // necessary information can be found in the moov box. + return 0; + } + + if (c->fc->nb_streams != 0) { + av_log(c->fc, AV_LOG_INFO, "Duplicate iloc box found\n"); + return 0; + } + + st = avformat_new_stream(c->fc, NULL); + if (!st) return AVERROR(ENOMEM); + st->id = c->fc->nb_streams; + sc = av_mallocz(sizeof(MOVStreamContext)); + if (!sc) return AVERROR(ENOMEM); + + st->priv_data = sc; + st->codecpar->codec_type = AVMEDIA_TYPE_VIDEO; + st->codecpar->codec_id = AV_CODEC_ID_AV1; + sc->ffindex = st->index; + c->trak_index = st->index; + st->avg_frame_rate.num = st->avg_frame_rate.den = 1; + st->time_base.num = st->time_base.den = 1; + st->nb_frames = 1; + sc->time_scale = 1; + sc = st->priv_data; + sc->pb = c->fc->pb; + sc->pb_is_copied = 1; + + version = avio_r8(pb); + avio_rb24(pb); // flags. + + value = avio_r8(pb); + offset_size = (value >> 4) & 0xF; + length_size = value & 0xF; + value = avio_r8(pb); + base_offset_size = (value >> 4) & 0xF; + index_size = (version == 0) ? 0 : (value & 0xF); + if (index_size != 0) { + return AVERROR_PATCHWELCOME; + } + item_count = (version < 2) ? avio_rb16(pb) : avio_rb32(pb); + if (item_count > 1) { + // For still AVIF images, we only support one item. Second item will + // generally be found for AVIF images with alpha channel. We don't + // support them as of now. + return AVERROR_PATCHWELCOME; + } + + // Populate the necessary fields used by mov_build_index. + sc->stsc_count = item_count; + sc->stsc_data = av_malloc_array(item_count, sizeof(*sc->stsc_data)); + if (!sc->stsc_data) { + return AVERROR(ENOMEM); + } + sc->stsc_data[0].first = 1; + sc->stsc_data[0].count = 1; + sc->stsc_data[0].id = 1; + sc->chunk_count = item_count; + sc->chunk_offsets = av_malloc_array(item_count, sizeof(*sc->chunk_offsets)); + if (!sc->chunk_offsets) { + return AVERROR(ENOMEM); + } + sc->sample_count = item_count; + sc->sample_sizes = av_malloc_array(item_count, sizeof(*sc->sample_sizes)); + if (!sc->sample_sizes) { + return AVERROR(ENOMEM); + } + sc->stts_count = item_count; + sc->stts_data = av_malloc_array(item_count, sizeof(*sc->stts_data)); + if (!sc->stts_data) { + return AVERROR(ENOMEM); + } + sc->stts_data[0].count = 1; + sc->stts_data[0].duration = 0; // Not used for still images. But needed by mov_build_index. + + for (i = 0; i < item_count; ++i) { + (version < 2) ? avio_rb16(pb) : avio_rb32(pb); // item_id; + if (version > 0) { + avio_rb16(pb); // construction_method. + } + avio_rb16(pb); // data_reference_index. + if (rb_size(pb, &base_offset, base_offset_size) < 0) { + return AVERROR_INVALIDDATA; + } + extent_count = avio_rb16(pb); + if (extent_count > 1) { + // For still AVIF images, we only support one extent item. + return AVERROR_PATCHWELCOME; + } + for (j = 0; j < extent_count; ++j) { + if (rb_size(pb, &extent_offset, offset_size) < 0 || + rb_size(pb, &extent_length, length_size) < 0) { + return AVERROR_INVALIDDATA; + } + sc->sample_sizes[0] = extent_length; + sc->chunk_offsets[0] = base_offset + extent_offset; + } + } + + mov_build_index(c, st); + + // For still AVIF images, the iloc box contains all the necessary + // information that would generally be provided by the moov box. So simply + // mark that we have found the moov box so that parsing can continue. + c->found_moov = 1; + + return atom.size; +} + static const MOVParseTableEntry mov_default_parse_table[] = { { MKTAG('A','C','L','R'), mov_read_aclr }, { MKTAG('A','P','R','G'), mov_read_avid }, @@ -7272,6 +7413,7 @@ static const MOVParseTableEntry mov_default_parse_table[] = { { MKTAG('d','v','v','C'), mov_read_dvcc_dvvc }, { MKTAG('d','v','w','C'), mov_read_dvcc_dvvc }, { MKTAG('k','i','n','d'), mov_read_kind }, +{ MKTAG('i','l','o','c'), mov_read_iloc }, { 0, NULL } };