From patchwork Tue Jan 9 19:55:45 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Almer X-Patchwork-Id: 45547 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:bf2f:b0:199:de12:6fa6 with SMTP id gc47csp622014pzb; Tue, 9 Jan 2024 11:56:14 -0800 (PST) X-Google-Smtp-Source: AGHT+IF4eH8JPgArVrLMoYpmH38HCRSzdbHEYd5YW77PzX6Y0sYnKaFlBvF0CdmYOHYNWh2i1D6F X-Received: by 2002:a05:6512:2212:b0:50e:9e95:5290 with SMTP id h18-20020a056512221200b0050e9e955290mr6276911lfu.1.1704830173773; Tue, 09 Jan 2024 11:56:13 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1704830173; cv=none; d=google.com; s=arc-20160816; b=AKQk3BDxcoQhbbQkI8yMduAiWEf2F+URcjFmWcE3OrD7kFlSjy2aANNsF0WP/zwFPB ohEEVIdRHAj8kedVuynYUSlADTjT74pPtnw3yoBIgCmRRN+quGpy46Gc6WppSiAX/KXz JVVJ2Na7tdcdPOhuFJCgipc1+RbCCjanvEjrUv0NPUbkDZP/tsxmhBwka44NyJJKXLb/ h+uZViI8e3B43LSziATvxz9Ic0OHow3SmitIIdUjeDa7epFLB4+WiyqIr6RC9YMIZa3e SslXEcP/ljf2MFwAFEEf5GbTSjhqqyJjL+PHIMbxWgy1eWAVDuiX5PyzHwXb6fxx7CMW 1Rrg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:dkim-signature:delivered-to; bh=uq4LtoHdM9kKX5DHmceSuppN/uM4j5LCUxiF3ANebNQ=; fh=YOA8vD9MJZuwZ71F/05pj6KdCjf6jQRmzLS+CATXUQk=; b=z83q/rEF0PJf3hA/YQZr3lgm66Ihz8k9SJa0lweSbIgpuHL0dL13Uw6jNdTmQi+mYM mKT8+UYFsTPKg5FFgd5PaKNdO8QrxDhVi9I69ugGMRet3cneOg2OSyRioHx7itxs8pVe Z21uAzD/wze3oU7ST4r/2oPBKddqdUMHM0oDjGpVLP913vvkY8lJbAyZ4Zqz0qRXIQqD d9HdxBPP2mCgb2HcFeViNoxtGo+BtIiei79qpKzxAo1A5SMtarBi5KcKDABDWuckr25W RCJLb4axuqTYtPNxgVlYNJVS3lJaOF3839Jf2aNkL1L4taBgchvjRnFyRbSeQlmHx6Ip pSMQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=IAAhb8i1; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id u18-20020a170906b11200b00a2a399a9bd1si1017220ejy.714.2024.01.09.11.56.13; Tue, 09 Jan 2024 11:56:13 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=IAAhb8i1; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id E21FA68BEC2; Tue, 9 Jan 2024 21:56:01 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pl1-f178.google.com (mail-pl1-f178.google.com [209.85.214.178]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 04C5C68AC21 for ; Tue, 9 Jan 2024 21:55:54 +0200 (EET) Received: by mail-pl1-f178.google.com with SMTP id d9443c01a7336-1d3ed1ca402so29714665ad.2 for ; Tue, 09 Jan 2024 11:55:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1704830151; x=1705434951; darn=ffmpeg.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=ier+V58lkRSqilrU9XTepp62YLs4fCxfchlcliREgPM=; b=IAAhb8i1aSmykEMelJ3RbRFYYMu/2V5uU2ntbBybdshC1Xj4qieYvwGZUHgKIwMepn If1K5MKWCEQ6BMpNmEM5mHPmoeWV/4hZDhC/HtgThtAze7w7TtlARj1rYodJ2jHlFHg/ 7C47EJn6XjZXfSqILZ2ojrs1+yvUuOOf4miJjAwmm0pCRGOjX65CglkP8dDDt5gIF1FU 7piWkFiPaTww/1yyK+olsZ502ppttUGIW45X3n4TJvqKCPILv+CK9GwRiD6m7kIq+7Pj pF/MDyz9fx6GAXGrY9jnpO9P0Qi/uHMprN/8E46ar7Y85I+K+Blxbt5ngXSKfD2yRrA3 6lEw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1704830151; x=1705434951; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ier+V58lkRSqilrU9XTepp62YLs4fCxfchlcliREgPM=; b=RlmQEPLLGhAF26hjh9yFPZInDG8xSmerZTcno+MJov0YQKeisfgYb/Yg9z4MF8BjdS O2wNHetuI7AMY7u+8fUJsAJuuCwbxA3lrxDZJi4zjZniPTiyOG6hIW3B7GCr3hrpogUc 8k1IU8kr2R/k8rS99gtpG8rNBdRin0vptTigoxjTvY8WTQ/Afu3RCrizggx0Ip8jKvXD jyghvNPvrnt/Ws+okV6L9eLvTZ49TG26qsvZ2+iBw8zgkB1JFW25VRVXoc9wIlu44JWi wrZ/xWqITAm+JTY+93Xoii0NwLYBKXDH45kJJQj4pPG23Ij1w3QqvKvSdtov3daDPahL /9Xg== X-Gm-Message-State: AOJu0YxTVRPtagE6FRr0wlfREcm8JmdWxKppd/yXyyxNRDzn5Z6Ff+h1 J0793HfKEt57hVDw+dvMmF5LqdXcr/c= X-Received: by 2002:a17:902:e850:b0:1d4:79b6:101a with SMTP id t16-20020a170902e85000b001d479b6101amr7218779plg.41.1704830151380; Tue, 09 Jan 2024 11:55:51 -0800 (PST) Received: from localhost.localdomain (host197.190-225-105.telecom.net.ar. [190.225.105.197]) by smtp.gmail.com with ESMTPSA id o7-20020a1709026b0700b001d0c151d325sm2160551plk.209.2024.01.09.11.55.50 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Jan 2024 11:55:50 -0800 (PST) From: James Almer To: ffmpeg-devel@ffmpeg.org Date: Tue, 9 Jan 2024 16:55:45 -0300 Message-ID: <20240109195545.59231-2-jamrial@gmail.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240109195545.59231-1-jamrial@gmail.com> References: <20240109195545.59231-1-jamrial@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 2/2] avformat/mov: improve HEIF parsing X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: ZAnUI/M0c6iU Parse iinf boxes and its child infe boxes to get the actual codec used (AV1 for avif, HEVC for heic), and properly export extradata in a generic way. The avif tests reference files are updated as the extradata is now exported. Signed-off-by: James Almer --- libavformat/isom.h | 3 +- libavformat/mov.c | 157 ++++++++++-------- .../fate/mov-avif-demux-still-image-1-item | 2 +- .../mov-avif-demux-still-image-multiple-items | 2 +- 4 files changed, 95 insertions(+), 69 deletions(-) diff --git a/libavformat/isom.h b/libavformat/isom.h index 90c4fb5530..107e2ce9bb 100644 --- a/libavformat/isom.h +++ b/libavformat/isom.h @@ -268,6 +268,7 @@ typedef struct MOVContext { int time_scale; int64_t duration; ///< duration of the longest track int found_moov; ///< 'moov' atom has been found + int found_iloc; ///< 'iloc' atom has been found int found_mdat; ///< 'mdat' atom has been found int found_hdlr_mdta; ///< 'hdlr' atom with type 'mdta' has been found int trak_index; ///< Index of the current 'trak' @@ -327,8 +328,6 @@ typedef struct MOVContext { int64_t extent_offset; } *heif_info; int heif_info_size; - int64_t hvcC_offset; - int hvcC_size; int interleaved_read; } MOVContext; diff --git a/libavformat/mov.c b/libavformat/mov.c index 12e82c66a9..47d0d41214 100644 --- a/libavformat/mov.c +++ b/libavformat/mov.c @@ -1233,8 +1233,6 @@ static int mov_read_ftyp(MOVContext *c, AVIOContext *pb, MOVAtom atom) c->isom = 1; av_log(c->fc, AV_LOG_DEBUG, "ISO: File Type Major Brand: %.4s\n",(char *)&type); av_dict_set(&c->fc->metadata, "major_brand", type, 0); - c->is_still_picture_avif = !strncmp(type, "avif", 4) || - !strncmp(type, "mif1", 4); minor_ver = avio_rb32(pb); /* minor version */ av_dict_set_int(&c->fc->metadata, "minor_version", minor_ver, 0); @@ -4641,10 +4639,6 @@ static int mov_read_trak(MOVContext *c, AVIOContext *pb, MOVAtom atom) MOVStreamContext *sc; int ret; - if (c->is_still_picture_avif) { - return AVERROR_INVALIDDATA; - } - st = avformat_new_stream(c->fc, NULL); if (!st) return AVERROR(ENOMEM); st->id = -1; @@ -4916,7 +4910,7 @@ static int mov_read_custom(MOVContext *c, AVIOContext *pb, MOVAtom atom) return ret; } -static int heif_add_stream(MOVContext *c, int item_id) +static int heif_add_stream(MOVContext *c, int item_id, uint32_t item_type) { MOVStreamContext *sc; AVStream *st; @@ -4940,20 +4934,7 @@ static int heif_add_stream(MOVContext *c, int item_id) st->priv_data = sc; st->codecpar->codec_type = AVMEDIA_TYPE_VIDEO; - st->codecpar->codec_id = AV_CODEC_ID_AV1; - if (c->hvcC_offset >= 0) { - int ret; - int64_t pos = avio_tell(c->fc->pb); - st->codecpar->codec_id = AV_CODEC_ID_HEVC; - if (avio_seek(c->fc->pb, c->hvcC_offset, SEEK_SET) != c->hvcC_offset) { - av_log(c->fc, AV_LOG_ERROR, "Failed to seek to hvcC data.\n"); - return AVERROR_UNKNOWN; - } - ret = ff_get_extradata(c->fc, st->codecpar, c->fc->pb, c->hvcC_size); - if (ret < 0) - return ret; - avio_seek(c->fc->pb, pos, SEEK_SET); - } + st->codecpar->codec_id = mov_codec_id(st, item_type); sc->ffindex = st->index; c->trak_index = st->index; st->avg_frame_rate.num = st->avg_frame_rate.den = 1; @@ -4996,8 +4977,6 @@ static int heif_add_stream(MOVContext *c, int item_id) static int mov_read_meta(MOVContext *c, AVIOContext *pb, MOVAtom atom) { - c->hvcC_offset = -1; - c->hvcC_size = 0; while (atom.size > 8) { uint32_t tag; if (avio_feof(pb)) @@ -5005,23 +4984,9 @@ static int mov_read_meta(MOVContext *c, AVIOContext *pb, MOVAtom atom) tag = avio_rl32(pb); atom.size -= 4; if (tag == MKTAG('h','d','l','r')) { - int ret; avio_seek(pb, -8, SEEK_CUR); atom.size += 8; - if ((ret = mov_read_default(c, pb, atom)) < 0) - return ret; - if (c->is_still_picture_avif) { - int ret; - // Add a stream for the YUV planes (primary item). - if ((ret = heif_add_stream(c, c->primary_item_id)) < 0) - return ret; - // For still AVIF images, the meta box contains all the - // necessary information that would generally be provided by the - // moov box. So simply mark that we have found the moov box so - // that parsing can continue. - c->found_moov = 1; - } - return ret; + return mov_read_default(c, pb, atom); } } return 0; @@ -7813,18 +7778,10 @@ static int mov_read_iloc(MOVContext *c, AVIOContext *pb, MOVAtom atom) uint64_t base_offset, extent_offset, extent_length; uint8_t value; - if (!c->is_still_picture_avif) { - // * For non-avif, we simply ignore the iloc box. - // * For animated avif, we don't care about the iloc box as all the - // necessary information can be found in the moov box. - return 0; - } - - if (c->heif_info) { + if (c->found_iloc) { av_log(c->fc, AV_LOG_INFO, "Duplicate iloc box found\n"); return 0; } - av_assert0(!c->fc->nb_streams); version = avio_r8(pb); avio_rb24(pb); // flags. @@ -7872,29 +7829,98 @@ static int mov_read_iloc(MOVContext *c, AVIOContext *pb, MOVAtom atom) } } + c->found_iloc = 1; return atom.size; } -static int mov_read_iprp(MOVContext *c, AVIOContext *pb, MOVAtom atom) +static int mov_read_infe(MOVContext *c, AVIOContext *pb, MOVAtom atom) { - int size = avio_rb32(pb); - if (avio_rl32(pb) != MKTAG('i','p','c','o')) - return AVERROR_INVALIDDATA; + char item_name[128]; + int64_t size = atom.size; + uint32_t item_type; + int item_id; + int version, ret; + + version = avio_r8(pb); + avio_rb24(pb); // flags. + size -= 4; + + if (version != 2) { + av_log(c->fc, AV_LOG_ERROR, "infe: version != 2 not supported.\n"); + return AVERROR_PATCHWELCOME; + } + + item_id = avio_rb16(pb); + avio_rb16(pb); // item_protection_index + item_type = avio_rl32(pb); size -= 8; - while (size > 0) { - int sub_size, sub_type; - sub_size = avio_rb32(pb); - sub_type = avio_rl32(pb); - sub_size -= 8; - size -= sub_size + 8; - if (sub_type == MKTAG('h','v','c','C')) { - c->hvcC_offset = avio_tell(pb); - c->hvcC_size = sub_size; - break; + size -= avio_get_str(pb, INT_MAX, item_name, sizeof(item_name)); + + // Skip all but the primary item until support is added + if (item_id != c->primary_item_id) + return 0; + + if (size > 0) + avio_skip(pb, size); + + switch (item_type) { + case MKTAG('a','v','0','1'): + case MKTAG('h','v','c','1'): + if (!c->heif_info) { + av_log(c->fc, AV_LOG_ERROR, "Missing iloc before iinf\n"); + return AVERROR_INVALIDDATA; } - avio_skip(pb, sub_size); + ret = heif_add_stream(c, item_id, item_type); + if (ret < 0) + return ret; + break; + default: + av_log(c->fc, AV_LOG_DEBUG, "infe: ignoring item_type %s.\n", av_fourcc2str(item_type)); + break; } - return atom.size; + + return 0; +} + +static int mov_read_iinf(MOVContext *c, AVIOContext *pb, MOVAtom atom) +{ + int entry_count; + int version, ret; + + version = avio_r8(pb); + avio_rb24(pb); // flags. + entry_count = version ? avio_rb32(pb) : avio_rb16(pb); + + for (int i = 0; i < entry_count; i++) { + MOVAtom infe; + + infe.size = avio_rb32(pb) - 8; + infe.type = avio_rl32(pb); + ret = mov_read_infe(c, pb, infe); + if (ret < 0) + return ret; + } + + return 0; +} + +static int mov_read_iprp(MOVContext *c, AVIOContext *pb, MOVAtom atom) +{ + MOVAtom ipco; + int ret; + + ipco.size = avio_rb32(pb) - 8; + ipco.type = avio_rl32(pb); + if (ipco.type != MKTAG('i','p','c','o')) + return AVERROR_INVALIDDATA; + if (avio_feof(pb)) + return AVERROR_INVALIDDATA; + + ret = mov_read_default(c, pb, ipco); + if (ret < 0) + return ret; + + return 0; } static const MOVParseTableEntry mov_default_parse_table[] = { @@ -8005,6 +8031,7 @@ static const MOVParseTableEntry mov_default_parse_table[] = { { MKTAG('p','i','t','m'), mov_read_pitm }, { MKTAG('e','v','c','C'), mov_read_glbl }, { MKTAG('i','p','r','p'), mov_read_iprp }, +{ MKTAG('i','i','n','f'), mov_read_iinf }, { 0, NULL } }; @@ -8672,8 +8699,8 @@ static int mov_read_header(AVFormatContext *s) av_log(s, AV_LOG_ERROR, "error reading header\n"); return err; } - } while ((pb->seekable & AVIO_SEEKABLE_NORMAL) && !mov->found_moov && !mov->moov_retry++); - if (!mov->found_moov) { + } while ((pb->seekable & AVIO_SEEKABLE_NORMAL) && !mov->found_moov && !mov->found_iloc && !mov->moov_retry++); + if (!mov->found_moov && !mov->found_iloc) { av_log(s, AV_LOG_ERROR, "moov atom not found\n"); return AVERROR_INVALIDDATA; } diff --git a/tests/ref/fate/mov-avif-demux-still-image-1-item b/tests/ref/fate/mov-avif-demux-still-image-1-item index 93773afd4e..770c98f868 100644 --- a/tests/ref/fate/mov-avif-demux-still-image-1-item +++ b/tests/ref/fate/mov-avif-demux-still-image-1-item @@ -1,7 +1,7 @@ #format: frame checksums #version: 2 #hash: MD5 -#extradata 0, 13, b52ae298d37128862ef1918cf916239c +#extradata 0, 4, b24b71499a8480fa4469bcbcba2140aa #tb 0: 1/1 #media_type 0: video #codec_id 0: av1 diff --git a/tests/ref/fate/mov-avif-demux-still-image-multiple-items b/tests/ref/fate/mov-avif-demux-still-image-multiple-items index 93773afd4e..770c98f868 100644 --- a/tests/ref/fate/mov-avif-demux-still-image-multiple-items +++ b/tests/ref/fate/mov-avif-demux-still-image-multiple-items @@ -1,7 +1,7 @@ #format: frame checksums #version: 2 #hash: MD5 -#extradata 0, 13, b52ae298d37128862ef1918cf916239c +#extradata 0, 4, b24b71499a8480fa4469bcbcba2140aa #tb 0: 1/1 #media_type 0: video #codec_id 0: av1