From patchwork Fri Jun 10 17:30:12 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vignesh Venkat X-Patchwork-Id: 36147 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:6914:b0:82:6b11:2509 with SMTP id q20csp1063479pzj; Fri, 10 Jun 2022 10:30:29 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwe+AwB/Or5i/kgnbKbDKGp94WapI84hiZ1rQ6ExmFB7yCYwiq3q7Qn5O3sQi4w46BFmOPM X-Received: by 2002:a17:906:649b:b0:712:24cd:6102 with SMTP id e27-20020a170906649b00b0071224cd6102mr2930630ejm.664.1654882229626; Fri, 10 Jun 2022 10:30:29 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1654882229; cv=none; d=google.com; s=arc-20160816; b=zezlFieIpwUX2HV8zi3w6vObTfKNPdpuyJ/fMb/OAcYPhgbczkawUsAh7jOI1qEFnk mhBzXycfrNlAixdSv0CzwZHcc90dXuEPj7K9fnVVztf8kGtIP5FLXF2xLd9Z2OJWfiJn 6FFa0THYsSqI7rJfzcCrQn38t6HxeBzHiFiT6kCxch38orNBGgnHE00TwicUDuVf8HXa 9fpgM6cQscigUrebPfEida9MELKJVPx3PV/dOoWv15doqIH3dksqBHeiTh81aeJlLPRv /nWRKgdXCDUe3QiW2KWRFuOZb36MJenOKYBi+nilE3qFp1mivaVxL/+m8DUF16k/3qUB 6oOw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:to:from:references:mime-version :message-id:in-reply-to:date:dkim-signature:delivered-to; bh=xEPae2+Pi4oIzdeslppAvArXvhyxviaupOH/+iDOiuY=; b=Tb7z+h63b6yATIbH1adkitt8pszhC+9iM3tVUozQpDY1sj9Rq1SrQzyVtB3DlVmGmI O35NEAcvuM4VHqZjnLY7kE83zjKz4PY5wR2l357xs7jTJDM9SPzgcN837aIPv+iXCR5v MXqcSMEtrhRFodTMLh4xu6ntR5sg3tstKe8XTv2k17gXWWmKacimdOLafP3nTtvj4q+o 6xmWSVbr13JxQUfiyAqp66EqZWlFW950p7JqqP1SdB3UW/5a7DQWbbJSjawTwCtnQFyR z0ZrGiQ60oGZ+CphoM8+lT9R9BSseypSZi1CAeKIEpvEIiwbXYcnjL8I/XQOqrnv/l60 JxyA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@google.com header.s=20210112 header.b=C9YCwPg9; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id i2-20020a05640242c200b0043343b373e9si5594899edc.592.2022.06.10.10.30.28; Fri, 10 Jun 2022 10:30:29 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@google.com header.s=20210112 header.b=C9YCwPg9; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 2D13968B358; Fri, 10 Jun 2022 20:30:25 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-yb1-f201.google.com (mail-yb1-f201.google.com [209.85.219.201]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 28F5968B2CC for ; Fri, 10 Jun 2022 20:30:18 +0300 (EEST) Received: by mail-yb1-f201.google.com with SMTP id c2-20020a25a2c2000000b0066288641421so17447044ybn.14 for ; Fri, 10 Jun 2022 10:30:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc:content-transfer-encoding; bh=61ijAwK+nPn0d6+T5eAIR4RWMJ9UWdFYPWnWEq9xdkk=; b=C9YCwPg9hoa9fniTtKR9MlM5ayyGKruZK1to7zXiFOoe/Oj+uAKCS8k95sQznRN1lB PfadOHFQckvE2dv8AIhi35f0Dfunu3AWFYJTwjnBGyLRVbtlweaUj/XF6/Wxpp8nr4bm FN3qxIkwms4e2KglLKotqnYVSqjIAWkvF+ewgLbQpGTUeTimUbGI8dIeNJBimfZvZLto mre9Wqf/7qQ+WMc+yZDW4qe7Uw0WsbzOcypGyILiaa5IaKSh+NVm01qGYexADN3kmtI1 UoMGQBJW9eAJS1y0L6XN1py4jI5B0yADCbQdpTumyBN2nK1dFh5UZSOEAikKYHoN9vaO NJdw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc:content-transfer-encoding; bh=61ijAwK+nPn0d6+T5eAIR4RWMJ9UWdFYPWnWEq9xdkk=; b=n3Au4+YMxy93++/2OmM3uU1ld3UV5G+wN58XQB288iDEx5CXS94aXuZE0JYAH89ef6 db0JVtIhMEj1x20a/GjFi8fNdkeGPDTIVAVSKYAnwMK3jFG0TG9UmB/+a+LAl0FY4gMm Lk0AYVTnfA770w3Zv+ZJIpiGCzWzwPaLNr+m7N02C1lxBX2BRSW/kFWXdfpKQVoFA5ho qlsVy5ZnQN4WJ5WFrKMJB1PpCC08mPTSxUrKzqz7cG0qJXuPOtDF7fwedowURsMV+x2b eKuNLhvBrShuhg2Y/bBOWa9LQgIQ76N/81X96xoxl4elZB5AlGhP5uGzHyx6vH2mDVPy Z9Ag== X-Gm-Message-State: AOAM533LxCpmYsMe1asXexgVTwuNhXYB3UOpx0+aAXpy5EiYvs8LOW1x QUg0wiMJzSXhMPwb8omF1i8h2tvrKKXv6orh7FVqKf41T16s6o3xRJ1OdiM1dezc2CCpzITHQsi MMMF6aNCAXLqgVX8Qi+YgGc5BTWrhiD79zIHG57x9ojdG2aY6hQi1pOKYhqEoh+ibt3hH X-Received: from vigneshv3.mtv.corp.google.com ([2620:0:1000:2511:68e4:fc9d:650:55b4]) (user=vigneshv job=sendgmr) by 2002:a25:8408:0:b0:664:381c:532a with SMTP id u8-20020a258408000000b00664381c532amr8682376ybk.448.1654882216273; Fri, 10 Jun 2022 10:30:16 -0700 (PDT) Date: Fri, 10 Jun 2022 10:30:12 -0700 In-Reply-To: <085f0277-e678-4d20-f2bf-fbf55ed959d8@gyani.pro> Message-Id: <20220610173012.3573186-1-vigneshv@google.com> Mime-Version: 1.0 References: <085f0277-e678-4d20-f2bf-fbf55ed959d8@gyani.pro> X-Mailer: git-send-email 2.36.1.476.g0c4daa206d-goog From: Vignesh Venkatasubramanian To: ffmpeg-devel@ffmpeg.org Subject: [FFmpeg-devel] [PATCH] avformat/mov: Only read the primary item for AVIF X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Vignesh Venkatasubramanian Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: NVEPmPHV+i2h Update the still AVIF parser to only read the primary item. With this patch, AVIF still images with exif/icc/alpha channel will no longer fail to parse. For example, this patch enables parsing of files in: https://github.com/AOMediaCodec/av1-avif/tree/master/testFiles/Microsoft Adding two fate tests: 1) demuxing of still image with 1 item - this test will pass regardlesss of this patch. 2) demuxing of still image with 2 items - this test will fail without this patch and will pass with patch applied. Partially fixes trac ticket #7621 Signed-off-by: Vignesh Venkatasubramanian --- libavformat/isom.h | 1 + libavformat/mov.c | 41 ++++++++++--------- tests/fate/mov.mak | 9 ++++ .../fate/mov-avif-demux-still-image-1-item | 11 +++++ .../mov-avif-demux-still-image-multiple-items | 11 +++++ 5 files changed, 53 insertions(+), 20 deletions(-) create mode 100644 tests/ref/fate/mov-avif-demux-still-image-1-item create mode 100644 tests/ref/fate/mov-avif-demux-still-image-multiple-items diff --git a/libavformat/isom.h b/libavformat/isom.h index cf36f04d5b..f05c2d9c28 100644 --- a/libavformat/isom.h +++ b/libavformat/isom.h @@ -317,6 +317,7 @@ typedef struct MOVContext { uint32_t mfra_size; uint32_t max_stts_delta; int is_still_picture_avif; + int primary_item_id; } MOVContext; int ff_mp4_read_descr_len(AVIOContext *pb); diff --git a/libavformat/mov.c b/libavformat/mov.c index d7be593a86..9310a393fe 100644 --- a/libavformat/mov.c +++ b/libavformat/mov.c @@ -7445,6 +7445,13 @@ static int rb_size(AVIOContext *pb, uint64_t* value, int size) return size; } +static int mov_read_pitm(MOVContext *c, AVIOContext *pb, MOVAtom atom) +{ + avio_rb32(pb); // version & flags. + c->primary_item_id = avio_rb16(pb); + return atom.size; +} + static int mov_read_iloc(MOVContext *c, AVIOContext *pb, MOVAtom atom) { int version, offset_size, length_size, base_offset_size, index_size; @@ -7501,34 +7508,25 @@ static int mov_read_iloc(MOVContext *c, AVIOContext *pb, MOVAtom atom) return AVERROR_PATCHWELCOME; } item_count = (version < 2) ? avio_rb16(pb) : avio_rb32(pb); - if (item_count > 1) { - // For still AVIF images, we only support one item. Second item will - // generally be found for AVIF images with alpha channel. We don't - // support them as of now. - av_log(c->fc, AV_LOG_ERROR, "iloc: item_count > 1 not supported.\n"); - return AVERROR_PATCHWELCOME; - } // Populate the necessary fields used by mov_build_index. - sc->stsc_count = item_count; - sc->stsc_data = av_malloc_array(item_count, sizeof(*sc->stsc_data)); + sc->stsc_count = 1; + sc->stsc_data = av_malloc_array(1, sizeof(*sc->stsc_data)); if (!sc->stsc_data) return AVERROR(ENOMEM); sc->stsc_data[0].first = 1; sc->stsc_data[0].count = 1; sc->stsc_data[0].id = 1; - sc->chunk_count = item_count; - sc->chunk_offsets = - av_malloc_array(item_count, sizeof(*sc->chunk_offsets)); + sc->chunk_count = 1; + sc->chunk_offsets = av_malloc_array(1, sizeof(*sc->chunk_offsets)); if (!sc->chunk_offsets) return AVERROR(ENOMEM); - sc->sample_count = item_count; - sc->sample_sizes = - av_malloc_array(item_count, sizeof(*sc->sample_sizes)); + sc->sample_count = 1; + sc->sample_sizes = av_malloc_array(1, sizeof(*sc->sample_sizes)); if (!sc->sample_sizes) return AVERROR(ENOMEM); - sc->stts_count = item_count; - sc->stts_data = av_malloc_array(item_count, sizeof(*sc->stts_data)); + sc->stts_count = 1; + sc->stts_data = av_malloc_array(1, sizeof(*sc->stts_data)); if (!sc->stts_data) return AVERROR(ENOMEM); sc->stts_data[0].count = 1; @@ -7536,7 +7534,7 @@ static int mov_read_iloc(MOVContext *c, AVIOContext *pb, MOVAtom atom) sc->stts_data[0].duration = 0; for (int i = 0; i < item_count; i++) { - (version < 2) ? avio_rb16(pb) : avio_rb32(pb); // item_id; + int item_id = (version < 2) ? avio_rb16(pb) : avio_rb32(pb); if (version > 0) avio_rb16(pb); // construction_method. avio_rb16(pb); // data_reference_index. @@ -7552,8 +7550,10 @@ static int mov_read_iloc(MOVContext *c, AVIOContext *pb, MOVAtom atom) if (rb_size(pb, &extent_offset, offset_size) < 0 || rb_size(pb, &extent_length, length_size) < 0) return AVERROR_INVALIDDATA; - sc->sample_sizes[0] = extent_length; - sc->chunk_offsets[0] = base_offset + extent_offset; + if (item_id == c->primary_item_id) { + sc->sample_sizes[0] = extent_length; + sc->chunk_offsets[0] = base_offset + extent_offset; + } } } @@ -7670,6 +7670,7 @@ static const MOVParseTableEntry mov_default_parse_table[] = { { MKTAG('S','A','3','D'), mov_read_SA3D }, /* ambisonic audio box */ { MKTAG('S','A','N','D'), mov_read_SAND }, /* non diegetic audio box */ { MKTAG('i','l','o','c'), mov_read_iloc }, +{ MKTAG('p','i','t','m'), mov_read_pitm }, { 0, NULL } }; diff --git a/tests/fate/mov.mak b/tests/fate/mov.mak index 2fae054423..842c7f9aa1 100644 --- a/tests/fate/mov.mak +++ b/tests/fate/mov.mak @@ -17,6 +17,8 @@ FATE_MOV = fate-mov-3elist \ fate-mov-bbi-elst-starts-b \ fate-mov-neg-firstpts-discard-frames \ fate-mov-stream-shorter-than-movie \ + fate-mov-avif-demux-still-image-1-item \ + fate-mov-avif-demux-still-image-multiple-items \ FATE_MOV_FFPROBE = fate-mov-neg-firstpts-discard \ fate-mov-neg-firstpts-discard-vorbis \ @@ -138,6 +140,13 @@ FATE_MOV_FFMPEG_FFPROBE-$(call TRANSCODE, TTML SUBRIP, MP4 MOV, SRT_DEMUXER TTML fate-mov-mp4-ttml-stpp: CMD = transcode srt $(TARGET_SAMPLES)/sub/SubRip_capability_tester.srt mp4 "-map 0:s -c:s ttml -time_base:s 1:1000" "-map 0 -c copy" "-of json -show_entries packet:stream=index,codec_type,codec_tag_string,codec_tag,codec_name,time_base,start_time,duration_ts,duration,nb_frames,nb_read_packets:stream_tags" fate-mov-mp4-ttml-dfxp: CMD = transcode srt $(TARGET_SAMPLES)/sub/SubRip_capability_tester.srt mp4 "-map 0:s -c:s ttml -time_base:s 1:1000 -tag:s dfxp -strict unofficial" "-map 0 -c copy" "-of json -show_entries packet:stream=index,codec_type,codec_tag_string,codec_tag,codec_name,time_base,start_time,duration_ts,duration,nb_frames,nb_read_packets:stream_tags" +# avif demuxing - still image with 1 item. +fate-mov-avif-demux-still-image-1-item: CMD = framemd5 -i $(TARGET_SAMPLES)/avif/still_image.avif -c:v copy + +# avif demuxing - still image with multiple items. only the primary item will be +# parsed. +fate-mov-avif-demux-still-image-multiple-items: CMD = framemd5 -i $(TARGET_SAMPLES)/avif/still_image_exif.avif -c:v copy + # Resulting remux should have: # 1. first audio stream with AV_DISPOSITION_HEARING_IMPAIRED # 2. second audio stream with AV_DISPOSITION_VISUAL_IMPAIRED | DESCRIPTIONS diff --git a/tests/ref/fate/mov-avif-demux-still-image-1-item b/tests/ref/fate/mov-avif-demux-still-image-1-item new file mode 100644 index 0000000000..93773afd4e --- /dev/null +++ b/tests/ref/fate/mov-avif-demux-still-image-1-item @@ -0,0 +1,11 @@ +#format: frame checksums +#version: 2 +#hash: MD5 +#extradata 0, 13, b52ae298d37128862ef1918cf916239c +#tb 0: 1/1 +#media_type 0: video +#codec_id 0: av1 +#dimensions 0: 352x288 +#sar 0: 1/1 +#stream#, dts, pts, duration, size, hash +0, 0, 0, 1, 36265, 235b0c6e389c4084845981e08d60db04 diff --git a/tests/ref/fate/mov-avif-demux-still-image-multiple-items b/tests/ref/fate/mov-avif-demux-still-image-multiple-items new file mode 100644 index 0000000000..93773afd4e --- /dev/null +++ b/tests/ref/fate/mov-avif-demux-still-image-multiple-items @@ -0,0 +1,11 @@ +#format: frame checksums +#version: 2 +#hash: MD5 +#extradata 0, 13, b52ae298d37128862ef1918cf916239c +#tb 0: 1/1 +#media_type 0: video +#codec_id 0: av1 +#dimensions 0: 352x288 +#sar 0: 1/1 +#stream#, dts, pts, duration, size, hash +0, 0, 0, 1, 36265, 235b0c6e389c4084845981e08d60db04