From patchwork Wed Oct 4 16:40:15 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vignesh Venkat X-Patchwork-Id: 44149 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:4f21:b0:15d:8365:d4b8 with SMTP id gi33csp642934pzb; Wed, 4 Oct 2023 09:40:42 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEKVGKFKsuVKCmRkIN9nXGyCpPhMCih80xb7gCy4ZUIvBj08Nirf8mnb6gdc6Kwq+8px2wN X-Received: by 2002:a05:6402:205:b0:534:8bdf:a258 with SMTP id t5-20020a056402020500b005348bdfa258mr2601926edv.31.1696437642105; Wed, 04 Oct 2023 09:40:42 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1696437642; cv=none; d=google.com; s=arc-20160816; b=ltAau+Z3hCKkdycToXBv90dCvK8EvmBEY4G/9GwKPelzXYggnKBiRCBqMeFRA9a5P8 ghxRQWCSHXP7FNrMHEaOUfwtDyepZp8jbtT9j82UlQrtyQXi9aximtC0q+LODPt/++tZ gC64n8ilEc66y39YcuzA8ReuoLA/7xzcaTPBtgik1FcSMfsC2SOHtRSZSt/ey3OTFJQJ XJot3lirKq5TiFXTP9DUbBmI85jqn3Y7KAfAsT2js+bxz9NVJr0GLRiS2axFp9i70Pto ap+shSYAWKpppUXKrhkOa8YOu83X3u31398hfEVz/ErDmCfQR5W2/9hkK86dCTpmFqZH SaVg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to:from :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:to:message-id:references:mime-version :in-reply-to:date:delivered-to; bh=8eKUKnXUrNtib3695Co5fI/4G70zG07bBFA8lE95Nbk=; fh=zLTX2kwa8L5gNw1IDWXMzP8nX5h2CbyibdEHyEbl1/E=; b=YS7P2ztVeTbeyME/MvEIPI19MCZCpQ8HraBxOJHbvLc5QFx3I3q1uvYd/j4c6T6WKE 5ccdkLAKx9XgB2Qssdx8I7cphfx9nE3l+93EEb7uT2ZexRCc1JUW2YXi1cf/fFzlxf9U Z4xdq1pXpETXBf+Ll9TJpIrf0gNi99KCApjxMXH3PPxAmaVjfIX+d6zMv9Y+qmnBd99s twWPWri71UuGurp8V16rg+NII92ACcVLHqHvSI2zSWpMk0sUHty4TsTtXRzgOy/YebZk autSlHyuJjW1vISChaqhyIJZAhh4Tfy/FMpSjfgJsi9w50w1HIZb4AkBHaTH9RWh3lyp n6VA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id l18-20020aa7d952000000b0052be6311acbsi65823eds.421.2023.10.04.09.40.41; Wed, 04 Oct 2023 09:40:42 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id E0EFF68CB68; Wed, 4 Oct 2023 19:40:38 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-yb1-f201.google.com (mail-yb1-f201.google.com [209.85.219.201]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 52EFF68C9B4 for ; Wed, 4 Oct 2023 19:40:32 +0300 (EEST) Received: by mail-yb1-f201.google.com with SMTP id 3f1490d57ef6-d816fa2404aso92160276.0 for ; Wed, 04 Oct 2023 09:40:32 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1696437630; x=1697042430; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=TFDk3e+p+GPhxfJt6yw7fFSasSjQQcqCSXN5/XFl7NU=; b=E5803hzURUon/DXcMfaTQhAkPRzAp3wptTFURwuUJi6eiqIA1e8pI82OK+xxM8A/QN yN3c0YD8rZuhg4wBiW+sPjy9Mmbw6fFzgEfnNkyKd7wUU2Ndo/cemyOk/PXfuzI7pBWx RxEw5cdt1NRLnWmHbixDLeZHja6SnqFaO82QU2ro4BB56DkrIoa9f7EJ3NQi8uhMumZJ a6i9h8Swfft5Q9WyPJhe/p/lW1nc21rtEEEAp19uYbCWHYuzsNl1s30lGtQB9AaKDHt3 DKeBZDNLHdTbKhuJoEjHsqAm+yVkv4WQdVPEj2khK4ekrDoVII1DIhB/mGukOO2BhHin WQ+w== X-Gm-Message-State: AOJu0YzG0A9Rrlgph4tRBWNL13rSqFjGjs/9qQtiv8Saanka+1y65N69 woshnrROrwR41jejry01GZy5TVi1SJIMhOptu0N8BofbY6h6+pMa6i7RZwaRKy8e8otDOmpTA2L cG8sIrJJp4vgBY8JDZ5NNVBae7hgVpIx+l9cBR2/mUqkzegOU94of9xYVsz7mu68jaG3Z X-Received: from vigneshv3.mtv.corp.google.com ([2620:0:1000:2511:4f1:ef9c:a0c0:745c]) (user=vigneshv job=sendgmr) by 2002:a05:6902:1603:b0:d7a:bfcf:2d7 with SMTP id bw3-20020a056902160300b00d7abfcf02d7mr42234ybb.6.1696437630495; Wed, 04 Oct 2023 09:40:30 -0700 (PDT) Date: Wed, 4 Oct 2023 09:40:15 -0700 In-Reply-To: Mime-Version: 1.0 References: X-Mailer: git-send-email 2.42.0.582.g8ccd20d70d-goog Message-ID: <20231004164019.1428483-1-vigneshv@google.com> To: ffmpeg-devel@ffmpeg.org Subject: [FFmpeg-devel] [PATCH v3] avformat/mov: Add support for demuxing still HEIC images X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Vignesh Venkatasubramanian via ffmpeg-devel From: Vignesh Venkat Reply-To: FFmpeg development discussions and patches Cc: Vignesh Venkatasubramanian Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 6YC3sac9Fynh They are similar to AVIF images (both use the HEIF container). The only additional work needed is to parse the hvcC box and put it in the extradata. With this patch applied, ffmpeg (when built with an HEVC decoder) is able to decode the files in https://github.com/nokiatech/heif/tree/gh-pages/content/images Also add a couple of fate tests with samples from https://github.com/nokiatech/heif_conformance/tree/master/conformance_files Partially fixes trac ticket #6521. Signed-off-by: Vignesh Venkatasubramanian --- libavformat/isom.h | 2 + libavformat/mov.c | 41 ++++++++++++++++++- tests/fate/mov.mak | 6 +++ .../fate/mov-heic-demux-still-image-1-item | 11 +++++ .../mov-heic-demux-still-image-multiple-items | 11 +++++ 5 files changed, 70 insertions(+), 1 deletion(-) create mode 100644 tests/ref/fate/mov-heic-demux-still-image-1-item create mode 100644 tests/ref/fate/mov-heic-demux-still-image-multiple-items diff --git a/libavformat/isom.h b/libavformat/isom.h index 3d375d7a46..b30b9da65e 100644 --- a/libavformat/isom.h +++ b/libavformat/isom.h @@ -327,6 +327,8 @@ typedef struct MOVContext { int64_t extent_offset; } *avif_info; int avif_info_size; + int64_t hvcC_offset; + int hvcC_size; int interleaved_read; } MOVContext; diff --git a/libavformat/mov.c b/libavformat/mov.c index 294c864fbd..d3747022bd 100644 --- a/libavformat/mov.c +++ b/libavformat/mov.c @@ -1218,7 +1218,8 @@ static int mov_read_ftyp(MOVContext *c, AVIOContext *pb, MOVAtom atom) c->isom = 1; av_log(c->fc, AV_LOG_DEBUG, "ISO: File Type Major Brand: %.4s\n",(char *)&type); av_dict_set(&c->fc->metadata, "major_brand", type, 0); - c->is_still_picture_avif = !strncmp(type, "avif", 4); + c->is_still_picture_avif = !strncmp(type, "avif", 4) || + !strncmp(type, "mif1", 4); minor_ver = avio_rb32(pb); /* minor version */ av_dict_set_int(&c->fc->metadata, "minor_version", minor_ver, 0); @@ -4911,6 +4912,19 @@ static int avif_add_stream(MOVContext *c, int item_id) st->priv_data = sc; st->codecpar->codec_type = AVMEDIA_TYPE_VIDEO; st->codecpar->codec_id = AV_CODEC_ID_AV1; + if (c->hvcC_offset >= 0) { + int ret; + int64_t pos = avio_tell(c->fc->pb); + st->codecpar->codec_id = AV_CODEC_ID_HEVC; + if (avio_seek(c->fc->pb, c->hvcC_offset, SEEK_SET) != c->hvcC_offset) { + av_log(c->fc, AV_LOG_ERROR, "Failed to seek to hvcC data.\n"); + return AVERROR_UNKNOWN; + } + ret = ff_get_extradata(c->fc, st->codecpar, c->fc->pb, c->hvcC_size); + if (ret < 0) + return ret; + avio_seek(c->fc->pb, pos, SEEK_SET); + } sc->ffindex = st->index; c->trak_index = st->index; st->avg_frame_rate.num = st->avg_frame_rate.den = 1; @@ -4953,6 +4967,8 @@ static int avif_add_stream(MOVContext *c, int item_id) static int mov_read_meta(MOVContext *c, AVIOContext *pb, MOVAtom atom) { + c->hvcC_offset = -1; + c->hvcC_size = 0; while (atom.size > 8) { uint32_t tag; if (avio_feof(pb)) @@ -7827,6 +7843,28 @@ static int mov_read_iloc(MOVContext *c, AVIOContext *pb, MOVAtom atom) return atom.size; } +static int mov_read_iprp(MOVContext *c, AVIOContext *pb, MOVAtom atom) +{ + int size = avio_rb32(pb); + if (avio_rl32(pb) != MKTAG('i','p','c','o')) + return AVERROR_INVALIDDATA; + size -= 8; + while (size > 0) { + int sub_size, sub_type; + sub_size = avio_rb32(pb); + sub_type = avio_rl32(pb); + sub_size -= 8; + size -= sub_size + 8; + if (sub_type == MKTAG('h','v','c','C')) { + c->hvcC_offset = avio_tell(pb); + c->hvcC_size = sub_size; + break; + } + avio_skip(pb, sub_size); + } + return atom.size; +} + static const MOVParseTableEntry mov_default_parse_table[] = { { MKTAG('A','C','L','R'), mov_read_aclr }, { MKTAG('A','P','R','G'), mov_read_avid }, @@ -7934,6 +7972,7 @@ static const MOVParseTableEntry mov_default_parse_table[] = { { MKTAG('p','c','m','C'), mov_read_pcmc }, /* PCM configuration box */ { MKTAG('p','i','t','m'), mov_read_pitm }, { MKTAG('e','v','c','C'), mov_read_glbl }, +{ MKTAG('i','p','r','p'), mov_read_iprp }, { 0, NULL } }; diff --git a/tests/fate/mov.mak b/tests/fate/mov.mak index 6cb493ceab..a2d3cc8013 100644 --- a/tests/fate/mov.mak +++ b/tests/fate/mov.mak @@ -18,6 +18,8 @@ FATE_MOV = fate-mov-3elist \ fate-mov-neg-firstpts-discard-frames \ fate-mov-stream-shorter-than-movie \ fate-mov-pcm-remux \ + fate-mov-heic-demux-still-image-1-item \ + fate-mov-heic-demux-still-image-multiple-items \ # FIXME: Uncomment these two lines once the test files are uploaded to the fate # server. # fate-mov-avif-demux-still-image-1-item \ @@ -152,6 +154,10 @@ fate-mov-mp4-ttml-dfxp: CMD = transcode srt $(TARGET_SAMPLES)/sub/SubRip_capabil # parsed. #fate-mov-avif-demux-still-image-multiple-items: CMD = framemd5 -i $(TARGET_SAMPLES)/avif/still_image_exif.avif -c:v copy +fate-mov-heic-demux-still-image-1-item: CMD = framemd5 -i $(TARGET_SAMPLES)/heif-conformance/C002.heic -c:v copy + +fate-mov-heic-demux-still-image-multiple-items: CMD = framemd5 -i $(TARGET_SAMPLES)/heif-conformance/C003.heic -c:v copy + # Resulting remux should have: # 1. first audio stream with AV_DISPOSITION_HEARING_IMPAIRED # 2. second audio stream with AV_DISPOSITION_VISUAL_IMPAIRED | DESCRIPTIONS diff --git a/tests/ref/fate/mov-heic-demux-still-image-1-item b/tests/ref/fate/mov-heic-demux-still-image-1-item new file mode 100644 index 0000000000..c850c1ff9c --- /dev/null +++ b/tests/ref/fate/mov-heic-demux-still-image-1-item @@ -0,0 +1,11 @@ +#format: frame checksums +#version: 2 +#hash: MD5 +#extradata 0, 100, 5444bf01e03182c73ae957179d560f4d +#tb 0: 1/1 +#media_type 0: video +#codec_id 0: hevc +#dimensions 0: 1280x720 +#sar 0: 0/1 +#stream#, dts, pts, duration, size, hash +0, 0, 0, 1, 111554, 03ceabfab39afd2e2e796b9362111f32 diff --git a/tests/ref/fate/mov-heic-demux-still-image-multiple-items b/tests/ref/fate/mov-heic-demux-still-image-multiple-items new file mode 100644 index 0000000000..c850c1ff9c --- /dev/null +++ b/tests/ref/fate/mov-heic-demux-still-image-multiple-items @@ -0,0 +1,11 @@ +#format: frame checksums +#version: 2 +#hash: MD5 +#extradata 0, 100, 5444bf01e03182c73ae957179d560f4d +#tb 0: 1/1 +#media_type 0: video +#codec_id 0: hevc +#dimensions 0: 1280x720 +#sar 0: 0/1 +#stream#, dts, pts, duration, size, hash +0, 0, 0, 1, 111554, 03ceabfab39afd2e2e796b9362111f32