From patchwork Thu May 12 16:23:28 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vignesh Venkat X-Patchwork-Id: 35751 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:a885:b0:7f:4be2:bd17 with SMTP id ca5csp531651pzb; Thu, 12 May 2022 09:23:43 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzaBGAKjP89fPyd81jy+soCJ6kThKZ26laQ7zHv+D9scHDffI/FRw+2nj6S51jgQTA0ZCZu X-Received: by 2002:a17:907:d02:b0:6f4:5a54:edb3 with SMTP id gn2-20020a1709070d0200b006f45a54edb3mr668497ejc.308.1652372623704; Thu, 12 May 2022 09:23:43 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1652372623; cv=none; d=google.com; s=arc-20160816; b=sm2cWM0+KssK+HuwiexApDr4o2/Szy6QUoNG/6AxPrr9hmFxKYWlZRcjm213ljpqFB aJbKZNsnppQD1dz0zIiqxePJLl3BPUIH1SZH1ndJHkemkstjokSkwPryIvBZeCOANhud yTJDRPAvgenikSnzCuLhsSKc9E0ocG9kteyTJXK5rwT6naq58F2yJ1CbUsIqWW0o/c1+ p4bAgcAmE6QD6ruY6qZtCPMOe4TgRGjCNpF7tLGYbP5tsgxPZqKWPpZ4VprAJXgtNVRf g4VkBwpZND40ps4r9DnAse1bTEfb28rNXxcqLlGtsfrRfjoSyyUXyiBt1sLvYPvcpA73 v+ZA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:to:from:references:mime-version :message-id:in-reply-to:date:dkim-signature:delivered-to; bh=pgEBkeTjG6jgsRiDIeLS7X+0BCZfZlaYL5SFSIBrHfM=; b=0h+mNxfG2gGKAvqqz2MyFZwpNH4znjGg3eRa0JB7VqWmhr5Mt2z45nIdaHWHuEjCYt LsD9SgKUC1+o25Ww4QtCmFPL15yah3l47k++d6nkspVCD3boD0ePJMZS1HaXINPtt13P jc8Vo1A8wpGlYkjYzrSX98vfnkA4RSTWM+mKH7Sea9aTmMJHP9Cs2HhKYLgbGkH2Mxmz mjev6PvNTV+VQIg9o+5vWK9G2qy6PLEEnd/xrruYZXNTAMRjJwImLl5lF1TudrSNbtZE lxOsDdvU0/UlSOahA0D+77fyN5Cn5dK82ltOLK/ELM1to4qprFyymQoUgFm7byngOrSJ XX1A== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@google.com header.s=20210112 header.b=k9z0SM2D; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id hc39-20020a17090716a700b006f39af9ad2esi7393731ejc.436.2022.05.12.09.23.43; Thu, 12 May 2022 09:23:43 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@google.com header.s=20210112 header.b=k9z0SM2D; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 4A2D568B439; Thu, 12 May 2022 19:23:40 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-yw1-f202.google.com (mail-yw1-f202.google.com [209.85.128.202]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 3D1EB68B3F4 for ; Thu, 12 May 2022 19:23:34 +0300 (EEST) Received: by mail-yw1-f202.google.com with SMTP id 00721157ae682-2f825f8c944so49172137b3.15 for ; Thu, 12 May 2022 09:23:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=zGcjuIwkccFJp+kwwjqO8h1K0iGBpChsyGBU+Wnlwvo=; b=k9z0SM2DIyfAqvTtbRuIK2RUmj192WQ7EfBgskRnG0sk1SMSuqwbWpYgUdgpsxN6kX cnkCohoCwdQVhi/ytmKvrJqQkHTfGN5Zz9GiPnrLh3GS5k9rEVYx+DI0loL52Oqqgu7m 9JhNkUPChK7ImY1Jmej0InLp9BhqvYTCwznWKGn0phm9MPd5vY+gdrYlmOYWrYGxQ7gy 5uV9r3l3kvBo7GkefOOFXagyxDfeKM26lIHAdfrg0o45zbczPpmRY5GdUmvm56YWdBA0 TZ6bBaIfS/0Q9+vMmThERDs5F90HikctEOn1/N9IGBM6wHrfjjyfsM4BLO6SzKZ+pjIx P7Nw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=zGcjuIwkccFJp+kwwjqO8h1K0iGBpChsyGBU+Wnlwvo=; b=Vo7HCvRH6BgkoNbfm5qpxP9ajha/erGdKCgvmho0M6/6TG66IW+8X23Vt6JvTIEG1h HYWXU0GG2Iv4WpnGwwfq+brXm4OibXgaff5/D20DKSdd6ghvxJkaa+za7YvDs5Qdmuky tYxKrkEvW5BEvJZX0vXJZYqCnfgm2i+pRZCiYBbL/t0bovNZm1UIFaJmKXcBGjW18m10 1SQxMKG488GB3UyKNkzhZo0WJ+BynEX36dG94WjFP8F43xYTV6FRKZeMA7qnnk02Ee/e LpMNFVnNF7J4NSaelE7RoD2uCBe0wWr9A8P3LS1Xa6uN518Rl51ejcH30jrV3De0KiTb fKog== X-Gm-Message-State: AOAM530+cbJ/qnJaDHd4UCYIYPY22iu9K8GAzcNOkU4kOXOPamrP1TMf dFgmld6nUY2XNEGOTJ6dy1Hdy3PNFZQl0lL18EvLD0IRL3g7Asf/cEN4Il1TH+onmo6ZQcoz7jT ntVn3CY+66AjfBzIBflQlWx5IzHl1t0dV9hXySWksj7jg7JgJGwBczmhGZh7kKOw5abl/ X-Received: from vigneshv3.mtv.corp.google.com ([2620:0:1000:2511:ccbd:7e9d:b125:afa9]) (user=vigneshv job=sendgmr) by 2002:a81:7743:0:b0:2fe:9a0c:278f with SMTP id s64-20020a817743000000b002fe9a0c278fmr920442ywc.43.1652372612665; Thu, 12 May 2022 09:23:32 -0700 (PDT) Date: Thu, 12 May 2022 09:23:28 -0700 In-Reply-To: Message-Id: <20220512162328.3648920-1-vigneshv@google.com> Mime-Version: 1.0 References: X-Mailer: git-send-email 2.36.0.550.gb090851708-goog From: Vignesh Venkatasubramanian To: ffmpeg-devel@ffmpeg.org Subject: [FFmpeg-devel] [PATCH 3/3] avformat/movenc: Add support for AVIF muxing X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Vignesh Venkatasubramanian Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 6jQ7/gBp06B3 Add an AVIF muxer by re-using the existing the mov/mp4 muxer. AVIF Specification: https://aomediacodec.github.io/av1-avif Sample usage for still image: ffmpeg -i image.png -c:v libaom-av1 -still-picture 1 image.avif Sample usage for animated AVIF image: ffmpeg -i video.mp4 animated.avif We can re-use any of the AV1 encoding options that will make sense for image encoding (like bitrate, tiles, encoding speed, etc). The files generated by this muxer has been verified to be valid AVIF files by the following: 1) Displays on Chrome (both still and animated images). 2) Displays on Firefox (only still images, firefox does not support animated AVIF yet). 3) Verified to be valid by Compliance Warden: https://github.com/gpac/ComplianceWarden Fixes the encoder/muxer part of Trac Ticket #7621 Signed-off-by: Vignesh Venkatasubramanian --- configure | 1 + libavformat/allformats.c | 1 + libavformat/movenc.c | 341 ++++++++++++++++++++++++++++++++++++--- libavformat/movenc.h | 5 + 4 files changed, 323 insertions(+), 25 deletions(-) diff --git a/configure b/configure index 196873c4aa..2992f9760e 100755 --- a/configure +++ b/configure @@ -3404,6 +3404,7 @@ asf_stream_muxer_select="asf_muxer" av1_demuxer_select="av1_frame_merge_bsf av1_parser" avi_demuxer_select="riffdec exif" avi_muxer_select="riffenc" +avif_muxer_select="mov_muxer" caf_demuxer_select="iso_media" caf_muxer_select="iso_media" dash_muxer_select="mp4_muxer" diff --git a/libavformat/allformats.c b/libavformat/allformats.c index 63876c468f..1802536633 100644 --- a/libavformat/allformats.c +++ b/libavformat/allformats.c @@ -81,6 +81,7 @@ extern const AVOutputFormat ff_au_muxer; extern const AVInputFormat ff_av1_demuxer; extern const AVInputFormat ff_avi_demuxer; extern const AVOutputFormat ff_avi_muxer; +extern const AVOutputFormat ff_avif_muxer; extern const AVInputFormat ff_avisynth_demuxer; extern const AVOutputFormat ff_avm2_muxer; extern const AVInputFormat ff_avr_demuxer; diff --git a/libavformat/movenc.c b/libavformat/movenc.c index 271db99b46..a07c0ae2b4 100644 --- a/libavformat/movenc.c +++ b/libavformat/movenc.c @@ -1335,7 +1335,7 @@ static int mov_write_av1c_tag(AVIOContext *pb, MOVTrack *track) avio_wb32(pb, 0); ffio_wfourcc(pb, "av1C"); - ff_isom_write_av1c(pb, track->vos_data, track->vos_len, 1); + ff_isom_write_av1c(pb, track->vos_data, track->vos_len, track->mode != MODE_AVIF); return update_size(pb, pos); } @@ -2037,12 +2037,13 @@ static int mov_write_colr_tag(AVIOContext *pb, MOVTrack *track, int prefer_icc) } } - /* We should only ever be called by MOV or MP4. */ - av_assert0(track->mode == MODE_MOV || track->mode == MODE_MP4); + /* We should only ever be called for MOV, MP4 and AVIF. */ + av_assert0(track->mode == MODE_MOV || track->mode == MODE_MP4 || + track->mode == MODE_AVIF); avio_wb32(pb, 0); /* size */ ffio_wfourcc(pb, "colr"); - if (track->mode == MODE_MP4) + if (track->mode == MODE_MP4 || track->mode == MODE_AVIF) ffio_wfourcc(pb, "nclx"); else ffio_wfourcc(pb, "nclc"); @@ -2052,7 +2053,7 @@ static int mov_write_colr_tag(AVIOContext *pb, MOVTrack *track, int prefer_icc) avio_wb16(pb, track->par->color_primaries); avio_wb16(pb, track->par->color_trc); avio_wb16(pb, track->par->color_space); - if (track->mode == MODE_MP4) { + if (track->mode == MODE_MP4 || track->mode == MODE_AVIF) { int full_range = track->par->color_range == AVCOL_RANGE_JPEG; avio_w8(pb, full_range << 7); } @@ -2118,7 +2119,7 @@ static void find_compressor(char * compressor_name, int len, MOVTrack *track) || (track->par->width == 1440 && track->par->height == 1080) || (track->par->width == 1920 && track->par->height == 1080); - if (track->mode == MODE_MOV && + if ((track->mode == MODE_AVIF || track->mode == MODE_MOV) && (encoder = av_dict_get(track->st->metadata, "encoder", NULL, 0))) { av_strlcpy(compressor_name, encoder->value, 32); } else if (track->par->codec_id == AV_CODEC_ID_MPEG2VIDEO && xdcam_res) { @@ -2139,6 +2140,25 @@ static void find_compressor(char * compressor_name, int len, MOVTrack *track) } } +static int mov_write_ccst_tag(AVIOContext *pb) +{ + int64_t pos = avio_tell(pb); + // Write sane defaults: + // all_ref_pics_intra = 0 : all samples can use any type of reference. + // intra_pred_used = 1 : intra prediction may or may not be used. + // max_ref_per_pic = 15 : reserved value to indicate that any number of + // reference images can be used. + uint8_t ccstValue = (0 << 7) | /* all_ref_pics_intra */ + (1 << 6) | /* intra_pred_used */ + (15 << 2); /* max_ref_per_pic */ + avio_wb32(pb, 0); /* size */ + ffio_wfourcc(pb, "ccst"); + avio_wb32(pb, 0); /* Version & flags */ + avio_w8(pb, ccstValue); + avio_wb24(pb, 0); /* reserved */ + return update_size(pb, pos); +} + static int mov_write_video_tag(AVFormatContext *s, AVIOContext *pb, MOVMuxContext *mov, MOVTrack *track) { int ret = AVERROR_BUG; @@ -2272,7 +2292,7 @@ static int mov_write_video_tag(AVFormatContext *s, AVIOContext *pb, MOVMuxContex else av_log(mov->fc, AV_LOG_WARNING, "Not writing 'gama' atom. Format is not MOV.\n"); } - if (track->mode == MODE_MOV || track->mode == MODE_MP4) { + if (track->mode == MODE_MOV || track->mode == MODE_MP4 || track->mode == MODE_AVIF) { int has_color_info = track->par->color_primaries != AVCOL_PRI_UNSPECIFIED && track->par->color_trc != AVCOL_TRC_UNSPECIFIED && track->par->color_space != AVCOL_SPC_UNSPECIFIED; @@ -2324,6 +2344,9 @@ static int mov_write_video_tag(AVFormatContext *s, AVIOContext *pb, MOVMuxContex if (avid) avio_wb32(pb, 0); + if (track->mode == MODE_AVIF) + mov_write_ccst_tag(pb); + return update_size(pb, pos); } @@ -2826,8 +2849,13 @@ static int mov_write_hdlr_tag(AVFormatContext *s, AVIOContext *pb, MOVTrack *tra if (track) { hdlr = (track->mode == MODE_MOV) ? "mhlr" : "\0\0\0\0"; if (track->par->codec_type == AVMEDIA_TYPE_VIDEO) { - hdlr_type = "vide"; - descr = "VideoHandler"; + if (track->mode == MODE_AVIF) { + hdlr_type = "pict"; + descr = "PictureHandler"; + } else { + hdlr_type = "vide"; + descr = "VideoHandler"; + } } else if (track->par->codec_type == AVMEDIA_TYPE_AUDIO) { hdlr_type = "soun"; descr = "SoundHandler"; @@ -2892,6 +2920,128 @@ static int mov_write_hdlr_tag(AVFormatContext *s, AVIOContext *pb, MOVTrack *tra return update_size(pb, pos); } +static int mov_write_pitm_tag(AVIOContext *pb, int item_id) +{ + int64_t pos = avio_tell(pb); + avio_wb32(pb, 0); /* size */ + ffio_wfourcc(pb, "pitm"); + avio_wb32(pb, 0); /* Version & flags */ + avio_wb16(pb, item_id); /* item_id */ + return update_size(pb, pos); +} + +static int mov_write_iloc_tag(AVIOContext *pb, MOVMuxContext *mov, AVFormatContext *s) +{ + int64_t pos = avio_tell(pb); + avio_wb32(pb, 0); /* size */ + ffio_wfourcc(pb, "iloc"); + avio_wb32(pb, 0); /* Version & flags */ + avio_w8(pb, (4 << 4) + 4); /* offset_size(4) and length_size(4) */ + avio_w8(pb, 0); /* base_offset_size(4) and reserved(4) */ + avio_wb16(pb, 1); /* item_count */ + + avio_wb16(pb, 1); /* item_id */ + avio_wb16(pb, 0); /* data_reference_index */ + avio_wb16(pb, 1); /* extent_count */ + mov->avif_extent_pos = avio_tell(pb); + avio_wb32(pb, 0); /* extent_offset (written later) */ + // For animated AVIF, we simply write the first packet's size. + avio_wb32(pb, mov->avif_extent_length); /* extent_length */ + + return update_size(pb, pos); +} + +static int mov_write_iinf_tag(AVIOContext *pb, MOVMuxContext *mov, AVFormatContext *s) +{ + int64_t infe_pos; + int64_t iinf_pos = avio_tell(pb); + avio_wb32(pb, 0); /* size */ + ffio_wfourcc(pb, "iinf"); + avio_wb32(pb, 0); /* Version & flags */ + avio_wb16(pb, 1); /* entry_count */ + + infe_pos = avio_tell(pb); + avio_wb32(pb, 0); /* size */ + ffio_wfourcc(pb, "infe"); + avio_w8(pb, 0x2); /* Version */ + avio_wb24(pb, 0); /* flags */ + avio_wb16(pb, 1); /* item_id */ + avio_wb16(pb, 0); /* item_protection_index */ + avio_write(pb, "av01", 4); /* item_type */ + avio_write(pb, "Color\0", 6); /* item_name */ + update_size(pb, infe_pos); + + return update_size(pb, iinf_pos); +} + +static int mov_write_ispe_tag(AVIOContext *pb, MOVMuxContext *mov, AVFormatContext *s) +{ + int64_t pos = avio_tell(pb); + avio_wb32(pb, 0); /* size */ + ffio_wfourcc(pb, "ispe"); + avio_wb32(pb, 0); /* Version & flags */ + avio_wb32(pb, s->streams[0]->codecpar->width); /* image_width */ + avio_wb32(pb, s->streams[0]->codecpar->height); /* image_height */ + return update_size(pb, pos); +} + +static int mov_write_pixi_tag(AVIOContext *pb, MOVMuxContext *mov, AVFormatContext *s) +{ + int64_t pos = avio_tell(pb); + const AVPixFmtDescriptor *pixdesc = av_pix_fmt_desc_get(s->streams[0]->codecpar->format); + avio_wb32(pb, 0); /* size */ + ffio_wfourcc(pb, "pixi"); + avio_wb32(pb, 0); /* Version & flags */ + avio_w8(pb, pixdesc->nb_components); /* num_channels */ + for (int i = 0; i < pixdesc->nb_components; ++i) { + avio_w8(pb, pixdesc->comp[i].depth); /* bits_per_channel */ + } + return update_size(pb, pos); +} + +static int mov_write_ipco_tag(AVIOContext *pb, MOVMuxContext *mov, AVFormatContext *s) +{ + int64_t pos = avio_tell(pb); + avio_wb32(pb, 0); /* size */ + ffio_wfourcc(pb, "ipco"); + mov_write_ispe_tag(pb, mov, s); + mov_write_pixi_tag(pb, mov, s); + mov_write_av1c_tag(pb, &mov->tracks[0]); + mov_write_colr_tag(pb, &mov->tracks[0], 0); + return update_size(pb, pos); +} + +static int mov_write_ipma_tag(AVIOContext *pb, MOVMuxContext *mov, AVFormatContext *s) +{ + int64_t pos = avio_tell(pb); + avio_wb32(pb, 0); /* size */ + ffio_wfourcc(pb, "ipma"); + avio_wb32(pb, 0); /* Version & flags */ + avio_wb32(pb, 1); /* entry_count */ + avio_wb16(pb, 1); /* item_ID */ + avio_w8(pb, 4); /* association_count */ + + // ispe association. + avio_w8(pb, 1); /* essential and property_index */ + // pixi association. + avio_w8(pb, 2); /* essential and property_index */ + // av1C association. + avio_w8(pb, 0x80 | 3); /* essential and property_index */ + // colr association. + avio_w8(pb, 4); /* essential and property_index */ + return update_size(pb, pos); +} + +static int mov_write_iprp_tag(AVIOContext *pb, MOVMuxContext *mov, AVFormatContext *s) +{ + int64_t pos = avio_tell(pb); + avio_wb32(pb, 0); /* size */ + ffio_wfourcc(pb, "iprp"); + mov_write_ipco_tag(pb, mov, s); + mov_write_ipma_tag(pb, mov, s); + return update_size(pb, pos); +} + static int mov_write_hmhd_tag(AVIOContext *pb) { /* This atom must be present, but leaving the values at zero @@ -3137,7 +3287,7 @@ static int mov_write_tkhd_tag(AVIOContext *pb, MOVMuxContext *mov, if (st && (track->par->codec_type == AVMEDIA_TYPE_VIDEO || track->par->codec_type == AVMEDIA_TYPE_SUBTITLE)) { int64_t track_width_1616; - if (track->mode == MODE_MOV) { + if (track->mode == MODE_MOV || track->mode == MODE_AVIF) { track_width_1616 = track->par->width * 0x10000ULL; } else { track_width_1616 = av_rescale(st->sample_aspect_ratio.num, @@ -3536,6 +3686,7 @@ static int mov_write_mvhd_tag(AVIOContext *pb, MOVMuxContext *mov) int max_track_id = 1, i; int64_t max_track_len = 0; int version; + int timescale; for (i = 0; i < mov->nb_streams; i++) { if (mov->tracks[i].entry > 0 && mov->tracks[i].timescale) { @@ -3570,7 +3721,12 @@ static int mov_write_mvhd_tag(AVIOContext *pb, MOVMuxContext *mov) avio_wb32(pb, mov->time); /* creation time */ avio_wb32(pb, mov->time); /* modification time */ } - avio_wb32(pb, mov->movie_timescale); + + timescale = mov->movie_timescale; + if (mov->mode == MODE_AVIF && !timescale) + timescale = mov->tracks[0].timescale; + + avio_wb32(pb, timescale); (version == 1) ? avio_wb64(pb, max_track_len) : avio_wb32(pb, max_track_len); /* duration of longest track */ avio_wb32(pb, 0x00010000); /* reserved (preferred rate) 1.0 = normal */ @@ -3947,8 +4103,15 @@ static int mov_write_meta_tag(AVIOContext *pb, MOVMuxContext *mov, mov_write_mdta_hdlr_tag(pb, mov, s); mov_write_mdta_keys_tag(pb, mov, s); mov_write_mdta_ilst_tag(pb, mov, s); - } - else { + } else if (mov->mode == MODE_AVIF) { + mov_write_hdlr_tag(s, pb, &mov->tracks[0]); + // We always write the primary item id as 1 since only one track is + // supported for AVIF. + mov_write_pitm_tag(pb, 1); + mov_write_iloc_tag(pb, mov, s); + mov_write_iinf_tag(pb, mov, s); + mov_write_iprp_tag(pb, mov, s); + } else { /* iTunes metadata tag */ mov_write_itunes_hdlr_tag(pb, mov, s); mov_write_ilst_tag(pb, mov, s); @@ -4278,10 +4441,11 @@ static int mov_write_moov_tag(AVIOContext *pb, MOVMuxContext *mov, } mov_write_mvhd_tag(pb, mov); - if (mov->mode != MODE_MOV && !mov->iods_skip) + if (mov->mode != MODE_MOV && mov->mode != MODE_AVIF && !mov->iods_skip) mov_write_iods_tag(pb, mov); for (i = 0; i < mov->nb_streams; i++) { - if (mov->tracks[i].entry > 0 || mov->flags & FF_MOV_FLAG_FRAGMENT) { + if (mov->tracks[i].entry > 0 || mov->flags & FF_MOV_FLAG_FRAGMENT || + mov->mode == MODE_AVIF) { int ret = mov_write_trak_tag(s, pb, mov, &(mov->tracks[i]), i < s->nb_streams ? s->streams[i] : NULL); if (ret < 0) return ret; @@ -4292,7 +4456,7 @@ static int mov_write_moov_tag(AVIOContext *pb, MOVMuxContext *mov, if (mov->mode == MODE_PSP) mov_write_uuidusmt_tag(pb, s); - else + else if (mov->mode != MODE_AVIF) mov_write_udta_tag(pb, mov, s); return update_size(pb, pos); @@ -5039,6 +5203,9 @@ static void mov_write_ftyp_tag_internal(AVIOContext *pb, AVFormatContext *s, else if (mov->mode == MODE_3GP) { ffio_wfourcc(pb, has_h264 ? "3gp6" : "3gp4"); minor = has_h264 ? 0x100 : 0x200; + } else if (mov->mode == MODE_AVIF) { + ffio_wfourcc(pb, mov->is_animated_avif ? "avis" : "avif"); + minor = 0; } else if (mov->mode & MODE_3G2) { ffio_wfourcc(pb, has_h264 ? "3g2b" : "3g2a"); minor = has_h264 ? 0x20000 : 0x10000; @@ -5102,6 +5269,31 @@ static int mov_write_ftyp_tag(AVIOContext *pb, AVFormatContext *s) // compatible brand a second time. if (mov->mode == MODE_ISM) { ffio_wfourcc(pb, "piff"); + } else if (mov->mode == MODE_AVIF) { + const AVPixFmtDescriptor *pix_fmt_desc = + av_pix_fmt_desc_get(s->streams[0]->codecpar->format); + const int depth = pix_fmt_desc->comp[0].depth; + if (mov->is_animated_avif) { + // For animated AVIF, major brand is "avis". Add "avif" as a + // compatible brand. + ffio_wfourcc(pb, "avif"); + ffio_wfourcc(pb, "msf1"); + ffio_wfourcc(pb, "iso8"); + } + ffio_wfourcc(pb, "mif1"); + ffio_wfourcc(pb, "miaf"); + if (depth == 8 || depth == 10) { + // MA1B and MA1A brands are based on AV1 profile. Short hand for + // computing that is based on chroma subsampling type. 420 chroma + // subsampling is MA1B. 444 chroma subsampling is MA1A. + if (!pix_fmt_desc->log2_chroma_w && !pix_fmt_desc->log2_chroma_h) { + // 444 chroma subsampling. + ffio_wfourcc(pb, "MA1A"); + } else { + // 420 chroma subsampling. + ffio_wfourcc(pb, "MA1B"); + } + } } else if (mov->mode != MODE_MOV) { // We add tfdt atoms when fragmenting, signal this with the iso6 compatible // brand, if not already the major brand. This is compatible with users that @@ -5705,7 +5897,7 @@ int ff_mov_write_packet(AVFormatContext *s, AVPacket *pkt) if (ret < 0) return ret; - if (mov->flags & FF_MOV_FLAG_FRAGMENT) { + if (mov->flags & FF_MOV_FLAG_FRAGMENT || mov->mode == MODE_AVIF) { int ret; if (mov->moov_written || mov->flags & FF_MOV_FLAG_EMPTY_MOOV) { if (mov->frag_interleave && mov->fragments > 0) { @@ -5846,7 +6038,11 @@ int ff_mov_write_packet(AVFormatContext *s, AVPacket *pkt) avio_write(pb, reformatted_data, size); } else { size = ff_av1_filter_obus(pb, pkt->data, pkt->size); + if (trk->mode == MODE_AVIF && !mov->avif_extent_length) { + mov->avif_extent_length = size; + } } + #if CONFIG_AC3_PARSER } else if (par->codec_id == AV_CODEC_ID_EAC3) { size = handle_eac3(mov, pkt, trk); @@ -6579,11 +6775,15 @@ static int mov_init(AVFormatContext *s) else if (IS_MODE(ipod, IPOD)) mov->mode = MODE_IPOD; else if (IS_MODE(ismv, ISMV)) mov->mode = MODE_ISM; else if (IS_MODE(f4v, F4V)) mov->mode = MODE_F4V; + else if (IS_MODE(avif, AVIF)) mov->mode = MODE_AVIF; #undef IS_MODE if (mov->flags & FF_MOV_FLAG_DELAY_MOOV) mov->flags |= FF_MOV_FLAG_EMPTY_MOOV; + if (mov->mode == MODE_AVIF) + mov->flags |= FF_MOV_FLAG_DELAY_MOOV; + /* Set the FRAGMENT flag if any of the fragmentation methods are * enabled. */ if (mov->max_fragment_duration || mov->max_fragment_size || @@ -6664,11 +6864,25 @@ static int mov_init(AVFormatContext *s) /* Non-seekable output is ok if using fragmentation. If ism_lookahead * is enabled, we don't support non-seekable output at all. */ if (!(s->pb->seekable & AVIO_SEEKABLE_NORMAL) && - (!(mov->flags & FF_MOV_FLAG_FRAGMENT) || mov->ism_lookahead)) { + (!(mov->flags & FF_MOV_FLAG_FRAGMENT) || mov->ism_lookahead || + mov->mode == MODE_AVIF)) { av_log(s, AV_LOG_ERROR, "muxer does not support non seekable output\n"); return AVERROR(EINVAL); } + /* AVIF output must have exactly one video stream */ + if (mov->mode == MODE_AVIF) { + if (s->nb_streams > 1) { + av_log(s, AV_LOG_ERROR, "AVIF output requires exactly one stream\n"); + return AVERROR(EINVAL); + } + if (s->streams[0]->codecpar->codec_type != AVMEDIA_TYPE_VIDEO) { + av_log(s, AV_LOG_ERROR, "AVIF output requires one video stream\n"); + return AVERROR(EINVAL); + } + s->streams[0]->disposition |= AV_DISPOSITION_DEFAULT; + } + mov->nb_streams = s->nb_streams; if (mov->mode & (MODE_MP4|MODE_MOV|MODE_IPOD) && s->nb_chapters) mov->chapter_track = mov->nb_streams++; @@ -6811,12 +7025,13 @@ static int mov_init(AVFormatContext *s) pix_fmt == AV_PIX_FMT_MONOWHITE || pix_fmt == AV_PIX_FMT_MONOBLACK; } - if (track->par->codec_id == AV_CODEC_ID_VP9 || - track->par->codec_id == AV_CODEC_ID_AV1) { - if (track->mode != MODE_MP4) { - av_log(s, AV_LOG_ERROR, "%s only supported in MP4.\n", avcodec_get_name(track->par->codec_id)); - return AVERROR(EINVAL); - } + if (track->par->codec_id == AV_CODEC_ID_VP9 && track->mode != MODE_MP4) { + av_log(s, AV_LOG_ERROR, "%s only supported in MP4.\n", avcodec_get_name(track->par->codec_id)); + return AVERROR(EINVAL); + } else if (track->par->codec_id == AV_CODEC_ID_AV1 && + track->mode != MODE_MP4 && track->mode != MODE_AVIF) { + av_log(s, AV_LOG_ERROR, "%s only supported in MP4 and AVIF.\n", avcodec_get_name(track->par->codec_id)); + return AVERROR(EINVAL); } else if (track->par->codec_id == AV_CODEC_ID_VP8) { /* altref frames handling is not defined in the spec as of version v1.0, * so just forbid muxing VP8 streams altogether until a new version does */ @@ -7034,7 +7249,7 @@ static int mov_write_header(AVFormatContext *s) FF_MOV_FLAG_FRAG_EVERY_FRAME)) && !mov->max_fragment_duration && !mov->max_fragment_size) mov->flags |= FF_MOV_FLAG_FRAG_KEYFRAME; - } else { + } else if (mov->mode != MODE_AVIF) { if (mov->flags & FF_MOV_FLAG_FASTSTART) mov->reserved_header_pos = avio_tell(pb); mov_write_mdat_tag(pb, mov); @@ -7322,6 +7537,50 @@ static int mov_check_bitstream(AVFormatContext *s, AVStream *st, return ret; } +static int avif_write_trailer(AVFormatContext *s) +{ + AVIOContext *pb = s->pb; + MOVMuxContext *mov = s->priv_data; + int64_t pos_backup, mdat_pos; + uint8_t *buf; + int buf_size, moov_size; + + if (mov->moov_written) return 0; + + mov->is_animated_avif = s->streams[0]->nb_frames > 1; + mov_write_identification(pb, s); + mov_write_meta_tag(pb, mov, s); + + moov_size = get_moov_size(s); + mov->tracks[0].data_offset = avio_tell(pb) + moov_size + 8; + + if (mov->is_animated_avif) { + int ret; + if ((ret = mov_write_moov_tag(pb, mov, s)) < 0) + return ret; + } + + buf_size = avio_get_dyn_buf(mov->mdat_buf, &buf); + avio_wb32(pb, buf_size + 8); + ffio_wfourcc(pb, "mdat"); + mdat_pos = avio_tell(pb); + + if (mdat_pos != (uint32_t)mdat_pos) { + av_log(s, AV_LOG_ERROR, "mdat offset does not fit in 32 bits\n"); + return AVERROR_INVALIDDATA; + } + + avio_write(pb, buf, buf_size); + + // write extent offset. + pos_backup = avio_tell(pb); + avio_seek(pb, mov->avif_extent_pos, SEEK_SET); + avio_wb32(pb, mdat_pos); /* rewrite offset */ + avio_seek(pb, pos_backup, SEEK_SET); + + return 0; +} + #if CONFIG_TGP_MUXER || CONFIG_TG2_MUXER static const AVCodecTag codec_3gp_tags[] = { { AV_CODEC_ID_H263, MKTAG('s','2','6','3') }, @@ -7404,6 +7663,20 @@ static const AVCodecTag codec_f4v_tags[] = { { AV_CODEC_ID_NONE, 0 }, }; +#if CONFIG_AVIF_MUXER +static const AVCodecTag codec_avif_tags[] = { + { AV_CODEC_ID_AV1, MKTAG('a','v','0','1') }, + { AV_CODEC_ID_NONE, 0 }, +}; +static const AVCodecTag *const codec_avif_tags_list[] = { codec_avif_tags, NULL }; + +static const AVClass mov_avif_muxer_class = { + .class_name = "avif muxer", + .item_name = av_default_item_name, + .version = LIBAVUTIL_VERSION_INT, +}; +#endif + #if CONFIG_MOV_MUXER const AVOutputFormat ff_mov_muxer = { .name = "mov", @@ -7566,3 +7839,21 @@ const AVOutputFormat ff_f4v_muxer = { .priv_class = &mov_isobmff_muxer_class, }; #endif +#if CONFIG_AVIF_MUXER +const AVOutputFormat ff_avif_muxer = { + .name = "avif", + .long_name = NULL_IF_CONFIG_SMALL("AVIF"), + .mime_type = "image/avif", + .extensions = "avif", + .priv_data_size = sizeof(MOVMuxContext), + .video_codec = AV_CODEC_ID_AV1, + .init = mov_init, + .write_header = mov_write_header, + .write_packet = mov_write_packet, + .write_trailer = avif_write_trailer, + .deinit = mov_free, + .flags = AVFMT_GLOBALHEADER | AVFMT_ALLOW_FLUSH, + .codec_tag = codec_avif_tags_list, + .priv_class = &mov_avif_muxer_class, +}; +#endif diff --git a/libavformat/movenc.h b/libavformat/movenc.h index ca507e0e04..281576cc66 100644 --- a/libavformat/movenc.h +++ b/libavformat/movenc.h @@ -43,6 +43,7 @@ #define MODE_IPOD 0x20 #define MODE_ISM 0x40 #define MODE_F4V 0x80 +#define MODE_AVIF 0x100 typedef struct MOVIentry { uint64_t pos; @@ -244,6 +245,10 @@ typedef struct MOVMuxContext { MOVPrftBox write_prft; int empty_hdlr_name; int movie_timescale; + + int64_t avif_extent_pos; + int avif_extent_length; + int is_animated_avif; } MOVMuxContext; #define FF_MOV_FLAG_RTP_HINT (1 << 0)