From patchwork Thu Feb 17 05:51:15 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vignesh Venkat X-Patchwork-Id: 34349 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6838:d078:0:0:0:0 with SMTP id x24csp352471nkx; Wed, 16 Feb 2022 21:51:36 -0800 (PST) X-Google-Smtp-Source: ABdhPJwvh24MymvcIn8S9G40GKL/KHnxnD7VRBD9EmOS9t7EaAmEp7lSP9fNxVcgbBtfTwN5tWZc X-Received: by 2002:a05:6402:d6e:b0:410:8169:46de with SMTP id ec46-20020a0564020d6e00b00410816946demr1020252edb.328.1645077096638; Wed, 16 Feb 2022 21:51:36 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1645077096; cv=none; d=google.com; s=arc-20160816; b=CDDWh3QPX1QRXlrnS/VWsgIAM60lnUOg7+UcI0hg9QPecK8Ic/s7/8RN31vG4GRiF9 wXXZJ5RaswGcg1JSz0MRtSLBhPTwxRG+riH6tVIMt3+k7EzKcZ9pKH7xoUv4+6fECr1w wT+ATGt+F/1qTMC5NHWEEpl2guwPH+xPxIthSLd01hoPIdJzA9NE/24mOEQX3uiMSSbs nFlguHTKVZMRyG2yFesr1nu98VedaoaT6HMBSdNKqmmVQAOH16tv9XObZsDRMAMh4hda FZRW767Le9nvz+CIf08p2HKB97pHYo+xDcFgJM9WM8X9pCtf4TkrA8KeKsZNJ0aD+YDN +njQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:to:from:mime-version:message-id:date :dkim-signature:delivered-to; bh=pi4x5EGmtt8a1ibniT14whIg6iFv4HMNTU5lKPE0OGM=; b=h0c9qFMODYl8CjykWRAjW6LPd6ECXxXIqj9mC31BJVY3aUEyJLoHopWDdepX4M9o7z zw7mkXnzT6xGaTYLaB8yB8swH40CTY2DmIIUukzu+W5Mumx0xo2FGILyMiBR73ID2AIN xEvbF14MT56RXKIsEzfSMACLJ0bZx5scuzOcPgmW6Ide2roUacz7O7cL+TBC4DRDocVz KH6uVyvSOM1CPwr+T3ILbSzDHip7izsxhQoryNFA04D8UrzQxChOVWCOn1UougHK71n7 qdVJefDjLOfGYOzyMt474marwLt+iRkXPtHzmoBOsRYgAYS3WcaJlqz3/THgnx+jQk2s reXA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@google.com header.s=20210112 header.b=osvzdGt5; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id nd3si1567903ejc.49.2022.02.16.21.51.35; Wed, 16 Feb 2022 21:51:36 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@google.com header.s=20210112 header.b=osvzdGt5; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 6828068B1E0; Thu, 17 Feb 2022 07:51:31 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-yb1-f201.google.com (mail-yb1-f201.google.com [209.85.219.201]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 36B8E68AEB1 for ; Thu, 17 Feb 2022 07:51:24 +0200 (EET) Received: by mail-yb1-f201.google.com with SMTP id i4-20020a25a0c4000000b00623ab738437so6326696ybm.0 for ; Wed, 16 Feb 2022 21:51:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:message-id:mime-version:subject:from:to:cc; bh=jY8aC5fIgBpYhbh86WoX7XQz/lswekl54M4wR7waLfk=; b=osvzdGt5OyX+hXxpKuF9M3ej3yxIvDZGHX22pbW7mGVtYFZleCBjUktTgPTvrXOzIe Ywe+IUfoOc46+Vg7zGI3TYFSdjp1ffA21AZ2KPn7gZuEe6MAd3zIUJaEX9TutqM9zHnF 1CH2MiGjrpYBXbw4969vWKb2gEldhf9eEVRUj9l/N8g7t7653QlZP3kdjLQpi8j1Y+wA NyyXU/wb/U5bAMSXUhBOVzm0QKdZ04N+PiNV+uLFEryieSOQTIMXgVfxW0tphF8qwPWj UutiwPjGRvNv90tBFSwQLP5QOml/XyLjDsAjHH1SPhx/xUTAXQ8agJAUzOihPSW5Fc7z w/1w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:message-id:mime-version:subject:from:to:cc; bh=jY8aC5fIgBpYhbh86WoX7XQz/lswekl54M4wR7waLfk=; b=KQk4axMJuDVICgY/5AGtgK/Esd/Jg83gT/xS5PxVcPof2+zaQLIimJBe8EmK62/bET AMX46D0VJgV9TsVrZGaEDJSU/If+mm+Y6a7y/hLzbBi7sTbZdlwkVPAVpeYDdToF9e/G /e8TJQ33RUq29czpnRI/6sXjBtnnnNUEgqTRhLRwwUVrYFy6YZ1/XlYrvUt7YvIx5gTI I9axBRdd4aVLeJG6oduRgdA2vUEiUfSBDQpcPbHfh0KGpa+xxtD+O2xUhE7GVzXOhs3n yPtpnXYAoeUyB6o9ZdJ/TJVu1jzqEqcBNqtPc3RvUUwJT2Gid72LuGlAbG3dF+91/2qc lmFw== X-Gm-Message-State: AOAM532yYWK8fZA6iZvv/qODksUKuVOnKcb/HAQMeuuY+ZeLUYo+gjyt WBJ2pu08mkmvYV7S7+sqN4yoK1wdfAqM7WKVtoKx8luHC422YgM2qVrXGgLzZEW7voI1/AVF1Ve BC+M6f8Rett9ApyvUXgMgBUEBkYcZoFk7vVYU3qzhwN9PWyWiUCPivT35r4qwIzP7elhc X-Received: from vigneshv3.mtv.corp.google.com ([2620:0:1000:2511:4804:9763:5ad6:9d1f]) (user=vigneshv job=sendgmr) by 2002:a81:644:0:b0:2d0:d34c:9170 with SMTP id 65-20020a810644000000b002d0d34c9170mr1178351ywg.204.1645077082250; Wed, 16 Feb 2022 21:51:22 -0800 (PST) Date: Wed, 16 Feb 2022 21:51:15 -0800 Message-Id: <20220217055117.3233501-1-vigneshv@google.com> Mime-Version: 1.0 X-Mailer: git-send-email 2.35.1.265.g69c8d7142f-goog From: Vignesh Venkatasubramanian To: ffmpeg-devel@ffmpeg.org Subject: [FFmpeg-devel] [PATCH 1/3] avcodec/libaomenc: Add parameter for avif single image encoding X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Vignesh Venkatasubramanian Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 6jsrGFH2oTGs Add a parameter to libaom-av1 encoder to enforce some of the single image constraints in the AV1 encoder. Setting this flag will limit the encoder to producing exactly one frame and the sequence header that is produced by the encoder will be conformant to the AVIF specification [1]. Part of Fixing Trac ticket #7621 [1] https://aomediacodec.github.io/av1-avif Signed-off-by:: Vignesh Venkatasubramanian --- libavcodec/libaomenc.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/libavcodec/libaomenc.c b/libavcodec/libaomenc.c index 963cc1bcbc..0398060a2f 100644 --- a/libavcodec/libaomenc.c +++ b/libavcodec/libaomenc.c @@ -99,6 +99,7 @@ typedef struct AOMEncoderContext { int enable_restoration; int usage; int tune; + int is_avif; int enable_rect_partitions; int enable_1to4_partitions; int enable_ab_partitions; @@ -746,6 +747,18 @@ static av_cold int aom_init(AVCodecContext *avctx, if (res < 0) return res; + if (ctx->is_avif) { + // Set the maximum number of frames to 1. This will let libaom set + // still_picture and reduced_still_picture_header to 1 in the Sequence + // Header as required by AVIF still images. + enccfg.g_limit = 1; + // Reduce memory usage for still images. + enccfg.g_lag_in_frames = 0; + // All frames will be key frames. + enccfg.kf_max_dist = 0; + enccfg.kf_mode = AOM_KF_DISABLED; + } + /* Construct Encoder Context */ res = aom_codec_enc_init(&ctx->encoder, iface, &enccfg, flags); if (res != AOM_CODEC_OK) { @@ -1290,6 +1303,7 @@ static const AVOption options[] = { { "psnr", NULL, 0, AV_OPT_TYPE_CONST, {.i64 = AOM_TUNE_PSNR}, 0, 0, VE, "tune"}, { "ssim", NULL, 0, AV_OPT_TYPE_CONST, {.i64 = AOM_TUNE_SSIM}, 0, 0, VE, "tune"}, FF_AV1_PROFILE_OPTS + { "avif-image", "Encode in single frame mode for still AVIF images.", OFFSET(is_avif), AV_OPT_TYPE_BOOL, {.i64 = -1}, -1, 1, VE }, { "enable-rect-partitions", "Enable rectangular partitions", OFFSET(enable_rect_partitions), AV_OPT_TYPE_BOOL, {.i64 = -1}, -1, 1, VE}, { "enable-1to4-partitions", "Enable 1:4/4:1 partitions", OFFSET(enable_1to4_partitions), AV_OPT_TYPE_BOOL, {.i64 = -1}, -1, 1, VE}, { "enable-ab-partitions", "Enable ab shape partitions", OFFSET(enable_ab_partitions), AV_OPT_TYPE_BOOL, {.i64 = -1}, -1, 1, VE}, From patchwork Thu Feb 17 05:51:16 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vignesh Venkat X-Patchwork-Id: 34350 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6838:d078:0:0:0:0 with SMTP id x24csp352725nkx; Wed, 16 Feb 2022 21:52:11 -0800 (PST) X-Google-Smtp-Source: ABdhPJxMvQyR4qed8GEMAIm9dgxCSPJf4CglEZjUphxpwo27gROsD4Kh7LkbXHKDZK8Wv6bsdEMP X-Received: by 2002:a17:906:6c8f:b0:6cf:95c:5d78 with SMTP id s15-20020a1709066c8f00b006cf095c5d78mr1114199ejr.74.1645077130906; Wed, 16 Feb 2022 21:52:10 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1645077130; cv=none; d=google.com; s=arc-20160816; b=dnv/2zIlAChUv1t49gUAdzkodiUnK5yHhufq6xiFSKNUGH3zcHKoolMpFXPoauW2yW wQfXyyatwBjhYo6/FPVS2w20u+S8sUxemW3vh6Zsnwoy3wXY1snu4kize2JD68rX3Rg4 EFxUya+v29nnxdj06D4BNqeJFoy97JVU/X5eWsK+tHvZyDr3fYbyX8+HN/7QHB2lFmZt K/wF+zGlwi10Ob94FKFGrQSI5zS3kenkGelPsBFGBGvunsujZ1VFnv12nmBmDNUWgL0T QVjRmXYQLx1RQ5cYzIiBPV+NnXWvAt3+ycPbqsgFy9LlpeUWuX6dY9kKjFvd58rp8FQ9 nctg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:to:from:references:mime-version :message-id:in-reply-to:date:dkim-signature:delivered-to; bh=DiBUl03j8snRqoFmYXlE2wRMavK4KiscBsg9TdQPv7E=; b=ZiXAXvD7rMbLqmi8+O2e5tuukzU/NVl2djLD2l3LRH6IFNqRv2UgBaBsQjWuYs1O8t QddxLdmkAQmEM0//fN0V+rWk8Q2/6A6C8OZ8f4kEsyRZA/VcUpDArbp+rn3tuVWu+Xh8 1tmgBC6m2obvNZjy6ewywLRC3ypvKYWNhsoSwMwLrDUWqD6fA5oSWUrFBFioOSJSOUIA GszqUxLx0nLmxNdmeckaQsyuQsZpjOY0zwtwDOcEeQ6KjFldb4aLyNCkwBOfJuQ9ilF9 FZ4tDaG4hBZbyCd8hA+smFzk2IPXxywXMmMCESDYrPV11yFys2SJVshaGzepK5zTs9an pjAw== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@google.com header.s=20210112 header.b=WAvTDv1t; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id d21si1102351ejc.473.2022.02.16.21.51.47; Wed, 16 Feb 2022 21:52:10 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@google.com header.s=20210112 header.b=WAvTDv1t; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 914FB68B26D; Thu, 17 Feb 2022 07:51:40 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-yb1-f201.google.com (mail-yb1-f201.google.com [209.85.219.201]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id BE51368B15E for ; Thu, 17 Feb 2022 07:51:34 +0200 (EET) Received: by mail-yb1-f201.google.com with SMTP id m10-20020a25800a000000b0061daa5b7151so8795243ybk.10 for ; Wed, 16 Feb 2022 21:51:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=a77bd7zC6/6SkEBOItFsKS/zPUsCnRt9cpCmZ3BTAhM=; b=WAvTDv1tQqemcK8XQm8lsH94qA9hgqC9YNO1EKFpEmRu7Ayh5M7eMdCwDiKdQCUB2q ZXXD2AocQdyPaxL8UVfAl+3J/ZENG4x/bzyGPqA/5Rv3wKYg3pAoluTAhQlSaXJTwVFo okb3ya6Uuca+ytKkYnt4D8GCfig6StVSuWC4gyTpZ5APDAu4GtaegQ6ivG39f6LrfcjA AbkyBUZin2lh8ZuF9KyOl4ABg/Aujip1xlySqpyblkQ8tFlrFgAqiiLJ5EP71xzwvB1e V2lBTHsIcNsBl/PGbP8tVBK3wKgWY39zIi5CEn76wGo0rrQ9q2mA5bKCMceng7Q3G3qx jwkQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=a77bd7zC6/6SkEBOItFsKS/zPUsCnRt9cpCmZ3BTAhM=; b=wmJN9opCWEMS0CBKOcipIqOyXTMHD9R/7rw6F1H1jXkLRqLtJ5RDpMbF+tTbgfb2Ls +UY4v/ViM5PpQ8zO/DJrMFDUj+XeQ5zsf/MozK04M4VviNf/my35o+pH/u6dF32Hfn+H V9NErEVk0D7yYwTbz5aPXI+2HjPvY3ztfGrOmVMbxsGlErxOeUoSiaCuDsyR3kalRHe8 +amx6CaHB4j8gylFC+lzjkRXiJBSzWaM/nGlJ8TMzLtSn8FadLcVuFZY0uL2MhyWCxGt mUfZQ/BVuW+KRtT98VKYWtaFsjHMtMmvHDTf852T195hDDSxjvoUuQY2LnNZ+PX9gosR /s3A== X-Gm-Message-State: AOAM531GJqubgnwsdtS3YrtfK/y9QHuLAkt/KZiC4GBJiZL9MYDubxWv T5duGid8Es3w+qGiN9wwTisATUazWEUiDmUuJQ1/bPVmrNDa3jE6Jd+25AJ1ZVPnbQIz5A/hE38 3NFb47/dfFh4Wz6hplbC3dSk16SZvlxnu36d0z1V5vbJLqAfnZvRXQDBSKA0ykMw+I16d X-Received: from vigneshv3.mtv.corp.google.com ([2620:0:1000:2511:4804:9763:5ad6:9d1f]) (user=vigneshv job=sendgmr) by 2002:a81:bb51:0:b0:2d6:997b:131c with SMTP id a17-20020a81bb51000000b002d6997b131cmr772334ywl.64.1645077093583; Wed, 16 Feb 2022 21:51:33 -0800 (PST) Date: Wed, 16 Feb 2022 21:51:16 -0800 In-Reply-To: <20220217055117.3233501-1-vigneshv@google.com> Message-Id: <20220217055117.3233501-2-vigneshv@google.com> Mime-Version: 1.0 References: <20220217055117.3233501-1-vigneshv@google.com> X-Mailer: git-send-email 2.35.1.265.g69c8d7142f-goog From: Vignesh Venkatasubramanian To: ffmpeg-devel@ffmpeg.org Subject: [FFmpeg-devel] [PATCH 2/3] avformat/av1: Add a parameter to av1c to omit seq header X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Vignesh Venkatasubramanian Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 4g8OQiuCng23 Add a parameter to omit seq header when generating the av1C atom. For now, this does not change any behavior. This will be used by a follow-up patch to add AVIF support. Signed-off-by: Vignesh Venkatasubramanian --- libavformat/av1.c | 7 +++++-- libavformat/av1.h | 4 +++- libavformat/matroskaenc.c | 4 ++-- libavformat/movenc.c | 2 +- 4 files changed, 11 insertions(+), 6 deletions(-) diff --git a/libavformat/av1.c b/libavformat/av1.c index 1fcfac2356..95ca7cc47f 100644 --- a/libavformat/av1.c +++ b/libavformat/av1.c @@ -361,7 +361,8 @@ int ff_av1_parse_seq_header(AV1SequenceParameters *seq, const uint8_t *buf, int return AVERROR_INVALIDDATA; } -int ff_isom_write_av1c(AVIOContext *pb, const uint8_t *buf, int size) +int ff_isom_write_av1c(AVIOContext *pb, const uint8_t *buf, int size, + int write_seq_header) { AVIOContext *meta_pb; AV1SequenceParameters seq_params; @@ -451,7 +452,9 @@ int ff_isom_write_av1c(AVIOContext *pb, const uint8_t *buf, int size) flush_put_bits(&pbc); avio_write(pb, header, sizeof(header)); - avio_write(pb, seq, seq_size); + if (write_seq_header) { + avio_write(pb, seq, seq_size); + } meta_size = avio_get_dyn_buf(meta_pb, &meta); if (meta_size) diff --git a/libavformat/av1.h b/libavformat/av1.h index f57dabe986..a393fbb78f 100644 --- a/libavformat/av1.h +++ b/libavformat/av1.h @@ -96,9 +96,11 @@ int ff_av1_parse_seq_header(AV1SequenceParameters *seq, const uint8_t *buf, int * @param pb pointer to the AVIOContext where the av1C box shall be written * @param buf input data buffer * @param size size in bytes of the input data buffer + * @param write_seq_header If 1, Sequence Header OBU will be written inside the + * av1C box. Otherwise, Sequence Header OBU will be omitted. * * @return >= 0 in case of success, a negative AVERROR code in case of failure */ -int ff_isom_write_av1c(AVIOContext *pb, const uint8_t *buf, int size); +int ff_isom_write_av1c(AVIOContext *pb, const uint8_t *buf, int size, int write_seq_header); #endif /* AVFORMAT_AV1_H */ diff --git a/libavformat/matroskaenc.c b/libavformat/matroskaenc.c index 38d9485288..5061961283 100644 --- a/libavformat/matroskaenc.c +++ b/libavformat/matroskaenc.c @@ -1087,7 +1087,7 @@ static int mkv_write_native_codecprivate(AVFormatContext *s, AVIOContext *pb, case AV_CODEC_ID_AV1: if (par->extradata_size) return ff_isom_write_av1c(dyn_cp, par->extradata, - par->extradata_size); + par->extradata_size, 1); else put_ebml_void(pb, 4 + 3); break; @@ -2663,7 +2663,7 @@ static int mkv_check_new_extra_data(AVFormatContext *s, const AVPacket *pkt) ret = avio_open_dyn_buf(&dyn_cp); if (ret < 0) return ret; - ff_isom_write_av1c(dyn_cp, side_data, side_data_size); + ff_isom_write_av1c(dyn_cp, side_data, side_data_size, 1); codecpriv_size = avio_get_dyn_buf(dyn_cp, &codecpriv); if ((ret = dyn_cp->error) < 0 || !codecpriv_size && (ret = AVERROR_INVALIDDATA)) { diff --git a/libavformat/movenc.c b/libavformat/movenc.c index 4c868919ae..1a746a67fd 100644 --- a/libavformat/movenc.c +++ b/libavformat/movenc.c @@ -1303,7 +1303,7 @@ static int mov_write_av1c_tag(AVIOContext *pb, MOVTrack *track) avio_wb32(pb, 0); ffio_wfourcc(pb, "av1C"); - ff_isom_write_av1c(pb, track->vos_data, track->vos_len); + ff_isom_write_av1c(pb, track->vos_data, track->vos_len, 1); return update_size(pb, pos); } From patchwork Thu Feb 17 05:51:17 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vignesh Venkat X-Patchwork-Id: 34351 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6838:d078:0:0:0:0 with SMTP id x24csp352731nkx; Wed, 16 Feb 2022 21:52:11 -0800 (PST) X-Google-Smtp-Source: ABdhPJzXEkkBE3U548Khog6TwZQgFpZS4lWkGDrEHlEVao5fsudB++RtxciFKU46Cy1ffflF3mE2 X-Received: by 2002:a17:906:3905:b0:6cf:7ef5:fee0 with SMTP id f5-20020a170906390500b006cf7ef5fee0mr1065315eje.307.1645077131245; Wed, 16 Feb 2022 21:52:11 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1645077131; cv=none; d=google.com; s=arc-20160816; b=fetzI3vsTaR7vnbBkFfh+OFxWhZfd7jbzoDFKeg47eefIvBPojbo5MDFLXXjMNm9bH ZpxiuXOhR4Ik6zpG6GpjUnPE/zP+XT3oNqEYZuq11K40dmsj0nUiuxGPo0zf9yUNSevH AEUaJS/kPf7xnUwtD+RBubIbjvox+gq5rERL3fTcWsiplsxu7D1eqWp2+KiCtFgYrSz5 39aCmo3zqpDRq1gnERkJtm0kH+hTzVs2cmNgpJAoIQnnBzxvLlcB4r52mqr/AuirZ+dU saWKtvEpgxzTyTixaKNQJKPy2gt6iGlxmyINB7T1Nq2RSdgSEd1Jd7kAUDB2fmQG432r m/vA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:to:from:references:mime-version :message-id:in-reply-to:date:dkim-signature:delivered-to; bh=lxqXqDI1Xf7MBmpnNVttFtukDjn3yL+/70YMqAnFmyo=; b=KvqA6jQkBjuWsdAR/Ina+dY1gS1KAAFIoSRig5/XohgCclpywPUIUrfjW+fQfXSpID c0flaPBntALq9cGIWL6bm4k+fEal9m4eeSM2Ni/+TbTFFPfDC0T5ebx5mbouOi7cs1a7 YDMAiS4caxXRaWn0RvdcNgxt12kyImsFhwGnh5PcLkySjhtSno4FlpqQ6zj7TAjr06xy oVDmBASQ/AATgZB1erQQRVI0piMSobOxCAaNVrVc4iNfA0cASEzDPMc5ItoW6FVG8fAP wBAwtHTItr3255pBoQyO+WpSqXoVFY+zxIA9WIhl98BT0D8qr4pmWklLiI8I5AVwwtlp Ei4g== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@google.com header.s=20210112 header.b=HmAmIUG7; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id b14si3871908edz.641.2022.02.16.21.51.57; Wed, 16 Feb 2022 21:52:11 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@google.com header.s=20210112 header.b=HmAmIUG7; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id B91A368B28F; Thu, 17 Feb 2022 07:51:44 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-yb1-f202.google.com (mail-yb1-f202.google.com [209.85.219.202]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 282DB68B1F5 for ; Thu, 17 Feb 2022 07:51:38 +0200 (EET) Received: by mail-yb1-f202.google.com with SMTP id i205-20020a2522d6000000b00622c778ac7cso8915104ybi.3 for ; Wed, 16 Feb 2022 21:51:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=LYnPhQQZeu/oYqdzSArTTnU7+Kn5hGSL2MOmpHrocLA=; b=HmAmIUG7r3xGr08LWqtVCTMdrca27rfG3FEJKM34HhSZ9nN52WF/e8cYGOrn7jBOFE geVNFZw7XdJMXngEIPnwl0eX2uChw5cTW44ibwv0GzfZa0+1nHRpchccAsJUovzm5KpF uA+RiPTPVHGx+PkZlp3FUgS9MdJE6U/7huVGAbE8UlfXee7WTj4XocsrjvTonHq6Tm12 CLjpUsMyDHXZoDVOm2XYTXKlmDdMarvLVHDZZZiKm2sFsoJImRm80ij3ShiGT6/OuYAP 8QED2pCelvmfOZt93qwDePCOqSw5JsCUz26cswlQHV8229lIcVk8p8UkBi16xykeZLhG U/SQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=LYnPhQQZeu/oYqdzSArTTnU7+Kn5hGSL2MOmpHrocLA=; b=Zr0FrwIKb6ECzPd806dnqIskLc4XtyxlleOOHLklvDl2SQP3pFUbuLkEWWxxy7HEpz BQi6Q4fmfyHfYywIrhySkWi7AFWpnhx8LB0ny2PAUcEMKepzFfN1Ob9ssqEJc6b9NJrb 0fzMk2TZD4A0bGd8tWKLvaLN3I1Z61RPAdxTiclsUYhz5vBzHSMFTlpQwPk2RzV9nk4H kopXmpDMSN80/L8YPiIMuUIPXAm8q5p/neOYqeFiDCydjHcntV8Vnl0Bu9wiibxfcnnZ 58BuJFXTaVslU+Frv9msMTan3nE4delbnU+zEdX0eLB1NOmRSzBIpCgAF5hW1Nv53Ma/ 7RqA== X-Gm-Message-State: AOAM5331OrjT2fnsWxIZIjOHPynRU+x2ndMI8aQQtUOczK0hJT89WlBX j/p9aK42JmTElURu7WZwrjiu/lcZaB+ZI//J2EReGze1vDXkPUwzBFJL9Z27STF0lWy1FznEt/V E3z+ByhLYziw1mx7h4613P4EuFlQZC8DWHZ8+NnawuwdhtKfiqM4xEOvF5tDouS+dSM+I X-Received: from vigneshv3.mtv.corp.google.com ([2620:0:1000:2511:4804:9763:5ad6:9d1f]) (user=vigneshv job=sendgmr) by 2002:a25:3d87:0:b0:61e:170c:aa9 with SMTP id k129-20020a253d87000000b0061e170c0aa9mr1149755yba.89.1645077096819; Wed, 16 Feb 2022 21:51:36 -0800 (PST) Date: Wed, 16 Feb 2022 21:51:17 -0800 In-Reply-To: <20220217055117.3233501-1-vigneshv@google.com> Message-Id: <20220217055117.3233501-3-vigneshv@google.com> Mime-Version: 1.0 References: <20220217055117.3233501-1-vigneshv@google.com> X-Mailer: git-send-email 2.35.1.265.g69c8d7142f-goog From: Vignesh Venkatasubramanian To: ffmpeg-devel@ffmpeg.org Subject: [FFmpeg-devel] [PATCH 3/3] avformat/movenc: Add support for AVIF muxing X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Vignesh Venkatasubramanian Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: qd53B/Zw8EGu Add an AVIF muxer by re-using the existing the mov/mp4 muxer. AVIF Specifiation: https://aomediacodec.github.io/av1-avif Sample usage for still image: ffmpeg -i image.png -c:v libaom-av1 -avif-image 1 image.avif Sample usage for animated AVIF image: ffmpeg -i video.mp4 animated.avif We can re-use any of the AV1 encoding options that will make sense for image encoding (like bitrate, tiles, encoding speed, etc). The files generated by this muxer has been verified to be valid AVIF files by the following: 1) Displays on Chrome (both still and animated images). 2) Displays on Firefox (only still images, firefox does not support animated AVIF yet). 3) Verfied to be valid by Compliance Warden: https://github.com/gpac/ComplianceWarden Fixes the encoder/muxer part of Trac Ticket #7621 Signed-off-by: Vignesh Venkatasubramanian --- configure | 1 + libavformat/allformats.c | 1 + libavformat/movenc.c | 300 +++++++++++++++++++++++++++++++++++---- libavformat/movenc.h | 5 + 4 files changed, 283 insertions(+), 24 deletions(-) diff --git a/configure b/configure index 1535dc3c5b..87b380fe3a 100755 --- a/configure +++ b/configure @@ -3393,6 +3393,7 @@ asf_stream_muxer_select="asf_muxer" av1_demuxer_select="av1_frame_merge_bsf av1_parser" avi_demuxer_select="riffdec exif" avi_muxer_select="riffenc" +avif_muxer_select="mov_muxer" caf_demuxer_select="iso_media" caf_muxer_select="iso_media" dash_muxer_select="mp4_muxer" diff --git a/libavformat/allformats.c b/libavformat/allformats.c index d066a7745b..400c17afbd 100644 --- a/libavformat/allformats.c +++ b/libavformat/allformats.c @@ -81,6 +81,7 @@ extern const AVOutputFormat ff_au_muxer; extern const AVInputFormat ff_av1_demuxer; extern const AVInputFormat ff_avi_demuxer; extern const AVOutputFormat ff_avi_muxer; +extern const AVOutputFormat ff_avif_muxer; extern const AVInputFormat ff_avisynth_demuxer; extern const AVOutputFormat ff_avm2_muxer; extern const AVInputFormat ff_avr_demuxer; diff --git a/libavformat/movenc.c b/libavformat/movenc.c index 1a746a67fd..69b5f4bc76 100644 --- a/libavformat/movenc.c +++ b/libavformat/movenc.c @@ -1303,7 +1303,7 @@ static int mov_write_av1c_tag(AVIOContext *pb, MOVTrack *track) avio_wb32(pb, 0); ffio_wfourcc(pb, "av1C"); - ff_isom_write_av1c(pb, track->vos_data, track->vos_len, 1); + ff_isom_write_av1c(pb, track->vos_data, track->vos_len, track->mode != MODE_AVIF); return update_size(pb, pos); } @@ -2004,12 +2004,13 @@ static int mov_write_colr_tag(AVIOContext *pb, MOVTrack *track, int prefer_icc) } } - /* We should only ever be called by MOV or MP4. */ - av_assert0(track->mode == MODE_MOV || track->mode == MODE_MP4); + /* We should only ever be called for MOV, MP4 and AVIF. */ + av_assert0(track->mode == MODE_MOV || track->mode == MODE_MP4 || + track->mode == MODE_AVIF); avio_wb32(pb, 0); /* size */ ffio_wfourcc(pb, "colr"); - if (track->mode == MODE_MP4) + if (track->mode == MODE_MP4 || track->mode == MODE_AVIF) ffio_wfourcc(pb, "nclx"); else ffio_wfourcc(pb, "nclc"); @@ -2019,7 +2020,7 @@ static int mov_write_colr_tag(AVIOContext *pb, MOVTrack *track, int prefer_icc) avio_wb16(pb, track->par->color_primaries); avio_wb16(pb, track->par->color_trc); avio_wb16(pb, track->par->color_space); - if (track->mode == MODE_MP4) { + if (track->mode == MODE_MP4 || track->mode == MODE_AVIF) { int full_range = track->par->color_range == AVCOL_RANGE_JPEG; avio_w8(pb, full_range << 7); } @@ -2103,6 +2104,8 @@ static void find_compressor(char * compressor_name, int len, MOVTrack *track) av_strlcatf(compressor_name, len, " %d%c", track->par->height, interlaced ? 'i' : 'p'); av_strlcatf(compressor_name, len, "%d", rate * (interlaced + 1)); + } else if (track->par->codec_id == AV_CODEC_ID_AV1) { + av_strlcatf(compressor_name, len, "libaom Encoder"); } } @@ -2123,6 +2126,8 @@ static int mov_write_video_tag(AVFormatContext *s, AVIOContext *pb, MOVMuxContex avio_wb32(pb, 0); /* size */ if (mov->encryption_scheme != MOV_ENC_NONE) { ffio_wfourcc(pb, "encv"); + } else if (track->mode == MODE_AVIF) { + ffio_wfourcc(pb, "av01"); } else { avio_wl32(pb, track->tag); // store it byteswapped } @@ -2239,7 +2244,7 @@ static int mov_write_video_tag(AVFormatContext *s, AVIOContext *pb, MOVMuxContex else av_log(mov->fc, AV_LOG_WARNING, "Not writing 'gama' atom. Format is not MOV.\n"); } - if (track->mode == MODE_MOV || track->mode == MODE_MP4) { + if (track->mode == MODE_MOV || track->mode == MODE_MP4 || track->mode == MODE_AVIF) { int has_color_info = track->par->color_primaries != AVCOL_PRI_UNSPECIFIED && track->par->color_trc != AVCOL_TRC_UNSPECIFIED && track->par->color_space != AVCOL_SPC_UNSPECIFIED; @@ -2792,7 +2797,10 @@ static int mov_write_hdlr_tag(AVFormatContext *s, AVIOContext *pb, MOVTrack *tra if (track) { hdlr = (track->mode == MODE_MOV) ? "mhlr" : "\0\0\0\0"; - if (track->par->codec_type == AVMEDIA_TYPE_VIDEO) { + if (track->mode == MODE_AVIF) { + hdlr_type = "pict"; + descr = "ffmpeg"; + } else if (track->par->codec_type == AVMEDIA_TYPE_VIDEO) { hdlr_type = "vide"; descr = "VideoHandler"; } else if (track->par->codec_type == AVMEDIA_TYPE_AUDIO) { @@ -2859,6 +2867,131 @@ static int mov_write_hdlr_tag(AVFormatContext *s, AVIOContext *pb, MOVTrack *tra return update_size(pb, pos); } +static int mov_write_pitm_tag(AVIOContext *pb, int item_id) +{ + int64_t pos = avio_tell(pb); + avio_wb32(pb, 0); /* size */ + ffio_wfourcc(pb, "pitm"); + avio_wb32(pb, 0); /* Version & flags */ + avio_wb16(pb, item_id); /* item_id */ + return update_size(pb, pos); +} + +static int mov_write_iloc_tag(AVIOContext *pb, MOVMuxContext *mov, AVFormatContext *s) +{ + int64_t pos = avio_tell(pb); + avio_wb32(pb, 0); /* size */ + ffio_wfourcc(pb, "iloc"); + avio_wb32(pb, 0); /* Version & flags */ + avio_w8(pb, (4 << 4) + 4); /* offset_size(4) and length_size(4) */ + avio_w8(pb, 0); /* base_offset_size(4) and reserved(4) */ + avio_wb16(pb, 1); /* item_count */ + + avio_wb16(pb, 1); /* item_id */ + avio_wb16(pb, 0); /* data_reference_index */ + avio_wb16(pb, 1); /* extent_count */ + mov->avif_extent_pos = avio_tell(pb); + avio_wb32(pb, 0); /* extent_offset (written later) */ + // For animated AVIF, we simply write the first packet's size. + avio_wb32(pb, mov->avif_extent_length); /* extent_length */ + + return update_size(pb, pos); +} + +static int mov_write_iinf_tag(AVIOContext *pb, MOVMuxContext *mov, AVFormatContext *s) +{ + int64_t infe_pos; + int64_t iinf_pos = avio_tell(pb); + avio_wb32(pb, 0); /* size */ + ffio_wfourcc(pb, "iinf"); + avio_wb32(pb, 0); /* Version & flags */ + avio_wb16(pb, 1); /* entry_count */ + + infe_pos = avio_tell(pb); + avio_wb32(pb, 0); /* size */ + ffio_wfourcc(pb, "infe"); + avio_w8(pb, 0x2); /* Version */ + avio_wb24(pb, 0); /* flags */ + avio_wb16(pb, 1); /* item_id */ + avio_wb16(pb, 0); /* item_protection_index */ + avio_write(pb, "av01", 4); /* item_type */ + avio_write(pb, "Color\0", 6); /* item_name */ + update_size(pb, infe_pos); + + return update_size(pb, iinf_pos); +} + +static int mov_write_ispe_tag(AVIOContext *pb, MOVMuxContext *mov, AVFormatContext *s) +{ + int64_t pos = avio_tell(pb); + avio_wb32(pb, 0); /* size */ + ffio_wfourcc(pb, "ispe"); + avio_wb32(pb, 0); /* Version & flags */ + avio_wb32(pb, s->streams[0]->codecpar->width); /* image_width */ + avio_wb32(pb, s->streams[0]->codecpar->height); /* image_height */ + return update_size(pb, pos); +} + + +static int mov_write_pixi_tag(AVIOContext *pb, MOVMuxContext *mov, AVFormatContext *s) +{ + int64_t pos = avio_tell(pb); + int num_channels = av_pix_fmt_count_planes(s->streams[0]->codecpar->format); + const AVPixFmtDescriptor *pixdesc = av_pix_fmt_desc_get(s->streams[0]->codecpar->format); + int i; + avio_wb32(pb, 0); /* size */ + ffio_wfourcc(pb, "pixi"); + avio_wb32(pb, 0); /* Version & flags */ + avio_w8(pb, num_channels); /* num_channels */ + for (i = 0; i < num_channels; ++i) { + avio_w8(pb, pixdesc->comp[i].depth); /* bits_per_channel */ + } + return update_size(pb, pos); +} + +static int mov_write_ipco_tag(AVIOContext *pb, MOVMuxContext *mov, AVFormatContext *s) +{ + int64_t pos = avio_tell(pb); + avio_wb32(pb, 0); /* size */ + ffio_wfourcc(pb, "ipco"); + mov_write_ispe_tag(pb, mov, s); + mov_write_pixi_tag(pb, mov, s); + mov_write_av1c_tag(pb, &mov->tracks[0]); + mov_write_colr_tag(pb, &mov->tracks[0], 0); + return update_size(pb, pos); +} + +static int mov_write_ipma_tag(AVIOContext *pb, MOVMuxContext *mov, AVFormatContext *s) +{ + int64_t pos = avio_tell(pb); + avio_wb32(pb, 0); /* size */ + ffio_wfourcc(pb, "ipma"); + avio_wb32(pb, 0); /* Version & flags */ + avio_wb32(pb, 1); /* entry_count */ + avio_wb16(pb, 1); /* item_ID */ + avio_w8(pb, 4); /* association_count */ + + // ispe association. + avio_w8(pb, 1); /* essential and property_index */ + // pixi association. + avio_w8(pb, 2); /* essential and property_index */ + // av1C association. + avio_w8(pb, 0x80 | 3); /* essential and property_index */ + // colr association. + avio_w8(pb, 4); /* essential and property_index */ + return update_size(pb, pos); +} + +static int mov_write_iprp_tag(AVIOContext *pb, MOVMuxContext *mov, AVFormatContext *s) +{ + int64_t pos = avio_tell(pb); + avio_wb32(pb, 0); /* size */ + ffio_wfourcc(pb, "iprp"); + mov_write_ipco_tag(pb, mov, s); + mov_write_ipma_tag(pb, mov, s); + return update_size(pb, pos); +} + static int mov_write_hmhd_tag(AVIOContext *pb) { /* This atom must be present, but leaving the values at zero @@ -3056,7 +3189,7 @@ static int mov_write_tkhd_tag(AVIOContext *pb, MOVMuxContext *mov, display_matrix = NULL; } - if (track->flags & MOV_TRACK_ENABLED) + if (track->flags & MOV_TRACK_ENABLED || track->mode == MODE_AVIF) flags |= MOV_TKHD_FLAG_ENABLED; if (track->mode == MODE_ISM) @@ -3104,7 +3237,7 @@ static int mov_write_tkhd_tag(AVIOContext *pb, MOVMuxContext *mov, if (st && (track->par->codec_type == AVMEDIA_TYPE_VIDEO || track->par->codec_type == AVMEDIA_TYPE_SUBTITLE)) { int64_t track_width_1616; - if (track->mode == MODE_MOV) { + if (track->mode == MODE_MOV || track->mode == MODE_AVIF) { track_width_1616 = track->par->width * 0x10000ULL; } else { track_width_1616 = av_rescale(st->sample_aspect_ratio.num, @@ -3439,7 +3572,8 @@ static int mov_write_trak_tag(AVFormatContext *s, AVIOContext *pb, MOVMuxContext mov_write_tapt_tag(pb, track); } } - mov_write_track_udta_tag(pb, mov, st); + if (track->mode != MODE_AVIF) + mov_write_track_udta_tag(pb, mov, st); track->entry = entry_backup; track->chunkCount = chunk_backup; return update_size(pb, pos); @@ -3914,8 +4048,15 @@ static int mov_write_meta_tag(AVIOContext *pb, MOVMuxContext *mov, mov_write_mdta_hdlr_tag(pb, mov, s); mov_write_mdta_keys_tag(pb, mov, s); mov_write_mdta_ilst_tag(pb, mov, s); - } - else { + } else if (mov->mode == MODE_AVIF) { + mov_write_hdlr_tag(s, pb, &mov->tracks[0]); + // We always write the primary item id as 1 since only one track is + // supported for AVIF. + mov_write_pitm_tag(pb, 1); + mov_write_iloc_tag(pb, mov, s); + mov_write_iinf_tag(pb, mov, s); + mov_write_iprp_tag(pb, mov, s); + } else { /* iTunes metadata tag */ mov_write_itunes_hdlr_tag(pb, mov, s); mov_write_ilst_tag(pb, mov, s); @@ -4245,10 +4386,11 @@ static int mov_write_moov_tag(AVIOContext *pb, MOVMuxContext *mov, } mov_write_mvhd_tag(pb, mov); - if (mov->mode != MODE_MOV && !mov->iods_skip) + if (mov->mode != MODE_MOV && mov->mode != MODE_AVIF && !mov->iods_skip) mov_write_iods_tag(pb, mov); for (i = 0; i < mov->nb_streams; i++) { - if (mov->tracks[i].entry > 0 || mov->flags & FF_MOV_FLAG_FRAGMENT) { + if (mov->tracks[i].entry > 0 || mov->flags & FF_MOV_FLAG_FRAGMENT || + mov->mode == MODE_AVIF) { int ret = mov_write_trak_tag(s, pb, mov, &(mov->tracks[i]), i < s->nb_streams ? s->streams[i] : NULL); if (ret < 0) return ret; @@ -4259,7 +4401,7 @@ static int mov_write_moov_tag(AVIOContext *pb, MOVMuxContext *mov, if (mov->mode == MODE_PSP) mov_write_uuidusmt_tag(pb, s); - else + else if (mov->mode != MODE_AVIF) mov_write_udta_tag(pb, mov, s); return update_size(pb, pos); @@ -5002,6 +5144,9 @@ static void mov_write_ftyp_tag_internal(AVIOContext *pb, AVFormatContext *s, else if (mov->mode == MODE_3GP) { ffio_wfourcc(pb, has_h264 ? "3gp6" : "3gp4"); minor = has_h264 ? 0x100 : 0x200; + } else if (mov->mode == MODE_AVIF) { + ffio_wfourcc(pb, mov->is_animated_avif ? "avis" : "avif"); + minor = 0; } else if (mov->mode & MODE_3G2) { ffio_wfourcc(pb, has_h264 ? "3g2b" : "3g2a"); minor = has_h264 ? 0x20000 : 0x10000; @@ -5065,6 +5210,30 @@ static int mov_write_ftyp_tag(AVIOContext *pb, AVFormatContext *s) // compatible brand a second time. if (mov->mode == MODE_ISM) { ffio_wfourcc(pb, "piff"); + } else if (mov->mode == MODE_AVIF) { + const AVPixFmtDescriptor *pix_fmt_desc = + av_pix_fmt_desc_get(s->streams[0]->codecpar->format); + const int depth = pix_fmt_desc->comp[0].depth; + if (mov->is_animated_avif) { + // For animated AVIF, major brand is "avis". Add "avif" as a + // compatible brand. + ffio_wfourcc(pb, "avif"); + ffio_wfourcc(pb, "msf1"); + } + ffio_wfourcc(pb, "mif1"); + ffio_wfourcc(pb, "miaf"); + if (depth == 8 || depth == 10) { + // MA1B and MA1A brands are based on AV1 profile. Short hand for + // computing that is based on chroma subsampling type. 420 chroma + // subsampling is MA1B. 444 chroma subsampling is MA1A. + if (pix_fmt_desc->log2_chroma_w == 0 && pix_fmt_desc->log2_chroma_h == 0) { + // 444 chroma subsampling. + ffio_wfourcc(pb, "MA1A"); + } else { + // 420 chroma subsampling. + ffio_wfourcc(pb, "MA1B"); + } + } } else if (mov->mode != MODE_MOV) { // We add tfdt atoms when fragmenting, signal this with the iso6 compatible // brand, if not already the major brand. This is compatible with users that @@ -5669,7 +5838,7 @@ int ff_mov_write_packet(AVFormatContext *s, AVPacket *pkt) if (ret < 0) return ret; - if (mov->flags & FF_MOV_FLAG_FRAGMENT) { + if (mov->flags & FF_MOV_FLAG_FRAGMENT || mov->mode == MODE_AVIF) { int ret; if (mov->moov_written || mov->flags & FF_MOV_FLAG_EMPTY_MOOV) { if (mov->frag_interleave && mov->fragments > 0) { @@ -5802,7 +5971,9 @@ int ff_mov_write_packet(AVFormatContext *s, AVPacket *pkt) } } } else if (par->codec_id == AV_CODEC_ID_AV1) { - if (trk->hint_track >= 0 && trk->hint_track < mov->nb_streams) { + if (trk->mode == MODE_AVIF) { + avio_write(pb, pkt->data, pkt->size); + } else if (trk->hint_track >= 0 && trk->hint_track < mov->nb_streams) { ret = ff_av1_filter_obus_buf(pkt->data, &reformatted_data, &size, &offset); if (ret < 0) @@ -6230,6 +6401,10 @@ fail: } } + if (trk->mode == MODE_AVIF && !mov->avif_extent_length) { + mov->avif_extent_length = pkt->size; + } + return mov_write_single_packet(s, pkt); } } @@ -6569,11 +6744,15 @@ static int mov_init(AVFormatContext *s) else if (IS_MODE(ipod, IPOD)) mov->mode = MODE_IPOD; else if (IS_MODE(ismv, ISMV)) mov->mode = MODE_ISM; else if (IS_MODE(f4v, F4V)) mov->mode = MODE_F4V; + else if (IS_MODE(avif, AVIF)) mov->mode = MODE_AVIF; #undef IS_MODE if (mov->flags & FF_MOV_FLAG_DELAY_MOOV) mov->flags |= FF_MOV_FLAG_EMPTY_MOOV; + if (mov->mode == MODE_AVIF) + mov->flags |= FF_MOV_FLAG_DELAY_MOOV; + /* Set the FRAGMENT flag if any of the fragmentation methods are * enabled. */ if (mov->max_fragment_duration || mov->max_fragment_size || @@ -6797,12 +6976,13 @@ static int mov_init(AVFormatContext *s) pix_fmt == AV_PIX_FMT_MONOWHITE || pix_fmt == AV_PIX_FMT_MONOBLACK; } - if (track->par->codec_id == AV_CODEC_ID_VP9 || - track->par->codec_id == AV_CODEC_ID_AV1) { - if (track->mode != MODE_MP4) { - av_log(s, AV_LOG_ERROR, "%s only supported in MP4.\n", avcodec_get_name(track->par->codec_id)); - return AVERROR(EINVAL); - } + if (track->par->codec_id == AV_CODEC_ID_VP9 && track->mode != MODE_MP4) { + av_log(s, AV_LOG_ERROR, "%s only supported in MP4.\n", avcodec_get_name(track->par->codec_id)); + return AVERROR(EINVAL); + } else if (track->par->codec_id == AV_CODEC_ID_AV1 && + track->mode != MODE_MP4 && track->mode != MODE_AVIF) { + av_log(s, AV_LOG_ERROR, "%s only supported in MP4 and AVIF.\n", avcodec_get_name(track->par->codec_id)); + return AVERROR(EINVAL); } else if (track->par->codec_id == AV_CODEC_ID_VP8) { /* altref frames handling is not defined in the spec as of version v1.0, * so just forbid muxing VP8 streams altogether until a new version does */ @@ -7003,7 +7183,7 @@ static int mov_write_header(AVFormatContext *s) FF_MOV_FLAG_FRAG_EVERY_FRAME)) && !mov->max_fragment_duration && !mov->max_fragment_size) mov->flags |= FF_MOV_FLAG_FRAG_KEYFRAME; - } else { + } else if (mov->mode != MODE_AVIF) { if (mov->flags & FF_MOV_FLAG_FASTSTART) mov->reserved_header_pos = avio_tell(pb); mov_write_mdat_tag(pb, mov); @@ -7291,6 +7471,54 @@ static int mov_check_bitstream(AVFormatContext *s, AVStream *st, return ret; } +static int avif_write_trailer(AVFormatContext *s) +{ + AVIOContext *pb = s->pb; + MOVMuxContext *mov = s->priv_data; + int64_t pos_backup, mdat_pos; + uint8_t *buf; + int buf_size, moov_size; + int i; + + if (mov->moov_written) return 0; + + mov->is_animated_avif = s->streams[0]->nb_frames > 1; + mov_write_identification(pb, s); + mov_write_meta_tag(pb, mov, s); + + moov_size = get_moov_size(s); + for (i = 0; i < mov->nb_streams; i++) + mov->tracks[i].data_offset = avio_tell(pb) + moov_size + 8; + + if (mov->is_animated_avif) { + int ret; + if ((ret = mov_write_moov_tag(pb, mov, s)) < 0) + return ret; + } + + buf_size = avio_get_dyn_buf(mov->mdat_buf, &buf); + avio_wb32(pb, buf_size + 8); + ffio_wfourcc(pb, "mdat"); + mdat_pos = avio_tell(pb); + + avio_write(pb, buf, buf_size); + ffio_free_dyn_buf(&mov->mdat_buf); + + // write extent offset. + pos_backup = avio_tell(pb); + avio_seek(pb, mov->avif_extent_pos, SEEK_SET); + avio_wb32(pb, mdat_pos); /* rewrite offset */ + avio_seek(pb, pos_backup, SEEK_SET); + + mov->moov_written = 1; + mov->mdat_size = 0; + for (i = 0; i < mov->nb_streams; i++) { + mov->tracks[i].entry = 0; + mov->tracks[i].end_reliable = 0; + } + return 0; +} + #if CONFIG_TGP_MUXER || CONFIG_TG2_MUXER static const AVCodecTag codec_3gp_tags[] = { { AV_CODEC_ID_H263, MKTAG('s','2','6','3') }, @@ -7373,6 +7601,12 @@ static const AVCodecTag codec_f4v_tags[] = { { AV_CODEC_ID_NONE, 0 }, }; +static const AVCodecTag codec_avif_tags[] = { + { AV_CODEC_ID_AV1, MKTAG('a','v','0','1') }, + { AV_CODEC_ID_NONE, 0 }, +}; +static const AVCodecTag *const codec_avif_tags_list[] = { codec_avif_tags, NULL }; + #if CONFIG_MOV_MUXER const AVOutputFormat ff_mov_muxer = { .name = "mov", @@ -7535,3 +7769,21 @@ const AVOutputFormat ff_f4v_muxer = { .priv_class = &mov_isobmff_muxer_class, }; #endif +#if CONFIG_AVIF_MUXER +const AVOutputFormat ff_avif_muxer = { + .name = "avif", + .long_name = NULL_IF_CONFIG_SMALL("AVIF"), + .mime_type = "image/avif", + .extensions = "avif", + .priv_data_size = sizeof(MOVMuxContext), + .video_codec = AV_CODEC_ID_AV1, + .init = mov_init, + .write_header = mov_write_header, + .write_packet = mov_write_packet, + .write_trailer = avif_write_trailer, + .deinit = mov_free, + .flags = AVFMT_GLOBALHEADER | AVFMT_ALLOW_FLUSH, + .codec_tag = codec_avif_tags_list, + .priv_class = &mov_isobmff_muxer_class, +}; +#endif diff --git a/libavformat/movenc.h b/libavformat/movenc.h index 2ac84ed070..55b8469f68 100644 --- a/libavformat/movenc.h +++ b/libavformat/movenc.h @@ -43,6 +43,7 @@ #define MODE_IPOD 0x20 #define MODE_ISM 0x40 #define MODE_F4V 0x80 +#define MODE_AVIF 0x100 typedef struct MOVIentry { uint64_t pos; @@ -242,6 +243,10 @@ typedef struct MOVMuxContext { MOVPrftBox write_prft; int empty_hdlr_name; int movie_timescale; + + int64_t avif_extent_pos; + int avif_extent_length; + int is_animated_avif; } MOVMuxContext; #define FF_MOV_FLAG_RTP_HINT (1 << 0)