From patchwork Wed Apr 13 20:39:18 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vignesh Venkat X-Patchwork-Id: 35309 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:671c:b0:7c:62c8:b2d1 with SMTP id q28csp404466pzh; Wed, 13 Apr 2022 13:39:40 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzBVmAynX7BW4k4W7CXSijvAVRZMhb4ZNYF/pLznUqAcxmC8WoioweeLIPgVwEfuozLDaVA X-Received: by 2002:a17:907:6e04:b0:6e0:736b:d786 with SMTP id sd4-20020a1709076e0400b006e0736bd786mr40182327ejc.667.1649882380458; Wed, 13 Apr 2022 13:39:40 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1649882380; cv=none; d=google.com; s=arc-20160816; b=OXDcgSIwbEU/VLOHqRkAS/JvtUQeH7Aiy8xdAcF4IRJpYxtt9ZoZOgSOGhJyOLtFQT +wtW/V/HaXVANWzUnA8VWqM5I8ckoSSJ7Mm0KepG0zw3fUu0HeIBbUDfGNl212vkpOWc bK9vqS2KcEKWsOzbjAzYSqqkpOn5nqcbfEy6C5Y84zuwcHlSNivFTKRvdficVwqXzUu+ T7K5N4Rzye2t35f76240W4VQ0osxnfzIkpFweh4GO3QF7QaSxp10XUJIDm9lFH/FtNCJ x/RxyF63qCskE1SBt3YyxNQgE4jHpKjJdCx3XQoGX7xNWOkgyX6yyFGqRS+7+QKuHthO iK/Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:to:from:references:mime-version :message-id:in-reply-to:date:dkim-signature:delivered-to; bh=T+PPCcSLuWhctvasOW6bpuhuWPQvLZXt0dnatXDCNHs=; b=E5VnZtFaTCuL3Sz0mayNecpHqQuU1LSGpsuheFmajUeL23mM8FbTshawIleTL+b6AA VUqGcCBGEoxOHdeCtT8OjX8YaI/xmosfYjt6ywTwr5iO+GxDDDf0hPPbymd+gyIqln20 iDBthmiTP+cPGC0cnokBTWr5JqEORolHVxi6vpOX/tQCU6/3jgmvqKHiv3NTGyBJH67U R2jYeh6Uw6lx0jSpvTZdu8lhKiyfjmkUh4gINk1aekHnScRZl/R1krFgWnxJnXhp4rGL 8W+V3RUHJ+1/arjiiG2Mm35qDHxb95AkbRhX92behtHjpV7s5t02HnhtlhcwkNsjsbHp Hcqg== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@google.com header.s=20210112 header.b=TNxSVTrV; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id s16-20020a056402037000b0041d664093e1si2248694edw.482.2022.04.13.13.39.39; Wed, 13 Apr 2022 13:39:40 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@google.com header.s=20210112 header.b=TNxSVTrV; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 2C5CB68B43E; Wed, 13 Apr 2022 23:39:36 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-yw1-f202.google.com (mail-yw1-f202.google.com [209.85.128.202]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id CA43368B360 for ; Wed, 13 Apr 2022 23:39:29 +0300 (EEST) Received: by mail-yw1-f202.google.com with SMTP id 00721157ae682-2e61c5a0eb7so25748657b3.22 for ; Wed, 13 Apr 2022 13:39:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=rDn2haR5vqJ4UydnzvgLnIpleYuPmJrdn8GZOUHJG74=; b=TNxSVTrVWEqEv15nN3kTzd3yDylazayxDzwGD2SCvp8N4N58l5c9EhKE8x7W3BmgFt H6cSgBRZ0QbSi94juR4r1qEeFTrbJ4+TQt1mosHQJKvkPTaIU9lJ3wk4HqfZPNIwyIQC SQ2oWYAhVQh97lE7HoWPCwOyjD766UtpgyBmlj4ptpt15oz45Bk+jowYpC+JT6WeUr0A GcrUttEpvJn9ypNvfm1vsIQhjdK4mLL9mrwHRIgvN0csu16eabJcfmI7y93r65M+kIXA VbV+tYPy1Bq/TovikeT8UIc+g0Om90V+azKMzohm6LbOc+N1Ijw+6Cz+Sr3AIw/E4gpk uWsA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=rDn2haR5vqJ4UydnzvgLnIpleYuPmJrdn8GZOUHJG74=; b=2JCeYyewr+kTxuIRofqZsEkvb0Qh4AIUmwE/d/e3RQMy/DanYpelKndvvMzKyFWIwM Gjrp8zdSqbLCUu9LtO8NqI7DT/eyTHip7umc8QlXhwcdqg3ZbTMlpTBgxOAWslkHEg2I kbnJngeIr0yvUG3GTVcZReVdYZI+kx5RDZJ1p0fIuW5p4bS8gb9ICF6PDXOZvCws56Qu rM/U6/ILJuvEeq+JQge3CWnRek3P21OSVErCaCIkpTXacRYmxIqXAEcs8SPcAnn0Y6a7 AMhgYWBO+w65bTIdpMy2ZaKCGpikfg1PmEbukK34jGcJ3+KG278r77BjfPnq5T5I3idX mesg== X-Gm-Message-State: AOAM533HNlR6YMtnZl9akA+yA5ii7RIm91LU4EDH/xJCo19FZ4kqZWvx p++pFrcSsjxbvpq7Fx9ajLA0koO9IfXjzcquLMmPyyt6kxiw4bQ8iS/Bv+ymUmlDJVBkbXbGfaK ftsxkb05+VTFzyc+G+lmD7JNiUdZaDdmCrHnPIKuVBxMD6K+Lyk6wpYvbLm3cK59zrM9/ X-Received: from vigneshv3.mtv.corp.google.com ([2620:0:1000:2511:a185:f162:f2de:5a9]) (user=vigneshv job=sendgmr) by 2002:a05:6902:150d:b0:63d:f457:9384 with SMTP id q13-20020a056902150d00b0063df4579384mr589635ybu.601.1649882367601; Wed, 13 Apr 2022 13:39:27 -0700 (PDT) Date: Wed, 13 Apr 2022 13:39:18 -0700 In-Reply-To: <20220328204752.3339971-1-vigneshv@google.com> Message-Id: <20220413203918.3695742-1-vigneshv@google.com> Mime-Version: 1.0 References: <20220328204752.3339971-1-vigneshv@google.com> X-Mailer: git-send-email 2.35.1.1178.g4f1659d476-goog From: Vignesh Venkatasubramanian To: ffmpeg-devel@ffmpeg.org Subject: [FFmpeg-devel] [PATCH 1/3] avcodec/libaomenc: Add parameter for avif single image encoding X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Vignesh Venkatasubramanian Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: D/iCfwAft8RD Add a parameter to libaom-av1 encoder to enforce some of the single image constraints in the AV1 encoder. Setting this flag will limit the encoder to producing exactly one frame and the sequence header that is produced by the encoder will be conformant to the AVIF specification [1]. Part of Fixing Trac ticket #7621 [1] https://aomediacodec.github.io/av1-avif Signed-off-by:: Vignesh Venkatasubramanian --- libavcodec/libaomenc.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/libavcodec/libaomenc.c b/libavcodec/libaomenc.c index 054903e6e2..0411773bbf 100644 --- a/libavcodec/libaomenc.c +++ b/libavcodec/libaomenc.c @@ -100,6 +100,7 @@ typedef struct AOMEncoderContext { int enable_restoration; int usage; int tune; + int still_picture; int enable_rect_partitions; int enable_1to4_partitions; int enable_ab_partitions; @@ -747,6 +748,18 @@ static av_cold int aom_init(AVCodecContext *avctx, if (res < 0) return res; + if (ctx->still_picture) { + // Set the maximum number of frames to 1. This will let libaom set + // still_picture and reduced_still_picture_header to 1 in the Sequence + // Header as required by AVIF still images. + enccfg.g_limit = 1; + // Reduce memory usage for still images. + enccfg.g_lag_in_frames = 0; + // All frames will be key frames. + enccfg.kf_max_dist = 0; + enccfg.kf_mode = AOM_KF_DISABLED; + } + /* Construct Encoder Context */ res = aom_codec_enc_init(&ctx->encoder, iface, &enccfg, flags); if (res != AOM_CODEC_OK) { @@ -1291,6 +1304,7 @@ static const AVOption options[] = { { "psnr", NULL, 0, AV_OPT_TYPE_CONST, {.i64 = AOM_TUNE_PSNR}, 0, 0, VE, "tune"}, { "ssim", NULL, 0, AV_OPT_TYPE_CONST, {.i64 = AOM_TUNE_SSIM}, 0, 0, VE, "tune"}, FF_AV1_PROFILE_OPTS + { "still-picture", "Encode in single frame mode (typically used for still AVIF images).", OFFSET(still_picture), AV_OPT_TYPE_BOOL, {.i64 = 0}, -1, 1, VE }, { "enable-rect-partitions", "Enable rectangular partitions", OFFSET(enable_rect_partitions), AV_OPT_TYPE_BOOL, {.i64 = -1}, -1, 1, VE}, { "enable-1to4-partitions", "Enable 1:4/4:1 partitions", OFFSET(enable_1to4_partitions), AV_OPT_TYPE_BOOL, {.i64 = -1}, -1, 1, VE}, { "enable-ab-partitions", "Enable ab shape partitions", OFFSET(enable_ab_partitions), AV_OPT_TYPE_BOOL, {.i64 = -1}, -1, 1, VE}, From patchwork Wed Apr 13 20:40:00 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vignesh Venkat X-Patchwork-Id: 35310 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:671c:b0:7c:62c8:b2d1 with SMTP id q28csp404704pzh; Wed, 13 Apr 2022 13:40:15 -0700 (PDT) X-Google-Smtp-Source: ABdhPJw/X6cLmSm5u3eAhZ1Zl++ZH2KC1O8VOjwtKPq6O+PiylGopUy2bg2n388AoATpCepGvoDw X-Received: by 2002:a17:907:6d83:b0:6e7:f485:925a with SMTP id sb3-20020a1709076d8300b006e7f485925amr39802632ejc.190.1649882415049; Wed, 13 Apr 2022 13:40:15 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1649882415; cv=none; d=google.com; s=arc-20160816; b=RMeJTlzGzReNXUiwa5PNI0/Ag5jfZZRzdqLO6B//UgiMMeXS58ldFzk9k1RK17HQB8 32mOz5QiQH53TtPLFWs6iw24Ri9WqQTYcR1uy92mDVfoeZ00W5oF5XH8zfY+oH6ipLa9 dXE36mVs2L4EPsMBuDjnwvKE3OstGSIQrRQKucPdvuXJR7MyjxzdNftY6mzgaAmYfyE5 3Ilj/rEp/hF18515UCYvjgQS6705DZUTkc3qiBTeOOK/Vg/Q5p+0/hSuEa5ZDKH9BOKS ED0IPmPE5WQk9QPPzhpk4FzvtvA+q8HnTnluWvDwKDFzaZFgXWzTKm+Bv294JcPRiHnu klsA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:to:from:references:mime-version :message-id:in-reply-to:date:dkim-signature:delivered-to; bh=pqqOk2ryKI6jhEben79DVIWp1Y8bsgWx+jzw8iJ1x7c=; b=xmsZZ+nt+xw5Q1Zo5p4WVFApWmsgK5O7r95wa6hcXnWBkSIu2JHkp9xXb8K14x2SLm MIaj1KzJovQ+TGxj4Gw64LNOD0k3c6Vy8HL9sJxAIBz0vtJTh/ipPZGVXYjBwbgxjXfW 9xjKO2JwwksIQ3gfACVUO2CyAHrsEqa7rMOoXOt9LYAA+4vf+yKOB0f6Z9EotsaTUEHW Vy/DVY5wXHaKs735lZe2wbyaxGx8I0/p6mX7jrelZeUvEgSDsI3xcQDV8cCynYXFJ7lQ pNQF4NmH5KsuJLqfF7pCaSvPVauFn2xJKlBo8N94b0yKRK0j3ewGSW4gqH60kuDYVOgX D+rA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@google.com header.s=20210112 header.b=g4e11kdD; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id tl11-20020a170907c30b00b006e8b0055f81si576153ejc.453.2022.04.13.13.40.13; Wed, 13 Apr 2022 13:40:15 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@google.com header.s=20210112 header.b=g4e11kdD; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 2BD0B68B44F; Wed, 13 Apr 2022 23:40:11 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-yw1-f201.google.com (mail-yw1-f201.google.com [209.85.128.201]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 3ACAC68B05A for ; Wed, 13 Apr 2022 23:40:05 +0300 (EEST) Received: by mail-yw1-f201.google.com with SMTP id 00721157ae682-2dc7bdd666fso26211697b3.7 for ; Wed, 13 Apr 2022 13:40:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=5yCqD9/AO1aKbttthPEDK3Sqz4Rt56wCEuSX4UZmnuc=; b=g4e11kdDQ93bgBww9A1BGkLTCLlJw6e5HBwSVhZavmYgM/Y0ig1KDNTTSnBolOiFAg 8HWLOcgTiljwqs8ULS1a4i6Aq63LTfeS8FGWkipzb7+ixBA1a8/7djp/+e/YLIf8bLTj 3d4oFwQ5/l25dGJh9BXEqXuBRIlV2c2CYJQ9a/y031IS0UcnIf+YkY1E6al+vp+m8n/f 2ecjgfalX74lDA2WSBxwKMG/3MsOBwyzQF8E6Wcwo0LpiBBbMw2UgAWImQJU0P0JGdaT XQgi5eMsVK/IyyJzD0+El+Uq5HN5rwC/QzoCfFQ03kigB7FOUOXVQ8d4qp8a26tLEM8M kjUQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=5yCqD9/AO1aKbttthPEDK3Sqz4Rt56wCEuSX4UZmnuc=; b=XO/6LIlDOn8CbY5JG9NWI51xnl0fNyykyZ/ZGoI/8oGFWVLtkkl6PlVapvDjhebuM6 tEJQlIEGjVnO+jGLOC3P7lt/PxedWNuR8BRcnTeSkQ60LO5qGar5nXDegYxy4bkU1GUN azkjyYcEZMk4TjLMASInB03G/NxD1Xx/ZJGL4l9YoORPlJGB+WFe3/ylVLPLur9+S++U PLeYTkmHW6Gx0udQDYAVKJ1LD2Ixgw1GPyR1sV7gNe5SItbi2O7hMocIvkgT/kvR5/kx lxpK/fVmHaP94kaZVpUeCzrGH7kW6rWyW45KYRwn15R2MOW51j9XZRKBUgFOWL2KIehR MRNA== X-Gm-Message-State: AOAM533/6b1xpXvbqwLTXUMBHr8Kx1P6np3G9iTMAfsHIyWFbClIbqS6 6BjxWKHXwd9OfgPWbJU+g9fUWKpLpe40GgtO13rxCAjN7Upwdy0RtPEaAj3UR4ZuzoCiv29n1yU jdIT3t7Azn/c6KVXYASjfYuVbSOd5kwOk38dOclh0fkPMefgbs4wyl5WkUv+J7p7i+7zy X-Received: from vigneshv3.mtv.corp.google.com ([2620:0:1000:2511:a185:f162:f2de:5a9]) (user=vigneshv job=sendgmr) by 2002:a25:ca87:0:b0:641:e72:6037 with SMTP id a129-20020a25ca87000000b006410e726037mr596786ybg.438.1649882403592; Wed, 13 Apr 2022 13:40:03 -0700 (PDT) Date: Wed, 13 Apr 2022 13:40:00 -0700 In-Reply-To: <20220328204833.3342663-1-vigneshv@google.com> Message-Id: <20220413204000.3696281-1-vigneshv@google.com> Mime-Version: 1.0 References: <20220328204833.3342663-1-vigneshv@google.com> X-Mailer: git-send-email 2.35.1.1178.g4f1659d476-goog From: Vignesh Venkatasubramanian To: ffmpeg-devel@ffmpeg.org Subject: [FFmpeg-devel] [PATCH 2/3] avformat/av1: Add a parameter to av1c to omit seq header X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Vignesh Venkatasubramanian Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: haPMpx1XOHaE Add a parameter to omit seq header when generating the av1C atom. For now, this does not change any behavior. This will be used by a follow-up patch to add AVIF support. Signed-off-by: Vignesh Venkatasubramanian --- libavformat/av1.c | 7 +++++-- libavformat/av1.h | 4 +++- libavformat/matroskaenc.c | 4 ++-- libavformat/movenc.c | 2 +- 4 files changed, 11 insertions(+), 6 deletions(-) diff --git a/libavformat/av1.c b/libavformat/av1.c index 79065d0c9f..b6eaf50627 100644 --- a/libavformat/av1.c +++ b/libavformat/av1.c @@ -395,7 +395,8 @@ int ff_av1_parse_seq_header(AV1SequenceParameters *seq, const uint8_t *buf, int return is_av1c ? 0 : AVERROR_INVALIDDATA; } -int ff_isom_write_av1c(AVIOContext *pb, const uint8_t *buf, int size) +int ff_isom_write_av1c(AVIOContext *pb, const uint8_t *buf, int size, + int write_seq_header) { AVIOContext *meta_pb; AV1SequenceParameters seq_params; @@ -485,7 +486,9 @@ int ff_isom_write_av1c(AVIOContext *pb, const uint8_t *buf, int size) flush_put_bits(&pbc); avio_write(pb, header, sizeof(header)); - avio_write(pb, seq, seq_size); + if (write_seq_header) { + avio_write(pb, seq, seq_size); + } meta_size = avio_get_dyn_buf(meta_pb, &meta); if (meta_size) diff --git a/libavformat/av1.h b/libavformat/av1.h index f57dabe986..a393fbb78f 100644 --- a/libavformat/av1.h +++ b/libavformat/av1.h @@ -96,9 +96,11 @@ int ff_av1_parse_seq_header(AV1SequenceParameters *seq, const uint8_t *buf, int * @param pb pointer to the AVIOContext where the av1C box shall be written * @param buf input data buffer * @param size size in bytes of the input data buffer + * @param write_seq_header If 1, Sequence Header OBU will be written inside the + * av1C box. Otherwise, Sequence Header OBU will be omitted. * * @return >= 0 in case of success, a negative AVERROR code in case of failure */ -int ff_isom_write_av1c(AVIOContext *pb, const uint8_t *buf, int size); +int ff_isom_write_av1c(AVIOContext *pb, const uint8_t *buf, int size, int write_seq_header); #endif /* AVFORMAT_AV1_H */ diff --git a/libavformat/matroskaenc.c b/libavformat/matroskaenc.c index 3b8ca11f28..d789a618a4 100644 --- a/libavformat/matroskaenc.c +++ b/libavformat/matroskaenc.c @@ -1089,7 +1089,7 @@ static int mkv_write_native_codecprivate(AVFormatContext *s, AVIOContext *pb, case AV_CODEC_ID_AV1: if (par->extradata_size) return ff_isom_write_av1c(dyn_cp, par->extradata, - par->extradata_size); + par->extradata_size, 1); else put_ebml_void(pb, 4 + 3); break; @@ -2665,7 +2665,7 @@ static int mkv_check_new_extra_data(AVFormatContext *s, const AVPacket *pkt) ret = avio_open_dyn_buf(&dyn_cp); if (ret < 0) return ret; - ff_isom_write_av1c(dyn_cp, side_data, side_data_size); + ff_isom_write_av1c(dyn_cp, side_data, side_data_size, 1); codecpriv_size = avio_get_dyn_buf(dyn_cp, &codecpriv); if ((ret = dyn_cp->error) < 0 || !codecpriv_size && (ret = AVERROR_INVALIDDATA)) { diff --git a/libavformat/movenc.c b/libavformat/movenc.c index b9956e699c..4202d0b79a 100644 --- a/libavformat/movenc.c +++ b/libavformat/movenc.c @@ -1334,7 +1334,7 @@ static int mov_write_av1c_tag(AVIOContext *pb, MOVTrack *track) avio_wb32(pb, 0); ffio_wfourcc(pb, "av1C"); - ff_isom_write_av1c(pb, track->vos_data, track->vos_len); + ff_isom_write_av1c(pb, track->vos_data, track->vos_len, 1); return update_size(pb, pos); } From patchwork Wed Apr 13 20:40:28 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vignesh Venkat X-Patchwork-Id: 35311 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:671c:b0:7c:62c8:b2d1 with SMTP id q28csp404851pzh; Wed, 13 Apr 2022 13:40:39 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxivLORcIg01bGERAUJkVNhu8I6ViMqzB2leG5OmRzGzYOYMRBuOxITNcasBM6zbDOeOjdA X-Received: by 2002:a05:6402:4396:b0:418:d776:14c1 with SMTP id o22-20020a056402439600b00418d77614c1mr45813864edc.127.1649882438761; Wed, 13 Apr 2022 13:40:38 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1649882438; cv=none; d=google.com; s=arc-20160816; b=YXOOfvKGClbL1G0+ld22rqC1zIatrhoPsKYSinZ1f0LiyE9B7q8Ci/hIZaMO8wFTxE YWtTrnbaerKZy/1Z9PUFKJhTVTssOTCQMxmDK54JZEP3tWys7XxZFkg1zD16lF+g7yom wNyLHv13mDQ8DNL5fD2oEMEfWFXmUfxCnVkD7Fl/hd1M3JEja5mRp69qtwVKhyzFJlUx 2QHKPo1GxljMOFbnkXUs7sX4B/jYymBeHFrlHl/4GwMKKgBPVijCgXpGqJOwNGtM1eeH TeJ/VrktkDq9/fp+04kR6A+moNV4qm37U53Ca2jcrEZtEV23xBEeSwb3uJ3fxB/86u/i RlrQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:to:from:references:mime-version :message-id:in-reply-to:date:dkim-signature:delivered-to; bh=gmnLLce1I7Utkm9t4e6lIUiDYbCcMbwlu04JxbZ6He0=; b=dtjJWoxRmCdz0wc9NmcUXOsV1nuMafDkY8n09EfhXinwhBIU/4V3e8P+ZCGNn34jSU z+cEkjKEx8lo9pc+VqHJUmc8rVCKQpi2VBJuAwsbL5RnDf26keZMEZ+oWxNqt9yAYxw0 GYEldTkpY0cCTjxp2hz7ZArZwC0tq4h5tuySFmPzB7jZZHmG9fzbHS/BAqnpb1dgaddD IKsvyR5D/e4xFgB4M1ESa+E5IAjg+XwbNmq0iKLKuVazGpN2+RQ3j2WDErpbycjcUY0x 2dTiTpSarNgU1CmkkXhr9/ad2Vf20e7+FukYABaPTEAPlnm2jukF+GWeceMNnisya8MX CTuQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@google.com header.s=20210112 header.b=VKGm79k5; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id j7-20020a17090643c700b006e7eb9aa825si628292ejn.416.2022.04.13.13.40.37; Wed, 13 Apr 2022 13:40:38 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@google.com header.s=20210112 header.b=VKGm79k5; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 1DA9168B452; Wed, 13 Apr 2022 23:40:35 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-yw1-f202.google.com (mail-yw1-f202.google.com [209.85.128.202]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 0AFCA68B272 for ; Wed, 13 Apr 2022 23:40:32 +0300 (EEST) Received: by mail-yw1-f202.google.com with SMTP id 00721157ae682-2ec08c1dd2dso25972047b3.20 for ; Wed, 13 Apr 2022 13:40:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=9nJZF8Obf9gqolgsRUhCCzuVLeK3OZtPnrQRuMValbU=; b=VKGm79k5DxuXtQIh337slTEaWxqmmlKGIJUPg3Ri9k9P16EPGTEspI6+xMZ66tI9yT 1iSHADeK3vpFsrv11Zzo4+MO63OGLesYKbvFvlg+3wkYACHP8ZGUS/mQ4XIOY6rDnkNu omUrkqgyROeEpvBPycJ1+AJ1r5na91JtiprMTp2sGKwFLE058ocjxN+nSK/AMHPGHR33 uO2NBav75OKnYk7a3ramnnLoS8Vpu+KlQLdNIDbOb8bluAdMn744SODaq7QlLSvOegtI d2CJ4LbbW3R5LkoxjccUh6vnlHh+cFA+wNOZv+u7kKXo7ksWsFxZBhWy8WqwoT1lZomm s0rQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=9nJZF8Obf9gqolgsRUhCCzuVLeK3OZtPnrQRuMValbU=; b=uMfeZY3OT7n6h90egjUbP24nQpXoBhFlBvVXQ6pPvQLs0L6TcDBMKgdkh5LULDCT51 542M6flUl1MeVCkHEsu+xS6GFOfQDT2UDQh+ucO8GGwo+agfw6wlAX87DH6L5NXMRvt5 b98AGs+Rkv2Vp8ee+56jf5ue4VqTjwIbQo19O6nhxaznFp/sF5TdMWzWeXVyUAS1fF9c vHMMI3Yw2Qajah8lamofB7bySiApICauMXG95GO3+rDDDvEFGxyRahMpZK0cRspAvbIg WHaGklES3NtzgFjf950c9eWd1gsy4J1jNrMcO8J16yJqj5biyzLCkH3WS6l9MQX4Fj9s CtSw== X-Gm-Message-State: AOAM533AXY66HcZ3IdiXXqZTFT1ZBc34Jrpn3K2vmD0L42qtQl2YZo/5 Br9ENeDK8PRAY9Y/R2r3JBmpSPV2dPtcfW9TOy3bQiqHinOIU8V15T1k7HtSTUeEIYGBshVzFng Ht5+3jeGQfOBhmgsbI7ArJYAWN93pfYoeU/V4kllRfaKvGbrZHFNAe8rOcJfRdW8MC/1p X-Received: from vigneshv3.mtv.corp.google.com ([2620:0:1000:2511:a185:f162:f2de:5a9]) (user=vigneshv job=sendgmr) by 2002:a25:2e52:0:b0:641:e3a7:ee8b with SMTP id b18-20020a252e52000000b00641e3a7ee8bmr615289ybn.254.1649882431722; Wed, 13 Apr 2022 13:40:31 -0700 (PDT) Date: Wed, 13 Apr 2022 13:40:28 -0700 In-Reply-To: Message-Id: <20220413204028.3696947-1-vigneshv@google.com> Mime-Version: 1.0 References: X-Mailer: git-send-email 2.35.1.1178.g4f1659d476-goog From: Vignesh Venkatasubramanian To: ffmpeg-devel@ffmpeg.org Subject: [FFmpeg-devel] [PATCH 3/3] avformat/movenc: Add support for AVIF muxing X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Vignesh Venkatasubramanian Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: nw/Ox3vgr03K Add an AVIF muxer by re-using the existing the mov/mp4 muxer. AVIF Specification: https://aomediacodec.github.io/av1-avif Sample usage for still image: ffmpeg -i image.png -c:v libaom-av1 -avif-image 1 image.avif Sample usage for animated AVIF image: ffmpeg -i video.mp4 animated.avif We can re-use any of the AV1 encoding options that will make sense for image encoding (like bitrate, tiles, encoding speed, etc). The files generated by this muxer has been verified to be valid AVIF files by the following: 1) Displays on Chrome (both still and animated images). 2) Displays on Firefox (only still images, firefox does not support animated AVIF yet). 3) Verified to be valid by Compliance Warden: https://github.com/gpac/ComplianceWarden Fixes the encoder/muxer part of Trac Ticket #7621 Signed-off-by: Vignesh Venkatasubramanian --- configure | 1 + libavformat/allformats.c | 1 + libavformat/movenc.c | 337 ++++++++++++++++++++++++++++++++++++--- libavformat/movenc.h | 5 + 4 files changed, 319 insertions(+), 25 deletions(-) diff --git a/configure b/configure index 358a614854..ef9d6cdc92 100755 --- a/configure +++ b/configure @@ -3398,6 +3398,7 @@ asf_stream_muxer_select="asf_muxer" av1_demuxer_select="av1_frame_merge_bsf av1_parser" avi_demuxer_select="riffdec exif" avi_muxer_select="riffenc" +avif_muxer_select="mov_muxer" caf_demuxer_select="iso_media" caf_muxer_select="iso_media" dash_muxer_select="mp4_muxer" diff --git a/libavformat/allformats.c b/libavformat/allformats.c index 7c1d0ac38f..320ddf9898 100644 --- a/libavformat/allformats.c +++ b/libavformat/allformats.c @@ -81,6 +81,7 @@ extern const AVOutputFormat ff_au_muxer; extern const AVInputFormat ff_av1_demuxer; extern const AVInputFormat ff_avi_demuxer; extern const AVOutputFormat ff_avi_muxer; +extern const AVOutputFormat ff_avif_muxer; extern const AVInputFormat ff_avisynth_demuxer; extern const AVOutputFormat ff_avm2_muxer; extern const AVInputFormat ff_avr_demuxer; diff --git a/libavformat/movenc.c b/libavformat/movenc.c index 4202d0b79a..302ee1fda5 100644 --- a/libavformat/movenc.c +++ b/libavformat/movenc.c @@ -1334,7 +1334,7 @@ static int mov_write_av1c_tag(AVIOContext *pb, MOVTrack *track) avio_wb32(pb, 0); ffio_wfourcc(pb, "av1C"); - ff_isom_write_av1c(pb, track->vos_data, track->vos_len, 1); + ff_isom_write_av1c(pb, track->vos_data, track->vos_len, track->mode != MODE_AVIF); return update_size(pb, pos); } @@ -2035,12 +2035,13 @@ static int mov_write_colr_tag(AVIOContext *pb, MOVTrack *track, int prefer_icc) } } - /* We should only ever be called by MOV or MP4. */ - av_assert0(track->mode == MODE_MOV || track->mode == MODE_MP4); + /* We should only ever be called for MOV, MP4 and AVIF. */ + av_assert0(track->mode == MODE_MOV || track->mode == MODE_MP4 || + track->mode == MODE_AVIF); avio_wb32(pb, 0); /* size */ ffio_wfourcc(pb, "colr"); - if (track->mode == MODE_MP4) + if (track->mode == MODE_MP4 || track->mode == MODE_AVIF) ffio_wfourcc(pb, "nclx"); else ffio_wfourcc(pb, "nclc"); @@ -2050,7 +2051,7 @@ static int mov_write_colr_tag(AVIOContext *pb, MOVTrack *track, int prefer_icc) avio_wb16(pb, track->par->color_primaries); avio_wb16(pb, track->par->color_trc); avio_wb16(pb, track->par->color_space); - if (track->mode == MODE_MP4) { + if (track->mode == MODE_MP4 || track->mode == MODE_AVIF) { int full_range = track->par->color_range == AVCOL_RANGE_JPEG; avio_w8(pb, full_range << 7); } @@ -2116,7 +2117,7 @@ static void find_compressor(char * compressor_name, int len, MOVTrack *track) || (track->par->width == 1440 && track->par->height == 1080) || (track->par->width == 1920 && track->par->height == 1080); - if (track->mode == MODE_MOV && + if ((track->mode == MODE_AVIF || track->mode == MODE_MOV) && (encoder = av_dict_get(track->st->metadata, "encoder", NULL, 0))) { av_strlcpy(compressor_name, encoder->value, 32); } else if (track->par->codec_id == AV_CODEC_ID_MPEG2VIDEO && xdcam_res) { @@ -2137,6 +2138,25 @@ static void find_compressor(char * compressor_name, int len, MOVTrack *track) } } +static int mov_write_ccst_tag(AVIOContext *pb) +{ + int64_t pos = avio_tell(pb); + // Write sane defaults: + // all_ref_pics_intra = 0 : all samples can use any type of reference. + // intra_pred_used = 1 : intra prediction may or may not be used. + // max_ref_per_pic = 15 : reserved value to indicate that any number of + // reference images can be used. + uint8_t ccstValue = (0 << 7) | /* all_ref_pics_intra */ + (1 << 6) | /* intra_pred_used */ + (15 << 2); /* max_ref_per_pic */ + avio_wb32(pb, 0); /* size */ + ffio_wfourcc(pb, "ccst"); + avio_wb32(pb, 0); /* Version & flags */ + avio_w8(pb, ccstValue); + avio_wb24(pb, 0); /* reserved */ + return update_size(pb, pos); +} + static int mov_write_video_tag(AVFormatContext *s, AVIOContext *pb, MOVMuxContext *mov, MOVTrack *track) { int ret = AVERROR_BUG; @@ -2154,6 +2174,8 @@ static int mov_write_video_tag(AVFormatContext *s, AVIOContext *pb, MOVMuxContex avio_wb32(pb, 0); /* size */ if (mov->encryption_scheme != MOV_ENC_NONE) { ffio_wfourcc(pb, "encv"); + } else if (track->mode == MODE_AVIF) { + ffio_wfourcc(pb, "av01"); } else { avio_wl32(pb, track->tag); // store it byteswapped } @@ -2270,7 +2292,7 @@ static int mov_write_video_tag(AVFormatContext *s, AVIOContext *pb, MOVMuxContex else av_log(mov->fc, AV_LOG_WARNING, "Not writing 'gama' atom. Format is not MOV.\n"); } - if (track->mode == MODE_MOV || track->mode == MODE_MP4) { + if (track->mode == MODE_MOV || track->mode == MODE_MP4 || track->mode == MODE_AVIF) { int has_color_info = track->par->color_primaries != AVCOL_PRI_UNSPECIFIED && track->par->color_trc != AVCOL_TRC_UNSPECIFIED && track->par->color_space != AVCOL_SPC_UNSPECIFIED; @@ -2322,6 +2344,9 @@ static int mov_write_video_tag(AVFormatContext *s, AVIOContext *pb, MOVMuxContex if (avid) avio_wb32(pb, 0); + if (track->mode == MODE_AVIF) + mov_write_ccst_tag(pb); + return update_size(pb, pos); } @@ -2823,7 +2848,10 @@ static int mov_write_hdlr_tag(AVFormatContext *s, AVIOContext *pb, MOVTrack *tra if (track) { hdlr = (track->mode == MODE_MOV) ? "mhlr" : "\0\0\0\0"; - if (track->par->codec_type == AVMEDIA_TYPE_VIDEO) { + if (track->mode == MODE_AVIF) { + hdlr_type = "pict"; + descr = "ffmpeg"; + } else if (track->par->codec_type == AVMEDIA_TYPE_VIDEO) { hdlr_type = "vide"; descr = "VideoHandler"; } else if (track->par->codec_type == AVMEDIA_TYPE_AUDIO) { @@ -2890,6 +2918,129 @@ static int mov_write_hdlr_tag(AVFormatContext *s, AVIOContext *pb, MOVTrack *tra return update_size(pb, pos); } +static int mov_write_pitm_tag(AVIOContext *pb, int item_id) +{ + int64_t pos = avio_tell(pb); + avio_wb32(pb, 0); /* size */ + ffio_wfourcc(pb, "pitm"); + avio_wb32(pb, 0); /* Version & flags */ + avio_wb16(pb, item_id); /* item_id */ + return update_size(pb, pos); +} + +static int mov_write_iloc_tag(AVIOContext *pb, MOVMuxContext *mov, AVFormatContext *s) +{ + int64_t pos = avio_tell(pb); + avio_wb32(pb, 0); /* size */ + ffio_wfourcc(pb, "iloc"); + avio_wb32(pb, 0); /* Version & flags */ + avio_w8(pb, (4 << 4) + 4); /* offset_size(4) and length_size(4) */ + avio_w8(pb, 0); /* base_offset_size(4) and reserved(4) */ + avio_wb16(pb, 1); /* item_count */ + + avio_wb16(pb, 1); /* item_id */ + avio_wb16(pb, 0); /* data_reference_index */ + avio_wb16(pb, 1); /* extent_count */ + mov->avif_extent_pos = avio_tell(pb); + avio_wb32(pb, 0); /* extent_offset (written later) */ + // For animated AVIF, we simply write the first packet's size. + avio_wb32(pb, mov->avif_extent_length); /* extent_length */ + + return update_size(pb, pos); +} + +static int mov_write_iinf_tag(AVIOContext *pb, MOVMuxContext *mov, AVFormatContext *s) +{ + int64_t infe_pos; + int64_t iinf_pos = avio_tell(pb); + avio_wb32(pb, 0); /* size */ + ffio_wfourcc(pb, "iinf"); + avio_wb32(pb, 0); /* Version & flags */ + avio_wb16(pb, 1); /* entry_count */ + + infe_pos = avio_tell(pb); + avio_wb32(pb, 0); /* size */ + ffio_wfourcc(pb, "infe"); + avio_w8(pb, 0x2); /* Version */ + avio_wb24(pb, 0); /* flags */ + avio_wb16(pb, 1); /* item_id */ + avio_wb16(pb, 0); /* item_protection_index */ + avio_write(pb, "av01", 4); /* item_type */ + avio_write(pb, "Color\0", 6); /* item_name */ + update_size(pb, infe_pos); + + return update_size(pb, iinf_pos); +} + +static int mov_write_ispe_tag(AVIOContext *pb, MOVMuxContext *mov, AVFormatContext *s) +{ + int64_t pos = avio_tell(pb); + avio_wb32(pb, 0); /* size */ + ffio_wfourcc(pb, "ispe"); + avio_wb32(pb, 0); /* Version & flags */ + avio_wb32(pb, s->streams[0]->codecpar->width); /* image_width */ + avio_wb32(pb, s->streams[0]->codecpar->height); /* image_height */ + return update_size(pb, pos); +} + + +static int mov_write_pixi_tag(AVIOContext *pb, MOVMuxContext *mov, AVFormatContext *s) +{ + int64_t pos = avio_tell(pb); + const AVPixFmtDescriptor *pixdesc = av_pix_fmt_desc_get(s->streams[0]->codecpar->format); + avio_wb32(pb, 0); /* size */ + ffio_wfourcc(pb, "pixi"); + avio_wb32(pb, 0); /* Version & flags */ + avio_w8(pb, pixdesc->nb_components); /* num_channels */ + for (int i = 0; i < pixdesc->nb_components; ++i) { + avio_w8(pb, pixdesc->comp[i].depth); /* bits_per_channel */ + } + return update_size(pb, pos); +} + +static int mov_write_ipco_tag(AVIOContext *pb, MOVMuxContext *mov, AVFormatContext *s) +{ + int64_t pos = avio_tell(pb); + avio_wb32(pb, 0); /* size */ + ffio_wfourcc(pb, "ipco"); + mov_write_ispe_tag(pb, mov, s); + mov_write_pixi_tag(pb, mov, s); + mov_write_av1c_tag(pb, &mov->tracks[0]); + mov_write_colr_tag(pb, &mov->tracks[0], 0); + return update_size(pb, pos); +} + +static int mov_write_ipma_tag(AVIOContext *pb, MOVMuxContext *mov, AVFormatContext *s) +{ + int64_t pos = avio_tell(pb); + avio_wb32(pb, 0); /* size */ + ffio_wfourcc(pb, "ipma"); + avio_wb32(pb, 0); /* Version & flags */ + avio_wb32(pb, 1); /* entry_count */ + avio_wb16(pb, 1); /* item_ID */ + avio_w8(pb, 4); /* association_count */ + + // ispe association. + avio_w8(pb, 1); /* essential and property_index */ + // pixi association. + avio_w8(pb, 2); /* essential and property_index */ + // av1C association. + avio_w8(pb, 0x80 | 3); /* essential and property_index */ + // colr association. + avio_w8(pb, 4); /* essential and property_index */ + return update_size(pb, pos); +} + +static int mov_write_iprp_tag(AVIOContext *pb, MOVMuxContext *mov, AVFormatContext *s) +{ + int64_t pos = avio_tell(pb); + avio_wb32(pb, 0); /* size */ + ffio_wfourcc(pb, "iprp"); + mov_write_ipco_tag(pb, mov, s); + mov_write_ipma_tag(pb, mov, s); + return update_size(pb, pos); +} + static int mov_write_hmhd_tag(AVIOContext *pb) { /* This atom must be present, but leaving the values at zero @@ -3087,7 +3238,7 @@ static int mov_write_tkhd_tag(AVIOContext *pb, MOVMuxContext *mov, display_matrix = NULL; } - if (track->flags & MOV_TRACK_ENABLED) + if (track->flags & MOV_TRACK_ENABLED || track->mode == MODE_AVIF) flags |= MOV_TKHD_FLAG_ENABLED; if (track->mode == MODE_ISM) @@ -3135,7 +3286,7 @@ static int mov_write_tkhd_tag(AVIOContext *pb, MOVMuxContext *mov, if (st && (track->par->codec_type == AVMEDIA_TYPE_VIDEO || track->par->codec_type == AVMEDIA_TYPE_SUBTITLE)) { int64_t track_width_1616; - if (track->mode == MODE_MOV) { + if (track->mode == MODE_MOV || track->mode == MODE_AVIF) { track_width_1616 = track->par->width * 0x10000ULL; } else { track_width_1616 = av_rescale(st->sample_aspect_ratio.num, @@ -3470,7 +3621,8 @@ static int mov_write_trak_tag(AVFormatContext *s, AVIOContext *pb, MOVMuxContext mov_write_tapt_tag(pb, track); } } - mov_write_track_udta_tag(pb, mov, st); + if (track->mode != MODE_AVIF) + mov_write_track_udta_tag(pb, mov, st); track->entry = entry_backup; track->chunkCount = chunk_backup; return update_size(pb, pos); @@ -3945,8 +4097,15 @@ static int mov_write_meta_tag(AVIOContext *pb, MOVMuxContext *mov, mov_write_mdta_hdlr_tag(pb, mov, s); mov_write_mdta_keys_tag(pb, mov, s); mov_write_mdta_ilst_tag(pb, mov, s); - } - else { + } else if (mov->mode == MODE_AVIF) { + mov_write_hdlr_tag(s, pb, &mov->tracks[0]); + // We always write the primary item id as 1 since only one track is + // supported for AVIF. + mov_write_pitm_tag(pb, 1); + mov_write_iloc_tag(pb, mov, s); + mov_write_iinf_tag(pb, mov, s); + mov_write_iprp_tag(pb, mov, s); + } else { /* iTunes metadata tag */ mov_write_itunes_hdlr_tag(pb, mov, s); mov_write_ilst_tag(pb, mov, s); @@ -4276,10 +4435,11 @@ static int mov_write_moov_tag(AVIOContext *pb, MOVMuxContext *mov, } mov_write_mvhd_tag(pb, mov); - if (mov->mode != MODE_MOV && !mov->iods_skip) + if (mov->mode != MODE_MOV && mov->mode != MODE_AVIF && !mov->iods_skip) mov_write_iods_tag(pb, mov); for (i = 0; i < mov->nb_streams; i++) { - if (mov->tracks[i].entry > 0 || mov->flags & FF_MOV_FLAG_FRAGMENT) { + if (mov->tracks[i].entry > 0 || mov->flags & FF_MOV_FLAG_FRAGMENT || + mov->mode == MODE_AVIF) { int ret = mov_write_trak_tag(s, pb, mov, &(mov->tracks[i]), i < s->nb_streams ? s->streams[i] : NULL); if (ret < 0) return ret; @@ -4290,7 +4450,7 @@ static int mov_write_moov_tag(AVIOContext *pb, MOVMuxContext *mov, if (mov->mode == MODE_PSP) mov_write_uuidusmt_tag(pb, s); - else + else if (mov->mode != MODE_AVIF) mov_write_udta_tag(pb, mov, s); return update_size(pb, pos); @@ -5033,6 +5193,9 @@ static void mov_write_ftyp_tag_internal(AVIOContext *pb, AVFormatContext *s, else if (mov->mode == MODE_3GP) { ffio_wfourcc(pb, has_h264 ? "3gp6" : "3gp4"); minor = has_h264 ? 0x100 : 0x200; + } else if (mov->mode == MODE_AVIF) { + ffio_wfourcc(pb, mov->is_animated_avif ? "avis" : "avif"); + minor = 0; } else if (mov->mode & MODE_3G2) { ffio_wfourcc(pb, has_h264 ? "3g2b" : "3g2a"); minor = has_h264 ? 0x20000 : 0x10000; @@ -5096,6 +5259,31 @@ static int mov_write_ftyp_tag(AVIOContext *pb, AVFormatContext *s) // compatible brand a second time. if (mov->mode == MODE_ISM) { ffio_wfourcc(pb, "piff"); + } else if (mov->mode == MODE_AVIF) { + const AVPixFmtDescriptor *pix_fmt_desc = + av_pix_fmt_desc_get(s->streams[0]->codecpar->format); + const int depth = pix_fmt_desc->comp[0].depth; + if (mov->is_animated_avif) { + // For animated AVIF, major brand is "avis". Add "avif" as a + // compatible brand. + ffio_wfourcc(pb, "avif"); + ffio_wfourcc(pb, "msf1"); + ffio_wfourcc(pb, "iso8"); + } + ffio_wfourcc(pb, "mif1"); + ffio_wfourcc(pb, "miaf"); + if (depth == 8 || depth == 10) { + // MA1B and MA1A brands are based on AV1 profile. Short hand for + // computing that is based on chroma subsampling type. 420 chroma + // subsampling is MA1B. 444 chroma subsampling is MA1A. + if (!pix_fmt_desc->log2_chroma_w && !pix_fmt_desc->log2_chroma_h) { + // 444 chroma subsampling. + ffio_wfourcc(pb, "MA1A"); + } else { + // 420 chroma subsampling. + ffio_wfourcc(pb, "MA1B"); + } + } } else if (mov->mode != MODE_MOV) { // We add tfdt atoms when fragmenting, signal this with the iso6 compatible // brand, if not already the major brand. This is compatible with users that @@ -5699,7 +5887,7 @@ int ff_mov_write_packet(AVFormatContext *s, AVPacket *pkt) if (ret < 0) return ret; - if (mov->flags & FF_MOV_FLAG_FRAGMENT) { + if (mov->flags & FF_MOV_FLAG_FRAGMENT || mov->mode == MODE_AVIF) { int ret; if (mov->moov_written || mov->flags & FF_MOV_FLAG_EMPTY_MOOV) { if (mov->frag_interleave && mov->fragments > 0) { @@ -5840,7 +6028,11 @@ int ff_mov_write_packet(AVFormatContext *s, AVPacket *pkt) avio_write(pb, reformatted_data, size); } else { size = ff_av1_filter_obus(pb, pkt->data, pkt->size); + if (trk->mode == MODE_AVIF && !mov->avif_extent_length) { + mov->avif_extent_length = size; + } } + #if CONFIG_AC3_PARSER } else if (par->codec_id == AV_CODEC_ID_EAC3) { size = handle_eac3(mov, pkt, trk); @@ -6573,11 +6765,15 @@ static int mov_init(AVFormatContext *s) else if (IS_MODE(ipod, IPOD)) mov->mode = MODE_IPOD; else if (IS_MODE(ismv, ISMV)) mov->mode = MODE_ISM; else if (IS_MODE(f4v, F4V)) mov->mode = MODE_F4V; + else if (IS_MODE(avif, AVIF)) mov->mode = MODE_AVIF; #undef IS_MODE if (mov->flags & FF_MOV_FLAG_DELAY_MOOV) mov->flags |= FF_MOV_FLAG_EMPTY_MOOV; + if (mov->mode == MODE_AVIF) + mov->flags |= FF_MOV_FLAG_DELAY_MOOV; + /* Set the FRAGMENT flag if any of the fragmentation methods are * enabled. */ if (mov->max_fragment_duration || mov->max_fragment_size || @@ -6658,11 +6854,25 @@ static int mov_init(AVFormatContext *s) /* Non-seekable output is ok if using fragmentation. If ism_lookahead * is enabled, we don't support non-seekable output at all. */ if (!(s->pb->seekable & AVIO_SEEKABLE_NORMAL) && - (!(mov->flags & FF_MOV_FLAG_FRAGMENT) || mov->ism_lookahead)) { + (!(mov->flags & FF_MOV_FLAG_FRAGMENT) || mov->ism_lookahead || + mov->mode == MODE_AVIF)) { av_log(s, AV_LOG_ERROR, "muxer does not support non seekable output\n"); return AVERROR(EINVAL); } + /* AVIF output must have exactly one video stream */ + if (mov->mode == MODE_AVIF) { + if (s->nb_streams > 1) { + av_log(s, AV_LOG_ERROR, "AVIF output requires exactly one stream\n"); + return AVERROR(EINVAL); + } + if (s->streams[0]->codecpar->codec_type != AVMEDIA_TYPE_VIDEO) { + av_log(s, AV_LOG_ERROR, "AVIF output requires one video stream\n"); + return AVERROR(EINVAL); + } + } + + mov->nb_streams = s->nb_streams; if (mov->mode & (MODE_MP4|MODE_MOV|MODE_IPOD) && s->nb_chapters) mov->chapter_track = mov->nb_streams++; @@ -6801,12 +7011,13 @@ static int mov_init(AVFormatContext *s) pix_fmt == AV_PIX_FMT_MONOWHITE || pix_fmt == AV_PIX_FMT_MONOBLACK; } - if (track->par->codec_id == AV_CODEC_ID_VP9 || - track->par->codec_id == AV_CODEC_ID_AV1) { - if (track->mode != MODE_MP4) { - av_log(s, AV_LOG_ERROR, "%s only supported in MP4.\n", avcodec_get_name(track->par->codec_id)); - return AVERROR(EINVAL); - } + if (track->par->codec_id == AV_CODEC_ID_VP9 && track->mode != MODE_MP4) { + av_log(s, AV_LOG_ERROR, "%s only supported in MP4.\n", avcodec_get_name(track->par->codec_id)); + return AVERROR(EINVAL); + } else if (track->par->codec_id == AV_CODEC_ID_AV1 && + track->mode != MODE_MP4 && track->mode != MODE_AVIF) { + av_log(s, AV_LOG_ERROR, "%s only supported in MP4 and AVIF.\n", avcodec_get_name(track->par->codec_id)); + return AVERROR(EINVAL); } else if (track->par->codec_id == AV_CODEC_ID_VP8) { /* altref frames handling is not defined in the spec as of version v1.0, * so just forbid muxing VP8 streams altogether until a new version does */ @@ -7024,7 +7235,7 @@ static int mov_write_header(AVFormatContext *s) FF_MOV_FLAG_FRAG_EVERY_FRAME)) && !mov->max_fragment_duration && !mov->max_fragment_size) mov->flags |= FF_MOV_FLAG_FRAG_KEYFRAME; - } else { + } else if (mov->mode != MODE_AVIF) { if (mov->flags & FF_MOV_FLAG_FASTSTART) mov->reserved_header_pos = avio_tell(pb); mov_write_mdat_tag(pb, mov); @@ -7312,6 +7523,50 @@ static int mov_check_bitstream(AVFormatContext *s, AVStream *st, return ret; } +static int avif_write_trailer(AVFormatContext *s) +{ + AVIOContext *pb = s->pb; + MOVMuxContext *mov = s->priv_data; + int64_t pos_backup, mdat_pos; + uint8_t *buf; + int buf_size, moov_size; + + if (mov->moov_written) return 0; + + mov->is_animated_avif = s->streams[0]->nb_frames > 1; + mov_write_identification(pb, s); + mov_write_meta_tag(pb, mov, s); + + moov_size = get_moov_size(s); + mov->tracks[0].data_offset = avio_tell(pb) + moov_size + 8; + + if (mov->is_animated_avif) { + int ret; + if ((ret = mov_write_moov_tag(pb, mov, s)) < 0) + return ret; + } + + buf_size = avio_get_dyn_buf(mov->mdat_buf, &buf); + avio_wb32(pb, buf_size + 8); + ffio_wfourcc(pb, "mdat"); + mdat_pos = avio_tell(pb); + + if (mdat_pos != (uint32_t)mdat_pos) { + av_log(s, AV_LOG_ERROR, "mdat offset does not fit in 32 bits\n"); + return AVERROR_INVALIDDATA; + } + + avio_write(pb, buf, buf_size); + + // write extent offset. + pos_backup = avio_tell(pb); + avio_seek(pb, mov->avif_extent_pos, SEEK_SET); + avio_wb32(pb, mdat_pos); /* rewrite offset */ + avio_seek(pb, pos_backup, SEEK_SET); + + return 0; +} + #if CONFIG_TGP_MUXER || CONFIG_TG2_MUXER static const AVCodecTag codec_3gp_tags[] = { { AV_CODEC_ID_H263, MKTAG('s','2','6','3') }, @@ -7394,6 +7649,20 @@ static const AVCodecTag codec_f4v_tags[] = { { AV_CODEC_ID_NONE, 0 }, }; +#if CONFIG_AVIF_MUXER +static const AVCodecTag codec_avif_tags[] = { + { AV_CODEC_ID_AV1, MKTAG('a','v','0','1') }, + { AV_CODEC_ID_NONE, 0 }, +}; +static const AVCodecTag *const codec_avif_tags_list[] = { codec_avif_tags, NULL }; + +static const AVClass mov_avif_muxer_class = { + .class_name = "avif muxer", + .item_name = av_default_item_name, + .version = LIBAVUTIL_VERSION_INT, +}; +#endif + #if CONFIG_MOV_MUXER const AVOutputFormat ff_mov_muxer = { .name = "mov", @@ -7556,3 +7825,21 @@ const AVOutputFormat ff_f4v_muxer = { .priv_class = &mov_isobmff_muxer_class, }; #endif +#if CONFIG_AVIF_MUXER +const AVOutputFormat ff_avif_muxer = { + .name = "avif", + .long_name = NULL_IF_CONFIG_SMALL("AVIF"), + .mime_type = "image/avif", + .extensions = "avif", + .priv_data_size = sizeof(MOVMuxContext), + .video_codec = AV_CODEC_ID_AV1, + .init = mov_init, + .write_header = mov_write_header, + .write_packet = mov_write_packet, + .write_trailer = avif_write_trailer, + .deinit = mov_free, + .flags = AVFMT_GLOBALHEADER | AVFMT_ALLOW_FLUSH, + .codec_tag = codec_avif_tags_list, + .priv_class = &mov_avif_muxer_class, +}; +#endif diff --git a/libavformat/movenc.h b/libavformat/movenc.h index 67d6d4fb66..c72bca1208 100644 --- a/libavformat/movenc.h +++ b/libavformat/movenc.h @@ -43,6 +43,7 @@ #define MODE_IPOD 0x20 #define MODE_ISM 0x40 #define MODE_F4V 0x80 +#define MODE_AVIF 0x100 typedef struct MOVIentry { uint64_t pos; @@ -243,6 +244,10 @@ typedef struct MOVMuxContext { MOVPrftBox write_prft; int empty_hdlr_name; int movie_timescale; + + int64_t avif_extent_pos; + int avif_extent_length; + int is_animated_avif; } MOVMuxContext; #define FF_MOV_FLAG_RTP_HINT (1 << 0)