From patchwork Thu Aug 16 21:22:19 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Almer X-Patchwork-Id: 10013 Delivered-To: ffmpegpatchwork@gmail.com Received: by 2002:a02:104:0:0:0:0:0 with SMTP id c4-v6csp2507580jad; Thu, 16 Aug 2018 14:23:28 -0700 (PDT) X-Google-Smtp-Source: AA+uWPxFZ9KA3edf3BUs4VsQrFL2CNou2RLMW4Gpckl7LyrlY9uqxQWZQAlNFWI1iGuY51MyHcpn X-Received: by 2002:adf:dbc3:: with SMTP id e3-v6mr19364913wrj.217.1534454608091; Thu, 16 Aug 2018 14:23:28 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1534454608; cv=none; d=google.com; s=arc-20160816; b=uCcZcVXba3SkMkSw73is7Pb7F7X+4gUAeplBh00X6xVyByowiX42gR3YJV2Vuj8UDk h1CnDguKcQxV2Dpx7eV1LYARyRkcJ/EXZ2yq9KENSUdX1IEV7I1c7II58ifN1BFfRCeh jgGTDRx6KkSXDwRIch//sM70bEuPEGwoFOTsMMEK6kYqVJl46YpdpQhZcHJpf6++mAhp S/EghI+LDAEN17DSHMy+uRlvOIKhR7r3egHjS9W6A5szIQX3ktEh9Vb085pZuNZul6Hs naKUJxBBD3yrSRI1ui28IMV6dgB6FHtwgimYPQvsmCScbbwStTNxcb32Y4tZNhx2CtxV 9Z2Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:dkim-signature:delivered-to:arc-authentication-results; bh=oQxkJk43bAG1lMm+d4GRAi37eyImlqarR8T9ooYtXjk=; b=XEKESo/MCiatgp68pSfnXYABHw6HCnjiLx4sKbckaEQq4xgl+5Z57AbQKq5ciO1pwy E1JnlGk9Lgu3DzirK1uA7HbmQntcIZTQ9dVXSnyxcI9IM+nLPXO0OIcJ+CuAntsp6PsV zMIjtZoPG3gw24LVfwg4Ow3n3RlmAhvG82ovJTzXLhqd5wv1mf+fzq6ZO2NSbui6J+pG RGlQgSEDkLPkHSsrgb7hU+dAtgOPnO4lrhDNGXKobeFiPScwfbuA1NVg0gMgco3Z/a32 LKNVs+wZ717o1i4GrkvXGhXDMu6cD69RlfaMh+u7lgFZgQIMT41MB7m5xYDvYS3p9EPT jisA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b="f/djVHaA"; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id p10-v6si257777wrq.50.2018.08.16.14.23.27; Thu, 16 Aug 2018 14:23:28 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b="f/djVHaA"; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 13C8968A782; Fri, 17 Aug 2018 00:22:53 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-qk0-f174.google.com (mail-qk0-f174.google.com [209.85.220.174]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id E3F9B68A6B7 for ; Fri, 17 Aug 2018 00:22:49 +0300 (EEST) Received: by mail-qk0-f174.google.com with SMTP id b66-v6so4876914qkj.1 for ; Thu, 16 Aug 2018 14:23:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id:in-reply-to:references; bh=lglvFFchLrByfun34OnfWHHPQYwjFBzP8E2R3r4xBUg=; b=f/djVHaASeIBjswBrGsnzKutRgXoCMJwt9enUSBeQyMX7QQFHLpKLFUWi5nwcgi/rp wUK+f6P/+pHg+wVAVB1ZD+z1Du5FzdLZ0MdEn7ParbMRsTMBogxpRviYCtcvdzRziUAr RlhmjgOpMH2stDtiMRoW7PP0KTLuDSSvySVddRJ8yHSQ+HS22QwugHK6fWepiYoeRRhO sRnUup0FopltWHPNUnWYY25HwhDO9qjJXz1Cuxax8jJwiLiUH18Lru4wpmJDqTpWXjzz epkiJoG/pqC/n+aqq7h9lJJVHNvXx83LNYAmXefbo0gwmyRyQK07vB5VuGax6TwJ0Uq9 Euog== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=lglvFFchLrByfun34OnfWHHPQYwjFBzP8E2R3r4xBUg=; b=iaL0v5jnBWFr6P+Bcpchun7VWQX/MGBUoy4KkR/uFc/51YNMjL8TdNKtjQluOxoHuO ODxMck+z8cbhplT6MtxxTA/B/Mg9p+o6Dq+d9LMDLz0/r/Icgkogt4EG+q2HaCfehvGT 6wbbWrIB+3IaJPFG4l0foO8ZEH1ovaD58aSJx2agOz6r/QGVJzK8LkYo0glS4u6X/P8o UVyZE0x6mQD2rQHNcsSw04iFdeOHQlNz5QYO+gebNO05ZIPaMqzd1NyicXghsiqNeYEN niLjo107FyhHRa5PGMqXlZZDnXE+GmrVjUDTvxxO6FvduQqymiVC/YMfo3fORiumdfVK 763A== X-Gm-Message-State: AOUpUlFgeVhY6GpPWh/mleLftg2Oq87BiNoo8zw/0EplR33BSg6ImyDm wf0PjI0An1GTdlQnIkRD8ZI2zW/4 X-Received: by 2002:a37:1866:: with SMTP id j99-v6mr514000qkh.344.1534454593579; Thu, 16 Aug 2018 14:23:13 -0700 (PDT) Received: from localhost.localdomain ([181.231.186.237]) by smtp.gmail.com with ESMTPSA id 134-v6sm194624qke.94.2018.08.16.14.23.12 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 16 Aug 2018 14:23:13 -0700 (PDT) From: James Almer To: ffmpeg-devel@ffmpeg.org Date: Thu, 16 Aug 2018 18:22:19 -0300 Message-Id: <20180816212219.10596-2-jamrial@gmail.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: References: Subject: [FFmpeg-devel] [PATCH 2/2 v2] avformat/av1: update ff_isom_write_av1c() to the latest revision of the spec X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" This will get ISOBMFF and Matroska up to date with the revised AV1 Codec Configuration Box spec. For now keep propagating raw OBUs as extradata until all libavcodec modules are adapted to handle AV1CodecConfigurationRecordv1 formatted extradata. Signed-off-by: James Almer --- New in v2: - AV1CodecConfigurationBox is now a Box instead of a FullBox. - Trailing bits are now taken into account when checking that the entire Sequence Header was parsed. "All libavcodec modules" means basically the upcoming AV1 decoder and CBS implementation. I prefer to wait until those are posted/pushed in order to better coordinate how to do this. libavformat/av1.c | 247 ++++++++++++++++++++++++++++++++++++++ libavformat/matroskadec.c | 4 + libavformat/mov.c | 18 ++- libavformat/movenc.c | 3 - 4 files changed, 259 insertions(+), 13 deletions(-) diff --git a/libavformat/av1.c b/libavformat/av1.c index db7f281fc7..ded990fd36 100644 --- a/libavformat/av1.c +++ b/libavformat/av1.c @@ -22,6 +22,8 @@ #include "libavutil/mem.h" #include "libavcodec/av1.h" #include "libavcodec/av1_parse.h" +#include "libavcodec/profiles.h" +#include "libavcodec/put_bits.h" #include "av1.h" #include "avio.h" @@ -73,9 +75,233 @@ int ff_av1_filter_obus_buf(const uint8_t *buf, uint8_t **out, int *size) return ret; } +typedef struct AV1SequenceParameters { + uint8_t seq_profile; + uint8_t seq_level_idx_0; + uint8_t seq_tier_0; + uint8_t high_bitdepth; + uint8_t twelve_bit; + uint8_t monochrome; + uint8_t chroma_subsampling_x; + uint8_t chroma_subsampling_y; + uint8_t chroma_sample_position; +} AV1SequenceParameters; + +static inline void uvlc(GetBitContext *gb) +{ + int leading_zeros = 0; + + while (1) { + if (get_bits1(gb)) + break; + leading_zeros++; + } + + if (leading_zeros >= 32) + return; + + skip_bits_long(gb, leading_zeros); +} + +static int parse_color_config(AV1SequenceParameters *seq_params, GetBitContext *gb) +{ + int color_primaries, transfer_characteristics, matrix_coefficients; + + seq_params->high_bitdepth = get_bits1(gb); + if (seq_params->seq_profile == FF_PROFILE_AV1_PROFESSIONAL && seq_params->high_bitdepth) + seq_params->twelve_bit = get_bits1(gb); + else + seq_params->twelve_bit = 0; + + if (seq_params->seq_profile == FF_PROFILE_AV1_HIGH) + seq_params->monochrome = 0; + else + seq_params->monochrome = get_bits1(gb); + + if (get_bits1(gb)) { // color_description_present_flag + color_primaries = get_bits(gb, 8); + transfer_characteristics = get_bits(gb, 8); + matrix_coefficients = get_bits(gb, 8); + } else { + color_primaries = AVCOL_PRI_UNSPECIFIED; + transfer_characteristics = AVCOL_TRC_UNSPECIFIED; + matrix_coefficients = AVCOL_SPC_UNSPECIFIED; + } + + if (seq_params->monochrome) { + skip_bits1(gb); // color_range + seq_params->chroma_subsampling_x = 1; + seq_params->chroma_subsampling_y = 1; + seq_params->chroma_sample_position = 0; + return 0; + } else if (color_primaries == AVCOL_PRI_BT709 && + transfer_characteristics == AVCOL_TRC_IEC61966_2_1 && + matrix_coefficients == AVCOL_SPC_RGB) { + seq_params->chroma_subsampling_x = 0; + seq_params->chroma_subsampling_y = 0; + } else { + skip_bits1(gb); // color_range + + if (seq_params->seq_profile == FF_PROFILE_AV1_MAIN) { + seq_params->chroma_subsampling_x = 1; + seq_params->chroma_subsampling_y = 1; + } else if (seq_params->seq_profile == FF_PROFILE_AV1_HIGH) { + seq_params->chroma_subsampling_x = 0; + seq_params->chroma_subsampling_y = 0; + } else { + if (seq_params->twelve_bit) { + seq_params->chroma_subsampling_x = get_bits1(gb); + if (seq_params->chroma_subsampling_x) + seq_params->chroma_subsampling_y = get_bits1(gb); + else + seq_params->chroma_subsampling_y = 0; + } else { + seq_params->chroma_subsampling_x = 1; + seq_params->chroma_subsampling_y = 0; + } + } + if (seq_params->chroma_subsampling_x && seq_params->chroma_subsampling_y) + seq_params->chroma_sample_position = get_bits(gb, 2); + } + + skip_bits1(gb); // separate_uv_delta_q + + return 0; +} + +static int parse_sequence_header(AV1SequenceParameters *seq_params, const uint8_t *buf, int size) +{ + GetBitContext gb; + int reduced_still_picture_header; + int frame_width_bits_minus_1, frame_height_bits_minus_1; + int size_bits, ret; + + size_bits = get_obu_bit_length(buf, size, AV1_OBU_SEQUENCE_HEADER); + if (size_bits < 0) + return size_bits; + + ret = init_get_bits(&gb, buf, size_bits); + if (ret < 0) + return ret; + + seq_params->seq_profile = get_bits(&gb, 3); + + skip_bits1(&gb); // still_picture + reduced_still_picture_header = get_bits1(&gb); + + if (reduced_still_picture_header) { + seq_params->seq_level_idx_0 = get_bits(&gb, 5); + seq_params->seq_tier_0 = 0; + } else { + int initial_display_delay_present_flag, operating_points_cnt_minus_1; + int decoder_model_info_present_flag, buffer_delay_length_minus_1; + + if (get_bits1(&gb)) { // timing_info_present_flag + skip_bits_long(&gb, 32); // num_units_in_display_tick + skip_bits_long(&gb, 32); // time_scale + + if (get_bits1(&gb)) // equal_picture_interval + uvlc(&gb); // num_ticks_per_picture_minus_1 + + decoder_model_info_present_flag = get_bits1(&gb); + if (decoder_model_info_present_flag) { + buffer_delay_length_minus_1 = get_bits(&gb, 5); + skip_bits_long(&gb, 32); // num_units_in_decoding_tick + skip_bits(&gb, 10); // buffer_removal_time_length_minus_1 (5) + // frame_presentation_time_length_minus_1 (5) + } + } else + decoder_model_info_present_flag = 0; + + initial_display_delay_present_flag = get_bits1(&gb); + + operating_points_cnt_minus_1 = get_bits(&gb, 5); + for (int i = 0; i <= operating_points_cnt_minus_1; i++) { + int seq_level_idx, seq_tier; + + skip_bits(&gb, 12); // operating_point_idc + seq_level_idx = get_bits(&gb, 5); + + if (seq_level_idx > 7) + seq_tier = get_bits1(&gb); + else + seq_tier = 0; + + if (decoder_model_info_present_flag) { + if (get_bits1(&gb)) { // decoder_model_present_for_this_op + skip_bits_long(&gb, buffer_delay_length_minus_1 + 1); // decoder_buffer_delay + skip_bits_long(&gb, buffer_delay_length_minus_1 + 1); // encoder_buffer_delay + skip_bits1(&gb); // low_delay_mode_flag + } + } + + if (initial_display_delay_present_flag) { + if (get_bits1(&gb)) // initial_display_delay_present_for_this_op + skip_bits(&gb, 4); // initial_display_delay_minus_1 + } + + if (i == 0) { + seq_params->seq_level_idx_0 = seq_level_idx; + seq_params->seq_tier_0 = seq_tier; + } + } + } + + frame_width_bits_minus_1 = get_bits(&gb, 4); + frame_height_bits_minus_1 = get_bits(&gb, 4); + + skip_bits(&gb, frame_width_bits_minus_1 + 1); // max_frame_width_minus_1 + skip_bits(&gb, frame_height_bits_minus_1 + 1); // max_frame_height_minus_1 + + if (!reduced_still_picture_header) { + if (get_bits1(&gb)) // frame_id_numbers_present_flag + skip_bits(&gb, 7); // delta_frame_id_length_minus_2 (4), additional_frame_id_length_minus_1 (3) + } + + skip_bits(&gb, 3); // use_128x128_superblock (1), enable_filter_intra (1), enable_intra_edge_filter (1) + + if (!reduced_still_picture_header) { + int enable_order_hint, seq_force_screen_content_tools; + + skip_bits(&gb, 4); // enable_intraintra_compound (1), enable_masked_compound (1) + // enable_warped_motion (1), enable_dual_filter (1) + + enable_order_hint = get_bits1(&gb); + if (enable_order_hint) + skip_bits(&gb, 2); // enable_jnt_comp (1), enable_ref_frame_mvs (1) + + if (get_bits1(&gb)) // seq_choose_screen_content_tools + seq_force_screen_content_tools = 2; + else + seq_force_screen_content_tools = get_bits1(&gb); + + if (seq_force_screen_content_tools) { + if (!get_bits1(&gb)) // seq_choose_integer_mv + skip_bits1(&gb); // seq_force_integer_mv + } + + if (enable_order_hint) + skip_bits(&gb, 3); // order_hint_bits_minus_1 + } + + skip_bits(&gb, 3); // enable_superres (1), enable_cdef (1), enable_restoration (1) + + parse_color_config(seq_params, &gb); + + skip_bits1(&gb); // film_grain_params_present + + if (get_bits_left(&gb)) + return AVERROR_INVALIDDATA; + + return 0; +} + int ff_isom_write_av1c(AVIOContext *pb, const uint8_t *buf, int size) { AVIOContext *seq_pb = NULL, *meta_pb = NULL; + AV1SequenceParameters seq_params; + PutBitContext pbc; + uint8_t header[4]; uint8_t *seq = NULL, *meta = NULL; int64_t obu_size; int start_pos, type, temporal_id, spatial_id; @@ -106,6 +332,10 @@ int ff_isom_write_av1c(AVIOContext *pb, const uint8_t *buf, int size) ret = AVERROR_INVALIDDATA; goto fail; } + ret = parse_sequence_header(&seq_params, buf + start_pos, obu_size); + if (ret < 0) + goto fail; + avio_write(seq_pb, buf, len); break; case AV1_OBU_METADATA: @@ -127,6 +357,23 @@ int ff_isom_write_av1c(AVIOContext *pb, const uint8_t *buf, int size) ret = AVERROR_INVALIDDATA; goto fail; } + + init_put_bits(&pbc, header, sizeof(header)); + + put_bits(&pbc, 1, 1); // marker + put_bits(&pbc, 7, 1); // version + put_bits(&pbc, 3, seq_params.seq_profile); + put_bits(&pbc, 5, seq_params.seq_level_idx_0); + put_bits(&pbc, 1, seq_params.seq_tier_0); + put_bits(&pbc, 1, seq_params.high_bitdepth); + put_bits(&pbc, 1, seq_params.twelve_bit); + put_bits(&pbc, 1, seq_params.monochrome); + put_bits(&pbc, 1, seq_params.chroma_subsampling_x); + put_bits(&pbc, 1, seq_params.chroma_subsampling_y); + put_bits(&pbc, 2, seq_params.chroma_sample_position); + flush_put_bits(&pbc); + + avio_write(pb, header, sizeof(header)); avio_write(pb, seq, seq_size); meta_size = avio_close_dyn_buf(meta_pb, &meta); diff --git a/libavformat/matroskadec.c b/libavformat/matroskadec.c index 1ded431b80..e6793988e1 100644 --- a/libavformat/matroskadec.c +++ b/libavformat/matroskadec.c @@ -2421,6 +2421,10 @@ static int matroska_parse_tracks(AVFormatContext *s) /* we don't need any value stored in CodecPrivate. make sure that it's not exported as extradata. */ track->codec_priv.size = 0; + } else if (codec_id == AV_CODEC_ID_AV1 && track->codec_priv.size) { + /* For now, propagate only the OBUs, if any. Once libavcodec is + updated to handle isobmff style extradata this can be removed. */ + extradata_offset = 4; } track->codec_priv.size -= extradata_offset; diff --git a/libavformat/mov.c b/libavformat/mov.c index c863047d79..1bd7d7e483 100644 --- a/libavformat/mov.c +++ b/libavformat/mov.c @@ -5190,27 +5190,25 @@ static int mov_read_tmcd(MOVContext *c, AVIOContext *pb, MOVAtom atom) static int mov_read_av1c(MOVContext *c, AVIOContext *pb, MOVAtom atom) { AVStream *st; - int ret, version; + int ret; if (c->fc->nb_streams < 1) return 0; st = c->fc->streams[c->fc->nb_streams - 1]; - if (atom.size < 5) { + if (atom.size < 4) { av_log(c->fc, AV_LOG_ERROR, "Empty AV1 Codec Configuration Box\n"); return AVERROR_INVALIDDATA; } - version = avio_r8(pb); - if (version != 0) { - av_log(c->fc, AV_LOG_WARNING, "Unknown AV1 Codec Configuration Box version %d\n", version); - return 0; - } - avio_skip(pb, 3); /* flags */ + /* For now, propagate only the OBUs, if any. Once libavcodec is + updated to handle isobmff style extradata this can be removed. */ + avio_skip(pb, 4); - avio_skip(pb, 1); /* reserved, initial_presentation_delay_present, initial_presentation_delay_minus_one */ + if (atom.size == 4) + return 0; - ret = ff_get_extradata(c->fc, st->codecpar, pb, atom.size - 5); + ret = ff_get_extradata(c->fc, st->codecpar, pb, atom.size - 4); if (ret < 0) return ret; diff --git a/libavformat/movenc.c b/libavformat/movenc.c index d530f40cab..8ad7026741 100644 --- a/libavformat/movenc.c +++ b/libavformat/movenc.c @@ -1170,9 +1170,6 @@ static int mov_write_av1c_tag(AVIOContext *pb, MOVTrack *track) avio_wb32(pb, 0); ffio_wfourcc(pb, "av1C"); - avio_w8(pb, 0); /* version */ - avio_wb24(pb, 0); /* flags */ - avio_w8(pb, 0); /* reserved (3), initial_presentation_delay_present (1), initial_presentation_delay_minus_one/reserved (4) */ ff_isom_write_av1c(pb, track->vos_data, track->vos_len); return update_size(pb, pos); }