From patchwork Wed Jan 8 23:00:04 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wonkap Jang X-Patchwork-Id: 17262 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 5842844BB68 for ; Thu, 9 Jan 2020 01:00:17 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 213B668A8F1; Thu, 9 Jan 2020 01:00:17 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-vs1-f74.google.com (mail-vs1-f74.google.com [209.85.217.74]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 8A04D680AE5 for ; Thu, 9 Jan 2020 01:00:10 +0200 (EET) Received: by mail-vs1-f74.google.com with SMTP id l1so836262vsr.21 for ; Wed, 08 Jan 2020 15:00:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:message-id:mime-version:subject:from:to; bh=biZJd7KEh2RgkqKzO1GQWjnserSvaw7AsjKG+EVA3sU=; b=IkoU07esG2ikhntDlDYrAtkH7b8yjmU5ZBpdne1EY1jwyztld8o6HPYNIrrfg5c8Qv Z23ckeiJLBKgXgXNyleepgrAQ4v7Z7gmwMzscY5jYrAcH1TUwIpxcyaI1QeeXpUgKWi4 eG5RbOFhnMYc2GU1OWG41Z8+tv81Qg4VjhTsW++WRH9dW4nm1Q6b7/bys6ZVI0NS6P0u 3Ut2UDzILll5Fxk7GunxhN8vKD+ZzufWCkSDxKN+sSHigq9sYrfihhnVoVIyvqZZ/JUx n/aeCID+Cydgp1DHVCwlcSMTuF4kl/4IHPCICvo29fQJVwz+F649mssIiTMWSIwsdmVF jlrA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:message-id:mime-version:subject:from:to; bh=biZJd7KEh2RgkqKzO1GQWjnserSvaw7AsjKG+EVA3sU=; b=t8O5BvaOeBXXVl/pKCZ7+fw7+sjyNMNmrnEJPjbfm3W1CQFDnMHEmHV04GoSspkwMH EUpJvg60UN8HlfR84pjCWUUC4AkJibttne1usBcaWePBbAyz5mHNWtf1APhbgfr0XOk5 5CnLqffjSTaSoMje+8KR+WXrMpg5/LoM3h5nm9P7eNWZxGVuMegZHbg0R+um1QJ8F38+ oXHf/ZpYc9ZUVrH3uGg/MD8OiS9/rVlB1Cayxqtb3pOtbcUh61OYzIDUqjkJKoXujN19 PZEbE3uEQuJfr9svF02wXzR+ojBppfqxK9pkBn0g7Ydr8er73TnLvH2rTZhVQ8lv5i7v g5IQ== X-Gm-Message-State: APjAAAXKu4UsWylmmITadTd6or2J5aAQo6dHEUHOAsP+xBic1ZNI2nx/ 6IomTZvY+VMP3N25ee/Twh2J+J5KNi/Y9fDXIq8kAygeoOl7EVkeERQN9Up/uAOYMpM18jS0xdV RhVN3xgX1g6VDcjKuTiHcwQdCY/q7eMDoco2zcaxX82kCJWUuitTWxKuidZpxWn4= X-Google-Smtp-Source: APXvYqwnaGmkoBI/umwCdwNsrRJMGWu0UlMyKwKI7PmSR4IhZWxo+SZ4lwJXdtrSYfXVQ45fRjnblkXVlWw= X-Received: by 2002:a67:3187:: with SMTP id x129mr4444863vsx.23.1578524408829; Wed, 08 Jan 2020 15:00:08 -0800 (PST) Date: Wed, 8 Jan 2020 15:00:04 -0800 Message-Id: <20200108230006.169044-1-wonkap@google.com> Mime-Version: 1.0 X-Mailer: git-send-email 2.25.0.rc1.283.g88dfdc4193-goog From: Wonkap Jang To: ffmpeg-devel@ffmpeg.org Subject: [FFmpeg-devel] [PATCH v3 1/3] avcodec/libvpxenc: add VP9 temporal scalability encoding option X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" This commit reuses the configuration options for VP8 that enables temporal scalability for VP9. It also adds a way to enable three preset temporal structures (refer to the documentation for more detail) that can be used in offline encoding. --- libavcodec/libvpxenc.c | 251 +++++++++++++++++++++++++++++++++++++---- 1 file changed, 228 insertions(+), 23 deletions(-) diff --git a/libavcodec/libvpxenc.c b/libavcodec/libvpxenc.c index 0b8a070304..14cc1e7158 100644 --- a/libavcodec/libvpxenc.c +++ b/libavcodec/libvpxenc.c @@ -100,7 +100,9 @@ typedef struct VPxEncoderContext { int rc_undershoot_pct; int rc_overshoot_pct; - AVDictionary *vp8_ts_parameters; + AVDictionary *vpx_ts_parameters; + int *ts_layer_flags; + int current_temporal_idx; // VP9-only int lossless; @@ -137,6 +139,7 @@ static const char *const ctlidstr[] = { [VP8E_SET_CQ_LEVEL] = "VP8E_SET_CQ_LEVEL", [VP8E_SET_MAX_INTRA_BITRATE_PCT] = "VP8E_SET_MAX_INTRA_BITRATE_PCT", [VP8E_SET_SHARPNESS] = "VP8E_SET_SHARPNESS", + [VP8E_SET_TEMPORAL_LAYER_ID] = "VP8E_SET_TEMPORAL_LAYER_ID", #if CONFIG_LIBVPX_VP9_ENCODER [VP9E_SET_LOSSLESS] = "VP9E_SET_LOSSLESS", [VP9E_SET_TILE_COLUMNS] = "VP9E_SET_TILE_COLUMNS", @@ -144,6 +147,11 @@ static const char *const ctlidstr[] = { [VP9E_SET_FRAME_PARALLEL_DECODING] = "VP9E_SET_FRAME_PARALLEL_DECODING", [VP9E_SET_AQ_MODE] = "VP9E_SET_AQ_MODE", [VP9E_SET_COLOR_SPACE] = "VP9E_SET_COLOR_SPACE", + [VP9E_SET_SVC_LAYER_ID] = "VP9E_SET_SVC_LAYER_ID", +#if VPX_ENCODER_ABI_VERSION >= 12 + [VP9E_SET_SVC_PARAMETERS] = "VP9E_SET_SVC_PARAMETERS", +#endif + [VP9E_SET_SVC] = "VP9E_SET_SVC", #if VPX_ENCODER_ABI_VERSION >= 11 [VP9E_SET_COLOR_RANGE] = "VP9E_SET_COLOR_RANGE", #endif @@ -223,8 +231,16 @@ static av_cold void dump_enc_cfg(AVCodecContext *avctx, " %*s%u\n", width, "ts_number_layers:", cfg->ts_number_layers); av_log(avctx, level, "\n %*s", width, "ts_target_bitrate:"); - for (i = 0; i < VPX_TS_MAX_LAYERS; i++) - av_log(avctx, level, "%u ", cfg->ts_target_bitrate[i]); + if (avctx->codec_id == AV_CODEC_ID_VP8) { + for (i = 0; i < VPX_TS_MAX_LAYERS; i++) + av_log(avctx, level, "%u ", cfg->ts_target_bitrate[i]); + } +#if (VPX_ENCODER_ABI_VERSION >= 12) && CONFIG_LIBVPX_VP9_ENCODER + if (avctx->codec_id == AV_CODEC_ID_VP9) { + for (i = 0; i < VPX_TS_MAX_LAYERS; i++) + av_log(avctx, level, "%u ", cfg->layer_target_bitrate[i]); + } +#endif av_log(avctx, level, "\n"); av_log(avctx, level, "\n %*s", width, "ts_rate_decimator:"); @@ -346,6 +362,8 @@ static av_cold int vpx_free(AVCodecContext *avctx) } #endif + av_freep(&ctx->ts_layer_flags); + vpx_codec_destroy(&ctx->encoder); if (ctx->is_alpha) { vpx_codec_destroy(&ctx->encoder_alpha); @@ -370,23 +388,154 @@ static void vp8_ts_parse_int_array(int *dest, char *value, size_t value_len, int } } -static int vp8_ts_param_parse(struct vpx_codec_enc_cfg *enccfg, char *key, char *value) +static void set_temporal_layer_pattern(int layering_mode, + vpx_codec_enc_cfg_t *cfg, + int *layer_flags, + int *flag_periodicity) +{ + switch (layering_mode) { + case 2: { + /** + * 2-layers, 2-frame period. + */ + int ids[2] = { 0, 1 }; + cfg->ts_periodicity = 2; + *flag_periodicity = 2; + cfg->ts_number_layers = 2; + cfg->ts_rate_decimator[0] = 2; + cfg->ts_rate_decimator[1] = 1; + memcpy(cfg->ts_layer_id, ids, sizeof(ids)); + + layer_flags[0] = + VP8_EFLAG_NO_REF_GF | VP8_EFLAG_NO_REF_ARF | + VP8_EFLAG_NO_UPD_GF | VP8_EFLAG_NO_UPD_ARF; + + layer_flags[1] = + VP8_EFLAG_NO_UPD_ARF | VP8_EFLAG_NO_UPD_GF | + VP8_EFLAG_NO_UPD_LAST | + VP8_EFLAG_NO_REF_ARF | VP8_EFLAG_NO_REF_GF; + break; + } + case 3: { + /** + * 3-layers structure with one reference frame. + * This works same as temporal_layering_mode 3. + * + * 3-layers, 4-frame period. + */ + int ids[4] = { 0, 2, 1, 2 }; + cfg->ts_periodicity = 4; + *flag_periodicity = 4; + cfg->ts_number_layers = 3; + cfg->ts_rate_decimator[0] = 4; + cfg->ts_rate_decimator[1] = 2; + cfg->ts_rate_decimator[2] = 1; + memcpy(cfg->ts_layer_id, ids, sizeof(ids)); + + /** + * 0=L, 1=GF, 2=ARF, + * Intra-layer prediction disabled. + */ + layer_flags[0] = + VP8_EFLAG_NO_REF_GF | VP8_EFLAG_NO_REF_ARF | + VP8_EFLAG_NO_UPD_GF | VP8_EFLAG_NO_UPD_ARF; + layer_flags[2] = + VP8_EFLAG_NO_REF_GF | VP8_EFLAG_NO_REF_ARF | + VP8_EFLAG_NO_UPD_ARF | VP8_EFLAG_NO_UPD_LAST; + layer_flags[1] = + VP8_EFLAG_NO_REF_GF | VP8_EFLAG_NO_REF_ARF | + VP8_EFLAG_NO_UPD_LAST | VP8_EFLAG_NO_UPD_GF | + VP8_EFLAG_NO_UPD_ARF; + layer_flags[3] = + VP8_EFLAG_NO_REF_LAST | VP8_EFLAG_NO_REF_ARF | + VP8_EFLAG_NO_UPD_LAST | VP8_EFLAG_NO_UPD_GF | + VP8_EFLAG_NO_UPD_ARF; + break; + } + case 4: { + /** + * 3-layers structure. + * added dependency between the two TL2 frames (on top of case 3). + * 3-layers, 4-frame period. + */ + int ids[4] = { 0, 2, 1, 2 }; + cfg->ts_periodicity = 4; + *flag_periodicity = 4; + cfg->ts_number_layers = 3; + cfg->ts_rate_decimator[0] = 4; + cfg->ts_rate_decimator[1] = 2; + cfg->ts_rate_decimator[2] = 1; + memcpy(cfg->ts_layer_id, ids, sizeof(ids)); + + /** + * 0=L, 1=GF, 2=ARF, Intra-layer prediction disabled. + */ + layer_flags[0] = + VP8_EFLAG_NO_REF_GF | VP8_EFLAG_NO_REF_ARF | + VP8_EFLAG_NO_UPD_GF | VP8_EFLAG_NO_UPD_ARF; + layer_flags[2] = + VP8_EFLAG_NO_REF_GF | VP8_EFLAG_NO_REF_ARF | + VP8_EFLAG_NO_UPD_ARF | VP8_EFLAG_NO_UPD_LAST; + layer_flags[1] = + VP8_EFLAG_NO_REF_GF | VP8_EFLAG_NO_REF_ARF | + VP8_EFLAG_NO_UPD_LAST | VP8_EFLAG_NO_UPD_GF; + layer_flags[3] = + VP8_EFLAG_NO_REF_LAST | + VP8_EFLAG_NO_UPD_LAST | VP8_EFLAG_NO_UPD_GF | + VP8_EFLAG_NO_UPD_ARF; + break; + } + default: + /** + * do not change the layer_flags or the flag_periodicity in this case; + * it might be that the code is using external flags to be used. + */ + break; + + } +} + +static int vpx_ts_param_parse(VPxContext *ctx, + struct vpx_codec_enc_cfg *enccfg, + char *key, char *value, + const enum AVCodecID codec_id) { size_t value_len = strlen(value); + int ts_layering_mode = 0; if (!value_len) return -1; if (!strcmp(key, "ts_number_layers")) enccfg->ts_number_layers = strtoul(value, &value, 10); - else if (!strcmp(key, "ts_target_bitrate")) - vp8_ts_parse_int_array(enccfg->ts_target_bitrate, value, value_len, VPX_TS_MAX_LAYERS); - else if (!strcmp(key, "ts_rate_decimator")) - vp8_ts_parse_int_array(enccfg->ts_rate_decimator, value, value_len, VPX_TS_MAX_LAYERS); - else if (!strcmp(key, "ts_periodicity")) + else if (!strcmp(key, "ts_target_bitrate")) { + if (codec_id == AV_CODEC_ID_VP8) + vp8_ts_parse_int_array(enccfg->ts_target_bitrate, value, value_len, VPX_TS_MAX_LAYERS); +#if (VPX_ENCODER_ABI_VERSION >= 12) && CONFIG_LIBVPX_VP9_ENCODER + if (codec_id == AV_CODEC_ID_VP9) + vp8_ts_parse_int_array(enccfg->layer_target_bitrate, value, value_len, VPX_TS_MAX_LAYERS); +#endif + } else if (!strcmp(key, "ts_rate_decimator")) { + vp8_ts_parse_int_array(enccfg->ts_rate_decimator, value, value_len, VPX_TS_MAX_LAYERS); + } else if (!strcmp(key, "ts_periodicity")) { enccfg->ts_periodicity = strtoul(value, &value, 10); - else if (!strcmp(key, "ts_layer_id")) + } else if (!strcmp(key, "ts_layer_id")) { vp8_ts_parse_int_array(enccfg->ts_layer_id, value, value_len, VPX_TS_MAX_PERIODICITY); + } else if (!strcmp(key, "ts_layering_mode")) { + /* option for pre-defined temporal structures in function set_temporal_layer_pattern. */ + ts_layering_mode = strtoul(value, &value, 4); + } + +#if (VPX_ENCODER_ABI_VERSION >= 12) && CONFIG_LIBVPX_VP9_ENCODER + enccfg->temporal_layering_mode = 1; // only bypass mode is being supported for now. + enccfg->ss_number_layers = 1; // making sure the spatial scalability is off. Will support this in future. +#endif + if (ts_layering_mode) { + // make sure the ts_layering_mode comes at the end of the ts_parameter string to ensure that + // correct configuration is done. + ctx->ts_layer_flags = av_malloc(sizeof(int) * VPX_TS_MAX_PERIODICITY); + set_temporal_layer_pattern(ts_layering_mode, enccfg, ctx->ts_layer_flags, &enccfg->ts_periodicity); + } return 0; } @@ -590,7 +739,9 @@ static av_cold int vpx_init(AVCodecContext *avctx, vpx_img_fmt_t img_fmt = VPX_IMG_FMT_I420; #if CONFIG_LIBVPX_VP9_ENCODER vpx_codec_caps_t codec_caps = vpx_codec_get_caps(iface); + vpx_svc_extra_cfg_t svc_params; #endif + AVDictionaryEntry* en = NULL; av_log(avctx, AV_LOG_INFO, "%s\n", vpx_codec_version_str()); av_log(avctx, AV_LOG_VERBOSE, "%s\n", vpx_codec_build_config()); @@ -611,8 +762,8 @@ static av_cold int vpx_init(AVCodecContext *avctx, } #endif - if(!avctx->bit_rate) - if(avctx->rc_max_rate || avctx->rc_buffer_size || avctx->rc_initial_buffer_occupancy) { + if (!avctx->bit_rate) + if (avctx->rc_max_rate || avctx->rc_buffer_size || avctx->rc_initial_buffer_occupancy) { av_log( avctx, AV_LOG_ERROR, "Rate control parameters set without a bitrate\n"); return AVERROR(EINVAL); } @@ -648,6 +799,9 @@ static av_cold int vpx_init(AVCodecContext *avctx, if (avctx->bit_rate) { enccfg.rc_target_bitrate = av_rescale_rnd(avctx->bit_rate, 1, 1000, AV_ROUND_NEAR_INF); +#if CONFIG_LIBVPX_VP9_ENCODER + enccfg.ss_target_bitrate[0] = enccfg.rc_target_bitrate; +#endif } else { // Set bitrate to default value. Also sets CRF to default if needed. set_vpx_defaults(avctx, &enccfg); @@ -757,14 +911,11 @@ FF_ENABLE_DEPRECATION_WARNINGS enccfg.g_error_resilient = ctx->error_resilient || ctx->flags & VP8F_ERROR_RESILIENT; - if (CONFIG_LIBVPX_VP8_ENCODER && avctx->codec_id == AV_CODEC_ID_VP8) { - AVDictionaryEntry* en = NULL; - while ((en = av_dict_get(ctx->vp8_ts_parameters, "", en, AV_DICT_IGNORE_SUFFIX))) { - if (vp8_ts_param_parse(&enccfg, en->key, en->value) < 0) - av_log(avctx, AV_LOG_WARNING, - "Error parsing option '%s = %s'.\n", - en->key, en->value); - } + while ((en = av_dict_get(ctx->vpx_ts_parameters, "", en, AV_DICT_IGNORE_SUFFIX))) { + if (vpx_ts_param_parse(ctx, &enccfg, en->key, en->value, avctx->codec_id) < 0) + av_log(avctx, AV_LOG_WARNING, + "Error parsing option '%s = %s'.\n", + en->key, en->value); } dump_enc_cfg(avctx, &enccfg); @@ -774,7 +925,21 @@ FF_ENABLE_DEPRECATION_WARNINGS log_encoder_error(avctx, "Failed to initialize encoder"); return AVERROR(EINVAL); } - +#if CONFIG_LIBVPX_VP9_ENCODER + if (avctx->codec_id == AV_CODEC_ID_VP9 && enccfg.ts_number_layers > 1) { + memset(&svc_params, 0, sizeof(svc_params)); + for (int i = 0; i < enccfg.ts_number_layers; ++i) { + svc_params.max_quantizers[i] = enccfg.rc_max_quantizer; + svc_params.min_quantizers[i] = enccfg.rc_min_quantizer; + } + svc_params.scaling_factor_num[0] = enccfg.g_h; + svc_params.scaling_factor_den[0] = enccfg.g_h; +#if VPX_ENCODER_ABI_VERSION >= 12 + codecctl_int(avctx, VP9E_SET_SVC, 1); + codecctl_intp(avctx, VP9E_SET_SVC_PARAMETERS, (int *) &svc_params); +#endif + } +#endif if (ctx->is_alpha) { enccfg_alpha = enccfg; res = vpx_codec_enc_init(&ctx->encoder_alpha, iface, &enccfg_alpha, flags); @@ -1321,6 +1486,9 @@ static int vpx_encode(AVCodecContext *avctx, AVPacket *pkt, int64_t timestamp = 0; int res, coded_size; vpx_enc_frame_flags_t flags = 0; + const struct vpx_codec_enc_cfg *enccfg = ctx->encoder.config.enc; + vpx_svc_layer_id_t layer_id; + int layer_id_valid = 0; if (frame) { const AVFrameSideData *sd = av_frame_get_side_data(frame, AV_FRAME_DATA_REGIONS_OF_INTEREST); @@ -1368,6 +1536,42 @@ static int vpx_encode(AVCodecContext *avctx, AVPacket *pkt, } } + // this is for encoding with preset temporal layering patterns defined in + // set_temporal_layer_pattern function. + if (enccfg->ts_number_layers > 1 && ctx->ts_layer_flags) { + if (flags & VPX_EFLAG_FORCE_KF) { + // keyframe, reset temporal layering. + ctx->current_temporal_idx = 0; + flags = VPX_EFLAG_FORCE_KF; + } else { + flags = 0; + } + + /* get the flags from the temporal layer configuration. */ + flags |= ctx->ts_layer_flags[ctx->current_temporal_idx]; + + memset(&layer_id, 0, sizeof(layer_id)); +#if VPX_ENCODER_ABI_VERSION >= 12 + layer_id.spatial_layer_id = 0; +#endif + layer_id.temporal_layer_id = enccfg->ts_layer_id[ctx->current_temporal_idx]; +#ifdef VPX_CTRL_VP9E_SET_MAX_INTER_BITRATE_PCT + layer_id.temporal_layer_id_per_spatial[0] = layer_id.temporal_layer_id; +#endif + layer_id_valid = 1; + } + + if (layer_id_valid) { + if (avctx->codec_id == AV_CODEC_ID_VP8) { + codecctl_int(avctx, VP8E_SET_TEMPORAL_LAYER_ID, layer_id.temporal_layer_id); + } +#if CONFIG_LIBVPX_VP9_ENCODER && VPX_ENCODER_ABI_VERSION >= 12 + else if (avctx->codec_id == AV_CODEC_ID_VP9) { + codecctl_intp(avctx, VP9E_SET_SVC_LAYER_ID, (int *) &layer_id); + } +#endif + } + res = vpx_codec_encode(&ctx->encoder, rawimg, timestamp, avctx->ticks_per_frame, flags, ctx->deadline); if (res != VPX_CODEC_OK) { @@ -1397,6 +1601,8 @@ static int vpx_encode(AVCodecContext *avctx, AVPacket *pkt, } av_base64_encode(avctx->stats_out, b64_size, ctx->twopass_stats.buf, ctx->twopass_stats.sz); + } else if (enccfg->ts_number_layers > 1 && ctx->ts_layer_flags) { + ctx->current_temporal_idx = (ctx->current_temporal_idx + 1) % enccfg->ts_periodicity; } *got_packet = !!coded_size; @@ -1435,6 +1641,7 @@ static int vpx_encode(AVCodecContext *avctx, AVPacket *pkt, { "noise-sensitivity", "Noise sensitivity", OFFSET(noise_sensitivity), AV_OPT_TYPE_INT, {.i64 = 0 }, 0, 4, VE}, \ { "undershoot-pct", "Datarate undershoot (min) target (%)", OFFSET(rc_undershoot_pct), AV_OPT_TYPE_INT, { .i64 = -1 }, -1, 100, VE }, \ { "overshoot-pct", "Datarate overshoot (max) target (%)", OFFSET(rc_overshoot_pct), AV_OPT_TYPE_INT, { .i64 = -1 }, -1, 1000, VE }, \ + { "ts-parameters", "Temporal scaling configuration using a :-separated list of key=value parameters", OFFSET(vpx_ts_parameters), AV_OPT_TYPE_DICT, {.str=NULL}, 0, 0, VE}, \ #define LEGACY_OPTIONS \ {"speed", "", offsetof(VPxContext, cpu_used), AV_OPT_TYPE_INT, {.i64 = 1}, -16, 16, VE}, \ @@ -1454,8 +1661,6 @@ static const AVOption vp8_options[] = { { "auto-alt-ref", "Enable use of alternate reference " "frames (2-pass only)", OFFSET(auto_alt_ref), AV_OPT_TYPE_INT, {.i64 = -1}, -1, 2, VE}, { "cpu-used", "Quality/Speed ratio modifier", OFFSET(cpu_used), AV_OPT_TYPE_INT, {.i64 = 1}, -16, 16, VE}, - { "ts-parameters", "Temporal scaling configuration using a " - ":-separated list of key=value parameters", OFFSET(vp8_ts_parameters), AV_OPT_TYPE_DICT, {.str=NULL}, 0, 0, VE}, LEGACY_OPTIONS { NULL } }; From patchwork Wed Jan 8 23:00:05 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wonkap Jang X-Patchwork-Id: 17263 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 4505044BB68 for ; Thu, 9 Jan 2020 01:00:19 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 2EC7768AB72; Thu, 9 Jan 2020 01:00:19 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-vk1-f201.google.com (mail-vk1-f201.google.com [209.85.221.201]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 51B7568A97C for ; Thu, 9 Jan 2020 01:00:12 +0200 (EET) Received: by mail-vk1-f201.google.com with SMTP id x74so1727541vke.3 for ; Wed, 08 Jan 2020 15:00:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to; bh=bF2I3mayMMGXwWiujUfaXiDk14T4n6PZl25jsFsfJPk=; b=d2eqUA63nBKC9jCXbpp4sJXbk3u7JStdsoIeFOR/XNdRphpjB6IoaP4/NTD8WJxaKc +d2B4BvT+P1zoQRChJG54ag1amJHla/1Viw0DqwKdUQb6mGmNR2Q5qCRarpoV6o7rYvH L/Mh/HbFCjtxRgi2PXeemAUs7nTn5xb94X5Fie4ZNNNymfUwF5RfWpMSzhxiYsVSUmFU fGC7xeljSKy4tO+YqW5KB/zg4bpgIG/5MHZNpsl4em1a+HczLLjjC7LQvBYAzAanEutj OAY9DHO0sFvR+ICZOL15ZdKVf5EjqhdRgqk1B0VKfED5LEAOFya2i5b/JNHo1pKAmafy PDZw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to; bh=bF2I3mayMMGXwWiujUfaXiDk14T4n6PZl25jsFsfJPk=; b=XumUMVjIF/L0R4yKrnUDBTK3tLLtSE3fHilLk7iddqTA0OJpYqqQ4Zdysl+mgUeKGH 2oSa08Rm1G/hNI8kb/tNRfYL8x8IasuNvbucFmJ72ni/tqWqE1JE6/lzP+0fCYxQB9Mz V3G53TZPaTw0cjegmOeWgRGDmhYErgLoPR3rBVL4+wlzUZ8LsMInerhRa28f22OjY+Aq Ze8O66YJxuOytcu75Ibrwc1NZVEn/JK+B4GxSsVp/uMn4Cujm0KH3cxW64klWdIRFdAz mCh4rbq+RazJGJALKCkOnSk1L7IrwyHEQaC1kHrw+iuEI0BxIJTNqG/AiKtdJj9rt624 6JGQ== X-Gm-Message-State: APjAAAW00FIWoCBJN3KZdyolMEJ6E1RJOtBvwTgLK2ayhznO3Nxg44Gz pyXpcgN/kGYXbGYxrpZYZmDD6AIomIP5fvG32tErkCJ9SLK8mFbumZaxEA2V2Ji09jk9xslBAyS VBZLXqnNa/l2bBQqRpXlkhYtSIX0tKSaq91h+VK6qT63AWnKyBgTOUymWupBKOGc= X-Google-Smtp-Source: APXvYqxSXJy2aFW9ZFdvqW+0UvfzAkLuD4sK00CtKKuHErtUZWrTgG04KYZMw5eE7/UTIfP8XJdAFizSKJM= X-Received: by 2002:ab0:6954:: with SMTP id c20mr4711698uas.82.1578524410921; Wed, 08 Jan 2020 15:00:10 -0800 (PST) Date: Wed, 8 Jan 2020 15:00:05 -0800 In-Reply-To: <20200108230006.169044-1-wonkap@google.com> Message-Id: <20200108230006.169044-2-wonkap@google.com> Mime-Version: 1.0 References: <20200108230006.169044-1-wonkap@google.com> X-Mailer: git-send-email 2.25.0.rc1.283.g88dfdc4193-goog From: Wonkap Jang To: ffmpeg-devel@ffmpeg.org Subject: [FFmpeg-devel] [PATCH v3 2/3] avcodec/libvpxenc: add a way to explicitly set temporal layer id X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" In order for rate control to correctly allocate bitrate to each temporal layer, correct temporal layer id has to be set to each frame. This commit provides the ability to set correct temporal layer id for each frame. --- libavcodec/libvpxenc.c | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-) diff --git a/libavcodec/libvpxenc.c b/libavcodec/libvpxenc.c index 14cc1e7158..8d24e96241 100644 --- a/libavcodec/libvpxenc.c +++ b/libavcodec/libvpxenc.c @@ -1520,11 +1520,22 @@ static int vpx_encode(AVCodecContext *avctx, AVPacket *pkt, #endif if (frame->pict_type == AV_PICTURE_TYPE_I) flags |= VPX_EFLAG_FORCE_KF; - if (CONFIG_LIBVPX_VP8_ENCODER && avctx->codec_id == AV_CODEC_ID_VP8 && frame->metadata) { + if (frame->metadata) { AVDictionaryEntry* en = av_dict_get(frame->metadata, "vp8-flags", NULL, 0); if (en) { flags |= strtoul(en->value, NULL, 10); } + + memset(&layer_id, 0, sizeof(layer_id)); + + en = av_dict_get(frame->metadata, "temporal_id", NULL, 0); + if (en) { + layer_id.temporal_layer_id = strtoul(en->value, NULL, 10); +#ifdef VPX_CTRL_VP9E_SET_MAX_INTER_BITRATE_PCT + layer_id.temporal_layer_id_per_spatial[0] = layer_id.temporal_layer_id; +#endif + layer_id_valid = 1; + } } if (sd) { From patchwork Wed Jan 8 23:00:06 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wonkap Jang X-Patchwork-Id: 17264 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 206C144BB68 for ; Thu, 9 Jan 2020 01:00:22 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 0A67C68ACCE; Thu, 9 Jan 2020 01:00:22 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pg1-f201.google.com (mail-pg1-f201.google.com [209.85.215.201]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 7329968A610 for ; Thu, 9 Jan 2020 01:00:15 +0200 (EET) Received: by mail-pg1-f201.google.com with SMTP id y15so2552817pgk.20 for ; Wed, 08 Jan 2020 15:00:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to; bh=yu8KfE8Q+/wvVAzbGhZKfmv8p8scqF1nb6WYThsnVpM=; b=Rl+85dPYkD6joepg/erFkUlMIYjyO/QabFvEOyHnkys5uPUfx4P1lSZAtg+kAFMkwd XwtA8RV+/3QcE4AiWCKvQjumiGTASxj1h5GFkjB68NXIfm4PDH3+N7oWyNuUB7P7B8cr BIfUC5NBw2k0pHrh/t79dqf/7kZa9cO/FEv+m03vJv0jWkYxn/UZvb76q3vt/Xa3ipHM 9ScIE7wxnBsSlXvYe+C2MrF9NhNooheV4G4jIfz/zUGuoqC1fQ1WKB8YHiy8L+qwrfhu 25Kw/O2nZ7veHoCdPCVNgQnEsa1LnWDygW5x1WdlCBPPiG0553pEvp8kKHxHnkSckGfH 7yPg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to; bh=yu8KfE8Q+/wvVAzbGhZKfmv8p8scqF1nb6WYThsnVpM=; b=CFCKDcrNQcGZtSAYOiUJysLlcLjuIfqvCYFxKa8ag1adfJ9rHaqQCO1KHnpCaBmol7 tKEKv2HjGtI0PkmXddCxtlF8XsoR81km4Q5Rwl0kzUrnfYbfsYB3zKjfek0xCOfX/rGx c5uCwbqtwk0zYOMil2zle541aaf1pxs2uFPwVO84p9NXRmpOvN4FA1BB959Hkd+b+qqH Y4MaEd9d/HkmggxPp3niMC2+HmmQec0wUIClAS/1sGrOS6PBPHuGbYGW/jnafytdTNN8 +yH0BTe4BKdYX3nuUpvpRzHA238CukbkqCCsIpiDM3xEBjN0c6W8m4BEPazXY0u4D74f ocQA== X-Gm-Message-State: APjAAAWi0X75ZkSTHPTMjYAP5ThQ2K0d7h8wv6rXG/kXGl4Vc77xMiLs RGwV6L///0MRPZN84E4Y3JV2wa9eTkw+Jd3+Zao1PvaSNe3EVGWgOagxUC73dZTTiVZEk4aiW75 fdkHmI+HLJ0aUKVD68oIVifQUMGD4mzs2cPJ7CGrTSIryg9X4APRTF2rebC+ZVw4= X-Google-Smtp-Source: APXvYqzGshOykIHnyo6f5LHqljqbLAyG1oHjlCsa78CRbjpTMD2hM/5N/I3k4+BOlRapaoF9qdxfk/aPXfI= X-Received: by 2002:a63:a43:: with SMTP id z3mr7841596pgk.232.1578524413033; Wed, 08 Jan 2020 15:00:13 -0800 (PST) Date: Wed, 8 Jan 2020 15:00:06 -0800 In-Reply-To: <20200108230006.169044-1-wonkap@google.com> Message-Id: <20200108230006.169044-3-wonkap@google.com> Mime-Version: 1.0 References: <20200108230006.169044-1-wonkap@google.com> X-Mailer: git-send-email 2.25.0.rc1.283.g88dfdc4193-goog From: Wonkap Jang To: ffmpeg-devel@ffmpeg.org Subject: [FFmpeg-devel] [PATCH v3 3/3] doc/encoders: add VP9 temporal scalability encoding option X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" Documentation change for adding support for encoding with temporal scalability in VP9. --- doc/encoders.texi | 18 +++++++++++++++--- 1 file changed, 15 insertions(+), 3 deletions(-) diff --git a/doc/encoders.texi b/doc/encoders.texi index 61e674cf96..88429aed4c 100644 --- a/doc/encoders.texi +++ b/doc/encoders.texi @@ -1885,8 +1885,6 @@ Enable error resiliency features. Increase sharpness at the expense of lower PSNR. The valid range is [0, 7]. -@item VP8-specific options -@table @option @item ts-parameters Sets the temporal scalability configuration using a :-separated list of key=value pairs. For example, to specify temporal scalability parameters @@ -1894,7 +1892,7 @@ with @code{ffmpeg}: @example ffmpeg -i INPUT -c:v libvpx -ts-parameters ts_number_layers=3:\ ts_target_bitrate=250,500,1000:ts_rate_decimator=4,2,1:\ -ts_periodicity=4:ts_layer_id=0,2,1,2 OUTPUT +ts_periodicity=4:ts_layer_id=0,2,1,2:ts_layering_mode=3 OUTPUT @end example Below is a brief explanation of each of the parameters, please refer to @code{struct vpx_codec_enc_cfg} in @code{vpx/vpx_encoder.h} for more @@ -1911,6 +1909,20 @@ Frame rate decimation factor for each temporal layer. Length of the sequence defining frame temporal layer membership. @item ts_layer_id Template defining the membership of frames to temporal layers. +@item ts_layering_mode +(optional) Selecting the temporal structure from a set of pre-defined temporal layering modes. +Currently supports the following options. +@table @option +@item 0 +No temporal layering flags are provided internally, +relies on flags being passed in using metadata in AVFrame. +@item 2 +Two temporal layers. 0-1... +@item 3 +Three temporal layers. 0-2-1-2...; with single reference frame. +@item 4 +Same as option "3", except there is a dependency between +the two temporal layer 2 frames within the temporal period. @end table @end table