From patchwork Tue Dec 29 19:51:53 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wonkap Jang X-Patchwork-Id: 24679 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 3882A44BAEC for ; Tue, 29 Dec 2020 21:52:04 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 1AB8E68A5E3; Tue, 29 Dec 2020 21:52:04 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-yb1-f201.google.com (mail-yb1-f201.google.com [209.85.219.201]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 81558689CC9 for ; Tue, 29 Dec 2020 21:51:57 +0200 (EET) Received: by mail-yb1-f201.google.com with SMTP id w17so25238199ybl.15 for ; Tue, 29 Dec 2020 11:51:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to; bh=uUiRUobRnETLVL9MdqODLc/gb1iQHuKf7YhTSH30pSo=; b=EOM1JGxJqFeP0FF9YbyAqCpFnoQmVNNHvAIYWiADzBgJdBcf2iOgcF4prKTvJ3qvQd hNJ8NgUMLItiFeBJcQMLy67Bq9ZJZgj5PmYPpRvrZVvYrW1fkzym817GLHmecsJolYMj e5kYPnEEYdDFeBy4ko/ffOMvc0nKAvNXRU6dmXhNwal9K36gZFrkQY5UoXF7uJAXrB3T i8g10FOA3jHncqf6ZM7HuNJLTeefSiZPUD61fXtoVLTPPSbBYwcYULYc8/YYBkkhmt3m kywjqMH+JOsSwSJtSwD1189oC2G4WKKlVSB8seh3FPpXYnq03rFeWVY03U40o5RBjafv loeA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to; bh=uUiRUobRnETLVL9MdqODLc/gb1iQHuKf7YhTSH30pSo=; b=AH+Iv7P6lSGL3tqNOPM9N16idofe0W6BNvlRWP4Qa4OABWEcR1VWqlNBY39Ff0AeXt 8We4ZjiFdvGDUyZmmt63khKxKruOSJzO1X3Fm0IF16RQj4n5aFP2Hu1UQI/KlfuZXZzG H8bRcpwwDlbJHTNjfZR81U8UexAkQzWhO+47yfPy65xe+4LBJmUjiBcdv+zI+ibpkvlD QI+yWSNB/JLjDi8xVTCVP+UdsDEa1iep9MtuokzKROjFI8TqYOLeNFmIiRIy3FxCAuUo 3BnyDKTADR21sRJeDEr2kpow8BtDlbFeaRFvAKRQyr+T8QEXa+PtPaK/LhABpp2Y11Sw 5hxw== X-Gm-Message-State: AOAM531h5lB1wNUBNZ485+OAI4jjB4PENFb2/nUw2kSx9crrbFSfm5ob UWilth4Wv/XPW5MpJGsa5Krd09AqexhRWYrLv796dES+872vlUF20L/zTURTiCsgYqoE/YMyvBa Fb8v5JNdGFhfdlPEhRUVf4ER0ZzAU78HzXflVNcOA94y302ypwkMmsJmZLoFTXs8= X-Google-Smtp-Source: ABdhPJwkO8fDfWqxH0BE1E5Qh2bGTWbxrL5fK05rikCdn0NAwmOdPtky3g8lWMtJwytiu3VMCLXQJAzh0HE= X-Received: from wonkap.kir.corp.google.com ([2620:15c:17:10:f693:9fff:feed:e069]) (user=wonkap job=sendgmr) by 2002:a05:6902:6cc:: with SMTP id m12mr69144168ybt.56.1609271515613; Tue, 29 Dec 2020 11:51:55 -0800 (PST) Date: Tue, 29 Dec 2020 11:51:53 -0800 In-Reply-To: <20201223213203.220869-1-wonkap@google.com> Message-Id: <20201229195153.1871025-1-wonkap@google.com> Mime-Version: 1.0 References: <20201223213203.220869-1-wonkap@google.com> X-Mailer: git-send-email 2.29.2.729.g45daf8777d-goog From: Wonkap Jang To: ffmpeg-devel@ffmpeg.org Subject: [FFmpeg-devel] [PATCH v4] avcodec/libvpxenc: add a way to set VP9E_SET_SVC_REF_FRAME_CONFIG X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" In order to fine-control referencing schemes in VP9 encoding, there is a need to use VP9E_SET_SVC_REF_FRAME_CONFIG method. This commit provides a way to use the API through frame metadata. --- doc/encoders.texi | 32 +++++++++++++++++ libavcodec/libvpxenc.c | 81 ++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 113 insertions(+) diff --git a/doc/encoders.texi b/doc/encoders.texi index 0b1c69e982..11cf4d89d0 100644 --- a/doc/encoders.texi +++ b/doc/encoders.texi @@ -2129,6 +2129,38 @@ midpoint is passed in rather than calculated for a specific clip or chunk. The valid range is [0, 10000]. 0 (default) uses standard VBR. @item enable-tpl @var{boolean} Enable temporal dependency model. +@item ref-frame-config +Using per-frame metadata, set members of the structure @code{vpx_svc_ref_frame_config_t} in @code{vpx/vp8cx.h} to fine-control referencing schemes and frame buffer management. +@*Use a :-separated list of key=value pairs. +For example, +@example +av_dict_set(&av_frame->metadata, "ref-frame-config", \ +"rfc_update_buffer_slot=7:rfc_lst_fb_idx=0:rfc_gld_fb_idx=1:rfc_alt_fb_idx=2:rfc_reference_last=0:rfc_reference_golden=0:rfc_reference_alt_ref=0"); +@end example +@table @option +@item rfc_update_buffer_slot +Indicates the buffer slot number to update +@item rfc_update_last +Indicates whether to update the LAST frame +@item rfc_update_golden +Indicates whether to update GOLDEN frame +@item rfc_update_alt_ref +Indicates whether to update ALT_REF frame +@item rfc_lst_fb_idx +LAST frame buffer index +@item rfc_gld_fb_idx +GOLDEN frame buffer index +@item rfc_alt_fb_idx +ALT_REF frame buffer index +@item rfc_reference_last +Indicates whetehr to reference LAST frame +@item rfc_reference_golden +Indicates whether to reference GOLDEN frame +@item rfc_reference_alt_ref +Indicates whether to reference ALT_REF frame +@item rfc_reference_duration +Indicates frame duration +@end table @end table @end table diff --git a/libavcodec/libvpxenc.c b/libavcodec/libvpxenc.c index a7c76eb835..4943a9e32c 100644 --- a/libavcodec/libvpxenc.c +++ b/libavcodec/libvpxenc.c @@ -125,6 +125,10 @@ typedef struct VPxEncoderContext { * encounter a frame with ROI side data. */ int roi_warned; +#if CONFIG_LIBVPX_VP9_ENCODER && defined (VPX_CTRL_VP9E_SET_MAX_INTER_BITRATE_PCT) + vpx_svc_ref_frame_config_t ref_frame_config; + AVDictionary *vpx_ref_frame_config; +#endif } VPxContext; /** String mappings for enum vp8e_enc_control_id */ @@ -152,6 +156,7 @@ static const char *const ctlidstr[] = { [VP9E_SET_SVC_LAYER_ID] = "VP9E_SET_SVC_LAYER_ID", #if VPX_ENCODER_ABI_VERSION >= 12 [VP9E_SET_SVC_PARAMETERS] = "VP9E_SET_SVC_PARAMETERS", + [VP9E_SET_SVC_REF_FRAME_CONFIG] = "VP9E_SET_SVC_REF_FRAME_CONFIG", #endif [VP9E_SET_SVC] = "VP9E_SET_SVC", #if VPX_ENCODER_ABI_VERSION >= 11 @@ -394,6 +399,21 @@ static void vp8_ts_parse_int_array(int *dest, char *value, size_t value_len, int } } +#if CONFIG_LIBVPX_VP9_ENCODER && defined (VPX_CTRL_VP9E_SET_MAX_INTER_BITRATE_PCT) +static void vp8_ts_parse_int64_array(int64_t *dest, char *value, size_t value_len, int max_entries) +{ + int dest_idx = 0; + char *saveptr = NULL; + char *token = av_strtok(value, ",", &saveptr); + + while (token && dest_idx < max_entries) { + dest[dest_idx++] = strtoul(token, NULL, 10); + token = av_strtok(NULL, ",", &saveptr); + } +} + +#endif + static void set_temporal_layer_pattern(int layering_mode, vpx_codec_enc_cfg_t *cfg, int *layer_flags, int *flag_periodicity) { @@ -541,6 +561,49 @@ static int vpx_ts_param_parse(VPxContext *ctx, struct vpx_codec_enc_cfg *enccfg, return 0; } +#if CONFIG_LIBVPX_VP9_ENCODER && defined (VPX_CTRL_VP9E_SET_MAX_INTER_BITRATE_PCT) +static int vpx_ref_frame_config_parse(VPxContext *ctx, const struct vpx_codec_enc_cfg *enccfg, + char *key, char *value, enum AVCodecID codec_id) +{ + size_t value_len = strlen(value); + int ss_number_layers = enccfg->ss_number_layers; + vpx_svc_ref_frame_config_t *ref_frame_config = &ctx->ref_frame_config; + + if (!value_len) + return -1; + + if (codec_id != AV_CODEC_ID_VP9) + return -1; + + if (!strcmp(key, "rfc_update_buffer_slot")) { + vp8_ts_parse_int_array(ref_frame_config->update_buffer_slot, value, value_len, ss_number_layers); + } else if (!strcmp(key, "rfc_update_last")) { + vp8_ts_parse_int_array(ref_frame_config->update_last, value, value_len, ss_number_layers); + } else if (!strcmp(key, "rfc_update_golden")) { + vp8_ts_parse_int_array(ref_frame_config->update_golden, value, value_len, ss_number_layers); + } else if (!strcmp(key, "rfc_update_alt_ref")) { + vp8_ts_parse_int_array(ref_frame_config->update_alt_ref, value, value_len, ss_number_layers); + } else if (!strcmp(key, "rfc_lst_fb_idx")) { + vp8_ts_parse_int_array(ref_frame_config->lst_fb_idx, value, value_len, ss_number_layers); + } else if (!strcmp(key, "rfc_gld_fb_idx")) { + vp8_ts_parse_int_array(ref_frame_config->gld_fb_idx, value, value_len, ss_number_layers); + } else if (!strcmp(key, "rfc_alt_fb_idx")) { + vp8_ts_parse_int_array(ref_frame_config->alt_fb_idx, value, value_len, ss_number_layers); + } else if (!strcmp(key, "rfc_reference_last")) { + vp8_ts_parse_int_array(ref_frame_config->reference_last, value, value_len, ss_number_layers); + } else if (!strcmp(key, "rfc_reference_golden")) { + vp8_ts_parse_int_array(ref_frame_config->reference_golden, value, value_len, ss_number_layers); + } else if (!strcmp(key, "rfc_reference_alt_ref")) { + vp8_ts_parse_int_array(ref_frame_config->reference_alt_ref, value, value_len, ss_number_layers); + } else if (!strcmp(key, "rfc_reference_duration")) { + vp8_ts_parse_int64_array(ref_frame_config->duration, value, value_len, ss_number_layers); + } + + return 0; +} +#endif + + #if CONFIG_LIBVPX_VP9_ENCODER static int set_pix_fmt(AVCodecContext *avctx, vpx_codec_caps_t codec_caps, struct vpx_codec_enc_cfg *enccfg, vpx_codec_flags_t *flags, @@ -1528,6 +1591,24 @@ static int vpx_encode(AVCodecContext *avctx, AVPacket *pkt, #endif layer_id_valid = 1; } +#if CONFIG_LIBVPX_VP9_ENCODER && defined (VPX_CTRL_VP9E_SET_MAX_INTER_BITRATE_PCT) + en = av_dict_get(frame->metadata, "ref-frame-config", NULL, 0); + + if (en) { + AVDictionaryEntry* en2 = NULL; + av_dict_parse_string(&ctx->vpx_ref_frame_config, en->value, "=", ":", 0); + + while ((en2 = av_dict_get(ctx->vpx_ref_frame_config, "", en2, AV_DICT_IGNORE_SUFFIX))) { + if (vpx_ref_frame_config_parse(ctx, enccfg, en2->key, en2->value, avctx->codec_id) < 0) + av_log(avctx, AV_LOG_WARNING, + "Error parsing option '%s = %s'.\n", + en2->key, en2->value); + } + + codecctl_intp(avctx, VP9E_SET_SVC_REF_FRAME_CONFIG, (int *)&ctx->ref_frame_config); + } +#endif + } if (sd) {