From patchwork Thu Feb 8 21:56:49 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Dariusz Marcinkiewicz X-Patchwork-Id: 46119 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a21:a586:b0:19e:8a94:b663 with SMTP id gd6csp562849pzc; Thu, 8 Feb 2024 13:58:11 -0800 (PST) X-Forwarded-Encrypted: i=2; AJvYcCWFsFr42cAgpucTOXXby9d74kNiO7Xq+47nEGACj6OWFYX2Qg59N8sUrf84AJktwjzmoJVqI/EN+v1iQ4ReTGVnD1rQBnkonuRsRQ== X-Google-Smtp-Source: AGHT+IG2OVkZBdEQRrkEP6YLv+yeQy53pjdmBsHMK8DzS9B8Ly7MP+tDbtiYkmaUXLZcwUbbsw4E X-Received: by 2002:a17:906:4742:b0:a33:b64f:48c1 with SMTP id j2-20020a170906474200b00a33b64f48c1mr388379ejs.21.1707429491203; Thu, 08 Feb 2024 13:58:11 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1707429491; cv=none; d=google.com; s=arc-20160816; b=R/yg0e2uwmok5w8GT3oV2yvlb+7e5BDhFhFA45OH9Su7WwZ+629PDK5561IWSCQ+XC E1a0IWVqdKDZkDKxdSgs4/uJohFZSf65EFXNM4XIUvFSS3PAFobpRHEV4Jx4oR0CLFxy KyY+58EzNb3DHhum+H3WU0djiYF4RfPDE9e2skfqsbUlq1KOuStQKQOz1EeMIGwERJY9 CLsRWkEr4OkiPgwIVTWRMxcD0nQyT4p23zcdoH4O5sRrEjRbOE7SKpAd/5EnrDbTOcKL NQnJpIAJdASZs0SrP4oBga2PdPPxSbwnkfncRvcQCdA5xCay67lUGV61gy2mkinjkLnO nqrg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to:from :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:to:message-id:references:mime-version :in-reply-to:date:delivered-to; bh=yWAdT9A3l5GRH8YoNKig86emMbkEN0AD8De1W4V7fyc=; fh=NBEEd1bjq//upXSgckQSCrOc1DbZWjj7p5whncmAYUI=; b=mFGrvD3ls4xcRG7YZNNorDf6wblWb9hEr/FwunEK8UBQRCJRki9i1rwZjVHK9qLTv9 yDA3JFk5PNF2dRrgsLNoEyS9L5+9JiPZ4sZ55V187e2P8FtaT18F6QgejC2/BJiHyTo+ vy59ASgyCxOWd/R95H89BEBCaAG7juTyQfa0WnvM+lMeWgePw8kX1X6ap3Lz322hKKa6 FPP5W/8NM9BhRVomSTnGLIk/PcHy8KQF4/hLspjHsXnbN9rWl7Ns0AdnTirXh4M1M92C J9eKNKiMn/bRieXWNxDnEQ9xiPrQMZKgLD/SftBN8zshF6pahqGqsGw7cYtVmB1oCyP0 3uTw==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org X-Forwarded-Encrypted: i=1; AJvYcCWVcrYk1WFyeXeAhteF+W5Aaos0aKSkRocainw+YlEZLpY+85Ipx7x7IHq4vY1H8+dM1oZtEHnxSSlIdc8xEK7obuG+crMsY/ede7cqSv7+6xC+1JjVlRbLpB0ijYRqSmh+a4p4TiMjLlGRwEki/H9VTARXvD1yPynjHzrcv0YrmxSGoIR/vmYd8oQkpGyXKIbZJlTkEHIFMl0DL3EEq0NMcSoyi1l4u7KTaa6A7GpiGtINNJFRD9NrSKU4JZ1TD2AsXxqtqWinpMK7zanJr+6OD0u5+7QaMuWQZl5LCsYySv/Fwu0SDtP69uKWGMxxCydUmKk79hXfYzrxYwAs3B8VpwD01hnRiy7t0Jrm6icSKsealSUpo4rD4OzsKoukRPI4XlMwdxRld2jFVu3TcAMW+sNS/hbOY8lesLOYUIQvigQ5scIf9Lp0xvNkxs8SjGZlqFOW9i+aRvpxTCmz6Adocd5AoRg4e1kGFSIn9fFnCTgJh5uvamnPoQiIZ7ZOxwNshTqK7++ScdgneBjC0r7J1UbukQ8n1VpSkiUwoldUr6LH/l+rzPGko20SDYjIzzBWYSDzMqx3f/GoGur114S373UdSLOijvLMEE8e0Sluva6fIyW6kz3Ffn383ObGtwIuiNhOaysSIoET3xdAsBpnECqlVa3ra7ftBm+4SaeZIE8AS6cryGhtSC8eLiwGXsvY8ab7cqnMZahfFf6GHJnqPR8qDq+1o/tHZ2ngN4VpKSe4spZRUHUKaZg3MfDmafO+MXn3r/SWo1cGPsmaIUcS25jZMqBHSn9RXGc5Ply7JNgeYop5XdX9Db/yGcunb8dw+J03JKAM+t+42xqAvNzmYD6hVSthQceIpMH/6bbzhZVyPgsagjPzfzxPdxYpEICQzJ74nSpqqDHWotvoDv9wd2TwEXMItDh0vGC5momn8OyU1+AOKdT4BkAymW02/dF0hE 7S19/qqasEs7eDBnXNwcZINVMn580dDdgq4Ni1ViHfTInRaTJUShZK/0tD1bdT3Ad9F9b6ueh5MygQx+aQ3ADZjX9xw0ZyR/2eLOOsI/x6iPcWNZpTCFkNgBRWU4QR8qu/Ayz6cISF7/vufBJszYZeCRsJSOf1c9BdFaW5/z6p5kRZmRckLH6Osh65vE3AJ9YlPbBsvROm8YZrvrC75ITVY08e5ngY/DCm7QHaS57Ai/RFBDWCwcw+2NKL7bHJCf7vwmKuD6iAX4TUJqBcOCGI+vEPnp8IzYwWOC3XqE7SP3qmLxjATmvGuPov5qpe7xpSnN4ANJIkF/n7lAZ00H/xZVljre+BbXvjC2GXQDUQ1wF2vhwCyVpsIP6hacIA4Frw0kGOkCG1rAXE+ee10NpaluxQx8VsrNQmYweNt4S+RDpCCSZYhBFbDSGlt9mLl95+gcBrp2lfeksUVBtwbpzr3/ve1gZJ2K1u54IqG/z00z0ZPDZgAaxOVulUp1mj1shuU2JeLzk7fDt5AiE9s4iBN9EQrD0SU8uQBtKpo82PWVye7um/cPFV7hjnl6KXM0ipXGKp/2NOfhfudRn3WjAurbhXfNih9VtRglonLX0iVSpBBWyESH4HSbvdJLneK/ZRl1Tt Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id x8-20020a1709065ac800b00a3773acdb02si104453ejs.154.2024.02.08.13.58.10; Thu, 08 Feb 2024 13:58:11 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id D629E68C954; Thu, 8 Feb 2024 23:58:07 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-yw1-f202.google.com (mail-yw1-f202.google.com [209.85.128.202]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 194EB68C954 for ; Thu, 8 Feb 2024 23:58:00 +0200 (EET) Received: by mail-yw1-f202.google.com with SMTP id 00721157ae682-5f38d676cecso18286387b3.0 for ; Thu, 08 Feb 2024 13:58:00 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1707429479; x=1708034279; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=UbzScIPQplXZM7h60qs3bmEq2VYpk/1K6qu2qpAlzCU=; b=B6+PrtMwGfqGbifoxB86akfeX0NrBvzhirqRMhDTxSVg4wWurL969i5tv3MZgVgUlH +rAOI4FTmivl/XFVZWZWdj8esISbSVOkDwJ+TFgfLYYvfSuX4S+Ue58pvQu94dNe/rwd 86U+rnw0dgdwJazGZ+MMWDQstWiZw1xJLZnEuDA40J/r9mHwqwTYQ5UqrmRM89K4IZK+ u/rL/CE9SR0JmVLcvV8ngKM6WjCIBDsuKlD4zY3vHKEa0FrGl7uB1mUzzRSnR769jeqr PfiPSYx2Mppnf5JEXTcyhoLNcXrDYUHa0HC+AkBMyAKkTKi6qCiPUoBaHhtO1/xNaYxn wf9g== X-Gm-Message-State: AOJu0YySLQzJAR2CZq9+H4STPL89IiPNZCY+BYDoPAxHc8gGMp4stlRz nuoG9D1Be+FUq8ouAP1GaMZOO+wjSY234o0fzTz/eUzcjwZ9yQuVb8j3BAHJ0C6Uq27aCOJ3//P Fmsf2CXWcm2jq/1zook1d2FIT1x24XJeEbfIa/X4dV6n02W2NwHstkPWYQ48hQSVyDvZ17WxmwK o92t7VXQjGGxjdmk0r90x3ZgmFCVpDK8o72Q== X-Received: from reksio.c.googlers.com ([fda3:e722:ac3:cc00:28:9cb1:c0a8:1be8]) (user=darekm job=sendgmr) by 2002:a25:946:0:b0:dc6:19b9:5b83 with SMTP id u6-20020a250946000000b00dc619b95b83mr829834ybm.6.1707429476463; Thu, 08 Feb 2024 13:57:56 -0800 (PST) Date: Thu, 8 Feb 2024 21:56:49 +0000 In-Reply-To: Mime-Version: 1.0 References: X-Mailer: git-send-email 2.43.0.687.g38aa6559b0-goog Message-ID: <20240208215649.3090001-2-darekm@google.com> To: ffmpeg-devel@ffmpeg.org Subject: [FFmpeg-devel] [PATCH v3] lavc/libvpxenc: add screen-content-mode option X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Dariusz Marcinkiewicz via ffmpeg-devel From: Dariusz Marcinkiewicz Reply-To: FFmpeg development discussions and patches Cc: Dariusz Marcinkiewicz , =?utf-8?q?Erik_Spr=C3=A5ng?= Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: MPD7DnlaNcTg This exposes VP8E_SET_SCREEN_CONTENT_MODE option from libvpx and makes us retry encode if screen_content_mode == 2 and no output was produced by encoder. Co-authored-by: Erik Språng Signed-off-by: Dariusz Marcinkiewicz --- doc/encoders.texi | 3 +++ libavcodec/libvpxenc.c | 57 ++++++++++++++++++++++++++++++++++++++---- libavcodec/version.h | 2 +- 3 files changed, 56 insertions(+), 6 deletions(-) diff --git a/doc/encoders.texi b/doc/encoders.texi index c9fe6d6143..0868aa66db 100644 --- a/doc/encoders.texi +++ b/doc/encoders.texi @@ -2150,6 +2150,9 @@ of quality. Set a change threshold on blocks below which they will be skipped by the encoder. +@item screen-content-mode +Screen content mode, one of: off (0), screen (1), screen with more aggressive rate control (2). + @item slices (@emph{token-parts}) Note that FFmpeg's @option{slices} option gives the total number of partitions, while @command{vpxenc}'s @option{token-parts} is given as diff --git a/libavcodec/libvpxenc.c b/libavcodec/libvpxenc.c index 80988a2608..7571a8577a 100644 --- a/libavcodec/libvpxenc.c +++ b/libavcodec/libvpxenc.c @@ -114,6 +114,7 @@ typedef struct VPxEncoderContext { int crf; int static_thresh; int max_intra_rate; + int screen_content_mode; int rc_undershoot_pct; int rc_overshoot_pct; @@ -164,6 +165,7 @@ static const char *const ctlidstr[] = { [VP8E_SET_MAX_INTRA_BITRATE_PCT] = "VP8E_SET_MAX_INTRA_BITRATE_PCT", [VP8E_SET_SHARPNESS] = "VP8E_SET_SHARPNESS", [VP8E_SET_TEMPORAL_LAYER_ID] = "VP8E_SET_TEMPORAL_LAYER_ID", + [VP8E_SET_SCREEN_CONTENT_MODE] = "VP8E_SET_SCREEN_CONTENT_MODE", #if CONFIG_LIBVPX_VP9_ENCODER [VP9E_SET_LOSSLESS] = "VP9E_SET_LOSSLESS", [VP9E_SET_TILE_COLUMNS] = "VP9E_SET_TILE_COLUMNS", @@ -1262,6 +1264,16 @@ static av_cold int vpx_init(AVCodecContext *avctx, #endif } #endif +#ifdef VPX_CTRL_VP8E_SET_SCREEN_CONTENT_MODE + if (avctx->codec_id == AV_CODEC_ID_VP8 && ctx->screen_content_mode >= 0) { + if (ctx->screen_content_mode == 2 && ctx->is_alpha) { + av_log(avctx, AV_LOG_ERROR, + "Transparency encoding with screen mode with aggressive rate control not supported\n"); + return AVERROR(EINVAL); + } + codecctl_int(avctx, VP8E_SET_SCREEN_CONTENT_MODE, ctx->screen_content_mode); + } +#endif av_log(avctx, AV_LOG_DEBUG, "Using deadline: %d\n", ctx->deadline); @@ -1379,14 +1391,15 @@ static int storeframe(AVCodecContext *avctx, struct FrameListData *cx_frame, * @return AVERROR(ENOMEM) on coded frame queue data allocation error */ static int queue_frames(AVCodecContext *avctx, struct vpx_codec_ctx *encoder, - struct FrameListData **frame_list, AVPacket *pkt_out) + struct FrameListData **frame_list, AVPacket *pkt_out, + int queue_only, int *frame_enc) { VPxContext *ctx = avctx->priv_data; const struct vpx_codec_cx_pkt *pkt; const void *iter = NULL; int size = 0; - if (!ctx->is_alpha && *frame_list) { + if (!queue_only && *frame_list) { struct FrameListData *cx_frame = *frame_list; /* return the leading frame if we've already begun queueing */ size = storeframe(avctx, cx_frame, NULL, pkt_out); @@ -1401,7 +1414,7 @@ static int queue_frames(AVCodecContext *avctx, struct vpx_codec_ctx *encoder, while (pkt = vpx_codec_get_cx_data(encoder, &iter)) { switch (pkt->kind) { case VPX_CODEC_CX_FRAME_PKT: - if (!ctx->is_alpha && !size) { + if (!queue_only && !size) { struct FrameListData cx_frame; /* avoid storing the frame when the list is empty and we haven't yet @@ -1411,6 +1424,8 @@ static int queue_frames(AVCodecContext *avctx, struct vpx_codec_ctx *encoder, size = storeframe(avctx, &cx_frame, NULL, pkt_out); if (size < 0) return size; + if (size > 0) + *frame_enc = 1; } else { struct FrameListData *cx_frame = av_malloc(sizeof(*cx_frame)); @@ -1430,6 +1445,8 @@ static int queue_frames(AVCodecContext *avctx, struct vpx_codec_ctx *encoder, return AVERROR(ENOMEM); } memcpy(cx_frame->buf, pkt->data.frame.buf, pkt->data.frame.sz); + if (pkt->data.frame.sz > 0) + *frame_enc = 1; coded_frame_add(frame_list, cx_frame); } break; @@ -1693,6 +1710,7 @@ static int vpx_encode(AVCodecContext *avctx, AVPacket *pkt, vpx_svc_layer_id_t layer_id; int layer_id_valid = 0; unsigned long duration = 0; + int frame_enc = 0; if (avctx->qmax >= 0 && enccfg->rc_max_quantizer != avctx->qmax) { struct vpx_codec_enc_cfg cfg = *enccfg; @@ -1856,9 +1874,32 @@ FF_ENABLE_DEPRECATION_WARNINGS } } - coded_size = queue_frames(avctx, &ctx->encoder, &ctx->coded_frame_list, pkt); + coded_size = queue_frames(avctx, &ctx->encoder, &ctx->coded_frame_list, + pkt, ctx->is_alpha, &frame_enc); + if (avctx->codec_id == AV_CODEC_ID_VP8 && frame_enc == 0 && + ctx->screen_content_mode == 2 && frame) { + // VP8 tuned for screen content with aggresive rate control - returned + // OK status code but produced no output, this indicates frame was + // rolled back due to bitrate overshoot - try to encode it again. + av_log(avctx, AV_LOG_VERBOSE, + "Attempting to reencode dropped frame.\n"); + res = vpx_codec_encode(&ctx->encoder, rawimg, timestamp, + duration, flags, ctx->deadline); + if (res != VPX_CODEC_OK) { + log_encoder_error(avctx, "Error encoding frame"); + return AVERROR_INVALIDDATA; + } + if (!coded_size) + coded_size = queue_frames(avctx, &ctx->encoder, &ctx->coded_frame_list, + pkt, /*queue_only*/0, &frame_enc); + else + queue_frames(avctx, &ctx->encoder, &ctx->coded_frame_list, pkt, + /*queue_only*/1, &frame_enc); + } + if (ctx->is_alpha) { - queue_frames(avctx, &ctx->encoder_alpha, &ctx->alpha_coded_frame_list, NULL); + queue_frames(avctx, &ctx->encoder_alpha, &ctx->alpha_coded_frame_list, + NULL, /*queue_only*/1, &frame_enc); if (ctx->coded_frame_list && ctx->alpha_coded_frame_list) { struct FrameListData *cx_frame = ctx->coded_frame_list; @@ -1946,6 +1987,12 @@ static const AVOption vp8_options[] = { { "auto-alt-ref", "Enable use of alternate reference " "frames (2-pass only)", OFFSET(auto_alt_ref), AV_OPT_TYPE_INT, {.i64 = -1}, -1, 2, VE}, { "cpu-used", "Quality/Speed ratio modifier", OFFSET(cpu_used), AV_OPT_TYPE_INT, {.i64 = 1}, -16, 16, VE}, +#ifdef VPX_CTRL_VP8E_SET_SCREEN_CONTENT_MODE + { "screen-content-mode", "Encoder screen content mode", OFFSET(screen_content_mode), AV_OPT_TYPE_INT, {.i64 = -1}, -1, 2, VE, "screen_content_mode"}, + { "off", "Screen content mode off", 0, AV_OPT_TYPE_CONST, {.i64 = 0}, 0, 0, VE, "screen_content_mode"}, + { "on", "Screen content mode on", 0, AV_OPT_TYPE_CONST, {.i64 = 1}, 0, 0, VE, "screen_content_mode"}, + { "on-agressive-rate-control", "Screen content mode on with aggressive rate control", 0, AV_OPT_TYPE_CONST, {.i64 = 2}, 0, 0, VE, "screen_content_mode"}, +#endif LEGACY_OPTIONS { NULL } }; diff --git a/libavcodec/version.h b/libavcodec/version.h index 0fae3d06d3..4b618d740f 100644 --- a/libavcodec/version.h +++ b/libavcodec/version.h @@ -30,7 +30,7 @@ #include "version_major.h" #define LIBAVCODEC_VERSION_MINOR 38 -#define LIBAVCODEC_VERSION_MICRO 100 +#define LIBAVCODEC_VERSION_MICRO 101 #define LIBAVCODEC_VERSION_INT AV_VERSION_INT(LIBAVCODEC_VERSION_MAJOR, \ LIBAVCODEC_VERSION_MINOR, \