From patchwork Wed Feb 7 16:03:40 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Dariusz Marcinkiewicz X-Patchwork-Id: 46097 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:a38a:b0:19e:8a94:b663 with SMTP id w10csp629166pzk; Wed, 7 Feb 2024 08:04:26 -0800 (PST) X-Google-Smtp-Source: AGHT+IH/bFU9Yxe8BWTJjL2CpeB0CCIuGzqKlrVxGbiMMDyu90Om00xt+TEuyJl+BFw5ohg2Gxx3 X-Received: by 2002:a17:906:b1cb:b0:a36:fd11:3f98 with SMTP id bv11-20020a170906b1cb00b00a36fd113f98mr5127057ejb.17.1707321866321; Wed, 07 Feb 2024 08:04:26 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1707321866; cv=none; d=google.com; s=arc-20160816; b=kGsZB+CfiyV34Qmun+rYCzp+weE8yA5Q6JOAlh15obJ9weJrGm0LkN8/mvffcoQ9nc /ZQjY1cLE7zhRqmAmFqNv+lJYnavmIwU7DwB2cp8zckIcWUEv/W9ZPc0pt/AigyfRS9r 0dY0CxYBB61R7n+n901Zc80cwZvekOF+noHc5uOGU1H7cbTrLRC2nna9veqDhLb0Ldw1 rTdw/SYrQ+yXF/pfldwHRX4zq1qUj1ITQABfdbIf+3d9/H9ZrCnjbCkekgegaj0N/EI7 fNovrNEJGgOGNFLj/At/Ueydp2iGZ00uBZZgJ8EqXEnADCRVXIIMjGPt0n6mE0toKLNI grQA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to:from :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:to:references:mime-version:message-id :in-reply-to:date:delivered-to; bh=eTPona1N7Z5XEcxb3/bGH4EaMGqq/qhj4w3060C1uwI=; fh=2/t4B3FdxO+tjg3XYHmKCZVURHQaOomj9yL6YxhTcfM=; b=WsXE2NGZrnCNtHsdrVZ+1K4QAUdTmTPW8ViVZmo7kNRCbG/8FkoBfetlEBqDHJemeP XEkM18GtP+HDhE2angcL2zLqKxhU4i11Ju6Wq/fr1DBxWbWjDUwFgIpdi7rwYUpkyfaL mHIYBjw0bO+Yy5HWrAIl1YKlN0VECg0RPW1/qR4883Rg6Q4SvbomCa1jKPfijFTlgUsa fx3TMmcVt+v88zpM1Iy7YGfpJFPC+Rv/+BrMKKFBdFyk6lPnug7A8N5sUGHuR77OfTsz cY+QspHpC/lNVMu8VSSDK5a5FPlv29vHBZGS5yDXQbX+EWjuzxsa/NDq0ODeYxjNhzyR uhdQ==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org X-Forwarded-Encrypted: i=1; AJvYcCXHfueDiZ1SLLz42+BuPEJvvI4TGhFFejHt683F3FLRIA+ZhV0bgpCIXllFYN8IqFBlYjSjILcqZNQfAuklC/6FXdTGV0LVBX77+qePCxJdrU2UtBks8Dt0uLBVhC5Jt84FVvuoC46I5gKBUPKBONMZE0qI1I/9BpuyXs8lhtAzHwDZr6QU1hrNBI4OhDqo1VXfgGj9mh33btw7QUPXvYn0wUQ6NQ782Kg0YJ2GEDI+ahGvVrEKHmYFuHMDXzgbUVRtkWQ01L3vvQZco+oD6lRp5Y0A+zR2eSs6hnv/X9beegMyxk0vw3mzUJ9CAOsFChdjah5+3IlJPZ9GddsJXoIrT3HA6W4lyg/4vgJL6FRRfXzWJgDoA4q74zaqVNsBME++s6uJorHGi1FCgC5HV/SSebxV3OzCMxCuzfU30Y/p2Fp3ZxsYZXnbJEd+91IW3DXKNwxM72CrfeiqY6xGaBfvbjmVEkncCwmk8Tj1xsM+HUlxrZAUGDPudZMmWbg2TFWTlwD5L/pYL22IB7hvJoAwCqUj5JNGbG/Wvcqakgf2yqWP8Z/GX5ndsOHxul7IEO/5zGgXWLjdYyFf+FQPPW5HaBkV4kqkQxIDGWlj9E87bCeATZANEm2ou0yOiolN5YlMEkgrgU88EKbkkPW0hcbjMa3CF63KM4MkCIePWZCLF/+xkf5a02N8oL0jZLegVK07uDtQidXOEMTI9+tAKar3VIbZ5zPahi4rLnfuMs+LcbYTrS5n8skX6IbmiMxhc1C0eW6IDdBT0fQHyfdQa39oxCJj4dTaEfacH4ygu85t6qJ5Xc5C/MzjKGeiVRL4Cv0khhS9rgDQwOOtEsEv2WzAK/DvzLcjNbeHgzZg6aQxpPSCQvjkfeiM2EUo9w/0YSs9nJFAdy8cV50JwkCr/HBsXF5IRw9Ta9wmaXmdJno+LDJx7Pq0/Aa3Sc3jEH/8qu1pUI 2ndn4c2q5LGITch8BIbLPwpGauw6uL0CURIPhX3orMSzHHn09bmkMJSFe3E5OmrTjeViAHlNsu+DQ6VPhmB8yhRC1j0GeoVeYMpmx+W4ag+8/6Hod/h7NsEUcShpfK5gYlsRyKIZIzQQF8HTCxvt00LX7KY+7CTyhe+OQQh5+kCElfiO5Gj1sfsun35Fignn7/3vQV8tfls5azcNUWF/cj9+BE+HB2FgYL1fq2WQNXe0Vya6ZOGnOEpNBW+oudVVcNxGQ3PkUxer2c0qUT/tTPrUNM04WYapX+HaGpE5PfY5wi5ZWz8k4XyrVOCYnbGE8jA+mcK4yKX6ty1pe6qxd12+gf3aDmLRIj4ptJh29dIxX2eOoAqA0ps1sKI+/DPOq/jVPNcYff8U16+HJH2zSEqDnrwvIcSh+oyhOqrctyhe1QdsUhOYpjkgy8mg96qCs7GVSt8gga78nRv0M54F1hmHMw/1d5uFKGecmjo+/sMxgPlyjgphwOJiLZRVHcUBMCEGn3ljfmt21Ot6UqbAIULVm3hNOi8SVbj/qRJNLDAT27XbzQpw+O+Y7TtYfNqc2+u/KCzD0snPZFVVP9mS/naMoBojyPLeEHuEsLHocVUNWL/bQtqi5bX34IfLPpJ3HZ Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id rh9-20020a17090720e900b00a38461ed233si1066373ejb.193.2024.02.07.08.04.24; Wed, 07 Feb 2024 08:04:26 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 899BC68D150; Wed, 7 Feb 2024 18:04:20 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-yw1-f201.google.com (mail-yw1-f201.google.com [209.85.128.201]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 64C3068CBCB for ; Wed, 7 Feb 2024 18:04:14 +0200 (EET) Received: by mail-yw1-f201.google.com with SMTP id 00721157ae682-6043fdfddfeso14300847b3.3 for ; Wed, 07 Feb 2024 08:04:14 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1707321853; x=1707926653; h=content-transfer-encoding:cc:to:from:subject:references :mime-version:message-id:in-reply-to:date:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=qWbLraEqBOpfKyrmKFzRhjEG5ddPLNRhqh6kQdqTdgc=; b=PNbfgUAEDK6Y7/ea3IbehWwswrbg6JlXbSIeSBuJd7iGwAjwnoR7LXjeCFERqJzpyI fDTelJZ1zMIYJekGtNmbQxjUlkyOKlrq2uREfTAHtGSG6gsiFSWJOq3UWrBPl1fgVx7d 7bdC4WBJVP6MA7C3sE3IzkybpS4QoJhHtk15u6z1UFEuBrmCNwST+1mG3PqKvG8L4zhG fcHfcHt5/hCg6eFQtawljpagvqR5esyGYJKgQH3IsE8b0iWjopwo17pR/YJRqLxmrHjo pE9IfHwsOzQxgY6AkBGJME9mVuzJcrN3ui4wP4PJ523KMGQ8A/1d8nWfpqeFG6EdUTSt Jchw== X-Gm-Message-State: AOJu0YxDRmI1xDk0ONKX9R+kcheu9pbLZLWEqtqESTyFrpcJtpvNYMia t9HOXikNj6sVlhQdzHK4gX9rr6vBmkvyTo9Ae9XM+HwBvp0aObve+PSDNBcYGX4MEVStoZChH0V zp6mc5gZjajdyatL4J8XynxdLhoN5om40q+ZmduH5iXr+LkVYH1dW8B3c0X281ozCtK7ItSAo8c v2Kc8GWlzzShLBpiLQEP6TRG5uoW/w8vTH4A== X-Received: from reksio2.lul.corp.google.com ([2620:0:1043:12:7990:c928:a41a:1dfc]) (user=darekm job=sendgmr) by 2002:a0d:e650:0:b0:604:9df3:3ccb with SMTP id p77-20020a0de650000000b006049df33ccbmr139062ywe.9.1707321850581; Wed, 07 Feb 2024 08:04:10 -0800 (PST) Date: Wed, 7 Feb 2024 17:03:40 +0100 In-Reply-To: <20240131162303.3134027-1-darekm@google.com> Message-Id: <20240207160340.357523-1-darekm@google.com> Mime-Version: 1.0 References: <20240131162303.3134027-1-darekm@google.com> X-Mailer: git-send-email 2.43.0.594.gd9cf4e227d-goog To: ffmpeg-devel@ffmpeg.org Subject: [FFmpeg-devel] [PATCH v2] libavcodec: add tune_content option also for VP8. X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Dariusz Marcinkiewicz via ffmpeg-devel From: Dariusz Marcinkiewicz Reply-To: FFmpeg development discussions and patches Cc: Dariusz Marcinkiewicz , =?utf-8?q?Erik_Spr=C3=A5ng?= Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: IVXD4PZMQwbn This exposes VP8E_SET_SCREEN_CONTENT_MODE option from libvpx. Changes since v1: - Put the new param initialzation in the right place, - Account for cases when the encoder's output is queued up. Co-authored-by: Erik Språng Signed-off-by: Dariusz Marcinkiewicz --- doc/encoders.texi | 7 ++++-- libavcodec/libvpxenc.c | 56 ++++++++++++++++++++++++++++++++++++++---- 2 files changed, 56 insertions(+), 7 deletions(-) diff --git a/doc/encoders.texi b/doc/encoders.texi index c9fe6d6143..2a9b38f62a 100644 --- a/doc/encoders.texi +++ b/doc/encoders.texi @@ -2186,6 +2186,11 @@ Enable error resiliency features. Increase sharpness at the expense of lower PSNR. The valid range is [0, 7]. +@item tune-content +Set content type. +For VP8: default (0), screen (1), screen with aggressive rate control (2). +For VP9: default (0), screen (1), film (2). + @item ts-parameters Sets the temporal scalability configuration using a :-separated list of key=value pairs. For example, to specify temporal scalability parameters @@ -2268,8 +2273,6 @@ colorspaces: @end table @item row-mt @var{boolean} Enable row based multi-threading. -@item tune-content -Set content type: default (0), screen (1), film (2). @item corpus-complexity Corpus VBR mode is a variant of standard VBR where the complexity distribution midpoint is passed in rather than calculated for a specific clip or chunk. diff --git a/libavcodec/libvpxenc.c b/libavcodec/libvpxenc.c index 80988a2608..c73c92d49b 100644 --- a/libavcodec/libvpxenc.c +++ b/libavcodec/libvpxenc.c @@ -164,6 +164,7 @@ static const char *const ctlidstr[] = { [VP8E_SET_MAX_INTRA_BITRATE_PCT] = "VP8E_SET_MAX_INTRA_BITRATE_PCT", [VP8E_SET_SHARPNESS] = "VP8E_SET_SHARPNESS", [VP8E_SET_TEMPORAL_LAYER_ID] = "VP8E_SET_TEMPORAL_LAYER_ID", + [VP8E_SET_SCREEN_CONTENT_MODE] = "VP8E_SET_SCREEN_CONTENT_MODE", #if CONFIG_LIBVPX_VP9_ENCODER [VP9E_SET_LOSSLESS] = "VP9E_SET_LOSSLESS", [VP9E_SET_TILE_COLUMNS] = "VP9E_SET_TILE_COLUMNS", @@ -1262,6 +1263,16 @@ static av_cold int vpx_init(AVCodecContext *avctx, #endif } #endif +#ifdef VPX_CTRL_VP8E_SET_SCREEN_CONTENT_MODE + if (avctx->codec_id == AV_CODEC_ID_VP8 && ctx->tune_content >= 0) { + if (ctx->tune_content == 2 && ctx->is_alpha) { + av_log(avctx, AV_LOG_ERROR, + "Transparency encoding with screen mode with aggressive rate control not supported\n"); + return AVERROR(EINVAL); + } + codecctl_int(avctx, VP8E_SET_SCREEN_CONTENT_MODE, ctx->tune_content); + } +#endif av_log(avctx, AV_LOG_DEBUG, "Using deadline: %d\n", ctx->deadline); @@ -1379,14 +1390,15 @@ static int storeframe(AVCodecContext *avctx, struct FrameListData *cx_frame, * @return AVERROR(ENOMEM) on coded frame queue data allocation error */ static int queue_frames(AVCodecContext *avctx, struct vpx_codec_ctx *encoder, - struct FrameListData **frame_list, AVPacket *pkt_out) + struct FrameListData **frame_list, AVPacket *pkt_out, + int queue_only, int *frame_enc) { VPxContext *ctx = avctx->priv_data; const struct vpx_codec_cx_pkt *pkt; const void *iter = NULL; int size = 0; - if (!ctx->is_alpha && *frame_list) { + if (!queue_only && *frame_list) { struct FrameListData *cx_frame = *frame_list; /* return the leading frame if we've already begun queueing */ size = storeframe(avctx, cx_frame, NULL, pkt_out); @@ -1401,7 +1413,7 @@ static int queue_frames(AVCodecContext *avctx, struct vpx_codec_ctx *encoder, while (pkt = vpx_codec_get_cx_data(encoder, &iter)) { switch (pkt->kind) { case VPX_CODEC_CX_FRAME_PKT: - if (!ctx->is_alpha && !size) { + if (!queue_only && !size) { struct FrameListData cx_frame; /* avoid storing the frame when the list is empty and we haven't yet @@ -1411,6 +1423,8 @@ static int queue_frames(AVCodecContext *avctx, struct vpx_codec_ctx *encoder, size = storeframe(avctx, &cx_frame, NULL, pkt_out); if (size < 0) return size; + if (size > 0) + *frame_enc = 1; } else { struct FrameListData *cx_frame = av_malloc(sizeof(*cx_frame)); @@ -1430,6 +1444,8 @@ static int queue_frames(AVCodecContext *avctx, struct vpx_codec_ctx *encoder, return AVERROR(ENOMEM); } memcpy(cx_frame->buf, pkt->data.frame.buf, pkt->data.frame.sz); + if (pkt->data.frame.sz > 0) + *frame_enc = 1; coded_frame_add(frame_list, cx_frame); } break; @@ -1693,6 +1709,7 @@ static int vpx_encode(AVCodecContext *avctx, AVPacket *pkt, vpx_svc_layer_id_t layer_id; int layer_id_valid = 0; unsigned long duration = 0; + int frame_enc = 0; if (avctx->qmax >= 0 && enccfg->rc_max_quantizer != avctx->qmax) { struct vpx_codec_enc_cfg cfg = *enccfg; @@ -1856,9 +1873,32 @@ FF_ENABLE_DEPRECATION_WARNINGS } } - coded_size = queue_frames(avctx, &ctx->encoder, &ctx->coded_frame_list, pkt); + coded_size = queue_frames(avctx, &ctx->encoder, &ctx->coded_frame_list, + pkt, ctx->is_alpha, &frame_enc); + if (avctx->codec_id == AV_CODEC_ID_VP8 && frame_enc == 0 && + ctx->tune_content == 2 && frame) { + // VP8 tuned for screen content with aggresive rate control - returned + // OK status code but produced no output, this indicates frame was + // rolled back due to bitrate overshoot - try to encode it again. + av_log(avctx, AV_LOG_VERBOSE, + "Attempting to reencode dropped VP8 screencast frame.\n"); + res = vpx_codec_encode(&ctx->encoder, rawimg, timestamp, + duration, flags, ctx->deadline); + if (res != VPX_CODEC_OK) { + log_encoder_error(avctx, "Error encoding frame"); + return AVERROR_INVALIDDATA; + } + if (!coded_size) + coded_size = queue_frames(avctx, &ctx->encoder, &ctx->coded_frame_list, + pkt, /*queue_only*/0, &frame_enc); + else + queue_frames(avctx, &ctx->encoder, &ctx->coded_frame_list, pkt, + /*queue_only*/1, &frame_enc); + } + if (ctx->is_alpha) { - queue_frames(avctx, &ctx->encoder_alpha, &ctx->alpha_coded_frame_list, NULL); + queue_frames(avctx, &ctx->encoder_alpha, &ctx->alpha_coded_frame_list, + NULL, /*queue_only*/1, &frame_enc); if (ctx->coded_frame_list && ctx->alpha_coded_frame_list) { struct FrameListData *cx_frame = ctx->coded_frame_list; @@ -1946,6 +1986,12 @@ static const AVOption vp8_options[] = { { "auto-alt-ref", "Enable use of alternate reference " "frames (2-pass only)", OFFSET(auto_alt_ref), AV_OPT_TYPE_INT, {.i64 = -1}, -1, 2, VE}, { "cpu-used", "Quality/Speed ratio modifier", OFFSET(cpu_used), AV_OPT_TYPE_INT, {.i64 = 1}, -16, 16, VE}, +#ifdef VPX_CTRL_VP8E_SET_SCREEN_CONTENT_MODE + { "tune-content", "Tune content type", OFFSET(tune_content), AV_OPT_TYPE_INT, {.i64 = -1}, -1, 2, VE, "tune_content"}, + { "default", "Regular video content", 0, AV_OPT_TYPE_CONST, {.i64 = 0}, 0, 0, VE, "tune_content"}, + { "screen", "Screen content mode on", 0, AV_OPT_TYPE_CONST, {.i64 = 1}, 0, 0, VE, "tune_content"}, + { "screen-rate-control", "Screen content mode with aggressive rate control", 0, AV_OPT_TYPE_CONST, {.i64 = 2}, 0, 0, VE, "tune_content"}, +#endif LEGACY_OPTIONS { NULL } };