From patchwork Wed Jan 31 16:23:03 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Dariusz Marcinkiewicz X-Patchwork-Id: 45935 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a21:8786:b0:199:de12:6fa6 with SMTP id ph6csp2943082pzb; Wed, 31 Jan 2024 08:23:32 -0800 (PST) X-Google-Smtp-Source: AGHT+IH4Sb3rdr0U+0lxfJkVQtPlJ3OoeF6Oi8RdstxLugHK7XcaJXV8tmv7RutiJiHv859k6g8m X-Received: by 2002:aa7:d58f:0:b0:55e:f9c4:129 with SMTP id r15-20020aa7d58f000000b0055ef9c40129mr1345890edq.40.1706718211841; Wed, 31 Jan 2024 08:23:31 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1706718211; cv=none; d=google.com; s=arc-20160816; b=N5xWCRVWRKc1SwGwWPFfHUVAwU6C8xYWL8ye/MQCvXkXoH6XBtdTr3qKKLS2qCYRzu vIac5omQuvU9VnYxTvSJdYhGhRSBZVlF/NPnUCWiyh9aV9yKvPtWTFbfGrF6eWcji2n6 2fxsYcFon2q2ResUGSqZNRHru0ok/Tp6zqWNnQ4CsoQA/CpvwBBFf8gYOlZGuqmSIUqt fG+kaRod41IVmEUz2wz/IQ8txCXHxujny6r6Aa0oBUqk+3ZVO/wJc+O0nThGtfWOPKCR uyeCqqB8wmcH/6mpocXA9jOPc0fxwJXyFHGFP070tDagVLPplFXWaWaYxMev2TvjZRTd G2Fw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to:from :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:to:mime-version:message-id:date :delivered-to; bh=eTPona1N7Z5XEcxb3/bGH4EaMGqq/qhj4w3060C1uwI=; fh=WRVZSNL4CLq6exlZZ0LmFXCFI+aiRW9z792QSMV7zcE=; b=bBxOtNOiyVqlmJuQV0nwCeS+0g2wnrbkHb8rcAwTuTHE9Zn/BhhDqBfVDb8s/2De4N 7OMLpPXeRz80G4iImyBba0miaPoOa0kvoED+k8nZC5r7OnVUotOKQ7P8LVpl5/RpsmSe QjmGTnPZ4YG+GXx0oAiAwbPMMgbLbLUsOSAGPxty1mbE4/TKEmF/oiBMK/aFl/AaMSIn E3206j4PBSUlfQivVY80rhypTOAU2ymgKHydtFcp/aF02QPPZQm6MzXr9Rk5xIbu7ilg POHc6NGeYEI0oJv7wxJYB2i1VMJCVOu37KSGxLVmedh0xB4ET9mgUoinHzl02AD0iXNq yOmA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id ij12-20020a056402158c00b0055cd3f3153csi5697038edb.16.2024.01.31.08.23.31; Wed, 31 Jan 2024 08:23:31 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 291CC68D08B; Wed, 31 Jan 2024 18:23:28 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-yb1-f201.google.com (mail-yb1-f201.google.com [209.85.219.201]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 1138C68CD4B for ; Wed, 31 Jan 2024 18:23:21 +0200 (EET) Received: by mail-yb1-f201.google.com with SMTP id 3f1490d57ef6-dc6bad01539so1420273276.3 for ; Wed, 31 Jan 2024 08:23:21 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706718200; x=1707323000; h=content-transfer-encoding:cc:to:from:subject:mime-version :message-id:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=qWbLraEqBOpfKyrmKFzRhjEG5ddPLNRhqh6kQdqTdgc=; b=Is2+WqqQ5NZE20WLsEeZDftsJBMUsqCvV3U5NVuntJ6z8JgCkbdDlzYF/B1nUdmyDL Bt13gZtLI1hyC49O9NRG67oM8uHMYGqyEpSRxbXh87jDfUq18XO62UVtT9Hx5FljTuRY wp7qXQexNeoZgZ2pnJFEtVJ47v8dGa4CuvSxBxbctplBI9jT+5fdQAvsUDgyK75SWRU1 MBkc/dmPGk+aAYFPI/U0dWrk6lE71AITE9aDy9v3rCSTrdUR3PIXCPqCRPfNKmCyeQU0 Z1j69NbsSpXNzETM5MmGvtPga/vanMZ1WOj1NTTyHzK+08ao8VfRZCjjMM2pXIu3D86r 8qsw== X-Gm-Message-State: AOJu0YzCdky5V/c2lbwqMIXQHCUURiw9kFO/Zp6iCdFDyaMsbxsI2RUg xfex7vAdOSjXZ3//D49teY/eBGNu0ZcFIPL6seKabH2qqdYcnQmRyMVZqOxDMO847WNrWuvu/rM RvdYnonan23nKZkiuz/nim4uBogz2iuls8BLRJ0yT5jMFuN+azEgsjs6uoAsaNqAjeuj9gH6j5l fwR5Ks58WTl9yUGaw/zBz/QhWmLwUTczSL0A== X-Received: from reksio2.lul.corp.google.com ([2620:0:1043:12:cde4:cff4:2c51:fec6]) (user=darekm job=sendgmr) by 2002:a05:6902:110f:b0:dc6:c9ff:58b7 with SMTP id o15-20020a056902110f00b00dc6c9ff58b7mr49304ybu.7.1706718196307; Wed, 31 Jan 2024 08:23:16 -0800 (PST) Date: Wed, 31 Jan 2024 17:23:03 +0100 Message-Id: <20240131162303.3134027-1-darekm@google.com> Mime-Version: 1.0 X-Mailer: git-send-email 2.43.0.429.g432eaa2c6b-goog To: ffmpeg-devel@ffmpeg.org Subject: [FFmpeg-devel] [PATCH v2] libavcodec: add tune_content option also for VP8. X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Dariusz Marcinkiewicz via ffmpeg-devel From: Dariusz Marcinkiewicz Reply-To: FFmpeg development discussions and patches Cc: Dariusz Marcinkiewicz , =?utf-8?q?Erik_Spr=C3=A5ng?= Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: hhveJeXUgQJy This exposes VP8E_SET_SCREEN_CONTENT_MODE option from libvpx. Changes since v1: - Put the new param initialzation in the right place, - Account for cases when the encoder's output is queued up. Co-authored-by: Erik Språng Signed-off-by: Dariusz Marcinkiewicz --- doc/encoders.texi | 7 ++++-- libavcodec/libvpxenc.c | 56 ++++++++++++++++++++++++++++++++++++++---- 2 files changed, 56 insertions(+), 7 deletions(-) diff --git a/doc/encoders.texi b/doc/encoders.texi index c9fe6d6143..2a9b38f62a 100644 --- a/doc/encoders.texi +++ b/doc/encoders.texi @@ -2186,6 +2186,11 @@ Enable error resiliency features. Increase sharpness at the expense of lower PSNR. The valid range is [0, 7]. +@item tune-content +Set content type. +For VP8: default (0), screen (1), screen with aggressive rate control (2). +For VP9: default (0), screen (1), film (2). + @item ts-parameters Sets the temporal scalability configuration using a :-separated list of key=value pairs. For example, to specify temporal scalability parameters @@ -2268,8 +2273,6 @@ colorspaces: @end table @item row-mt @var{boolean} Enable row based multi-threading. -@item tune-content -Set content type: default (0), screen (1), film (2). @item corpus-complexity Corpus VBR mode is a variant of standard VBR where the complexity distribution midpoint is passed in rather than calculated for a specific clip or chunk. diff --git a/libavcodec/libvpxenc.c b/libavcodec/libvpxenc.c index 80988a2608..c73c92d49b 100644 --- a/libavcodec/libvpxenc.c +++ b/libavcodec/libvpxenc.c @@ -164,6 +164,7 @@ static const char *const ctlidstr[] = { [VP8E_SET_MAX_INTRA_BITRATE_PCT] = "VP8E_SET_MAX_INTRA_BITRATE_PCT", [VP8E_SET_SHARPNESS] = "VP8E_SET_SHARPNESS", [VP8E_SET_TEMPORAL_LAYER_ID] = "VP8E_SET_TEMPORAL_LAYER_ID", + [VP8E_SET_SCREEN_CONTENT_MODE] = "VP8E_SET_SCREEN_CONTENT_MODE", #if CONFIG_LIBVPX_VP9_ENCODER [VP9E_SET_LOSSLESS] = "VP9E_SET_LOSSLESS", [VP9E_SET_TILE_COLUMNS] = "VP9E_SET_TILE_COLUMNS", @@ -1262,6 +1263,16 @@ static av_cold int vpx_init(AVCodecContext *avctx, #endif } #endif +#ifdef VPX_CTRL_VP8E_SET_SCREEN_CONTENT_MODE + if (avctx->codec_id == AV_CODEC_ID_VP8 && ctx->tune_content >= 0) { + if (ctx->tune_content == 2 && ctx->is_alpha) { + av_log(avctx, AV_LOG_ERROR, + "Transparency encoding with screen mode with aggressive rate control not supported\n"); + return AVERROR(EINVAL); + } + codecctl_int(avctx, VP8E_SET_SCREEN_CONTENT_MODE, ctx->tune_content); + } +#endif av_log(avctx, AV_LOG_DEBUG, "Using deadline: %d\n", ctx->deadline); @@ -1379,14 +1390,15 @@ static int storeframe(AVCodecContext *avctx, struct FrameListData *cx_frame, * @return AVERROR(ENOMEM) on coded frame queue data allocation error */ static int queue_frames(AVCodecContext *avctx, struct vpx_codec_ctx *encoder, - struct FrameListData **frame_list, AVPacket *pkt_out) + struct FrameListData **frame_list, AVPacket *pkt_out, + int queue_only, int *frame_enc) { VPxContext *ctx = avctx->priv_data; const struct vpx_codec_cx_pkt *pkt; const void *iter = NULL; int size = 0; - if (!ctx->is_alpha && *frame_list) { + if (!queue_only && *frame_list) { struct FrameListData *cx_frame = *frame_list; /* return the leading frame if we've already begun queueing */ size = storeframe(avctx, cx_frame, NULL, pkt_out); @@ -1401,7 +1413,7 @@ static int queue_frames(AVCodecContext *avctx, struct vpx_codec_ctx *encoder, while (pkt = vpx_codec_get_cx_data(encoder, &iter)) { switch (pkt->kind) { case VPX_CODEC_CX_FRAME_PKT: - if (!ctx->is_alpha && !size) { + if (!queue_only && !size) { struct FrameListData cx_frame; /* avoid storing the frame when the list is empty and we haven't yet @@ -1411,6 +1423,8 @@ static int queue_frames(AVCodecContext *avctx, struct vpx_codec_ctx *encoder, size = storeframe(avctx, &cx_frame, NULL, pkt_out); if (size < 0) return size; + if (size > 0) + *frame_enc = 1; } else { struct FrameListData *cx_frame = av_malloc(sizeof(*cx_frame)); @@ -1430,6 +1444,8 @@ static int queue_frames(AVCodecContext *avctx, struct vpx_codec_ctx *encoder, return AVERROR(ENOMEM); } memcpy(cx_frame->buf, pkt->data.frame.buf, pkt->data.frame.sz); + if (pkt->data.frame.sz > 0) + *frame_enc = 1; coded_frame_add(frame_list, cx_frame); } break; @@ -1693,6 +1709,7 @@ static int vpx_encode(AVCodecContext *avctx, AVPacket *pkt, vpx_svc_layer_id_t layer_id; int layer_id_valid = 0; unsigned long duration = 0; + int frame_enc = 0; if (avctx->qmax >= 0 && enccfg->rc_max_quantizer != avctx->qmax) { struct vpx_codec_enc_cfg cfg = *enccfg; @@ -1856,9 +1873,32 @@ FF_ENABLE_DEPRECATION_WARNINGS } } - coded_size = queue_frames(avctx, &ctx->encoder, &ctx->coded_frame_list, pkt); + coded_size = queue_frames(avctx, &ctx->encoder, &ctx->coded_frame_list, + pkt, ctx->is_alpha, &frame_enc); + if (avctx->codec_id == AV_CODEC_ID_VP8 && frame_enc == 0 && + ctx->tune_content == 2 && frame) { + // VP8 tuned for screen content with aggresive rate control - returned + // OK status code but produced no output, this indicates frame was + // rolled back due to bitrate overshoot - try to encode it again. + av_log(avctx, AV_LOG_VERBOSE, + "Attempting to reencode dropped VP8 screencast frame.\n"); + res = vpx_codec_encode(&ctx->encoder, rawimg, timestamp, + duration, flags, ctx->deadline); + if (res != VPX_CODEC_OK) { + log_encoder_error(avctx, "Error encoding frame"); + return AVERROR_INVALIDDATA; + } + if (!coded_size) + coded_size = queue_frames(avctx, &ctx->encoder, &ctx->coded_frame_list, + pkt, /*queue_only*/0, &frame_enc); + else + queue_frames(avctx, &ctx->encoder, &ctx->coded_frame_list, pkt, + /*queue_only*/1, &frame_enc); + } + if (ctx->is_alpha) { - queue_frames(avctx, &ctx->encoder_alpha, &ctx->alpha_coded_frame_list, NULL); + queue_frames(avctx, &ctx->encoder_alpha, &ctx->alpha_coded_frame_list, + NULL, /*queue_only*/1, &frame_enc); if (ctx->coded_frame_list && ctx->alpha_coded_frame_list) { struct FrameListData *cx_frame = ctx->coded_frame_list; @@ -1946,6 +1986,12 @@ static const AVOption vp8_options[] = { { "auto-alt-ref", "Enable use of alternate reference " "frames (2-pass only)", OFFSET(auto_alt_ref), AV_OPT_TYPE_INT, {.i64 = -1}, -1, 2, VE}, { "cpu-used", "Quality/Speed ratio modifier", OFFSET(cpu_used), AV_OPT_TYPE_INT, {.i64 = 1}, -16, 16, VE}, +#ifdef VPX_CTRL_VP8E_SET_SCREEN_CONTENT_MODE + { "tune-content", "Tune content type", OFFSET(tune_content), AV_OPT_TYPE_INT, {.i64 = -1}, -1, 2, VE, "tune_content"}, + { "default", "Regular video content", 0, AV_OPT_TYPE_CONST, {.i64 = 0}, 0, 0, VE, "tune_content"}, + { "screen", "Screen content mode on", 0, AV_OPT_TYPE_CONST, {.i64 = 1}, 0, 0, VE, "tune_content"}, + { "screen-rate-control", "Screen content mode with aggressive rate control", 0, AV_OPT_TYPE_CONST, {.i64 = 2}, 0, 0, VE, "tune_content"}, +#endif LEGACY_OPTIONS { NULL } };