From patchwork Sun Sep 8 20:07:43 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cameron Gutman X-Patchwork-Id: 51398 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a59:9c4f:0:b0:48e:c0f8:d0de with SMTP id w15csp1284903vqu; Sun, 8 Sep 2024 13:19:08 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCW7CDL7fvorNhPjjNU8LJdn8xUvrhBwd1l+hxnE5yLeV8VW5yuTy4r38CqVBiXRElbt5/5GcPJaKVAyMItU5zoy@gmail.com X-Google-Smtp-Source: AGHT+IHyDnNsG6gZLo5NKOfvTAvB8u+OuwRTo8SZo99ur4EZFXQGEB4OsVbM5nluwDKp5E76F0/I X-Received: by 2002:a05:6512:b84:b0:52f:c0dd:d168 with SMTP id 2adb3069b0e04-536587f1801mr2962997e87.7.1725826748332; Sun, 08 Sep 2024 13:19:08 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1725826748; cv=none; d=google.com; s=arc-20240605; b=b9EvhhED1YewJCHJpR6z+nuTyflzOTUQCm0x2QypdsqJ5r22C1iU1h/l8DCkyEMPBR qwNRb7SbG8WAlGSaWC007TEMBwtue/1hxfrTaP4k7Sk+biO92qeu9dVS1+mDhzTUUoFI OiIuXPs3gGADZmmOf8P+lBKvSNfGzvq5+2czGza55v7XTdF7WV1WJRHqvOXLvgrgWAcQ lXt2/NUL+vc+1UlRbhz1YSu4JqbnPTztoWiXGBUgNW3eKU4HRVVpe9Gcv+FtMHkwJCPm naInxzZb/DX88P7jTcoOitL6+0lVgPmbShO/MFJDuiRAa58iA4XQQTMxL/vWxBh2/wAH 9Izw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:message-id:date:to:from :dkim-signature:delivered-to; bh=zqlX3Y19BkrBOeBuInQn+ynSpxuM84d7pw6bPZ90wig=; fh=w4fzhbMgkwtsNjTAKfmbaWyUdBeE1a22iU1vLh+Ut4o=; b=BFGBv09KFnJdasDLCH8nY2jNY+mnxhw7g0wDgIH4QwnzdSP/1dR4upIKEMI+GsERnA WgBd+Sr0QT75Dp/GEZL2fHXcPN8fegqTJheUlSCrEWUgzVGY7lPhKZsoca6BLcYmAUpY DQ2ffMSjwzgerzSuZQknHDyB6M2RgM+TxdGoxcVEtrQ2Aushv8xfLAxVVQpsa8raReUB Eh9T5YJOLeJKNrZPaCklTLj45R0VDYWKuJVS/Iwo0EqdTX7heB2T+gfBpOgMt9DzOIoD BtKTbPPoGpLJ2DIYndFHzR3ed2odC6VrLrdNWFwzalHVkPXlwNsiKX0qHExGMLsh43fI 9B3g==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=YFe1EO3Q; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com; dara=fail header.i=@gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id a640c23a62f3a-a8d2585eca2si269418066b.183.2024.09.08.13.19.07; Sun, 08 Sep 2024 13:19:08 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=YFe1EO3Q; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com; dara=fail header.i=@gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id B6BF168D79B; Sun, 8 Sep 2024 23:11:17 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-ot1-f43.google.com (mail-ot1-f43.google.com [209.85.210.43]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id CCFF468D4BF for ; Sun, 8 Sep 2024 23:11:11 +0300 (EEST) Received: by mail-ot1-f43.google.com with SMTP id 46e09a7af769-710da8668b3so786961a34.1 for ; Sun, 08 Sep 2024 13:11:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1725826270; x=1726431070; darn=ffmpeg.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=zo1wRRgqzrlv6/qApz70NCHEFVjGsc/gw/Olsip2Vuw=; b=YFe1EO3Q4M1g+IQ/BuKq2fduEMYT2XK/eTFXww5N3/GRnh7dv5T0v4IPhuGAtH/ZE/ 9JFaIDUzH5vRjMyrcTfDFFqaxDIxZ/XdONB+ux5umO/iLwRtnghJoUA1NVS2ELTtQhpK ++1vbMIxx8+iKciM4ZEKnYAzIEJrMiHtvLxin+kXtncqRJ3tzxvafni5igOTmY/JKOu5 dByUWmXxrqnyjRKReYffGVIh51ymgJbW90A68VbwFzfI4mht5xppnvNjc50ji+VQSRSZ zQ7I9ISTxk2CMo+whpfd4d5aPUeZCVc6K1T1IpAi/yUm1W2poAhLpgmOs/MpdCWpnq8g KIvA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725826270; x=1726431070; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=zo1wRRgqzrlv6/qApz70NCHEFVjGsc/gw/Olsip2Vuw=; b=cWv5XqlT8P1/jmvH9WmouMXmoI/xhS0i87GLlB8czXhi2z3TfdbKSPeRfThTE+fLvW IgcAlsfLEFCTk1IdbgPPQr2D9IJAoWF+9gP1dNGMzITAIKdnKzE7wG45EHjvthQrVPNJ vU9X0+64p0fFmJhLVcQLFbzsjDCxb44lBhxHm03/b+xMuxc62SJvS6p9o23d+ng4zLzF WFkWGrSMDV8+pawN5/zV3LyTD2fFgPe5lcZq0lElnDkBse0/zpDtEvnSBZ/ntZsDSkDf CfUK9bQsN/PmMSHppZyOXTlSy6t/8pqFAt/rDKS6ZbapCXVvN7scbPd3+wHntml8PQLL 9ZVQ== X-Gm-Message-State: AOJu0YzanxVyhlXndpfLEKHvc/ZT5Mvy4uS3HGmVwmLWc0hHhOkSPyH+ TjtDx4t9wPNTa46KOyTSMswlRjREgzw+WxVLRiBkh4u8nhAPnXjmlfhZYw== X-Received: by 2002:a05:6830:3499:b0:709:3921:7920 with SMTP id 46e09a7af769-710cc25d71dmr12542068a34.26.1725826269574; Sun, 08 Sep 2024 13:11:09 -0700 (PDT) Received: from localhost.localdomain ([2600:1702:50c6:cf1f:9548:4dc9:5394:e50d]) by smtp.gmail.com with ESMTPSA id 46e09a7af769-710d9dd871esm1009488a34.77.2024.09.08.13.11.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 08 Sep 2024 13:11:09 -0700 (PDT) From: Cameron Gutman To: ffmpeg-devel@ffmpeg.org Date: Sun, 8 Sep 2024 15:07:43 -0500 Message-ID: <20240908201058.541-1-aicommander@gmail.com> X-Mailer: git-send-email 2.43.0.windows.1 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2] avcodec/amfenc: Add support for on-demand key frames X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: ovchinnikov.dmitrii@gmail.com, anton@khirnov.net Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: RM0NxclyN7JJ v2: Use forced_idr option instead of AV_FRAME_FLAG_KEY Signed-off-by: Cameron Gutman --- In response to the feedback from Anton on the previous patch, I opted to switch to a new codec option plus pict_type rather than triggering IDR frames on AV_FRAME_FLAG_KEY. However, I chose to use 'forced_idr' as the name rather than 'forced-idr' to match the rest of the AMF options which all use '_' instead of '-'. --- libavcodec/amfenc.c | 41 +++++++++++++++++++++++++++++++++++++++- libavcodec/amfenc.h | 1 + libavcodec/amfenc_av1.c | 1 + libavcodec/amfenc_h264.c | 1 + libavcodec/amfenc_hevc.c | 1 + 5 files changed, 44 insertions(+), 1 deletion(-) diff --git a/libavcodec/amfenc.c b/libavcodec/amfenc.c index 41eaef9758..9d16345b30 100644 --- a/libavcodec/amfenc.c +++ b/libavcodec/amfenc.c @@ -766,11 +766,50 @@ int ff_amf_receive_packet(AVCodecContext *avctx, AVPacket *avpkt) switch (avctx->codec->id) { case AV_CODEC_ID_H264: AMF_ASSIGN_PROPERTY_INT64(res, surface, AMF_VIDEO_ENCODER_INSERT_AUD, !!ctx->aud); + switch (frame->pict_type) { + case AV_PICTURE_TYPE_I: + if (ctx->forced_idr) { + AMF_ASSIGN_PROPERTY_INT64(res, surface, AMF_VIDEO_ENCODER_INSERT_SPS, 1); + AMF_ASSIGN_PROPERTY_INT64(res, surface, AMF_VIDEO_ENCODER_INSERT_PPS, 1); + AMF_ASSIGN_PROPERTY_INT64(res, surface, AMF_VIDEO_ENCODER_FORCE_PICTURE_TYPE, AMF_VIDEO_ENCODER_PICTURE_TYPE_IDR); + } else { + AMF_ASSIGN_PROPERTY_INT64(res, surface, AMF_VIDEO_ENCODER_FORCE_PICTURE_TYPE, AMF_VIDEO_ENCODER_PICTURE_TYPE_I); + } + break; + case AV_PICTURE_TYPE_P: + AMF_ASSIGN_PROPERTY_INT64(res, surface, AMF_VIDEO_ENCODER_FORCE_PICTURE_TYPE, AMF_VIDEO_ENCODER_PICTURE_TYPE_P); + break; + case AV_PICTURE_TYPE_B: + AMF_ASSIGN_PROPERTY_INT64(res, surface, AMF_VIDEO_ENCODER_FORCE_PICTURE_TYPE, AMF_VIDEO_ENCODER_PICTURE_TYPE_B); + break; + } break; case AV_CODEC_ID_HEVC: AMF_ASSIGN_PROPERTY_INT64(res, surface, AMF_VIDEO_ENCODER_HEVC_INSERT_AUD, !!ctx->aud); + switch (frame->pict_type) { + case AV_PICTURE_TYPE_I: + if (ctx->forced_idr) { + AMF_ASSIGN_PROPERTY_INT64(res, surface, AMF_VIDEO_ENCODER_HEVC_INSERT_HEADER, 1); + AMF_ASSIGN_PROPERTY_INT64(res, surface, AMF_VIDEO_ENCODER_HEVC_FORCE_PICTURE_TYPE, AMF_VIDEO_ENCODER_HEVC_PICTURE_TYPE_IDR); + } else { + AMF_ASSIGN_PROPERTY_INT64(res, surface, AMF_VIDEO_ENCODER_HEVC_FORCE_PICTURE_TYPE, AMF_VIDEO_ENCODER_HEVC_PICTURE_TYPE_I); + } + break; + case AV_PICTURE_TYPE_P: + AMF_ASSIGN_PROPERTY_INT64(res, surface, AMF_VIDEO_ENCODER_HEVC_FORCE_PICTURE_TYPE, AMF_VIDEO_ENCODER_HEVC_PICTURE_TYPE_P); + break; + } + break; + case AV_CODEC_ID_AV1: + if (frame->pict_type == AV_PICTURE_TYPE_I) { + if (ctx->forced_idr) { + AMF_ASSIGN_PROPERTY_INT64(res, surface, AMF_VIDEO_ENCODER_AV1_FORCE_INSERT_SEQUENCE_HEADER, 1); + AMF_ASSIGN_PROPERTY_INT64(res, surface, AMF_VIDEO_ENCODER_AV1_FORCE_FRAME_TYPE, AMF_VIDEO_ENCODER_AV1_FORCE_FRAME_TYPE_KEY); + } else { + AMF_ASSIGN_PROPERTY_INT64(res, surface, AMF_VIDEO_ENCODER_AV1_FORCE_FRAME_TYPE, AMF_VIDEO_ENCODER_AV1_FORCE_FRAME_TYPE_INTRA_ONLY); + } + } break; - //case AV_CODEC_ID_AV1 not supported default: break; } diff --git a/libavcodec/amfenc.h b/libavcodec/amfenc.h index d985d01bb1..0f2abcbd82 100644 --- a/libavcodec/amfenc.h +++ b/libavcodec/amfenc.h @@ -114,6 +114,7 @@ typedef struct AmfContext { int max_b_frames; int qvbr_quality_level; int hw_high_motion_quality_boost; + int forced_idr; // HEVC - specific options diff --git a/libavcodec/amfenc_av1.c b/libavcodec/amfenc_av1.c index 2a7a782063..b40d54f70c 100644 --- a/libavcodec/amfenc_av1.c +++ b/libavcodec/amfenc_av1.c @@ -116,6 +116,7 @@ static const AVOption options[] = { { "none", "no adaptive quantization", 0, AV_OPT_TYPE_CONST, {.i64 = AMF_VIDEO_ENCODER_AV1_AQ_MODE_NONE }, 0, 0, VE, .unit = "adaptive_quantisation_mode" }, { "caq", "context adaptive quantization", 0, AV_OPT_TYPE_CONST, {.i64 = AMF_VIDEO_ENCODER_AV1_AQ_MODE_CAQ }, 0, 0, VE, .unit = "adaptive_quantisation_mode" }, + { "forced_idr", "Force I frames to be IDR frames", OFFSET(forced_idr), AV_OPT_TYPE_BOOL, { .i64 = 0 }, 0, 1, VE }, { "align", "alignment mode", OFFSET(align), AV_OPT_TYPE_INT, {.i64 = AMF_VIDEO_ENCODER_AV1_ALIGNMENT_MODE_NO_RESTRICTIONS }, AMF_VIDEO_ENCODER_AV1_ALIGNMENT_MODE_64X16_ONLY, AMF_VIDEO_ENCODER_AV1_ALIGNMENT_MODE_NO_RESTRICTIONS, VE, .unit = "align" }, { "64x16", "", 0, AV_OPT_TYPE_CONST, {.i64 = AMF_VIDEO_ENCODER_AV1_ALIGNMENT_MODE_64X16_ONLY }, 0, 0, VE, .unit = "align" }, diff --git a/libavcodec/amfenc_h264.c b/libavcodec/amfenc_h264.c index 8edd39c633..959be9eab6 100644 --- a/libavcodec/amfenc_h264.c +++ b/libavcodec/amfenc_h264.c @@ -133,6 +133,7 @@ static const AVOption options[] = { { "me_half_pel", "Enable ME Half Pixel", OFFSET(me_half_pel), AV_OPT_TYPE_BOOL, { .i64 = -1 }, -1, 1, VE }, { "me_quarter_pel", "Enable ME Quarter Pixel", OFFSET(me_quarter_pel),AV_OPT_TYPE_BOOL, { .i64 = -1 }, -1, 1, VE }, + { "forced_idr", "Force I frames to be IDR frames", OFFSET(forced_idr) , AV_OPT_TYPE_BOOL, { .i64 = 0 }, 0, 1, VE }, { "aud", "Inserts AU Delimiter NAL unit", OFFSET(aud) , AV_OPT_TYPE_BOOL, { .i64 = -1 }, -1, 1, VE }, diff --git a/libavcodec/amfenc_hevc.c b/libavcodec/amfenc_hevc.c index 4898824f3a..fc25dcec1a 100644 --- a/libavcodec/amfenc_hevc.c +++ b/libavcodec/amfenc_hevc.c @@ -100,6 +100,7 @@ static const AVOption options[] = { { "me_half_pel", "Enable ME Half Pixel", OFFSET(me_half_pel), AV_OPT_TYPE_BOOL,{ .i64 = -1 }, -1, 1, VE }, { "me_quarter_pel", "Enable ME Quarter Pixel ", OFFSET(me_quarter_pel),AV_OPT_TYPE_BOOL,{ .i64 = -1 }, -1, 1, VE }, + { "forced_idr", "Force I frames to be IDR frames", OFFSET(forced_idr) ,AV_OPT_TYPE_BOOL,{ .i64 = 0 }, 0, 1, VE }, { "aud", "Inserts AU Delimiter NAL unit", OFFSET(aud) ,AV_OPT_TYPE_BOOL,{ .i64 = -1 }, -1, 1, VE },