From patchwork Sun Jul 21 22:53:47 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Almer X-Patchwork-Id: 50670 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a59:a742:0:b0:482:c625:d099 with SMTP id f2csp1711693vqm; Sun, 21 Jul 2024 15:53:59 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCUKdaHgb+x7+1WcH6SQUyvOdYzMqwqchDNz9H1Ygn3jOkJyRnN+B2hYT1jn/xDUib2a7VF6oGcpkS/FDyW1aY9MBY8RRpLCx/53dA== X-Google-Smtp-Source: AGHT+IFKyJtIMIfSuPD8ECZrlHm3H0RVAUmx8iUnDCL4uvtErhKknb+5v+WgAA6UTObUG1S4ZXDG X-Received: by 2002:a2e:9e01:0:b0:2ef:2cbc:9072 with SMTP id 38308e7fff4ca-2ef2cbc911emr14557721fa.49.1721602438863; Sun, 21 Jul 2024 15:53:58 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1721602438; cv=none; d=google.com; s=arc-20160816; b=iUmRcsApNgA8yxera9i9QuWjbdbvfCUM6YmkzlW4yPurWQlso97LbpeVIssCE6y7TA 0BgDl0j+HOo6xZ2tMOL+E6tHoPXMtwcjTsvDEBfmBHUqKgm1DjziK/iZnA3ZXyVcIqVb wBitPB76AvDoOUZFzAqDh7ISeUXh1qsG/qDxArUWf70+xWqmK+QABqj7svxN0tbOL9xB 76xv+kvQ05oOKSwZoY977e9ENjQ2NsJejMDCUBXW3PbrIcFxAsBMRxVl/THbVQVrwU4v 7h4MhIpoJO0hUtRWBA6dnEnvwVCTWOpXO28qIGWY+MHyvFJvMAzMgmfJf0+UxAoeTNsq LK6w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:message-id:date:to:from :dkim-signature:delivered-to; bh=9pGUr8bQMitae/x7wU9HYCGY+rsH6lMGlaNsAusoUPY=; fh=YOA8vD9MJZuwZ71F/05pj6KdCjf6jQRmzLS+CATXUQk=; b=uJBtylniG0eJ2GRL9tzKy0bPnO8SLoZsXlhOISe9e0D38AkVvYN85/vW/e0XXI1q9M 3lrBWi0kILMZ9OXEvoZfGEs4D/KgITBgEvwCD0VIJT95hdzNFe8agRug9nJQThN1J9JO A+JWJC/g1z7DImcdcKT1jd4kLDY8XeLslAybe4z4+Nq9APx7V5/hRBkvNmlvgIzHuidU rHGtDNxTFhMhgaACmJtqHQNvgqHUon+tB++Zbj69HPgRXhhZsbPOavl3kjWNBYfPdp3p x1Bx2JqjJAA/yii45fS9zqo1g5Xv/VrIcfflcrbnr8AYjpoAzgVYvouIuzOUJsI9cDfC YeqQ==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=Foa2iTgo; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com; dara=fail header.i=@gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 38308e7fff4ca-2ef1d8acf9bsi9817301fa.50.2024.07.21.15.53.58; Sun, 21 Jul 2024 15:53:58 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=Foa2iTgo; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com; dara=fail header.i=@gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 56A4D68D6F4; Mon, 22 Jul 2024 01:53:55 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pf1-f176.google.com (mail-pf1-f176.google.com [209.85.210.176]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 841BF68D49F for ; Mon, 22 Jul 2024 01:53:48 +0300 (EEST) Received: by mail-pf1-f176.google.com with SMTP id d2e1a72fcca58-70d18112b60so520436b3a.1 for ; Sun, 21 Jul 2024 15:53:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1721602426; x=1722207226; darn=ffmpeg.org; h=content-transfer-encoding:mime-version:message-id:date:subject:to :from:from:to:cc:subject:date:message-id:reply-to; bh=OWkoOooC2Vp8K0bnjG2uyRK2Hx+L8xHPHTiiysP3VNg=; b=Foa2iTgo/EOXB2j6Bm4L1Sv0/L1Ea4q0sq/HYiBriFCeAiGgm/QzwnbdWk4wr06c9s Y3RomOL/DqMKqQ9mf0wenexH+7ltCmtMeUt6BLY/F2c7YLAsYZL5qMUYhwk3/b9PJQJ6 5Qy64PzG2Rj94ZpNm4jEbP4T0ECu7ONjCa7OgtqlIRMYdhPBMv/PVlqLN3ZOBNmzy6o/ TVK8dvJI97TCLqsYF8nG4bWeUvYRhiEsESV6uI2FP83lzGZ/Mrd6aOAVKNNiWM/VsDeB Q45OKhgL5FFPjdTyk8o2R/NX4MIy/dM6HE3TNV6lmOsWpSmehkE5zLsjqBvmehecpqsH H2fw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1721602426; x=1722207226; h=content-transfer-encoding:mime-version:message-id:date:subject:to :from:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=OWkoOooC2Vp8K0bnjG2uyRK2Hx+L8xHPHTiiysP3VNg=; b=IB+1IEVKeOGdRyqwWDq+kEwVxR1sgs8Vd5W4X0oNGgSjP7IvEz8KBpI8bqXgkujAEz LJoFmkCo0chP5DOcd1rnWfyuW0kSkoV8/b3rY/muAHiTzGiOPLDVkhav+v57fkFaApXN Mps6fej0UMCV6fb4YFO1RNoLyLHQT2/B86fRd4AKR8z23OTZphIHDEz+XT12wCzLyvUu cuM/RlY6ye9A2L3WnI1oYLJiq0RtdHGwATCD83vbtKNe+xOV+mD2embael5DE81xJc8f ja43d5ymR3QzVSFDmZcg7viowb5KHtT7E7yGJWitOuStipCkTsPg3PwIFSs/+NwZyZDk Qv7g== X-Gm-Message-State: AOJu0Yz4q0eKihouWP72vWVpmqCvl6FhiVPdXY6aJkQADNEMnlGEwYPk jflDUyp+mOjSvDdjMS+O32H4YIxXhB348Wv+6OCgP3AD87+GZKr9UfXe2Q== X-Received: by 2002:a05:6a20:7f97:b0:1be:f834:4509 with SMTP id adf61e73a8af0-1c407852fd4mr18316852637.14.1721602425970; Sun, 21 Jul 2024 15:53:45 -0700 (PDT) Received: from localhost.localdomain ([190.194.167.233]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-70cff5538f6sm4223710b3a.100.2024.07.21.15.53.44 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 21 Jul 2024 15:53:45 -0700 (PDT) From: James Almer To: ffmpeg-devel@ffmpeg.org Date: Sun, 21 Jul 2024 19:53:47 -0300 Message-ID: <20240721225350.298-1-jamrial@gmail.com> X-Mailer: git-send-email 2.45.2 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 1/4 v2] avutil/frame: add an LCEVC enhancement data payload side data type X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: xyeUnGIjkzKW Signed-off-by: James Almer --- libavutil/frame.c | 1 + libavutil/frame.h | 5 +++++ 2 files changed, 6 insertions(+) diff --git a/libavutil/frame.c b/libavutil/frame.c index 673a9afb3b..8da5eb28c8 100644 --- a/libavutil/frame.c +++ b/libavutil/frame.c @@ -46,6 +46,7 @@ static const AVSideDataDescriptor sd_props[] = { [AV_FRAME_DATA_DETECTION_BBOXES] = { "Bounding boxes for object detection and classification" }, [AV_FRAME_DATA_DOVI_RPU_BUFFER] = { "Dolby Vision RPU Data" }, [AV_FRAME_DATA_DOVI_METADATA] = { "Dolby Vision Metadata" }, + [AV_FRAME_DATA_LCEVC] = { "Raw LCEVC metadata" }, [AV_FRAME_DATA_STEREO3D] = { "Stereo 3D", AV_SIDE_DATA_PROP_GLOBAL }, [AV_FRAME_DATA_REPLAYGAIN] = { "AVReplayGain", AV_SIDE_DATA_PROP_GLOBAL }, [AV_FRAME_DATA_DISPLAYMATRIX] = { "3x3 displaymatrix", AV_SIDE_DATA_PROP_GLOBAL }, diff --git a/libavutil/frame.h b/libavutil/frame.h index 60bb966f8b..973980ad58 100644 --- a/libavutil/frame.h +++ b/libavutil/frame.h @@ -228,6 +228,11 @@ enum AVFrameSideDataType { * encoding. */ AV_FRAME_DATA_VIDEO_HINT, + + /** + * Raw LCEVC payload data, as a uint8_t array. + */ + AV_FRAME_DATA_LCEVC, }; enum AVActiveFormatDescription { From patchwork Sun Jul 21 22:53:48 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Almer X-Patchwork-Id: 50671 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a59:a742:0:b0:482:c625:d099 with SMTP id f2csp1711721vqm; Sun, 21 Jul 2024 15:54:06 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCUOjKGpntQVtlzDbejnX12M69x7VthnpO2CrnavugdkzZkuV9bE1KQAhPuwoudQ+YQ9wy89IBNPvPOmogayaFp9DVqIXGSrFd64Vg== X-Google-Smtp-Source: AGHT+IElg5uNu7Rh+ddoO9gkK/RFfGmROEyV+ZFkBo2sQEpAPSK4HYL+eto+8o88BBkkI9SjFZXg X-Received: by 2002:ac2:4bc2:0:b0:52c:e07d:229c with SMTP id 2adb3069b0e04-52efb6343fbmr3263734e87.22.1721602446701; Sun, 21 Jul 2024 15:54:06 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1721602446; cv=none; d=google.com; s=arc-20160816; b=sROIlp+JcMw42IstrbNPNVaSYTuf0I/TCRzKu6j95oTDq5n5/9jM11dZ2/F7HiBSg1 RoE8rusTCjPIc8JelLgznLt/AJS/+ZI9M/IggigQqaM5UMYiLbdbfMPCjgBi9u5NXMO2 42SULrIg2zbWT3oKIyORthKQFDujwU/jw7L6jb9a6c5bMCxyW7W/vP5YzsJJpCl5WGne pAAATKQxGRNu3W9CaQiTKQ5mUrh1Pd0xAzB/vwZh/F8XXvLCNBdEAf4cxHNkT29WDeJy ydr+3x4TQ3XojcicP6B2dooGLvgPMc048gGZr9niAhxnMEO3TZVVjGS0DgWMIPFjv0K2 Ty5w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:dkim-signature:delivered-to; bh=xpV6cztm7WyU0rDAlbAdIH6J/zMpVkQFBqD4qSX5LgA=; fh=YOA8vD9MJZuwZ71F/05pj6KdCjf6jQRmzLS+CATXUQk=; b=wQWuwajiB8msCBCdoOKV8ISu7NfquHh9i7IInSt3fNtbxcJdSB+bxt3FQNEbyT7tRG piByYpttogvGLha1EA3Wf8MFxgC4u9dIUj2dNBBMczAwhqATUYun668cZJLh3In4o4BV W5oFMTrpJvzwJ4VRJeO2AKmG1CMcB+QbJ6WIYu3TV0NZLIUCyYmX9Djk1C9u87sEp7Mo jMB/9sKWZbsRiqRpmsTwC5ygWhlcWOmUY7q1hmZimMQer1P5Q/2woTe3159de1Hdjdqc J/trOFWNdKPpd5e5DQ5CkcfiTo03KfA7GdzdM5vVuxHdvDqO71cHZspuvjf5C9yDauBa l2aQ==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=Thr4kcQX; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com; dara=fail header.i=@gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id a640c23a62f3a-a7a3c926eeasi316909766b.682.2024.07.21.15.54.06; Sun, 21 Jul 2024 15:54:06 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=Thr4kcQX; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com; dara=fail header.i=@gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 7117868D75E; Mon, 22 Jul 2024 01:53:58 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-yw1-f181.google.com (mail-yw1-f181.google.com [209.85.128.181]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id A2C8368D49F for ; Mon, 22 Jul 2024 01:53:49 +0300 (EEST) Received: by mail-yw1-f181.google.com with SMTP id 00721157ae682-65f9e25fffaso39563867b3.3 for ; Sun, 21 Jul 2024 15:53:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1721602427; x=1722207227; darn=ffmpeg.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=bQAZY9rqqHgZatvQso+PteZU+7dfhLICax/qGZws1Qw=; b=Thr4kcQXr6dBb7x6SzDyepDlE9F/mI9WruiWasQoh2iXDXFecqE4rWRqq/r8KKGfKo xfweRU7E6615zNIA6Msr2nGB+6GQH4Pe0E2yUHrmPNU4uEnvkEYG0MODoqDB+sH9Q/en Y67GeV05+CuP8PKsFqPnqSib/id9UojxviOjNEBtlqixf72TDdlMCpyWmT0wDEMHkT4n 6E4SF28fVFQqJeJQmrdzzEQ6jU3kl+y76UK2EXjAkl/oRZnIy9EpVsdAAKpS4Uw21wIh QVYNw/xu2l2OTR0JNm8Rb2UjacIa4JsP2yrTq90oSumcczG8aZBqG56srGqJimvRta6s IdLg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1721602427; x=1722207227; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=bQAZY9rqqHgZatvQso+PteZU+7dfhLICax/qGZws1Qw=; b=FwpAzkx7XVz1Xl6RbCA+uN9bv0tAoiSNA1Di56Xl6wImgd7ZhJ+xLI1ZmSmulgsijH Tnd4Ko63tgBenQOhKnKT1C0NsHYmvK8+f6Dhul8apzcilg43BTzKDlQLGVhMD79C2N3E XNgTplZiWEJp1+QJklyOTIjCgL+sY7e/3NcNIM4Etzo9kbBuctTocrDHqd8Y/HkiG5eX SyafW1mqU/GdarPrqzvFSGB3d+e1D1e+jFdEnIcP8+0p9qCEeyjIYlqGtLvT/JQtui5T HxOCODn6mnY9yXvd+HknweWMM6nOnjjK0X6u1uoq7bCEPdTxFCCZO1sbI+wmk3GJTJsg CGxw== X-Gm-Message-State: AOJu0YyMmV9Mgmh7Nt9yKB5wnAyQQzNH3MnIPWyBdJ/t1b5wL4zQTkhK 3yfVfNu7daRjcyeG2JpNTIeW+BHRWJnknUE2iy+lm6H15VthGh1DYV74gg== X-Received: by 2002:a81:69c5:0:b0:64b:44b4:e13 with SMTP id 00721157ae682-66ad8eb8906mr55697447b3.28.1721602427415; Sun, 21 Jul 2024 15:53:47 -0700 (PDT) Received: from localhost.localdomain ([190.194.167.233]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-70cff5538f6sm4223710b3a.100.2024.07.21.15.53.46 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 21 Jul 2024 15:53:46 -0700 (PDT) From: James Almer To: ffmpeg-devel@ffmpeg.org Date: Sun, 21 Jul 2024 19:53:48 -0300 Message-ID: <20240721225350.298-2-jamrial@gmail.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240721225350.298-1-jamrial@gmail.com> References: <20240721225350.298-1-jamrial@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 2/4 v2] libavcodec/h2645_sei: export raw LCEVC metadata X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: yJniROr7xtuK Signed-off-by: James Almer --- libavcodec/h2645_sei.c | 37 +++++++++++++++++++++++++++++++++++++ libavcodec/h2645_sei.h | 5 +++++ libavcodec/itut35.h | 2 ++ 3 files changed, 44 insertions(+) diff --git a/libavcodec/h2645_sei.c b/libavcodec/h2645_sei.c index 7c83747cd0..61d18e14b3 100644 --- a/libavcodec/h2645_sei.c +++ b/libavcodec/h2645_sei.c @@ -99,6 +99,20 @@ static int decode_registered_user_data_dynamic_hdr_vivid(HEVCSEIDynamicHDRVivid } #endif +static int decode_registered_user_data_lcevc(HEVCSEILCEVC *s, + GetByteContext *gb) +{ + int size = bytestream2_get_bytes_left(gb); + + av_buffer_unref(&s->info); + s->info = av_buffer_alloc(size); + if (!s->info) + return AVERROR(ENOMEM); + + bytestream2_get_bufferu(gb, s->info->data, size); + return 0; +} + static int decode_registered_user_data_afd(H2645SEIAFD *h, GetByteContext *gb) { int flag; @@ -142,6 +156,7 @@ static int decode_registered_user_data(H2645SEI *h, GetByteContext *gb, } if (country_code != ITU_T_T35_COUNTRY_CODE_US && + country_code != ITU_T_T35_COUNTRY_CODE_UK && country_code != ITU_T_T35_COUNTRY_CODE_CN) { av_log(logctx, AV_LOG_VERBOSE, "Unsupported User Data Registered ITU-T T35 SEI message (country_code = %d)\n", @@ -173,6 +188,13 @@ static int decode_registered_user_data(H2645SEI *h, GetByteContext *gb, } break; } + case ITU_T_T35_PROVIDER_CODE_LCEVC: { + if (bytestream2_get_bytes_left(gb) < 2) + return AVERROR_INVALIDDATA; + + bytestream2_skipu(gb, 1); // user_data_type_code + return decode_registered_user_data_lcevc(&h->lcevc, gb); + } #if CONFIG_HEVC_SEI case ITU_T_T35_PROVIDER_CODE_CUVA: { const uint16_t cuva_provider_oriented_code = 0x0005; @@ -501,6 +523,11 @@ int ff_h2645_sei_ctx_replace(H2645SEI *dst, const H2645SEI *src) av_buffer_unref(&dst->unregistered.buf_ref[i]); dst->unregistered.nb_buf_ref = 0; + ret = av_buffer_replace(&dst->lcevc.info, + src->lcevc.info); + if (ret < 0) + return ret; + if (src->unregistered.nb_buf_ref) { ret = av_reallocp_array(&dst->unregistered.buf_ref, src->unregistered.nb_buf_ref, @@ -666,6 +693,15 @@ static int h2645_sei_to_side_data(AVCodecContext *avctx, H2645SEI *sei, } } + if (sei->lcevc.info) { + HEVCSEILCEVC *lcevc = &sei->lcevc; + AVFrameSideData *entry = av_frame_side_data_add(sd, nb_sd, AV_FRAME_DATA_LCEVC, + &lcevc->info, 0); + if (!entry) + av_buffer_unref(&lcevc->info); + lcevc->info = NULL; + } + return 0; } @@ -875,6 +911,7 @@ void ff_h2645_sei_reset(H2645SEI *s) av_freep(&s->unregistered.buf_ref); av_buffer_unref(&s->dynamic_hdr_plus.info); av_buffer_unref(&s->dynamic_hdr_vivid.info); + av_buffer_unref(&s->lcevc.info); s->ambient_viewing_environment.present = 0; s->mastering_display.present = 0; diff --git a/libavcodec/h2645_sei.h b/libavcodec/h2645_sei.h index 488dbcad7e..598f78b585 100644 --- a/libavcodec/h2645_sei.h +++ b/libavcodec/h2645_sei.h @@ -50,6 +50,10 @@ typedef struct HEVCSEIDynamicHDRVivid { AVBufferRef *info; } HEVCSEIDynamicHDRVivid; +typedef struct HEVCSEILCEVC { + AVBufferRef *info; +} HEVCSEILCEVC; + typedef struct H2645SEIUnregistered { AVBufferRef **buf_ref; unsigned nb_buf_ref; @@ -126,6 +130,7 @@ typedef struct H2645SEI { H2645SEIAFD afd; HEVCSEIDynamicHDRPlus dynamic_hdr_plus; //< HEVC only HEVCSEIDynamicHDRVivid dynamic_hdr_vivid; //< HEVC only + HEVCSEILCEVC lcevc; H2645SEIUnregistered unregistered; H2645SEIFramePacking frame_packing; H2645SEIDisplayOrientation display_orientation; diff --git a/libavcodec/itut35.h b/libavcodec/itut35.h index ffa7024981..a75ef37929 100644 --- a/libavcodec/itut35.h +++ b/libavcodec/itut35.h @@ -20,11 +20,13 @@ #define AVCODEC_ITUT35_H #define ITU_T_T35_COUNTRY_CODE_CN 0x26 +#define ITU_T_T35_COUNTRY_CODE_UK 0xB4 #define ITU_T_T35_COUNTRY_CODE_US 0xB5 #define ITU_T_T35_PROVIDER_CODE_ATSC 0x31 #define ITU_T_T35_PROVIDER_CODE_CUVA 0x04 #define ITU_T_T35_PROVIDER_CODE_DOLBY 0x3B +#define ITU_T_T35_PROVIDER_CODE_LCEVC 0x50 #define ITU_T_T35_PROVIDER_CODE_SMTPE 0x3C #endif /* AVCODEC_ITUT35_H */ From patchwork Sun Jul 21 22:53:49 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Almer X-Patchwork-Id: 50672 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a59:a742:0:b0:482:c625:d099 with SMTP id f2csp1711745vqm; Sun, 21 Jul 2024 15:54:15 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCUdfaYIrtnONQsPvRQYCOFBtO7PUwaa7ZzIyUU3lccRgBUlPAsAVCq5ULR/xRysXFa2njLE9J0EpvIW948Ab3FAsNEMks3jeimulg== X-Google-Smtp-Source: AGHT+IG/YU4Mj3yB0LIEFxtJVl3VXQ3Za5PDa3UtWZK/h0H+3VIW88ex9LA2FQZb5He016WNOZU9 X-Received: by 2002:a05:6512:3049:b0:52c:e184:3dd6 with SMTP id 2adb3069b0e04-52ef8c4b73cmr4081788e87.0.1721602455068; Sun, 21 Jul 2024 15:54:15 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1721602455; cv=none; d=google.com; s=arc-20160816; b=mLobYFE48p4yZVO4H7dk466Xvj+OVzv3ZbsE7EhtBXGUUuol3d9+3MLoMY2nA5eSeB krsNzPi1AI7U4tf6lZXMGGrYqfQQTZyX1bMS9Kcgr6FYFd4GpgXzEKYqHmhOkNF19b0L NRJO0zwK1B3G2cTlxk4W5jElABvPSkoD3MCIxT7E03TJJ66nwAocY2FEooscnjZGKwOC hFfASZnzHo47STYVNCUcDPVyp+r1IxaCEKs64OAcHrSBnnB7laP4yoe4Ji+ZJC54yjq2 i7oVzxu77ALfL0aypoD8+t/2Lyr4AUU/td0jlZ4xJG6PN+CECiY5Iv7P3Fx4W3RC0ygL HrOA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:dkim-signature:delivered-to; bh=BLwBntY7stqaOKdJ3LCaXiHsaQdW7pJ3jhX7Wc7KdXw=; fh=YOA8vD9MJZuwZ71F/05pj6KdCjf6jQRmzLS+CATXUQk=; b=ONWpUMSpV/ASzGsZHResoC+sSWSNwAapnBXkeN9CB4Ea9+njcGmcZ5gPa9g2jMbP0U H98XZqJ4IuMCzB83pEjK3IPslN45Hc9jCMLnuH/P5x22zeDCmE163nK/dowzWU0aiTJ6 2mLzzZsrNeBL8sLO34VbyC2y/aeowUcImgum4bl5ROqFijA+sVeuPRdIG7N/9MN7O8Vx QK2RZNWScfTjCZWULUo7QQpskplH8ik5xzZ7o8jd2u9chT9Wl4CQFibZunWRZs5v2eq9 ztcYgsDw9sqdhEJjKGDCTHxFL7QegJLqHoBDykLU6czytX5QNibH89MBD6CBCqy1pW+U VxOw==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=U9WGT24a; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com; dara=fail header.i=@gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 2adb3069b0e04-52f030e33d9si310602e87.269.2024.07.21.15.54.14; Sun, 21 Jul 2024 15:54:15 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=U9WGT24a; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com; dara=fail header.i=@gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 154A368D756; Mon, 22 Jul 2024 01:54:00 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pf1-f174.google.com (mail-pf1-f174.google.com [209.85.210.174]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id AC14168D553 for ; Mon, 22 Jul 2024 01:53:51 +0300 (EEST) Received: by mail-pf1-f174.google.com with SMTP id d2e1a72fcca58-70d2e68f5a8so69011b3a.3 for ; Sun, 21 Jul 2024 15:53:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1721602429; x=1722207229; darn=ffmpeg.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=FcFUWYCHnYGhg5h5cHFPEbx3B/SnSSXhINKetfr/LOk=; b=U9WGT24aB4p/BddZWFrzj78Jgk3VxbGB+HAvZLgazrzBxC6A4fkItpcpe3u/+jikp+ 7jGWlZewWDiM3gHsGUjT4PS4A+jAvkejpETZt+5v7yN4NKGNKuVayDahzY51745yx07+ 4B27xA97sQ/prT8AQO9sHzQc30mOlTxHflTH33rx7p9vsS0yg+58ZC9FYrZt3cJQW1/x Ww7jB7j0kjb5wAXJ5lbxK1mw4L/9kMcj9Foi97ik5rWks4uoRfpRfqfxuc3BPG53aVI/ 1ZuIwUNmiUn05IZIxByQs2ZIuZXZhvPvtoukP5B92HykB30Upcj+eFTNcqplChmtG3Ov aS6Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1721602429; x=1722207229; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=FcFUWYCHnYGhg5h5cHFPEbx3B/SnSSXhINKetfr/LOk=; b=oyBYP0Wa8WDPEixgsgEEv44h4b09WSACBSmpg5JR0FZeyCieTE0KwTtjO/4a8ivcf8 uUoEA4cdkUoGWx6jGwyKcuLzdCX0Qg1XxXs9IHtDpVJnMBxS8OgtqnfHrkxZUh8k6dj3 uwJC22DP+bSS+yhcG2ZVEE+ojhHd9p8EAi9buBIKeBD9b9lWCkRIt4zCrxaIHOUesV+B 1oMR1/Zok1uCL8XWcQ0pRrL68nkBpN5IC9gO5wN0B4xwcwOBG3hSBjkNK1ONts0JI62Y VMNxJ4Xr8T9jwaR0nauX+g9o4ua6jvJWfnPv96FYRkiCHPmo7kKbkidc13j/xo3hVGkJ cGZw== X-Gm-Message-State: AOJu0Yz2kZRh/CNEwyxrUiVQA/YZJytNnPn2srXHYQklAD+56bT+P75n XTkX4XywP8BetWHkK9p19QfM0mlhbbI1uVx2cY5JoEbsWxfF3UNfDB48cQ== X-Received: by 2002:a05:6a00:2ea0:b0:70d:2e24:af75 with SMTP id d2e1a72fcca58-70d2e24bb7cmr663511b3a.24.1721602428810; Sun, 21 Jul 2024 15:53:48 -0700 (PDT) Received: from localhost.localdomain ([190.194.167.233]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-70cff5538f6sm4223710b3a.100.2024.07.21.15.53.47 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 21 Jul 2024 15:53:48 -0700 (PDT) From: James Almer To: ffmpeg-devel@ffmpeg.org Date: Sun, 21 Jul 2024 19:53:49 -0300 Message-ID: <20240721225350.298-3-jamrial@gmail.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240721225350.298-1-jamrial@gmail.com> References: <20240721225350.298-1-jamrial@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 3/4 v2] avcodec: add LCEVC decoding support via LCEVCdec X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: d0jRVCPeJQjU Signed-off-by: James Almer --- configure | 3 + doc/general_contents.texi | 13 ++ libavcodec/Makefile | 1 + libavcodec/lcevcdec.c | 276 ++++++++++++++++++++++++++++++++++++++ libavcodec/lcevcdec.h | 44 ++++++ 5 files changed, 337 insertions(+) create mode 100644 libavcodec/lcevcdec.c create mode 100644 libavcodec/lcevcdec.h diff --git a/configure b/configure index f6f5c29fea..d1f32684a6 100755 --- a/configure +++ b/configure @@ -225,6 +225,7 @@ External library support: --enable-libcdio enable audio CD grabbing with libcdio [no] --enable-libcodec2 enable codec2 en/decoding using libcodec2 [no] --enable-libdav1d enable AV1 decoding via libdav1d [no] + --enable-liblcevc_dec enable LCEVC decoding via liblcevc_dec [no] --enable-libdavs2 enable AVS2 decoding via libdavs2 [no] --enable-libdc1394 enable IIDC-1394 grabbing using libdc1394 and libraw1394 [no] @@ -1914,6 +1915,7 @@ EXTERNAL_LIBRARY_LIST=" libcelt libcodec2 libdav1d + liblcevc_dec libdc1394 libflite libfontconfig @@ -6854,6 +6856,7 @@ enabled libcelt && require libcelt celt/celt.h celt_decode -lcelt0 && enabled libcaca && require_pkg_config libcaca caca caca.h caca_create_canvas enabled libcodec2 && require libcodec2 codec2/codec2.h codec2_create -lcodec2 enabled libdav1d && require_pkg_config libdav1d "dav1d >= 0.5.0" "dav1d/dav1d.h" dav1d_version +enabled liblcevc_dec && require_pkg_config liblcevc_dec "lcevc_dec >= 2.0.0" "LCEVC/lcevc_dec.h" LCEVC_CreateDecoder enabled libdavs2 && require_pkg_config libdavs2 "davs2 >= 1.6.0" davs2.h davs2_decoder_open enabled libdc1394 && require_pkg_config libdc1394 libdc1394-2 dc1394/dc1394.h dc1394_new enabled libdrm && check_pkg_config libdrm libdrm xf86drm.h drmGetVersion diff --git a/doc/general_contents.texi b/doc/general_contents.texi index e7cf4f8239..ecaf3979ce 100644 --- a/doc/general_contents.texi +++ b/doc/general_contents.texi @@ -245,6 +245,19 @@ Go to @url{https://github.com/google/liblc3/} and follow the instructions for installing the library. Then pass @code{--enable-liblc3} to configure to enable it. +@section LCEVCdec + +FFmpeg can make use of the liblcevc_dec library for LCEVC enhacement layer +decoding on supported bitstreams. + +Go to @url{https://github.com/v-novaltd/LCEVCdec} and follow the instructions +for installing the library. Then pass @code{--enable-libvpx} to configure to +enable it. + +@float NOTE +LCEVCdec is under the BSD-3-Clause-Clear License. +@end float + @section OpenH264 FFmpeg can make use of the OpenH264 library for H.264 decoding and encoding. diff --git a/libavcodec/Makefile b/libavcodec/Makefile index 771e2b597e..71bc3c8075 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -121,6 +121,7 @@ OBJS-$(CONFIG_INTRAX8) += intrax8.o intrax8dsp.o msmpeg4_vc1_dat OBJS-$(CONFIG_IVIDSP) += ivi_dsp.o OBJS-$(CONFIG_JNI) += ffjni.o jni.o OBJS-$(CONFIG_JPEGTABLES) += jpegtables.o +OBJS-$(CONFIG_LIBLCEVC_DEC) += lcevcdec.o OBJS-$(CONFIG_LCMS2) += fflcms2.o OBJS-$(CONFIG_LLAUDDSP) += lossless_audiodsp.o OBJS-$(CONFIG_LLVIDDSP) += lossless_videodsp.o diff --git a/libavcodec/lcevcdec.c b/libavcodec/lcevcdec.c new file mode 100644 index 0000000000..4edb0b72dc --- /dev/null +++ b/libavcodec/lcevcdec.c @@ -0,0 +1,276 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "libavutil/avassert.h" +#include "libavutil/frame.h" +#include "libavutil/imgutils.h" +#include "libavutil/log.h" +#include "libavutil/mem.h" +#include "decode.h" +#include "lcevcdec.h" + +static LCEVC_ColorFormat map_format(int format) +{ + switch (format) { + case AV_PIX_FMT_YUV420P: + return LCEVC_I420_8; + case AV_PIX_FMT_YUV420P10: + return LCEVC_I420_10_LE; + case AV_PIX_FMT_NV12: + return LCEVC_NV12_8; + case AV_PIX_FMT_NV21: + return LCEVC_NV21_8; + case AV_PIX_FMT_GRAY8: + return LCEVC_GRAY_8; + } + + return LCEVC_ColorFormat_Unknown; +} + +static int alloc_base_frame(void *logctx, LCEVC_DecoderHandle decoder, + const AVFrame *frame, LCEVC_PictureHandle *picture) +{ + LCEVC_PictureDesc desc; + LCEVC_ColorFormat fmt = map_format(frame->format); + LCEVC_PictureLockHandle lock; + uint8_t *data[4] = { NULL }; + int linesizes[4] = { 0 }; + uint32_t planes; + LCEVC_ReturnCode res; + + res = LCEVC_DefaultPictureDesc(&desc, fmt, frame->width, frame->height); + if (res != LCEVC_Success) + return AVERROR_EXTERNAL; + + desc.cropTop = frame->crop_top; + desc.cropBottom = frame->crop_bottom; + desc.cropLeft = frame->crop_left; + desc.cropRight = frame->crop_right; + desc.sampleAspectRatioNum = frame->sample_aspect_ratio.num; + desc.sampleAspectRatioDen = frame->sample_aspect_ratio.den; + + /* Allocate LCEVC Picture */ + res = LCEVC_AllocPicture(decoder, &desc, picture); + if (res != LCEVC_Success) + return AVERROR_EXTERNAL; + + res = LCEVC_LockPicture(decoder, *picture, LCEVC_Access_Write, &lock); + if (res != LCEVC_Success) + return AVERROR_EXTERNAL; + + res = LCEVC_GetPicturePlaneCount(decoder, *picture, &planes); + if (res != LCEVC_Success) + return AVERROR_EXTERNAL; + + for (unsigned i = 0; i < planes; i++) { + LCEVC_PicturePlaneDesc plane; + + res = LCEVC_GetPictureLockPlaneDesc(decoder, lock, i, &plane); + if (res != LCEVC_Success) + return AVERROR_EXTERNAL; + + data[i] = plane.firstSample; + linesizes[i] = plane.rowByteStride; + } + + av_image_copy2(data, linesizes, frame->data, frame->linesize, + frame->format, frame->width, frame->height); + + res = LCEVC_UnlockPicture(decoder, lock); + if (res != LCEVC_Success) + return AVERROR_EXTERNAL; + + return 0; +} + +static int alloc_enhanced_frame(void *logctx, LCEVC_DecoderHandle decoder, + const AVFrame *frame, LCEVC_PictureHandle *picture) +{ + LCEVC_PictureDesc desc ; + LCEVC_ColorFormat fmt = map_format(frame->format); + LCEVC_PicturePlaneDesc planes[4] = { 0 }; + int width = frame->width * 2 / FFMAX(frame->sample_aspect_ratio.den, 1); + int height = frame->height * 2 / FFMAX(frame->sample_aspect_ratio.num, 1); + LCEVC_ReturnCode res; + + res = LCEVC_DefaultPictureDesc(&desc, fmt, width, height); + if (res != LCEVC_Success) + return AVERROR_EXTERNAL; + + /* Set plane description */ + for (int i = 0; i < 4; i++) { + planes[i].firstSample = frame->data[i]; + planes[i].rowByteStride = frame->linesize[i]; + } + + /* Allocate LCEVC Picture */ + res = LCEVC_AllocPictureExternal(decoder, &desc, NULL, planes, picture); + if (res != LCEVC_Success) + return AVERROR_EXTERNAL; + + return 0; +} + +int ff_lcevc_send_frame(void *logctx, FFLCEVCContext *lcevc, const AVFrame *in) +{ + const AVFrameSideData *sd = av_frame_get_side_data(in, AV_FRAME_DATA_LCEVC); + LCEVC_PictureHandle picture; + LCEVC_ReturnCode res; + int ret = 0; + + if (!sd) + return 0; + + res = LCEVC_SendDecoderEnhancementData(lcevc->decoder, in->pts, 0, sd->data, sd->size); + if (res != LCEVC_Success) + return AVERROR_EXTERNAL; + + ret = alloc_base_frame(logctx, lcevc->decoder, in, &picture); + if (ret < 0) + return ret; + + res = LCEVC_SendDecoderBase(lcevc->decoder, in->pts, 0, picture, -1, NULL); + if (res != LCEVC_Success) + return AVERROR_EXTERNAL; + + memset(&picture, 0, sizeof(picture)); + ret = alloc_enhanced_frame(logctx, lcevc->decoder, in, &picture); + if (ret < 0) + return ret; + + res = LCEVC_SendDecoderPicture(lcevc->decoder, picture); + if (res != LCEVC_Success) + return AVERROR_EXTERNAL; + + return 0; +} + +static int generate_output(void *logctx, FFLCEVCContext *lcevc, AVFrame *out) +{ + LCEVC_PictureDesc desc; + LCEVC_DecodeInformation info; + LCEVC_PictureHandle picture; + LCEVC_ReturnCode res; + + res = LCEVC_ReceiveDecoderPicture(lcevc->decoder, &picture, &info); + if (res != LCEVC_Success) + return AVERROR_EXTERNAL; + + res = LCEVC_GetPictureDesc(lcevc->decoder, picture, &desc); + if (res != LCEVC_Success) + return AVERROR_EXTERNAL; + + out->crop_top = desc.cropTop; + out->crop_bottom = desc.cropBottom; + out->crop_left = desc.cropLeft; + out->crop_right = desc.cropRight; + out->sample_aspect_ratio.num = desc.sampleAspectRatioNum; + out->sample_aspect_ratio.den = desc.sampleAspectRatioDen; + out->width = desc.width + out->crop_left + out->crop_right; + out->height = desc.height + out->crop_top + out->crop_bottom; + + return 0; +} + +int ff_lcevc_receive_frame(void *logctx, FFLCEVCContext *lcevc, AVFrame *out) +{ + LCEVC_PictureHandle picture; + LCEVC_ReturnCode res; + int ret; + + ret = generate_output(logctx, lcevc, out); + if (ret < 0) + return ret; + + res = LCEVC_ReceiveDecoderBase (lcevc->decoder, &picture); + if (res != LCEVC_Success) + return AVERROR_EXTERNAL; + + res = LCEVC_FreePicture(lcevc->decoder, picture); + if (res != LCEVC_Success) + return AVERROR_EXTERNAL; + + return 0; +} + +int ff_lcevc_process(void *logctx, AVFrame *frame) +{ + FrameDecodeData *fdd = (FrameDecodeData*)frame->private_ref->data; + AVBufferRef *ref = (AVBufferRef*)fdd->post_process_opaque; + FFLCEVCContext *lcevc = (FFLCEVCContext*)ref->data; + int ret; + + ret = ff_lcevc_send_frame(logctx, lcevc, frame); + if (ret < 0) + return ret; + + ff_lcevc_receive_frame(logctx, lcevc, frame); + if (ret < 0) + return ret; + + return 0; +} + +static void event_callback(LCEVC_DecoderHandle dec, LCEVC_Event event, + LCEVC_PictureHandle pic, const LCEVC_DecodeInformation *info, + const uint8_t *data, uint32_t size, void *logctx) +{ + switch (event) { + case LCEVC_Log: + av_log(logctx, AV_LOG_INFO, "%s\n", data); + break; + default: + break; + } +} + +int ff_lcevc_init(FFLCEVCContext *lcevc, void *logctx) +{ + LCEVC_AccelContextHandle dummy = { 0 }; + + int32_t events[] = { LCEVC_Log }; + + if (LCEVC_CreateDecoder(&lcevc->decoder, dummy) != LCEVC_Success) { + av_log(logctx, AV_LOG_ERROR, "Failed to create LCEVC decoder\n"); + return AVERROR_EXTERNAL; + } + + LCEVC_ConfigureDecoderInt(lcevc->decoder, "log_level", 4); + LCEVC_ConfigureDecoderIntArray(lcevc->decoder, "events", FF_ARRAY_ELEMS(events), events); + LCEVC_SetDecoderEventCallback(lcevc->decoder, event_callback, logctx); + + if (LCEVC_InitializeDecoder(lcevc->decoder) != LCEVC_Success) { + av_log(logctx, AV_LOG_ERROR, "Failed to initialize LCEVC decoder\n"); + return AVERROR_EXTERNAL; + } + + return 0; +} + +void ff_lcevc_free(void *opaque, uint8_t *data) +{ + FFLCEVCContext *lcevc = opaque; + LCEVC_DestroyDecoder(lcevc->decoder); + av_free(data); +} + +void ff_lcevc_unref(void *opaque) +{ + AVBufferRef *ref = opaque; + av_buffer_unref(&ref); +} diff --git a/libavcodec/lcevcdec.h b/libavcodec/lcevcdec.h new file mode 100644 index 0000000000..fe105371d3 --- /dev/null +++ b/libavcodec/lcevcdec.h @@ -0,0 +1,44 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#ifndef AVCODEC_LCEVCDEC_H +#define AVCODEC_LCEVCDEC_H + +#include "config_components.h" + +#include +#if CONFIG_LIBLCEVC_DEC +#include +#else +typedef uintptr_t LCEVC_DecoderHandle; +#endif + +typedef struct FFLCEVCContext { + LCEVC_DecoderHandle decoder; +} FFLCEVCContext; + +struct AVFrame; + +int ff_lcevc_process(void *logctx, struct AVFrame *frame); +int ff_lcevc_send_frame(void *logctx, FFLCEVCContext *lcevc, const AVFrame *in); +int ff_lcevc_receive_frame(void *logctx, FFLCEVCContext *lcevc, AVFrame *out); +int ff_lcevc_init(FFLCEVCContext *lcevc, void *logctx); +void ff_lcevc_unref(void *opaque); +void ff_lcevc_free(void *opaque, uint8_t *data); + +#endif /* AVCODEC_LCEVCDEC_H */ From patchwork Sun Jul 21 22:53:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Almer X-Patchwork-Id: 50673 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a59:a742:0:b0:482:c625:d099 with SMTP id f2csp1711780vqm; Sun, 21 Jul 2024 15:54:23 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCWnYIsOPBK5CDvnW5voQKIvelK0zUxanwmwc2tvysg2DSxcwlT/kgJAJXFMYG+JV/T2rp0lHHbfvsxc1WDy/ZvJp5DlcMQPHVJYEw== X-Google-Smtp-Source: AGHT+IEBIRqA3xbiQkaYvbCuepWj51q+Yl33qGfsbJNKkMiTKIYO3NKJbL7fOBs2A52sYW0oYmlx X-Received: by 2002:a05:6512:2209:b0:52c:dd94:bda9 with SMTP id 2adb3069b0e04-52efb7e8c11mr4175535e87.56.1721602462952; Sun, 21 Jul 2024 15:54:22 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1721602462; cv=none; d=google.com; s=arc-20160816; b=z10fCHTW6v0sks+OuhK6lghqo6JdTpIFpJqGE2oWEfSofWlUgYkzVcdDlfxZ8xVYoF +jYuLXKJXbg5eaWdzzPesAMk0gkkrkU4M7VRjA0KKXLmnQsHg9PTq64B+6xHoqRxSsR8 H+eYL+l3GuyUhyg9Zxkk31/2bAOVICHwyRG+DDZf46ZyyNufNJGFA0jlmxOyPpEGkMvU 1IOOJl05/RGZWKKZ9fiUwlHS1TA2Cs3DoKvjOLgPmkC85Ig1/vPXy/5opK/sBhYb8F95 SIZi4i/xF/dOLibbH3gyqK88qgFo0wk3KWkODh8WCjO+0OJmFR5pqy/Eo5aBExE4o8// kHtQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:dkim-signature:delivered-to; bh=p16gNUVCZrC0oHA0MGJnSLcyfuOZko2FcccvFbf4lCw=; fh=YOA8vD9MJZuwZ71F/05pj6KdCjf6jQRmzLS+CATXUQk=; b=s5c2oVwi8N2bQAxllibX5RrVhb9JDB8T+H1e9eStXCcMNWsf7alH7aimyzu0fgZ1Q9 waC1PjygqUl9vr6TwZb/+EncIh7QWtWAkzqirxNCZX3rBxSUJbKtBz2HFgd6SflTJIDz 8qOMD4Pj3+jSBtWk53Nw86OTeaHhlrt+Xqk4IIWYHk3oszqSxPiFpAM22onLldk0Szpc iVQOZa4MdLkrsVht9haCfh2hj6tW1iErEAuLE4sAyi9poO5cbNSYTPPsssiakIyMClem spL+8o1K3U2wC4SJV2ftQoQSIE9S94OeSc0Mr5OzhaCGt4vEIlCpGgdpdkf4PAVJp++G ZT0Q==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=J0HEB5iI; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com; dara=fail header.i=@gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id a640c23a62f3a-a7a3c78827fsi321269466b.131.2024.07.21.15.54.22; Sun, 21 Jul 2024 15:54:22 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=J0HEB5iI; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com; dara=fail header.i=@gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 5450168D774; Mon, 22 Jul 2024 01:54:01 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-yw1-f170.google.com (mail-yw1-f170.google.com [209.85.128.170]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 0A13968D73B for ; Mon, 22 Jul 2024 01:53:53 +0300 (EEST) Received: by mail-yw1-f170.google.com with SMTP id 00721157ae682-65f9e25fffaso39564167b3.3 for ; Sun, 21 Jul 2024 15:53:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1721602431; x=1722207231; darn=ffmpeg.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=74KXyri8yN9FpuSONNo3lyVL8Bbyj1aTCZBJc8foCqE=; b=J0HEB5iI1+OOdDhF5jwDt4Lb+koTZs6/Mmm2r2RaJZZfmwI3fPhTEN7ed5c/wUWJuz G+GGiT3QPkr1jFoIdOi3lVY+DgZwynUC5/Cxy3u5e8GWlFst16sBMQPm2z66DRudIDpC AW1mhrecDdZkqkGLkyyNY4xKNFm465IWw5pcOMFP51j7HqLRUY5x/huq9sQm2y8quWsO eojtZdtntOFKWI2Bfzrn+Ooah7VaO7fSMYEz/o7r+oXFo3a6vu/dk8VEf7mJ0F4HtKQ0 tyRSaaHMk+igwT4ox7UyeZf4a0yjKAZZfkzEoS8f/3xqqeDIHY3YLGtCLR5DY5HUJfyo tvzA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1721602431; x=1722207231; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=74KXyri8yN9FpuSONNo3lyVL8Bbyj1aTCZBJc8foCqE=; b=FrnZh25uHkfPVUVGQ37rThMIoG8RonQT4z10S8cXqCZtkmhuyvRo+55G3QMoHu+9KR ltU/B+rxE6XyEz3zhA3Nat3tWVV7nW0TMb2sw1XsjDlUSOm5X4W0Byiv4P9saVCG2/QQ MPWmyQ6qzBCBc3jP52Hi8D0V7K5U6KYX30W4ypU5OVkDyyWg/701mkmSsFATBsqjkcMM v3p0VnafPDl0dwjL1BELExtJ3LLaVC97nx7oALoCPGO6lDXRnSYbHbPcgG5jmI/CGDSl uXaq+8MFstIe5d/vC14NX5bGqF3d+FJzP0c69TFTue95bglEQZ3dq7teTCx27yyDoFmD D0TQ== X-Gm-Message-State: AOJu0YxOQ3gMJIK+F9/iHaLA8V5leCNHrf0xPJTar3v+CAO7le3womOg kkl+tA3P3pY+hq+U7aRUwkJf8q69AoEv8xiGcxvM+Y5JWDZmQceWvKilLw== X-Received: by 2002:a05:690c:d8f:b0:643:92a8:ba00 with SMTP id 00721157ae682-66ad5717b8dmr70191727b3.0.1721602430679; Sun, 21 Jul 2024 15:53:50 -0700 (PDT) Received: from localhost.localdomain ([190.194.167.233]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-70cff5538f6sm4223710b3a.100.2024.07.21.15.53.49 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 21 Jul 2024 15:53:49 -0700 (PDT) From: James Almer To: ffmpeg-devel@ffmpeg.org Date: Sun, 21 Jul 2024 19:53:50 -0300 Message-ID: <20240721225350.298-4-jamrial@gmail.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240721225350.298-1-jamrial@gmail.com> References: <20240721225350.298-1-jamrial@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 4/4 v2] avcodec/h264dec: add support for LCEVC enhancement X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 3zVSjqo4d0js Signed-off-by: James Almer --- configure | 2 +- libavcodec/avcodec.h | 5 +++++ libavcodec/h264_picture.c | 1 + libavcodec/h264_slice.c | 15 ++++++++++++++ libavcodec/h264dec.c | 41 ++++++++++++++++++++++++++++++++++++--- libavcodec/h264dec.h | 3 +++ 6 files changed, 63 insertions(+), 4 deletions(-) diff --git a/configure b/configure index d1f32684a6..7bc8c3b0af 100755 --- a/configure +++ b/configure @@ -2968,7 +2968,7 @@ h263i_decoder_select="h263_decoder" h263p_decoder_select="h263_decoder" h263p_encoder_select="h263_encoder" h264_decoder_select="cabac golomb h264chroma h264dsp h264parse h264pred h264qpel h264_sei videodsp" -h264_decoder_suggest="error_resilience" +h264_decoder_suggest="error_resilience liblcevc_dec" hap_decoder_select="snappy texturedsp" hap_encoder_deps="libsnappy" hap_encoder_select="texturedspenc" diff --git a/libavcodec/avcodec.h b/libavcodec/avcodec.h index 2da63c87ea..6fc46207db 100644 --- a/libavcodec/avcodec.h +++ b/libavcodec/avcodec.h @@ -418,6 +418,11 @@ typedef struct RcOverride{ * Do not apply film grain, export it instead. */ #define AV_CODEC_EXPORT_DATA_FILM_GRAIN (1 << 3) +/** + * Decoding only. + * Do not apply picture enhancement layers, export them instead. + */ +#define AV_CODEC_EXPORT_DATA_ENHANCEMENTS (1 << 4) /** * The decoder will keep a reference to the frame and may reuse it later. diff --git a/libavcodec/h264_picture.c b/libavcodec/h264_picture.c index 3234141dbd..1cdb926d15 100644 --- a/libavcodec/h264_picture.c +++ b/libavcodec/h264_picture.c @@ -103,6 +103,7 @@ static void h264_copy_picture_params(H264Picture *dst, const H264Picture *src) dst->mb_height = src->mb_height; dst->mb_stride = src->mb_stride; dst->needs_fg = src->needs_fg; + dst->needs_lcevc = src->needs_lcevc; } int ff_h264_ref_picture(H264Picture *dst, const H264Picture *src) diff --git a/libavcodec/h264_slice.c b/libavcodec/h264_slice.c index a66b75ca80..60b6fa4624 100644 --- a/libavcodec/h264_slice.c +++ b/libavcodec/h264_slice.c @@ -191,6 +191,11 @@ static int alloc_picture(H264Context *h, H264Picture *pic) av_assert0(!pic->f->data[0]); + if (pic->needs_lcevc) { + pic->f->width = FFMAX(h->avctx->width, h->avctx->coded_width) * 2 / FFMAX(h->avctx->sample_aspect_ratio.den, 1); + pic->f->height = FFMAX(h->avctx->height, h->avctx->coded_height) * 2 / FFMAX(h->avctx->sample_aspect_ratio.num, 1); + } + pic->tf.f = pic->f; ret = ff_thread_get_ext_buffer(h->avctx, &pic->tf, pic->reference ? AV_GET_BUFFER_FLAG_REF : 0); @@ -206,6 +211,11 @@ static int alloc_picture(H264Context *h, H264Picture *pic) goto fail; } + if (pic->needs_lcevc) { + pic->f->width = FFMAX(h->avctx->width, h->avctx->coded_width); + pic->f->height = FFMAX(h->avctx->height, h->avctx->coded_height); + } + ret = ff_hwaccel_frame_priv_alloc(h->avctx, &pic->hwaccel_picture_private); if (ret < 0) goto fail; @@ -459,6 +469,8 @@ int ff_h264_update_thread_context(AVCodecContext *dst, h->recovery_frame = h1->recovery_frame; h->non_gray = h1->non_gray; + av_buffer_replace(&h->lcevc, h1->lcevc); + return err; } @@ -519,6 +531,9 @@ static int h264_frame_start(H264Context *h) pic->needs_fg = h->sei.common.film_grain_characteristics.present && !h->avctx->hwaccel && !(h->avctx->export_side_data & AV_CODEC_EXPORT_DATA_FILM_GRAIN); + pic->needs_lcevc = CONFIG_LIBLCEVC_DEC && h->sei.common.lcevc.info && !h->avctx->hwaccel && + !(h->avctx->export_side_data & AV_CODEC_EXPORT_DATA_ENHANCEMENTS); + if ((ret = alloc_picture(h, pic)) < 0) return ret; diff --git a/libavcodec/h264dec.c b/libavcodec/h264dec.c index c77d8f42db..1c3a84f081 100644 --- a/libavcodec/h264dec.c +++ b/libavcodec/h264dec.c @@ -38,6 +38,7 @@ #include "libavutil/video_enc_params.h" #include "codec_internal.h" +#include "decode.h" #include "internal.h" #include "error_resilience.h" #include "avcodec.h" @@ -49,6 +50,7 @@ #include "golomb.h" #include "hwaccel_internal.h" #include "hwconfig.h" +#include "lcevcdec.h" #include "mpegutils.h" #include "profiles.h" #include "rectangle.h" @@ -377,6 +379,8 @@ static av_cold int h264_decode_end(AVCodecContext *avctx) h264_free_pic(h, &h->cur_pic); h264_free_pic(h, &h->last_pic_for_ec); + av_buffer_unref(&h->lcevc); + return 0; } @@ -404,6 +408,9 @@ FF_ENABLE_DEPRECATION_WARNINGS #endif if (!avctx->internal->is_copy) { +#if CONFIG_LIBLCEVC_DEC + FFLCEVCContext *lcevc; +#endif if (avctx->extradata_size > 0 && avctx->extradata) { ret = ff_h264_decode_extradata(avctx->extradata, avctx->extradata_size, &h->ps, &h->is_avc, &h->nal_length_size, @@ -418,6 +425,21 @@ FF_ENABLE_DEPRECATION_WARNINGS ret = 0; } } +#if CONFIG_LIBLCEVC_DEC + lcevc = av_mallocz(sizeof(FFLCEVCContext)); + ret = ff_lcevc_init(lcevc, avctx); + if (ret < 0) { + int explode = avctx->err_recognition & AV_EF_EXPLODE; + av_log(avctx, explode ? AV_LOG_ERROR: AV_LOG_WARNING, + "Error initializing LCEVC\n"); + if (explode) { + av_free(lcevc); + return ret; + } + ret = 0; + } + h->lcevc = av_buffer_create(lcevc, sizeof(FFLCEVCContext), ff_lcevc_free, lcevc, 0); +#endif } if (h->ps.sps && h->ps.sps->bitstream_restriction_flag && @@ -667,9 +689,20 @@ static int decode_nal_units(H264Context *h, const uint8_t *buf, int buf_size) h->setup_finished = 1; } - if (h->avctx->hwaccel && - (ret = FF_HW_CALL(h->avctx, start_frame, buf, buf_size)) < 0) - goto end; + if (h->avctx->hwaccel) { + ret = FF_HW_CALL(h->avctx, start_frame, buf, buf_size); + if (ret < 0) + goto end; + } else if (CONFIG_LIBLCEVC_DEC && h->cur_pic_ptr->needs_lcevc) { + FrameDecodeData *fdd = (FrameDecodeData*)h->cur_pic_ptr->f->private_ref->data; + fdd->post_process_opaque = av_buffer_ref(h->lcevc); + if (!fdd->post_process_opaque) { + ret = -1; + goto end; + } + fdd->post_process_opaque_free = ff_lcevc_unref; + fdd->post_process = ff_lcevc_process; + } } max_slice_ctx = avctx->hwaccel ? 1 : h->nb_slice_ctx; @@ -904,6 +937,8 @@ static int output_frame(H264Context *h, AVFrame *dst, H264Picture *srcp) if (!(h->avctx->export_side_data & AV_CODEC_EXPORT_DATA_FILM_GRAIN)) av_frame_remove_side_data(dst, AV_FRAME_DATA_FILM_GRAIN_PARAMS); + if (!(h->avctx->export_side_data & AV_CODEC_EXPORT_DATA_ENHANCEMENTS)) + av_frame_remove_side_data(dst, AV_FRAME_DATA_LCEVC); return 0; fail: diff --git a/libavcodec/h264dec.h b/libavcodec/h264dec.h index ccd7583bf4..419d699bab 100644 --- a/libavcodec/h264dec.h +++ b/libavcodec/h264dec.h @@ -154,6 +154,7 @@ typedef struct H264Picture { int invalid_gap; int sei_recovery_frame_cnt; int needs_fg; ///< whether picture needs film grain synthesis (see `f_grain`) + int needs_lcevc; ///< whether picture needs LCEVC enhancement const PPS *pps; @@ -580,6 +581,8 @@ typedef struct H264Context { int non_gray; ///< Did we encounter a intra frame after a gray gap frame int noref_gray; int skip_gray; + + AVBufferRef *lcevc; // FFLCEVCContext } H264Context; extern const uint16_t ff_h264_mb_sizes[4];