From patchwork Tue Mar 19 14:06:34 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Almer X-Patchwork-Id: 47218 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:958e:b0:1a3:31a3:7958 with SMTP id iu14csp1687671pzb; Tue, 19 Mar 2024 07:06:46 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCWtUkbV3EPJKTANbw9C1v1UD8MM01P01W+Ky2QKHnZb/PTFXi4MIfUSttkBdTR6fY1PRY1G+/Ro+Imlz7eOg8zjALHtAC5eHOQeGg== X-Google-Smtp-Source: AGHT+IF3DCp4ajM442yZ1XjW13Hc2GmbPQ3t6IZMPg+p5DwZmmdgSYGENZM/HWFm4JyX0YC9f5UE X-Received: by 2002:a17:906:1c8e:b0:a46:61e0:d99 with SMTP id g14-20020a1709061c8e00b00a4661e00d99mr2051597ejh.54.1710857206052; Tue, 19 Mar 2024 07:06:46 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1710857206; cv=none; d=google.com; s=arc-20160816; b=0NY85VTlWXxrfPNwUu778/WLqITfJi6i4VTxPCY7rvbSSWOcNEvOHQVaLWFvkY0n8X BxVve1hK2IUSshhQ4L73QHPfPddCHAQnexU9DQaNOkv5NP+ZnUG0sT194SmGP/tI6riw IeL2bEC08VHudxM1CwsgbBw6pa/eHsirC//fnQWpXT8f4UU00tFCw3mJbF7DzvfTrbOp FTkaEpUEupEEiC+BKOBCnWrMzkMlK/ynz4cQ13ywr/2jrknfLDYoOYM9A7beo7EWp4u6 WvfUir+cyXT6xSWO8PxoZpIrWfagHPtwFfLpIuLNDX+aZgNkFPAmAOhUmvPc0L7I2Kat Y4Fg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:message-id:date:to:from :dkim-signature:delivered-to; bh=DYZ7jdLbdo7Q2RiMVrENWnYzgkxS4Izd6SRQcEqbsJ0=; fh=YOA8vD9MJZuwZ71F/05pj6KdCjf6jQRmzLS+CATXUQk=; b=qmkp2lRJ59d877ISCVYyVUyRUF9Sgo5bEw8/7cTW64jVJpn62JsvdcqOtz+wDdkj03 Y0JwCpqPlY6a+nuhTAAcMzN9OUC+V//NCFLH8x/aCTP2kxAcXaYoBxQr891fRCPLRRlS aD/AwUzOpRm2xOqAWTEffHv/TZNiKxX6j/4XZ6/xeqPoYzUzaEUyyqyVH3/jJ5Tuq2tZ fSu7/L+zTrIWQ1/THt54+C/WNM37MzuB8mew7AETAh1YNYzlvS7yXlfjhtmdgFBM5FFP OfPIwc3OvdXTUTUGS+fsvALT6NkezbjiXpg0k+7wsSJPm1uJD2W52AAks1nx7WTjweXM I9FA==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=WHIf1YGe; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id d15-20020a17090648cf00b00a46e5820c9fsi397035ejt.300.2024.03.19.07.06.45; Tue, 19 Mar 2024 07:06:46 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=WHIf1YGe; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 5689F68D38D; Tue, 19 Mar 2024 16:06:41 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pl1-f182.google.com (mail-pl1-f182.google.com [209.85.214.182]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id DC1CB68D316 for ; Tue, 19 Mar 2024 16:06:34 +0200 (EET) Received: by mail-pl1-f182.google.com with SMTP id d9443c01a7336-1def2a1aafaso26637025ad.3 for ; Tue, 19 Mar 2024 07:06:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1710857192; x=1711461992; darn=ffmpeg.org; h=content-transfer-encoding:mime-version:message-id:date:subject:to :from:from:to:cc:subject:date:message-id:reply-to; bh=x+hXhcGDEBTw2GP0KB8JyYGhLB9g6aYuCtE3Egmlt/g=; b=WHIf1YGeceaQ1JC44jfSwaze5kQhnHyARrfViwmLMR6oHU6Yuxn0Uoz2FYRFET4xgG MTJB1j+Kgwqh7IIZETXE/5bueSMtT71JEOlodUIifLV95hVBJrAPDgUo6m4N+dnD0lE5 6DWzYcXONu1VsuOXDX1ADnTMU6BKeKyVJh5+OjTy9pVaZlQaEhF/0Ba6KVj+jRAbi9vh v9icmCJZEtmusVNgtVRPpXPQVXQ6KER/Dvfmmp5YWPl750aEPbBnBEyQ/JIcpGSkGq0B 0PcSpCN6Oy+f4i+1g1EZRsjh0JZFupgo/IXh2UPUGVUnS7WQP+xiirEysZEQMf6Ha8E3 +N+A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1710857192; x=1711461992; h=content-transfer-encoding:mime-version:message-id:date:subject:to :from:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=x+hXhcGDEBTw2GP0KB8JyYGhLB9g6aYuCtE3Egmlt/g=; b=N3jgpyQC48TbqoTuQmfqXdLvfDmQYqdQVyz4afEkH9vNSZ7RzA/yJGK221xe4VtSo1 X5mSsJLgvhffS2RDX3tDVzUaxSRwW5Dxq9VGZBUxCEJc1h04CjrfiAj3fyRqPOeqXu9j JWMMJkjloCqzndiKbVJ7mCrr5OoI0AFNI74c+4wKsw+DlT3L2cAMoiDwRTzRLk5nT7gS ZL2WEIyuWNbsplk0ib4KxaPVKgzhoLRAfwBWCOOsWvgAE1nk1F/VDLSg8d0FIhzfuNmG FCK7itVe/D61WH1f5I4OZNAkVm61OdmeqT3EkZhoe1QzhjDe2kqkTdReOHgXmI4x2RNF onvw== X-Gm-Message-State: AOJu0YwB43abGRLZpXI+9/Tm5EXhYU/G1cbGMsB2eBaH5XpotohqZaCM Et9Mh5OtQMLFcHN9Q60cLftQQv9TNYgS+z9Q83rUvLrCzwXNGKJBpV0KTOMA X-Received: by 2002:a17:903:2312:b0:1de:f555:164e with SMTP id d18-20020a170903231200b001def555164emr13766017plh.61.1710857191803; Tue, 19 Mar 2024 07:06:31 -0700 (PDT) Received: from localhost.localdomain ([190.194.167.233]) by smtp.gmail.com with ESMTPSA id b11-20020a170902650b00b001d9aa663282sm10397008plk.266.2024.03.19.07.06.30 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 19 Mar 2024 07:06:31 -0700 (PDT) From: James Almer To: ffmpeg-devel@ffmpeg.org Date: Tue, 19 Mar 2024 11:06:34 -0300 Message-ID: <20240319140636.4323-1-jamrial@gmail.com> X-Mailer: git-send-email 2.44.0 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 1/3] libavcodec/h2645_sei: export raw LCEVC metadata X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: roQxDWi8bURl Signed-off-by: James Almer --- libavcodec/h2645_sei.c | 36 ++++++++++++++++++++++++++++++++++++ libavcodec/h2645_sei.h | 5 +++++ libavcodec/itut35.h | 2 ++ libavutil/frame.c | 1 + libavutil/frame.h | 5 +++++ 5 files changed, 49 insertions(+) diff --git a/libavcodec/h2645_sei.c b/libavcodec/h2645_sei.c index e8eb15524f..a6e71894ff 100644 --- a/libavcodec/h2645_sei.c +++ b/libavcodec/h2645_sei.c @@ -99,6 +99,20 @@ static int decode_registered_user_data_dynamic_hdr_vivid(HEVCSEIDynamicHDRVivid } #endif +static int decode_registered_user_data_lcevc(HEVCSEILCEVC *s, + GetByteContext *gb) +{ + int size = bytestream2_get_bytes_left(gb); + + av_buffer_unref(&s->info); + s->info = av_buffer_alloc(size); + if (!s->info) + return AVERROR(ENOMEM); + + bytestream2_get_bufferu(gb, s->info->data, size); + return 0; +} + static int decode_registered_user_data_afd(H2645SEIAFD *h, GetByteContext *gb) { int flag; @@ -142,6 +156,7 @@ static int decode_registered_user_data(H2645SEI *h, GetByteContext *gb, } if (country_code != ITU_T_T35_COUNTRY_CODE_US && + country_code != ITU_T_T35_COUNTRY_CODE_UK && country_code != ITU_T_T35_COUNTRY_CODE_CN) { av_log(logctx, AV_LOG_VERBOSE, "Unsupported User Data Registered ITU-T T35 SEI message (country_code = %d)\n", @@ -173,6 +188,13 @@ static int decode_registered_user_data(H2645SEI *h, GetByteContext *gb, } break; } + case ITU_T_T35_PROVIDER_CODE_LCEVC: { + if (bytestream2_get_bytes_left(gb) < 2) + return AVERROR_INVALIDDATA; + + bytestream2_skipu(gb, 1); // user_data_type_code + return decode_registered_user_data_lcevc(&h->lcevc, gb); + } #if CONFIG_HEVC_SEI case ITU_T_T35_PROVIDER_CODE_CUVA: { const uint16_t cuva_provider_oriented_code = 0x0005; @@ -483,6 +505,11 @@ int ff_h2645_sei_ctx_replace(H2645SEI *dst, const H2645SEI *src) av_buffer_unref(&dst->unregistered.buf_ref[i]); dst->unregistered.nb_buf_ref = 0; + ret = av_buffer_replace(&dst->lcevc.info, + src->lcevc.info); + if (ret < 0) + return ret; + if (src->unregistered.nb_buf_ref) { ret = av_reallocp_array(&dst->unregistered.buf_ref, src->unregistered.nb_buf_ref, @@ -773,6 +800,14 @@ int ff_h2645_sei_to_frame(AVFrame *frame, H2645SEI *sei, } } + if (sei->lcevc.info) { + HEVCSEILCEVC *lcevc = &sei->lcevc; + AVFrameSideData *sd = av_frame_new_side_data_from_buf(frame, AV_FRAME_DATA_LCEVC, lcevc->info); + if (!sd) + av_buffer_unref(&lcevc->info); + lcevc->info = NULL; + } + return 0; } @@ -786,6 +821,7 @@ void ff_h2645_sei_reset(H2645SEI *s) av_freep(&s->unregistered.buf_ref); av_buffer_unref(&s->dynamic_hdr_plus.info); av_buffer_unref(&s->dynamic_hdr_vivid.info); + av_buffer_unref(&s->lcevc.info); s->ambient_viewing_environment.present = 0; s->mastering_display.present = 0; diff --git a/libavcodec/h2645_sei.h b/libavcodec/h2645_sei.h index 0ebf48011a..70d7c2277c 100644 --- a/libavcodec/h2645_sei.h +++ b/libavcodec/h2645_sei.h @@ -48,6 +48,10 @@ typedef struct HEVCSEIDynamicHDRVivid { AVBufferRef *info; } HEVCSEIDynamicHDRVivid; +typedef struct HEVCSEILCEVC { + AVBufferRef *info; +} HEVCSEILCEVC; + typedef struct H2645SEIUnregistered { AVBufferRef **buf_ref; unsigned nb_buf_ref; @@ -124,6 +128,7 @@ typedef struct H2645SEI { H2645SEIAFD afd; HEVCSEIDynamicHDRPlus dynamic_hdr_plus; //< HEVC only HEVCSEIDynamicHDRVivid dynamic_hdr_vivid; //< HEVC only + HEVCSEILCEVC lcevc; H2645SEIUnregistered unregistered; H2645SEIFramePacking frame_packing; H2645SEIDisplayOrientation display_orientation; diff --git a/libavcodec/itut35.h b/libavcodec/itut35.h index ffa7024981..a75ef37929 100644 --- a/libavcodec/itut35.h +++ b/libavcodec/itut35.h @@ -20,11 +20,13 @@ #define AVCODEC_ITUT35_H #define ITU_T_T35_COUNTRY_CODE_CN 0x26 +#define ITU_T_T35_COUNTRY_CODE_UK 0xB4 #define ITU_T_T35_COUNTRY_CODE_US 0xB5 #define ITU_T_T35_PROVIDER_CODE_ATSC 0x31 #define ITU_T_T35_PROVIDER_CODE_CUVA 0x04 #define ITU_T_T35_PROVIDER_CODE_DOLBY 0x3B +#define ITU_T_T35_PROVIDER_CODE_LCEVC 0x50 #define ITU_T_T35_PROVIDER_CODE_SMTPE 0x3C #endif /* AVCODEC_ITUT35_H */ diff --git a/libavutil/frame.c b/libavutil/frame.c index 079cf6595b..f872b1c67d 100644 --- a/libavutil/frame.c +++ b/libavutil/frame.c @@ -822,6 +822,7 @@ const char *av_frame_side_data_name(enum AVFrameSideDataType type) case AV_FRAME_DATA_DOVI_RPU_BUFFER: return "Dolby Vision RPU Data"; case AV_FRAME_DATA_DOVI_METADATA: return "Dolby Vision Metadata"; case AV_FRAME_DATA_AMBIENT_VIEWING_ENVIRONMENT: return "Ambient viewing environment"; + case AV_FRAME_DATA_LCEVC: return "Raw LCEVC metadata"; } return NULL; } diff --git a/libavutil/frame.h b/libavutil/frame.h index b94687941d..8959922f85 100644 --- a/libavutil/frame.h +++ b/libavutil/frame.h @@ -224,6 +224,11 @@ enum AVFrameSideDataType { * encoding. */ AV_FRAME_DATA_VIDEO_HINT, + + /** + * Raw LCEVC payload data, as a uint8_t array. + */ + AV_FRAME_DATA_LCEVC, }; enum AVActiveFormatDescription { From patchwork Tue Mar 19 14:06:35 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Almer X-Patchwork-Id: 47219 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:958e:b0:1a3:31a3:7958 with SMTP id iu14csp1687872pzb; Tue, 19 Mar 2024 07:06:59 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCWRFKfZe2qJeTsdaEf0RDiSnTYhPUfzk7UF2PWUpveOuKRnSOg1Y4RufpXYsNlnV52S2eoUCCJmH+ghRd3a42wQ0aX7RovYmOL7hg== X-Google-Smtp-Source: AGHT+IFzijSjPA6dmZ8vz2ZVPAT4I3YuVwAfv/Z4CNPzxBT3rE7WOWOXIwUWHr2XJH4TwhiNzgcL X-Received: by 2002:ac2:4d9a:0:b0:513:d82a:8d3a with SMTP id g26-20020ac24d9a000000b00513d82a8d3amr7958610lfe.35.1710857218895; Tue, 19 Mar 2024 07:06:58 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1710857218; cv=none; d=google.com; s=arc-20160816; b=J/AgfhJJNm3QfS6mE+uz5lnlRTIIS3dC6xbj6NEVlHI6SDN3AgHPOf/K36hyZmYzwM PgfIAvSNcA1k2wb2FENJDrzIM3PQBl6n/VOr6HDmQRZqEIA2rpWEa2Gef57lbZaMbvfF J+vyhEGXSV7LMkVzTCh/TPUW+aLtFf8YZ4qGMh9RqinMMARwnjZtYx3bB/DzRDh1DIOd 4sUgwa3nreGj+L9BHYpgEhJSIryFoCBmIOE+GRByrSpSW2Aw7zcI/HNt5O3PGzvxM/qj lm2b13YwcwSmTX1Ik+W/NNTuDMqTuoNIZDA71PStFCQljXsZDFN1NQecmlnNrILkGL2w rj0w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:dkim-signature:delivered-to; bh=RDTkUabGU7ixvW3VF5vrH1/Kt6gQn2TmC0vwZKaHgSk=; fh=YOA8vD9MJZuwZ71F/05pj6KdCjf6jQRmzLS+CATXUQk=; b=PxkV3hr3DDmvln9nbNw62o1FdXKZJ3kGF9KrPQ23AZ4iPu8NnlhbYHH3Kyb/rGC11R PoyhscYPe8sO+qT+AG+XjsO5W89lee47xi+OQi0h4+CrRtzv+kSApxTLySz6JSyPqXkw 2twgGFDC/g4bdoaeg0/6A4tQs4s++9Y2kx520qfoIuAIWMwzSoj+6c53fQHUpfOuARBB yQarGHP2WT+qmRIJ7qP38gnPgaIAgJorT3WqkQZeKoJvL7/8Qt8Db7IrZKoDP227JUm0 Drp9Tla8zrIjf9g+otbeThwOQG3mifoMBDSXFXRt4VCz9wR0KjRZsxu7ww2EDjzjUE+A /cxA==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=FFDR0q7S; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id qk15-20020a1709077f8f00b00a46af0b9616si2867670ejc.1054.2024.03.19.07.06.58; Tue, 19 Mar 2024 07:06:58 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=FFDR0q7S; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id B466068D39A; Tue, 19 Mar 2024 16:06:42 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pl1-f172.google.com (mail-pl1-f172.google.com [209.85.214.172]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id E843B68D38F for ; Tue, 19 Mar 2024 16:06:36 +0200 (EET) Received: by mail-pl1-f172.google.com with SMTP id d9443c01a7336-1dffa5e3f2dso17831365ad.2 for ; Tue, 19 Mar 2024 07:06:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1710857194; x=1711461994; darn=ffmpeg.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=4KMymU2lNcpDP5vU1qEkMBvlnpBBb+MbVTnBPWixxM4=; b=FFDR0q7SAsQUCGBfp0Y90Vd/Lf5x9kZ3nqF4vM/wuaAOjXpEiK0fR5ORE/gvSeXCod i2VVzK/70+ZxIjfk6KN1/0+PjtSFXSy/l8lTsj4w878ifwfC0r1dK1I2k7OPq7QSIpF5 mI2r6ucj1uaJwmfVb6QiXtwA0x2xhiNVYdLHYebyjyTxPqhBuMUj8OLpUCwj7WfGbgzq J+ZqA8ecVLSixDoMkCUYveVK3c8iiccEwWQGsH2vQCfmKpkhURO4KxcHDCu/7Iseiwi/ PttydfGybEpX/vMJ+TXIhhP7T1F/j8HF9leB4vKuxtDfREyr5ZSSskD0dJuzhqLrghzJ ZEIw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1710857194; x=1711461994; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=4KMymU2lNcpDP5vU1qEkMBvlnpBBb+MbVTnBPWixxM4=; b=nXzZ6NhBLwZlEbGGfYdEEqeCd82zdLNNLNNHM7apr0y+TyMuCMCTMqTc21c37+vBIf iKzx9iWFEnhxL0FgkQ0UKuoal0eVd+CvBocXSNSy1TA5ueao47790oaPiSvP6PrZpvQw y8NTvUwv7oCLWNlF5tk7W51CqXkpLxd/uwnHlA5rSwmyIXjhIepMIJF6NGVDpmjlxniG bopjpYns4vAGn18MwykW0z9ZCsfphY48nJ8btjnUJJMvF26gKrdCF1QUVZAbtUt4u+Gt d6s6KruTucVSKLyGWUEOG4ZB25Nsz+anq5ZAjaMlDYddDDekmuD4HQJNBaHio6B0BLPj kIFg== X-Gm-Message-State: AOJu0Ywt2KnTjP1y3lfw+U1W0St/EmQhp6RcgbWLqp4MzwDTKlBUBhHS nyTdiTML0RZgP4BfwYxq+o1uGdu/LDbY/Odabgc6L2iuIGNUjh7ZytNSnH1U X-Received: by 2002:a17:902:c949:b0:1dd:819f:9503 with SMTP id i9-20020a170902c94900b001dd819f9503mr3230188pla.12.1710857193791; Tue, 19 Mar 2024 07:06:33 -0700 (PDT) Received: from localhost.localdomain ([190.194.167.233]) by smtp.gmail.com with ESMTPSA id b11-20020a170902650b00b001d9aa663282sm10397008plk.266.2024.03.19.07.06.32 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 19 Mar 2024 07:06:32 -0700 (PDT) From: James Almer To: ffmpeg-devel@ffmpeg.org Date: Tue, 19 Mar 2024 11:06:35 -0300 Message-ID: <20240319140636.4323-2-jamrial@gmail.com> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240319140636.4323-1-jamrial@gmail.com> References: <20240319140636.4323-1-jamrial@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 2/3] avfilter: add an LCEVC decoding filter X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: kb+XzxMYfAja Signed-off-by: James Almer --- configure | 4 + libavfilter/Makefile | 1 + libavfilter/allfilters.c | 1 + libavfilter/vf_lcevc.c | 466 +++++++++++++++++++++++++++++++++++++++ 4 files changed, 472 insertions(+) create mode 100644 libavfilter/vf_lcevc.c diff --git a/configure b/configure index e019d1b996..4022d13f76 100755 --- a/configure +++ b/configure @@ -224,6 +224,7 @@ External library support: --enable-libcdio enable audio CD grabbing with libcdio [no] --enable-libcodec2 enable codec2 en/decoding using libcodec2 [no] --enable-libdav1d enable AV1 decoding via libdav1d [no] + --enable-liblcevc_dec enable AV1 decoding via liblcevc_dec [no] --enable-libdavs2 enable AVS2 decoding via libdavs2 [no] --enable-libdc1394 enable IIDC-1394 grabbing using libdc1394 and libraw1394 [no] @@ -1861,6 +1862,7 @@ EXTERNAL_LIBRARY_LIST=" libcelt libcodec2 libdav1d + liblcevc_dec libdc1394 libflite libfontconfig @@ -3789,6 +3791,7 @@ iccgen_filter_deps="lcms2" interlace_filter_deps="gpl" kerndeint_filter_deps="gpl" ladspa_filter_deps="ladspa libdl" +lcevc_filter_deps="liblcevc_dec" lensfun_filter_deps="liblensfun version3" libplacebo_filter_deps="libplacebo vulkan" lv2_filter_deps="lv2" @@ -6801,6 +6804,7 @@ enabled libjxl && require_pkg_config libjxl "libjxl >= 0.7.0" jxl/dec enabled libklvanc && require libklvanc libklvanc/vanc.h klvanc_context_create -lklvanc enabled libkvazaar && require_pkg_config libkvazaar "kvazaar >= 2.0.0" kvazaar.h kvz_api_get enabled liblensfun && require_pkg_config liblensfun lensfun lensfun.h lf_db_create +enabled liblcevc_dec && require_pkg_config liblcevc_dec "lcevc_dec >= 2.0.0" "LCEVC/lcevc_dec.h" LCEVC_CreateDecoder if enabled libmfx && enabled libvpl; then die "ERROR: can not use libmfx and libvpl together" diff --git a/libavfilter/Makefile b/libavfilter/Makefile index 994d9773ba..57f27a1221 100644 --- a/libavfilter/Makefile +++ b/libavfilter/Makefile @@ -359,6 +359,7 @@ OBJS-$(CONFIG_INTERLEAVE_FILTER) += f_interleave.o OBJS-$(CONFIG_KERNDEINT_FILTER) += vf_kerndeint.o OBJS-$(CONFIG_KIRSCH_FILTER) += vf_convolution.o OBJS-$(CONFIG_LAGFUN_FILTER) += vf_lagfun.o +OBJS-$(CONFIG_LCEVC_FILTER) += vf_lcevc.o OBJS-$(CONFIG_LATENCY_FILTER) += f_latency.o OBJS-$(CONFIG_LENSCORRECTION_FILTER) += vf_lenscorrection.o OBJS-$(CONFIG_LENSFUN_FILTER) += vf_lensfun.o diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c index 149bf50997..54d6ea1043 100644 --- a/libavfilter/allfilters.c +++ b/libavfilter/allfilters.c @@ -337,6 +337,7 @@ extern const AVFilter ff_vf_kerndeint; extern const AVFilter ff_vf_kirsch; extern const AVFilter ff_vf_lagfun; extern const AVFilter ff_vf_latency; +extern const AVFilter ff_vf_lcevc; extern const AVFilter ff_vf_lenscorrection; extern const AVFilter ff_vf_lensfun; extern const AVFilter ff_vf_libplacebo; diff --git a/libavfilter/vf_lcevc.c b/libavfilter/vf_lcevc.c new file mode 100644 index 0000000000..e18f0945f9 --- /dev/null +++ b/libavfilter/vf_lcevc.c @@ -0,0 +1,466 @@ +/* + * This file is part of FFmpeg. + * + * Copyright (c) 2024 James Almer + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include + +#include + +#include "libavutil/internal.h" +#include "libavutil/opt.h" +#include "filters.h" +#include "internal.h" +#include "video.h" + +typedef struct LCEVCContext { + LCEVC_DecoderHandle decoder; + LCEVC_PictureHandle base; + AVFrame *in; + int w, h; + int status; +} LCEVCContext; + +static LCEVC_ColorFormat map_format(int format) +{ + switch (format) { + case AV_PIX_FMT_YUV420P: + return LCEVC_I420_8; + case AV_PIX_FMT_YUV420P10LE: + return LCEVC_I420_10_LE; + case AV_PIX_FMT_YUV420P12LE: + return LCEVC_I420_12_LE; + case AV_PIX_FMT_NV12: + return LCEVC_NV12_8; + case AV_PIX_FMT_NV21: + return LCEVC_NV21_8; + case AV_PIX_FMT_GRAY8: + return LCEVC_GRAY_8; + case AV_PIX_FMT_GRAY10LE: + return LCEVC_GRAY_10_LE; + case AV_PIX_FMT_GRAY12LE: + return LCEVC_GRAY_12_LE; + case AV_PIX_FMT_RGB24: + return LCEVC_RGB_8; + case AV_PIX_FMT_BGR24: + return LCEVC_BGR_8; + case AV_PIX_FMT_RGBA: + return LCEVC_RGBA_8; + case AV_PIX_FMT_ARGB: + return LCEVC_ARGB_8; + case AV_PIX_FMT_ABGR: + return LCEVC_ABGR_8; + case AV_PIX_FMT_BGRA: + return LCEVC_BGRA_8; + } + + return LCEVC_ColorFormat_Unknown; +} + +static inline LCEVC_ColorRange map_range(int range) +{ + switch (range) { + case AVCOL_RANGE_MPEG: + return LCEVC_ColorRange_Limited; + case AVCOL_RANGE_JPEG: + return LCEVC_ColorRange_Full; + } + + return LCEVC_ColorRange_Unknown; +} + +static inline enum AVColorRange map_av_range(int range) +{ + switch (range) { + case LCEVC_ColorRange_Limited: + return AVCOL_RANGE_MPEG; + case LCEVC_ColorRange_Full: + return AVCOL_RANGE_JPEG; + } + + return AVCOL_RANGE_UNSPECIFIED; +} + +static int alloc_base_frame(AVFilterLink *inlink, const AVFrame *in, + LCEVC_PictureHandle *picture) +{ + AVFilterContext *ctx = inlink->dst; + LCEVCContext *lcevc = ctx->priv; + LCEVC_PictureDesc desc; + LCEVC_PicturePlaneDesc planes[AV_VIDEO_MAX_PLANES] = { 0 }; + LCEVC_ColorFormat fmt = map_format(in->format); + int width = in->width - in->crop_left - in->crop_right; + int height = in->height - in->crop_top - in->crop_bottom; + LCEVC_ReturnCode res; + + res = LCEVC_DefaultPictureDesc(&desc, fmt, width, height); + if (res != LCEVC_Success) { + av_log(ctx, AV_LOG_ERROR, "LCEVC_DefaultPictureDesc failed\n"); + return AVERROR_EXTERNAL; + } + + for (int i = 0; i < AV_VIDEO_MAX_PLANES; i++) { + planes[i].firstSample = in->data[i]; + planes[i].rowByteStride = in->linesize[i]; + } + + desc.cropTop = in->crop_top; + desc.cropBottom = in->crop_bottom; + desc.cropLeft = in->crop_left; + desc.cropRight = in->crop_right; + desc.sampleAspectRatioNum = in->sample_aspect_ratio.num; + desc.sampleAspectRatioDen = in->sample_aspect_ratio.den; + desc.colorRange = map_range(in->color_range); + desc.colorPrimaries = (LCEVC_ColorPrimaries)in->color_primaries; + desc.matrixCoefficients = (LCEVC_MatrixCoefficients)in->colorspace; + desc.transferCharacteristics = (LCEVC_TransferCharacteristics)in->color_trc; + av_log(ctx, AV_LOG_DEBUG, "in PTS %"PRId64", %dx%d, " + "%"SIZE_SPECIFIER"/%"SIZE_SPECIFIER"/%"SIZE_SPECIFIER"/%"SIZE_SPECIFIER", " + "SAR %d:%d\n", + in->pts, in->width, in->height, + in->crop_top, in->crop_bottom, in->crop_left, in->crop_right, + in->sample_aspect_ratio.num, in->sample_aspect_ratio.den); + + res = LCEVC_AllocPictureExternal(lcevc->decoder, &desc, NULL, planes, picture); + if (res != LCEVC_Success) { + av_log(ctx, AV_LOG_ERROR, "LCEVC_AllocPictureExternal to allocate a buffer for a base frame\n"); + return AVERROR_EXTERNAL; + } + + return 0; +} + +static int send_frame(AVFilterLink *inlink) +{ + AVFilterContext *ctx = inlink->dst; + LCEVCContext *lcevc = ctx->priv; + const AVFrameSideData *sd = av_frame_get_side_data(lcevc->in, AV_FRAME_DATA_LCEVC); + LCEVC_ReturnCode res; + + // lcevc->base will be set only if a previous LCEVC_SendDecoderBase() call failed with LCEVC_Again, + // in which case we'll try sending the picture again after having fetched at least one enhanced frame. + if (!lcevc->base.hdl) { + int ret = alloc_base_frame(inlink, lcevc->in, &lcevc->base); + if (ret < 0) + return ret; + } + + if (sd) { + res = LCEVC_SendDecoderEnhancementData(lcevc->decoder, lcevc->in->pts, 0, sd->data, sd->size); + if (res == LCEVC_Again) + return AVERROR(EAGAIN); + else if (res != LCEVC_Success) { + av_log(ctx, AV_LOG_ERROR, "LCEVC_SendDecoderEnhancementData failed\n"); + return AVERROR_EXTERNAL; + } + } + + res = LCEVC_SendDecoderBase(lcevc->decoder, lcevc->in->pts, 0, lcevc->base, -1, lcevc->in); + if (res == LCEVC_Success) { + lcevc->in = NULL; + memset(&lcevc->base, 0, sizeof(lcevc->base)); + } else if (res == LCEVC_Again) + return AVERROR(EAGAIN); + else { + av_log(ctx, AV_LOG_ERROR, "LCEVC_SendDecoderBase failed\n"); + return AVERROR_EXTERNAL; + } + + return 0; +} + +static int alloc_enhanced_frame(AVFilterLink *inlink, const AVFrame *out, + LCEVC_PictureHandle *picture) +{ + AVFilterContext *ctx = inlink->dst; + LCEVCContext *lcevc = ctx->priv; + LCEVC_PictureDesc desc; + LCEVC_PicturePlaneDesc planes[AV_VIDEO_MAX_PLANES] = { 0 }; + LCEVC_ColorFormat fmt = map_format(out->format); + LCEVC_ReturnCode res; + + res = LCEVC_DefaultPictureDesc(&desc, fmt, out->width, out->height); + if (res != LCEVC_Success) + return AVERROR_EXTERNAL; + + for (int i = 0; i < AV_VIDEO_MAX_PLANES; i++) { + planes[i].firstSample = out->data[i]; + planes[i].rowByteStride = out->linesize[i]; + } + + res = LCEVC_AllocPictureExternal(lcevc->decoder, &desc, NULL, planes, picture); + if (res != LCEVC_Success) { + av_log(ctx, AV_LOG_ERROR, "LCEVC_AllocPictureExternal to allocate a buffer for an enhanced frame\n"); + return AVERROR_EXTERNAL; + } + + return 0; +} + +static int generate_output(AVFilterLink *inlink, AVFrame *out) +{ + AVFilterContext *ctx = inlink->dst; + AVFilterLink *outlink = ctx->outputs[0]; + LCEVCContext *lcevc = ctx->priv; + LCEVC_PictureDesc desc; + LCEVC_DecodeInformation info; + LCEVC_PictureHandle picture; + LCEVC_ReturnCode res; + + res = LCEVC_ReceiveDecoderPicture(lcevc->decoder, &picture, &info); + if (res == LCEVC_Again) { + int64_t pts; + if (ff_inlink_acknowledge_status(inlink, &lcevc->status, &pts)) { + av_frame_free(&out); + ff_outlink_set_status(outlink, lcevc->status, pts); + return 0; + } + // this shouldn't be reachable, but instead of asserting, just error out + return AVERROR_EXTERNAL; + } else if (res != LCEVC_Success) { + av_log(ctx, AV_LOG_ERROR, "LCEVC_ReceiveDecoderPicture failed\n"); + return AVERROR_EXTERNAL; + } + + av_frame_copy_props(out, (AVFrame *)info.baseUserData); + av_frame_remove_side_data(out, AV_FRAME_DATA_LCEVC); + + av_frame_free((AVFrame **)&info.baseUserData); + + res = LCEVC_GetPictureDesc(lcevc->decoder, picture, &desc); + if (res != LCEVC_Success) { + av_log(ctx, AV_LOG_ERROR, "LCEVC_GetPictureDesc failed\n"); + return AVERROR_EXTERNAL; + } + res = LCEVC_FreePicture(lcevc->decoder, picture); + if (res != LCEVC_Success) { + av_log(ctx, AV_LOG_ERROR, "LCEVC_FreePicture enhanced failed\n"); + return AVERROR_EXTERNAL; + } + + out->crop_top = desc.cropTop; + out->crop_bottom = desc.cropBottom; + out->crop_left = desc.cropLeft; + out->crop_right = desc.cropRight; + out->sample_aspect_ratio.num = outlink->sample_aspect_ratio.num = desc.sampleAspectRatioNum; + out->sample_aspect_ratio.den = outlink->sample_aspect_ratio.den = desc.sampleAspectRatioDen; + out->color_range = map_range(desc.colorRange); + out->color_primaries = (enum AVColorPrimaries)desc.colorPrimaries; + out->colorspace = (enum AVColorSpace)desc.matrixCoefficients; + out->color_trc = (enum AVColorTransferCharacteristic)desc.transferCharacteristics; + out->width = outlink->w = desc.width + out->crop_left + out->crop_right; + out->height = outlink->h = desc.height + out->crop_top + out->crop_bottom; + + av_log(ctx, AV_LOG_DEBUG, "out PTS %"PRId64", %dx%d, " + "%"SIZE_SPECIFIER"/%"SIZE_SPECIFIER"/%"SIZE_SPECIFIER"/%"SIZE_SPECIFIER", " + "SAR %d:%d, " + "hasEnhancement %d, enhanced %d\n", + out->pts, out->width, out->height, + out->crop_top, out->crop_bottom, out->crop_left, out->crop_right, + out->sample_aspect_ratio.num, out->sample_aspect_ratio.den, + info.hasEnhancement, info.enhanced); + + return ff_filter_frame(outlink, out); +} + +static int receive_frame(AVFilterLink *inlink, AVFrame *out) +{ + AVFilterContext *ctx = inlink->dst; + LCEVCContext *lcevc = ctx->priv; + LCEVC_PictureHandle picture; + LCEVC_ReturnCode res; + int ret; + + ret = alloc_enhanced_frame(inlink, out, &picture); + if (ret < 0) + return ret; + + res = LCEVC_SendDecoderPicture(lcevc->decoder, picture); + if (res != LCEVC_Success) { + av_log(ctx, AV_LOG_ERROR, "LCEVC_SendDecoderPicture failed\n"); + return AVERROR_EXTERNAL; + } + + return generate_output(inlink, out); +} + +static int config_props(AVFilterLink *outlink) +{ + AVFilterContext *ctx = outlink->src; + AVFilterLink *inlink = ctx->inputs[0]; + LCEVCContext *lcevc = ctx->priv; + + outlink->w = lcevc->w = inlink->w * 2 / FFMAX(inlink->sample_aspect_ratio.den, 1); + outlink->h = lcevc->h = inlink->h * 2 / FFMAX(inlink->sample_aspect_ratio.den, 1); + outlink->sample_aspect_ratio = (AVRational) { 0, 1 }; + + return 0; +} + +static void flush_bases(AVFilterContext *ctx) +{ + LCEVCContext *lcevc = ctx->priv; + LCEVC_PictureHandle picture; + + while (LCEVC_ReceiveDecoderBase(lcevc->decoder, &picture) == LCEVC_Success) + LCEVC_FreePicture(lcevc->decoder, picture); +} + +static int activate(AVFilterContext *ctx) +{ + LCEVCContext *lcevc = ctx->priv; + AVFilterLink *inlink = ctx->inputs[0]; + AVFilterLink *outlink = ctx->outputs[0]; + AVFrame *out; + int ret; + + FF_FILTER_FORWARD_STATUS_BACK(outlink, inlink); + + if (!lcevc->in) { + ret = ff_inlink_consume_frame(inlink, &lcevc->in); + if (ret < 0) + return ret; + if (!ret) { + int64_t pts; + if (ff_inlink_acknowledge_status(inlink, &lcevc->status, &pts)) { + if (!lcevc->status) + ff_outlink_set_status(outlink, lcevc->status, pts); + } + if (!lcevc->status) + FF_FILTER_FORWARD_WANTED(outlink, inlink); + } + } + + if (lcevc->in) { + if (lcevc->in->width != inlink->w || + lcevc->in->height != inlink->h || + lcevc->in->sample_aspect_ratio.den != inlink->sample_aspect_ratio.den || + lcevc->in->sample_aspect_ratio.num != inlink->sample_aspect_ratio.num) { + inlink->dst->inputs[0]->w = lcevc->in->width; + inlink->dst->inputs[0]->h = lcevc->in->height; + inlink->dst->inputs[0]->sample_aspect_ratio.den = lcevc->in->sample_aspect_ratio.den; + inlink->dst->inputs[0]->sample_aspect_ratio.num = lcevc->in->sample_aspect_ratio.num; + + config_props(outlink); + } + + ret = send_frame(inlink); + // Buffer as many base frames as the decoder allows + if (!ret) { + FF_FILTER_FORWARD_WANTED(outlink, inlink); + return FFERROR_NOT_READY; + } else if (ret < 0 && ret != AVERROR(EAGAIN)) + return ret; + } + + out = ff_get_video_buffer(outlink, lcevc->w, lcevc->h); + if (!out) + return AVERROR(ENOMEM); + + ret = receive_frame(inlink, out); + if (ret < 0) { + av_frame_free(&out); + return ret; + } + + flush_bases(ctx); + + return ret; +} + +static void log_callback(LCEVC_DecoderHandle dec, LCEVC_Event event, + LCEVC_PictureHandle pic, const LCEVC_DecodeInformation *info, + const uint8_t *data, uint32_t size, void *logctx) +{ + if (event != LCEVC_Log) // shouldn't happen + return; + + if (strlen(data) != size) // sanitize input + return; + + av_log(logctx, AV_LOG_INFO, "LCEVC Log: %s\n", data); +} + +static av_cold int init(AVFilterContext *ctx) +{ + LCEVCContext *lcevc = ctx->priv; + LCEVC_AccelContextHandle dummy = { 0 }; + const int32_t event = LCEVC_Log; + LCEVC_ReturnCode res; + + res = LCEVC_CreateDecoder(&lcevc->decoder, dummy); + if (res != LCEVC_Success) { + av_log(ctx, AV_LOG_ERROR, "LCEVC_CreateDecoder failed\n"); + return AVERROR_EXTERNAL; + } + + res = LCEVC_ConfigureDecoderIntArray(lcevc->decoder, "events", 1, &event); + if (res != LCEVC_Success) { + av_log(ctx, AV_LOG_ERROR, "LCEVC_ConfigureDecoderIntArray failed to set \"events\"\n"); + return AVERROR_EXTERNAL; + } + res = LCEVC_SetDecoderEventCallback(lcevc->decoder, log_callback, ctx); + if (res != LCEVC_Success) { + av_log(ctx, AV_LOG_ERROR, "LCEVC_SetDecoderEventCallback failed\n"); + return AVERROR_EXTERNAL; + } + + res = LCEVC_InitializeDecoder(lcevc->decoder); + if (res != LCEVC_Success) { + av_log(ctx, AV_LOG_ERROR, "LCEVC_InitializeDecoder failed\n"); + return AVERROR_EXTERNAL; + } + + return 0; +} + +static av_cold void uninit(AVFilterContext *ctx) +{ + LCEVCContext *lcevc = ctx->priv; + + LCEVC_DestroyDecoder(lcevc->decoder); +} + +static const AVFilterPad lcevc_outputs[] = { + { + .name = "default", + .type = AVMEDIA_TYPE_VIDEO, + .config_props = config_props, + }, +}; + +static const enum AVPixelFormat pix_fmts[] = { + AV_PIX_FMT_YUV420P, AV_PIX_FMT_YUV420P10LE, AV_PIX_FMT_YUV420P12LE, + AV_PIX_FMT_NV12, AV_PIX_FMT_NV21, + AV_PIX_FMT_GRAY8, AV_PIX_FMT_GRAY10LE, AV_PIX_FMT_GRAY12LE, + AV_PIX_FMT_RGB24, AV_PIX_FMT_BGR24, + AV_PIX_FMT_RGBA, AV_PIX_FMT_ARGB, AV_PIX_FMT_ABGR, AV_PIX_FMT_BGRA, + AV_PIX_FMT_NONE +}; + +const AVFilter ff_vf_lcevc = { + .name = "lcevc", + .description = NULL_IF_CONFIG_SMALL("LCEVC"), + .activate = activate, + FILTER_INPUTS(ff_video_default_filterpad), + FILTER_OUTPUTS(lcevc_outputs), + FILTER_PIXFMTS_ARRAY(pix_fmts), + .priv_size = sizeof(LCEVCContext), + .init = init, + .uninit = uninit, +}; From patchwork Tue Mar 19 14:06:36 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Almer X-Patchwork-Id: 47220 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:958e:b0:1a3:31a3:7958 with SMTP id iu14csp1688050pzb; Tue, 19 Mar 2024 07:07:11 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCX/BufQtehjMhSvNyJGp9f7GQn5yuvUgZCfSYgB7UC6SasJ/7nGgHjB1MuMbcggBgd1bFh1zVrjYjrnVHoABKGXkglsvv/9OeWOCw== X-Google-Smtp-Source: AGHT+IEpuBDR6m0rtI1QZR89t8Zq0X9TT1P0SCFqfF6ep+DORK7JCjwN6TmASXPzgDRJ621QJ99d X-Received: by 2002:a17:906:2349:b0:a45:b930:b3ed with SMTP id m9-20020a170906234900b00a45b930b3edmr1686578eja.3.1710857231106; Tue, 19 Mar 2024 07:07:11 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1710857230; cv=none; d=google.com; s=arc-20160816; b=NDzOo/4eM+EKnZSfS/4XEgyYsDJXhXAq5aE2EOAH6CH14XYsuW0GEqxZiSY2WPeLlC ov9261uCn+sv4rR5WuhQ5Uy+G5mSpoUY3WRXyu7wTTbkOaa2G0+0Oh3epehqPMrzCXO9 1GgWuJBjrrgQRoO0W0OMn5iV0LQ5nWwR+X2AfBPyqKy/hNc+MIhZNDfCc9MO6QxvhdnG dWnK6YJWkFdRpRaKwTjLl71yeBk2sfB7scZwOUJIZxeCeI7szIdzymSKDVdrryuVRj+T 8/EiQuDHv7HZCmq7BDLsYcfSWTDbMVIxsCOcI2z2r/+KjJDiLWay9S6HIhds2kj6mHkn 9d/g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:dkim-signature:delivered-to; bh=8+3awPHOlMPqrCi3kVfid9xzqIlNvDeUlYhXy5ODtqc=; fh=YOA8vD9MJZuwZ71F/05pj6KdCjf6jQRmzLS+CATXUQk=; b=bU9IubMLZEjANm7hs+zsrrFQEutZ0eF0WlBNnJD3qcapZB/cwJmeovVfsKTKGbyS6k x9HA//Z38r2Yj5xt0qO9BTzL2X0zmCvRv3l1FmUc9s1VdDVzXHir1rX9m76zyAagtn3n EylN5NdtDwfASowlsHK7RWP9esU2+iDkbPbju2wpUdMfbLWlfsn1njQhQ2kMynUYrExW jaCYUXT6hd482F6CkXgDAaxHRU1/51BAccsAGGuSzD5wXwHy7KVJIQW2u7d1qLdJSzis VtPDXrzBH69RVbpCcVvoNAn9WB/b5uPcaNvPsj5jViSbZZAKQULjNHe3ChH0hoc9rj5m ITPQ==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=RNps8lIB; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id dt17-20020a170906b79100b00a46bb77973asi2217468ejb.244.2024.03.19.07.07.10; Tue, 19 Mar 2024 07:07:10 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=RNps8lIB; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id C960768D3BD; Tue, 19 Mar 2024 16:06:44 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pl1-f173.google.com (mail-pl1-f173.google.com [209.85.214.173]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 012F168D27A for ; Tue, 19 Mar 2024 16:06:38 +0200 (EET) Received: by mail-pl1-f173.google.com with SMTP id d9443c01a7336-1dfff641d10so17817325ad.2 for ; Tue, 19 Mar 2024 07:06:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1710857196; x=1711461996; darn=ffmpeg.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=1vtH5jIS6AxGphXHlrlUMiby8SpOdJUFLwdPDdNlKoU=; b=RNps8lIBLLhNrVnRGdIT4g6vRZujtUVOMFDjbHHqSNbAHasxYWdGb1bLnVlMnPQ0Ft 8HGOPxMJKSdvZY8kek/z8eegDQG7pllK5319iHS6Az1ME3oliDkIirzFAXcCRgLGqB/y x1CWGjStrOR3wR9B8vnIhqLS4yF2GJOaj8ig5ROfA4UusNXyX13UOCrr6SqTBtQS1X8o 1tfBiWvDjD14oC2yD2K7ff+dfXt7RqcbjfPFE+CnyYtq4wx5b74INEEUWdlDKQEroBmw X8JqY3GdyhOgXQZs5hkym1W8oAN8AZVWXxS9l0HW06btlmh2fV1AXcFU1kik+gMTbqtM LZ3w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1710857196; x=1711461996; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=1vtH5jIS6AxGphXHlrlUMiby8SpOdJUFLwdPDdNlKoU=; b=AU3vzv5mgMSn/sgaN83FgLYoovET5jJuZ4LIBI8KzpE9Cxo/+gqFTj5KBDBp4kRykK ZMRoh+3ZueXsHNUMp0tyBsuYp4MS0Jes/NH0AczM9ztrHYIVM/yfFRJdEugJc8ZoIICf Mkw+gk4XWzdhICrT8kVocHKZREk6uTaa/uxTi/xgcVfLxP+pXvWepo4ELf51lpx6uQ4D fUZZeD2WRNXYh2ZD8Kpk8zut6m72crieKGqzmQ0TLV/cG2EeryR4h1RWFwtcHZ0RJRjf jvQzdQtAu4q3VxWpOKbV6v/SNKcQ8ZvQsVgErGGoNTQzcWdpQ9ZCJXAqdMcnYnnukocp h1Yw== X-Gm-Message-State: AOJu0YwdbyuP0OXjI8uqJsub0Wl8hHy3cdOEzPHrqntQ4+oAvyLvdxaG SS/H9N60c5aApXvouX3X5UJeuwU5yYQWeg4gUsNNwervTm8IvfhnOqu4q997 X-Received: by 2002:a17:903:22d0:b0:1e0:1e87:c9d8 with SMTP id y16-20020a17090322d000b001e01e87c9d8mr6376621plg.7.1710857195484; Tue, 19 Mar 2024 07:06:35 -0700 (PDT) Received: from localhost.localdomain ([190.194.167.233]) by smtp.gmail.com with ESMTPSA id b11-20020a170902650b00b001d9aa663282sm10397008plk.266.2024.03.19.07.06.34 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 19 Mar 2024 07:06:34 -0700 (PDT) From: James Almer To: ffmpeg-devel@ffmpeg.org Date: Tue, 19 Mar 2024 11:06:36 -0300 Message-ID: <20240319140636.4323-3-jamrial@gmail.com> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240319140636.4323-1-jamrial@gmail.com> References: <20240319140636.4323-1-jamrial@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 3/3] fftools/ffmpeg: auto insert enhancement filters when available X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 4h70GYAPidUb Signed-off-by: James Almer --- fftools/ffmpeg.h | 3 +++ fftools/ffmpeg_demux.c | 4 ++++ fftools/ffmpeg_filter.c | 28 +++++++++++++++++++++++++++- fftools/ffmpeg_opt.c | 3 +++ 4 files changed, 37 insertions(+), 1 deletion(-) diff --git a/fftools/ffmpeg.h b/fftools/ffmpeg.h index 7454089c2d..8d54affc0a 100644 --- a/fftools/ffmpeg.h +++ b/fftools/ffmpeg.h @@ -155,6 +155,7 @@ typedef struct OptionsContext { SpecifierOptList hwaccel_devices; SpecifierOptList hwaccel_output_formats; SpecifierOptList autorotate; + SpecifierOptList autoenhance; /* output options */ StreamMap *stream_maps; @@ -239,6 +240,7 @@ enum IFilterFlags { IFILTER_FLAG_AUTOROTATE = (1 << 0), IFILTER_FLAG_REINIT = (1 << 1), IFILTER_FLAG_CFR = (1 << 2), + IFILTER_FLAG_AUTOENHANCE = (1 << 3), }; typedef struct InputFilterOptions { @@ -369,6 +371,7 @@ typedef struct InputStream { #endif int autorotate; + int autoenhance; int fix_sub_duration; diff --git a/fftools/ffmpeg_demux.c b/fftools/ffmpeg_demux.c index 47312c9fe1..0b4ab3d7ec 100644 --- a/fftools/ffmpeg_demux.c +++ b/fftools/ffmpeg_demux.c @@ -1056,6 +1056,7 @@ int ist_filter_add(InputStream *ist, InputFilter *ifilter, int is_simple, return AVERROR(ENOMEM); opts->flags |= IFILTER_FLAG_AUTOROTATE * !!(ist->autorotate) | + IFILTER_FLAG_AUTOENHANCE * !!(ist->autoenhance) | IFILTER_FLAG_REINIT * !!(ds->reinit_filters); return ds->sch_idx_dec; @@ -1238,6 +1239,9 @@ static int ist_add(const OptionsContext *o, Demuxer *d, AVStream *st) ist->autorotate = 1; MATCH_PER_STREAM_OPT(autorotate, i, ist->autorotate, ic, st); + ist->autoenhance = 1; + MATCH_PER_STREAM_OPT(autoenhance, i, ist->autoenhance, ic, st); + MATCH_PER_STREAM_OPT(codec_tags, str, codec_tag, ic, st); if (codec_tag) { uint32_t tag = strtol(codec_tag, &next, 0); diff --git a/fftools/ffmpeg_filter.c b/fftools/ffmpeg_filter.c index 3d88482d07..c4d900d95b 100644 --- a/fftools/ffmpeg_filter.c +++ b/fftools/ffmpeg_filter.c @@ -18,6 +18,8 @@ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA */ +#include "config_components.h" + #include #include "ffmpeg.h" @@ -145,6 +147,8 @@ typedef struct InputFilterPriv { int displaymatrix_present; int32_t displaymatrix[9]; + int enhancement_present; + // fallback parameters to use when no input is ever sent struct { AVRational time_base; @@ -1567,6 +1571,15 @@ static int configure_input_video_filter(FilterGraph *fg, AVFilterGraph *graph, desc = av_pix_fmt_desc_get(ifp->format); av_assert0(desc); +#if CONFIG_LCEVC_FILTER + if ((ifp->opts.flags & IFILTER_FLAG_AUTOENHANCE) && + !(desc->flags & AV_PIX_FMT_FLAG_HWACCEL)) { + ret = insert_filter(&last_filter, &pad_idx, "lcevc", NULL); + if (ret < 0) + return ret; + } +#endif + // TODO: insert hwaccel enabled filters like transpose_vaapi into the graph if ((ifp->opts.flags & IFILTER_FLAG_AUTOROTATE) && !(desc->flags & AV_PIX_FMT_FLAG_HWACCEL)) { @@ -1883,6 +1896,10 @@ static int ifilter_parameters_from_frame(InputFilter *ifilter, const AVFrame *fr memcpy(ifp->displaymatrix, sd->data, sizeof(ifp->displaymatrix)); ifp->displaymatrix_present = !!sd; +#if CONFIG_LCEVC_FILTER + ifp->enhancement_present = !!(av_frame_get_side_data(frame, AV_FRAME_DATA_LCEVC)); +#endif + return 0; } @@ -2584,7 +2601,8 @@ enum ReinitReason { VIDEO_CHANGED = (1 << 0), AUDIO_CHANGED = (1 << 1), MATRIX_CHANGED = (1 << 2), - HWACCEL_CHANGED = (1 << 3) + HWACCEL_CHANGED = (1 << 3), + ENHANCEMENT_CHANGED = (1 << 4), }; static const char *unknown_if_null(const char *str) @@ -2625,6 +2643,12 @@ static int send_frame(FilterGraph *fg, FilterGraphThread *fgt, } else if (ifp->displaymatrix_present) need_reinit |= MATRIX_CHANGED; +#if CONFIG_LCEVC_FILTER + if (sd = av_frame_get_side_data(frame, AV_FRAME_DATA_LCEVC)) + if (!ifp->enhancement_present) + need_reinit |= ENHANCEMENT_CHANGED; +#endif + if (!(ifp->opts.flags & IFILTER_FLAG_REINIT) && fgt->graph) need_reinit = 0; @@ -2681,6 +2705,8 @@ static int send_frame(FilterGraph *fg, FilterGraphThread *fgt, av_bprintf(&reason, "display matrix changed, "); if (need_reinit & HWACCEL_CHANGED) av_bprintf(&reason, "hwaccel changed, "); + if (need_reinit & ENHANCEMENT_CHANGED) + av_bprintf(&reason, "enhancement data found, "); if (reason.len > 1) reason.str[reason.len - 2] = '\0'; // remove last comma av_log(fg, AV_LOG_INFO, "Reconfiguring filter graph%s%s\n", reason.len ? " because " : "", reason.str); diff --git a/fftools/ffmpeg_opt.c b/fftools/ffmpeg_opt.c index 4b3f9789ba..b87782806e 100644 --- a/fftools/ffmpeg_opt.c +++ b/fftools/ffmpeg_opt.c @@ -1738,6 +1738,9 @@ const OptionDef options[] = { { "autorotate", OPT_TYPE_BOOL, OPT_PERSTREAM | OPT_EXPERT | OPT_INPUT, { .off = OFFSET(autorotate) }, "automatically insert correct rotate filters" }, + { "autoenhance", OPT_TYPE_BOOL, OPT_PERSTREAM | OPT_EXPERT | OPT_INPUT, + { .off = OFFSET(autoenhance) }, + "automatically insert enhancement filters (LCEVC)" }, { "autoscale", OPT_TYPE_BOOL, OPT_PERSTREAM | OPT_EXPERT | OPT_OUTPUT, { .off = OFFSET(autoscale) }, "automatically insert a scale filter at the end of the filter graph" },