From patchwork Sat Aug 31 16:31:02 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Almer X-Patchwork-Id: 51252 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:612c:4027:b0:48e:c0f8:d0de with SMTP id ky39csp1017487vqb; Sat, 31 Aug 2024 09:31:01 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCXxPc18miKin0NRzLr0JGN50E1Sfs5SgV8VVYF9fGY6vv+VE5bl/LS0z0qdysIkzpwNhnNgnNo31sjcm1dcHs51@gmail.com X-Google-Smtp-Source: AGHT+IFgxfq3YnCy6HcbsUQFSMnnH9JqnscpoPM7hzM9MjOGOnTXQ5U68rYTeTD0htuswiPuxw2z X-Received: by 2002:a05:6512:b0d:b0:530:e0fd:4a85 with SMTP id 2adb3069b0e04-53546b2d158mr1982883e87.4.1725121861451; Sat, 31 Aug 2024 09:31:01 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1725121861; cv=none; d=google.com; s=arc-20240605; b=bcfdIdyLpM00eFKaosVz0PnJfUbX0jRiUU6G73sDCe8E4TtXwloY18YCv/o3qTcqxX gpZd4qSWlJCMMi4UYU+YZEuBjT1rv8AQdGY8GZcZ6PVBLf2uBMYk5hix+1tAl4/NPZrr zhN46sXbpcQfCvf9tdrejYDch8h01u+pGM2bgv0SPngU7Z0hKTidQOOX/+razesVsgVj O04y9KMpcBp4j/X9HeHD5E+i7iNBC1lDSC4MWLdwYogCkhvw8NqT4bSl5lG287x6yR6S vF13mQ0mptFGcQnLWiTz0dlaNgMeexbPk/NqDyZy22R+fr+3vxwyl6ySGkrjs5XdhzPu rZsw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:message-id:date:to:from :dkim-signature:delivered-to; bh=IydnY7OgdECYL45s/Ezcl1qawuTxqZSItsa/bfg/TL0=; fh=YOA8vD9MJZuwZ71F/05pj6KdCjf6jQRmzLS+CATXUQk=; b=ksMvTbu68RsesUpWZU92RdnP+JTgmJbMy+NwBX8NS1iSk+3oz5M1I9jA2Vensgz/el Iur9CIPSyYN8eTb70hZfs8eH+U/0s8KacYf0q8xEqZ5C78to1OLcdGDs3ZxYGsYXjCK8 Z9aB/LjxOfYE2aVQkToEQO3w3aKEaxhfmkycahYC5ZwmxWpbeISdTwM/dUbGGRtzrU6G WXUk61aW6hdnJOgj4Ur/iY40OGhYqEXQilO9/fNQFoSd/xK18dVCtqC18xw9lj7eTXl6 DH7cjDKYNuXLpYWmkRz4WjoW/G1tJ7nyR4MYKycd8yqGC9pleo1ixm2e2vSQhnZmCx/r MOog==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=Nlf12GKg; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com; dara=fail header.i=@gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 2adb3069b0e04-535407d90f3si2145039e87.239.2024.08.31.09.31.00; Sat, 31 Aug 2024 09:31:01 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=Nlf12GKg; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com; dara=fail header.i=@gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id D8F6368DD82; Sat, 31 Aug 2024 19:30:54 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pj1-f43.google.com (mail-pj1-f43.google.com [209.85.216.43]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 431D168DCA7 for ; Sat, 31 Aug 2024 19:30:48 +0300 (EEST) Received: by mail-pj1-f43.google.com with SMTP id 98e67ed59e1d1-2d892997913so454339a91.3 for ; Sat, 31 Aug 2024 09:30:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1725121845; x=1725726645; darn=ffmpeg.org; h=content-transfer-encoding:mime-version:message-id:date:subject:to :from:from:to:cc:subject:date:message-id:reply-to; bh=+ctStw3bOWGUJAQ9bc9eSxb9+oJrEDR0Sdk6JhydoWg=; b=Nlf12GKgMSk1HO5UJZ+cak2vF3o45hh+Pp5/LK1miCGB5McWAABVycnDn5y8JBTmxG mu3G3OykZbpKvW91LgaeiARI/LuFlU9tCRjpefxfqj+LFxovVIB+9SDUssO5GG+PoHb7 X4k4WS/lBwQbmx5YcDHyn+y03nP5V1kvVfccqTrKdmhkteI7kw3W/qwLPlRHrLi78Y+g yrrNO6OusMs97VODX3IIMmLFE5BljEi2BHI8n6q/nHzEpLv/L3M6dwBK3eh94wm+z8wt PwnKoEPkJv3ACuxGrc1+/3xKcbgbOakvTONHlu0nS6KLI5MfvDv6flTx5x9kgUUYxXRT fGaw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725121845; x=1725726645; h=content-transfer-encoding:mime-version:message-id:date:subject:to :from:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=+ctStw3bOWGUJAQ9bc9eSxb9+oJrEDR0Sdk6JhydoWg=; b=DvWy+/gU/RudUeMoEkbifeBh7tmgm0ntniqskyDGOOgccz7lWkLNJ2LPEE3CFsv0Mr 6zw4x/JJDi4SCWUCMPBOMvMLBjTpBzSdT7+EUh/f2MWNfUsNqdAe7Nkt/JAxbCCnq6WU DW8n+WPb/aPJft8TFFYlxEdLGpZGxvoQT/lexAbL/jh/WMhBv7FV9hfXMkAE6O1UbFH3 bvLCka3byBHbH/YD+xuki74u/ZUNbM9h3URswkrLeclIPijLG2Koirui59Y0GbVOn9o2 Fw4O6sA7Oc1Kn6CeygW3JHydy72oGmYhvTkGB6hNSfR8bfWFBZUj4SiSvnf7Wwls3vAz pzcw== X-Gm-Message-State: AOJu0YwyVa1FHDf91HsZiU6JNH0HXEw+ncmJU5CXVuXBIinMyr3mntKn 8PWZ+ED9XsnSkFl6RmdajdSmvBdERXnMDU3J8LNXKEHEL+xueKbQGtP3sw== X-Received: by 2002:a17:90a:600f:b0:2d8:8509:85d1 with SMTP id 98e67ed59e1d1-2d890620499mr2651018a91.38.1725121844868; Sat, 31 Aug 2024 09:30:44 -0700 (PDT) Received: from localhost.localdomain ([190.194.167.233]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2d85b3b9522sm5935328a91.50.2024.08.31.09.30.43 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 31 Aug 2024 09:30:44 -0700 (PDT) From: James Almer To: ffmpeg-devel@ffmpeg.org Date: Sat, 31 Aug 2024 13:31:02 -0300 Message-ID: <20240831163114.4197-1-jamrial@gmail.com> X-Mailer: git-send-email 2.46.0 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 01/13 v3] avutil/frame: add an LCEVC enhancement data payload side data type X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 80TIjWhlMZGF Signed-off-by: James Almer --- libavutil/frame.c | 1 + libavutil/frame.h | 5 +++++ 2 files changed, 6 insertions(+) diff --git a/libavutil/frame.c b/libavutil/frame.c index 5cbfc6a48b..2758f90e27 100644 --- a/libavutil/frame.c +++ b/libavutil/frame.c @@ -46,6 +46,7 @@ static const AVSideDataDescriptor sd_props[] = { [AV_FRAME_DATA_DETECTION_BBOXES] = { "Bounding boxes for object detection and classification" }, [AV_FRAME_DATA_DOVI_RPU_BUFFER] = { "Dolby Vision RPU Data" }, [AV_FRAME_DATA_DOVI_METADATA] = { "Dolby Vision Metadata" }, + [AV_FRAME_DATA_LCEVC] = { "LCEVC NAL data" }, [AV_FRAME_DATA_STEREO3D] = { "Stereo 3D", AV_SIDE_DATA_PROP_GLOBAL }, [AV_FRAME_DATA_REPLAYGAIN] = { "AVReplayGain", AV_SIDE_DATA_PROP_GLOBAL }, [AV_FRAME_DATA_DISPLAYMATRIX] = { "3x3 displaymatrix", AV_SIDE_DATA_PROP_GLOBAL }, diff --git a/libavutil/frame.h b/libavutil/frame.h index 60bb966f8b..973980ad58 100644 --- a/libavutil/frame.h +++ b/libavutil/frame.h @@ -228,6 +228,11 @@ enum AVFrameSideDataType { * encoding. */ AV_FRAME_DATA_VIDEO_HINT, + + /** + * Raw LCEVC payload data, as a uint8_t array. + */ + AV_FRAME_DATA_LCEVC, }; enum AVActiveFormatDescription { From patchwork Sat Aug 31 16:31:03 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Almer X-Patchwork-Id: 51253 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:612c:4027:b0:48e:c0f8:d0de with SMTP id ky39csp1017579vqb; Sat, 31 Aug 2024 09:31:14 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCXTjPg9zDcUbEAevXAQkAVzWpzeeNbfmRBU8GjBg0eZ70JiK69MUWfn3gZ6UXSXppdoPKe7GE5VbQv9TKDTjiPy@gmail.com X-Google-Smtp-Source: AGHT+IHEOZnVKnBJhXrIDuknJEaD4fqIxq1gPG9PFdeq9nc1hztTreWkvSKN1B17gg+gLYt/ylHh X-Received: by 2002:a05:651c:2211:b0:2ef:1c03:73e6 with SMTP id 38308e7fff4ca-2f61e09b9d7mr19052681fa.5.1725121874595; Sat, 31 Aug 2024 09:31:14 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1725121874; cv=none; d=google.com; s=arc-20240605; b=h0w0vfnhy7qTR+jcGOA0lFclEIimQW0OObIaiPA6dKImEDF4U7OXLCfcFwWGsYqtuA mwqaBK3ROtJqAqUab9Z+KmUvARtari186oXFSyelChQT5cNhGL9VSolbT0U92mbdF/jK 9oEPNUt2oa53Zhk8tQ9A1z+R0HqT9o6kgrD9UO6yEJRbG9xlpGl4sECtJRZnwk5tZQpb ehf7tVbIn11Hz+zisKIROQPlsxewxN8gwZ8y943oOfMJpJqGRwKvZehnwYX7vj6XDzXC ZsNd8DrbkruaPVCcOOFV7IFdqlqftDgyF7Vjv4zgV7AdPslWq1YdyIlsK2npTrYdqGJ9 2hvA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:dkim-signature:delivered-to; bh=EPgxFUl3C7qs9+hBcNP30X+HUsCxyPjVw8hbg9vNTMI=; fh=YOA8vD9MJZuwZ71F/05pj6KdCjf6jQRmzLS+CATXUQk=; b=UKmwovr1WWdw6crI1tf0IZlEVR9RZwghsqi52NU3NHygyyXHdg2mIsj75ra61W9ZFl OjdgsmvY0F5xEBc7UZnQhpMm7CvwE6wYHJ5XS9qwkZvqVepFSWgQDrEWDSrWyFcmvDGN HPnye82qpWWu3dmqhqvAiVDci8wHmcQQDBv902y8sqraL8cKm4k9162SPmK534fGrmeu eiMHKv7kY5FTBSXChr4dZHOEF1vq5JHqar/rorbKtcc/xctsfId+2Boi3jMrluZKWrKc tODxI1vwzfFnGwTdHhbnJlJYamziOGxzIDDHiAQPQEbUylUlJioiYbkyNBKFsHmjNVpi WPPw==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=INe6qSbW; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com; dara=fail header.i=@gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 38308e7fff4ca-2f614e95317si19673091fa.104.2024.08.31.09.31.14; Sat, 31 Aug 2024 09:31:14 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=INe6qSbW; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com; dara=fail header.i=@gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id F029168DD89; Sat, 31 Aug 2024 19:30:55 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pg1-f174.google.com (mail-pg1-f174.google.com [209.85.215.174]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id A565F68DD65 for ; Sat, 31 Aug 2024 19:30:49 +0300 (EEST) Received: by mail-pg1-f174.google.com with SMTP id 41be03b00d2f7-7c691c8f8dcso1918791a12.1 for ; Sat, 31 Aug 2024 09:30:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1725121847; x=1725726647; darn=ffmpeg.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=h6j0vest497igZL+/dZvG8eC1m/PzsqVeOlkQDO5Z6Q=; b=INe6qSbW6GmavF6JE4uR2llV+R4AgV3Lzi/FGXN/L1kFcQI8Wwuk3TMr516zqg3vTC Bi+/xr5jxn09enl8f0SzsLAWzfu1y7Q8PMstukoUhztujMbHOuH1HzBbYDKOTPclLnwN xJNKbi5IPhJvi3DsPPJM5graWHfmL29A8Xe/duWHItSYSxa6BhZ0n0C0GkmCW9qiJnKI qSIttxwP44tJHozmtDvj6Sfnd+6JlVWhLMLUu7zu33TJXn19YfeMc4fkCNBDDtqifDBx n8ZhNTiJ6GZlhrp5WSJsG4pHGf6v0+hMhz7HDXof6KVslMlvgOswhYah/spMlk1v/XAh CYXg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725121847; x=1725726647; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=h6j0vest497igZL+/dZvG8eC1m/PzsqVeOlkQDO5Z6Q=; b=UMZhso5uMej1nVPxzA0Juj0jgqRXp66oPOMEqh4lO9xKnzoaLZ1qQVuJ5F7ltoZhzH /DqgXOu4ljVCcE40TYgCXp1Rj5dEviTqWXqcCaSd+oOqLrBdKLBKzReQ79ZJlnmu6mgJ apeyONTcgtU2/Cu36vsbYjD+0+klKnKYlXP+9si0RkCHQ45yxWgYluWeDvTt8ynoerxn IYygcfqey36sIh9NCY2kkevL92TLwspMVj4T8iXUyBx41W5Te8Qjq9hdSL5LG56stiHM kKKK2+2UfuMnZunv/lJcTQgu5bp3qRrm9FvOZJpz4Qu7NjTyPORXUQAK6acQyba9ZP4L yvzA== X-Gm-Message-State: AOJu0YzjdFaFJ1XivYNr3HLoob9TbpM97eRGTceCWJW7aWKXfuXZCoAS VtDFSa90+AI6+D04twGTwOQj/Che1cRwtM8YvJD+y63cuotGsY5HoeZYuw== X-Received: by 2002:a17:90b:30d5:b0:2c9:df1c:4a58 with SMTP id 98e67ed59e1d1-2d890564136mr2670465a91.23.1725121846833; Sat, 31 Aug 2024 09:30:46 -0700 (PDT) Received: from localhost.localdomain ([190.194.167.233]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2d85b3b9522sm5935328a91.50.2024.08.31.09.30.45 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 31 Aug 2024 09:30:45 -0700 (PDT) From: James Almer To: ffmpeg-devel@ffmpeg.org Date: Sat, 31 Aug 2024 13:31:03 -0300 Message-ID: <20240831163114.4197-2-jamrial@gmail.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240831163114.4197-1-jamrial@gmail.com> References: <20240831163114.4197-1-jamrial@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 02/13 v3] avcodec: add LCEVC decoding support via LCEVCdec X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: SJyeJPkNhaBt Signed-off-by: James Almer --- configure | 3 + doc/general_contents.texi | 13 ++ libavcodec/Makefile | 1 + libavcodec/lcevcdec.c | 289 ++++++++++++++++++++++++++++++++++++++ libavcodec/lcevcdec.h | 45 ++++++ 5 files changed, 351 insertions(+) create mode 100644 libavcodec/lcevcdec.c create mode 100644 libavcodec/lcevcdec.h diff --git a/configure b/configure index 63f0429b02..261ab6252e 100755 --- a/configure +++ b/configure @@ -225,6 +225,7 @@ External library support: --enable-libcdio enable audio CD grabbing with libcdio [no] --enable-libcodec2 enable codec2 en/decoding using libcodec2 [no] --enable-libdav1d enable AV1 decoding via libdav1d [no] + --enable-liblcevc-dec enable LCEVC decoding via liblcevc-dec [no] --enable-libdavs2 enable AVS2 decoding via libdavs2 [no] --enable-libdc1394 enable IIDC-1394 grabbing using libdc1394 and libraw1394 [no] @@ -1914,6 +1915,7 @@ EXTERNAL_LIBRARY_LIST=" libcelt libcodec2 libdav1d + liblcevc_dec libdc1394 libflite libfontconfig @@ -6859,6 +6861,7 @@ enabled libcelt && require libcelt celt/celt.h celt_decode -lcelt0 && enabled libcaca && require_pkg_config libcaca caca caca.h caca_create_canvas enabled libcodec2 && require libcodec2 codec2/codec2.h codec2_create -lcodec2 enabled libdav1d && require_pkg_config libdav1d "dav1d >= 0.5.0" "dav1d/dav1d.h" dav1d_version +enabled liblcevc_dec && require_pkg_config liblcevc_dec "lcevc_dec >= 2.0.0" "LCEVC/lcevc_dec.h" LCEVC_CreateDecoder enabled libdavs2 && require_pkg_config libdavs2 "davs2 >= 1.6.0" davs2.h davs2_decoder_open enabled libdc1394 && require_pkg_config libdc1394 libdc1394-2 dc1394/dc1394.h dc1394_new enabled libdrm && check_pkg_config libdrm libdrm xf86drm.h drmGetVersion diff --git a/doc/general_contents.texi b/doc/general_contents.texi index e7cf4f8239..5309db9ba8 100644 --- a/doc/general_contents.texi +++ b/doc/general_contents.texi @@ -245,6 +245,19 @@ Go to @url{https://github.com/google/liblc3/} and follow the instructions for installing the library. Then pass @code{--enable-liblc3} to configure to enable it. +@section LCEVCdec + +FFmpeg can make use of the liblcevc_dec library for LCEVC enhacement layer +decoding on supported bitstreams. + +Go to @url{https://github.com/v-novaltd/LCEVCdec} and follow the instructions +for installing the library. Then pass @code{--enable-liblcevc-dec} to configure to +enable it. + +@float NOTE +LCEVCdec is under the BSD-3-Clause-Clear License. +@end float + @section OpenH264 FFmpeg can make use of the OpenH264 library for H.264 decoding and encoding. diff --git a/libavcodec/Makefile b/libavcodec/Makefile index 3b4b8681f5..1333db82d0 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -121,6 +121,7 @@ OBJS-$(CONFIG_INTRAX8) += intrax8.o intrax8dsp.o msmpeg4_vc1_dat OBJS-$(CONFIG_IVIDSP) += ivi_dsp.o OBJS-$(CONFIG_JNI) += ffjni.o jni.o OBJS-$(CONFIG_JPEGTABLES) += jpegtables.o +OBJS-$(CONFIG_LIBLCEVC_DEC) += lcevcdec.o OBJS-$(CONFIG_LCMS2) += fflcms2.o OBJS-$(CONFIG_LLAUDDSP) += lossless_audiodsp.o OBJS-$(CONFIG_LLVIDDSP) += lossless_videodsp.o diff --git a/libavcodec/lcevcdec.c b/libavcodec/lcevcdec.c new file mode 100644 index 0000000000..ae1d150986 --- /dev/null +++ b/libavcodec/lcevcdec.c @@ -0,0 +1,289 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "libavutil/avassert.h" +#include "libavutil/frame.h" +#include "libavutil/imgutils.h" +#include "libavutil/log.h" +#include "libavutil/mem.h" +#include "decode.h" +#include "lcevcdec.h" + +static LCEVC_ColorFormat map_format(int format) +{ + switch (format) { + case AV_PIX_FMT_YUV420P: + return LCEVC_I420_8; + case AV_PIX_FMT_YUV420P10: + return LCEVC_I420_10_LE; + case AV_PIX_FMT_NV12: + return LCEVC_NV12_8; + case AV_PIX_FMT_NV21: + return LCEVC_NV21_8; + case AV_PIX_FMT_GRAY8: + return LCEVC_GRAY_8; + } + + return LCEVC_ColorFormat_Unknown; +} + +static int alloc_base_frame(void *logctx, LCEVC_DecoderHandle decoder, + const AVFrame *frame, LCEVC_PictureHandle *picture) +{ + LCEVC_PictureDesc desc; + LCEVC_ColorFormat fmt = map_format(frame->format); + LCEVC_PictureLockHandle lock; + uint8_t *data[4] = { NULL }; + int linesizes[4] = { 0 }; + uint32_t planes; + LCEVC_ReturnCode res; + + res = LCEVC_DefaultPictureDesc(&desc, fmt, frame->width, frame->height); + if (res != LCEVC_Success) + return AVERROR_EXTERNAL; + + desc.cropTop = frame->crop_top; + desc.cropBottom = frame->crop_bottom; + desc.cropLeft = frame->crop_left; + desc.cropRight = frame->crop_right; + desc.sampleAspectRatioNum = frame->sample_aspect_ratio.num; + desc.sampleAspectRatioDen = frame->sample_aspect_ratio.den; + + /* Allocate LCEVC Picture */ + res = LCEVC_AllocPicture(decoder, &desc, picture); + if (res != LCEVC_Success) + return AVERROR_EXTERNAL; + + res = LCEVC_LockPicture(decoder, *picture, LCEVC_Access_Write, &lock); + if (res != LCEVC_Success) + return AVERROR_EXTERNAL; + + res = LCEVC_GetPicturePlaneCount(decoder, *picture, &planes); + if (res != LCEVC_Success) + return AVERROR_EXTERNAL; + + for (unsigned i = 0; i < planes; i++) { + LCEVC_PicturePlaneDesc plane; + + res = LCEVC_GetPictureLockPlaneDesc(decoder, lock, i, &plane); + if (res != LCEVC_Success) + return AVERROR_EXTERNAL; + + data[i] = plane.firstSample; + linesizes[i] = plane.rowByteStride; + } + + av_image_copy2(data, linesizes, frame->data, frame->linesize, + frame->format, frame->width, frame->height); + + res = LCEVC_UnlockPicture(decoder, lock); + if (res != LCEVC_Success) + return AVERROR_EXTERNAL; + + return 0; +} + +static int alloc_enhanced_frame(void *logctx, LCEVC_DecoderHandle decoder, + const AVFrame *frame, LCEVC_PictureHandle *picture) +{ + LCEVC_PictureDesc desc ; + LCEVC_ColorFormat fmt = map_format(frame->format); + LCEVC_PicturePlaneDesc planes[4] = { 0 }; + int width = frame->width * 2 / FFMAX(frame->sample_aspect_ratio.den, 1); + int height = frame->height * 2 / FFMAX(frame->sample_aspect_ratio.num, 1); + LCEVC_ReturnCode res; + + res = LCEVC_DefaultPictureDesc(&desc, fmt, width, height); + if (res != LCEVC_Success) + return AVERROR_EXTERNAL; + + /* Set plane description */ + for (int i = 0; i < 4; i++) { + planes[i].firstSample = frame->data[i]; + planes[i].rowByteStride = frame->linesize[i]; + } + + /* Allocate LCEVC Picture */ + res = LCEVC_AllocPictureExternal(decoder, &desc, NULL, planes, picture); + if (res != LCEVC_Success) + return AVERROR_EXTERNAL; + + return 0; +} + +int ff_lcevc_send_frame(void *logctx, FFLCEVCContext *lcevc, const AVFrame *in) +{ + const AVFrameSideData *sd = av_frame_get_side_data(in, AV_FRAME_DATA_LCEVC); + LCEVC_PictureHandle picture; + LCEVC_ReturnCode res; + int ret = 0; + + if (!sd) + return 1; + + res = LCEVC_SendDecoderEnhancementData(lcevc->decoder, in->pts, 0, sd->data, sd->size); + if (res != LCEVC_Success) + return AVERROR_EXTERNAL; + + ret = alloc_base_frame(logctx, lcevc->decoder, in, &picture); + if (ret < 0) + return ret; + + res = LCEVC_SendDecoderBase(lcevc->decoder, in->pts, 0, picture, -1, NULL); + if (res != LCEVC_Success) + return AVERROR_EXTERNAL; + + memset(&picture, 0, sizeof(picture)); + ret = alloc_enhanced_frame(logctx, lcevc->decoder, in, &picture); + if (ret < 0) + return ret; + + res = LCEVC_SendDecoderPicture(lcevc->decoder, picture); + if (res != LCEVC_Success) + return AVERROR_EXTERNAL; + + return 0; +} + +static int generate_output(void *logctx, FFLCEVCContext *lcevc, AVFrame *out) +{ + LCEVC_PictureDesc desc; + LCEVC_DecodeInformation info; + LCEVC_PictureHandle picture; + LCEVC_ReturnCode res; + + res = LCEVC_ReceiveDecoderPicture(lcevc->decoder, &picture, &info); + if (res != LCEVC_Success) + return AVERROR_EXTERNAL; + + res = LCEVC_GetPictureDesc(lcevc->decoder, picture, &desc); + if (res != LCEVC_Success) + return AVERROR_EXTERNAL; + + out->crop_top = desc.cropTop; + out->crop_bottom = desc.cropBottom; + out->crop_left = desc.cropLeft; + out->crop_right = desc.cropRight; + out->sample_aspect_ratio.num = desc.sampleAspectRatioNum; + out->sample_aspect_ratio.den = desc.sampleAspectRatioDen; + out->width = desc.width + out->crop_left + out->crop_right; + out->height = desc.height + out->crop_top + out->crop_bottom; + + return 0; +} + +int ff_lcevc_receive_frame(void *logctx, FFLCEVCContext *lcevc, AVFrame *out) +{ + LCEVC_PictureHandle picture; + LCEVC_ReturnCode res; + int ret; + + ret = generate_output(logctx, lcevc, out); + if (ret < 0) + return ret; + + res = LCEVC_ReceiveDecoderBase (lcevc->decoder, &picture); + if (res != LCEVC_Success) + return AVERROR_EXTERNAL; + + res = LCEVC_FreePicture(lcevc->decoder, picture); + if (res != LCEVC_Success) + return AVERROR_EXTERNAL; + + return 0; +} + +int ff_lcevc_process(void *logctx, AVFrame *frame) +{ + FrameDecodeData *fdd = (FrameDecodeData*)frame->private_ref->data; + AVBufferRef *ref = (AVBufferRef*)fdd->post_process_opaque; + FFLCEVCContext *lcevc = (FFLCEVCContext*)ref->data; + int ret; + + if (!lcevc->initialized) { + ret = ff_lcevc_init(lcevc, logctx); + if (ret < 0) + return ret; + } + + ret = ff_lcevc_send_frame(logctx, lcevc, frame); + if (ret) + return ret < 0 ? ret : 0; + + ff_lcevc_receive_frame(logctx, lcevc, frame); + if (ret < 0) + return ret; + + return 0; +} + +static void event_callback(LCEVC_DecoderHandle dec, LCEVC_Event event, + LCEVC_PictureHandle pic, const LCEVC_DecodeInformation *info, + const uint8_t *data, uint32_t size, void *logctx) +{ + switch (event) { + case LCEVC_Log: + av_log(logctx, AV_LOG_INFO, "%s\n", data); + break; + default: + break; + } +} + +int ff_lcevc_init(FFLCEVCContext *lcevc, void *logctx) +{ + LCEVC_AccelContextHandle dummy = { 0 }; + int32_t events[] = { LCEVC_Log }; + + if (lcevc->initialized) + return 0; + + if (LCEVC_CreateDecoder(&lcevc->decoder, dummy) != LCEVC_Success) { + av_log(logctx, AV_LOG_ERROR, "Failed to create LCEVC decoder\n"); + return AVERROR_EXTERNAL; + } + + LCEVC_ConfigureDecoderInt(lcevc->decoder, "log_level", 4); + LCEVC_ConfigureDecoderIntArray(lcevc->decoder, "events", FF_ARRAY_ELEMS(events), events); + LCEVC_SetDecoderEventCallback(lcevc->decoder, event_callback, logctx); + + if (LCEVC_InitializeDecoder(lcevc->decoder) != LCEVC_Success) { + av_log(logctx, AV_LOG_ERROR, "Failed to initialize LCEVC decoder\n"); + LCEVC_DestroyDecoder(lcevc->decoder); + return AVERROR_EXTERNAL; + } + + lcevc->initialized = 1; + + return 0; +} + +void ff_lcevc_free(void *opaque, uint8_t *data) +{ + FFLCEVCContext *lcevc = opaque; + if (lcevc->initialized) + LCEVC_DestroyDecoder(lcevc->decoder); + lcevc->initialized = 0; + av_free(data); +} + +void ff_lcevc_unref(void *opaque) +{ + AVBufferRef *ref = opaque; + av_buffer_unref(&ref); +} diff --git a/libavcodec/lcevcdec.h b/libavcodec/lcevcdec.h new file mode 100644 index 0000000000..45a3ac6800 --- /dev/null +++ b/libavcodec/lcevcdec.h @@ -0,0 +1,45 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#ifndef AVCODEC_LCEVCDEC_H +#define AVCODEC_LCEVCDEC_H + +#include "config_components.h" + +#include +#if CONFIG_LIBLCEVC_DEC +#include +#else +typedef uintptr_t LCEVC_DecoderHandle; +#endif + +typedef struct FFLCEVCContext { + LCEVC_DecoderHandle decoder; + int initialized; +} FFLCEVCContext; + +struct AVFrame; + +int ff_lcevc_process(void *logctx, struct AVFrame *frame); +int ff_lcevc_send_frame(void *logctx, FFLCEVCContext *lcevc, const AVFrame *in); +int ff_lcevc_receive_frame(void *logctx, FFLCEVCContext *lcevc, AVFrame *out); +int ff_lcevc_init(FFLCEVCContext *lcevc, void *logctx); +void ff_lcevc_unref(void *opaque); +void ff_lcevc_free(void *opaque, uint8_t *data); + +#endif /* AVCODEC_LCEVCDEC_H */ From patchwork Sat Aug 31 16:31:04 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Almer X-Patchwork-Id: 51254 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:612c:4027:b0:48e:c0f8:d0de with SMTP id ky39csp1017661vqb; Sat, 31 Aug 2024 09:31:24 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCWXrzF1AJ74TYOragqCQjP3DmnT3XiKs6QLnmEVdWXDYAVtd2i0yCbvxj5oaHZFigiNW9pi3aFKKashNTujI20H@gmail.com X-Google-Smtp-Source: AGHT+IGubdCZG8QejwaWxEiSI6h0HOX4QOJyj80Xrt6PwaWHKlrcjLGfT/8bpEAv7QArjp3PQPHo X-Received: by 2002:a05:651c:2221:b0:2ef:17df:6314 with SMTP id 38308e7fff4ca-2f61e08e3a6mr19293231fa.4.1725121884302; Sat, 31 Aug 2024 09:31:24 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1725121884; cv=none; d=google.com; s=arc-20240605; b=K5vEO71lRYfXkWzP5x0eTviefTmx+IaGkKnOlKM7E/1ogi0UnuvWNh2S2CU7wX92WU rG/xE2Wm0K04KVxND/FtjlTpYyqi2CM/KgGQMgclErTf42yXbldvBUDd/0JR39O1RDtH B5pjdpZABTeFTtp6K1LO4PbbyIOaHCNHNL3OvaLlBrSycQ/ZJgo8K9ga4eBWvxOccqCM 6WibFqBHEr88EFaVQV10F5ndQJdGEM+O8nILjlt5EICpvFyX3+bNhVNa/Z+iaWCcNN/i f38O6xqRE8cqojkrgX7wMf84ZLSRLPGk7Yua+LVVrjqzjxAXXWX6TuSDyGqRehZyFhQl lQZw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:dkim-signature:delivered-to; bh=Bx8K1nocb36B67JQ0gIdfZomVQ7lbh4NK60xfduKPl8=; fh=YOA8vD9MJZuwZ71F/05pj6KdCjf6jQRmzLS+CATXUQk=; b=e8YY9A2LlPgpJhzMk3qNYlOP6oVRKFqAkjvpkNr514BYSHmrPkKT1DHQQXuqkE49iF r6LhfisEW8EFv9h72EgsOnW2Yds4cUc6gMokT3PSK/ZVVJOuyHZ/5jstZNeY69DtEdfA R2KKoeHljdHkeISToRDhK/nnauBk2VeXIgpa/vjx0fxHMojf/v4ysF5++WtLk34/gH/U D0cjncmVsusu9tWtP8/zi9t5bluS1LbFL8dye53HkqssOq3RtldAlJi6F1h7TtGVZisW qgABiWTrRrbLBzFLV40QzPfk82PLFCULyg7+5FioFnSDU0gRVrBeah+2zTefIw6XZHeE qdkA==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=jqKvl8l2; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com; dara=fail header.i=@gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 38308e7fff4ca-2f615171f1bsi17885201fa.457.2024.08.31.09.31.23; Sat, 31 Aug 2024 09:31:24 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=jqKvl8l2; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com; dara=fail header.i=@gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 5548568DDAF; Sat, 31 Aug 2024 19:30:58 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pj1-f42.google.com (mail-pj1-f42.google.com [209.85.216.42]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 9397E68DD83 for ; Sat, 31 Aug 2024 19:30:51 +0300 (EEST) Received: by mail-pj1-f42.google.com with SMTP id 98e67ed59e1d1-2d87a1f0791so1042281a91.2 for ; Sat, 31 Aug 2024 09:30:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1725121849; x=1725726649; darn=ffmpeg.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=a2fWDowJC+xAEVEawnDnymPiyaxHYr1G1E3DohPB7Gk=; b=jqKvl8l24eYMgNVC7uXxmNAskBY6RZkqTGcoVUwdqYIXHo3MsM80Je9gBkeUqgAfqa 6fw6QqSyQXRUG/gcPjMVkrnDyW3vwZM6Eu1eTSmShg9O+p7iPuqFN40XcJb6vkBTUS+H 0LbmZIEEkH2xhFP2HfnCkvFQaMStVRTDkBMijm6uJuh3aJF42OOkAwVN14wvaxjFtkFd 5bMtPFMM0XY9annDEtYqs9E5QTaYJck6AHS2ZMWT/+29SN/rq2Ma6VVUpOeWFYrDtEGe SrEBKeY2fVyeCIPX8iCC4Wum6SKxE2zPIlHTAYLm6CX8w8wgqK51ywIztKjf6x72tV5F +6fQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725121849; x=1725726649; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=a2fWDowJC+xAEVEawnDnymPiyaxHYr1G1E3DohPB7Gk=; b=KP8MXTF/xyg9AkbZaI2wpq70gktvm7sixQGLD3T+KButzKiGUkbOMc6R/lX2K/DeQ8 ZKZi/1jZw2AVumKsizImkwekSzZZLFQcO0P+ZgOT97bvle/jGTSjZJf5UViXZ4Bf8WEU UjCSRfHWQgwuCxFHvXvXq9Sb5HyI3UyHgwBH8rf9jZAYn4eQtFZSMaHbEtBSVRK8qVni glLs7HpxP3P2lGF4P/nBzQYtTsw82q6e5ADzeAbaAaMAl+NHzojrHJJnszhuzBj0NB18 Gs9Om8DG0on6a+iTGAhkxg8JlkUDIQX0kkCYL59QwdHYIC46mUEAUzEIDP8CHqUbpAmq +Iig== X-Gm-Message-State: AOJu0YwdFWIi2KVwF2qM2Al3VkWpButuV1EI72mRwA2mnMiTPyqkDffw g3TJKMMjl3PnUQI99Yw4urPb2fpJpMyV/CuP3i/QK6gHClXzKF+kUHR5lQ== X-Received: by 2002:a17:90a:43a5:b0:2d8:7f5c:6030 with SMTP id 98e67ed59e1d1-2d8973c40d5mr1994778a91.38.1725121848517; Sat, 31 Aug 2024 09:30:48 -0700 (PDT) Received: from localhost.localdomain ([190.194.167.233]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2d85b3b9522sm5935328a91.50.2024.08.31.09.30.47 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 31 Aug 2024 09:30:47 -0700 (PDT) From: James Almer To: ffmpeg-devel@ffmpeg.org Date: Sat, 31 Aug 2024 13:31:04 -0300 Message-ID: <20240831163114.4197-3-jamrial@gmail.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240831163114.4197-1-jamrial@gmail.com> References: <20240831163114.4197-1-jamrial@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 03/13] avcodec/decode: support applying LCEVC enhacements X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: /77TC0TWtmuW Signed-off-by: James Almer --- libavcodec/avcodec.c | 2 ++ libavcodec/avcodec.h | 5 +++++ libavcodec/decode.c | 42 +++++++++++++++++++++++++++++++++++++- libavcodec/internal.h | 2 ++ libavcodec/lcevcdec.c | 2 ++ libavcodec/options_table.h | 1 + libavcodec/pthread_frame.c | 7 +++++++ 7 files changed, 60 insertions(+), 1 deletion(-) diff --git a/libavcodec/avcodec.c b/libavcodec/avcodec.c index 6065f1b689..380df267a0 100644 --- a/libavcodec/avcodec.c +++ b/libavcodec/avcodec.c @@ -446,6 +446,8 @@ av_cold void ff_codec_close(AVCodecContext *avctx) ff_refstruct_unref(&avci->pool); ff_refstruct_pool_uninit(&avci->progress_frame_pool); + av_buffer_unref(&avci->lcevc); + ff_hwaccel_uninit(avctx); av_bsf_free(&avci->bsf); diff --git a/libavcodec/avcodec.h b/libavcodec/avcodec.h index 7a67300134..f6e01da738 100644 --- a/libavcodec/avcodec.h +++ b/libavcodec/avcodec.h @@ -418,6 +418,11 @@ typedef struct RcOverride{ * Do not apply film grain, export it instead. */ #define AV_CODEC_EXPORT_DATA_FILM_GRAIN (1 << 3) +/** + * Decoding only. + * Do not apply picture enhancement layers, export them instead. + */ +#define AV_CODEC_EXPORT_DATA_ENHANCEMENTS (1 << 4) /** * The decoder will keep a reference to the frame and may reuse it later. diff --git a/libavcodec/decode.c b/libavcodec/decode.c index 27dba8a1f3..dc868714ac 100644 --- a/libavcodec/decode.c +++ b/libavcodec/decode.c @@ -48,6 +48,7 @@ #include "hwaccel_internal.h" #include "hwconfig.h" #include "internal.h" +#include "lcevcdec.h" #include "packet_internal.h" #include "progressframe.h" #include "refstruct.h" @@ -1599,6 +1600,7 @@ int ff_attach_decode_data(AVFrame *frame) int ff_get_buffer(AVCodecContext *avctx, AVFrame *frame, int flags) { const FFHWAccel *hwaccel = ffhwaccel(avctx->hwaccel); + int lcevc = 0, width = 0, height = 0; int override_dimensions = 1; int ret; @@ -1639,8 +1641,17 @@ int ff_get_buffer(AVCodecContext *avctx, AVFrame *frame, int flags) ret = hwaccel->alloc_frame(avctx, frame); goto end; } - } else + } else { avctx->sw_pix_fmt = avctx->pix_fmt; + lcevc = CONFIG_LIBLCEVC_DEC && avctx->codec_type == AVMEDIA_TYPE_VIDEO && + avctx->internal->lcevc && av_frame_get_side_data(frame, AV_FRAME_DATA_LCEVC); + if (lcevc) { + width = frame->width; + height = frame->height; + frame->width = frame->width * 2 / FFMAX(frame->sample_aspect_ratio.den, 1); + frame->height = frame->height * 2 / FFMAX(frame->sample_aspect_ratio.num, 1); + } + } ret = avctx->get_buffer2(avctx, frame, flags); if (ret < 0) @@ -1652,6 +1663,20 @@ int ff_get_buffer(AVCodecContext *avctx, AVFrame *frame, int flags) if (ret < 0) goto fail; + if (CONFIG_LIBLCEVC_DEC && lcevc) { + FrameDecodeData *fdd = (FrameDecodeData*)frame->private_ref->data; + fdd->post_process_opaque = av_buffer_ref(avctx->internal->lcevc); + + frame->width = width; + frame->height = height; + if (!fdd->post_process_opaque) { + ret = AVERROR(ENOMEM); + goto fail; + } + fdd->post_process_opaque_free = ff_lcevc_unref; + fdd->post_process = ff_lcevc_process; + } + end: if (avctx->codec_type == AVMEDIA_TYPE_VIDEO && !override_dimensions && !(ffcodec(avctx->codec)->caps_internal & FF_CODEC_CAP_EXPORTS_CROPPING)) { @@ -1949,6 +1974,21 @@ int ff_decode_preinit(AVCodecContext *avctx) if (ret < 0) return ret; + if (CONFIG_LIBLCEVC_DEC && !(avctx->export_side_data & AV_CODEC_EXPORT_DATA_ENHANCEMENTS)) { + FFLCEVCContext *lcevc = av_mallocz(sizeof(*lcevc)); + + if (!lcevc || !(avci->lcevc = av_buffer_create((uint8_t *)lcevc, sizeof(*lcevc), + ff_lcevc_free, lcevc, 0))) { + int explode = avctx->err_recognition & AV_EF_EXPLODE; + av_log(avctx, explode ? AV_LOG_ERROR: AV_LOG_WARNING, + "Error allocating LCEVC context\n"); + if (explode) { + av_free(lcevc); + return AVERROR(ENOMEM); + } + } + } + #if FF_API_DROPCHANGED if (avctx->flags & AV_CODEC_FLAG_DROPCHANGED) av_log(avctx, AV_LOG_WARNING, "The dropchanged flag is deprecated.\n"); diff --git a/libavcodec/internal.h b/libavcodec/internal.h index 98ab2797ce..aae78854cc 100644 --- a/libavcodec/internal.h +++ b/libavcodec/internal.h @@ -158,6 +158,8 @@ typedef struct AVCodecInternal { FFIccContext icc; /* used to read and write embedded ICC profiles */ #endif + AVBufferRef *lcevc; + /** * Set when the user has been warned about a failed allocation from * a fixed frame pool. diff --git a/libavcodec/lcevcdec.c b/libavcodec/lcevcdec.c index ae1d150986..47fdaaa9ef 100644 --- a/libavcodec/lcevcdec.c +++ b/libavcodec/lcevcdec.c @@ -229,6 +229,8 @@ int ff_lcevc_process(void *logctx, AVFrame *frame) if (ret < 0) return ret; + av_frame_remove_side_data(frame, AV_FRAME_DATA_LCEVC); + return 0; } diff --git a/libavcodec/options_table.h b/libavcodec/options_table.h index 33f1bce887..332f354fa6 100644 --- a/libavcodec/options_table.h +++ b/libavcodec/options_table.h @@ -93,6 +93,7 @@ static const AVOption avcodec_options[] = { {"prft", "export Producer Reference Time through packet side data", 0, AV_OPT_TYPE_CONST, {.i64 = AV_CODEC_EXPORT_DATA_PRFT}, INT_MIN, INT_MAX, A|V|S|E, .unit = "export_side_data"}, {"venc_params", "export video encoding parameters through frame side data", 0, AV_OPT_TYPE_CONST, {.i64 = AV_CODEC_EXPORT_DATA_VIDEO_ENC_PARAMS}, INT_MIN, INT_MAX, V|D, .unit = "export_side_data"}, {"film_grain", "export film grain parameters through frame side data", 0, AV_OPT_TYPE_CONST, {.i64 = AV_CODEC_EXPORT_DATA_FILM_GRAIN}, INT_MIN, INT_MAX, V|D, .unit = "export_side_data"}, +{"enhancements", "export film grain parameters through frame side data", 0, AV_OPT_TYPE_CONST, {.i64 = AV_CODEC_EXPORT_DATA_ENHANCEMENTS}, INT_MIN, INT_MAX, V|D, .unit = "export_side_data"}, {"time_base", NULL, OFFSET(time_base), AV_OPT_TYPE_RATIONAL, {.dbl = 0}, 0, INT_MAX}, {"g", "set the group of picture (GOP) size", OFFSET(gop_size), AV_OPT_TYPE_INT, {.i64 = 12 }, INT_MIN, INT_MAX, V|E}, {"ar", "set audio sampling rate (in Hz)", OFFSET(sample_rate), AV_OPT_TYPE_INT, {.i64 = DEFAULT }, 0, INT_MAX, A|D|E}, diff --git a/libavcodec/pthread_frame.c b/libavcodec/pthread_frame.c index 019e33b7b2..00ad93cbdf 100644 --- a/libavcodec/pthread_frame.c +++ b/libavcodec/pthread_frame.c @@ -782,6 +782,7 @@ void ff_frame_thread_free(AVCodecContext *avctx, int thread_count) ff_refstruct_unref(&ctx->internal->pool); av_packet_free(&ctx->internal->in_pkt); av_packet_free(&ctx->internal->last_pkt_props); + av_buffer_unref(&ctx->internal->lcevc); av_freep(&ctx->internal); av_buffer_unref(&ctx->hw_frames_ctx); av_frame_side_data_free(&ctx->decoded_side_data, @@ -878,6 +879,12 @@ static av_cold int init_thread(PerThreadContext *p, int *threads_to_free, if (!copy->internal->in_pkt) return AVERROR(ENOMEM); + if (avctx->internal->lcevc) { + copy->internal->lcevc = av_buffer_ref(avctx->internal->lcevc); + if (!copy->internal->lcevc) + return AVERROR(ENOMEM); + } + copy->internal->last_pkt_props = av_packet_alloc(); if (!copy->internal->last_pkt_props) return AVERROR(ENOMEM); From patchwork Sat Aug 31 16:31:05 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Almer X-Patchwork-Id: 51257 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:612c:4027:b0:48e:c0f8:d0de with SMTP id ky39csp1020782vqb; Sat, 31 Aug 2024 09:40:14 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCWF5G/pQfoIj/YbvgHzGkEbtlaMlAEpDncQbTCCVLpF1KSRIYeIfVzYneMN/EUa3Cqt9ozZ5dCR8CzfatY30VdN@gmail.com X-Google-Smtp-Source: AGHT+IG+zT5B+ttl8IrzDBuzsJ5ZdhcdwqA5UujYQTsBJPNpZtaLCsUAC95xnvb0gNaQq2+hYqot X-Received: by 2002:a2e:a98e:0:b0:2ef:2d71:e23c with SMTP id 38308e7fff4ca-2f61e02592dmr18018421fa.2.1725122414186; Sat, 31 Aug 2024 09:40:14 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1725122414; cv=none; d=google.com; s=arc-20240605; b=FgUWhgiNmQIPgVe7dKsB2YL5NSr4iJt8M8hS708JvCcNqUEDhv3aVw6vQxa/IFqBjo AdjsUZrwkE9UDS1kpl5N1zede3SjHYOCva3ufZC6CMxGJT8Dsik/1ary2NROFUURkVP+ 97skwt8wmM7GJqY5zp8tkAG/hO14lxo0G7Ad/NMyvz61W29XZoGTA7jbUZdZAFxqkRhw jMg6mN1ISKfxEf/1yO9uIJY4hinZfGiWwRyCqb1MaMwdJjq2QD/xyy5gqYk1AZ/33B6J 4ZeedkySTlersxnJXVLO78f5vszR2vnfpct23Ixyfj0c4JuVVV/nLwa9u61YFGdNk838 D02g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:dkim-signature:delivered-to; bh=yzE4+Y16g2PwLDgrV/Wb0icRMnQzZuyKENLjYulLJic=; fh=YOA8vD9MJZuwZ71F/05pj6KdCjf6jQRmzLS+CATXUQk=; b=BhTnS/dNwmw3MnHZCMoJTSNnQMb3dBHkBqN+WsNuto5C6Zwd/1x8//jeD2L9Obonhw rIwA/y1vIiIOLaMJjuF0HCMtu6Qegf4PZ6yqhUiQ3vmtvqsK+B39vA4M+uNEi+5kIwBg fy5DpQdEJCdu/OQcss6ijNHqMX/5ROlRJEV8crMt1cG8RfxUhYUaz9rNJFUttBWh2Eru DiqKEuvfKyxbYfEJPKasHN2uzD1706/pOOMC+U4x4Jq/ah+3cNOsXjss3eE/vG8YDgct Ii/OPr3l+FKkbIGR6ZxbOlbIb3OwkGTBRK6bdVe5CTSVJwflJCoNvwGVSRz2gOicCWwX 23jA==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=AhluNThp; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com; dara=fail header.i=@gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 38308e7fff4ca-2f615171d73si18430541fa.450.2024.08.31.09.40.13; Sat, 31 Aug 2024 09:40:14 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=AhluNThp; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com; dara=fail header.i=@gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 921C668DDD6; Sat, 31 Aug 2024 19:31:01 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pj1-f42.google.com (mail-pj1-f42.google.com [209.85.216.42]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 659BA68DDB0 for ; Sat, 31 Aug 2024 19:30:53 +0300 (EEST) Received: by mail-pj1-f42.google.com with SMTP id 98e67ed59e1d1-2d88690837eso893208a91.2 for ; Sat, 31 Aug 2024 09:30:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1725121851; x=1725726651; darn=ffmpeg.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=+Dhsu1aV7u8PaQrpiRufV9h/GwuJeN0UpHAddg4+3H4=; b=AhluNThp2WLHV399AGa+nHuFZBoBeY86lxHlhEPGavaZ/4vpK37ExQQoShBl9krgvk 7lukPaJUFLo0y7IJ21vaZZi0RDRz1qWbETgK6eQGLQi4WTp1CGEoyvaiwm/dBM30mT5T ZKK374a5j6JVVhRspfAibMM1j0izJQoyD3DglbbQgO4ZSDofNg47YsATUCjcW5OWh3Ja Uu1uLYmRegnTbYtWEf9wQ570p+/COGnHiTOhxaYNZOF71mnRdVzZtnRkMRGN4m/DV6NI XlOPNH+j5ct/MAZR3JglzvDGIhVEHFTD3RauAQS6gjIDTt+M0xvn51qXlEhUYoWln787 Qwxg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725121851; x=1725726651; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=+Dhsu1aV7u8PaQrpiRufV9h/GwuJeN0UpHAddg4+3H4=; b=AuL1YEtxFT9XcKmQhD2un/KzViim3iKboSiPVey3pxsI+Rsv3ZR4SXCTD+SA9BmXzo LGj3IJfrcJbITstHpTMRzBIVmZOmOryUjD13YXo1a4NOh723l8FLfA4ksqskE5hwyWy4 nwEMRRjhmyj+mqdbHWEMN8q9F0VpFcJzsDspOdCQRqRHRw137buxYVuas9YGynjQhv5m cXAA01xrWcun99Uc6WvfVI0Vs/EW/7Yq5uiAEgCKpzDxBuTO9Pa3a3DQX7Ps8r67Ch8b U8MKaCkCJe2jhDPtRd0ztHTMEEtGfVjvS93+BVRGEkj4EiBuRw2MtEasCN+4ITEpwJDn U17g== X-Gm-Message-State: AOJu0Yy89S4b50tiWqqOfppskUIR0lToc9KBD5t30W9q6UzzqTuujyhZ O1Lo/EwPcoLm9a7bdSNMWNb6DDM1bDho0DRFw9LHAmMO9/Tbt/TW3ia2Qw== X-Received: by 2002:a17:90b:524a:b0:2c5:10a6:e989 with SMTP id 98e67ed59e1d1-2d8564b9bc5mr9806844a91.35.1725121850504; Sat, 31 Aug 2024 09:30:50 -0700 (PDT) Received: from localhost.localdomain ([190.194.167.233]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2d85b3b9522sm5935328a91.50.2024.08.31.09.30.48 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 31 Aug 2024 09:30:49 -0700 (PDT) From: James Almer To: ffmpeg-devel@ffmpeg.org Date: Sat, 31 Aug 2024 13:31:05 -0300 Message-ID: <20240831163114.4197-4-jamrial@gmail.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240831163114.4197-1-jamrial@gmail.com> References: <20240831163114.4197-1-jamrial@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 04/13] avcodec/decode: split ProgressFrame allocator into two functions X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: oTEgTmb9DsU0 Signed-off-by: James Almer --- libavcodec/decode.c | 11 +++++++---- libavcodec/progressframe.h | 16 ++++++++++++++-- 2 files changed, 21 insertions(+), 6 deletions(-) diff --git a/libavcodec/decode.c b/libavcodec/decode.c index dc868714ac..033c4913c0 100644 --- a/libavcodec/decode.c +++ b/libavcodec/decode.c @@ -1749,7 +1749,7 @@ static void check_progress_consistency(const ProgressFrame *f) av_assert1(!f->progress || f->progress->f == f->f); } -static int progress_frame_get(AVCodecContext *avctx, ProgressFrame *f) +int ff_progress_frame_alloc(AVCodecContext *avctx, ProgressFrame *f) { FFRefStructPool *pool = avctx->internal->progress_frame_pool; @@ -1767,9 +1767,12 @@ int ff_progress_frame_get_buffer(AVCodecContext *avctx, ProgressFrame *f, int fl { int ret; - ret = progress_frame_get(avctx, f); - if (ret < 0) - return ret; + check_progress_consistency(f); + if (!f->f) { + ret = ff_progress_frame_alloc(avctx, f); + if (ret < 0) + return ret; + } ret = ff_thread_get_buffer(avctx, f->progress->f, flags); if (ret < 0) { diff --git a/libavcodec/progressframe.h b/libavcodec/progressframe.h index 428a461659..32a345beec 100644 --- a/libavcodec/progressframe.h +++ b/libavcodec/progressframe.h @@ -102,12 +102,24 @@ void ff_progress_frame_report(ProgressFrame *f, int progress); void ff_progress_frame_await(const ProgressFrame *f, int progress); /** - * This function sets up the ProgressFrame, i.e. gets ProgressFrame.f - * and also calls ff_thread_get_buffer() on the frame. + * This function allocates ProgressFrame.f + * May be called before ff_progress_frame_get_buffer() in the cases where the + * AVFrame needs to be accessed before the ff_thread_get_buffer() call in + * ff_progress_frame_alloc(). * * @note: This must only be called by codecs with the * FF_CODEC_CAP_USES_PROGRESSFRAMES internal cap. */ +int ff_progress_frame_alloc(struct AVCodecContext *avctx, ProgressFrame *f); + +/** + * This function sets up the ProgressFrame, i.e. allocates ProgressFrame.f + * if needed, and also calls ff_thread_get_buffer() on the frame. + * + * @note: This must only be called by codecs with the + * FF_CODEC_CAP_USES_PROGRESSFRAMES internal cap. + * @see ff_progress_frame_alloc + */ int ff_progress_frame_get_buffer(struct AVCodecContext *avctx, ProgressFrame *f, int flags); From patchwork Sat Aug 31 16:31:06 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Almer X-Patchwork-Id: 51263 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:612c:4027:b0:48e:c0f8:d0de with SMTP id ky39csp1031736vqb; Sat, 31 Aug 2024 10:10:19 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCUFdACfhH/oh4J3OMu1ISxBA9qa1huhUJONiMYvtIN9TNLK3550iRinmaOrvic9C7m8A+RDSG6oy3lMjJnLSoWl@gmail.com X-Google-Smtp-Source: AGHT+IFpLl+xUaseZ/IZYPcGnv1ei8fTZ3PpZ5LAbN52qvrEjF+eDbsL7Zs6aTwImPaD4Uwf/ezy X-Received: by 2002:a05:6402:5c9:b0:5a2:68a2:ae57 with SMTP id 4fb4d7f45d1cf-5c243781be8mr1789225a12.31.1725124219676; Sat, 31 Aug 2024 10:10:19 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1725124219; cv=none; d=google.com; s=arc-20240605; b=jlgpYZ4lm9s+rs7jBOdUIbpOp+beuVrcnnCaHq55iqTFK6gKFvhIy8uJ70irTNXx5J UwELd+Xwv9zC8CCLWdB4fIaf0KSjhnbqw+iZRxeoFMTgeLqy7Pv344S4nSsjy/Wt2JJk vTgLPojF3dxi6RS2EdxX0d/urbOUpEImBimZUEM5zSiUVAx2sVGBOOpsHpqZsapJYcDC bM0rcvJsINFudrdhFDnzUQCL8DUGL5rkvsHboTNlugnmJarxy+DbJcoTM2XWqiQ+Cv4T EyP6X5fxWinar4ozOY4bpuZKs8LWh1qVRyCHCDmlUgbkq/vzHTRPXG03n98GCsh+OOcK Lh7w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:dkim-signature:delivered-to; bh=9Art5aUa8PW6KcXiFJbHF7WTICdAaMLWKZ+MsHGQyA8=; fh=YOA8vD9MJZuwZ71F/05pj6KdCjf6jQRmzLS+CATXUQk=; b=Xw+xOcKg6Q9H3o+lvFjvSeXjpDkrZQEvIIAO6wGX3/7lhbsaxARMlCDZkwlqs4SXSs i4QLDm88vCaj4p4LtdsWKYnYXOxtR3Xt061yPaLzrz0YzXCWhuXWWj44x+kMDx4ffCzX r8wE0U5VR1OXjzAuYzow6oZG5mqrjvN7vwZ0OBbtgQeGiQH/vWiesgcceJ8icybpnRK3 kMSKWfsPamcsas66nsNZI/Ucmiweotw6KTWV7hDIVlQPhPX9g9UuUzdhCuw/wE66NdOp Wrfo1Cfz3ws1Duga9JdzVLb15yiAZCqrluUf4ZhtErzrNjqH8qLYD1x4vDZolBE/CbaP YMVg==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=P2dKjCTS; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com; dara=fail header.i=@gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 4fb4d7f45d1cf-5c240f4c622si1768233a12.591.2024.08.31.10.10.19; Sat, 31 Aug 2024 10:10:19 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=P2dKjCTS; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com; dara=fail header.i=@gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id EC98568DDE4; Sat, 31 Aug 2024 19:31:02 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pj1-f47.google.com (mail-pj1-f47.google.com [209.85.216.47]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id B6ACD68DD93 for ; Sat, 31 Aug 2024 19:30:54 +0300 (EEST) Received: by mail-pj1-f47.google.com with SMTP id 98e67ed59e1d1-2d889ba25f7so771256a91.0 for ; Sat, 31 Aug 2024 09:30:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1725121852; x=1725726652; darn=ffmpeg.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=nvvIhLCt5HrPhKWu/R2Ozd1vUWAE7PV94Rk9nWw8tzI=; b=P2dKjCTSkBTLkmWh0oEiZHOmHeXUi3n1AjQywJg1a0dFA1/Yb/lkUqKKBOq/FOnvhw 4Tnt7KMNRi+TkGxdmGRl/qrU/3cp5MJ3W8dDv57kwD++FQtxc2s/e6nB6bXYtTxxZJQN ou8SNKe8a4sAIaxtLyJ2MbRW6U4CSr2vofxn0V2cBlrVpDr5E8QPWYktplg7pJ475RQp Sq1rE8eKTDLXieQi/ZtAPMjI5C8bKVQ/0qQCFD0ICCoZTwJLTJAv3szAyOaY6S245KCb F7dRIMInecP+JBRlkKTPtvn+O47gKLKjOCFkslxCnZ59yrLAoQZEORZJEKOhDMRHxaw5 jl0A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725121852; x=1725726652; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=nvvIhLCt5HrPhKWu/R2Ozd1vUWAE7PV94Rk9nWw8tzI=; b=KZNK5+sIzgq0Pjn9RB9L7s96BeztWmVMWn0uVH0JzVigpMTfliWy9esdZIm3ihcNAU Db/XhdGP7Cg/JnjmCdheWpabK7iu3UvQUMMPgclFYol8KTXZkiLxPRfCAladG7ZUTCvH SKKF6rDgg0rfQD+me/rh5p5PXF597WzvZtvE9dDwft0GPe4oDGJM52N6sx70ayeCUJv2 kT6jJ0Nsgs0WkocGIUAOlxQzn52wLvCideaLuq/DbtpNj45P6mQ3tujOjnO4fvimVqlO Z0IreyQ67rj/hjys3OuxDvwPTw8m7SHBVcTCH0i+MLjypxqLSeFbOMA4ojG73t+qGAhM wYQA== X-Gm-Message-State: AOJu0YzlBYgCe2GYwpeiUdY8WsNA7tl42j18MKvXVQVX7VWXMH88L3T4 vxoxm7sGC6flSlDUldKKKCZkGl0MZ4+Si04WQ/YXsUvg6/xYKvRvF25xIg== X-Received: by 2002:a17:90b:358c:b0:2d8:79bf:3873 with SMTP id 98e67ed59e1d1-2d8973a5a01mr1766170a91.29.1725121852255; Sat, 31 Aug 2024 09:30:52 -0700 (PDT) Received: from localhost.localdomain ([190.194.167.233]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2d85b3b9522sm5935328a91.50.2024.08.31.09.30.50 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 31 Aug 2024 09:30:51 -0700 (PDT) From: James Almer To: ffmpeg-devel@ffmpeg.org Date: Sat, 31 Aug 2024 13:31:06 -0300 Message-ID: <20240831163114.4197-5-jamrial@gmail.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240831163114.4197-1-jamrial@gmail.com> References: <20240831163114.4197-1-jamrial@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 05/13 v3] avcodec/h2645_sei: export raw LCEVC metadata X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: udleXvMuDBgI Signed-off-by: James Almer --- libavcodec/h2645_sei.c | 34 ++++++++++++++++++++++++++++++++++ libavcodec/h2645_sei.h | 5 +++++ libavcodec/itut35.h | 2 ++ 3 files changed, 41 insertions(+) diff --git a/libavcodec/h2645_sei.c b/libavcodec/h2645_sei.c index 7c83747cd0..c46a563308 100644 --- a/libavcodec/h2645_sei.c +++ b/libavcodec/h2645_sei.c @@ -99,6 +99,20 @@ static int decode_registered_user_data_dynamic_hdr_vivid(HEVCSEIDynamicHDRVivid } #endif +static int decode_registered_user_data_lcevc(HEVCSEILCEVC *s, + GetByteContext *gb) +{ + int size = bytestream2_get_bytes_left(gb); + + av_buffer_unref(&s->info); + s->info = av_buffer_alloc(size); + if (!s->info) + return AVERROR(ENOMEM); + + bytestream2_get_bufferu(gb, s->info->data, size); + return 0; +} + static int decode_registered_user_data_afd(H2645SEIAFD *h, GetByteContext *gb) { int flag; @@ -142,6 +156,7 @@ static int decode_registered_user_data(H2645SEI *h, GetByteContext *gb, } if (country_code != ITU_T_T35_COUNTRY_CODE_US && + country_code != ITU_T_T35_COUNTRY_CODE_UK && country_code != ITU_T_T35_COUNTRY_CODE_CN) { av_log(logctx, AV_LOG_VERBOSE, "Unsupported User Data Registered ITU-T T35 SEI message (country_code = %d)\n", @@ -173,6 +188,13 @@ static int decode_registered_user_data(H2645SEI *h, GetByteContext *gb, } break; } + case ITU_T_T35_PROVIDER_CODE_LCEVC: { + if (bytestream2_get_bytes_left(gb) < 2) + return AVERROR_INVALIDDATA; + + bytestream2_skipu(gb, 1); // user_data_type_code + return decode_registered_user_data_lcevc(&h->lcevc, gb); + } #if CONFIG_HEVC_SEI case ITU_T_T35_PROVIDER_CODE_CUVA: { const uint16_t cuva_provider_oriented_code = 0x0005; @@ -501,6 +523,10 @@ int ff_h2645_sei_ctx_replace(H2645SEI *dst, const H2645SEI *src) av_buffer_unref(&dst->unregistered.buf_ref[i]); dst->unregistered.nb_buf_ref = 0; + ret = av_buffer_replace(&dst->lcevc.info, src->lcevc.info); + if (ret < 0) + return ret; + if (src->unregistered.nb_buf_ref) { ret = av_reallocp_array(&dst->unregistered.buf_ref, src->unregistered.nb_buf_ref, @@ -778,6 +804,13 @@ int ff_h2645_sei_to_frame(AVFrame *frame, H2645SEI *sei, } } + if (sei->lcevc.info) { + HEVCSEILCEVC *lcevc = &sei->lcevc; + ret = ff_frame_new_side_data_from_buf(avctx, frame, AV_FRAME_DATA_LCEVC, &lcevc->info); + if (ret < 0) + return ret; + } + if (sei->film_grain_characteristics.present) { H2645SEIFilmGrainCharacteristics *fgc = &sei->film_grain_characteristics; AVFilmGrainParams *fgp = av_film_grain_params_create_side_data(frame); @@ -875,6 +908,7 @@ void ff_h2645_sei_reset(H2645SEI *s) av_freep(&s->unregistered.buf_ref); av_buffer_unref(&s->dynamic_hdr_plus.info); av_buffer_unref(&s->dynamic_hdr_vivid.info); + av_buffer_unref(&s->lcevc.info); s->ambient_viewing_environment.present = 0; s->mastering_display.present = 0; diff --git a/libavcodec/h2645_sei.h b/libavcodec/h2645_sei.h index 488dbcad7e..598f78b585 100644 --- a/libavcodec/h2645_sei.h +++ b/libavcodec/h2645_sei.h @@ -50,6 +50,10 @@ typedef struct HEVCSEIDynamicHDRVivid { AVBufferRef *info; } HEVCSEIDynamicHDRVivid; +typedef struct HEVCSEILCEVC { + AVBufferRef *info; +} HEVCSEILCEVC; + typedef struct H2645SEIUnregistered { AVBufferRef **buf_ref; unsigned nb_buf_ref; @@ -126,6 +130,7 @@ typedef struct H2645SEI { H2645SEIAFD afd; HEVCSEIDynamicHDRPlus dynamic_hdr_plus; //< HEVC only HEVCSEIDynamicHDRVivid dynamic_hdr_vivid; //< HEVC only + HEVCSEILCEVC lcevc; H2645SEIUnregistered unregistered; H2645SEIFramePacking frame_packing; H2645SEIDisplayOrientation display_orientation; diff --git a/libavcodec/itut35.h b/libavcodec/itut35.h index ffa7024981..a75ef37929 100644 --- a/libavcodec/itut35.h +++ b/libavcodec/itut35.h @@ -20,11 +20,13 @@ #define AVCODEC_ITUT35_H #define ITU_T_T35_COUNTRY_CODE_CN 0x26 +#define ITU_T_T35_COUNTRY_CODE_UK 0xB4 #define ITU_T_T35_COUNTRY_CODE_US 0xB5 #define ITU_T_T35_PROVIDER_CODE_ATSC 0x31 #define ITU_T_T35_PROVIDER_CODE_CUVA 0x04 #define ITU_T_T35_PROVIDER_CODE_DOLBY 0x3B +#define ITU_T_T35_PROVIDER_CODE_LCEVC 0x50 #define ITU_T_T35_PROVIDER_CODE_SMTPE 0x3C #endif /* AVCODEC_ITUT35_H */ From patchwork Sat Aug 31 16:31:07 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Almer X-Patchwork-Id: 51255 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:612c:4027:b0:48e:c0f8:d0de with SMTP id ky39csp1020753vqb; Sat, 31 Aug 2024 09:40:10 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCWWtG8raICpyST0TtFWHPuu0x6iOkLIG2Zfig4INPSWAcnO7s5h3YVL/vdcSfsXOk+cTcoa1EQ2hQuynjQHCKj5@gmail.com X-Google-Smtp-Source: AGHT+IFMxsKIAyi1NAgGULJwecurMxYQ/KVZZc4snsHgLc9sEycMxPm8FuYhP3INEZvkjjTn7b6C X-Received: by 2002:a05:6512:334e:b0:535:4d89:5b52 with SMTP id 2adb3069b0e04-5354d895d38mr708800e87.1.1725122410603; Sat, 31 Aug 2024 09:40:10 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1725122410; cv=none; d=google.com; s=arc-20240605; b=Dov5VhXPHa7SyW+E6zjFVkAYZfUFs7vNABBIiQlgktBYOIhE11hauq6fhRmYon+ede imtQnlNmj6CvMYb49cRcRaypqGhrLD6JiRE/M1h9loGTsWjfGyulx4Xw8WzptcN/Gkvi FWLB/7RF7kcHml1Dq75cpHTkvsTtXHvRnex/8wg3YthMNkKLni6OGMnpiVqyNENRuawv JMZ7+x86e+sslJAimdx8T0HBRsW7uujNRQ/ThZv7klCO1zH1v7/dbEKt7OEsCNwGblRJ jaQ5Q0/nl9F9qNO93HxanlBNFC+RZfIolTYchmi8H732yw9AaHLPe3MWOS6eOBf7oDnE M3jQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:dkim-signature:delivered-to; bh=D1LRZzYzol7mLquTtl/LgNlr+iJzv4jaRLoFdXjPqmg=; fh=YOA8vD9MJZuwZ71F/05pj6KdCjf6jQRmzLS+CATXUQk=; b=LvhUB5QRWdPp9aOnHG9cwfXRSg4Nx9ojK77Vkg4uXV70PJlft9BX+a3ErhKfVe5g5f CCqvKip6LsK9y6gsL+7spDzS81//xut1ULPNPy1gDmOCEsf1e9lapN2ddOGbF5hZ6FwH POdjoVYECCHvln10zN5Iy8jt2iCG/rjA+mDh6HBfFu2+Ugvf52NIXp8Vm/Q5x2Rdxlwa 9hpoRT4YlDlFdZTTZEFpEN5lKjNBmHQF1bzXKQKpzc0OnYjly09hsVca55k1g3gkg/xE xyBa+UUshhYh/PwzaetPNyjX1TqRefFhrIvuQwatWv5/HdTKf/oJWPXJYTwmHg55rZnj t+bg==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=PJQWka33; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com; dara=fail header.i=@gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 2adb3069b0e04-535407d8c55si1865643e87.258.2024.08.31.09.40.10; Sat, 31 Aug 2024 09:40:10 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=PJQWka33; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com; dara=fail header.i=@gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 27B9F68DDE3; Sat, 31 Aug 2024 19:31:04 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pf1-f171.google.com (mail-pf1-f171.google.com [209.85.210.171]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 3F72268DDA0 for ; Sat, 31 Aug 2024 19:30:56 +0300 (EEST) Received: by mail-pf1-f171.google.com with SMTP id d2e1a72fcca58-714114be925so2337311b3a.2 for ; Sat, 31 Aug 2024 09:30:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1725121854; x=1725726654; darn=ffmpeg.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=vOlV8WW2iLOZRx8RD9zPfQKz/yqmrJp8B57cZgpDtAs=; b=PJQWka33R2dkCGrEJ3UdMWGsWZyd9WHGMAX6gqTTEea6GYPyeYgOS35iUJIHiIfkdl J/q9/CQw3h/fzJLdrKVPg6IvmZHUCQq+RvLlvcMhdDA+bzLdudYCtlxejVzYipLp+BTn tCJWU4igwghuFTqJt97ci0AQDhQyH4vgGP04XwO5HdUqIfp2q8lmXaPartQKRi5wB5vj IprykF18WMxUseu7zKZw7aKGBqgRvwr4vwYbETkW2P3RaacCym5atf8bl0Sv3PhdodGE FSK4fvkE40neWbvPyeYtagUgbDKZ+aJTAXNn7EYc5eS3n49LWEYKp0Bo3J2aIIJLnXgq Efmg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725121854; x=1725726654; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=vOlV8WW2iLOZRx8RD9zPfQKz/yqmrJp8B57cZgpDtAs=; b=M1mSfQ+mjVvEXbHyS5u3eTccWFSWdZAx1PRNTiyFGhywXC/fVaaLExZCLkG6T+EKtW ZhAe8wAi0KZ0gnb/zmu2A4B6zkfUicgrh+lToevo4fBQlIJsIV/kiJfIXVv7MxlSvApO GwNFg1v7zYmrmQ1RhkFP5lW8jy6NM8E/ALvl7uJuEPjYJT5WcPdNumZSWk9bnheLr1b1 MIF3qWbHePostcY0D8aX+9T2sBHAM8rWzDjsEks5JHPxxVsM+QOkb+YBfY+tKZj2fBXW 47GGgl4sl/nhY/5/8Nxz31PWvEo5A+c8cMgBDvCxcsK32qYnthLnCp4Nv/n8thzwMLUj wvQQ== X-Gm-Message-State: AOJu0Yy/khuLMadzMdOWMAeAVtw/CBiln50c3rRcJqnSZEm9gCFbruzM GnKJxKH3n5Y080kr1oEMNhRyFKJuWrjO2zSPmeuu/0rbrKbt33uEApNp0w== X-Received: by 2002:a05:6a20:cf8f:b0:1cc:e5bc:37de with SMTP id adf61e73a8af0-1cecf757f22mr2857325637.39.1725121853741; Sat, 31 Aug 2024 09:30:53 -0700 (PDT) Received: from localhost.localdomain ([190.194.167.233]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2d85b3b9522sm5935328a91.50.2024.08.31.09.30.52 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 31 Aug 2024 09:30:53 -0700 (PDT) From: James Almer To: ffmpeg-devel@ffmpeg.org Date: Sat, 31 Aug 2024 13:31:07 -0300 Message-ID: <20240831163114.4197-6-jamrial@gmail.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240831163114.4197-1-jamrial@gmail.com> References: <20240831163114.4197-1-jamrial@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 06/13] avcodec/h264dec: export LCEVC metadata as frame side data X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: o2Km3DNCdrgN Signed-off-by: James Almer --- configure | 2 +- libavcodec/h264_slice.c | 8 ++++++++ libavcodec/h264dec.c | 2 ++ 3 files changed, 11 insertions(+), 1 deletion(-) diff --git a/configure b/configure index 261ab6252e..84f63bbb87 100755 --- a/configure +++ b/configure @@ -2970,7 +2970,7 @@ h263i_decoder_select="h263_decoder" h263p_decoder_select="h263_decoder" h263p_encoder_select="h263_encoder" h264_decoder_select="cabac golomb h264chroma h264dsp h264parse h264pred h264qpel h264_sei videodsp" -h264_decoder_suggest="error_resilience" +h264_decoder_suggest="error_resilience liblcevc_dec" hap_decoder_select="snappy texturedsp" hap_encoder_deps="libsnappy" hap_encoder_select="texturedspenc" diff --git a/libavcodec/h264_slice.c b/libavcodec/h264_slice.c index a66b75ca80..89172dc05a 100644 --- a/libavcodec/h264_slice.c +++ b/libavcodec/h264_slice.c @@ -518,6 +518,14 @@ static int h264_frame_start(H264Context *h) pic->needs_fg = h->sei.common.film_grain_characteristics.present && !h->avctx->hwaccel && !(h->avctx->export_side_data & AV_CODEC_EXPORT_DATA_FILM_GRAIN); + if (CONFIG_LIBLCEVC_DEC && h->sei.common.lcevc.info && !h->avctx->hwaccel && + !(h->avctx->export_side_data & AV_CODEC_EXPORT_DATA_ENHANCEMENTS)) { + HEVCSEILCEVC *lcevc = &h->sei.common.lcevc; + ret = ff_frame_new_side_data_from_buf(h->avctx, pic->f, + AV_FRAME_DATA_LCEVC, &lcevc->info); + if (ret < 0) + return ret; + } if ((ret = alloc_picture(h, pic)) < 0) return ret; diff --git a/libavcodec/h264dec.c b/libavcodec/h264dec.c index 0154fe17b6..976b21c30e 100644 --- a/libavcodec/h264dec.c +++ b/libavcodec/h264dec.c @@ -904,6 +904,8 @@ static int output_frame(H264Context *h, AVFrame *dst, H264Picture *srcp) if (!(h->avctx->export_side_data & AV_CODEC_EXPORT_DATA_FILM_GRAIN)) av_frame_remove_side_data(dst, AV_FRAME_DATA_FILM_GRAIN_PARAMS); + if (!CONFIG_LIBLCEVC_DEC && !(h->avctx->export_side_data & AV_CODEC_EXPORT_DATA_ENHANCEMENTS)) + av_frame_remove_side_data(dst, AV_FRAME_DATA_LCEVC); return 0; fail: From patchwork Sat Aug 31 16:31:08 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Almer X-Patchwork-Id: 51261 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:612c:4027:b0:48e:c0f8:d0de with SMTP id ky39csp1031681vqb; Sat, 31 Aug 2024 10:10:14 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCXUrkAdJ8EoPSQ4hyT9VVM4tQljULC8bCYrepLLmqG2aXbv4Zhc2vWW0Pq3+JYOa7tHIx7mPPrR8L2xOVzCIDmR@gmail.com X-Google-Smtp-Source: AGHT+IHeqfk9dsS18CED1l/BzXtjUZmVgc79JJU5HYv53slGUaeEVk65mp0DCKg8XIz1Eds23dIo X-Received: by 2002:a05:6512:124f:b0:530:e228:779c with SMTP id 2adb3069b0e04-53546b04212mr4108684e87.19.1725124213823; Sat, 31 Aug 2024 10:10:13 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1725124213; cv=none; d=google.com; s=arc-20240605; b=E/B0MQM9Ftwg8D8++3rSxZWcX0Z35N2gIpqYmTOVQapqAWNI2dxOuOOGnDEANHDXgY kF7070RMjlG6Voy+Ep1di2PGQ02fUEzIa3eqlsLN57PwvJ4ZHRm9v7n1sl2IYKmfx9bS j/crmeO506fAaumkOdsbpS0jhT2UVucB4LS6hFzLcY2mWwoelwcPVrRIluZ+85b+7Bye fHsvLQFIbAVphBHeMOLP4GddNHsGVwOdjIJY5dZCUXUOZvL/fgQ8fKBVbSpP/0Jo33Az Y2lPAm127BhIIdCk94lLBounRbSTRl4C6VnmTNDxp4L9+K/4HXD/o82a/xLKIbHjXOFz 6Q1g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:dkim-signature:delivered-to; bh=vXZ2sxgfwXzJ0Rtq+312+y1tC+Gd5mdvNlGdkiIoVQ0=; fh=YOA8vD9MJZuwZ71F/05pj6KdCjf6jQRmzLS+CATXUQk=; b=JtW6nOzpigLWqR4uQmKr5a+tb2SjH5yQ7/GJuovPuSVvz55pFS0Ql7DjHOQzLmQWHT yvGze4y3ZGQtI3lEe+0kT8hjksVUyDI6dBeuDNPKxpaIQfPLq2iXAkbSEwY5BWUFbACq 5MKcNwF0uHW5nN02C/dpmwNqKHudoGHLOHdsbBJsUi5POLKSdn/pahrXq0lAt7yKYXOK vPUidF9QK6z0MxDdUZ/tToNJvydGE8yTyREL7YVk/iirJCSN9O+wMZkk/5j2jJ2t9x1z WY/IqyGSj7JQGTQ2mDQ/RFOkHNqUdrQKK6Vw55VmHjYmbcctVmMerS+K7wzlCInuozo+ X3sQ==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=BlJD4fiv; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com; dara=fail header.i=@gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 2adb3069b0e04-5354082a07esi2166173e87.474.2024.08.31.10.10.13; Sat, 31 Aug 2024 10:10:13 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=BlJD4fiv; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com; dara=fail header.i=@gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id A3D3D68DDA0; Sat, 31 Aug 2024 19:31:06 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pj1-f42.google.com (mail-pj1-f42.google.com [209.85.216.42]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id E0DE068DDE1 for ; Sat, 31 Aug 2024 19:30:57 +0300 (EEST) Received: by mail-pj1-f42.google.com with SMTP id 98e67ed59e1d1-2d88c0f8e79so745013a91.3 for ; Sat, 31 Aug 2024 09:30:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1725121855; x=1725726655; darn=ffmpeg.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=4SPt3Z6cMaFR4mIx50eMascMoC0APJmemF/ohxWFdXE=; b=BlJD4fiv6j/Zccb3CLyX8LvheJsVVhjxEEAisQC7r9vBX8UXc3rITHCzHC57V96hvT IDflV4dUASi3wB+Jh//Xg0dJgVFqh8Hranllj+0v3okxoHo4Pqv5prSI8lYJwdLE+fjh XSS14ESnXShyLzEUs9fCLfTRt/fEkMMhFes7mY58xSbDLcWNpMqJglufcFx8ZXqkmZE9 xxUuJblMoMZdvxiafn8itwfcqlY+tn4E8WGUGjQ3h6XvO6xpT/cyPE7md8yy9hfz1E+M dIUWF+Agp0oAThKxP8DzrBO4LgVMMroxMpaNNCaRs8Yb62YuneE8dpGeq3zoe5qBdOJu QU0A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725121855; x=1725726655; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=4SPt3Z6cMaFR4mIx50eMascMoC0APJmemF/ohxWFdXE=; b=CxYrBK2Nh91VIl2qEDqn6FaPakmyvM6F9IxC+NmsWBJNoZ90aqBS/TAhmVRh00p4ss 5388BkQu98fpVonkPiX85zRbjV/hK398HTIC33akM2RujHARFOkFg9zhrm8+KbR9GMB+ mXxsbgjmh6ha+RkqKhLJFhahomWDkpnVJwuAMIzV1Z2XFh9R7LLuLVsySnkUNiBWDI89 z3Y4JsDHKfWD5zKzM5qYBRgbGqjXpgQPvgppSOW4InNDQMqwiUGSPFFUcBx4vTJuMGTE AkCdMZVOeqWSp/UmwTDBFvgTI8hEXl4EhokERhpMXFXw0nLwPtsbbPx9J7/UidNkMv+w EEBQ== X-Gm-Message-State: AOJu0YyvdKKFprud1NhS5pglckea2+5IS4Dkkj8SfqG2DNsqhbE1Wcev uQn1koZLWGoLMKbAZNkg10k9in3zqRO5RZ7UzqJiXp6hQuJdESCfp6G5Mw== X-Received: by 2002:a17:90a:c684:b0:2d8:8cef:3d64 with SMTP id 98e67ed59e1d1-2d88cef3f05mr3631688a91.6.1725121855182; Sat, 31 Aug 2024 09:30:55 -0700 (PDT) Received: from localhost.localdomain ([190.194.167.233]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2d85b3b9522sm5935328a91.50.2024.08.31.09.30.54 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 31 Aug 2024 09:30:54 -0700 (PDT) From: James Almer To: ffmpeg-devel@ffmpeg.org Date: Sat, 31 Aug 2024 13:31:08 -0300 Message-ID: <20240831163114.4197-7-jamrial@gmail.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240831163114.4197-1-jamrial@gmail.com> References: <20240831163114.4197-1-jamrial@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 07/13] avcodec/hevcdec: export LCEVC metadata as frame side data X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: PGwqvZx4FjTX Signed-off-by: James Almer --- configure | 1 + libavcodec/hevc/hevcdec.c | 3 +++ libavcodec/hevc/refs.c | 15 ++++++++++++++- 3 files changed, 18 insertions(+), 1 deletion(-) diff --git a/configure b/configure index 84f63bbb87..3b7cf05bb5 100755 --- a/configure +++ b/configure @@ -2975,6 +2975,7 @@ hap_decoder_select="snappy texturedsp" hap_encoder_deps="libsnappy" hap_encoder_select="texturedspenc" hevc_decoder_select="bswapdsp cabac dovi_rpudec golomb hevcparse hevc_sei videodsp" +hevc_decoder_suggest="liblcevc_dec" huffyuv_decoder_select="bswapdsp huffyuvdsp llviddsp" huffyuv_encoder_select="bswapdsp huffman huffyuvencdsp llvidencdsp" hymt_decoder_select="huffyuv_decoder" diff --git a/libavcodec/hevc/hevcdec.c b/libavcodec/hevc/hevcdec.c index 4077ed3ac5..41bb7fdfe1 100644 --- a/libavcodec/hevc/hevcdec.c +++ b/libavcodec/hevc/hevcdec.c @@ -3459,6 +3459,9 @@ do_output: if (ff_container_fifo_read(s->output_fifo, frame) >= 0) { if (!(avctx->export_side_data & AV_CODEC_EXPORT_DATA_FILM_GRAIN)) av_frame_remove_side_data(frame, AV_FRAME_DATA_FILM_GRAIN_PARAMS); + if (!CONFIG_LIBLCEVC_DEC && + !(avctx->export_side_data & AV_CODEC_EXPORT_DATA_ENHANCEMENTS)) + av_frame_remove_side_data(frame, AV_FRAME_DATA_LCEVC); return 0; } diff --git a/libavcodec/hevc/refs.c b/libavcodec/hevc/refs.c index 65abd09a21..88c5c5ea52 100644 --- a/libavcodec/hevc/refs.c +++ b/libavcodec/hevc/refs.c @@ -83,10 +83,23 @@ static HEVCFrame *alloc_frame(HEVCContext *s) if (frame->f) continue; + ret = ff_progress_frame_alloc(s->avctx, &frame->tf); + if (ret < 0) + return NULL; + + if (CONFIG_LIBLCEVC_DEC && s->sei.common.lcevc.info && !s->avctx->hwaccel && + !(s->avctx->export_side_data & AV_CODEC_EXPORT_DATA_ENHANCEMENTS)) { + HEVCSEILCEVC *lcevc = &s->sei.common.lcevc; + ret = ff_frame_new_side_data_from_buf(s->avctx, frame->tf.f, + AV_FRAME_DATA_LCEVC, &lcevc->info); + if (ret < 0) + goto fail; + } + ret = ff_progress_frame_get_buffer(s->avctx, &frame->tf, AV_GET_BUFFER_FLAG_REF); if (ret < 0) - return NULL; + goto fail; frame->rpl = ff_refstruct_allocz(s->pkt.nb_nals * sizeof(*frame->rpl)); if (!frame->rpl) From patchwork Sat Aug 31 16:31:09 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Almer X-Patchwork-Id: 51262 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:612c:4027:b0:48e:c0f8:d0de with SMTP id ky39csp1031712vqb; Sat, 31 Aug 2024 10:10:17 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCVGPSQhRRHLD/HC8bC0i8yhQZTy6V5PuPWO2qE5jxiarzC1+R27zFKVdhKcPqAu5s1AUxY8vUzvwoH2fcgFjr+9@gmail.com X-Google-Smtp-Source: AGHT+IF0MyfoSmT/uqOrKE4Wb+AyIycok4uiwxXPTUVjM9SikA4HaX2YXAcObgCF4+ZrLCMsNdTQ X-Received: by 2002:a17:907:3f89:b0:a80:a193:a509 with SMTP id a640c23a62f3a-a89a34e1a7emr199955166b.2.1725124217209; Sat, 31 Aug 2024 10:10:17 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1725124217; cv=none; d=google.com; s=arc-20240605; b=fqSMrpUU6JJYig4KfhziaCFcRv6xQ1OC7AdsjHZl/CqNFrZHivscfbbfa26wpvb75b z61NGtoJU8ewBkJkzL+v35mVHyooTkdNz2KgmWBVwemDutqSvOYSCIKCacLj6jQs+c6c npZo/Wa420Mt/7Hfr89Ip2FWNzssqAx+Oyb4aslvMYa2jxOOpy5LN8pqnLhuwhnNk5Px awImEjzz+NyBrJaij/UTS4r3GgQUoY8TxT4euQ+WL/nZ7ow0I+WgH83gFg1lfZmFZR27 40MHdlkmQ+yMfZNsb4TifjQ2pxQRY813i0jt4WTENKaGBW4AF7g10+gSYN1xQGXWOfxU Wo0Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:dkim-signature:delivered-to; bh=J05P8rmGc4H0AaOf0VXbKGaV9z5C4L/PIP8J1wWn6Ao=; fh=YOA8vD9MJZuwZ71F/05pj6KdCjf6jQRmzLS+CATXUQk=; b=W9QOR7w7nkp+Q2XF942f2qQ7/gZj1JfHFtx8YqnQ6D+TddPw9C7/6AVtQfHx/ZVUPm z1rIzWr28QKcddrI3WHYOZOhCLKEUewvYDfCZbsAl377/32ScQ3J0TRoUYrdKv9NQ9xV +o6+pvQo8xEJOW00B+9jc84nrGp0hhl/cTNLepU1iWxn90NpUQW/EU+rrtfDVtiCo+gv uKd69ivfWB0W9rpqmkPlOiy43o60kvGz9BZyS9VfB8bb7TlJeFNa+jEIli7wtr+snFkJ eA4/ivWzXpCYGJt9DpwGc+n3+h6dIEPWIjdPQqStIuKRlmyvgMvgEIw19b5oUqinBbrw 6bgw==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=e6ZgkwND; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com; dara=fail header.i=@gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id a640c23a62f3a-a89891cbd14si499897866b.807.2024.08.31.10.10.16; Sat, 31 Aug 2024 10:10:17 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=e6ZgkwND; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com; dara=fail header.i=@gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 0F8F068DE2C; Sat, 31 Aug 2024 19:31:09 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pj1-f45.google.com (mail-pj1-f45.google.com [209.85.216.45]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 3C94868DE17 for ; Sat, 31 Aug 2024 19:31:00 +0300 (EEST) Received: by mail-pj1-f45.google.com with SMTP id 98e67ed59e1d1-2d885019558so884300a91.2 for ; Sat, 31 Aug 2024 09:31:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1725121857; x=1725726657; darn=ffmpeg.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=Kio4RJKrR274LSSFCDIMdnZQCvHFqnmHvzP/sszPBos=; b=e6ZgkwND07EEtb0n7zRBxHvLtbgKS0uNTOwhK+sqFwa+XISBuiV2YJ9O9Xhwu3ssDQ BJJjMvbOiKGub0+8dFZqrt4r6qs52o8ADTKOS80aWJmrd2rfLDPTGOyeUuAUCs2AvVLk CzJ7W/DMsCkjoVsa5pTAQQgW5EfClN3GePqY275qc0UDzpDeRtrsuUmU1Cc1DVh8skX5 4lfDyE9A72fl86Vuj7B+G6DB+s/lbjGiUUctiURHB9DsJPBwnSL3Gn/UECAtEYSbYnOv klBkc3rQIV8merDSukNKzU4g+hc6LgyYJMYDHNYjgq26Qn9vdlKveilyW6UJYgdnbO85 X35w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725121857; x=1725726657; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Kio4RJKrR274LSSFCDIMdnZQCvHFqnmHvzP/sszPBos=; b=Fez4IntdCCiTXs7ebwWHOUkM4vAcfGen+7Oy2FYLLdaWvko67Xc9PCCyljsfhtYWP6 8YNAYbjfokRYiQ4YCbscKJu6mDnrX0C3V3Ssu1va00NSJVS09jsXlJ/dki8XDVGI8hLR 4bE6hHjQ+n3nXo1xJBeJU36TPFIorqG8ODIVbdWP4iWFqdoEMmqMcIWhS1si/sq2PROv WrhtbXc/Xl+7x/ikfVgO0PE4xrKJjj6nmJJoX0026s1M8hAS6a38UacLmGBpxBBtad2k Ob3zKtXUUYyt8v7NIDcPcXynupj514es6LlpCooJ9tLxDtaW52VRLSbWdhBRzvGML1x4 2iAg== X-Gm-Message-State: AOJu0YzAXsvLYGfM0J5QqzYL/YUKzT3uO4QEAq1nVw+dEAEisDvUSxZa ig5iIP6POSH4r+l+yJWOl5VatJUFiaW9Z4XQYMjWo/lCnmtm7noHI5lq+g== X-Received: by 2002:a17:90a:130a:b0:2d3:b438:725f with SMTP id 98e67ed59e1d1-2d893a9dc2amr2388243a91.24.1725121857028; Sat, 31 Aug 2024 09:30:57 -0700 (PDT) Received: from localhost.localdomain ([190.194.167.233]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2d85b3b9522sm5935328a91.50.2024.08.31.09.30.55 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 31 Aug 2024 09:30:56 -0700 (PDT) From: James Almer To: ffmpeg-devel@ffmpeg.org Date: Sat, 31 Aug 2024 13:31:09 -0300 Message-ID: <20240831163114.4197-8-jamrial@gmail.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240831163114.4197-1-jamrial@gmail.com> References: <20240831163114.4197-1-jamrial@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 08/13] avcodec/codec_id: add an LCEVC codec id for raw LCEVC data X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: TDTaxJrplnEU Signed-off-by: James Almer --- libavcodec/codec_desc.c | 6 ++++++ libavcodec/codec_id.h | 1 + 2 files changed, 7 insertions(+) diff --git a/libavcodec/codec_desc.c b/libavcodec/codec_desc.c index a28ef68061..03dea5751a 100644 --- a/libavcodec/codec_desc.c +++ b/libavcodec/codec_desc.c @@ -3696,6 +3696,12 @@ static const AVCodecDescriptor codec_descriptors[] = { .name = "smpte_2038", .long_name = NULL_IF_CONFIG_SMALL("SMPTE ST 2038 VANC in MPEG-2 TS"), }, + { + .id = AV_CODEC_ID_LCEVC, + .type = AVMEDIA_TYPE_DATA, + .name = "lcevc", + .long_name = NULL_IF_CONFIG_SMALL("LCEVC (Low Complexity Enhancement Video Coding) / MPEG-5 LCEVC / MPEG-5 part 2"), + }, { .id = AV_CODEC_ID_MPEG2TS, .type = AVMEDIA_TYPE_DATA, diff --git a/libavcodec/codec_id.h b/libavcodec/codec_id.h index 0ab1e34a61..0a8d3bed1e 100644 --- a/libavcodec/codec_id.h +++ b/libavcodec/codec_id.h @@ -589,6 +589,7 @@ enum AVCodecID { AV_CODEC_ID_TIMED_ID3, AV_CODEC_ID_BIN_DATA, AV_CODEC_ID_SMPTE_2038, + AV_CODEC_ID_LCEVC, AV_CODEC_ID_PROBE = 0x19000, ///< codec_id is not known (like AV_CODEC_ID_NONE) but lavf should attempt to identify it From patchwork Sat Aug 31 16:31:10 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Almer X-Patchwork-Id: 51260 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:612c:4027:b0:48e:c0f8:d0de with SMTP id ky39csp1025389vqb; Sat, 31 Aug 2024 09:55:12 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCW6ri/79gD1sLsjE9pvjgR5S5UHd8mkyew7t+6kHkv0lGwZBW7Zp0r+cgVylbSBnyWReC/vpynODMYZek0dZ2n7@gmail.com X-Google-Smtp-Source: AGHT+IGStqELqz29e6v4Nnjvq1Dmkj6N24aNsRtbo08YtR71dCzYshdZwZ5eCqHm8cs/Aj5KQ9QH X-Received: by 2002:a05:651c:198a:b0:2f4:f4da:d584 with SMTP id 38308e7fff4ca-2f61e0ad3b8mr21903291fa.7.1725123312322; Sat, 31 Aug 2024 09:55:12 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1725123312; cv=none; d=google.com; s=arc-20240605; b=DI5pvL7uaKrP1fq2yryyeVRSbUPhcMkXlcQNgkXLm1dbBLJGj2gwigvPS9G/BsI7YF 1Idk9BB14Gon+ucWcxjlSTRHNhyRR6sbPhbTZTTXYCoxpAcwFsV5s6uzr+iGvXrmRjtR lHkdofumzTyZ9VAGwHF2zzb3TTod4LzLEYVlbsB77sAIVWZJ4nILY8AZvvW6GJiKteXK bC5dD9qebLt8LjfHk4dgrbu+OvikW5OUuTJG6dlkyQfp+5/Uzh0KNZU369gayvA2O/r9 SxfZy//h5D/1xuXLvcR6A/JA4+gBrRfHErX68mOqENEbhORJR4afgzQa5ZQXYKytS/gL yWCQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:dkim-signature:delivered-to; bh=NZqsteGy7kuyMrJomskZrWgXn1kUb7ktRJtXOLk7QIU=; fh=YOA8vD9MJZuwZ71F/05pj6KdCjf6jQRmzLS+CATXUQk=; b=ktMAfUSBx63/oNRYdxVrxogSh1z/z5IBpxrYZc+yM2VAMQqF7K2ksP8YsTLlDYI0ZD mdah9Vas8Fd+cu0dEnn5oyKzZdg01KwrBgm0zgPuHHPc2XcgRjpIueHjlSTtuKaf0FXo Ekt+KL9p9MdV2actkPf6xyQz7JPupDJhVzqZFaqG1fltk5Y7RJuVNvwQXPy5tkHOWClV fvbDPsndllb4oJMnuwpXUeVuEm60cgiEpNCbUvfHPL7+ts1STRPFhZXO4Z2k0OzVIf2E jO7ek9Rd4VHbVNNbrNYu+tbGHVjlpZ01UyYyfrkCR9RZR2dn48DESkPZN2+Dc/BAhzgo lMIg==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=DTWm5KUg; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com; dara=fail header.i=@gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 4fb4d7f45d1cf-5c24e7f7c28si456056a12.215.2024.08.31.09.55.11; Sat, 31 Aug 2024 09:55:12 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=DTWm5KUg; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com; dara=fail header.i=@gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 4AA1568DE18; Sat, 31 Aug 2024 19:31:05 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pf1-f176.google.com (mail-pf1-f176.google.com [209.85.210.176]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 541EB68DDA2 for ; Sat, 31 Aug 2024 19:31:02 +0300 (EEST) Received: by mail-pf1-f176.google.com with SMTP id d2e1a72fcca58-715cc93694fso2580769b3a.2 for ; Sat, 31 Aug 2024 09:31:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1725121859; x=1725726659; darn=ffmpeg.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=+zQwg0/WBQog/vwh4LIIcpphuuNHbCICpNZ2JW+xZ14=; b=DTWm5KUghPDJIYRH1vy64zLY3CsxFXKuzo8grbw/BnAQJDOrUjLUa/QWHHiWqgHMxQ upNil1sfgghaYOp5BVwrAYQPKyKIUb2EyvKVe6S99u46IijmulUhDMebf9fTSlzC/hzN tbWq5byLLyw2jSy3jJsoYNYoKVzFJss6hKuieYbBtN/ZQg9xyTkdzsJtDBmmMdr+M44h iENRdUYg8tO6iE+dA9R+tHLFQq0sEOMGTW2k1uv9obkXvzBDTmAb421U1l2dv45DCqa9 fh6mC8zfEAnEnEkci6a2nC1SczH4gt/JWRKFMJnekUhNIUquVsNh9f9Ush4nERSk5pLt 5CXA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725121859; x=1725726659; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=+zQwg0/WBQog/vwh4LIIcpphuuNHbCICpNZ2JW+xZ14=; b=r+Nvwu6WKzTEkMJYunAmPQ6BuQOUrDLVztAsfrNXPZT7LsPEgHrxopmiUlQUfUWIM/ wnPAj/l2mZjgpEWIaSDA3TMgCCYvgtln08Cza1xJuNCcChkUAJOO+7HFM570sJRHt71n boYBM5bwB8vLGzlN2BKJXsVz0JpiQMWcHA/jB4SkxxCfJz80C1hHX+5D1jV183Pe0GPN yPOANTexkMdmKeWKH2BUYXCQuyS/SoezbuxmURLbD+WFbgoKfUscwSQxxObIeadkrfPy WWmeYrxS334UF6nhzly3406nsnBWVWbeHu03EtGgkySTeOTB7KUbODMKT41gXaZEKhul ohjg== X-Gm-Message-State: AOJu0YwaRd+KYohbGKxtxaPlSlWxO11mFcOVzS8ZVmo1/NvheL4a6bqm bkXLXWgqItCXKK+uio8/bUFG30FHAOzRvq8lSSBN0aly6v5Q6kaL2H4lyQ== X-Received: by 2002:a05:6a20:9f08:b0:1c0:ec1c:f4a5 with SMTP id adf61e73a8af0-1ced04bd8aamr2538765637.25.1725121859193; Sat, 31 Aug 2024 09:30:59 -0700 (PDT) Received: from localhost.localdomain ([190.194.167.233]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2d85b3b9522sm5935328a91.50.2024.08.31.09.30.57 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 31 Aug 2024 09:30:58 -0700 (PDT) From: James Almer To: ffmpeg-devel@ffmpeg.org Date: Sat, 31 Aug 2024 13:31:10 -0300 Message-ID: <20240831163114.4197-9-jamrial@gmail.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240831163114.4197-1-jamrial@gmail.com> References: <20240831163114.4197-1-jamrial@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 09/13] avformat: add an LCEVC stream group X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: m0ISNdtrqAyc Signed-off-by: James Almer --- libavformat/avformat.c | 5 +++++ libavformat/avformat.h | 15 +++++++++++++++ libavformat/dump.c | 27 +++++++++++++++++++++++++++ libavformat/options.c | 29 ++++++++++++++++++++++++++++- 4 files changed, 75 insertions(+), 1 deletion(-) diff --git a/libavformat/avformat.c b/libavformat/avformat.c index b4f20502fb..06dcde0565 100644 --- a/libavformat/avformat.c +++ b/libavformat/avformat.c @@ -104,6 +104,10 @@ void ff_free_stream_group(AVStreamGroup **pstg) av_freep(&stg->params.tile_grid->offsets); av_freep(&stg->params.tile_grid); break; + case AV_STREAM_GROUP_PARAMS_LCEVC: + av_opt_free(stg->params.lcevc); + av_freep(&stg->params.lcevc); + break; default: break; } @@ -327,6 +331,7 @@ const char *avformat_stream_group_name(enum AVStreamGroupParamsType type) case AV_STREAM_GROUP_PARAMS_IAMF_AUDIO_ELEMENT: return "IAMF Audio Element"; case AV_STREAM_GROUP_PARAMS_IAMF_MIX_PRESENTATION: return "IAMF Mix Presentation"; case AV_STREAM_GROUP_PARAMS_TILE_GRID: return "Tile Grid"; + case AV_STREAM_GROUP_PARAMS_LCEVC: return "LCEVC (Split video and enhancement)"; } return NULL; } diff --git a/libavformat/avformat.h b/libavformat/avformat.h index 4a3fb00529..b611e110e1 100644 --- a/libavformat/avformat.h +++ b/libavformat/avformat.h @@ -1084,11 +1084,25 @@ typedef struct AVStreamGroupTileGrid { int height; } AVStreamGroupTileGrid; +typedef struct AVStreamGroupLCEVC { + const AVClass *av_class; + + /** + * Width of the final stream for presentation. + */ + int width; + /** + * Height of the final image for presentation. + */ + int height; +} AVStreamGroupLCEVC; + enum AVStreamGroupParamsType { AV_STREAM_GROUP_PARAMS_NONE, AV_STREAM_GROUP_PARAMS_IAMF_AUDIO_ELEMENT, AV_STREAM_GROUP_PARAMS_IAMF_MIX_PRESENTATION, AV_STREAM_GROUP_PARAMS_TILE_GRID, + AV_STREAM_GROUP_PARAMS_LCEVC, }; struct AVIAMFAudioElement; @@ -1130,6 +1144,7 @@ typedef struct AVStreamGroup { struct AVIAMFAudioElement *iamf_audio_element; struct AVIAMFMixPresentation *iamf_mix_presentation; struct AVStreamGroupTileGrid *tile_grid; + struct AVStreamGroupLCEVC *lcevc; } params; /** diff --git a/libavformat/dump.c b/libavformat/dump.c index 5e1f367742..f20c2c4953 100644 --- a/libavformat/dump.c +++ b/libavformat/dump.c @@ -789,6 +789,33 @@ static void dump_stream_group(const AVFormatContext *ic, uint8_t *printed, } break; } + case AV_STREAM_GROUP_PARAMS_LCEVC: { + const AVStreamGroupLCEVC *lcevc = stg->params.lcevc; + AVCodecContext *avctx = avcodec_alloc_context3(NULL); + const char *ptr = NULL; + av_log(NULL, AV_LOG_INFO, " LCEVC:"); + if (avctx && stg->nb_streams && !avcodec_parameters_to_context(avctx, stg->streams[0]->codecpar)) { + avctx->width = lcevc->width; + avctx->height = lcevc->height; + avctx->coded_width = lcevc->width; + avctx->coded_height = lcevc->height; + if (ic->dump_separator) + av_opt_set(avctx, "dump_separator", ic->dump_separator, 0); + buf[0] = 0; + avcodec_string(buf, sizeof(buf), avctx, is_output); + ptr = av_stristr(buf, " "); + } + avcodec_free_context(&avctx); + if (ptr) + av_log(NULL, AV_LOG_INFO, "%s", ptr); + av_log(NULL, AV_LOG_INFO, "\n"); + for (int i = 0; i < stg->nb_streams; i++) { + const AVStream *st = stg->streams[i]; + dump_stream_format(ic, st->index, i, index, is_output, AV_LOG_VERBOSE); + printed[st->index] = 1; + } + break; + } default: break; } diff --git a/libavformat/options.c b/libavformat/options.c index 485265df52..039f1eea42 100644 --- a/libavformat/options.c +++ b/libavformat/options.c @@ -348,7 +348,6 @@ static const AVOption tile_grid_options[] = { { "vertical_offset", NULL, OFFSET(vertical_offset), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, INT_MAX, FLAGS }, { NULL }, }; -#undef FLAGS #undef OFFSET static const AVClass tile_grid_class = { @@ -357,6 +356,20 @@ static const AVClass tile_grid_class = { .option = tile_grid_options, }; +#define OFFSET(x) offsetof(AVStreamGroupLCEVC, x) +static const AVOption lcevc_options[] = { + { "video_size", "size of video after LCEVC enhancement has been applied", OFFSET(width), + AV_OPT_TYPE_IMAGE_SIZE, { .str = NULL }, 0, INT_MAX, FLAGS }, + { NULL }, +}; +#undef OFFSET + +static const AVClass lcevc_class = { + .class_name = "AVStreamGroupLCEVC", + .version = LIBAVUTIL_VERSION_INT, + .option = lcevc_options, +}; + static void *stream_group_child_next(void *obj, void *prev) { AVStreamGroup *stg = obj; @@ -368,6 +381,8 @@ static void *stream_group_child_next(void *obj, void *prev) return stg->params.iamf_mix_presentation; case AV_STREAM_GROUP_PARAMS_TILE_GRID: return stg->params.tile_grid; + case AV_STREAM_GROUP_PARAMS_LCEVC: + return stg->params.lcevc; default: break; } @@ -375,6 +390,8 @@ static void *stream_group_child_next(void *obj, void *prev) return NULL; } +#undef FLAGS + static const AVClass *stream_group_child_iterate(void **opaque) { uintptr_t i = (uintptr_t)*opaque; @@ -393,6 +410,9 @@ static const AVClass *stream_group_child_iterate(void **opaque) case AV_STREAM_GROUP_PARAMS_TILE_GRID: ret = &tile_grid_class; break; + case AV_STREAM_GROUP_PARAMS_LCEVC: + ret = &lcevc_class; + break; default: break; } @@ -462,6 +482,13 @@ AVStreamGroup *avformat_stream_group_create(AVFormatContext *s, stg->params.tile_grid->av_class = &tile_grid_class; av_opt_set_defaults(stg->params.tile_grid); break; + case AV_STREAM_GROUP_PARAMS_LCEVC: + stg->params.lcevc = av_mallocz(sizeof(*stg->params.lcevc)); + if (!stg->params.lcevc) + goto fail; + stg->params.lcevc->av_class = &lcevc_class; + av_opt_set_defaults(stg->params.lcevc); + break; default: goto fail; } From patchwork Sat Aug 31 16:31:11 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Almer X-Patchwork-Id: 51265 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:612c:4027:b0:48e:c0f8:d0de with SMTP id ky39csp1099021vqb; Sat, 31 Aug 2024 14:20:12 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCVQZZSb7CbqD/OAD8UXJPC9NcLTfsemyYZPfDsr/JgcE7TFs+J9vYXg2zFOfd2gz9B2nI2H8uMxYQRl0em0nh80@gmail.com X-Google-Smtp-Source: AGHT+IHvBSR9NvXiIjiTYwvVXMoa5p0s+C2+zf9JLHuDiWf8p/Wn22Pe5x78eJ1ZHxAILB2eWCym X-Received: by 2002:a05:6512:224a:b0:52f:e5:3765 with SMTP id 2adb3069b0e04-53546b8f4c7mr2315910e87.6.1725139211707; Sat, 31 Aug 2024 14:20:11 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1725139211; cv=none; d=google.com; s=arc-20240605; b=JYIQEsfpHDQqm2N5beNew5unvssUFJb36iP5T6VHuTYvmHcrZQ7RYSLtzvHbt3qG5q EXuBrTm4OkL3q/YEKU7pU8m79AxoKdemslsQGPfo2YkdqJRe4/uOMgsvqZEEhVZOAoe1 2ybKB7aDGgv1xLgAyJaobw6RYROvVJC7wSCdGe0HhrVGMBz4EBfNy+bPATg3AaTpa0on jE4lJPuFqmPishh+q58gCuHdxNTw4NdRDk2XzlxIFtn2WotiN6tvvW/p9CoQXgzFzHl+ K+q0qoZDTC4XYpWC62oOVu/0fO6WNAvJsY4E7PdlQbBhtQtSsdAxR+l7GbaxVVABhCIF VnwA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:dkim-signature:delivered-to; bh=bhU4Gmu1iLa5G79paiLOF2K6JOFvJyldtHUizh2R2lI=; fh=YOA8vD9MJZuwZ71F/05pj6KdCjf6jQRmzLS+CATXUQk=; b=auIcSMYlz92ebs1/tQDLFfTIah1X0ywdRpJzNOTkYbO1qU4+g21XN7J2osGGcfYUNg tvV+dl8FWmUUCfGhRLgnFpqWVh78fS0Zc9XhVkbDa69uWMSmrWJf5e/nuIbHeGAN32LX FGPP1aNPoJXFVrGzS8u6T2n+I9c7cvevVyTX4GISQHd7nYC/WivpQ1ed/+xzJcm7EZdA A9H+Mf2CyBEVn0dBHMNK3Z00l+eX63IYcG7R3S0A62uKjws2LZePqzuwe3Jz0H4Gejt4 7dFsRxCqx/fcd02o/nhtAFVTsSjql/qPfZiLn297iXdC5oax810BUZhBOUBF5Cp64Nif VuGg==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=FQNcldRh; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com; dara=fail header.i=@gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id a640c23a62f3a-a89891c89aesi524644266b.779.2024.08.31.14.20.11; Sat, 31 Aug 2024 14:20:11 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=FQNcldRh; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com; dara=fail header.i=@gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id F189968DE1C; Sat, 31 Aug 2024 19:31:07 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pj1-f47.google.com (mail-pj1-f47.google.com [209.85.216.47]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id A699268DDF0 for ; Sat, 31 Aug 2024 19:31:03 +0300 (EEST) Received: by mail-pj1-f47.google.com with SMTP id 98e67ed59e1d1-2d5f5d8cc01so2005873a91.0 for ; Sat, 31 Aug 2024 09:31:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1725121861; x=1725726661; darn=ffmpeg.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=OUU/0Gh5GQchFcJYaErkakompCZYhMY6dxcX6xX++Qo=; b=FQNcldRhmhMieVSnkvEzvpuR2ycnRt3z3HyEHTFALd8nbiy47k78rUpg2hlyYeWhn/ E2XdoVWfgITf05ICU2TsKTNCeF8RGt2oFnbLjo0ieQDtl2hFc7dQ4UA7gLKr8YRe8J5N 8IhO21hhkTPC9HFdX1NJoiOFHC0krERFfuzvxGAlfhQnRuR3fZT23QjoAy8KfWh26U0R qhIgq2mf2S/blWnUrE9AJdRa9/EC9h6rZb9sGcq/2xNCtUWs/jauOjjKT0zkbuyK+8Mx 5sLT+9yHc8DMAeAmUo+k8JQwh2AuzxLf5BGLv52Hnczrarjgc/UL2EyAHZ9uS3T2OdOz C74A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725121861; x=1725726661; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=OUU/0Gh5GQchFcJYaErkakompCZYhMY6dxcX6xX++Qo=; b=eyNFzlrgrD78AwiOQof6J+mm62ja80SVeT7fzmqKcy8FIKwq+y+4vlJHOZ4aU9dqZF Bts/2xQj5WaFg4/nvAlOt971VvA0kbx/D8cvfXDL3Zp6BtMpgWwm9W0dQ7xRWSjqhzyr 9oH8pArrf0fUEK7QS5viI2NvQYGOOEYhLdUSX0IXzNhHm/QpUs9Apgo3UHzuE0N5NQV0 K1yldC1Gi/IUc4ba8BOO4Nxzml0vANwoUV4SBnLeaveaN98txpTmDKRAJ6XYGdW5O2aj a6oDRLV6EtkbrqTUUD/3KTD3BT9Evv/r6YOJ2/qAX50TV5oJq4IRHMmK+oQXgtlDgp7v gXjw== X-Gm-Message-State: AOJu0YyUlQjb6Wp35TZ2iisstv6wHAK2XsdLEy0kOOCfoNYmHGlYkohq cjW9N+CP5JWhFmtwgsWL7dD1owrlqDdLJ5K+lrhGemEJzVMhKyLMwUPe1g== X-Received: by 2002:a17:90b:23d6:b0:2d8:89ff:db1a with SMTP id 98e67ed59e1d1-2d889ffddc1mr5584216a91.8.1725121861108; Sat, 31 Aug 2024 09:31:01 -0700 (PDT) Received: from localhost.localdomain ([190.194.167.233]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2d85b3b9522sm5935328a91.50.2024.08.31.09.30.59 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 31 Aug 2024 09:31:00 -0700 (PDT) From: James Almer To: ffmpeg-devel@ffmpeg.org Date: Sat, 31 Aug 2024 13:31:11 -0300 Message-ID: <20240831163114.4197-10-jamrial@gmail.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240831163114.4197-1-jamrial@gmail.com> References: <20240831163114.4197-1-jamrial@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 10/13] avformat/mov: support for LCEVC tracks X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: hWwR9TXdtXaf Co-authored-by: V-Nova Team Signed-off-by: James Almer --- libavformat/isom.h | 3 ++ libavformat/isom_tags.c | 2 + libavformat/mov.c | 86 +++++++++++++++++++++++++++++++++++++++++ 3 files changed, 91 insertions(+) diff --git a/libavformat/isom.h b/libavformat/isom.h index 15e9466e41..4723397048 100644 --- a/libavformat/isom.h +++ b/libavformat/isom.h @@ -212,6 +212,8 @@ typedef struct MOVStreamContext { unsigned drefs_count; MOVDref *drefs; int dref_id; + unsigned tref_flags; + int tref_id; int timecode_track; int width; ///< tkhd width int height; ///< tkhd height @@ -408,6 +410,7 @@ void ff_mp4_parse_es_descr(AVIOContext *pb, int *es_id); #define MOV_SAMPLE_DEPENDENCY_YES 0x1 #define MOV_SAMPLE_DEPENDENCY_NO 0x2 +#define MOV_TREF_FLAG_ENHANCEMENT 0x1 #define TAG_IS_AVCI(tag) \ ((tag) == MKTAG('a', 'i', '5', 'p') || \ diff --git a/libavformat/isom_tags.c b/libavformat/isom_tags.c index 058f0f2a59..5dd72d570e 100644 --- a/libavformat/isom_tags.c +++ b/libavformat/isom_tags.c @@ -290,6 +290,8 @@ const AVCodecTag ff_codec_movvideo_tags[] = { { AV_CODEC_ID_CFHD, MKTAG('C', 'F', 'H', 'D') }, + { AV_CODEC_ID_LCEVC, MKTAG('l', 'v', 'c', '1') }, /* LCEVC raw payload */ + { AV_CODEC_ID_NONE, 0 }, }; diff --git a/libavformat/mov.c b/libavformat/mov.c index d57c4f150b..7cb50fdbf8 100644 --- a/libavformat/mov.c +++ b/libavformat/mov.c @@ -2434,6 +2434,30 @@ static int mov_read_dvc1(MOVContext *c, AVIOContext *pb, MOVAtom atom) return 0; } +static int mov_read_sbas(MOVContext* c, AVIOContext* pb, MOVAtom atom) +{ + AVStream* st; + MOVStreamContext* sc; + + if (c->fc->nb_streams < 1) + return 0; + + /* For SBAS this should be fine - though beware if someone implements a + * tref atom processor that doesn't drop down to default then this may + * be lost. */ + if (atom.size > 4) { + av_log(c->fc, AV_LOG_ERROR, "Only a single tref of type sbas is supported\n"); + return AVERROR_PATCHWELCOME; + } + + st = c->fc->streams[c->fc->nb_streams - 1]; + sc = st->priv_data; + sc->tref_id = avio_rb32(pb); + sc->tref_flags |= MOV_TREF_FLAG_ENHANCEMENT; + + return 0; +} + /** * An strf atom is a BITMAPINFOHEADER struct. This struct is 40 bytes itself, * but can have extradata appended at the end after the 40 bytes belonging @@ -4995,6 +5019,8 @@ static int mov_read_trak(MOVContext *c, AVIOContext *pb, MOVAtom atom) st->codecpar->codec_type = AVMEDIA_TYPE_DATA; sc->ffindex = st->index; c->trak_index = st->index; + sc->tref_flags = 0; + sc->tref_id = -1; sc->refcount = 1; if ((ret = mov_read_default(c, pb, atom)) < 0) @@ -9052,6 +9078,7 @@ static const MOVParseTableEntry mov_default_parse_table[] = { { MKTAG('a','v','c','C'), mov_read_glbl }, { MKTAG('p','a','s','p'), mov_read_pasp }, { MKTAG('c','l','a','p'), mov_read_clap }, +{ MKTAG('s','b','a','s'), mov_read_sbas }, { MKTAG('s','i','d','x'), mov_read_sidx }, { MKTAG('s','t','b','l'), mov_read_default }, { MKTAG('s','t','c','o'), mov_read_stco }, @@ -9132,6 +9159,7 @@ static const MOVParseTableEntry mov_default_parse_table[] = { { MKTAG('i','i','n','f'), mov_read_iinf }, { MKTAG('a','m','v','e'), mov_read_amve }, /* ambient viewing environment box */ { MKTAG('l','h','v','C'), mov_read_lhvc }, +{ MKTAG('l','v','c','C'), mov_read_glbl }, #if CONFIG_IAMFDEC { MKTAG('i','a','c','b'), mov_read_iacb }, #endif @@ -10029,6 +10057,20 @@ static int mov_parse_tiles(AVFormatContext *s) return 0; } +static AVStream *mov_find_reference_track(AVFormatContext *s, AVStream *st) +{ + MOVStreamContext *sc = st->priv_data; + + if (sc->tref_id < 0) + return NULL; + + for (int i = 0; i < s->nb_streams; i++) + if (s->streams[i]->id == sc->tref_id) + return s->streams[i]; + + return NULL; +} + static int mov_read_header(AVFormatContext *s) { MOVContext *mov = s->priv_data; @@ -10154,6 +10196,50 @@ static int mov_read_header(AVFormatContext *s) } export_orphan_timecode(s); + /* Update the LCEVC stream as a data stream. */ + for (i = 0; i < s->nb_streams; i++) { + AVStreamGroup *stg; + AVStream *st = s->streams[i]; + AVStream *st_base; + MOVStreamContext *sc = st->priv_data; + MOVStreamContext *sc_base; + + /* Find the enhancement stream. */ + if (!(sc->tref_flags & MOV_TREF_FLAG_ENHANCEMENT)) + continue; + + st_base = mov_find_reference_track(s, st); + if (!st_base) { + av_log(s, AV_LOG_ERROR, "Failed to find base stream for enhancement stream\n"); + return AVERROR_INVALIDDATA; + } + sc_base = st_base->priv_data; + + /* Make reference bidirectional. */ + if (sc_base->tref_id >= 0) { + av_log(s, AV_LOG_ERROR, "Base stream must not already have a stream reference\n"); + return AVERROR_INVALIDDATA; + } + + st->codecpar->codec_type = AVMEDIA_TYPE_DATA; + sc_base->tref_id = st->id; + + stg = avformat_stream_group_create(s, AV_STREAM_GROUP_PARAMS_LCEVC, NULL); + if (!stg) + return AVERROR(ENOMEM); + + stg->id = st->id; + stg->params.lcevc->width = st->codecpar->width; + stg->params.lcevc->height = st->codecpar->height; + + err = avformat_stream_group_add_stream(stg, st_base); + if (err < 0) + return err; + err = avformat_stream_group_add_stream(stg, st); + if (err < 0) + return err; + } + for (i = 0; i < s->nb_streams; i++) { AVStream *st = s->streams[i]; FFStream *const sti = ffstream(st); From patchwork Sat Aug 31 16:31:12 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Almer X-Patchwork-Id: 51259 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:612c:4027:b0:48e:c0f8:d0de with SMTP id ky39csp1023866vqb; Sat, 31 Aug 2024 09:50:12 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCXuIXNRHm/F97yf3gd39e4FUiIQglewkByDtMmNIKbyCkmRvkINYGmr/XHdM6adwCqeM6h9qJalEGKfch9/1d72@gmail.com X-Google-Smtp-Source: AGHT+IGE204wFbZb7+659YQEDjcjNESXb0jBlnOj6TQ/qUovQnC9NrKC5pMdWLCMenNtZRQ+i+6j X-Received: by 2002:a2e:be03:0:b0:2f4:f253:ec7 with SMTP id 38308e7fff4ca-2f6104270b5mr88517181fa.0.1725123012031; Sat, 31 Aug 2024 09:50:12 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1725123012; cv=none; d=google.com; s=arc-20240605; b=QFbv8PlnBPJcnn1cY0cW0fffnYPFspiMHGBSXvNGPMgVSZ9F9emh/mwlcThhYRzsZv 8XTSfzcktXo3ud++Tkm/obOrvFGxIyye4PUMfbYZj5t1UKcDpEJHgh66jzRLUN3FJ+hC 6v4HSObYQvUtBtML0a9dpKm4Jvm0t/seAwDPxYDnjPlSbcYwUXTN1aEs+J688BFPtVHj H/XH0SRDdKQK5qLhKHLvZFhhPSEHpk+IpO3o0L/7gZIRRkYHDd+czJKPaV49sPpi8pQN 0k/Ucy68BoQ/6lSQ2cWlwNxb/qn1VTYI0PqXvmkc2ipJ4yuACF1p0tgf6JI+pdASqAFJ vPWg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:dkim-signature:delivered-to; bh=F+GCeic/gH0xe3f7k3QT/ipiIyno5LbQ2QiBNhaaoa8=; fh=YOA8vD9MJZuwZ71F/05pj6KdCjf6jQRmzLS+CATXUQk=; b=WxbSthWAhdt3xsTKjD3MRM9DOTokaqvxu58k11/vRRyokI4vk3truO0PcfO/L+3PB3 d/4hLKt4aYZLK4kf8PR64KblVUhBfHHwr2ENwqJX5Rlc69G/td4kPxpbxKd7mtt1TTtB gj1nsyS1Uqf2Ha3ftTuNwC1xMi3r5GWEd8UNEXjvj2iroITB4Lr+2FjmjyomMZgj9LPV v86BTi6fJfm8paXg9ji8DfwIJ9gxpxzyTe1SF0/5EmhvwIi/jk2snN1VjzXDrPp/thfE s9juhtXa+6vE6xG+ZS6/VxJeko1U+cA53RNiJEKGmktwUw4ui6ohF7urKhx2zW9p/2Vm Tcug==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=cJJ1ieRB; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com; dara=fail header.i=@gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 4fb4d7f45d1cf-5c24c55b9aasi755959a12.307.2024.08.31.09.50.11; Sat, 31 Aug 2024 09:50:12 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=cJJ1ieRB; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com; dara=fail header.i=@gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 2B60468DE53; Sat, 31 Aug 2024 19:31:10 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pf1-f179.google.com (mail-pf1-f179.google.com [209.85.210.179]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 742E868DE17 for ; Sat, 31 Aug 2024 19:31:05 +0300 (EEST) Received: by mail-pf1-f179.google.com with SMTP id d2e1a72fcca58-71456acebe8so2277214b3a.3 for ; Sat, 31 Aug 2024 09:31:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1725121863; x=1725726663; darn=ffmpeg.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=Jl38kWUkhUfCSKMPdHpb5PYGCpv8S6uV0HOktyI9OOE=; b=cJJ1ieRBwrXkrCTRVX1NXei9IsX8o2NDr4V/UfLIjbrDME9HC82cCaKbN9PIucl5HE lzJYuHNZGoizAj1TDVrWAeQoiPICJjuBK8cZDeYQ2K8fS1zbx7MfJexdS4es7fNyyZ7/ TXS7awZFpVIr2fqNnaM57FKJzShbcTr4tUzgzOmfC56ObemUJibqaZhrFhqR+BbJJfdY fraFom7cZDhXpQP94k3J5G+EzKngY3NKleFAA0OezoRrhLLxlve342UbQb63SwJER1cL 89redNhOSn7CxReKxr2VgLCgaE4rmsIkDGrdLORsgrtCbE0Izdt1+OauR8zEuh/TIqG6 8WkQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725121863; x=1725726663; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Jl38kWUkhUfCSKMPdHpb5PYGCpv8S6uV0HOktyI9OOE=; b=XS9LlmNilNsYgeck2F96xq8OHzLknozSw7xWLntRytswIkwHFm+gEIhsEI3j6NeQAq ABaCaP6xEYR4X1OxAOTCrOiXtz73bRopTnScyzxw8f13OTCLYVyWodMvWwAcAm3spMZv tImfxKk56FIJ6USSnJxQekqfFmLBjMwitHaT7nbejAhjjVXw5WZfVWXm12itGjNGnwRf d0moDLNGg1M231BKbCuwsOR46NTvNx05uIEXvf7WlxV6OG//p9OEy5Zbhdn7Pbn4f4NC E5BCMBqaRX1GEZrAQ9koCxg6+yq2lZIi7eoJRIOt+RnY3G1vWBcW8W2dgJ3cdK6oEXvy 43/Q== X-Gm-Message-State: AOJu0Yx8kJNY5VmUcpZsCwwBl5BcKbNITw7axRj9W/dWpiRMs5eaHAXu 5fPWg4BZwR78ttm1H4e3TrOrH6HdTru0Xn7bYVIbLWa5avSSiZnCDsTGVw== X-Received: by 2002:a05:6a20:4386:b0:1c4:214c:cc0d with SMTP id adf61e73a8af0-1cce10ab1ccmr8755378637.36.1725121862612; Sat, 31 Aug 2024 09:31:02 -0700 (PDT) Received: from localhost.localdomain ([190.194.167.233]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2d85b3b9522sm5935328a91.50.2024.08.31.09.31.01 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 31 Aug 2024 09:31:02 -0700 (PDT) From: James Almer To: ffmpeg-devel@ffmpeg.org Date: Sat, 31 Aug 2024 13:31:12 -0300 Message-ID: <20240831163114.4197-11-jamrial@gmail.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240831163114.4197-1-jamrial@gmail.com> References: <20240831163114.4197-1-jamrial@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 11/13] avcodec/packet: add a LCEVC packet side data type X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: pMhjLf3l8nw8 Signed-off-by: James Almer --- libavcodec/decode.c | 1 + libavcodec/packet.c | 1 + libavcodec/packet.h | 5 +++++ 3 files changed, 7 insertions(+) diff --git a/libavcodec/decode.c b/libavcodec/decode.c index 033c4913c0..7303a08dc1 100644 --- a/libavcodec/decode.c +++ b/libavcodec/decode.c @@ -1435,6 +1435,7 @@ int ff_decode_frame_props_from_pkt(const AVCodecContext *avctx, { AV_PKT_DATA_DYNAMIC_HDR10_PLUS, AV_FRAME_DATA_DYNAMIC_HDR_PLUS }, { AV_PKT_DATA_S12M_TIMECODE, AV_FRAME_DATA_S12M_TIMECODE }, { AV_PKT_DATA_SKIP_SAMPLES, AV_FRAME_DATA_SKIP_SAMPLES }, + { AV_PKT_DATA_LCEVC, AV_FRAME_DATA_LCEVC }, }; frame->pts = pkt->pts; diff --git a/libavcodec/packet.c b/libavcodec/packet.c index 032f270777..381001fd65 100644 --- a/libavcodec/packet.c +++ b/libavcodec/packet.c @@ -306,6 +306,7 @@ const char *av_packet_side_data_name(enum AVPacketSideDataType type) case AV_PKT_DATA_IAMF_DEMIXING_INFO_PARAM: return "IAMF Demixing Info Parameter Data"; case AV_PKT_DATA_IAMF_RECON_GAIN_INFO_PARAM: return "IAMF Recon Gain Info Parameter Data"; case AV_PKT_DATA_FRAME_CROPPING: return "Frame Cropping"; + case AV_PKT_DATA_LCEVC: return "LCEVC NAL data"; } return NULL; } diff --git a/libavcodec/packet.h b/libavcodec/packet.h index 13667ffa36..6e9a301884 100644 --- a/libavcodec/packet.h +++ b/libavcodec/packet.h @@ -339,6 +339,11 @@ enum AVPacketSideDataType { */ AV_PKT_DATA_FRAME_CROPPING, + /** + * LCEVC raw NAL data. + */ + AV_PKT_DATA_LCEVC, + /** * The number of side data types. * This is not part of the public API/ABI in the sense that it may From patchwork Sat Aug 31 16:31:13 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Almer X-Patchwork-Id: 51256 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:612c:4027:b0:48e:c0f8:d0de with SMTP id ky39csp1020762vqb; Sat, 31 Aug 2024 09:40:12 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCUJXOH7Cf5KjbHHEfvjyp76a4046qe/v+H81gGdLgECnk/PFz9yCtoDnKbH72x+clux7HKY92qJqFu6+kEKLy5d@gmail.com X-Google-Smtp-Source: AGHT+IFhe19ZkWwb39+s1XcqhULcmhvN+nxXE04i5pXCn7JqEVxYdQUvuhl4GHeCWGgcovY4p9zU X-Received: by 2002:a05:651c:1991:b0:2f5:10c5:9111 with SMTP id 38308e7fff4ca-2f61e0aaa72mr22355721fa.8.1725122411724; Sat, 31 Aug 2024 09:40:11 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1725122411; cv=none; d=google.com; s=arc-20240605; b=J0oLhMSOXMCZdN6Ru5jLZ7UP1kASfXUGQnD4PAxESs94nQkyOIaV5aeAEAZUQj/keQ Va0IsJbWbd3To9x/yDoOcFQJI8tPj3EcmDxiyxE8yu1fEstKiv6xufmV8xxOoBIDdCKa BNkY1NXlObl6ixPly86MnKA/+xq6N9MX54kJcTdTA4m6I4DWvrimFLLKOPUeBBuna2eg 4FB2ks7SfPCUWXTo3l7iq+fW1owFWz0kvtwryP3TamlDvIvjY1uW8WfF/CMVRhnu15tv aW7p9EYuantY6CxN6QOJ2qXLu0D4U4fQ1nen6uZJ5tmz5F2ToycbXicyEkqHPip9xxe2 xkmw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:dkim-signature:delivered-to; bh=broJgnM8Nbdiv+MEDqH2cWgXndW8dWWY1Fh4ejEsnE8=; fh=YOA8vD9MJZuwZ71F/05pj6KdCjf6jQRmzLS+CATXUQk=; b=ZY5rt4vR9DP9x0SMzAR1YtK6ZfZRIzEofPF3g2SY7WLPyV3S1EJkmOChafVXxtjLR/ D/C5dlMLAuO+JcmLMv4q4874licRb+9XihfAnHMG+XXKGhCe9m4zSos0HLNWDVY35Yq8 g+BV3nHJp1GS9UFXP6B0l0isdlyhm9nM2ucRGN9WWyLGH3h29QLirMvRpzMOKuGFpCq5 YhlMAkEej2tmsFVhu56NUAtokRoPR9TrytMNhGsJ7fFVQWYhEY7S16yYqyXJiHN3khrU GQKu5jeTC4267JzxfhQCnti5tk2TM1Jv1RiuxlL7fflw4bSiM8hXGgiaRuJrVzRKrEAd DZKA==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=TgxzmUPl; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com; dara=fail header.i=@gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 4fb4d7f45d1cf-5c226d6d241si4840579a12.631.2024.08.31.09.40.10; Sat, 31 Aug 2024 09:40:11 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=TgxzmUPl; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com; dara=fail header.i=@gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 2DF6B68DE5E; Sat, 31 Aug 2024 19:31:11 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pj1-f47.google.com (mail-pj1-f47.google.com [209.85.216.47]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 5EC5568DDEA for ; Sat, 31 Aug 2024 19:31:07 +0300 (EEST) Received: by mail-pj1-f47.google.com with SMTP id 98e67ed59e1d1-2d3b5f2f621so2072955a91.1 for ; Sat, 31 Aug 2024 09:31:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1725121865; x=1725726665; darn=ffmpeg.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=4RStph+HDRefuNP4WS4Gtca82902/H+C8j9HlpzwFZw=; b=TgxzmUPlRpVtwxFQIPc0t5R0KmnnID9r56HfFLGljFQBthczqk2rssR3sGw5e0YzZ6 5VpRaTS2Wsh3MyxYYvBuFHiCrL5tILIXVeP7tEkNA8hJP/pfz0nafNeNfajGUcS8X3gg Gh7KrefB9yhuRBBeZgVWAz317niYfbxB2VsTyRzsDUbW6TPzo2bsHa5+tmYnzRPZes/4 1AtL6VF/W/z77+laJe1DtnV+OCxt1Xi2mAEcN8JsZNhm0hgiKITyees5KRxGoUcCdJFh lTzngdWQPfcIZcEeXNlfLSxFdAuvYuGyMzlX+Tzf/RimUAOpygRtPK9rO5OPRuVhoehz pq/Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725121865; x=1725726665; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=4RStph+HDRefuNP4WS4Gtca82902/H+C8j9HlpzwFZw=; b=CUHiP6fkq+QzAXzjSMf2TXw3K7gcr8hGGhP6AlhMswvYgCFunQiTvQHA0Xxm2yF6y8 90pZpa5u9pD09BWIl8oHz+L8LEBEAHsscgghL2bh6F+jZBibp7jxfINVNpqEmDHlUpCs XVw0/Orh6clFk+LX/xF6O0xS4OEehvdxVaK3a3QmFgatxeKbUeG5GEi/J43Q3h6sv5LC WoziMJSl4NfQCrQCapWKa1RRQ+4td8Kz+B1h6pe10zDoN0mMNGf88PgakIltSdqd5C71 Z7ZkC1LiWJWSSyeAypL/lrUHoKLHn6k+5UtUwJqBx+84xU52tiiLL9eHT1IY5CDbQMkY hTQg== X-Gm-Message-State: AOJu0YylRzVdBqQo/7cM+x8jeyK7J1on998kxDrupa9jsft1jxOkKUp3 kpldnwT3S2ZuFGrfK0KhnEV/mc9olLY22lriL6gpserXJ1SyUUf4crF8ug== X-Received: by 2002:a17:90a:fe18:b0:2cd:1e60:9c31 with SMTP id 98e67ed59e1d1-2d894453fd1mr2065079a91.30.1725121864466; Sat, 31 Aug 2024 09:31:04 -0700 (PDT) Received: from localhost.localdomain ([190.194.167.233]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2d85b3b9522sm5935328a91.50.2024.08.31.09.31.02 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 31 Aug 2024 09:31:03 -0700 (PDT) From: James Almer To: ffmpeg-devel@ffmpeg.org Date: Sat, 31 Aug 2024 13:31:13 -0300 Message-ID: <20240831163114.4197-12-jamrial@gmail.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240831163114.4197-1-jamrial@gmail.com> References: <20240831163114.4197-1-jamrial@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 12/13] avcodec: add an LCEVC merger bsf X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: ecloHbC5BIhR Signed-off-by: James Almer --- libavcodec/bitstream_filters.c | 1 + libavcodec/bsf/Makefile | 3 +- libavcodec/bsf/lcevc_merge_bsf.c | 265 +++++++++++++++++++++++++++++++ 3 files changed, 268 insertions(+), 1 deletion(-) create mode 100644 libavcodec/bsf/lcevc_merge_bsf.c diff --git a/libavcodec/bitstream_filters.c b/libavcodec/bitstream_filters.c index f923411bee..fdd4fcf01b 100644 --- a/libavcodec/bitstream_filters.c +++ b/libavcodec/bitstream_filters.c @@ -45,6 +45,7 @@ extern const FFBitStreamFilter ff_hapqa_extract_bsf; extern const FFBitStreamFilter ff_hevc_metadata_bsf; extern const FFBitStreamFilter ff_hevc_mp4toannexb_bsf; extern const FFBitStreamFilter ff_imx_dump_header_bsf; +extern const FFBitStreamFilter ff_lcevc_merge_bsf; extern const FFBitStreamFilter ff_media100_to_mjpegb_bsf; extern const FFBitStreamFilter ff_mjpeg2jpeg_bsf; extern const FFBitStreamFilter ff_mjpega_dump_header_bsf; diff --git a/libavcodec/bsf/Makefile b/libavcodec/bsf/Makefile index 40b7fc6e9b..850bcaf377 100644 --- a/libavcodec/bsf/Makefile +++ b/libavcodec/bsf/Makefile @@ -1,4 +1,4 @@ -clean:: +\clean:: $(RM) $(CLEANSUFFIXES:%=libavcodec/bsf/%) OBJS-$(CONFIG_AAC_ADTSTOASC_BSF) += bsf/aac_adtstoasc.o @@ -22,6 +22,7 @@ OBJS-$(CONFIG_HEVC_METADATA_BSF) += bsf/h265_metadata.o OBJS-$(CONFIG_DOVI_RPU_BSF) += bsf/dovi_rpu.o OBJS-$(CONFIG_HEVC_MP4TOANNEXB_BSF) += bsf/hevc_mp4toannexb.o OBJS-$(CONFIG_IMX_DUMP_HEADER_BSF) += bsf/imx_dump_header.o +OBJS-$(CONFIG_LCEVC_MERGE_BSF) += bsf/lcevc_merge_bsf.o OBJS-$(CONFIG_MEDIA100_TO_MJPEGB_BSF) += bsf/media100_to_mjpegb.o OBJS-$(CONFIG_MJPEG2JPEG_BSF) += bsf/mjpeg2jpeg.o OBJS-$(CONFIG_MJPEGA_DUMP_HEADER_BSF) += bsf/mjpega_dump_header.o diff --git a/libavcodec/bsf/lcevc_merge_bsf.c b/libavcodec/bsf/lcevc_merge_bsf.c new file mode 100644 index 0000000000..fcaeb7ef7f --- /dev/null +++ b/libavcodec/bsf/lcevc_merge_bsf.c @@ -0,0 +1,265 @@ +/* + * Copyright (c) 2022 James Almer + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * @file + * Derive PTS by reordering DTS from supported streams + */ + +#include "libavutil/avassert.h" +#include "libavutil/attributes.h" +#include "libavutil/fifo.h" +#include "libavutil/mem.h" +#include "libavutil/opt.h" +#include "libavutil/tree.h" + +#include "bsf.h" +#include "bsf_internal.h" + +typedef struct LCEVCMergeContext { + const AVClass *class; + + struct AVTreeNode *base; + struct AVTreeNode *enhancement; + AVFifo *nodes; + + int base_idx, enhancement_idx; +} LCEVCMergeContext; + +// AVTreeNode callbacks +static int cmp_insert(const void *_key, const void *_node) +{ + const AVPacket *key = _key, *node = _node; + + return FFDIFFSIGN(key->pts, node->pts); +} + +static int cmp_find(const void *_key, const void *_node) +{ + int64_t key = *(const int64_t *)_key; + const AVPacket *node = _node; + + return FFDIFFSIGN(key, node->pts); +} + +#define WARN_BUFFERED(type) \ +static int warn_##type##_buffered(void *logctx, void *elem) \ +{ \ + const AVPacket *pkt = (const AVPacket *)elem; \ + av_log(logctx, AV_LOG_WARNING, #type" packet with PTS %"PRId64 \ + " left buffered at EOF\n", pkt->pts); \ + return 0; \ +} + +WARN_BUFFERED(base) +WARN_BUFFERED(enhanced) + +static int lcevc_merge_init(AVBSFContext *ctx) +{ + LCEVCMergeContext *s = ctx->priv_data; + + if (s->base_idx < 0 || s->enhancement_idx < 0) { + av_log(ctx, AV_LOG_ERROR, "Both base and enhancement stream index must be set\n"); + return AVERROR(EINVAL); + } + + s->nodes = av_fifo_alloc2(1, sizeof(struct AVTreeNode*), AV_FIFO_FLAG_AUTO_GROW); + if (!s->nodes) + return AVERROR(ENOMEM); + + return 0; +} + +static int merge_packet(AVBSFContext *ctx, struct AVTreeNode **root, + AVPacket *out, AVPacket *in) +{ + LCEVCMergeContext *s = ctx->priv_data; + struct AVTreeNode *node = NULL; + uint8_t *side_data; + int ret; + + // It doesn't matter if the packet is from the base or enhancement stream + // as both share the pts cmp_insert() will look for to remove the element. + av_tree_insert(root, in, cmp_insert, &node); + memset(node, 0, av_tree_node_size); + ret = av_fifo_write(s->nodes, &node, 1); + if (ret < 0) { + av_free(node); + return ret; + } + + side_data = av_packet_new_side_data(out, AV_PKT_DATA_LCEVC, in->size); + if (!side_data) + return AVERROR(ENOMEM); + + memcpy(side_data, in->data, in->size); + + return 0; +} + +static int buffer_packet(AVBSFContext *ctx, struct AVTreeNode **root, + AVPacket **p_in) +{ + LCEVCMergeContext *s = ctx->priv_data; + AVPacket *in = *p_in, *pkt; + struct AVTreeNode *node = NULL; + + if (av_fifo_can_read(s->nodes)) { + int av_unused ret = av_fifo_read(s->nodes, &node, 1); + av_assert2(ret >= 0); + } else + node = av_tree_node_alloc(); + if (!node) + return AVERROR(ENOMEM); + + pkt = av_tree_insert(root, in, cmp_insert, &node); + if (pkt && pkt != in) { + av_log(ctx, AV_LOG_ERROR, "Duplicate packet with PTS %"PRId64 + " for stream_index %d \n", in->pts, in->stream_index); + av_free(node); + return AVERROR_INVALIDDATA; + } + *p_in = NULL; + + return 0; +} + +#define HANDLE_PACKET(type1, type2, pkt1, pkt2) \ +static int handle_##type1##_packet(AVBSFContext *ctx, AVPacket *out, AVPacket **p_in) \ +{ \ + LCEVCMergeContext *s = ctx->priv_data; \ + AVPacket *in = *p_in, *pkt; \ + int ret; \ + \ + pkt = av_tree_find(s->type2, &in->pts, cmp_find, NULL); \ + if (pkt) { \ + ret = merge_packet(ctx, &s->type2, pkt1, pkt2); \ + if (!ret) \ + av_packet_move_ref(out, pkt1); \ + av_packet_free(&pkt); \ + av_packet_free(p_in); \ + return ret; \ + } \ + \ + return buffer_packet(ctx, &s->type1, p_in); \ +} + +HANDLE_PACKET(base, enhancement, in, pkt) +HANDLE_PACKET(enhancement, base, pkt, in) + +static int lcevc_merge_filter(AVBSFContext *ctx, AVPacket *out) +{ + LCEVCMergeContext *s = ctx->priv_data; + AVPacket *in; + int ret; + + do { + ret = ff_bsf_get_packet(ctx, &in); + if (ret < 0) { + if (ret == AVERROR_EOF) { + av_tree_enumerate(s->base, ctx, NULL, warn_base_buffered); + av_tree_enumerate(s->enhancement, ctx, NULL, warn_enhanced_buffered); + } + return ret; + } + + if (!in->size || in->pts < 0) { + ret = AVERROR_INVALIDDATA; + goto fail; + } + + if (in->stream_index == s->base_idx) + ret = handle_base_packet(ctx, out, &in); + else if (in->stream_index == s->enhancement_idx) + ret = handle_enhancement_packet(ctx, out, &in); + else { + av_log(ctx, AV_LOG_ERROR, "Input packet neither base or enhacement\n"); + ret = AVERROR(EINVAL); + } + if (ret < 0) + goto fail; + } while (!out->data); + + ret = 0; +fail: + if (ret < 0) + av_packet_free(&in); + + return ret; +} + +static int free_node(void *opaque, void *elem) +{ + AVPacket *pkt = elem; + av_packet_free(&pkt); + return 0; +} + +static void lcevc_merge_flush(AVBSFContext *ctx) +{ + LCEVCMergeContext *s = ctx->priv_data; + + av_tree_enumerate(s->base, NULL, NULL, free_node); + av_tree_enumerate(s->enhancement, NULL, NULL, free_node); + av_tree_destroy(s->base); + av_tree_destroy(s->enhancement); + s->base = NULL; + s->enhancement = NULL; +} + +static void lcevc_merge_close(AVBSFContext *ctx) +{ + LCEVCMergeContext *s = ctx->priv_data; + + lcevc_merge_flush(ctx); + + if (s->nodes) { + struct AVTreeNode *node; + while (av_fifo_read(s->nodes, &node, 1) >= 0) + av_free(node); + } + + av_fifo_freep2(&s->nodes); +} + +#define OFFSET(x) offsetof(LCEVCMergeContext, x) +#define FLAGS (AV_OPT_FLAG_VIDEO_PARAM|AV_OPT_FLAG_BSF_PARAM) +static const AVOption lcevc_merge_options[] = { + { "base_idx", NULL, OFFSET(base_idx), AV_OPT_TYPE_INT, { .i64 = -1 }, -1, INT_MAX, FLAGS }, + { "enhancement_idx", NULL, OFFSET(enhancement_idx), AV_OPT_TYPE_INT, { .i64 = -1 }, -1, INT_MAX, FLAGS }, + { NULL } +}; + +static const AVClass lcevc_merge_class = { + .class_name = "lcevc_merge_bsf", + .item_name = av_default_item_name, + .option = lcevc_merge_options, + .version = LIBAVUTIL_VERSION_INT, +}; + +const FFBitStreamFilter ff_lcevc_merge_bsf = { + .p.name = "lcevc_merge", + .p.priv_class = &lcevc_merge_class, + .priv_data_size = sizeof(LCEVCMergeContext), + .init = lcevc_merge_init, + .flush = lcevc_merge_flush, + .close = lcevc_merge_close, + .filter = lcevc_merge_filter, +}; From patchwork Sat Aug 31 16:31:14 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Almer X-Patchwork-Id: 51258 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:612c:4027:b0:48e:c0f8:d0de with SMTP id ky39csp1023863vqb; Sat, 31 Aug 2024 09:50:11 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCXHJg+Wp9IxtXNPPMlI9tbZv9lDAaz+008WC1bhZ6255ALTt6OtDvsjGBQSoZmcvMV6K6eKEY8Uzmipr1VZKdHh@gmail.com X-Google-Smtp-Source: AGHT+IHfXRwaAOBvODmXTM/6f+uewE4XnNdRlOUNAEYf10F5W2xRnx2OaFUeGnaZqWYYU8fzOyzA X-Received: by 2002:a05:651c:b23:b0:2ef:29fc:f950 with SMTP id 38308e7fff4ca-2f61e0ac3b3mr22335511fa.6.1725123011542; Sat, 31 Aug 2024 09:50:11 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1725123011; cv=none; d=google.com; s=arc-20240605; b=VdgrNoHr3SuhgnnKNHfp78wKnfnVblKHHyfc44kQVrfUv+A0C6uc+iQDK8QmQSAJjv Y35R2K4JNA1WyiDYVOsDNIvGeXfxk4DU6nDmNf/41Bmhpu2gQ/ErqgHAV+eFFYZGNY8N 2YDWPElroe4gTIs3mi+Yi4WHvuhyCVGN41YGLXsDTqf4SqkuGckgtez5oNLX/I5yS6KL lgtmjCLXpDyTb9vkC5nbFkb9UOHv25SjEq0P8r0TAYpDowT8/vaU1Tp+UGVf/mUAZYug 5/3i5on0GJypqCgQ/VMXJZwjqLJFIlzRBOBsk3UghoATKNKsXJ0LpdmjSFyteO6sSHnr jkcw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:dkim-signature:delivered-to; bh=oWHFnp/76ArcOpHwOaaO4OrbMYTfPH1PFHdN9ip5Yws=; fh=YOA8vD9MJZuwZ71F/05pj6KdCjf6jQRmzLS+CATXUQk=; b=jm8IB2HDBlY6g4Vh0gyiG9O40JQYUS3cdaqLr3YJg1Kt+T0dthvUqltgM3JfX70hD9 HtJa1Y032CIHFsAcR7WtdBsnsVKnWHfgNqTK3bHitjoNdkt42QbpzpymAckEeqS8yw+t dEpNX7jNvDgbmavqeKdJVsL5ZRQSqYgnuGefO+IrkdGyBXj7f6PvckcKg04nprk5EaNH dcDc8qS95xn/3blXp73WPwaTyGeEtMCdIulsbJscY6BzGu+y6SkQwhvJE71H5kdk7Yl3 GMZe+njb/rrLjX15GGKPbdCisvHpwPblqa9pLDkNf7AH0sZBUgoNz7pc766vJFpOxElx O9Kg==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=T5tlusei; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com; dara=fail header.i=@gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 38308e7fff4ca-2f614e95ac1si19805931fa.42.2024.08.31.09.50.11; Sat, 31 Aug 2024 09:50:11 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=T5tlusei; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com; dara=fail header.i=@gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 2592968DE6B; Sat, 31 Aug 2024 19:31:12 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pg1-f172.google.com (mail-pg1-f172.google.com [209.85.215.172]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 1356B68DE33 for ; Sat, 31 Aug 2024 19:31:08 +0300 (EEST) Received: by mail-pg1-f172.google.com with SMTP id 41be03b00d2f7-7c1324be8easo2603917a12.1 for ; Sat, 31 Aug 2024 09:31:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1725121867; x=1725726667; darn=ffmpeg.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=keaJrEh5WciReZS8FrHT9RmBTWd8fkf8W86gcrr+NQw=; b=T5tlusei1AnH9BadTJTnjlJuRGv2VOW7rONbd4/ioF79xedl1GnhYDo/ZkwE9UDBy9 E0WNnAC6rWvb1E9WKroBeqaojiwv//D8/JWre+OihronOfdYTW+wo+eQEifa5Y7BXcUh Vo2EB39rDZL3D0MOqq3l5MB7HrVO639obGkpDFITaqbJxbx2JQsag1hx3YvFttoJi03s rDbkPh+56sqIv2Sorsq5IEVpSSmiiPLULKC1+FHkd1t9KlO351SaoeJx2oqtxBgdkyAM FqaSR4mOqOsjjdiM29JxMsdStqXUgZOSP63ScE2P9McNHURScMjtxua1flP3dqsKnEe/ LXqA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725121867; x=1725726667; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=keaJrEh5WciReZS8FrHT9RmBTWd8fkf8W86gcrr+NQw=; b=YAiUGd3Q8MNifFGu88m4RAMFHjw0ldAV/WJe67BDR/V4RzbTqv55Jcjt1JLrxNa5vh /q5QjWgyru1ULCV4yLNWkLW8NxZysP/D2EN27zT2OMwtMRPQ10mkfrVkDKChh7NSIOd+ CyPCPnxNtpbDA7XmewmJ3i9GpcwIE5wvLWJXyWDL8oCAJFzYVVNzJhy61eInsIBmCVPU tVM0PnBhVvmrzUk9aroyrE265ka3pJS5AclOPM6G3IJuLkK/PMzdtOR0S7kuuZioXqZ1 ItRrOT4baNmMtDV412TdoOctFOI7dGj4GeM1LkVaCL4Gpm0lCbL3LmjOYmtGPvp1o6Ob hO8Q== X-Gm-Message-State: AOJu0YwApNMHgNYfweFhIE0EelE8y6d5kc8feNHgNx2re/xWIAvxzxCh Et/x7OTBAJdrXtg2IvJXNiWj8UiC0QfKCMlJksp4U+vlyYJEdOWrfqKgPQ== X-Received: by 2002:a17:90b:2750:b0:2c9:7343:71f1 with SMTP id 98e67ed59e1d1-2d86b00ece4mr8999206a91.14.1725121866342; Sat, 31 Aug 2024 09:31:06 -0700 (PDT) Received: from localhost.localdomain ([190.194.167.233]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2d85b3b9522sm5935328a91.50.2024.08.31.09.31.04 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 31 Aug 2024 09:31:05 -0700 (PDT) From: James Almer To: ffmpeg-devel@ffmpeg.org Date: Sat, 31 Aug 2024 13:31:14 -0300 Message-ID: <20240831163114.4197-13-jamrial@gmail.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240831163114.4197-1-jamrial@gmail.com> References: <20240831163114.4197-1-jamrial@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 13/13] fftools/ffmpeg_demux: merge streams in a LCEVC stream group X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: A2BGWNe2Vt0K Add the LCEVC data stream payloads as packet side data to the main video stream, ensuring the former is always output by the demuxer even if not used by the process. Signed-off-by: James Almer --- configure | 2 +- fftools/ffmpeg.h | 17 +++ fftools/ffmpeg_demux.c | 307 ++++++++++++++++++++++++++++++++++++----- 3 files changed, 292 insertions(+), 34 deletions(-) diff --git a/configure b/configure index 3b7cf05bb5..3af3654483 100755 --- a/configure +++ b/configure @@ -4044,7 +4044,7 @@ ffmpeg_deps="avcodec avfilter avformat threads" ffmpeg_select="aformat_filter anull_filter atrim_filter format_filter hflip_filter null_filter transpose_filter trim_filter vflip_filter" -ffmpeg_suggest="ole32 psapi shell32" +ffmpeg_suggest="ole32 psapi shell32 lcevc_merge_bsf" ffplay_deps="avcodec avformat avfilter swscale swresample sdl2" ffplay_select="crop_filter transpose_filter hflip_filter vflip_filter rotate_filter" ffplay_suggest="shell32 libplacebo vulkan" diff --git a/fftools/ffmpeg.h b/fftools/ffmpeg.h index 3c5d933e17..a9fb55fb6e 100644 --- a/fftools/ffmpeg.h +++ b/fftools/ffmpeg.h @@ -440,6 +440,17 @@ typedef struct InputStream { int nb_outputs; } InputStream; +typedef struct InputStreamGroup { + const AVClass *class; + + /* parent source */ + struct InputFile *file; + + int index; + + AVStreamGroup *stg; +} InputStreamGroup; + typedef struct InputFile { const AVClass *class; @@ -461,6 +472,12 @@ typedef struct InputFile { * if new streams appear dynamically during demuxing */ InputStream **streams; int nb_streams; + + /** + * stream groups that ffmpeg is aware of + */ + InputStreamGroup **stream_groups; + int nb_stream_groups; } InputFile; enum forced_keyframes_const { diff --git a/fftools/ffmpeg_demux.c b/fftools/ffmpeg_demux.c index 039ee0c785..eb7cc96f4c 100644 --- a/fftools/ffmpeg_demux.c +++ b/fftools/ffmpeg_demux.c @@ -90,12 +90,29 @@ typedef struct DemuxStream { AVBSFContext *bsf; + InputStreamGroup **stream_groups; + int nb_stream_groups; + /* number of packets successfully read for this stream */ uint64_t nb_packets; // combined size of all the packets read uint64_t data_size; } DemuxStream; +typedef struct DemuxStreamGroup { + InputStreamGroup istg; + + // main stream for merged output + InputStream *stream; + + // name used for logging + char log_name[32]; + + int discard; + + AVBSFContext *bsf; +} DemuxStreamGroup; + typedef struct Demuxer { InputFile f; @@ -142,6 +159,7 @@ typedef struct DemuxThreadContext { AVPacket *pkt_demux; // packet for reading from BSFs AVPacket *pkt_bsf; + AVPacket *pkt_group_bsf; } DemuxThreadContext; static DemuxStream *ds_from_ist(InputStream *ist) @@ -149,6 +167,11 @@ static DemuxStream *ds_from_ist(InputStream *ist) return (DemuxStream*)ist; } +static DemuxStreamGroup *dsg_from_istg(InputStreamGroup *istg) +{ + return (DemuxStreamGroup*)istg; +} + static Demuxer *demuxer_from_ifile(InputFile *f) { return (Demuxer*)f; @@ -537,17 +560,69 @@ static int do_send(Demuxer *d, DemuxStream *ds, AVPacket *pkt, unsigned flags, return 0; } +static int demux_filter(Demuxer *d, DemuxThreadContext *dt, DemuxStream *ds, + AVBSFContext *bsf, AVPacket *pkt, void *logctx) +{ + int ret; + + if (pkt) + av_packet_rescale_ts(pkt, pkt->time_base, bsf->time_base_in); + + ret = av_bsf_send_packet(bsf, pkt); + if (ret < 0) { + if (pkt) + av_packet_unref(pkt); + av_log(logctx, AV_LOG_ERROR, "Error submitting a packet for filtering: %s\n", + av_err2str(ret)); + return ret; + } + + while (1) { + ret = av_bsf_receive_packet(bsf, dt->pkt_bsf); + if (ret == AVERROR(EAGAIN)) + return 0; + else if (ret < 0) { + if (ret != AVERROR_EOF) + av_log(logctx, AV_LOG_ERROR, + "Error applying bitstream filters to a packet: %s\n", + av_err2str(ret)); + break; + } + + dt->pkt_bsf->time_base = bsf->time_base_out; + + ret = do_send(d, ds, dt->pkt_bsf, 0, "filtered"); + if (ret < 0) { + av_packet_unref(dt->pkt_bsf); + break; + } + } + + return ret; +} + static int demux_send(Demuxer *d, DemuxThreadContext *dt, DemuxStream *ds, AVPacket *pkt, unsigned flags) { InputFile *f = &d->f; - int ret; + int ret = 0; // pkt can be NULL only when flushing BSFs av_assert0(ds->bsf || pkt); + // a stream can only be disabled if it's needed by a group + av_assert0(ds->nb_stream_groups || !ds->discard); + + // create a reference for the packet to be filtered by group bsfs + if (pkt && ds->nb_stream_groups) { + av_packet_unref(dt->pkt_group_bsf); + ret = av_packet_ref(dt->pkt_group_bsf, pkt); + if (ret < 0) + return ret; + } + // send heartbeat for sub2video streams - if (d->pkt_heartbeat && pkt && pkt->pts != AV_NOPTS_VALUE) { + if (d->pkt_heartbeat && pkt && !ds->discard && pkt->pts != AV_NOPTS_VALUE) { for (int i = 0; i < f->nb_streams; i++) { DemuxStream *ds1 = ds_from_ist(f->streams[i]); @@ -564,39 +639,30 @@ static int demux_send(Demuxer *d, DemuxThreadContext *dt, DemuxStream *ds, } } - if (ds->bsf) { - if (pkt) - av_packet_rescale_ts(pkt, pkt->time_base, ds->bsf->time_base_in); + for (int i = 0; i < ds->nb_stream_groups; i++) { + DemuxStreamGroup *dsg = dsg_from_istg(ds->stream_groups[i]); - ret = av_bsf_send_packet(ds->bsf, pkt); - if (ret < 0) { - if (pkt) - av_packet_unref(pkt); - av_log(ds, AV_LOG_ERROR, "Error submitting a packet for filtering: %s\n", - av_err2str(ret)); + // if the main stream is disabled, we don't want to filter + if (ds == ds_from_ist(dsg->stream) && ds->discard) + continue; + + ret = demux_filter(d, dt, ds_from_ist(dsg->stream), dsg->bsf, + pkt ? dt->pkt_group_bsf : NULL, dsg); + if (ret < 0) return ret; - } - while (1) { - ret = av_bsf_receive_packet(ds->bsf, dt->pkt_bsf); - if (ret == AVERROR(EAGAIN)) - return 0; - else if (ret < 0) { - if (ret != AVERROR_EOF) - av_log(ds, AV_LOG_ERROR, - "Error applying bitstream filters to a packet: %s\n", - av_err2str(ret)); - return ret; - } + // TODO handle streams belonging to more than one Stream group + if (i == (ds->nb_stream_groups - 1) && ds == ds_from_ist(dsg->stream)) + return 0; + } - dt->pkt_bsf->time_base = ds->bsf->time_base_out; + if (ds->discard) + return 0; - ret = do_send(d, ds, dt->pkt_bsf, 0, "filtered"); - if (ret < 0) { - av_packet_unref(dt->pkt_bsf); - return ret; - } - } + if (ds->bsf) { + ret = demux_filter(d, dt, ds, ds->bsf, pkt, ds); + if (ret < 0) + return ret; } else { ret = do_send(d, ds, pkt, flags, "demuxed"); if (ret < 0) @@ -660,6 +726,7 @@ static void demux_thread_uninit(DemuxThreadContext *dt) { av_packet_free(&dt->pkt_demux); av_packet_free(&dt->pkt_bsf); + av_packet_free(&dt->pkt_group_bsf); memset(dt, 0, sizeof(*dt)); } @@ -676,6 +743,10 @@ static int demux_thread_init(DemuxThreadContext *dt) if (!dt->pkt_bsf) return AVERROR(ENOMEM); + dt->pkt_group_bsf = av_packet_alloc(); + if (!dt->pkt_group_bsf) + return AVERROR(ENOMEM); + return 0; } @@ -749,9 +820,16 @@ static int input_thread(void *arg) ds = dt.pkt_demux->stream_index < f->nb_streams ? ds_from_ist(f->streams[dt.pkt_demux->stream_index]) : NULL; if (!ds || ds->discard || ds->finished) { - report_new_stream(d, dt.pkt_demux); - av_packet_unref(dt.pkt_demux); - continue; + int i = 0; + /* Is the stream disabled, but still needed to handle a group? */ + for (; ds && i < ds->nb_stream_groups; i++) + if (!dsg_from_istg(ds->stream_groups[i])->discard) + break; + if (!ds || i == ds->nb_stream_groups) { + report_new_stream(d, dt.pkt_demux); + av_packet_unref(dt.pkt_demux); + continue; + } } if (dt.pkt_demux->flags & AV_PKT_FLAG_CORRUPT) { @@ -849,9 +927,25 @@ static void ist_free(InputStream **pist) av_bsf_free(&ds->bsf); + av_freep(&ds->stream_groups); + av_freep(pist); } +static void istg_free(InputStreamGroup **pistg) +{ + InputStreamGroup *istg = *pistg; + DemuxStreamGroup *dsg; + + if (!istg) + return; + dsg = dsg_from_istg(istg); + + av_bsf_free(&dsg->bsf); + + av_freep(pistg); +} + void ifile_close(InputFile **pf) { InputFile *f = *pf; @@ -866,6 +960,9 @@ void ifile_close(InputFile **pf) for (int i = 0; i < f->nb_streams; i++) ist_free(&f->streams[i]); av_freep(&f->streams); + for (int i = 0; i < f->nb_stream_groups; i++) + istg_free(&f->stream_groups[i]); + av_freep(&f->stream_groups); avformat_close_input(&f->ctx); @@ -961,6 +1058,19 @@ static int ist_use(InputStream *ist, int decoding_needed) d->have_audio_dec |= is_audio; } + // if this stream is the main one in any group, enable said group and + // all its streams, so lavf will return their packets + for (int i = 0; i < ds->nb_stream_groups; i++) { + DemuxStreamGroup *dsg = dsg_from_istg(ds->stream_groups[i]); + AVStreamGroup *stg = ds->stream_groups[i]->stg; + + if (ist != dsg->stream) + continue; + for (int j = 0; j < stg->nb_streams; j++) + stg->streams[j]->discard = 0; + dsg->discard = 0; + } + return 0; } @@ -1524,6 +1634,128 @@ static int ist_add(const OptionsContext *o, Demuxer *d, AVStream *st, AVDictiona return 0; } +static const char *input_stream_group_item_name(void *obj) +{ + const DemuxStreamGroup *dsg = obj; + + return dsg->log_name; +} + +static const AVClass input_stream_group_class = { + .class_name = "InputStreamGroup", + .version = LIBAVUTIL_VERSION_INT, + .item_name = input_stream_group_item_name, + .category = AV_CLASS_CATEGORY_DEMUXER, +}; + +static DemuxStreamGroup *demux_stream_group_alloc(Demuxer *d, AVStreamGroup *stg) +{ + InputFile *f = &d->f; + DemuxStreamGroup *dsg; + + dsg = allocate_array_elem(&f->stream_groups, sizeof(*dsg), &f->nb_stream_groups); + if (!dsg) + return NULL; + + dsg->istg.stg = stg; + dsg->istg.file = f; + dsg->istg.index = stg->index; + dsg->istg.class = &input_stream_group_class; + + snprintf(dsg->log_name, sizeof(dsg->log_name), "istg#%d:%d/%s", + d->f.index, stg->index, avformat_stream_group_name(stg->type)); + + return dsg; +} + +static int istg_add(Demuxer *d, AVStreamGroup *stg) +{ + InputFile *f = &d->f; + DemuxStreamGroup *dsg; + const AVBitStreamFilter *filter; + int base_idx = -1, enhancement_idx = -1; + int ret; + + // TODO: generic handling of groups, once support for more is added + if (stg->type != AV_STREAM_GROUP_PARAMS_LCEVC) + return 0; + + if (stg->nb_streams != 2) + return AVERROR_BUG; + + filter = av_bsf_get_by_name("lcevc_merge"); + if (!filter) + return 0; + + dsg = demux_stream_group_alloc(d, stg); + if (!dsg) + return AVERROR(ENOMEM); + + dsg->discard = 1; + + // set the main stream for the group + for (int i = 0; i < stg->nb_streams; i++) { + int j; + + for (j = 0; j < f->nb_streams; j++) + if (stg->streams[i] == f->streams[j]->st) + break; + + if (j == f->nb_streams) + return AVERROR_BUG; + + if (stg->streams[i]->codecpar->codec_type != AVMEDIA_TYPE_VIDEO) { + if (stg->streams[i]->codecpar->codec_type != AVMEDIA_TYPE_DATA || + enhancement_idx > 0) + return AVERROR_BUG; + enhancement_idx = f->streams[j]->st->index; + continue; + } else if (base_idx > 0) + return AVERROR_BUG; + + dsg->stream = f->streams[j]; + base_idx = f->streams[j]->st->index; + } + + /* since the API lets us know what streams belong to a given group, but + * not what groups a given stream is part of, add a pointer to the + * DemuxStreamGroup to all relevant DemuxStream structs for this purpose */ + for (int i = 0; i < stg->nb_streams; i++) { + DemuxStreamGroup **dsg1; + DemuxStream *ds; + int j; + + for (j = 0; j < f->nb_streams; j++) + if (stg->streams[i] == f->streams[j]->st) + break; + + if (j == f->nb_streams) + return AVERROR_BUG; + + ds = ds_from_ist(f->streams[j]); + dsg1 = av_dynarray2_add((void **)&ds->stream_groups, &ds->nb_stream_groups, sizeof(*dsg1), NULL); + if (!dsg1) + return AVERROR(ENOMEM); + + *dsg1 = dsg; + } + + ret = av_bsf_alloc(filter, &dsg->bsf); + if (ret < 0) + return ret; + + av_opt_set_int(dsg->bsf->priv_data, "base_idx", base_idx, 0); + av_opt_set_int(dsg->bsf->priv_data, "enhancement_idx", enhancement_idx, 0); + + dsg->bsf->time_base_in = stg->streams[0]->time_base; + + ret = av_bsf_init(dsg->bsf); + if (ret < 0) + return ret; + + return 0; +} + static int dump_attachment(InputStream *ist, const char *filename) { AVStream *st = ist->st; @@ -1878,6 +2110,15 @@ int ifile_open(const OptionsContext *o, const char *filename, Scheduler *sch) } } + /* Add all the stream groups from the given input file to the demuxer */ + for (int i = 0; i < ic->nb_stream_groups; i++) { + ret = istg_add(d, ic->stream_groups[i]); + if (ret < 0) { + av_dict_free(&opts_used); + return ret; + } + } + /* dump the file content */ av_dump_format(ic, f->index, filename, 0);