From patchwork Wed Jun 22 06:49:04 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: "Dawid Kozinski/Multimedia \\(PLT\\) /SRPOL/Staff Engineer/Samsung Electronics" X-Patchwork-Id: 36378 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:1a22:b0:84:42e0:ad30 with SMTP id cj34csp2657095pzb; Tue, 21 Jun 2022 23:49:24 -0700 (PDT) X-Google-Smtp-Source: AGRyM1uVlXp/TTKGQJBu2Uv6DJwfwyZxKwS2y/GRpTvd0Z2+7pslwXexu/dKKimg0Tsp+mhOkhFr X-Received: by 2002:a17:907:3d8a:b0:70d:e791:f82d with SMTP id he10-20020a1709073d8a00b0070de791f82dmr1730124ejc.528.1655880563923; Tue, 21 Jun 2022 23:49:23 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1655880563; cv=none; d=google.com; s=arc-20160816; b=vG0kgZ4lit8oTi71PsT4E4r4Lt8BSiNMni9ZIw0jR2jju5da4DXzSkQLC1M6p1NuPc ymC2TealWKLhEUhLI+kdBjFnRDug5f1Z55ldqnOebjMgCKYZWnbnG4OioUjRnBaeON5e 6Ufn/hdIN2uUZY1YBkxyfyEndtqrm8iuetZBZNevKb7uanxdtOR0ePUjzABsFRwOKVwx h49umFWPNjRv/KJsb8+Y8wlPAiLK7JSoGvHFTvIFZtKra14vlqWp0Qa2lPsqg9zG/eVN OW/we4Hw7koyJ4SMMCLKgrRIIh7gnMC5SxvcUDSrY5oynTtP8BtUeFpNKF2C+LqZlJ8A 8olg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:references:cms-type:content-language :thread-index:mime-version:message-id:date:to:from:dkim-signature :dkim-filter:delivered-to; bh=mDROYwWOE1fcNx1N7LXJijHz7M/85FS8ipDJDALN1lU=; b=q5d7BjW+TZVSfeqhRhMf7lzqJYu4GeKaUDtBP/ONjP0LumbXEcR8z2wNY8nwDQAPOV ChAVgJ8lkALcKmlCUzDVRaUWWwqpEV5g7tEmLWaM80dQAFWV15rtqZ+4H5TQtHO8ux1F 20LZzggH7bmTkwTQl98XbXbrfrcHpfctHtGYTkbanmM++gy8JeC1j7Ov61OIr2GJmyRT k4hu47lEUL0G6eocMuEzSWwXfQJElJwy32gJy+84qWuIlcVUpWAHkcAovzODTlVEl85m 33iIyA2EIdx6VC3vcUyZ93qwdoaWmi6KZJpQTsULHU6X/16mkXvlVMdp6SBWgtpA2ao6 FoqA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@samsung.com header.s=mail20170921 header.b="LZO/tGoh"; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=samsung.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id ss28-20020a170907c01c00b00722e85dcd94si2347989ejc.375.2022.06.21.23.49.23; Tue, 21 Jun 2022 23:49:23 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@samsung.com header.s=mail20170921 header.b="LZO/tGoh"; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=samsung.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id C464168B6D3; Wed, 22 Jun 2022 09:49:13 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mailout1.w1.samsung.com (mailout1.w1.samsung.com [210.118.77.11]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id C958068B0CD for ; Wed, 22 Jun 2022 09:49:06 +0300 (EEST) Received: from eucas1p2.samsung.com (unknown [182.198.249.207]) by mailout1.w1.samsung.com (KnoxPortal) with ESMTP id 20220622064905euoutp0124aa9db9ad8805ddf0ef30820be91bd4~63pU4bJTd2816028160euoutp01V for ; Wed, 22 Jun 2022 06:49:05 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout1.w1.samsung.com 20220622064905euoutp0124aa9db9ad8805ddf0ef30820be91bd4~63pU4bJTd2816028160euoutp01V DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1655880545; bh=jZmCsfWZpCF1+rOWYZBgQhwKXriFWnh8UfnnVQ7y+zU=; h=From:To:Subject:Date:References:From; b=LZO/tGohiw/ory5uu5cISrFxfrg/ueDsOGwb1evIzoRuACTbzPgyJaaV38V+/+OG9 SVMiam/3CHBUXG/zn7fzP8WkcHcfJIRrMPIZ2ivxQzg3CkUFn/4MnykHdIjKWZxB6h Vyqnsigxc4fi+B94Qmr+uawUNPrZCVt+K7r1/O0c= Received: from eusmges2new.samsung.com (unknown [203.254.199.244]) by eucas1p1.samsung.com (KnoxPortal) with ESMTP id 20220622064905eucas1p12772a2dce6a9cc9ea22d325628b4829b~63pUmtqbS2963529635eucas1p1y for ; Wed, 22 Jun 2022 06:49:05 +0000 (GMT) Received: from eucas1p2.samsung.com ( [182.198.249.207]) by eusmges2new.samsung.com (EUCPMTA) with SMTP id 69.3E.10067.06BB2B26; Wed, 22 Jun 2022 07:49:05 +0100 (BST) Received: from eusmtrp2.samsung.com (unknown [182.198.249.139]) by eucas1p1.samsung.com (KnoxPortal) with ESMTPA id 20220622064904eucas1p19bbbc922c4f1b734730c48169bada35a~63pULbhq92949329493eucas1p17 for ; Wed, 22 Jun 2022 06:49:04 +0000 (GMT) Received: from eusmgms1.samsung.com (unknown [182.198.249.179]) by eusmtrp2.samsung.com (KnoxPortal) with ESMTP id 20220622064904eusmtrp2008cb688a148f4b45bcd6af2a5bdd022~63pUK1eaV0070700707eusmtrp2i for ; Wed, 22 Jun 2022 06:49:04 +0000 (GMT) X-AuditID: cbfec7f4-dd7ff70000002753-0e-62b2bb6069a8 Received: from eusmtip2.samsung.com ( [203.254.199.222]) by eusmgms1.samsung.com (EUCPMTA) with SMTP id 5B.40.09095.06BB2B26; Wed, 22 Jun 2022 07:49:04 +0100 (BST) Received: from AMDN3260 (unknown [106.210.132.171]) by eusmtip2.samsung.com (KnoxPortal) with ESMTPA id 20220622064904eusmtip29003988b3670c2ab4f59e8a533e8e467~63pTv6SE22990829908eusmtip2F for ; Wed, 22 Jun 2022 06:49:04 +0000 (GMT) From: "Dawid Kozinski" To: Date: Wed, 22 Jun 2022 08:49:04 +0200 Message-ID: <00bb01d88604$29cbc180$7d634480$@samsung.com> MIME-Version: 1.0 X-Mailer: Microsoft Outlook 16.0 Thread-Index: AdiGAjXpq6O0EN1BQtC9UmcGVuTcVA== Content-Language: pl X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrNIsWRmVeSWpSXmKPExsWy7djP87qJuzclGax/zmjx7dMZZgdGjz+L NrMEMEZx2aSk5mSWpRbp2yVwZew5sJKpYP8e9oquI61sDYy3ZrN3MXJySAiYSLx6/o6xi5GL Q0hgBaPEjKUfoZxJTBJPjjczQTgTmSQ2d+5jgmlZ8vsKC0RiOaPE7lXf2SGcNiaJY31f2UCq 2AT0JWYu+gRmiwjISqz+NwXMFhYIlbg2bxWYzSKgKjH/6R0wm1fAUuLj9pOMELagxMmZT1hA bGYBbYllC18zQ2xWkPj5dBkrxEw9iZ93XzNB1IhI3HjUAna3hMBXdonrjf8ZIRpcJLb+2cAK YQtLvDq+BeprGYn/O+cDNXMA2cUSh/odIMwaiUM/0iEqrCXeNh5nBAkzC2hKrN+lDxF2lHi8 aRILRDWfxI23ghAH8ElM2jadGSLMK9HRJgRhqkj0dYpBNEpJPF02B+oND4mWR6+YJzAqzkLy 7Swk385C8tUshBMWMLKsYhRPLS3OTU8tNspLLdcrTswtLs1L10vOz93ECEwPp/8d/7KDcfmr j3qHGJk4GA8xSnAwK4nw2nBvSBLiTUmsrEotyo8vKs1JLT7EKM3BoiTOm5y5IVFIID2xJDU7 NbUgtQgmy8TBKdXAtP1n9KRJG2Yd0FcVUdnsVZ62mO+nSO/iM9P/XX7cee5g9x8H5sW3Dp7v zI1e86767Knak4lvPYJ4NPZbXqmYxTrRyU4j1nRK+HPum3aZZdcb7j4yvTSVRVyCa0XeB6n+ xU8/Pb/9T+/aFuf9M1UWOK65dcdWuSso4WPYTrsDb5x9JkwpSe9/qdfx9sTRGhlvwf73dWrz 9tp+O1k/xUyjznhn1d1i8fnqs7fzeJ6erfBeecoMb+5NOhW7jrL8PTMjryWGc2ZCRoBS4Lsr e+v6esz2iXNmCm0+q/5xN0fBQp8lzrzSjJcqOMvV3zh4C0WsYrZKCVDbKTO3fM+mI1FfxM1X KZRbMO/77v1i0/vl+5RYijMSDbWYi4oTAXPdAgp+AwAA X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrJLMWRmVeSWpSXmKPExsVy+t/xe7oJuzclGTzaqW7x7dMZZgdGjz+L NrMEMEbp2RTll5akKmTkF5fYKkUbWhjpGVpa6BmZWOoZGpvHWhmZKunb2aSk5mSWpRbp2yXo ZfSuf8hS0HCPpWLVqj7mBsbPu5m7GDk5JARMJJb8vsLSxcjFISSwlFHid+d6NoiElMTSpYsY IWxhiT/XutggilqYJKY1d7KCJNgE9CVmLvoE1iAiICux+t8UMFtYIFjiWcsNsA0sAqoS85/e AYvzClhKfNx+khHCFpQ4OfMJC4jNLKAt8fTmUzh72cLXUNcpSPx8uowVYr6exM+7r5kgakQk bjxqYZzAKDALyahZSEbNQjJqFpKWBYwsqxhFUkuLc9Nziw31ihNzi0vz0vWS83M3MQLDfNux n5t3MM579VHvECMTB+MhRgkOZiURXhvuDUlCvCmJlVWpRfnxRaU5qcWHGE2BfpvILCWanA+M tLySeEMzA1NDEzNLA1NLM2MlcV7Pgo5EIYH0xJLU7NTUgtQimD4mDk6pBqaZK1ewq8/h6Zpq eOHSuYthB6a5tLdYbDl3vDiTj+VJ+3/zTWuO9qVEF/PK1UaYNrj8yG0+6tJg/idt9ZfJRzpO JpfcyGbOk3H4eHH/lctm048vvHyTI4djXbiPzdpPTO8ty1IZuBcExk+Z9DToUnd/xl7miCd3 rlxj++bxeYGbIa9q466nh4PqN39JOib6ZOPPKX+Pzr5nU3vhM2v51JD7Rmp5KRYKuoYRJr8z IoT/6r/lmN74aq2EFn8a75OVTj05E5UU33z4KN1yZe16v/B/l2W+Ns24dWJZrH2vQSX3i+pF /63Pmd07NM+x5VNIwxbPbyxsFjXWrVNKnX7tNSuzvSCb9bgqZar8ijPVXCZKLMUZiYZazEXF iQCMrKHN/AIAAA== X-CMS-MailID: 20220622064904eucas1p19bbbc922c4f1b734730c48169bada35a X-Msg-Generator: CA X-RootMTR: 20220622064904eucas1p19bbbc922c4f1b734730c48169bada35a X-EPHeader: CA CMS-TYPE: 201P X-CMS-RootMailID: 20220622064904eucas1p19bbbc922c4f1b734730c48169bada35a References: Subject: [FFmpeg-devel] [PATCH 2/3] Provided support for MPEG-5 EVC (Essential Video Coding) codec X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: iDR37I6scA8a - Added xeve encoder wrapper - Added xevd dencoder wrapper - Added documentation for xeve and xevd wrappers - Added parser for EVC format - Changes in project configuration file and libavcodec Makefile Signed-off-by: Dawid Kozinski --- configure | 8 + doc/decoders.texi | 24 ++ doc/encoders.texi | 112 ++++++ doc/general_contents.texi | 19 + libavcodec/Makefile | 3 + libavcodec/allcodecs.c | 2 + libavcodec/avcodec.h | 3 + libavcodec/evc_parser.c | 527 ++++++++++++++++++++++++++ libavcodec/libxevd.c | 440 ++++++++++++++++++++++ libavcodec/libxeve.c | 753 ++++++++++++++++++++++++++++++++++++++ libavcodec/parsers.c | 1 + 11 files changed, 1892 insertions(+) create mode 100644 libavcodec/evc_parser.c create mode 100644 libavcodec/libxevd.c create mode 100644 libavcodec/libxeve.c -- 2.17.1 diff --git a/configure b/configure index 3a97610209..8071e22859 100755 --- a/configure +++ b/configure @@ -291,6 +291,8 @@ External library support: --enable-libwebp enable WebP encoding via libwebp [no] --enable-libx264 enable H.264 encoding via x264 [no] --enable-libx265 enable HEVC encoding via x265 [no] + --enable-libxeve enable MPEG-5 Essential Video Coding (EVC) encoding via libxeve [no] + --enable-libxevd enable MPEG-5 Essential Video Coding (EVC) decoding via libxevd [no] --enable-libxavs enable AVS encoding via xavs [no] --enable-libxavs2 enable AVS2 encoding via xavs2 [no] --enable-libxcb enable X11 grabbing using XCB [autodetect] @@ -1874,6 +1876,8 @@ EXTERNAL_LIBRARY_LIST=" libvorbis libvpx libwebp + libxevd + libxeve libxml2 libzimg libzmq @@ -3384,6 +3388,8 @@ libx264rgb_encoder_select="libx264_encoder" libx265_encoder_deps="libx265" libxavs_encoder_deps="libxavs" libxavs2_encoder_deps="libxavs2" +libxevd_decoder_deps="libxevd" +libxeve_encoder_deps="libxeve" libxvid_encoder_deps="libxvid" libzvbi_teletext_decoder_deps="libzvbi" vapoursynth_demuxer_deps="vapoursynth" @@ -6668,6 +6674,8 @@ enabled libx265 && require_pkg_config libx265 x265 x265.h x265_api_get require_cpp_condition libx265 x265.h "X265_BUILD >= 70" enabled libxavs && require libxavs "stdint.h xavs.h" xavs_encoder_encode "-lxavs $pthreads_extralibs $libm_extralibs" enabled libxavs2 && require_pkg_config libxavs2 "xavs2 >= 1.3.0" "stdint.h xavs2.h" xavs2_api_get +enabled libxevd && require_pkg_config libxevd "xevd >= 0.3.3" "xevd.h" xevd_decode +enabled libxeve && require_pkg_config libxeve "xeve >= 0.3.5" "xeve.h" xeve_encode enabled libxvid && require libxvid xvid.h xvid_global -lxvidcore enabled libzimg && require_pkg_config libzimg "zimg >= 2.7.0" zimg.h zimg_get_api_version enabled libzmq && require_pkg_config libzmq "libzmq >= 4.2.1" zmq.h zmq_ctx_new diff --git a/doc/decoders.texi b/doc/decoders.texi index e2fcbf5dc9..0bd9096114 100644 --- a/doc/decoders.texi +++ b/doc/decoders.texi @@ -126,6 +126,30 @@ Set amount of frame threads to use during decoding. The default value is 0 (auto @end table +@section libxevd + +eXtra-fast Essential Video Decoder (XEVD) MPEG-5 EVC decoder wrapper. + +This decoder requires the presence of the libxevd headers and library +during configuration. You need to explicitly configure the build with +@option{--enable-libxevd}. + +The xevd project website is at @url{https://github.com/mpeg5/xevd}. + +@subsection Options + +The following options are supported by the libxevd wrapper. +The xevd-equivalent options or values are listed in parentheses for easy migration. + +To get a more accurate and extensive documentation of the libxevd options, +invoke the command @code{xevd_app --help} or consult the libxevd documentation. + +@table @option +@item threads (@emph{threads}) +Force to use a specific number of threads + +@end table + @section QSV Decoders The family of Intel QuickSync Video decoders (VC1, MPEG-2, H.264, HEVC, diff --git a/doc/encoders.texi b/doc/encoders.texi index 1850c99fe9..245df680d2 100644 --- a/doc/encoders.texi +++ b/doc/encoders.texi @@ -2892,6 +2892,118 @@ ffmpeg -i input -c:v libxavs2 -xavs2-params RdoqLevel=0 output.avs2 @end example @end table +@section libxeve + +eXtra-fast Essential Video Encoder (XEVE) MPEG-5 EVC encoder wrapper. + +This encoder requires the presence of the libxeve headers and library +during configuration. You need to explicitly configure the build with +@option{--enable-libxeve}. + +@float NOTE +Many libxeve encoder options are mapped to FFmpeg global codec options, +while unique encoder options are provided through private options. +Additionally the xeve-params private options allows one to pass a list +of key=value tuples as accepted by the libxeve @code{parse_xeve_params} function. +@end float + +The xeve project website is at @url{https://github.com/mpeg5/xeve}. + +@subsection Options + +The following options are supported by the libxeve wrapper. +The xeve-equivalent options or values are listed in parentheses for easy migration. + +@float NOTE +To reduce the duplication of documentation, only the private options +and some others requiring special attention are documented here. For +the documentation of the undocumented generic options, see +@ref{codec-options,,the Codec Options chapter}. +@end float + +@float NOTE +To get a more accurate and extensive documentation of the libxeve options, +invoke the command @code{xeve_app --help} or consult the libxeve documentation. +@end float + +@table @option +@item b (@emph{bitrate}) +Set target video bitrate in bits/s. +Note that FFmpeg’s b option is expressed in bits/s, while xeve’s bitrate is in kilobits/s. + +@item bf (@emph{bframes}) +Set the maximum number of B frames (1,3,7,15). + +@item g (@emph{keyint}) +Set the GOP size (I-picture period). + +@item preset (@emph{preset}) +Set the xeve preset. +Set the encoder preset value to determine encoding speed [fast, medium, slow, placebo] + +@item tune (@emph{tune}) +Set the encoder tune parameter [psnr, zerolatency] + +@item profile (@emph{profile}) +Set the encoder profile [0: baselie; 1: main] + +@item crf (@emph{crf}) +Set the quality for constant quality mode. +Constant rate factor <-1..51> [default: 32] + +@item qp (@emph{qp}) +Set constant quantization rate control method parameter. +Quantization parameter qp <0..51> [default: 32] + +@item threads (@emph{threads}) +Force to use a specific number of threads + +@item xeve-params +Set xeve options using a list of @var{key}=@var{value} couples separated +by ":". + +For example to specify libxeve encoding options with @option{-xeve-params}: + +@example +ffmpeg -i input -c:v libxeve -xeve-params profile=main:tune=zerolatency:preset=fast:q=17:bframes=15:bitrate=100k output.mp4 +@end example +@end table + +@subsubsection Supported key values for xeve-params + +Set xeve options using a list of @var{key}=@var{value} couples separated + +@table @option + +@item hash +Embed the picture signature (HASH) for conformance checking in decoding, (0: Off, 1: On) + +@item vbv-bufsize +Set the VBV buffer size: Kbits(none,K,k), Mbits(M,m) + +@item rc-type +Set the rate control type, (0: OFF, 1: ABR, 2: CRF) + +@item bframes +Set the maximum number of B frames (1,3,7,15) + +@item profile +Set the PROFILE (main, baseline) + +@item preset +Set the encoder PRESET (fast, medium, slow, placebo) + +@item tune +Set the encoder TUNE (psnr, zerolatency) + +@item bitrate +Set the Bitrate in terms of kilo-bits per second: Kbps(none,K,k), Mbps(M,m) + +@item qp +Set the QP value (0~51) + +@end table + @section libxvid Xvid MPEG-4 Part 2 encoder wrapper. diff --git a/doc/general_contents.texi b/doc/general_contents.texi index 987a2f82fb..6c4b7506ba 100644 --- a/doc/general_contents.texi +++ b/doc/general_contents.texi @@ -343,6 +343,22 @@ libxavs2 is under the GNU Public License Version 2 or later details), you must upgrade FFmpeg's license to GPL in order to use it. @end float +@section eXtra-fast Essential Video Encoder (XEVE) + +FFmpeg can make use of the XEVE library for EVC video encoding. + +Go to @url{https://github.com/mpeg5/xeve} and follow the instructions for +installing the XEVE library. Then pass @code{--enable-libxeve} to configure to +enable it. + +@section eXtra-fast Essential Video Decoder (XEVD) + +FFmpeg can make use of the XEVD library for EVC video decoding. + +Go to @url{https://github.com/mpeg5/xevd} and follow the instructions for +installing the XEVD library. Then pass @code{--enable-libxevd} to configure to +enable it. + @section ZVBI ZVBI is a VBI decoding library which can be used by FFmpeg to decode DVB @@ -591,6 +607,7 @@ library: @item raw DTS @tab X @tab X @item raw DTS-HD @tab @tab X @item raw E-AC-3 @tab X @tab X +@item raw EVC @tab X @tab X @item raw FLAC @tab X @tab X @item raw GSM @tab @tab X @item raw H.261 @tab X @tab X @@ -924,6 +941,8 @@ following image formats are supported: @item Electronic Arts TQI video @tab @tab X @item Escape 124 @tab @tab X @item Escape 130 @tab @tab X +@item EVC / MPEG-5 Part 1 @tab X @tab X + @tab encoding and decoding supported through external libraries libxeve and libxevd @item FFmpeg video codec #1 @tab X @tab X @tab lossless codec (fourcc: FFV1) @item Flash Screen Video v1 @tab X @tab X diff --git a/libavcodec/Makefile b/libavcodec/Makefile index 3b8f7b5e01..cb2d438a14 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -1103,6 +1103,8 @@ OBJS-$(CONFIG_LIBX264_ENCODER) += libx264.o OBJS-$(CONFIG_LIBX265_ENCODER) += libx265.o OBJS-$(CONFIG_LIBXAVS_ENCODER) += libxavs.o OBJS-$(CONFIG_LIBXAVS2_ENCODER) += libxavs2.o +OBJS-$(CONFIG_LIBXEVD_DECODER) += libxevd.o +OBJS-$(CONFIG_LIBXEVE_ENCODER) += libxeve.o OBJS-$(CONFIG_LIBXVID_ENCODER) += libxvid.o OBJS-$(CONFIG_LIBZVBI_TELETEXT_DECODER) += libzvbi-teletextdec.o ass.o @@ -1130,6 +1132,7 @@ OBJS-$(CONFIG_DVAUDIO_PARSER) += dvaudio_parser.o OBJS-$(CONFIG_DVBSUB_PARSER) += dvbsub_parser.o OBJS-$(CONFIG_DVD_NAV_PARSER) += dvd_nav_parser.o OBJS-$(CONFIG_DVDSUB_PARSER) += dvdsub_parser.o +OBJS-$(CONFIG_EVC_PARSER) += evc_parser.o OBJS-$(CONFIG_FLAC_PARSER) += flac_parser.o flacdata.o flac.o OBJS-$(CONFIG_G723_1_PARSER) += g723_1_parser.o OBJS-$(CONFIG_G729_PARSER) += g729_parser.o diff --git a/libavcodec/allcodecs.c b/libavcodec/allcodecs.c index f0b01051b0..2e0acc6f9a 100644 --- a/libavcodec/allcodecs.c +++ b/libavcodec/allcodecs.c @@ -794,6 +794,8 @@ extern LIBX264_CONST FFCodec ff_libx264_encoder; #endif extern const FFCodec ff_libx264rgb_encoder; extern FFCodec ff_libx265_encoder; +extern const FFCodec ff_libxeve_encoder; +extern const FFCodec ff_libxevd_decoder; extern const FFCodec ff_libxavs_encoder; extern const FFCodec ff_libxavs2_encoder; extern const FFCodec ff_libxvid_encoder; diff --git a/libavcodec/avcodec.h b/libavcodec/avcodec.h index 4dae23d06e..b83870c174 100644 --- a/libavcodec/avcodec.h +++ b/libavcodec/avcodec.h @@ -1666,6 +1666,9 @@ typedef struct AVCodecContext { #define FF_PROFILE_KLVA_SYNC 0 #define FF_PROFILE_KLVA_ASYNC 1 +#define FF_PROFILE_EVC_BASELINE 0 +#define FF_PROFILE_EVC_MAIN 1 + /** * level * - encoding: Set by user. diff --git a/libavcodec/evc_parser.c b/libavcodec/evc_parser.c new file mode 100644 index 0000000000..03faf12d62 --- /dev/null +++ b/libavcodec/evc_parser.c @@ -0,0 +1,527 @@ +/* + * EVC AVC format parser + * + * Copyright (C) 2021 Dawid Kozinski + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include + +#include "libavutil/common.h" + +#include "parser.h" +#include "golomb.h" + +// The length field that indicates the length in bytes of the following NAL unit is configured to be of 4 bytes +#define EVC_NAL_UNIT_LENGTH_BYTE (4) /* byte */ + +#define EVC_NAL_HEADER_SIZE (2) /* byte */ +#define MAX_SPS_CNT (16) /* defined value in EVC standard */ + +// NALU types +// @see ISO_IEC_23094-1_2020 7.4.2.2 NAL unit header semantics +// +#define EVC_NUT_NONIDR (0) /* Coded slice of a non-IDR picture */ +#define EVC_NUT_IDR (1) /* Coded slice of an IDR picture */ +#define EVC_NUT_SPS (24) /* Sequence parameter set */ +#define EVC_NUT_PPS (25) /* Picture paremeter set */ +#define EVC_NUT_APS (26) /* Adaptation parameter set */ +#define EVC_NUT_FD (27) /* Filler data */ +#define EVC_NUT_SEI (28) /* Supplemental enhancement information */ + +// The sturcture reflects SPS RBSP(raw byte sequence payload) layout +// @see ISO_IEC_23094-1 section 7.3.2.1 +// +// The following descriptors specify the parsing process of each element +// u(n) - unsigned integer using n bits +// ue(v) - unsigned integer 0-th order Exp_Golomb-coded syntax element with the left bit first +typedef struct EVCParserSPS { + int sps_seq_parameter_set_id; // ue(v) + int profile_idc; // u(8) + int level_idc; // u(8) + int toolset_idc_h; // u(32) + int toolset_idc_l; // u(32) + int chroma_format_idc; // ue(v) + int pic_width_in_luma_samples; // ue(v) + int pic_height_in_luma_samples; // ue(v) + int bit_depth_luma_minus8; // ue(v) + int bit_depth_chroma_minus8; // ue(v) + + int sps_btt_flag; // u(1) + int log2_ctu_size_minus5; // ue(v) + int log2_min_cb_size_minus2; // ue(v) + int log2_diff_ctu_max_14_cb_size; // ue(v) + int log2_diff_ctu_max_tt_cb_size; // ue(v) + int log2_diff_min_cb_min_tt_cb_size_minus2; // ue(v) + + int sps_suco_flag; // u(1) + int log2_diff_ctu_size_max_suco_cb_size; // ue(v) + int log2_diff_max_suco_min_suco_cb_size; // ue(v) + + int sps_admvp_flag; // u(1) + int sps_affine_flag; // u(1) + int sps_amvr_flag; // u(1) + int sps_dmvr_flag; // u(1) + int sps_mmvd_flag; // u(1) + int sps_hmvp_flag; // u(1) + + int sps_eipd_flag; // u(1) + int sps_ibc_flag; // u(1) + int log2_max_ibc_cand_size_minus2; // ue(v) + + int sps_cm_init_flag; // u(1) + int sps_adcc_flag; // u(1) + + int sps_iqt_flag; // u(1) + int sps_ats_flag; // u(1) + + int sps_addb_flag; // u(1) + int sps_alf_flag; // u(1) + int sps_htdf_flag; // u(1) + int sps_rpl_flag; // u(1) + int sps_pocs_flag; // u(1) + int sps_dquant_flag; // u(1) + int sps_dra_flag; // u(1) + + int log2_max_pic_order_cnt_lsb_minus4; // ue(v) + int log2_sub_gop_length; // ue(v) + int log2_ref_pic_gap_length; // ue(v) + + int max_num_tid0_ref_pics; // ue(v) + + int sps_max_dec_pic_buffering_minus1; // ue(v) + int long_term_ref_pic_flag; // u(1) + int rpl1_same_as_rpl0_flag; // u(1) + int num_ref_pic_list_in_sps[2]; // ue(v) + + int picture_cropping_flag; // u(1) + int picture_crop_left_offset; // ue(v) + int picture_crop_right_offset; // ue(v) + int picture_crop_top_offset; // ue(v) + int picture_crop_bottom_offset; // ue(v) + +} EVCParserSPS; + +typedef struct EVCParserContext { + ParseContext pc; + EVCParserSPS sps[MAX_SPS_CNT]; + int is_avc; + int nal_length_size; + int to_read; + int incomplete_nalu_prefix_read; // The flag is set to 1 when an incomplete NAL unit prefix has been read + + int got_sps; + int got_pps; + int got_sei; + int got_slice; +} EVCParserContext; + +static int get_nalu_type(const uint8_t *bits, int bits_size, AVCodecContext *avctx) +{ + int unit_type_plus1 = 0; + + if(bits_size >= EVC_NAL_HEADER_SIZE) { + unsigned char *p = (unsigned char *)bits; + // forbidden_zero_bit + if ((p[0] & 0x80) != 0) { + av_log(avctx, AV_LOG_ERROR, "Cannot get bitstream information. Malformed bitstream.\n"); + return -1; + } + + // nal_unit_type + unit_type_plus1 = (p[0] >> 1) & 0x3F; + } + + return unit_type_plus1 - 1; +} + +static uint32_t read_nal_unit_length(const uint8_t *bits, int bits_size, AVCodecContext *avctx) +{ + uint32_t nalu_len = 0; + + if(bits_size >= EVC_NAL_UNIT_LENGTH_BYTE) { + + int t = 0; + unsigned char *p = (unsigned char *)bits; + + for(int i=0; i= MAX_SPS_CNT) + return NULL; + + sps = &ev->sps[sps_seq_parameter_set_id]; + sps->sps_seq_parameter_set_id = sps_seq_parameter_set_id; + + // the Baseline profile is indicated by profile_idc eqal to 0 + // the Main profile is indicated by profile_idc eqal to 1 + sps->profile_idc = get_bits(&gb, 8); + + sps->level_idc = get_bits(&gb, 8); + + skip_bits_long(&gb, 32); /* skip toolset_idc_h */ + skip_bits_long(&gb, 32); /* skip toolset_idc_l */ + + // 0 - monochrome + // 1 - 4:2:0 + // 2 - 4:2:2 + // 3 - 4:4:4 + sps->chroma_format_idc = get_ue_golomb(&gb); + + sps->pic_width_in_luma_samples = get_ue_golomb(&gb); + sps->pic_height_in_luma_samples = get_ue_golomb(&gb); + + sps->bit_depth_luma_minus8 = get_ue_golomb(&gb); + sps->bit_depth_chroma_minus8 = get_ue_golomb(&gb); + + sps->sps_btt_flag = get_bits(&gb, 1); + if(sps->sps_btt_flag) { + sps->log2_ctu_size_minus5 = get_ue_golomb(&gb); + sps->log2_min_cb_size_minus2 = get_ue_golomb(&gb); + sps->log2_diff_ctu_max_14_cb_size = get_ue_golomb(&gb); + sps->log2_diff_ctu_max_tt_cb_size = get_ue_golomb(&gb); + sps->log2_diff_min_cb_min_tt_cb_size_minus2 = get_ue_golomb(&gb); + } + + sps->sps_suco_flag = get_bits(&gb, 1); + if(sps->sps_suco_flag) { + sps->log2_diff_ctu_size_max_suco_cb_size = get_ue_golomb(&gb); + sps->log2_diff_max_suco_min_suco_cb_size = get_ue_golomb(&gb); + } + + sps->sps_admvp_flag = get_bits(&gb, 1); + if(sps->sps_admvp_flag) { + sps->sps_affine_flag = get_bits(&gb, 1); + sps->sps_amvr_flag = get_bits(&gb, 1); + sps->sps_dmvr_flag = get_bits(&gb, 1); + sps->sps_mmvd_flag = get_bits(&gb, 1); + sps->sps_hmvp_flag = get_bits(&gb, 1); + } + + sps->sps_eipd_flag = get_bits(&gb, 1); + if(sps->sps_eipd_flag) { + sps->sps_ibc_flag = get_bits(&gb, 1); + if(sps->sps_ibc_flag) + sps->log2_max_ibc_cand_size_minus2 = get_ue_golomb(&gb); + } + + sps->sps_cm_init_flag = get_bits(&gb, 1); + if(sps->sps_cm_init_flag) + sps->sps_adcc_flag = get_bits(&gb, 1); + + sps->sps_iqt_flag = get_bits(&gb, 1); + if(sps->sps_iqt_flag) + sps->sps_ats_flag = get_bits(&gb, 1); + + sps->sps_addb_flag = get_bits(&gb, 1); + sps->sps_alf_flag = get_bits(&gb, 1); + sps->sps_htdf_flag = get_bits(&gb, 1); + sps->sps_rpl_flag = get_bits(&gb, 1); + sps->sps_pocs_flag = get_bits(&gb, 1); + sps->sps_dquant_flag = get_bits(&gb, 1); + sps->sps_dra_flag = get_bits(&gb, 1); + + if(sps->sps_pocs_flag) + sps->log2_max_pic_order_cnt_lsb_minus4 = get_ue_golomb(&gb); + + if(!sps->sps_pocs_flag || !sps->sps_rpl_flag) { + sps->log2_sub_gop_length = get_ue_golomb(&gb); + if(sps->log2_sub_gop_length == 0) + sps->log2_ref_pic_gap_length = get_ue_golomb(&gb); + } + + return sps; +} + +static int parse_nal_units(AVCodecParserContext *s, const uint8_t *bs, + int bs_size, AVCodecContext *avctx) +{ + EVCParserContext *ev = s->priv_data; + int nalu_type, nalu_size; + unsigned char *bits = (unsigned char *)bs; + int bits_size = bs_size; + + avctx->codec_id = AV_CODEC_ID_EVC; + + nalu_size = read_nal_unit_length(bits, bits_size, avctx); + if(nalu_size == 0) { + av_log(avctx, AV_LOG_ERROR, "Invalid NAL unit size: (%d)\n", nalu_size); + return -1; + } + + bits += EVC_NAL_UNIT_LENGTH_BYTE; + bits_size -= EVC_NAL_UNIT_LENGTH_BYTE; + + nalu_type = get_nalu_type(bits, bits_size, avctx); + + bits += EVC_NAL_HEADER_SIZE; + bits_size -= EVC_NAL_HEADER_SIZE; + + if (nalu_type == EVC_NUT_SPS) { // NAL Unit type: SPS (Sequence Parameter Set) + EVCParserSPS *sps; + + sps = parse_sps(bits, bits_size, ev); + + avctx->coded_width = sps->pic_width_in_luma_samples; + avctx->coded_height = sps->pic_height_in_luma_samples; + avctx->width = sps->pic_width_in_luma_samples; + avctx->height = sps->pic_height_in_luma_samples; + + if(sps->profile_idc == 0) avctx->profile = FF_PROFILE_EVC_BASELINE; + else if (sps->profile_idc == 1) avctx->profile = FF_PROFILE_EVC_MAIN; + else { + av_log(avctx, AV_LOG_ERROR, "Not supported profile (%d)\n", sps->profile_idc); + return -1; + } + + // Currently XEVD decoder supports ony YCBCR420_10LE chroma format for EVC stream + switch(sps->chroma_format_idc) { + case 0: /* YCBCR400_10LE */ + av_log(avctx, AV_LOG_ERROR, "YCBCR400_10LE: Not supported chroma format\n"); + avctx->pix_fmt = AV_PIX_FMT_GRAY10LE; + return -1; + case 1: /* YCBCR420_10LE */ + avctx->pix_fmt = AV_PIX_FMT_YUV420P10LE; + break; + case 2: /* YCBCR422_10LE */ + av_log(avctx, AV_LOG_ERROR, "YCBCR422_10LE: Not supported chroma format\n"); + avctx->pix_fmt = AV_PIX_FMT_YUV422P10LE; + return -1; + case 3: /* YCBCR444_10LE */ + av_log(avctx, AV_LOG_ERROR, "YCBCR444_10LE: Not supported chroma format\n"); + avctx->pix_fmt = AV_PIX_FMT_YUV444P10LE; + return -1; + default: + avctx->pix_fmt = AV_PIX_FMT_NONE; + av_log(avctx, AV_LOG_ERROR, "Unknown supported chroma format\n"); + return -1; + } + + // The current implementation of parse_sps function doesn't handle VUI parameters parsing, + // so at the moment it's impossible to initialize has_b_frames and max_b_frames AVCodecContex fields here. + // Currently, initialization of has_b_frames and max_b_frames AVCodecContex fields have been moved to + // libxevd_decode function where we can use xevd_config function being a part of xevd library API + // to get the needed information. + // However, if it will be needed, parse_sps function should be extended to handle VUI parameters parsing + // and the following lines should be used to initialize has_b_frames and max_b_frames fields of the AVCodecContex. + // + // sps->vui_parameters.num_reorder_pics + // if (sps->bitstream_restriction_flag && sps->vui_parameters.num_reorder_pics) { + // avctx->has_b_frames = sps->vui_parameters.num_reorder_pics; + // } + + ev->got_sps = 1; + + } else if (nalu_type == EVC_NUT_PPS) // NAL Unit type: PPS (Video Parameter Set) + ev->got_pps = 1; + else if(nalu_type == EVC_NUT_SEI) // NAL unit type: SEI (Supplemental Enhancement Information) + ev->got_sei = 1; + else if (nalu_type == EVC_NUT_IDR || nalu_type == EVC_NUT_NONIDR) // NAL Unit type: Coded slice of a IDR or non-IDR picture + ev->got_slice++; + else { + av_log(avctx, AV_LOG_ERROR, "Invalid NAL unit type: %d\n", nalu_type); + return -1; + } + + return 0; +} + +/** + * Find the end of the current frame in the bitstream. + * @return the position of the first byte of the next frame, or END_NOT_FOUND + */ +static int evc_find_frame_end(AVCodecParserContext *s, const uint8_t *buf, + int buf_size, AVCodecContext *avctx) +{ + EVCParserContext *ev = s->priv_data; + + if(!ev->to_read) { + int nal_unit_size = 0; + int next = END_NOT_FOUND; + + // This is the case when buffer size is not enough for buffer to store NAL unit length + if(buf_size < EVC_NAL_UNIT_LENGTH_BYTE) { + ev->to_read = EVC_NAL_UNIT_LENGTH_BYTE; + ev->nal_length_size = buf_size; + ev->incomplete_nalu_prefix_read = 1; + + return END_NOT_FOUND; + } + + nal_unit_size = read_nal_unit_length(buf, buf_size, avctx); + ev->nal_length_size = EVC_NAL_UNIT_LENGTH_BYTE; + + next = nal_unit_size + EVC_NAL_UNIT_LENGTH_BYTE; + ev->to_read = next; + if(next < buf_size) + return next; + else + return END_NOT_FOUND; + } else if(ev->to_read > buf_size) { + return END_NOT_FOUND; + } else { + if(ev->incomplete_nalu_prefix_read == 1) { + EVCParserContext *ev = s->priv_data; + ParseContext *pc = &ev->pc; + uint8_t nalu_len[EVC_NAL_UNIT_LENGTH_BYTE] = {0}; + int nal_unit_size = 0; + + // 1. pc->buffer contains previously read bytes of NALU prefix + // 2. buf contains the rest of NAL unit prefix bytes + // + // ~~~~~~~ + // EXAMPLE + // ~~~~~~~ + // + // In the following example we assumed that the number of already read NAL Unit prefix bytes is equal 1 + // + // ---------- + // pc->buffer -> conatins already read bytes + // ---------- + // __ pc->index == 1 + // | + // V + // ------------------------------------------------------- + // | 0 | 1 | 2 | 3 | 4 | ... | N | + // ------------------------------------------------------- + // | 0x00 | 0xXX | 0xXX | 0xXX | 0xXX | ... | 0xXX | + // ------------------------------------------------------- + // + // ---------- + // buf -> contains newly read bytes + // ---------- + // ------------------------------------------------------- + // | 0 | 1 | 2 | 3 | 4 | ... | N | + // ------------------------------------------------------- + // | 0x00 | 0x00 | 0x3C | 0xXX | 0xXX | ... | 0xXX | + // ------------------------------------------------------- + // + for(int i = 0; i < EVC_NAL_UNIT_LENGTH_BYTE; i++) { + if(i < pc->index) + nalu_len[i] = pc->buffer[i]; + else + nalu_len[i] = buf[i - pc->index]; + } + + // ---------- + // nalu_len + // ---------- + // --------------------------------- + // | 0 | 1 | 2 | 3 | + // --------------------------------- + // | 0x00 | 0x00 | 0x00 | 0x3C | + // --------------------------------- + // | NALU LENGTH | + // --------------------------------- + // NAL Unit lenght = 60 (0x0000003C) + + nal_unit_size = read_nal_unit_length(nalu_len, EVC_NAL_UNIT_LENGTH_BYTE, avctx); + + ev->to_read = nal_unit_size + EVC_NAL_UNIT_LENGTH_BYTE - pc->index; + + ev->incomplete_nalu_prefix_read = 0; + + if(ev->to_read > buf_size) + return END_NOT_FOUND; + else + return ev->to_read; + } + return ev->to_read; + } + + return END_NOT_FOUND; +} + +static int evc_parser_init(AVCodecParserContext *s) +{ + EVCParserContext *ev = s->priv_data; + + ev->got_sps = 0; + ev->got_pps = 0; + ev->got_sei = 0; + ev->got_slice = 0; + ev->nal_length_size = EVC_NAL_UNIT_LENGTH_BYTE; + ev->incomplete_nalu_prefix_read = 0; + + return 0; +} + +static int evc_parse(AVCodecParserContext *s, AVCodecContext *avctx, + const uint8_t **poutbuf, int *poutbuf_size, + const uint8_t *buf, int buf_size) +{ + int next; + EVCParserContext *ev = s->priv_data; + ParseContext *pc = &ev->pc; + int is_dummy_buf = !buf_size; + const uint8_t *dummy_buf = buf; + + if (s->flags & PARSER_FLAG_COMPLETE_FRAMES) + next = buf_size; + else { + next = evc_find_frame_end(s, buf, buf_size, avctx); + if (ff_combine_frame(pc, next, &buf, &buf_size) < 0) { + *poutbuf = NULL; + *poutbuf_size = 0; + ev->to_read -= buf_size; + return buf_size; + } + } + + is_dummy_buf &= (dummy_buf == buf); + + if (!is_dummy_buf) + parse_nal_units(s, buf, buf_size, avctx); + + *poutbuf = buf; + *poutbuf_size = buf_size; + ev->to_read -= next; + + return next; +} + +const AVCodecParser ff_evc_parser = { + .codec_ids = { AV_CODEC_ID_EVC }, + .priv_data_size = sizeof(EVCParserContext), + .parser_init = evc_parser_init, + .parser_parse = evc_parse, + .parser_close = ff_parse_close, +}; diff --git a/libavcodec/libxevd.c b/libavcodec/libxevd.c new file mode 100644 index 0000000000..ce6862d72f --- /dev/null +++ b/libavcodec/libxevd.c @@ -0,0 +1,440 @@ +/* + * libxevd decoder + * EVC (MPEG-5 Essential Video Coding) decoding using XEVD MPEG-5 EVC decoder library + * + * Copyright (C) 2021 Dawid Kozinski + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include +#include + +#include + +#include "libavutil/internal.h" +#include "libavutil/common.h" +#include "libavutil/opt.h" +#include "libavutil/pixdesc.h" +#include "libavutil/pixfmt.h" +#include "libavutil/imgutils.h" +#include "libavutil/cpu.h" + +#include "avcodec.h" +#include "internal.h" +#include "packet_internal.h" +#include "codec_internal.h" + +#define XEVD_PARAM_BAD_NAME -1 +#define XEVD_PARAM_BAD_VALUE -2 + +#define EVC_NAL_HEADER_SIZE 2 /* byte */ + +/** + * The structure stores all the states associated with the instance of Xeve MPEG-5 EVC decoder + */ +typedef struct XevdContext { + const AVClass *class; + + XEVD id; // XEVD instance identifier @see xevd.h + XEVD_CDSC cdsc; // decoding parameters @see xevd.h + + int decod_frames; // number of decoded frames + int packet_count; // number of packets created by decoder + + // configuration parameters + AVDictionary *xevd_opts; // xevd configuration read from a : separated list of key=value parameters + +} XevdContext; + +/** + * The function returns a pointer to a variable of type XEVD_CDSC. + * XEVD_CDSC contains all decoder parameters that should be initialized before its use. + * + * The field values of the XEVD_CDSC structure are populated based on: + * - the corresponding field values of the AvCodecConetxt structure, + * - the xevd decoder specific option values, + * (the full list of options available for the xevd encoder is displayed after executing the command ./ffmpeg --help decoder = libxevd) + * - and the xevd encoder options specified as a list of key-value pairs following the xevd-params option + * + * Order of input processing and populating the XEVD_CDSC structure + * 1. first, the corresponding fields of the AVCodecContext structure are processed, (i.e -threads 4) + * 2. then xevd-specific options are added as AVOption to the xevd AVCodec implementation (i.e -threads 3) + * 3. finally, the options specified after the xevd-params option as the parameter list of type key value are processed (i.e -xevd-params "m=2") + * + * Some options can be set in different ways. In this case, please follow the above-mentioned order of processing. + * The most recent assignments overwrite the previous values. + * + * @param[in] avctx codec context + * @param[out] cdsc contains all encoder parameters that should be initialized before its use + * + * @return 0 on success, negative error code on failure + */ +static int get_conf(AVCodecContext *avctx, XEVD_CDSC *cdsc) +{ + int cpu_count = av_cpu_count(); + + /* clear XEVS_CDSC structure */ + memset(cdsc, 0, sizeof(XEVD_CDSC)); + + /* init XEVD_CDSC */ + if(avctx->thread_count <= 0) + cdsc->threads = (cpu_count < XEVD_MAX_TASK_CNT) ? cpu_count : XEVD_MAX_TASK_CNT; + else if(avctx->thread_count > XEVD_MAX_TASK_CNT) + cdsc->threads = XEVD_MAX_TASK_CNT; + else + cdsc->threads = avctx->thread_count; + + return 0; +} + +/** + * Read NAL unit length + * @param bs input data (bitstream) + * @return the length of NAL unit on success, 0 value on failure + */ +static uint32_t read_nal_unit_length(const uint8_t *bs, int bs_size, AVCodecContext *avctx) +{ + uint32_t len = 0; + XEVD_INFO info; + int ret; + + if(bs_size == XEVD_NAL_UNIT_LENGTH_BYTE) { + ret = xevd_info((void *)bs, XEVD_NAL_UNIT_LENGTH_BYTE, 1, &info); + if (XEVD_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Cannot get bitstream information\n"); + return 0; + } + len = info.nalu_len; + if(len == 0) { + av_log(avctx, AV_LOG_ERROR, "Invalid bitstream size! [%d]\n", bs_size); + return 0; + } + } + + return len; +} + +/** + * @param[in] xectx the structure that stores all the state associated with the instance of Xeve MPEG-5 EVC decoder + * @param[out] avctx codec context + * @return 0 on success, negative value on failure + */ +static int export_stream_params(const XevdContext *xectx, AVCodecContext *avctx) +{ + int ret; + int size; + int color_space; + + avctx->pix_fmt = AV_PIX_FMT_YUV420P10; + + size = 4; + ret = xevd_config(xectx->id, XEVD_CFG_GET_CODED_WIDTH, &avctx->coded_width, &size); + if (XEVD_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to get coded_width\n"); + return AVERROR_EXTERNAL; + } + + ret = xevd_config(xectx->id, XEVD_CFG_GET_CODED_HEIGHT, &avctx->coded_height, &size); + if (XEVD_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to get coded_height\n"); + return AVERROR_EXTERNAL; + } + + ret = xevd_config(xectx->id, XEVD_CFG_GET_WIDTH, &avctx->width, &size); + if (XEVD_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to get width\n"); + return AVERROR_EXTERNAL; + } + + ret = xevd_config(xectx->id, XEVD_CFG_GET_HEIGHT, &avctx->height, &size); + if (XEVD_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to get height\n"); + return AVERROR_EXTERNAL; + } + + ret = xevd_config(xectx->id, XEVD_CFG_GET_COLOR_SPACE, &color_space, &size); + if (XEVD_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to get color_space\n"); + return AVERROR_EXTERNAL; + } + switch(color_space) { + case XEVD_CS_YCBCR400_10LE: + avctx->pix_fmt = AV_PIX_FMT_GRAY10LE; + break; + case XEVD_CS_YCBCR420_10LE: + avctx->pix_fmt = AV_PIX_FMT_YUV420P10LE; + break; + case XEVD_CS_YCBCR422_10LE: + avctx->pix_fmt = AV_PIX_FMT_YUV422P10LE; + break; + case XEVD_CS_YCBCR444_10LE: + avctx->pix_fmt = AV_PIX_FMT_YUV444P10LE; + break; + default: + av_log(avctx, AV_LOG_ERROR, "Unknown color space\n"); + avctx->pix_fmt = AV_PIX_FMT_NONE; + return AVERROR_INVALIDDATA; + } + + // the function returns sps->num_reorder_pics + ret = xevd_config(xectx->id, XEVD_CFG_GET_MAX_CODING_DELAY, &avctx->max_b_frames, &size); + if (XEVD_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to get max_coding_delay\n"); + return AVERROR_EXTERNAL; + } + + avctx->has_b_frames = (avctx->max_b_frames) ? 1 : 0; + + avctx->color_primaries = AVCOL_PRI_UNSPECIFIED; + avctx->color_trc = AVCOL_TRC_UNSPECIFIED; + avctx->colorspace = AVCOL_SPC_UNSPECIFIED; + + return 0; +} + +/** + * Initialize decoder + * Create a decoder instance and allocate all the needed resources + * + * @param avctx codec context + * @return 0 on success, negative error code on failure + */ +static av_cold int libxevd_init(AVCodecContext *avctx) +{ + XevdContext *xectx = avctx->priv_data; + int ret = 0; + XEVD_CDSC *cdsc = &(xectx->cdsc); + + /* read configurations and set values for created descriptor (XEVD_CDSC) */ + ret = get_conf(avctx, cdsc); + if (ret != 0) { + av_log(avctx, AV_LOG_ERROR, "Cannot get configuration\n"); + return AVERROR_INVALIDDATA; + } + + /* create decoder */ + xectx->id = xevd_create(&(xectx->cdsc), NULL); + if(xectx->id == NULL) { + av_log(avctx, AV_LOG_ERROR, "Cannot create XEVD encoder\n"); + return AVERROR_EXTERNAL; + } + + xectx->packet_count = 0; + xectx->decod_frames = 0; + + return 0; +} + +/** + * Decode picture + * + * @param avctx codec context + * @param[out] frame decoded frame + * @param[out] got_frame_ptr decoder sets to 0 or 1 to indicate that a + * non-empty frame or subtitle was returned in + * outdata. + * @param[in] avpkt AVPacket containing encoded data to be decoded + * + * @return amount of bytes read from the packet on success, negative error + * code on failure + */ + static int libxevd_decode(struct AVCodecContext *avctx, struct AVFrame *frame, int *got_frame_ptr, AVPacket *avpkt) +{ + XevdContext *xectx = NULL; + XEVD_IMGB *imgb = NULL; + XEVD_STAT stat; + XEVD_BITB bitb; + int xevd_ret, nalu_size, bs_read_pos; + int ret = 0; + + if(avctx == NULL) { + av_log(avctx, AV_LOG_ERROR, "Invalid input parameter: AVCodecContext\n"); + return AVERROR(EINVAL); + } + xectx = avctx->priv_data; + if(xectx == NULL) { + av_log(avctx, AV_LOG_ERROR, "Invalid XEVD context\n"); + return AVERROR(EINVAL); + } + + if(avpkt->size > 0) { + bs_read_pos = 0; + imgb = NULL; + while(avpkt->size > (bs_read_pos + XEVD_NAL_UNIT_LENGTH_BYTE)) { + + memset(&stat, 0, sizeof(XEVD_STAT)); + memset(&bitb, 0, sizeof(XEVD_BITB)); + + nalu_size = read_nal_unit_length(avpkt->data + bs_read_pos, XEVD_NAL_UNIT_LENGTH_BYTE, avctx); + if(nalu_size == 0) { + av_log(avctx, AV_LOG_ERROR, "Invalid bitstream\n"); + ret = AVERROR_INVALIDDATA; + goto ERR; + } + bs_read_pos += XEVD_NAL_UNIT_LENGTH_BYTE; + + bitb.addr = avpkt->data + bs_read_pos; + bitb.ssize = nalu_size; + + /* main decoding block */ + xevd_ret = xevd_decode(xectx->id, &bitb, &stat); + if(XEVD_FAILED(xevd_ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to decode bitstream\n"); + ret = AVERROR_EXTERNAL; + goto ERR; + } + + bs_read_pos += nalu_size; + + if(stat.nalu_type == XEVD_NUT_SPS) { // EVC stream parameters changed + if((ret = export_stream_params(xectx, avctx)) != 0) + goto ERR; + } + + if(stat.read != nalu_size) + av_log(avctx, AV_LOG_INFO, "Different reading of bitstream (in:%d, read:%d)\n,", nalu_size, stat.read); + if(stat.fnum >= 0) { + // already has a decoded image + if (imgb) { + // xevd_pull uses pool of objects of type XEVD_IMGB. + // The pool size is equal MAX_PB_SIZE (26), so release object when it is no more needed + imgb->release(imgb); + imgb = NULL; + } + xevd_ret = xevd_pull(xectx->id, &imgb); + if(XEVD_FAILED(xevd_ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to pull the decoded image (xevd error code: %d, frame#=%d)\n", xevd_ret, stat.fnum); + ret = AVERROR_EXTERNAL; + goto ERR; + } else if (xevd_ret == XEVD_OK_FRM_DELAYED) { + if(imgb) { + // xevd_pull uses pool of objects of type XEVD_IMGB. + // The pool size is equal MAX_PB_SIZE (26), so release object when it is no more needed + imgb->release(imgb); + imgb = NULL; + } + } + } + } + } else { // bumping + xevd_ret = xevd_pull(xectx->id, &(imgb)); + if(xevd_ret == XEVD_ERR_UNEXPECTED) { // bumping process completed + *got_frame_ptr = 0; + return 0; + } else if(XEVD_FAILED(xevd_ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to pull the decoded image (xevd error code: %d)\n", xevd_ret); + ret = AVERROR_EXTERNAL; + goto ERR; + } + } + + if(imgb) { + // @todo supports other color space and bit depth + if(imgb->cs != XEVD_CS_YCBCR420_10LE) { + av_log(avctx, AV_LOG_ERROR, "Not supported pixel format: %s\n", av_get_pix_fmt_name(avctx->pix_fmt)); + ret = AVERROR_INVALIDDATA; + goto ERR; + } + + if (imgb->w[0] != avctx->width || imgb->h[0] != avctx->height) { // stream resolution changed + if(ff_set_dimensions(avctx, imgb->w[0], imgb->h[0]) < 0) { + av_log(avctx, AV_LOG_ERROR, "Cannot set new dimension\n"); + ret = AVERROR_INVALIDDATA; + goto ERR; + } + } + + frame->coded_picture_number++; + frame->display_picture_number++; + frame->format = AV_PIX_FMT_YUV420P10LE; + + if (ff_get_buffer(avctx, frame, 0) < 0) { + av_log(avctx, AV_LOG_ERROR, "Cannot get AV buffer\n"); + ret = AVERROR(ENOMEM); + goto ERR; + } + + frame->pts = avpkt->pts; + + av_image_copy(frame->data, frame->linesize, (const uint8_t **)imgb->a, + imgb->s, avctx->pix_fmt, + imgb->w[0], imgb->h[0]); + + xectx->decod_frames++; + *got_frame_ptr = 1; + + // xevd_pull uses pool of objects of type XEVD_IMGB. + // The pool size is equal MAX_PB_SIZE (26), so release object when it is no more needed + imgb->release(imgb); + imgb = NULL; + } else + *got_frame_ptr = 0; + + xectx->packet_count++; + + return avpkt->size; + +ERR: + if(imgb) { + imgb->release(imgb); + imgb = NULL; + } + *got_frame_ptr = 0; + + return ret; +} + +/** + * Destroy decoder + * + * @param avctx codec context + * @return 0 on success + */ +static av_cold int libxevd_close(AVCodecContext *avctx) +{ + XevdContext *xectx = avctx->priv_data; + if(xectx->id) { + xevd_delete(xectx->id); + xectx->id = NULL; + } + + return 0; +} + +#define OFFSET(x) offsetof(XevdContext, x) +#define VD AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_DECODING_PARAM + +static const AVClass xevd_class = { + .class_name = "libxevd", + .item_name = av_default_item_name, + .version = LIBAVUTIL_VERSION_INT, +}; + +const FFCodec ff_libxevd_decoder = { + .p.name = "evc", + .p.long_name = NULL_IF_CONFIG_SMALL("EVC / MPEG-5 Essential Video Coding (EVC)"), + .p.type = AVMEDIA_TYPE_VIDEO, + .p.id = AV_CODEC_ID_EVC, + .init = libxevd_init, + FF_CODEC_DECODE_CB(libxevd_decode), + .close = libxevd_close, + .priv_data_size = sizeof(XevdContext), + .p.priv_class = &xevd_class, + .p.capabilities = AV_CODEC_CAP_DELAY | AV_CODEC_CAP_AUTO_THREADS | AV_CODEC_CAP_AVOID_PROBING | AV_CODEC_CAP_DR1, + .p.wrapper_name = "libxevd", +}; diff --git a/libavcodec/libxeve.c b/libavcodec/libxeve.c new file mode 100644 index 0000000000..e115ce63d2 --- /dev/null +++ b/libavcodec/libxeve.c @@ -0,0 +1,753 @@ +/* + * libxeve encoder + * EVC (MPEG-5 Essential Video Coding) encoding using XEVE MPEG-5 EVC encoder library + * + * Copyright (C) 2021 Dawid Kozinski + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include +#include + +#include + +#include "libavutil/internal.h" +#include "libavutil/common.h" +#include "libavutil/opt.h" +#include "libavutil/pixdesc.h" +#include "libavutil/pixfmt.h" +#include "libavutil/time.h" +#include "libavutil/cpu.h" +#include "libavutil/avstring.h" + +#include "avcodec.h" +#include "internal.h" +#include "packet_internal.h" +#include "codec_internal.h" +#include "encode.h" + +#define MAX_BS_BUF (16*1024*1024) + +/** + * Error codes + */ +#define XEVE_PARAM_BAD_NAME -100 +#define XEVE_PARAM_BAD_VALUE -200 + +/** + * Encoder states + * + * STATE_ENCODING - the encoder receives and processes input frames + * STATE_BUMPING - there are no more input frames, however the encoder still processes previously received data + * STATE_SKIPPING - skipping input frames + */ +typedef enum State { + STATE_ENCODING, + STATE_BUMPING, + STATE_SKIPPING +} State; + +/** + * The structure stores all the states associated with the instance of Xeve MPEG-5 EVC encoder + */ +typedef struct XeveContext { + const AVClass *class; + + XEVE id; // XEVE instance identifier + XEVE_CDSC cdsc; // coding parameters i.e profile, width & height of input frame, num of therads, frame rate ... + XEVE_BITB bitb; // bitstream buffer (output) + XEVE_STAT stat; // encoding status (output) + XEVE_IMGB imgb; // image buffer (input) + + State state; // encoder state (skipping, encoding, bumping) + + // Chroma subsampling + int width_luma; + int height_luma; + int width_chroma; + int height_chroma; + + int profile_id; // encoder profile (main, baseline) + int preset_id; // preset of xeve ( fast, medium, slow, placebo) + int tune_id; // tune of xeve (psnr, zerolatency) + int input_depth; // input bit-depth: 8bit, 10bit + int color_format; // input data color format: currently only XEVE_CF_YCBCR420 is supported + int hash; // embed picture signature (HASH) for conformance checking in decoding + + /* variables for input parameter */ + char *op_preset; + char *op_tune; + int op_qp; + int op_crf; + + // configuration parameters + // xeve configuration read from a : separated list of key=value parameters + AVDictionary *xeve_params; +} XeveContext; + +/** + * Gets Xeve pre-defined preset + * + * @param preset string describing Xeve encoder preset (fast, medium, slow, placebo) + * @return XEVE pre-defined profile on success, negative value on failure + */ +static int get_preset_id(const char *preset) +{ + if((!strcmp(preset, "fast"))) + return XEVE_PRESET_FAST; + else if (!strcmp(preset, "medium")) + return XEVE_PRESET_MEDIUM; + else if (!strcmp(preset, "slow")) + return XEVE_PRESET_SLOW; + else if (!strcmp(preset, "placebo")) + return XEVE_PRESET_PLACEBO; + else + return AVERROR(EINVAL); +} + +/** + * Gets Xeve pre-defined tune id + * + * @param[in] tune string describing Xeve encoder tune (psnr, zerolatency) + * @return XEVE pre-defined profile on success, negative value on failure + */ +static int get_tune_id(const char *tune) +{ + if((!strcmp(tune, "psnr"))) + return XEVE_TUNE_PSNR; + else if (!strcmp(tune, "zerolatency")) + return XEVE_TUNE_ZEROLATENCY; + else + return AVERROR(EINVAL); +} + +/** + * Convert FFmpeg pixel format (AVPixelFormat) to XEVE pre-defined color format + * + * @param[in] av_pix_fmt pixel format (@see https://ffmpeg.org/doxygen/trunk/pixfmt_8h.html#a9a8e335cf3be472042bc9f0cf80cd4c5) + * @param[out] xeve_col_fmt XEVE pre-defined color format (@see xeve.h) + * + * @return 0 on success, negative value on failure + */ +static int xeve_color_fmt(enum AVPixelFormat av_pix_fmt, int *xeve_col_fmt) +{ + switch (av_pix_fmt) { + case AV_PIX_FMT_YUV420P: + *xeve_col_fmt = XEVE_CF_YCBCR420; + break; + case AV_PIX_FMT_YUV420P10: + *xeve_col_fmt = XEVE_CF_YCBCR420; + break; + default: + *xeve_col_fmt = XEVE_CF_UNKNOWN; + return AVERROR_INVALIDDATA; + } + + return 0; +} + +/** + * Convert FFmpeg pixel format (AVPixelFormat) into XEVE pre-defined color space + * + * @param[in] px_fmt pixel format (@see https://ffmpeg.org/doxygen/trunk/pixfmt_8h.html#a9a8e335cf3be472042bc9f0cf80cd4c5) + * + * @return XEVE pre-defined color space (@see xeve.h) on success, XEVE_CF_UNKNOWN on failure + */ +static int xeve_color_space(enum AVPixelFormat av_pix_fmt) +{ + /* color space of input image */ + int cs = XEVE_CF_UNKNOWN; + + switch (av_pix_fmt) { + case AV_PIX_FMT_YUV420P: + cs = XEVE_CS_YCBCR420; + break; + case AV_PIX_FMT_YUV420P10: +#if AV_HAVE_BIGENDIAN + cs = XEVE_CS_SET(XEVE_CF_YCBCR420, 10, 1); +#else + cs = XEVE_CS_YCBCR420_10LE; +#endif + + break; + default: + cs = XEVE_CF_UNKNOWN; + break; + } + + return cs; +} + +static int kbps_str_to_int(char *str) +{ + int kbps = 0; + char *saveptr = NULL; + if (strchr(str, 'K') || strchr(str, 'k')) { + char *tmp = av_strtok(str, "Kk ", &saveptr); + kbps = (int)(strtof(tmp, NULL)); + } else if (strchr(str, 'M') || strchr(str, 'm')) { + char *tmp = av_strtok(str, "Mm ", &saveptr); + kbps = (int)(strtof(tmp, NULL) * 1000); + } else { + kbps = strtol(str, NULL, 10); + } + + return kbps; +} + +/** + * Parse : separated list of key=value parameters + * + * @param[in] avctx context for logger + * @param[in] key + * @param[in] value + * @param[out] cdsc contains all Xeve MPEG-5 EVC encoder encoder parameters that + * should be initialized before the encoder is use + * + * @return 0 on success, negative value on failure + */ +static int parse_xeve_params(AVCodecContext *avctx, const char *key, const char *value, XEVE_CDSC* cdsc) +{ + XeveContext *xectx = NULL; + xectx = avctx->priv_data; + + if(!key) { + av_log(avctx, AV_LOG_ERROR, "Ivalid argument: key string is NULL\n"); + return XEVE_PARAM_BAD_VALUE; + } + if(!value) { + if (strcmp(key, "hash") == 0) { + xectx->hash = 1; + av_log(avctx, AV_LOG_INFO, "embedding signature is enabled\n"); + } else { + av_log(avctx, AV_LOG_ERROR, "Ivalid argument: value string is NULL\n"); + return XEVE_PARAM_BAD_VALUE; + } + } else if (strcmp(key, "vbv-bufsize") == 0 ) { + cdsc->param.vbv_bufsize = kbps_str_to_int((char *)value); + av_log(avctx, AV_LOG_INFO, "VBV buffer size: %dkbits\n", cdsc->param.vbv_bufsize); + } else { + av_log(avctx, AV_LOG_ERROR, "Unknown xeve codec option: %s\n", key); + return XEVE_PARAM_BAD_NAME; + } + + return 0; +} + +/** + * The function returns a pointer to the object of the XEVE_CDSC type. + * XEVE_CDSC contains all encoder parameters that should be initialized before the encoder is used. + * + * The field values of the XEVE_CDSC structure are populated based on: + * - the corresponding field values of the AvCodecConetxt structure, + * - the xeve encoder specific option values, + * (the full list of options available for xeve encoder is displayed after executing the command ./ffmpeg --help encoder = libxeve) + * - and the xeve encoder options specified as a list of key-value pairs following the xeve-params option + * + * The order of processing input data and populating the XEVE_CDSC structure + * 1) first, the fields of the AVCodecContext structure corresponding to the provided input options are processed, + * (i.e -pix_fmt yuv420p -s:v 1920x1080 -r 30 -profile:v 0) + * 2) then xeve-specific options added as AVOption to the xeve AVCodec implementation + * (i.e -threads_cnt 3 -preset 0) + * 3) finally, the options specified after the xeve-params option as the parameter list of type key value are processed + * (i.e -xeve-params "m=2:q=17") + * + * Keep in mind that, there are options that can be set in different ways. + * In this case, please follow the above-mentioned order of processing. + * The most recent assignments overwrite the previous values. + * + * @param[in] avctx codec context (AVCodecContext) + * @param[out] cdsc contains all Xeve MPEG-5 EVC encoder encoder parameters that should be initialized before the encoder is use + * + * @return 0 on success, negative error code on failure + */ +static int get_conf(AVCodecContext *avctx, XEVE_CDSC *cdsc) +{ + XeveContext *xectx = NULL; + int ret; + + xectx = avctx->priv_data; + xectx->hash = 0; + + /* initialize xeve_param struct with default values */ + ret = xeve_param_default(&cdsc->param); + if (XEVE_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Cannot set_default parameter\n"); + return AVERROR_EXTERNAL; + } + + /* read options from AVCodecContext */ + if(avctx->width > 0) + cdsc->param.w = xectx->width_luma = avctx->width; + + if(avctx->height > 0) + cdsc->param.h = xectx->height_luma = avctx->height; + + if(avctx->framerate.num > 0) { + // fps can be float number, but xeve API doesn't support it + cdsc->param.fps = (int)(((float)avctx->framerate.num / avctx->framerate.den) + 0.5); + } + + if(avctx->gop_size >= 0) { // GOP size (key-frame interval) + cdsc->param.keyint = avctx->gop_size; // 0: only one I-frame at the first time; 1: every frame is coded in I-frame + } + + if (avctx->max_b_frames == 0 || avctx->max_b_frames == 1 || avctx->max_b_frames == 3 || + avctx->max_b_frames == 7 || avctx->max_b_frames == 15) { // number of b-frames + cdsc->param.bframes = avctx->max_b_frames; + } else { + av_log(avctx, AV_LOG_ERROR, "Incorrect value for maximum number of B frames: (%d) \n" + "Acceptable values for bf option (maximum number of B frames) are 0,1,3,7 or 15\n", avctx->max_b_frames); + return AVERROR_INVALIDDATA; + } + + if (avctx->level >= 0) + cdsc->param.level_idc = avctx->level; + + if (avctx->rc_buffer_size > 0) { // VBV buf size + cdsc->param.vbv_bufsize = (int)(avctx->rc_buffer_size / 1000); + } + + // rc_type: Rate control type [ 0(CQP) / 1(ABR) / 2(CRF) ] + if (avctx->bit_rate > 0) { + if (avctx->bit_rate / 1000 > INT_MAX || avctx->rc_max_rate / 1000 > INT_MAX) { + av_log(avctx, AV_LOG_ERROR, "Not supported bitrate bit_rate and rc_max_rate > %d000\n", INT_MAX); + return AVERROR_INVALIDDATA; + } + cdsc->param.bitrate = (int)(avctx->bit_rate / 1000); + cdsc->param.rc_type = XEVE_RC_ABR; + } + + if (xectx->op_crf >= 0) { + cdsc->param.crf = xectx->op_crf; + cdsc->param.rc_type = XEVE_RC_CRF; + } + + if(avctx->thread_count <= 0) { + int cpu_count = av_cpu_count(); + cdsc->param.threads = (cpu_count < XEVE_MAX_THREADS) ? cpu_count : XEVE_MAX_THREADS; + } else if(avctx->thread_count > XEVE_MAX_THREADS) + cdsc->param.threads = XEVE_MAX_THREADS; + else + cdsc->param.threads = avctx->thread_count; + + + xeve_color_fmt(avctx->pix_fmt, &xectx->color_format); + +#if AV_HAVE_BIGENDIAN + cdsc->param.cs = XEVE_CS_SET(xectx->color_format, cdsc->param.codec_bit_depth, 1); +#else + cdsc->param.cs = XEVE_CS_SET(xectx->color_format, cdsc->param.codec_bit_depth, 0); +#endif + + cdsc->max_bs_buf_size = MAX_BS_BUF; + + if(avctx->profile == FF_PROFILE_EVC_BASELINE) + xectx->profile_id = XEVE_PROFILE_BASELINE; + else if(avctx->profile == FF_PROFILE_EVC_MAIN) + xectx->profile_id = XEVE_PROFILE_MAIN; + else { + av_log(avctx, AV_LOG_ERROR, "Unknown encoder profile (%d)\n" + "Acceptable values for profile option are 0 and 1 (0: baseline profile; 1: main profile)\n", avctx->profile); + return AVERROR_INVALIDDATA; + } + + if (xectx->op_preset) { // preset + xectx->preset_id = get_preset_id(xectx->op_preset); + } + + if (xectx->op_tune) { // tune + xectx->tune_id = get_tune_id(xectx->op_tune); + } + + ret = xeve_param_ppt(&cdsc->param, xectx->profile_id, xectx->preset_id, xectx->tune_id); + if (XEVE_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Cannot set profile(%d), preset(%d), tune(%d)\n", xectx->profile_id, xectx->preset_id, xectx->tune_id); + return AVERROR_EXTERNAL; + } + + /* parse : separated list of key=value parameters and set values for created descriptor (XEVE_CDSC) */ + { + const AVDictionaryEntry *en = NULL; + + // Start to parse xeve_params + while ((en = av_dict_get(xectx->xeve_params, "", en, AV_DICT_IGNORE_SUFFIX))) { + int parse_ret = parse_xeve_params(avctx, en->key, en->value, cdsc); + + switch (parse_ret) { + case XEVE_PARAM_BAD_NAME: + av_log(avctx, AV_LOG_WARNING, "Unknown option: %s.\n", en->key); + break; + case XEVE_PARAM_BAD_VALUE: + av_log(avctx, AV_LOG_WARNING, "Invalid value for %s: %s.\n", en->key, en->value); + break; + default: + break; + } + } + } + + return 0; +} + +/** + * Set XEVE_CFG_SET_USE_PIC_SIGNATURE for encoder + * + * @param[in] logger context + * @param[in] id XEVE encodec instance identifier + * @param[in] ctx the structure stores all the states associated with the instance of Xeve MPEG-5 EVC encoder + * + * @return 0 on success, negative error code on failure + */ +static int set_extra_config(AVCodecContext* avctx, XEVE id, XeveContext *ctx) +{ + int ret, size, value; + size = 4; + + // embed SEI messages identifying encoder parameters and command line arguments + // - 0: off\n" + // - 1: emit sei info" + // + // SEI - Supplemental enhancement information contains information + // that is not necessary to decode the samples of coded pictures from VCL NAL units. + // Some SEI message information is required to check bitstream conformance + // and for output timing decoder conformance. + // @see ISO_IEC_23094-1_2020 7.4.3.5 + // @see ISO_IEC_23094-1_2020 Annex D + ret = xeve_config(id, XEVE_CFG_SET_SEI_CMD, &value, &size); // sei_cmd_info + if (XEVE_FAILED(ret)) + { + av_log(avctx, AV_LOG_ERROR, "Failed to set config for sei command info messages\n"); + return AVERROR_EXTERNAL; + } + + if(ctx->hash) { + value = 1; + ret = xeve_config(id, XEVE_CFG_SET_USE_PIC_SIGNATURE, &value, &size); + if(XEVE_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to set config for picture signature\n"); + return AVERROR_EXTERNAL; + } + } + + return 0; +} + +/** + * @brief Switch encoder to bumping mode + * + * @param id XEVE encodec instance identifier + * @return 0 on success, negative error code on failure + */ +static int setup_bumping(XEVE id) +{ + int val = 1; + int size = sizeof(int); + if(XEVE_FAILED(xeve_config(id, XEVE_CFG_SET_FORCE_OUT, (void *)(&val), &size))) + return AVERROR_EXTERNAL; + + return 0; +} + +/** + * @brief Initialize eXtra-fast Essential Video Encoder codec + * Create an encoder instance and allocate all the needed resources + * + * @param avctx codec context + * @return 0 on success, negative error code on failure + */ +static av_cold int libxeve_init(AVCodecContext *avctx) +{ + XeveContext *xectx = avctx->priv_data; + unsigned char *bs_buf = NULL; + int i; + int shift_h = 0; + int shift_v = 0; + XEVE_IMGB *imgb = NULL; + int ret = 0; + + XEVE_CDSC *cdsc = &(xectx->cdsc); + + /* allocate bitstream buffer */ + bs_buf = av_malloc(MAX_BS_BUF); + if(bs_buf == NULL) { + av_log(avctx, AV_LOG_ERROR, "Cannot allocate bitstream buffer\n"); + return AVERROR(ENOMEM); + } + xectx->bitb.addr = bs_buf; + xectx->bitb.bsize = MAX_BS_BUF; + + /* read configurations and set values for created descriptor (XEVE_CDSC) */ + if ((ret = get_conf(avctx, cdsc)) != 0) { + av_log(avctx, AV_LOG_ERROR, "Cannot get configuration\n"); + return AVERROR(EINVAL); + } + + if ((ret = xeve_param_check(&cdsc->param)) != 0) { + av_log(avctx, AV_LOG_ERROR, "Invalid configuration\n"); + return AVERROR(EINVAL); + } + + /* create encoder */ + xectx->id = xeve_create(cdsc, NULL); + if(xectx->id == NULL) { + av_log(avctx, AV_LOG_ERROR, "Cannot create XEVE encoder\n"); + return AVERROR_EXTERNAL; + } + + if((ret = set_extra_config(avctx, xectx->id, xectx))!=0) { + av_log(avctx, AV_LOG_ERROR, "Cannot set extra configuration\n"); + return AVERROR(EINVAL); + } + + if((ret = av_pix_fmt_get_chroma_sub_sample(avctx->pix_fmt, &shift_h, &shift_v)) != 0) { + av_log(avctx, AV_LOG_ERROR, "Failed to get chroma shift\n"); + return AVERROR(EINVAL); + } + + // YUV format explanation + // shift_h == 1 && shift_v == 1 : YUV420 + // shift_h == 1 && shift_v == 0 : YUV422 + // shift_h == 0 && shift_v == 0 : YUV444 + // + xectx->width_chroma = AV_CEIL_RSHIFT(xectx->width_luma, shift_h); + xectx->height_chroma = AV_CEIL_RSHIFT(xectx->height_luma, shift_v); + + /* set default values for input image buffer */ + imgb = &xectx->imgb; + imgb->cs = xeve_color_space(avctx->pix_fmt); + imgb->np = 3; /* only for yuv420p, yuv420ple */ + + for (i = 0; i < imgb->np; i++) + imgb->x[i] = imgb->y[i] = 0; + + imgb->w[0] = imgb->aw[0] = xectx->width_luma; + imgb->w[1] = imgb->w[2] = imgb->aw[1] = imgb->aw[2] = xectx->width_chroma; + imgb->h[0] = imgb->ah[0] = xectx->height_luma; + imgb->h[1] = imgb->h[2] = imgb->ah[1] = imgb->ah[2] = xectx->height_chroma; + + xectx->state = STATE_ENCODING; + + return 0; +} + +/** + * Encode raw data frame into EVC packet + * + * @param[in] avctx codec context + * @param[out] pkt output AVPacket containing encoded data + * @param[in] frame AVFrame containing the raw data to be encoded + * @param[out] got_packet encoder sets to 0 or 1 to indicate that a + * non-empty packet was returned in pkt + * + * @return 0 on success, negative error code on failure + */ +static int libxeve_encode(AVCodecContext *avctx, AVPacket *avpkt, + const AVFrame *frame, int *got_packet) +{ + XeveContext *xectx = avctx->priv_data; + int ret = -1; + int xeve_cs; + + if(xectx->state == STATE_SKIPPING && frame ) { + xectx->state = STATE_ENCODING; // Entering encoding process + } else if(xectx->state == STATE_ENCODING && frame == NULL) { + if (setup_bumping(xectx->id) == 0) + xectx->state = STATE_BUMPING; // Entering bumping process + else { + av_log(avctx, AV_LOG_ERROR, "Failed to setup bumping\n"); + xectx->state = STATE_SKIPPING; + } + } + + if(xectx->state == STATE_ENCODING) { + const AVPixFmtDescriptor *pixel_fmt_desc = av_pix_fmt_desc_get (frame->format); + if(!pixel_fmt_desc) { + av_log(avctx, AV_LOG_ERROR, "Invalid pixel format descriptor for pixel format: %s\n", av_get_pix_fmt_name(avctx->pix_fmt)); + return AVERROR_INVALIDDATA; + } + + xeve_cs = xeve_color_space(avctx->pix_fmt); + if(xeve_cs != XEVE_CS_YCBCR420 && xeve_cs != XEVE_CS_YCBCR420_10LE) { + av_log(avctx, AV_LOG_ERROR, "Invalid pixel format: %s\n", av_get_pix_fmt_name(avctx->pix_fmt)); + return AVERROR_INVALIDDATA; + } + + { + int i; + XEVE_IMGB *imgb = NULL; + + imgb = &xectx->imgb; + + for (i = 0; i < imgb->np; i++) { + imgb->a[i] = frame->data[i]; + imgb->s[i] = frame->linesize[i]; + } + + if(xectx->id == NULL) { + av_log(avctx, AV_LOG_ERROR, "Invalid XEVE encoder\n"); + return AVERROR_INVALIDDATA; + } + + imgb->ts[0] = frame->pts; + imgb->ts[1] = 0; + + /* push image to encoder */ + ret = xeve_push(xectx->id, imgb); + if(XEVE_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "xeve_push() failed\n"); + return AVERROR_EXTERNAL; + } + } + } + if(xectx->state == STATE_ENCODING || xectx->state == STATE_BUMPING) { + + /* encoding */ + ret = xeve_encode(xectx->id, &(xectx->bitb), &(xectx->stat)); + if(XEVE_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "xeve_encode() failed\n"); + return AVERROR_EXTERNAL; + } + + /* store bitstream */ + if (ret == XEVE_OK_OUT_NOT_AVAILABLE) { // Return OK but picture is not available yet + *got_packet = 0; + return 0; + } else if(ret == XEVE_OK) { + int av_pic_type; + + if(xectx->stat.write > 0) { + + ret = ff_get_encode_buffer(avctx, avpkt, xectx->stat.write, 0); + if (ret < 0) { + return ret; + } + + memcpy(avpkt->data, xectx->bitb.addr, xectx->stat.write); + + avpkt->pts = xectx->bitb.ts[0]; + avpkt->dts = xectx->bitb.ts[1]; + + switch(xectx->stat.stype) { + case XEVE_ST_I: + av_pic_type = AV_PICTURE_TYPE_I; + avpkt->flags |= AV_PKT_FLAG_KEY; + break; + case XEVE_ST_P: + av_pic_type = AV_PICTURE_TYPE_P; + break; + case XEVE_ST_B: + av_pic_type = AV_PICTURE_TYPE_B; + break; + case XEVE_ST_UNKNOWN: + av_log(avctx, AV_LOG_ERROR, "Unknown slice type\n"); + return AVERROR_INVALIDDATA; + } + + ff_side_data_set_encoder_stats(avpkt, xectx->stat.qp * FF_QP2LAMBDA, NULL, 0, av_pic_type); + + *got_packet = 1; + } + } else if (ret == XEVE_OK_NO_MORE_FRM) { + // Return OK but no more frames + return 0; + } else { + av_log(avctx, AV_LOG_ERROR, "Invalid return value: %d\n", ret); + return AVERROR_EXTERNAL; + } + } else { + av_log(avctx, AV_LOG_ERROR, "Udefined encoder state\n"); + return AVERROR_INVALIDDATA; + } + + return 0; +} + +/** + * Destroy the encoder and release all the allocated resources + * + * @param avctx codec context + * @return 0 on success, negative error code on failure + */ +static av_cold int libxeve_close(AVCodecContext *avctx) +{ + XeveContext *xectx = avctx->priv_data; + + xeve_delete(xectx->id); + + av_free(xectx->bitb.addr); /* release bitstream buffer */ + + return 0; +} + +#define OFFSET(x) offsetof(XeveContext, x) +#define VE AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_ENCODING_PARAM + +static const enum AVPixelFormat supported_pixel_formats[] = { + AV_PIX_FMT_YUV420P, + AV_PIX_FMT_YUV420P10, + AV_PIX_FMT_NONE +}; + +// Example of using: ./ffmpeg -xeve-params "m=2:q=17" +// Consider using following options (./ffmpeg --help encoder=libxeve) +// +static const AVOption xeve_options[] = { + { "preset", "Encoding preset for setting encoding speed [fast, medium, slow, placebo]", OFFSET(op_preset), AV_OPT_TYPE_STRING, { .str = "medium" }, 0, 0, VE }, + { "tune", "Tuneing parameter for special purpose operation [psnr, zerolatency]", OFFSET(op_tune), AV_OPT_TYPE_STRING, { 0 }, 0, 0, VE}, + { "qp", "quantization parameter qp <0..51> [default: 32]", OFFSET(op_qp), AV_OPT_TYPE_INT, { .i64 = 32 }, 0, 51, VE }, + { "crf", "constant rate factor <-1..51> [default: 32]", OFFSET(op_crf), AV_OPT_TYPE_INT, { .i64 = -1 }, -1, 51, VE }, + { "xeve-params", "override the xeve configuration using a : separated list of key=value parameters", OFFSET(xeve_params), AV_OPT_TYPE_DICT, { 0 }, 0, 0, VE }, + { NULL } +}; + +static const AVClass xeve_class = { + .class_name = "libxeve", + .item_name = av_default_item_name, + .option = xeve_options, + .version = LIBAVUTIL_VERSION_INT, +}; + +/** + * libavcodec generic global options, which can be set on all the encoders and decoders + * @see https://www.ffmpeg.org/ffmpeg-codecs.html#Codec-Options + */ +static const FFCodecDefault xeve_defaults[] = { + { "b", "0" }, // bitrate + { "g", "0" }, // gop_size (key-frame interval 0: only one I-frame at the first time; 1: every frame is coded in I-frame) + { "bf", "15"}, // bframes (0: no B-frames) + { "profile", "0"}, // encoder codec profile (0: baselie; 1: main) + { "threads", "0"}, // number of threads to be used (0: automatically select the number of threads to set) + { NULL }, +}; + +const FFCodec ff_libxeve_encoder = { + .p.name = "libxeve", + .p.long_name = NULL_IF_CONFIG_SMALL("libxeve MPEG-5 EVC"), + .p.type = AVMEDIA_TYPE_VIDEO, + .p.id = AV_CODEC_ID_EVC, + .init = libxeve_init, + FF_CODEC_ENCODE_CB(libxeve_encode), + .close = libxeve_close, + .priv_data_size = sizeof(XeveContext), + .p.priv_class = &xeve_class, + .defaults = xeve_defaults, + .p.capabilities = FF_CODEC_CAP_INIT_CLEANUP | AV_CODEC_CAP_DELAY | AV_CODEC_CAP_AUTO_THREADS | AV_CODEC_CAP_DR1, + .p.wrapper_name = "libxeve", + .p.pix_fmts = supported_pixel_formats, +}; diff --git a/libavcodec/parsers.c b/libavcodec/parsers.c index b59388835d..553f7ee23e 100644 --- a/libavcodec/parsers.c +++ b/libavcodec/parsers.c @@ -41,6 +41,7 @@ extern const AVCodecParser ff_dvaudio_parser; extern const AVCodecParser ff_dvbsub_parser; extern const AVCodecParser ff_dvdsub_parser; extern const AVCodecParser ff_dvd_nav_parser; +extern const AVCodecParser ff_evc_parser; extern const AVCodecParser ff_flac_parser; extern const AVCodecParser ff_g723_1_parser; extern const AVCodecParser ff_g729_parser;