From patchwork Mon Aug 1 09:28:57 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dawid Kozinski X-Patchwork-Id: 37059 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a21:1649:b0:8b:613a:194d with SMTP id no9csp2384617pzb; Mon, 1 Aug 2022 02:29:15 -0700 (PDT) X-Google-Smtp-Source: AGRyM1s9zNUqmUmcIlgf2vQMFkpmJDEhFZJIxvNCGSUb7v5SlUaDwaFb+FGqVoU/ThQ9rodEciRR X-Received: by 2002:a17:907:7241:b0:72b:347b:17a1 with SMTP id ds1-20020a170907724100b0072b347b17a1mr12242436ejc.32.1659346155559; Mon, 01 Aug 2022 02:29:15 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1659346155; cv=none; d=google.com; s=arc-20160816; b=cBgpxY7+UXl/zqWWoOteyxRN2H/7NOLBRsdDPBKE4B1Lx3bNYYDsP19pj1pf4KZXBT hkLhcHZTlMaeq5kwPyPSyLFPQGAoNSLH0FJGl9rYue+v9nGPax3yuZd2RKWNr5bT6CBI VLnTDaNYrJDniz1bmu6QSCbes4BBwjZiTfw3uD0khcGka1CGuqz7F1gQ6RA+ukQS1p24 xUQRSUG7Y7RXZM81l6WJ3iOeFByL7ZBfe4QxTfIixa02/Af2tIWGXqJpZZBOtTy2iGVN VcM2QyXqCBhz2EtCJZ/Hormzs4PKVNgv6XaWngRE9OuEMySYhQarq8AcRQKcfNSepUsF g8KQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:references:cms-type:content-language :thread-index:mime-version:message-id:date:to:from:dkim-signature :dkim-filter:delivered-to; bh=LwNe6vya4tf9SkhKgqDE9aJ+rAy6oS2mtbQ5dBDzJ6g=; b=mQVeT90XSwuJxp70VHDIj3H9gAHhDR1JyGOFMCtkPYdYQcaqoBXA8rnH0XBVCifQvV r20nzSlp8sGHPYOwsaYLoHFodbiZErr8F3hWhkUZ8nj7ZeEhVerJNUxq/hGjjNxlEWjX UGPH3QFr3HV0YzU52dBZ4zEcxuqKcP4dEk7GWJagJchelMuaUjb9omdbrph+Yy1AvvaP F0yj31Lbmp9aEBbiPNZBmOZYjjBp2pQAzMwLX19ZaPWWtde4m/nvZmka5o9aNt/zhIqG bmdIT5aR+CYK7NXFLt8I9+A7c6ZSSogf495+GM9ER0QUSROowB0DtykPU+ftzeXORzlz tRGA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@samsung.com header.s=mail20170921 header.b=CEJhy9UD; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=samsung.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id nb39-20020a1709071ca700b0072b3a316cc6si10666794ejc.977.2022.08.01.02.29.14; Mon, 01 Aug 2022 02:29:15 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@samsung.com header.s=mail20170921 header.b=CEJhy9UD; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=samsung.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id DBD5F68BAA7; Mon, 1 Aug 2022 12:29:09 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mailout2.w1.samsung.com (mailout2.w1.samsung.com [210.118.77.12]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id A00F368BA89 for ; Mon, 1 Aug 2022 12:29:02 +0300 (EEST) Received: from eucas1p2.samsung.com (unknown [182.198.249.207]) by mailout2.w1.samsung.com (KnoxPortal) with ESMTP id 20220801092859euoutp021f7d5874e7d3897a2a18be2e284a820a~HLoW-188z1250512505euoutp02R for ; Mon, 1 Aug 2022 09:28:59 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout2.w1.samsung.com 20220801092859euoutp021f7d5874e7d3897a2a18be2e284a820a~HLoW-188z1250512505euoutp02R DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1659346139; bh=8F0u/pTpy9TwxMNESNF+OIaPF8sl/fbK1zh9dHA/bN8=; h=From:To:Subject:Date:References:From; b=CEJhy9UDsDHfsrytg89mjlAPAQkTTP83svYQHhjj47nb22jFbW0eLaZHTzbjBvNh7 eYCYkHMAIcFPCnWKYXwTz856fmzEsck/t1C0P3CR4o5Q0NRNDHU4ilsvD/RH3YYGWW 6yjg090mK2FJsG4dWZ44yrLquuR3aas2vmZ8mdEQ= Received: from eusmges2new.samsung.com (unknown [203.254.199.244]) by eucas1p2.samsung.com (KnoxPortal) with ESMTP id 20220801092859eucas1p2eb37b41d667fa812ba8f9aa861fac40a~HLoW1TxkG1971419714eucas1p2u for ; Mon, 1 Aug 2022 09:28:59 +0000 (GMT) Received: from eucas1p2.samsung.com ( [182.198.249.207]) by eusmges2new.samsung.com (EUCPMTA) with SMTP id 6E.90.10067.BDC97E26; Mon, 1 Aug 2022 10:28:59 +0100 (BST) Received: from eusmtrp2.samsung.com (unknown [182.198.249.139]) by eucas1p2.samsung.com (KnoxPortal) with ESMTPA id 20220801092858eucas1p2eb2d2732543be955865cc083ebf582b3~HLoWV5-b71970819708eucas1p2K for ; Mon, 1 Aug 2022 09:28:58 +0000 (GMT) Received: from eusmgms2.samsung.com (unknown [182.198.249.180]) by eusmtrp2.samsung.com (KnoxPortal) with ESMTP id 20220801092858eusmtrp248d36dca55a641480ca3660b95b577c4~HLoWVPlvO0640506405eusmtrp2e for ; Mon, 1 Aug 2022 09:28:58 +0000 (GMT) X-AuditID: cbfec7f4-dc1ff70000002753-46-62e79cdb093a Received: from eusmtip2.samsung.com ( [203.254.199.222]) by eusmgms2.samsung.com (EUCPMTA) with SMTP id 0E.C6.09038.ADC97E26; Mon, 1 Aug 2022 10:28:58 +0100 (BST) Received: from AMDN3260 (unknown [106.210.132.171]) by eusmtip2.samsung.com (KnoxPortal) with ESMTPA id 20220801092858eusmtip25fe956f0d21df264b33a6bd39312ecd4~HLoVwxwL22591325913eusmtip2G for ; Mon, 1 Aug 2022 09:28:58 +0000 (GMT) From: "Dawid Kozinski" To: Date: Mon, 1 Aug 2022 11:28:57 +0200 Message-ID: <001f01d8a589$20ad4ac0$6207e040$@samsung.com> MIME-Version: 1.0 X-Mailer: Microsoft Outlook 16.0 Thread-Index: Adilhq+8IbVVkR0QRNafaptA+fO6oA== Content-Language: pl X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrBIsWRmVeSWpSXmKPExsWy7djP87q35zxPMlh5WcLi26czzA6MHn8W bWYJYIzisklJzcksSy3St0vgypi7cTJTwf0vLBV/Ps1mb2B8f5m5i5GTQ0LAROL8o3bGLkYu DiGBFYwSV66dZIJwJjFJtE19zwpSJSQwkUni0zu9LkYOsI6Ne6ohapYzSvx7f58JoqaNSeLZ PTEQm01AX2Lmok9sILaIgKzE6n9TwGxhgVCJped2gW1mEVCReHhzLlicV8BS4tPqr6wQtqDE yZlPWEBsZgFtiWULX0NdqiDx8+kyVoiZehLXt51jhagRkbjxqAXsAwmBr+wSLxq2s0E0uEi0 npvIDmELS7w6vgXKlpH4v3M+E8QzxRKH+h0gzBqJQz/SISqsJd42HmcECTMLaEqs36UPEXaU eL/iEgtENZ/EjbeCEAfwSUzaNp0ZIswr0dEmBGGqSPR1ikE0Skk8XTYH6g0PiXufb7NPYFSc heTbWUi+nYXkq1kIJyxgZFnFKJ5aWpybnlpslJdarlecmFtcmpeul5yfu4kRmBhO/zv+ZQfj 8lcf9Q4xMnEwHmKU4GBWEuG94/I8SYg3JbGyKrUoP76oNCe1+BCjNAeLkjhvcuaGRCGB9MSS 1OzU1ILUIpgsEwenVAPTrvm+D6d9/bq16VL/2w2B3jtmuzDyhebvNJdPKelNeS/rX5Ps/PF5 gVjN/FTrWsd9mWsux625+ZUzVebdW8X1Oquqtn5wXTS1iMHw8q9WpiZpK6bEm7fVCvV2xx9o /ZXf9iXAf9Jb9cbTrNMz/qe81ZmjK8yk/ElOR+zJ9s+qecvsFsTq3tn7s6HB/Nrd10eKAuSP tWYdjP/QFTJtaqL/hLXayzm2/7i04VWPv2pm64MNC5j/ziv6JX/c5u6ZTbumvj03b+3f+99v 24UHfe+afn7F52vMW+qU3s6cGV2+YuqZN5m3rvAYXLPSK3M/bJzx43NY0KpJ6Z9UH6XwhPKn iaR8MbxSOctycgKTeBZzhBJLcUaioRZzUXEiAGb3wOF7AwAA X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrBLMWRmVeSWpSXmKPExsVy+t/xe7q35jxPMpjXZmrx7dMZZgdGjz+L NrMEMEbp2RTll5akKmTkF5fYKkUbWhjpGVpa6BmZWOoZGpvHWhmZKunb2aSk5mSWpRbp2yXo ZZydNZOx4NFdloofS6ayNzC+2MPcxcjBISFgIrFxT3UXIxeHkMBSRomnc5cydjFyAsWlJJYu XQRlC0v8udbFBlHUwiRx/vhPNpAEm4C+xMxFn8BsEQFZidX/poDZwgLBEstOX2cBsVkEVCQe 3pwLFucVsJT4tPorK4QtKHFy5hOwGmYBbYneh62MMPayha+ZIRYrSPx8uowVYr6exPVt51gh akQkbjxqYZzAKDALyahZSEbNQjJqFpKWBYwsqxhFUkuLc9Nzi430ihNzi0vz0vWS83M3MQKD fNuxn1t2MK589VHvECMTB+MhRgkOZiUR3jsuz5OEeFMSK6tSi/Lji0pzUosPMZoC/TaRWUo0 OR8YZ3kl8YZmBqaGJmaWBqaWZsZK4ryeBR2JQgLpiSWp2ampBalFMH1MHJxSDUyb9h2zfto1 R+NpA7foh0tlzU13rWas2XNbrTY99fQWXY3eiZUSDav3SWxa9HnfCZ8HHsZul7XlxSdvvXjy oMXyhUJNhq/EYlrPxFn8vpSSyf2KNTcoJOQmZ1Ql7w8LeeHyb3WXO8QNX5gfYGj4yfqqd5Ol w5l6kYa0hrzkyKa6/evv7swKnXj4enz+2Su/GG4uPSawuaH1sf/N8+GPz76X3DJ9/adlnSxf v+XIvWysycjNMxcT00u1NW3pE8lj6PtTs0Erv2PZQ71OpWaZvPt6AvpH/C893/kt00qpxkE1 uk59quxylt7aC6aZRQeY5LVu+yqU/XY6wD/3Qe/FH1FxjLsi5+/P+xuwkJnrqRJLcUaioRZz UXEiAEXBqTP7AgAA X-CMS-MailID: 20220801092858eucas1p2eb2d2732543be955865cc083ebf582b3 X-Msg-Generator: CA X-RootMTR: 20220801092858eucas1p2eb2d2732543be955865cc083ebf582b3 X-EPHeader: CA CMS-TYPE: 201P X-CMS-RootMailID: 20220801092858eucas1p2eb2d2732543be955865cc083ebf582b3 References: Subject: [FFmpeg-devel] [PATCH 1/2] Provided support for MPEG-5 EVC (Essential Video Coding) codec X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: wHFDZ3TMAcFG - Added new entry to codec IDs list - Added new entry to the codec descriptor list - Bumped libavcodec minor version - Changes in Changelog and MAINTAINERS files - Added xeve encoder wrapper - Added xevd dencoder wrapper - Added documentation for xeve and xevd wrappers - Added parser for EVC format - Changes in project configuration file and libavcodec Makefile Signed-off-by: Dawid Kozinski --- Changelog | 3 +- MAINTAINERS | 2 + configure | 8 + doc/decoders.texi | 24 ++ doc/encoders.texi | 92 ++++++ doc/general_contents.texi | 19 ++ libavcodec/Makefile | 3 + libavcodec/allcodecs.c | 2 + libavcodec/avcodec.h | 3 + libavcodec/codec_desc.c | 8 + libavcodec/codec_id.h | 1 + libavcodec/evc_parser.c | 526 +++++++++++++++++++++++++++++++++ libavcodec/libxevd.c | 436 ++++++++++++++++++++++++++++ libavcodec/libxeve.c | 597 ++++++++++++++++++++++++++++++++++++++ libavcodec/parsers.c | 1 + libavcodec/profiles.c | 6 + libavcodec/profiles.h | 1 + libavcodec/version.h | 2 +- 18 files changed, 1732 insertions(+), 2 deletions(-) create mode 100644 libavcodec/evc_parser.c create mode 100644 libavcodec/libxevd.c create mode 100644 libavcodec/libxeve.c diff --git a/Changelog b/Changelog index 40ea7ccecf..5e14f0a8a4 100644 --- a/Changelog +++ b/Changelog @@ -80,7 +80,8 @@ version 5.0: - VideoToolbox ProRes encoder - anlmf audio filter - IMF demuxer (experimental) - +- eXtra-fast Essential Video Encoder (XEVE) +- eXtra-fast Essential Video Decoder (XEVD) version 4.4: - AudioToolbox output device diff --git a/MAINTAINERS b/MAINTAINERS index 274fc89203..4b33d3c1b2 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -205,6 +205,7 @@ Codecs: libvpx* James Zern libxavs.c Stefan Gehrer libxavs2.c Huiwen Ren + libxev*.c, evc_parser.c Dawid Kozinski libzvbi-teletextdec.c Marton Balint lzo.h, lzo.c Reimar Doeffinger mdec.c Michael Niedermayer @@ -425,6 +426,7 @@ Muxers/Demuxers: dv.c Roman Shaposhnik electronicarts.c Peter Ross epafdec.c Paul B Mahol + evcdec.c Dawid Kozinski ffm* Baptiste Coudurier flic.c Mike Melanson flvdec.c Michael Niedermayer diff --git a/configure b/configure index 8c7e8c9d1d..3c12140721 100755 --- a/configure +++ b/configure @@ -291,6 +291,8 @@ External library support: --enable-libwebp enable WebP encoding via libwebp [no] --enable-libx264 enable H.264 encoding via x264 [no] --enable-libx265 enable HEVC encoding via x265 [no] + --enable-libxeve enable EVC encoding via libxeve [no] + --enable-libxevd enable EVC decoding via libxevd [no] --enable-libxavs enable AVS encoding via xavs [no] --enable-libxavs2 enable AVS2 encoding via xavs2 [no] --enable-libxcb enable X11 grabbing using XCB [autodetect] @@ -1874,6 +1876,8 @@ EXTERNAL_LIBRARY_LIST=" libvorbis libvpx libwebp + libxevd + libxeve libxml2 libzimg libzmq @@ -3393,6 +3397,8 @@ libx264rgb_encoder_select="libx264_encoder" libx265_encoder_deps="libx265" libxavs_encoder_deps="libxavs" libxavs2_encoder_deps="libxavs2" +libxevd_decoder_deps="libxevd" +libxeve_encoder_deps="libxeve" libxvid_encoder_deps="libxvid" libzvbi_teletext_decoder_deps="libzvbi" vapoursynth_demuxer_deps="vapoursynth" @@ -6682,6 +6688,8 @@ enabled libx265 && require_pkg_config libx265 x265 x265.h x265_api_get require_cpp_condition libx265 x265.h "X265_BUILD >= 70" enabled libxavs && require libxavs "stdint.h xavs.h" xavs_encoder_encode "-lxavs $pthreads_extralibs $libm_extralibs" enabled libxavs2 && require_pkg_config libxavs2 "xavs2 >= 1.3.0" "stdint.h xavs2.h" xavs2_api_get +enabled libxevd && require_pkg_config libxevd "xevd >= 0.4.0" "xevd.h" xevd_decode +enabled libxeve && require_pkg_config libxeve "xeve >= 0.4.0" "xeve.h" xeve_encode enabled libxvid && require libxvid xvid.h xvid_global -lxvidcore enabled libzimg && require_pkg_config libzimg "zimg >= 2.7.0" zimg.h zimg_get_api_version enabled libzmq && require_pkg_config libzmq "libzmq >= 4.2.1" zmq.h zmq_ctx_new diff --git a/doc/decoders.texi b/doc/decoders.texi index e2fcbf5dc9..0bd9096114 100644 --- a/doc/decoders.texi +++ b/doc/decoders.texi @@ -126,6 +126,30 @@ Set amount of frame threads to use during decoding. The default value is 0 (auto @end table +@section libxevd + +eXtra-fast Essential Video Decoder (XEVD) MPEG-5 EVC decoder wrapper. + +This decoder requires the presence of the libxevd headers and library +during configuration. You need to explicitly configure the build with +@option{--enable-libxevd}. + +The xevd project website is at @url{https://github.com/mpeg5/xevd}. + +@subsection Options + +The following options are supported by the libxevd wrapper. +The xevd-equivalent options or values are listed in parentheses for easy migration. + +To get a more accurate and extensive documentation of the libxevd options, +invoke the command @code{xevd_app --help} or consult the libxevd documentation. + +@table @option +@item threads (@emph{threads}) +Force to use a specific number of threads + +@end table + @section QSV Decoders The family of Intel QuickSync Video decoders (VC1, MPEG-2, H.264, HEVC, diff --git a/doc/encoders.texi b/doc/encoders.texi index 6d73f74196..76f55e6721 100644 --- a/doc/encoders.texi +++ b/doc/encoders.texi @@ -2892,6 +2892,98 @@ ffmpeg -i input -c:v libxavs2 -xavs2-params RdoqLevel=0 output.avs2 @end example @end table +@section libxeve + +eXtra-fast Essential Video Encoder (XEVE) MPEG-5 EVC encoder wrapper. +The xeve-equivalent options or values are listed in parentheses for easy migration. + +This encoder requires the presence of the libxeve headers and library +during configuration. You need to explicitly configure the build with +@option{--enable-libxeve}. + +@float NOTE +Many libxeve encoder options are mapped to FFmpeg global codec options, +while unique encoder options are provided through private options. +Additionally the xeve-params private options allows one to pass a list +of key=value tuples as accepted by the libxeve @code{parse_xeve_params} function. +@end float + +The xeve project website is at @url{https://github.com/mpeg5/xeve}. + +@subsection Options + +The following options are supported by the libxeve wrapper. +The xeve-equivalent options or values are listed in parentheses for easy migration. + +@float NOTE +To reduce the duplication of documentation, only the private options +and some others requiring special attention are documented here. For +the documentation of the undocumented generic options, see +@ref{codec-options,,the Codec Options chapter}. +@end float + +@float NOTE +To get a more accurate and extensive documentation of the libxeve options, +invoke the command @code{xeve_app --help} or consult the libxeve documentation. +@end float + +@table @option +@item b (@emph{bitrate}) +Set target video bitrate in bits/s. +Note that FFmpeg's b option is expressed in bits/s, while xeve's bitrate is in kilobits/s. + +@item bf (@emph{bframes}) +Set the maximum number of B frames (1,3,7,15). + +@item g (@emph{keyint}) +Set the GOP size (I-picture period). + +@item preset (@emph{preset}) +Set the xeve preset. +Set the encoder preset value to determine encoding speed [fast, medium, slow, placebo] + +@item tune (@emph{tune}) +Set the encoder tune parameter [psnr, zerolatency] + +@item profile (@emph{profile}) +Set the encoder profile [0: baselie; 1: main] + +@item crf (@emph{crf}) +Set the quality for constant quality mode. +Constant rate factor <10..49> [default: 32] + +@item qp (@emph{qp}) +Set constant quantization rate control method parameter. +Quantization parameter qp <0..51> [default: 32] + +@item threads (@emph{threads}) +Force to use a specific number of threads + +@item xeve-params +Set xeve options using a list of @var{key}=@var{value} couples separated +by ":". + +For example to specify libxeve encoding options with @option{-xeve-params}: + +@example +ffmpeg -i input -c:v libxeve -xeve-params profile=main:tune=zerolatency:preset=fast:q=17:bframes=15:bitrate=100k output.mp4 +@end example +@end table + +@subsubsection Supported key values for xeve-params + +Set xeve options using a list of @var{key}=@var{value} couples separated + +@table @option + +@item hash +Embed the picture signature (HASH) for conformance checking in decoding, (0: Off, 1: On) + +@item vbv-bufsize +Set the VBV buffer size: Kbits(none,K,k), Mbits(M,m) + +@end table + @section libxvid Xvid MPEG-4 Part 2 encoder wrapper. diff --git a/doc/general_contents.texi b/doc/general_contents.texi index f25c784d3b..d72304dd82 100644 --- a/doc/general_contents.texi +++ b/doc/general_contents.texi @@ -343,6 +343,22 @@ libxavs2 is under the GNU Public License Version 2 or later details), you must upgrade FFmpeg's license to GPL in order to use it. @end float +@section eXtra-fast Essential Video Encoder (XEVE) + +FFmpeg can make use of the XEVE library for EVC video encoding. + +Go to @url{https://github.com/mpeg5/xeve} and follow the instructions for +installing the XEVE library. Then pass @code{--enable-libxeve} to configure to +enable it. + +@section eXtra-fast Essential Video Decoder (XEVD) + +FFmpeg can make use of the XEVD library for EVC video decoding. + +Go to @url{https://github.com/mpeg5/xevd} and follow the instructions for +installing the XEVD library. Then pass @code{--enable-libxevd} to configure to +enable it. + @section ZVBI ZVBI is a VBI decoding library which can be used by FFmpeg to decode DVB @@ -591,6 +607,7 @@ library: @item raw DTS @tab X @tab X @item raw DTS-HD @tab @tab X @item raw E-AC-3 @tab X @tab X +@item raw EVC @tab X @tab X @item raw FLAC @tab X @tab X @item raw GSM @tab @tab X @item raw H.261 @tab X @tab X @@ -928,6 +945,8 @@ following image formats are supported: @item Electronic Arts TQI video @tab @tab X @item Escape 124 @tab @tab X @item Escape 130 @tab @tab X +@item EVC / MPEG-5 Part 1 @tab X @tab X + @tab encoding and decoding supported through external libraries libxeve and libxevd @item FFmpeg video codec #1 @tab X @tab X @tab lossless codec (fourcc: FFV1) @item Flash Screen Video v1 @tab X @tab X diff --git a/libavcodec/Makefile b/libavcodec/Makefile index aff7752856..80adc7d54f 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -1109,6 +1109,8 @@ OBJS-$(CONFIG_LIBX264_ENCODER) += libx264.o OBJS-$(CONFIG_LIBX265_ENCODER) += libx265.o OBJS-$(CONFIG_LIBXAVS_ENCODER) += libxavs.o OBJS-$(CONFIG_LIBXAVS2_ENCODER) += libxavs2.o +OBJS-$(CONFIG_LIBXEVD_DECODER) += libxevd.o +OBJS-$(CONFIG_LIBXEVE_ENCODER) += libxeve.o OBJS-$(CONFIG_LIBXVID_ENCODER) += libxvid.o OBJS-$(CONFIG_LIBZVBI_TELETEXT_DECODER) += libzvbi-teletextdec.o ass.o @@ -1136,6 +1138,7 @@ OBJS-$(CONFIG_DVAUDIO_PARSER) += dvaudio_parser.o OBJS-$(CONFIG_DVBSUB_PARSER) += dvbsub_parser.o OBJS-$(CONFIG_DVD_NAV_PARSER) += dvd_nav_parser.o OBJS-$(CONFIG_DVDSUB_PARSER) += dvdsub_parser.o +OBJS-$(CONFIG_EVC_PARSER) += evc_parser.o OBJS-$(CONFIG_FLAC_PARSER) += flac_parser.o flacdata.o flac.o OBJS-$(CONFIG_G723_1_PARSER) += g723_1_parser.o OBJS-$(CONFIG_G729_PARSER) += g729_parser.o diff --git a/libavcodec/allcodecs.c b/libavcodec/allcodecs.c index 31d2c5979c..8a3ca15d36 100644 --- a/libavcodec/allcodecs.c +++ b/libavcodec/allcodecs.c @@ -798,6 +798,8 @@ extern LIBX264_CONST FFCodec ff_libx264_encoder; #endif extern const FFCodec ff_libx264rgb_encoder; extern FFCodec ff_libx265_encoder; +extern const FFCodec ff_libxeve_encoder; +extern const FFCodec ff_libxevd_decoder; extern const FFCodec ff_libxavs_encoder; extern const FFCodec ff_libxavs2_encoder; extern const FFCodec ff_libxvid_encoder; diff --git a/libavcodec/avcodec.h b/libavcodec/avcodec.h index 60b215d2e9..ba33c985cd 100644 --- a/libavcodec/avcodec.h +++ b/libavcodec/avcodec.h @@ -1671,6 +1671,9 @@ typedef struct AVCodecContext { #define FF_PROFILE_KLVA_SYNC 0 #define FF_PROFILE_KLVA_ASYNC 1 +#define FF_PROFILE_EVC_BASELINE 0 +#define FF_PROFILE_EVC_MAIN 1 + /** * level * - encoding: Set by user. diff --git a/libavcodec/codec_desc.c b/libavcodec/codec_desc.c index fdcf8198fe..ebeddd5fa6 100644 --- a/libavcodec/codec_desc.c +++ b/libavcodec/codec_desc.c @@ -1900,6 +1900,14 @@ static const AVCodecDescriptor codec_descriptors[] = { .long_name = NULL_IF_CONFIG_SMALL("HDR (Radiance RGBE format) image"), .props = AV_CODEC_PROP_INTRA_ONLY | AV_CODEC_PROP_LOSSY, }, + { + .id = AV_CODEC_ID_EVC, + .type = AVMEDIA_TYPE_VIDEO, + .name = "evc", + .long_name = NULL_IF_CONFIG_SMALL("MPEG-5 EVC (Essential Video Coding)"), + .props = AV_CODEC_PROP_LOSSY | AV_CODEC_PROP_REORDER, + .profiles = NULL_IF_CONFIG_SMALL(ff_evc_profiles), + }, /* various PCM "codecs" */ { diff --git a/libavcodec/codec_id.h b/libavcodec/codec_id.h index 27bf68ec1d..cebbf63245 100644 --- a/libavcodec/codec_id.h +++ b/libavcodec/codec_id.h @@ -313,6 +313,7 @@ enum AVCodecID { AV_CODEC_ID_QOI, AV_CODEC_ID_PHM, AV_CODEC_ID_RADIANCE_HDR, + AV_CODEC_ID_EVC, /* various PCM "codecs" */ AV_CODEC_ID_FIRST_AUDIO = 0x10000, ///< A dummy id pointing at the start of audio codecs diff --git a/libavcodec/evc_parser.c b/libavcodec/evc_parser.c new file mode 100644 index 0000000000..b2136a6b45 --- /dev/null +++ b/libavcodec/evc_parser.c @@ -0,0 +1,526 @@ +/* + * EVC format parser + * + * Copyright (C) 2021 Dawid Kozinski + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include + +#include "libavutil/common.h" + +#include "parser.h" +#include "golomb.h" + +// The length field that indicates the length in bytes of the following NAL unit is configured to be of 4 bytes +#define EVC_NAL_UNIT_LENGTH_BYTE (4) /* byte */ + +#define EVC_NAL_HEADER_SIZE (2) /* byte */ +#define MAX_SPS_CNT (16) /* defined value in EVC standard */ + +// NALU types +// @see ISO_IEC_23094-1_2020 7.4.2.2 NAL unit header semantics +// +#define EVC_NUT_NONIDR (0) /* Coded slice of a non-IDR picture */ +#define EVC_NUT_IDR (1) /* Coded slice of an IDR picture */ +#define EVC_NUT_SPS (24) /* Sequence parameter set */ +#define EVC_NUT_PPS (25) /* Picture paremeter set */ +#define EVC_NUT_APS (26) /* Adaptation parameter set */ +#define EVC_NUT_FD (27) /* Filler data */ +#define EVC_NUT_SEI (28) /* Supplemental enhancement information */ + +// The sturcture reflects SPS RBSP(raw byte sequence payload) layout +// @see ISO_IEC_23094-1 section 7.3.2.1 +// +// The following descriptors specify the parsing process of each element +// u(n) - unsigned integer using n bits +// ue(v) - unsigned integer 0-th order Exp_Golomb-coded syntax element with the left bit first +typedef struct EVCParserSPS { + int sps_seq_parameter_set_id; // ue(v) + int profile_idc; // u(8) + int level_idc; // u(8) + int toolset_idc_h; // u(32) + int toolset_idc_l; // u(32) + int chroma_format_idc; // ue(v) + int pic_width_in_luma_samples; // ue(v) + int pic_height_in_luma_samples; // ue(v) + int bit_depth_luma_minus8; // ue(v) + int bit_depth_chroma_minus8; // ue(v) + + int sps_btt_flag; // u(1) + int log2_ctu_size_minus5; // ue(v) + int log2_min_cb_size_minus2; // ue(v) + int log2_diff_ctu_max_14_cb_size; // ue(v) + int log2_diff_ctu_max_tt_cb_size; // ue(v) + int log2_diff_min_cb_min_tt_cb_size_minus2; // ue(v) + + int sps_suco_flag; // u(1) + int log2_diff_ctu_size_max_suco_cb_size; // ue(v) + int log2_diff_max_suco_min_suco_cb_size; // ue(v) + + int sps_admvp_flag; // u(1) + int sps_affine_flag; // u(1) + int sps_amvr_flag; // u(1) + int sps_dmvr_flag; // u(1) + int sps_mmvd_flag; // u(1) + int sps_hmvp_flag; // u(1) + + int sps_eipd_flag; // u(1) + int sps_ibc_flag; // u(1) + int log2_max_ibc_cand_size_minus2; // ue(v) + + int sps_cm_init_flag; // u(1) + int sps_adcc_flag; // u(1) + + int sps_iqt_flag; // u(1) + int sps_ats_flag; // u(1) + + int sps_addb_flag; // u(1) + int sps_alf_flag; // u(1) + int sps_htdf_flag; // u(1) + int sps_rpl_flag; // u(1) + int sps_pocs_flag; // u(1) + int sps_dquant_flag; // u(1) + int sps_dra_flag; // u(1) + + int log2_max_pic_order_cnt_lsb_minus4; // ue(v) + int log2_sub_gop_length; // ue(v) + int log2_ref_pic_gap_length; // ue(v) + + int max_num_tid0_ref_pics; // ue(v) + + int sps_max_dec_pic_buffering_minus1; // ue(v) + int long_term_ref_pic_flag; // u(1) + int rpl1_same_as_rpl0_flag; // u(1) + int num_ref_pic_list_in_sps[2]; // ue(v) + + int picture_cropping_flag; // u(1) + int picture_crop_left_offset; // ue(v) + int picture_crop_right_offset; // ue(v) + int picture_crop_top_offset; // ue(v) + int picture_crop_bottom_offset; // ue(v) + +} EVCParserSPS; + +typedef struct EVCParserContext { + ParseContext pc; + EVCParserSPS sps[MAX_SPS_CNT]; + int is_avc; + int nal_length_size; + int to_read; + int incomplete_nalu_prefix_read; // The flag is set to 1 when an incomplete NAL unit prefix has been read + + int got_sps; + int got_pps; + int got_sei; + int got_slice; +} EVCParserContext; + +static int get_nalu_type(const uint8_t *bits, int bits_size, AVCodecContext *avctx) +{ + int unit_type_plus1 = 0; + + if(bits_size >= EVC_NAL_HEADER_SIZE) { + unsigned char *p = (unsigned char *)bits; + // forbidden_zero_bit + if ((p[0] & 0x80) != 0) { + av_log(avctx, AV_LOG_ERROR, "Cannot get bitstream information. Malformed bitstream.\n"); + return -1; + } + + // nal_unit_type + unit_type_plus1 = (p[0] >> 1) & 0x3F; + } + + return unit_type_plus1 - 1; +} + +static uint32_t read_nal_unit_length(const uint8_t *bits, int bits_size, AVCodecContext *avctx) +{ + uint32_t nalu_len = 0; + + if(bits_size >= EVC_NAL_UNIT_LENGTH_BYTE) { + + int t = 0; + unsigned char *p = (unsigned char *)bits; + + for(int i = 0; i < EVC_NAL_UNIT_LENGTH_BYTE; i++) + t = (t << 8) | p[i]; + + nalu_len = t; + if(nalu_len == 0) { + av_log(avctx, AV_LOG_ERROR, "Invalid bitstream size!\n"); + return 0; + } + } + + return nalu_len; +} + +// @see ISO_IEC_23094-1 (7.3.2.1 SPS RBSP syntax) +static EVCParserSPS *parse_sps(const uint8_t *bs, int bs_size, EVCParserContext *ev) +{ + GetBitContext gb; + EVCParserSPS *sps; + int sps_seq_parameter_set_id; + + if(init_get_bits8(&gb, bs, bs_size) < 0) + return NULL; + + sps_seq_parameter_set_id = get_ue_golomb(&gb); + + if(sps_seq_parameter_set_id >= MAX_SPS_CNT) + return NULL; + + sps = &ev->sps[sps_seq_parameter_set_id]; + sps->sps_seq_parameter_set_id = sps_seq_parameter_set_id; + + // the Baseline profile is indicated by profile_idc eqal to 0 + // the Main profile is indicated by profile_idc eqal to 1 + sps->profile_idc = get_bits(&gb, 8); + + sps->level_idc = get_bits(&gb, 8); + + skip_bits_long(&gb, 32); /* skip toolset_idc_h */ + skip_bits_long(&gb, 32); /* skip toolset_idc_l */ + + // 0 - monochrome + // 1 - 4:2:0 + // 2 - 4:2:2 + // 3 - 4:4:4 + sps->chroma_format_idc = get_ue_golomb(&gb); + + sps->pic_width_in_luma_samples = get_ue_golomb(&gb); + sps->pic_height_in_luma_samples = get_ue_golomb(&gb); + + sps->bit_depth_luma_minus8 = get_ue_golomb(&gb); + sps->bit_depth_chroma_minus8 = get_ue_golomb(&gb); + + sps->sps_btt_flag = get_bits(&gb, 1); + if(sps->sps_btt_flag) { + sps->log2_ctu_size_minus5 = get_ue_golomb(&gb); + sps->log2_min_cb_size_minus2 = get_ue_golomb(&gb); + sps->log2_diff_ctu_max_14_cb_size = get_ue_golomb(&gb); + sps->log2_diff_ctu_max_tt_cb_size = get_ue_golomb(&gb); + sps->log2_diff_min_cb_min_tt_cb_size_minus2 = get_ue_golomb(&gb); + } + + sps->sps_suco_flag = get_bits(&gb, 1); + if(sps->sps_suco_flag) { + sps->log2_diff_ctu_size_max_suco_cb_size = get_ue_golomb(&gb); + sps->log2_diff_max_suco_min_suco_cb_size = get_ue_golomb(&gb); + } + + sps->sps_admvp_flag = get_bits(&gb, 1); + if(sps->sps_admvp_flag) { + sps->sps_affine_flag = get_bits(&gb, 1); + sps->sps_amvr_flag = get_bits(&gb, 1); + sps->sps_dmvr_flag = get_bits(&gb, 1); + sps->sps_mmvd_flag = get_bits(&gb, 1); + sps->sps_hmvp_flag = get_bits(&gb, 1); + } + + sps->sps_eipd_flag = get_bits(&gb, 1); + if(sps->sps_eipd_flag) { + sps->sps_ibc_flag = get_bits(&gb, 1); + if(sps->sps_ibc_flag) + sps->log2_max_ibc_cand_size_minus2 = get_ue_golomb(&gb); + } + + sps->sps_cm_init_flag = get_bits(&gb, 1); + if(sps->sps_cm_init_flag) + sps->sps_adcc_flag = get_bits(&gb, 1); + + sps->sps_iqt_flag = get_bits(&gb, 1); + if(sps->sps_iqt_flag) + sps->sps_ats_flag = get_bits(&gb, 1); + + sps->sps_addb_flag = get_bits(&gb, 1); + sps->sps_alf_flag = get_bits(&gb, 1); + sps->sps_htdf_flag = get_bits(&gb, 1); + sps->sps_rpl_flag = get_bits(&gb, 1); + sps->sps_pocs_flag = get_bits(&gb, 1); + sps->sps_dquant_flag = get_bits(&gb, 1); + sps->sps_dra_flag = get_bits(&gb, 1); + + if(sps->sps_pocs_flag) + sps->log2_max_pic_order_cnt_lsb_minus4 = get_ue_golomb(&gb); + + if(!sps->sps_pocs_flag || !sps->sps_rpl_flag) { + sps->log2_sub_gop_length = get_ue_golomb(&gb); + if(sps->log2_sub_gop_length == 0) + sps->log2_ref_pic_gap_length = get_ue_golomb(&gb); + } + + return sps; +} + +static int parse_nal_units(AVCodecParserContext *s, const uint8_t *bs, + int bs_size, AVCodecContext *avctx) +{ + EVCParserContext *ev = s->priv_data; + int nalu_type, nalu_size; + unsigned char *bits = (unsigned char *)bs; + int bits_size = bs_size; + + avctx->codec_id = AV_CODEC_ID_EVC; + + nalu_size = read_nal_unit_length(bits, bits_size, avctx); + if(nalu_size == 0) { + av_log(avctx, AV_LOG_ERROR, "Invalid NAL unit size: (%d)\n", nalu_size); + return -1; + } + + bits += EVC_NAL_UNIT_LENGTH_BYTE; + bits_size -= EVC_NAL_UNIT_LENGTH_BYTE; + + nalu_type = get_nalu_type(bits, bits_size, avctx); + + bits += EVC_NAL_HEADER_SIZE; + bits_size -= EVC_NAL_HEADER_SIZE; + + if (nalu_type == EVC_NUT_SPS) { // NAL Unit type: SPS (Sequence Parameter Set) + EVCParserSPS *sps; + + sps = parse_sps(bits, bits_size, ev); + + avctx->coded_width = sps->pic_width_in_luma_samples; + avctx->coded_height = sps->pic_height_in_luma_samples; + avctx->width = sps->pic_width_in_luma_samples; + avctx->height = sps->pic_height_in_luma_samples; + + if(sps->profile_idc == 0) avctx->profile = FF_PROFILE_EVC_BASELINE; + else if (sps->profile_idc == 1) avctx->profile = FF_PROFILE_EVC_MAIN; + else { + av_log(avctx, AV_LOG_ERROR, "Not supported profile (%d)\n", sps->profile_idc); + return -1; + } + + // Currently XEVD decoder supports ony YCBCR420_10LE chroma format for EVC stream + switch(sps->chroma_format_idc) { + case 0: /* YCBCR400_10LE */ + av_log(avctx, AV_LOG_ERROR, "YCBCR400_10LE: Not supported chroma format\n"); + avctx->pix_fmt = AV_PIX_FMT_GRAY10LE; + return -1; + case 1: /* YCBCR420_10LE */ + avctx->pix_fmt = AV_PIX_FMT_YUV420P10LE; + break; + case 2: /* YCBCR422_10LE */ + av_log(avctx, AV_LOG_ERROR, "YCBCR422_10LE: Not supported chroma format\n"); + avctx->pix_fmt = AV_PIX_FMT_YUV422P10LE; + return -1; + case 3: /* YCBCR444_10LE */ + av_log(avctx, AV_LOG_ERROR, "YCBCR444_10LE: Not supported chroma format\n"); + avctx->pix_fmt = AV_PIX_FMT_YUV444P10LE; + return -1; + default: + avctx->pix_fmt = AV_PIX_FMT_NONE; + av_log(avctx, AV_LOG_ERROR, "Unknown supported chroma format\n"); + return -1; + } + + // The current implementation of parse_sps function doesn't handle VUI parameters parsing, + // so at the moment it's impossible to initialize has_b_frames and max_b_frames AVCodecContex fields here. + // Currently, initialization of has_b_frames and max_b_frames AVCodecContex fields have been moved to + // libxevd_decode function where we can use xevd_config function being a part of xevd library API + // to get the needed information. + // However, if it will be needed, parse_sps function should be extended to handle VUI parameters parsing + // and the following lines should be used to initialize has_b_frames and max_b_frames fields of the AVCodecContex. + // + // sps->vui_parameters.num_reorder_pics + // if (sps->bitstream_restriction_flag && sps->vui_parameters.num_reorder_pics) { + // avctx->has_b_frames = sps->vui_parameters.num_reorder_pics; + // } + + ev->got_sps = 1; + + } else if (nalu_type == EVC_NUT_PPS) // NAL Unit type: PPS (Video Parameter Set) + ev->got_pps = 1; + else if(nalu_type == EVC_NUT_SEI) // NAL unit type: SEI (Supplemental Enhancement Information) + ev->got_sei = 1; + else if (nalu_type == EVC_NUT_IDR || nalu_type == EVC_NUT_NONIDR) // NAL Unit type: Coded slice of a IDR or non-IDR picture + ev->got_slice++; + else { + av_log(avctx, AV_LOG_ERROR, "Invalid NAL unit type: %d\n", nalu_type); + return -1; + } + + return 0; +} + +/** + * Find the end of the current frame in the bitstream. + * @return the position of the first byte of the next frame, or END_NOT_FOUND + */ +static int evc_find_frame_end(AVCodecParserContext *s, const uint8_t *buf, + int buf_size, AVCodecContext *avctx) +{ + EVCParserContext *ev = s->priv_data; + + if(!ev->to_read) { + int nal_unit_size = 0; + int next = END_NOT_FOUND; + + // This is the case when buffer size is not enough for buffer to store NAL unit length + if(buf_size < EVC_NAL_UNIT_LENGTH_BYTE) { + ev->to_read = EVC_NAL_UNIT_LENGTH_BYTE; + ev->nal_length_size = buf_size; + ev->incomplete_nalu_prefix_read = 1; + + return END_NOT_FOUND; + } + + nal_unit_size = read_nal_unit_length(buf, buf_size, avctx); + ev->nal_length_size = EVC_NAL_UNIT_LENGTH_BYTE; + + next = nal_unit_size + EVC_NAL_UNIT_LENGTH_BYTE; + ev->to_read = next; + if(next < buf_size) + return next; + else + return END_NOT_FOUND; + } else if(ev->to_read > buf_size) + return END_NOT_FOUND; + else { + if(ev->incomplete_nalu_prefix_read == 1) { + EVCParserContext *ev = s->priv_data; + ParseContext *pc = &ev->pc; + uint8_t nalu_len[EVC_NAL_UNIT_LENGTH_BYTE] = {0}; + int nal_unit_size = 0; + + // 1. pc->buffer contains previously read bytes of NALU prefix + // 2. buf contains the rest of NAL unit prefix bytes + // + // ~~~~~~~ + // EXAMPLE + // ~~~~~~~ + // + // In the following example we assumed that the number of already read NAL Unit prefix bytes is equal 1 + // + // ---------- + // pc->buffer -> conatins already read bytes + // ---------- + // __ pc->index == 1 + // | + // V + // ------------------------------------------------------- + // | 0 | 1 | 2 | 3 | 4 | ... | N | + // ------------------------------------------------------- + // | 0x00 | 0xXX | 0xXX | 0xXX | 0xXX | ... | 0xXX | + // ------------------------------------------------------- + // + // ---------- + // buf -> contains newly read bytes + // ---------- + // ------------------------------------------------------- + // | 0 | 1 | 2 | 3 | 4 | ... | N | + // ------------------------------------------------------- + // | 0x00 | 0x00 | 0x3C | 0xXX | 0xXX | ... | 0xXX | + // ------------------------------------------------------- + // + for(int i = 0; i < EVC_NAL_UNIT_LENGTH_BYTE; i++) { + if(i < pc->index) + nalu_len[i] = pc->buffer[i]; + else + nalu_len[i] = buf[i - pc->index]; + } + + // ---------- + // nalu_len + // ---------- + // --------------------------------- + // | 0 | 1 | 2 | 3 | + // --------------------------------- + // | 0x00 | 0x00 | 0x00 | 0x3C | + // --------------------------------- + // | NALU LENGTH | + // --------------------------------- + // NAL Unit lenght = 60 (0x0000003C) + + nal_unit_size = read_nal_unit_length(nalu_len, EVC_NAL_UNIT_LENGTH_BYTE, avctx); + + ev->to_read = nal_unit_size + EVC_NAL_UNIT_LENGTH_BYTE - pc->index; + + ev->incomplete_nalu_prefix_read = 0; + + if(ev->to_read > buf_size) + return END_NOT_FOUND; + else + return ev->to_read; + } + return ev->to_read; + } + + return END_NOT_FOUND; +} + +static int evc_parser_init(AVCodecParserContext *s) +{ + EVCParserContext *ev = s->priv_data; + + ev->got_sps = 0; + ev->got_pps = 0; + ev->got_sei = 0; + ev->got_slice = 0; + ev->nal_length_size = EVC_NAL_UNIT_LENGTH_BYTE; + ev->incomplete_nalu_prefix_read = 0; + + return 0; +} + +static int evc_parse(AVCodecParserContext *s, AVCodecContext *avctx, + const uint8_t **poutbuf, int *poutbuf_size, + const uint8_t *buf, int buf_size) +{ + int next; + EVCParserContext *ev = s->priv_data; + ParseContext *pc = &ev->pc; + int is_dummy_buf = !buf_size; + const uint8_t *dummy_buf = buf; + + if (s->flags & PARSER_FLAG_COMPLETE_FRAMES) + next = buf_size; + else { + next = evc_find_frame_end(s, buf, buf_size, avctx); + if (ff_combine_frame(pc, next, &buf, &buf_size) < 0) { + *poutbuf = NULL; + *poutbuf_size = 0; + ev->to_read -= buf_size; + return buf_size; + } + } + + is_dummy_buf &= (dummy_buf == buf); + + if (!is_dummy_buf) + parse_nal_units(s, buf, buf_size, avctx); + + *poutbuf = buf; + *poutbuf_size = buf_size; + ev->to_read -= next; + + return next; +} + +const AVCodecParser ff_evc_parser = { + .codec_ids = { AV_CODEC_ID_EVC }, + .priv_data_size = sizeof(EVCParserContext), + .parser_init = evc_parser_init, + .parser_parse = evc_parse, + .parser_close = ff_parse_close, +}; diff --git a/libavcodec/libxevd.c b/libavcodec/libxevd.c new file mode 100644 index 0000000000..8ba2d89cbd --- /dev/null +++ b/libavcodec/libxevd.c @@ -0,0 +1,436 @@ +/* + * libxevd decoder + * EVC (MPEG-5 Essential Video Coding) decoding using XEVD MPEG-5 EVC decoder library + * + * Copyright (C) 2021 Dawid Kozinski + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include +#include + +#include + +#include "libavutil/internal.h" +#include "libavutil/common.h" +#include "libavutil/opt.h" +#include "libavutil/pixdesc.h" +#include "libavutil/pixfmt.h" +#include "libavutil/imgutils.h" +#include "libavutil/cpu.h" + +#include "avcodec.h" +#include "internal.h" +#include "packet_internal.h" +#include "codec_internal.h" + +#define XEVD_PARAM_BAD_NAME -1 +#define XEVD_PARAM_BAD_VALUE -2 + +#define EVC_NAL_HEADER_SIZE 2 /* byte */ + +/** + * The structure stores all the states associated with the instance of Xeve MPEG-5 EVC decoder + */ +typedef struct XevdContext { + const AVClass *class; + + XEVD id; // XEVD instance identifier @see xevd.h + XEVD_CDSC cdsc; // decoding parameters @see xevd.h + + int decod_frames; // number of decoded frames + int packet_count; // number of packets created by decoder +} XevdContext; + +/** + * The function returns a pointer to a variable of type XEVD_CDSC. + * XEVD_CDSC contains all decoder parameters that should be initialized before its use. + * + * The field values of the XEVD_CDSC structure are populated based on: + * - the corresponding field values of the AvCodecConetxt structure, + * - the xevd decoder specific option values, + * (the full list of options available for the xevd encoder is displayed after executing the command ./ffmpeg --help decoder = libxevd) + * - and the xevd encoder options specified as a list of key-value pairs following the xevd-params option + * + * Order of input processing and populating the XEVD_CDSC structure + * 1. first, the corresponding fields of the AVCodecContext structure are processed, (i.e -threads 4) + * 2. then xevd-specific options are added as AVOption to the xevd AVCodec implementation (i.e -threads 3) + * 3. finally, the options specified after the xevd-params option as the parameter list of type key value are processed (i.e -xevd-params "m=2") + * + * Some options can be set in different ways. In this case, please follow the above-mentioned order of processing. + * The most recent assignments overwrite the previous values. + * + * @param[in] avctx codec context + * @param[out] cdsc contains all encoder parameters that should be initialized before its use + * + * @return 0 on success, negative error code on failure + */ +static int get_conf(AVCodecContext *avctx, XEVD_CDSC *cdsc) +{ + int cpu_count = av_cpu_count(); + + /* clear XEVS_CDSC structure */ + memset(cdsc, 0, sizeof(XEVD_CDSC)); + + /* init XEVD_CDSC */ + if (avctx->thread_count <= 0) + cdsc->threads = (cpu_count < XEVD_MAX_TASK_CNT) ? cpu_count : XEVD_MAX_TASK_CNT; + else if (avctx->thread_count > XEVD_MAX_TASK_CNT) + cdsc->threads = XEVD_MAX_TASK_CNT; + else + cdsc->threads = avctx->thread_count; + + return 0; +} + +/** + * Read NAL unit length + * @param bs input data (bitstream) + * @return the length of NAL unit on success, 0 value on failure + */ +static uint32_t read_nal_unit_length(const uint8_t *bs, int bs_size, AVCodecContext *avctx) +{ + uint32_t len = 0; + XEVD_INFO info; + int ret; + + if (bs_size == XEVD_NAL_UNIT_LENGTH_BYTE) { + ret = xevd_info((void *)bs, XEVD_NAL_UNIT_LENGTH_BYTE, 1, &info); + if (XEVD_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Cannot get bitstream information\n"); + return 0; + } + len = info.nalu_len; + if (len == 0) { + av_log(avctx, AV_LOG_ERROR, "Invalid bitstream size! [%d]\n", bs_size); + return 0; + } + } + + return len; +} + +/** + * @param[in] xectx the structure that stores all the state associated with the instance of Xeve MPEG-5 EVC decoder + * @param[out] avctx codec context + * @return 0 on success, negative value on failure + */ +static int export_stream_params(const XevdContext *xectx, AVCodecContext *avctx) +{ + int ret; + int size; + int color_space; + + avctx->pix_fmt = AV_PIX_FMT_YUV420P10; + + size = 4; + ret = xevd_config(xectx->id, XEVD_CFG_GET_CODED_WIDTH, &avctx->coded_width, &size); + if (XEVD_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to get coded_width\n"); + return AVERROR_EXTERNAL; + } + + ret = xevd_config(xectx->id, XEVD_CFG_GET_CODED_HEIGHT, &avctx->coded_height, &size); + if (XEVD_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to get coded_height\n"); + return AVERROR_EXTERNAL; + } + + ret = xevd_config(xectx->id, XEVD_CFG_GET_WIDTH, &avctx->width, &size); + if (XEVD_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to get width\n"); + return AVERROR_EXTERNAL; + } + + ret = xevd_config(xectx->id, XEVD_CFG_GET_HEIGHT, &avctx->height, &size); + if (XEVD_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to get height\n"); + return AVERROR_EXTERNAL; + } + + ret = xevd_config(xectx->id, XEVD_CFG_GET_COLOR_SPACE, &color_space, &size); + if (XEVD_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to get color_space\n"); + return AVERROR_EXTERNAL; + } + switch(color_space) { + case XEVD_CS_YCBCR400_10LE: + avctx->pix_fmt = AV_PIX_FMT_GRAY10LE; + break; + case XEVD_CS_YCBCR420_10LE: + avctx->pix_fmt = AV_PIX_FMT_YUV420P10LE; + break; + case XEVD_CS_YCBCR422_10LE: + avctx->pix_fmt = AV_PIX_FMT_YUV422P10LE; + break; + case XEVD_CS_YCBCR444_10LE: + avctx->pix_fmt = AV_PIX_FMT_YUV444P10LE; + break; + default: + av_log(avctx, AV_LOG_ERROR, "Unknown color space\n"); + avctx->pix_fmt = AV_PIX_FMT_NONE; + return AVERROR_INVALIDDATA; + } + + // the function returns sps->num_reorder_pics + ret = xevd_config(xectx->id, XEVD_CFG_GET_MAX_CODING_DELAY, &avctx->max_b_frames, &size); + if (XEVD_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to get max_coding_delay\n"); + return AVERROR_EXTERNAL; + } + + avctx->has_b_frames = (avctx->max_b_frames) ? 1 : 0; + + avctx->color_primaries = AVCOL_PRI_UNSPECIFIED; + avctx->color_trc = AVCOL_TRC_UNSPECIFIED; + avctx->colorspace = AVCOL_SPC_UNSPECIFIED; + + return 0; +} + +/** + * Initialize decoder + * Create a decoder instance and allocate all the needed resources + * + * @param avctx codec context + * @return 0 on success, negative error code on failure + */ +static av_cold int libxevd_init(AVCodecContext *avctx) +{ + XevdContext *xectx = avctx->priv_data; + int ret = 0; + XEVD_CDSC *cdsc = &(xectx->cdsc); + + /* read configurations and set values for created descriptor (XEVD_CDSC) */ + ret = get_conf(avctx, cdsc); + if (ret != 0) { + av_log(avctx, AV_LOG_ERROR, "Cannot get configuration\n"); + return AVERROR_INVALIDDATA; + } + + /* create decoder */ + xectx->id = xevd_create(&(xectx->cdsc), NULL); + if (xectx->id == NULL) { + av_log(avctx, AV_LOG_ERROR, "Cannot create XEVD encoder\n"); + return AVERROR_EXTERNAL; + } + + xectx->packet_count = 0; + xectx->decod_frames = 0; + + return 0; +} + +/** + * Decode picture + * + * @param avctx codec context + * @param[out] frame decoded frame + * @param[out] got_frame_ptr decoder sets to 0 or 1 to indicate that a + * non-empty frame or subtitle was returned in + * outdata. + * @param[in] avpkt AVPacket containing encoded data to be decoded + * + * @return amount of bytes read from the packet on success, negative error + * code on failure + */ +static int libxevd_decode(struct AVCodecContext *avctx, struct AVFrame *frame, int *got_frame_ptr, AVPacket *avpkt) +{ + XevdContext *xectx = NULL; + XEVD_IMGB *imgb = NULL; + XEVD_STAT stat; + XEVD_BITB bitb; + int xevd_ret, nalu_size, bs_read_pos; + int ret = 0; + + if (avctx == NULL) { + av_log(avctx, AV_LOG_ERROR, "Invalid input parameter: AVCodecContext\n"); + return AVERROR(EINVAL); + } + xectx = avctx->priv_data; + if (xectx == NULL) { + av_log(avctx, AV_LOG_ERROR, "Invalid XEVD context\n"); + return AVERROR(EINVAL); + } + + if (avpkt->size > 0) { + bs_read_pos = 0; + imgb = NULL; + while(avpkt->size > (bs_read_pos + XEVD_NAL_UNIT_LENGTH_BYTE)) { + + memset(&stat, 0, sizeof(XEVD_STAT)); + memset(&bitb, 0, sizeof(XEVD_BITB)); + + nalu_size = read_nal_unit_length(avpkt->data + bs_read_pos, XEVD_NAL_UNIT_LENGTH_BYTE, avctx); + if (nalu_size == 0) { + av_log(avctx, AV_LOG_ERROR, "Invalid bitstream\n"); + ret = AVERROR_INVALIDDATA; + goto ERR; + } + bs_read_pos += XEVD_NAL_UNIT_LENGTH_BYTE; + + bitb.addr = avpkt->data + bs_read_pos; + bitb.ssize = nalu_size; + + /* main decoding block */ + xevd_ret = xevd_decode(xectx->id, &bitb, &stat); + if (XEVD_FAILED(xevd_ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to decode bitstream\n"); + ret = AVERROR_EXTERNAL; + goto ERR; + } + + bs_read_pos += nalu_size; + + if (stat.nalu_type == XEVD_NUT_SPS) { // EVC stream parameters changed + if ((ret = export_stream_params(xectx, avctx)) != 0) + goto ERR; + } + + if (stat.read != nalu_size) + av_log(avctx, AV_LOG_INFO, "Different reading of bitstream (in:%d, read:%d)\n,", nalu_size, stat.read); + if (stat.fnum >= 0) { + // already has a decoded image + if (imgb) { + // xevd_pull uses pool of objects of type XEVD_IMGB. + // The pool size is equal MAX_PB_SIZE (26), so release object when it is no more needed + imgb->release(imgb); + imgb = NULL; + } + xevd_ret = xevd_pull(xectx->id, &imgb); + if (XEVD_FAILED(xevd_ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to pull the decoded image (xevd error code: %d, frame#=%d)\n", xevd_ret, stat.fnum); + ret = AVERROR_EXTERNAL; + goto ERR; + } else if (xevd_ret == XEVD_OK_FRM_DELAYED) { + if (imgb) { + // xevd_pull uses pool of objects of type XEVD_IMGB. + // The pool size is equal MAX_PB_SIZE (26), so release object when it is no more needed + imgb->release(imgb); + imgb = NULL; + } + } + } + } + } else { // bumping + xevd_ret = xevd_pull(xectx->id, &(imgb)); + if (xevd_ret == XEVD_ERR_UNEXPECTED) { // bumping process completed + *got_frame_ptr = 0; + return 0; + } else if (XEVD_FAILED(xevd_ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to pull the decoded image (xevd error code: %d)\n", xevd_ret); + ret = AVERROR_EXTERNAL; + goto ERR; + } + } + + if (imgb) { + // @todo supports other color space and bit depth + if (imgb->cs != XEVD_CS_YCBCR420_10LE) { + av_log(avctx, AV_LOG_ERROR, "Not supported pixel format: %s\n", av_get_pix_fmt_name(avctx->pix_fmt)); + ret = AVERROR_INVALIDDATA; + goto ERR; + } + + if (imgb->w[0] != avctx->width || imgb->h[0] != avctx->height) { // stream resolution changed + if (ff_set_dimensions(avctx, imgb->w[0], imgb->h[0]) < 0) { + av_log(avctx, AV_LOG_ERROR, "Cannot set new dimension\n"); + ret = AVERROR_INVALIDDATA; + goto ERR; + } + } + + frame->coded_picture_number++; + frame->display_picture_number++; + frame->format = AV_PIX_FMT_YUV420P10LE; + + if (ff_get_buffer(avctx, frame, 0) < 0) { + av_log(avctx, AV_LOG_ERROR, "Cannot get AV buffer\n"); + ret = AVERROR(ENOMEM); + goto ERR; + } + + frame->pts = avpkt->pts; + + av_image_copy(frame->data, frame->linesize, (const uint8_t **)imgb->a, + imgb->s, avctx->pix_fmt, + imgb->w[0], imgb->h[0]); + + xectx->decod_frames++; + *got_frame_ptr = 1; + + // xevd_pull uses pool of objects of type XEVD_IMGB. + // The pool size is equal MAX_PB_SIZE (26), so release object when it is no more needed + imgb->release(imgb); + imgb = NULL; + } else + *got_frame_ptr = 0; + + xectx->packet_count++; + + return avpkt->size; + +ERR: + if (imgb) { + imgb->release(imgb); + imgb = NULL; + } + *got_frame_ptr = 0; + + return ret; +} + +/** + * Destroy decoder + * + * @param avctx codec context + * @return 0 on success + */ +static av_cold int libxevd_close(AVCodecContext *avctx) +{ + XevdContext *xectx = avctx->priv_data; + if (xectx->id) { + xevd_delete(xectx->id); + xectx->id = NULL; + } + + return 0; +} + +#define OFFSET(x) offsetof(XevdContext, x) +#define VD AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_DECODING_PARAM + +static const AVClass libxevd_class = { + .class_name = "libxevd", + .item_name = av_default_item_name, + .version = LIBAVUTIL_VERSION_INT, +}; + +const FFCodec ff_libxevd_decoder = { + .p.name = "evc", + .p.long_name = NULL_IF_CONFIG_SMALL("EVC / MPEG-5 Essential Video Coding (EVC)"), + .p.type = AVMEDIA_TYPE_VIDEO, + .p.id = AV_CODEC_ID_EVC, + .init = libxevd_init, + FF_CODEC_DECODE_CB(libxevd_decode), + .close = libxevd_close, + .priv_data_size = sizeof(XevdContext), + .p.priv_class = &libxevd_class, + .p.capabilities = AV_CODEC_CAP_DELAY | AV_CODEC_CAP_AUTO_THREADS | AV_CODEC_CAP_AVOID_PROBING | AV_CODEC_CAP_DR1, + .p.wrapper_name = "libxevd", +}; diff --git a/libavcodec/libxeve.c b/libavcodec/libxeve.c new file mode 100644 index 0000000000..4808e6d853 --- /dev/null +++ b/libavcodec/libxeve.c @@ -0,0 +1,597 @@ +/* + * libxeve encoder + * EVC (MPEG-5 Essential Video Coding) encoding using XEVE MPEG-5 EVC encoder library + * + * Copyright (C) 2021 Dawid Kozinski + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include +#include + +#include + +#include "libavutil/internal.h" +#include "libavutil/common.h" +#include "libavutil/opt.h" +#include "libavutil/pixdesc.h" +#include "libavutil/pixfmt.h" +#include "libavutil/time.h" +#include "libavutil/cpu.h" +#include "libavutil/avstring.h" + +#include "avcodec.h" +#include "internal.h" +#include "packet_internal.h" +#include "codec_internal.h" +#include "encode.h" + +#define MAX_BS_BUF (16*1024*1024) + +/** + * Error codes + */ +#define XEVE_PARAM_BAD_NAME -100 +#define XEVE_PARAM_BAD_VALUE -200 + +/** + * Encoder states + * + * STATE_ENCODING - the encoder receives and processes input frames + * STATE_BUMPING - there are no more input frames, however the encoder still processes previously received data + */ +typedef enum State { + STATE_ENCODING, + STATE_BUMPING, +} State; + +/** + * The structure stores all the states associated with the instance of Xeve MPEG-5 EVC encoder + */ +typedef struct XeveContext { + const AVClass *class; + + XEVE id; // XEVE instance identifier + XEVE_CDSC cdsc; // coding parameters i.e profile, width & height of input frame, num of therads, frame rate ... + XEVE_BITB bitb; // bitstream buffer (output) + XEVE_STAT stat; // encoding status (output) + XEVE_IMGB imgb; // image buffer (input) + + State state; // encoder state (skipping, encoding, bumping) + + int profile_id; // encoder profile (main, baseline) + int preset_id; // preset of xeve ( fast, medium, slow, placebo) + int tune_id; // tune of xeve (psnr, zerolatency) + + // variables for rate control modes + int rc_mode; // Rate control mode [ 0(CQP) / 1(ABR) / 2(CRF) ] + int qp; // quantization parameter (QP) [0,51] + int crf; // constant rate factor (CRF) [10,49] + + int hash; // embed picture signature (HASH) for conformance checking in decoding + int sei_info; // embed Supplemental enhancement information while encoding + + int color_format; // input data color format: currently only XEVE_CF_YCBCR420 is supported +} XeveContext; + +/** + * Convert FFmpeg pixel format (AVPixelFormat) to XEVE pre-defined color format + * + * @param[in] av_pix_fmt pixel format (@see https://ffmpeg.org/doxygen/trunk/pixfmt_8h.html#a9a8e335cf3be472042bc9f0cf80cd4c5) + * @param[out] xeve_col_fmt XEVE pre-defined color format (@see xeve.h) + * + * @return 0 on success, negative value on failure + */ +static int libxeve_color_fmt(enum AVPixelFormat av_pix_fmt, int *xeve_col_fmt) +{ + switch (av_pix_fmt) { + case AV_PIX_FMT_YUV420P: + *xeve_col_fmt = XEVE_CF_YCBCR420; + break; + case AV_PIX_FMT_YUV420P10: + *xeve_col_fmt = XEVE_CF_YCBCR420; + break; + default: + *xeve_col_fmt = XEVE_CF_UNKNOWN; + return AVERROR_INVALIDDATA; + } + + return 0; +} + +/** + * Convert FFmpeg pixel format (AVPixelFormat) into XEVE pre-defined color space + * + * @param[in] px_fmt pixel format (@see https://ffmpeg.org/doxygen/trunk/pixfmt_8h.html#a9a8e335cf3be472042bc9f0cf80cd4c5) + * + * @return XEVE pre-defined color space (@see xeve.h) on success, XEVE_CF_UNKNOWN on failure + */ +static int libxeve_color_space(enum AVPixelFormat av_pix_fmt) +{ + /* color space of input image */ + int cs = XEVE_CF_UNKNOWN; + + switch (av_pix_fmt) { + case AV_PIX_FMT_YUV420P: + cs = XEVE_CS_YCBCR420; + break; + case AV_PIX_FMT_YUV420P10: +#if AV_HAVE_BIGENDIAN + cs = XEVE_CS_SET(XEVE_CF_YCBCR420, 10, 1); +#else + cs = XEVE_CS_YCBCR420_10LE; +#endif + + break; + default: + cs = XEVE_CF_UNKNOWN; + break; + } + + return cs; +} + +/** + * The function returns a pointer to the object of the XEVE_CDSC type. + * XEVE_CDSC contains all encoder parameters that should be initialized before the encoder is used. + * + * The field values of the XEVE_CDSC structure are populated based on: + * - the corresponding field values of the AvCodecConetxt structure, + * - the xeve encoder specific option values, + * (the full list of options available for xeve encoder is displayed after executing the command ./ffmpeg --help encoder = libxeve) + * - and the xeve encoder options specified as a list of key-value pairs following the xeve-params option + * + * The order of processing input data and populating the XEVE_CDSC structure + * 1) first, the fields of the AVCodecContext structure corresponding to the provided input options are processed, + * (i.e -pix_fmt yuv420p -s:v 1920x1080 -r 30 -profile:v 0) + * 2) then xeve-specific options added as AVOption to the xeve AVCodec implementation + * (i.e -preset 0) + * + * Keep in mind that, there are options that can be set in different ways. + * In this case, please follow the above-mentioned order of processing. + * The most recent assignments overwrite the previous values. + * + * @param[in] avctx codec context (AVCodecContext) + * @param[out] cdsc contains all Xeve MPEG-5 EVC encoder encoder parameters that should be initialized before the encoder is use + * + * @return 0 on success, negative error code on failure + */ +static int get_conf(AVCodecContext *avctx, XEVE_CDSC *cdsc) +{ + XeveContext *xectx = NULL; + int ret; + + xectx = avctx->priv_data; + + /* initialize xeve_param struct with default values */ + ret = xeve_param_default(&cdsc->param); + if (XEVE_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Cannot set_default parameter\n"); + return AVERROR_EXTERNAL; + } + + /* read options from AVCodecContext */ + if (avctx->width > 0) + cdsc->param.w = avctx->width; + + if (avctx->height > 0) + cdsc->param.h = avctx->height; + + if (avctx->framerate.num > 0) { + // fps can be float number, but xeve API doesn't support it + cdsc->param.fps = (int)(((float)avctx->framerate.num / avctx->framerate.den) + 0.5); + } + + if (avctx->gop_size >= 0) // GOP size (key-frame interval, I-picture period) + cdsc->param.keyint = avctx->gop_size; // 0: only one I-frame at the first time; 1: every frame is coded in I-frame + + if (avctx->max_b_frames == 0 || avctx->max_b_frames == 1 || avctx->max_b_frames == 3 || + avctx->max_b_frames == 7 || avctx->max_b_frames == 15) // number of b-frames + cdsc->param.bframes = avctx->max_b_frames; + else { + av_log(avctx, AV_LOG_ERROR, "Incorrect value for maximum number of B frames: (%d) \n" + "Acceptable values for bf option (maximum number of B frames) are 0,1,3,7 or 15\n", avctx->max_b_frames); + return AVERROR_INVALIDDATA; + } + + if (avctx->level >= 0) + cdsc->param.level_idc = avctx->level; + + if (avctx->rc_buffer_size) // VBV buf size + cdsc->param.vbv_bufsize = (int)(avctx->rc_buffer_size / 1000); + + cdsc->param.rc_type = xectx->rc_mode; + + if (xectx->rc_mode == XEVE_RC_CQP) { + cdsc->param.qp = xectx->qp; + } else if (xectx->rc_mode == XEVE_RC_ABR) { + if (avctx->bit_rate / 1000 > INT_MAX || avctx->rc_max_rate / 1000 > INT_MAX) { + av_log(avctx, AV_LOG_ERROR, "Not supported bitrate bit_rate and rc_max_rate > %d000\n", INT_MAX); + return AVERROR_INVALIDDATA; + } + cdsc->param.bitrate = (int)(avctx->bit_rate / 1000); + } else if (xectx->rc_mode == XEVE_RC_CRF) { + cdsc->param.crf = xectx->crf; + } else { + av_log(avctx, AV_LOG_ERROR, "Not supported rate control type: %d\n", xectx->rc_mode); + return AVERROR_INVALIDDATA; + } + + if (avctx->thread_count <= 0) { + int cpu_count = av_cpu_count(); + cdsc->param.threads = (cpu_count < XEVE_MAX_THREADS) ? cpu_count : XEVE_MAX_THREADS; + } else if (avctx->thread_count > XEVE_MAX_THREADS) + cdsc->param.threads = XEVE_MAX_THREADS; + else + cdsc->param.threads = avctx->thread_count; + + + libxeve_color_fmt(avctx->pix_fmt, &xectx->color_format); + + cdsc->param.cs = XEVE_CS_SET(xectx->color_format, cdsc->param.codec_bit_depth, AV_HAVE_BIGENDIAN); + + cdsc->max_bs_buf_size = MAX_BS_BUF; + + ret = xeve_param_ppt(&cdsc->param, xectx->profile_id, xectx->preset_id, xectx->tune_id); + if (XEVE_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Cannot set profile(%d), preset(%d), tune(%d)\n", xectx->profile_id, xectx->preset_id, xectx->tune_id); + return AVERROR_EXTERNAL; + } + + return 0; +} + +/** + * Set XEVE_CFG_SET_USE_PIC_SIGNATURE for encoder + * + * @param[in] logger context + * @param[in] id XEVE encodec instance identifier + * @param[in] ctx the structure stores all the states associated with the instance of Xeve MPEG-5 EVC encoder + * + * @return 0 on success, negative error code on failure + */ +static int set_extra_config(AVCodecContext *avctx, XEVE id, XeveContext *ctx) +{ + int ret, size; + size = 4; + + // embed SEI messages identifying encoder parameters and command line arguments + // - 0: off\n" + // - 1: emit sei info" + // + // SEI - Supplemental enhancement information contains information + // that is not necessary to decode the samples of coded pictures from VCL NAL units. + // Some SEI message information is required to check bitstream conformance + // and for output timing decoder conformance. + // @see ISO_IEC_23094-1_2020 7.4.3.5 + // @see ISO_IEC_23094-1_2020 Annex D + ret = xeve_config(id, XEVE_CFG_SET_SEI_CMD, &ctx->sei_info, &size); // sei_cmd_info + if (XEVE_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to set config for sei command info messages\n"); + return AVERROR_EXTERNAL; + } + + ret = xeve_config(id, XEVE_CFG_SET_USE_PIC_SIGNATURE, &ctx->hash, &size); + if (XEVE_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to set config for picture signature\n"); + return AVERROR_EXTERNAL; + } + + return 0; +} + +/** + * @brief Switch encoder to bumping mode + * + * @param id XEVE encodec instance identifier + * @return 0 on success, negative error code on failure + */ +static int setup_bumping(XEVE id) +{ + int val = 1; + int size = sizeof(int); + if (XEVE_FAILED(xeve_config(id, XEVE_CFG_SET_FORCE_OUT, (void *)(&val), &size))) + return AVERROR_EXTERNAL; + + return 0; +} + +/** + * @brief Initialize eXtra-fast Essential Video Encoder codec + * Create an encoder instance and allocate all the needed resources + * + * @param avctx codec context + * @return 0 on success, negative error code on failure + */ +static av_cold int libxeve_init(AVCodecContext *avctx) +{ + XeveContext *xectx = avctx->priv_data; + unsigned char *bs_buf = NULL; + int i; + int shift_h = 0; + int shift_v = 0; + int width_chroma = 0; + int height_chroma = 0; + XEVE_IMGB *imgb = NULL; + int ret = 0; + + XEVE_CDSC *cdsc = &(xectx->cdsc); + + /* allocate bitstream buffer */ + bs_buf = av_malloc(MAX_BS_BUF); + if (bs_buf == NULL) { + av_log(avctx, AV_LOG_ERROR, "Cannot allocate bitstream buffer\n"); + return AVERROR(ENOMEM); + } + xectx->bitb.addr = bs_buf; + xectx->bitb.bsize = MAX_BS_BUF; + + /* read configurations and set values for created descriptor (XEVE_CDSC) */ + if ((ret = get_conf(avctx, cdsc)) != 0) { + av_log(avctx, AV_LOG_ERROR, "Cannot get configuration\n"); + return AVERROR(EINVAL); + } + + if ((ret = xeve_param_check(&cdsc->param)) != 0) { + av_log(avctx, AV_LOG_ERROR, "Invalid configuration\n"); + return AVERROR(EINVAL); + } + + /* create encoder */ + xectx->id = xeve_create(cdsc, NULL); + if (xectx->id == NULL) { + av_log(avctx, AV_LOG_ERROR, "Cannot create XEVE encoder\n"); + return AVERROR_EXTERNAL; + } + + if ((ret = set_extra_config(avctx, xectx->id, xectx)) != 0) { + av_log(avctx, AV_LOG_ERROR, "Cannot set extra configuration\n"); + return AVERROR(EINVAL); + } + + if ((ret = av_pix_fmt_get_chroma_sub_sample(avctx->pix_fmt, &shift_h, &shift_v)) != 0) { + av_log(avctx, AV_LOG_ERROR, "Failed to get chroma shift\n"); + return AVERROR(EINVAL); + } + + // Chroma subsampling + // + // YUV format explanation + // shift_h == 1 && shift_v == 1 : YUV420 + // shift_h == 1 && shift_v == 0 : YUV422 + // shift_h == 0 && shift_v == 0 : YUV444 + // + width_chroma = AV_CEIL_RSHIFT(avctx->width, shift_h); + height_chroma = AV_CEIL_RSHIFT(avctx->height, shift_v); + + /* set default values for input image buffer */ + imgb = &xectx->imgb; + imgb->cs = libxeve_color_space(avctx->pix_fmt); + imgb->np = 3; /* only for yuv420p, yuv420ple */ + + for (i = 0; i < imgb->np; i++) + imgb->x[i] = imgb->y[i] = 0; + + imgb->w[0] = imgb->aw[0] = avctx->width; // width luma + imgb->w[1] = imgb->w[2] = imgb->aw[1] = imgb->aw[2] = width_chroma; + imgb->h[0] = imgb->ah[0] = avctx->height; // height luma + imgb->h[1] = imgb->h[2] = imgb->ah[1] = imgb->ah[2] = height_chroma; + + xectx->state = STATE_ENCODING; + + return 0; +} + +/** + * Encode raw data frame into EVC packet + * + * @param[in] avctx codec context + * @param[out] pkt output AVPacket containing encoded data + * @param[in] frame AVFrame containing the raw data to be encoded + * @param[out] got_packet encoder sets to 0 or 1 to indicate that a + * non-empty packet was returned in pkt + * + * @return 0 on success, negative error code on failure + */ +static int libxeve_encode(AVCodecContext *avctx, AVPacket *avpkt, + const AVFrame *frame, int *got_packet) +{ + XeveContext *xectx = avctx->priv_data; + int ret = -1; + + // No more input frames are available but encoder still can have some data in its internal buffer to process + // and some frames to dump. + if (xectx->state == STATE_ENCODING && frame == NULL) { + if (setup_bumping(xectx->id) == 0) + xectx->state = STATE_BUMPING; // Entering bumping process + else { + av_log(avctx, AV_LOG_ERROR, "Failed to setup bumping\n"); + return 0; + } + } + + if (xectx->state == STATE_ENCODING) { + int i; + XEVE_IMGB *imgb = NULL; + + imgb = &xectx->imgb; + + for (i = 0; i < imgb->np; i++) { + imgb->a[i] = frame->data[i]; + imgb->s[i] = frame->linesize[i]; + } + + imgb->ts[0] = frame->pts; + imgb->ts[1] = 0; + + /* push image to encoder */ + ret = xeve_push(xectx->id, imgb); + if (XEVE_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "xeve_push() failed\n"); + return AVERROR_EXTERNAL; + } + } + if (xectx->state == STATE_ENCODING || xectx->state == STATE_BUMPING) { + /* encoding */ + ret = xeve_encode(xectx->id, &(xectx->bitb), &(xectx->stat)); + if (XEVE_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "xeve_encode() failed\n"); + return AVERROR_EXTERNAL; + } + + /* store bitstream */ + if (ret == XEVE_OK_OUT_NOT_AVAILABLE) { // Return OK but picture is not available yet + *got_packet = 0; + return 0; + } else if (ret == XEVE_OK) { + int av_pic_type; + + if (xectx->stat.write > 0) { + + ret = ff_get_encode_buffer(avctx, avpkt, xectx->stat.write, 0); + if (ret < 0) + return ret; + + memcpy(avpkt->data, xectx->bitb.addr, xectx->stat.write); + + avpkt->pts = xectx->bitb.ts[0]; + avpkt->dts = xectx->bitb.ts[1]; + + switch(xectx->stat.stype) { + case XEVE_ST_I: + av_pic_type = AV_PICTURE_TYPE_I; + avpkt->flags |= AV_PKT_FLAG_KEY; + break; + case XEVE_ST_P: + av_pic_type = AV_PICTURE_TYPE_P; + break; + case XEVE_ST_B: + av_pic_type = AV_PICTURE_TYPE_B; + break; + case XEVE_ST_UNKNOWN: + av_log(avctx, AV_LOG_ERROR, "Unknown slice type\n"); + return AVERROR_INVALIDDATA; + } + + ff_side_data_set_encoder_stats(avpkt, xectx->stat.qp * FF_QP2LAMBDA, NULL, 0, av_pic_type); + + *got_packet = 1; + } + } else if (ret == XEVE_OK_NO_MORE_FRM) { + // Return OK but no more frames + return 0; + } else { + av_log(avctx, AV_LOG_ERROR, "Invalid return value: %d\n", ret); + return AVERROR_EXTERNAL; + } + } else { + av_log(avctx, AV_LOG_ERROR, "Udefined encoder state\n"); + return AVERROR_INVALIDDATA; + } + + return 0; +} + +/** + * Destroy the encoder and release all the allocated resources + * + * @param avctx codec context + * @return 0 on success, negative error code on failure + */ +static av_cold int libxeve_close(AVCodecContext *avctx) +{ + XeveContext *xectx = avctx->priv_data; + + xeve_delete(xectx->id); + + av_free(xectx->bitb.addr); /* release bitstream buffer */ + + return 0; +} + +#define OFFSET(x) offsetof(XeveContext, x) +#define VE AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_ENCODING_PARAM + +static const enum AVPixelFormat supported_pixel_formats[] = { + AV_PIX_FMT_YUV420P, + AV_PIX_FMT_YUV420P10, + AV_PIX_FMT_NONE +}; + +// Example of using: ./ffmpeg -xeve-params "m=2:q=17" +// Consider using following options (./ffmpeg --help encoder=libxeve) +// +static const AVOption libxeve_options[] = { + { "preset", "Encoding preset for setting encoding speed", OFFSET(preset_id), AV_OPT_TYPE_INT, { .i64 = XEVE_PRESET_MEDIUM }, XEVE_PRESET_DEFAULT, XEVE_PRESET_PLACEBO, VE, "preset" }, + { "default", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_PRESET_DEFAULT }, INT_MIN, INT_MAX, VE, "preset" }, + { "fast", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_PRESET_FAST }, INT_MIN, INT_MAX, VE, "preset" }, + { "medium", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_PRESET_MEDIUM }, INT_MIN, INT_MAX, VE, "preset" }, + { "slow", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_PRESET_SLOW }, INT_MIN, INT_MAX, VE, "preset" }, + { "placebo", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_PRESET_PLACEBO }, INT_MIN, INT_MAX, VE, "preset" }, + { "tune", "Tuning parameter for special purpose operation", OFFSET(tune_id), AV_OPT_TYPE_INT, { .i64 = XEVE_TUNE_NONE }, XEVE_TUNE_NONE, XEVE_TUNE_PSNR, VE, "tune"}, + { "none", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_TUNE_NONE }, INT_MIN, INT_MAX, VE, "tune" }, + { "zerolatency", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_TUNE_ZEROLATENCY }, INT_MIN, INT_MAX, VE, "tune" }, + { "psnr", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_TUNE_PSNR }, INT_MIN, INT_MAX, VE, "tune" }, + { "profile", "Encoding profile", OFFSET(profile_id), AV_OPT_TYPE_INT, { .i64 = XEVE_PROFILE_BASELINE }, XEVE_PROFILE_BASELINE, XEVE_PROFILE_MAIN, VE, "profile" }, + { "baseline", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_PROFILE_BASELINE }, INT_MIN, INT_MAX, VE, "profile" }, + { "main", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_PROFILE_MAIN }, INT_MIN, INT_MAX, VE, "profile" }, + { "rc_mode", "Rate control mode", OFFSET(rc_mode), AV_OPT_TYPE_INT, { .i64 = XEVE_RC_CQP }, XEVE_RC_CQP, XEVE_RC_CRF, VE, "rc_mode" }, + { "CQP", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_RC_CQP }, INT_MIN, INT_MAX, VE, "rc_mode" }, + { "ABR", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_RC_ABR }, INT_MIN, INT_MAX, VE, "rc_mode" }, + { "CRF", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_RC_CRF }, INT_MIN, INT_MAX, VE, "rc_mode" }, + { "qp", "Quantization parameter value for CQP rate control mode", OFFSET(qp), AV_OPT_TYPE_INT, { .i64 = 32 }, 0, 51, VE }, + { "crf", "Constant rate factor value for CRF rate control mode", OFFSET(crf), AV_OPT_TYPE_INT, { .i64 = 32 }, 10, 49, VE }, + { "hash", "Embed picture signature (HASH) for conformance checking in decoding", OFFSET(hash), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, 1, VE }, + { "sei_info", "Embed SEI messages identifying encoder parameters and command line arguments", OFFSET(sei_info), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, 1, VE }, + { NULL } +}; + +static const AVClass libxeve_class = { + .class_name = "libxeve", + .item_name = av_default_item_name, + .option = libxeve_options, + .version = LIBAVUTIL_VERSION_INT, +}; + +/** + * libavcodec generic global options, which can be set on all the encoders and decoders + * @see https://www.ffmpeg.org/ffmpeg-codecs.html#Codec-Options + */ +static const FFCodecDefault libxeve_defaults[] = { + { "b", "0" }, // bitrate in terms of kilo-bits per second + { "g", "0" }, // gop_size (key-frame interval 0: only one I-frame at the first time; 1: every frame is coded in I-frame) + { "bf", "15"}, // maximum number of B frames (0: no B-frames, 1,3,7,15) + { "threads", "0"}, // number of threads to be used (0: automatically select the number of threads to set) + { NULL }, +}; + +const FFCodec ff_libxeve_encoder = { + .p.name = "libxeve", + .p.long_name = NULL_IF_CONFIG_SMALL("libxeve MPEG-5 EVC"), + .p.type = AVMEDIA_TYPE_VIDEO, + .p.id = AV_CODEC_ID_EVC, + .init = libxeve_init, + FF_CODEC_ENCODE_CB(libxeve_encode), + .close = libxeve_close, + .priv_data_size = sizeof(XeveContext), + .p.priv_class = &libxeve_class, + .defaults = libxeve_defaults, + .p.capabilities = FF_CODEC_CAP_INIT_CLEANUP | AV_CODEC_CAP_DELAY | AV_CODEC_CAP_AUTO_THREADS | AV_CODEC_CAP_DR1, + .p.wrapper_name = "libxeve", + .p.pix_fmts = supported_pixel_formats, +}; diff --git a/libavcodec/parsers.c b/libavcodec/parsers.c index a8d52af6cb..ca00c63176 100644 --- a/libavcodec/parsers.c +++ b/libavcodec/parsers.c @@ -41,6 +41,7 @@ extern const AVCodecParser ff_dvaudio_parser; extern const AVCodecParser ff_dvbsub_parser; extern const AVCodecParser ff_dvdsub_parser; extern const AVCodecParser ff_dvd_nav_parser; +extern const AVCodecParser ff_evc_parser; extern const AVCodecParser ff_flac_parser; extern const AVCodecParser ff_g723_1_parser; extern const AVCodecParser ff_g729_parser; diff --git a/libavcodec/profiles.c b/libavcodec/profiles.c index 7af7fbeb13..a31244e0db 100644 --- a/libavcodec/profiles.c +++ b/libavcodec/profiles.c @@ -181,4 +181,10 @@ const AVProfile ff_arib_caption_profiles[] = { { FF_PROFILE_UNKNOWN } }; +const AVProfile ff_evc_profiles[] = { + { FF_PROFILE_EVC_BASELINE, "Baseline" }, + { FF_PROFILE_EVC_MAIN, "Main" }, + { FF_PROFILE_UNKNOWN }, +}; + #endif /* !CONFIG_SMALL */ diff --git a/libavcodec/profiles.h b/libavcodec/profiles.h index 41a19aa9ad..cf92b5f126 100644 --- a/libavcodec/profiles.h +++ b/libavcodec/profiles.h @@ -72,5 +72,6 @@ extern const AVProfile ff_sbc_profiles[]; extern const AVProfile ff_prores_profiles[]; extern const AVProfile ff_mjpeg_profiles[]; extern const AVProfile ff_arib_caption_profiles[]; +extern const AVProfile ff_evc_profiles[]; #endif /* AVCODEC_PROFILES_H */ diff --git a/libavcodec/version.h b/libavcodec/version.h index 19f3f4a272..4a1205c718 100644 --- a/libavcodec/version.h +++ b/libavcodec/version.h @@ -29,7 +29,7 @@ #include "version_major.h" -#define LIBAVCODEC_VERSION_MINOR 40 +#define LIBAVCODEC_VERSION_MINOR 41 #define LIBAVCODEC_VERSION_MICRO 100 #define LIBAVCODEC_VERSION_INT AV_VERSION_INT(LIBAVCODEC_VERSION_MAJOR, \