From patchwork Tue Mar 28 13:49:14 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dawid Kozinski X-Patchwork-Id: 40894 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:7a30:b0:df:834d:2c1a with SMTP id t48csp2197491pzh; Tue, 28 Mar 2023 06:49:58 -0700 (PDT) X-Google-Smtp-Source: AKy350YQGOkZ8abkCYkEMCwV1Aq3MiXsuAdOHjnRmvj05kgCPnEYDwwHq7r9X0leq92WlgWwU/cq X-Received: by 2002:a17:906:5589:b0:88a:2e57:9813 with SMTP id y9-20020a170906558900b0088a2e579813mr17478401ejp.33.1680011398685; Tue, 28 Mar 2023 06:49:58 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1680011398; cv=none; d=google.com; s=arc-20160816; b=Nx63Glq7AK+23LM81a3EY5gXrYjHsk1yuO4bESFGb83jSJoZcQmVENX8weRPfbg3QW c+QG1qoWxEw43UCZ5s8Lz5lMgAwTQuwsrqs+MVieNfVG5wMA74SE95SrBNPXzU9n8Zu4 X/kXptCQ8XD7ojCphLdA9WsuZMUKQWb9myrPTNdlkJhgTr6MTkh/dnkbE/tov8pcsQ86 X7weksHDxJdDrZZ6LXZWnFptOPSlVBt28i7hLKlJIu5D6c41WxwN325rP6xpZtWWU1Mu XiWv1T/BXAig8gQ5JTKkc9LK+dhgv5/qKY3KcfEqrXTQ8W3a07r6D18rVHW4ITKCs6DP UABQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:cms-type:mime-version :message-id:date:to:from:dkim-signature:dkim-filter:delivered-to; bh=cBk7Db1k+WWlCsmwVo269b9cUYHLfh8v/sOpen79w/I=; b=sx9rFrghNY4Rv/6l4lKSWcK+JgRJS5bQB1J4ZyGE6dV/9pdhqUjBtu4/eURRaNtFSE zEXV0UbfJ9KfEFWEX2iPCFkH1EZBWgVjzn/UmSuMq7nuVCfZ3DYykFq4xBh3ZO39ZDfk jJHJVUAFfE1ts+Ma4EiB9Q4W7OoLTYc0BZbjjp5UkBk5vXUTsdvH3EQWHRYuGe2ksJRc 2H1v8kRbuwnm8Od+/kb9LtSngc4TbX85dI1v7E4gEs51xfuMI4bCWo5g/1gw55Fx7T1x p7ykPCO7BiWhWnQSFl9LPheC6pOH0GRITjDpZ1TLr9lsJ0LxXN5rtFDrJ3XeA0ukLnFx dXdw== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@samsung.com header.s=mail20170921 header.b=uXZvCGs1; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=samsung.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id i13-20020a170906a28d00b00932dac6402csi26171896ejz.370.2023.03.28.06.49.58; Tue, 28 Mar 2023 06:49:58 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@samsung.com header.s=mail20170921 header.b=uXZvCGs1; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=samsung.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 3290A68CB79; Tue, 28 Mar 2023 16:49:56 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mailout2.w1.samsung.com (mailout2.w1.samsung.com [210.118.77.12]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 7CF7968C1DA for ; Tue, 28 Mar 2023 16:49:49 +0300 (EEST) Received: from eucas1p1.samsung.com (unknown [182.198.249.206]) by mailout2.w1.samsung.com (KnoxPortal) with ESMTP id 20230328134949euoutp02e76f2a9c374b7dccbdb1ae575623798d~QmXUOWCdN2752727527euoutp023 for ; Tue, 28 Mar 2023 13:49:49 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout2.w1.samsung.com 20230328134949euoutp02e76f2a9c374b7dccbdb1ae575623798d~QmXUOWCdN2752727527euoutp023 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1680011389; bh=eb61XYZO89xig2M0t+C7U/+HBajMaerCYsgLznt0mTs=; h=From:To:Cc:Subject:Date:References:From; b=uXZvCGs1E+tW7PCBv6+PcHIPU2GsBFOY8CmZK9Dp1DR4Ffj+rycT0l+SwL+m9h2z5 lEq5aP0Q6DkoHHPjvDWR4oa31fIanW3Z5Ekz/R/7Tyc7+WyOM/fNVtI26BF6s087Ag jiKh6zxwdyd8u7F8Mfx47W8s93Y09kZWkgu/IpDs= Received: from eusmges3new.samsung.com (unknown [203.254.199.245]) by eucas1p1.samsung.com (KnoxPortal) with ESMTP id 20230328134948eucas1p1d818c1dd891239b401ff622cc6ac2abd~QmXTngSVk1571815718eucas1p1p; Tue, 28 Mar 2023 13:49:48 +0000 (GMT) Received: from eucas1p2.samsung.com ( [182.198.249.207]) by eusmges3new.samsung.com (EUCPMTA) with SMTP id CD.FB.10014.C70F2246; Tue, 28 Mar 2023 14:49:48 +0100 (BST) Received: from eusmtrp1.samsung.com (unknown [182.198.249.138]) by eucas1p1.samsung.com (KnoxPortal) with ESMTPA id 20230328134948eucas1p1ec170b3992293246ccbab88fa2b24797~QmXTVIgpY1366813668eucas1p15; Tue, 28 Mar 2023 13:49:48 +0000 (GMT) Received: from eusmgms1.samsung.com (unknown [182.198.249.179]) by eusmtrp1.samsung.com (KnoxPortal) with ESMTP id 20230328134948eusmtrp1ecbaf655eeaa392b9d8ccea20158c9ef~QmXTUgOXP0181801818eusmtrp1t; Tue, 28 Mar 2023 13:49:48 +0000 (GMT) X-AuditID: cbfec7f5-b8bff7000000271e-af-6422f07c663f Received: from eusmtip2.samsung.com ( [203.254.199.222]) by eusmgms1.samsung.com (EUCPMTA) with SMTP id DA.62.08862.B70F2246; Tue, 28 Mar 2023 14:49:47 +0100 (BST) Received: from AMDN5164.EU.corp.samsungelectronics.net (unknown [106.210.132.171]) by eusmtip2.samsung.com (KnoxPortal) with ESMTPA id 20230328134947eusmtip2554fe046a5c757674c7a6aa68e5c6911~QmXS8lPi42048320483eusmtip2Y; Tue, 28 Mar 2023 13:49:47 +0000 (GMT) From: Dawid Kozinski To: d.frankiewic@samsung.com, ffmpeg-devel@ffmpeg.org Date: Tue, 28 Mar 2023 15:49:14 +0200 Message-Id: <20230328134914.1043-1-d.kozinski@samsung.com> X-Mailer: git-send-email 2.37.3.windows.1 MIME-Version: 1.0 X-Unsent: 1 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrHIsWRmVeSWpSXmKPExsWy7djP87o1H5RSDPYvELb4cu0Ls8XKb79Z LL59OsPswOzxZ9FmFo++LasYA5iiuGxSUnMyy1KL9O0SuDK+bFzHVnDtN2PF9luHGBsY3+1l 7GLk5JAQMJFYveQKkM3FISSwglFi1aNeKOcLo8SG2ztYIJzPjBIHd2xnhWmZNuctO0RiOaPE 9VWfmSCcdiaJY6tPgA1mE9CVePnhEROILSJgKnGyeS8biM0soC2x7u53MFtYIEji9vYF7CA2 i4CqxLWGnSwgNq+AlcSK/x+gtmlKTNgyhwkiLihxcuYTFog4t8S6WZdYIWbKSzRvnc0McoSE wF92iZkH/zBDFLlILFvewQRhC0u8Or6FHcKWkTg9uQdoEAeQXSxxqN8BwqyROPQjHaLCWuJt 43FGkDAz0Anrd+lDhB0lZp3czApRzSdx460gxAF8EpO2TWeGCPNKdLQJQZgqEn2dYhCNUhJP l82BuspDYunUQ4wTGBVnIflqFpJPZiGsXcDIvIpRPLW0ODc9tdg4L7Vcrzgxt7g0L10vOT93 EyMwSZz+d/zrDsYVrz7qHWJk4mA8xCjBwawkwvv7mlKKEG9KYmVValF+fFFpTmrxIUZpDhYl cV5t25PJQgLpiSWp2ampBalFMFkmDk6pBiZv/7zjr3Uzd3a6CZiZiyxY5P7szf49e6vzf59f WXLiv1yH/tkDrE/N9/YlmLAoGM8U0ZW0KBLb2XfPufFr+uw++Swh5XXqzbq9cbE+Ex+8VfFZ 8/TjnvtfYu4oae/OXm0cI6bSE/pte9pCK5td8YyHfHOuMRhciuvr49l+YOGxauXWbFaHM+Eb VjzLDC7eGh/s/2udsYPP7ibtOVP2PbglcmGLtc3EWZseT72k/mhF4zr/04ztBk8yXu8t1L32 5GhvR32KIMsyFYcmkwt1D/yP2h75qlO56uAvc5Wmlekz3atz1f1u82urcTvIcl7kPSBfy6zD ITsndILfizvzJlYtunDjusl6P4OSmdss9yixFGckGmoxFxUnAgBIBLEKgQMAAA== X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFrrMLMWRmVeSWpSXmKPExsVy+t/xe7o1H5RSDM6qW3y59oXZYuW33ywW 3z6dYXZg9vizaDOLR9+WVYwBTFF6NkX5pSWpChn5xSW2StGGFkZ6hpYWekYmlnqGxuaxVkam Svp2NimpOZllqUX6dgl6GV82rmMruPabsWL7rUOMDYzv9jJ2MXJySAiYSEyb85a9i5GLQ0hg KaPEs9M/2CESUhJLly6CKhKW+HOtiw2iqJVJ4viB/2wgCTYBXYmXHx4xgdgiAuYSF9fdAmtg FtCWWHf3O1iNsECAxPvFl8FqWARUJa417GQBsXkFrCRW/P/ACrFAU2LCljlMEHFBiZMzn7BA xLkl1s26xAoxU16ieets5gmM/LOQlM1CklrAyLSKUSS1tDg3PbfYUK84Mbe4NC9dLzk/dxMj MHS3Hfu5eQfjvFcf9Q4xMnEwHmKU4GBWEuH9fU0pRYg3JbGyKrUoP76oNCe1+BCjKdCtE5ml RJPzgdGTVxJvaGZgamhiZmlgamlmrCTO61nQkSgkkJ5YkpqdmlqQWgTTx8TBKdXAxCpzTMJ1 0rOJQT7X2L5ctTxyQfdPbM+dW78jSy5fmSh55LTb2n3/YqWY4l4e8pfwffamp2B7xYyKNu8j md2vulf6t/quyjjLM/XBxZQ+vt232SYyCImveP/RxWnd04uTL/gd//2a1fZ2dua1RfNqbwrW i4U80U0+9F9oC7N4p9u6pFkHZTalppvelit7a3iBqeHSTUYN677agtgFfHpHDf8ZJPIYvvI3 1xdpnbJgw40Lft5Ks6tumS99+ekKa8Ql8dnfP7x5/P8jn+Pv2MiHyYdc/1h+SPH0+r18/qTX F23naGWK5FyvmTKVu+OntUYS/7F9fS8SD9d+35Kyh0f4TJE2u9SCz9u3Fav58ar9dFViKc5I NNRiLipOBAAR+U7O5gIAAA== X-CMS-MailID: 20230328134948eucas1p1ec170b3992293246ccbab88fa2b24797 X-Msg-Generator: CA X-RootMTR: 20230328134948eucas1p1ec170b3992293246ccbab88fa2b24797 X-EPHeader: CA CMS-TYPE: 201P X-CMS-RootMailID: 20230328134948eucas1p1ec170b3992293246ccbab88fa2b24797 References: Subject: [FFmpeg-devel] [PATCH v18 05/10] avcodec/evc_encoder: Provided support for EVC encoder X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Dawid Kozinski Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: ni12pTuVqH1J - Added EVC encoder wrapper - Changes in project configuration file and libavcodec Makefile - Added documentation for xeve wrapper Signed-off-by: Dawid Kozinski --- configure | 4 + doc/encoders.texi | 69 +++++ doc/general_contents.texi | 11 + libavcodec/Makefile | 1 + libavcodec/allcodecs.c | 1 + libavcodec/libxeve.c | 616 ++++++++++++++++++++++++++++++++++++++ 6 files changed, 702 insertions(+) create mode 100644 libavcodec/libxeve.c diff --git a/configure b/configure index cec001fb16..4b6db91de5 100755 --- a/configure +++ b/configure @@ -291,6 +291,7 @@ External library support: --enable-libwebp enable WebP encoding via libwebp [no] --enable-libx264 enable H.264 encoding via x264 [no] --enable-libx265 enable HEVC encoding via x265 [no] + --enable-libxeve enable EVC encoding via libxeve [no] --enable-libxavs enable AVS encoding via xavs [no] --enable-libxavs2 enable AVS2 encoding via xavs2 [no] --enable-libxcb enable X11 grabbing using XCB [autodetect] @@ -1862,6 +1863,7 @@ EXTERNAL_LIBRARY_LIST=" libvorbis libvpx libwebp + libxeve libxml2 libzimg libzmq @@ -3397,6 +3399,7 @@ libx265_encoder_deps="libx265" libx265_encoder_select="atsc_a53" libxavs_encoder_deps="libxavs" libxavs2_encoder_deps="libxavs2" +libxeve_encoder_deps="libxeve" libxvid_encoder_deps="libxvid" libzvbi_teletext_decoder_deps="libzvbi" vapoursynth_demuxer_deps="vapoursynth" @@ -6735,6 +6738,7 @@ enabled libx265 && require_pkg_config libx265 x265 x265.h x265_api_get require_cpp_condition libx265 x265.h "X265_BUILD >= 89" enabled libxavs && require libxavs "stdint.h xavs.h" xavs_encoder_encode "-lxavs $pthreads_extralibs $libm_extralibs" enabled libxavs2 && require_pkg_config libxavs2 "xavs2 >= 1.3.0" "stdint.h xavs2.h" xavs2_api_get +enabled libxeve && require_pkg_config libxeve "xeve >= 0.4.0" "xeve.h" xeve_encode enabled libxvid && require libxvid xvid.h xvid_global -lxvidcore enabled libzimg && require_pkg_config libzimg "zimg >= 2.7.0" zimg.h zimg_get_api_version enabled libzmq && require_pkg_config libzmq "libzmq >= 4.2.1" zmq.h zmq_ctx_new diff --git a/doc/encoders.texi b/doc/encoders.texi index af3bd43cbe..45ae61b3fb 100644 --- a/doc/encoders.texi +++ b/doc/encoders.texi @@ -2878,6 +2878,75 @@ ffmpeg -i input -c:v libxavs2 -xavs2-params RdoqLevel=0 output.avs2 @end example @end table +@section libxeve + +eXtra-fast Essential Video Encoder (XEVE) MPEG-5 EVC encoder wrapper. +The xeve-equivalent options or values are listed in parentheses for easy migration. + +This encoder requires the presence of the libxeve headers and library +during configuration. You need to explicitly configure the build with +@option{--enable-libxeve}. + +@float NOTE +Many libxeve encoder options are mapped to FFmpeg global codec options, +while unique encoder options are provided through private options. +Additionally the xeve-params private options allows one to pass a list +of key=value tuples as accepted by the libxeve @code{parse_xeve_params} function. +@end float + +The xeve project website is at @url{https://github.com/mpeg5/xeve}. + +@subsection Options + +The following options are supported by the libxeve wrapper. +The xeve-equivalent options or values are listed in parentheses for easy migration. + +@float NOTE +To reduce the duplication of documentation, only the private options +and some others requiring special attention are documented here. For +the documentation of the undocumented generic options, see +@ref{codec-options,,the Codec Options chapter}. +@end float + +@float NOTE +To get a more accurate and extensive documentation of the libxeve options, +invoke the command @code{xeve_app --help} or consult the libxeve documentation. +@end float + +@table @option +@item b (@emph{bitrate}) +Set target video bitrate in bits/s. +Note that FFmpeg's b option is expressed in bits/s, while xeve's bitrate is in kilobits/s. + +@item bf (@emph{bframes}) +Set the maximum number of B frames (1,3,7,15). + +@item g (@emph{keyint}) +Set the GOP size (I-picture period). + +@item preset (@emph{preset}) +Set the xeve preset. +Set the encoder preset value to determine encoding speed [fast, medium, slow, placebo] + +@item tune (@emph{tune}) +Set the encoder tune parameter [psnr, zerolatency] + +@item profile (@emph{profile}) +Set the encoder profile [0: baselie; 1: main] + +@item crf (@emph{crf}) +Set the quality for constant quality mode. +Constant rate factor <10..49> [default: 32] + +@item qp (@emph{qp}) +Set constant quantization rate control method parameter. +Quantization parameter qp <0..51> [default: 32] + +@item threads (@emph{threads}) +Force to use a specific number of threads + +@end table + @section libxvid Xvid MPEG-4 Part 2 encoder wrapper. diff --git a/doc/general_contents.texi b/doc/general_contents.texi index 2eeebd847d..31f1793b9e 100644 --- a/doc/general_contents.texi +++ b/doc/general_contents.texi @@ -343,6 +343,14 @@ libxavs2 is under the GNU Public License Version 2 or later details), you must upgrade FFmpeg's license to GPL in order to use it. @end float +@section eXtra-fast Essential Video Encoder (XEVE) + +FFmpeg can make use of the XEVE library for EVC video encoding. + +Go to @url{https://github.com/mpeg5/xeve} and follow the instructions for +installing the XEVE library. Then pass @code{--enable-libxeve} to configure to +enable it. + @section ZVBI ZVBI is a VBI decoding library which can be used by FFmpeg to decode DVB @@ -598,6 +606,7 @@ library: @item raw DTS @tab X @tab X @item raw DTS-HD @tab @tab X @item raw E-AC-3 @tab X @tab X +@item raw EVC @tab X @tab X @item raw FLAC @tab X @tab X @item raw GSM @tab @tab X @item raw H.261 @tab X @tab X @@ -943,6 +952,8 @@ following image formats are supported: @item Electronic Arts TQI video @tab @tab X @item Escape 124 @tab @tab X @item Escape 130 @tab @tab X +@item EVC / MPEG-5 Part 1 @tab X @tab X + @tab encoding and decoding supported through external librariy libxeve @item FFmpeg video codec #1 @tab X @tab X @tab lossless codec (fourcc: FFV1) @item Flash Screen Video v1 @tab X @tab X diff --git a/libavcodec/Makefile b/libavcodec/Makefile index 635b936cac..53ffdc5bbe 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -1130,6 +1130,7 @@ OBJS-$(CONFIG_LIBX264_ENCODER) += libx264.o OBJS-$(CONFIG_LIBX265_ENCODER) += libx265.o OBJS-$(CONFIG_LIBXAVS_ENCODER) += libxavs.o OBJS-$(CONFIG_LIBXAVS2_ENCODER) += libxavs2.o +OBJS-$(CONFIG_LIBXEVE_ENCODER) += libxeve.o OBJS-$(CONFIG_LIBXVID_ENCODER) += libxvid.o OBJS-$(CONFIG_LIBZVBI_TELETEXT_DECODER) += libzvbi-teletextdec.o ass.o diff --git a/libavcodec/allcodecs.c b/libavcodec/allcodecs.c index 385ee34803..acfade1893 100644 --- a/libavcodec/allcodecs.c +++ b/libavcodec/allcodecs.c @@ -814,6 +814,7 @@ extern LIBX264_CONST FFCodec ff_libx264_encoder; #endif extern const FFCodec ff_libx264rgb_encoder; extern FFCodec ff_libx265_encoder; +extern const FFCodec ff_libxeve_encoder; extern const FFCodec ff_libxavs_encoder; extern const FFCodec ff_libxavs2_encoder; extern const FFCodec ff_libxvid_encoder; diff --git a/libavcodec/libxeve.c b/libavcodec/libxeve.c new file mode 100644 index 0000000000..c10b7de023 --- /dev/null +++ b/libavcodec/libxeve.c @@ -0,0 +1,616 @@ +/* + * libxeve encoder + * EVC (MPEG-5 Essential Video Coding) encoding using XEVE MPEG-5 EVC encoder library + * + * Copyright (C) 2021 Dawid Kozinski + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include +#include + +#include + +#include "libavutil/internal.h" +#include "libavutil/common.h" +#include "libavutil/opt.h" +#include "libavutil/pixdesc.h" +#include "libavutil/pixfmt.h" +#include "libavutil/time.h" +#include "libavutil/cpu.h" +#include "libavutil/avstring.h" + +#include "avcodec.h" +#include "internal.h" +#include "packet_internal.h" +#include "codec_internal.h" +#include "profiles.h" +#include "encode.h" + +#define MAX_BS_BUF (16*1024*1024) + +/** + * Error codes + */ +#define XEVE_PARAM_BAD_NAME -100 +#define XEVE_PARAM_BAD_VALUE -200 + +/** + * Encoder states + * + * STATE_ENCODING - the encoder receives and processes input frames + * STATE_BUMPING - there are no more input frames, however the encoder still processes previously received data + */ +typedef enum State { + STATE_ENCODING, + STATE_BUMPING, +} State; + +/** + * The structure stores all the states associated with the instance of Xeve MPEG-5 EVC encoder + */ +typedef struct XeveContext { + const AVClass *class; + + XEVE id; // XEVE instance identifier + XEVE_CDSC cdsc; // coding parameters i.e profile, width & height of input frame, num of therads, frame rate ... + XEVE_BITB bitb; // bitstream buffer (output) + XEVE_STAT stat; // encoding status (output) + XEVE_IMGB imgb; // image buffer (input) + + State state; // encoder state (skipping, encoding, bumping) + + int profile_id; // encoder profile (main, baseline) + int preset_id; // preset of xeve ( fast, medium, slow, placebo) + int tune_id; // tune of xeve (psnr, zerolatency) + + // variables for rate control modes + int rc_mode; // Rate control mode [ 0(CQP) / 1(ABR) / 2(CRF) ] + int qp; // quantization parameter (QP) [0,51] + int crf; // constant rate factor (CRF) [10,49] + + int hash; // embed picture signature (HASH) for conformance checking in decoding + int sei_info; // embed Supplemental enhancement information while encoding + + int color_format; // input data color format: currently only XEVE_CF_YCBCR420 is supported + + AVDictionary *xeve_params; +} XeveContext; + +/** + * Convert FFmpeg pixel format (AVPixelFormat) to XEVE pre-defined color format + * + * @param[in] av_pix_fmt pixel format (@see https://ffmpeg.org/doxygen/trunk/pixfmt_8h.html#a9a8e335cf3be472042bc9f0cf80cd4c5) + * @param[out] xeve_col_fmt XEVE pre-defined color format (@see xeve.h) + * + * @return 0 on success, negative value on failure + */ +static int libxeve_color_fmt(enum AVPixelFormat av_pix_fmt, int *xeve_col_fmt) +{ + switch (av_pix_fmt) { + case AV_PIX_FMT_YUV420P: + *xeve_col_fmt = XEVE_CF_YCBCR420; + break; + case AV_PIX_FMT_YUV420P10: + *xeve_col_fmt = XEVE_CF_YCBCR420; + break; + default: + *xeve_col_fmt = XEVE_CF_UNKNOWN; + return AVERROR_INVALIDDATA; + } + + return 0; +} + +/** + * Convert FFmpeg pixel format (AVPixelFormat) into XEVE pre-defined color space + * + * @param[in] px_fmt pixel format (@see https://ffmpeg.org/doxygen/trunk/pixfmt_8h.html#a9a8e335cf3be472042bc9f0cf80cd4c5) + * + * @return XEVE pre-defined color space (@see xeve.h) on success, XEVE_CF_UNKNOWN on failure + */ +static int libxeve_color_space(enum AVPixelFormat av_pix_fmt) +{ + /* color space of input image */ + int cs = XEVE_CF_UNKNOWN; + + switch (av_pix_fmt) { + case AV_PIX_FMT_YUV420P: + cs = XEVE_CS_YCBCR420; + break; + case AV_PIX_FMT_YUV420P10: +#if AV_HAVE_BIGENDIAN + cs = XEVE_CS_SET(XEVE_CF_YCBCR420, 10, 1); +#else + cs = XEVE_CS_YCBCR420_10LE; +#endif + + break; + default: + cs = XEVE_CF_UNKNOWN; + break; + } + + return cs; +} + +/** + * The function returns a pointer to the object of the XEVE_CDSC type. + * XEVE_CDSC contains all encoder parameters that should be initialized before the encoder is used. + * + * The field values of the XEVE_CDSC structure are populated based on: + * - the corresponding field values of the AvCodecConetxt structure, + * - the xeve encoder specific option values, + * (the full list of options available for xeve encoder is displayed after executing the command ./ffmpeg --help encoder = libxeve) + * + * The order of processing input data and populating the XEVE_CDSC structure + * 1) first, the fields of the AVCodecContext structure corresponding to the provided input options are processed, + * (i.e -pix_fmt yuv420p -s:v 1920x1080 -r 30 -profile:v 0) + * 2) then xeve-specific options added as AVOption to the xeve AVCodec implementation + * (i.e -preset 0) + * + * Keep in mind that, there are options that can be set in different ways. + * In this case, please follow the above-mentioned order of processing. + * The most recent assignments overwrite the previous values. + * + * @param[in] avctx codec context (AVCodecContext) + * @param[out] cdsc contains all Xeve MPEG-5 EVC encoder encoder parameters that should be initialized before the encoder is use + * + * @return 0 on success, negative error code on failure + */ +static int get_conf(AVCodecContext *avctx, XEVE_CDSC *cdsc) +{ + XeveContext *xectx = NULL; + int ret; + + xectx = avctx->priv_data; + + /* initialize xeve_param struct with default values */ + ret = xeve_param_default(&cdsc->param); + if (XEVE_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Cannot set_default parameter\n"); + return AVERROR_EXTERNAL; + } + + /* read options from AVCodecContext */ + if (avctx->width > 0) + cdsc->param.w = avctx->width; + + if (avctx->height > 0) + cdsc->param.h = avctx->height; + + if (avctx->framerate.num > 0) { + // fps can be float number, but xeve API doesn't support it + cdsc->param.fps = lrintf(av_q2d(avctx->framerate)); + } + + // GOP size (key-frame interval, I-picture period) + cdsc->param.keyint = avctx->gop_size; // 0: only one I-frame at the first time; 1: every frame is coded in I-frame + + if (avctx->max_b_frames == 0 || avctx->max_b_frames == 1 || avctx->max_b_frames == 3 || + avctx->max_b_frames == 7 || avctx->max_b_frames == 15) // number of b-frames + cdsc->param.bframes = avctx->max_b_frames; + else { + av_log(avctx, AV_LOG_ERROR, "Incorrect value for maximum number of B frames: (%d) \n" + "Acceptable values for bf option (maximum number of B frames) are 0,1,3,7 or 15\n", avctx->max_b_frames); + return AVERROR_INVALIDDATA; + } + + cdsc->param.level_idc = avctx->level; + + if (avctx->rc_buffer_size) // VBV buf size + cdsc->param.vbv_bufsize = (int)(avctx->rc_buffer_size / 1000); + + cdsc->param.rc_type = xectx->rc_mode; + + if (xectx->rc_mode == XEVE_RC_CQP) + cdsc->param.qp = xectx->qp; + else if (xectx->rc_mode == XEVE_RC_ABR) { + if (avctx->bit_rate / 1000 > INT_MAX || avctx->rc_max_rate / 1000 > INT_MAX) { + av_log(avctx, AV_LOG_ERROR, "Not supported bitrate bit_rate and rc_max_rate > %d000\n", INT_MAX); + return AVERROR_INVALIDDATA; + } + cdsc->param.bitrate = (int)(avctx->bit_rate / 1000); + } else if (xectx->rc_mode == XEVE_RC_CRF) + cdsc->param.crf = xectx->crf; + else { + av_log(avctx, AV_LOG_ERROR, "Not supported rate control type: %d\n", xectx->rc_mode); + return AVERROR_INVALIDDATA; + } + + if (avctx->thread_count <= 0) { + int cpu_count = av_cpu_count(); + cdsc->param.threads = (cpu_count < XEVE_MAX_THREADS) ? cpu_count : XEVE_MAX_THREADS; + } else if (avctx->thread_count > XEVE_MAX_THREADS) + cdsc->param.threads = XEVE_MAX_THREADS; + else + cdsc->param.threads = avctx->thread_count; + + + libxeve_color_fmt(avctx->pix_fmt, &xectx->color_format); + + cdsc->param.cs = XEVE_CS_SET(xectx->color_format, cdsc->param.codec_bit_depth, AV_HAVE_BIGENDIAN); + + cdsc->max_bs_buf_size = MAX_BS_BUF; + + ret = xeve_param_ppt(&cdsc->param, xectx->profile_id, xectx->preset_id, xectx->tune_id); + if (XEVE_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Cannot set profile(%d), preset(%d), tune(%d)\n", xectx->profile_id, xectx->preset_id, xectx->tune_id); + return AVERROR_EXTERNAL; + } + + return 0; +} + +/** + * Set XEVE_CFG_SET_USE_PIC_SIGNATURE for encoder + * + * @param[in] logger context + * @param[in] id XEVE encodec instance identifier + * @param[in] ctx the structure stores all the states associated with the instance of Xeve MPEG-5 EVC encoder + * + * @return 0 on success, negative error code on failure + */ +static int set_extra_config(AVCodecContext *avctx, XEVE id, XeveContext *ctx) +{ + int ret, size; + size = 4; + + // embed SEI messages identifying encoder parameters and command line arguments + // - 0: off\n" + // - 1: emit sei info" + // + // SEI - Supplemental enhancement information contains information + // that is not necessary to decode the samples of coded pictures from VCL NAL units. + // Some SEI message information is required to check bitstream conformance + // and for output timing decoder conformance. + // @see ISO_IEC_23094-1_2020 7.4.3.5 + // @see ISO_IEC_23094-1_2020 Annex D + ret = xeve_config(id, XEVE_CFG_SET_SEI_CMD, &ctx->sei_info, &size); // sei_cmd_info + if (XEVE_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to set config for sei command info messages\n"); + return AVERROR_EXTERNAL; + } + + ret = xeve_config(id, XEVE_CFG_SET_USE_PIC_SIGNATURE, &ctx->hash, &size); + if (XEVE_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to set config for picture signature\n"); + return AVERROR_EXTERNAL; + } + + return 0; +} + +/** + * @brief Switch encoder to bumping mode + * + * @param id XEVE encodec instance identifier + * @return 0 on success, negative error code on failure + */ +static int setup_bumping(XEVE id) +{ + int val = 1; + int size = sizeof(int); + if (XEVE_FAILED(xeve_config(id, XEVE_CFG_SET_FORCE_OUT, (void *)(&val), &size))) + return AVERROR_EXTERNAL; + + return 0; +} + +/** + * @brief Initialize eXtra-fast Essential Video Encoder codec + * Create an encoder instance and allocate all the needed resources + * + * @param avctx codec context + * @return 0 on success, negative error code on failure + */ +static av_cold int libxeve_init(AVCodecContext *avctx) +{ + XeveContext *xectx = avctx->priv_data; + unsigned char *bs_buf = NULL; + int i; + int shift_h = 0; + int shift_v = 0; + int width_chroma = 0; + int height_chroma = 0; + XEVE_IMGB *imgb = NULL; + int ret = 0; + + XEVE_CDSC *cdsc = &(xectx->cdsc); + + /* allocate bitstream buffer */ + bs_buf = av_malloc(MAX_BS_BUF); + if (bs_buf == NULL) { + av_log(avctx, AV_LOG_ERROR, "Cannot allocate bitstream buffer\n"); + return AVERROR(ENOMEM); + } + xectx->bitb.addr = bs_buf; + xectx->bitb.bsize = MAX_BS_BUF; + + /* read configurations and set values for created descriptor (XEVE_CDSC) */ + if ((ret = get_conf(avctx, cdsc)) != 0) { + av_log(avctx, AV_LOG_ERROR, "Cannot get configuration\n"); + return AVERROR(EINVAL); + } + + if ((ret = xeve_param_check(&cdsc->param)) != 0) { + av_log(avctx, AV_LOG_ERROR, "Invalid configuration\n"); + return AVERROR(EINVAL); + } + + { + AVDictionaryEntry *en = NULL; + while (en = av_dict_get(xectx->xeve_params, "", en, AV_DICT_IGNORE_SUFFIX)) { + if ((ret = xeve_param_parse(&cdsc->param, en->key, en->value)) < 0) { + av_log(avctx, AV_LOG_WARNING, + "Error parsing option '%s = %s'.\n", + en->key, en->value); + } + } + } + + /* create encoder */ + xectx->id = xeve_create(cdsc, NULL); + if (xectx->id == NULL) { + av_log(avctx, AV_LOG_ERROR, "Cannot create XEVE encoder\n"); + return AVERROR_EXTERNAL; + } + + if ((ret = set_extra_config(avctx, xectx->id, xectx)) != 0) { + av_log(avctx, AV_LOG_ERROR, "Cannot set extra configuration\n"); + return AVERROR(EINVAL); + } + + if ((ret = av_pix_fmt_get_chroma_sub_sample(avctx->pix_fmt, &shift_h, &shift_v)) != 0) { + av_log(avctx, AV_LOG_ERROR, "Failed to get chroma shift\n"); + return AVERROR(EINVAL); + } + + // Chroma subsampling + // + // YUV format explanation + // shift_h == 1 && shift_v == 1 : YUV420 + // shift_h == 1 && shift_v == 0 : YUV422 + // shift_h == 0 && shift_v == 0 : YUV444 + // + width_chroma = AV_CEIL_RSHIFT(avctx->width, shift_h); + height_chroma = AV_CEIL_RSHIFT(avctx->height, shift_v); + + /* set default values for input image buffer */ + imgb = &xectx->imgb; + imgb->cs = libxeve_color_space(avctx->pix_fmt); + imgb->np = 3; /* only for yuv420p, yuv420ple */ + + for (i = 0; i < imgb->np; i++) + imgb->x[i] = imgb->y[i] = 0; + + imgb->w[0] = imgb->aw[0] = avctx->width; // width luma + imgb->w[1] = imgb->w[2] = imgb->aw[1] = imgb->aw[2] = width_chroma; + imgb->h[0] = imgb->ah[0] = avctx->height; // height luma + imgb->h[1] = imgb->h[2] = imgb->ah[1] = imgb->ah[2] = height_chroma; + + xectx->state = STATE_ENCODING; + + return 0; +} + +/** + * Encode raw data frame into EVC packet + * + * @param[in] avctx codec context + * @param[out] avpkt output AVPacket containing encoded data + * @param[in] frame AVFrame containing the raw data to be encoded + * @param[out] got_packet encoder sets to 0 or 1 to indicate that a + * non-empty packet was returned in pkt + * + * @return 0 on success, negative error code on failure + */ +static int libxeve_encode(AVCodecContext *avctx, AVPacket *avpkt, + const AVFrame *frame, int *got_packet) +{ + XeveContext *xectx = avctx->priv_data; + int ret = -1; + + // No more input frames are available but encoder still can have some data in its internal buffer to process + // and some frames to dump. + if (xectx->state == STATE_ENCODING && frame == NULL) { + if (setup_bumping(xectx->id) == 0) + xectx->state = STATE_BUMPING; // Entering bumping process + else { + av_log(avctx, AV_LOG_ERROR, "Failed to setup bumping\n"); + return 0; + } + } + + if (xectx->state == STATE_ENCODING) { + int i; + XEVE_IMGB *imgb = NULL; + + imgb = &xectx->imgb; + + for (i = 0; i < imgb->np; i++) { + imgb->a[i] = frame->data[i]; + imgb->s[i] = frame->linesize[i]; + } + + imgb->ts[XEVE_TS_PTS] = frame->pts; + + /* push image to encoder */ + ret = xeve_push(xectx->id, imgb); + if (XEVE_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "xeve_push() failed\n"); + return AVERROR_EXTERNAL; + } + } + if (xectx->state == STATE_ENCODING || xectx->state == STATE_BUMPING) { + /* encoding */ + ret = xeve_encode(xectx->id, &(xectx->bitb), &(xectx->stat)); + if (XEVE_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "xeve_encode() failed\n"); + return AVERROR_EXTERNAL; + } + + /* store bitstream */ + if (ret == XEVE_OK_OUT_NOT_AVAILABLE) { // Return OK but picture is not available yet + *got_packet = 0; + return 0; + } else if (ret == XEVE_OK) { + int av_pic_type; + + if (xectx->stat.write > 0) { + + ret = ff_get_encode_buffer(avctx, avpkt, xectx->stat.write, 0); + if (ret < 0) + return ret; + + memcpy(avpkt->data, xectx->bitb.addr, xectx->stat.write); + + avpkt->time_base.num = 1; + avpkt->time_base.den = xectx->cdsc.param.fps; + + avpkt->pts = xectx->bitb.ts[XEVE_TS_PTS]; + avpkt->dts = xectx->bitb.ts[XEVE_TS_DTS]; + + switch(xectx->stat.stype) { + case XEVE_ST_I: + av_pic_type = AV_PICTURE_TYPE_I; + avpkt->flags |= AV_PKT_FLAG_KEY; + break; + case XEVE_ST_P: + av_pic_type = AV_PICTURE_TYPE_P; + break; + case XEVE_ST_B: + av_pic_type = AV_PICTURE_TYPE_B; + break; + case XEVE_ST_UNKNOWN: + av_log(avctx, AV_LOG_ERROR, "Unknown slice type\n"); + return AVERROR_INVALIDDATA; + } + + ff_side_data_set_encoder_stats(avpkt, xectx->stat.qp * FF_QP2LAMBDA, NULL, 0, av_pic_type); + + *got_packet = 1; + } + } else if (ret == XEVE_OK_NO_MORE_FRM) { + // Return OK but no more frames + return 0; + } else { + av_log(avctx, AV_LOG_ERROR, "Invalid return value: %d\n", ret); + return AVERROR_EXTERNAL; + } + } else { + av_log(avctx, AV_LOG_ERROR, "Udefined encoder state\n"); + return AVERROR_INVALIDDATA; + } + + return 0; +} + +/** + * Destroy the encoder and release all the allocated resources + * + * @param avctx codec context + * @return 0 on success, negative error code on failure + */ +static av_cold int libxeve_close(AVCodecContext *avctx) +{ + XeveContext *xectx = avctx->priv_data; + + if (xectx->id) { + xeve_delete(xectx->id); + xectx->id = NULL; + } + + av_free(xectx->bitb.addr); /* release bitstream buffer */ + + return 0; +} + +#define OFFSET(x) offsetof(XeveContext, x) +#define VE AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_ENCODING_PARAM + +static const enum AVPixelFormat supported_pixel_formats[] = { + AV_PIX_FMT_YUV420P, + AV_PIX_FMT_YUV420P10, + AV_PIX_FMT_NONE +}; + +// Consider using following options (./ffmpeg --help encoder=libxeve) +// +static const AVOption libxeve_options[] = { + { "preset", "Encoding preset for setting encoding speed", OFFSET(preset_id), AV_OPT_TYPE_INT, { .i64 = XEVE_PRESET_MEDIUM }, XEVE_PRESET_DEFAULT, XEVE_PRESET_PLACEBO, VE, "preset" }, + { "default", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_PRESET_DEFAULT }, INT_MIN, INT_MAX, VE, "preset" }, + { "fast", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_PRESET_FAST }, INT_MIN, INT_MAX, VE, "preset" }, + { "medium", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_PRESET_MEDIUM }, INT_MIN, INT_MAX, VE, "preset" }, + { "slow", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_PRESET_SLOW }, INT_MIN, INT_MAX, VE, "preset" }, + { "placebo", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_PRESET_PLACEBO }, INT_MIN, INT_MAX, VE, "preset" }, + { "tune", "Tuning parameter for special purpose operation", OFFSET(tune_id), AV_OPT_TYPE_INT, { .i64 = XEVE_TUNE_NONE }, XEVE_TUNE_NONE, XEVE_TUNE_PSNR, VE, "tune"}, + { "none", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_TUNE_NONE }, INT_MIN, INT_MAX, VE, "tune" }, + { "zerolatency", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_TUNE_ZEROLATENCY }, INT_MIN, INT_MAX, VE, "tune" }, + { "psnr", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_TUNE_PSNR }, INT_MIN, INT_MAX, VE, "tune" }, + { "profile", "Encoding profile", OFFSET(profile_id), AV_OPT_TYPE_INT, { .i64 = XEVE_PROFILE_BASELINE }, XEVE_PROFILE_BASELINE, XEVE_PROFILE_MAIN, VE, "profile" }, + { "baseline", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_PROFILE_BASELINE }, INT_MIN, INT_MAX, VE, "profile" }, + { "main", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_PROFILE_MAIN }, INT_MIN, INT_MAX, VE, "profile" }, + { "rc_mode", "Rate control mode", OFFSET(rc_mode), AV_OPT_TYPE_INT, { .i64 = XEVE_RC_CQP }, XEVE_RC_CQP, XEVE_RC_CRF, VE, "rc_mode" }, + { "CQP", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_RC_CQP }, INT_MIN, INT_MAX, VE, "rc_mode" }, + { "ABR", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_RC_ABR }, INT_MIN, INT_MAX, VE, "rc_mode" }, + { "CRF", NULL, 0, AV_OPT_TYPE_CONST, { .i64 = XEVE_RC_CRF }, INT_MIN, INT_MAX, VE, "rc_mode" }, + { "qp", "Quantization parameter value for CQP rate control mode", OFFSET(qp), AV_OPT_TYPE_INT, { .i64 = 32 }, 0, 51, VE }, + { "crf", "Constant rate factor value for CRF rate control mode", OFFSET(crf), AV_OPT_TYPE_INT, { .i64 = 32 }, 10, 49, VE }, + { "hash", "Embed picture signature (HASH) for conformance checking in decoding", OFFSET(hash), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, 1, VE }, + { "sei_info", "Embed SEI messages identifying encoder parameters and command line arguments", OFFSET(sei_info), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, 1, VE }, + { "xeve-params", "Override the xeve configuration using a :-separated list of key=value parameters", OFFSET(xeve_params), AV_OPT_TYPE_DICT, { 0 }, 0, 0, VE }, + { NULL } +}; + +static const AVClass libxeve_class = { + .class_name = "libxeve", + .item_name = av_default_item_name, + .option = libxeve_options, + .version = LIBAVUTIL_VERSION_INT, +}; + +/** + * libavcodec generic global options, which can be set on all the encoders and decoders + * @see https://www.ffmpeg.org/ffmpeg-codecs.html#Codec-Options + */ +static const FFCodecDefault libxeve_defaults[] = { + { "b", "0" }, // bitrate in terms of kilo-bits per second + { "g", "0" }, // gop_size (key-frame interval 0: only one I-frame at the first time; 1: every frame is coded in I-frame) + { "bf", "15"}, // maximum number of B frames (0: no B-frames, 1,3,7,15) + { "threads", "0"}, // number of threads to be used (0: automatically select the number of threads to set) + { NULL }, +}; + +const FFCodec ff_libxeve_encoder = { + .p.name = "libxeve", + .p.long_name = NULL_IF_CONFIG_SMALL("libxeve MPEG-5 EVC"), + .p.type = AVMEDIA_TYPE_VIDEO, + .p.id = AV_CODEC_ID_EVC, + .init = libxeve_init, + FF_CODEC_ENCODE_CB(libxeve_encode), + .close = libxeve_close, + .priv_data_size = sizeof(XeveContext), + .p.priv_class = &libxeve_class, + .defaults = libxeve_defaults, + .p.capabilities = AV_CODEC_CAP_DELAY | AV_CODEC_CAP_OTHER_THREADS | AV_CODEC_CAP_DR1, + .p.profiles = NULL_IF_CONFIG_SMALL(ff_evc_profiles), + .p.wrapper_name = "libxeve", + .p.pix_fmts = supported_pixel_formats, + .caps_internal = FF_CODEC_CAP_INIT_CLEANUP | FF_CODEC_CAP_NOT_INIT_THREADSAFE, +};