Message ID | 20230102125458.1642-1-d.kozinski@samsung.com |
---|---|
State | New |
Headers | show |
Series | [FFmpeg-devel,v16,1/9] avcodec/evc: MPEG-5 EVC codec registration | expand |
Context | Check | Description |
---|---|---|
andriy/make_x86 | success | Make finished |
andriy/make_fate_x86 | success | Make fate finished |
Jan 2, 2023, 13:55 by d.kozinski@samsung.com: > - Added EVC decoder wrapper > - Changes in project configuration file and libavcodec Makefile > - Added documentation for xevd wrapper > > Signed-off-by: Dawid Kozinski <d.kozinski@samsung.com> > --- > configure | 4 + > doc/decoders.texi | 24 +++ > doc/general_contents.texi | 10 +- > libavcodec/Makefile | 1 + > libavcodec/allcodecs.c | 1 + > libavcodec/libxevd.c | 400 ++++++++++++++++++++++++++++++++++++++ > 6 files changed, 439 insertions(+), 1 deletion(-) > create mode 100644 libavcodec/libxevd.c > > diff --git a/configure b/configure > index 392169a4ee..f9b83b7166 100755 > --- a/configure > +++ b/configure > @@ -292,6 +292,7 @@ External library support: > --enable-libx264 enable H.264 encoding via x264 [no] > --enable-libx265 enable HEVC encoding via x265 [no] > --enable-libxeve enable EVC encoding via libxeve [no] > + --enable-libxevd enable EVC decoding via libxevd [no] > --enable-libxavs enable AVS encoding via xavs [no] > --enable-libxavs2 enable AVS2 encoding via xavs2 [no] > --enable-libxcb enable X11 grabbing using XCB [autodetect] > @@ -1863,6 +1864,7 @@ EXTERNAL_LIBRARY_LIST=" > libvorbis > libvpx > libwebp > + libxevd > libxeve > libxml2 > libzimg > @@ -3396,6 +3398,7 @@ libx265_encoder_deps="libx265" > libx265_encoder_select="atsc_a53" > libxavs_encoder_deps="libxavs" > libxavs2_encoder_deps="libxavs2" > +libxevd_decoder_deps="libxevd" > libxeve_encoder_deps="libxeve" > libxvid_encoder_deps="libxvid" > libzvbi_teletext_decoder_deps="libzvbi" > @@ -6721,6 +6724,7 @@ enabled libx265 && require_pkg_config libx265 x265 x265.h x265_api_get > require_cpp_condition libx265 x265.h "X265_BUILD >= 89" > enabled libxavs && require libxavs "stdint.h xavs.h" xavs_encoder_encode "-lxavs $pthreads_extralibs $libm_extralibs" > enabled libxavs2 && require_pkg_config libxavs2 "xavs2 >= 1.3.0" "stdint.h xavs2.h" xavs2_api_get > +enabled libxevd && require_pkg_config libxevd "xevd >= 0.4.0" "xevd.h" xevd_decode > enabled libxeve && require_pkg_config libxeve "xeve >= 0.4.0" "xeve.h" xeve_encode > enabled libxvid && require libxvid xvid.h xvid_global -lxvidcore > enabled libzimg && require_pkg_config libzimg "zimg >= 2.7.0" zimg.h zimg_get_api_version > diff --git a/doc/decoders.texi b/doc/decoders.texi > index 5ba85cf9b1..54720ee8b4 100644 > --- a/doc/decoders.texi > +++ b/doc/decoders.texi > @@ -130,6 +130,30 @@ Set amount of frame threads to use during decoding. The default value is 0 (auto > > @end table > > +@section libxevd > + > +eXtra-fast Essential Video Decoder (XEVD) MPEG-5 EVC decoder wrapper. > + > +This decoder requires the presence of the libxevd headers and library > +during configuration. You need to explicitly configure the build with > +@option{--enable-libxevd}. > + > +The xevd project website is at @url{https://github.com/mpeg5/xevd}. > + > +@subsection Options > + > +The following options are supported by the libxevd wrapper. > +The xevd-equivalent options or values are listed in parentheses for easy migration. > + > +To get a more accurate and extensive documentation of the libxevd options, > +invoke the command @code{xevd_app --help} or consult the libxevd documentation. > + > +@table @option > +@item threads (@emph{threads}) > +Force to use a specific number of threads > + > +@end table > + > @section QSV Decoders > > The family of Intel QuickSync Video decoders (VC1, MPEG-2, H.264, HEVC, > diff --git a/doc/general_contents.texi b/doc/general_contents.texi > index bcff3e29b7..38940d497a 100644 > --- a/doc/general_contents.texi > +++ b/doc/general_contents.texi > @@ -351,6 +351,14 @@ Go to @url{https://github.com/mpeg5/xeve} and follow the instructions for > installing the XEVE library. Then pass @code{--enable-libxeve} to configure to > enable it. > > +@section eXtra-fast Essential Video Decoder (XEVD) > + > +FFmpeg can make use of the XEVD library for EVC video decoding. > + > +Go to @url{https://github.com/mpeg5/xevd} and follow the instructions for > +installing the XEVD library. Then pass @code{--enable-libxevd} to configure to > +enable it. > + > @section ZVBI > > ZVBI is a VBI decoding library which can be used by FFmpeg to decode DVB > @@ -944,7 +952,7 @@ following image formats are supported: > @item Escape 124 @tab @tab X > @item Escape 130 @tab @tab X > @item EVC / MPEG-5 Part 1 @tab X @tab X > - @tab encoding supported through external library libxeve > + @tab encoding and decoding supported through external libraries libxeve and libxevd > @item FFmpeg video codec #1 @tab X @tab X > @tab lossless codec (fourcc: FFV1) > @item Flash Screen Video v1 @tab X @tab X > diff --git a/libavcodec/Makefile b/libavcodec/Makefile > index 8119a4cb6d..f0c41fb021 100644 > --- a/libavcodec/Makefile > +++ b/libavcodec/Makefile > @@ -1119,6 +1119,7 @@ OBJS-$(CONFIG_LIBX264_ENCODER) += libx264.o > OBJS-$(CONFIG_LIBX265_ENCODER) += libx265.o > OBJS-$(CONFIG_LIBXAVS_ENCODER) += libxavs.o > OBJS-$(CONFIG_LIBXAVS2_ENCODER) += libxavs2.o > +OBJS-$(CONFIG_LIBXEVD_DECODER) += libxevd.o > OBJS-$(CONFIG_LIBXEVE_ENCODER) += libxeve.o > OBJS-$(CONFIG_LIBXVID_ENCODER) += libxvid.o > OBJS-$(CONFIG_LIBZVBI_TELETEXT_DECODER) += libzvbi-teletextdec.o ass.o > diff --git a/libavcodec/allcodecs.c b/libavcodec/allcodecs.c > index 514f791fdb..956e9d454f 100644 > --- a/libavcodec/allcodecs.c > +++ b/libavcodec/allcodecs.c > @@ -810,6 +810,7 @@ extern LIBX264_CONST FFCodec ff_libx264_encoder; > extern const FFCodec ff_libx264rgb_encoder; > extern FFCodec ff_libx265_encoder; > extern const FFCodec ff_libxeve_encoder; > +extern const FFCodec ff_libxevd_decoder; > extern const FFCodec ff_libxavs_encoder; > extern const FFCodec ff_libxavs2_encoder; > extern const FFCodec ff_libxvid_encoder; > diff --git a/libavcodec/libxevd.c b/libavcodec/libxevd.c > new file mode 100644 > index 0000000000..5f9976f357 > --- /dev/null > +++ b/libavcodec/libxevd.c > @@ -0,0 +1,400 @@ > +/* > + * libxevd decoder > + * EVC (MPEG-5 Essential Video Coding) decoding using XEVD MPEG-5 EVC decoder library > + * > + * Copyright (C) 2021 Dawid Kozinski <d.kozinski@samsung.com> > + * > + * This file is part of FFmpeg. > + * > + * FFmpeg is free software; you can redistribute it and/or > + * modify it under the terms of the GNU Lesser General Public > + * License as published by the Free Software Foundation; either > + * version 2.1 of the License, or (at your option) any later version. > + * > + * FFmpeg is distributed in the hope that it will be useful, > + * but WITHOUT ANY WARRANTY; without even the implied warranty of > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > + * Lesser General Public License for more details. > + * > + * You should have received a copy of the GNU Lesser General Public > + * License along with FFmpeg; if not, write to the Free Software > + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA > + */ > + > +#include <float.h> > +#include <stdlib.h> > + > +#include <xevd.h> > + > +#include "libavutil/internal.h" > +#include "libavutil/common.h" > +#include "libavutil/opt.h" > +#include "libavutil/pixdesc.h" > +#include "libavutil/pixfmt.h" > +#include "libavutil/imgutils.h" > +#include "libavutil/cpu.h" > + > +#include "avcodec.h" > +#include "internal.h" > +#include "packet_internal.h" > +#include "codec_internal.h" > +#include "profiles.h" > +#include "decode.h" > + > +#define XEVD_PARAM_BAD_NAME -1 > +#define XEVD_PARAM_BAD_VALUE -2 > + > +#define EVC_NALU_HEADER_SIZE 2 /* byte */ > + > +/** > + * The structure stores all the states associated with the instance of Xeve MPEG-5 EVC decoder > + */ > +typedef struct XevdContext { > + const AVClass *class; > + > + XEVD id; // XEVD instance identifier @see xevd.h > + XEVD_CDSC cdsc; // decoding parameters @see xevd.h > + int coded_picture_number; > +} XevdContext; > + > +/** > + * The function populates the XEVD_CDSC structure. > + * XEVD_CDSC contains all decoder parameters that should be initialized before its use. > + * > + * @param[in] avctx codec context > + * @param[out] cdsc contains all decoder parameters that should be initialized before its use > + * > + */ > +static void get_conf(AVCodecContext *avctx, XEVD_CDSC *cdsc) > +{ > + int cpu_count = av_cpu_count(); > + > + /* clear XEVS_CDSC structure */ > + memset(cdsc, 0, sizeof(XEVD_CDSC)); > + > + /* init XEVD_CDSC */ > + if (avctx->thread_count <= 0) > + cdsc->threads = (cpu_count < XEVD_MAX_TASK_CNT) ? cpu_count : XEVD_MAX_TASK_CNT; > + else if (avctx->thread_count > XEVD_MAX_TASK_CNT) > + cdsc->threads = XEVD_MAX_TASK_CNT; > + else > + cdsc->threads = avctx->thread_count; > +} > + > +/** > + * Read NAL unit length > + * @param bs input data (bitstream) > + * @return the length of NAL unit on success, 0 value on failure > + */ > +static uint32_t read_nal_unit_length(const uint8_t *bs, int bs_size, AVCodecContext *avctx) > +{ > + uint32_t len = 0; > + XEVD_INFO info; > + int ret; > + > + if (bs_size == XEVD_NAL_UNIT_LENGTH_BYTE) { > + ret = xevd_info((void *)bs, XEVD_NAL_UNIT_LENGTH_BYTE, 1, &info); > + if (XEVD_FAILED(ret)) { > + av_log(avctx, AV_LOG_ERROR, "Cannot get bitstream information\n"); > + return 0; > + } > + len = info.nalu_len; > + if (len == 0) { > + av_log(avctx, AV_LOG_ERROR, "Invalid bitstream size! [%d]\n", bs_size); > + return 0; > + } > + } > + > + return len; > +} > + > +/** > + * @param[in] xectx the structure that stores all the state associated with the instance of Xeve MPEG-5 EVC decoder > + * @param[out] avctx codec context > + * @return 0 on success, negative value on failure > + */ > +static int export_stream_params(const XevdContext *xectx, AVCodecContext *avctx) > +{ > + int ret; > + int size; > + int color_space; > + > + avctx->pix_fmt = AV_PIX_FMT_YUV420P10; > + > + size = 4; > + ret = xevd_config(xectx->id, XEVD_CFG_GET_CODED_WIDTH, &avctx->coded_width, &size); > + if (XEVD_FAILED(ret)) { > + av_log(avctx, AV_LOG_ERROR, "Failed to get coded_width\n"); > + return AVERROR_EXTERNAL; > + } > + > + ret = xevd_config(xectx->id, XEVD_CFG_GET_CODED_HEIGHT, &avctx->coded_height, &size); > + if (XEVD_FAILED(ret)) { > + av_log(avctx, AV_LOG_ERROR, "Failed to get coded_height\n"); > + return AVERROR_EXTERNAL; > + } > + > + ret = xevd_config(xectx->id, XEVD_CFG_GET_WIDTH, &avctx->width, &size); > + if (XEVD_FAILED(ret)) { > + av_log(avctx, AV_LOG_ERROR, "Failed to get width\n"); > + return AVERROR_EXTERNAL; > + } > + > + ret = xevd_config(xectx->id, XEVD_CFG_GET_HEIGHT, &avctx->height, &size); > + if (XEVD_FAILED(ret)) { > + av_log(avctx, AV_LOG_ERROR, "Failed to get height\n"); > + return AVERROR_EXTERNAL; > + } > + > + ret = xevd_config(xectx->id, XEVD_CFG_GET_COLOR_SPACE, &color_space, &size); > + if (XEVD_FAILED(ret)) { > + av_log(avctx, AV_LOG_ERROR, "Failed to get color_space\n"); > + return AVERROR_EXTERNAL; > + } > + switch(color_space) { > + case XEVD_CS_YCBCR400_10LE: > + avctx->pix_fmt = AV_PIX_FMT_GRAY10LE; > + break; > + case XEVD_CS_YCBCR420_10LE: > + avctx->pix_fmt = AV_PIX_FMT_YUV420P10LE; > + break; > + case XEVD_CS_YCBCR422_10LE: > + avctx->pix_fmt = AV_PIX_FMT_YUV422P10LE; > + break; > + case XEVD_CS_YCBCR444_10LE: > + avctx->pix_fmt = AV_PIX_FMT_YUV444P10LE; > + break; > + default: > + av_log(avctx, AV_LOG_ERROR, "Unknown color space\n"); > + avctx->pix_fmt = AV_PIX_FMT_NONE; > + return AVERROR_INVALIDDATA; > + } > + > + // the function returns sps->num_reorder_pics > + ret = xevd_config(xectx->id, XEVD_CFG_GET_MAX_CODING_DELAY, &avctx->max_b_frames, &size); > + if (XEVD_FAILED(ret)) { > + av_log(avctx, AV_LOG_ERROR, "Failed to get max_coding_delay\n"); > + return AVERROR_EXTERNAL; > + } > + > + avctx->has_b_frames = (avctx->max_b_frames) ? 1 : 0; > + > + avctx->color_primaries = AVCOL_PRI_UNSPECIFIED; > + avctx->color_trc = AVCOL_TRC_UNSPECIFIED; > + avctx->colorspace = AVCOL_SPC_UNSPECIFIED; > Even if the library doesn't currently support getting those, could you at least not reset them? Containers may carry their own colorspace fields that will be set.. > + > + return 0; > +} > + > +/** > + * @brief Copy image in imgb to frame. > + * > + * @param avctx codec context > + * @param[in] imgb > + * @param[out] frame > + * @return 0 on success, negative value on failure > + */ > +static int libxevd_image_copy(struct AVCodecContext *avctx, XEVD_IMGB *imgb, struct AVFrame *frame) > +{ > + int ret; > + if (imgb->cs != XEVD_CS_YCBCR420_10LE) { > + av_log(avctx, AV_LOG_ERROR, "Not supported pixel format: %s\n", av_get_pix_fmt_name(avctx->pix_fmt)); > + return AVERROR_INVALIDDATA; > + } > + > + if (imgb->w[0] != avctx->width || imgb->h[0] != avctx->height) { // stream resolution changed > + if (ff_set_dimensions(avctx, imgb->w[0], imgb->h[0]) < 0) { > + av_log(avctx, AV_LOG_ERROR, "Cannot set new dimension\n"); > + return AVERROR_INVALIDDATA; > + } > + } > + > + if (ret = ff_get_buffer(avctx, frame, 0) < 0) > + return ret; > + > + av_image_copy(frame->data, frame->linesize, (const uint8_t **)imgb->a, > + imgb->s, avctx->pix_fmt, > + imgb->w[0], imgb->h[0]); > + > + return 0; > +} > + > +/** > + * Initialize decoder > + * Create a decoder instance and allocate all the needed resources > + * > + * @param avctx codec context > + * @return 0 on success, negative error code on failure > + */ > +static av_cold int libxevd_init(AVCodecContext *avctx) > +{ > + XevdContext *xectx = avctx->priv_data; > + XEVD_CDSC *cdsc = &(xectx->cdsc); > + > + /* read configurations and set values for created descriptor (XEVD_CDSC) */ > + get_conf(avctx, cdsc); > + > + /* create decoder */ > + xectx->id = xevd_create(&(xectx->cdsc), NULL); > + if (xectx->id == NULL) { > + av_log(avctx, AV_LOG_ERROR, "Cannot create XEVD encoder\n"); > + return AVERROR_EXTERNAL; > + } > + > + return 0; > +} > + > +/** > + * Decode frame with decoupled packet/frame dataflow > + * > + * @param avctx codec context > + * @param[out] frame decoded frame > + * > + * @return 0 on success, negative error code on failure > + */ > +static int libxevd_receive_frame(AVCodecContext *avctx, AVFrame *frame) > +{ > + XevdContext *xectx = NULL; > + int xevd_ret; > + int ret = 0; > + > + xectx = avctx->priv_data; > + > + /* poll for new frame */ > + { > + XEVD_IMGB *imgb = NULL; > + > + xevd_ret = xevd_pull(xectx->id, &imgb); > + if (XEVD_SUCCEEDED(ret)) { > + if (imgb) { > + > + frame->coded_picture_number = imgb->ts[XEVD_TS_DTS]; > + frame->display_picture_number = imgb->ts[XEVD_TS_PTS]; > You're not setting any output timestamps on the frame? Shouldn't you at least copy the properties from the source packet? I get that it's just the first version of the decoder, so it's expected to be spartan, but not having proper timestamp handling is a bit below our standards. > + > + ret = libxevd_image_copy(avctx, imgb, frame); > + > + // xevd_pull uses pool of objects of type XEVD_IMGB. > + // The pool size is equal MAX_PB_SIZE (26), so release object when it is no more needed > + imgb->release(imgb); > + imgb = NULL; > + return ret; > + } > + } else { > + if (imgb) { // already has a decoded image > + imgb->release(imgb); > + imgb = NULL; > + } > + } > + } > + > + /* feed decoder */ > + { > + XEVD_STAT stat; > + XEVD_BITB bitb; > + int nalu_size, bs_read_pos, dec_read_bytes; > + > + AVPacket pkt = {0}; > + > + pkt.data = NULL; > + pkt.size = 0; > + > Use av_packet_alloc, we've deprecated sizeof(AVPacket) from the API and shouldn't use more of it internally. > + > + nalu_size = read_nal_unit_length(pkt.data + bs_read_pos, XEVD_NAL_UNIT_LENGTH_BYTE, avctx); > + if (nalu_size == 0) { > + av_log(avctx, AV_LOG_ERROR, "Invalid bitstream\n"); > + ret = AVERROR_INVALIDDATA; > + goto ERR; > + } > + bs_read_pos += XEVD_NAL_UNIT_LENGTH_BYTE; > + > + bitb.addr = pkt.data + bs_read_pos; > + bitb.ssize = nalu_size; > + bitb.ts[XEVD_TS_DTS] = xectx->coded_picture_number; > + > + /* main decoding block */ > + xevd_ret = xevd_decode(xectx->id, &bitb, &stat); > + if (XEVD_FAILED(xevd_ret)) { > + av_log(avctx, AV_LOG_ERROR, "Failed to decode bitstream\n"); > + ret = AVERROR_EXTERNAL; > + goto ERR; > + } > + > + bs_read_pos += nalu_size; > + dec_read_bytes += nalu_size; > + > + if (stat.nalu_type == XEVD_NUT_SPS) { // EVC stream parameters changed > + if ((ret = export_stream_params(xectx, avctx)) != 0) > + goto ERR; > + } else if (stat.nalu_type == XEVD_NUT_IDR || stat.nalu_type == XEVD_NUT_NONIDR) { > + xectx->coded_picture_number++; > + } > + > + if (stat.read != dec_read_bytes) { > + av_log(avctx, AV_LOG_INFO, "Different reading of bitstream (in:%d, read:%d)\n", nalu_size, stat.read); > + ret = AVERROR_EXTERNAL; > + goto ERR; > + } > + } > + } > + > +ERR: > + av_packet_unref(&pkt); > + return ret; > + } > + > + return 0; > +} > + > +/** > + * Destroy decoder > + * > + * @param avctx codec context > + * @return 0 on success > + */ > +static av_cold int libxevd_close(AVCodecContext *avctx) > +{ > + XevdContext *xectx = avctx->priv_data; > + if (xectx->id) { > + xevd_delete(xectx->id); > + xectx->id = NULL; > + } > + return 0; > +} > + > +#define OFFSET(x) offsetof(XevdContext, x) > +#define VD AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_DECODING_PARAM > + > +static const AVClass libxevd_class = { > + .class_name = "libxevd", > + .item_name = av_default_item_name, > + .version = LIBAVUTIL_VERSION_INT, > +}; > + > +const FFCodec ff_libxevd_decoder = { > + .p.name = "evc", > + .p.long_name = NULL_IF_CONFIG_SMALL("EVC / MPEG-5 Essential Video Coding (EVC)"), > + .p.type = AVMEDIA_TYPE_VIDEO, > + .p.id = AV_CODEC_ID_EVC, > + .init = libxevd_init, > + FF_CODEC_RECEIVE_FRAME_CB(libxevd_receive_frame), > + .close = libxevd_close, > + .priv_data_size = sizeof(XevdContext), > + .p.priv_class = &libxevd_class, > + .p.capabilities = AV_CODEC_CAP_DELAY | AV_CODEC_CAP_OTHER_THREADS | AV_CODEC_CAP_AVOID_PROBING, > + .p.profiles = NULL_IF_CONFIG_SMALL(ff_evc_profiles), > + .p.wrapper_name = "libxevd", > + .caps_internal = FF_CODEC_CAP_INIT_CLEANUP | FF_CODEC_CAP_NOT_INIT_THREADSAFE, > +}; > Rest doesn't look too bad, except for the coding style and excessive doxy.
diff --git a/configure b/configure index 392169a4ee..f9b83b7166 100755 --- a/configure +++ b/configure @@ -292,6 +292,7 @@ External library support: --enable-libx264 enable H.264 encoding via x264 [no] --enable-libx265 enable HEVC encoding via x265 [no] --enable-libxeve enable EVC encoding via libxeve [no] + --enable-libxevd enable EVC decoding via libxevd [no] --enable-libxavs enable AVS encoding via xavs [no] --enable-libxavs2 enable AVS2 encoding via xavs2 [no] --enable-libxcb enable X11 grabbing using XCB [autodetect] @@ -1863,6 +1864,7 @@ EXTERNAL_LIBRARY_LIST=" libvorbis libvpx libwebp + libxevd libxeve libxml2 libzimg @@ -3396,6 +3398,7 @@ libx265_encoder_deps="libx265" libx265_encoder_select="atsc_a53" libxavs_encoder_deps="libxavs" libxavs2_encoder_deps="libxavs2" +libxevd_decoder_deps="libxevd" libxeve_encoder_deps="libxeve" libxvid_encoder_deps="libxvid" libzvbi_teletext_decoder_deps="libzvbi" @@ -6721,6 +6724,7 @@ enabled libx265 && require_pkg_config libx265 x265 x265.h x265_api_get require_cpp_condition libx265 x265.h "X265_BUILD >= 89" enabled libxavs && require libxavs "stdint.h xavs.h" xavs_encoder_encode "-lxavs $pthreads_extralibs $libm_extralibs" enabled libxavs2 && require_pkg_config libxavs2 "xavs2 >= 1.3.0" "stdint.h xavs2.h" xavs2_api_get +enabled libxevd && require_pkg_config libxevd "xevd >= 0.4.0" "xevd.h" xevd_decode enabled libxeve && require_pkg_config libxeve "xeve >= 0.4.0" "xeve.h" xeve_encode enabled libxvid && require libxvid xvid.h xvid_global -lxvidcore enabled libzimg && require_pkg_config libzimg "zimg >= 2.7.0" zimg.h zimg_get_api_version diff --git a/doc/decoders.texi b/doc/decoders.texi index 5ba85cf9b1..54720ee8b4 100644 --- a/doc/decoders.texi +++ b/doc/decoders.texi @@ -130,6 +130,30 @@ Set amount of frame threads to use during decoding. The default value is 0 (auto @end table +@section libxevd + +eXtra-fast Essential Video Decoder (XEVD) MPEG-5 EVC decoder wrapper. + +This decoder requires the presence of the libxevd headers and library +during configuration. You need to explicitly configure the build with +@option{--enable-libxevd}. + +The xevd project website is at @url{https://github.com/mpeg5/xevd}. + +@subsection Options + +The following options are supported by the libxevd wrapper. +The xevd-equivalent options or values are listed in parentheses for easy migration. + +To get a more accurate and extensive documentation of the libxevd options, +invoke the command @code{xevd_app --help} or consult the libxevd documentation. + +@table @option +@item threads (@emph{threads}) +Force to use a specific number of threads + +@end table + @section QSV Decoders The family of Intel QuickSync Video decoders (VC1, MPEG-2, H.264, HEVC, diff --git a/doc/general_contents.texi b/doc/general_contents.texi index bcff3e29b7..38940d497a 100644 --- a/doc/general_contents.texi +++ b/doc/general_contents.texi @@ -351,6 +351,14 @@ Go to @url{https://github.com/mpeg5/xeve} and follow the instructions for installing the XEVE library. Then pass @code{--enable-libxeve} to configure to enable it. +@section eXtra-fast Essential Video Decoder (XEVD) + +FFmpeg can make use of the XEVD library for EVC video decoding. + +Go to @url{https://github.com/mpeg5/xevd} and follow the instructions for +installing the XEVD library. Then pass @code{--enable-libxevd} to configure to +enable it. + @section ZVBI ZVBI is a VBI decoding library which can be used by FFmpeg to decode DVB @@ -944,7 +952,7 @@ following image formats are supported: @item Escape 124 @tab @tab X @item Escape 130 @tab @tab X @item EVC / MPEG-5 Part 1 @tab X @tab X - @tab encoding supported through external library libxeve + @tab encoding and decoding supported through external libraries libxeve and libxevd @item FFmpeg video codec #1 @tab X @tab X @tab lossless codec (fourcc: FFV1) @item Flash Screen Video v1 @tab X @tab X diff --git a/libavcodec/Makefile b/libavcodec/Makefile index 8119a4cb6d..f0c41fb021 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -1119,6 +1119,7 @@ OBJS-$(CONFIG_LIBX264_ENCODER) += libx264.o OBJS-$(CONFIG_LIBX265_ENCODER) += libx265.o OBJS-$(CONFIG_LIBXAVS_ENCODER) += libxavs.o OBJS-$(CONFIG_LIBXAVS2_ENCODER) += libxavs2.o +OBJS-$(CONFIG_LIBXEVD_DECODER) += libxevd.o OBJS-$(CONFIG_LIBXEVE_ENCODER) += libxeve.o OBJS-$(CONFIG_LIBXVID_ENCODER) += libxvid.o OBJS-$(CONFIG_LIBZVBI_TELETEXT_DECODER) += libzvbi-teletextdec.o ass.o diff --git a/libavcodec/allcodecs.c b/libavcodec/allcodecs.c index 514f791fdb..956e9d454f 100644 --- a/libavcodec/allcodecs.c +++ b/libavcodec/allcodecs.c @@ -810,6 +810,7 @@ extern LIBX264_CONST FFCodec ff_libx264_encoder; extern const FFCodec ff_libx264rgb_encoder; extern FFCodec ff_libx265_encoder; extern const FFCodec ff_libxeve_encoder; +extern const FFCodec ff_libxevd_decoder; extern const FFCodec ff_libxavs_encoder; extern const FFCodec ff_libxavs2_encoder; extern const FFCodec ff_libxvid_encoder; diff --git a/libavcodec/libxevd.c b/libavcodec/libxevd.c new file mode 100644 index 0000000000..5f9976f357 --- /dev/null +++ b/libavcodec/libxevd.c @@ -0,0 +1,400 @@ +/* + * libxevd decoder + * EVC (MPEG-5 Essential Video Coding) decoding using XEVD MPEG-5 EVC decoder library + * + * Copyright (C) 2021 Dawid Kozinski <d.kozinski@samsung.com> + * + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include <float.h> +#include <stdlib.h> + +#include <xevd.h> + +#include "libavutil/internal.h" +#include "libavutil/common.h" +#include "libavutil/opt.h" +#include "libavutil/pixdesc.h" +#include "libavutil/pixfmt.h" +#include "libavutil/imgutils.h" +#include "libavutil/cpu.h" + +#include "avcodec.h" +#include "internal.h" +#include "packet_internal.h" +#include "codec_internal.h" +#include "profiles.h" +#include "decode.h" + +#define XEVD_PARAM_BAD_NAME -1 +#define XEVD_PARAM_BAD_VALUE -2 + +#define EVC_NALU_HEADER_SIZE 2 /* byte */ + +/** + * The structure stores all the states associated with the instance of Xeve MPEG-5 EVC decoder + */ +typedef struct XevdContext { + const AVClass *class; + + XEVD id; // XEVD instance identifier @see xevd.h + XEVD_CDSC cdsc; // decoding parameters @see xevd.h + int coded_picture_number; +} XevdContext; + +/** + * The function populates the XEVD_CDSC structure. + * XEVD_CDSC contains all decoder parameters that should be initialized before its use. + * + * @param[in] avctx codec context + * @param[out] cdsc contains all decoder parameters that should be initialized before its use + * + */ +static void get_conf(AVCodecContext *avctx, XEVD_CDSC *cdsc) +{ + int cpu_count = av_cpu_count(); + + /* clear XEVS_CDSC structure */ + memset(cdsc, 0, sizeof(XEVD_CDSC)); + + /* init XEVD_CDSC */ + if (avctx->thread_count <= 0) + cdsc->threads = (cpu_count < XEVD_MAX_TASK_CNT) ? cpu_count : XEVD_MAX_TASK_CNT; + else if (avctx->thread_count > XEVD_MAX_TASK_CNT) + cdsc->threads = XEVD_MAX_TASK_CNT; + else + cdsc->threads = avctx->thread_count; +} + +/** + * Read NAL unit length + * @param bs input data (bitstream) + * @return the length of NAL unit on success, 0 value on failure + */ +static uint32_t read_nal_unit_length(const uint8_t *bs, int bs_size, AVCodecContext *avctx) +{ + uint32_t len = 0; + XEVD_INFO info; + int ret; + + if (bs_size == XEVD_NAL_UNIT_LENGTH_BYTE) { + ret = xevd_info((void *)bs, XEVD_NAL_UNIT_LENGTH_BYTE, 1, &info); + if (XEVD_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Cannot get bitstream information\n"); + return 0; + } + len = info.nalu_len; + if (len == 0) { + av_log(avctx, AV_LOG_ERROR, "Invalid bitstream size! [%d]\n", bs_size); + return 0; + } + } + + return len; +} + +/** + * @param[in] xectx the structure that stores all the state associated with the instance of Xeve MPEG-5 EVC decoder + * @param[out] avctx codec context + * @return 0 on success, negative value on failure + */ +static int export_stream_params(const XevdContext *xectx, AVCodecContext *avctx) +{ + int ret; + int size; + int color_space; + + avctx->pix_fmt = AV_PIX_FMT_YUV420P10; + + size = 4; + ret = xevd_config(xectx->id, XEVD_CFG_GET_CODED_WIDTH, &avctx->coded_width, &size); + if (XEVD_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to get coded_width\n"); + return AVERROR_EXTERNAL; + } + + ret = xevd_config(xectx->id, XEVD_CFG_GET_CODED_HEIGHT, &avctx->coded_height, &size); + if (XEVD_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to get coded_height\n"); + return AVERROR_EXTERNAL; + } + + ret = xevd_config(xectx->id, XEVD_CFG_GET_WIDTH, &avctx->width, &size); + if (XEVD_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to get width\n"); + return AVERROR_EXTERNAL; + } + + ret = xevd_config(xectx->id, XEVD_CFG_GET_HEIGHT, &avctx->height, &size); + if (XEVD_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to get height\n"); + return AVERROR_EXTERNAL; + } + + ret = xevd_config(xectx->id, XEVD_CFG_GET_COLOR_SPACE, &color_space, &size); + if (XEVD_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to get color_space\n"); + return AVERROR_EXTERNAL; + } + switch(color_space) { + case XEVD_CS_YCBCR400_10LE: + avctx->pix_fmt = AV_PIX_FMT_GRAY10LE; + break; + case XEVD_CS_YCBCR420_10LE: + avctx->pix_fmt = AV_PIX_FMT_YUV420P10LE; + break; + case XEVD_CS_YCBCR422_10LE: + avctx->pix_fmt = AV_PIX_FMT_YUV422P10LE; + break; + case XEVD_CS_YCBCR444_10LE: + avctx->pix_fmt = AV_PIX_FMT_YUV444P10LE; + break; + default: + av_log(avctx, AV_LOG_ERROR, "Unknown color space\n"); + avctx->pix_fmt = AV_PIX_FMT_NONE; + return AVERROR_INVALIDDATA; + } + + // the function returns sps->num_reorder_pics + ret = xevd_config(xectx->id, XEVD_CFG_GET_MAX_CODING_DELAY, &avctx->max_b_frames, &size); + if (XEVD_FAILED(ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to get max_coding_delay\n"); + return AVERROR_EXTERNAL; + } + + avctx->has_b_frames = (avctx->max_b_frames) ? 1 : 0; + + avctx->color_primaries = AVCOL_PRI_UNSPECIFIED; + avctx->color_trc = AVCOL_TRC_UNSPECIFIED; + avctx->colorspace = AVCOL_SPC_UNSPECIFIED; + + return 0; +} + +/** + * @brief Copy image in imgb to frame. + * + * @param avctx codec context + * @param[in] imgb + * @param[out] frame + * @return 0 on success, negative value on failure + */ +static int libxevd_image_copy(struct AVCodecContext *avctx, XEVD_IMGB *imgb, struct AVFrame *frame) +{ + int ret; + if (imgb->cs != XEVD_CS_YCBCR420_10LE) { + av_log(avctx, AV_LOG_ERROR, "Not supported pixel format: %s\n", av_get_pix_fmt_name(avctx->pix_fmt)); + return AVERROR_INVALIDDATA; + } + + if (imgb->w[0] != avctx->width || imgb->h[0] != avctx->height) { // stream resolution changed + if (ff_set_dimensions(avctx, imgb->w[0], imgb->h[0]) < 0) { + av_log(avctx, AV_LOG_ERROR, "Cannot set new dimension\n"); + return AVERROR_INVALIDDATA; + } + } + + if (ret = ff_get_buffer(avctx, frame, 0) < 0) + return ret; + + av_image_copy(frame->data, frame->linesize, (const uint8_t **)imgb->a, + imgb->s, avctx->pix_fmt, + imgb->w[0], imgb->h[0]); + + return 0; +} + +/** + * Initialize decoder + * Create a decoder instance and allocate all the needed resources + * + * @param avctx codec context + * @return 0 on success, negative error code on failure + */ +static av_cold int libxevd_init(AVCodecContext *avctx) +{ + XevdContext *xectx = avctx->priv_data; + XEVD_CDSC *cdsc = &(xectx->cdsc); + + /* read configurations and set values for created descriptor (XEVD_CDSC) */ + get_conf(avctx, cdsc); + + /* create decoder */ + xectx->id = xevd_create(&(xectx->cdsc), NULL); + if (xectx->id == NULL) { + av_log(avctx, AV_LOG_ERROR, "Cannot create XEVD encoder\n"); + return AVERROR_EXTERNAL; + } + + return 0; +} + +/** + * Decode frame with decoupled packet/frame dataflow + * + * @param avctx codec context + * @param[out] frame decoded frame + * + * @return 0 on success, negative error code on failure + */ +static int libxevd_receive_frame(AVCodecContext *avctx, AVFrame *frame) +{ + XevdContext *xectx = NULL; + int xevd_ret; + int ret = 0; + + xectx = avctx->priv_data; + + /* poll for new frame */ + { + XEVD_IMGB *imgb = NULL; + + xevd_ret = xevd_pull(xectx->id, &imgb); + if (XEVD_SUCCEEDED(ret)) { + if (imgb) { + + frame->coded_picture_number = imgb->ts[XEVD_TS_DTS]; + frame->display_picture_number = imgb->ts[XEVD_TS_PTS]; + + ret = libxevd_image_copy(avctx, imgb, frame); + + // xevd_pull uses pool of objects of type XEVD_IMGB. + // The pool size is equal MAX_PB_SIZE (26), so release object when it is no more needed + imgb->release(imgb); + imgb = NULL; + return ret; + } + } else { + if (imgb) { // already has a decoded image + imgb->release(imgb); + imgb = NULL; + } + } + } + + /* feed decoder */ + { + XEVD_STAT stat; + XEVD_BITB bitb; + int nalu_size, bs_read_pos, dec_read_bytes; + + AVPacket pkt = {0}; + + pkt.data = NULL; + pkt.size = 0; + + // obtain input data + ret = ff_decode_get_packet(avctx, &pkt); + if (ret < 0) // no data is currently available or end of stream has been reached + return ret; + + memset(&stat, 0, sizeof(XEVD_STAT)); + memset(&bitb, 0, sizeof(XEVD_BITB)); + + if (pkt.size) { + + bs_read_pos = 0; + dec_read_bytes = 0; + while (pkt.size > (bs_read_pos + XEVD_NAL_UNIT_LENGTH_BYTE)) { + + nalu_size = read_nal_unit_length(pkt.data + bs_read_pos, XEVD_NAL_UNIT_LENGTH_BYTE, avctx); + if (nalu_size == 0) { + av_log(avctx, AV_LOG_ERROR, "Invalid bitstream\n"); + ret = AVERROR_INVALIDDATA; + goto ERR; + } + bs_read_pos += XEVD_NAL_UNIT_LENGTH_BYTE; + + bitb.addr = pkt.data + bs_read_pos; + bitb.ssize = nalu_size; + bitb.ts[XEVD_TS_DTS] = xectx->coded_picture_number; + + /* main decoding block */ + xevd_ret = xevd_decode(xectx->id, &bitb, &stat); + if (XEVD_FAILED(xevd_ret)) { + av_log(avctx, AV_LOG_ERROR, "Failed to decode bitstream\n"); + ret = AVERROR_EXTERNAL; + goto ERR; + } + + bs_read_pos += nalu_size; + dec_read_bytes += nalu_size; + + if (stat.nalu_type == XEVD_NUT_SPS) { // EVC stream parameters changed + if ((ret = export_stream_params(xectx, avctx)) != 0) + goto ERR; + } else if (stat.nalu_type == XEVD_NUT_IDR || stat.nalu_type == XEVD_NUT_NONIDR) { + xectx->coded_picture_number++; + } + + if (stat.read != dec_read_bytes) { + av_log(avctx, AV_LOG_INFO, "Different reading of bitstream (in:%d, read:%d)\n", nalu_size, stat.read); + ret = AVERROR_EXTERNAL; + goto ERR; + } + } + } + +ERR: + av_packet_unref(&pkt); + return ret; + } + + return 0; +} + +/** + * Destroy decoder + * + * @param avctx codec context + * @return 0 on success + */ +static av_cold int libxevd_close(AVCodecContext *avctx) +{ + XevdContext *xectx = avctx->priv_data; + if (xectx->id) { + xevd_delete(xectx->id); + xectx->id = NULL; + } + return 0; +} + +#define OFFSET(x) offsetof(XevdContext, x) +#define VD AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_DECODING_PARAM + +static const AVClass libxevd_class = { + .class_name = "libxevd", + .item_name = av_default_item_name, + .version = LIBAVUTIL_VERSION_INT, +}; + +const FFCodec ff_libxevd_decoder = { + .p.name = "evc", + .p.long_name = NULL_IF_CONFIG_SMALL("EVC / MPEG-5 Essential Video Coding (EVC)"), + .p.type = AVMEDIA_TYPE_VIDEO, + .p.id = AV_CODEC_ID_EVC, + .init = libxevd_init, + FF_CODEC_RECEIVE_FRAME_CB(libxevd_receive_frame), + .close = libxevd_close, + .priv_data_size = sizeof(XevdContext), + .p.priv_class = &libxevd_class, + .p.capabilities = AV_CODEC_CAP_DELAY | AV_CODEC_CAP_OTHER_THREADS | AV_CODEC_CAP_AVOID_PROBING, + .p.profiles = NULL_IF_CONFIG_SMALL(ff_evc_profiles), + .p.wrapper_name = "libxevd", + .caps_internal = FF_CODEC_CAP_INIT_CLEANUP | FF_CODEC_CAP_NOT_INIT_THREADSAFE, +};
- Added EVC decoder wrapper - Changes in project configuration file and libavcodec Makefile - Added documentation for xevd wrapper Signed-off-by: Dawid Kozinski <d.kozinski@samsung.com> --- configure | 4 + doc/decoders.texi | 24 +++ doc/general_contents.texi | 10 +- libavcodec/Makefile | 1 + libavcodec/allcodecs.c | 1 + libavcodec/libxevd.c | 400 ++++++++++++++++++++++++++++++++++++++ 6 files changed, 439 insertions(+), 1 deletion(-) create mode 100644 libavcodec/libxevd.c