From patchwork Thu Sep 12 00:59:24 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Almer X-Patchwork-Id: 51552 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:612c:14c:b0:48e:c0f8:d0de with SMTP id h12csp1222899vqi; Thu, 12 Sep 2024 15:44:14 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCVYgjz9XZBT7WbQDRQoEMtuW9VcygxBNRkVbjicI+ZfWv6W2k8rEeFX5wC2rOBXu0IE6EotT5J9oeN3z0K/r0dQ@gmail.com X-Google-Smtp-Source: AGHT+IGSoDWBRHtclba/uGXg0I1sKXJUCEE2HR22QBylSPkdIHZSelwdINv4C6TPhTHNCQatEbVU X-Received: by 2002:a2e:a0cf:0:b0:2f6:62f3:9709 with SMTP id 38308e7fff4ca-2f787dd1917mr20925401fa.20.1726181054201; Thu, 12 Sep 2024 15:44:14 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1726181054; cv=none; d=google.com; s=arc-20240605; b=eZ4POsnTsw8ceIOil1td7lzNf9y3ocjEchZrev0j/w4gGAtO0YGc+b78EDlec4QCTU 1kif5Pp7jdegHkjYqabxt4Pla7YC1YgPwgL/c6ivzj4c1YmDc73+UK/w10oC93wQgyiS rwxnhaYizsb/t/IYyOw6cg8fvzsc5pQa1zVkFqoQftVFvRClxzobiOQmakpv+d0s9XNs bUDaEQmuI7BgJpyHCgr2QyMnDAtZBO1QS6/nLmK1WdcPy0EfE8q9nz1QZkm0DpOe3Yq7 8UXdI8OZ/x+aL6Wu+ykewNGYjS3RhViQTHL5U2mh/9ZnG/c1SVKoQyl1vYN5aVvjH+td /xsA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:dkim-signature:delivered-to; bh=u6CgCq4KWLlyY8txCyHHqrn2rW8bJRDJIzNzWnoS+rU=; fh=YOA8vD9MJZuwZ71F/05pj6KdCjf6jQRmzLS+CATXUQk=; b=JaLM/qXAb+YWltdKpaecOKMucXTj0uaWwDVUk+FVRE+RUtZ/KhlTBYXcBVd2dQ7PcP IpNPRWD2fXwPnnKn4/Jv8HSnG9XJhyoNdoQQHPx9IfM7z69NdT+8b1Pc9mO7AGotcxnf jq6bIa/A6rzLqlTY449dXhseKLM1BU/5A6azxWt8zY94PfVgz2sZZlVMTRB3xiIprwoO 9Xvjzha/vkVTG7ZUQkE/K+dmg6DWKQm/mAgw+Plq43aoJEpgSGDTyWlXxXpQ9dsjUVCZ ttqpGjQb/y23JhJJkz7Or6IdY1Q0mmOe4Aq5OLXmBqWbT9cMtBvuGwFbwJnKsSDRR85s 8UHQ==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b="C/h9VGOY"; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com; dara=fail header.i=@gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 4fb4d7f45d1cf-5c3ebd43215si9517009a12.29.2024.09.12.15.44.13; Thu, 12 Sep 2024 15:44:14 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b="C/h9VGOY"; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com; dara=fail header.i=@gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 0F77F68D8F9; Thu, 12 Sep 2024 03:59:54 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-oa1-f50.google.com (mail-oa1-f50.google.com [209.85.160.50]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 91B0E68D8FB for ; Thu, 12 Sep 2024 03:59:51 +0300 (EEST) Received: by mail-oa1-f50.google.com with SMTP id 586e51a60fabf-27b88b8a24aso204481fac.1 for ; Wed, 11 Sep 2024 17:59:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1726102789; x=1726707589; darn=ffmpeg.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=jFyEcHowfY8z0wH+lhl11/TvKsJ66LuJjX90G8HF1GU=; b=C/h9VGOY5qGDZ24cUe/4tUt4HalUD1zhFOM5BDSRBYy7Zr2pxn+KQkV4PHfSjBXKZ6 RgUmrpzzIlk93nxpJP5+HfB9iR6Fzl7PK55rb0Wu69ZCRjjpDG0dGI9reRICPyzZehPC TA95Uh28y78vexFeD76OF8GHQUvV1Y/yR/c0iF8lvI88lzOg2D8LCm5wqT3HviiutPQ2 cjRaapC92AlY1dKem+z4ePZgnuKXnp2NTGaQ2Z1tlmwY8QTHHwHnraiom6cSeZgG2YRe fdSARJeGOkMJRUP34qCov+7kg0SlstaM/Hi4OB9xLV03ZxJgnoz/FSwBUxRWfc+vrZZ/ gGFg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1726102789; x=1726707589; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=jFyEcHowfY8z0wH+lhl11/TvKsJ66LuJjX90G8HF1GU=; b=dYsF5H/6zM/ExPNQwfLczvdyPJ3J09Bm7io/EJK9AzQ2Y1ZnuCyJVhK9TwE27WepYx Rs7wjhRqcQghliVoen6lGGMjn3W4eNsY0ev7kcCKB0Atw2UIviH3BHg77TsEWDpapQIA KHNQC+FOogS3C2d9YHFkvbYaRjBqNEb84oUO/9SIROoZGmepDpjKe+6xQsIWoSl4ORRf IKUY3wAdQCSJGi0zVzi9+A3KIhv9dhGy0mnhKhvC4gyYPIvTAFNOeOFlWMTTzIj+avZE AY7VgVpkGguVYR8+4BGoi1tILhoQsxcVA7bChRsBvgn1ZcWL9GWLZOjCcgmlUGWFRaIJ LeeA== X-Gm-Message-State: AOJu0YwBT8XcXSYjrdUUSwMyAU4u1lUap5eBxY/9xUFKJkeQK7MuqOwO nP6OQ6/8tyoh57Hv1FwhCaZg0LL/NhlO3oYShL6sEszqPVEYzoZYBLD+XQ== X-Received: by 2002:a05:6870:910f:b0:277:eb68:2878 with SMTP id 586e51a60fabf-27c3f6a6e6emr787729fac.44.1726102788664; Wed, 11 Sep 2024 17:59:48 -0700 (PDT) Received: from localhost.localdomain ([181.92.233.116]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-71908fc8210sm3557769b3a.36.2024.09.11.17.59.47 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 11 Sep 2024 17:59:48 -0700 (PDT) From: James Almer To: ffmpeg-devel@ffmpeg.org Date: Wed, 11 Sep 2024 21:59:24 -0300 Message-ID: <20240912005925.10151-10-jamrial@gmail.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240912005925.10151-1-jamrial@gmail.com> References: <20240912005925.10151-1-jamrial@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 10/10 v4] avcodec: add LCEVC decoding support via LCEVCdec X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: PyZxBmb0fr0g Signed-off-by: James Almer --- Somewhat improved the glue, making it all internal to decode.c and no longer touching AVCodecInternal. I also tried abstracting the delayed processing API in FrameDecodeData by moving it into a standalone API, but the result was quite a bit of complexity that may only become worth applying once there's more than just a few hwaccels and LCEVC using it. I can send a PoC of it if there's interest. configure | 5 +- doc/general_contents.texi | 13 ++ libavcodec/Makefile | 1 + libavcodec/avcodec_internal.h | 4 + libavcodec/decode.c | 65 ++++++- libavcodec/lcevcdec.c | 318 ++++++++++++++++++++++++++++++++++ libavcodec/lcevcdec.h | 42 +++++ libavcodec/pthread_frame.c | 3 + 8 files changed, 449 insertions(+), 2 deletions(-) create mode 100644 libavcodec/lcevcdec.c create mode 100644 libavcodec/lcevcdec.h diff --git a/configure b/configure index d3bd46f382..db9638a1ae 100755 --- a/configure +++ b/configure @@ -225,6 +225,7 @@ External library support: --enable-libcdio enable audio CD grabbing with libcdio [no] --enable-libcodec2 enable codec2 en/decoding using libcodec2 [no] --enable-libdav1d enable AV1 decoding via libdav1d [no] + --enable-liblcevc-dec enable LCEVC decoding via liblcevc-dec [no] --enable-libdavs2 enable AVS2 decoding via libdavs2 [no] --enable-libdc1394 enable IIDC-1394 grabbing using libdc1394 and libraw1394 [no] @@ -1914,6 +1915,7 @@ EXTERNAL_LIBRARY_LIST=" libcelt libcodec2 libdav1d + liblcevc_dec libdc1394 libflite libfontconfig @@ -4027,7 +4029,7 @@ cws2fws_extralibs="zlib_extralibs" # libraries, in any order avcodec_deps="avutil" -avcodec_suggest="libm stdatomic" +avcodec_suggest="libm stdatomic liblcevc_dec" avdevice_deps="avformat avcodec avutil" avdevice_suggest="libm stdatomic" avfilter_deps="avutil" @@ -6870,6 +6872,7 @@ enabled libcelt && require libcelt celt/celt.h celt_decode -lcelt0 && enabled libcaca && require_pkg_config libcaca caca caca.h caca_create_canvas enabled libcodec2 && require libcodec2 codec2/codec2.h codec2_create -lcodec2 enabled libdav1d && require_pkg_config libdav1d "dav1d >= 0.5.0" "dav1d/dav1d.h" dav1d_version +enabled liblcevc_dec && require_pkg_config liblcevc_dec "lcevc_dec >= 2.0.0" "LCEVC/lcevc_dec.h" LCEVC_CreateDecoder enabled libdavs2 && require_pkg_config libdavs2 "davs2 >= 1.6.0" davs2.h davs2_decoder_open enabled libdc1394 && require_pkg_config libdc1394 libdc1394-2 dc1394/dc1394.h dc1394_new enabled libdrm && check_pkg_config libdrm libdrm xf86drm.h drmGetVersion diff --git a/doc/general_contents.texi b/doc/general_contents.texi index e7cf4f8239..5309db9ba8 100644 --- a/doc/general_contents.texi +++ b/doc/general_contents.texi @@ -245,6 +245,19 @@ Go to @url{https://github.com/google/liblc3/} and follow the instructions for installing the library. Then pass @code{--enable-liblc3} to configure to enable it. +@section LCEVCdec + +FFmpeg can make use of the liblcevc_dec library for LCEVC enhacement layer +decoding on supported bitstreams. + +Go to @url{https://github.com/v-novaltd/LCEVCdec} and follow the instructions +for installing the library. Then pass @code{--enable-liblcevc-dec} to configure to +enable it. + +@float NOTE +LCEVCdec is under the BSD-3-Clause-Clear License. +@end float + @section OpenH264 FFmpeg can make use of the OpenH264 library for H.264 decoding and encoding. diff --git a/libavcodec/Makefile b/libavcodec/Makefile index 1d27e554c8..18663e332f 100644 --- a/libavcodec/Makefile +++ b/libavcodec/Makefile @@ -46,6 +46,7 @@ OBJS = ac3_parser.o \ get_buffer.o \ imgconvert.o \ jni.o \ + lcevcdec.o \ mathtables.o \ mediacodec.o \ mpeg12framerate.o \ diff --git a/libavcodec/avcodec_internal.h b/libavcodec/avcodec_internal.h index 31745b89b1..f4ec3595c5 100644 --- a/libavcodec/avcodec_internal.h +++ b/libavcodec/avcodec_internal.h @@ -68,6 +68,10 @@ void ff_decode_flush_buffers(struct AVCodecContext *avctx); void ff_encode_flush_buffers(struct AVCodecContext *avctx); struct AVCodecInternal *ff_decode_internal_alloc(void); +void ff_decode_internal_sync(struct AVCodecContext *dst, + const struct AVCodecContext *src); +void ff_decode_internal_uninit(struct AVCodecInternal *avci); + struct AVCodecInternal *ff_encode_internal_alloc(void); void ff_codec_close(struct AVCodecContext *avctx); diff --git a/libavcodec/decode.c b/libavcodec/decode.c index e4e92e34e4..1f9263a567 100644 --- a/libavcodec/decode.c +++ b/libavcodec/decode.c @@ -48,6 +48,7 @@ #include "hwaccel_internal.h" #include "hwconfig.h" #include "internal.h" +#include "lcevcdec.h" #include "packet_internal.h" #include "progressframe.h" #include "refstruct.h" @@ -89,6 +90,11 @@ typedef struct DecodeContext { * (global or attached to packets) side data over bytestream. */ uint64_t side_data_pref_mask; + + FFLCEVCContext *lcevc; + int lcevc_frame; + int width; + int height; } DecodeContext; static DecodeContext *decode_ctx(AVCodecInternal *avci) @@ -1597,6 +1603,40 @@ int ff_attach_decode_data(AVFrame *frame) return 0; } +static void update_frame_props(AVCodecContext *avctx, AVFrame *frame) +{ + AVCodecInternal *avci = avctx->internal; + DecodeContext *dc = decode_ctx(avci); + + dc->lcevc_frame = dc->lcevc && avctx->codec_type == AVMEDIA_TYPE_VIDEO && + av_frame_get_side_data(frame, AV_FRAME_DATA_LCEVC); + + if (dc->lcevc_frame) { + dc->width = frame->width; + dc->height = frame->height; + frame->width = frame->width * 2 / FFMAX(frame->sample_aspect_ratio.den, 1); + frame->height = frame->height * 2 / FFMAX(frame->sample_aspect_ratio.num, 1); + } +} + +static void attach_post_process_data(AVCodecContext *avctx, AVFrame *frame) +{ + AVCodecInternal *avci = avctx->internal; + DecodeContext *dc = decode_ctx(avci); + + if (dc->lcevc_frame) { + FrameDecodeData *fdd = (FrameDecodeData*)frame->private_ref->data; + + fdd->post_process_opaque = ff_refstruct_ref(dc->lcevc); + fdd->post_process_opaque_free = ff_lcevc_unref; + fdd->post_process = ff_lcevc_process; + + frame->width = dc->width; + frame->height = dc->height; + } + dc->lcevc_frame = 0; +} + int ff_get_buffer(AVCodecContext *avctx, AVFrame *frame, int flags) { const FFHWAccel *hwaccel = ffhwaccel(avctx->hwaccel); @@ -1640,8 +1680,10 @@ int ff_get_buffer(AVCodecContext *avctx, AVFrame *frame, int flags) ret = hwaccel->alloc_frame(avctx, frame); goto end; } - } else + } else { avctx->sw_pix_fmt = avctx->pix_fmt; + update_frame_props(avctx, frame); + } ret = avctx->get_buffer2(avctx, frame, flags); if (ret < 0) @@ -1653,6 +1695,8 @@ int ff_get_buffer(AVCodecContext *avctx, AVFrame *frame, int flags) if (ret < 0) goto fail; + attach_post_process_data(avctx, frame); + end: if (avctx->codec_type == AVMEDIA_TYPE_VIDEO && !override_dimensions && !(ffcodec(avctx->codec)->caps_internal & FF_CODEC_CAP_EXPORTS_CROPPING)) { @@ -1953,6 +1997,12 @@ int ff_decode_preinit(AVCodecContext *avctx) if (ret < 0) return ret; + if (!(avctx->export_side_data & AV_CODEC_EXPORT_DATA_ENHANCEMENTS)) { + ret = ff_lcevc_alloc(&dc->lcevc); + if (ret < 0 && (avctx->err_recognition & AV_EF_EXPLODE)) + return ret; + } + #if FF_API_DROPCHANGED if (avctx->flags & AV_CODEC_FLAG_DROPCHANGED) av_log(avctx, AV_LOG_WARNING, "The dropchanged flag is deprecated.\n"); @@ -2187,3 +2237,16 @@ AVCodecInternal *ff_decode_internal_alloc(void) { return av_mallocz(sizeof(DecodeContext)); } + +void ff_decode_internal_sync(AVCodecContext *dst, const AVCodecContext *src) +{ + ff_refstruct_replace(&decode_ctx(dst->internal)->lcevc, + decode_ctx(src->internal)->lcevc); +} + +void ff_decode_internal_uninit(AVCodecInternal *avci) +{ + DecodeContext *dc = decode_ctx(avci); + + ff_refstruct_unref(&dc->lcevc); +} diff --git a/libavcodec/lcevcdec.c b/libavcodec/lcevcdec.c new file mode 100644 index 0000000000..9dcc2c4afc --- /dev/null +++ b/libavcodec/lcevcdec.c @@ -0,0 +1,318 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#include "config_components.h" + +#include "libavutil/avassert.h" +#include "libavutil/frame.h" +#include "libavutil/imgutils.h" +#include "libavutil/log.h" +#include "libavutil/mem.h" +#include "decode.h" +#include "lcevcdec.h" + +#if CONFIG_LIBLCEVC_DEC +static LCEVC_ColorFormat map_format(int format) +{ + switch (format) { + case AV_PIX_FMT_YUV420P: + return LCEVC_I420_8; + case AV_PIX_FMT_YUV420P10: + return LCEVC_I420_10_LE; + case AV_PIX_FMT_NV12: + return LCEVC_NV12_8; + case AV_PIX_FMT_NV21: + return LCEVC_NV21_8; + case AV_PIX_FMT_GRAY8: + return LCEVC_GRAY_8; + } + + return LCEVC_ColorFormat_Unknown; +} + +static int alloc_base_frame(void *logctx, LCEVC_DecoderHandle decoder, + const AVFrame *frame, LCEVC_PictureHandle *picture) +{ + LCEVC_PictureDesc desc; + LCEVC_ColorFormat fmt = map_format(frame->format); + LCEVC_PictureLockHandle lock; + uint8_t *data[4] = { NULL }; + int linesizes[4] = { 0 }; + uint32_t planes; + LCEVC_ReturnCode res; + + res = LCEVC_DefaultPictureDesc(&desc, fmt, frame->width, frame->height); + if (res != LCEVC_Success) + return AVERROR_EXTERNAL; + + desc.cropTop = frame->crop_top; + desc.cropBottom = frame->crop_bottom; + desc.cropLeft = frame->crop_left; + desc.cropRight = frame->crop_right; + desc.sampleAspectRatioNum = frame->sample_aspect_ratio.num; + desc.sampleAspectRatioDen = frame->sample_aspect_ratio.den; + + /* Allocate LCEVC Picture */ + res = LCEVC_AllocPicture(decoder, &desc, picture); + if (res != LCEVC_Success) { + return AVERROR_EXTERNAL; + } + res = LCEVC_LockPicture(decoder, *picture, LCEVC_Access_Write, &lock); + if (res != LCEVC_Success) + return AVERROR_EXTERNAL; + + res = LCEVC_GetPicturePlaneCount(decoder, *picture, &planes); + if (res != LCEVC_Success) + return AVERROR_EXTERNAL; + + for (unsigned i = 0; i < planes; i++) { + LCEVC_PicturePlaneDesc plane; + + res = LCEVC_GetPictureLockPlaneDesc(decoder, lock, i, &plane); + if (res != LCEVC_Success) + return AVERROR_EXTERNAL; + + data[i] = plane.firstSample; + linesizes[i] = plane.rowByteStride; + } + + av_image_copy2(data, linesizes, frame->data, frame->linesize, + frame->format, frame->width, frame->height); + + res = LCEVC_UnlockPicture(decoder, lock); + if (res != LCEVC_Success) + return AVERROR_EXTERNAL; + + return 0; +} + +static int alloc_enhanced_frame(void *logctx, LCEVC_DecoderHandle decoder, + const AVFrame *frame, LCEVC_PictureHandle *picture) +{ + LCEVC_PictureDesc desc ; + LCEVC_ColorFormat fmt = map_format(frame->format); + LCEVC_PicturePlaneDesc planes[4] = { 0 }; + int width = frame->width * 2 / FFMAX(frame->sample_aspect_ratio.den, 1); + int height = frame->height * 2 / FFMAX(frame->sample_aspect_ratio.num, 1); + LCEVC_ReturnCode res; + + res = LCEVC_DefaultPictureDesc(&desc, fmt, width, height); + if (res != LCEVC_Success) + return AVERROR_EXTERNAL; + + /* Set plane description */ + for (int i = 0; i < 4; i++) { + planes[i].firstSample = frame->data[i]; + planes[i].rowByteStride = frame->linesize[i]; + } + + /* Allocate LCEVC Picture */ + res = LCEVC_AllocPictureExternal(decoder, &desc, NULL, planes, picture); + if (res != LCEVC_Success) { + return AVERROR_EXTERNAL; + } + return 0; +} + +static int lcevc_send_frame(void *logctx, FFLCEVCContext *lcevc, const AVFrame *in) +{ + const AVFrameSideData *sd = av_frame_get_side_data(in, AV_FRAME_DATA_LCEVC); + LCEVC_PictureHandle picture; + LCEVC_ReturnCode res; + int ret = 0; + + if (!sd) + return 1; + + res = LCEVC_SendDecoderEnhancementData(lcevc->decoder, in->pts, 0, sd->data, sd->size); + if (res != LCEVC_Success) + return AVERROR_EXTERNAL; + + ret = alloc_base_frame(logctx, lcevc->decoder, in, &picture); + if (ret < 0) + return ret; + + res = LCEVC_SendDecoderBase(lcevc->decoder, in->pts, 0, picture, -1, NULL); + if (res != LCEVC_Success) + return AVERROR_EXTERNAL; + + memset(&picture, 0, sizeof(picture)); + ret = alloc_enhanced_frame(logctx, lcevc->decoder, in, &picture); + if (ret < 0) + return ret; + + res = LCEVC_SendDecoderPicture(lcevc->decoder, picture); + if (res != LCEVC_Success) + return AVERROR_EXTERNAL; + + return 0; +} + +static int generate_output(void *logctx, FFLCEVCContext *lcevc, AVFrame *out) +{ + LCEVC_PictureDesc desc; + LCEVC_DecodeInformation info; + LCEVC_PictureHandle picture; + LCEVC_ReturnCode res; + + res = LCEVC_ReceiveDecoderPicture(lcevc->decoder, &picture, &info); + if (res != LCEVC_Success) + return AVERROR_EXTERNAL; + + res = LCEVC_GetPictureDesc(lcevc->decoder, picture, &desc); + if (res != LCEVC_Success) + return AVERROR_EXTERNAL; + + out->crop_top = desc.cropTop; + out->crop_bottom = desc.cropBottom; + out->crop_left = desc.cropLeft; + out->crop_right = desc.cropRight; + out->sample_aspect_ratio.num = desc.sampleAspectRatioNum; + out->sample_aspect_ratio.den = desc.sampleAspectRatioDen; + out->width = desc.width + out->crop_left + out->crop_right; + out->height = desc.height + out->crop_top + out->crop_bottom; + + res = LCEVC_FreePicture(lcevc->decoder, picture); + if (res != LCEVC_Success) + return AVERROR_EXTERNAL; + + return 0; +} + +static int lcevc_receive_frame(void *logctx, FFLCEVCContext *lcevc, AVFrame *out) +{ + LCEVC_PictureHandle picture; + LCEVC_ReturnCode res; + int ret; + + ret = generate_output(logctx, lcevc, out); + if (ret < 0) + return ret; + + while (1) { + res = LCEVC_ReceiveDecoderBase (lcevc->decoder, &picture); + if (res != LCEVC_Success && res != LCEVC_Again) + return AVERROR_EXTERNAL; + + if (res == LCEVC_Again) + break; + + res = LCEVC_FreePicture(lcevc->decoder, picture); + if (res != LCEVC_Success) + return AVERROR_EXTERNAL; + } + + return 0; +} + +static void event_callback(LCEVC_DecoderHandle dec, LCEVC_Event event, + LCEVC_PictureHandle pic, const LCEVC_DecodeInformation *info, + const uint8_t *data, uint32_t size, void *logctx) +{ + switch (event) { + case LCEVC_Log: + av_log(logctx, AV_LOG_INFO, "%s\n", data); + break; + default: + break; + } +} + +static void lcevc_free(FFRefStructOpaque unused, void *obj) +{ + FFLCEVCContext *lcevc = obj; + if (lcevc->initialized) + LCEVC_DestroyDecoder(lcevc->decoder); + memset(lcevc, 0, sizeof(*lcevc)); +} +#endif + +static int lcevc_init(FFLCEVCContext *lcevc, void *logctx) +{ +#if CONFIG_LIBLCEVC_DEC + LCEVC_AccelContextHandle dummy = { 0 }; +#endif + + if (lcevc->initialized) + return 0; + +#if CONFIG_LIBLCEVC_DEC + if (LCEVC_CreateDecoder(&lcevc->decoder, dummy) != LCEVC_Success) { + av_log(logctx, AV_LOG_ERROR, "Failed to create LCEVC decoder\n"); + return AVERROR_EXTERNAL; + } + + LCEVC_ConfigureDecoderInt(lcevc->decoder, "log_level", 4); + LCEVC_ConfigureDecoderInt(lcevc->decoder, "events", LCEVC_Log); + LCEVC_SetDecoderEventCallback(lcevc->decoder, event_callback, logctx); + + if (LCEVC_InitializeDecoder(lcevc->decoder) != LCEVC_Success) { + av_log(logctx, AV_LOG_ERROR, "Failed to initialize LCEVC decoder\n"); + LCEVC_DestroyDecoder(lcevc->decoder); + return AVERROR_EXTERNAL; + } + +#endif + lcevc->initialized = 1; + + return 0; +} + +int ff_lcevc_process(void *logctx, AVFrame *frame) +{ + FrameDecodeData *fdd = (FrameDecodeData*)frame->private_ref->data; + FFLCEVCContext *lcevc = fdd->post_process_opaque; + int ret; + + if (!lcevc->initialized) { + ret = lcevc_init(lcevc, logctx); + if (ret < 0) + return ret; + } + +#if CONFIG_LIBLCEVC_DEC + ret = lcevc_send_frame(logctx, lcevc, frame); + if (ret) + return ret < 0 ? ret : 0; + + lcevc_receive_frame(logctx, lcevc, frame); + if (ret < 0) + return ret; + + av_frame_remove_side_data(frame, AV_FRAME_DATA_LCEVC); +#endif + + return 0; +} + +int ff_lcevc_alloc(FFLCEVCContext **plcevc) +{ + FFLCEVCContext *lcevc = NULL; +#if CONFIG_LIBLCEVC_DEC + lcevc = ff_refstruct_alloc_ext(sizeof(*lcevc), 0, NULL, lcevc_free); + if (!lcevc) + return AVERROR(ENOMEM); +#endif + *plcevc = lcevc; + return 0; +} + +void ff_lcevc_unref(void *opaque) +{ + ff_refstruct_unref(&opaque); +} diff --git a/libavcodec/lcevcdec.h b/libavcodec/lcevcdec.h new file mode 100644 index 0000000000..7334d3a645 --- /dev/null +++ b/libavcodec/lcevcdec.h @@ -0,0 +1,42 @@ +/* + * This file is part of FFmpeg. + * + * FFmpeg is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * FFmpeg is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with FFmpeg; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#ifndef AVCODEC_LCEVCDEC_H +#define AVCODEC_LCEVCDEC_H + +#include "config_components.h" + +#include +#if CONFIG_LIBLCEVC_DEC +#include +#else +typedef uintptr_t LCEVC_DecoderHandle; +#endif +#include "refstruct.h" + +typedef struct FFLCEVCContext { + LCEVC_DecoderHandle decoder; + int initialized; +} FFLCEVCContext; + +struct AVFrame; + +int ff_lcevc_alloc(FFLCEVCContext **plcevc); +int ff_lcevc_process(void *logctx, struct AVFrame *frame); +void ff_lcevc_unref(void *opaque); +#endif /* AVCODEC_LCEVCDEC_H */ diff --git a/libavcodec/pthread_frame.c b/libavcodec/pthread_frame.c index 019e33b7b2..87d796fd90 100644 --- a/libavcodec/pthread_frame.c +++ b/libavcodec/pthread_frame.c @@ -406,6 +406,7 @@ FF_ENABLE_DEPRECATION_WARNINGS dst->hwaccel_flags = src->hwaccel_flags; ff_refstruct_replace(&dst->internal->pool, src->internal->pool); + ff_decode_internal_sync(dst, src); } if (for_user) { @@ -782,6 +783,7 @@ void ff_frame_thread_free(AVCodecContext *avctx, int thread_count) ff_refstruct_unref(&ctx->internal->pool); av_packet_free(&ctx->internal->in_pkt); av_packet_free(&ctx->internal->last_pkt_props); + ff_decode_internal_uninit(ctx->internal); av_freep(&ctx->internal); av_buffer_unref(&ctx->hw_frames_ctx); av_frame_side_data_free(&ctx->decoded_side_data, @@ -845,6 +847,7 @@ static av_cold int init_thread(PerThreadContext *p, int *threads_to_free, copy->internal = ff_decode_internal_alloc(); if (!copy->internal) return AVERROR(ENOMEM); + ff_decode_internal_sync(copy, avctx); copy->internal->thread_ctx = p; copy->internal->progress_frame_pool = avctx->internal->progress_frame_pool;