From patchwork Wed Dec 13 20:53:35 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Zsolt_Vad=C3=A1sz?= X-Patchwork-Id: 45128 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:1225:b0:181:818d:5e7f with SMTP id v37csp4542969pzf; Wed, 13 Dec 2023 12:54:02 -0800 (PST) X-Google-Smtp-Source: AGHT+IGhCVh96Sy9ei/Y0wO9MH8dGzmeg8e/C/2VOmaQJ5GOLQMBxfglVbN5QZ/iiZDsJeSHBnk+ X-Received: by 2002:a17:907:c311:b0:a19:9b79:8b4e with SMTP id tl17-20020a170907c31100b00a199b798b4emr4023541ejc.95.1702500841863; Wed, 13 Dec 2023 12:54:01 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1702500841; cv=none; d=google.com; s=arc-20160816; b=Oc1kskrF7tFo+ENPh+fZISr148DVwSFsfVMGRuZ0x9DzESpCG6aaBQEWMoCwGF6OJa pey4Of2zYvZmIhQDc7nnUe0dA/sXeH1lXBUQBpDihpnlyFEZVmFiIWMH/xNwxDaKWVct 2aBT01khLlzzAX7PMKzD/2+dENsYcKix4RVJU/3qQWB6IwcoZB3TLTwfbYBbKBAALiNz C47kpzwXhGsSaLBt5QhJ1H5tLkzaaKqtHC0tJgMYV61iJFDTOHtYciDrda6Ior+BCy3T uE0wZywaXFDp88vvEbg0ur1cUGi8PCjnXLkYvSD4KeI0niXl/R2BpxssgFsqwreA+6fv Am7g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to:from :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:feedback-id:message-id:to :date:delivered-to; bh=zHwUOQaJjXjsE3lpjbkS4I4Dy/U6fRDzslt2HlGdfa4=; fh=lvOXre93Kc+LM49rLl0yJcoSym9484cIB4O1jbJgoXk=; b=H/bck1qlbjOzVHE8ZSeIk/kQiEIuqj0nSgn5OJuY+ofmFtlmn+jy8smCNOyB2OX8pJ e+ZCeAKysnKE6qrpsXyKyHT6OSAqH+vt/PBQMER0Z8AVyYj6ExqmNWt58+ePa4uth6m6 UtxdKmksdOQtAR3ujsm0ByfLWuil+wAE02P7T+bQwTtqqlTLgdFxQtFr0R0zjS110G09 4katPYAHW6rglBzzlq1AAXgmyrIqW4mRIU/imgGHpHjsxHmZsi9DbMl1W2q4W/+mLNjG 8AX1VnoFzpxpyVtyfXi6+ryeL/t/PU/qrws5dnCr7PLIhNOFybb0RBFE2bGlA2RaYUbH MKyQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id bj22-20020a170906b05600b00a0fe0ed680esi5682510ejb.178.2023.12.13.12.54.01; Wed, 13 Dec 2023 12:54:01 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 2861D68CC0B; Wed, 13 Dec 2023 22:53:59 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-40140.protonmail.ch (mail-40140.protonmail.ch [185.70.40.140]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id B247768CC0B for ; Wed, 13 Dec 2023 22:53:51 +0200 (EET) Date: Wed, 13 Dec 2023 20:53:35 +0000 To: FFmpeg development discussions and patches Message-ID: Feedback-ID: 28710920:user:proton MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 1/1] avcodec/libjxlenc: Add libjxl_animated encoder X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: =?utf-8?q?Zsolt_Vad=C3=A1sz_via_ffmpeg-devel?= From: =?utf-8?q?Zsolt_Vad=C3=A1sz?= Reply-To: FFmpeg development discussions and patches Cc: =?utf-8?q?Zsolt_Vad=C3=A1sz?= Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: SZ51nr9Y6QGn --- configure | 1 + libavcodec/allcodecs.c | 1 + libavcodec/libjxlenc.c | 214 +++++++++++++++++++++++++++++++++-------- 3 files changed, 177 insertions(+), 39 deletions(-) diff --git a/configure b/configure index 7d2ee66000..a334a9b1e0 100755 --- a/configure +++ b/configure @@ -3418,6 +3418,7 @@ libilbc_decoder_deps="libilbc" libilbc_encoder_deps="libilbc" libjxl_decoder_deps="libjxl libjxl_threads" libjxl_encoder_deps="libjxl libjxl_threads" +libjxl_animated_encoder_deps="libjxl libjxl_threads" libkvazaar_encoder_deps="libkvazaar" libmodplug_demuxer_deps="libmodplug" libmp3lame_encoder_deps="libmp3lame" diff --git a/libavcodec/allcodecs.c b/libavcodec/allcodecs.c index b0f004e15c..e6733b0d4f 100644 --- a/libavcodec/allcodecs.c +++ b/libavcodec/allcodecs.c @@ -784,6 +784,7 @@ extern const FFCodec ff_libilbc_encoder; extern const FFCodec ff_libilbc_decoder; extern const FFCodec ff_libjxl_decoder; extern const FFCodec ff_libjxl_encoder; +extern const FFCodec ff_libjxl_animated_encoder; extern const FFCodec ff_libmp3lame_encoder; extern const FFCodec ff_libopencore_amrnb_encoder; extern const FFCodec ff_libopencore_amrnb_decoder; diff --git a/libavcodec/libjxlenc.c b/libavcodec/libjxlenc.c index d707f3a61b..bf44307a34 100644 --- a/libavcodec/libjxlenc.c +++ b/libavcodec/libjxlenc.c @@ -39,6 +39,7 @@ #include "avcodec.h" #include "encode.h" #include "codec_internal.h" +#include "internal.h" #include #include @@ -49,11 +50,15 @@ typedef struct LibJxlEncodeContext { void *runner; JxlEncoder *encoder; JxlEncoderFrameSettings *options; + JxlPixelFormat jxl_fmt; int effort; float distance; int modular; uint8_t *buffer; size_t buffer_size; + /* Only used by libjxl_animated */ + int animated; + AVFrame *last; } LibJxlEncodeContext; /** @@ -183,6 +188,8 @@ static av_cold int libjxl_encode_init(AVCodecContext *avctx) return AVERROR(ENOMEM); } + ctx->animated = 0; + return 0; } @@ -237,28 +244,19 @@ static int libjxl_populate_primaries(void *avctx, JxlColorEncoding *jxl_color, e return 0; } -/** - * Encode an entire frame. Currently animation, is not supported by - * this encoder, so this will always reinitialize a new still image - * and encode a one-frame image (for image2 and image2pipe). - */ -static int libjxl_encode_frame(AVCodecContext *avctx, AVPacket *pkt, const AVFrame *frame, int *got_packet) +static int libjxl_encode_init_image(AVCodecContext *avctx, const AVFrame *frame) { LibJxlEncodeContext *ctx = avctx->priv_data; AVFrameSideData *sd; const AVPixFmtDescriptor *pix_desc = av_pix_fmt_desc_get(frame->format); JxlBasicInfo info; JxlColorEncoding jxl_color; - JxlPixelFormat jxl_fmt; + JxlPixelFormat *jxl_fmt = &ctx->jxl_fmt; int bits_per_sample; #if JPEGXL_NUMERIC_VERSION >= JPEGXL_COMPUTE_NUMERIC_VERSION(0, 8, 0) JxlBitDepth jxl_bit_depth; #endif - JxlEncoderStatus jret; int ret; - size_t available = ctx->buffer_size; - size_t bytes_written = 0; - uint8_t *next_out = ctx->buffer; ret = libjxl_init_jxl_encoder(avctx); if (ret) { @@ -268,23 +266,30 @@ static int libjxl_encode_frame(AVCodecContext *avctx, AVPacket *pkt, const AVFra /* populate the basic info settings */ JxlEncoderInitBasicInfo(&info); - jxl_fmt.num_channels = pix_desc->nb_components; + jxl_fmt->num_channels = pix_desc->nb_components; info.xsize = frame->width; info.ysize = frame->height; - info.num_extra_channels = (jxl_fmt.num_channels + 1) % 2; - info.num_color_channels = jxl_fmt.num_channels - info.num_extra_channels; - bits_per_sample = av_get_bits_per_pixel(pix_desc) / jxl_fmt.num_channels; + info.num_extra_channels = (jxl_fmt->num_channels + 1) % 2; + info.num_color_channels = jxl_fmt->num_channels - info.num_extra_channels; + bits_per_sample = av_get_bits_per_pixel(pix_desc) / jxl_fmt->num_channels; info.bits_per_sample = avctx->bits_per_raw_sample > 0 && !(pix_desc->flags & AV_PIX_FMT_FLAG_FLOAT) ? avctx->bits_per_raw_sample : bits_per_sample; info.alpha_bits = (info.num_extra_channels > 0) * info.bits_per_sample; if (pix_desc->flags & AV_PIX_FMT_FLAG_FLOAT) { info.exponent_bits_per_sample = info.bits_per_sample > 16 ? 8 : 5; info.alpha_exponent_bits = info.alpha_bits ? info.exponent_bits_per_sample : 0; - jxl_fmt.data_type = info.bits_per_sample > 16 ? JXL_TYPE_FLOAT : JXL_TYPE_FLOAT16; + jxl_fmt->data_type = info.bits_per_sample > 16 ? JXL_TYPE_FLOAT : JXL_TYPE_FLOAT16; } else { info.exponent_bits_per_sample = 0; info.alpha_exponent_bits = 0; - jxl_fmt.data_type = info.bits_per_sample <= 8 ? JXL_TYPE_UINT8 : JXL_TYPE_UINT16; + jxl_fmt->data_type = info.bits_per_sample <= 8 ? JXL_TYPE_UINT8 : JXL_TYPE_UINT16; + } + if(ctx->animated) { + info.have_animation = 1; + info.animation.have_timecodes = 0; + info.animation.num_loops = 0; + info.animation.tps_numerator = frame->time_base.den; + info.animation.tps_denominator = frame->time_base.num; } #if JPEGXL_NUMERIC_VERSION >= JPEGXL_COMPUTE_NUMERIC_VERSION(0, 8, 0) @@ -382,19 +387,19 @@ static int libjxl_encode_frame(AVCodecContext *avctx, AVPacket *pkt, const AVFra av_log(avctx, AV_LOG_WARNING, "Could not increase codestream level\n"); } - jxl_fmt.endianness = JXL_NATIVE_ENDIAN; - jxl_fmt.align = frame->linesize[0]; + jxl_fmt->endianness = JXL_NATIVE_ENDIAN; + jxl_fmt->align = frame->linesize[0]; - if (JxlEncoderAddImageFrame(ctx->options, &jxl_fmt, frame->data[0], jxl_fmt.align * info.ysize) != JXL_ENC_SUCCESS) { - av_log(avctx, AV_LOG_ERROR, "Failed to add Image Frame\n"); - return AVERROR_EXTERNAL; - } - /* - * Run this after the last frame in the image has been passed. - * TODO support animation - */ - JxlEncoderCloseInput(ctx->encoder); + return 0; +} + +static int libjxl_encode_process_output(AVCodecContext *avctx, size_t *bytes_written) +{ + LibJxlEncodeContext *ctx = avctx->priv_data; + JxlEncoderStatus jret; + size_t available = ctx->buffer_size; + uint8_t *next_out = ctx->buffer; while (1) { jret = JxlEncoderProcessOutput(ctx->encoder, &next_out, &available); @@ -402,7 +407,7 @@ static int libjxl_encode_frame(AVCodecContext *avctx, AVPacket *pkt, const AVFra av_log(avctx, AV_LOG_ERROR, "Unspecified libjxl error occurred\n"); return AVERROR_EXTERNAL; } - bytes_written = ctx->buffer_size - available; + *bytes_written = ctx->buffer_size - available; /* all data passed has been encoded */ if (jret == JXL_ENC_SUCCESS) break; @@ -419,14 +424,48 @@ static int libjxl_encode_frame(AVCodecContext *avctx, AVPacket *pkt, const AVFra return AVERROR(ENOMEM); ctx->buffer = temp; ctx->buffer_size = new_size; - next_out = ctx->buffer + bytes_written; - available = new_size - bytes_written; + next_out = ctx->buffer + *bytes_written; + available = new_size - *bytes_written; continue; } av_log(avctx, AV_LOG_ERROR, "Bad libjxl event: %d\n", jret); return AVERROR_EXTERNAL; } + return 0; +} + +/** + * Encode an entire frame. Currently animation, is not supported by + * this encoder, so this will always reinitialize a new still image + * and encode a one-frame image (for image2 and image2pipe). + */ +static int libjxl_encode_frame(AVCodecContext *avctx, AVPacket *pkt, const AVFrame *frame, int *got_packet) +{ + LibJxlEncodeContext *ctx = avctx->priv_data; + JxlPixelFormat *jxl_fmt = &ctx->jxl_fmt; + int ret; + size_t bytes_written = 0; + + if((ret = libjxl_encode_init_image(avctx, frame)) < 0) + return ret; + + if (JxlEncoderAddImageFrame(ctx->options, + jxl_fmt, + frame->data[0], + jxl_fmt->align * frame->height) != JXL_ENC_SUCCESS) { + av_log(avctx, AV_LOG_ERROR, "Failed to add Image Frame\n"); + return AVERROR_EXTERNAL; + } + + /* + * Run this after the last frame in the image has been passed. + */ + JxlEncoderCloseInput(ctx->encoder); + + if((ret = libjxl_encode_process_output(avctx, &bytes_written)) < 0) + return ret; + ret = ff_get_encode_buffer(avctx, pkt, bytes_written, 0); if (ret < 0) return ret; @@ -437,6 +476,70 @@ static int libjxl_encode_frame(AVCodecContext *avctx, AVPacket *pkt, const AVFra return 0; } +static int libjxl_animated_encode_frame(AVCodecContext *avctx, AVPacket *pkt, const AVFrame *frame, int *got_packet) +{ + LibJxlEncodeContext *ctx = avctx->priv_data; + int ret = 0; + + if(avctx->frame_num == 0) + if((ret = libjxl_encode_init_image(avctx, frame)) < 0) + return ret; + + if(!ctx->last && !avctx->internal->draining) { + ctx->last = av_frame_clone(frame); + *got_packet = 0; + return AVERROR(EAGAIN); + } else if(!ctx->last && avctx->internal->draining && !frame) { + *got_packet = 0; + } else { + JxlPixelFormat *jxl_fmt = &ctx->jxl_fmt; + JxlFrameHeader frame_header; + size_t bytes_written = 0; + + JxlEncoderInitFrameHeader(&frame_header); + frame_header.duration = ctx->last->duration; + pkt->duration = ctx->last->duration; + pkt->pts = AV_NOPTS_VALUE; + pkt->dts = AV_NOPTS_VALUE; + + if(JxlEncoderSetFrameHeader(ctx->options, &frame_header) != JXL_ENC_SUCCESS) { + av_log(avctx, AV_LOG_ERROR, "Failed to set frame header\n"); + return AVERROR_EXTERNAL; + } + if (JxlEncoderAddImageFrame(ctx->options, + jxl_fmt, + ctx->last->data[0], + jxl_fmt->align * ctx->last->height) != JXL_ENC_SUCCESS) { + av_log(avctx, AV_LOG_ERROR, "Failed to add Image Frame\n"); + return AVERROR_EXTERNAL; + } + + /* + * Run this after the last frame in the image has been passed. + */ + if(avctx->internal->draining) + JxlEncoderCloseInput(ctx->encoder); + + if((ret = libjxl_encode_process_output(avctx, &bytes_written)) < 0) + return ret; + + ret = ff_get_encode_buffer(avctx, pkt, bytes_written, 0); + if (ret < 0) + return ret; + + memcpy(pkt->data, ctx->buffer, bytes_written); + *got_packet = 1; + + if(!avctx->internal->draining) { + av_frame_replace(ctx->last, frame); + } else { + av_frame_free(&ctx->last); + } + } + + return 0; +} + static av_cold int libjxl_encode_close(AVCodecContext *avctx) { LibJxlEncodeContext *ctx = avctx->priv_data; @@ -458,6 +561,17 @@ static av_cold int libjxl_encode_close(AVCodecContext *avctx) return 0; } +static av_cold int libjxl_animated_encode_init(AVCodecContext *avctx) +{ + int ret; + LibJxlEncodeContext *ctx = avctx->priv_data; + if((ret = libjxl_encode_init(avctx)) < 0) + return ret; + ctx->animated = 1; + ret = libjxl_init_jxl_encoder(avctx); + return ret; +} + #define OFFSET(x) offsetof(LibJxlEncodeContext, x) #define VE AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_ENCODING_PARAM @@ -476,6 +590,15 @@ static const AVClass libjxl_encode_class = { .version = LIBAVUTIL_VERSION_INT, }; +static const enum AVPixelFormat libjxl_pix_fmts[] = { + AV_PIX_FMT_RGB24, AV_PIX_FMT_RGBA, + AV_PIX_FMT_RGB48, AV_PIX_FMT_RGBA64, + AV_PIX_FMT_GRAY8, AV_PIX_FMT_YA8, + AV_PIX_FMT_GRAY16, AV_PIX_FMT_YA16, + AV_PIX_FMT_GRAYF32, + AV_PIX_FMT_NONE +}; + const FFCodec ff_libjxl_encoder = { .p.name = "libjxl", CODEC_LONG_NAME("libjxl JPEG XL"), @@ -490,14 +613,27 @@ const FFCodec ff_libjxl_encoder = { .caps_internal = FF_CODEC_CAP_NOT_INIT_THREADSAFE | FF_CODEC_CAP_AUTO_THREADS | FF_CODEC_CAP_INIT_CLEANUP | FF_CODEC_CAP_ICC_PROFILES, - .p.pix_fmts = (const enum AVPixelFormat[]) { - AV_PIX_FMT_RGB24, AV_PIX_FMT_RGBA, - AV_PIX_FMT_RGB48, AV_PIX_FMT_RGBA64, - AV_PIX_FMT_GRAY8, AV_PIX_FMT_YA8, - AV_PIX_FMT_GRAY16, AV_PIX_FMT_YA16, - AV_PIX_FMT_GRAYF32, - AV_PIX_FMT_NONE - }, + .p.pix_fmts = libjxl_pix_fmts, + .p.priv_class = &libjxl_encode_class, + .p.wrapper_name = "libjxl", +}; + +const FFCodec ff_libjxl_animated_encoder = { + .p.name = "libjxl_animated", + CODEC_LONG_NAME("libjxl Animated JPEG XL"), + .p.type = AVMEDIA_TYPE_VIDEO, + .p.id = AV_CODEC_ID_JPEGXL, + .priv_data_size = sizeof(LibJxlEncodeContext), + .init = libjxl_animated_encode_init, + FF_CODEC_ENCODE_CB(libjxl_animated_encode_frame), + .close = libjxl_encode_close, + .p.capabilities = AV_CODEC_CAP_OTHER_THREADS | + AV_CODEC_CAP_ENCODER_REORDERED_OPAQUE | + AV_CODEC_CAP_DELAY, + .caps_internal = FF_CODEC_CAP_NOT_INIT_THREADSAFE | + FF_CODEC_CAP_AUTO_THREADS | FF_CODEC_CAP_INIT_CLEANUP | + FF_CODEC_CAP_ICC_PROFILES | FF_CODEC_CAP_EOF_FLUSH, + .p.pix_fmts = libjxl_pix_fmts, .p.priv_class = &libjxl_encode_class, .p.wrapper_name = "libjxl", };