From patchwork Fri Mar 3 20:31:45 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Leo Izen X-Patchwork-Id: 40577 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:d046:b0:cd:afd7:272c with SMTP id hv6csp1056716pzb; Fri, 3 Mar 2023 12:32:19 -0800 (PST) X-Google-Smtp-Source: AK7set/CK+LMTM+3roDICWijVxwg9t0KhUdqdNL4FdMIVPuT6kkA61Urs9X9OR/dJSpPrht2EEzy X-Received: by 2002:a17:907:31ca:b0:88d:f759:15b0 with SMTP id xf10-20020a17090731ca00b0088df75915b0mr3658497ejb.45.1677875539276; Fri, 03 Mar 2023 12:32:19 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1677875539; cv=none; d=google.com; s=arc-20160816; b=SsU0LFdtXz69e+kvIS1V3Onr8bhBNP9GsbHDL7Fk+W+6YZQAWsb7vUtQr343nN1stu oucveq9o5zBV92I4s3JgHjAQagTyd8y3OeEyBiAbFIBZzSP/q5mWuoECmYntsXbynp5p wt34ZfCV+CJTTaauV7Om4L2vt6wW6msy61dFHj78VUiy2RADlDJ/kmO9iF3u0xVzmGiT XQ6XO3waUttP4dQqaHPwh+5sd7cUg+5/NEZ0OcE95s/3QRQ6PiJLXn5Wo1GufYxC8Ksw f5ZqBPKNn7Q1RNYPYuFO8XqJU4h0BauRClbR8xnhMY78HHVmXsYVMsgPFDMRNQYJL9jU 2ATg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=fb/iJH0uwqfo2QPjud1ewLeXc1Q6RhUvmDKmSmyu3Yk=; b=rF60Z8dnvlBO1571dMW46XuKIgF2Mr2ZQqBvKAiANPtK2CVm4xfta8h7aqZXl6PJfJ J+lO3O9Txuh749eeegF5qvLeZ2Az5YfXuiIaQJ2VoxATkVJYcGE6fh5+keiC5WhOJsny X5+O9I9vGJ5F/drF7rXTAxlxigdXMHW8935yZRCJe3z7nbfOpwr5VmYkW0d+Us7KnTnC gMOLYL1n5MkPdGCGbZFePTwlLQYXUgIqYhJamqR4EroZAIA3xX8Yy5ZOSs9F31iNZJIg oNqy/qhaMSITJKwJ7C8zaQ0RPVXos+MmrTyML1kUzNHqfMQGd+BmuK+yx+AqhHhxgf5M po5g== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=LrVBYYQi; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id s26-20020a170906bc5a00b008e21b9235ccsi2756442ejv.304.2023.03.03.12.32.18; Fri, 03 Mar 2023 12:32:19 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=LrVBYYQi; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 1F4B568BA19; Fri, 3 Mar 2023 22:32:05 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-io1-f47.google.com (mail-io1-f47.google.com [209.85.166.47]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 1B08B689D3A for ; Fri, 3 Mar 2023 22:31:58 +0200 (EET) Received: by mail-io1-f47.google.com with SMTP id bf15so1471519iob.7 for ; Fri, 03 Mar 2023 12:31:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; t=1677875516; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=5tmuI0mAvRNn/M+GvLZQ9WMXHkZ1lWfTVqm6cJ0c6NM=; b=LrVBYYQifoYnOeTcPDR24iTHG0jyero8ubjs8ySBKwpSvpOLWAWxJs6Hdvg6AXYBld ZUlAkujUTmq6j1Z44uiPeWSptCK/S7bbnSrbabz2Zl40lhsuLpRZCATWXzS8bKtNKlnQ gEb++obGrlyv7/ToQtHPuo7r3RrucT7BRimNZm+Yk37ZvtKvD7w4ZLK0daqHbFOAZVsx /maogw+RhYi4KUmzmIIDLN2VknG+Z5xgEeuDcANPGoZekU9DuFzzcD3lvxqPS/bD4INi F8oc0kO7rRpRhd89ssrQY85kDFrhK/YJa5hgmnlS4Yc/BBr8MgWEW5uwIMT5n2A3Cgz3 PJ4A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1677875516; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=5tmuI0mAvRNn/M+GvLZQ9WMXHkZ1lWfTVqm6cJ0c6NM=; b=I0Ji8a0zFL0K0/TlPdqQQzuYcOmsxMXJHPRO75YttQS4F7mxRgBUmDWv0Gdvm84+LY uthNkEsVauvTvheE1as4po41gP6C8lkXwK5csAnt6Llg5QoN5YtRU+SpvC/uqhLQrIH+ NPXLLqNBAn0v7V7qiOt/glmFnn9yLCjsBaTF25reWoWWvGS8/LlWzxcMAr7Z3SrbRZv9 9VxCw5z/u1u+OiXBh8ZewXSZ+/TVrKnRn/lskqfApJd1FsxXs6oWVyOETNju7qiiM59o x9iiwV7eqXDr1Ep2TfdZR7RmMvYEkyF2C4GSm0HFFbv7gcsN+UlIo6DUqasBJWWl3PYB X66w== X-Gm-Message-State: AO0yUKX+KIpw1tEmJvWh9bdvqhdbT/C7cVj3RyO0lAsmFOu/52OIBPBt 5z7LNHeD5F8BDwJbw7Xijax8zJuQoKDZxg== X-Received: by 2002:a05:6602:2b81:b0:740:7d21:d96f with SMTP id r1-20020a0566022b8100b007407d21d96fmr2193922iov.1.1677875516647; Fri, 03 Mar 2023 12:31:56 -0800 (PST) Received: from localhost.localdomain (d-75-118-216-66.oh.cpe.breezeline.net. [75.118.216.66]) by smtp.gmail.com with ESMTPSA id a14-20020a5d958e000000b00746c45ff173sm971324ioo.5.2023.03.03.12.31.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 03 Mar 2023 12:31:56 -0800 (PST) From: Leo Izen To: ffmpeg-devel@ffmpeg.org Date: Fri, 3 Mar 2023 15:31:45 -0500 Message-Id: <20230303203146.377726-2-leo.izen@gmail.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230303203146.377726-1-leo.izen@gmail.com> References: <20230303203146.377726-1-leo.izen@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 1/2] avcodec/libjxldec: add animated decode support X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Leo Izen Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 4sMPZbeBlEp+ Migrate the libjxl decoder wrapper from the decode_frame method to the receive_frame method, which allows sending more than one frame from a single packet. This allows the libjxl decoder to decode JPEG XL files that are animated, and emit every frame of the animation. Now, clients that feed the libjxl decoder with an animated JPEG XL file will be able to receieve the full animation. Signed-off-by: Leo Izen --- libavcodec/libjxldec.c | 103 ++++++++++++++++++++++++++++++----------- libavcodec/version.h | 2 +- 2 files changed, 78 insertions(+), 27 deletions(-) diff --git a/libavcodec/libjxldec.c b/libavcodec/libjxldec.c index 045a1535f9..394fd8698a 100644 --- a/libavcodec/libjxldec.c +++ b/libavcodec/libjxldec.c @@ -52,13 +52,20 @@ typedef struct LibJxlDecodeContext { #endif JxlDecoderStatus events; AVBufferRef *iccp; + AVPacket *avpkt; + size_t remaining; + int64_t pts; + int64_t frame_duration; + int prev_is_last; + AVRational timebase; } LibJxlDecodeContext; static int libjxl_init_jxl_decoder(AVCodecContext *avctx) { LibJxlDecodeContext *ctx = avctx->priv_data; - ctx->events = JXL_DEC_BASIC_INFO | JXL_DEC_FULL_IMAGE | JXL_DEC_COLOR_ENCODING; + ctx->events = JXL_DEC_BASIC_INFO | JXL_DEC_FULL_IMAGE + | JXL_DEC_COLOR_ENCODING | JXL_DEC_FRAME; if (JxlDecoderSubscribeEvents(ctx->decoder, ctx->events) != JXL_DEC_SUCCESS) { av_log(avctx, AV_LOG_ERROR, "Error subscribing to JXL events\n"); return AVERROR_EXTERNAL; @@ -71,6 +78,8 @@ static int libjxl_init_jxl_decoder(AVCodecContext *avctx) memset(&ctx->basic_info, 0, sizeof(JxlBasicInfo)); memset(&ctx->jxl_pixfmt, 0, sizeof(JxlPixelFormat)); + ctx->prev_is_last = 1; + ctx->frame_duration = 1; return 0; } @@ -93,6 +102,11 @@ static av_cold int libjxl_decode_init(AVCodecContext *avctx) return AVERROR_EXTERNAL; } + ctx->avpkt = av_packet_alloc(); + if (!ctx->avpkt) + return AVERROR(ENOMEM); + ctx->pts = 0; + return libjxl_init_jxl_decoder(avctx); } @@ -328,18 +342,33 @@ static int libjxl_color_encoding_event(AVCodecContext *avctx, AVFrame *frame) return 0; } -static int libjxl_decode_frame(AVCodecContext *avctx, AVFrame *frame, int *got_frame, AVPacket *avpkt) +static int libjxl_receive_frame(AVCodecContext *avctx, AVFrame *frame) { LibJxlDecodeContext *ctx = avctx->priv_data; - const uint8_t *buf = avpkt->data; - size_t remaining = avpkt->size; JxlDecoderStatus jret; int ret; - *got_frame = 0; + AVPacket *pkt = ctx->avpkt; + + if (!pkt->size) { + av_packet_unref(pkt); + ret = ff_decode_get_packet(avctx, pkt); + if (ret < 0 && ret != AVERROR_EOF) + return ret; + ctx->remaining = pkt->size; + if (!pkt->size) { + /* empty packet means eof */ + if (ret >= 0) { + av_packet_unref(pkt); + return AVERROR(EAGAIN); + } else { + return AVERROR_EOF; + } + } + } while (1) { - jret = JxlDecoderSetInput(ctx->decoder, buf, remaining); + jret = JxlDecoderSetInput(ctx->decoder, pkt->data + (pkt->size - ctx->remaining), ctx->remaining); if (jret == JXL_DEC_ERROR) { /* this should never happen here unless there's a bug in libjxl */ @@ -353,19 +382,18 @@ static int libjxl_decode_frame(AVCodecContext *avctx, AVFrame *frame, int *got_f * of bytes remaining to be read, rather than * the number of bytes that it did read */ - remaining = JxlDecoderReleaseInput(ctx->decoder); - buf = avpkt->data + avpkt->size - remaining; + ctx->remaining = JxlDecoderReleaseInput(ctx->decoder); switch(jret) { case JXL_DEC_ERROR: av_log(avctx, AV_LOG_ERROR, "Unknown libjxl decode error\n"); return AVERROR_INVALIDDATA; case JXL_DEC_NEED_MORE_INPUT: - if (remaining == 0) { + av_log(avctx, AV_LOG_DEBUG, "NEED_MORE_INPUT event emitted\n"); + if (ctx->remaining == 0) { av_log(avctx, AV_LOG_ERROR, "Unexpected end of JXL codestream\n"); return AVERROR_INVALIDDATA; } - av_log(avctx, AV_LOG_DEBUG, "NEED_MORE_INPUT event emitted\n"); continue; case JXL_DEC_BASIC_INFO: av_log(avctx, AV_LOG_DEBUG, "BASIC_INFO event emitted\n"); @@ -384,6 +412,13 @@ static int libjxl_decode_frame(AVCodecContext *avctx, AVFrame *frame, int *got_f } if ((ret = ff_set_dimensions(avctx, ctx->basic_info.xsize, ctx->basic_info.ysize)) < 0) return ret; + if (ctx->basic_info.have_animation) { + ctx->timebase = av_make_q( + ctx->basic_info.animation.tps_denominator, + ctx->basic_info.animation.tps_numerator); + } else { + ctx->timebase = avctx->pkt_timebase; + } continue; case JXL_DEC_COLOR_ENCODING: av_log(avctx, AV_LOG_DEBUG, "COLOR_ENCODING event emitted\n"); @@ -407,11 +442,28 @@ static int libjxl_decode_frame(AVCodecContext *avctx, AVFrame *frame, int *got_f } #endif continue; + case JXL_DEC_FRAME: + av_log(avctx, AV_LOG_DEBUG, "FRAME event emitted\n"); + if (!ctx->basic_info.have_animation || ctx->prev_is_last) { + frame->pict_type = AV_PICTURE_TYPE_I; + frame->key_frame = 1; + } + if (ctx->basic_info.have_animation) { + JxlFrameHeader header; + if (JxlDecoderGetFrameHeader(ctx->decoder, &header) != JXL_DEC_SUCCESS) { + av_log(avctx, AV_LOG_ERROR, "Bad libjxl dec frame event\n"); + return AVERROR_EXTERNAL; + } + ctx->prev_is_last = header.is_last; + ctx->frame_duration = header.duration; + } else { + ctx->prev_is_last = 1; + ctx->frame_duration = 1; + } + continue; case JXL_DEC_FULL_IMAGE: /* full image is one frame, even if animated */ av_log(avctx, AV_LOG_DEBUG, "FULL_IMAGE event emitted\n"); - frame->pict_type = AV_PICTURE_TYPE_I; - frame->key_frame = 1; if (ctx->iccp) { AVFrameSideData *sd = av_frame_new_side_data_from_buf(frame, AV_FRAME_DATA_ICC_PROFILE, ctx->iccp); if (!sd) @@ -419,25 +471,23 @@ static int libjxl_decode_frame(AVCodecContext *avctx, AVFrame *frame, int *got_f /* ownership is transfered, and it is not ref-ed */ ctx->iccp = NULL; } - *got_frame = 1; - return avpkt->size - remaining; + frame->pts = av_rescale_q(ctx->pts, ctx->timebase, avctx->pkt_timebase); + ctx->pts += ctx->frame_duration; + return 0; case JXL_DEC_SUCCESS: av_log(avctx, AV_LOG_DEBUG, "SUCCESS event emitted\n"); /* - * The SUCCESS event isn't fired until after JXL_DEC_FULL_IMAGE. If this - * stream only contains one JXL image then JXL_DEC_SUCCESS will never fire. - * If the image2 sequence being decoded contains several JXL files, then - * libjxl will fire this event after the next AVPacket has been passed, - * which means the current packet is actually the next image in the sequence. - * This is why we reset the decoder and populate the packet data now, since - * this is the next packet and it has not been decoded yet. The decoder does - * have to be reset to allow us to use it for the next image, or libjxl - * will become very confused if the header information is not identical. + * this event will be fired when the zero-length EOF + * packet is sent to the decoder by the client, + * but it will also be fired when the next image of + * an image2pipe sequence is loaded up */ JxlDecoderReset(ctx->decoder); libjxl_init_jxl_decoder(avctx); - buf = avpkt->data; - remaining = avpkt->size; + if (!ctx->remaining) { + av_packet_unref(pkt); + return AVERROR_EOF; + } continue; default: av_log(avctx, AV_LOG_ERROR, "Bad libjxl event: %d\n", jret); @@ -457,6 +507,7 @@ static av_cold int libjxl_decode_close(AVCodecContext *avctx) JxlDecoderDestroy(ctx->decoder); ctx->decoder = NULL; av_buffer_unref(&ctx->iccp); + av_packet_free(&ctx->avpkt); return 0; } @@ -468,7 +519,7 @@ const FFCodec ff_libjxl_decoder = { .p.id = AV_CODEC_ID_JPEGXL, .priv_data_size = sizeof(LibJxlDecodeContext), .init = libjxl_decode_init, - FF_CODEC_DECODE_CB(libjxl_decode_frame), + FF_CODEC_RECEIVE_FRAME_CB(libjxl_receive_frame), .close = libjxl_decode_close, .p.capabilities = AV_CODEC_CAP_DR1 | AV_CODEC_CAP_OTHER_THREADS, .caps_internal = FF_CODEC_CAP_NOT_INIT_THREADSAFE | diff --git a/libavcodec/version.h b/libavcodec/version.h index da54f87887..39dbec0208 100644 --- a/libavcodec/version.h +++ b/libavcodec/version.h @@ -30,7 +30,7 @@ #include "version_major.h" #define LIBAVCODEC_VERSION_MINOR 6 -#define LIBAVCODEC_VERSION_MICRO 100 +#define LIBAVCODEC_VERSION_MICRO 101 #define LIBAVCODEC_VERSION_INT AV_VERSION_INT(LIBAVCODEC_VERSION_MAJOR, \ LIBAVCODEC_VERSION_MINOR, \