From patchwork Fri May 26 20:55:04 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Leo Izen X-Patchwork-Id: 41847 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:c51c:b0:10c:5e6f:955f with SMTP id gm28csp9776pzb; Fri, 26 May 2023 13:55:30 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7FTvseA68I3JbtALYqB9Y/xriFp+6bgvDQeVodTphpH5X/CRzvElBcupvJlrmXxM9c2SEQ X-Received: by 2002:a17:907:72c7:b0:96f:9cea:a351 with SMTP id du7-20020a17090772c700b0096f9ceaa351mr3065415ejc.37.1685134530099; Fri, 26 May 2023 13:55:30 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1685134530; cv=none; d=google.com; s=arc-20160816; b=IZqiZ6CtvkB39M6Xy5D9REjQ/Rbne5PV9iWm3zZiqHCC920SJClvF529ge/4JnWQMw WOR5xGUt+dQTdiwpQqX8fxrR4VAS1tJUv3C6AddyJGlWCtHuaSO3y2yzPnCZRY292itJ CrhDYJNRsSdhuWwUFxPPtx6x54XXUbTe6alFcTyjU0pgIErHGvtpxPE62MjbjLeJYq8T 7Q14D0UyZ5PVkEFlQ12KcUOiOS5U/H0pzQ82RiKvV9pCa0b8Q9NaddPvD39iopgLSE7y YmNEakWk38rAocr6C2HUMl4GjYaj0yEiFZx8JgnuvdCythGRXYnSdpVrXu1K03ifmwix l5hg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=4Qp6rF19Pm42v6euH4C999+tic3rbSuYTOPK5D3S8vg=; b=oEBb45Oe2zqvoRFUZltBiZlsk+jkCA1gAas0eysQtWnw+Wz5ChHjTgvueDFccEAoZD cbFi5A9zJQJcMDRe0hzhG7fNM7A7bsx+Nf8IWYlGvxDpN3B7gFxsAd6iJAUUo9XaHO9F Qoig32ehnYRZd97r0RYTf0h7LFt8hRs4t3/lPS10F0HcX1vtM0D8yLHIJo2/VBT16ybu oOg0bBVEC/ks9ZzccKRDCLqPSIT7mYBovFNeduX85nhw01ny7HvyrhKzxz5/OzF9arwt YLHIQ6arrDhYHeo0e4Z0yeDK/PujNfs75/X0n3h36+k43+Nn7FdAxU0gWiN+zDPXMe59 tCCw== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20221208 header.b=fF0Hvt4T; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id l4-20020a170906938400b0096f647c3a73si2899393ejx.1028.2023.05.26.13.55.29; Fri, 26 May 2023 13:55:30 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20221208 header.b=fF0Hvt4T; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 5F0D768C1E5; Fri, 26 May 2023 23:55:19 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-yb1-f169.google.com (mail-yb1-f169.google.com [209.85.219.169]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id B95FE68C1C5 for ; Fri, 26 May 2023 23:55:10 +0300 (EEST) Received: by mail-yb1-f169.google.com with SMTP id 3f1490d57ef6-bad1b745168so165330276.0 for ; Fri, 26 May 2023 13:55:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1685134509; x=1687726509; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=AhgGwqP7B0EgSSI81x1Wcu2SplYnh/0mpUoSA7zHJJc=; b=fF0Hvt4TtgeqtCk1lYkNCN9FfeNgibckEG+Ne86wDRd8wAK+7A5ss4zfI42d+pogkV kmr9WFMh509Ygo2oBKLtp362ANe69B8wqvZVICg/vCjuToV7AcQTw5ltFUNvQ4wbi3Mu KO37BHamS+d3VNjSTqOkv6Qb2D0/nam7hU4FXcqdLZfKaCtCxXuNYLnS5ykNDXfzSsoH PAkjPYu9kf7RwA0X13WBmp6UxKDgMMM+1KomX0qrVYmeeFJoDnRa7xJb7nvOyFRqtAir OZMvjjH3UKgrMrGT8K2aPjbI2hWkTrKIGQOmU/WpW/uVK3C6n8vXzNrQCzMBqexmR6I3 fyAg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1685134509; x=1687726509; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=AhgGwqP7B0EgSSI81x1Wcu2SplYnh/0mpUoSA7zHJJc=; b=Um30N+rDZe9Rucf6Y4m4fImRy4epvdUg5W22PIy/60IqrnuTiZ3llWSejao2Nq8Zdy 3DyMsCCA+F1C2Ex0P/RTz24qWWmBgXHwVkHJ4sc9bMj4p11BvTBuMidK7qZXZzlYcSUA S1z/Rcz7whuRracetQ7E8oKsbGlNUgEiQGsE/PhFnRt41PIg2lpDITpf7tiX6KWwyNlG pk8F9qiinSfvC2cmvKaKtPGj5axRpZpek2G7VKSJlY5/d5lTUulNUL6Vbru42FpdutIW o7yhMnAapsK4RPj1lJbmllZa//8JRPbRTqVJTUB8FS1sy8DBqSISGPCBkSiHtTX5tbb/ kllg== X-Gm-Message-State: AC+VfDx6BZDbvK/CLic9hy0V5SooSK66TIyKRCx+ZdR83c4iRvZobbvd fAorX1L84n8cdsa6nNkdw6ujiYPcObY= X-Received: by 2002:a81:784c:0:b0:565:afae:f3fb with SMTP id t73-20020a81784c000000b00565afaef3fbmr3046108ywc.2.1685134509222; Fri, 26 May 2023 13:55:09 -0700 (PDT) Received: from gauss.local (c-98-224-219-15.hsd1.mi.comcast.net. [98.224.219.15]) by smtp.gmail.com with ESMTPSA id m186-20020a0dfcc3000000b0054bfc94a10dsm1457972ywf.47.2023.05.26.13.55.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 26 May 2023 13:55:08 -0700 (PDT) From: Leo Izen To: ffmpeg-devel@ffmpeg.org Date: Fri, 26 May 2023 16:55:04 -0400 Message-Id: <20230526205505.23569-2-leo.izen@gmail.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230526205505.23569-1-leo.izen@gmail.com> References: <20230526205505.23569-1-leo.izen@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v4 1/2] avcodec/libjxldec: add animated decode support X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Leo Izen Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: o10NIECSVTSF Migrate the libjxl decoder wrapper from the decode_frame method to the receive_frame method, which allows sending more than one frame from a single packet. This allows the libjxl decoder to decode JPEG XL files that are animated, and emit every frame of the animation. Now, clients that feed the libjxl decoder with an animated JPEG XL file will be able to receieve the full animation. Signed-off-by: Leo Izen --- libavcodec/libjxldec.c | 107 ++++++++++++++++++++++++++++++----------- libavcodec/version.h | 2 +- 2 files changed, 80 insertions(+), 29 deletions(-) diff --git a/libavcodec/libjxldec.c b/libavcodec/libjxldec.c index 65966b7b57..50417bcb02 100644 --- a/libavcodec/libjxldec.c +++ b/libavcodec/libjxldec.c @@ -37,6 +37,7 @@ #include "avcodec.h" #include "codec_internal.h" #include "decode.h" +#include "internal.h" #include #include @@ -52,13 +53,19 @@ typedef struct LibJxlDecodeContext { #endif JxlDecoderStatus events; AVBufferRef *iccp; + AVPacket *avpkt; + int64_t pts; + int64_t frame_duration; + int prev_is_last; + AVRational timebase; } LibJxlDecodeContext; static int libjxl_init_jxl_decoder(AVCodecContext *avctx) { LibJxlDecodeContext *ctx = avctx->priv_data; - ctx->events = JXL_DEC_BASIC_INFO | JXL_DEC_FULL_IMAGE | JXL_DEC_COLOR_ENCODING; + ctx->events = JXL_DEC_BASIC_INFO | JXL_DEC_FULL_IMAGE + | JXL_DEC_COLOR_ENCODING | JXL_DEC_FRAME; if (JxlDecoderSubscribeEvents(ctx->decoder, ctx->events) != JXL_DEC_SUCCESS) { av_log(avctx, AV_LOG_ERROR, "Error subscribing to JXL events\n"); return AVERROR_EXTERNAL; @@ -71,6 +78,8 @@ static int libjxl_init_jxl_decoder(AVCodecContext *avctx) memset(&ctx->basic_info, 0, sizeof(JxlBasicInfo)); memset(&ctx->jxl_pixfmt, 0, sizeof(JxlPixelFormat)); + ctx->prev_is_last = 1; + ctx->frame_duration = 1; return 0; } @@ -93,6 +102,9 @@ static av_cold int libjxl_decode_init(AVCodecContext *avctx) return AVERROR_EXTERNAL; } + ctx->avpkt = avctx->internal->in_pkt; + ctx->pts = 0; + return libjxl_init_jxl_decoder(avctx); } @@ -328,19 +340,33 @@ static int libjxl_color_encoding_event(AVCodecContext *avctx, AVFrame *frame) return 0; } -static int libjxl_decode_frame(AVCodecContext *avctx, AVFrame *frame, int *got_frame, AVPacket *avpkt) +static int libjxl_receive_frame(AVCodecContext *avctx, AVFrame *frame) { LibJxlDecodeContext *ctx = avctx->priv_data; - const uint8_t *buf = avpkt->data; - size_t remaining = avpkt->size; - JxlDecoderStatus jret; + JxlDecoderStatus jret = JXL_DEC_SUCCESS; int ret; - *got_frame = 0; + AVPacket *pkt = ctx->avpkt; while (1) { + size_t remaining; - jret = JxlDecoderSetInput(ctx->decoder, buf, remaining); + if (!pkt->size) { + av_packet_unref(pkt); + ret = ff_decode_get_packet(avctx, pkt); + if (ret < 0 && ret != AVERROR_EOF) + return ret; + if (!pkt->size) { + /* jret set by the last iteration of the loop */ + if (jret == JXL_DEC_NEED_MORE_INPUT) { + av_log(avctx, AV_LOG_ERROR, "Unexpected end of JXL codestream\n"); + return AVERROR_INVALIDDATA; + } else { + return AVERROR_EOF; + } + } + } + jret = JxlDecoderSetInput(ctx->decoder, pkt->data, pkt->size); if (jret == JXL_DEC_ERROR) { /* this should never happen here unless there's a bug in libjxl */ av_log(avctx, AV_LOG_ERROR, "Unknown libjxl decode error\n"); @@ -354,18 +380,19 @@ static int libjxl_decode_frame(AVCodecContext *avctx, AVFrame *frame, int *got_f * the number of bytes that it did read */ remaining = JxlDecoderReleaseInput(ctx->decoder); - buf = avpkt->data + avpkt->size - remaining; + pkt->data += pkt->size - remaining; + pkt->size = remaining; switch(jret) { case JXL_DEC_ERROR: av_log(avctx, AV_LOG_ERROR, "Unknown libjxl decode error\n"); return AVERROR_INVALIDDATA; case JXL_DEC_NEED_MORE_INPUT: - if (remaining == 0) { - av_log(avctx, AV_LOG_ERROR, "Unexpected end of JXL codestream\n"); - return AVERROR_INVALIDDATA; - } av_log(avctx, AV_LOG_DEBUG, "NEED_MORE_INPUT event emitted\n"); + if (!pkt->size) { + av_packet_unref(pkt); + return AVERROR(EAGAIN); + } continue; case JXL_DEC_BASIC_INFO: av_log(avctx, AV_LOG_DEBUG, "BASIC_INFO event emitted\n"); @@ -384,6 +411,13 @@ static int libjxl_decode_frame(AVCodecContext *avctx, AVFrame *frame, int *got_f } if ((ret = ff_set_dimensions(avctx, ctx->basic_info.xsize, ctx->basic_info.ysize)) < 0) return ret; + if (ctx->basic_info.have_animation) + ctx->timebase = av_make_q(ctx->basic_info.animation.tps_denominator, + ctx->basic_info.animation.tps_numerator); + else if (avctx->pkt_timebase.num) + ctx->timebase = avctx->pkt_timebase; + else + ctx->timebase = AV_TIME_BASE_Q; continue; case JXL_DEC_COLOR_ENCODING: av_log(avctx, AV_LOG_DEBUG, "COLOR_ENCODING event emitted\n"); @@ -407,11 +441,28 @@ static int libjxl_decode_frame(AVCodecContext *avctx, AVFrame *frame, int *got_f } #endif continue; + case JXL_DEC_FRAME: + av_log(avctx, AV_LOG_DEBUG, "FRAME event emitted\n"); + if (!ctx->basic_info.have_animation || ctx->prev_is_last) { + frame->pict_type = AV_PICTURE_TYPE_I; + frame->flags |= AV_FRAME_FLAG_KEY; + } + if (ctx->basic_info.have_animation) { + JxlFrameHeader header; + if (JxlDecoderGetFrameHeader(ctx->decoder, &header) != JXL_DEC_SUCCESS) { + av_log(avctx, AV_LOG_ERROR, "Bad libjxl dec frame event\n"); + return AVERROR_EXTERNAL; + } + ctx->prev_is_last = header.is_last; + ctx->frame_duration = header.duration; + } else { + ctx->prev_is_last = 1; + ctx->frame_duration = 1; + } + continue; case JXL_DEC_FULL_IMAGE: /* full image is one frame, even if animated */ av_log(avctx, AV_LOG_DEBUG, "FULL_IMAGE event emitted\n"); - frame->pict_type = AV_PICTURE_TYPE_I; - frame->flags |= AV_FRAME_FLAG_KEY; if (ctx->iccp) { AVFrameSideData *sd = av_frame_new_side_data_from_buf(frame, AV_FRAME_DATA_ICC_PROFILE, ctx->iccp); if (!sd) @@ -419,25 +470,25 @@ static int libjxl_decode_frame(AVCodecContext *avctx, AVFrame *frame, int *got_f /* ownership is transfered, and it is not ref-ed */ ctx->iccp = NULL; } - *got_frame = 1; - return avpkt->size - remaining; + if (avctx->pkt_timebase.num) { + frame->pts = av_rescale_q(ctx->pts, ctx->timebase, avctx->pkt_timebase); + frame->duration = av_rescale_q(ctx->frame_duration, ctx->timebase, avctx->pkt_timebase); + } else { + frame->pts = ctx->pts; + frame->duration = ctx->frame_duration; + } + ctx->pts += ctx->frame_duration; + return 0; case JXL_DEC_SUCCESS: av_log(avctx, AV_LOG_DEBUG, "SUCCESS event emitted\n"); /* - * The SUCCESS event isn't fired until after JXL_DEC_FULL_IMAGE. If this - * stream only contains one JXL image then JXL_DEC_SUCCESS will never fire. - * If the image2 sequence being decoded contains several JXL files, then - * libjxl will fire this event after the next AVPacket has been passed, - * which means the current packet is actually the next image in the sequence. - * This is why we reset the decoder and populate the packet data now, since - * this is the next packet and it has not been decoded yet. The decoder does - * have to be reset to allow us to use it for the next image, or libjxl - * will become very confused if the header information is not identical. + * this event will be fired when the zero-length EOF + * packet is sent to the decoder by the client, + * but it will also be fired when the next image of + * an image2pipe sequence is loaded up */ JxlDecoderReset(ctx->decoder); libjxl_init_jxl_decoder(avctx); - buf = avpkt->data; - remaining = avpkt->size; continue; default: av_log(avctx, AV_LOG_ERROR, "Bad libjxl event: %d\n", jret); @@ -468,7 +519,7 @@ const FFCodec ff_libjxl_decoder = { .p.id = AV_CODEC_ID_JPEGXL, .priv_data_size = sizeof(LibJxlDecodeContext), .init = libjxl_decode_init, - FF_CODEC_DECODE_CB(libjxl_decode_frame), + FF_CODEC_RECEIVE_FRAME_CB(libjxl_receive_frame), .close = libjxl_decode_close, .p.capabilities = AV_CODEC_CAP_DR1 | AV_CODEC_CAP_OTHER_THREADS, .caps_internal = FF_CODEC_CAP_NOT_INIT_THREADSAFE | diff --git a/libavcodec/version.h b/libavcodec/version.h index c20072197d..7fd9f256cf 100644 --- a/libavcodec/version.h +++ b/libavcodec/version.h @@ -30,7 +30,7 @@ #include "version_major.h" #define LIBAVCODEC_VERSION_MINOR 14 -#define LIBAVCODEC_VERSION_MICRO 101 +#define LIBAVCODEC_VERSION_MICRO 102 #define LIBAVCODEC_VERSION_INT AV_VERSION_INT(LIBAVCODEC_VERSION_MAJOR, \ LIBAVCODEC_VERSION_MINOR, \