From patchwork Thu Mar 30 22:42:45 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Leo Izen X-Patchwork-Id: 40937 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:4645:b0:e3:3194:9d20 with SMTP id eb5csp273367pzb; Thu, 30 Mar 2023 15:43:08 -0700 (PDT) X-Google-Smtp-Source: AKy350YuwywTu37vux7cMD+L2/mOJ/4p0rbLJeeeD2lbrPla5rGQtEyEeAsHp1JSExfWUou9UPaw X-Received: by 2002:aa7:da12:0:b0:4ff:7116:189e with SMTP id r18-20020aa7da12000000b004ff7116189emr28682996eds.31.1680216188652; Thu, 30 Mar 2023 15:43:08 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1680216188; cv=none; d=google.com; s=arc-20160816; b=e5D/+cgOOVWuamSHltPutYpC7WPS5RZGN5Lid2ro47oIeDJAH2hFtMUQQ7jdan7jpC kN+i7nkAiVsNRrZS47f58zZ7hNssvY0IqDd/Cl2nqeZ+/w4BbEtAyWuaBZnO/z3WZo0U zCmx4C9pZi5CGZrrNPkvEGUMkb6SDAmJ404oVJWeVh0zNamyINp7fMJMX+ocRd9IPcFI 3F0pbKsUNa3fKxHN1WapLP5yAmD6bNjuuVONQWdYJnv3DS2yToBdyiDTGMZPoDOC7Z/R bWS4dpn+qyKoR0etdJSyyNoyWTOGNbdfqN7mRQIwhmmjNyjsMy/WmaoheAC1JlHFrwNs BJpg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=g1iM81c23Yw0CuUlmibZtrJ/5h+bTTSg5CRL/Regw7A=; b=p8lPR4vA2gemC6od4OeCXhAIII5voLNHuQAv30Pi6hCjPiE8/7tYxxnmuPid4LZp/Y smsjpePDlRCtiQ9b/vbKP2+CDkuUtlWPQXmq051POp1rZ22aAcr5gjfSatUuz8D4+7AI I7DL8kLmlbKjyPEnwY3q5tQZLLTgBCvSoFBRqk0Y0GSiOGjZOUD2qSFR8xnY7+lq12rd dO5G9xwErqBkziUHfh+Wmdr60kTmE4E+IpYmyBCCSCeregehXF/QLQ5mxSRskQGVhciz aCJkWWR/UMQ4TsLgOsmI4DbCAfuU0BNLVomNHxVBVgWM8j6eEtySLGwkemD134gdNGcm j+Ug== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=TPHfSKjV; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id d9-20020aa7c1c9000000b004acb7e10eb1si822951edp.238.2023.03.30.15.43.08; Thu, 30 Mar 2023 15:43:08 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20210112 header.b=TPHfSKjV; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id AD86F68C2EF; Fri, 31 Mar 2023 01:42:57 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-yw1-f178.google.com (mail-yw1-f178.google.com [209.85.128.178]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id C5A5268C145 for ; Fri, 31 Mar 2023 01:42:50 +0300 (EEST) Received: by mail-yw1-f178.google.com with SMTP id 00721157ae682-5463fa0c2bfso52188117b3.1 for ; Thu, 30 Mar 2023 15:42:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; t=1680216169; x=1682808169; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=XaiMUgG4LCW8vxPVrElK21ndveWydqjxi5BZDMPGHuY=; b=TPHfSKjV8CDzlBLg8CKv7Zlgx+PdYaMeD1XwwYZr6qJGVn7mS10EJcA1DXtNfX+fwn a1LB/s344+5DyGAwPf2/y0EOn75B81f+XFWlRA0i+krHyrIG4q03uPHNPQO4/lyxs3Qp 8ev0M15/u936s1BuUZxs3w/ozRB2NZQ5GEHA6DyhhCqTfrhh0reKbIeTF1fuYMfmTGHh hTVyqAAWEX++CcOZWGuyW2HAjmlngGaMNJH4P/XTobfNvj441UDifYT7f7d+eWofkKZr uixL08PFFqYlOy/fJYDOTHLFFFKj/Q/Kod/1t1LK6Xr4s5kH70JgEBkpUYObB02fU5cC 9xhQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1680216169; x=1682808169; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=XaiMUgG4LCW8vxPVrElK21ndveWydqjxi5BZDMPGHuY=; b=YHo1mf0qznXE0XbTaAK/CM4ONptACdtzFGs8jCvxoMe97rsWbT3OSgja7DcaQvYrcB S1O4BUURelJYzFSyjDlTC6kGIe01qHw1QpRtmgwDibLSB2aveOrtEdfoA6RqbNyDhYea SIqFW039PglvZiNk5bXD0vulIhjPnyMglX3GSM71tS5QIHddZ3GpI8PvEwu7ki2pOKkY /Ve4p4GoEFuGmNrH+QC8DT5nG1HG1pPye8eI0z7jKNxVuVH1bar5Oy62gFJDV2reoVbI bzgeKhdxLjU/6Uo4+JyoBe5RzuL0tQh1C90wHOMCLKccsXCL6X+f1Bj0kuqQlWVz7b6d 7yyA== X-Gm-Message-State: AAQBX9fnPMgcFdKkMkaKIn3K/qKb3sJ4KF7w8ZLn88kK36yAxkR9MjH2 AO5Ap3YvpGo+c2V59dIdLOywTlajVX400w== X-Received: by 2002:a81:4cc7:0:b0:544:a86e:db72 with SMTP id z190-20020a814cc7000000b00544a86edb72mr3233850ywa.4.1680216169333; Thu, 30 Mar 2023 15:42:49 -0700 (PDT) Received: from gauss.local (c-98-224-219-15.hsd1.mi.comcast.net. [98.224.219.15]) by smtp.gmail.com with ESMTPSA id 203-20020a8113d4000000b00545a08184a9sm136364ywt.57.2023.03.30.15.42.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 30 Mar 2023 15:42:49 -0700 (PDT) From: Leo Izen To: ffmpeg-devel@ffmpeg.org Date: Thu, 30 Mar 2023 18:42:45 -0400 Message-Id: <20230330224246.47109-2-leo.izen@gmail.com> X-Mailer: git-send-email 2.40.0 In-Reply-To: <20230330224246.47109-1-leo.izen@gmail.com> References: <20230330224246.47109-1-leo.izen@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2 1/2] avcodec/libjxldec: add animated decode support X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Leo Izen Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: BShdOrLsGvJF Migrate the libjxl decoder wrapper from the decode_frame method to the receive_frame method, which allows sending more than one frame from a single packet. This allows the libjxl decoder to decode JPEG XL files that are animated, and emit every frame of the animation. Now, clients that feed the libjxl decoder with an animated JPEG XL file will be able to receieve the full animation. Signed-off-by: Leo Izen --- libavcodec/libjxldec.c | 109 ++++++++++++++++++++++++++++++----------- libavcodec/version.h | 2 +- 2 files changed, 82 insertions(+), 29 deletions(-) diff --git a/libavcodec/libjxldec.c b/libavcodec/libjxldec.c index 045a1535f9..5940d0f407 100644 --- a/libavcodec/libjxldec.c +++ b/libavcodec/libjxldec.c @@ -52,13 +52,19 @@ typedef struct LibJxlDecodeContext { #endif JxlDecoderStatus events; AVBufferRef *iccp; + AVPacket *avpkt; + int64_t pts; + int64_t frame_duration; + int prev_is_last; + AVRational timebase; } LibJxlDecodeContext; static int libjxl_init_jxl_decoder(AVCodecContext *avctx) { LibJxlDecodeContext *ctx = avctx->priv_data; - ctx->events = JXL_DEC_BASIC_INFO | JXL_DEC_FULL_IMAGE | JXL_DEC_COLOR_ENCODING; + ctx->events = JXL_DEC_BASIC_INFO | JXL_DEC_FULL_IMAGE + | JXL_DEC_COLOR_ENCODING | JXL_DEC_FRAME; if (JxlDecoderSubscribeEvents(ctx->decoder, ctx->events) != JXL_DEC_SUCCESS) { av_log(avctx, AV_LOG_ERROR, "Error subscribing to JXL events\n"); return AVERROR_EXTERNAL; @@ -71,6 +77,8 @@ static int libjxl_init_jxl_decoder(AVCodecContext *avctx) memset(&ctx->basic_info, 0, sizeof(JxlBasicInfo)); memset(&ctx->jxl_pixfmt, 0, sizeof(JxlPixelFormat)); + ctx->prev_is_last = 1; + ctx->frame_duration = 1; return 0; } @@ -93,6 +101,11 @@ static av_cold int libjxl_decode_init(AVCodecContext *avctx) return AVERROR_EXTERNAL; } + ctx->avpkt = av_packet_alloc(); + if (!ctx->avpkt) + return AVERROR(ENOMEM); + ctx->pts = 0; + return libjxl_init_jxl_decoder(avctx); } @@ -328,19 +341,33 @@ static int libjxl_color_encoding_event(AVCodecContext *avctx, AVFrame *frame) return 0; } -static int libjxl_decode_frame(AVCodecContext *avctx, AVFrame *frame, int *got_frame, AVPacket *avpkt) +static int libjxl_receive_frame(AVCodecContext *avctx, AVFrame *frame) { LibJxlDecodeContext *ctx = avctx->priv_data; - const uint8_t *buf = avpkt->data; - size_t remaining = avpkt->size; - JxlDecoderStatus jret; + JxlDecoderStatus jret = JXL_DEC_SUCCESS; int ret; - *got_frame = 0; + AVPacket *pkt = ctx->avpkt; while (1) { + size_t remaining; - jret = JxlDecoderSetInput(ctx->decoder, buf, remaining); + if (!pkt->size) { + av_packet_unref(pkt); + ret = ff_decode_get_packet(avctx, pkt); + if (ret < 0 && ret != AVERROR_EOF) + return ret; + if (!pkt->size) { + /* jret set by the last iteration of the loop */ + if (jret == JXL_DEC_NEED_MORE_INPUT) { + av_log(avctx, AV_LOG_ERROR, "Unexpected end of JXL codestream\n"); + return AVERROR_INVALIDDATA; + } else { + return AVERROR_EOF; + } + } + } + jret = JxlDecoderSetInput(ctx->decoder, pkt->data, pkt->size); if (jret == JXL_DEC_ERROR) { /* this should never happen here unless there's a bug in libjxl */ av_log(avctx, AV_LOG_ERROR, "Unknown libjxl decode error\n"); @@ -354,18 +381,19 @@ static int libjxl_decode_frame(AVCodecContext *avctx, AVFrame *frame, int *got_f * the number of bytes that it did read */ remaining = JxlDecoderReleaseInput(ctx->decoder); - buf = avpkt->data + avpkt->size - remaining; + pkt->data += pkt->size - remaining; + pkt->size = remaining; switch(jret) { case JXL_DEC_ERROR: av_log(avctx, AV_LOG_ERROR, "Unknown libjxl decode error\n"); return AVERROR_INVALIDDATA; case JXL_DEC_NEED_MORE_INPUT: - if (remaining == 0) { - av_log(avctx, AV_LOG_ERROR, "Unexpected end of JXL codestream\n"); - return AVERROR_INVALIDDATA; - } av_log(avctx, AV_LOG_DEBUG, "NEED_MORE_INPUT event emitted\n"); + if (!pkt->size) { + av_packet_unref(pkt); + return AVERROR(EAGAIN); + } continue; case JXL_DEC_BASIC_INFO: av_log(avctx, AV_LOG_DEBUG, "BASIC_INFO event emitted\n"); @@ -384,6 +412,13 @@ static int libjxl_decode_frame(AVCodecContext *avctx, AVFrame *frame, int *got_f } if ((ret = ff_set_dimensions(avctx, ctx->basic_info.xsize, ctx->basic_info.ysize)) < 0) return ret; + if (ctx->basic_info.have_animation) + ctx->timebase = av_make_q(ctx->basic_info.animation.tps_denominator, + ctx->basic_info.animation.tps_numerator); + else if (avctx->pkt_timebase.num) + ctx->timebase = avctx->pkt_timebase; + else + ctx->timebase = AV_TIME_BASE_Q; continue; case JXL_DEC_COLOR_ENCODING: av_log(avctx, AV_LOG_DEBUG, "COLOR_ENCODING event emitted\n"); @@ -407,11 +442,28 @@ static int libjxl_decode_frame(AVCodecContext *avctx, AVFrame *frame, int *got_f } #endif continue; + case JXL_DEC_FRAME: + av_log(avctx, AV_LOG_DEBUG, "FRAME event emitted\n"); + if (!ctx->basic_info.have_animation || ctx->prev_is_last) { + frame->pict_type = AV_PICTURE_TYPE_I; + frame->key_frame = 1; + } + if (ctx->basic_info.have_animation) { + JxlFrameHeader header; + if (JxlDecoderGetFrameHeader(ctx->decoder, &header) != JXL_DEC_SUCCESS) { + av_log(avctx, AV_LOG_ERROR, "Bad libjxl dec frame event\n"); + return AVERROR_EXTERNAL; + } + ctx->prev_is_last = header.is_last; + ctx->frame_duration = header.duration; + } else { + ctx->prev_is_last = 1; + ctx->frame_duration = 1; + } + continue; case JXL_DEC_FULL_IMAGE: /* full image is one frame, even if animated */ av_log(avctx, AV_LOG_DEBUG, "FULL_IMAGE event emitted\n"); - frame->pict_type = AV_PICTURE_TYPE_I; - frame->key_frame = 1; if (ctx->iccp) { AVFrameSideData *sd = av_frame_new_side_data_from_buf(frame, AV_FRAME_DATA_ICC_PROFILE, ctx->iccp); if (!sd) @@ -419,25 +471,25 @@ static int libjxl_decode_frame(AVCodecContext *avctx, AVFrame *frame, int *got_f /* ownership is transfered, and it is not ref-ed */ ctx->iccp = NULL; } - *got_frame = 1; - return avpkt->size - remaining; + if (avctx->pkt_timebase.num) { + frame->pts = av_rescale_q(ctx->pts, ctx->timebase, avctx->pkt_timebase); + frame->duration = av_rescale_q(ctx->frame_duration, ctx->timebase, avctx->pkt_timebase); + } else { + frame->pts = ctx->pts; + frame->duration = ctx->frame_duration; + } + ctx->pts += ctx->frame_duration; + return 0; case JXL_DEC_SUCCESS: av_log(avctx, AV_LOG_DEBUG, "SUCCESS event emitted\n"); /* - * The SUCCESS event isn't fired until after JXL_DEC_FULL_IMAGE. If this - * stream only contains one JXL image then JXL_DEC_SUCCESS will never fire. - * If the image2 sequence being decoded contains several JXL files, then - * libjxl will fire this event after the next AVPacket has been passed, - * which means the current packet is actually the next image in the sequence. - * This is why we reset the decoder and populate the packet data now, since - * this is the next packet and it has not been decoded yet. The decoder does - * have to be reset to allow us to use it for the next image, or libjxl - * will become very confused if the header information is not identical. + * this event will be fired when the zero-length EOF + * packet is sent to the decoder by the client, + * but it will also be fired when the next image of + * an image2pipe sequence is loaded up */ JxlDecoderReset(ctx->decoder); libjxl_init_jxl_decoder(avctx); - buf = avpkt->data; - remaining = avpkt->size; continue; default: av_log(avctx, AV_LOG_ERROR, "Bad libjxl event: %d\n", jret); @@ -457,6 +509,7 @@ static av_cold int libjxl_decode_close(AVCodecContext *avctx) JxlDecoderDestroy(ctx->decoder); ctx->decoder = NULL; av_buffer_unref(&ctx->iccp); + av_packet_free(&ctx->avpkt); return 0; } @@ -468,7 +521,7 @@ const FFCodec ff_libjxl_decoder = { .p.id = AV_CODEC_ID_JPEGXL, .priv_data_size = sizeof(LibJxlDecodeContext), .init = libjxl_decode_init, - FF_CODEC_DECODE_CB(libjxl_decode_frame), + FF_CODEC_RECEIVE_FRAME_CB(libjxl_receive_frame), .close = libjxl_decode_close, .p.capabilities = AV_CODEC_CAP_DR1 | AV_CODEC_CAP_OTHER_THREADS, .caps_internal = FF_CODEC_CAP_NOT_INIT_THREADSAFE | diff --git a/libavcodec/version.h b/libavcodec/version.h index 7acb261bb3..ecb096f38b 100644 --- a/libavcodec/version.h +++ b/libavcodec/version.h @@ -30,7 +30,7 @@ #include "version_major.h" #define LIBAVCODEC_VERSION_MINOR 7 -#define LIBAVCODEC_VERSION_MICRO 100 +#define LIBAVCODEC_VERSION_MICRO 101 #define LIBAVCODEC_VERSION_INT AV_VERSION_INT(LIBAVCODEC_VERSION_MAJOR, \ LIBAVCODEC_VERSION_MINOR, \