From patchwork Fri Dec 15 21:06:33 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Leo Izen X-Patchwork-Id: 45170 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:1225:b0:181:818d:5e7f with SMTP id v37csp5943432pzf; Fri, 15 Dec 2023 13:07:08 -0800 (PST) X-Google-Smtp-Source: AGHT+IES21/l7cvI6eZKHcgd8dFzgcvPpV2lQQETUuIbS/KzbfZZYqsz3Ng7WHp4pBoQzkjIpuFP X-Received: by 2002:a05:600c:43c9:b0:40b:5e21:cc1b with SMTP id f9-20020a05600c43c900b0040b5e21cc1bmr6555754wmn.70.1702674428037; Fri, 15 Dec 2023 13:07:08 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1702674428; cv=none; d=google.com; s=arc-20160816; b=oEpF1cjjDX8QfE2iAFIFR1iNNKB4pOB4dizB7OYZpvdI4/lOjFRWZWtYT34AkW6ktv 5Pi6u58/7G7fB5RJ/oMuG30HgUX9xagnkUsOkDjYRvOvspdi3X/LsZanJOqqd5Px8278 5F4kz42BnWon54gcDvqUzFzfa+4V1d9OtVXCRz6t4QIDmi34tTF70rBvd+o0dnTJ8vGC rHGnhrfI2DWFFP+rRFpelhZZ2rSmAKYRNLx5NuyPKd3Qn6687RGZebt+wc96oikSeiVw Q00LH2fDvn/0AvbAd0LwW3OWfygYxJSAS/VV4+w76butD7lb/TdJtUvxAY+j25ugXj0M tvZA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:message-id:date:to:from :dkim-signature:delivered-to; bh=f+IBWseYgYF/xL40FWQNbFt3MKw75tcQtZtrSsVlDu8=; fh=+bdjGe20eEUjtjncwA1dnEEVYNfJL4vyhV+sIRR4l+g=; b=ljqO7XbHN2UXy2oCtaKB5KkjxNVBuuMNoXfqFRoD44FGvR05M30/Z41wiO1nUgKeTW 9M6dajIqB2SzlxJpHvcgW+lTk4xZlppeXK6uLKpnT2TrPY3CrznRlH/41vcHabhY1hqb csUIC7M58iP0qVTUXEHoom64xXu1cjAdRgHAvWMmo2UNpc10y1YfegZquVyH8BpkNOdg LNSQtk6kz8ztLwYl+JfS+jDPlFQYj2lmzLj4ZRnKCiQ1+ejR9Rvgts2zDubm6O03SjSU KeDydOYhD/3JHiPtEGlcdrnnOISgKwc8zt2Z4VO2bA+9eXZkdiX/FBbNjcqtTRRhIJWH gvyg== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=OGVWZFzM; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id hg22-20020a170906f35600b00a1f93956364si5454625ejb.928.2023.12.15.13.06.47; Fri, 15 Dec 2023 13:07:08 -0800 (PST) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=OGVWZFzM; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id BF46368D0E1; Fri, 15 Dec 2023 23:06:44 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-qk1-f182.google.com (mail-qk1-f182.google.com [209.85.222.182]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 7FF3A68CFCB for ; Fri, 15 Dec 2023 23:06:38 +0200 (EET) Received: by mail-qk1-f182.google.com with SMTP id af79cd13be357-77f971bbb05so12929585a.0 for ; Fri, 15 Dec 2023 13:06:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1702674397; x=1703279197; darn=ffmpeg.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=Ld+yTYncP/jUWtvDPSHHlXWyBc1pKoXKeALoJGftp8g=; b=OGVWZFzMMvaJkJ5tEsqpzXxdUO/JtvbpsPs+RP6+8EAVHSEb0DfmAL/GGWqeGzjLQP 4in93kwCv4aUCLMCIDGluVncAkdGXeiqsXCLy3IqfQj9LtqAKjlquHyFti89H8uLoZaP H5dQVq3EwpRbqKOhOtbPwpESlAggoUqJcnvWm14y2YOT68h2jvxi6wcEgE2tredwQNET TfKOq+mEVA0R8NOzm831O7RE2A1rOzsw/T//kBYLB/SpuSLN+DJ/GEEH5Q6fJ1Cjmfy0 //uHDfCe/dBuJXCMiyOAMsNee7MFjb+6EZFRd1XmzPPIkvdXdBkQ2CKPt9Z0w5lwHr4t tzhA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1702674397; x=1703279197; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=Ld+yTYncP/jUWtvDPSHHlXWyBc1pKoXKeALoJGftp8g=; b=umPvL2s2/WaDWY3D8Nfm7inLGzS31F/MN+VGlSzVuevgZLdkJjgqZV4C4OckQWE1jl 2/+jumRSEq4Fsi3sxREj53vxzuk+mYxg9SA+/kRX28pq7Co4l+VJDfL300oK1g2Xzktq mIgcXkgG9BUPM98uuXfdJha8AOqpIxVxw4bOyzvbqPZ6zqj8K8OjIxBYOH2nIgWFyVj2 m/GUxNEDfo0QZbsav71AiepWPInISRj5kaWmA8bdCB5bcgSxYWdQ7tVLSSl2BHEno4EY 6+nDyz7sJzeDa8tJRwNeu+rY/r6MPW0moUMg17MJWyWdi9T+kN7kWqDvO5GOZRNtQipx CCXg== X-Gm-Message-State: AOJu0YyGrALsrOELyrXLMk/ZdcBr35QLaCgPE1OdXumRluteZTWoOKtz r/GIy9V04XZ58EFnp5AnCZe3QqCTV2o= X-Received: by 2002:a05:6214:401a:b0:67e:f9a7:6bf2 with SMTP id kd26-20020a056214401a00b0067ef9a76bf2mr9709382qvb.3.1702674396953; Fri, 15 Dec 2023 13:06:36 -0800 (PST) Received: from gauss.local (c-68-56-149-176.hsd1.mi.comcast.net. [68.56.149.176]) by smtp.gmail.com with ESMTPSA id v16-20020a0cf910000000b0067a53aa6df2sm7156644qvn.46.2023.12.15.13.06.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 15 Dec 2023 13:06:36 -0800 (PST) From: Leo Izen To: ffmpeg-devel@ffmpeg.org Date: Fri, 15 Dec 2023 16:06:33 -0500 Message-ID: <20231215210633.24100-1-leo.izen@gmail.com> X-Mailer: git-send-email 2.43.0 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v2] avcodec/libjxldec: emit proper PTS to decoded AVFrame X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Leo Izen Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: YR21Nk5ie/hC If a sequence of JXL images is encapsulated in a container that has PTS information, we should use the PTS information from the container. At this time there is no container that does this, but if JPEG XL support is ever added to NUT, AVTransport, or some other container, this commit should allow the PTS information those containers provide to work as expected. Signed-off-by: Leo Izen --- libavcodec/libjxldec.c | 57 +++++++++++++++++++++--------------------- 1 file changed, 29 insertions(+), 28 deletions(-) diff --git a/libavcodec/libjxldec.c b/libavcodec/libjxldec.c index 002740d9c1..6e630fb1b0 100644 --- a/libavcodec/libjxldec.c +++ b/libavcodec/libjxldec.c @@ -54,10 +54,10 @@ typedef struct LibJxlDecodeContext { JxlDecoderStatus events; AVBufferRef *iccp; AVPacket *avpkt; - int64_t pts; + int64_t accumulated_pts; int64_t frame_duration; int prev_is_last; - AVRational timebase; + AVRational anim_timebase; AVFrame *frame; } LibJxlDecodeContext; @@ -80,7 +80,6 @@ static int libjxl_init_jxl_decoder(AVCodecContext *avctx) memset(&ctx->basic_info, 0, sizeof(JxlBasicInfo)); memset(&ctx->jxl_pixfmt, 0, sizeof(JxlPixelFormat)); ctx->prev_is_last = 1; - ctx->frame_duration = 1; return 0; } @@ -104,7 +103,6 @@ static av_cold int libjxl_decode_init(AVCodecContext *avctx) } ctx->avpkt = avctx->internal->in_pkt; - ctx->pts = 0; ctx->frame = av_frame_alloc(); if (!ctx->frame) return AVERROR(ENOMEM); @@ -370,12 +368,15 @@ static int libjxl_receive_frame(AVCodecContext *avctx, AVFrame *frame) while (1) { size_t remaining; + JxlFrameHeader header; if (!pkt->size) { av_packet_unref(pkt); ret = ff_decode_get_packet(avctx, pkt); if (ret < 0 && ret != AVERROR_EOF) return ret; + ctx->accumulated_pts = 0; + ctx->frame_duration = 0; if (!pkt->size) { /* jret set by the last iteration of the loop */ if (jret == JXL_DEC_NEED_MORE_INPUT) { @@ -429,12 +430,8 @@ static int libjxl_receive_frame(AVCodecContext *avctx, AVFrame *frame) if ((ret = ff_set_dimensions(avctx, ctx->basic_info.xsize, ctx->basic_info.ysize)) < 0) return ret; if (ctx->basic_info.have_animation) - ctx->timebase = av_make_q(ctx->basic_info.animation.tps_denominator, - ctx->basic_info.animation.tps_numerator); - else if (avctx->pkt_timebase.num) - ctx->timebase = avctx->pkt_timebase; - else - ctx->timebase = AV_TIME_BASE_Q; + ctx->anim_timebase = av_make_q(ctx->basic_info.animation.tps_denominator, + ctx->basic_info.animation.tps_numerator); continue; case JXL_DEC_COLOR_ENCODING: av_log(avctx, AV_LOG_DEBUG, "COLOR_ENCODING event emitted\n"); @@ -462,23 +459,24 @@ static int libjxl_receive_frame(AVCodecContext *avctx, AVFrame *frame) #endif continue; case JXL_DEC_FRAME: + /* Frame here refers to the Frame bundle, not a decoded picture */ av_log(avctx, AV_LOG_DEBUG, "FRAME event emitted\n"); - if (!ctx->basic_info.have_animation || ctx->prev_is_last) { + if (ctx->prev_is_last) { + /* + * The last frame sent was tagged as "is_last" which + * means this is a new image file altogether. + */ ctx->frame->pict_type = AV_PICTURE_TYPE_I; ctx->frame->flags |= AV_FRAME_FLAG_KEY; } - if (ctx->basic_info.have_animation) { - JxlFrameHeader header; - if (JxlDecoderGetFrameHeader(ctx->decoder, &header) != JXL_DEC_SUCCESS) { - av_log(avctx, AV_LOG_ERROR, "Bad libjxl dec frame event\n"); - return AVERROR_EXTERNAL; - } - ctx->prev_is_last = header.is_last; - ctx->frame_duration = header.duration; - } else { - ctx->prev_is_last = 1; - ctx->frame_duration = 1; + if (JxlDecoderGetFrameHeader(ctx->decoder, &header) != JXL_DEC_SUCCESS) { + av_log(avctx, AV_LOG_ERROR, "Bad libjxl dec frame event\n"); + return AVERROR_EXTERNAL; } + ctx->prev_is_last = header.is_last; + /* zero duration for animation means the frame is not presented */ + if (ctx->basic_info.have_animation && header.duration) + ctx->frame_duration = header.duration; continue; case JXL_DEC_FULL_IMAGE: /* full image is one frame, even if animated */ @@ -490,14 +488,17 @@ static int libjxl_receive_frame(AVCodecContext *avctx, AVFrame *frame) /* ownership is transfered, and it is not ref-ed */ ctx->iccp = NULL; } - if (avctx->pkt_timebase.num) { - ctx->frame->pts = av_rescale_q(ctx->pts, ctx->timebase, avctx->pkt_timebase); - ctx->frame->duration = av_rescale_q(ctx->frame_duration, ctx->timebase, avctx->pkt_timebase); + if (ctx->basic_info.have_animation) { + ctx->frame->pts = av_rescale_q(ctx->accumulated_pts, ctx->anim_timebase, avctx->pkt_timebase); + ctx->frame->duration = av_rescale_q(ctx->frame_duration, ctx->anim_timebase, avctx->pkt_timebase); } else { - ctx->frame->pts = ctx->pts; - ctx->frame->duration = ctx->frame_duration; + ctx->frame->pts = 0; + ctx->frame->duration = pkt->duration; } - ctx->pts += ctx->frame_duration; + if (pkt->pts != AV_NOPTS_VALUE) + ctx->frame->pts += pkt->pts; + ctx->accumulated_pts += ctx->frame_duration; + ctx->frame->pkt_dts = pkt->dts; av_frame_move_ref(frame, ctx->frame); return 0; case JXL_DEC_SUCCESS: