From patchwork Mon Mar 25 20:06:02 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Almer X-Patchwork-Id: 47459 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:c889:b0:1a3:b6bb:3029 with SMTP id hb9csp1416854pzb; Mon, 25 Mar 2024 13:07:15 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCXufvKFAwWeKYIrdLxPq3pMU+sjWMOXd8DbVg2n0tDC3GPiYNqyUlFa0Ay5kLkj/FVVs3JW2d2FfcukceWRtIcqqZp2SXvS59iDIQ== X-Google-Smtp-Source: AGHT+IHidhA9Ep/ml9VXsATLTjYckf//debSC0ZxLnxesu9h+9dJ7qzeSlTqInOzAAXWnUYoLr7i X-Received: by 2002:a50:ab5d:0:b0:56b:deff:4402 with SMTP id t29-20020a50ab5d000000b0056bdeff4402mr4608587edc.42.1711397235200; Mon, 25 Mar 2024 13:07:15 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1711397235; cv=none; d=google.com; s=arc-20160816; b=uVXxyYmnvPSHsubWk6lS6UktcY6dg67gkzIq0f+yjfeImmRtilhy9bHsj58NFVXqO2 rQlyoe9Yz7wyKLdd4QI0IjQFLBvrL3y6mKRu0gUwkWBkRkhJfP2HUroZwtZyjiUW8Lzk l7NrMC6tfBaA1/aldX1CH8PvP6TIhNp6KPrKkCsy8jjoUhKqxeX614aKxWQhvRgwLclz a5jXQjMPfQgTR1iOkn1bhE8QCqpTVgjNLEwHtCDoPjzfgbXeXODdvmSJQg50qp7ByL6u hl+Ep/OpY5M7MrlaFzGEQhnZhm9v/3d+wW6JJPQz/0jwQVgPOJ8pXxFe2eLN6vUm1kjq sdkA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:dkim-signature:delivered-to; bh=8wJS1UC1JpNbRmU0WFmDPzw/bKv6la0JdNYYJ07gl+A=; fh=YOA8vD9MJZuwZ71F/05pj6KdCjf6jQRmzLS+CATXUQk=; b=Ub30FRLMW0MFWPLVUnnRHW96+ADNm0n+Dhq2HJt87uTGZ3/m/GplsbIU1W0MTT8v9p vjJrKtbuqgguOGDov2k488CTSF6OZsd53fAjJMcYwpqkh5r/rfCIOw77SoC+XDBPfw5E QvbWK8Nj2fOoO5HfCZ4uUR/5c3RIMYG8E9AbMJM7yIQG6YrUFoEvLmZ3vJYUS1rsVk+N vUi6cmvOBW4yxkmJuBng+r3RiIY6yeEoYhuDBqCg6it31Bp4iDlrB1rHD3zQDuTv0baJ AjrgRVI7Jwx4MnulRH6A4wmlO5QNLpuBikIfie6h3hDdYRMGaO8ahOnw2QqfpuTQb5gR 9hbA==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=IvXWrus5; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id s26-20020a056402015a00b0056b9023d0cbsi2756633edu.522.2024.03.25.13.07.14; Mon, 25 Mar 2024 13:07:15 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=IvXWrus5; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 5BB7C68D592; Mon, 25 Mar 2024 22:06:36 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pf1-f175.google.com (mail-pf1-f175.google.com [209.85.210.175]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 6170768D547 for ; Mon, 25 Mar 2024 22:06:28 +0200 (EET) Received: by mail-pf1-f175.google.com with SMTP id d2e1a72fcca58-6e703e0e5deso3342347b3a.3 for ; Mon, 25 Mar 2024 13:06:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1711397186; x=1712001986; darn=ffmpeg.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=wMydzzvJzBlUC5WSgQQ9ltrwDLDhZJy6R4YjAn8B9ro=; b=IvXWrus55Ah8ZUIM7j/alMCRYTmJTbE7NwSYMJnnRRVEAtJjI9zXorbIP9Et+T6YZa lJHN+wz2CltQf7//3x5ma//0J7DtKYHZL4ZckIPVvTTmtdVY1CtyD7jU5jUIzL/jJTEK dE2eQt6Z/32Fu/cuqagY6Nd9wtYnrDfmpwJ2nTcRiznSBMI/BhXSP3AeWXrxEaJlZQSz f7e3lEuAXANrumQiva6onDq2Vog3tkfzty1x1EfSE/8KCa8d9m0bC3vFc9vn8zIu+817 3WXpx9fBsQPLoTtcckjxQyJwybKHe7CYtSnCnCZSs+QGPd0UZdcupdrSkziO9MiMnu6Z xDZg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1711397186; x=1712001986; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=wMydzzvJzBlUC5WSgQQ9ltrwDLDhZJy6R4YjAn8B9ro=; b=wyt0n08Nfd3Hwsh2WLHinNoGY+tQ/DYpStUEwRtAxHgD4AaxbWU75OKOwCPvUNRzwW JQSyLWzFo/3Gu4MSSJvlyGj/sgtiFc2f2DgLjnD9qqFBn+CsTbLcOcSUBQFUpWUtwTsg P5H3GuYrLRK0TECtiorXnlNLquby+MG3Yjpncc0T8pqGrst6HyMJf4ckokG+yJKiPnhN PoYWBkxknC9VfR7ooLleILo6aONf5CAt9CoHufMwXOqe8wzAAJ6mCggJdiZ0HxSIpDJw TTZGABnxCQh403m8EX8Pp/eWnhd9gXoi8lbyo8MYW4NVI2n4MQBdFHpSBjejTqf7zT8s jLfg== X-Gm-Message-State: AOJu0YxFh3VETTKaeBRXlFG8I6NRkSm9T4WsQR6VJ6uVU9F5THJkDol1 StLS1uPuIQ26fSp079s0D8YRxP0C9ywD5Xl6QK+YBUBZ/4vgfjYi9wgWoU/G X-Received: by 2002:a05:6a00:4615:b0:6ea:7468:1f31 with SMTP id ko21-20020a056a00461500b006ea74681f31mr9028777pfb.30.1711397186177; Mon, 25 Mar 2024 13:06:26 -0700 (PDT) Received: from localhost.localdomain ([190.194.167.233]) by smtp.gmail.com with ESMTPSA id kt1-20020a056a004ba100b006ea98271e43sm4431120pfb.144.2024.03.25.13.06.24 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 25 Mar 2024 13:06:25 -0700 (PDT) From: James Almer To: ffmpeg-devel@ffmpeg.org Date: Mon, 25 Mar 2024 17:06:02 -0300 Message-ID: <20240325200602.63020-6-jamrial@gmail.com> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240325200602.63020-1-jamrial@gmail.com> References: <20240325200602.63020-1-jamrial@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 6/6 v2] avcodec/hevcdec: export global side data in AVCodecContext X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 55uo9yNa9Wlb Signed-off-by: James Almer --- libavcodec/avcodec.h | 2 +- libavcodec/h2645_sei.c | 217 ++++++++++++++++++++++--------------- libavcodec/h2645_sei.h | 2 + libavcodec/hevcdec.c | 4 + libavcodec/pthread_frame.c | 10 ++ 5 files changed, 146 insertions(+), 89 deletions(-) diff --git a/libavcodec/avcodec.h b/libavcodec/avcodec.h index 83dc487251..968009a192 100644 --- a/libavcodec/avcodec.h +++ b/libavcodec/avcodec.h @@ -2071,7 +2071,7 @@ typedef struct AVCodecContext { * - encoding: may be set by user before calling avcodec_open2() for * encoder configuration. Afterwards owned and freed by the * encoder. - * - decoding: unused + * - decoding: may be set by libavcodec in avcodec_open2(). */ AVFrameSideData **decoded_side_data; int nb_decoded_side_data; diff --git a/libavcodec/h2645_sei.c b/libavcodec/h2645_sei.c index a9f4e01de8..7c11d7e449 100644 --- a/libavcodec/h2645_sei.c +++ b/libavcodec/h2645_sei.c @@ -529,6 +529,124 @@ static int is_frame_packing_type_valid(SEIFpaType type, enum AVCodecID codec_id) type >= SEI_FPA_TYPE_SIDE_BY_SIDE; } +static int h2645_sei_to_side_data(AVCodecContext *avctx, H2645SEI *sei, + AVFrameSideData ***sd, int *nb_sd) +{ + int ret; + + for (unsigned i = 0; i < sei->unregistered.nb_buf_ref; i++) { + H2645SEIUnregistered *unreg = &sei->unregistered; + AVBufferRef *buf; + + if (!unreg->buf_ref[i]) + continue; + + buf = av_buffer_ref(unreg->buf_ref[i]); + if (!buf) + return AVERROR(ENOMEM); + + if (!av_frame_side_data_add(sd, nb_sd, AV_FRAME_DATA_SEI_UNREGISTERED, &buf, 0)) { + av_buffer_unref(&buf); + return AVERROR(ENOMEM); + } + } + + if (sei->ambient_viewing_environment.present) { + H2645SEIAmbientViewingEnvironment *env = + &sei->ambient_viewing_environment; + AVBufferRef *buf; + size_t size; + + AVAmbientViewingEnvironment *dst_env = + av_ambient_viewing_environment_alloc(&size); + if (!dst_env) + return AVERROR(ENOMEM); + + buf = av_buffer_create((uint8_t *)dst_env, size, NULL, NULL, 0); + if (!buf) { + av_free(dst_env); + return AVERROR(ENOMEM); + } + + ret = ff_frame_new_side_data_from_buf(avctx, sd, nb_sd, + AV_FRAME_DATA_AMBIENT_VIEWING_ENVIRONMENT, &buf); + + if (ret < 0) + return ret; + + dst_env->ambient_illuminance = av_make_q(env->ambient_illuminance, 10000); + dst_env->ambient_light_x = av_make_q(env->ambient_light_x, 50000); + dst_env->ambient_light_y = av_make_q(env->ambient_light_y, 50000); + } + + if (sei->mastering_display.present) { + // HEVC uses a g,b,r ordering, which we convert to a more natural r,g,b + const int mapping[3] = {2, 0, 1}; + const int chroma_den = 50000; + const int luma_den = 10000; + int i; + AVMasteringDisplayMetadata *metadata; + + ret = ff_decode_mastering_display_new(avctx, sd, nb_sd, &metadata); + if (ret < 0) + return ret; + + if (metadata) { + for (i = 0; i < 3; i++) { + const int j = mapping[i]; + metadata->display_primaries[i][0].num = sei->mastering_display.display_primaries[j][0]; + metadata->display_primaries[i][0].den = chroma_den; + metadata->display_primaries[i][1].num = sei->mastering_display.display_primaries[j][1]; + metadata->display_primaries[i][1].den = chroma_den; + } + metadata->white_point[0].num = sei->mastering_display.white_point[0]; + metadata->white_point[0].den = chroma_den; + metadata->white_point[1].num = sei->mastering_display.white_point[1]; + metadata->white_point[1].den = chroma_den; + + metadata->max_luminance.num = sei->mastering_display.max_luminance; + metadata->max_luminance.den = luma_den; + metadata->min_luminance.num = sei->mastering_display.min_luminance; + metadata->min_luminance.den = luma_den; + metadata->has_luminance = 1; + metadata->has_primaries = 1; + + av_log(avctx, AV_LOG_DEBUG, "Mastering Display Metadata:\n"); + av_log(avctx, AV_LOG_DEBUG, + "r(%5.4f,%5.4f) g(%5.4f,%5.4f) b(%5.4f %5.4f) wp(%5.4f, %5.4f)\n", + av_q2d(metadata->display_primaries[0][0]), + av_q2d(metadata->display_primaries[0][1]), + av_q2d(metadata->display_primaries[1][0]), + av_q2d(metadata->display_primaries[1][1]), + av_q2d(metadata->display_primaries[2][0]), + av_q2d(metadata->display_primaries[2][1]), + av_q2d(metadata->white_point[0]), av_q2d(metadata->white_point[1])); + av_log(avctx, AV_LOG_DEBUG, + "min_luminance=%f, max_luminance=%f\n", + av_q2d(metadata->min_luminance), av_q2d(metadata->max_luminance)); + } + } + + if (sei->content_light.present) { + AVContentLightMetadata *metadata; + + ret = ff_decode_content_light_new(avctx, sd, nb_sd, &metadata); + if (ret < 0) + return ret; + + if (metadata) { + metadata->MaxCLL = sei->content_light.max_content_light_level; + metadata->MaxFALL = sei->content_light.max_pic_average_light_level; + + av_log(avctx, AV_LOG_DEBUG, "Content Light Level Metadata:\n"); + av_log(avctx, AV_LOG_DEBUG, "MaxCLL=%d, MaxFALL=%d\n", + metadata->MaxCLL, metadata->MaxFALL); + } + } + + return 0; +} + int ff_h2645_sei_to_frame(AVFrame *frame, H2645SEI *sei, enum AVCodecID codec_id, AVCodecContext *avctx, const H2645VUI *vui, @@ -625,17 +743,13 @@ int ff_h2645_sei_to_frame(AVFrame *frame, H2645SEI *sei, avctx->properties |= FF_CODEC_PROPERTY_CLOSED_CAPTIONS; } + ret = h2645_sei_to_side_data(avctx, sei, &frame->side_data, &frame->nb_side_data); + if (ret < 0) + return ret; + for (unsigned i = 0; i < sei->unregistered.nb_buf_ref; i++) { H2645SEIUnregistered *unreg = &sei->unregistered; - - if (unreg->buf_ref[i]) { - AVFrameSideData *sd = av_frame_new_side_data_from_buf(frame, - AV_FRAME_DATA_SEI_UNREGISTERED, - unreg->buf_ref[i]); - if (!sd) - av_buffer_unref(&unreg->buf_ref[i]); - unreg->buf_ref[i] = NULL; - } + av_buffer_unref(&unreg->buf_ref[i]); } sei->unregistered.nb_buf_ref = 0; @@ -728,88 +842,15 @@ FF_ENABLE_DEPRECATION_WARNINGS return ret; #endif - if (sei->ambient_viewing_environment.present) { - H2645SEIAmbientViewingEnvironment *env = - &sei->ambient_viewing_environment; - - AVAmbientViewingEnvironment *dst_env = - av_ambient_viewing_environment_create_side_data(frame); - if (!dst_env) - return AVERROR(ENOMEM); - - dst_env->ambient_illuminance = av_make_q(env->ambient_illuminance, 10000); - dst_env->ambient_light_x = av_make_q(env->ambient_light_x, 50000); - dst_env->ambient_light_y = av_make_q(env->ambient_light_y, 50000); - } - - if (sei->mastering_display.present) { - // HEVC uses a g,b,r ordering, which we convert to a more natural r,g,b - const int mapping[3] = {2, 0, 1}; - const int chroma_den = 50000; - const int luma_den = 10000; - int i; - AVMasteringDisplayMetadata *metadata; - - ret = ff_decode_mastering_display_new(avctx, &frame->side_data, &frame->nb_side_data, &metadata); - if (ret < 0) - return ret; - - if (metadata) { - for (i = 0; i < 3; i++) { - const int j = mapping[i]; - metadata->display_primaries[i][0].num = sei->mastering_display.display_primaries[j][0]; - metadata->display_primaries[i][0].den = chroma_den; - metadata->display_primaries[i][1].num = sei->mastering_display.display_primaries[j][1]; - metadata->display_primaries[i][1].den = chroma_den; - } - metadata->white_point[0].num = sei->mastering_display.white_point[0]; - metadata->white_point[0].den = chroma_den; - metadata->white_point[1].num = sei->mastering_display.white_point[1]; - metadata->white_point[1].den = chroma_den; - - metadata->max_luminance.num = sei->mastering_display.max_luminance; - metadata->max_luminance.den = luma_den; - metadata->min_luminance.num = sei->mastering_display.min_luminance; - metadata->min_luminance.den = luma_den; - metadata->has_luminance = 1; - metadata->has_primaries = 1; - - av_log(avctx, AV_LOG_DEBUG, "Mastering Display Metadata:\n"); - av_log(avctx, AV_LOG_DEBUG, - "r(%5.4f,%5.4f) g(%5.4f,%5.4f) b(%5.4f %5.4f) wp(%5.4f, %5.4f)\n", - av_q2d(metadata->display_primaries[0][0]), - av_q2d(metadata->display_primaries[0][1]), - av_q2d(metadata->display_primaries[1][0]), - av_q2d(metadata->display_primaries[1][1]), - av_q2d(metadata->display_primaries[2][0]), - av_q2d(metadata->display_primaries[2][1]), - av_q2d(metadata->white_point[0]), av_q2d(metadata->white_point[1])); - av_log(avctx, AV_LOG_DEBUG, - "min_luminance=%f, max_luminance=%f\n", - av_q2d(metadata->min_luminance), av_q2d(metadata->max_luminance)); - } - } - - if (sei->content_light.present) { - AVContentLightMetadata *metadata; - - ret = ff_decode_content_light_new(avctx, &frame->side_data, &frame->nb_side_data, &metadata); - if (ret < 0) - return ret; - - if (metadata) { - metadata->MaxCLL = sei->content_light.max_content_light_level; - metadata->MaxFALL = sei->content_light.max_pic_average_light_level; - - av_log(avctx, AV_LOG_DEBUG, "Content Light Level Metadata:\n"); - av_log(avctx, AV_LOG_DEBUG, "MaxCLL=%d, MaxFALL=%d\n", - metadata->MaxCLL, metadata->MaxFALL); - } - } - return 0; } +int ff_h2645_sei_to_context(AVCodecContext *avctx, H2645SEI *sei) +{ + return h2645_sei_to_side_data(avctx, sei, &avctx->decoded_side_data, + &avctx->nb_decoded_side_data); +} + void ff_h2645_sei_reset(H2645SEI *s) { av_buffer_unref(&s->a53_caption.buf_ref); diff --git a/libavcodec/h2645_sei.h b/libavcodec/h2645_sei.h index b9a6c7587b..488dbcad7e 100644 --- a/libavcodec/h2645_sei.h +++ b/libavcodec/h2645_sei.h @@ -168,4 +168,6 @@ int ff_h2645_sei_to_frame(AVFrame *frame, H2645SEI *sei, unsigned bit_depth_luma, unsigned bit_depth_chroma, int seed); +int ff_h2645_sei_to_context(AVCodecContext *avctx, H2645SEI *sei); + #endif /* AVCODEC_H2645_SEI_H */ diff --git a/libavcodec/hevcdec.c b/libavcodec/hevcdec.c index d744e6f598..70d22e5976 100644 --- a/libavcodec/hevcdec.c +++ b/libavcodec/hevcdec.c @@ -3658,6 +3658,10 @@ static av_cold int hevc_decode_init(AVCodecContext *avctx) if (ret < 0) { return ret; } + + ret = ff_h2645_sei_to_context(avctx, &s->sei.common); + if (ret < 0) + return ret; } sd = ff_get_coded_side_data(avctx, AV_PKT_DATA_DOVI_CONF); diff --git a/libavcodec/pthread_frame.c b/libavcodec/pthread_frame.c index fd356bd190..bbf8a546dc 100644 --- a/libavcodec/pthread_frame.c +++ b/libavcodec/pthread_frame.c @@ -334,6 +334,14 @@ FF_ENABLE_DEPRECATION_WARNINGS if (for_user) { if (codec->update_thread_context_for_user) err = codec->update_thread_context_for_user(dst, src); + + av_frame_side_data_free(&dst->decoded_side_data, &dst->nb_decoded_side_data); + for (int i = 0; i < src->nb_decoded_side_data; i++) { + int ret = av_frame_side_data_clone(&dst->decoded_side_data, &dst->nb_decoded_side_data, + src->decoded_side_data[i], AV_FRAME_SIDE_DATA_FLAG_UNIQUE); + if (ret < 0) + return ret; + } } else { const PerThreadContext *p_src = src->internal->thread_ctx; PerThreadContext *p_dst = dst->internal->thread_ctx; @@ -767,6 +775,8 @@ static av_cold int init_thread(PerThreadContext *p, int *threads_to_free, if (!copy) return AVERROR(ENOMEM); copy->priv_data = NULL; + copy->decoded_side_data = NULL; + copy->nb_decoded_side_data = 0; /* From now on, this PerThreadContext will be cleaned up by * ff_frame_thread_free in case of errors. */