From patchwork Sun Jul 21 22:53:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Almer X-Patchwork-Id: 50673 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a59:a742:0:b0:482:c625:d099 with SMTP id f2csp1711780vqm; Sun, 21 Jul 2024 15:54:23 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCWnYIsOPBK5CDvnW5voQKIvelK0zUxanwmwc2tvysg2DSxcwlT/kgJAJXFMYG+JV/T2rp0lHHbfvsxc1WDy/ZvJp5DlcMQPHVJYEw== X-Google-Smtp-Source: AGHT+IEBIRqA3xbiQkaYvbCuepWj51q+Yl33qGfsbJNKkMiTKIYO3NKJbL7fOBs2A52sYW0oYmlx X-Received: by 2002:a05:6512:2209:b0:52c:dd94:bda9 with SMTP id 2adb3069b0e04-52efb7e8c11mr4175535e87.56.1721602462952; Sun, 21 Jul 2024 15:54:22 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1721602462; cv=none; d=google.com; s=arc-20160816; b=z10fCHTW6v0sks+OuhK6lghqo6JdTpIFpJqGE2oWEfSofWlUgYkzVcdDlfxZ8xVYoF +jYuLXKJXbg5eaWdzzPesAMk0gkkrkU4M7VRjA0KKXLmnQsHg9PTq64B+6xHoqRxSsR8 H+eYL+l3GuyUhyg9Zxkk31/2bAOVICHwyRG+DDZf46ZyyNufNJGFA0jlmxOyPpEGkMvU 1IOOJl05/RGZWKKZ9fiUwlHS1TA2Cs3DoKvjOLgPmkC85Ig1/vPXy/5opK/sBhYb8F95 SIZi4i/xF/dOLibbH3gyqK88qgFo0wk3KWkODh8WCjO+0OJmFR5pqy/Eo5aBExE4o8// kHtQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:dkim-signature:delivered-to; bh=p16gNUVCZrC0oHA0MGJnSLcyfuOZko2FcccvFbf4lCw=; fh=YOA8vD9MJZuwZ71F/05pj6KdCjf6jQRmzLS+CATXUQk=; b=s5c2oVwi8N2bQAxllibX5RrVhb9JDB8T+H1e9eStXCcMNWsf7alH7aimyzu0fgZ1Q9 waC1PjygqUl9vr6TwZb/+EncIh7QWtWAkzqirxNCZX3rBxSUJbKtBz2HFgd6SflTJIDz 8qOMD4Pj3+jSBtWk53Nw86OTeaHhlrt+Xqk4IIWYHk3oszqSxPiFpAM22onLldk0Szpc iVQOZa4MdLkrsVht9haCfh2hj6tW1iErEAuLE4sAyi9poO5cbNSYTPPsssiakIyMClem spL+8o1K3U2wC4SJV2ftQoQSIE9S94OeSc0Mr5OzhaCGt4vEIlCpGgdpdkf4PAVJp++G ZT0Q==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=J0HEB5iI; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com; dara=fail header.i=@gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id a640c23a62f3a-a7a3c78827fsi321269466b.131.2024.07.21.15.54.22; Sun, 21 Jul 2024 15:54:22 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20230601 header.b=J0HEB5iI; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com; dara=fail header.i=@gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 5450168D774; Mon, 22 Jul 2024 01:54:01 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-yw1-f170.google.com (mail-yw1-f170.google.com [209.85.128.170]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 0A13968D73B for ; Mon, 22 Jul 2024 01:53:53 +0300 (EEST) Received: by mail-yw1-f170.google.com with SMTP id 00721157ae682-65f9e25fffaso39564167b3.3 for ; Sun, 21 Jul 2024 15:53:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1721602431; x=1722207231; darn=ffmpeg.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=74KXyri8yN9FpuSONNo3lyVL8Bbyj1aTCZBJc8foCqE=; b=J0HEB5iI1+OOdDhF5jwDt4Lb+koTZs6/Mmm2r2RaJZZfmwI3fPhTEN7ed5c/wUWJuz G+GGiT3QPkr1jFoIdOi3lVY+DgZwynUC5/Cxy3u5e8GWlFst16sBMQPm2z66DRudIDpC AW1mhrecDdZkqkGLkyyNY4xKNFm465IWw5pcOMFP51j7HqLRUY5x/huq9sQm2y8quWsO eojtZdtntOFKWI2Bfzrn+Ooah7VaO7fSMYEz/o7r+oXFo3a6vu/dk8VEf7mJ0F4HtKQ0 tyRSaaHMk+igwT4ox7UyeZf4a0yjKAZZfkzEoS8f/3xqqeDIHY3YLGtCLR5DY5HUJfyo tvzA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1721602431; x=1722207231; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=74KXyri8yN9FpuSONNo3lyVL8Bbyj1aTCZBJc8foCqE=; b=FrnZh25uHkfPVUVGQ37rThMIoG8RonQT4z10S8cXqCZtkmhuyvRo+55G3QMoHu+9KR ltU/B+rxE6XyEz3zhA3Nat3tWVV7nW0TMb2sw1XsjDlUSOm5X4W0Byiv4P9saVCG2/QQ MPWmyQ6qzBCBc3jP52Hi8D0V7K5U6KYX30W4ypU5OVkDyyWg/701mkmSsFATBsqjkcMM v3p0VnafPDl0dwjL1BELExtJ3LLaVC97nx7oALoCPGO6lDXRnSYbHbPcgG5jmI/CGDSl uXaq+8MFstIe5d/vC14NX5bGqF3d+FJzP0c69TFTue95bglEQZ3dq7teTCx27yyDoFmD D0TQ== X-Gm-Message-State: AOJu0YxOQ3gMJIK+F9/iHaLA8V5leCNHrf0xPJTar3v+CAO7le3womOg kkl+tA3P3pY+hq+U7aRUwkJf8q69AoEv8xiGcxvM+Y5JWDZmQceWvKilLw== X-Received: by 2002:a05:690c:d8f:b0:643:92a8:ba00 with SMTP id 00721157ae682-66ad5717b8dmr70191727b3.0.1721602430679; Sun, 21 Jul 2024 15:53:50 -0700 (PDT) Received: from localhost.localdomain ([190.194.167.233]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-70cff5538f6sm4223710b3a.100.2024.07.21.15.53.49 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 21 Jul 2024 15:53:49 -0700 (PDT) From: James Almer To: ffmpeg-devel@ffmpeg.org Date: Sun, 21 Jul 2024 19:53:50 -0300 Message-ID: <20240721225350.298-4-jamrial@gmail.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240721225350.298-1-jamrial@gmail.com> References: <20240721225350.298-1-jamrial@gmail.com> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 4/4 v2] avcodec/h264dec: add support for LCEVC enhancement X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 3zVSjqo4d0js Signed-off-by: James Almer --- configure | 2 +- libavcodec/avcodec.h | 5 +++++ libavcodec/h264_picture.c | 1 + libavcodec/h264_slice.c | 15 ++++++++++++++ libavcodec/h264dec.c | 41 ++++++++++++++++++++++++++++++++++++--- libavcodec/h264dec.h | 3 +++ 6 files changed, 63 insertions(+), 4 deletions(-) diff --git a/configure b/configure index d1f32684a6..7bc8c3b0af 100755 --- a/configure +++ b/configure @@ -2968,7 +2968,7 @@ h263i_decoder_select="h263_decoder" h263p_decoder_select="h263_decoder" h263p_encoder_select="h263_encoder" h264_decoder_select="cabac golomb h264chroma h264dsp h264parse h264pred h264qpel h264_sei videodsp" -h264_decoder_suggest="error_resilience" +h264_decoder_suggest="error_resilience liblcevc_dec" hap_decoder_select="snappy texturedsp" hap_encoder_deps="libsnappy" hap_encoder_select="texturedspenc" diff --git a/libavcodec/avcodec.h b/libavcodec/avcodec.h index 2da63c87ea..6fc46207db 100644 --- a/libavcodec/avcodec.h +++ b/libavcodec/avcodec.h @@ -418,6 +418,11 @@ typedef struct RcOverride{ * Do not apply film grain, export it instead. */ #define AV_CODEC_EXPORT_DATA_FILM_GRAIN (1 << 3) +/** + * Decoding only. + * Do not apply picture enhancement layers, export them instead. + */ +#define AV_CODEC_EXPORT_DATA_ENHANCEMENTS (1 << 4) /** * The decoder will keep a reference to the frame and may reuse it later. diff --git a/libavcodec/h264_picture.c b/libavcodec/h264_picture.c index 3234141dbd..1cdb926d15 100644 --- a/libavcodec/h264_picture.c +++ b/libavcodec/h264_picture.c @@ -103,6 +103,7 @@ static void h264_copy_picture_params(H264Picture *dst, const H264Picture *src) dst->mb_height = src->mb_height; dst->mb_stride = src->mb_stride; dst->needs_fg = src->needs_fg; + dst->needs_lcevc = src->needs_lcevc; } int ff_h264_ref_picture(H264Picture *dst, const H264Picture *src) diff --git a/libavcodec/h264_slice.c b/libavcodec/h264_slice.c index a66b75ca80..60b6fa4624 100644 --- a/libavcodec/h264_slice.c +++ b/libavcodec/h264_slice.c @@ -191,6 +191,11 @@ static int alloc_picture(H264Context *h, H264Picture *pic) av_assert0(!pic->f->data[0]); + if (pic->needs_lcevc) { + pic->f->width = FFMAX(h->avctx->width, h->avctx->coded_width) * 2 / FFMAX(h->avctx->sample_aspect_ratio.den, 1); + pic->f->height = FFMAX(h->avctx->height, h->avctx->coded_height) * 2 / FFMAX(h->avctx->sample_aspect_ratio.num, 1); + } + pic->tf.f = pic->f; ret = ff_thread_get_ext_buffer(h->avctx, &pic->tf, pic->reference ? AV_GET_BUFFER_FLAG_REF : 0); @@ -206,6 +211,11 @@ static int alloc_picture(H264Context *h, H264Picture *pic) goto fail; } + if (pic->needs_lcevc) { + pic->f->width = FFMAX(h->avctx->width, h->avctx->coded_width); + pic->f->height = FFMAX(h->avctx->height, h->avctx->coded_height); + } + ret = ff_hwaccel_frame_priv_alloc(h->avctx, &pic->hwaccel_picture_private); if (ret < 0) goto fail; @@ -459,6 +469,8 @@ int ff_h264_update_thread_context(AVCodecContext *dst, h->recovery_frame = h1->recovery_frame; h->non_gray = h1->non_gray; + av_buffer_replace(&h->lcevc, h1->lcevc); + return err; } @@ -519,6 +531,9 @@ static int h264_frame_start(H264Context *h) pic->needs_fg = h->sei.common.film_grain_characteristics.present && !h->avctx->hwaccel && !(h->avctx->export_side_data & AV_CODEC_EXPORT_DATA_FILM_GRAIN); + pic->needs_lcevc = CONFIG_LIBLCEVC_DEC && h->sei.common.lcevc.info && !h->avctx->hwaccel && + !(h->avctx->export_side_data & AV_CODEC_EXPORT_DATA_ENHANCEMENTS); + if ((ret = alloc_picture(h, pic)) < 0) return ret; diff --git a/libavcodec/h264dec.c b/libavcodec/h264dec.c index c77d8f42db..1c3a84f081 100644 --- a/libavcodec/h264dec.c +++ b/libavcodec/h264dec.c @@ -38,6 +38,7 @@ #include "libavutil/video_enc_params.h" #include "codec_internal.h" +#include "decode.h" #include "internal.h" #include "error_resilience.h" #include "avcodec.h" @@ -49,6 +50,7 @@ #include "golomb.h" #include "hwaccel_internal.h" #include "hwconfig.h" +#include "lcevcdec.h" #include "mpegutils.h" #include "profiles.h" #include "rectangle.h" @@ -377,6 +379,8 @@ static av_cold int h264_decode_end(AVCodecContext *avctx) h264_free_pic(h, &h->cur_pic); h264_free_pic(h, &h->last_pic_for_ec); + av_buffer_unref(&h->lcevc); + return 0; } @@ -404,6 +408,9 @@ FF_ENABLE_DEPRECATION_WARNINGS #endif if (!avctx->internal->is_copy) { +#if CONFIG_LIBLCEVC_DEC + FFLCEVCContext *lcevc; +#endif if (avctx->extradata_size > 0 && avctx->extradata) { ret = ff_h264_decode_extradata(avctx->extradata, avctx->extradata_size, &h->ps, &h->is_avc, &h->nal_length_size, @@ -418,6 +425,21 @@ FF_ENABLE_DEPRECATION_WARNINGS ret = 0; } } +#if CONFIG_LIBLCEVC_DEC + lcevc = av_mallocz(sizeof(FFLCEVCContext)); + ret = ff_lcevc_init(lcevc, avctx); + if (ret < 0) { + int explode = avctx->err_recognition & AV_EF_EXPLODE; + av_log(avctx, explode ? AV_LOG_ERROR: AV_LOG_WARNING, + "Error initializing LCEVC\n"); + if (explode) { + av_free(lcevc); + return ret; + } + ret = 0; + } + h->lcevc = av_buffer_create(lcevc, sizeof(FFLCEVCContext), ff_lcevc_free, lcevc, 0); +#endif } if (h->ps.sps && h->ps.sps->bitstream_restriction_flag && @@ -667,9 +689,20 @@ static int decode_nal_units(H264Context *h, const uint8_t *buf, int buf_size) h->setup_finished = 1; } - if (h->avctx->hwaccel && - (ret = FF_HW_CALL(h->avctx, start_frame, buf, buf_size)) < 0) - goto end; + if (h->avctx->hwaccel) { + ret = FF_HW_CALL(h->avctx, start_frame, buf, buf_size); + if (ret < 0) + goto end; + } else if (CONFIG_LIBLCEVC_DEC && h->cur_pic_ptr->needs_lcevc) { + FrameDecodeData *fdd = (FrameDecodeData*)h->cur_pic_ptr->f->private_ref->data; + fdd->post_process_opaque = av_buffer_ref(h->lcevc); + if (!fdd->post_process_opaque) { + ret = -1; + goto end; + } + fdd->post_process_opaque_free = ff_lcevc_unref; + fdd->post_process = ff_lcevc_process; + } } max_slice_ctx = avctx->hwaccel ? 1 : h->nb_slice_ctx; @@ -904,6 +937,8 @@ static int output_frame(H264Context *h, AVFrame *dst, H264Picture *srcp) if (!(h->avctx->export_side_data & AV_CODEC_EXPORT_DATA_FILM_GRAIN)) av_frame_remove_side_data(dst, AV_FRAME_DATA_FILM_GRAIN_PARAMS); + if (!(h->avctx->export_side_data & AV_CODEC_EXPORT_DATA_ENHANCEMENTS)) + av_frame_remove_side_data(dst, AV_FRAME_DATA_LCEVC); return 0; fail: diff --git a/libavcodec/h264dec.h b/libavcodec/h264dec.h index ccd7583bf4..419d699bab 100644 --- a/libavcodec/h264dec.h +++ b/libavcodec/h264dec.h @@ -154,6 +154,7 @@ typedef struct H264Picture { int invalid_gap; int sei_recovery_frame_cnt; int needs_fg; ///< whether picture needs film grain synthesis (see `f_grain`) + int needs_lcevc; ///< whether picture needs LCEVC enhancement const PPS *pps; @@ -580,6 +581,8 @@ typedef struct H264Context { int non_gray; ///< Did we encounter a intra frame after a gray gap frame int noref_gray; int skip_gray; + + AVBufferRef *lcevc; // FFLCEVCContext } H264Context; extern const uint16_t ff_h264_mb_sizes[4];