From patchwork Fri Jul 21 13:37:44 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "J. Dekker" X-Patchwork-Id: 42887 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:be1d:b0:130:ccc6:6c4b with SMTP id ge29csp228708pzb; Fri, 21 Jul 2023 06:38:42 -0700 (PDT) X-Google-Smtp-Source: APBJJlE/FgDMi+rXxHAeP+92zveqwi9QvGU/p7slmkn/OAfhdnlcW1yZx7srG66CnjL9euGYdHYg X-Received: by 2002:a17:906:7691:b0:991:c566:979 with SMTP id o17-20020a170906769100b00991c5660979mr1614332ejm.36.1689946722131; Fri, 21 Jul 2023 06:38:42 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1689946722; cv=none; d=google.com; s=arc-20160816; b=0h4TiRXMYDRhhFZUZ4LG5EXbQ2CuF2GvgbtRAjvhvSRbBW4LJtP2UjuYKVHNUMRCjT hnCLX6jTfNzAW34P8wVEGt5KDSmkhSO9z0dmyClSjRxImgkuYpSIfPwVq3MFNRQbjOiQ gaWBYLU/U75I0meRNzHOl1sJxo1Vu2uQvMXK+iUyQ/469MmKKWv+kMOAykXemA0c+LWm Qb844eoCjWeBawy490Z5/tgeJHadIPXA+eC4eArB+dZklY56Rb6lL4xFnTUwNJinrNSz mBsEf5WuP4f7OCfdG9HWnJy2hjPFGLqUdH6aTGA/NE3umK2LY4LUgmhBGZjes/RmPhnQ Qj6A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:feedback-id:dkim-signature:dkim-signature:delivered-to; bh=9g7AagCK5OlZ39f89EeiQGZELkXTQMsDNdzMHhayVPY=; fh=RX379NAUHdZIU7CQlxbXTMt5HBBB5CUKf4o4OtPlQjY=; b=Dk4TJ5tryJaQnh1cRxUkgJ780EZCY6In2Y97AwHoqGtGWFNOhBuPOGptgI8P4kmvNB womeOhaWZnAOHHOjjxltPZPmbX19k8Q7uKcPZJS9W210wCcSDfSxX8+Q58P0h32FEp7d WdWvC+UtRy7mKXaTkIDfz5tBn0DOhcgHOn2G4w/MI6FN0wqf0k+YK1VkKRBGMNA8/y73 wjK7U4R/Khu2hODUSsaH/WLeW9CLNXoK0T94poTr/Q2wIReT/WlAKXeB1UUqHet7tKUo 9gm4T/1G/9gI7nLPlIfywtJ4jKNMTx1Cw6+lz75hPPl55hRW11PzVSLKO+Yq03MSLEaT hMdA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@itanimul.li header.s=fm1 header.b=pbctysA3; dkim=neutral (body hash did not verify) header.i=@messagingengine.com header.s=fm3 header.b="Z3/kIZsL"; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id n23-20020a170906379700b00988be9d8c53si2166964ejc.946.2023.07.21.06.38.41; Fri, 21 Jul 2023 06:38:42 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@itanimul.li header.s=fm1 header.b=pbctysA3; dkim=neutral (body hash did not verify) header.i=@messagingengine.com header.s=fm3 header.b="Z3/kIZsL"; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 4FAA868C3D6; Fri, 21 Jul 2023 16:38:03 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from out2-smtp.messagingengine.com (out2-smtp.messagingengine.com [66.111.4.26]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 2574868C5DC for ; Fri, 21 Jul 2023 16:37:54 +0300 (EEST) Received: from compute5.internal (compute5.nyi.internal [10.202.2.45]) by mailout.nyi.internal (Postfix) with ESMTP id 466905C0131 for ; Fri, 21 Jul 2023 09:37:53 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute5.internal (MEProxy); Fri, 21 Jul 2023 09:37:53 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=itanimul.li; h= cc:content-transfer-encoding:content-type:date:date:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:sender:subject:subject:to:to; s=fm1; t=1689946673; x= 1690033073; bh=+URvZCdRy/h0qcZIfId2bkiRXUOKMDGwHgtD8iG+rLw=; b=p bctysA35RPUbVt1a28JKUC7RU7/vhvMcqym3JSqPA0vh5upz2SZIE+nqb5ZNC9hL igW5IjfMZtfh+KAmrZ+jtiheeRz4IS8nTsksUBWnkJlRpDfmoX8Rj4Cc7ZD5E0Xf Yx8fw/CYoL6YDLEHvIfaTZEqh2DfcRVp5KIy9TY3bKgh3S5To7QEHUThyUJebpgy GoHE9fvvuOSWANrtSBKCQAROl2X5xO7EbAFYOWhWDtd3JeTOm3BCKJiXiLrfBxPO P8NZ3LtcqRvVtW5MWp6P0F9V+BrcEV0mKsc/UJ0zqk6ijl/O4e/Wh1GzDzLA0lad O/1PruzglUGxeW6E/VnYg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:date:feedback-id:feedback-id:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:sender :subject:subject:to:to:x-me-proxy:x-me-proxy:x-me-sender :x-me-sender:x-sasl-enc; s=fm3; t=1689946673; x=1690033073; bh=+ URvZCdRy/h0qcZIfId2bkiRXUOKMDGwHgtD8iG+rLw=; b=Z3/kIZsLJWa9k9AN+ VCJrVe+EjIZPCtK2firMaRgN3RIq3cqSSnrvxrqqglNNhU2n/oT1b1AWwVwNN2R7 Apv0Fa5Sm5CFdaj1PMGWgEZdaR2RkMe+y0tyECUnAM+C4GQmyyTqEteVlIMXD3Ot puoocEAideKI29oDYm1N/d/XVYTiur7gEseW4bZz1tuxvH0KgS+o5IlsMpJZXR6E 1jM44p4oBCjxeyZQjFu+1ZKupjNsA0E1EBP5eekYT18HRnty9Rp70ErKmCT+2NU3 nFiWjkazXhOXXWlTaU1IHrW8wmQudWLPVCfQY44sFazVBxQ9YPuWRZa+HlioUsJL qFXhg== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedviedrhedvgdeivdcutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecunecujfgurhephffvufffkffojghfggfgsedtkeertd ertddtnecuhfhrohhmpedflfdrucffvghkkhgvrhdfuceojhguvghksehithgrnhhimhhu lhdrlhhiqeenucggtffrrghtthgvrhhnpeeuvddukefftdfggfekteekteekvefgheekvd duueethefhvdefkeekteelhfeljeenucffohhmrghinheptghomhhmohhnrdhfihhlmhen ucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehjuggvkh esihhtrghnihhmuhhlrdhlih X-ME-Proxy: Feedback-ID: i84994747:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA for ; Fri, 21 Jul 2023 09:37:52 -0400 (EDT) From: "J. Dekker" To: ffmpeg-devel@ffmpeg.org Date: Fri, 21 Jul 2023 15:37:44 +0200 Message-ID: <20230721133746.33335-5-jdek@itanimul.li> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230721133746.33335-1-jdek@itanimul.li> References: <20230721133746.33335-1-jdek@itanimul.li> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 5/7] lavc/h264: export ecinfo X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: wcLEuY1rkZAl Export ecinfo to the user when AV_CODEC_EXPORT_DATA_ERROR is set. Co-Authored-By: Thomas Guillem Signed-off-by: J. Dekker --- libavcodec/h264_picture.c | 10 ++++++++-- libavcodec/h264_slice.c | 16 +++++++++++----- libavcodec/h264dec.c | 33 +++++++++++++++++++++++++++------ libavcodec/h264dec.h | 8 +++++++- 4 files changed, 53 insertions(+), 14 deletions(-) diff --git a/libavcodec/h264_picture.c b/libavcodec/h264_picture.c index dcaf0fdb0a..aa7c5ac673 100644 --- a/libavcodec/h264_picture.c +++ b/libavcodec/h264_picture.c @@ -48,6 +48,7 @@ void ff_h264_unref_picture(H264Context *h, H264Picture *pic) av_buffer_unref(&pic->qscale_table_buf); av_buffer_unref(&pic->mb_type_buf); av_buffer_unref(&pic->pps_buf); + av_buffer_unref(&pic->ec_info_buf); for (i = 0; i < 2; i++) { av_buffer_unref(&pic->motion_val_buf[i]); av_buffer_unref(&pic->ref_index_buf[i]); @@ -61,6 +62,7 @@ static void h264_copy_picture_params(H264Picture *dst, const H264Picture *src) dst->qscale_table = src->qscale_table; dst->mb_type = src->mb_type; dst->pps = src->pps; + dst->ec_info = src->ec_info; for (int i = 0; i < 2; i++) { dst->motion_val[i] = src->motion_val[i]; @@ -111,7 +113,9 @@ int ff_h264_ref_picture(H264Context *h, H264Picture *dst, H264Picture *src) dst->qscale_table_buf = av_buffer_ref(src->qscale_table_buf); dst->mb_type_buf = av_buffer_ref(src->mb_type_buf); dst->pps_buf = av_buffer_ref(src->pps_buf); - if (!dst->qscale_table_buf || !dst->mb_type_buf || !dst->pps_buf) { + dst->ec_info_buf = av_buffer_ref(src->ec_info_buf); + if (!dst->qscale_table_buf || !dst->mb_type_buf || !dst->pps_buf + || !dst->ec_info_buf) { ret = AVERROR(ENOMEM); goto fail; } @@ -168,6 +172,7 @@ int ff_h264_replace_picture(H264Context *h, H264Picture *dst, const H264Picture ret = av_buffer_replace(&dst->qscale_table_buf, src->qscale_table_buf); ret |= av_buffer_replace(&dst->mb_type_buf, src->mb_type_buf); ret |= av_buffer_replace(&dst->pps_buf, src->pps_buf); + ret |= av_buffer_replace(&dst->ec_info_buf, src->ec_info_buf); if (ret < 0) goto fail; @@ -192,7 +197,7 @@ fail: return ret; } -void ff_h264_set_erpic(ERPicture *dst, H264Picture *src) +void ff_h264_set_erpic(ERPicture *dst, H264Picture *src, bool export_error) { #if CONFIG_ERROR_RESILIENCE int i; @@ -202,6 +207,7 @@ void ff_h264_set_erpic(ERPicture *dst, H264Picture *src) if (!src) return; + dst->info = export_error ? src->ec_info : NULL; dst->f = src->f; dst->tf = &src->tf; diff --git a/libavcodec/h264_slice.c b/libavcodec/h264_slice.c index 41bf30eefc..b291405c2a 100644 --- a/libavcodec/h264_slice.c +++ b/libavcodec/h264_slice.c @@ -176,12 +176,15 @@ static int init_table_pools(H264Context *h) sizeof(int16_t), av_buffer_allocz); h->ref_index_pool = av_buffer_pool_init(4 * mb_array_size, av_buffer_allocz); + h->ec_info_pool = av_buffer_pool_init(sizeof(AVECInfo), av_buffer_allocz); + if (!h->qscale_table_pool || !h->mb_type_pool || !h->motion_val_pool || - !h->ref_index_pool) { + !h->ref_index_pool || !h->ec_info_pool) { av_buffer_pool_uninit(&h->qscale_table_pool); av_buffer_pool_uninit(&h->mb_type_pool); av_buffer_pool_uninit(&h->motion_val_pool); av_buffer_pool_uninit(&h->ref_index_pool); + av_buffer_pool_uninit(&h->ec_info_pool); return AVERROR(ENOMEM); } @@ -240,11 +243,13 @@ static int alloc_picture(H264Context *h, H264Picture *pic) pic->qscale_table_buf = av_buffer_pool_get(h->qscale_table_pool); pic->mb_type_buf = av_buffer_pool_get(h->mb_type_pool); - if (!pic->qscale_table_buf || !pic->mb_type_buf) + pic->ec_info_buf = av_buffer_pool_get(h->ec_info_pool); + if (!pic->qscale_table_buf || !pic->mb_type_buf || !pic->ec_info_buf) goto fail; pic->mb_type = (uint32_t*)pic->mb_type_buf->data + 2 * h->mb_stride + 1; pic->qscale_table = pic->qscale_table_buf->data + 2 * h->mb_stride + 1; + pic->ec_info = (AVECInfo*) pic->ec_info_buf->data; for (i = 0; i < 2; i++) { pic->motion_val_buf[i] = av_buffer_pool_get(h->motion_val_pool); @@ -514,6 +519,7 @@ FF_ENABLE_DEPRECATION_WARNINGS pic->f->crop_right = h->crop_right; pic->f->crop_top = h->crop_top; pic->f->crop_bottom = h->crop_bottom; + pic->error_decode_slices = 0; pic->needs_fg = h->sei.common.film_grain_characteristics.present && !h->avctx->hwaccel && !(h->avctx->export_side_data & AV_CODEC_EXPORT_DATA_FILM_GRAIN); @@ -524,7 +530,7 @@ FF_ENABLE_DEPRECATION_WARNINGS h->cur_pic_ptr = pic; ff_h264_unref_picture(h, &h->cur_pic); if (CONFIG_ERROR_RESILIENCE) { - ff_h264_set_erpic(&h->er.cur_pic, NULL); + ff_h264_set_erpic(&h->er.cur_pic, NULL, false); } if ((ret = ff_h264_ref_picture(h, &h->cur_pic, h->cur_pic_ptr)) < 0) @@ -537,8 +543,8 @@ FF_ENABLE_DEPRECATION_WARNINGS if (CONFIG_ERROR_RESILIENCE && h->enable_er) { ff_er_frame_start(&h->er); - ff_h264_set_erpic(&h->er.last_pic, NULL); - ff_h264_set_erpic(&h->er.next_pic, NULL); + ff_h264_set_erpic(&h->er.last_pic, NULL, false); + ff_h264_set_erpic(&h->er.next_pic, NULL, false); } for (i = 0; i < 16; i++) { diff --git a/libavcodec/h264dec.c b/libavcodec/h264dec.c index 19f8dba131..fa30f75f99 100644 --- a/libavcodec/h264dec.c +++ b/libavcodec/h264dec.c @@ -153,6 +153,7 @@ void ff_h264_free_tables(H264Context *h) av_buffer_pool_uninit(&h->mb_type_pool); av_buffer_pool_uninit(&h->motion_val_pool); av_buffer_pool_uninit(&h->ref_index_pool); + av_buffer_pool_uninit(&h->ec_info_pool); #if CONFIG_ERROR_RESILIENCE av_freep(&h->er.mb_index2xy); @@ -741,7 +742,7 @@ static int decode_nal_units(H264Context *h, const uint8_t *buf, int buf_size) // set decode_error_flags to allow users to detect concealed decoding errors if ((ret < 0 || h->er.error_occurred) && h->cur_pic_ptr) { - h->cur_pic_ptr->f->decode_error_flags |= FF_DECODE_ERROR_DECODE_SLICES; + h->cur_pic_ptr->error_decode_slices = 1; } ret = 0; @@ -764,24 +765,34 @@ end: H264SliceContext *sl = h->slice_ctx; int use_last_pic = h->last_pic_for_ec.f->buf[0] && !sl->ref_count[0]; + bool export_error = avctx->export_side_data & AV_CODEC_EXPORT_DATA_ERROR; - ff_h264_set_erpic(&h->er.cur_pic, h->cur_pic_ptr); + ff_h264_set_erpic(&h->er.cur_pic, h->cur_pic_ptr, export_error); if (use_last_pic) { - ff_h264_set_erpic(&h->er.last_pic, &h->last_pic_for_ec); + ff_h264_set_erpic(&h->er.last_pic, &h->last_pic_for_ec, export_error); sl->ref_list[0][0].parent = &h->last_pic_for_ec; memcpy(sl->ref_list[0][0].data, h->last_pic_for_ec.f->data, sizeof(sl->ref_list[0][0].data)); memcpy(sl->ref_list[0][0].linesize, h->last_pic_for_ec.f->linesize, sizeof(sl->ref_list[0][0].linesize)); sl->ref_list[0][0].reference = h->last_pic_for_ec.reference; } else if (sl->ref_count[0]) { - ff_h264_set_erpic(&h->er.last_pic, sl->ref_list[0][0].parent); + ff_h264_set_erpic(&h->er.last_pic, sl->ref_list[0][0].parent, export_error); } else - ff_h264_set_erpic(&h->er.last_pic, NULL); + ff_h264_set_erpic(&h->er.last_pic, NULL, export_error); if (sl->ref_count[1]) - ff_h264_set_erpic(&h->er.next_pic, sl->ref_list[1][0].parent); + ff_h264_set_erpic(&h->er.next_pic, sl->ref_list[1][0].parent, export_error); ff_er_frame_end(&h->er); + + /* Copy the ec_info into out_ec_info now since the ec_info might be + * written by an other thread while outputing the picture. Don't add + * the ec_info in the AVFrame side data since the AVFrame metadata + * can't be modified after calling ff_thread_finish_setup() (prevent + * data-races). */ + if (export_error) + h->cur_pic_ptr->out_ec_info = *h->cur_pic_ptr->ec_info; + if (use_last_pic) memset(&sl->ref_list[0][0], 0, sizeof(sl->ref_list[0][0])); } @@ -850,6 +861,16 @@ static int output_frame(H264Context *h, AVFrame *dst, H264Picture *srcp) if (ret < 0) return ret; + if (srcp->out_ec_info.error | srcp->out_ec_info.ref_error) { + AVECInfo *eci = av_eci_create_side_data(dst); + if (eci) + *eci = srcp->out_ec_info; + dst->decode_error_flags |= FF_DECODE_ERROR_CONCEALMENT_ACTIVE; + av_log(h, AV_LOG_DEBUG, "ecinfo error: %" PRIu64 ", ref_error: %" PRIu64 "\n", eci->error, eci->ref_error); + } + if (srcp->error_decode_slices) + dst->decode_error_flags |= FF_DECODE_ERROR_DECODE_SLICES; + if (srcp->needs_fg && (ret = av_frame_copy_props(dst, srcp->f)) < 0) return ret; diff --git a/libavcodec/h264dec.h b/libavcodec/h264dec.h index 9a1ec1bace..5a541e2c0b 100644 --- a/libavcodec/h264dec.h +++ b/libavcodec/h264dec.h @@ -154,6 +154,11 @@ typedef struct H264Picture { int mb_width, mb_height; int mb_stride; + + AVBufferRef *ec_info_buf; + AVECInfo *ec_info; + AVECInfo out_ec_info; + int error_decode_slices; } H264Picture; typedef struct H264Ref { @@ -551,6 +556,7 @@ typedef struct H264Context { AVBufferPool *mb_type_pool; AVBufferPool *motion_val_pool; AVBufferPool *ref_index_pool; + AVBufferPool *ec_info_pool; int ref2frm[MAX_SLICES][2][64]; ///< reference to frame number lists, used in the loop filter, the first 2 are for -2,-1 } H264Context; @@ -806,6 +812,6 @@ void ff_h264_flush_change(H264Context *h); void ff_h264_free_tables(H264Context *h); -void ff_h264_set_erpic(ERPicture *dst, H264Picture *src); +void ff_h264_set_erpic(ERPicture *dst, H264Picture *src, bool export_error); #endif /* AVCODEC_H264DEC_H */