From patchwork Fri Jul 21 13:37:42 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "J. Dekker" X-Patchwork-Id: 42885 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6a20:be1d:b0:130:ccc6:6c4b with SMTP id ge29csp228528pzb; Fri, 21 Jul 2023 06:38:25 -0700 (PDT) X-Google-Smtp-Source: APBJJlE1d7v6sYFT66qu4nXnZJF+rN+Vm11K4Q2gRImd8C8+MmB6ITFxEGBieOMnLyzqO4qYCHqX X-Received: by 2002:a17:907:c29:b0:993:f664:ce25 with SMTP id ga41-20020a1709070c2900b00993f664ce25mr10491910ejc.19.1689946704732; Fri, 21 Jul 2023 06:38:24 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1689946704; cv=none; d=google.com; s=arc-20160816; b=EgQMcokKggSZiIQk8sshQFTV8zFgR4jV/07tiw8c0pr0DIMYUfu9aDaOKk7dThqP/r EPo/Wp9s6aQGjNX6DDDAjiECFk4FcOSJPx6wHqK4oz8vKPbCnxLkqeVONp7ZjTlWflFv TK1zzP/8Ih0OpmsKmmRyxduTBnJ35EKdTWsUlrz/C6FNqaKDKv+vrLRdvKggITv10YzI C3BLB5N9OyteD9L1/FWBbZ8ACoaNAYDxau5s4D338B74NAzPFegPIpfKEebu300G5hAI 0petmsM2zxiOL4/qLkDQO1XRj0doja4y7EOSBRBeNBPUUuDx4VoM91lV6dALjsW21HbX 3Jfw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:reply-to:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:from:feedback-id:dkim-signature:dkim-signature:delivered-to; bh=XPDQvvMxc2VDuQHi2gV4j0t4ShI7LMPOSxpKoSGMy7A=; fh=RX379NAUHdZIU7CQlxbXTMt5HBBB5CUKf4o4OtPlQjY=; b=CskbZofbetdzv+8Yiba+OaQ74vnpQ8a0DKd+Qrxxt65jyY7o8X0+Wc5lYn+Fy8/3T/ +JfU55uqAoKzADL4A+oJ2ZAjHUWfrPRvWhHG8BE+vvL9HRU5pmCHMM3IZvteen6cpYIy kCIpn8zNX5OAN3hAEX8xpDh0OhOnBdxcC/4flKiQ3VTEdIFkIUyf8MRmkxxNHnW6Subg w23nH8wR952ZLXNSR5/JGgvjKDZ3kNjUDwKe/dY9U34Umu11jBvI0a8yj5Q4/0h5uoR7 UHB8pYb37Vuo0xlvdypbpaMIzDiqh3JW9TlvmiIh094q28QVOrETiW4uAANNdy25lpeA EbaQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@itanimul.li header.s=fm1 header.b=K2ATTkPf; dkim=neutral (body hash did not verify) header.i=@messagingengine.com header.s=fm3 header.b=RQmt0SAa; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id h11-20020aa7de0b000000b0051e05cd6191si2301365edv.28.2023.07.21.06.38.24; Fri, 21 Jul 2023 06:38:24 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@itanimul.li header.s=fm1 header.b=K2ATTkPf; dkim=neutral (body hash did not verify) header.i=@messagingengine.com header.s=fm3 header.b=RQmt0SAa; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id E9DA268BE88; Fri, 21 Jul 2023 16:38:00 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from out2-smtp.messagingengine.com (out2-smtp.messagingengine.com [66.111.4.26]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 4686F68C50A for ; Fri, 21 Jul 2023 16:37:52 +0300 (EEST) Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailout.nyi.internal (Postfix) with ESMTP id 6828A5C0152 for ; Fri, 21 Jul 2023 09:37:51 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute4.internal (MEProxy); Fri, 21 Jul 2023 09:37:51 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=itanimul.li; h= cc:content-transfer-encoding:content-type:date:date:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:sender:subject:subject:to:to; s=fm1; t=1689946671; x= 1690033071; bh=a99xMkyUHPVI5hgR8eKAChGM4msynWHyYwPHg21+YBU=; b=K 2ATTkPftS7BtGGi5oHiJcwGVou70rhrLl1IlWauzhi4cCTQzSrLoPjbbY7uj+LWu Dn6OZREhMjcgDSHyX4pSZ7D9H6MDb1uoL3yXVxegf+FsGvZrUARTRx9Zv6wtAU80 Dp8REFwF+/U/PpHt8UN2agvXMRFi7A0fWLDafS4GobRTpyKhsysrxcctr4Z8j3z6 KphIPm+8vvIjF+fZzjou0ZPZVihTJQCGPTuQ/igVNH74p++GDZAR3D9QK4lAwfh4 z2rsYhRKc1juDEkfKXsk41ds/Dr8Inm+sxLlg/20aaBJtf2vlni/6Eax416Vb3Ww KV6By6jyvNKyuaHtBBmIQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:date:feedback-id:feedback-id:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:sender :subject:subject:to:to:x-me-proxy:x-me-proxy:x-me-sender :x-me-sender:x-sasl-enc; s=fm3; t=1689946671; x=1690033071; bh=a 99xMkyUHPVI5hgR8eKAChGM4msynWHyYwPHg21+YBU=; b=RQmt0SAabMrhZEuvv I4CZE+OFTIY26WtppVperIh/g4quxti3l5e367zMxSx1XW1mVab6xCzJpLCiRG0y W6o/tHKlEFrx2x4iiJIbtmp7gxb+2gkvC16Hp6NCnGIAA/56AQLhsAByVQa7AEGa u40ar3fIykgLVNDmOQHmZ9SResahxiV01jmRM30BVqeRo5TzGHJoyNi2uzhUcpVb OPjzA3gu/VsP/HVvVnWGsb5j50ruaDXr9UySW1iUn1kagsxngVTHnU7yFf3UPpLH 0G2lovyXkornEInc0zLbLkeleXFPKdRFoAdLvEtDPjB4/B282/8J7GoTN+2lKbVf di0Qw== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedviedrhedvgdeivdcutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecunecujfgurhephffvufffkffojghfggfgsedtkeertd ertddtnecuhfhrohhmpedflfdrucffvghkkhgvrhdfuceojhguvghksehithgrnhhimhhu lhdrlhhiqeenucggtffrrghtthgvrhhnpeegvdfhffduudegheevieekieeffefgveffke dtheekieefhfevueetueefhfetteenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgr mhepmhgrihhlfhhrohhmpehjuggvkhesihhtrghnihhmuhhlrdhlih X-ME-Proxy: Feedback-ID: i84994747:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA for ; Fri, 21 Jul 2023 09:37:50 -0400 (EDT) From: "J. Dekker" To: ffmpeg-devel@ffmpeg.org Date: Fri, 21 Jul 2023 15:37:42 +0200 Message-ID: <20230721133746.33335-3-jdek@itanimul.li> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230721133746.33335-1-jdek@itanimul.li> References: <20230721133746.33335-1-jdek@itanimul.li> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 3/7] lavc/error_resilience: fill ecinfo X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: RTamLCcDtwkp Fill ECInfo inside error resilience using references set by the decoder. Co-Authored-By: Thomas Guillem Signed-off-by: J. Dekker --- libavcodec/error_resilience.c | 91 +++++++++++++++++++++++++++++------ libavcodec/error_resilience.h | 4 +- 2 files changed, 79 insertions(+), 16 deletions(-) diff --git a/libavcodec/error_resilience.c b/libavcodec/error_resilience.c index 2aa6f1d864..c1417ecf07 100644 --- a/libavcodec/error_resilience.c +++ b/libavcodec/error_resilience.c @@ -28,6 +28,7 @@ #include #include "libavutil/internal.h" +#include "libavutil/ec.h" #include "avcodec.h" #include "error_resilience.h" #include "me_cmp.h" @@ -411,7 +412,7 @@ static void guess_mv(ERContext *s) num_avail = 0; if (s->last_pic.motion_val[0]) - ff_thread_await_progress(s->last_pic.tf, mb_height-1, 0); + ff_thread_await_progress(s->last_pic.tf, INT_MAX, 0); for (i = 0; i < mb_width * mb_height; i++) { const int mb_xy = s->mb_index2xy[i]; int f = 0; @@ -889,25 +890,61 @@ void ff_er_add_slice(ERContext *s, int startx, int starty, } } -void ff_er_frame_end(ERContext *s) +static void +er_fill_info_ref(ERContext *s) +{ + uint64_t acc_ok = 0, acc_err = 0; + av_assert0(s->cur_pic.info); + + if (s->cur_pic.f->pict_type != AV_PICTURE_TYPE_I) { + ERPicture *reffs[2] = {&s->last_pic, &s->next_pic}; + int i, nb_ref_pics = s->cur_pic.f->pict_type == AV_PICTURE_TYPE_B ? 2 : 1; + + for (i = 0; i < nb_ref_pics; i++) { + ERPicture *reff = reffs[i]; + + if (reff->info == NULL) + continue; + + ff_thread_await_progress(reff->tf, INT_MAX, 0); + + /* should check more accurately how refs are used */ + if (reff->info->error == 0 && reff->info->ref_error == 0) + continue; + + if (acc_err < (reff->info->error + reff->info->ref_error)){ + acc_err = reff->info->error + reff->info->ref_error; + acc_ok = reff->info->ok + reff->info->ref_ok; + } + } + } + + s->cur_pic.info->ref_error = acc_err; + s->cur_pic.info->ref_ok = acc_ok; +} + +int ff_er_frame_end(ERContext *s) { int *linesize = NULL; - int i, mb_x, mb_y, error, error_type, dc_error, mv_error, ac_error; + int i, mb_x, mb_y, error, error_type, dc_error, mv_error, ac_error, terror; int distance; int threshold_part[4] = { 100, 100, 100 }; int threshold = 50; int is_intra_likely; int size = s->b8_stride * 2 * s->mb_height; + /* We do not support ER of field pictures yet, * though it should not crash if enabled. */ - if (!s->avctx->error_concealment || !atomic_load(&s->error_count) || - s->avctx->lowres || - !er_supported(s) || - atomic_load(&s->error_count) == 3 * s->mb_width * - (s->avctx->skip_top + s->avctx->skip_bottom)) { - return; + if (!s->avctx->error_concealment || s->avctx->lowres || !er_supported(s)) + return 0; + + if (!atomic_load(&s->error_count) || + atomic_load(&s->error_count) == 3 * s->mb_width * (s->avctx->skip_top + s->avctx->skip_bottom)) + { + goto end_find_ref_errs; } + linesize = s->cur_pic.f->linesize; if ( s->avctx->codec_id == AV_CODEC_ID_MPEG2VIDEO @@ -921,7 +958,7 @@ void ff_er_frame_end(ERContext *s) if (mb_x == s->mb_width) { av_log(s->avctx, AV_LOG_DEBUG, "ignoring last missing slice\n"); - return; + goto end_find_ref_errs; } } @@ -960,7 +997,7 @@ void ff_er_frame_end(ERContext *s) s->cur_pic.ref_index[i] = NULL; s->cur_pic.motion_val[i] = NULL; } - return; + goto end_find_ref_errs; } } @@ -1100,10 +1137,12 @@ void ff_er_frame_end(ERContext *s) } #endif - dc_error = ac_error = mv_error = 0; + terror = dc_error = ac_error = mv_error = 0; for (i = 0; i < s->mb_num; i++) { const int mb_xy = s->mb_index2xy[i]; int error = s->error_status_table[mb_xy]; + if (error) + terror++; if (error & ER_DC_ERROR) dc_error++; if (error & ER_AC_ERROR) @@ -1111,10 +1150,9 @@ void ff_er_frame_end(ERContext *s) if (error & ER_MV_ERROR) mv_error++; } - av_log(s->avctx, AV_LOG_INFO, "concealing %d DC, %d AC, %d MV errors in %c frame\n", - dc_error, ac_error, mv_error, av_get_picture_type_char(s->cur_pic.f->pict_type)); - s->cur_pic.f->decode_error_flags |= FF_DECODE_ERROR_CONCEALMENT_ACTIVE; + fprintf(stderr, "concealing %d DC, %d AC, %d MV errors in %c frame: %d\n", + dc_error, ac_error, mv_error, av_get_picture_type_char(s->cur_pic.f->pict_type), terror * 256); is_intra_likely = is_intra_more_likely(s); @@ -1349,7 +1387,30 @@ void ff_er_frame_end(ERContext *s) s->cur_pic.motion_val[i] = NULL; } + av_assert0(terror); + if (s->cur_pic.info) { + /* assumes 16x16 mbs */ + s->cur_pic.info->error = terror * 256; + s->cur_pic.info->ok = (s->mb_num - terror) * 256; + er_fill_info_ref(s); + } + + memset(&s->cur_pic, 0, sizeof(ERPicture)); + memset(&s->last_pic, 0, sizeof(ERPicture)); + memset(&s->next_pic, 0, sizeof(ERPicture)); + + return 1; + +end_find_ref_errs: + if (s->cur_pic.info) + { + er_fill_info_ref(s); + s->cur_pic.info->error = s->cur_pic.info->ok = 0; + } + memset(&s->cur_pic, 0, sizeof(ERPicture)); memset(&s->last_pic, 0, sizeof(ERPicture)); memset(&s->next_pic, 0, sizeof(ERPicture)); + + return 0; } diff --git a/libavcodec/error_resilience.h b/libavcodec/error_resilience.h index 47cc8a4fc6..97479d2c0c 100644 --- a/libavcodec/error_resilience.h +++ b/libavcodec/error_resilience.h @@ -25,6 +25,7 @@ #include "avcodec.h" #include "me_cmp.h" #include "threadframe.h" +#include "libavutil/ec.h" ///< current MB is the first after a resync marker #define VP_START 1 @@ -48,6 +49,7 @@ typedef struct ERPicture { uint32_t *mb_type; int field_picture; + AVECInfo *info; } ERPicture; typedef struct ERContext { @@ -90,7 +92,7 @@ typedef struct ERContext { } ERContext; void ff_er_frame_start(ERContext *s); -void ff_er_frame_end(ERContext *s); +int ff_er_frame_end(ERContext *s); void ff_er_add_slice(ERContext *s, int startx, int starty, int endx, int endy, int status);