From patchwork Sat Aug 14 11:36:18 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Niklas Haas X-Patchwork-Id: 29508 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a05:6602:2a4a:0:0:0:0 with SMTP id k10csp380306iov; Sat, 14 Aug 2021 04:36:52 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwXQwlNtKIEkEALEpEDAeON+3Tu1YA7zNzOJSzEMialMv/Ng6VDXGr0Whxg1voopAqhGmVA X-Received: by 2002:a05:6402:22ab:: with SMTP id cx11mr8605246edb.240.1628941012188; Sat, 14 Aug 2021 04:36:52 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1628941012; cv=none; d=google.com; s=arc-20160816; b=zZAcIAm+CCR1iUseoRkxmvxawcH9esomtyQsgYli8W253Wom5yALcWk0OfzofLgl17 5/8O30Ewj8EAjHcD7cbZ3RoaTWk8mjLx7/9uEFPUvco6NYqwy91e5eaTH6X7KSUoMhSh NInpM0zWo2hd2/WQIgaO1sKx0UUnMCoqarMX159brMgEJHiIfoFpmJEYzt25pzi7hUm6 OLfcWE5U4mb2L5bdZeI6pa/8/jkJjPTozS3/LtOK4X9rRiZ8PaULcGy10YbkJvVvWb7k lPEJQom9EexFjFGogUFbmSraiN8clgciiWKfqXWzyEQmmR9SfUtABpYSJxw1AoFQCEql 6VvA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:delivered-to; bh=es52bofGWQHATlDm6yQwXTirl6bbiXYFYUnQvE4vDx4=; b=XwRli2RWc51taDf9jlmJbygkK5bxglWZCpuGoq82IdtNIu/0WnGf55fyBzJ6JERx2D xb9aMtSM4OrRnp7zK0T1Z4kGEVisEcXTw0jfdr73N9ycGHEnmKIqZ3y2Tqrr/QKmf63p 1txw57lyxV+H25MHrF2D+WIn1wEYZypD6d2vcEWBer9x9EtY1WCOwHvM9Y7ItUp1pIwC ck23PZyYClt4MGVX49YQ3ADZCV0ufRwrHsnWHXUKag82tMsY5mki0h4ZC00cJV70fHob tgxcPhWGQJSONrVySFkLH/aeK8GR9y/sXNI1Xon0sKyYKWJS324kkZjGJWJwj6xEml7z CkwQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@haasn.xyz header.s=mail header.b=Ck95ooOf; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id d22si4600454edx.92.2021.08.14.04.36.51; Sat, 14 Aug 2021 04:36:52 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@haasn.xyz header.s=mail header.b=Ck95ooOf; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 1CAC268A62F; Sat, 14 Aug 2021 14:36:41 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from haasn.dev (haasn.dev [78.46.187.166]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 875DE68A613 for ; Sat, 14 Aug 2021 14:36:34 +0300 (EEST) Received: from haasn.dev (unknown [10.30.0.2]) by haasn.dev (Postfix) with ESMTP id 4E839435AB; Sat, 14 Aug 2021 13:36:34 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=haasn.xyz; s=mail; t=1628940994; bh=vuug1d7qXqvGypHVWcGojTdmPV26kJ4dg/X57RU9/yg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Ck95ooOfVh1bY3rhlLgIYZpNZDHLgMWWkCxmLYBvSz/rEXN5ZldCID49qU+5Da2tH o94ClZT1AbxYLamxpRvQzpmSRuaNDRLZO5I78CZvmIDajI08MnPjj3oG7GTMOYNJpG ZHsAZRw/P9DkP5TDVo+0KrMxJvBtMBHEVxSMSCck= From: Niklas Haas To: ffmpeg-devel@ffmpeg.org Date: Sat, 14 Aug 2021 13:36:18 +0200 Message-Id: <20210814113620.39290-2-ffmpeg@haasn.xyz> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210814113620.39290-1-ffmpeg@haasn.xyz> References: <20210814113620.39290-1-ffmpeg@haasn.xyz> MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH 2/4] avcodec/h264_slice: compute and export film grain seed X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Niklas Haas Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: eKsMiIZZmQTs From: Niklas Haas From SMPTE RDD 5-2006, the grain seed is to be computed from the following definition of `pic_offset`: > When decoding H.264 | MPEG-4 AVC bitstreams, pic_offset is defined as > follows: > - pic_offset = PicOrderCnt(CurrPic) + (PicOrderCnt_offset << 5) > where: > - PicOrderCnt(CurrPic) is the picture order count of the current frame, > which shall be derived from [the video stream]. > > - PicOrderCnt_offset is set to idr_pic_id on IDR frames. idr_pic_id > shall be read from the slice header of [the video stream]. On non-IDR I > frames, PicOrderCnt_offset is set to 0. A frame shall be classified as I > frame when all its slices are I slices, which may be optionally > designated by setting primary_pic_type to 0 in the access delimiter NAL > unit. Otherwise, PicOrderCnt_offset it not changed. PicOrderCnt_offset is > updated in decoding order. To forward this information correctly, we need to make a number of annoying changes to h264_slice: - We need to keep track of the `PicOrderCnt_offset`, and update it to the decoded value of `idr_pic_id` on IDR frames. - We need to keep track of whether or not a frame contained any non-I slices, so we can detect the "non-IDR I frame" condition, and reset `PicOrderCnt_offset` back to 0. This is *not* synonymous with existing fields such as `pic->f->pict_type` because that is somehow only set based on the first slice in a frame, so we introduce a new field `picture_intra_only` which is set to 0 when decoding any non-I slice. - We need to compute this derived `pic_offset` at the *end* of the frame, rather than at the beginning (where the SEI is decoded/set), because it now depends on the types of the slices present in the same frame. If there's a less ugly way to accomplish the above, I don't see it. Suggestions welcome(tm). Signed-off-by: Niklas Haas --- libavcodec/h264_picture.c | 13 ++++++++++++- libavcodec/h264_slice.c | 11 ++++++++++- libavcodec/h264dec.h | 7 +++++++ libavutil/film_grain_params.h | 3 +++ 4 files changed, 32 insertions(+), 2 deletions(-) diff --git a/libavcodec/h264_picture.c b/libavcodec/h264_picture.c index ff30166b4d..b79944b794 100644 --- a/libavcodec/h264_picture.c +++ b/libavcodec/h264_picture.c @@ -27,6 +27,7 @@ #include "libavutil/avassert.h" #include "libavutil/imgutils.h" +#include "libavutil/film_grain_params.h" #include "internal.h" #include "cabac.h" #include "cabac_functions.h" @@ -212,6 +213,8 @@ void ff_h264_set_erpic(ERPicture *dst, H264Picture *src) int ff_h264_field_end(H264Context *h, H264SliceContext *sl, int in_setup) { AVCodecContext *const avctx = h->avctx; + H264Picture *cur = h->cur_pic_ptr; + AVFrameSideData *sd; int err = 0; h->mb_y = 0; @@ -225,6 +228,9 @@ int ff_h264_field_end(H264Context *h, H264SliceContext *sl, int in_setup) h->poc.prev_frame_num = h->poc.frame_num; } + if (!h->picture_idr && h->picture_intra_only) + h->poc_offset = 0; + if (avctx->hwaccel) { err = avctx->hwaccel->end_frame(avctx); if (err < 0) @@ -232,8 +238,13 @@ int ff_h264_field_end(H264Context *h, H264SliceContext *sl, int in_setup) "hardware accelerator failed to decode picture\n"); } + if ((sd = av_frame_get_side_data(cur->f, AV_FRAME_DATA_FILM_GRAIN_PARAMS))) { + AVFilmGrainParams *fgp = (AVFilmGrainParams *) sd->data; + fgp->seed = cur->poc + (h->poc_offset << 5); + } + if (!in_setup && !h->droppable) - ff_thread_report_progress(&h->cur_pic_ptr->tf, INT_MAX, + ff_thread_report_progress(&cur->tf, INT_MAX, h->picture_structure == PICT_BOTTOM_FIELD); emms_c(); diff --git a/libavcodec/h264_slice.c b/libavcodec/h264_slice.c index 0d7107d455..65d0259a71 100644 --- a/libavcodec/h264_slice.c +++ b/libavcodec/h264_slice.c @@ -406,6 +406,7 @@ int ff_h264_update_thread_context(AVCodecContext *dst, h->next_output_pic = h1->next_output_pic; h->next_outputed_poc = h1->next_outputed_poc; + h->poc_offset = h1->poc_offset; memcpy(h->mmco, h1->mmco, sizeof(h->mmco)); h->nb_mmco = h1->nb_mmco; @@ -510,6 +511,7 @@ static int h264_frame_start(H264Context *h) pic->sei_recovery_frame_cnt = h->sei.recovery_point.recovery_frame_cnt; pic->f->pict_type = h->slice_ctx[0].slice_type; + h->picture_intra_only = 1; // set to 0 by h264_slice_init pic->f->crop_left = h->crop_left; pic->f->crop_right = h->crop_right; @@ -1335,6 +1337,7 @@ static int h264_export_frame_props(H264Context *h) return AVERROR(ENOMEM); fgp->type = AV_FILM_GRAIN_PARAMS_H274; + /* fgp->seed is set by ff_h264_field_end */ fgp->codec.h274.model_id = fgc->model_id; if (fgc->separate_colour_description_present_flag) { @@ -1543,6 +1546,9 @@ static int h264_field_start(H264Context *h, const H264SliceContext *sl, h->poc.delta_poc[0] = sl->delta_poc[0]; h->poc.delta_poc[1] = sl->delta_poc[1]; + if (nal->type == H264_NAL_IDR_SLICE) + h->poc_offset = sl->idr_pic_id; + /* Shorten frame num gaps so we don't have to allocate reference * frames just to throw them away */ if (h->poc.frame_num != h->poc.prev_frame_num) { @@ -1891,7 +1897,7 @@ static int h264_slice_header_parse(const H264Context *h, H264SliceContext *sl, } if (nal->type == H264_NAL_IDR_SLICE) - get_ue_golomb_long(&sl->gb); /* idr_pic_id */ + sl->idr_pic_id = get_ue_golomb_long(&sl->gb); sl->poc_lsb = 0; sl->delta_poc_bottom = 0; @@ -2024,6 +2030,9 @@ static int h264_slice_init(H264Context *h, H264SliceContext *sl, return AVERROR_INVALIDDATA; } + if (sl->slice_type_nos != AV_PICTURE_TYPE_I) + h->picture_intra_only = 0; + av_assert1(h->mb_num == h->mb_width * h->mb_height); if (sl->first_mb_addr << FIELD_OR_MBAFF_PICTURE(h) >= h->mb_num || sl->first_mb_addr >= h->mb_num) { diff --git a/libavcodec/h264dec.h b/libavcodec/h264dec.h index 125966aa04..7c419de051 100644 --- a/libavcodec/h264dec.h +++ b/libavcodec/h264dec.h @@ -331,6 +331,7 @@ typedef struct H264SliceContext { int explicit_ref_marking; int frame_num; + int idr_pic_id; int poc_lsb; int delta_poc_bottom; int delta_poc[2]; @@ -384,6 +385,11 @@ typedef struct H264Context { */ int picture_idr; + /* + * Set to 1 when the current picture contains only I slices, 0 otherwise. + */ + int picture_intra_only; + int crop_left; int crop_right; int crop_top; @@ -473,6 +479,7 @@ typedef struct H264Context { int last_pocs[MAX_DELAYED_PIC_COUNT]; H264Picture *next_output_pic; int next_outputed_poc; + int poc_offset; ///< PicOrderCnt_offset from SMPTE RDD-2006 /** * memory management control operations buffer. diff --git a/libavutil/film_grain_params.h b/libavutil/film_grain_params.h index 7350dfc5b8..f3bd0a4a6a 100644 --- a/libavutil/film_grain_params.h +++ b/libavutil/film_grain_params.h @@ -221,6 +221,9 @@ typedef struct AVFilmGrainParams { /** * Seed to use for the synthesis process, if the codec allows for it. + * + * @note For H.264, this refers to `pic_offset` as defined in + * SMPTE RDD 5-2006. */ uint64_t seed;