From patchwork Wed Aug 4 17:00:48 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Niklas Haas X-Patchwork-Id: 29249 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a6b:6c0f:0:0:0:0:0 with SMTP id a15csp3546067ioh; Wed, 4 Aug 2021 10:01:03 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwFVxiOpjdGDyUG6+ovuLuvGGTzWSS4Lg3g1aKQBsWI+Y0IoBA/8uZFExCGjp2cl/ejlsKJ X-Received: by 2002:a17:906:8cd:: with SMTP id o13mr161109eje.341.1628096463123; Wed, 04 Aug 2021 10:01:03 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1628096463; cv=none; d=google.com; s=arc-20160816; b=md7qVkPJwLwD8Cb8YbjZykPTR3zhPyfyA/VOVsrTa+kaxHzIZzM4LLFlj29sBtKawj ipqi7/rpv7jkAyS5sApkm5/5ERBOfe999VMRla2ylPwHKVvSp5IJsyWKOzxgZ/fOfunF Y18vUEUWrxNN53dRzaLQPqBVy2fa/aVKhPh9jvlGG7n3NpFFvyTGjzAdN2LHin1G3OlW 7XOJYMItiTwGHXFnMXIotipWVnqSV2ZV7o8aTOBGgAzWzNwb3Ff9nSM3lk9mMvrCkjoN FXY22Ot6IINSgrqkP1I/krytvD+/PSGEL7JT3kW8dCJf0YI9TW5KfNYTGkPlltoCNHY6 TZdQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:mime-version:message-id:date:to:from :dkim-signature:delivered-to; bh=rY39XOtrzuyOdeSjYcic6r4nY0BAjriziF4DHchtTvk=; b=W2DchWlxLdSSxhl6qMhIUSYZoPfLHVGF8HvMBO9hF6Iz0HJRIBoGZQQ5dW7MueFBai CS67DwXtGIf/d4HB5eSofzuwJ9+iXBzOirfPnfR3Ca4ML0sf7l4+Cu91a5DNI0oyNz3Y 8+EaziWlPc8nbnjbEjB5lAf9zTgwXWvG9CeK3JTKTOBmHVM/i6NgezC14cL/hS4HFylR bAM1CLLxdJ30/Q8gQiVSAep4kx4kjBPTnaGu3FDLIvvsWE7fqqUPVbOSgUAP7UaNSU2r oPlNbemcax8uwohtM/O2A/vxGcYWurJQGKJZhnb5QkpMwWljiOogM27Pmtco6ue4Qpt3 aq8A== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@haasn.xyz header.s=mail header.b=DF3TOdNP; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id f19si2658111eds.480.2021.08.04.10.01.02; Wed, 04 Aug 2021 10:01:03 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@haasn.xyz header.s=mail header.b=DF3TOdNP; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id B6F3F687EBB; Wed, 4 Aug 2021 20:00:59 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from haasn.dev (haasn.dev [78.46.187.166]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 743C8687EBB for ; Wed, 4 Aug 2021 20:00:53 +0300 (EEST) Received: from haasn.dev (unknown [10.30.0.2]) by haasn.dev (Postfix) with ESMTP id F103A47426; Wed, 4 Aug 2021 19:00:52 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=haasn.xyz; s=mail; t=1628096453; bh=HGHxA2QjXUyUSqnd9Zs2M2CTrJedwPuB1oUE30nPR+I=; h=From:To:Cc:Subject:Date:From; b=DF3TOdNPmJ93RjDHlR33OzmeFxNPEoa081sgfgeKuRg8xiZ6vcvUUTmDhxk4hc2aA wgrzBORzOceBfHSLdwk0dZydmqgZrPCtzgQt9vTmIhKfqe8/ANoSA7TKVBxc9fiEOA XM5XyCbqUARdpBFEJq5Wqr3HdzGsmblnLO+3pwkU= From: Niklas Haas To: ffmpeg-devel@ffmpeg.org Date: Wed, 4 Aug 2021 19:00:48 +0200 Message-Id: <20210804170048.129779-1-ffmpeg@haasn.xyz> X-Mailer: git-send-email 2.32.0 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH] avcodec/h264_slice: compute and export film grain seed X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Niklas Haas Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: 5PxRiiaCvrvx From: Niklas Haas From SMPTE RDD 5-2006, the grain seed is to be computed from the following definition of `pic_offset`: > When decoding H.264 | MPEG-4 AVC bitstreams, pic_offset is defined as > follows: > - pic_offset = PicOrderCnt(CurrPic) + (PicOrderCnt_offset << 5) > where: > - PicOrderCnt(CurrPic) is the picture order count of the current frame, > which shall be derived from [the video stream]. > > - PicOrderCnt_offset is set to idr_pic_id on IDR frames. idr_pic_id > shall be read from the slice header of [the video stream]. On non-IDR I > frames, PicOrderCnt_offset is set to 0. A frame shall be classified as I > frame when all its slices are I slices, which may be optionally > designated by setting primary_pic_type to 0 in the access delimiter NAL > unit. Otherwise, PicOrderCnt_offset it not changed. PicOrderCnt_offset is > updated in decoding order. To forward this information correctly, we need to make a number of annoying changes to h264_slice: - We need to keep track of the `PicOrderCnt_offset`, and update it to the decoded value of `idr_pic_id` on IDR frames. - We need to keep track of whether or not a frame contained any non-I slices, so we can detect the "non-IDR I frame" condition, and reset `PicOrderCnt_offset` back to 0. This is *not* synonymous with existing fields such as `pic->f->pict_type` because that is somehow only set based on the first slice in a frame, so we introduce a new field `picture_intra_only` which is set to 0 when decoding any non-I slice. - We need to compute this derived `pic_offset` at the *end* of the frame, rather than at the beginning (where the SEI is decoded/set), because it now depends on the types of the slices present in the same frame. If there's a less ugly way to accomplish the above, I don't see it. Suggestions welcome(tm). Signed-off-by: Niklas Haas --- libavcodec/h264_picture.c | 13 ++++++++++++- libavcodec/h264_slice.c | 11 ++++++++++- libavcodec/h264dec.h | 7 +++++++ libavutil/film_grain_params.h | 3 +++ 4 files changed, 32 insertions(+), 2 deletions(-) diff --git a/libavcodec/h264_picture.c b/libavcodec/h264_picture.c index eec5e9fb9a..18daf423b5 100644 --- a/libavcodec/h264_picture.c +++ b/libavcodec/h264_picture.c @@ -27,6 +27,7 @@ #include "libavutil/avassert.h" #include "libavutil/imgutils.h" +#include "libavutil/film_grain_params.h" #include "internal.h" #include "cabac.h" #include "cabac_functions.h" @@ -159,6 +160,8 @@ void ff_h264_set_erpic(ERPicture *dst, H264Picture *src) int ff_h264_field_end(H264Context *h, H264SliceContext *sl, int in_setup) { AVCodecContext *const avctx = h->avctx; + H264Picture *cur = h->cur_pic_ptr; + AVFrameSideData *sd; int err = 0; h->mb_y = 0; @@ -172,6 +175,9 @@ int ff_h264_field_end(H264Context *h, H264SliceContext *sl, int in_setup) h->poc.prev_frame_num = h->poc.frame_num; } + if (!h->picture_idr && h->picture_intra_only) + h->poc_offset = 0; + if (avctx->hwaccel) { err = avctx->hwaccel->end_frame(avctx); if (err < 0) @@ -179,8 +185,13 @@ int ff_h264_field_end(H264Context *h, H264SliceContext *sl, int in_setup) "hardware accelerator failed to decode picture\n"); } + if ((sd = av_frame_get_side_data(cur->f, AV_FRAME_DATA_FILM_GRAIN_PARAMS))) { + AVFilmGrainParams *fgp = (AVFilmGrainParams *) sd->data; + fgp->seed = cur->poc + h->poc_offset << 5; + } + if (!in_setup && !h->droppable) - ff_thread_report_progress(&h->cur_pic_ptr->tf, INT_MAX, + ff_thread_report_progress(&cur->tf, INT_MAX, h->picture_structure == PICT_BOTTOM_FIELD); emms_c(); diff --git a/libavcodec/h264_slice.c b/libavcodec/h264_slice.c index 41338fbcb6..19283fb4ec 100644 --- a/libavcodec/h264_slice.c +++ b/libavcodec/h264_slice.c @@ -410,6 +410,7 @@ int ff_h264_update_thread_context(AVCodecContext *dst, h->next_output_pic = h1->next_output_pic; h->next_outputed_poc = h1->next_outputed_poc; + h->poc_offset = h1->poc_offset; memcpy(h->mmco, h1->mmco, sizeof(h->mmco)); h->nb_mmco = h1->nb_mmco; @@ -514,6 +515,7 @@ static int h264_frame_start(H264Context *h) pic->sei_recovery_frame_cnt = h->sei.recovery_point.recovery_frame_cnt; pic->f->pict_type = h->slice_ctx[0].slice_type; + h->picture_intra_only = 1; // set to 0 by h264_slice_init pic->f->crop_left = h->crop_left; pic->f->crop_right = h->crop_right; @@ -1339,6 +1341,7 @@ static int h264_export_frame_props(H264Context *h) return AVERROR(ENOMEM); fgp->type = AV_FILM_GRAIN_PARAMS_H274; + /* fgp->seed is set by ff_h264_field_end */ fgp->codec.h274.model_id = fgc->model_id; if (fgc->separate_colour_description_present_flag) { @@ -1547,6 +1550,9 @@ static int h264_field_start(H264Context *h, const H264SliceContext *sl, h->poc.delta_poc[0] = sl->delta_poc[0]; h->poc.delta_poc[1] = sl->delta_poc[1]; + if (nal->type == H264_NAL_IDR_SLICE) + h->poc_offset = sl->idr_pic_id; + /* Shorten frame num gaps so we don't have to allocate reference * frames just to throw them away */ if (h->poc.frame_num != h->poc.prev_frame_num) { @@ -1895,7 +1901,7 @@ static int h264_slice_header_parse(const H264Context *h, H264SliceContext *sl, } if (nal->type == H264_NAL_IDR_SLICE) - get_ue_golomb_long(&sl->gb); /* idr_pic_id */ + sl->idr_pic_id = get_ue_golomb_long(&sl->gb); if (sps->poc_type == 0) { sl->poc_lsb = get_bits(&sl->gb, sps->log2_max_poc_lsb); @@ -2025,6 +2031,9 @@ static int h264_slice_init(H264Context *h, H264SliceContext *sl, return AVERROR_INVALIDDATA; } + if (sl->slice_type_nos != AV_PICTURE_TYPE_I) + h->picture_intra_only = 0; + av_assert1(h->mb_num == h->mb_width * h->mb_height); if (sl->first_mb_addr << FIELD_OR_MBAFF_PICTURE(h) >= h->mb_num || sl->first_mb_addr >= h->mb_num) { diff --git a/libavcodec/h264dec.h b/libavcodec/h264dec.h index 8954b74795..3a1dc5d239 100644 --- a/libavcodec/h264dec.h +++ b/libavcodec/h264dec.h @@ -331,6 +331,7 @@ typedef struct H264SliceContext { int explicit_ref_marking; int frame_num; + int idr_pic_id; int poc_lsb; int delta_poc_bottom; int delta_poc[2]; @@ -384,6 +385,11 @@ typedef struct H264Context { */ int picture_idr; + /* + * Set to 1 when the current picture contains only I slices, 0 otherwise. + */ + int picture_intra_only; + int crop_left; int crop_right; int crop_top; @@ -473,6 +479,7 @@ typedef struct H264Context { int last_pocs[MAX_DELAYED_PIC_COUNT]; H264Picture *next_output_pic; int next_outputed_poc; + int poc_offset; ///< PicOrderCnt_offset from SMPTE RDD-2006 /** * memory management control operations buffer. diff --git a/libavutil/film_grain_params.h b/libavutil/film_grain_params.h index 7350dfc5b8..f3bd0a4a6a 100644 --- a/libavutil/film_grain_params.h +++ b/libavutil/film_grain_params.h @@ -221,6 +221,9 @@ typedef struct AVFilmGrainParams { /** * Seed to use for the synthesis process, if the codec allows for it. + * + * @note For H.264, this refers to `pic_offset` as defined in + * SMPTE RDD 5-2006. */ uint64_t seed;