From patchwork Mon Jul 5 08:26:40 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fei Wang X-Patchwork-Id: 28776 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a5d:965a:0:0:0:0:0 with SMTP id d26csp3945176ios; Mon, 5 Jul 2021 01:29:57 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxG8Wd0zt/6s/LJDZtk15WhMs1hx6o3EZQOJ4SXj35x/uYOQqGM9DwY0NwhFURJ4Xv3h/K0 X-Received: by 2002:a17:907:397:: with SMTP id ss23mr6480358ejb.120.1625473797727; Mon, 05 Jul 2021 01:29:57 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1625473797; cv=none; d=google.com; s=arc-20160816; b=nlOVS1US6t77N4C2+fdv3SWXxa989YYEde3ns4qkbkolJszwb039QrsHfwQWqSRCu0 JU4Ml+lHK5isqaaGFeaIaw1zSMHFs+8OWdAQOCD+V3D6mhScHEWY5Bs7+e8UssRBoicv SuGCMWPqGjDxVhUAkT4dndcCzSPT73cNwya6uCiN3bsAeaJ3DJR849TwZgfTPLOrVcOe ViWjVcIpTG20TXorGsxzHMD1+B3tjlnQSSa8K+myUdKWsKUcRLXukz729wZh6itsF9x+ CaXxcKs4KXeRE3JAz1k+nXHANtFQaxvRw88uFW8XBXgxnUYYdG/iMr7Jd8JZ+qXdF+HL KAMg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:delivered-to; bh=WiEy5wlN2MCegx/Vp31TB50wCFUrKfRfvMmJPZT+sb0=; b=J93z566S0QqOUTSYa2+IjWhXGnJIPk3fgUgPrnzzBjH5oWjwUN561W51XeUivanSeA 4x7g8BF+NPlaNeaw0ck1aB10vYfK+3XktDUCLbyya7z6LTB6cD4nPBcew2foaz6kXSBk fXcatixwJQ0T0ZphofXNQ+MtYRO7B9vZeed58B+XFxO+h1D20oTZxSSQuVLT7IIk2ysc 4z+lc/3CueYsA7+5evhHhHksZxrnNn2JysiQwEsn5wgaNHBmHQ4SzNYhVBHbhWDplGb0 Dp4OO8hd7fy5oRlX+L7r6SCeGSfx3DI/y2pr3FdZCGBSf3UgXEH/P86n7dIExV4AT6ca sucw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id dt16si10919936ejc.556.2021.07.05.01.29.57; Mon, 05 Jul 2021 01:29:57 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id ED1EE68A6F6; Mon, 5 Jul 2021 11:29:05 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id B2B3168A6CC for ; Mon, 5 Jul 2021 11:28:58 +0300 (EEST) X-IronPort-AV: E=McAfee;i="6200,9189,10035"; a="230662732" X-IronPort-AV: E=Sophos;i="5.83,325,1616482800"; d="scan'208";a="230662732" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Jul 2021 01:28:50 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.83,325,1616482800"; d="scan'208";a="496045713" Received: from f.sh.intel.com ([10.239.159.147]) by fmsmga002.fm.intel.com with ESMTP; 05 Jul 2021 01:28:49 -0700 From: Fei Wang To: ffmpeg-devel@ffmpeg.org Date: Mon, 5 Jul 2021 16:26:40 +0800 Message-Id: <20210705082643.28259-6-fei.w.wang@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210705082643.28259-1-fei.w.wang@intel.com> References: <20210705082643.28259-1-fei.w.wang@intel.com> Subject: [FFmpeg-devel] [PATCH v3 6/9] avcodec/av1dec: add display frame for film grain usage X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Fei Wang MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: sP2J78Crg+FR Make it flexible if decode frame(without apply film grain) and display frame(applied film grain) both needed when decode film grain clips. Signed-off-by: Fei Wang --- libavcodec/av1dec.c | 43 ++++++++++++++++++++++++++++++++++++++++++- libavcodec/av1dec.h | 3 +++ 2 files changed, 45 insertions(+), 1 deletion(-) diff --git a/libavcodec/av1dec.c b/libavcodec/av1dec.c index 9cf25e3409..10ee6c6084 100644 --- a/libavcodec/av1dec.c +++ b/libavcodec/av1dec.c @@ -570,6 +570,8 @@ static int get_pixel_format(AVCodecContext *avctx) static void av1_frame_unref(AVCodecContext *avctx, AV1Frame *f) { ff_thread_release_buffer(avctx, &f->tf); + if (f->tf_display.f->buf[0]) + ff_thread_release_buffer(avctx, &f->tf_display); av_buffer_unref(&f->hwaccel_priv_buf); f->hwaccel_picture_private = NULL; av_buffer_unref(&f->header_ref); @@ -589,6 +591,12 @@ static int av1_frame_ref(AVCodecContext *avctx, AV1Frame *dst, const AV1Frame *s if (ret < 0) return ret; + if (src->tf_display.f->buf[0]) { + ret = ff_thread_ref_frame(&dst->tf_display, &src->tf_display); + if (ret < 0) + return ret; + } + dst->header_ref = av_buffer_ref(src->header_ref); if (!dst->header_ref) goto fail; @@ -635,9 +643,11 @@ static av_cold int av1_decode_free(AVCodecContext *avctx) for (int i = 0; i < FF_ARRAY_ELEMS(s->ref); i++) { av1_frame_unref(avctx, &s->ref[i]); av_frame_free(&s->ref[i].tf.f); + av_frame_free(&s->ref[i].tf_display.f); } av1_frame_unref(avctx, &s->cur_frame); av_frame_free(&s->cur_frame.tf.f); + av_frame_free(&s->cur_frame.tf_display.f); av_buffer_unref(&s->seq_ref); av_buffer_unref(&s->header_ref); @@ -739,6 +749,13 @@ static av_cold int av1_decode_init(AVCodecContext *avctx) "Failed to allocate reference frame buffer %d.\n", i); return AVERROR(ENOMEM); } + + s->ref[i].tf_display.f = av_frame_alloc(); + if (!s->ref[i].tf_display.f) { + av_log(avctx, AV_LOG_ERROR, + "Failed to allocate display frame buffer %d.\n", i); + return AVERROR(ENOMEM); + } } s->cur_frame.tf.f = av_frame_alloc(); @@ -748,6 +765,13 @@ static av_cold int av1_decode_init(AVCodecContext *avctx) return AVERROR(ENOMEM); } + s->cur_frame.tf_display.f = av_frame_alloc(); + if (!s->cur_frame.tf_display.f) { + av_log(avctx, AV_LOG_ERROR, + "Failed to allocate current display frame buffer.\n"); + return AVERROR(ENOMEM); + } + ret = ff_cbs_init(&s->cbc, AV_CODEC_ID_AV1, avctx); if (ret < 0) return ret; @@ -820,6 +844,16 @@ static int av1_frame_alloc(AVCodecContext *avctx, AV1Frame *f) break; } + if (header->film_grain.apply_grain && + (avctx->pix_fmt == AV_PIX_FMT_VAAPI) && + !(avctx->export_side_data & AV_CODEC_EXPORT_DATA_FILM_GRAIN)) { + if ((ret = ff_thread_get_buffer(avctx, &f->tf_display, AV_GET_BUFFER_FLAG_REF)) < 0) + goto fail; + + f->tf_display.f->key_frame = frame->key_frame; + f->tf_display.f->pict_type = frame->pict_type; + } + if (avctx->hwaccel) { const AVHWAccel *hwaccel = avctx->hwaccel; if (hwaccel->frame_priv_data_size) { @@ -903,9 +937,16 @@ static int set_output_frame(AVCodecContext *avctx, AVFrame *frame, const AVPacket *pkt, int *got_frame) { AV1DecContext *s = avctx->priv_data; - const AVFrame *srcframe = s->cur_frame.tf.f; + const AVFrame *srcframe; int ret; + /* Use tf_display as output when it is available. */ + if (s->cur_frame.tf_display.f->buf[0]) { + srcframe = s->cur_frame.tf_display.f; + } else { + srcframe= s->cur_frame.tf.f; + } + // TODO: all layers if (s->operating_point_idc && av_log2(s->operating_point_idc >> 8) > s->cur_frame.spatial_id) diff --git a/libavcodec/av1dec.h b/libavcodec/av1dec.h index 4e140588b9..5af3dfc867 100644 --- a/libavcodec/av1dec.h +++ b/libavcodec/av1dec.h @@ -33,6 +33,9 @@ typedef struct AV1Frame { ThreadFrame tf; + /** current tf_display is only used for VA-API to apply film grain */ + ThreadFrame tf_display; + AVBufferRef *hwaccel_priv_buf; void *hwaccel_picture_private;