From patchwork Mon Jul 5 02:29:30 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fei Wang X-Patchwork-Id: 28763 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a5d:965a:0:0:0:0:0 with SMTP id d26csp3736590ios; Sun, 4 Jul 2021 19:32:44 -0700 (PDT) X-Google-Smtp-Source: ABdhPJz4k8sPSJ34RaGgSWumYmDkZAR+9nqyM2Tc20cqrEl0axmrsgew1HslcjkQrCARJblP02mK X-Received: by 2002:a17:906:b298:: with SMTP id q24mr10807630ejz.69.1625452364733; Sun, 04 Jul 2021 19:32:44 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1625452364; cv=none; d=google.com; s=arc-20160816; b=pl2GyEqzwj3AoX8VJO1IMabeGQEKua7KA6ggMxqLXw78pYJ5Eydn+cHn1AeQRgiZZS mkT+9Xs5GlkzDd1QIhAMCK/WpOB+yRBVAiT9+VB98WXugcIkSYqxl79qvDhuU5WtLi2U dD/Dha9pdoGzoxjwCh8cx3N0gihqM25RToS3Voct2w1ZzvljSeEcNekK2VldevnLW+gq nZ4rMJHwEASsSh4pZOQgENMtbBXL26LeiZA76fbr5HxCCN1Yxal+0r6gZF9QrAuQ3ki6 G0XnCkEixwEivS3oNzEwQmYcc6e/rjb1PF8CQq/LOHykn677cajrjKrUjkYsIVFlpnt9 g5OA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:delivered-to; bh=pswVKK2ik6I5+4gyjrSXM3LXnGxIybk9plEgtn2R+Qs=; b=ccenaSOVriWwoqjXiwQcIsOXPcJqzSwnrMe0XQi2NAlAl7sAAHyvjjb/MeThBlQf9v OtHN0wj90YXAtbQPRyg8fJogqDeuKdl6QjOmCTIKGZUjW1zUaSz9tkwE63+8fHltS+V3 3JKTch0G63JOaT97tBbjx/8X4KgW+4p6Y74Bgqg3FwBj0ECv2Y4y4wLsxu5U/7S6+Kp/ p/vs4wZLw8nfE4Ph+oCe9erU8a8ytCJzCHgOKmtXa6UaZmBlvr8LZvq5gwjlEyJZPUQB 2cFP1yBh7Y7LWKP9Qbjlx3F9UQRE9QdXpRAWdf4JZZBi3QjFrYe6ckdQvvAJyxR3o9Lh nl1A== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id w6si10015123edd.335.2021.07.04.19.32.44; Sun, 04 Jul 2021 19:32:44 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id A373A68A5DE; Mon, 5 Jul 2021 05:31:54 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id C607168A642 for ; Mon, 5 Jul 2021 05:31:46 +0300 (EEST) X-IronPort-AV: E=McAfee;i="6200,9189,10035"; a="208869458" X-IronPort-AV: E=Sophos;i="5.83,325,1616482800"; d="scan'208";a="208869458" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Jul 2021 19:31:41 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.83,325,1616482800"; d="scan'208";a="456571458" Received: from f.sh.intel.com ([10.239.159.147]) by orsmga008.jf.intel.com with ESMTP; 04 Jul 2021 19:31:40 -0700 From: Fei Wang To: ffmpeg-devel@ffmpeg.org Date: Mon, 5 Jul 2021 10:29:30 +0800 Message-Id: <20210705022933.29068-6-fei.w.wang@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210705022933.29068-1-fei.w.wang@intel.com> References: <20210705022933.29068-1-fei.w.wang@intel.com> Subject: [FFmpeg-devel] [PATCH v2 6/9] avcodec/av1dec: add display frame for film grain usage X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Fei Wang MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: fRAfP4JAyTyr Make it flexible if decode frame(without apply film grain) and display frame(applied film grain) both needed when decode film grain clips. Signed-off-by: Fei Wang --- libavcodec/av1dec.c | 43 ++++++++++++++++++++++++++++++++++++++++++- libavcodec/av1dec.h | 3 +++ 2 files changed, 45 insertions(+), 1 deletion(-) diff --git a/libavcodec/av1dec.c b/libavcodec/av1dec.c index 3fca17e84b..af2ecb4f0c 100644 --- a/libavcodec/av1dec.c +++ b/libavcodec/av1dec.c @@ -569,6 +569,8 @@ static int get_pixel_format(AVCodecContext *avctx) static void av1_frame_unref(AVCodecContext *avctx, AV1Frame *f) { ff_thread_release_buffer(avctx, &f->tf); + if (f->tf_display.f->buf[0]) + ff_thread_release_buffer(avctx, &f->tf_display); av_buffer_unref(&f->hwaccel_priv_buf); f->hwaccel_picture_private = NULL; av_buffer_unref(&f->header_ref); @@ -588,6 +590,12 @@ static int av1_frame_ref(AVCodecContext *avctx, AV1Frame *dst, const AV1Frame *s if (ret < 0) return ret; + if (src->tf_display.f->buf[0]) { + ret = ff_thread_ref_frame(&dst->tf_display, &src->tf_display); + if (ret < 0) + return ret; + } + dst->header_ref = av_buffer_ref(src->header_ref); if (!dst->header_ref) goto fail; @@ -634,9 +642,11 @@ static av_cold int av1_decode_free(AVCodecContext *avctx) for (int i = 0; i < FF_ARRAY_ELEMS(s->ref); i++) { av1_frame_unref(avctx, &s->ref[i]); av_frame_free(&s->ref[i].tf.f); + av_frame_free(&s->ref[i].tf_display.f); } av1_frame_unref(avctx, &s->cur_frame); av_frame_free(&s->cur_frame.tf.f); + av_frame_free(&s->cur_frame.tf_display.f); av_buffer_unref(&s->seq_ref); av_buffer_unref(&s->header_ref); @@ -738,6 +748,13 @@ static av_cold int av1_decode_init(AVCodecContext *avctx) "Failed to allocate reference frame buffer %d.\n", i); return AVERROR(ENOMEM); } + + s->ref[i].tf_display.f = av_frame_alloc(); + if (!s->ref[i].tf_display.f) { + av_log(avctx, AV_LOG_ERROR, + "Failed to allocate display frame buffer %d.\n", i); + return AVERROR(ENOMEM); + } } s->cur_frame.tf.f = av_frame_alloc(); @@ -747,6 +764,13 @@ static av_cold int av1_decode_init(AVCodecContext *avctx) return AVERROR(ENOMEM); } + s->cur_frame.tf_display.f = av_frame_alloc(); + if (!s->cur_frame.tf_display.f) { + av_log(avctx, AV_LOG_ERROR, + "Failed to allocate current display frame buffer.\n"); + return AVERROR(ENOMEM); + } + ret = ff_cbs_init(&s->cbc, AV_CODEC_ID_AV1, avctx); if (ret < 0) return ret; @@ -819,6 +843,16 @@ static int av1_frame_alloc(AVCodecContext *avctx, AV1Frame *f) break; } + if (header->film_grain.apply_grain && + (avctx->pix_fmt == AV_PIX_FMT_VAAPI) && + !(avctx->export_side_data & AV_CODEC_EXPORT_DATA_FILM_GRAIN)) { + if ((ret = ff_thread_get_buffer(avctx, &f->tf_display, AV_GET_BUFFER_FLAG_REF)) < 0) + goto fail; + + f->tf_display.f->key_frame = frame->key_frame; + f->tf_display.f->pict_type = frame->pict_type; + } + if (avctx->hwaccel) { const AVHWAccel *hwaccel = avctx->hwaccel; if (hwaccel->frame_priv_data_size) { @@ -902,9 +936,16 @@ static int set_output_frame(AVCodecContext *avctx, AVFrame *frame, const AVPacket *pkt, int *got_frame) { AV1DecContext *s = avctx->priv_data; - const AVFrame *srcframe = s->cur_frame.tf.f; + const AVFrame *srcframe; int ret; + /* Use tf_display as output when it is available. */ + if (s->cur_frame.tf_display.f->buf[0]) { + srcframe = s->cur_frame.tf_display.f; + } else { + srcframe= s->cur_frame.tf.f; + } + // TODO: all layers if (s->operating_point_idc && av_log2(s->operating_point_idc >> 8) > s->cur_frame.spatial_id) diff --git a/libavcodec/av1dec.h b/libavcodec/av1dec.h index 4e140588b9..5af3dfc867 100644 --- a/libavcodec/av1dec.h +++ b/libavcodec/av1dec.h @@ -33,6 +33,9 @@ typedef struct AV1Frame { ThreadFrame tf; + /** current tf_display is only used for VA-API to apply film grain */ + ThreadFrame tf_display; + AVBufferRef *hwaccel_priv_buf; void *hwaccel_picture_private;