From patchwork Fri Jul 9 03:32:26 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fei Wang X-Patchwork-Id: 28874 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a5d:965a:0:0:0:0:0 with SMTP id d26csp270696ios; Thu, 8 Jul 2021 20:36:01 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyWhsNdYO2PZetcT9fIVlxJ9MLusc83LbIJf35OUQ1+O1vja/91i3mMn8t95bdouuxwrqWz X-Received: by 2002:a05:6402:3514:: with SMTP id b20mr23873950edd.187.1625801760858; Thu, 08 Jul 2021 20:36:00 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1625801760; cv=none; d=google.com; s=arc-20160816; b=oE5Z7kt+8FI/3zAo0lIwco3x3G0qNl3lxOPwKXUHlxEctPy5RybAKQ20EPqae+lmtN p4zi0YhU4Y2bY0WnPQTzo2cZSDtXfUSgqU5rQeJWapOq3/3FK1SiVsVYP9uCTQ/yY9Ei ccNOsRrBCsYQKNgd5orwexpFtqdvnYdciNsuimu+RFyhOgPr+XJfF0ahHQrPH///cVUm unufPci/qwYuMrDKLRQfO+m0kmm56yrIJaZ6AITqRHeqs65SbXfQgbDSFV+MD4Go5Yxp owJklBnTpO3oSK1LcrGGIoDyDmgL1G2oNaqhRdwD2hBybpuwQKsrNV28lgqpFhH+6qAq 2f+w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:delivered-to; bh=Ab4gcMvZ8kTUKZfAWKIR/Fx1d0LjB1G6Zi/aYD+jwIU=; b=JUCzOBVGvjiUbWkclnwG6vdJ2ZW2IZSYR2TKERyk2CnpYF73twncFmvRVyy3WTsaAW CSsUId+BWEsxd2MzXtNg7o+lN0VvQUOlrOlmN/IHEO9+DZ4YGO6IrSuLWxldgr5NdVlQ gbX2oN758zRFyh2Lqdzgc3R2CSr+C97EK1RnADP5jtyJacIEBuzBEe0asVZlFrY3mfl/ 9Vcb2LItOEC2YcfIQLRTBTXapNyNEONeJuyEQ5zf109uTzSsXA0oIdb7MUKG5sBlVXU7 n4CvBvexKyWIeyaiTk4Z2gqNCbARspUus7elchTbJLBj5rGdW/yHknyW7TRz7kt+Zm7R L4+g== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id 4si4510170ejr.300.2021.07.08.20.36.00; Thu, 08 Jul 2021 20:36:00 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 7844968A5D0; Fri, 9 Jul 2021 06:35:13 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 187D268A569 for ; Fri, 9 Jul 2021 06:35:04 +0300 (EEST) X-IronPort-AV: E=McAfee;i="6200,9189,10039"; a="196814909" X-IronPort-AV: E=Sophos;i="5.84,225,1620716400"; d="scan'208";a="196814909" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by orsmga101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Jul 2021 20:34:56 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.84,225,1620716400"; d="scan'208";a="461973677" Received: from f.sh.intel.com ([10.239.159.147]) by fmsmga008.fm.intel.com with ESMTP; 08 Jul 2021 20:34:55 -0700 From: Fei Wang To: ffmpeg-devel@ffmpeg.org Date: Fri, 9 Jul 2021 11:32:26 +0800 Message-Id: <20210709033229.19125-6-fei.w.wang@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210709033229.19125-1-fei.w.wang@intel.com> References: <20210709033229.19125-1-fei.w.wang@intel.com> Subject: [FFmpeg-devel] [PATCH v4 6/9] avcodec/av1dec: add display frame for film grain usage X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Fei Wang MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: Un+KzGnoZVQE Make it flexible if decode frame(without apply film grain) and display frame(applied film grain) both needed when decode film grain clips. Signed-off-by: Fei Wang --- libavcodec/av1dec.c | 43 ++++++++++++++++++++++++++++++++++++++++++- libavcodec/av1dec.h | 3 +++ 2 files changed, 45 insertions(+), 1 deletion(-) diff --git a/libavcodec/av1dec.c b/libavcodec/av1dec.c index d808a9d4c6..6bba2cb7a2 100644 --- a/libavcodec/av1dec.c +++ b/libavcodec/av1dec.c @@ -570,6 +570,8 @@ static int get_pixel_format(AVCodecContext *avctx) static void av1_frame_unref(AVCodecContext *avctx, AV1Frame *f) { ff_thread_release_buffer(avctx, &f->tf); + if (f->tf_display.f->buf[0]) + ff_thread_release_buffer(avctx, &f->tf_display); av_buffer_unref(&f->hwaccel_priv_buf); f->hwaccel_picture_private = NULL; av_buffer_unref(&f->header_ref); @@ -589,6 +591,12 @@ static int av1_frame_ref(AVCodecContext *avctx, AV1Frame *dst, const AV1Frame *s if (ret < 0) return ret; + if (src->tf_display.f->buf[0]) { + ret = ff_thread_ref_frame(&dst->tf_display, &src->tf_display); + if (ret < 0) + return ret; + } + dst->header_ref = av_buffer_ref(src->header_ref); if (!dst->header_ref) goto fail; @@ -635,9 +643,11 @@ static av_cold int av1_decode_free(AVCodecContext *avctx) for (int i = 0; i < FF_ARRAY_ELEMS(s->ref); i++) { av1_frame_unref(avctx, &s->ref[i]); av_frame_free(&s->ref[i].tf.f); + av_frame_free(&s->ref[i].tf_display.f); } av1_frame_unref(avctx, &s->cur_frame); av_frame_free(&s->cur_frame.tf.f); + av_frame_free(&s->cur_frame.tf_display.f); av_buffer_unref(&s->seq_ref); av_buffer_unref(&s->header_ref); @@ -739,6 +749,13 @@ static av_cold int av1_decode_init(AVCodecContext *avctx) "Failed to allocate reference frame buffer %d.\n", i); return AVERROR(ENOMEM); } + + s->ref[i].tf_display.f = av_frame_alloc(); + if (!s->ref[i].tf_display.f) { + av_log(avctx, AV_LOG_ERROR, + "Failed to allocate display frame buffer %d.\n", i); + return AVERROR(ENOMEM); + } } s->cur_frame.tf.f = av_frame_alloc(); @@ -748,6 +765,13 @@ static av_cold int av1_decode_init(AVCodecContext *avctx) return AVERROR(ENOMEM); } + s->cur_frame.tf_display.f = av_frame_alloc(); + if (!s->cur_frame.tf_display.f) { + av_log(avctx, AV_LOG_ERROR, + "Failed to allocate current display frame buffer.\n"); + return AVERROR(ENOMEM); + } + ret = ff_cbs_init(&s->cbc, AV_CODEC_ID_AV1, avctx); if (ret < 0) return ret; @@ -820,6 +844,16 @@ static int av1_frame_alloc(AVCodecContext *avctx, AV1Frame *f) break; } + if (header->film_grain.apply_grain && + (avctx->pix_fmt == AV_PIX_FMT_VAAPI) && + !(avctx->export_side_data & AV_CODEC_EXPORT_DATA_FILM_GRAIN)) { + if ((ret = ff_thread_get_buffer(avctx, &f->tf_display, AV_GET_BUFFER_FLAG_REF)) < 0) + goto fail; + + f->tf_display.f->key_frame = frame->key_frame; + f->tf_display.f->pict_type = frame->pict_type; + } + if (avctx->hwaccel) { const AVHWAccel *hwaccel = avctx->hwaccel; if (hwaccel->frame_priv_data_size) { @@ -903,9 +937,16 @@ static int set_output_frame(AVCodecContext *avctx, AVFrame *frame, const AVPacket *pkt, int *got_frame) { AV1DecContext *s = avctx->priv_data; - const AVFrame *srcframe = s->cur_frame.tf.f; + const AVFrame *srcframe; int ret; + /* Use tf_display as output when it is available. */ + if (s->cur_frame.tf_display.f->buf[0]) { + srcframe = s->cur_frame.tf_display.f; + } else { + srcframe= s->cur_frame.tf.f; + } + // TODO: all layers if (s->operating_point_idc && av_log2(s->operating_point_idc >> 8) > s->cur_frame.spatial_id) diff --git a/libavcodec/av1dec.h b/libavcodec/av1dec.h index 4e140588b9..5af3dfc867 100644 --- a/libavcodec/av1dec.h +++ b/libavcodec/av1dec.h @@ -33,6 +33,9 @@ typedef struct AV1Frame { ThreadFrame tf; + /** current tf_display is only used for VA-API to apply film grain */ + ThreadFrame tf_display; + AVBufferRef *hwaccel_priv_buf; void *hwaccel_picture_private;