From patchwork Thu Jun 17 06:10:34 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fei Wang X-Patchwork-Id: 28563 Delivered-To: ffmpegpatchwork2@gmail.com Received: by 2002:a5e:c91a:0:0:0:0:0 with SMTP id z26csp330599iol; Wed, 16 Jun 2021 23:13:42 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyXQ3YncVThQqwUy+ZBaSWt/yOdmCcOtveX14QgXZ8CygwyLNU/lUq5HdTbw06h4YrqbxEQ X-Received: by 2002:a17:906:c0c7:: with SMTP id bn7mr3332785ejb.393.1623910422419; Wed, 16 Jun 2021 23:13:42 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1623910422; cv=none; d=google.com; s=arc-20160816; b=Mh34p83wV7c8zlS6LwbnIcEagHhT8+w9Da/BbbCKc5uUIiDocbuNLL35nCPGhIfB9+ w3XDE9YrSjAxxcW9+8Jm4SjXMCZGscSePrEjUdol5gNG5jEyGWxRPWCYJfrnCqjrxgJz hxYch+6A8LVFWPjdIodtMz+1ktor5Q/WWG0JZZDUTxz0X9aFf0ozz4p8YBo5JBQBFex1 hoSQFOYZmiCeX78H9HmH5WtRvFEy5TKAZoduoFO7DCHdBdP+1CN1U82zzYEeTLuMFcMe oGkHnshX1qwKVbqmcaS9H25xMc3Ijd/BSBgb/hXieACWo4dSuizRhAu8nlV17aYAJT+Y GVtQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc:reply-to :list-subscribe:list-help:list-post:list-archive:list-unsubscribe :list-id:precedence:subject:references:in-reply-to:message-id:date :to:from:ironport-sdr:ironport-sdr:delivered-to; bh=pswVKK2ik6I5+4gyjrSXM3LXnGxIybk9plEgtn2R+Qs=; b=rd5Aqp5i75F7z3rRWTbsZx3h5Gwmf671TRO48gA0OzIQr6tDjvJzzCynjtoPg7e3wP N1y6q/zY+KbCCw4FFxVsu9H4NSKuWvPIi+7OPMTJYH8Czdr+/nEi1+5PV9uLNigUI4ql VTgRldTy/MDax1m4+R/KOPgc+GJ7D2TN4WGkCOhaROZq7rU2CL0P/Y508sSugCaTRJKy /083UcjlNl6oFmAaVricYLrP5d+OGOKPtKByb57s0GV/qpOXUhYqm4I8WIjU/hHg8L/T ZIu63ZjDTo+J6ClPApELemm9epSACD184vdJ5+wqpAedPMk4OgwGYsFgZwBJ+Zk/wuvV MD/g== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id v7si5176649edj.318.2021.06.16.23.13.41; Wed, 16 Jun 2021 23:13:42 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 589326899A2; Thu, 17 Jun 2021 09:12:29 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 81416688081 for ; Thu, 17 Jun 2021 09:12:18 +0300 (EEST) IronPort-SDR: 5clAT4W11yId/in+xnMkccW+m0Ky+iCawKSM4QLRErLvpeInOqqgbOiL6CKmM7WUoJTrBtLxWB iAuF9Z6+iOMQ== X-IronPort-AV: E=McAfee;i="6200,9189,10017"; a="291942566" X-IronPort-AV: E=Sophos;i="5.83,278,1616482800"; d="scan'208";a="291942566" Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 16 Jun 2021 23:12:12 -0700 IronPort-SDR: E5cszs8NT/sC7hvG/J3HsN/MPYSzOTHu5ZwMs5e51+q8XM8Gw8zYjLlLORl4kS3XpE7E1ZcqPS fG2xwkiWJoNw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.83,278,1616482800"; d="scan'208";a="485177031" Received: from f.sh.intel.com ([10.239.159.147]) by orsmga001.jf.intel.com with ESMTP; 16 Jun 2021 23:12:11 -0700 From: Fei Wang To: ffmpeg-devel@ffmpeg.org Date: Thu, 17 Jun 2021 14:10:34 +0800 Message-Id: <20210617061037.20335-6-fei.w.wang@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210617061037.20335-1-fei.w.wang@intel.com> References: <20210617061037.20335-1-fei.w.wang@intel.com> Subject: [FFmpeg-devel] [PATCH 6/9] avcodec/av1dec: add display frame for film grain usage X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: Fei Wang MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" X-TUID: FcX/SVdafeEk Make it flexible if decode frame(without apply film grain) and display frame(applied film grain) both needed when decode film grain clips. Signed-off-by: Fei Wang --- libavcodec/av1dec.c | 43 ++++++++++++++++++++++++++++++++++++++++++- libavcodec/av1dec.h | 3 +++ 2 files changed, 45 insertions(+), 1 deletion(-) diff --git a/libavcodec/av1dec.c b/libavcodec/av1dec.c index 3fca17e84b..af2ecb4f0c 100644 --- a/libavcodec/av1dec.c +++ b/libavcodec/av1dec.c @@ -569,6 +569,8 @@ static int get_pixel_format(AVCodecContext *avctx) static void av1_frame_unref(AVCodecContext *avctx, AV1Frame *f) { ff_thread_release_buffer(avctx, &f->tf); + if (f->tf_display.f->buf[0]) + ff_thread_release_buffer(avctx, &f->tf_display); av_buffer_unref(&f->hwaccel_priv_buf); f->hwaccel_picture_private = NULL; av_buffer_unref(&f->header_ref); @@ -588,6 +590,12 @@ static int av1_frame_ref(AVCodecContext *avctx, AV1Frame *dst, const AV1Frame *s if (ret < 0) return ret; + if (src->tf_display.f->buf[0]) { + ret = ff_thread_ref_frame(&dst->tf_display, &src->tf_display); + if (ret < 0) + return ret; + } + dst->header_ref = av_buffer_ref(src->header_ref); if (!dst->header_ref) goto fail; @@ -634,9 +642,11 @@ static av_cold int av1_decode_free(AVCodecContext *avctx) for (int i = 0; i < FF_ARRAY_ELEMS(s->ref); i++) { av1_frame_unref(avctx, &s->ref[i]); av_frame_free(&s->ref[i].tf.f); + av_frame_free(&s->ref[i].tf_display.f); } av1_frame_unref(avctx, &s->cur_frame); av_frame_free(&s->cur_frame.tf.f); + av_frame_free(&s->cur_frame.tf_display.f); av_buffer_unref(&s->seq_ref); av_buffer_unref(&s->header_ref); @@ -738,6 +748,13 @@ static av_cold int av1_decode_init(AVCodecContext *avctx) "Failed to allocate reference frame buffer %d.\n", i); return AVERROR(ENOMEM); } + + s->ref[i].tf_display.f = av_frame_alloc(); + if (!s->ref[i].tf_display.f) { + av_log(avctx, AV_LOG_ERROR, + "Failed to allocate display frame buffer %d.\n", i); + return AVERROR(ENOMEM); + } } s->cur_frame.tf.f = av_frame_alloc(); @@ -747,6 +764,13 @@ static av_cold int av1_decode_init(AVCodecContext *avctx) return AVERROR(ENOMEM); } + s->cur_frame.tf_display.f = av_frame_alloc(); + if (!s->cur_frame.tf_display.f) { + av_log(avctx, AV_LOG_ERROR, + "Failed to allocate current display frame buffer.\n"); + return AVERROR(ENOMEM); + } + ret = ff_cbs_init(&s->cbc, AV_CODEC_ID_AV1, avctx); if (ret < 0) return ret; @@ -819,6 +843,16 @@ static int av1_frame_alloc(AVCodecContext *avctx, AV1Frame *f) break; } + if (header->film_grain.apply_grain && + (avctx->pix_fmt == AV_PIX_FMT_VAAPI) && + !(avctx->export_side_data & AV_CODEC_EXPORT_DATA_FILM_GRAIN)) { + if ((ret = ff_thread_get_buffer(avctx, &f->tf_display, AV_GET_BUFFER_FLAG_REF)) < 0) + goto fail; + + f->tf_display.f->key_frame = frame->key_frame; + f->tf_display.f->pict_type = frame->pict_type; + } + if (avctx->hwaccel) { const AVHWAccel *hwaccel = avctx->hwaccel; if (hwaccel->frame_priv_data_size) { @@ -902,9 +936,16 @@ static int set_output_frame(AVCodecContext *avctx, AVFrame *frame, const AVPacket *pkt, int *got_frame) { AV1DecContext *s = avctx->priv_data; - const AVFrame *srcframe = s->cur_frame.tf.f; + const AVFrame *srcframe; int ret; + /* Use tf_display as output when it is available. */ + if (s->cur_frame.tf_display.f->buf[0]) { + srcframe = s->cur_frame.tf_display.f; + } else { + srcframe= s->cur_frame.tf.f; + } + // TODO: all layers if (s->operating_point_idc && av_log2(s->operating_point_idc >> 8) > s->cur_frame.spatial_id) diff --git a/libavcodec/av1dec.h b/libavcodec/av1dec.h index 4e140588b9..5af3dfc867 100644 --- a/libavcodec/av1dec.h +++ b/libavcodec/av1dec.h @@ -33,6 +33,9 @@ typedef struct AV1Frame { ThreadFrame tf; + /** current tf_display is only used for VA-API to apply film grain */ + ThreadFrame tf_display; + AVBufferRef *hwaccel_priv_buf; void *hwaccel_picture_private;