From patchwork Mon Sep 12 07:41:30 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jun Zhao X-Patchwork-Id: 552 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.103.140.134 with SMTP id o128csp1480596vsd; Mon, 12 Sep 2016 00:41:46 -0700 (PDT) X-Received: by 10.194.192.195 with SMTP id hi3mr13538100wjc.108.1473666105929; Mon, 12 Sep 2016 00:41:45 -0700 (PDT) Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id s8si798375wjr.237.2016.09.12.00.41.45; Mon, 12 Sep 2016 00:41:45 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 9A38B689F19; Mon, 12 Sep 2016 10:41:31 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pf0-f169.google.com (mail-pf0-f169.google.com [209.85.192.169]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id A7361689EE1 for ; Mon, 12 Sep 2016 10:41:24 +0300 (EEST) Received: by mail-pf0-f169.google.com with SMTP id i75so9769846pfa.2 for ; Mon, 12 Sep 2016 00:41:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=to:from:subject:message-id:date:user-agent:mime-version; bh=UP7sQnjWiUn5ngqaXGAjUz3V2V9V3hcCBP3UUimGfCU=; b=JwU0YjjBwR7j4VweClC0Jc483MyxS4Z4J8zYgPvQxy5lrI4B9XY5yhfUO2AVKJho6l dS2mafCKtnphJkkgo9d5BLaSHMLI1uto9K2gwau4dFGOUoOp5Fl+SXgGyRtY3HHddA1u TmwcwKc6cDyUaQc3uwjBwRgjiaf6SXV9ER12uTi5D/yYTouFkELPMbHhiotVCFKzJbO0 zDEa8ABn/ZKomNj++2bVjF+8nwpnMJvqCDa0LOJehCcrJYCfTC4c4hFDwxdOy820d1rA mxXVUSYwEjPVRpvCxpEgYFtYp72I2QwLGS7Ap4OrGrmnQISJrOJPxTu2NY+0XTRUSlwy kmdg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:to:from:subject:message-id:date:user-agent :mime-version; bh=UP7sQnjWiUn5ngqaXGAjUz3V2V9V3hcCBP3UUimGfCU=; b=hnN+XdjlojaQzseUPIPOt7iyNRjzObehv+Z2wnoFcPbvZ+Db4iy0pHoBgl3VK68Jbb gBxRQP2aAGjLYs4smCvrGD4ZOvtzHpXOdlxqNcq3m33jbdsq4od3kJt17jYagbXZ2eBa iaJmpTRiz2EKupHEdKtDjoOXCrg+q7rQ7ECMJvdoQRL9VjjreKBCrBjv3VHEV7d+nhMd aLvuZMfWLsiziMLD0VOCMbz2M9ULYGz8M3hVUi+Zyz5vGF6ISeja14kLv3/xPFgpSsGs h0Yo3RXlpAr48pdjneMEQPTAiOdLScE9My7QGWzMAKg4/nD6QGiHwfIbuew3F0F15AKU jNtg== X-Gm-Message-State: AE9vXwP+tD//cBw3u2t+rM1X+kd6NQ9G6D+4ASAFCfwAsOZs8Ty7v5AN3LLiOCYvFouKUA== X-Received: by 10.98.73.202 with SMTP id r71mr4561393pfi.36.1473666094747; Mon, 12 Sep 2016 00:41:34 -0700 (PDT) Received: from [10.239.204.170] ([192.55.54.44]) by smtp.gmail.com with ESMTPSA id s1sm22088781paz.47.2016.09.12.00.41.32 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 12 Sep 2016 00:41:34 -0700 (PDT) To: ffmpeg development discussions and patches , Mark Thompson , Moritz Barsnick , jun.zhao@intel.com From: Jun Zhao Message-ID: Date: Mon, 12 Sep 2016 15:41:30 +0800 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.3.0 MIME-Version: 1.0 Subject: [FFmpeg-devel] [PATCH v5] lavfi/scale_vaapi: add denoise/sharpness support X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" v5 : - fix the commit message as review(Mark Thompson and Moritz Barsnick) - change the magic filter numbers 8 with VAProcFilterCount - check multiple filters because multiple filters aren't supported in the driver now v4 : - fix sharpless typo(sharpless -> sharpness) - when don't support denoise/sharpness, report the error and return - fix denoise/sharpness params buffer leak in error handle v3 : fix sharpless mapping issue v2 : fix filter support flag check logic issue From 21471c5e222873dd03bc3f9878598314710d7784 Mon Sep 17 00:00:00 2001 From: Jun Zhao Date: Tue, 30 Aug 2016 14:36:00 +0800 Subject: [PATCH v5] lavfi/scale_vaapi: add denoise/sharpness support. add denoise/sharpness support, used scope [-1, 100] as the input scope. Signed-off-by: Jun Zhao --- libavfilter/vf_scale_vaapi.c | 107 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 107 insertions(+) diff --git a/libavfilter/vf_scale_vaapi.c b/libavfilter/vf_scale_vaapi.c index 8dd5acf..736ce56 100644 --- a/libavfilter/vf_scale_vaapi.c +++ b/libavfilter/vf_scale_vaapi.c @@ -53,6 +53,14 @@ typedef struct ScaleVAAPIContext { int output_width; int output_height; + VAProcFilterCap denoise_caps; + int denoise; // enable denoise algo. level is the optional + // value from the interval [-1, 100], -1 means disabled + + VAProcFilterCap sharpness_caps; + int sharpness; // enable sharpness. level is the optional value + // from the interval [-1, 100], -1 means disabled + } ScaleVAAPIContext; @@ -117,6 +125,8 @@ static int scale_vaapi_config_output(AVFilterLink *outlink) AVVAAPIFramesContext *va_frames; VAStatus vas; int err, i; + unsigned int num_denoise_caps = 1; + unsigned int num_sharpness_caps = 1; scale_vaapi_pipeline_uninit(ctx); @@ -225,6 +235,40 @@ static int scale_vaapi_config_output(AVFilterLink *outlink) goto fail; } + // multiple filters aren't supported in the driver: + // sharpness can't work with noise reduction(de-noise), deinterlacing + // color balance, skin tone enhancement... + if (ctx->denoise != -1 && ctx->sharpness != -1) { + av_log(ctx, AV_LOG_ERROR, "Do not support multiply filters (sharpness " + "can't work with the other filters).\n"); + err = AVERROR(EINVAL); + goto fail; + } + + if (ctx->denoise != -1) { + vas = vaQueryVideoProcFilterCaps(ctx->hwctx->display, ctx->va_context, + VAProcFilterNoiseReduction, + &ctx->denoise_caps, &num_denoise_caps); + if (vas != VA_STATUS_SUCCESS) { + av_log(ctx, AV_LOG_ERROR, "Failed to query denoise caps " + "context: %d (%s).\n", vas, vaErrorStr(vas)); + err = AVERROR(EIO); + goto fail; + } + } + + if (ctx->sharpness != -1) { + vas = vaQueryVideoProcFilterCaps(ctx->hwctx->display, ctx->va_context, + VAProcFilterSharpening, + &ctx->sharpness_caps, &num_sharpness_caps); + if (vas != VA_STATUS_SUCCESS) { + av_log(ctx, AV_LOG_ERROR, "Failed to query sharpness caps " + "context: %d (%s).\n", vas, vaErrorStr(vas)); + err = AVERROR(EIO); + goto fail; + } + } + av_freep(&hwconfig); av_hwframe_constraints_free(&constraints); return 0; @@ -250,6 +294,14 @@ static int vaapi_proc_colour_standard(enum AVColorSpace av_cs) } } +static float map_to_range( + int input, int in_min, int in_max, + float out_min, float out_max) +{ + return (input - in_min) * (out_max - out_min) / (in_max - in_min) + out_min; +} + + static int scale_vaapi_filter_frame(AVFilterLink *inlink, AVFrame *input_frame) { AVFilterContext *avctx = inlink->dst; @@ -259,6 +311,10 @@ static int scale_vaapi_filter_frame(AVFilterLink *inlink, AVFrame *input_frame) VASurfaceID input_surface, output_surface; VAProcPipelineParameterBuffer params; VABufferID params_id; + VABufferID denoise_id; + VABufferID sharpness_id; + VABufferID filter_bufs[VAProcFilterCount]; + int num_filter_bufs = 0; VAStatus vas; int err; @@ -290,6 +346,43 @@ static int scale_vaapi_filter_frame(AVFilterLink *inlink, AVFrame *input_frame) av_log(ctx, AV_LOG_DEBUG, "Using surface %#x for scale output.\n", output_surface); + if (ctx->denoise != -1) { + VAProcFilterParameterBuffer denoise; + denoise.type = VAProcFilterNoiseReduction; + denoise.value = map_to_range(ctx->denoise, 0, 100, + ctx->denoise_caps.range.min_value, + ctx->denoise_caps.range.max_value); + vas = vaCreateBuffer(ctx->hwctx->display, ctx->va_context, + VAProcFilterParameterBufferType, sizeof(denoise), 1, + &denoise, &denoise_id); + if (vas != VA_STATUS_SUCCESS) { + av_log(ctx, AV_LOG_ERROR, "Failed to create denoise parameter buffer: " + "%d (%s).\n", vas, vaErrorStr(vas)); + err = AVERROR(EIO); + goto fail; + } + filter_bufs[num_filter_bufs++] = denoise_id; + } + + if (ctx->sharpness != -1) { + VAProcFilterParameterBuffer sharpness; + sharpness.type = VAProcFilterSharpening; + sharpness.value = map_to_range(ctx->sharpness, + 0, 100, + ctx->sharpness_caps.range.min_value, + ctx->sharpness_caps.range.max_value); + vas = vaCreateBuffer(ctx->hwctx->display, ctx->va_context, + VAProcFilterParameterBufferType, sizeof(sharpness), 1, + &sharpness, &sharpness_id); + if (vas != VA_STATUS_SUCCESS) { + av_log(ctx, AV_LOG_ERROR, "Failed to create sharpness parameter buffer: " + "%d (%s).\n", vas, vaErrorStr(vas)); + err = AVERROR(EIO); + goto fail; + } + filter_bufs[num_filter_bufs++] = sharpness_id; + } + memset(¶ms, 0, sizeof(params)); params.surface = input_surface; @@ -304,6 +397,11 @@ static int scale_vaapi_filter_frame(AVFilterLink *inlink, AVFrame *input_frame) params.pipeline_flags = 0; params.filter_flags = VA_FILTER_SCALING_HQ; + if (num_filter_bufs) { + params.filters = filter_bufs; + params.num_filters = num_filter_bufs; + } + vas = vaBeginPicture(ctx->hwctx->display, ctx->va_context, output_surface); if (vas != VA_STATUS_SUCCESS) { @@ -351,6 +449,9 @@ static int scale_vaapi_filter_frame(AVFilterLink *inlink, AVFrame *input_frame) goto fail; } + for (int i = 0; i < num_filter_bufs; i++) + vaDestroyBuffer(ctx->hwctx->display, filter_bufs[i]); + av_frame_copy_props(output_frame, input_frame); av_frame_free(&input_frame); @@ -369,6 +470,8 @@ fail_after_begin: fail_after_render: vaEndPicture(ctx->hwctx->display, ctx->va_context); fail: + for (int i = 0; i < num_filter_bufs; i++) + vaDestroyBuffer(ctx->hwctx->display, filter_bufs[i]); av_frame_free(&input_frame); av_frame_free(&output_frame); return err; @@ -418,6 +521,10 @@ static const AVOption scale_vaapi_options[] = { OFFSET(output_height), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, INT_MAX, .flags = FLAGS }, { "format", "Output video format (software format of hardware frames)", OFFSET(output_format_string), AV_OPT_TYPE_STRING, .flags = FLAGS }, + { "denoise", "denoise level [-1, 100], -1 means disabled", + OFFSET(denoise), AV_OPT_TYPE_INT, { .i64 = -1 }, -1, 100, .flags = FLAGS }, + { "sharpness", "sharpness level [-1, 100], -1 means disabled", + OFFSET(sharpness), AV_OPT_TYPE_INT, { .i64 = -1 }, -1, 100, .flags = FLAGS }, { NULL }, };