From patchwork Fri Aug 4 07:25:54 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jun Zhao X-Patchwork-Id: 4620 Delivered-To: ffmpegpatchwork@gmail.com Received: by 10.103.46.211 with SMTP id u202csp1924090vsu; Fri, 4 Aug 2017 00:34:21 -0700 (PDT) X-Received: by 10.223.169.131 with SMTP id b3mr1072892wrd.126.1501832060995; Fri, 04 Aug 2017 00:34:20 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1501832060; cv=none; d=google.com; s=arc-20160816; b=JCoqU5MK2xg4/QO9slePP6O9oa0//rHt7ijiCW7/12h+3ndQ1gnaOrUSHZhdZxgG1Q MM5XivWslHL62TYcCH4FtwSqCZm6c3x/izlxY5sqOWlxdMibR8MEnDRvBVkWK+ETf19U 7bmW9k+pj4ovX53krRFqDpTa6cZQNPygW9dmLvM1CNfWwxixlkwLuCaipWTHQFy8xlfv UFEFBaTBdAX5VMlSNBBqlQN94vkFk/LKycvcrHVGBIvKlll+lfPID0pyz/PjdDQxEimG 43iuujQ620RS2QFswQtQsFs/qd/iJKqUlPUx65Q1JMXOGEEenDcgM+n8Qry+iTA9caUx wBgg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:reply-to:list-subscribe:list-help:list-post :list-archive:list-unsubscribe:list-id:precedence:subject :content-language:mime-version:user-agent:date:message-id:from:to :dkim-signature:delivered-to:arc-authentication-results; bh=0uWp4htZHtulKDk6deC1+uONBNe/64jFSaIzExCslmk=; b=DLWfQ0ozD1vgeVpP7IIKupsaBFhtI7o9wkF06ww7+iLXuO0ghE7S5zSRGIThY3BBXf qx2gZmfKjhPbnq/qketxXMUmBrv3UT7++MojJdAv1MlUj9PNQ373kIiDoVfDAJ92CSlX dqOFUi2IVfcqk8vhAUBurj9kYAgbdTbB+EjTh2aicly3m/aLkOHbjXzcAMh4x+hVh+D8 RAUwOVkMiRwWmBjSyX5OYDfAs8cgL/l/ciXzezkbe2skigYO22P2aD/q2CQZiT7DjLm+ CSP7frssj+hk+TupNn7Ajzu3UVZ0+zgkJ6GuJcu5u3Pq+AtsXfz2YYmdPw51hVjHVi0I 3+vw== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b=sliPaQOS; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=gmail.com Return-Path: Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org. [79.124.17.100]) by mx.google.com with ESMTP id o203si2651192wmo.47.2017.08.04.00.34.18; Fri, 04 Aug 2017 00:34:20 -0700 (PDT) Received-SPF: pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) client-ip=79.124.17.100; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@gmail.com header.s=20161025 header.b=sliPaQOS; spf=pass (google.com: domain of ffmpeg-devel-bounces@ffmpeg.org designates 79.124.17.100 as permitted sender) smtp.mailfrom=ffmpeg-devel-bounces@ffmpeg.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=gmail.com Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 99D586891FB; Fri, 4 Aug 2017 10:34:15 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mail-pg0-f66.google.com (mail-pg0-f66.google.com [74.125.83.66]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 32FC5688379 for ; Fri, 4 Aug 2017 10:34:09 +0300 (EEST) Received: by mail-pg0-f66.google.com with SMTP id y192so1066698pgd.1 for ; Fri, 04 Aug 2017 00:34:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=to:from:subject:message-id:date:user-agent:mime-version :content-language; bh=0IG/La+RINqNiBa8rtlbuRYZuuOsv2aX7chcZGrIDTA=; b=sliPaQOSDB8oJ7uM6r4TO25Vc/dpxFc6IJLOqEZBVfMP1qaybX5nANr7sfs0YUXnSa SJcG3EAoH2WlxdI/7N3NVeWQOHIiFAOEkCapGCeaEBQ5FuZMnfY74NHmdTHWKv7S1+eG WHNNb9U8J+DXqTKGOCEuR+O8Hnx9MjcqQXK1IHvK4fNu9OO+98Vcs6cDGaX+yxLF/bIt VgR2v/Hhc68syOjMGLIoTMT9yLyZ8WqL4e4/Kib5YyDkmE4cxp6aKsQGlAymSJBqYKqs BLgaHl5IQXXoPiMk+Rt1JBmkmCEEvGQtLO/DdsUkgPW2adwp0Wzm/XyrRLTMM9e2N1Gd vylQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:to:from:subject:message-id:date:user-agent :mime-version:content-language; bh=0IG/La+RINqNiBa8rtlbuRYZuuOsv2aX7chcZGrIDTA=; b=QQaTj4pEhL0AKpYcenuDh4KAKnZPC+BXZIObxWyYUidq/Se7Dw9ja0LeIq4TIDOunF j5ZeXG7uWz2+vSrUlpog7sKOGynD0Q1s+7HO5jEezcspp5zqsVJ6leFxM64zMCoX51z9 4TekNpivVCFpR5dDZuED+pEY7vMRzWovFQerMgYN8ep2/9d1aGmod6kZWoHLSGg2c0xo e0LJwvOpVR2vvFVeiFV0vRDs9jbiFPobVMvGLk66NIO+RBsV76nHKqFie1RKKiI/j4BB YGmGKwCx1Moas8CBoix8c0DlEGDNUdi+C4fds1nqLqarQrVEylKAR6a2lGsvJT1mCNlC +0gg== X-Gm-Message-State: AIVw112IuKYKSDNG0PHQgOd1V6Yv5+cET94+jWlE7mFcTj0PwEXFQub+ 6M+GR8hu7nbzGQ== X-Received: by 10.98.138.22 with SMTP id y22mr1469883pfd.9.1501831559941; Fri, 04 Aug 2017 00:25:59 -0700 (PDT) Received: from [10.67.68.30] (fmdmzpr02-ext.fm.intel.com. [192.55.55.37]) by smtp.gmail.com with ESMTPSA id t186sm1424868pgc.28.2017.08.04.00.25.56 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 04 Aug 2017 00:25:58 -0700 (PDT) To: FFmpeg development discussions and patches , Mark Thompson , Moritz Barsnick From: Jun Zhao Message-ID: <3057e4ed-9f36-ba05-5b6c-a1fe51a535c8@gmail.com> Date: Fri, 4 Aug 2017 15:25:54 +0800 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.2.1 MIME-Version: 1.0 Content-Language: en-US Subject: [FFmpeg-devel] [PATCH V6] lavfi/scale_vaapi: add denoise/sharpness support. X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" V6 : - Re-work with current scale_vaapi and double check i965 have fix the sharpness issue. https://bugs.freedesktop.org/show_bug.cgi?id=96988 v5 : - fix the commit message as review(Mark Thompson and Moritz Barsnick) - change the magic filter numbers 8 with VAProcFilterCount - check multiple filters because multiple filters aren't supported in the driver now v4 : - fix sharpless typo(sharpless -> sharpness) - when don't support denoise/sharpness, report the error and return - fix denoise/sharpness params buffer leak in error handle v3 : fix sharpless mapping issue v2 : fix filter support flag check logic issue From 5a97b6c307480b59d27f3dc2aabdb0781912835d Mon Sep 17 00:00:00 2001 From: Jun Zhao Date: Fri, 4 Aug 2017 03:10:40 -0400 Subject: [PATCH V6] lavfi/scale_vaapi: add denoise/sharpness support. add denoise/sharpness support, used scope [-1, 100] as the input scope. Signed-off-by: Jun Zhao --- libavfilter/vf_scale_vaapi.c | 107 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 107 insertions(+) diff --git a/libavfilter/vf_scale_vaapi.c b/libavfilter/vf_scale_vaapi.c index 5f049a5d7b..cb5b03df61 100644 --- a/libavfilter/vf_scale_vaapi.c +++ b/libavfilter/vf_scale_vaapi.c @@ -58,6 +58,14 @@ typedef struct ScaleVAAPIContext { int output_width; // computed width int output_height; // computed height + + VAProcFilterCap denoise_caps; + int denoise; // enable denoise algo. level is the optional + // value from the interval [-1, 100], -1 means disabled + + VAProcFilterCap sharpness_caps; + int sharpness; // enable sharpness. level is the optional value + // from the interval [-1, 100], -1 means disabled } ScaleVAAPIContext; @@ -126,6 +134,8 @@ static int scale_vaapi_config_output(AVFilterLink *outlink) AVVAAPIFramesContext *va_frames; VAStatus vas; int err, i; + uint32_t num_denoise_caps = 1; + uint32_t num_sharpness_caps = 1; scale_vaapi_pipeline_uninit(ctx); @@ -240,6 +250,40 @@ static int scale_vaapi_config_output(AVFilterLink *outlink) goto fail; } + // multiple filters aren't supported in the driver: + // sharpness can't work with noise reduction(de-noise), deinterlacing + // color balance, skin tone enhancement... + if (ctx->denoise != -1 && ctx->sharpness != -1) { + av_log(ctx, AV_LOG_ERROR, "Do not support multiply filters (sharpness " + "can't work with the other filters).\n"); + err = AVERROR(EINVAL); + goto fail; + } + + if (ctx->denoise != -1) { + vas = vaQueryVideoProcFilterCaps(ctx->hwctx->display, ctx->va_context, + VAProcFilterNoiseReduction, + &ctx->denoise_caps, &num_denoise_caps); + if (vas != VA_STATUS_SUCCESS) { + av_log(ctx, AV_LOG_ERROR, "Failed to query denoise caps " + "context: %d (%s).\n", vas, vaErrorStr(vas)); + err = AVERROR(EIO); + goto fail; + } + } + + if (ctx->sharpness != -1) { + vas = vaQueryVideoProcFilterCaps(ctx->hwctx->display, ctx->va_context, + VAProcFilterSharpening, + &ctx->sharpness_caps, &num_sharpness_caps); + if (vas != VA_STATUS_SUCCESS) { + av_log(ctx, AV_LOG_ERROR, "Failed to query sharpness caps " + "context: %d (%s).\n", vas, vaErrorStr(vas)); + err = AVERROR(EIO); + goto fail; + } + } + av_freep(&hwconfig); av_hwframe_constraints_free(&constraints); return 0; @@ -265,6 +309,14 @@ static int vaapi_proc_colour_standard(enum AVColorSpace av_cs) } } +static float map_to_range( + int input, int in_min, int in_max, + float out_min, float out_max) +{ + return (input - in_min) * (out_max - out_min) / (in_max - in_min) + out_min; +} + + static int scale_vaapi_filter_frame(AVFilterLink *inlink, AVFrame *input_frame) { AVFilterContext *avctx = inlink->dst; @@ -274,6 +326,10 @@ static int scale_vaapi_filter_frame(AVFilterLink *inlink, AVFrame *input_frame) VASurfaceID input_surface, output_surface; VAProcPipelineParameterBuffer params; VABufferID params_id; + VABufferID denoise_id; + VABufferID sharpness_id; + VABufferID filter_bufs[VAProcFilterCount]; + int num_filter_bufs = 0; VARectangle input_region; VAStatus vas; int err; @@ -300,6 +356,43 @@ static int scale_vaapi_filter_frame(AVFilterLink *inlink, AVFrame *input_frame) av_log(ctx, AV_LOG_DEBUG, "Using surface %#x for scale output.\n", output_surface); + if (ctx->denoise != -1) { + VAProcFilterParameterBuffer denoise; + denoise.type = VAProcFilterNoiseReduction; + denoise.value = map_to_range(ctx->denoise, 0, 100, + ctx->denoise_caps.range.min_value, + ctx->denoise_caps.range.max_value); + vas = vaCreateBuffer(ctx->hwctx->display, ctx->va_context, + VAProcFilterParameterBufferType, sizeof(denoise), 1, + &denoise, &denoise_id); + if (vas != VA_STATUS_SUCCESS) { + av_log(ctx, AV_LOG_ERROR, "Failed to create denoise parameter buffer: " + "%d (%s).\n", vas, vaErrorStr(vas)); + err = AVERROR(EIO); + goto fail; + } + filter_bufs[num_filter_bufs++] = denoise_id; + } + + if (ctx->sharpness != -1) { + VAProcFilterParameterBuffer sharpness; + sharpness.type = VAProcFilterSharpening; + sharpness.value = map_to_range(ctx->sharpness, + 0, 100, + ctx->sharpness_caps.range.min_value, + ctx->sharpness_caps.range.max_value); + vas = vaCreateBuffer(ctx->hwctx->display, ctx->va_context, + VAProcFilterParameterBufferType, sizeof(sharpness), 1, + &sharpness, &sharpness_id); + if (vas != VA_STATUS_SUCCESS) { + av_log(ctx, AV_LOG_ERROR, "Failed to create sharpness parameter buffer: " + "%d (%s).\n", vas, vaErrorStr(vas)); + err = AVERROR(EIO); + goto fail; + } + filter_bufs[num_filter_bufs++] = sharpness_id; + } + memset(¶ms, 0, sizeof(params)); // If there were top/left cropping, it could be taken into @@ -323,6 +416,11 @@ static int scale_vaapi_filter_frame(AVFilterLink *inlink, AVFrame *input_frame) params.pipeline_flags = 0; params.filter_flags = VA_FILTER_SCALING_HQ; + if (num_filter_bufs) { + params.filters = filter_bufs; + params.num_filters = num_filter_bufs; + } + vas = vaBeginPicture(ctx->hwctx->display, ctx->va_context, output_surface); if (vas != VA_STATUS_SUCCESS) { @@ -371,6 +469,9 @@ static int scale_vaapi_filter_frame(AVFilterLink *inlink, AVFrame *input_frame) } } + for (int i = 0; i < num_filter_bufs; i++) + vaDestroyBuffer(ctx->hwctx->display, filter_bufs[i]); + av_frame_copy_props(output_frame, input_frame); av_frame_free(&input_frame); @@ -389,6 +490,8 @@ fail_after_begin: fail_after_render: vaEndPicture(ctx->hwctx->display, ctx->va_context); fail: + for (int i = 0; i < num_filter_bufs; i++) + vaDestroyBuffer(ctx->hwctx->display, filter_bufs[i]); av_frame_free(&input_frame); av_frame_free(&output_frame); return err; @@ -438,6 +541,10 @@ static const AVOption scale_vaapi_options[] = { OFFSET(h_expr), AV_OPT_TYPE_STRING, {.str = "ih"}, .flags = FLAGS }, { "format", "Output video format (software format of hardware frames)", OFFSET(output_format_string), AV_OPT_TYPE_STRING, .flags = FLAGS }, + { "denoise", "denoise level [-1, 100], -1 means disabled", + OFFSET(denoise), AV_OPT_TYPE_INT, { .i64 = -1 }, -1, 100, .flags = FLAGS }, + { "sharpness", "sharpness level [-1, 100], -1 means disabled", + OFFSET(sharpness), AV_OPT_TYPE_INT, { .i64 = -1 }, -1, 100, .flags = FLAGS }, { NULL }, };