From patchwork Mon Mar 23 04:08:32 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Guo, Yejun" X-Patchwork-Id: 18364 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id 55FC744B13D for ; Mon, 23 Mar 2020 06:09:29 +0200 (EET) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 3CF4968B796; Mon, 23 Mar 2020 06:09:29 +0200 (EET) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id 8D2AE68B787 for ; Mon, 23 Mar 2020 06:09:27 +0200 (EET) IronPort-SDR: 81Ghq1VCDvHVkz7Stzu8eyoS7YSWkWybCk5qBBXjwJEcJVeWh00jzVhrSp9R9mk5I5HFUS3H7Z WNEV2zn2SLJw== X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Mar 2020 21:09:27 -0700 IronPort-SDR: 63bzrl1Uh+LnR8ADIo0AWYDkjCwqtXa3gMX55ROzJxVD7nJ9Joq5jTE8gJc+0meNKmd/Wo9yIX +ZJYqKRmH7ug== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.72,295,1580803200"; d="scan'208";a="447269583" Received: from yguo18-skl-u1604.sh.intel.com ([10.239.159.53]) by fmsmga006.fm.intel.com with ESMTP; 22 Mar 2020 21:09:25 -0700 From: "Guo, Yejun" To: ffmpeg-devel@ffmpeg.org Date: Mon, 23 Mar 2020 12:08:32 +0800 Message-Id: <1584936512-9306-1-git-send-email-yejun.guo@intel.com> X-Mailer: git-send-email 2.7.4 Subject: [FFmpeg-devel] [PATCH 4/4] avfilter/vf_derain.c: put all the calculation in model file. X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches Cc: yejun.guo@intel.com MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" currently, the model outputs the rain, and so need a subtraction in filter c code to get the final derain result. I've sent a PR to update the model file and accepted, see at https://github.com/XueweiMeng/derain_filter/pull/3 Signed-off-by: Guo, Yejun --- doc/filters.texi | 7 +++++-- libavfilter/vf_derain.c | 6 +----- 2 files changed, 6 insertions(+), 7 deletions(-) diff --git a/doc/filters.texi b/doc/filters.texi index 328e984..3dc7715 100644 --- a/doc/filters.texi +++ b/doc/filters.texi @@ -8878,6 +8878,7 @@ delogo=x=0:y=0:w=100:h=77:band=10 @end itemize +@anchor{derain} @section derain Remove the rain in the input image/video by applying the derain methods based on @@ -8932,6 +8933,8 @@ Note that different backends use different file formats. TensorFlow and native backend can load files for only its format. @end table +It can also be finished with @ref{dnn_processing} filter. + @section deshake Attempt to fix small changes in horizontal and/or vertical shift. This @@ -9201,9 +9204,9 @@ Set the output name of the dnn network. @itemize @item -Halve the red channle of the frame with format rgb24: +Remove rain in rgb24 frame with can.pb (see @ref{derain} filter): @example -ffmpeg -i input.jpg -vf format=rgb24,dnn_processing=model=halve_first_channel.model:input=dnn_in:output=dnn_out:dnn_backend=native out.native.png +./ffmpeg -i rain.jpg -vf format=rgb24,dnn_processing=dnn_backend=tensorflow:model=can.pb:input=x:output=y derain.jpg @end example @item diff --git a/libavfilter/vf_derain.c b/libavfilter/vf_derain.c index 89f9d5a..7432260 100644 --- a/libavfilter/vf_derain.c +++ b/libavfilter/vf_derain.c @@ -100,7 +100,6 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *in) AVFilterLink *outlink = ctx->outputs[0]; DRContext *dr_context = ctx->priv; DNNReturnType dnn_result; - int pad_size; AVFrame *out = ff_get_video_buffer(outlink, outlink->w, outlink->h); if (!out) { @@ -129,15 +128,12 @@ static int filter_frame(AVFilterLink *inlink, AVFrame *in) out->width = dr_context->output.width; outlink->h = dr_context->output.height; outlink->w = dr_context->output.width; - pad_size = (in->height - out->height) >> 1; for (int i = 0; i < out->height; i++){ for(int j = 0; j < out->width * 3; j++){ int k = i * out->linesize[0] + j; int t = i * out->width * 3 + j; - - int t_in = (i + pad_size) * in->width * 3 + j + pad_size * 3; - out->data[0][k] = CLIP((int)((((float *)dr_context->input.data)[t_in] - ((float *)dr_context->output.data)[t]) * 255), 0, 255); + out->data[0][k] = CLIP((int)((((float *)dr_context->output.data)[t]) * 255), 0, 255); } }