From patchwork Sat Jun 6 12:12:46 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Fu, Ting" X-Patchwork-Id: 20169 Return-Path: X-Original-To: patchwork@ffaux-bg.ffmpeg.org Delivered-To: patchwork@ffaux-bg.ffmpeg.org Received: from ffbox0-bg.mplayerhq.hu (ffbox0-bg.ffmpeg.org [79.124.17.100]) by ffaux.localdomain (Postfix) with ESMTP id CE9C144872F for ; Sat, 6 Jun 2020 15:18:10 +0300 (EEST) Received: from [127.0.1.1] (localhost [127.0.0.1]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTP id 77F6368B06B; Sat, 6 Jun 2020 15:18:10 +0300 (EEST) X-Original-To: ffmpeg-devel@ffmpeg.org Delivered-To: ffmpeg-devel@ffmpeg.org Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by ffbox0-bg.mplayerhq.hu (Postfix) with ESMTPS id C1255680A7C for ; Sat, 6 Jun 2020 15:18:03 +0300 (EEST) IronPort-SDR: E2doq6vyuVrtNyGI7/+92gnFTcSxpWqEHdi68NTylMdvzYnXpE/g5M11hpogNV8WugKl4zLeEw JWZlaXhGrdaw== X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Jun 2020 05:18:01 -0700 IronPort-SDR: C97YlpDBpIUJxEZ5v5Hmb7OPKhgedDEAFxgUCeh3ziEbPlV4M3NaZbvR0E52Wn+qdI4Q3N/m+j ATFt5/PvD+xA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.73,480,1583222400"; d="scan'208";a="378996773" Received: from semmer-ubuntu.sh.intel.com ([10.239.159.126]) by fmsmga001.fm.intel.com with ESMTP; 06 Jun 2020 05:18:00 -0700 From: Ting Fu To: ffmpeg-devel@ffmpeg.org Date: Sat, 6 Jun 2020 20:12:46 +0800 Message-Id: <20200606121251.11934-1-ting.fu@intel.com> X-Mailer: git-send-email 2.17.1 Subject: [FFmpeg-devel] [PATCH 1/6] dnn_backend_native_layer_mathunary: add sin support X-BeenThere: ffmpeg-devel@ffmpeg.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: FFmpeg development discussions and patches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: FFmpeg development discussions and patches MIME-Version: 1.0 Errors-To: ffmpeg-devel-bounces@ffmpeg.org Sender: "ffmpeg-devel" It can be tested with the model file generated with below python scripy: import tensorflow as tf import numpy as np import imageio in_img = imageio.imread('input.jpeg') in_img = in_img.astype(np.float32)/255.0 in_data = in_img[np.newaxis, :] x = tf.placeholder(tf.float32, shape=[1, None, None, 3], name='dnn_in') x1 = tf.multiply(x, 3.14) x2 = tf.sin(x1) y = tf.identity(x2, name='dnn_out') sess=tf.Session() sess.run(tf.global_variables_initializer()) graph_def = tf.graph_util.convert_variables_to_constants(sess, sess.graph_def, ['dnn_out']) tf.train.write_graph(graph_def, '.', 'image_process.pb', as_text=False) print("image_process.pb generated, please use \ path_to_ffmpeg/tools/python/convert.py to generate image_process.model\n") output = sess.run(y, feed_dict={x: in_data}) imageio.imsave("out.jpg", np.squeeze(output)) Signed-off-by: Ting Fu --- libavfilter/dnn/dnn_backend_native_layer_mathunary.c | 6 ++++++ libavfilter/dnn/dnn_backend_native_layer_mathunary.h | 1 + tools/python/convert_from_tensorflow.py | 2 +- tools/python/convert_header.py | 2 +- 4 files changed, 9 insertions(+), 2 deletions(-) diff --git a/libavfilter/dnn/dnn_backend_native_layer_mathunary.c b/libavfilter/dnn/dnn_backend_native_layer_mathunary.c index d65af151cd..5324d15bc3 100644 --- a/libavfilter/dnn/dnn_backend_native_layer_mathunary.c +++ b/libavfilter/dnn/dnn_backend_native_layer_mathunary.c @@ -23,6 +23,8 @@ * DNN native backend implementation. */ +#include + #include "dnn_backend_native.h" #include "libavutil/avassert.h" #include "dnn_backend_native_layer_mathunary.h" @@ -74,6 +76,10 @@ int dnn_execute_layer_math_unary(DnnOperand *operands, const int32_t *input_oper for (int i = 0; i < dims_count; ++i) dst[i] = FFABS(src[i]); return 0; + case DMUO_SIN: + for (int i = 0; i < dims_count; ++i) + dst[i] = sin(src[i]); + return 0; default: return -1; } diff --git a/libavfilter/dnn/dnn_backend_native_layer_mathunary.h b/libavfilter/dnn/dnn_backend_native_layer_mathunary.h index 4e44003b66..31a1ea8fb6 100644 --- a/libavfilter/dnn/dnn_backend_native_layer_mathunary.h +++ b/libavfilter/dnn/dnn_backend_native_layer_mathunary.h @@ -31,6 +31,7 @@ typedef enum { DMUO_ABS = 0, + DMUO_SIN = 1, DMUO_COUNT } DNNMathUnaryOperation; diff --git a/tools/python/convert_from_tensorflow.py b/tools/python/convert_from_tensorflow.py index 8c0a9be7be..b17facdda8 100644 --- a/tools/python/convert_from_tensorflow.py +++ b/tools/python/convert_from_tensorflow.py @@ -72,7 +72,7 @@ class TFConverter: self.conv2d_scopename_inputname_dict = {} self.op2code = {'Conv2D':1, 'DepthToSpace':2, 'MirrorPad':3, 'Maximum':4, 'MathBinary':5, 'MathUnary':6} self.mathbin2code = {'Sub':0, 'Add':1, 'Mul':2, 'RealDiv':3, 'Minimum':4} - self.mathun2code = {'Abs':0} + self.mathun2code = {'Abs':0, 'Sin':1} self.mirrorpad_mode = {'CONSTANT':0, 'REFLECT':1, 'SYMMETRIC':2} self.name_operand_dict = {} diff --git a/tools/python/convert_header.py b/tools/python/convert_header.py index ad4491729a..c79fef4be8 100644 --- a/tools/python/convert_header.py +++ b/tools/python/convert_header.py @@ -23,4 +23,4 @@ str = 'FFMPEGDNNNATIVE' major = 1 # increase minor when we don't have to re-convert the model file -minor = 6 +minor = 7