tensorflow模型转ncnn的操作方式


Posted in Python onMay 25, 2020

第一步把tensorflow保存的.ckpt模型转为pb模型, 并记下模型的输入输出名字.

第二步去ncnn的github上把仓库clone下来, 按照上面的要求装好依赖并make.

第三步是修改ncnn的CMakeList, 具体修改的位置有:

ncnn/CMakeList.txt 文件, 在文件开头处加入add_definitions(-std=c++11), 末尾处加上add_subdirectory(examples), 如果ncnn没有examples文件夹,就新建一个, 并加上CMakeList.txt文件.

ncnn/tools/CMakeList.txt 文件, 加入add_subdirectory(tensorflow)

原版的tools/tensorflow/tensorflow2ncnn.cpp里, 不支持tensorflow的elu, FusedBathNormalization, Conv2dBackpropback操作, 其实elu是支持的,只需要仿照relu的格式, 在.cpp文件里加上就行. FusedBatchNormalization就是ncnn/layer/里实现的batchnorm.cpp, 只是`tensorflow2ncnn里没有写上, 可以增加下面的内容:

else if (node.op() == "FusedBatchNorm")
{
 fprintf(pp, "%-16s", "BatchNorm");
}
...
else if (node.op() == "FusedBatchNorm")
{
 std::cout << "node name is FusedBatchNorm" << std::endl;
 tensorflow::TensorProto tensor;
 find_tensor_proto(weights, node, tensor);
 const tensorflow::TensorShapeProto& shape = tensor.tensor_shape();

 const tensorflow::TensorProto& gamma = weights[node.input(1)];
 const tensorflow::TensorProto& Beta = weights[node.input(2)];
 const tensorflow::TensorProto& mean = weights[node.input(3)];
 const tensorflow::TensorProto& var = weights[node.input(4)];

 int channels = gamma.tensor_shape().dim(0).size(); // data size
 int dtype = gamma.dtype();

 switch (dtype){
  case 1: 
  {

   const float * gamma_tensor = reinterpret_cast<const float *>(gamma.tensor_content().c_str());
   const float * mean_data = reinterpret_cast<const float *>(mean.tensor_content().c_str());
   const float * var_data = reinterpret_cast<const float *>(var.tensor_content().c_str());
   const float * b_data = reinterpret_cast<const float *>(Beta.tensor_content().c_str());
   for (int i=0; i< channels; ++i)
   {
    fwrite(gamma_tensor+i, sizeof(float), 1, bp);
   }
   for (int i=0; i< channels; ++i)
   {
    fwrite(mean_data+i, sizeof(float), 1, bp);
   }
   for (int i=0; i< channels; ++i)
   {
    fwrite(var_data+i, sizeof(float), 1, bp);
   }
   for (int i=0; i< channels; ++i)
   {
    fwrite(b_data+i, sizeof(float), 1, bp);
   }
  }
  default:
   std::cerr << "Type is not supported." << std::endl;

 }
 fprintf(pp, " 0=%d", channels);

 tensorflow::AttrValue value_epsilon;
 if (find_attr_value(node, "epsilon", value_epsilon)){
  float epsilon = value_epsilon.f();
  fprintf(pp, " 1=%f", epsilon);
 }
}

同理, Conv2dBackpropback其实就是ncnn里的反卷积操作, 只不过ncnn实现反卷积的操作和tensorflow内部实现反卷积的操作过程不一样, 但结果是一致的, 需要仿照普通卷积的写法加上去.

ncnn同样支持空洞卷积, 但无法识别tensorflow的空洞卷积, 具体原理可以看tensorflow空洞卷积的原理, tensorflow是改变featuremap做空洞卷积, 而ncnn是改变kernel做空洞卷积, 结果都一样. 需要对.proto文件修改即可完成空洞卷积.

总之ncnn对tensorflow的支持很不友好, 有的层还需要自己手动去实现, 还是很麻烦.

补充知识:pytorch模型转mxnet

介绍

gluon把mxnet再进行封装,封装的风格非常接近pytorch

使用gluon的好处是非常容易把pytorch模型向mxnet转化

唯一的问题是gluon封装还不成熟,封装好的layer不多,很多常用的layer 如concat,upsampling等layer都没有

这里关注如何把pytorch 模型快速转换成 mxnet基于symbol 和 exector设计的网络

pytorch转mxnet module

关键点:

mxnet 设计网络时symbol 名称要和pytorch初始化中各网络层名称对应

torch.load()读入pytorch模型checkpoint 字典,取当中的'state_dict'元素,也是一个字典

pytorch state_dict 字典中key是网络层参数的名称,val是参数ndarray

pytorch 的参数名称的组织形式和mxnet一样,但是连接符号不同,pytorch是'.',而mxnet是'_'比如:

pytorch '0.conv1.0.weight'
mxnet '0_conv1_0_weight'

pytorch 的参数array 和mxnet 的参数array 完全一样,只要名称对上,直接赋值即可初始化mxnet模型

需要做的有以下几点:

设计和pytorch网络对应的mxnet网络

加载pytorch checkpoint

调整pytorch checkpoint state_dict 的key名称和mxnet命名格式一致

FlowNet2S PytorchToMxnet

pytorch flownet2S 的checkpoint 可以在github上搜到

import mxnet as mx
from symbol_util import *
import pickle
 
def get_loss(data, label, loss_scale, name, get_input=False, is_sparse = False, type='stereo'):
 
 if type == 'stereo':
  data = mx.sym.Activation(data=data, act_type='relu',name=name+'relu')
 # loss
 if is_sparse:
  loss =mx.symbol.Custom(data=data, label=label, name=name, loss_scale= loss_scale, is_l1=True,
   op_type='SparseRegressionLoss')
 else:
  loss = mx.sym.MAERegressionOutput(data=data, label=label, name=name, grad_scale=loss_scale)
 return (loss,data) if get_input else loss
 
def flownet_s(loss_scale, is_sparse=False, name=''):
 img1 = mx.symbol.Variable('img1')
 img2 = mx.symbol.Variable('img2')
 data = mx.symbol.concat(img1,img2,dim=1)
 labels = {'loss{}'.format(i): mx.sym.Variable('loss{}_label'.format(i)) for i in range(0, 7)}
 # print('labels: ',labels)
 prediction = {}# a dict for loss collection
 loss = []#a list
 
 #normalize
 data = (data-125)/255
 
 # extract featrue
 conv1 = mx.sym.Convolution(data, pad=(3, 3), kernel=(7, 7), stride=(2, 2), num_filter=64, name=name + 'conv1_0')
 conv1 = mx.sym.LeakyReLU(data=conv1, act_type='leaky', slope=0.1)
 
 conv2 = mx.sym.Convolution(conv1, pad=(2, 2), kernel=(5, 5), stride=(2, 2), num_filter=128, name=name + 'conv2_0')
 conv2 = mx.sym.LeakyReLU(data=conv2, act_type='leaky', slope=0.1)
 
 conv3a = mx.sym.Convolution(conv2, pad=(2, 2), kernel=(5, 5), stride=(2, 2), num_filter=256, name=name + 'conv3_0')
 conv3a = mx.sym.LeakyReLU(data=conv3a, act_type='leaky', slope=0.1)
 
 conv3b = mx.sym.Convolution(conv3a, pad=(1, 1), kernel=(3, 3), stride=(1, 1), num_filter=256, name=name + 'conv3_1_0')
 conv3b = mx.sym.LeakyReLU(data=conv3b, act_type='leaky', slope=0.1)
 
 conv4a = mx.sym.Convolution(conv3b, pad=(1, 1), kernel=(3, 3), stride=(2, 2), num_filter=512, name=name + 'conv4_0')
 conv4a = mx.sym.LeakyReLU(data=conv4a, act_type='leaky', slope=0.1)
 
 conv4b = mx.sym.Convolution(conv4a, pad=(1, 1), kernel=(3, 3), stride=(1, 1), num_filter=512, name=name + 'conv4_1_0')
 conv4b = mx.sym.LeakyReLU(data=conv4b, act_type='leaky', slope=0.1)
 
 conv5a = mx.sym.Convolution(conv4b, pad=(1, 1), kernel=(3, 3), stride=(2, 2), num_filter=512, name=name + 'conv5_0')
 conv5a = mx.sym.LeakyReLU(data=conv5a, act_type='leaky', slope=0.1)
 
 conv5b = mx.sym.Convolution(conv5a, pad=(1, 1), kernel=(3, 3), stride=(1, 1), num_filter=512, name=name + 'conv5_1_0')
 conv5b = mx.sym.LeakyReLU(data=conv5b, act_type='leaky', slope=0.1)
 
 conv6a = mx.sym.Convolution(conv5b, pad=(1, 1), kernel=(3, 3), stride=(2, 2), num_filter=1024, name=name + 'conv6_0')
 conv6a = mx.sym.LeakyReLU(data=conv6a, act_type='leaky', slope=0.1)
 
 conv6b = mx.sym.Convolution(conv6a, pad=(1, 1), kernel=(3, 3), stride=(1, 1), num_filter=1024,
        name=name + 'conv6_1_0')
 conv6b = mx.sym.LeakyReLU(data=conv6b, act_type='leaky', slope=0.1, )
 
 #predict flow
 pr6 = mx.sym.Convolution(conv6b, pad=(1, 1), kernel=(3, 3), stride=(1, 1), num_filter=2,
        name=name + 'predict_flow6')
 prediction['loss6'] = pr6
 
 upsample_pr6to5 = mx.sym.Deconvolution(pr6, pad=(1, 1), kernel=(4, 4), stride=(2, 2), num_filter=2,
           name=name + 'upsampled_flow6_to_5', no_bias=True)
 upconv5 = mx.sym.Deconvolution(conv6b, pad=(1, 1), kernel=(4, 4), stride=(2, 2), num_filter=512,
         name=name + 'deconv5_0', no_bias=False)
 upconv5 = mx.sym.LeakyReLU(data=upconv5, act_type='leaky', slope=0.1)
 iconv5 = mx.sym.Concat(conv5b, upconv5, upsample_pr6to5, dim=1)
 
 
 pr5 = mx.sym.Convolution(iconv5, pad=(1, 1), kernel=(3, 3), stride=(1, 1), num_filter=2,
        name=name + 'predict_flow5')
 prediction['loss5'] = pr5
 
 upconv4 = mx.sym.Deconvolution(iconv5, pad=(1, 1), kernel=(4, 4), stride=(2, 2), num_filter=256,
         name=name + 'deconv4_0', no_bias=False)
 upconv4 = mx.sym.LeakyReLU(data=upconv4, act_type='leaky', slope=0.1)
 
 upsample_pr5to4 = mx.sym.Deconvolution(pr5, pad=(1, 1), kernel=(4, 4), stride=(2, 2), num_filter=2,
           name=name + 'upsampled_flow5_to_4', no_bias=True)
 
 iconv4 = mx.sym.Concat(conv4b, upconv4, upsample_pr5to4)
 
 pr4 = mx.sym.Convolution(iconv4, pad=(1, 1), kernel=(3, 3), stride=(1, 1), num_filter=2,
        name=name + 'predict_flow4')
 prediction['loss4'] = pr4
 
 upconv3 = mx.sym.Deconvolution(iconv4, pad=(1, 1), kernel=(4, 4), stride=(2, 2), num_filter=128,
         name=name + 'deconv3_0', no_bias=False)
 upconv3 = mx.sym.LeakyReLU(data=upconv3, act_type='leaky', slope=0.1)
 
 upsample_pr4to3 = mx.sym.Deconvolution(pr4, pad=(1, 1), kernel=(4, 4), stride=(2, 2), num_filter=2,
           name= name + 'upsampled_flow4_to_3', no_bias=True)
 iconv3 = mx.sym.Concat(conv3b, upconv3, upsample_pr4to3)
 
 pr3 = mx.sym.Convolution(iconv3, pad=(1, 1), kernel=(3, 3), stride=(1, 1), num_filter=2,
        name=name + 'predict_flow3')
 prediction['loss3'] = pr3
 
 upconv2 = mx.sym.Deconvolution(iconv3, pad=(1, 1), kernel=(4, 4), stride=(2, 2), num_filter=64,
         name=name + 'deconv2_0', no_bias=False)
 upconv2 = mx.sym.LeakyReLU(data=upconv2, act_type='leaky', slope=0.1)
 
 upsample_pr3to2 = mx.sym.Deconvolution(pr3, pad=(1, 1), kernel=(4, 4), stride=(2, 2), num_filter=2,
           name=name + 'upsampled_flow3_to_2', no_bias=True)
 iconv2 = mx.sym.Concat(conv2, upconv2, upsample_pr3to2)
 
 pr2 = mx.sym.Convolution(iconv2, pad=(1, 1), kernel=(3, 3), stride=(1, 1), num_filter=2,
        name=name + 'predict_flow2')
 prediction['loss2'] = pr2
 flow = mx.sym.UpSampling(arg0=pr2,scale=4,num_filter=2,num_args = 1,sample_type='nearest', name='upsample_flow2_to_1')
 # ignore the loss functions with loss scale of zero
 keys = loss_scale.keys()
 # keys.sort()
 #obtain the symbol of the losses
 for key in keys:
  # loss.append(get_loss(prediction[key] * 20, labels[key], loss_scale[key], name=key + name,get_input=False, is_sparse=is_sparse, type='flow'))
  loss.append(mx.sym.MAERegressionOutput(data=prediction[key] * 20, label=labels[key], name=key + name, grad_scale=loss_scale[key]))
 # print('loss: ',loss)
 #group 暂时不知道为嘛要group
 loss_group =mx.sym.Group(loss)
 # print('net: ',loss_group)
 return loss_group,flow
 
import gluonbook as gb
import torch
from utils.frame_utils import *
import numpy as np
if __name__ == '__main__':
 checkpoint = torch.load("C:/Users/junjie.huang/PycharmProjects/flownet2_mxnet/flownet2_pytorch/FlowNet2-S_checkpoint.pth.tar")
 # # checkpoint是一个字典
 print(isinstance(checkpoint['state_dict'], dict))
 # # 打印checkpoint字典中的key名
 print('keys of checkpoint:')
 for i in checkpoint:
  print(i)
 print('')
 # # pytorch 模型参数保存在一个key名为'state_dict'的元素中
 state_dict = checkpoint['state_dict']
 # # state_dict也是一个字典
 print('keys of state_dict:')
 for i in state_dict:
  print(i)
  # print(state_dict[i].size())
 print('')
 # print(state_dict)
 #字典的value是torch.tensor
 print(torch.is_tensor(state_dict['conv1.0.weight']))
 #查看某个value的size
 print(state_dict['conv1.0.weight'].size())
 
 #flownet-mxnet init
 loss_scale={'loss2': 1.00,
    'loss3': 1.00,
    'loss4': 1.00,
    'loss5': 1.00,
    'loss6': 1.00}
 loss,flow = flownet_s(loss_scale=loss_scale,is_sparse=False)
 print('loss information: ')
 print('loss:',loss)
 print('type:',type(loss))
 print('list_arguments:',loss.list_arguments())
 print('list_outputs:',loss.list_outputs())
 print('list_inputs:',loss.list_inputs())
 print('')
 
 print('flow information: ')
 print('flow:',flow)
 print('type:',type(flow))
 print('list_arguments:',flow.list_arguments())
 print('list_outputs:',flow.list_outputs())
 print('list_inputs:',flow.list_inputs())
 print('')
 name_mxnet = symbol.list_arguments()
 print(type(name_mxnet))
 for key in name_mxnet:
  print(key)
 
 name_mxnet.sort()
 for key in name_mxnet:
  print(key)
 print(name_mxnet)
 
 shapes = (1, 3, 384, 512)
 ctx = gb.try_gpu()
 # exe = symbol.simple_bind(ctx=ctx, img1=shapes,img2=shapes)
 exe = flow.simple_bind(ctx=ctx, img1=shapes, img2=shapes)
 print('exe type: ',type(exe))
 print('exe: ',exe)
 #module
 # mod = mx.mod.Module(flow)
 # print('mod type: ', type(exe))
 # print('mod: ', exe)
 
 pim1 = read_gen("C:/Users/junjie.huang/PycharmProjects/flownet2_mxnet/data/0000007-img0.ppm")
 pim2 = read_gen("C:/Users/junjie.huang/PycharmProjects/flownet2_mxnet/data/0000007-img1.ppm")
 print(pim1.shape)
 
 '''使用pytorch 的state_dict 初始化 mxnet 模型参数'''
 for key in state_dict:
  # print(type(key))
  k_split = key.split('.')
  key_mx = '_'.join(k_split)
  # print(key,key_mx)
  try:
   exe.arg_dict[key_mx][:]=state_dict[key].data
  except:
   print(key,exe.arg_dict[key_mx].shape,state_dict[key].data.shape)
 
 exe.arg_dict['img1'][:] = pim1[np.newaxis, :, :, :].transpose(0, 3, 1, 2).data
 exe.arg_dict['img2'][:] = pim2[np.newaxis, :, :, :].transpose(0, 3, 1, 2).data
 
 result = exe.forward()
 print('result: ',type(result))
 # for tmp in result:
 #  print(type(tmp))
 #  print(tmp.shape)
 # color = flow2color(exe.outputs[0].asnumpy()[0].transpose(1, 2, 0))
 outputs = exe.outputs
 print('output type: ',type(outputs))
 # for tmp in outputs:
 #  print(type(tmp))
 #  print(tmp.shape)
 
 #来自pytroch flownet2
 from visualize import flow2color
 # color = flow2color(exe.outputs[0].asnumpy()[0].transpose(1,2,0))
 flow_color = flow2color(exe.outputs[0].asnumpy()[0].transpose(1, 2, 0))
 print('color type:',type(flow_color))
 import matplotlib.pyplot as plt
 #来自pytorch
 from torchvision.transforms import ToPILImage
 TF = ToPILImage()
 images = TF(flow_color)
 images.show()
 # plt.imshow(color)

以上这篇tensorflow模型转ncnn的操作方式就是小编分享给大家的全部内容了,希望能给大家一个参考,也希望大家多多支持三水点靠木。

Python 相关文章推荐
Python正则表达式介绍
Aug 06 Python
进一步探究Python中的正则表达式
Apr 28 Python
Python读取键盘输入的2种方法
Jun 16 Python
python实现linux下抓包并存库功能
Jul 18 Python
python发送告警邮件脚本
Sep 17 Python
Python设计模式之迭代器模式原理与用法实例分析
Jan 10 Python
Python时间和字符串转换操作实例分析
Mar 16 Python
Pytorch的mean和std调查实例
Jan 02 Python
Python3操作读写CSV文件使用包过程解析
Apr 10 Python
python文件编写好后如何实践
Jul 07 Python
Matplotlib配色之Colormap详解
Jan 05 Python
Python之京东商品秒杀的实现示例
Jan 06 Python
MxNet预训练模型到Pytorch模型的转换方式
May 25 #Python
浅谈pytorch 模型 .pt, .pth, .pkl的区别及模型保存方式
May 25 #Python
Pytorch通过保存为ONNX模型转TensorRT5的实现
May 25 #Python
tensorflow pb to tflite 精度下降详解
May 25 #Python
Python HTMLTestRunner测试报告view按钮失效解决方案
May 25 #Python
python用opencv完成图像分割并进行目标物的提取
May 25 #Python
Pytorch转tflite方式
May 25 #Python
You might like
php中unlink()、mkdir()、rmdir()等方法的使用介绍
2012/12/21 PHP
PHP各种异常和错误的拦截方法及发生致命错误时进行报警
2016/01/19 PHP
PHP互换两个变量值的方法(不用第三变量)
2016/11/14 PHP
给页面渲染时间加速 干掉Dom Level 0 Event
2012/12/19 Javascript
js操作CheckBoxList实现全选/反选(在客服端完成)
2013/02/02 Javascript
为什么要在引入的css或者js文件后面加参数的详细讲解
2013/05/03 Javascript
JavaScript在for循环中绑定事件解决事件参数不同的情况
2014/01/20 Javascript
用javascript读取xml文件读取节点数据
2014/08/12 Javascript
js实现顶部可折叠的菜单工具栏效果实例
2015/05/09 Javascript
js代码实现随机颜色的小方块
2015/07/30 Javascript
BootStrap使用file-input插件上传图片的方法
2016/09/05 Javascript
js中利用cookie实现记住密码功能
2020/08/20 Javascript
vue的props实现子组件随父组件一起变化
2016/10/27 Javascript
Bootstrap面板(Panels)的简单实现代码
2017/03/17 Javascript
js模拟支付宝密码输入框
2017/04/11 Javascript
jquery实现点击a链接,跳转之后,该a链接处显示背景色的方法
2018/01/18 jQuery
javascript实现最长公共子序列实例代码
2018/02/05 Javascript
ES6使用export和import实现模块化的方法
2018/09/10 Javascript
jQuery实现的简单日历组件定义与用法示例
2018/12/24 jQuery
React降级配置及Ant Design配置详解
2018/12/27 Javascript
关于layui的动态图标不显示的解决方法
2019/09/04 Javascript
JS+html5实现异步上传图片显示上传文件进度条功能示例
2019/11/09 Javascript
[01:44]Ti10举办地公布
2019/08/25 DOTA
举例详解Python中循环语句的嵌套使用
2015/05/14 Python
在python3.5中使用OpenCV的实例讲解
2018/04/02 Python
Django中的Model操作表的实现
2018/07/24 Python
python队列Queue的详解
2019/05/10 Python
python实现提取str字符串/json中多级目录下的某个值
2020/02/27 Python
Python使用GitPython操作Git版本库的方法
2020/02/29 Python
在Anaconda3下使用清华镜像源安装TensorFlow(CPU版)
2020/04/19 Python
军训生自我鉴定范文
2013/12/27 职场文书
学校后勤人员职责
2013/12/27 职场文书
预备党员思想汇报1000字
2014/10/07 职场文书
使用css样式设计一个简单的html登陆界面的实现
2021/03/30 HTML / CSS
怎么用Python识别手势数字
2021/06/07 Python
MySQL外键约束(FOREIGN KEY)案例讲解
2021/08/23 MySQL