tensorflow模型转ncnn的操作方式


Posted in Python onMay 25, 2020

第一步把tensorflow保存的.ckpt模型转为pb模型, 并记下模型的输入输出名字.

第二步去ncnn的github上把仓库clone下来, 按照上面的要求装好依赖并make.

第三步是修改ncnn的CMakeList, 具体修改的位置有:

ncnn/CMakeList.txt 文件, 在文件开头处加入add_definitions(-std=c++11), 末尾处加上add_subdirectory(examples), 如果ncnn没有examples文件夹,就新建一个, 并加上CMakeList.txt文件.

ncnn/tools/CMakeList.txt 文件, 加入add_subdirectory(tensorflow)

原版的tools/tensorflow/tensorflow2ncnn.cpp里, 不支持tensorflow的elu, FusedBathNormalization, Conv2dBackpropback操作, 其实elu是支持的,只需要仿照relu的格式, 在.cpp文件里加上就行. FusedBatchNormalization就是ncnn/layer/里实现的batchnorm.cpp, 只是`tensorflow2ncnn里没有写上, 可以增加下面的内容:

else if (node.op() == "FusedBatchNorm")
{
 fprintf(pp, "%-16s", "BatchNorm");
}
...
else if (node.op() == "FusedBatchNorm")
{
 std::cout << "node name is FusedBatchNorm" << std::endl;
 tensorflow::TensorProto tensor;
 find_tensor_proto(weights, node, tensor);
 const tensorflow::TensorShapeProto& shape = tensor.tensor_shape();

 const tensorflow::TensorProto& gamma = weights[node.input(1)];
 const tensorflow::TensorProto& Beta = weights[node.input(2)];
 const tensorflow::TensorProto& mean = weights[node.input(3)];
 const tensorflow::TensorProto& var = weights[node.input(4)];

 int channels = gamma.tensor_shape().dim(0).size(); // data size
 int dtype = gamma.dtype();

 switch (dtype){
  case 1: 
  {

   const float * gamma_tensor = reinterpret_cast<const float *>(gamma.tensor_content().c_str());
   const float * mean_data = reinterpret_cast<const float *>(mean.tensor_content().c_str());
   const float * var_data = reinterpret_cast<const float *>(var.tensor_content().c_str());
   const float * b_data = reinterpret_cast<const float *>(Beta.tensor_content().c_str());
   for (int i=0; i< channels; ++i)
   {
    fwrite(gamma_tensor+i, sizeof(float), 1, bp);
   }
   for (int i=0; i< channels; ++i)
   {
    fwrite(mean_data+i, sizeof(float), 1, bp);
   }
   for (int i=0; i< channels; ++i)
   {
    fwrite(var_data+i, sizeof(float), 1, bp);
   }
   for (int i=0; i< channels; ++i)
   {
    fwrite(b_data+i, sizeof(float), 1, bp);
   }
  }
  default:
   std::cerr << "Type is not supported." << std::endl;

 }
 fprintf(pp, " 0=%d", channels);

 tensorflow::AttrValue value_epsilon;
 if (find_attr_value(node, "epsilon", value_epsilon)){
  float epsilon = value_epsilon.f();
  fprintf(pp, " 1=%f", epsilon);
 }
}

同理, Conv2dBackpropback其实就是ncnn里的反卷积操作, 只不过ncnn实现反卷积的操作和tensorflow内部实现反卷积的操作过程不一样, 但结果是一致的, 需要仿照普通卷积的写法加上去.

ncnn同样支持空洞卷积, 但无法识别tensorflow的空洞卷积, 具体原理可以看tensorflow空洞卷积的原理, tensorflow是改变featuremap做空洞卷积, 而ncnn是改变kernel做空洞卷积, 结果都一样. 需要对.proto文件修改即可完成空洞卷积.

总之ncnn对tensorflow的支持很不友好, 有的层还需要自己手动去实现, 还是很麻烦.

补充知识:pytorch模型转mxnet

介绍

gluon把mxnet再进行封装,封装的风格非常接近pytorch

使用gluon的好处是非常容易把pytorch模型向mxnet转化

唯一的问题是gluon封装还不成熟,封装好的layer不多,很多常用的layer 如concat,upsampling等layer都没有

这里关注如何把pytorch 模型快速转换成 mxnet基于symbol 和 exector设计的网络

pytorch转mxnet module

关键点:

mxnet 设计网络时symbol 名称要和pytorch初始化中各网络层名称对应

torch.load()读入pytorch模型checkpoint 字典,取当中的'state_dict'元素,也是一个字典

pytorch state_dict 字典中key是网络层参数的名称,val是参数ndarray

pytorch 的参数名称的组织形式和mxnet一样,但是连接符号不同,pytorch是'.',而mxnet是'_'比如:

pytorch '0.conv1.0.weight'
mxnet '0_conv1_0_weight'

pytorch 的参数array 和mxnet 的参数array 完全一样,只要名称对上,直接赋值即可初始化mxnet模型

需要做的有以下几点:

设计和pytorch网络对应的mxnet网络

加载pytorch checkpoint

调整pytorch checkpoint state_dict 的key名称和mxnet命名格式一致

FlowNet2S PytorchToMxnet

pytorch flownet2S 的checkpoint 可以在github上搜到

import mxnet as mx
from symbol_util import *
import pickle
 
def get_loss(data, label, loss_scale, name, get_input=False, is_sparse = False, type='stereo'):
 
 if type == 'stereo':
  data = mx.sym.Activation(data=data, act_type='relu',name=name+'relu')
 # loss
 if is_sparse:
  loss =mx.symbol.Custom(data=data, label=label, name=name, loss_scale= loss_scale, is_l1=True,
   op_type='SparseRegressionLoss')
 else:
  loss = mx.sym.MAERegressionOutput(data=data, label=label, name=name, grad_scale=loss_scale)
 return (loss,data) if get_input else loss
 
def flownet_s(loss_scale, is_sparse=False, name=''):
 img1 = mx.symbol.Variable('img1')
 img2 = mx.symbol.Variable('img2')
 data = mx.symbol.concat(img1,img2,dim=1)
 labels = {'loss{}'.format(i): mx.sym.Variable('loss{}_label'.format(i)) for i in range(0, 7)}
 # print('labels: ',labels)
 prediction = {}# a dict for loss collection
 loss = []#a list
 
 #normalize
 data = (data-125)/255
 
 # extract featrue
 conv1 = mx.sym.Convolution(data, pad=(3, 3), kernel=(7, 7), stride=(2, 2), num_filter=64, name=name + 'conv1_0')
 conv1 = mx.sym.LeakyReLU(data=conv1, act_type='leaky', slope=0.1)
 
 conv2 = mx.sym.Convolution(conv1, pad=(2, 2), kernel=(5, 5), stride=(2, 2), num_filter=128, name=name + 'conv2_0')
 conv2 = mx.sym.LeakyReLU(data=conv2, act_type='leaky', slope=0.1)
 
 conv3a = mx.sym.Convolution(conv2, pad=(2, 2), kernel=(5, 5), stride=(2, 2), num_filter=256, name=name + 'conv3_0')
 conv3a = mx.sym.LeakyReLU(data=conv3a, act_type='leaky', slope=0.1)
 
 conv3b = mx.sym.Convolution(conv3a, pad=(1, 1), kernel=(3, 3), stride=(1, 1), num_filter=256, name=name + 'conv3_1_0')
 conv3b = mx.sym.LeakyReLU(data=conv3b, act_type='leaky', slope=0.1)
 
 conv4a = mx.sym.Convolution(conv3b, pad=(1, 1), kernel=(3, 3), stride=(2, 2), num_filter=512, name=name + 'conv4_0')
 conv4a = mx.sym.LeakyReLU(data=conv4a, act_type='leaky', slope=0.1)
 
 conv4b = mx.sym.Convolution(conv4a, pad=(1, 1), kernel=(3, 3), stride=(1, 1), num_filter=512, name=name + 'conv4_1_0')
 conv4b = mx.sym.LeakyReLU(data=conv4b, act_type='leaky', slope=0.1)
 
 conv5a = mx.sym.Convolution(conv4b, pad=(1, 1), kernel=(3, 3), stride=(2, 2), num_filter=512, name=name + 'conv5_0')
 conv5a = mx.sym.LeakyReLU(data=conv5a, act_type='leaky', slope=0.1)
 
 conv5b = mx.sym.Convolution(conv5a, pad=(1, 1), kernel=(3, 3), stride=(1, 1), num_filter=512, name=name + 'conv5_1_0')
 conv5b = mx.sym.LeakyReLU(data=conv5b, act_type='leaky', slope=0.1)
 
 conv6a = mx.sym.Convolution(conv5b, pad=(1, 1), kernel=(3, 3), stride=(2, 2), num_filter=1024, name=name + 'conv6_0')
 conv6a = mx.sym.LeakyReLU(data=conv6a, act_type='leaky', slope=0.1)
 
 conv6b = mx.sym.Convolution(conv6a, pad=(1, 1), kernel=(3, 3), stride=(1, 1), num_filter=1024,
        name=name + 'conv6_1_0')
 conv6b = mx.sym.LeakyReLU(data=conv6b, act_type='leaky', slope=0.1, )
 
 #predict flow
 pr6 = mx.sym.Convolution(conv6b, pad=(1, 1), kernel=(3, 3), stride=(1, 1), num_filter=2,
        name=name + 'predict_flow6')
 prediction['loss6'] = pr6
 
 upsample_pr6to5 = mx.sym.Deconvolution(pr6, pad=(1, 1), kernel=(4, 4), stride=(2, 2), num_filter=2,
           name=name + 'upsampled_flow6_to_5', no_bias=True)
 upconv5 = mx.sym.Deconvolution(conv6b, pad=(1, 1), kernel=(4, 4), stride=(2, 2), num_filter=512,
         name=name + 'deconv5_0', no_bias=False)
 upconv5 = mx.sym.LeakyReLU(data=upconv5, act_type='leaky', slope=0.1)
 iconv5 = mx.sym.Concat(conv5b, upconv5, upsample_pr6to5, dim=1)
 
 
 pr5 = mx.sym.Convolution(iconv5, pad=(1, 1), kernel=(3, 3), stride=(1, 1), num_filter=2,
        name=name + 'predict_flow5')
 prediction['loss5'] = pr5
 
 upconv4 = mx.sym.Deconvolution(iconv5, pad=(1, 1), kernel=(4, 4), stride=(2, 2), num_filter=256,
         name=name + 'deconv4_0', no_bias=False)
 upconv4 = mx.sym.LeakyReLU(data=upconv4, act_type='leaky', slope=0.1)
 
 upsample_pr5to4 = mx.sym.Deconvolution(pr5, pad=(1, 1), kernel=(4, 4), stride=(2, 2), num_filter=2,
           name=name + 'upsampled_flow5_to_4', no_bias=True)
 
 iconv4 = mx.sym.Concat(conv4b, upconv4, upsample_pr5to4)
 
 pr4 = mx.sym.Convolution(iconv4, pad=(1, 1), kernel=(3, 3), stride=(1, 1), num_filter=2,
        name=name + 'predict_flow4')
 prediction['loss4'] = pr4
 
 upconv3 = mx.sym.Deconvolution(iconv4, pad=(1, 1), kernel=(4, 4), stride=(2, 2), num_filter=128,
         name=name + 'deconv3_0', no_bias=False)
 upconv3 = mx.sym.LeakyReLU(data=upconv3, act_type='leaky', slope=0.1)
 
 upsample_pr4to3 = mx.sym.Deconvolution(pr4, pad=(1, 1), kernel=(4, 4), stride=(2, 2), num_filter=2,
           name= name + 'upsampled_flow4_to_3', no_bias=True)
 iconv3 = mx.sym.Concat(conv3b, upconv3, upsample_pr4to3)
 
 pr3 = mx.sym.Convolution(iconv3, pad=(1, 1), kernel=(3, 3), stride=(1, 1), num_filter=2,
        name=name + 'predict_flow3')
 prediction['loss3'] = pr3
 
 upconv2 = mx.sym.Deconvolution(iconv3, pad=(1, 1), kernel=(4, 4), stride=(2, 2), num_filter=64,
         name=name + 'deconv2_0', no_bias=False)
 upconv2 = mx.sym.LeakyReLU(data=upconv2, act_type='leaky', slope=0.1)
 
 upsample_pr3to2 = mx.sym.Deconvolution(pr3, pad=(1, 1), kernel=(4, 4), stride=(2, 2), num_filter=2,
           name=name + 'upsampled_flow3_to_2', no_bias=True)
 iconv2 = mx.sym.Concat(conv2, upconv2, upsample_pr3to2)
 
 pr2 = mx.sym.Convolution(iconv2, pad=(1, 1), kernel=(3, 3), stride=(1, 1), num_filter=2,
        name=name + 'predict_flow2')
 prediction['loss2'] = pr2
 flow = mx.sym.UpSampling(arg0=pr2,scale=4,num_filter=2,num_args = 1,sample_type='nearest', name='upsample_flow2_to_1')
 # ignore the loss functions with loss scale of zero
 keys = loss_scale.keys()
 # keys.sort()
 #obtain the symbol of the losses
 for key in keys:
  # loss.append(get_loss(prediction[key] * 20, labels[key], loss_scale[key], name=key + name,get_input=False, is_sparse=is_sparse, type='flow'))
  loss.append(mx.sym.MAERegressionOutput(data=prediction[key] * 20, label=labels[key], name=key + name, grad_scale=loss_scale[key]))
 # print('loss: ',loss)
 #group 暂时不知道为嘛要group
 loss_group =mx.sym.Group(loss)
 # print('net: ',loss_group)
 return loss_group,flow
 
import gluonbook as gb
import torch
from utils.frame_utils import *
import numpy as np
if __name__ == '__main__':
 checkpoint = torch.load("C:/Users/junjie.huang/PycharmProjects/flownet2_mxnet/flownet2_pytorch/FlowNet2-S_checkpoint.pth.tar")
 # # checkpoint是一个字典
 print(isinstance(checkpoint['state_dict'], dict))
 # # 打印checkpoint字典中的key名
 print('keys of checkpoint:')
 for i in checkpoint:
  print(i)
 print('')
 # # pytorch 模型参数保存在一个key名为'state_dict'的元素中
 state_dict = checkpoint['state_dict']
 # # state_dict也是一个字典
 print('keys of state_dict:')
 for i in state_dict:
  print(i)
  # print(state_dict[i].size())
 print('')
 # print(state_dict)
 #字典的value是torch.tensor
 print(torch.is_tensor(state_dict['conv1.0.weight']))
 #查看某个value的size
 print(state_dict['conv1.0.weight'].size())
 
 #flownet-mxnet init
 loss_scale={'loss2': 1.00,
    'loss3': 1.00,
    'loss4': 1.00,
    'loss5': 1.00,
    'loss6': 1.00}
 loss,flow = flownet_s(loss_scale=loss_scale,is_sparse=False)
 print('loss information: ')
 print('loss:',loss)
 print('type:',type(loss))
 print('list_arguments:',loss.list_arguments())
 print('list_outputs:',loss.list_outputs())
 print('list_inputs:',loss.list_inputs())
 print('')
 
 print('flow information: ')
 print('flow:',flow)
 print('type:',type(flow))
 print('list_arguments:',flow.list_arguments())
 print('list_outputs:',flow.list_outputs())
 print('list_inputs:',flow.list_inputs())
 print('')
 name_mxnet = symbol.list_arguments()
 print(type(name_mxnet))
 for key in name_mxnet:
  print(key)
 
 name_mxnet.sort()
 for key in name_mxnet:
  print(key)
 print(name_mxnet)
 
 shapes = (1, 3, 384, 512)
 ctx = gb.try_gpu()
 # exe = symbol.simple_bind(ctx=ctx, img1=shapes,img2=shapes)
 exe = flow.simple_bind(ctx=ctx, img1=shapes, img2=shapes)
 print('exe type: ',type(exe))
 print('exe: ',exe)
 #module
 # mod = mx.mod.Module(flow)
 # print('mod type: ', type(exe))
 # print('mod: ', exe)
 
 pim1 = read_gen("C:/Users/junjie.huang/PycharmProjects/flownet2_mxnet/data/0000007-img0.ppm")
 pim2 = read_gen("C:/Users/junjie.huang/PycharmProjects/flownet2_mxnet/data/0000007-img1.ppm")
 print(pim1.shape)
 
 '''使用pytorch 的state_dict 初始化 mxnet 模型参数'''
 for key in state_dict:
  # print(type(key))
  k_split = key.split('.')
  key_mx = '_'.join(k_split)
  # print(key,key_mx)
  try:
   exe.arg_dict[key_mx][:]=state_dict[key].data
  except:
   print(key,exe.arg_dict[key_mx].shape,state_dict[key].data.shape)
 
 exe.arg_dict['img1'][:] = pim1[np.newaxis, :, :, :].transpose(0, 3, 1, 2).data
 exe.arg_dict['img2'][:] = pim2[np.newaxis, :, :, :].transpose(0, 3, 1, 2).data
 
 result = exe.forward()
 print('result: ',type(result))
 # for tmp in result:
 #  print(type(tmp))
 #  print(tmp.shape)
 # color = flow2color(exe.outputs[0].asnumpy()[0].transpose(1, 2, 0))
 outputs = exe.outputs
 print('output type: ',type(outputs))
 # for tmp in outputs:
 #  print(type(tmp))
 #  print(tmp.shape)
 
 #来自pytroch flownet2
 from visualize import flow2color
 # color = flow2color(exe.outputs[0].asnumpy()[0].transpose(1,2,0))
 flow_color = flow2color(exe.outputs[0].asnumpy()[0].transpose(1, 2, 0))
 print('color type:',type(flow_color))
 import matplotlib.pyplot as plt
 #来自pytorch
 from torchvision.transforms import ToPILImage
 TF = ToPILImage()
 images = TF(flow_color)
 images.show()
 # plt.imshow(color)

以上这篇tensorflow模型转ncnn的操作方式就是小编分享给大家的全部内容了,希望能给大家一个参考,也希望大家多多支持三水点靠木。

Python 相关文章推荐
python实现机械分词之逆向最大匹配算法代码示例
Dec 13 Python
Python爬虫实例_利用百度地图API批量获取城市所有的POI点
Jan 10 Python
Django处理文件上传File Uploads的实例
May 28 Python
python-str,list,set间的转换实例
Jun 27 Python
Python3获取拉勾网招聘信息的方法实例
Apr 03 Python
Django中使用 Closure Table 储存无限分级数据
Jun 06 Python
python3.5 cv2 获取视频特定帧生成jpg图片
Aug 28 Python
python django生成迁移文件的实例
Aug 31 Python
python的json中方法及jsonpath模块用法分析
Dec 06 Python
Python用K-means聚类算法进行客户分群的实现
Aug 23 Python
python实现定时发送邮件
Dec 23 Python
pandas统计重复值次数的方法实现
Feb 20 Python
MxNet预训练模型到Pytorch模型的转换方式
May 25 #Python
浅谈pytorch 模型 .pt, .pth, .pkl的区别及模型保存方式
May 25 #Python
Pytorch通过保存为ONNX模型转TensorRT5的实现
May 25 #Python
tensorflow pb to tflite 精度下降详解
May 25 #Python
Python HTMLTestRunner测试报告view按钮失效解决方案
May 25 #Python
python用opencv完成图像分割并进行目标物的提取
May 25 #Python
Pytorch转tflite方式
May 25 #Python
You might like
PHP使用数组实现队列
2012/02/05 PHP
利用php实现禁用IE和火狐的缓存问题
2012/12/03 PHP
php中0,null,empty,空,false,字符串关系的详细介绍
2013/06/20 PHP
PHP函数checkdnsrr用法详解(Windows平台用法)
2016/03/21 PHP
jQuery timers计时器简单应用说明
2010/10/28 Javascript
JavaScript中textRange对象使用方法小结
2015/03/24 Javascript
javascript中this的四种用法
2015/05/11 Javascript
jQuery操作Table技巧大汇总
2016/01/23 Javascript
JavaScript关于提高网站性能的几点建议(一)
2016/07/24 Javascript
Vue学习之路之登录注册实例代码
2017/07/06 Javascript
解决vue页面DOM操作不生效的问题
2018/03/17 Javascript
JS实现数组的增删改查操作示例
2018/08/29 Javascript
推荐一个基于Node.js的表单验证库
2019/02/15 Javascript
js中Function引用类型常见有用的方法和属性详解
2019/12/11 Javascript
Vue实现图片轮播组件思路及实例解析
2020/05/11 Javascript
使用Python3制作TCP端口扫描器
2017/04/17 Python
Python中表示字符串的三种方法
2017/09/06 Python
Python使用cx_Oracle调用Oracle存储过程的方法示例
2017/10/07 Python
python实现分页效果
2017/10/25 Python
Python网络编程详解
2017/10/31 Python
利用python为运维人员写一个监控脚本
2018/03/25 Python
python合并同类型excel表格的方法
2018/04/01 Python
Python输出\u编码将其转换成中文的实例
2018/12/15 Python
Django 创建新App及其常用命令的实现方法
2019/08/04 Python
Python的缺点和劣势分析
2019/11/19 Python
python 如何在测试中使用 Mock
2021/03/01 Python
Tripadvisor新西兰:阅读评论,比较价格和酒店预订
2018/02/10 全球购物
印刷工程专业应届生求职信
2013/09/29 职场文书
祖国在我心中演讲稿
2014/01/15 职场文书
关于抽烟的检讨书
2014/02/25 职场文书
迎新晚会主持词
2014/03/24 职场文书
篮球比赛策划方案
2014/06/05 职场文书
巾帼志愿者活动方案
2014/08/17 职场文书
2015年党员干部承诺书
2015/01/21 职场文书
JavaScript组合继承详解
2021/11/07 Javascript
CSS实现背景图片全屏铺满自适应的3种方式
2022/07/07 HTML / CSS