对Pytorch中nn.ModuleList 和 nn.Sequential详解


Posted in Python onAugust 18, 2019

简而言之就是,nn.Sequential类似于Keras中的贯序模型,它是Module的子类,在构建数个网络层之后会自动调用forward()方法,从而有网络模型生成。而nn.ModuleList仅仅类似于pytho中的list类型,只是将一系列层装入列表,并没有实现forward()方法,因此也不会有网络模型产生的副作用。

需要注意的是,nn.ModuleList接受的必须是subModule类型,例如:

nn.ModuleList(
      [nn.ModuleList([Conv(inp_dim + j * increase, oup_dim, 1, relu=False, bn=False) for j in range(5)]) for i in
       range(nstack)])

其中,二次嵌套的list内部也必须额外使用一个nn.ModuleList修饰实例化,否则会无法识别类型而报错!

摘录自

nn.ModuleList is just like a Python list. It was designed to store any desired number of nn.Module's. It may be useful, for instance, if you want to design a neural network whose number of layers is passed as input:

class LinearNet(nn.Module):
 def __init__(self, input_size, num_layers, layers_size, output_size):
   super(LinearNet, self).__init__()
 
   self.linears = nn.ModuleList([nn.Linear(input_size, layers_size)])
   self.linears.extend([nn.Linear(layers_size, layers_size) for i in range(1, self.num_layers-1)])
   self.linears.append(nn.Linear(layers_size, output_size)

nn.Sequential allows you to build a neural net by specifying sequentially the building blocks (nn.Module's) of that net. Here's an example:

class Flatten(nn.Module):
 def forward(self, x):
  N, C, H, W = x.size() # read in N, C, H, W
  return x.view(N, -1)
 
simple_cnn = nn.Sequential(
      nn.Conv2d(3, 32, kernel_size=7, stride=2),
      nn.ReLU(inplace=True),
      Flatten(), 
      nn.Linear(5408, 10),
     )

In nn.Sequential, the nn.Module's stored inside are connected in a cascaded way. For instance, in the example that I gave, I define a neural network that receives as input an image with 3 channels and outputs 10 neurons. That network is composed by the following blocks, in the following order: Conv2D -> ReLU -> Linear layer. Moreover, an object of type nn.Sequential has a forward() method, so if I have an input image x I can directly call y = simple_cnn(x) to obtain the scores for x. When you define an nn.Sequential you must be careful to make sure that the output size of a block matches the input size of the following block. Basically, it behaves just like a nn.Module

On the other hand, nn.ModuleList does not have a forward() method, because it does not define any neural network, that is, there is no connection between each of the nn.Module's that it stores. You may use it to store nn.Module's, just like you use Python lists to store other types of objects (integers, strings, etc). The advantage of using nn.ModuleList's instead of using conventional Python lists to store nn.Module's is that Pytorch is “aware” of the existence of the nn.Module's inside an nn.ModuleList, which is not the case for Python lists. If you want to understand exactly what I mean, just try to redefine my class LinearNet using a Python list instead of a nn.ModuleList and train it. When defining the optimizer() for that net, you'll get an error saying that your model has no parameters, because PyTorch does not see the parameters of the layers stored in a Python list. If you use a nn.ModuleList instead, you'll get no error.

以上这篇对Pytorch中nn.ModuleList 和 nn.Sequential详解就是小编分享给大家的全部内容了,希望能给大家一个参考,也希望大家多多支持三水点靠木。

Python 相关文章推荐
python处理文本文件实现生成指定格式文件的方法
Jul 31 Python
跟老齐学Python之深入变量和引用对象
Sep 24 Python
python类继承用法实例分析
Oct 10 Python
使用Python脚本来获取Cisco设备信息的示例
May 04 Python
基础的十进制按位运算总结与在Python中的计算示例
Jun 28 Python
python实现12306抢票及自动邮件发送提醒付款功能
Mar 08 Python
Python实现的凯撒密码算法示例
Apr 12 Python
浅谈pycharm的xmx和xms设置方法
Dec 03 Python
python使用Plotly绘图工具绘制柱状图
Apr 01 Python
Python autoescape标签用法解析
Jan 17 Python
Django如何批量创建Model
Sep 01 Python
python批量创建变量并赋值操作
Jun 03 Python
pytorch 自定义数据集加载方法
Aug 18 #Python
PyTorch的Optimizer训练工具的实现
Aug 18 #Python
Pytorch反向求导更新网络参数的方法
Aug 17 #Python
pytorch 模型可视化的例子
Aug 17 #Python
pytorch 输出中间层特征的实例
Aug 17 #Python
基于pytorch的保存和加载模型参数的方法
Aug 17 #Python
pytorch 固定部分参数训练的方法
Aug 17 #Python
You might like
PHP常用代码大全(新手入门必备)
2010/06/29 PHP
PHP setcookie指定domain参数后,在IE下设置cookie失效的解决方法
2011/09/09 PHP
php 生成唯一id的几种解决方法
2013/03/08 PHP
自写的利用PDO对mysql数据库增删改查操作类
2018/02/19 PHP
动态加载js文件 document.createElement
2006/10/14 Javascript
分析 JavaScript 中令人困惑的变量赋值
2007/08/13 Javascript
使弱类型的语言JavaScript变强势
2009/06/22 Javascript
解析Javascript中中括号“[]”的多义性
2013/12/03 Javascript
javascript引擎长时间独占线程造成卡顿的解决方案
2014/12/03 Javascript
node.js中的path.dirname方法使用说明
2014/12/09 Javascript
js检测用户输入密码强度
2015/10/22 Javascript
Javascript中级语法快速入手
2016/07/30 Javascript
微信小程序 wxapp内容组件 icon详细介绍
2016/10/31 Javascript
angularjs中ng-attr的用法详解
2016/12/31 Javascript
详解Javascript百度地图接口开发文档中的类和方法
2017/02/07 Javascript
基于vue的短信验证码倒计时demo
2017/09/13 Javascript
基于Vue、Vuex、Vue-router实现的购物商城(原生切换动画)效果
2018/01/09 Javascript
Vue实现简单分页器
2018/12/29 Javascript
小程序怎样让wx.navigateBack更好用的方法实现
2019/11/01 Javascript
python dataframe astype 字段类型转换方法
2018/04/11 Python
详解python中sort排序使用
2019/03/23 Python
10 行Python 代码实现 AI 目标检测技术【推荐】
2019/06/14 Python
Python 运行.py文件和交互式运行代码的区别详解
2019/07/02 Python
在python中实现调用可执行文件.exe的3种方法
2019/07/07 Python
python按比例随机切分数据的实现
2019/07/11 Python
h5移动端调用支付宝、微信支付的实现
2020/06/08 HTML / CSS
美国嘻哈首饰购物网站:Hip Hop Bling
2016/12/30 全球购物
应届生自我鉴定
2013/12/11 职场文书
企业环保标语
2014/06/10 职场文书
新农村建设标语
2014/06/24 职场文书
综合实践活动报告
2015/02/05 职场文书
个人自荐书怎么写
2015/03/26 职场文书
迎新生欢迎词2015
2015/07/16 职场文书
大学毕业生自我鉴定范文
2019/06/21 职场文书
python基础详解之if循环语句
2021/04/24 Python
【海涛教你打dota】体验一超神发条:咱是抢盾专业户
2022/04/01 DOTA