pytorch+lstm实现的pos示例


Posted in Python onJanuary 14, 2020

学了几天终于大概明白pytorch怎么用了

这个是直接搬运的官方文档的代码

之后会自己试着实现其他nlp的任务

# Author: Robert Guthrie

import torch
import torch.autograd as autograd
import torch.nn as nn
import torch.nn.functional as F
import torch.optim as optim

torch.manual_seed(1)


lstm = nn.LSTM(3, 3) # Input dim is 3, output dim is 3
inputs = [autograd.Variable(torch.randn((1, 3)))
     for _ in range(5)] # make a sequence of length 5

# initialize the hidden state.
hidden = (autograd.Variable(torch.randn(1, 1, 3)),
     autograd.Variable(torch.randn((1, 1, 3))))
for i in inputs:
  # Step through the sequence one element at a time.
  # after each step, hidden contains the hidden state.
  out, hidden = lstm(i.view(1, 1, -1), hidden)

# alternatively, we can do the entire sequence all at once.
# the first value returned by LSTM is all of the hidden states throughout
# the sequence. the second is just the most recent hidden state
# (compare the last slice of "out" with "hidden" below, they are the same)
# The reason for this is that:
# "out" will give you access to all hidden states in the sequence
# "hidden" will allow you to continue the sequence and backpropagate,
# by passing it as an argument to the lstm at a later time
# Add the extra 2nd dimension
inputs = torch.cat(inputs).view(len(inputs), 1, -1)
hidden = (autograd.Variable(torch.randn(1, 1, 3)), autograd.Variable(
  torch.randn((1, 1, 3)))) # clean out hidden state
out, hidden = lstm(inputs, hidden)
#print(out)
#print(hidden)

#准备数据
def prepare_sequence(seq, to_ix):
  idxs = [to_ix[w] for w in seq]
  tensor = torch.LongTensor(idxs)
  return autograd.Variable(tensor)

training_data = [
  ("The dog ate the apple".split(), ["DET", "NN", "V", "DET", "NN"]),
  ("Everybody read that book".split(), ["NN", "V", "DET", "NN"])
]
word_to_ix = {}
for sent, tags in training_data:
  for word in sent:
    if word not in word_to_ix:
      word_to_ix[word] = len(word_to_ix)
print(word_to_ix)
tag_to_ix = {"DET": 0, "NN": 1, "V": 2}

# These will usually be more like 32 or 64 dimensional.
# We will keep them small, so we can see how the weights change as we train.
EMBEDDING_DIM = 6
HIDDEN_DIM = 6

#继承自nn.module
class LSTMTagger(nn.Module):

  def __init__(self, embedding_dim, hidden_dim, vocab_size, tagset_size):
    super(LSTMTagger, self).__init__()
    self.hidden_dim = hidden_dim

    #一个单词数量到embedding维数的矩阵
    self.word_embeddings = nn.Embedding(vocab_size, embedding_dim)

    #传入两个维度参数
    # The LSTM takes word embeddings as inputs, and outputs hidden states
    # with dimensionality hidden_dim.
    self.lstm = nn.LSTM(embedding_dim, hidden_dim)

    #线性layer从隐藏状态空间映射到tag便签
    # The linear layer that maps from hidden state space to tag space
    self.hidden2tag = nn.Linear(hidden_dim, tagset_size)
    self.hidden = self.init_hidden()

  def init_hidden(self):
    # Before we've done anything, we dont have any hidden state.
    # Refer to the Pytorch documentation to see exactly
    # why they have this dimensionality.
    # The axes semantics are (num_layers, minibatch_size, hidden_dim)
    return (autograd.Variable(torch.zeros(1, 1, self.hidden_dim)),
        autograd.Variable(torch.zeros(1, 1, self.hidden_dim)))

  def forward(self, sentence):
    embeds = self.word_embeddings(sentence)
    lstm_out, self.hidden = self.lstm(embeds.view(len(sentence), 1, -1), self.hidden)
    tag_space = self.hidden2tag(lstm_out.view(len(sentence), -1))
    tag_scores = F.log_softmax(tag_space)
    return tag_scores

#embedding维度,hidden维度,词语数量,标签数量
model = LSTMTagger(EMBEDDING_DIM, HIDDEN_DIM, len(word_to_ix), len(tag_to_ix))

#optim中存了各种优化算法
loss_function = nn.NLLLoss()
optimizer = optim.SGD(model.parameters(), lr=0.1)

# See what the scores are before training
# Note that element i,j of the output is the score for tag j for word i.
inputs = prepare_sequence(training_data[0][0], word_to_ix)
tag_scores = model(inputs)
print(tag_scores)

for epoch in range(300): # again, normally you would NOT do 300 epochs, it is toy data
  for sentence, tags in training_data:
    # Step 1. Remember that Pytorch accumulates gradients.
    # We need to clear them out before each instance
    model.zero_grad()

    # Also, we need to clear out the hidden state of the LSTM,
    # detaching it from its history on the last instance.
    model.hidden = model.init_hidden()

    # Step 2. Get our inputs ready for the network, that is, turn them into
    # Variables of word indices.
    sentence_in = prepare_sequence(sentence, word_to_ix)
    targets = prepare_sequence(tags, tag_to_ix)

    # Step 3. Run our forward pass.
    tag_scores = model(sentence_in)

    # Step 4. Compute the loss, gradients, and update the parameters by
    # calling optimizer.step()
    loss = loss_function(tag_scores, targets)
    loss.backward()
    optimizer.step()

# See what the scores are after training
inputs = prepare_sequence(training_data[0][0], word_to_ix)
tag_scores = model(inputs)
# The sentence is "the dog ate the apple". i,j corresponds to score for tag j
# for word i. The predicted tag is the maximum scoring tag.
# Here, we can see the predicted sequence below is 0 1 2 0 1
# since 0 is index of the maximum value of row 1,
# 1 is the index of maximum value of row 2, etc.
# Which is DET NOUN VERB DET NOUN, the correct sequence!
print(tag_scores)

以上这篇pytorch+lstm实现的pos示例就是小编分享给大家的全部内容了,希望能给大家一个参考,也希望大家多多支持三水点靠木。

Python 相关文章推荐
windows下python之mysqldb模块安装方法
Sep 07 Python
Python实现判断一个字符串是否包含子串的方法总结
Nov 21 Python
python pandas dataframe 按列或者按行合并的方法
Apr 12 Python
解决Pycharm下面出现No R interpreter defined的问题
Oct 29 Python
python之pyqt5通过按钮改变Label的背景颜色方法
Jun 13 Python
python 列表、字典和集合的添加和删除操作
Dec 16 Python
Pytorch 神经网络—自定义数据集上实现教程
Jan 07 Python
Python使用Pandas库常见操作详解
Jan 16 Python
浅谈python之自动化运维(Paramiko)
Jan 31 Python
Python 分布式缓存之Reids数据类型操作详解
Jun 24 Python
python语音识别指南终极版(有这一篇足矣)
Sep 09 Python
python音频处理的示例详解
Dec 23 Python
Python中sorted()排序与字母大小写的问题
Jan 14 #Python
Pytorch实现LSTM和GRU示例
Jan 14 #Python
Python生成词云的实现代码
Jan 14 #Python
pytorch-RNN进行回归曲线预测方式
Jan 14 #Python
利用pytorch实现对CIFAR-10数据集的分类
Jan 14 #Python
pytorch下使用LSTM神经网络写诗实例
Jan 14 #Python
python使用openCV遍历文件夹里所有视频文件并保存成图片
Jan 14 #Python
You might like
十幅图告诉你什么是PHP引用
2015/02/22 PHP
PHP 微信支付类 demo
2015/11/30 PHP
php英文单词统计器
2016/06/23 PHP
php smtp实现发送邮件功能
2017/06/22 PHP
laravel通过创建自定义artisan make命令来新建类文件详解
2017/08/17 PHP
HR vs ForZe BO3 第一场 2.13
2021/03/10 DOTA
javascript 限制输入和粘贴(IE,firefox测试通过)
2008/11/14 Javascript
Date对象格式化函数代码
2010/07/17 Javascript
JavaScript插入动态样式实现代码
2012/02/22 Javascript
使用script的src实现跨域和类似ajax效果
2014/11/10 Javascript
javascript实现节点(div)名称编辑
2014/12/17 Javascript
JavaScript编写推箱子游戏
2015/07/07 Javascript
Angular实现可删除并计算总金额的购物车功能示例
2017/12/26 Javascript
Vue使用枚举类型实现HTML下拉框步骤详解
2018/02/05 Javascript
简单的三步vuex入门
2018/05/20 Javascript
基于element-ui的rules中正则表达式
2018/09/04 Javascript
JavaScript怎样在删除前添加确认弹出框?
2019/05/27 Javascript
ES6数组与对象的解构赋值详解
2019/06/14 Javascript
Vue.js中provide/inject实现响应式数据更新的方法示例
2019/10/16 Javascript
vue 避免变量赋值后双向绑定的操作
2020/11/07 Javascript
Python获取运行目录与当前脚本目录的方法
2015/06/01 Python
浅谈Python 字符串格式化输出(format/printf)
2016/07/21 Python
django开发教程之利用缓存文件进行页面缓存的方法
2017/11/10 Python
python文档字符串(函数使用说明)使用详解
2019/07/30 Python
Python Web框架之Django框架cookie和session用法分析
2019/08/16 Python
pandas实现导出数据的四种方式
2020/12/13 Python
联想台湾官网:Lenovo TW
2018/05/09 全球购物
世嘉游戏英国官方商店:SEGA Shop UK
2019/09/20 全球购物
DTD的含义以及作用
2014/01/26 面试题
技术总监的工作职责
2013/11/13 职场文书
2014学校庆三八妇女节活动总结
2014/03/01 职场文书
高中军训感言800字
2014/03/05 职场文书
2014年专项整治工作总结
2014/11/17 职场文书
2015年教师党员承诺书
2015/04/27 职场文书
幼儿园大班教学反思
2016/03/02 职场文书
Oracle 11g数据库使用expdp每周进行数据备份并上传到备份服务器
2022/06/28 Oracle