Python实现的递归神经网络简单示例


Posted in Python onAugust 11, 2017

本文实例讲述了Python实现的递归神经网络。分享给大家供大家参考,具体如下:

# Recurrent Neural Networks
import copy, numpy as np
np.random.seed(0)
# compute sigmoid nonlinearity
def sigmoid(x):
  output = 1/(1+np.exp(-x))
  return output
# convert output of sigmoid function to its derivative
def sigmoid_output_to_derivative(output):
  return output*(1-output)
# training dataset generation
int2binary = {}
binary_dim = 8
largest_number = pow(2,binary_dim)
binary = np.unpackbits(
  np.array([range(largest_number)],dtype=np.uint8).T,axis=1)
for i in range(largest_number):
  int2binary[i] = binary[i]
# input variables
alpha = 0.1
input_dim = 2
hidden_dim = 16
output_dim = 1
# initialize neural network weights
synapse_0 = 2*np.random.random((input_dim,hidden_dim)) - 1
synapse_1 = 2*np.random.random((hidden_dim,output_dim)) - 1
synapse_h = 2*np.random.random((hidden_dim,hidden_dim)) - 1
synapse_0_update = np.zeros_like(synapse_0)
synapse_1_update = np.zeros_like(synapse_1)
synapse_h_update = np.zeros_like(synapse_h)
# training logic
for j in range(10000):
  # generate a simple addition problem (a + b = c)
  a_int = np.random.randint(largest_number/2) # int version
  a = int2binary[a_int] # binary encoding
  b_int = np.random.randint(largest_number/2) # int version
  b = int2binary[b_int] # binary encoding
  # true answer
  c_int = a_int + b_int
  c = int2binary[c_int]
  # where we'll store our best guess (binary encoded)
  d = np.zeros_like(c)
  overallError = 0
  layer_2_deltas = list()
  layer_1_values = list()
  layer_1_values.append(np.zeros(hidden_dim))
  # moving along the positions in the binary encoding
  for position in range(binary_dim):
    # generate input and output
    X = np.array([[a[binary_dim - position - 1],b[binary_dim - position - 1]]])
    y = np.array([[c[binary_dim - position - 1]]]).T
    # hidden layer (input ~+ prev_hidden)
    layer_1 = sigmoid(np.dot(X,synapse_0) + np.dot(layer_1_values[-1],synapse_h))
    # output layer (new binary representation)
    layer_2 = sigmoid(np.dot(layer_1,synapse_1))
    # did we miss?... if so, by how much?
    layer_2_error = y - layer_2
    layer_2_deltas.append((layer_2_error)*sigmoid_output_to_derivative(layer_2))
    overallError += np.abs(layer_2_error[0])
    # decode estimate so we can print(it out)
    d[binary_dim - position - 1] = np.round(layer_2[0][0])
    # store hidden layer so we can use it in the next timestep
    layer_1_values.append(copy.deepcopy(layer_1))
  future_layer_1_delta = np.zeros(hidden_dim)
  for position in range(binary_dim):
    X = np.array([[a[position],b[position]]])
    layer_1 = layer_1_values[-position-1]
    prev_layer_1 = layer_1_values[-position-2]
    # error at output layer
    layer_2_delta = layer_2_deltas[-position-1]
    # error at hidden layer
    layer_1_delta = (future_layer_1_delta.dot(synapse_h.T) + layer_2_delta.dot(synapse_1.T)) * sigmoid_output_to_derivative(layer_1)
    # let's update all our weights so we can try again
    synapse_1_update += np.atleast_2d(layer_1).T.dot(layer_2_delta)
    synapse_h_update += np.atleast_2d(prev_layer_1).T.dot(layer_1_delta)
    synapse_0_update += X.T.dot(layer_1_delta)
    future_layer_1_delta = layer_1_delta
  synapse_0 += synapse_0_update * alpha
  synapse_1 += synapse_1_update * alpha
  synapse_h += synapse_h_update * alpha
  synapse_0_update *= 0
  synapse_1_update *= 0
  synapse_h_update *= 0
  # print(out progress)
  if j % 1000 == 0:
    print("Error:" + str(overallError))
    print("Pred:" + str(d))
    print("True:" + str(c))
    out = 0
    for index,x in enumerate(reversed(d)):
      out += x*pow(2,index)
    print(str(a_int) + " + " + str(b_int) + " = " + str(out))
    print("------------")

运行输出:

Error:[ 3.45638663]
Pred:[0 0 0 0 0 0 0 1]
True:[0 1 0 0 0 1 0 1]
9 + 60 = 1
------------
Error:[ 3.63389116]
Pred:[1 1 1 1 1 1 1 1]
True:[0 0 1 1 1 1 1 1]
28 + 35 = 255
------------
Error:[ 3.91366595]
Pred:[0 1 0 0 1 0 0 0]
True:[1 0 1 0 0 0 0 0]
116 + 44 = 72
------------
Error:[ 3.72191702]
Pred:[1 1 0 1 1 1 1 1]
True:[0 1 0 0 1 1 0 1]
4 + 73 = 223
------------
Error:[ 3.5852713]
Pred:[0 0 0 0 1 0 0 0]
True:[0 1 0 1 0 0 1 0]
71 + 11 = 8
------------
Error:[ 2.53352328]
Pred:[1 0 1 0 0 0 1 0]
True:[1 1 0 0 0 0 1 0]
81 + 113 = 162
------------
Error:[ 0.57691441]
Pred:[0 1 0 1 0 0 0 1]
True:[0 1 0 1 0 0 0 1]
81 + 0 = 81
------------
Error:[ 1.42589952]
Pred:[1 0 0 0 0 0 0 1]
True:[1 0 0 0 0 0 0 1]
4 + 125 = 129
------------
Error:[ 0.47477457]
Pred:[0 0 1 1 1 0 0 0]
True:[0 0 1 1 1 0 0 0]
39 + 17 = 56
------------
Error:[ 0.21595037]
Pred:[0 0 0 0 1 1 1 0]
True:[0 0 0 0 1 1 1 0]
11 + 3 = 14
------------

英文原文:https://iamtrask.github.io/2015/11/15/anyone-can-code-lstm/

希望本文所述对大家Python程序设计有所帮助。

Python 相关文章推荐
python网络编程学习笔记(二):socket建立网络客户端
Jun 09 Python
python中字典dict常用操作方法实例总结
Apr 04 Python
用Python中的wxPython实现最基本的浏览器功能
Apr 14 Python
Python存取XML的常见方法实例分析
Mar 21 Python
python 创建一个空dataframe 然后添加行数据的实例
Jun 07 Python
Python面向对象程序设计类的封装与继承用法示例
Apr 12 Python
PyQt5通信机制 信号与槽详解
Aug 07 Python
Python学习笔记之For循环用法详解
Aug 14 Python
PyCharm 2019.3发布增加了新功能一览
Dec 08 Python
解决Pycharm中恢复被exclude的项目问题(pycharm source root)
Feb 14 Python
numpy库ndarray多维数组的维度变换方法(reshape、resize、swapaxes、flatten)
Apr 28 Python
Python3 用matplotlib绘制sigmoid函数的案例
Dec 11 Python
Python调用系统底层API播放wav文件的方法
Aug 11 #Python
Django 导出 Excel 代码的实例详解
Aug 11 #Python
python技能之数据导出excel的实例代码
Aug 11 #Python
利用标准库fractions模块让Python支持分数类型的方法详解
Aug 11 #Python
Python对字符串实现去重操作的方法示例
Aug 11 #Python
python中模块查找的原理与方法详解
Aug 11 #Python
python利用lxml读写xml格式的文件
Aug 10 #Python
You might like
mysql5详细安装教程
2007/01/15 PHP
逐步提升php框架的性能
2008/01/10 PHP
php 清除网页病毒的方法
2008/12/05 PHP
PHP针对中英文混合字符串长度判断及截取方法示例
2017/03/31 PHP
Yii2框架自定义类统一处理url操作示例
2019/05/25 PHP
JS获取图片实际宽高及根据图片大小进行自适应
2013/08/11 Javascript
javascript中对Attr(dom中属性)的操作示例讲解
2013/12/02 Javascript
Egret引擎开发指南之创建项目
2014/09/03 Javascript
基于jquery实现表格内容筛选功能实例解析
2016/05/09 Javascript
针对BootStrap中tabs控件的美化和完善(推荐)
2016/07/06 Javascript
angular实现IM聊天图片发送实例
2017/05/08 Javascript
Nodejs调用Dll模块的方法
2018/09/17 NodeJs
简单了解Ajax表单序列化的实现方法
2019/06/14 Javascript
JS实现点击下拉列表文本框中出现对应的网址,点击跳转按钮实现跳转
2019/11/25 Javascript
使用 UniApp 实现小程序的微信登录功能
2020/06/09 Javascript
详解JavaScript类型判断的四种方法
2020/10/21 Javascript
[02:57]DOTA2英雄基础教程 风行者
2014/01/16 DOTA
[00:13]天涯墨客二技能展示
2018/08/25 DOTA
Python修改Excel数据的实例代码
2013/11/01 Python
python操作gmail实例
2015/01/14 Python
Python进阶篇之字典操作总结
2016/11/16 Python
Python get获取页面cookie代码实例
2018/09/12 Python
influx+grafana自定义python采集数据和一些坑的总结
2018/09/17 Python
Python常用的json标准库
2019/02/19 Python
Django应用程序入口WSGIHandler源码解析
2019/08/05 Python
如何基于Python实现自动扫雷
2020/01/06 Python
利用python+ffmpeg合并B站视频及格式转换的实例代码
2020/11/24 Python
手机配件第一品牌:ZAGG
2017/05/28 全球购物
Crocs欧洲官网:Crocs Europe
2020/01/14 全球购物
《与象共舞》教学反思
2014/02/24 职场文书
个人授权委托书样本
2014/09/13 职场文书
个人贷款收入证明
2014/10/26 职场文书
2015年银行客户经理工作总结
2015/04/01 职场文书
银行岗位培训心得体会
2016/01/09 职场文书
python 如何在list中找Topk的数值和索引
2021/05/20 Python
Golang 遍历二叉树
2022/04/19 Golang