详解python实现识别手写MNIST数字集的程序


Posted in Python onAugust 03, 2018

我们需要做的第⼀件事情是获取 MNIST 数据。如果你是⼀个 git ⽤⼾,那么你能够通过克隆这本书的代码仓库获得数据,实现我们的⽹络来分类数字

git clone https://github.com/mnielsen/neural-networks-and-deep-learning.git
class Network(object):
def __init__(self, sizes):
self.num_layers = len(sizes)
self.sizes = sizes
self.biases = [np.random.randn(y, 1) for y in sizes[1:]]
self.weights = [np.random.randn(y, x)
for x, y in zip(sizes[:-1], sizes[1:])]

在这段代码中,列表 sizes 包含各层神经元的数量。例如,如果我们想创建⼀个在第⼀层有2 个神经元,第⼆层有 3 个神经元,最后层有 1 个神经元的 Network 对象,我们应这样写代码:

net = Network([2, 3, 1])

Network 对象中的偏置和权重都是被随机初始化的,使⽤ Numpy 的 np.random.randn 函数来⽣成均值为 0,标准差为 1 的⾼斯分布。这样的随机初始化给了我们的随机梯度下降算法⼀个起点。在后⾯的章节中我们将会发现更好的初始化权重和偏置的⽅法,但是⽬前随机地将其初始化。注意 Network 初始化代码假设第⼀层神经元是⼀个输⼊层,并对这些神经元不设置任何偏置,因为偏置仅在后⾯的层中⽤于计算输出。有了这些,很容易写出从⼀个 Network 实例计算输出的代码。我们从定义 S 型函数开始:

def sigmoid(z):
return 1.0/(1.0+np.exp(-z))

注意,当输⼊ z 是⼀个向量或者 Numpy 数组时,Numpy ⾃动地按元素应⽤ sigmoid 函数,即以向量形式。

我们然后对 Network 类添加⼀个 feedforward ⽅法,对于⽹络给定⼀个输⼊ a,返回对应的输出 6 。这个⽅法所做的是对每⼀层应⽤⽅程 (22):

def feedforward(self, a):
"""Return the output of the network if "a" is input."""
for b, w in zip(self.biases, self.weights):
a = sigmoid(np.dot(w, a)+b)
return a

当然,我们想要 Network 对象做的主要事情是学习。为此我们给它们⼀个实现随即梯度下降算法的 SGD ⽅法。代码如下。其中⼀些地⽅看似有⼀点神秘,我会在代码后⾯逐个分析

def SGD(self, training_data, epochs, mini_batch_size, eta,
test_data=None):
"""Train the neural network using mini-batch stochastic
gradient descent. The "training_data" is a list of tuples
"(x, y)" representing the training inputs and the desired
outputs. The other non-optional parameters are
self-explanatory. If "test_data" is provided then the
network will be evaluated against the test data after each
epoch, and partial progress printed out. This is useful for
tracking progress, but slows things down substantially."""
if test_data: n_test = len(test_data)
n = len(training_data)
for j in xrange(epochs):
random.shuffle(training_data)
mini_batches = [
training_data[k:k+mini_batch_size]
for k in xrange(0, n, mini_batch_size)]
for mini_batch in mini_batches:
self.update_mini_batch(mini_batch, eta)
if test_data:
print "Epoch {0}: {1} / {2}".format(
j, self.evaluate(test_data), n_test)
else:
print "Epoch {0} complete".format(j)
 

training_data 是⼀个 (x, y) 元组的列表,表⽰训练输⼊和其对应的期望输出。变量 epochs 和mini_batch_size 正如你预料的——迭代期数量,和采样时的⼩批量数据的⼤⼩。 eta 是学习速率,η。如果给出了可选参数 test_data ,那么程序会在每个训练器后评估⽹络,并打印出部分进展。这对于追踪进度很有⽤,但相当拖慢执⾏速度。

在每个迭代期,它⾸先随机地将训练数据打乱,然后将它分成多个适当⼤⼩的⼩批量数据。这是⼀个简单的从训练数据的随机采样⽅法。然后对于每⼀个 mini_batch我们应⽤⼀次梯度下降。这是通过代码 self.update_mini_batch(mini_batch, eta) 完成的,它仅仅使⽤ mini_batch 中的训练数据,根据单次梯度下降的迭代更新⽹络的权重和偏置。这是update_mini_batch ⽅法的代码:

def update_mini_batch(self, mini_batch, eta):
"""Update the network's weights and biases by applying
gradient descent using backpropagation to a single mini batch.
The "mini_batch" is a list of tuples "(x, y)", and "eta"
is the learning rate."""
nabla_b = [np.zeros(b.shape) for b in self.biases]
nabla_w = [np.zeros(w.shape) for w in self.weights]
for x, y in mini_batch:
delta_nabla_b, delta_nabla_w = self.backprop(x, y)
nabla_b = [nb+dnb for nb, dnb in zip(nabla_b, delta_nabla_b)]
nabla_w = [nw+dnw for nw, dnw in zip(nabla_w, delta_nabla_w)]
self.weights = [w-(eta/len(mini_batch))*nw
for w, nw in zip(self.weights, nabla_w)]
self.biases = [b-(eta/len(mini_batch))*nb
for b, nb in zip(self.biases, nabla_b)]

⼤部分⼯作由这⾏代码完成:

delta_nabla_b, delta_nabla_w = self.backprop(x, y)

这⾏调⽤了⼀个称为反向传播的算法,⼀种快速计算代价函数的梯度的⽅法。因此update_mini_batch 的⼯作仅仅是对 mini_batch 中的每⼀个训练样本计算梯度,然后适当地更新 self.weights 和 self.biases 。我现在不会列出 self.backprop 的代码。我们将在下章中学习反向传播是怎样⼯作的,包括self.backprop 的代码。现在,就假设它按照我们要求的⼯作,返回与训练样本 x 相关代价的适当梯度

完整的程序

"""
network.py
~~~~~~~~~~
 
A module to implement the stochastic gradient descent learning
algorithm for a feedforward neural network. Gradients are calculated
using backpropagation. Note that I have focused on making the code
simple, easily readable, and easily modifiable. It is not optimized,
and omits many desirable features.
"""
 
#### Libraries
# Standard library
import random
 
# Third-party libraries
import numpy as np
 
class Network(object):
 
 def __init__(self, sizes):
  """The list ``sizes`` contains the number of neurons in the
  respective layers of the network. For example, if the list
  was [2, 3, 1] then it would be a three-layer network, with the
  first layer containing 2 neurons, the second layer 3 neurons,
  and the third layer 1 neuron. The biases and weights for the
  network are initialized randomly, using a Gaussian
  distribution with mean 0, and variance 1. Note that the first
  layer is assumed to be an input layer, and by convention we
  won't set any biases for those neurons, since biases are only
  ever used in computing the outputs from later layers."""
  self.num_layers = len(sizes)
  self.sizes = sizes
  self.biases = [np.random.randn(y, 1) for y in sizes[1:]]
  self.weights = [np.random.randn(y, x)
      for x, y in zip(sizes[:-1], sizes[1:])]
 
 def feedforward(self, a):
  """Return the output of the network if ``a`` is input."""
  for b, w in zip(self.biases, self.weights):
   a = sigmoid(np.dot(w, a)+b)
  return a
 
 def SGD(self, training_data, epochs, mini_batch_size, eta,
   test_data=None):
  """Train the neural network using mini-batch stochastic
  gradient descent. The ``training_data`` is a list of tuples
  ``(x, y)`` representing the training inputs and the desired
  outputs. The other non-optional parameters are
  self-explanatory. If ``test_data`` is provided then the
  network will be evaluated against the test data after each
  epoch, and partial progress printed out. This is useful for
  tracking progress, but slows things down substantially."""
  if test_data: n_test = len(test_data)
  n = len(training_data)
  for j in xrange(epochs):
   random.shuffle(training_data)
   mini_batches = [
    training_data[k:k+mini_batch_size]
    for k in xrange(0, n, mini_batch_size)]
   for mini_batch in mini_batches:
    self.update_mini_batch(mini_batch, eta)
   if test_data:
    print "Epoch {0}: {1} / {2}".format(
     j, self.evaluate(test_data), n_test)
   else:
    print "Epoch {0} complete".format(j)
 
 def update_mini_batch(self, mini_batch, eta):
  """Update the network's weights and biases by applying
  gradient descent using backpropagation to a single mini batch.
  The ``mini_batch`` is a list of tuples ``(x, y)``, and ``eta``
  is the learning rate."""
  nabla_b = [np.zeros(b.shape) for b in self.biases]
  nabla_w = [np.zeros(w.shape) for w in self.weights]
  for x, y in mini_batch:
   delta_nabla_b, delta_nabla_w = self.backprop(x, y)
   nabla_b = [nb+dnb for nb, dnb in zip(nabla_b, delta_nabla_b)]
   nabla_w = [nw+dnw for nw, dnw in zip(nabla_w, delta_nabla_w)]
  self.weights = [w-(eta/len(mini_batch))*nw
      for w, nw in zip(self.weights, nabla_w)]
  self.biases = [b-(eta/len(mini_batch))*nb
      for b, nb in zip(self.biases, nabla_b)]
 
 def backprop(self, x, y):
  """Return a tuple ``(nabla_b, nabla_w)`` representing the
  gradient for the cost function C_x. ``nabla_b`` and
  ``nabla_w`` are layer-by-layer lists of numpy arrays, similar
  to ``self.biases`` and ``self.weights``."""
  nabla_b = [np.zeros(b.shape) for b in self.biases]
  nabla_w = [np.zeros(w.shape) for w in self.weights]
  # feedforward
  activation = x
  activations = [x] # list to store all the activations, layer by layer
  zs = [] # list to store all the z vectors, layer by layer
  for b, w in zip(self.biases, self.weights):
   z = np.dot(w, activation)+b
   zs.append(z)
   activation = sigmoid(z)
   activations.append(activation)
  # backward pass
  delta = self.cost_derivative(activations[-1], y) * \
   sigmoid_prime(zs[-1])
  nabla_b[-1] = delta
  nabla_w[-1] = np.dot(delta, activations[-2].transpose())
  # Note that the variable l in the loop below is used a little
  # differently to the notation in Chapter 2 of the book. Here,
  # l = 1 means the last layer of neurons, l = 2 is the
  # second-last layer, and so on. It's a renumbering of the
  # scheme in the book, used here to take advantage of the fact
  # that Python can use negative indices in lists.
  for l in xrange(2, self.num_layers):
   z = zs[-l]
   sp = sigmoid_prime(z)
   delta = np.dot(self.weights[-l+1].transpose(), delta) * sp
   nabla_b[-l] = delta
   nabla_w[-l] = np.dot(delta, activations[-l-1].transpose())
  return (nabla_b, nabla_w)
 
 def evaluate(self, test_data):
  """Return the number of test inputs for which the neural
  network outputs the correct result. Note that the neural
  network's output is assumed to be the index of whichever
  neuron in the final layer has the highest activation."""
  test_results = [(np.argmax(self.feedforward(x)), y)
      for (x, y) in test_data]
  return sum(int(x == y) for (x, y) in test_results)
 
 def cost_derivative(self, output_activations, y):
  """Return the vector of partial derivatives \partial C_x /
  \partial a for the output activations."""
  return (output_activations-y)
 
#### Miscellaneous functions
def sigmoid(z):
 """The sigmoid function."""
 return 1.0/(1.0+np.exp(-z))
 
def sigmoid_prime(z):
 """Derivative of the sigmoid function."""
 return sigmoid(z)*(1-sigmoid(z))
"""
mnist_loader
~~~~~~~~~~~~
 
A library to load the MNIST image data. For details of the data
structures that are returned, see the doc strings for ``load_data``
and ``load_data_wrapper``. In practice, ``load_data_wrapper`` is the
function usually called by our neural network code.
"""
 
#### Libraries
# Standard library
import cPickle
import gzip
 
# Third-party libraries
import numpy as np
 
def load_data():
 """Return the MNIST data as a tuple containing the training data,
 the validation data, and the test data.
 
 The ``training_data`` is returned as a tuple with two entries.
 The first entry contains the actual training images. This is a
 numpy ndarray with 50,000 entries. Each entry is, in turn, a
 numpy ndarray with 784 values, representing the 28 * 28 = 784
 pixels in a single MNIST image.
 
 The second entry in the ``training_data`` tuple is a numpy ndarray
 containing 50,000 entries. Those entries are just the digit
 values (0...9) for the corresponding images contained in the first
 entry of the tuple.
 
 The ``validation_data`` and ``test_data`` are similar, except
 each contains only 10,000 images.
 
 This is a nice data format, but for use in neural networks it's
 helpful to modify the format of the ``training_data`` a little.
 That's done in the wrapper function ``load_data_wrapper()``, see
 below.
 """
 f = gzip.open('../data/mnist.pkl.gz', 'rb')
 training_data, validation_data, test_data = cPickle.load(f)
 f.close()
 return (training_data, validation_data, test_data)
 
def load_data_wrapper():
 """Return a tuple containing ``(training_data, validation_data,
 test_data)``. Based on ``load_data``, but the format is more
 convenient for use in our implementation of neural networks.
 
 In particular, ``training_data`` is a list containing 50,000
 2-tuples ``(x, y)``. ``x`` is a 784-dimensional numpy.ndarray
 containing the input image. ``y`` is a 10-dimensional
 numpy.ndarray representing the unit vector corresponding to the
 correct digit for ``x``.
 
 ``validation_data`` and ``test_data`` are lists containing 10,000
 2-tuples ``(x, y)``. In each case, ``x`` is a 784-dimensional
 numpy.ndarry containing the input image, and ``y`` is the
 corresponding classification, i.e., the digit values (integers)
 corresponding to ``x``.
 
 Obviously, this means we're using slightly different formats for
 the training data and the validation / test data. These formats
 turn out to be the most convenient for use in our neural network
 code."""
 tr_d, va_d, te_d = load_data()
 training_inputs = [np.reshape(x, (784, 1)) for x in tr_d[0]]
 training_results = [vectorized_result(y) for y in tr_d[1]]
 training_data = zip(training_inputs, training_results)
 validation_inputs = [np.reshape(x, (784, 1)) for x in va_d[0]]
 validation_data = zip(validation_inputs, va_d[1])
 test_inputs = [np.reshape(x, (784, 1)) for x in te_d[0]]
 test_data = zip(test_inputs, te_d[1])
 return (training_data, validation_data, test_data)
 
def vectorized_result(j):
 """Return a 10-dimensional unit vector with a 1.0 in the jth
 position and zeroes elsewhere. This is used to convert a digit
 (0...9) into a corresponding desired output from the neural
 network."""
 e = np.zeros((10, 1))
 e[j] = 1.0
 return e
# test network.py "cost function square func"
import mnist_loader
training_data, validation_data, test_data = mnist_loader.load_data_wrapper()
import network
net = network.Network([784, 10])
net.SGD(training_data, 5, 10, 5.0, test_data=test_data)

原英文查看:http://neuralnetworksanddeeplearning.com/chap1.html

以上就是本文的全部内容,希望对大家的学习有所帮助,也希望大家多多支持三水点靠木。

Python 相关文章推荐
python 正则式使用心得
May 07 Python
Python类的多重继承问题深入分析
Nov 09 Python
详解Python命令行解析工具Argparse
Apr 20 Python
用python写个自动SSH登录远程服务器的小工具(实例)
Jun 17 Python
Python中的self用法详解
Aug 06 Python
python实现实时视频流播放代码实例
Jan 11 Python
tensorflow 自定义损失函数示例代码
Feb 05 Python
python实例化对象的具体方法
Jun 17 Python
Python定义一个Actor任务
Jul 29 Python
Pycharm导入anaconda环境的教程图解
Jul 31 Python
基于Python的图像阈值化分割(迭代法)
Nov 20 Python
python 基于pygame实现俄罗斯方块
Mar 02 Python
opencv python 基于KNN的手写体识别的实例
Aug 03 #Python
Flask实现图片的上传、下载及展示示例代码
Aug 03 #Python
python使用opencv驱动摄像头的方法
Aug 03 #Python
python 简单照相机调用系统摄像头实现方法 pygame
Aug 03 #Python
Python判断字符串是否为字母或者数字(浮点数)的多种方法
Aug 03 #Python
OpenCV2从摄像头获取帧并写入视频文件的方法
Aug 03 #Python
python中copy()与deepcopy()的区别小结
Aug 03 #Python
You might like
mysql数据库差异比较的PHP代码
2012/02/05 PHP
PHP打开和关闭文件操作函数总结
2014/11/18 PHP
PHP实现文件上传和多文件上传
2015/12/24 PHP
PHP配置ZendOpcache插件加速
2019/02/14 PHP
PHP设计模式之 策略模式Strategy详解【对象行为型】
2020/05/01 PHP
关于javascript中this关键字(翻译+自我理解)
2010/10/20 Javascript
SyntaxHighlighter语法高亮插件使用说明
2011/08/14 Javascript
jquery多选项卡效果实例代码(附效果图)
2013/03/23 Javascript
JavaScript中split() 使用方法汇总
2015/04/17 Javascript
JQuery中属性过滤选择器用法实例分析
2015/05/18 Javascript
基于Bootstrap模态对话框只加载一次 remote 数据的解决方法
2017/07/09 Javascript
JS使用tofixed与round处理数据四舍五入的区别
2017/10/25 Javascript
vue-cli项目中使用公用的提示弹层tips或加载loading组件实例详解
2018/05/28 Javascript
koa2实现登录注册功能的示例代码
2018/12/03 Javascript
详解webpack 最简打包结果分析
2019/02/20 Javascript
如何使用jQuery操作Cookies方法解析
2020/09/08 jQuery
[50:38]DOTA2-DPC中国联赛 正赛 Phoenix vs CDEC BO3 第二场 3月7日
2021/03/11 DOTA
Python实现文件按照日期命名的方法
2015/07/09 Python
剖析Python的Twisted框架的核心特性
2016/05/25 Python
使用Python的Tornado框架实现一个Web端图书展示页面
2016/07/11 Python
python 实现在Excel末尾增加新行
2018/05/02 Python
python实现诗歌游戏(类继承)
2019/02/26 Python
Pycharm+django2.2+python3.6+MySQL实现简单的考试报名系统
2019/09/05 Python
python根据时间获取周数代码实例
2019/09/30 Python
python全局变量引用与修改过程解析
2020/01/07 Python
基于Keras的格式化输出Loss实现方式
2020/06/17 Python
利用Python优雅的登录校园网
2020/10/21 Python
结构工程个人自荐信范文
2013/11/30 职场文书
电气自动化个人求职信范文
2014/02/03 职场文书
端午节寄语2015
2015/03/23 职场文书
电影开国大典观后感
2015/06/04 职场文书
2015秋季开学典礼演讲稿
2015/07/16 职场文书
2016年小学“公民道德宣传日”活动总结
2016/04/01 职场文书
请假条应该怎么写?
2019/06/24 职场文书
vue首次渲染全过程
2021/04/21 Vue.js
python实现简易自习室座位预约系统
2021/06/30 Python