使用Keras建立模型并训练等一系列操作方式


Posted in Python onJuly 02, 2020

由于Keras是一种建立在已有深度学习框架上的二次框架,其使用起来非常方便,其后端实现有两种方法,theano和tensorflow。由于自己平时用tensorflow,所以选择后端用tensorflow的Keras,代码写起来更加方便。

1、建立模型

Keras分为两种不同的建模方式,

Sequential models:这种方法用于实现一些简单的模型。你只需要向一些存在的模型中添加层就行了。

Functional API:Keras的API是非常强大的,你可以利用这些API来构造更加复杂的模型,比如多输出模型,有向无环图等等。

这里采用sequential models方法。

构建序列模型。

def define_model():

  model = Sequential()

  # setup first conv layer
  model.add(Conv2D(32, (3, 3), activation="relu",
           input_shape=(120, 120, 3), padding='same')) # [10, 120, 120, 32]

  # setup first maxpooling layer
  model.add(MaxPooling2D(pool_size=(2, 2))) # [10, 60, 60, 32]

  # setup second conv layer
  model.add(Conv2D(8, kernel_size=(3, 3), activation="relu",
           padding='same')) # [10, 60, 60, 8]

  # setup second maxpooling layer
  model.add(MaxPooling2D(pool_size=(3, 3))) # [10, 20, 20, 8]

  # add bianping layer, 3200 = 20 * 20 * 8
  model.add(Flatten()) # [10, 3200]

  # add first full connection layer
  model.add(Dense(512, activation='sigmoid')) # [10, 512]

  # add dropout layer
  model.add(Dropout(0.5))

  # add second full connection layer
  model.add(Dense(4, activation='softmax')) # [10, 4]

  return model

可以看到定义模型时输出的网络结构。

使用Keras建立模型并训练等一系列操作方式

2、准备数据

def load_data(resultpath):
  datapath = os.path.join(resultpath, "data10_4.npz")
  if os.path.exists(datapath):
    data = np.load(datapath)
    X, Y = data["X"], data["Y"]
  else:
    X = np.array(np.arange(432000)).reshape(10, 120, 120, 3)
    Y = [0, 0, 1, 1, 2, 2, 3, 3, 2, 0]
    X = X.astype('float32')
    Y = np_utils.to_categorical(Y, 4)
    np.savez(datapath, X=X, Y=Y)
    print('Saved dataset to dataset.npz.')
  print('X_shape:{}\nY_shape:{}'.format(X.shape, Y.shape))
  return X, Y

使用Keras建立模型并训练等一系列操作方式

3、训练模型

def train_model(resultpath):
  model = define_model()

  # if want to use SGD, first define sgd, then set optimizer=sgd
  sgd = SGD(lr=0.001, decay=1e-6, momentum=0, nesterov=True)

  # select loss\optimizer\
  model.compile(loss=categorical_crossentropy,
         optimizer=Adam(), metrics=['accuracy'])
  model.summary()

  # draw the model structure
  plot_model(model, show_shapes=True,
        to_file=os.path.join(resultpath, 'model.png'))

  # load data
  X, Y = load_data(resultpath)

  # split train and test data
  X_train, X_test, Y_train, Y_test = train_test_split(
    X, Y, test_size=0.2, random_state=2)

  # input data to model and train
  history = model.fit(X_train, Y_train, batch_size=2, epochs=10,
            validation_data=(X_test, Y_test), verbose=1, shuffle=True)

  # evaluate the model
  loss, acc = model.evaluate(X_test, Y_test, verbose=0)
  print('Test loss:', loss)
  print('Test accuracy:', acc)

可以看到训练时输出的日志。因为是随机数据,没有意义,这里训练的结果不必计较,只是练习而已。

使用Keras建立模型并训练等一系列操作方式

保存下来的模型结构:

使用Keras建立模型并训练等一系列操作方式

4、保存与加载模型并测试

有两种保存方式

4.1 直接保存模型h5

保存:

def my_save_model(resultpath):

  model = train_model(resultpath)

  # the first way to save model
  model.save(os.path.join(resultpath, 'my_model.h5'))

加载:

def my_load_model(resultpath):

  # test data
  X = np.array(np.arange(86400)).reshape(2, 120, 120, 3)
  Y = [0, 1]
  X = X.astype('float32')
  Y = np_utils.to_categorical(Y, 4)

  # the first way of load model
  model2 = load_model(os.path.join(resultpath, 'my_model.h5'))
  model2.compile(loss=categorical_crossentropy,
         optimizer=Adam(), metrics=['accuracy'])

  test_loss, test_acc = model2.evaluate(X, Y, verbose=0)
  print('Test loss:', test_loss)
  print('Test accuracy:', test_acc)

  y = model2.predict_classes(X)
  print("predicct is: ", y)

使用Keras建立模型并训练等一系列操作方式

4.2 分别保存网络结构和权重

保存:

def my_save_model(resultpath):

  model = train_model(resultpath)

  # the secon way : save trained network structure and weights
  model_json = model.to_json()
  open(os.path.join(resultpath, 'my_model_structure.json'), 'w').write(model_json)
  model.save_weights(os.path.join(resultpath, 'my_model_weights.hd5'))

加载:

def my_load_model(resultpath):

  # test data
  X = np.array(np.arange(86400)).reshape(2, 120, 120, 3)
  Y = [0, 1]
  X = X.astype('float32')
  Y = np_utils.to_categorical(Y, 4)

  # the second way : load model structure and weights
  model = model_from_json(open(os.path.join(resultpath, 'my_model_structure.json')).read())
  model.load_weights(os.path.join(resultpath, 'my_model_weights.hd5'))
  model.compile(loss=categorical_crossentropy,
         optimizer=Adam(), metrics=['accuracy']) 

  test_loss, test_acc = model.evaluate(X, Y, verbose=0)
  print('Test loss:', test_loss)
  print('Test accuracy:', test_acc)

  y = model.predict_classes(X)
  print("predicct is: ", y)

使用Keras建立模型并训练等一系列操作方式

可以看到,两次的结果是一样的。

5、完整代码

from keras.models import Sequential
from keras.layers import Dense, Conv2D, MaxPooling2D, Flatten, Dropout
from keras.losses import categorical_crossentropy
from keras.optimizers import Adam
from keras.utils.vis_utils import plot_model
from keras.optimizers import SGD
from keras.models import model_from_json
from keras.models import load_model
from keras.utils import np_utils
import numpy as np
import os
from sklearn.model_selection import train_test_split

def load_data(resultpath):
  datapath = os.path.join(resultpath, "data10_4.npz")
  if os.path.exists(datapath):
    data = np.load(datapath)
    X, Y = data["X"], data["Y"]
  else:
    X = np.array(np.arange(432000)).reshape(10, 120, 120, 3)
    Y = [0, 0, 1, 1, 2, 2, 3, 3, 2, 0]
    X = X.astype('float32')
    Y = np_utils.to_categorical(Y, 4)
    np.savez(datapath, X=X, Y=Y)
    print('Saved dataset to dataset.npz.')
  print('X_shape:{}\nY_shape:{}'.format(X.shape, Y.shape))
  return X, Y

def define_model():
  model = Sequential()

  # setup first conv layer
  model.add(Conv2D(32, (3, 3), activation="relu",
           input_shape=(120, 120, 3), padding='same')) # [10, 120, 120, 32]

  # setup first maxpooling layer
  model.add(MaxPooling2D(pool_size=(2, 2))) # [10, 60, 60, 32]

  # setup second conv layer
  model.add(Conv2D(8, kernel_size=(3, 3), activation="relu",
           padding='same')) # [10, 60, 60, 8]

  # setup second maxpooling layer
  model.add(MaxPooling2D(pool_size=(3, 3))) # [10, 20, 20, 8]

  # add bianping layer, 3200 = 20 * 20 * 8
  model.add(Flatten()) # [10, 3200]

  # add first full connection layer
  model.add(Dense(512, activation='sigmoid')) # [10, 512]

  # add dropout layer
  model.add(Dropout(0.5))

  # add second full connection layer
  model.add(Dense(4, activation='softmax')) # [10, 4]

  return model

def train_model(resultpath):
  model = define_model()

  # if want to use SGD, first define sgd, then set optimizer=sgd
  sgd = SGD(lr=0.001, decay=1e-6, momentum=0, nesterov=True)

  # select loss\optimizer\
  model.compile(loss=categorical_crossentropy,
         optimizer=Adam(), metrics=['accuracy'])
  model.summary()

  # draw the model structure
  plot_model(model, show_shapes=True,
        to_file=os.path.join(resultpath, 'model.png'))

  # load data
  X, Y = load_data(resultpath)

  # split train and test data
  X_train, X_test, Y_train, Y_test = train_test_split(
    X, Y, test_size=0.2, random_state=2)

  # input data to model and train
  history = model.fit(X_train, Y_train, batch_size=2, epochs=10,
            validation_data=(X_test, Y_test), verbose=1, shuffle=True)

  # evaluate the model
  loss, acc = model.evaluate(X_test, Y_test, verbose=0)
  print('Test loss:', loss)
  print('Test accuracy:', acc)

  return model

def my_save_model(resultpath):

  model = train_model(resultpath)

  # the first way to save model
  model.save(os.path.join(resultpath, 'my_model.h5'))

  # the secon way : save trained network structure and weights
  model_json = model.to_json()
  open(os.path.join(resultpath, 'my_model_structure.json'), 'w').write(model_json)
  model.save_weights(os.path.join(resultpath, 'my_model_weights.hd5'))

def my_load_model(resultpath):

  # test data
  X = np.array(np.arange(86400)).reshape(2, 120, 120, 3)
  Y = [0, 1]
  X = X.astype('float32')
  Y = np_utils.to_categorical(Y, 4)

  # the first way of load model
  model2 = load_model(os.path.join(resultpath, 'my_model.h5'))
  model2.compile(loss=categorical_crossentropy,
          optimizer=Adam(), metrics=['accuracy'])

  test_loss, test_acc = model2.evaluate(X, Y, verbose=0)
  print('Test loss:', test_loss)
  print('Test accuracy:', test_acc)

  y = model2.predict_classes(X)
  print("predicct is: ", y)

  # the second way : load model structure and weights
  model = model_from_json(open(os.path.join(resultpath, 'my_model_structure.json')).read())
  model.load_weights(os.path.join(resultpath, 'my_model_weights.hd5'))
  model.compile(loss=categorical_crossentropy,
         optimizer=Adam(), metrics=['accuracy'])

  test_loss, test_acc = model.evaluate(X, Y, verbose=0)
  print('Test loss:', test_loss)
  print('Test accuracy:', test_acc)

  y = model.predict_classes(X)
  print("predicct is: ", y)

def main():
  resultpath = "result"
  #train_model(resultpath)
  #my_save_model(resultpath)
  my_load_model(resultpath)


if __name__ == "__main__":
  main()

以上这篇使用Keras建立模型并训练等一系列操作方式就是小编分享给大家的全部内容了,希望能给大家一个参考,也希望大家多多支持三水点靠木。

Python 相关文章推荐
python根据文件大小打log日志
Oct 09 Python
全面了解Python环境配置及项目建立
Jun 30 Python
python 循环遍历字典元素的简单方法
Sep 11 Python
python实现画圆功能
Jan 25 Python
Python设置在shell脚本中自动补全功能的方法
Jun 25 Python
numpy添加新的维度:newaxis的方法
Aug 02 Python
python远程连接MySQL数据库
Apr 19 Python
pandas 数据索引与选取的实现方法
Jun 21 Python
Python使用gluon/mxnet模块实现的mnist手写数字识别功能完整示例
Dec 18 Python
kafka监控获取指定topic的消息总量示例
Dec 23 Python
python 生成器需注意的小问题
Sep 29 Python
Python timeit模块原理及使用方法
Oct 10 Python
python解释器安装教程的方法步骤
Jul 02 #Python
Python分析最近大火的网剧《隐秘的角落》
Jul 02 #Python
keras训练浅层卷积网络并保存和加载模型实例
Jul 02 #Python
Python RabbitMQ实现简单的进程间通信示例
Jul 02 #Python
利用scikitlearn画ROC曲线实例
Jul 02 #Python
Python使用文件操作实现一个XX信息管理系统的示例
Jul 02 #Python
keras用auc做metrics以及早停实例
Jul 02 #Python
You might like
发一个php简单的伪原创程序,配合商城采集用的
2010/10/12 PHP
php中{}大括号是什么意思
2013/12/01 PHP
destoon之一键登录设置
2014/06/21 PHP
php中实现获取随机数组列表的自定义函数
2015/04/02 PHP
PHP文件操作方法汇总
2015/07/01 PHP
php上传图片生成缩略图(GD库)
2016/01/06 PHP
PHP框架自动加载类文件原理详解
2017/06/06 PHP
PHP实现随机数字、字母的验证码功能
2018/08/01 PHP
javascript笔试题目附答案@20081025_jb51.net
2008/10/26 Javascript
JS 页面计时器示例代码
2013/10/28 Javascript
javascript 中that的含义示例介绍
2014/05/14 Javascript
JavaScript代码性能优化总结(推荐)
2016/05/16 Javascript
AngularJS Phonecat实例讲解
2016/11/21 Javascript
详解jQuery简单的表单应用
2016/12/16 Javascript
详解基于Angular4+ server render(服务端渲染)开发教程
2017/08/28 Javascript
jQuery实现注册会员时密码强度提示信息功能示例
2017/09/05 jQuery
JS中跳出循环的示例代码
2017/09/14 Javascript
解决Vue打包之后文件路径出错的问题
2018/03/06 Javascript
Koa日志中间件封装开发详解
2019/03/09 Javascript
详解vue-flickity的fullScreen功能实现
2020/04/07 Javascript
[49:21]完美世界DOTA2联赛循环赛 Ink Ice vs LBZS BO2第二场 11.05
2020/11/06 DOTA
python自动zip压缩目录的方法
2015/06/28 Python
Python学习小技巧之列表项的拼接
2017/05/20 Python
Python cookbook(数据结构与算法)将多个映射合并为单个映射的方法
2018/04/19 Python
python3解析库pyquery的深入讲解
2018/06/26 Python
解决pyttsx3无法封装的问题
2018/12/24 Python
Python  word实现读取及导出代码解析
2020/07/09 Python
印度购买眼镜和太阳镜网站:Coolwinks
2018/09/26 全球购物
波兰最大的度假胜地和城市公寓租赁运营商:Sun & Snow
2018/10/18 全球购物
年终奖发放方案
2014/06/02 职场文书
运动员口号
2014/06/09 职场文书
革命英雄事迹演讲稿
2014/09/13 职场文书
关于运动会的广播稿
2014/09/22 职场文书
2015年办公室工作总结范文
2015/03/31 职场文书
2015年度公共机构节能工作总结
2015/05/26 职场文书
python代码实现备忘录案例讲解
2021/07/26 Python