TensorFlow2.0使用keras训练模型的实现


Posted in Python onFebruary 20, 2021

1.一般的模型构造、训练、测试流程

# 模型构造
inputs = keras.Input(shape=(784,), name='mnist_input')
h1 = layers.Dense(64, activation='relu')(inputs)
h1 = layers.Dense(64, activation='relu')(h1)
outputs = layers.Dense(10, activation='softmax')(h1)
model = keras.Model(inputs, outputs)
# keras.utils.plot_model(model, 'net001.png', show_shapes=True)

model.compile(optimizer=keras.optimizers.RMSprop(),
    loss=keras.losses.SparseCategoricalCrossentropy(),
    metrics=[keras.metrics.SparseCategoricalAccuracy()])

# 载入数据
(x_train, y_train), (x_test, y_test) = keras.datasets.mnist.load_data()
x_train = x_train.reshape(60000, 784).astype('float32') /255
x_test = x_test.reshape(10000, 784).astype('float32') /255

x_val = x_train[-10000:]
y_val = y_train[-10000:]

x_train = x_train[:-10000]
y_train = y_train[:-10000]

# 训练模型
history = model.fit(x_train, y_train, batch_size=64, epochs=3,
   validation_data=(x_val, y_val))
print('history:')
print(history.history)

result = model.evaluate(x_test, y_test, batch_size=128)
print('evaluate:')
print(result)
pred = model.predict(x_test[:2])
print('predict:')
print(pred)

2.自定义损失和指标

自定义指标只需继承Metric类, 并重写一下函数

_init_(self),初始化。

update_state(self,y_true,y_pred,sample_weight = None),它使用目标y_true和模型预测y_pred来更新状态变量。

result(self),它使用状态变量来计算最终结果。

reset_states(self),重新初始化度量的状态。

# 这是一个简单的示例,显示如何实现CatgoricalTruePositives指标,该指标计算正确分类为属于给定类的样本数量

class CatgoricalTruePostives(keras.metrics.Metric):
 def __init__(self, name='binary_true_postives', **kwargs):
  super(CatgoricalTruePostives, self).__init__(name=name, **kwargs)
  self.true_postives = self.add_weight(name='tp', initializer='zeros')
  
 def update_state(self, y_true, y_pred, sample_weight=None):
  y_pred = tf.argmax(y_pred)
  y_true = tf.equal(tf.cast(y_pred, tf.int32), tf.cast(y_true, tf.int32))
  
  y_true = tf.cast(y_true, tf.float32)
  
  if sample_weight is not None:
   sample_weight = tf.cast(sample_weight, tf.float32)
   y_true = tf.multiply(sample_weight, y_true)
   
  return self.true_postives.assign_add(tf.reduce_sum(y_true))
 
 def result(self):
  return tf.identity(self.true_postives)
 
 def reset_states(self):
  self.true_postives.assign(0.)
  

model.compile(optimizer=keras.optimizers.RMSprop(1e-3),
    loss=keras.losses.SparseCategoricalCrossentropy(),
    metrics=[CatgoricalTruePostives()])

model.fit(x_train, y_train,
   batch_size=64, epochs=3)
# 以定义网络层的方式添加网络loss
class ActivityRegularizationLayer(layers.Layer):
 def call(self, inputs):
  self.add_loss(tf.reduce_sum(inputs) * 0.1)
  return inputs

inputs = keras.Input(shape=(784,), name='mnist_input')
h1 = layers.Dense(64, activation='relu')(inputs)
h1 = ActivityRegularizationLayer()(h1)
h1 = layers.Dense(64, activation='relu')(h1)
outputs = layers.Dense(10, activation='softmax')(h1)
model = keras.Model(inputs, outputs)
# keras.utils.plot_model(model, 'net001.png', show_shapes=True)

model.compile(optimizer=keras.optimizers.RMSprop(),
    loss=keras.losses.SparseCategoricalCrossentropy(),
    metrics=[keras.metrics.SparseCategoricalAccuracy()])
model.fit(x_train, y_train, batch_size=32, epochs=1)
# 也可以以定义网络层的方式添加要统计的metric
class MetricLoggingLayer(layers.Layer):
 def call(self, inputs):
  self.add_metric(keras.backend.std(inputs),
      name='std_of_activation',
      aggregation='mean')
  
  return inputs

inputs = keras.Input(shape=(784,), name='mnist_input')
h1 = layers.Dense(64, activation='relu')(inputs)
h1 = MetricLoggingLayer()(h1)
h1 = layers.Dense(64, activation='relu')(h1)
outputs = layers.Dense(10, activation='softmax')(h1)
model = keras.Model(inputs, outputs)
# keras.utils.plot_model(model, 'net001.png', show_shapes=True)

model.compile(optimizer=keras.optimizers.RMSprop(),
    loss=keras.losses.SparseCategoricalCrossentropy(),
    metrics=[keras.metrics.SparseCategoricalAccuracy()])
model.fit(x_train, y_train, batch_size=32, epochs=1)
# 也可以直接在model上面加
# 也可以以定义网络层的方式添加要统计的metric
class MetricLoggingLayer(layers.Layer):
 def call(self, inputs):
  self.add_metric(keras.backend.std(inputs),
      name='std_of_activation',
      aggregation='mean')
  
  return inputs

inputs = keras.Input(shape=(784,), name='mnist_input')
h1 = layers.Dense(64, activation='relu')(inputs)
h2 = layers.Dense(64, activation='relu')(h1)
outputs = layers.Dense(10, activation='softmax')(h2)
model = keras.Model(inputs, outputs)

model.add_metric(keras.backend.std(inputs),
      name='std_of_activation',
      aggregation='mean')
model.add_loss(tf.reduce_sum(h1)*0.1)

# keras.utils.plot_model(model, 'net001.png', show_shapes=True)

model.compile(optimizer=keras.optimizers.RMSprop(),
    loss=keras.losses.SparseCategoricalCrossentropy(),
    metrics=[keras.metrics.SparseCategoricalAccuracy()])
model.fit(x_train, y_train, batch_size=32, epochs=1)

处理使用validation_data传入测试数据,还可以使用validation_split划分验证数据

ps:validation_split只能在用numpy数据训练的情况下使用

model.fit(x_train, y_train, batch_size=32, epochs=1, validation_split=0.2)

3.使用tf.data构造数据

def get_compiled_model():
 inputs = keras.Input(shape=(784,), name='mnist_input')
 h1 = layers.Dense(64, activation='relu')(inputs)
 h2 = layers.Dense(64, activation='relu')(h1)
 outputs = layers.Dense(10, activation='softmax')(h2)
 model = keras.Model(inputs, outputs)
 model.compile(optimizer=keras.optimizers.RMSprop(),
     loss=keras.losses.SparseCategoricalCrossentropy(),
     metrics=[keras.metrics.SparseCategoricalAccuracy()])
 return model
model = get_compiled_model()
train_dataset = tf.data.Dataset.from_tensor_slices((x_train, y_train))
train_dataset = train_dataset.shuffle(buffer_size=1024).batch(64)

val_dataset = tf.data.Dataset.from_tensor_slices((x_val, y_val))
val_dataset = val_dataset.batch(64)

# model.fit(train_dataset, epochs=3)
# steps_per_epoch 每个epoch只训练几步
# validation_steps 每次验证,验证几步
model.fit(train_dataset, epochs=3, steps_per_epoch=100,
   validation_data=val_dataset, validation_steps=3)

4.样本权重和类权重

“样本权重”数组是一个数字数组,用于指定批处理中每个样本在计算总损失时应具有多少权重。 它通常用于不平衡的分类问题(这个想法是为了给予很少见的类更多的权重)。 当使用的权重是1和0时,该数组可以用作损失函数的掩码(完全丢弃某些样本对总损失的贡献)。

“类权重”dict是同一概念的更具体的实例:它将类索引映射到应该用于属于该类的样本的样本权重。 例如,如果类“0”比数据中的类“1”少两倍,则可以使用class_weight = {0:1.,1:0.5}。

# 增加第5类的权重
import numpy as np
# 样本权重
model = get_compiled_model()
class_weight = {i:1.0 for i in range(10)}
class_weight[5] = 2.0
print(class_weight)
model.fit(x_train, y_train,
   class_weight=class_weight,
   batch_size=64,
   epochs=4)
# 类权重
model = get_compiled_model()
sample_weight = np.ones(shape=(len(y_train),))
sample_weight[y_train == 5] = 2.0
model.fit(x_train, y_train,
   sample_weight=sample_weight,
   batch_size=64,
   epochs=4)
# tf.data数据
model = get_compiled_model()

sample_weight = np.ones(shape=(len(y_train),))
sample_weight[y_train == 5] = 2.0

train_dataset = tf.data.Dataset.from_tensor_slices((x_train, y_train,
             sample_weight))
train_dataset = train_dataset.shuffle(buffer_size=1024).batch(64)

val_dataset = tf.data.Dataset.from_tensor_slices((x_val, y_val))
val_dataset = val_dataset.batch(64)

model.fit(train_dataset, epochs=3, )

5.多输入多输出模型

image_input = keras.Input(shape=(32, 32, 3), name='img_input')
timeseries_input = keras.Input(shape=(None, 10), name='ts_input')

x1 = layers.Conv2D(3, 3)(image_input)
x1 = layers.GlobalMaxPooling2D()(x1)

x2 = layers.Conv1D(3, 3)(timeseries_input)
x2 = layers.GlobalMaxPooling1D()(x2)

x = layers.concatenate([x1, x2])

score_output = layers.Dense(1, name='score_output')(x)
class_output = layers.Dense(5, activation='softmax', name='class_output')(x)

model = keras.Model(inputs=[image_input, timeseries_input],
     outputs=[score_output, class_output])
keras.utils.plot_model(model, 'multi_input_output_model.png'
      , show_shapes=True)
# 可以为模型指定不同的loss和metrics
model.compile(
 optimizer=keras.optimizers.RMSprop(1e-3),
 loss=[keras.losses.MeanSquaredError(),
   keras.losses.CategoricalCrossentropy()])

# 还可以指定loss的权重
model.compile(
 optimizer=keras.optimizers.RMSprop(1e-3),
 loss={'score_output': keras.losses.MeanSquaredError(),
   'class_output': keras.losses.CategoricalCrossentropy()},
 metrics={'score_output': [keras.metrics.MeanAbsolutePercentageError(),
        keras.metrics.MeanAbsoluteError()],
    'class_output': [keras.metrics.CategoricalAccuracy()]},
 loss_weight={'score_output': 2., 'class_output': 1.})

# 可以把不需要传播的loss置0
model.compile(
 optimizer=keras.optimizers.RMSprop(1e-3),
 loss=[None, keras.losses.CategoricalCrossentropy()])

# Or dict loss version
model.compile(
 optimizer=keras.optimizers.RMSprop(1e-3),
 loss={'class_output': keras.losses.CategoricalCrossentropy()})

6.使用回 调

Keras中的回调是在训练期间(在epoch开始时,batch结束时,epoch结束时等)在不同点调用的对象,可用于实现以下行为:

  • 在培训期间的不同时间点进行验证(超出内置的每个时期验证)
  • 定期检查模型或超过某个精度阈值
  • 在训练似乎平稳时改变模型的学习率
  • 在训练似乎平稳时对顶层进行微调
  • 在培训结束或超出某个性能阈值时发送电子邮件或即时消息通知等等。

可使用的内置回调有

  • ModelCheckpoint:定期保存模型。
  • EarlyStopping:当训练不再改进验证指标时停止培训。
  • TensorBoard:定期编写可在TensorBoard中显示的模型日志(更多细节见“可视化”)。
  • CSVLogger:将丢失和指标数据流式传输到CSV文件。
  • 等等

6.1回调使用

model = get_compiled_model()

callbacks = [
 keras.callbacks.EarlyStopping(
  # Stop training when `val_loss` is no longer improving
  monitor='val_loss',
  # "no longer improving" being defined as "no better than 1e-2 less"
  min_delta=1e-2,
  # "no longer improving" being further defined as "for at least 2 epochs"
  patience=2,
  verbose=1)
]
model.fit(x_train, y_train,
   epochs=20,
   batch_size=64,
   callbacks=callbacks,
   validation_split=0.2)
# checkpoint模型回调
model = get_compiled_model()
check_callback = keras.callbacks.ModelCheckpoint(
 filepath='mymodel_{epoch}.h5',
 save_best_only=True,
 monitor='val_loss',
 verbose=1
)

model.fit(x_train, y_train,
   epochs=3,
   batch_size=64,
   callbacks=[check_callback],
   validation_split=0.2)
# 动态调整学习率
initial_learning_rate = 0.1
lr_schedule = keras.optimizers.schedules.ExponentialDecay(
 initial_learning_rate,
 decay_steps=10000,
 decay_rate=0.96,
 staircase=True
)
optimizer = keras.optimizers.RMSprop(learning_rate=lr_schedule)
# 使用tensorboard
tensorboard_cbk = keras.callbacks.TensorBoard(log_dir='./full_path_to_your_logs')
model.fit(x_train, y_train,
   epochs=5,
   batch_size=64,
   callbacks=[tensorboard_cbk],
   validation_split=0.2)

6.2创建自己的回调方法

class LossHistory(keras.callbacks.Callback):
 def on_train_begin(self, logs):
  self.losses = []
 def on_epoch_end(self, batch, logs):
  self.losses.append(logs.get('loss'))
  print('\nloss:',self.losses[-1])
  
model = get_compiled_model()

callbacks = [
 LossHistory()
]
model.fit(x_train, y_train,
   epochs=3,
   batch_size=64,
   callbacks=callbacks,
   validation_split=0.2)

7.自己构造训练和验证循环

# Get the model.
inputs = keras.Input(shape=(784,), name='digits')
x = layers.Dense(64, activation='relu', name='dense_1')(inputs)
x = layers.Dense(64, activation='relu', name='dense_2')(x)
outputs = layers.Dense(10, activation='softmax', name='predictions')(x)
model = keras.Model(inputs=inputs, outputs=outputs)

# Instantiate an optimizer.
optimizer = keras.optimizers.SGD(learning_rate=1e-3)
# Instantiate a loss function.
loss_fn = keras.losses.SparseCategoricalCrossentropy()

# Prepare the training dataset.
batch_size = 64
train_dataset = tf.data.Dataset.from_tensor_slices((x_train, y_train))
train_dataset = train_dataset.shuffle(buffer_size=1024).batch(batch_size)

# 自己构造循环
for epoch in range(3):
 print('epoch: ', epoch)
 for step, (x_batch_train, y_batch_train) in enumerate(train_dataset):
  # 开一个gradient tape, 计算梯度
  with tf.GradientTape() as tape:
   logits = model(x_batch_train)
   
   loss_value = loss_fn(y_batch_train, logits)
   grads = tape.gradient(loss_value, model.trainable_variables)
   optimizer.apply_gradients(zip(grads, model.trainable_variables))
   
  if step % 200 == 0:
   print('Training loss (for one batch) at step %s: %s' % (step, float(loss_value)))
   print('Seen so far: %s samples' % ((step + 1) * 64))
# 训练并验证
# Get model
inputs = keras.Input(shape=(784,), name='digits')
x = layers.Dense(64, activation='relu', name='dense_1')(inputs)
x = layers.Dense(64, activation='relu', name='dense_2')(x)
outputs = layers.Dense(10, activation='softmax', name='predictions')(x)
model = keras.Model(inputs=inputs, outputs=outputs)

# Instantiate an optimizer to train the model.
optimizer = keras.optimizers.SGD(learning_rate=1e-3)
# Instantiate a loss function.
loss_fn = keras.losses.SparseCategoricalCrossentropy()

# Prepare the metrics.
train_acc_metric = keras.metrics.SparseCategoricalAccuracy() 
val_acc_metric = keras.metrics.SparseCategoricalAccuracy()

# Prepare the training dataset.
batch_size = 64
train_dataset = tf.data.Dataset.from_tensor_slices((x_train, y_train))
train_dataset = train_dataset.shuffle(buffer_size=1024).batch(batch_size)

# Prepare the validation dataset.
val_dataset = tf.data.Dataset.from_tensor_slices((x_val, y_val))
val_dataset = val_dataset.batch(64)


# Iterate over epochs.
for epoch in range(3):
 print('Start of epoch %d' % (epoch,))
 
 # Iterate over the batches of the dataset.
 for step, (x_batch_train, y_batch_train) in enumerate(train_dataset):
 with tf.GradientTape() as tape:
  logits = model(x_batch_train)
  loss_value = loss_fn(y_batch_train, logits)
 grads = tape.gradient(loss_value, model.trainable_variables)
 optimizer.apply_gradients(zip(grads, model.trainable_variables))
  
 # Update training metric.
 train_acc_metric(y_batch_train, logits)

 # Log every 200 batches.
 if step % 200 == 0:
  print('Training loss (for one batch) at step %s: %s' % (step, float(loss_value)))
  print('Seen so far: %s samples' % ((step + 1) * 64))

 # Display metrics at the end of each epoch.
 train_acc = train_acc_metric.result()
 print('Training acc over epoch: %s' % (float(train_acc),))
 # Reset training metrics at the end of each epoch
 train_acc_metric.reset_states()

 # Run a validation loop at the end of each epoch.
 for x_batch_val, y_batch_val in val_dataset:
 val_logits = model(x_batch_val)
 # Update val metrics
 val_acc_metric(y_batch_val, val_logits)
 val_acc = val_acc_metric.result()
 val_acc_metric.reset_states()
 print('Validation acc: %s' % (float(val_acc),))
## 添加自己构造的loss, 每次只能看到最新一次训练增加的loss
class ActivityRegularizationLayer(layers.Layer):
 
 def call(self, inputs):
 self.add_loss(1e-2 * tf.reduce_sum(inputs))
 return inputs
 
inputs = keras.Input(shape=(784,), name='digits')
x = layers.Dense(64, activation='relu', name='dense_1')(inputs)
# Insert activity regularization as a layer
x = ActivityRegularizationLayer()(x)
x = layers.Dense(64, activation='relu', name='dense_2')(x)
outputs = layers.Dense(10, activation='softmax', name='predictions')(x)

model = keras.Model(inputs=inputs, outputs=outputs)
logits = model(x_train[:64])
print(model.losses)
logits = model(x_train[:64])
logits = model(x_train[64: 128])
logits = model(x_train[128: 192])
print(model.losses)
# 将loss添加进求导中
optimizer = keras.optimizers.SGD(learning_rate=1e-3)

for epoch in range(3):
 print('Start of epoch %d' % (epoch,))

 for step, (x_batch_train, y_batch_train) in enumerate(train_dataset):
 with tf.GradientTape() as tape:
  logits = model(x_batch_train)
  loss_value = loss_fn(y_batch_train, logits)

  # Add extra losses created during this forward pass:
  loss_value += sum(model.losses)
  
 grads = tape.gradient(loss_value, model.trainable_variables)
 optimizer.apply_gradients(zip(grads, model.trainable_variables))

 # Log every 200 batches.
 if step % 200 == 0:
  print('Training loss (for one batch) at step %s: %s' % (step, float(loss_value)))
  print('Seen so far: %s samples' % ((step + 1) * 64))

到此这篇关于TensorFlow2.0使用keras训练模型的实现的文章就介绍到这了,更多相关TensorFlow2.0 keras训练模型内容请搜索三水点靠木以前的文章或继续浏览下面的相关文章希望大家以后多多支持三水点靠木!

Python 相关文章推荐
python的几种开发工具介绍
Mar 07 Python
收集的几个Python小技巧分享
Nov 22 Python
Python 实现文件的全备份和差异备份详解
Dec 27 Python
Python实现的人工神经网络算法示例【基于反向传播算法】
Nov 11 Python
基于Python实现的ID3决策树功能示例
Jan 02 Python
python批量修改图片大小的方法
Jul 24 Python
Python日期时间Time模块实例详解
Apr 15 Python
Python3视频转字符动画的实例代码
Aug 29 Python
如何基于Python制作有道翻译小工具
Dec 16 Python
Django 拼接两个queryset 或是两个不可以相加的对象实例
Mar 28 Python
完美解决pyinstaller打包报错找不到依赖pypiwin32或pywin32-ctypes的错误
Apr 01 Python
pycharm远程连接服务器并配置python interpreter的方法
Dec 23 Python
tensorflow2.0教程之Keras快速入门
Feb 20 #Python
在Pycharm中安装Pandas库方法(简单易懂)
Feb 20 #Python
Python3爬虫RedisDump的安装步骤
Feb 20 #Python
python爬取2021猫眼票房字体加密实例
Feb 19 #Python
Python之Sklearn使用入门教程
Feb 19 #Python
Python爬虫UA伪装爬取的实例讲解
Feb 19 #Python
Pycharm制作搞怪弹窗的实现代码
Feb 19 #Python
You might like
基于mysql的论坛(3)
2006/10/09 PHP
PHP图片处理类 phpThumb参数用法介绍
2012/03/11 PHP
PHP实现RSA签名生成订单功能【支付宝示例】
2017/06/06 PHP
laravel框架添加数据,显示数据,返回成功值的方法
2019/10/11 PHP
JQuery实现的在新窗口打开链接的方法小结
2010/04/22 Javascript
基于jquery的放大镜效果
2012/05/30 Javascript
JavaScript italics方法入门实例(把字符串显示为斜体)
2014/10/17 Javascript
JavaScript之数组(Array)详解
2015/04/01 Javascript
JavaScript面向对象的实现方法小结
2015/04/14 Javascript
JQuery validate插件验证用户注册信息
2016/05/11 Javascript
js实现日历的简单算法
2017/01/24 Javascript
详解VUE中v-bind的基本用法
2017/07/13 Javascript
vue2.x 父组件监听子组件事件并传回信息的方法
2017/07/17 Javascript
js生成word中图片处理方法
2018/01/06 Javascript
Node.js 进程平滑离场剖析小结
2019/01/24 Javascript
vue2 拖动排序 vuedraggable组件的实现
2019/08/08 Javascript
js正则匹配多个全部数据问题
2019/12/20 Javascript
python处理二进制数据的方法
2015/06/03 Python
python实现嵌套列表平铺的两种方法
2018/11/08 Python
Python比较配置文件的方法实例详解
2019/06/06 Python
python3模拟实现xshell远程执行liunx命令的方法
2019/07/12 Python
django创建超级用户过程解析
2019/09/18 Python
Django 自动生成api接口文档教程
2019/11/19 Python
利用OpenCV和Python实现查找图片差异
2019/12/19 Python
python使用梯度下降算法实现一个多线性回归
2020/03/24 Python
python使用numpy中的size()函数实例用法详解
2021/01/29 Python
CSS3下的渐变文字效果实现示例
2018/03/02 HTML / CSS
Html5基于canvas实现电子签名并生成PDF文档
2020/12/07 HTML / CSS
PUMA官方商城:世界领先的运动品牌之一
2016/11/16 全球购物
沙滩主题婚礼活动策划方案
2014/09/15 职场文书
离婚协议书范本及离婚须知
2014/10/15 职场文书
感谢信格式范文
2015/01/22 职场文书
2015年驾驶员工作总结
2015/04/29 职场文书
导游词之晋城蟒河
2019/12/12 职场文书
JavaScript控制台的更多功能
2021/04/28 Javascript
使用Redis实现实时排行榜功能
2021/07/02 Redis