解决keras使用cov1D函数的输入问题


Posted in Python onJune 29, 2020

解决了以下错误:

1.ValueError: Input 0 is incompatible with layer conv1d_1: expected ndim=3, found ndim=4

2.ValueError: Error when checking target: expected dense_3 to have 3 dimensions, but got array with …

1.ValueError: Input 0 is incompatible with layer conv1d_1: expected ndim=3, found ndim=4

错误代码:

model.add(Conv1D(8, kernel_size=3, strides=1, padding='same', input_shape=(x_train.shape))

或者

model.add(Conv1D(8, kernel_size=3, strides=1, padding='same', input_shape=(x_train.shape[1:])))

这是因为模型输入的维数有误,在使用基于tensorflow的keras中,cov1d的input_shape是二维的,应该:

1、reshape x_train的形状

x_train=x_train.reshape((x_train.shape[0],x_train.shape[1],1))
x_test = x_test.reshape((x_test.shape[0], x_test.shape[1],1))

2、改变input_shape

model = Sequential()
model.add(Conv1D(8, kernel_size=3, strides=1, padding='same', input_shape=(x_train.shape[1],1)))

大神原文:

The input shape is wrong, it should be input_shape = (1, 3253) for Theano or (3253, 1) for TensorFlow. The input shape doesn't include the number of samples.

Then you need to reshape your data to include the channels axis:

x_train = x_train.reshape((500000, 1, 3253))

Or move the channels dimension to the end if you use TensorFlow. After these changes it should work.

2.ValueError: Error when checking target: expected dense_3 to have 3 dimensions, but got array with …

出现此问题是因为ylabel的维数与x_train x_test不符,既然将x_train x_test都reshape了,那么也需要对y进行reshape。

解决办法:

同时对照x_train改变ylabel的形状

t_train=t_train.reshape((t_train.shape[0],1))
t_test = t_test.reshape((t_test.shape[0],1))

附:

修改完的代码:

import warnings
warnings.filterwarnings("ignore")
import os
os.environ["CUDA_VISIBLE_DEVICES"] = "0"

import pandas as pd
import numpy as np
import matplotlib
# matplotlib.use('Agg')
import matplotlib.pyplot as plt

from sklearn.model_selection import train_test_split
from sklearn import preprocessing

from keras.models import Sequential
from keras.layers import Dense, Dropout, BatchNormalization, Activation, Flatten, Conv1D
from keras.callbacks import LearningRateScheduler, EarlyStopping, ModelCheckpoint, ReduceLROnPlateau
from keras import optimizers
from keras.regularizers import l2
from keras.models import load_model
df_train = pd.read_csv('./input/train_V2.csv')
df_test = pd.read_csv('./input/test_V2.csv')
df_train.drop(df_train.index[[2744604]],inplace=True)#去掉nan值
df_train["distance"] = df_train["rideDistance"]+df_train["walkDistance"]+df_train["swimDistance"]
# df_train["healthpack"] = df_train["boosts"] + df_train["heals"]
df_train["skill"] = df_train["headshotKills"]+df_train["roadKills"]
df_test["distance"] = df_test["rideDistance"]+df_test["walkDistance"]+df_test["swimDistance"]
# df_test["healthpack"] = df_test["boosts"] + df_test["heals"]
df_test["skill"] = df_test["headshotKills"]+df_test["roadKills"]

df_train_size = df_train.groupby(['matchId','groupId']).size().reset_index(name='group_size')
df_test_size = df_test.groupby(['matchId','groupId']).size().reset_index(name='group_size')

df_train_mean = df_train.groupby(['matchId','groupId']).mean().reset_index()
df_test_mean = df_test.groupby(['matchId','groupId']).mean().reset_index()

df_train = pd.merge(df_train, df_train_mean, suffixes=["", "_mean"], how='left', on=['matchId', 'groupId'])
df_test = pd.merge(df_test, df_test_mean, suffixes=["", "_mean"], how='left', on=['matchId', 'groupId'])
del df_train_mean
del df_test_mean

df_train = pd.merge(df_train, df_train_size, how='left', on=['matchId', 'groupId'])
df_test = pd.merge(df_test, df_test_size, how='left', on=['matchId', 'groupId'])
del df_train_size
del df_test_size

target = 'winPlacePerc'
train_columns = list(df_test.columns)
""" remove some columns """
train_columns.remove("Id")
train_columns.remove("matchId")
train_columns.remove("groupId")
train_columns_new = []
for name in train_columns:
 if '_' in name:
  train_columns_new.append(name)
train_columns = train_columns_new
# print(train_columns)

X = df_train[train_columns]
Y = df_test[train_columns]
T = df_train[target]

del df_train
x_train, x_test, t_train, t_test = train_test_split(X, T, test_size = 0.2, random_state = 1234)

# scaler = preprocessing.MinMaxScaler(feature_range=(-1, 1)).fit(x_train)
scaler = preprocessing.QuantileTransformer().fit(x_train)

x_train = scaler.transform(x_train)
x_test = scaler.transform(x_test)
Y = scaler.transform(Y)
x_train=x_train.reshape((x_train.shape[0],x_train.shape[1],1))
x_test = x_test.reshape((x_test.shape[0], x_test.shape[1],1))
t_train=t_train.reshape((t_train.shape[0],1))
t_test = t_test.reshape((t_test.shape[0],1))

model = Sequential()
model.add(Conv1D(8, kernel_size=3, strides=1, padding='same', input_shape=(x_train.shape[1],1)))
model.add(BatchNormalization())
model.add(Conv1D(8, kernel_size=3, strides=1, padding='same'))
model.add(Conv1D(16, kernel_size=3, strides=1, padding='valid'))
model.add(BatchNormalization())
model.add(Conv1D(16, kernel_size=3, strides=1, padding='same'))
model.add(Conv1D(32, kernel_size=3, strides=1, padding='valid'))
model.add(BatchNormalization())
model.add(Conv1D(32, kernel_size=3, strides=1, padding='same'))
model.add(Conv1D(32, kernel_size=3, strides=1, padding='same'))
model.add(Conv1D(64, kernel_size=3, strides=1, padding='same'))
model.add(Activation('tanh'))
model.add(Flatten())
model.add(Dropout(0.5))
# model.add(Dropout(0.25))
model.add(Dense(512,kernel_initializer='he_normal', activation='relu', W_regularizer=l2(0.01)))
model.add(Dense(128,kernel_initializer='he_normal', activation='relu', W_regularizer=l2(0.01)))
model.add(Dense(1, kernel_initializer='normal', activation='sigmoid'))

optimizers.Adam(lr=0.01, epsilon=1e-8, decay=1e-4)

model.compile(optimizer=optimizer, loss='mse', metrics=['mae'])
model.summary()

ng = EarlyStopping(monitor='val_mean_absolute_error', mode='min', patience=4, verbose=1)
# model_checkpoint = ModelCheckpoint(filepath='best_model.h5', monitor='val_mean_absolute_error', mode = 'min', save_best_only=True, verbose=1)
# reduce_lr = ReduceLROnPlateau(monitor='val_mean_absolute_error', mode = 'min',factor=0.5, patience=3, min_lr=0.0001, verbose=1)
history = model.fit(x_train, t_train,
     validation_data=(x_test, t_test),
     epochs=30,
     batch_size=32768,
     callbacks=[early_stopping],
     verbose=1)predict(Y)
pred = pred.ravel()

补充知识:Keras Conv1d 参数及输入输出详解

Conv1d(in_channels,out_channels,kernel_size,stride=1,padding=0,dilation=1,groups=1,bias=True)

filters:卷积核的数目(即输出的维度)

kernel_size: 整数或由单个整数构成的list/tuple,卷积核的空域或时域窗长度

strides: 整数或由单个整数构成的list/tuple,为卷积的步长。任何不为1的strides均为任何不为1的dilation_rata均不兼容

padding: 补0策略,为”valid”,”same”或”casual”,”casual”将产生因果(膨胀的)卷积,即output[t]不依赖于input[t+1:]。当对不能违反事件顺序的时序信号建模时有用。“valid”代表只进行有效的卷积,即对边界数据不处理。“same”代表保留边界处的卷积结果,通常会导致输出shape与输入shape相同。

activation:激活函数,为预定义的激活函数名,或逐元素的Theano函数。如果不指定该函数,将不会使用任何激活函数(即使用线性激活函数:a(x)=x)

model.add(Conv1D(filters=nn_params["input_filters"],
      kernel_size=nn_params["filter_length"],
      strides=1,
      padding='valid',
      activation=nn_params["activation"],
      kernel_regularizer=l2(nn_params["reg"])))

例:输入维度为(None,1000,4)

第一维度:None

第二维度:

output_length = int((input_length - nn_params["filter_length"] + 1))

在此情况下为:

output_length = (1000 + 2*padding - filters +1)/ strides = (1000 + 2*0 -32 +1)/1 = 969

第三维度:filters

以上这篇解决keras使用cov1D函数的输入问题就是小编分享给大家的全部内容了,希望能给大家一个参考,也希望大家多多支持三水点靠木。

Python 相关文章推荐
使用Python实现windows下的抓包与解析
Jan 15 Python
对pandas写入读取h5文件的方法详解
Dec 28 Python
对python函数签名的方法详解
Jan 22 Python
详解js文件通过python访问数据库方法
Mar 03 Python
Python调用graphviz绘制结构化图形网络示例
Nov 22 Python
Python 过滤错误log并导出的实例
Dec 26 Python
快速查找Python安装路径方法
Feb 06 Python
pytorch进行上采样的种类实例
Feb 18 Python
在python中使用nohup命令说明
Apr 16 Python
Python使用urlretrieve实现直接远程下载图片的示例代码
Aug 17 Python
python实现每天自动签到领积分的示例代码
Aug 18 Python
Python第三方库安装缓慢的解决方法
Feb 06 Python
快速了解Python开发环境Spyder
Jun 29 #Python
使用Keras构造简单的CNN网络实例
Jun 29 #Python
基于K.image_data_format() == 'channels_first' 的理解
Jun 29 #Python
Python enumerate() 函数如何实现索引功能
Jun 29 #Python
解决Keras中CNN输入维度报错问题
Jun 29 #Python
Python字符串split及rsplit方法原理详解
Jun 29 #Python
浅谈Keras参数 input_shape、input_dim和input_length用法
Jun 29 #Python
You might like
提升PHP执行速度全攻略(上)
2006/10/09 PHP
PHP实现表单提交时去除斜杠的方法
2016/12/26 PHP
php+js实现裁剪任意形状图片
2018/10/31 PHP
JS创建优美的页面滑动块效果 - Glider.js
2007/09/27 Javascript
js实现页面打印功能实例代码(附去页眉页脚功能代码)
2009/12/15 Javascript
IE6下focus与blur错乱的解决方案
2011/07/31 Javascript
给js文件传参数(详解)
2014/07/13 Javascript
jQuery滚动加载图片实现原理
2015/12/14 Javascript
浅谈js中字符和数组一些基本算法题
2016/08/15 Javascript
HTML5 canvas 9绘制图片实例详解
2016/09/06 Javascript
jQuery+CSS3实现四种应用广泛的导航条制作实例详解
2016/09/17 Javascript
JQuery PHP图片在线裁剪实例
2020/07/27 Javascript
bootstrap读书笔记之CSS组件(上)
2016/10/17 Javascript
jQuery 实现双击编辑表格功能
2017/06/19 jQuery
使用JavaScript实现在页面中显示距离2017年中秋节的天数
2017/09/26 Javascript
vue中使用codemirror的实例详解
2018/11/01 Javascript
NodeJs crypto加密制作token的实现代码
2019/11/15 NodeJs
Vue 列表页带参数进详情页的操作(router-link)
2020/11/13 Javascript
Djang中静态文件配置方法
2015/07/30 Python
解析Python中的eval()、exec()及其相关函数
2017/12/20 Python
django2+uwsgi+nginx上线部署到服务器Ubuntu16.04
2018/06/26 Python
实例分析python3实现并发访问水平切分表
2018/09/29 Python
解决pycharm 工具栏Tool中找不到Run manager.py Task的问题
2019/07/01 Python
Python爬虫实现“盗取”微信好友信息的方法分析
2019/09/16 Python
Python如何读取文件中图片格式
2020/01/13 Python
Python 列表的清空方式
2020/01/13 Python
python字符串下标与切片及使用方法
2020/02/13 Python
Python装饰器用法与知识点小结
2020/03/09 Python
python使用PIL剪切和拼接图片
2020/03/23 Python
python 3.8.3 安装配置图文教程
2020/05/21 Python
TripAdvisor日本:全球领先的旅游网站
2019/02/14 全球购物
某公司C#程序员面试题笔试题
2014/05/26 面试题
《水上飞机》教学反思
2014/04/10 职场文书
领导干部群众路线教育实践活动剖析材料
2014/10/10 职场文书
汽车4S店销售经理岗位职责
2015/04/02 职场文书
victoriaMetrics库布隆过滤器初始化及使用详解
2022/04/05 Golang