解决keras使用cov1D函数的输入问题


Posted in Python onJune 29, 2020

解决了以下错误:

1.ValueError: Input 0 is incompatible with layer conv1d_1: expected ndim=3, found ndim=4

2.ValueError: Error when checking target: expected dense_3 to have 3 dimensions, but got array with …

1.ValueError: Input 0 is incompatible with layer conv1d_1: expected ndim=3, found ndim=4

错误代码:

model.add(Conv1D(8, kernel_size=3, strides=1, padding='same', input_shape=(x_train.shape))

或者

model.add(Conv1D(8, kernel_size=3, strides=1, padding='same', input_shape=(x_train.shape[1:])))

这是因为模型输入的维数有误,在使用基于tensorflow的keras中,cov1d的input_shape是二维的,应该:

1、reshape x_train的形状

x_train=x_train.reshape((x_train.shape[0],x_train.shape[1],1))
x_test = x_test.reshape((x_test.shape[0], x_test.shape[1],1))

2、改变input_shape

model = Sequential()
model.add(Conv1D(8, kernel_size=3, strides=1, padding='same', input_shape=(x_train.shape[1],1)))

大神原文:

The input shape is wrong, it should be input_shape = (1, 3253) for Theano or (3253, 1) for TensorFlow. The input shape doesn't include the number of samples.

Then you need to reshape your data to include the channels axis:

x_train = x_train.reshape((500000, 1, 3253))

Or move the channels dimension to the end if you use TensorFlow. After these changes it should work.

2.ValueError: Error when checking target: expected dense_3 to have 3 dimensions, but got array with …

出现此问题是因为ylabel的维数与x_train x_test不符,既然将x_train x_test都reshape了,那么也需要对y进行reshape。

解决办法:

同时对照x_train改变ylabel的形状

t_train=t_train.reshape((t_train.shape[0],1))
t_test = t_test.reshape((t_test.shape[0],1))

附:

修改完的代码:

import warnings
warnings.filterwarnings("ignore")
import os
os.environ["CUDA_VISIBLE_DEVICES"] = "0"

import pandas as pd
import numpy as np
import matplotlib
# matplotlib.use('Agg')
import matplotlib.pyplot as plt

from sklearn.model_selection import train_test_split
from sklearn import preprocessing

from keras.models import Sequential
from keras.layers import Dense, Dropout, BatchNormalization, Activation, Flatten, Conv1D
from keras.callbacks import LearningRateScheduler, EarlyStopping, ModelCheckpoint, ReduceLROnPlateau
from keras import optimizers
from keras.regularizers import l2
from keras.models import load_model
df_train = pd.read_csv('./input/train_V2.csv')
df_test = pd.read_csv('./input/test_V2.csv')
df_train.drop(df_train.index[[2744604]],inplace=True)#去掉nan值
df_train["distance"] = df_train["rideDistance"]+df_train["walkDistance"]+df_train["swimDistance"]
# df_train["healthpack"] = df_train["boosts"] + df_train["heals"]
df_train["skill"] = df_train["headshotKills"]+df_train["roadKills"]
df_test["distance"] = df_test["rideDistance"]+df_test["walkDistance"]+df_test["swimDistance"]
# df_test["healthpack"] = df_test["boosts"] + df_test["heals"]
df_test["skill"] = df_test["headshotKills"]+df_test["roadKills"]

df_train_size = df_train.groupby(['matchId','groupId']).size().reset_index(name='group_size')
df_test_size = df_test.groupby(['matchId','groupId']).size().reset_index(name='group_size')

df_train_mean = df_train.groupby(['matchId','groupId']).mean().reset_index()
df_test_mean = df_test.groupby(['matchId','groupId']).mean().reset_index()

df_train = pd.merge(df_train, df_train_mean, suffixes=["", "_mean"], how='left', on=['matchId', 'groupId'])
df_test = pd.merge(df_test, df_test_mean, suffixes=["", "_mean"], how='left', on=['matchId', 'groupId'])
del df_train_mean
del df_test_mean

df_train = pd.merge(df_train, df_train_size, how='left', on=['matchId', 'groupId'])
df_test = pd.merge(df_test, df_test_size, how='left', on=['matchId', 'groupId'])
del df_train_size
del df_test_size

target = 'winPlacePerc'
train_columns = list(df_test.columns)
""" remove some columns """
train_columns.remove("Id")
train_columns.remove("matchId")
train_columns.remove("groupId")
train_columns_new = []
for name in train_columns:
 if '_' in name:
  train_columns_new.append(name)
train_columns = train_columns_new
# print(train_columns)

X = df_train[train_columns]
Y = df_test[train_columns]
T = df_train[target]

del df_train
x_train, x_test, t_train, t_test = train_test_split(X, T, test_size = 0.2, random_state = 1234)

# scaler = preprocessing.MinMaxScaler(feature_range=(-1, 1)).fit(x_train)
scaler = preprocessing.QuantileTransformer().fit(x_train)

x_train = scaler.transform(x_train)
x_test = scaler.transform(x_test)
Y = scaler.transform(Y)
x_train=x_train.reshape((x_train.shape[0],x_train.shape[1],1))
x_test = x_test.reshape((x_test.shape[0], x_test.shape[1],1))
t_train=t_train.reshape((t_train.shape[0],1))
t_test = t_test.reshape((t_test.shape[0],1))

model = Sequential()
model.add(Conv1D(8, kernel_size=3, strides=1, padding='same', input_shape=(x_train.shape[1],1)))
model.add(BatchNormalization())
model.add(Conv1D(8, kernel_size=3, strides=1, padding='same'))
model.add(Conv1D(16, kernel_size=3, strides=1, padding='valid'))
model.add(BatchNormalization())
model.add(Conv1D(16, kernel_size=3, strides=1, padding='same'))
model.add(Conv1D(32, kernel_size=3, strides=1, padding='valid'))
model.add(BatchNormalization())
model.add(Conv1D(32, kernel_size=3, strides=1, padding='same'))
model.add(Conv1D(32, kernel_size=3, strides=1, padding='same'))
model.add(Conv1D(64, kernel_size=3, strides=1, padding='same'))
model.add(Activation('tanh'))
model.add(Flatten())
model.add(Dropout(0.5))
# model.add(Dropout(0.25))
model.add(Dense(512,kernel_initializer='he_normal', activation='relu', W_regularizer=l2(0.01)))
model.add(Dense(128,kernel_initializer='he_normal', activation='relu', W_regularizer=l2(0.01)))
model.add(Dense(1, kernel_initializer='normal', activation='sigmoid'))

optimizers.Adam(lr=0.01, epsilon=1e-8, decay=1e-4)

model.compile(optimizer=optimizer, loss='mse', metrics=['mae'])
model.summary()

ng = EarlyStopping(monitor='val_mean_absolute_error', mode='min', patience=4, verbose=1)
# model_checkpoint = ModelCheckpoint(filepath='best_model.h5', monitor='val_mean_absolute_error', mode = 'min', save_best_only=True, verbose=1)
# reduce_lr = ReduceLROnPlateau(monitor='val_mean_absolute_error', mode = 'min',factor=0.5, patience=3, min_lr=0.0001, verbose=1)
history = model.fit(x_train, t_train,
     validation_data=(x_test, t_test),
     epochs=30,
     batch_size=32768,
     callbacks=[early_stopping],
     verbose=1)predict(Y)
pred = pred.ravel()

补充知识:Keras Conv1d 参数及输入输出详解

Conv1d(in_channels,out_channels,kernel_size,stride=1,padding=0,dilation=1,groups=1,bias=True)

filters:卷积核的数目(即输出的维度)

kernel_size: 整数或由单个整数构成的list/tuple,卷积核的空域或时域窗长度

strides: 整数或由单个整数构成的list/tuple,为卷积的步长。任何不为1的strides均为任何不为1的dilation_rata均不兼容

padding: 补0策略,为”valid”,”same”或”casual”,”casual”将产生因果(膨胀的)卷积,即output[t]不依赖于input[t+1:]。当对不能违反事件顺序的时序信号建模时有用。“valid”代表只进行有效的卷积,即对边界数据不处理。“same”代表保留边界处的卷积结果,通常会导致输出shape与输入shape相同。

activation:激活函数,为预定义的激活函数名,或逐元素的Theano函数。如果不指定该函数,将不会使用任何激活函数(即使用线性激活函数:a(x)=x)

model.add(Conv1D(filters=nn_params["input_filters"],
      kernel_size=nn_params["filter_length"],
      strides=1,
      padding='valid',
      activation=nn_params["activation"],
      kernel_regularizer=l2(nn_params["reg"])))

例:输入维度为(None,1000,4)

第一维度:None

第二维度:

output_length = int((input_length - nn_params["filter_length"] + 1))

在此情况下为:

output_length = (1000 + 2*padding - filters +1)/ strides = (1000 + 2*0 -32 +1)/1 = 969

第三维度:filters

以上这篇解决keras使用cov1D函数的输入问题就是小编分享给大家的全部内容了,希望能给大家一个参考,也希望大家多多支持三水点靠木。

Python 相关文章推荐
Python的Django框架中settings文件的部署建议
May 30 Python
Python实现改变与矩形橡胶的线条的颜色代码示例
Jan 05 Python
python如何让类支持比较运算
Mar 20 Python
python操作mysql代码总结
Jun 01 Python
pandas使用apply多列生成一列数据的实例
Nov 28 Python
Python实现读取txt文件中的数据并绘制出图形操作示例
Feb 26 Python
详解Python使用Plotly绘图工具,绘制甘特图
Apr 02 Python
安装Pycharm2019以及配置anconda教程的方法步骤
Nov 11 Python
通过实例简单了解Python中yield的作用
Dec 11 Python
python名片管理系统开发
Jun 18 Python
python/golang 删除链表中的元素
Sep 14 Python
sublime3之内网安装python插件Anaconda的流程
Nov 10 Python
快速了解Python开发环境Spyder
Jun 29 #Python
使用Keras构造简单的CNN网络实例
Jun 29 #Python
基于K.image_data_format() == 'channels_first' 的理解
Jun 29 #Python
Python enumerate() 函数如何实现索引功能
Jun 29 #Python
解决Keras中CNN输入维度报错问题
Jun 29 #Python
Python字符串split及rsplit方法原理详解
Jun 29 #Python
浅谈Keras参数 input_shape、input_dim和input_length用法
Jun 29 #Python
You might like
几种显示数据的方法的比较
2006/10/09 PHP
求帮忙修改个php curl模拟post请求内容后并下载文件的解决思路
2015/09/20 PHP
javascript YUI 读码日记之 YAHOO.util.Dom - Part.4
2008/03/22 Javascript
js动态控制table的tr、td增加及删除的具体实现
2014/04/30 Javascript
jQuery实现动态表单验证时文本框抖动效果完整实例
2015/08/21 Javascript
Javascript页面跳转常见实现方式汇总
2015/11/28 Javascript
javascript 中iframe高度自适应(同域)实例详解
2017/05/16 Javascript
JQuery判断正整数整理小结
2017/08/21 jQuery
Bootstrap4如何定制自己的颜色和风格
2018/02/26 Javascript
p5.js入门教程之图片加载
2018/03/20 Javascript
create-react-app修改为多页面支持的方法
2018/05/17 Javascript
小程序跨页面交互的作用与方法详解
2020/01/07 Javascript
[15:56]Heroes18_暗影萨满(完美)
2014/10/31 DOTA
Windows系统下使用flup搭建Nginx和Python环境的方法
2015/12/25 Python
python实现自动登录
2018/09/17 Python
Python调用C++,通过Pybind11制作Python接口
2018/10/16 Python
numpy linalg模块的具体使用方法
2019/05/26 Python
python多项式拟合之np.polyfit 和 np.polyld详解
2020/02/18 Python
python脚本第一行如何写
2020/08/30 Python
Python Spyder 调出缩进对齐线的操作
2021/02/26 Python
微信html5页面调用第三方位置导航的示例
2018/03/14 HTML / CSS
新加坡网上花店:FlowerAdvisor新加坡
2018/10/05 全球购物
介绍下java.util.Arrays类
2012/10/16 面试题
幼儿教师个人求职信范文
2013/09/21 职场文书
创业计划书——互联网商机
2014/01/12 职场文书
教师简历自我评价
2014/02/03 职场文书
消防安全汇报材料
2014/02/08 职场文书
宾馆总经理岗位职责
2014/02/14 职场文书
《海伦?凯勒》教学反思
2014/04/17 职场文书
2014法制宣传日活动总结
2014/07/09 职场文书
车辆年检委托书范本
2014/10/14 职场文书
健康状况证明书
2014/11/26 职场文书
公司内部升职自荐信
2015/03/27 职场文书
单位政审意见范文
2015/06/04 职场文书
2016学习医德医风心得体会
2016/01/25 职场文书
Python趣味挑战之给幼儿园弟弟生成1000道算术题
2021/05/28 Python