神经网络python源码分享


Posted in Python onDecember 15, 2017

神经网络的逻辑应该都是熟知的了,在这里想说明一下交叉验证

交叉验证方法:

神经网络python源码分享

看图大概就能理解了,大致就是先将数据集分成K份,对这K份中每一份都取不一样的比例数据进行训练和测试。得出K个误差,将这K个误差平均得到最终误差

这第一个部分是BP神经网络的建立

参数选取参照论文:基于数据挖掘技术的股价指数分析与预测研究_胡林林

import math
import random
import tushare as ts
import pandas as pd
random.seed(0)
def getData(id,start,end):
  df = ts.get_hist_data(id,start,end)
  DATA=pd.DataFrame(columns=['rate1', 'rate2','rate3','pos1','pos2','pos3','amt1','amt2','amt3','MA20','MA5','r'])
  P1 = pd.DataFrame(columns=['high','low','close','open','volume'])
  DATA2=pd.DataFrame(columns=['R'])
  DATA['MA20']=df['ma20']
  DATA['MA5']=df['ma5']
  P=df['close']
  P1['high']=df['high']
  P1['low']=df['low']
  P1['close']=df['close']
  P1['open']=df['open']
  P1['volume']=df['volume']

  DATA['rate1']=(P1['close'].shift(1)-P1['open'].shift(1))/P1['open'].shift(1)
  DATA['rate2']=(P1['close'].shift(2)-P1['open'].shift(2))/P1['open'].shift(2)
  DATA['rate3']=(P1['close'].shift(3)-P1['open'].shift(3))/P1['open'].shift(3)
  DATA['pos1']=(P1['close'].shift(1)-P1['low'].shift(1))/(P1['high'].shift(1)-P1['low'].shift(1))
  DATA['pos2']=(P1['close'].shift(2)-P1['low'].shift(2))/(P1['high'].shift(2)-P1['low'].shift(2))
  DATA['pos3']=(P1['close'].shift(3)-P1['low'].shift(3))/(P1['high'].shift(3)-P1['low'].shift(3))
  DATA['amt1']=P1['volume'].shift(1)/((P1['volume'].shift(1)+P1['volume'].shift(2)+P1['volume'].shift(3))/3)
  DATA['amt2']=P1['volume'].shift(2)/((P1['volume'].shift(2)+P1['volume'].shift(3)+P1['volume'].shift(4))/3)
  DATA['amt3']=P1['volume'].shift(3)/((P1['volume'].shift(3)+P1['volume'].shift(4)+P1['volume'].shift(5))/3)
  templist=(P-P.shift(1))/P.shift(1)
  tempDATA = []
  for indextemp in templist:
    tempDATA.append(1/(1+math.exp(-indextemp*100)))
  DATA['r'] = tempDATA
  DATA=DATA.dropna(axis=0)
  DATA2['R']=DATA['r']
  del DATA['r']
  DATA=DATA.T
  DATA2=DATA2.T
  DATAlist=DATA.to_dict("list")
  result = []
  for key in DATAlist:
    result.append(DATAlist[key])
  DATAlist2=DATA2.to_dict("list")
  result2 = []
  for key in DATAlist2:
    result2.append(DATAlist2[key])
  return result
def getDataR(id,start,end):
  df = ts.get_hist_data(id,start,end)
  DATA=pd.DataFrame(columns=['rate1', 'rate2','rate3','pos1','pos2','pos3','amt1','amt2','amt3','MA20','MA5','r'])
  P1 = pd.DataFrame(columns=['high','low','close','open','volume'])
  DATA2=pd.DataFrame(columns=['R'])
  DATA['MA20']=df['ma20'].shift(1)
  DATA['MA5']=df['ma5'].shift(1)
  P=df['close']
  P1['high']=df['high']
  P1['low']=df['low']
  P1['close']=df['close']
  P1['open']=df['open']
  P1['volume']=df['volume']

  DATA['rate1']=(P1['close'].shift(1)-P1['open'].shift(1))/P1['open'].shift(1)
  DATA['rate2']=(P1['close'].shift(2)-P1['open'].shift(2))/P1['open'].shift(2)
  DATA['rate3']=(P1['close'].shift(3)-P1['open'].shift(3))/P1['open'].shift(3)
  DATA['pos1']=(P1['close'].shift(1)-P1['low'].shift(1))/(P1['high'].shift(1)-P1['low'].shift(1))
  DATA['pos2']=(P1['close'].shift(2)-P1['low'].shift(2))/(P1['high'].shift(2)-P1['low'].shift(2))
  DATA['pos3']=(P1['close'].shift(3)-P1['low'].shift(3))/(P1['high'].shift(3)-P1['low'].shift(3))
  DATA['amt1']=P1['volume'].shift(1)/((P1['volume'].shift(1)+P1['volume'].shift(2)+P1['volume'].shift(3))/3)
  DATA['amt2']=P1['volume'].shift(2)/((P1['volume'].shift(2)+P1['volume'].shift(3)+P1['volume'].shift(4))/3)
  DATA['amt3']=P1['volume'].shift(3)/((P1['volume'].shift(3)+P1['volume'].shift(4)+P1['volume'].shift(5))/3)
  templist=(P-P.shift(1))/P.shift(1)
  tempDATA = []
  for indextemp in templist:
    tempDATA.append(1/(1+math.exp(-indextemp*100)))
  DATA['r'] = tempDATA
  DATA=DATA.dropna(axis=0)
  DATA2['R']=DATA['r']
  del DATA['r']
  DATA=DATA.T
  DATA2=DATA2.T
  DATAlist=DATA.to_dict("list")
  result = []
  for key in DATAlist:
    result.append(DATAlist[key])
  DATAlist2=DATA2.to_dict("list")
  result2 = []
  for key in DATAlist2:
    result2.append(DATAlist2[key])
  return result2
def rand(a, b):
  return (b - a) * random.random() + a
def make_matrix(m, n, fill=0.0):
  mat = []
  for i in range(m):
    mat.append([fill] * n)
  return mat
def sigmoid(x):
  return 1.0 / (1.0 + math.exp(-x))
def sigmod_derivate(x):
  return x * (1 - x)
class BPNeuralNetwork:
  def __init__(self):
    self.input_n = 0
    self.hidden_n = 0
    self.output_n = 0
    self.input_cells = []
    self.hidden_cells = []
    self.output_cells = []
    self.input_weights = []
    self.output_weights = []
    self.input_correction = []
    self.output_correction = []

  def setup(self, ni, nh, no):
    self.input_n = ni + 1
    self.hidden_n = nh
    self.output_n = no
    # init cells
    self.input_cells = [1.0] * self.input_n
    self.hidden_cells = [1.0] * self.hidden_n
    self.output_cells = [1.0] * self.output_n
    # init weights
    self.input_weights = make_matrix(self.input_n, self.hidden_n)
    self.output_weights = make_matrix(self.hidden_n, self.output_n)
    # random activate
    for i in range(self.input_n):
      for h in range(self.hidden_n):
        self.input_weights[i][h] = rand(-0.2, 0.2)
    for h in range(self.hidden_n):
      for o in range(self.output_n):
        self.output_weights[h][o] = rand(-2.0, 2.0)
    # init correction matrix
    self.input_correction = make_matrix(self.input_n, self.hidden_n)
    self.output_correction = make_matrix(self.hidden_n, self.output_n)

  def predict(self, inputs):
    # activate input layer
    for i in range(self.input_n - 1):
      self.input_cells[i] = inputs[i]
    # activate hidden layer
    for j in range(self.hidden_n):
      total = 0.0
      for i in range(self.input_n):
        total += self.input_cells[i] * self.input_weights[i][j]
      self.hidden_cells[j] = sigmoid(total)
    # activate output layer
    for k in range(self.output_n):
      total = 0.0
      for j in range(self.hidden_n):
        total += self.hidden_cells[j] * self.output_weights[j][k]
      self.output_cells[k] = sigmoid(total)
    return self.output_cells[:]
  def back_propagate(self, case, label, learn, correct):
    # feed forward
    self.predict(case)
    # get output layer error
    output_deltas = [0.0] * self.output_n
    for o in range(self.output_n):
      error = label[o] - self.output_cells[o]
      output_deltas[o] = sigmod_derivate(self.output_cells[o]) * error
    # get hidden layer error
    hidden_deltas = [0.0] * self.hidden_n
    for h in range(self.hidden_n):
      error = 0.0
      for o in range(self.output_n):
        error += output_deltas[o] * self.output_weights[h][o]
      hidden_deltas[h] = sigmod_derivate(self.hidden_cells[h]) * error
    # update output weights
    for h in range(self.hidden_n):
      for o in range(self.output_n):
        change = output_deltas[o] * self.hidden_cells[h]
        self.output_weights[h][o] += learn * change + correct * self.output_correction[h][o]
        self.output_correction[h][o] = change
    # update input weights
    for i in range(self.input_n):
      for h in range(self.hidden_n):
        change = hidden_deltas[h] * self.input_cells[i]
        self.input_weights[i][h] += learn * change + correct * self.input_correction[i][h]
        self.input_correction[i][h] = change
    # get global error
    error = 0.0
    for o in range(len(label)):
      error += 0.5 * (label[o] - self.output_cells[o]) ** 2
    return error
  def train(self, cases, labels, limit=10000, learn=0.05, correct=0.1):
    for i in range(limit):
      error = 0.0
      for i in range(len(cases)):
        label = labels[i]
        case = cases[i]
        error += self.back_propagate(case, label, learn, correct)
  def test(self,id):
    result=getData("000001", "2015-01-05", "2015-01-09")
    result2=getDataR("000001", "2015-01-05", "2015-01-09")
    self.setup(11, 5, 1)
    self.train(result, result2, 10000, 0.05, 0.1)
    for t in resulttest:
      print(self.predict(t))

下面是选取14-15年数据进行训练,16年数据作为测试集,调仓周期为20个交易日,大约1个月,对上证50中的股票进行预测,选取预测的涨幅前10的股票买入,对每只股票分配一样的资金,初步运行没有问题,但就是太慢了,等哪天有空了再运行

import BPnet
import tushare as ts
import pandas as pd
import math
import xlrd
import datetime as dt
import time

#
#nn =BPnet.BPNeuralNetwork()
#nn.test('000001')
#for i in ts.get_sz50s()['code']:
holdList=pd.DataFrame(columns=['time','id','value'])
share=ts.get_sz50s()['code']
time2=ts.get_k_data('000001')['date']
newtime = time2[400:640]
newcount=0
for itime in newtime:
  print(itime)
  if newcount % 20 == 0:
        sharelist = pd.DataFrame(columns=['time','id','value'])
    for ishare in share:
      backwardtime = time.strftime('%Y-%m-%d',time.localtime(time.mktime(time.strptime(itime,'%Y-%m-%d'))-432000*4))
      trainData = BPnet.getData(ishare, '2014-05-22',itime)
      trainDataR = BPnet.getDataR(ishare, '2014-05-22',itime)
      testData = BPnet.getData(ishare, backwardtime,itime)
      try:
        print(testData)
        testData = testData[-1]
        print(testData)
        nn = BPnet.BPNeuralNetwork()
        nn.setup(11, 5, 1)
        nn.train(trainData, trainDataR, 10000, 0.05, 0.1)
        value = nn.predict(testData)
        newlist= pd.DataFrame({'time':itime,"id":ishare,"value":value},index=["0"])
        sharelist = sharelist.append(newlist,ignore_index=True)
      except: 
        pass
    sharelist=sharelist.sort(columns ='value',ascending=False)
    sharelist = sharelist[:10]
    holdList=holdList.append(sharelist,ignore_index=True)
  newcount+=1
  print(holdList)

总结

以上就是本文关于神经网络python源码分享的全部内容,希望对大家有所帮助。感兴趣的朋友可以继续参阅本站:

如有不足之处,欢迎留言指出。感谢朋友们对本站的支持!

Python 相关文章推荐
基于使用paramiko执行远程linux主机命令(详解)
Oct 16 Python
Python3结合Dlib实现人脸识别和剪切
Jan 24 Python
python实现人脸识别经典算法(一) 特征脸法
Mar 13 Python
Python连接Mssql基础教程之Python库pymssql
Sep 16 Python
python 划分数据集为训练集和测试集的方法
Dec 11 Python
django 环境变量配置过程详解
Aug 06 Python
Django 多表关联 存储 使用方法详解 ManyToManyField save
Aug 09 Python
Django 实现xadmin后台菜单改为中文
Nov 15 Python
Python加密模块的hashlib,hmac模块使用解析
Jan 02 Python
6行Python代码实现进度条效果(Progress、tqdm、alive-progress​​​​​​​和PySimpleGUI库)
Jan 06 Python
Python Selenium 设置元素等待的三种方式
Mar 18 Python
python实现批量转换图片为黑白
Jun 16 Python
神经网络理论基础及Python实现详解
Dec 15 #Python
浅谈机器学习需要的了解的十大算法
Dec 15 #Python
python实现协同过滤推荐算法完整代码示例
Dec 15 #Python
python3大文件解压和基本操作
Dec 15 #Python
Python数据结构与算法之常见的分配排序法示例【桶排序与基数排序】
Dec 15 #Python
numpy自动生成数组详解
Dec 15 #Python
Python3实现发送QQ邮件功能(附件)
Dec 23 #Python
You might like
文件系统基本操作类
2006/11/23 PHP
php单例模式的简单实现方法
2016/06/10 PHP
Nigma vs AM BO3 第一场2.13
2021/03/10 DOTA
基于jquery的划词搜索实现(备忘)
2010/09/14 Javascript
JQuery1.4+ Ajax IE8 内存泄漏问题
2010/10/15 Javascript
javascript获取下拉列表框当中的文本值示例代码
2013/07/31 Javascript
jquery中animate动画积累的解决方法
2013/10/05 Javascript
JS通过Cookie判断页面是否为首次打开
2016/02/05 Javascript
jQuery操作动态生成的内容的方法
2016/05/28 Javascript
每日十条JavaScript经验技巧(二)
2016/06/23 Javascript
JavaScript中闭包之浅析解读(必看篇)
2016/08/25 Javascript
Angular.Js中过滤器filter与自定义过滤器filter实例详解
2017/05/08 Javascript
全面解析Node.js 8 重要功能和修复
2017/06/02 Javascript
Vue编写多地区选择组件
2017/08/21 Javascript
探索Vue高阶组件的使用
2018/01/08 Javascript
详解VueJS应用中管理用户权限
2018/02/02 Javascript
Jquery的Ajax技术使用方法
2019/01/21 jQuery
超详细小程序定位地图模块全系列开发教学
2020/11/24 Javascript
利用JavaScript为句子加标题的3种方法示例
2021/01/05 Javascript
[04:19]DOTA2亚洲邀请赛 现场花絮
2015/03/11 DOTA
Python中实现对list做减法操作介绍
2015/01/09 Python
深入解析Python中的变量和赋值运算符
2015/10/12 Python
深入解读Python解析XML的几种方式
2016/02/16 Python
Python实现的矩阵类实例
2017/08/22 Python
Python实现发送与接收邮件的方法详解
2018/03/28 Python
python爬虫之线程池和进程池功能与用法详解
2018/08/02 Python
Python后台开发Django会话控制的实现
2019/04/15 Python
用canvas实现图片滤镜效果附演示
2013/11/05 HTML / CSS
canvas版人体时钟的实现示例
2021/01/29 HTML / CSS
应届护士推荐信
2013/11/16 职场文书
共产党员承诺书
2014/03/25 职场文书
培训班主持词
2014/03/28 职场文书
二胎满月酒致辞
2015/07/29 职场文书
商场广播稿范文
2015/08/19 职场文书
小学英语课教学反思
2016/02/15 职场文书
基于Go语言构建RESTful API服务
2021/07/25 Golang