python机器学习理论与实战(六)支持向量机


Posted in Python onJanuary 19, 2018

上节基本完成了SVM的理论推倒,寻找最大化间隔的目标最终转换成求解拉格朗日乘子变量alpha的求解问题,求出了alpha即可求解出SVM的权重W,有了权重也就有了最大间隔距离,但是其实上节我们有个假设:就是训练集是线性可分的,这样求出的alpha在[0,infinite]。但是如果数据不是线性可分的呢?此时我们就要允许部分的样本可以越过分类器,这样优化的目标函数就可以不变,只要引入松弛变量python机器学习理论与实战(六)支持向量机即可,它表示错分类样本点的代价,分类正确时它等于0,当分类错误时python机器学习理论与实战(六)支持向量机,其中Tn表示样本的真实标签-1或者1,回顾上节中,我们把支持向量到分类器的距离固定为1,因此两类的支持向量间的距离肯定大于1的,当分类错误时python机器学习理论与实战(六)支持向量机肯定也大于1,如(图五)所示(这里公式和图标序号都接上一节)。

python机器学习理论与实战(六)支持向量机

(图五)

       这样有了错分类的代价,我们把上节(公式四)的目标函数上添加上这一项错分类代价,得到如(公式八)的形式:

python机器学习理论与实战(六)支持向量机

(公式八)

重复上节的拉格朗日乘子法步骤,得到(公式九):

python机器学习理论与实战(六)支持向量机

(公式九)

         多了一个Un乘子,当然我们的工作就是继续求解此目标函数,继续重复上节的步骤,求导得到(公式十):

 python机器学习理论与实战(六)支持向量机

(公式十)

         又因为alpha大于0,而且Un大于0,所以0<alpha<C,为了解释的清晰一些,我们把(公式九)的KKT条件也发出来(上节中的第三类优化问题),注意Un是大于等于0

python机器学习理论与实战(六)支持向量机 

      推导到现在,优化函数的形式基本没变,只是多了一项错分类的价值,但是多了一个条件,0<alpha<C,C是一个常数,它的作用就是在允许有错误分类的情况下,控制最大化间距,它太大了会导致过拟合,太小了会导致欠拟合。接下来的步骤貌似大家都应该知道了,多了一个C常量的限制条件,然后继续用SMO算法优化求解二次规划,但是我想继续把核函数也一次说了,如果样本线性不可分,引入核函数后,把样本映射到高维空间就可以线性可分,如(图六)所示的线性不可分的样本:

python机器学习理论与实战(六)支持向量机

(图六)

         在(图六)中,现有的样本是很明显线性不可分,但是加入我们利用现有的样本X之间作些不同的运算,如(图六)右边所示的样子,而让f作为新的样本(或者说新的特征)是不是更好些?现在把X已经投射到高维度上去了,但是f我们不知道,此时核函数就该上场了,以高斯核函数为例,在(图七)中选几个样本点作为基准点,来利用核函数计算f,如(图七)所示:

python机器学习理论与实战(六)支持向量机

(图七)

       这样就有了f,而核函数此时相当于对样本的X和基准点一个度量,做权重衰减,形成依赖于x的新的特征f,把f放在上面说的SVM中继续求解alpha,然后得出权重就行了,原理很简单吧,为了显得有点学术味道,把核函数也做个样子加入目标函数中去吧,如(公式十一)所示:

 python机器学习理论与实战(六)支持向量机

(公式十一) 

        其中K(Xn,Xm)是核函数,和上面目标函数比没有多大的变化,用SMO优化求解就行了,代码如下:

def smoPK(dataMatIn, classLabels, C, toler, maxIter): #full Platt SMO 
 oS = optStruct(mat(dataMatIn),mat(classLabels).transpose(),C,toler) 
 iter = 0 
 entireSet = True; alphaPairsChanged = 0 
 while (iter < maxIter) and ((alphaPairsChanged > 0) or (entireSet)): 
  alphaPairsChanged = 0 
  if entireSet: #go over all 
   for i in range(oS.m):   
    alphaPairsChanged += innerL(i,oS) 
    print "fullSet, iter: %d i:%d, pairs changed %d" % (iter,i,alphaPairsChanged) 
   iter += 1 
  else:#go over non-bound (railed) alphas 
   nonBoundIs = nonzero((oS.alphas.A > 0) * (oS.alphas.A < C))[0] 
   for i in nonBoundIs: 
    alphaPairsChanged += innerL(i,oS) 
    print "non-bound, iter: %d i:%d, pairs changed %d" % (iter,i,alphaPairsChanged) 
   iter += 1 
  if entireSet: entireSet = False #toggle entire set loop 
  elif (alphaPairsChanged == 0): entireSet = True 
  print "iteration number: %d" % iter 
 return oS.b,oS.alphas

下面演示一个小例子,手写识别。

      (1)收集数据:提供文本文件

      (2)准备数据:基于二值图像构造向量

      (3)分析数据:对图像向量进行目测

      (4)训练算法:采用两种不同的核函数,并对径向基函数采用不同的设置来运行SMO算法。

       (5)测试算法:编写一个函数来测试不同的核函数,并计算错误率

       (6)使用算法:一个图像识别的完整应用还需要一些图像处理的只是,此demo略。

完整代码如下:

from numpy import * 
from time import sleep 
 
def loadDataSet(fileName): 
 dataMat = []; labelMat = [] 
 fr = open(fileName) 
 for line in fr.readlines(): 
  lineArr = line.strip().split('\t') 
  dataMat.append([float(lineArr[0]), float(lineArr[1])]) 
  labelMat.append(float(lineArr[2])) 
 return dataMat,labelMat 
 
def selectJrand(i,m): 
 j=i #we want to select any J not equal to i 
 while (j==i): 
  j = int(random.uniform(0,m)) 
 return j 
 
def clipAlpha(aj,H,L): 
 if aj > H: 
  aj = H 
 if L > aj: 
  aj = L 
 return aj 
 
def smoSimple(dataMatIn, classLabels, C, toler, maxIter): 
 dataMatrix = mat(dataMatIn); labelMat = mat(classLabels).transpose() 
 b = 0; m,n = shape(dataMatrix) 
 alphas = mat(zeros((m,1))) 
 iter = 0 
 while (iter < maxIter): 
  alphaPairsChanged = 0 
  for i in range(m): 
   fXi = float(multiply(alphas,labelMat).T*(dataMatrix*dataMatrix[i,:].T)) + b 
   Ei = fXi - float(labelMat[i])#if checks if an example violates KKT conditions 
   if ((labelMat[i]*Ei < -toler) and (alphas[i] < C)) or ((labelMat[i]*Ei > toler) and (alphas[i] > 0)): 
    j = selectJrand(i,m) 
    fXj = float(multiply(alphas,labelMat).T*(dataMatrix*dataMatrix[j,:].T)) + b 
    Ej = fXj - float(labelMat[j]) 
    alphaIold = alphas[i].copy(); alphaJold = alphas[j].copy(); 
    if (labelMat[i] != labelMat[j]): 
     L = max(0, alphas[j] - alphas[i]) 
     H = min(C, C + alphas[j] - alphas[i]) 
    else: 
     L = max(0, alphas[j] + alphas[i] - C) 
     H = min(C, alphas[j] + alphas[i]) 
    if L==H: print "L==H"; continue 
    eta = 2.0 * dataMatrix[i,:]*dataMatrix[j,:].T - dataMatrix[i,:]*dataMatrix[i,:].T - dataMatrix[j,:]*dataMatrix[j,:].T 
    if eta >= 0: print "eta>=0"; continue 
    alphas[j] -= labelMat[j]*(Ei - Ej)/eta 
    alphas[j] = clipAlpha(alphas[j],H,L) 
    if (abs(alphas[j] - alphaJold) < 0.00001): print "j not moving enough"; continue 
    alphas[i] += labelMat[j]*labelMat[i]*(alphaJold - alphas[j])#update i by the same amount as j 
                  #the update is in the oppostie direction 
    b1 = b - Ei- labelMat[i]*(alphas[i]-alphaIold)*dataMatrix[i,:]*dataMatrix[i,:].T - labelMat[j]*(alphas[j]-alphaJold)*dataMatrix[i,:]*dataMatrix[j,:].T 
    b2 = b - Ej- labelMat[i]*(alphas[i]-alphaIold)*dataMatrix[i,:]*dataMatrix[j,:].T - labelMat[j]*(alphas[j]-alphaJold)*dataMatrix[j,:]*dataMatrix[j,:].T 
    if (0 < alphas[i]) and (C > alphas[i]): b = b1 
    elif (0 < alphas[j]) and (C > alphas[j]): b = b2 
    else: b = (b1 + b2)/2.0 
    alphaPairsChanged += 1 
    print "iter: %d i:%d, pairs changed %d" % (iter,i,alphaPairsChanged) 
  if (alphaPairsChanged == 0): iter += 1 
  else: iter = 0 
  print "iteration number: %d" % iter 
 return b,alphas 
 
def kernelTrans(X, A, kTup): #calc the kernel or transform data to a higher dimensional space 
 m,n = shape(X) 
 K = mat(zeros((m,1))) 
 if kTup[0]=='lin': K = X * A.T #linear kernel 
 elif kTup[0]=='rbf': 
  for j in range(m): 
   deltaRow = X[j,:] - A 
   K[j] = deltaRow*deltaRow.T 
  K = exp(K/(-1*kTup[1]**2)) #divide in NumPy is element-wise not matrix like Matlab 
 else: raise NameError('Houston We Have a Problem -- \ 
 That Kernel is not recognized') 
 return K 
 
class optStruct: 
 def __init__(self,dataMatIn, classLabels, C, toler, kTup): # Initialize the structure with the parameters 
  self.X = dataMatIn 
  self.labelMat = classLabels 
  self.C = C 
  self.tol = toler 
  self.m = shape(dataMatIn)[0] 
  self.alphas = mat(zeros((self.m,1))) 
  self.b = 0 
  self.eCache = mat(zeros((self.m,2))) #first column is valid flag 
  self.K = mat(zeros((self.m,self.m))) 
  for i in range(self.m): 
   self.K[:,i] = kernelTrans(self.X, self.X[i,:], kTup) 
   
def calcEk(oS, k): 
 fXk = float(multiply(oS.alphas,oS.labelMat).T*oS.K[:,k] + oS.b) 
 Ek = fXk - float(oS.labelMat[k]) 
 return Ek 
   
def selectJ(i, oS, Ei):   #this is the second choice -heurstic, and calcs Ej 
 maxK = -1; maxDeltaE = 0; Ej = 0 
 oS.eCache[i] = [1,Ei] #set valid #choose the alpha that gives the maximum delta E 
 validEcacheList = nonzero(oS.eCache[:,0].A)[0] 
 if (len(validEcacheList)) > 1: 
  for k in validEcacheList: #loop through valid Ecache values and find the one that maximizes delta E 
   if k == i: continue #don't calc for i, waste of time 
   Ek = calcEk(oS, k) 
   deltaE = abs(Ei - Ek) 
   if (deltaE > maxDeltaE): 
    maxK = k; maxDeltaE = deltaE; Ej = Ek 
  return maxK, Ej 
 else: #in this case (first time around) we don't have any valid eCache values 
  j = selectJrand(i, oS.m) 
  Ej = calcEk(oS, j) 
 return j, Ej 
 
def updateEk(oS, k):#after any alpha has changed update the new value in the cache 
 Ek = calcEk(oS, k) 
 oS.eCache[k] = [1,Ek] 
   
def innerL(i, oS): 
 Ei = calcEk(oS, i) 
 if ((oS.labelMat[i]*Ei < -oS.tol) and (oS.alphas[i] < oS.C)) or ((oS.labelMat[i]*Ei > oS.tol) and (oS.alphas[i] > 0)): 
  j,Ej = selectJ(i, oS, Ei) #this has been changed from selectJrand 
  alphaIold = oS.alphas[i].copy(); alphaJold = oS.alphas[j].copy(); 
  if (oS.labelMat[i] != oS.labelMat[j]): 
   L = max(0, oS.alphas[j] - oS.alphas[i]) 
   H = min(oS.C, oS.C + oS.alphas[j] - oS.alphas[i]) 
  else: 
   L = max(0, oS.alphas[j] + oS.alphas[i] - oS.C) 
   H = min(oS.C, oS.alphas[j] + oS.alphas[i]) 
  if L==H: print "L==H"; return 0 
  eta = 2.0 * oS.K[i,j] - oS.K[i,i] - oS.K[j,j] #changed for kernel 
  if eta >= 0: print "eta>=0"; return 0 
  oS.alphas[j] -= oS.labelMat[j]*(Ei - Ej)/eta 
  oS.alphas[j] = clipAlpha(oS.alphas[j],H,L) 
  updateEk(oS, j) #added this for the Ecache 
  if (abs(oS.alphas[j] - alphaJold) < 0.00001): print "j not moving enough"; return 0 
  oS.alphas[i] += oS.labelMat[j]*oS.labelMat[i]*(alphaJold - oS.alphas[j])#update i by the same amount as j 
  updateEk(oS, i) #added this for the Ecache     #the update is in the oppostie direction 
  b1 = oS.b - Ei- oS.labelMat[i]*(oS.alphas[i]-alphaIold)*oS.K[i,i] - oS.labelMat[j]*(oS.alphas[j]-alphaJold)*oS.K[i,j] 
  b2 = oS.b - Ej- oS.labelMat[i]*(oS.alphas[i]-alphaIold)*oS.K[i,j]- oS.labelMat[j]*(oS.alphas[j]-alphaJold)*oS.K[j,j] 
  if (0 < oS.alphas[i]) and (oS.C > oS.alphas[i]): oS.b = b1 
  elif (0 < oS.alphas[j]) and (oS.C > oS.alphas[j]): oS.b = b2 
  else: oS.b = (b1 + b2)/2.0 
  return 1 
 else: return 0 
 
def smoP(dataMatIn, classLabels, C, toler, maxIter,kTup=('lin', 0)): #full Platt SMO 
 oS = optStruct(mat(dataMatIn),mat(classLabels).transpose(),C,toler, kTup) 
 iter = 0 
 entireSet = True; alphaPairsChanged = 0 
 while (iter < maxIter) and ((alphaPairsChanged > 0) or (entireSet)): 
  alphaPairsChanged = 0 
  if entireSet: #go over all 
   for i in range(oS.m):   
    alphaPairsChanged += innerL(i,oS) 
    print "fullSet, iter: %d i:%d, pairs changed %d" % (iter,i,alphaPairsChanged) 
   iter += 1 
  else:#go over non-bound (railed) alphas 
   nonBoundIs = nonzero((oS.alphas.A > 0) * (oS.alphas.A < C))[0] 
   for i in nonBoundIs: 
    alphaPairsChanged += innerL(i,oS) 
    print "non-bound, iter: %d i:%d, pairs changed %d" % (iter,i,alphaPairsChanged) 
   iter += 1 
  if entireSet: entireSet = False #toggle entire set loop 
  elif (alphaPairsChanged == 0): entireSet = True 
  print "iteration number: %d" % iter 
 return oS.b,oS.alphas 
 
def calcWs(alphas,dataArr,classLabels): 
 X = mat(dataArr); labelMat = mat(classLabels).transpose() 
 m,n = shape(X) 
 w = zeros((n,1)) 
 for i in range(m): 
  w += multiply(alphas[i]*labelMat[i],X[i,:].T) 
 return w 
 
def testRbf(k1=1.3): 
 dataArr,labelArr = loadDataSet('testSetRBF.txt') 
 b,alphas = smoP(dataArr, labelArr, 200, 0.0001, 10000, ('rbf', k1)) #C=200 important 
 datMat=mat(dataArr); labelMat = mat(labelArr).transpose() 
 svInd=nonzero(alphas.A>0)[0] 
 sVs=datMat[svInd] #get matrix of only support vectors 
 labelSV = labelMat[svInd]; 
 print "there are %d Support Vectors" % shape(sVs)[0] 
 m,n = shape(datMat) 
 errorCount = 0 
 for i in range(m): 
  kernelEval = kernelTrans(sVs,datMat[i,:],('rbf', k1)) 
  predict=kernelEval.T * multiply(labelSV,alphas[svInd]) + b 
  if sign(predict)!=sign(labelArr[i]): errorCount += 1 
 print "the training error rate is: %f" % (float(errorCount)/m) 
 dataArr,labelArr = loadDataSet('testSetRBF2.txt') 
 errorCount = 0 
 datMat=mat(dataArr); labelMat = mat(labelArr).transpose() 
 m,n = shape(datMat) 
 for i in range(m): 
  kernelEval = kernelTrans(sVs,datMat[i,:],('rbf', k1)) 
  predict=kernelEval.T * multiply(labelSV,alphas[svInd]) + b 
  if sign(predict)!=sign(labelArr[i]): errorCount += 1  
 print "the test error rate is: %f" % (float(errorCount)/m)  
  
def img2vector(filename): 
 returnVect = zeros((1,1024)) 
 fr = open(filename) 
 for i in range(32): 
  lineStr = fr.readline() 
  for j in range(32): 
   returnVect[0,32*i+j] = int(lineStr[j]) 
 return returnVect 
 
def loadImages(dirName): 
 from os import listdir 
 hwLabels = [] 
 trainingFileList = listdir(dirName)   #load the training set 
 m = len(trainingFileList) 
 trainingMat = zeros((m,1024)) 
 for i in range(m): 
  fileNameStr = trainingFileList[i] 
  fileStr = fileNameStr.split('.')[0]  #take off .txt 
  classNumStr = int(fileStr.split('_')[0]) 
  if classNumStr == 9: hwLabels.append(-1) 
  else: hwLabels.append(1) 
  trainingMat[i,:] = img2vector('%s/%s' % (dirName, fileNameStr)) 
 return trainingMat, hwLabels  
 
def testDigits(kTup=('rbf', 10)): 
 dataArr,labelArr = loadImages('trainingDigits') 
 b,alphas = smoP(dataArr, labelArr, 200, 0.0001, 10000, kTup) 
 datMat=mat(dataArr); labelMat = mat(labelArr).transpose() 
 svInd=nonzero(alphas.A>0)[0] 
 sVs=datMat[svInd] 
 labelSV = labelMat[svInd]; 
 print "there are %d Support Vectors" % shape(sVs)[0] 
 m,n = shape(datMat) 
 errorCount = 0 
 for i in range(m): 
  kernelEval = kernelTrans(sVs,datMat[i,:],kTup) 
  predict=kernelEval.T * multiply(labelSV,alphas[svInd]) + b 
  if sign(predict)!=sign(labelArr[i]): errorCount += 1 
 print "the training error rate is: %f" % (float(errorCount)/m) 
 dataArr,labelArr = loadImages('testDigits') 
 errorCount = 0 
 datMat=mat(dataArr); labelMat = mat(labelArr).transpose() 
 m,n = shape(datMat) 
 for i in range(m): 
  kernelEval = kernelTrans(sVs,datMat[i,:],kTup) 
  predict=kernelEval.T * multiply(labelSV,alphas[svInd]) + b 
  if sign(predict)!=sign(labelArr[i]): errorCount += 1  
 print "the test error rate is: %f" % (float(errorCount)/m) 
 
 
'''''#######******************************** 
Non-Kernel VErsions below 
'''#######******************************** 
 
class optStructK: 
 def __init__(self,dataMatIn, classLabels, C, toler): # Initialize the structure with the parameters 
  self.X = dataMatIn 
  self.labelMat = classLabels 
  self.C = C 
  self.tol = toler 
  self.m = shape(dataMatIn)[0] 
  self.alphas = mat(zeros((self.m,1))) 
  self.b = 0 
  self.eCache = mat(zeros((self.m,2))) #first column is valid flag 
   
def calcEkK(oS, k): 
 fXk = float(multiply(oS.alphas,oS.labelMat).T*(oS.X*oS.X[k,:].T)) + oS.b 
 Ek = fXk - float(oS.labelMat[k]) 
 return Ek 
   
def selectJK(i, oS, Ei):   #this is the second choice -heurstic, and calcs Ej 
 maxK = -1; maxDeltaE = 0; Ej = 0 
 oS.eCache[i] = [1,Ei] #set valid #choose the alpha that gives the maximum delta E 
 validEcacheList = nonzero(oS.eCache[:,0].A)[0] 
 if (len(validEcacheList)) > 1: 
  for k in validEcacheList: #loop through valid Ecache values and find the one that maximizes delta E 
   if k == i: continue #don't calc for i, waste of time 
   Ek = calcEk(oS, k) 
   deltaE = abs(Ei - Ek) 
   if (deltaE > maxDeltaE): 
    maxK = k; maxDeltaE = deltaE; Ej = Ek 
  return maxK, Ej 
 else: #in this case (first time around) we don't have any valid eCache values 
  j = selectJrand(i, oS.m) 
  Ej = calcEk(oS, j) 
 return j, Ej 
 
def updateEkK(oS, k):#after any alpha has changed update the new value in the cache 
 Ek = calcEk(oS, k) 
 oS.eCache[k] = [1,Ek] 
   
def innerLK(i, oS): 
 Ei = calcEk(oS, i) 
 if ((oS.labelMat[i]*Ei < -oS.tol) and (oS.alphas[i] < oS.C)) or ((oS.labelMat[i]*Ei > oS.tol) and (oS.alphas[i] > 0)): 
  j,Ej = selectJ(i, oS, Ei) #this has been changed from selectJrand 
  alphaIold = oS.alphas[i].copy(); alphaJold = oS.alphas[j].copy(); 
  if (oS.labelMat[i] != oS.labelMat[j]): 
   L = max(0, oS.alphas[j] - oS.alphas[i]) 
   H = min(oS.C, oS.C + oS.alphas[j] - oS.alphas[i]) 
  else: 
   L = max(0, oS.alphas[j] + oS.alphas[i] - oS.C) 
   H = min(oS.C, oS.alphas[j] + oS.alphas[i]) 
  if L==H: print "L==H"; return 0 
  eta = 2.0 * oS.X[i,:]*oS.X[j,:].T - oS.X[i,:]*oS.X[i,:].T - oS.X[j,:]*oS.X[j,:].T 
  if eta >= 0: print "eta>=0"; return 0 
  oS.alphas[j] -= oS.labelMat[j]*(Ei - Ej)/eta 
  oS.alphas[j] = clipAlpha(oS.alphas[j],H,L) 
  updateEk(oS, j) #added this for the Ecache 
  if (abs(oS.alphas[j] - alphaJold) < 0.00001): print "j not moving enough"; return 0 
  oS.alphas[i] += oS.labelMat[j]*oS.labelMat[i]*(alphaJold - oS.alphas[j])#update i by the same amount as j 
  updateEk(oS, i) #added this for the Ecache     #the update is in the oppostie direction 
  b1 = oS.b - Ei- oS.labelMat[i]*(oS.alphas[i]-alphaIold)*oS.X[i,:]*oS.X[i,:].T - oS.labelMat[j]*(oS.alphas[j]-alphaJold)*oS.X[i,:]*oS.X[j,:].T 
  b2 = oS.b - Ej- oS.labelMat[i]*(oS.alphas[i]-alphaIold)*oS.X[i,:]*oS.X[j,:].T - oS.labelMat[j]*(oS.alphas[j]-alphaJold)*oS.X[j,:]*oS.X[j,:].T 
  if (0 < oS.alphas[i]) and (oS.C > oS.alphas[i]): oS.b = b1 
  elif (0 < oS.alphas[j]) and (oS.C > oS.alphas[j]): oS.b = b2 
  else: oS.b = (b1 + b2)/2.0 
  return 1 
 else: return 0 
 
def smoPK(dataMatIn, classLabels, C, toler, maxIter): #full Platt SMO 
 oS = optStruct(mat(dataMatIn),mat(classLabels).transpose(),C,toler) 
 iter = 0 
 entireSet = True; alphaPairsChanged = 0 
 while (iter < maxIter) and ((alphaPairsChanged > 0) or (entireSet)): 
  alphaPairsChanged = 0 
  if entireSet: #go over all 
   for i in range(oS.m):   
    alphaPairsChanged += innerL(i,oS) 
    print "fullSet, iter: %d i:%d, pairs changed %d" % (iter,i,alphaPairsChanged) 
   iter += 1 
  else:#go over non-bound (railed) alphas 
   nonBoundIs = nonzero((oS.alphas.A > 0) * (oS.alphas.A < C))[0] 
   for i in nonBoundIs: 
    alphaPairsChanged += innerL(i,oS) 
    print "non-bound, iter: %d i:%d, pairs changed %d" % (iter,i,alphaPairsChanged) 
   iter += 1 
  if entireSet: entireSet = False #toggle entire set loop 
  elif (alphaPairsChanged == 0): entireSet = True 
  print "iteration number: %d" % iter 
 return oS.b,oS.alphas

运行结果如(图八)所示:

python机器学习理论与实战(六)支持向量机

(图八)

上面代码有兴趣的可以读读,用的话,建议使用libsvm。

参考文献:

    [1]machine learning in action. PeterHarrington

    [2] pattern recognition and machinelearning. Christopher M. Bishop

    [3]machine learning.Andrew Ng

以上就是本文的全部内容,希望对大家的学习有所帮助,也希望大家多多支持三水点靠木。

Python 相关文章推荐
Python中使用urllib2防止302跳转的代码例子
Jul 07 Python
python中requests模块的使用方法
Apr 08 Python
Python使用urllib2模块抓取HTML页面资源的实例分享
May 03 Python
Python入门_浅谈数据结构的4种基本类型
May 16 Python
DataFrame 将某列数据转为数组的方法
Apr 13 Python
Python + selenium自动化环境搭建的完整步骤
May 19 Python
python保存数据到本地文件的方法
Jun 23 Python
对python 生成拼接xml报文的示例详解
Dec 28 Python
django echarts饼图数据动态加载的实例
Aug 12 Python
Pytorch实现的手写数字mnist识别功能完整示例
Dec 13 Python
python raise的基本使用
Sep 10 Python
selenium框架中driver.close()和driver.quit()关闭浏览器
Dec 08 Python
Python logging管理不同级别log打印和存储实例
Jan 19 #Python
python机器学习理论与实战(五)支持向量机
Jan 19 #Python
Python读取图片为16进制表示简单代码
Jan 19 #Python
Python实现pdf文档转txt的方法示例
Jan 19 #Python
浅谈Python实现2种文件复制的方法
Jan 19 #Python
用Python进行简单图像识别(验证码)
Jan 19 #Python
flask中使用蓝图将路由分开写在不同文件实例解析
Jan 19 #Python
You might like
建立文件交换功能的脚本(二)
2006/10/09 PHP
MacOS 安装 PHP的图片裁剪扩展Tclip
2015/03/25 PHP
php 从指定数字中获取随机组合的简单方法(推荐)
2017/04/05 PHP
PHP实现发送微博消息功能完整示例
2019/12/04 PHP
Jquery 常用方法经典总结
2010/01/28 Javascript
基于jquery的一个浮动框(扩展性比较好 )
2010/08/27 Javascript
js和php如何获取当前url的内容
2013/09/22 Javascript
JS JSON对象转为字符串的简单实现方法
2013/11/18 Javascript
jquery中 $.expr使用实例介绍
2014/06/09 Javascript
基于NodeJS的前后端分离的思考与实践(四)安全问题解决方案
2014/09/26 NodeJs
JavaScript实现基于Cookie的存储类实例
2015/04/10 Javascript
javascript基本数据类型及类型检测常用方法小结
2016/12/14 Javascript
详解js产生对象的3种基本方式(工厂模式,构造函数模式,原型模式)
2017/01/09 Javascript
图片上传之FileAPI与NodeJs
2017/01/24 NodeJs
AngularJS自定义过滤器用法经典实例总结
2018/05/17 Javascript
微信小程序select下拉框实现效果
2019/05/15 Javascript
使用jQuery mobile NuGet让你的网站在移动设备上同样精彩
2019/06/18 jQuery
layui lay-verify form表单自定义验证规则详解
2019/09/18 Javascript
基于原生js实现判断元素是否有指定class名
2020/07/11 Javascript
Python使用htpasswd实现基本认证授权的例子
2014/06/10 Python
Python中MySQL数据迁移到MongoDB脚本的方法
2016/04/28 Python
Python基于Matplotlib库简单绘制折线图的方法示例
2017/08/14 Python
Python一句代码实现找出所有水仙花数的方法
2018/11/13 Python
Python3几个常见问题的处理方法
2019/02/26 Python
Python中的十大图像处理工具(小结)
2019/06/10 Python
Django接收自定义http header过程详解
2019/08/23 Python
python2.7实现复制大量文件及文件夹资料
2019/08/31 Python
Python的scikit-image模块实例讲解
2020/12/30 Python
Python实现一个论文下载器的过程
2021/01/18 Python
品质主管的岗位职责
2013/12/04 职场文书
30年同学聚会邀请函
2014/01/25 职场文书
擅自离岗检讨书
2014/09/12 职场文书
领导党的群众路线教育实践活动个人对照检查材料
2014/09/23 职场文书
应届生求职自荐信范文
2015/03/04 职场文书
公积金具结保证书
2015/05/11 职场文书
Python还能这么玩之用Python修改了班花的开机密码
2021/06/04 Python