python机器学习理论与实战(六)支持向量机


Posted in Python onJanuary 19, 2018

上节基本完成了SVM的理论推倒,寻找最大化间隔的目标最终转换成求解拉格朗日乘子变量alpha的求解问题,求出了alpha即可求解出SVM的权重W,有了权重也就有了最大间隔距离,但是其实上节我们有个假设:就是训练集是线性可分的,这样求出的alpha在[0,infinite]。但是如果数据不是线性可分的呢?此时我们就要允许部分的样本可以越过分类器,这样优化的目标函数就可以不变,只要引入松弛变量python机器学习理论与实战(六)支持向量机即可,它表示错分类样本点的代价,分类正确时它等于0,当分类错误时python机器学习理论与实战(六)支持向量机,其中Tn表示样本的真实标签-1或者1,回顾上节中,我们把支持向量到分类器的距离固定为1,因此两类的支持向量间的距离肯定大于1的,当分类错误时python机器学习理论与实战(六)支持向量机肯定也大于1,如(图五)所示(这里公式和图标序号都接上一节)。

python机器学习理论与实战(六)支持向量机

(图五)

       这样有了错分类的代价,我们把上节(公式四)的目标函数上添加上这一项错分类代价,得到如(公式八)的形式:

python机器学习理论与实战(六)支持向量机

(公式八)

重复上节的拉格朗日乘子法步骤,得到(公式九):

python机器学习理论与实战(六)支持向量机

(公式九)

         多了一个Un乘子,当然我们的工作就是继续求解此目标函数,继续重复上节的步骤,求导得到(公式十):

 python机器学习理论与实战(六)支持向量机

(公式十)

         又因为alpha大于0,而且Un大于0,所以0<alpha<C,为了解释的清晰一些,我们把(公式九)的KKT条件也发出来(上节中的第三类优化问题),注意Un是大于等于0

python机器学习理论与实战(六)支持向量机 

      推导到现在,优化函数的形式基本没变,只是多了一项错分类的价值,但是多了一个条件,0<alpha<C,C是一个常数,它的作用就是在允许有错误分类的情况下,控制最大化间距,它太大了会导致过拟合,太小了会导致欠拟合。接下来的步骤貌似大家都应该知道了,多了一个C常量的限制条件,然后继续用SMO算法优化求解二次规划,但是我想继续把核函数也一次说了,如果样本线性不可分,引入核函数后,把样本映射到高维空间就可以线性可分,如(图六)所示的线性不可分的样本:

python机器学习理论与实战(六)支持向量机

(图六)

         在(图六)中,现有的样本是很明显线性不可分,但是加入我们利用现有的样本X之间作些不同的运算,如(图六)右边所示的样子,而让f作为新的样本(或者说新的特征)是不是更好些?现在把X已经投射到高维度上去了,但是f我们不知道,此时核函数就该上场了,以高斯核函数为例,在(图七)中选几个样本点作为基准点,来利用核函数计算f,如(图七)所示:

python机器学习理论与实战(六)支持向量机

(图七)

       这样就有了f,而核函数此时相当于对样本的X和基准点一个度量,做权重衰减,形成依赖于x的新的特征f,把f放在上面说的SVM中继续求解alpha,然后得出权重就行了,原理很简单吧,为了显得有点学术味道,把核函数也做个样子加入目标函数中去吧,如(公式十一)所示:

 python机器学习理论与实战(六)支持向量机

(公式十一) 

        其中K(Xn,Xm)是核函数,和上面目标函数比没有多大的变化,用SMO优化求解就行了,代码如下:

def smoPK(dataMatIn, classLabels, C, toler, maxIter): #full Platt SMO 
 oS = optStruct(mat(dataMatIn),mat(classLabels).transpose(),C,toler) 
 iter = 0 
 entireSet = True; alphaPairsChanged = 0 
 while (iter < maxIter) and ((alphaPairsChanged > 0) or (entireSet)): 
  alphaPairsChanged = 0 
  if entireSet: #go over all 
   for i in range(oS.m):   
    alphaPairsChanged += innerL(i,oS) 
    print "fullSet, iter: %d i:%d, pairs changed %d" % (iter,i,alphaPairsChanged) 
   iter += 1 
  else:#go over non-bound (railed) alphas 
   nonBoundIs = nonzero((oS.alphas.A > 0) * (oS.alphas.A < C))[0] 
   for i in nonBoundIs: 
    alphaPairsChanged += innerL(i,oS) 
    print "non-bound, iter: %d i:%d, pairs changed %d" % (iter,i,alphaPairsChanged) 
   iter += 1 
  if entireSet: entireSet = False #toggle entire set loop 
  elif (alphaPairsChanged == 0): entireSet = True 
  print "iteration number: %d" % iter 
 return oS.b,oS.alphas

下面演示一个小例子,手写识别。

      (1)收集数据:提供文本文件

      (2)准备数据:基于二值图像构造向量

      (3)分析数据:对图像向量进行目测

      (4)训练算法:采用两种不同的核函数,并对径向基函数采用不同的设置来运行SMO算法。

       (5)测试算法:编写一个函数来测试不同的核函数,并计算错误率

       (6)使用算法:一个图像识别的完整应用还需要一些图像处理的只是,此demo略。

完整代码如下:

from numpy import * 
from time import sleep 
 
def loadDataSet(fileName): 
 dataMat = []; labelMat = [] 
 fr = open(fileName) 
 for line in fr.readlines(): 
  lineArr = line.strip().split('\t') 
  dataMat.append([float(lineArr[0]), float(lineArr[1])]) 
  labelMat.append(float(lineArr[2])) 
 return dataMat,labelMat 
 
def selectJrand(i,m): 
 j=i #we want to select any J not equal to i 
 while (j==i): 
  j = int(random.uniform(0,m)) 
 return j 
 
def clipAlpha(aj,H,L): 
 if aj > H: 
  aj = H 
 if L > aj: 
  aj = L 
 return aj 
 
def smoSimple(dataMatIn, classLabels, C, toler, maxIter): 
 dataMatrix = mat(dataMatIn); labelMat = mat(classLabels).transpose() 
 b = 0; m,n = shape(dataMatrix) 
 alphas = mat(zeros((m,1))) 
 iter = 0 
 while (iter < maxIter): 
  alphaPairsChanged = 0 
  for i in range(m): 
   fXi = float(multiply(alphas,labelMat).T*(dataMatrix*dataMatrix[i,:].T)) + b 
   Ei = fXi - float(labelMat[i])#if checks if an example violates KKT conditions 
   if ((labelMat[i]*Ei < -toler) and (alphas[i] < C)) or ((labelMat[i]*Ei > toler) and (alphas[i] > 0)): 
    j = selectJrand(i,m) 
    fXj = float(multiply(alphas,labelMat).T*(dataMatrix*dataMatrix[j,:].T)) + b 
    Ej = fXj - float(labelMat[j]) 
    alphaIold = alphas[i].copy(); alphaJold = alphas[j].copy(); 
    if (labelMat[i] != labelMat[j]): 
     L = max(0, alphas[j] - alphas[i]) 
     H = min(C, C + alphas[j] - alphas[i]) 
    else: 
     L = max(0, alphas[j] + alphas[i] - C) 
     H = min(C, alphas[j] + alphas[i]) 
    if L==H: print "L==H"; continue 
    eta = 2.0 * dataMatrix[i,:]*dataMatrix[j,:].T - dataMatrix[i,:]*dataMatrix[i,:].T - dataMatrix[j,:]*dataMatrix[j,:].T 
    if eta >= 0: print "eta>=0"; continue 
    alphas[j] -= labelMat[j]*(Ei - Ej)/eta 
    alphas[j] = clipAlpha(alphas[j],H,L) 
    if (abs(alphas[j] - alphaJold) < 0.00001): print "j not moving enough"; continue 
    alphas[i] += labelMat[j]*labelMat[i]*(alphaJold - alphas[j])#update i by the same amount as j 
                  #the update is in the oppostie direction 
    b1 = b - Ei- labelMat[i]*(alphas[i]-alphaIold)*dataMatrix[i,:]*dataMatrix[i,:].T - labelMat[j]*(alphas[j]-alphaJold)*dataMatrix[i,:]*dataMatrix[j,:].T 
    b2 = b - Ej- labelMat[i]*(alphas[i]-alphaIold)*dataMatrix[i,:]*dataMatrix[j,:].T - labelMat[j]*(alphas[j]-alphaJold)*dataMatrix[j,:]*dataMatrix[j,:].T 
    if (0 < alphas[i]) and (C > alphas[i]): b = b1 
    elif (0 < alphas[j]) and (C > alphas[j]): b = b2 
    else: b = (b1 + b2)/2.0 
    alphaPairsChanged += 1 
    print "iter: %d i:%d, pairs changed %d" % (iter,i,alphaPairsChanged) 
  if (alphaPairsChanged == 0): iter += 1 
  else: iter = 0 
  print "iteration number: %d" % iter 
 return b,alphas 
 
def kernelTrans(X, A, kTup): #calc the kernel or transform data to a higher dimensional space 
 m,n = shape(X) 
 K = mat(zeros((m,1))) 
 if kTup[0]=='lin': K = X * A.T #linear kernel 
 elif kTup[0]=='rbf': 
  for j in range(m): 
   deltaRow = X[j,:] - A 
   K[j] = deltaRow*deltaRow.T 
  K = exp(K/(-1*kTup[1]**2)) #divide in NumPy is element-wise not matrix like Matlab 
 else: raise NameError('Houston We Have a Problem -- \ 
 That Kernel is not recognized') 
 return K 
 
class optStruct: 
 def __init__(self,dataMatIn, classLabels, C, toler, kTup): # Initialize the structure with the parameters 
  self.X = dataMatIn 
  self.labelMat = classLabels 
  self.C = C 
  self.tol = toler 
  self.m = shape(dataMatIn)[0] 
  self.alphas = mat(zeros((self.m,1))) 
  self.b = 0 
  self.eCache = mat(zeros((self.m,2))) #first column is valid flag 
  self.K = mat(zeros((self.m,self.m))) 
  for i in range(self.m): 
   self.K[:,i] = kernelTrans(self.X, self.X[i,:], kTup) 
   
def calcEk(oS, k): 
 fXk = float(multiply(oS.alphas,oS.labelMat).T*oS.K[:,k] + oS.b) 
 Ek = fXk - float(oS.labelMat[k]) 
 return Ek 
   
def selectJ(i, oS, Ei):   #this is the second choice -heurstic, and calcs Ej 
 maxK = -1; maxDeltaE = 0; Ej = 0 
 oS.eCache[i] = [1,Ei] #set valid #choose the alpha that gives the maximum delta E 
 validEcacheList = nonzero(oS.eCache[:,0].A)[0] 
 if (len(validEcacheList)) > 1: 
  for k in validEcacheList: #loop through valid Ecache values and find the one that maximizes delta E 
   if k == i: continue #don't calc for i, waste of time 
   Ek = calcEk(oS, k) 
   deltaE = abs(Ei - Ek) 
   if (deltaE > maxDeltaE): 
    maxK = k; maxDeltaE = deltaE; Ej = Ek 
  return maxK, Ej 
 else: #in this case (first time around) we don't have any valid eCache values 
  j = selectJrand(i, oS.m) 
  Ej = calcEk(oS, j) 
 return j, Ej 
 
def updateEk(oS, k):#after any alpha has changed update the new value in the cache 
 Ek = calcEk(oS, k) 
 oS.eCache[k] = [1,Ek] 
   
def innerL(i, oS): 
 Ei = calcEk(oS, i) 
 if ((oS.labelMat[i]*Ei < -oS.tol) and (oS.alphas[i] < oS.C)) or ((oS.labelMat[i]*Ei > oS.tol) and (oS.alphas[i] > 0)): 
  j,Ej = selectJ(i, oS, Ei) #this has been changed from selectJrand 
  alphaIold = oS.alphas[i].copy(); alphaJold = oS.alphas[j].copy(); 
  if (oS.labelMat[i] != oS.labelMat[j]): 
   L = max(0, oS.alphas[j] - oS.alphas[i]) 
   H = min(oS.C, oS.C + oS.alphas[j] - oS.alphas[i]) 
  else: 
   L = max(0, oS.alphas[j] + oS.alphas[i] - oS.C) 
   H = min(oS.C, oS.alphas[j] + oS.alphas[i]) 
  if L==H: print "L==H"; return 0 
  eta = 2.0 * oS.K[i,j] - oS.K[i,i] - oS.K[j,j] #changed for kernel 
  if eta >= 0: print "eta>=0"; return 0 
  oS.alphas[j] -= oS.labelMat[j]*(Ei - Ej)/eta 
  oS.alphas[j] = clipAlpha(oS.alphas[j],H,L) 
  updateEk(oS, j) #added this for the Ecache 
  if (abs(oS.alphas[j] - alphaJold) < 0.00001): print "j not moving enough"; return 0 
  oS.alphas[i] += oS.labelMat[j]*oS.labelMat[i]*(alphaJold - oS.alphas[j])#update i by the same amount as j 
  updateEk(oS, i) #added this for the Ecache     #the update is in the oppostie direction 
  b1 = oS.b - Ei- oS.labelMat[i]*(oS.alphas[i]-alphaIold)*oS.K[i,i] - oS.labelMat[j]*(oS.alphas[j]-alphaJold)*oS.K[i,j] 
  b2 = oS.b - Ej- oS.labelMat[i]*(oS.alphas[i]-alphaIold)*oS.K[i,j]- oS.labelMat[j]*(oS.alphas[j]-alphaJold)*oS.K[j,j] 
  if (0 < oS.alphas[i]) and (oS.C > oS.alphas[i]): oS.b = b1 
  elif (0 < oS.alphas[j]) and (oS.C > oS.alphas[j]): oS.b = b2 
  else: oS.b = (b1 + b2)/2.0 
  return 1 
 else: return 0 
 
def smoP(dataMatIn, classLabels, C, toler, maxIter,kTup=('lin', 0)): #full Platt SMO 
 oS = optStruct(mat(dataMatIn),mat(classLabels).transpose(),C,toler, kTup) 
 iter = 0 
 entireSet = True; alphaPairsChanged = 0 
 while (iter < maxIter) and ((alphaPairsChanged > 0) or (entireSet)): 
  alphaPairsChanged = 0 
  if entireSet: #go over all 
   for i in range(oS.m):   
    alphaPairsChanged += innerL(i,oS) 
    print "fullSet, iter: %d i:%d, pairs changed %d" % (iter,i,alphaPairsChanged) 
   iter += 1 
  else:#go over non-bound (railed) alphas 
   nonBoundIs = nonzero((oS.alphas.A > 0) * (oS.alphas.A < C))[0] 
   for i in nonBoundIs: 
    alphaPairsChanged += innerL(i,oS) 
    print "non-bound, iter: %d i:%d, pairs changed %d" % (iter,i,alphaPairsChanged) 
   iter += 1 
  if entireSet: entireSet = False #toggle entire set loop 
  elif (alphaPairsChanged == 0): entireSet = True 
  print "iteration number: %d" % iter 
 return oS.b,oS.alphas 
 
def calcWs(alphas,dataArr,classLabels): 
 X = mat(dataArr); labelMat = mat(classLabels).transpose() 
 m,n = shape(X) 
 w = zeros((n,1)) 
 for i in range(m): 
  w += multiply(alphas[i]*labelMat[i],X[i,:].T) 
 return w 
 
def testRbf(k1=1.3): 
 dataArr,labelArr = loadDataSet('testSetRBF.txt') 
 b,alphas = smoP(dataArr, labelArr, 200, 0.0001, 10000, ('rbf', k1)) #C=200 important 
 datMat=mat(dataArr); labelMat = mat(labelArr).transpose() 
 svInd=nonzero(alphas.A>0)[0] 
 sVs=datMat[svInd] #get matrix of only support vectors 
 labelSV = labelMat[svInd]; 
 print "there are %d Support Vectors" % shape(sVs)[0] 
 m,n = shape(datMat) 
 errorCount = 0 
 for i in range(m): 
  kernelEval = kernelTrans(sVs,datMat[i,:],('rbf', k1)) 
  predict=kernelEval.T * multiply(labelSV,alphas[svInd]) + b 
  if sign(predict)!=sign(labelArr[i]): errorCount += 1 
 print "the training error rate is: %f" % (float(errorCount)/m) 
 dataArr,labelArr = loadDataSet('testSetRBF2.txt') 
 errorCount = 0 
 datMat=mat(dataArr); labelMat = mat(labelArr).transpose() 
 m,n = shape(datMat) 
 for i in range(m): 
  kernelEval = kernelTrans(sVs,datMat[i,:],('rbf', k1)) 
  predict=kernelEval.T * multiply(labelSV,alphas[svInd]) + b 
  if sign(predict)!=sign(labelArr[i]): errorCount += 1  
 print "the test error rate is: %f" % (float(errorCount)/m)  
  
def img2vector(filename): 
 returnVect = zeros((1,1024)) 
 fr = open(filename) 
 for i in range(32): 
  lineStr = fr.readline() 
  for j in range(32): 
   returnVect[0,32*i+j] = int(lineStr[j]) 
 return returnVect 
 
def loadImages(dirName): 
 from os import listdir 
 hwLabels = [] 
 trainingFileList = listdir(dirName)   #load the training set 
 m = len(trainingFileList) 
 trainingMat = zeros((m,1024)) 
 for i in range(m): 
  fileNameStr = trainingFileList[i] 
  fileStr = fileNameStr.split('.')[0]  #take off .txt 
  classNumStr = int(fileStr.split('_')[0]) 
  if classNumStr == 9: hwLabels.append(-1) 
  else: hwLabels.append(1) 
  trainingMat[i,:] = img2vector('%s/%s' % (dirName, fileNameStr)) 
 return trainingMat, hwLabels  
 
def testDigits(kTup=('rbf', 10)): 
 dataArr,labelArr = loadImages('trainingDigits') 
 b,alphas = smoP(dataArr, labelArr, 200, 0.0001, 10000, kTup) 
 datMat=mat(dataArr); labelMat = mat(labelArr).transpose() 
 svInd=nonzero(alphas.A>0)[0] 
 sVs=datMat[svInd] 
 labelSV = labelMat[svInd]; 
 print "there are %d Support Vectors" % shape(sVs)[0] 
 m,n = shape(datMat) 
 errorCount = 0 
 for i in range(m): 
  kernelEval = kernelTrans(sVs,datMat[i,:],kTup) 
  predict=kernelEval.T * multiply(labelSV,alphas[svInd]) + b 
  if sign(predict)!=sign(labelArr[i]): errorCount += 1 
 print "the training error rate is: %f" % (float(errorCount)/m) 
 dataArr,labelArr = loadImages('testDigits') 
 errorCount = 0 
 datMat=mat(dataArr); labelMat = mat(labelArr).transpose() 
 m,n = shape(datMat) 
 for i in range(m): 
  kernelEval = kernelTrans(sVs,datMat[i,:],kTup) 
  predict=kernelEval.T * multiply(labelSV,alphas[svInd]) + b 
  if sign(predict)!=sign(labelArr[i]): errorCount += 1  
 print "the test error rate is: %f" % (float(errorCount)/m) 
 
 
'''''#######******************************** 
Non-Kernel VErsions below 
'''#######******************************** 
 
class optStructK: 
 def __init__(self,dataMatIn, classLabels, C, toler): # Initialize the structure with the parameters 
  self.X = dataMatIn 
  self.labelMat = classLabels 
  self.C = C 
  self.tol = toler 
  self.m = shape(dataMatIn)[0] 
  self.alphas = mat(zeros((self.m,1))) 
  self.b = 0 
  self.eCache = mat(zeros((self.m,2))) #first column is valid flag 
   
def calcEkK(oS, k): 
 fXk = float(multiply(oS.alphas,oS.labelMat).T*(oS.X*oS.X[k,:].T)) + oS.b 
 Ek = fXk - float(oS.labelMat[k]) 
 return Ek 
   
def selectJK(i, oS, Ei):   #this is the second choice -heurstic, and calcs Ej 
 maxK = -1; maxDeltaE = 0; Ej = 0 
 oS.eCache[i] = [1,Ei] #set valid #choose the alpha that gives the maximum delta E 
 validEcacheList = nonzero(oS.eCache[:,0].A)[0] 
 if (len(validEcacheList)) > 1: 
  for k in validEcacheList: #loop through valid Ecache values and find the one that maximizes delta E 
   if k == i: continue #don't calc for i, waste of time 
   Ek = calcEk(oS, k) 
   deltaE = abs(Ei - Ek) 
   if (deltaE > maxDeltaE): 
    maxK = k; maxDeltaE = deltaE; Ej = Ek 
  return maxK, Ej 
 else: #in this case (first time around) we don't have any valid eCache values 
  j = selectJrand(i, oS.m) 
  Ej = calcEk(oS, j) 
 return j, Ej 
 
def updateEkK(oS, k):#after any alpha has changed update the new value in the cache 
 Ek = calcEk(oS, k) 
 oS.eCache[k] = [1,Ek] 
   
def innerLK(i, oS): 
 Ei = calcEk(oS, i) 
 if ((oS.labelMat[i]*Ei < -oS.tol) and (oS.alphas[i] < oS.C)) or ((oS.labelMat[i]*Ei > oS.tol) and (oS.alphas[i] > 0)): 
  j,Ej = selectJ(i, oS, Ei) #this has been changed from selectJrand 
  alphaIold = oS.alphas[i].copy(); alphaJold = oS.alphas[j].copy(); 
  if (oS.labelMat[i] != oS.labelMat[j]): 
   L = max(0, oS.alphas[j] - oS.alphas[i]) 
   H = min(oS.C, oS.C + oS.alphas[j] - oS.alphas[i]) 
  else: 
   L = max(0, oS.alphas[j] + oS.alphas[i] - oS.C) 
   H = min(oS.C, oS.alphas[j] + oS.alphas[i]) 
  if L==H: print "L==H"; return 0 
  eta = 2.0 * oS.X[i,:]*oS.X[j,:].T - oS.X[i,:]*oS.X[i,:].T - oS.X[j,:]*oS.X[j,:].T 
  if eta >= 0: print "eta>=0"; return 0 
  oS.alphas[j] -= oS.labelMat[j]*(Ei - Ej)/eta 
  oS.alphas[j] = clipAlpha(oS.alphas[j],H,L) 
  updateEk(oS, j) #added this for the Ecache 
  if (abs(oS.alphas[j] - alphaJold) < 0.00001): print "j not moving enough"; return 0 
  oS.alphas[i] += oS.labelMat[j]*oS.labelMat[i]*(alphaJold - oS.alphas[j])#update i by the same amount as j 
  updateEk(oS, i) #added this for the Ecache     #the update is in the oppostie direction 
  b1 = oS.b - Ei- oS.labelMat[i]*(oS.alphas[i]-alphaIold)*oS.X[i,:]*oS.X[i,:].T - oS.labelMat[j]*(oS.alphas[j]-alphaJold)*oS.X[i,:]*oS.X[j,:].T 
  b2 = oS.b - Ej- oS.labelMat[i]*(oS.alphas[i]-alphaIold)*oS.X[i,:]*oS.X[j,:].T - oS.labelMat[j]*(oS.alphas[j]-alphaJold)*oS.X[j,:]*oS.X[j,:].T 
  if (0 < oS.alphas[i]) and (oS.C > oS.alphas[i]): oS.b = b1 
  elif (0 < oS.alphas[j]) and (oS.C > oS.alphas[j]): oS.b = b2 
  else: oS.b = (b1 + b2)/2.0 
  return 1 
 else: return 0 
 
def smoPK(dataMatIn, classLabels, C, toler, maxIter): #full Platt SMO 
 oS = optStruct(mat(dataMatIn),mat(classLabels).transpose(),C,toler) 
 iter = 0 
 entireSet = True; alphaPairsChanged = 0 
 while (iter < maxIter) and ((alphaPairsChanged > 0) or (entireSet)): 
  alphaPairsChanged = 0 
  if entireSet: #go over all 
   for i in range(oS.m):   
    alphaPairsChanged += innerL(i,oS) 
    print "fullSet, iter: %d i:%d, pairs changed %d" % (iter,i,alphaPairsChanged) 
   iter += 1 
  else:#go over non-bound (railed) alphas 
   nonBoundIs = nonzero((oS.alphas.A > 0) * (oS.alphas.A < C))[0] 
   for i in nonBoundIs: 
    alphaPairsChanged += innerL(i,oS) 
    print "non-bound, iter: %d i:%d, pairs changed %d" % (iter,i,alphaPairsChanged) 
   iter += 1 
  if entireSet: entireSet = False #toggle entire set loop 
  elif (alphaPairsChanged == 0): entireSet = True 
  print "iteration number: %d" % iter 
 return oS.b,oS.alphas

运行结果如(图八)所示:

python机器学习理论与实战(六)支持向量机

(图八)

上面代码有兴趣的可以读读,用的话,建议使用libsvm。

参考文献:

    [1]machine learning in action. PeterHarrington

    [2] pattern recognition and machinelearning. Christopher M. Bishop

    [3]machine learning.Andrew Ng

以上就是本文的全部内容,希望对大家的学习有所帮助,也希望大家多多支持三水点靠木。

Python 相关文章推荐
python条件和循环的使用方法
Nov 01 Python
python实现html转ubb代码(html2ubb)
Jul 03 Python
TensorFLow用Saver保存和恢复变量
Mar 10 Python
Python基于property实现类的特性操作示例
Jun 15 Python
Python实现绘制双柱状图并显示数值功能示例
Jun 23 Python
Python使用itertools模块实现排列组合功能示例
Jul 02 Python
Python实现监控键盘鼠标操作示例【基于pyHook与pythoncom模块】
Sep 04 Python
Python嵌套式数据结构实例浅析
Mar 05 Python
详解python解压压缩包的五种方法
Jul 05 Python
python文件排序的方法总结
Sep 13 Python
详解python 支持向量机(SVM)算法
Sep 18 Python
Python读写锁实现实现代码解析
Nov 28 Python
Python logging管理不同级别log打印和存储实例
Jan 19 #Python
python机器学习理论与实战(五)支持向量机
Jan 19 #Python
Python读取图片为16进制表示简单代码
Jan 19 #Python
Python实现pdf文档转txt的方法示例
Jan 19 #Python
浅谈Python实现2种文件复制的方法
Jan 19 #Python
用Python进行简单图像识别(验证码)
Jan 19 #Python
flask中使用蓝图将路由分开写在不同文件实例解析
Jan 19 #Python
You might like
PHP正则提取不包含指定网址的图片地址的例子
2014/04/21 PHP
php使用date和strtotime函数输出指定日期的方法
2014/11/14 PHP
php+ajax无刷新分页实例详解
2015/12/07 PHP
PHP使用mongoclient简单操作mongodb数据库示例
2019/02/08 PHP
JavaScript 基础问答三
2008/12/03 Javascript
Jquery 实现Tab效果 思路是js思路
2010/03/02 Javascript
js各种验证文本框输入格式(正则表达式)
2010/10/22 Javascript
在IE和VB中支持png图片透明效果的实现方法(vb源码打包)
2011/04/01 Javascript
javascript动态加载三
2012/08/22 Javascript
jquery ready函数、css函数及text()使用示例
2013/09/27 Javascript
jquery 删除cookie失效的解决方法
2013/11/12 Javascript
jquery实现弹出层遮罩效果的简单实例
2014/03/03 Javascript
js判断滚动条是否已到页面最底部或顶部实例
2014/11/20 Javascript
JS模拟酷狗音乐播放器收缩折叠关闭效果代码
2015/10/29 Javascript
JS模仿手机端九宫格登录功能实现代码
2016/04/28 Javascript
防止重复发送 Ajax 请求
2017/02/15 Javascript
Ajax和Comet技术总结
2017/02/19 Javascript
AngularJS表格添加序号的方法
2017/03/03 Javascript
基于JavaScript+HTML5 实现打地鼠小游戏逻辑流程图文详解(附完整代码)
2017/11/02 Javascript
vue轮播图插件vue-awesome-swiper
2017/11/27 Javascript
Vue 中使用vue2-highcharts实现top功能的示例
2018/03/05 Javascript
mpvue网易云短信接口实现小程序短信登录的示例代码
2020/04/03 Javascript
在Python中用split()方法分割字符串的使用介绍
2015/05/20 Python
Python从零开始创建区块链
2018/03/06 Python
Python 中的lambda函数介绍
2018/10/10 Python
python3+opencv生成不规则黑白mask实例
2020/02/19 Python
python开发入门——列表生成式
2020/09/03 Python
纯css3实现思维导图样式示例
2018/11/01 HTML / CSS
HTML5的标签的代码的简单介绍 HTML5标签的简介
2012/05/28 HTML / CSS
贝斯特韦斯特酒店集团官网:Best Western
2019/01/03 全球购物
LORAC官网:美国彩妆品牌
2019/08/27 全球购物
会计电算化个人自我评价
2013/11/17 职场文书
化验室技术员岗位职责
2013/12/24 职场文书
自荐信写法介绍
2014/01/25 职场文书
消防安全宣传口号
2014/06/10 职场文书
MySQL学习必备条件查询数据
2022/03/25 MySQL