python 牛顿法实现逻辑回归(Logistic Regression)


Posted in Python onOctober 15, 2020

本文采用的训练方法是牛顿法(Newton Method)。

代码

import numpy as np

class LogisticRegression(object):
 """
 Logistic Regression Classifier training by Newton Method
 """

 def __init__(self, error: float = 0.7, max_epoch: int = 100):
  """
  :param error: float, if the distance between new weight and 
      old weight is less than error, the process 
      of traing will break.
  :param max_epoch: if training epoch >= max_epoch the process 
       of traing will break.
  """
  self.error = error
  self.max_epoch = max_epoch
  self.weight = None
  self.sign = np.vectorize(lambda x: 1 if x >= 0.5 else 0)

 def p_func(self, X_):
  """Get P(y=1 | x)
  :param X_: shape = (n_samples + 1, n_features)
  :return: shape = (n_samples)
  """
  tmp = np.exp(self.weight @ X_.T)
  return tmp / (1 + tmp)

 def diff(self, X_, y, p):
  """Get derivative
  :param X_: shape = (n_samples, n_features + 1) 
  :param y: shape = (n_samples)
  :param p: shape = (n_samples) P(y=1 | x)
  :return: shape = (n_features + 1) first derivative
  """
  return -(y - p) @ X_

 def hess_mat(self, X_, p):
  """Get Hessian Matrix
  :param p: shape = (n_samples) P(y=1 | x)
  :return: shape = (n_features + 1, n_features + 1) second derivative
  """
  hess = np.zeros((X_.shape[1], X_.shape[1]))
  for i in range(X_.shape[0]):
   hess += self.X_XT[i] * p[i] * (1 - p[i])
  return hess

 def newton_method(self, X_, y):
  """Newton Method to calculate weight
  :param X_: shape = (n_samples + 1, n_features)
  :param y: shape = (n_samples)
  :return: None
  """
  self.weight = np.ones(X_.shape[1])
  self.X_XT = []
  for i in range(X_.shape[0]):
   t = X_[i, :].reshape((-1, 1))
   self.X_XT.append(t @ t.T)

  for _ in range(self.max_epoch):
   p = self.p_func(X_)
   diff = self.diff(X_, y, p)
   hess = self.hess_mat(X_, p)
   new_weight = self.weight - (np.linalg.inv(hess) @ diff.reshape((-1, 1))).flatten()

   if np.linalg.norm(new_weight - self.weight) <= self.error:
    break
   self.weight = new_weight

 def fit(self, X, y):
  """
  :param X_: shape = (n_samples, n_features)
  :param y: shape = (n_samples)
  :return: self
  """
  X_ = np.c_[np.ones(X.shape[0]), X]
  self.newton_method(X_, y)
  return self

 def predict(self, X) -> np.array:
  """
  :param X: shape = (n_samples, n_features] 
  :return: shape = (n_samples]
  """
  X_ = np.c_[np.ones(X.shape[0]), X]
  return self.sign(self.p_func(X_))

测试代码

import matplotlib.pyplot as plt
import sklearn.datasets

def plot_decision_boundary(pred_func, X, y, title=None):
 """分类器画图函数,可画出样本点和决策边界
 :param pred_func: predict函数
 :param X: 训练集X
 :param y: 训练集Y
 :return: None
 """

 # Set min and max values and give it some padding
 x_min, x_max = X[:, 0].min() - .5, X[:, 0].max() + .5
 y_min, y_max = X[:, 1].min() - .5, X[:, 1].max() + .5
 h = 0.01
 # Generate a grid of points with distance h between them
 xx, yy = np.meshgrid(np.arange(x_min, x_max, h), np.arange(y_min, y_max, h))
 # Predict the function value for the whole gid
 Z = pred_func(np.c_[xx.ravel(), yy.ravel()])
 Z = Z.reshape(xx.shape)
 # Plot the contour and training examples
 plt.contourf(xx, yy, Z, cmap=plt.cm.Spectral)
 plt.scatter(X[:, 0], X[:, 1], s=40, c=y, cmap=plt.cm.Spectral)
 if title:
  plt.title(title)
 plt.show()

效果

python 牛顿法实现逻辑回归(Logistic Regression)

更多机器学习代码,请访问 https://github.com/WiseDoge/plume

以上就是python 牛顿法实现逻辑回归(Logistic Regression)的详细内容,更多关于python 逻辑回归的资料请关注三水点靠木其它相关文章!

Python 相关文章推荐
pymssql ntext字段调用问题解决方法
Dec 17 Python
python进阶教程之动态类型详解
Aug 30 Python
详解Python中的join()函数的用法
Apr 07 Python
python基于phantomjs实现导入图片
May 13 Python
Python调用服务接口的实例
Jan 03 Python
python turtle库画一个方格和圆实例
Jun 27 Python
python3中替换python2中cmp函数的实现
Aug 20 Python
Python Opencv提取图片中某种颜色组成的图形的方法
Sep 19 Python
基于Python检测动态物体颜色过程解析
Dec 04 Python
python 实现字符串下标的输出功能
Feb 13 Python
python实现用户名密码校验
Mar 18 Python
Django表单提交后实现获取相同name的不同value值
May 14 Python
PyCharm 2020.2.2 x64 下载并安装的详细教程
Oct 15 #Python
Python 实现3种回归模型(Linear Regression,Lasso,Ridge)的示例
Oct 15 #Python
Python在centos7.6上安装python3.9的详细教程(默认python版本为2.7.5)
Oct 15 #Python
Pycharm编辑器功能之代码折叠效果的实现代码
Oct 15 #Python
如何用Python 实现全连接神经网络(Multi-layer Perceptron)
Oct 15 #Python
python 实现非极大值抑制算法(Non-maximum suppression, NMS)
Oct 15 #Python
解决pip安装的第三方包在PyCharm无法导入的问题
Oct 15 #Python
You might like
PHP session有效期问题
2009/04/26 PHP
基于PHP服务端图片生成缩略图的方法详解
2013/06/20 PHP
php中try catch捕获异常实例详解
2014/11/21 PHP
ThinkPHP连接数据库的方式汇总
2014/12/05 PHP
一个原生的用户等级的进度条
2010/07/03 Javascript
jQuery Pagination Ajax分页插件(分页切换时无刷新与延迟)中文翻译版
2013/01/11 Javascript
Javascript简单改变表单元素背景的方法
2015/07/15 Javascript
jQuery实现div随意拖动的实例代码(通用代码)
2016/01/28 Javascript
Angularjs CURD 详解及实例代码
2016/09/14 Javascript
jquery实现轮播图效果
2017/02/13 Javascript
BootStrap实现文件上传并带有进度条效果
2017/09/11 Javascript
极简主义法编写JavaScript类
2017/11/02 Javascript
微信小程序模版渲染详解
2018/01/26 Javascript
JS加密插件CryptoJS实现的DES加密示例
2018/08/16 Javascript
vue生成文件本地打开查看效果的实例
2018/09/06 Javascript
基于Nodejs的Tcp封包和解包的理解
2018/09/19 NodeJs
vue调用本地摄像头实现拍照功能
2020/08/14 Javascript
以Flask为例讲解Python的框架的使用方法
2015/04/29 Python
在Python程序中操作文件之flush()方法的使用教程
2015/05/24 Python
Python 安装第三方库 pip install 安装慢安装不上的解决办法
2019/06/18 Python
python selenium 查找隐藏元素 自动播放视频功能
2019/07/24 Python
python多线程同步实例教程
2019/08/11 Python
Python实现Restful API的例子
2019/08/31 Python
python 爬虫百度地图的信息界面的实现方法
2019/10/27 Python
python中利用matplotlib读取灰度图的例子
2019/12/07 Python
python global和nonlocal用法解析
2020/02/03 Python
TensorFlow实现保存训练模型为pd文件并恢复
2020/02/06 Python
CSS3实现银灰色动画效果的导航菜单代码
2015/09/01 HTML / CSS
C语言编程练习
2012/04/02 面试题
个人自我鉴定范文
2013/10/04 职场文书
四群教育工作实施方案
2014/03/26 职场文书
数学高效课堂实施方案
2014/03/29 职场文书
2014年“四风”问题个人整改措施
2014/09/17 职场文书
毕业论文答辩演讲稿
2015/06/23 职场文书
解决Python字典查找报Keyerror的问题
2021/05/26 Python
Python序列化与反序列化相关知识总结
2021/06/08 Python