Pytorch 使用CNN图像分类的实现


Posted in Python onJune 16, 2020

需求

在4*4的图片中,比较外围黑色像素点和内圈黑色像素点个数的大小将图片分类

Pytorch 使用CNN图像分类的实现

如上图图片外围黑色像素点5个大于内圈黑色像素点1个分为0类反之1类

想法

  • 通过numpy、PIL构造4*4的图像数据集
  • 构造自己的数据集类
  • 读取数据集对数据集选取减少偏斜
  • cnn设计因为特征少,直接1*1卷积层
  • 或者在4*4外围添加padding成6*6,设计2*2的卷积核得出3*3再接上全连接层

代码

import torch
import torchvision
import torchvision.transforms as transforms
import numpy as np
from PIL import Image

构造数据集

import csv
import collections
import os
import shutil

def buildDataset(root,dataType,dataSize):
  """构造数据集
  构造的图片存到root/{dataType}Data
  图片地址和标签的csv文件存到 root/{dataType}DataInfo.csv
  Args:
    root:str
      项目目录
    dataType:str
      'train'或者‘test'
    dataNum:int
      数据大小
  Returns:
  """
  dataInfo = []
  dataPath = f'{root}/{dataType}Data'
  if not os.path.exists(dataPath):
    os.makedirs(dataPath)
  else:
    shutil.rmtree(dataPath)
    os.mkdir(dataPath)
    
  for i in range(dataSize):
    # 创建0,1 数组
    imageArray=np.random.randint(0,2,(4,4))
    # 计算0,1数量得到标签
    allBlackNum = collections.Counter(imageArray.flatten())[0]
    innerBlackNum = collections.Counter(imageArray[1:3,1:3].flatten())[0]
    label = 0 if (allBlackNum-innerBlackNum)>innerBlackNum else 1
    # 将图片保存
    path = f'{dataPath}/{i}.jpg'
    dataInfo.append([path,label])
    im = Image.fromarray(np.uint8(imageArray*255))
    im = im.convert('1') 
    im.save(path)
  # 将图片地址和标签存入csv文件
  filePath = f'{root}/{dataType}DataInfo.csv'
  with open(filePath, 'w') as f:
    writer = csv.writer(f)
    writer.writerows(dataInfo)
root=r'/Users/null/Documents/PythonProject/Classifier'

构造训练数据集

buildDataset(root,'train',20000)

构造测试数据集

buildDataset(root,'test',10000)

读取数据集

class MyDataset(torch.utils.data.Dataset):

  def __init__(self, root, datacsv, transform=None):
    super(MyDataset, self).__init__()
    with open(f'{root}/{datacsv}', 'r') as f:
      imgs = []
      # 读取csv信息到imgs列表
      for path,label in map(lambda line:line.rstrip().split(','),f):
        imgs.append((path, int(label)))
    self.imgs = imgs
    self.transform = transform if transform is not None else lambda x:x
    
  def __getitem__(self, index):
    path, label = self.imgs[index]
    img = self.transform(Image.open(path).convert('1'))
    return img, label

  def __len__(self):
    return len(self.imgs)
trainData=MyDataset(root = root,datacsv='trainDataInfo.csv', transform=transforms.ToTensor())
testData=MyDataset(root = root,datacsv='testDataInfo.csv', transform=transforms.ToTensor())

处理数据集使得数据集不偏斜

import itertools

def chooseData(dataset,scale):
  # 将类别为1的排序到前面
  dataset.imgs.sort(key=lambda x:x[1],reverse=True)
  # 获取类别1的数目 ,取scale倍的数组,得数据不那么偏斜
  trueNum =collections.Counter(itertools.chain.from_iterable(dataset.imgs))[1]
  end = min(trueNum*scale,len(dataset))
  dataset.imgs=dataset.imgs[:end]
scale = 4
chooseData(trainData,scale)
chooseData(testData,scale)
len(trainData),len(testData)
(2250, 1122)
import torch.utils.data as Data

# 超参数
batchSize = 50
lr = 0.1
numEpochs = 20

trainIter = Data.DataLoader(dataset=trainData, batch_size=batchSize, shuffle=True)
testIter = Data.DataLoader(dataset=testData, batch_size=batchSize)

定义模型

from torch import nn
from torch.autograd import Variable
from torch.nn import Module,Linear,Sequential,Conv2d,ReLU,ConstantPad2d
import torch.nn.functional as F
class Net(Module):  
  def __init__(self):
    super(Net, self).__init__()

    self.cnnLayers = Sequential(
      # padding添加1层常数1,设定卷积核为2*2
      ConstantPad2d(1, 1),
      Conv2d(1, 1, kernel_size=2, stride=2,bias=True)
    )
    self.linearLayers = Sequential(
      Linear(9, 2)
    )

  def forward(self, x):
    x = self.cnnLayers(x)
    x = x.view(x.shape[0], -1)
    x = self.linearLayers(x)
    return x
class Net2(Module):  
  def __init__(self):
    super(Net2, self).__init__()

    self.cnnLayers = Sequential(
      Conv2d(1, 1, kernel_size=1, stride=1,bias=True)
    )
    self.linearLayers = Sequential(
      ReLU(),
      Linear(16, 2)
    )

  def forward(self, x):
    x = self.cnnLayers(x)
    x = x.view(x.shape[0], -1)
    x = self.linearLayers(x)
    return x

定义损失函数

# 交叉熵损失函数
loss = nn.CrossEntropyLoss()
loss2 = nn.CrossEntropyLoss()

定义优化算法

net = Net()
optimizer = torch.optim.SGD(net.parameters(),lr = lr)
net2 = Net2()
optimizer2 = torch.optim.SGD(net2.parameters(),lr = lr)

训练模型

# 计算准确率
def evaluateAccuracy(dataIter, net):
  accSum, n = 0.0, 0
  with torch.no_grad():
    for X, y in dataIter:
      accSum += (net(X).argmax(dim=1) == y).float().sum().item()
      n += y.shape[0]
  return accSum / n
def train(net, trainIter, testIter, loss, numEpochs, batchSize,
       optimizer):
  for epoch in range(numEpochs):
    trainLossSum, trainAccSum, n = 0.0, 0.0, 0
    for X,y in trainIter:
      yHat = net(X)
      l = loss(yHat,y).sum()
      optimizer.zero_grad()
      l.backward()
      optimizer.step()
      # 计算训练准确度和loss
      trainLossSum += l.item()
      trainAccSum += (yHat.argmax(dim=1) == y).sum().item()
      n += y.shape[0]
    # 评估测试准确度
    testAcc = evaluateAccuracy(testIter, net)
    print('epoch {:d}, loss {:.4f}, train acc {:.3f}, test acc {:.3f}'.format(epoch + 1, trainLossSum / n, trainAccSum / n, testAcc))

Net模型训练

train(net, trainIter, testIter, loss, numEpochs, batchSize,optimizer)
epoch 1, loss 0.0128, train acc 0.667, test acc 0.667
epoch 2, loss 0.0118, train acc 0.683, test acc 0.760
epoch 3, loss 0.0104, train acc 0.742, test acc 0.807
epoch 4, loss 0.0093, train acc 0.769, test acc 0.772
epoch 5, loss 0.0085, train acc 0.797, test acc 0.745
epoch 6, loss 0.0084, train acc 0.798, test acc 0.807
epoch 7, loss 0.0082, train acc 0.804, test acc 0.816
epoch 8, loss 0.0078, train acc 0.816, test acc 0.812
epoch 9, loss 0.0077, train acc 0.818, test acc 0.817
epoch 10, loss 0.0074, train acc 0.824, test acc 0.826
epoch 11, loss 0.0072, train acc 0.836, test acc 0.819
epoch 12, loss 0.0075, train acc 0.823, test acc 0.829
epoch 13, loss 0.0071, train acc 0.839, test acc 0.797
epoch 14, loss 0.0067, train acc 0.849, test acc 0.824
epoch 15, loss 0.0069, train acc 0.848, test acc 0.843
epoch 16, loss 0.0064, train acc 0.864, test acc 0.851
epoch 17, loss 0.0062, train acc 0.867, test acc 0.780
epoch 18, loss 0.0060, train acc 0.871, test acc 0.864
epoch 19, loss 0.0057, train acc 0.881, test acc 0.890
epoch 20, loss 0.0055, train acc 0.885, test acc 0.897

Net2模型训练

# batchSize = 50 
# lr = 0.1
# numEpochs = 15 下得出的结果
train(net2, trainIter, testIter, loss2, numEpochs, batchSize,optimizer2)

epoch 1, loss 0.0119, train acc 0.638, test acc 0.676
epoch 2, loss 0.0079, train acc 0.823, test acc 0.986
epoch 3, loss 0.0046, train acc 0.987, test acc 0.977
epoch 4, loss 0.0030, train acc 0.983, test acc 0.973
epoch 5, loss 0.0023, train acc 0.981, test acc 0.976
epoch 6, loss 0.0019, train acc 0.980, test acc 0.988
epoch 7, loss 0.0016, train acc 0.984, test acc 0.984
epoch 8, loss 0.0014, train acc 0.985, test acc 0.986
epoch 9, loss 0.0013, train acc 0.987, test acc 0.992
epoch 10, loss 0.0011, train acc 0.989, test acc 0.993
epoch 11, loss 0.0010, train acc 0.989, test acc 0.996
epoch 12, loss 0.0010, train acc 0.992, test acc 0.994
epoch 13, loss 0.0009, train acc 0.993, test acc 0.994
epoch 14, loss 0.0008, train acc 0.995, test acc 0.996
epoch 15, loss 0.0008, train acc 0.994, test acc 0.998

测试

test = torch.Tensor([[[[0,0,0,0],[0,1,1,0],[0,1,1,0],[0,0,0,0]]],
         [[[1,1,1,1],[1,0,0,1],[1,0,0,1],[1,1,1,1]]],
         [[[0,1,0,1],[1,0,0,1],[1,0,0,1],[0,0,0,1]]],
         [[[0,1,1,1],[1,0,0,1],[1,0,0,1],[0,0,0,1]]],
         [[[0,0,1,1],[1,0,0,1],[1,0,0,1],[1,0,1,0]]],
         [[[0,0,1,0],[0,1,0,1],[0,0,1,1],[1,0,1,0]]],
         [[[1,1,1,0],[1,0,0,1],[1,0,1,1],[1,0,1,1]]]
         ])

target=torch.Tensor([0,1,0,1,1,0,1])
test
tensor([[[[0., 0., 0., 0.],
     [0., 1., 1., 0.],
     [0., 1., 1., 0.],
     [0., 0., 0., 0.]]],

​

    [[[1., 1., 1., 1.],
     [1., 0., 0., 1.],
     [1., 0., 0., 1.],
     [1., 1., 1., 1.]]],

​

    [[[0., 1., 0., 1.],
     [1., 0., 0., 1.],
     [1., 0., 0., 1.],
     [0., 0., 0., 1.]]],

​

    [[[0., 1., 1., 1.],
     [1., 0., 0., 1.],
     [1., 0., 0., 1.],
     [0., 0., 0., 1.]]],

​

    [[[0., 0., 1., 1.],
     [1., 0., 0., 1.],
     [1., 0., 0., 1.],
     [1., 0., 1., 0.]]],

​

    [[[0., 0., 1., 0.],
     [0., 1., 0., 1.],
     [0., 0., 1., 1.],
     [1., 0., 1., 0.]]],

​

    [[[1., 1., 1., 0.],
     [1., 0., 0., 1.],
     [1., 0., 1., 1.],
     [1., 0., 1., 1.]]]])



with torch.no_grad():
  output = net(test)
  output2 = net2(test)
predictions =output.argmax(dim=1)
predictions2 =output2.argmax(dim=1)
# 比较结果
print(f'Net测试结果{predictions.eq(target)}')
print(f'Net2测试结果{predictions2.eq(target)}')
Net测试结果tensor([ True, True, False, True, True, True, True])
Net2测试结果tensor([False, True, False, True, True, False, True])

到此这篇关于Pytorch 使用CNN图像分类的实现的文章就介绍到这了,更多相关Pytorch CNN图像分类内容请搜索三水点靠木以前的文章或继续浏览下面的相关文章希望大家以后多多支持三水点靠木!

Python 相关文章推荐
python模拟登陆阿里妈妈生成商品推广链接
Apr 03 Python
python的tkinter布局之简单的聊天窗口实现方法
Sep 03 Python
Python的Django框架中从url中捕捉文本的方法
Jul 20 Python
Python检测生僻字的实现方法
Oct 23 Python
Python正则表达式经典入门教程
May 22 Python
Python+PIL实现支付宝AR红包
Feb 09 Python
一百多行python代码实现抢票助手
Sep 25 Python
解决python中os.listdir()函数读取文件夹下文件的乱序和排序问题
Oct 17 Python
python 实现list或string按指定分段
Dec 25 Python
Python实现封装打包自己写的代码,被python import
Jul 12 Python
Python -m参数原理及使用方法解析
Aug 21 Python
python 5个实用的技巧
Sep 27 Python
利用python中的matplotlib打印混淆矩阵实例
Jun 16 #Python
Python SMTP配置参数并发送邮件
Jun 16 #Python
基于matplotlib中ion()和ioff()的使用详解
Jun 16 #Python
Python数据相关系数矩阵和热力图轻松实现教程
Jun 16 #Python
matplotlib.pyplot.matshow 矩阵可视化实例
Jun 16 #Python
使用python matploblib库绘制准确率,损失率折线图
Jun 16 #Python
为什么称python为胶水语言
Jun 16 #Python
You might like
php下批量挂马和批量清马代码
2011/02/27 PHP
PHPExcel内存泄漏问题解决方法
2015/01/23 PHP
PHPStrom 新建FTP项目以及在线操作教程
2016/10/16 PHP
laravel 框架结合关联查询 when()用法分析
2019/11/22 PHP
javascript 贪吃蛇实现代码
2008/11/22 Javascript
js中的window.open返回object的错误的解决方法
2009/08/15 Javascript
网站页面自动跳转实现方法PHP、JSP(下)
2010/08/01 Javascript
TimergliderJS 一个基于jQuery的时间轴插件
2011/12/07 Javascript
怎么清空javascript数组
2013/05/11 Javascript
node+express+jade制作简单网站指南
2014/11/26 Javascript
nodejs开发微博实例
2015/03/25 NodeJs
跟我学习javascript的异步脚本加载
2015/11/20 Javascript
javascript实现简单计算器效果【推荐】
2016/04/19 Javascript
老生常谈js-react组件生命周期
2017/05/02 Javascript
详解利用 Express 托管静态文件的方法
2017/09/18 Javascript
详解如何优雅地在React项目中使用Redux
2017/12/28 Javascript
vue axios 给生产环境和发布环境配置不同的接口地址(推荐)
2018/05/08 Javascript
vue devtools的安装与使用教程
2018/08/08 Javascript
使用layui日期控件laydate对开始和结束时间进行联动控制的方法
2019/09/06 Javascript
Vue 实现把表单form数据 转化成json格式的数据
2019/10/29 Javascript
jQuery实现推拉门效果
2020/10/19 jQuery
python中的计时器timeit的使用方法
2017/10/20 Python
Django中url的反向查询的方法
2018/03/14 Python
python脚本监控Tomcat服务器的方法
2018/07/06 Python
python快速编写单行注释多行注释的方法
2019/07/31 Python
HTTP状态码详解
2021/03/18 杂记
顶丰TOPPIK台湾官网:增发纤维假发,告别秃发困扰
2018/06/13 全球购物
Juicy Couture Beauty官方网站:香水和化妆品
2019/03/12 全球购物
物流仓储实习自我鉴定
2013/09/25 职场文书
外贸业务员的岗位职责
2013/11/23 职场文书
手术室护士长竞聘书
2014/03/31 职场文书
2014年安全生产责任书
2014/07/22 职场文书
委托培训协议书
2014/11/17 职场文书
门市房租房协议书
2014/12/04 职场文书
家长通知书家长意见
2014/12/30 职场文书
如何使用JavaScript策略模式校验表单
2021/04/29 Javascript