Python机器学习之基于Pytorch实现猫狗分类


Posted in Python onJune 08, 2021

一、环境配置

安装Anaconda

配置Pytorch

pip install -i https://pypi.tuna.tsinghua.edu.cn/simple torch
pip install -i https://pypi.tuna.tsinghua.edu.cn/simple torchvision

二、数据集的准备

1.数据集的下载

kaggle网站的数据集下载地址:
https://www.kaggle.com/lizhensheng/-2000

2.数据集的分类

将下载的数据集进行解压操作,然后进行分类
分类如下(每个文件夹下包括cats和dogs文件夹)

Python机器学习之基于Pytorch实现猫狗分类 

三、猫狗分类的实例

导入相应的库

# 导入库
import torch.nn.functional as F
import torch.optim as optim
import torch
import torch.nn as nn
import torch.nn.parallel
 
import torch.optim
import torch.utils.data
import torch.utils.data.distributed
import torchvision.transforms as transforms
import torchvision.datasets as datasets

设置超参数

# 设置超参数
#每次的个数
BATCH_SIZE = 20
#迭代次数
EPOCHS = 10
#采用cpu还是gpu进行计算
DEVICE = torch.device('cuda' if torch.cuda.is_available() else 'cpu')

图像处理与图像增强

# 数据预处理
 
transform = transforms.Compose([
    transforms.Resize(100),
    transforms.RandomVerticalFlip(),
    transforms.RandomCrop(50),
    transforms.RandomResizedCrop(150),
    transforms.ColorJitter(brightness=0.5, contrast=0.5, hue=0.5),
    transforms.ToTensor(),
    transforms.Normalize([0.5, 0.5, 0.5], [0.5, 0.5, 0.5])
])

读取数据集和导入数据

# 读取数据
 
dataset_train = datasets.ImageFolder('E:\\Cat_And_Dog\\kaggle\\cats_and_dogs_small\\train', transform)
 
print(dataset_train.imgs)
 
# 对应文件夹的label
 
print(dataset_train.class_to_idx)
 
dataset_test = datasets.ImageFolder('E:\\Cat_And_Dog\\kaggle\\cats_and_dogs_small\\validation', transform)
 
# 对应文件夹的label
 
print(dataset_test.class_to_idx)
 
# 导入数据
 
train_loader = torch.utils.data.DataLoader(dataset_train, batch_size=BATCH_SIZE, shuffle=True)
 
test_loader = torch.utils.data.DataLoader(dataset_test, batch_size=BATCH_SIZE, shuffle=True)

定义网络模型

# 定义网络
class ConvNet(nn.Module):
    def __init__(self):
        super(ConvNet, self).__init__()
        self.conv1 = nn.Conv2d(3, 32, 3)
        self.max_pool1 = nn.MaxPool2d(2)
        self.conv2 = nn.Conv2d(32, 64, 3) 
        self.max_pool2 = nn.MaxPool2d(2) 
        self.conv3 = nn.Conv2d(64, 64, 3) 
        self.conv4 = nn.Conv2d(64, 64, 3) 
        self.max_pool3 = nn.MaxPool2d(2) 
        self.conv5 = nn.Conv2d(64, 128, 3) 
        self.conv6 = nn.Conv2d(128, 128, 3) 
        self.max_pool4 = nn.MaxPool2d(2) 
        self.fc1 = nn.Linear(4608, 512) 
        self.fc2 = nn.Linear(512, 1)
  
    def forward(self, x): 
        in_size = x.size(0) 
        x = self.conv1(x) 
        x = F.relu(x) 
        x = self.max_pool1(x) 
        x = self.conv2(x) 
        x = F.relu(x) 
        x = self.max_pool2(x) 
        x = self.conv3(x) 
        x = F.relu(x) 
        x = self.conv4(x) 
        x = F.relu(x) 
        x = self.max_pool3(x) 
        x = self.conv5(x) 
        x = F.relu(x) 
        x = self.conv6(x) 
        x = F.relu(x)
        x = self.max_pool4(x) 
        # 展开
        x = x.view(in_size, -1)
        x = self.fc1(x)
        x = F.relu(x) 
        x = self.fc2(x) 
        x = torch.sigmoid(x) 
        return x
 
modellr = 1e-4
 
# 实例化模型并且移动到GPU
 
model = ConvNet().to(DEVICE)
 
# 选择简单暴力的Adam优化器,学习率调低
 
optimizer = optim.Adam(model.parameters(), lr=modellr)

调整学习率

def adjust_learning_rate(optimizer, epoch):
 
    """Sets the learning rate to the initial LR decayed by 10 every 30 epochs"""
    modellrnew = modellr * (0.1 ** (epoch // 5)) 
    print("lr:",modellrnew) 
    for param_group in optimizer.param_groups: 
        param_group['lr'] = modellrnew

定义训练过程

# 定义训练过程
def train(model, device, train_loader, optimizer, epoch):
 
    model.train() 
    for batch_idx, (data, target) in enumerate(train_loader):
 
        data, target = data.to(device), target.to(device).float().unsqueeze(1)
 
        optimizer.zero_grad()
 
        output = model(data)
 
        # print(output)
 
        loss = F.binary_cross_entropy(output, target)
 
        loss.backward()
 
        optimizer.step()
 
        if (batch_idx + 1) % 10 == 0:
 
            print('Train Epoch: {} [{}/{} ({:.0f}%)]\tLoss: {:.6f}'.format(
 
                epoch, (batch_idx + 1) * len(data), len(train_loader.dataset),
 
                    100. * (batch_idx + 1) / len(train_loader), loss.item()))
# 定义测试过程
 
def val(model, device, test_loader):
 
    model.eval()
 
    test_loss = 0
 
    correct = 0
 
    with torch.no_grad():
 
        for data, target in test_loader:
 
            data, target = data.to(device), target.to(device).float().unsqueeze(1)
 
            output = model(data)
            # print(output)
            test_loss += F.binary_cross_entropy(output, target, reduction='mean').item()
            pred = torch.tensor([[1] if num[0] >= 0.5 else [0] for num in output]).to(device)
            correct += pred.eq(target.long()).sum().item()
 
        print('\nTest set: Average loss: {:.4f}, Accuracy: {}/{} ({:.0f}%)\n'.format(
            test_loss, correct, len(test_loader.dataset),
            100. * correct / len(test_loader.dataset)))

定义保存模型和训练

# 训练
for epoch in range(1, EPOCHS + 1):
 
    adjust_learning_rate(optimizer, epoch)
    train(model, DEVICE, train_loader, optimizer, epoch) 
    val(model, DEVICE, test_loader)
 
torch.save(model, 'E:\\Cat_And_Dog\\kaggle\\model.pth')

训练结果

Python机器学习之基于Pytorch实现猫狗分类 

四、实现分类预测测试

准备预测的图片进行测试

from __future__ import print_function, division
from PIL import Image
 
from torchvision import transforms
import torch.nn.functional as F
 
import torch
import torch.nn as nn
import torch.nn.parallel
# 定义网络
class ConvNet(nn.Module):
    def __init__(self):
        super(ConvNet, self).__init__()
        self.conv1 = nn.Conv2d(3, 32, 3)
        self.max_pool1 = nn.MaxPool2d(2)
        self.conv2 = nn.Conv2d(32, 64, 3)
        self.max_pool2 = nn.MaxPool2d(2)
        self.conv3 = nn.Conv2d(64, 64, 3)
        self.conv4 = nn.Conv2d(64, 64, 3)
        self.max_pool3 = nn.MaxPool2d(2)
        self.conv5 = nn.Conv2d(64, 128, 3)
        self.conv6 = nn.Conv2d(128, 128, 3)
        self.max_pool4 = nn.MaxPool2d(2)
        self.fc1 = nn.Linear(4608, 512)
        self.fc2 = nn.Linear(512, 1)
 
    def forward(self, x):
        in_size = x.size(0)
        x = self.conv1(x)
        x = F.relu(x)
        x = self.max_pool1(x)
        x = self.conv2(x)
        x = F.relu(x)
        x = self.max_pool2(x)
        x = self.conv3(x)
        x = F.relu(x)
        x = self.conv4(x)
        x = F.relu(x)
        x = self.max_pool3(x)
        x = self.conv5(x)
        x = F.relu(x)
        x = self.conv6(x)
        x = F.relu(x)
        x = self.max_pool4(x)
        # 展开
        x = x.view(in_size, -1)
        x = self.fc1(x)
        x = F.relu(x)
        x = self.fc2(x)
        x = torch.sigmoid(x)
        return x
# 模型存储路径
model_save_path = 'E:\\Cat_And_Dog\\kaggle\\model.pth'
 
# ------------------------ 加载数据 --------------------------- #
# Data augmentation and normalization for training
# Just normalization for validation
# 定义预训练变换
# 数据预处理
transform_test = transforms.Compose([
    transforms.Resize(100),
    transforms.RandomVerticalFlip(),
    transforms.RandomCrop(50),
    transforms.RandomResizedCrop(150),
    transforms.ColorJitter(brightness=0.5, contrast=0.5, hue=0.5),
    transforms.ToTensor(),
    transforms.Normalize([0.5, 0.5, 0.5], [0.5, 0.5, 0.5])
])
 
 
class_names = ['cat', 'dog']  # 这个顺序很重要,要和训练时候的类名顺序一致
 
device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")
 
# ------------------------ 载入模型并且训练 --------------------------- #
model = torch.load(model_save_path)
model.eval()
# print(model)
 
image_PIL = Image.open('E:\\Cat_And_Dog\\kaggle\\cats_and_dogs_small\\test\\cats\\cat.1500.jpg')
#
image_tensor = transform_test(image_PIL)
# 以下语句等效于 image_tensor = torch.unsqueeze(image_tensor, 0)
image_tensor.unsqueeze_(0)
# 没有这句话会报错
image_tensor = image_tensor.to(device)
 
out = model(image_tensor)
pred = torch.tensor([[1] if num[0] >= 0.5 else [0] for num in out]).to(device)
print(class_names[pred])

预测结果

Python机器学习之基于Pytorch实现猫狗分类
Python机器学习之基于Pytorch实现猫狗分类

实际训练的过程来看,整体看准确度不高。而经过测试发现,该模型只能对于猫进行识别,对于狗则会误判。

到此这篇关于Python机器学习之基于Pytorch实现猫狗分类的文章就介绍到这了,更多相关Pytorch实现猫狗分类内容请搜索三水点靠木以前的文章或继续浏览下面的相关文章希望大家以后多多支持三水点靠木!

Python 相关文章推荐
Python简单连接MongoDB数据库的方法
Mar 15 Python
Python实现字典的遍历与排序功能示例
Dec 23 Python
python多进程实现文件下载传输功能
Jul 28 Python
python中的不可变数据类型与可变数据类型详解
Sep 16 Python
Python 使用Numpy对矩阵进行转置的方法
Jan 28 Python
python程序变成软件的实操方法
Jun 24 Python
python time.sleep()是睡眠线程还是进程
Jul 09 Python
我们为什么要减少Python中循环的使用
Jul 10 Python
python求最大公约数和最小公倍数的简单方法
Feb 13 Python
在Python中使用K-Means聚类和PCA主成分分析进行图像压缩
Apr 10 Python
python Django 反向访问器的外键冲突解决
May 20 Python
Python爬虫:从m3u8文件里提取小视频的正确操作
May 14 Python
Python中json.load()和json.loads()有哪些区别
python 爬取哔哩哔哩up主信息和投稿视频
Jun 07 #Python
OpenCV-Python直方图均衡化实现图像去雾
OpenCV-Python实现人脸磨皮算法
Python实现拼音转换
Python实现简繁体转换
在Python中如何使用yield
Jun 07 #Python
You might like
用PHP函数解决SQL injection
2006/12/09 PHP
PHP中的session永不过期的解决思路及实现方法分享
2011/04/20 PHP
去除php注释和去除空格函数分享
2014/03/13 PHP
php解决约瑟夫环示例
2014/04/09 PHP
PHP仿微信发红包领红包效果
2016/10/30 PHP
thinkphp5 框架结合plupload实现图片批量上传功能示例
2020/04/04 PHP
基于jquery1.4.2的仿flash超炫焦点图播放效果
2010/04/20 Javascript
JavaScript可否多线程? 深入理解JavaScript定时机制
2012/05/23 Javascript
终于解决了IE8不支持数组的indexOf方法
2013/04/03 Javascript
localResizeIMG先压缩后使用ajax无刷新上传(移动端)
2015/08/11 Javascript
表单中单选框添加选项和移除选项
2016/07/04 Javascript
14 个折磨人的 JavaScript 面试题
2016/08/08 Javascript
jquery html5 视频播放控制代码
2016/11/06 Javascript
javascript基于原型链的继承及call和apply函数用法分析
2016/12/15 Javascript
微信小程序 下拉菜单的实现
2017/04/06 Javascript
微信小程序 动态绑定事件并实现事件修改样式
2017/04/13 Javascript
通过示例彻底搞懂js闭包
2017/08/10 Javascript
[02:16]DOTA2英雄基础教程 干扰者
2014/01/15 DOTA
[51:28]EG vs Mineski 2018国际邀请赛小组赛BO2 第一场 8.16
2018/08/16 DOTA
利用python求相邻数的方法示例
2017/08/18 Python
python实现自动发送邮件发送多人、群发、多附件的示例
2018/01/23 Python
Python socket实现多对多全双工通信的方法
2019/02/13 Python
python在新的图片窗口显示图片(图像)的方法
2019/07/11 Python
基于Python获取照片的GPS位置信息
2020/01/20 Python
基于html5绘制圆形多角图案
2016/04/21 HTML / CSS
Otticanet美国:最顶尖的世界名牌眼镜, 能得到打折季的价格
2019/03/10 全球购物
初中高效课堂实施方案
2014/02/26 职场文书
《红军不怕远征难》教学反思
2014/04/14 职场文书
学校欢迎标语
2014/06/18 职场文书
企业读书活动总结
2014/06/30 职场文书
灵山大佛导游词
2015/02/04 职场文书
论文致谢词范文
2015/05/14 职场文书
考试后的感想
2015/08/07 职场文书
2016优秀大学生个人事迹材料范文
2016/03/01 职场文书
Mysql 如何批量插入数据
2021/04/06 MySQL
win10如何更改appdata文件夹的默认位置?
2022/07/15 数码科技