Python机器学习之基于Pytorch实现猫狗分类


Posted in Python onJune 08, 2021

一、环境配置

安装Anaconda

配置Pytorch

pip install -i https://pypi.tuna.tsinghua.edu.cn/simple torch
pip install -i https://pypi.tuna.tsinghua.edu.cn/simple torchvision

二、数据集的准备

1.数据集的下载

kaggle网站的数据集下载地址:
https://www.kaggle.com/lizhensheng/-2000

2.数据集的分类

将下载的数据集进行解压操作,然后进行分类
分类如下(每个文件夹下包括cats和dogs文件夹)

Python机器学习之基于Pytorch实现猫狗分类 

三、猫狗分类的实例

导入相应的库

# 导入库
import torch.nn.functional as F
import torch.optim as optim
import torch
import torch.nn as nn
import torch.nn.parallel
 
import torch.optim
import torch.utils.data
import torch.utils.data.distributed
import torchvision.transforms as transforms
import torchvision.datasets as datasets

设置超参数

# 设置超参数
#每次的个数
BATCH_SIZE = 20
#迭代次数
EPOCHS = 10
#采用cpu还是gpu进行计算
DEVICE = torch.device('cuda' if torch.cuda.is_available() else 'cpu')

图像处理与图像增强

# 数据预处理
 
transform = transforms.Compose([
    transforms.Resize(100),
    transforms.RandomVerticalFlip(),
    transforms.RandomCrop(50),
    transforms.RandomResizedCrop(150),
    transforms.ColorJitter(brightness=0.5, contrast=0.5, hue=0.5),
    transforms.ToTensor(),
    transforms.Normalize([0.5, 0.5, 0.5], [0.5, 0.5, 0.5])
])

读取数据集和导入数据

# 读取数据
 
dataset_train = datasets.ImageFolder('E:\\Cat_And_Dog\\kaggle\\cats_and_dogs_small\\train', transform)
 
print(dataset_train.imgs)
 
# 对应文件夹的label
 
print(dataset_train.class_to_idx)
 
dataset_test = datasets.ImageFolder('E:\\Cat_And_Dog\\kaggle\\cats_and_dogs_small\\validation', transform)
 
# 对应文件夹的label
 
print(dataset_test.class_to_idx)
 
# 导入数据
 
train_loader = torch.utils.data.DataLoader(dataset_train, batch_size=BATCH_SIZE, shuffle=True)
 
test_loader = torch.utils.data.DataLoader(dataset_test, batch_size=BATCH_SIZE, shuffle=True)

定义网络模型

# 定义网络
class ConvNet(nn.Module):
    def __init__(self):
        super(ConvNet, self).__init__()
        self.conv1 = nn.Conv2d(3, 32, 3)
        self.max_pool1 = nn.MaxPool2d(2)
        self.conv2 = nn.Conv2d(32, 64, 3) 
        self.max_pool2 = nn.MaxPool2d(2) 
        self.conv3 = nn.Conv2d(64, 64, 3) 
        self.conv4 = nn.Conv2d(64, 64, 3) 
        self.max_pool3 = nn.MaxPool2d(2) 
        self.conv5 = nn.Conv2d(64, 128, 3) 
        self.conv6 = nn.Conv2d(128, 128, 3) 
        self.max_pool4 = nn.MaxPool2d(2) 
        self.fc1 = nn.Linear(4608, 512) 
        self.fc2 = nn.Linear(512, 1)
  
    def forward(self, x): 
        in_size = x.size(0) 
        x = self.conv1(x) 
        x = F.relu(x) 
        x = self.max_pool1(x) 
        x = self.conv2(x) 
        x = F.relu(x) 
        x = self.max_pool2(x) 
        x = self.conv3(x) 
        x = F.relu(x) 
        x = self.conv4(x) 
        x = F.relu(x) 
        x = self.max_pool3(x) 
        x = self.conv5(x) 
        x = F.relu(x) 
        x = self.conv6(x) 
        x = F.relu(x)
        x = self.max_pool4(x) 
        # 展开
        x = x.view(in_size, -1)
        x = self.fc1(x)
        x = F.relu(x) 
        x = self.fc2(x) 
        x = torch.sigmoid(x) 
        return x
 
modellr = 1e-4
 
# 实例化模型并且移动到GPU
 
model = ConvNet().to(DEVICE)
 
# 选择简单暴力的Adam优化器,学习率调低
 
optimizer = optim.Adam(model.parameters(), lr=modellr)

调整学习率

def adjust_learning_rate(optimizer, epoch):
 
    """Sets the learning rate to the initial LR decayed by 10 every 30 epochs"""
    modellrnew = modellr * (0.1 ** (epoch // 5)) 
    print("lr:",modellrnew) 
    for param_group in optimizer.param_groups: 
        param_group['lr'] = modellrnew

定义训练过程

# 定义训练过程
def train(model, device, train_loader, optimizer, epoch):
 
    model.train() 
    for batch_idx, (data, target) in enumerate(train_loader):
 
        data, target = data.to(device), target.to(device).float().unsqueeze(1)
 
        optimizer.zero_grad()
 
        output = model(data)
 
        # print(output)
 
        loss = F.binary_cross_entropy(output, target)
 
        loss.backward()
 
        optimizer.step()
 
        if (batch_idx + 1) % 10 == 0:
 
            print('Train Epoch: {} [{}/{} ({:.0f}%)]\tLoss: {:.6f}'.format(
 
                epoch, (batch_idx + 1) * len(data), len(train_loader.dataset),
 
                    100. * (batch_idx + 1) / len(train_loader), loss.item()))
# 定义测试过程
 
def val(model, device, test_loader):
 
    model.eval()
 
    test_loss = 0
 
    correct = 0
 
    with torch.no_grad():
 
        for data, target in test_loader:
 
            data, target = data.to(device), target.to(device).float().unsqueeze(1)
 
            output = model(data)
            # print(output)
            test_loss += F.binary_cross_entropy(output, target, reduction='mean').item()
            pred = torch.tensor([[1] if num[0] >= 0.5 else [0] for num in output]).to(device)
            correct += pred.eq(target.long()).sum().item()
 
        print('\nTest set: Average loss: {:.4f}, Accuracy: {}/{} ({:.0f}%)\n'.format(
            test_loss, correct, len(test_loader.dataset),
            100. * correct / len(test_loader.dataset)))

定义保存模型和训练

# 训练
for epoch in range(1, EPOCHS + 1):
 
    adjust_learning_rate(optimizer, epoch)
    train(model, DEVICE, train_loader, optimizer, epoch) 
    val(model, DEVICE, test_loader)
 
torch.save(model, 'E:\\Cat_And_Dog\\kaggle\\model.pth')

训练结果

Python机器学习之基于Pytorch实现猫狗分类 

四、实现分类预测测试

准备预测的图片进行测试

from __future__ import print_function, division
from PIL import Image
 
from torchvision import transforms
import torch.nn.functional as F
 
import torch
import torch.nn as nn
import torch.nn.parallel
# 定义网络
class ConvNet(nn.Module):
    def __init__(self):
        super(ConvNet, self).__init__()
        self.conv1 = nn.Conv2d(3, 32, 3)
        self.max_pool1 = nn.MaxPool2d(2)
        self.conv2 = nn.Conv2d(32, 64, 3)
        self.max_pool2 = nn.MaxPool2d(2)
        self.conv3 = nn.Conv2d(64, 64, 3)
        self.conv4 = nn.Conv2d(64, 64, 3)
        self.max_pool3 = nn.MaxPool2d(2)
        self.conv5 = nn.Conv2d(64, 128, 3)
        self.conv6 = nn.Conv2d(128, 128, 3)
        self.max_pool4 = nn.MaxPool2d(2)
        self.fc1 = nn.Linear(4608, 512)
        self.fc2 = nn.Linear(512, 1)
 
    def forward(self, x):
        in_size = x.size(0)
        x = self.conv1(x)
        x = F.relu(x)
        x = self.max_pool1(x)
        x = self.conv2(x)
        x = F.relu(x)
        x = self.max_pool2(x)
        x = self.conv3(x)
        x = F.relu(x)
        x = self.conv4(x)
        x = F.relu(x)
        x = self.max_pool3(x)
        x = self.conv5(x)
        x = F.relu(x)
        x = self.conv6(x)
        x = F.relu(x)
        x = self.max_pool4(x)
        # 展开
        x = x.view(in_size, -1)
        x = self.fc1(x)
        x = F.relu(x)
        x = self.fc2(x)
        x = torch.sigmoid(x)
        return x
# 模型存储路径
model_save_path = 'E:\\Cat_And_Dog\\kaggle\\model.pth'
 
# ------------------------ 加载数据 --------------------------- #
# Data augmentation and normalization for training
# Just normalization for validation
# 定义预训练变换
# 数据预处理
transform_test = transforms.Compose([
    transforms.Resize(100),
    transforms.RandomVerticalFlip(),
    transforms.RandomCrop(50),
    transforms.RandomResizedCrop(150),
    transforms.ColorJitter(brightness=0.5, contrast=0.5, hue=0.5),
    transforms.ToTensor(),
    transforms.Normalize([0.5, 0.5, 0.5], [0.5, 0.5, 0.5])
])
 
 
class_names = ['cat', 'dog']  # 这个顺序很重要,要和训练时候的类名顺序一致
 
device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")
 
# ------------------------ 载入模型并且训练 --------------------------- #
model = torch.load(model_save_path)
model.eval()
# print(model)
 
image_PIL = Image.open('E:\\Cat_And_Dog\\kaggle\\cats_and_dogs_small\\test\\cats\\cat.1500.jpg')
#
image_tensor = transform_test(image_PIL)
# 以下语句等效于 image_tensor = torch.unsqueeze(image_tensor, 0)
image_tensor.unsqueeze_(0)
# 没有这句话会报错
image_tensor = image_tensor.to(device)
 
out = model(image_tensor)
pred = torch.tensor([[1] if num[0] >= 0.5 else [0] for num in out]).to(device)
print(class_names[pred])

预测结果

Python机器学习之基于Pytorch实现猫狗分类
Python机器学习之基于Pytorch实现猫狗分类

实际训练的过程来看,整体看准确度不高。而经过测试发现,该模型只能对于猫进行识别,对于狗则会误判。

到此这篇关于Python机器学习之基于Pytorch实现猫狗分类的文章就介绍到这了,更多相关Pytorch实现猫狗分类内容请搜索三水点靠木以前的文章或继续浏览下面的相关文章希望大家以后多多支持三水点靠木!

Python 相关文章推荐
Python字符串匹配算法KMP实例
Jul 18 Python
简单介绍使用Python解析并修改XML文档的方法
Oct 15 Python
使用Mixin设计模式进行Python编程的方法讲解
Jun 21 Python
python用Pygal如何生成漂亮的SVG图像详解
Feb 10 Python
Python字符串格式化%s%d%f详解
Feb 02 Python
Anaconda下配置python+opencv+contribx的实例讲解
Aug 06 Python
详解python读取和输出到txt
Mar 29 Python
python环境路径配置以及命令行运行脚本
Apr 02 Python
python实现支付宝转账接口
May 07 Python
Python学习笔记之错误和异常及访问错误消息详解
Aug 08 Python
python图形开发GUI库pyqt5的详细使用方法及各控件的属性与方法
Feb 14 Python
Python词云的正确实现方法实例
May 08 Python
Python中json.load()和json.loads()有哪些区别
python 爬取哔哩哔哩up主信息和投稿视频
Jun 07 #Python
OpenCV-Python直方图均衡化实现图像去雾
OpenCV-Python实现人脸磨皮算法
Python实现拼音转换
Python实现简繁体转换
在Python中如何使用yield
Jun 07 #Python
You might like
萌王史莱姆”萌王性别尴尬!那“萌战”归女组还是男?
2018/12/17 日漫
SONY ICF-SW07收音机电路分析
2021/03/02 无线电
PHP_Cooikes不同页面无法传递的解决方法
2014/03/07 PHP
PHP中通过fopen()函数访问远程文件示例
2014/11/18 PHP
如何用PHP来实现一个动态Web服务器
2015/07/29 PHP
Yii遍历行下每列数据的方法
2016/10/17 PHP
php in_array() 检查数组中是否存在某个值详解
2016/11/23 PHP
PHP用PDO如何封装简单易用的DB类详解
2017/07/30 PHP
PHP设计模式入门之迭代器模式原理与实现方法分析
2020/04/26 PHP
图像替换新技术 状态域方法
2010/01/28 Javascript
Tips 带三角可关闭的文字提示
2010/10/06 Javascript
对jQuery的事件绑定的一些思考(补充)
2013/04/20 Javascript
json中换行符的处理方法示例介绍
2014/06/10 Javascript
JS实现的表格操作类详解(添加,删除,排序,上移,下移)
2015/12/22 Javascript
ionic js 复选框 与普通的 HTML 复选框到底有没区别
2016/06/06 Javascript
AngularJS 模块化详解及实例代码
2016/09/14 Javascript
JS检测window.open打开的窗口是否关闭
2017/06/25 Javascript
Vue2.0 多 Tab切换组件的封装实例
2017/07/28 Javascript
JavaScript创建对象的四种常用模式实例分析
2019/01/11 Javascript
微信小程序绑定手机号获取验证码功能
2019/10/22 Javascript
node使用request请求的方法
2019/12/20 Javascript
基于PHP pthreads实现多线程代码实例
2020/06/24 Javascript
python操作xml文件示例
2014/04/07 Python
在Python中调用ggplot的三种方法
2015/04/08 Python
numpy排序与集合运算用法示例
2017/12/15 Python
python opencv之分水岭算法示例
2018/02/24 Python
深入浅析python with语句简介
2018/04/11 Python
python卸载后再次安装遇到的问题解决
2019/07/10 Python
Python中最好用的命令行参数解析工具(argparse)
2019/08/23 Python
numpy.random.shuffle打乱顺序函数的实现
2019/09/10 Python
python__new__内置静态方法使用解析
2020/01/07 Python
python 爬取疫情数据的源码
2020/02/09 Python
python实现无边框进度条的实例代码
2020/12/30 Python
阿里巴巴国际站:Alibaba.com
2016/07/21 全球购物
教师工作失职检讨书
2014/09/18 职场文书
校运会班级霸气口号
2015/12/24 职场文书