Python机器学习之基于Pytorch实现猫狗分类


Posted in Python onJune 08, 2021

一、环境配置

安装Anaconda

配置Pytorch

pip install -i https://pypi.tuna.tsinghua.edu.cn/simple torch
pip install -i https://pypi.tuna.tsinghua.edu.cn/simple torchvision

二、数据集的准备

1.数据集的下载

kaggle网站的数据集下载地址:
https://www.kaggle.com/lizhensheng/-2000

2.数据集的分类

将下载的数据集进行解压操作,然后进行分类
分类如下(每个文件夹下包括cats和dogs文件夹)

Python机器学习之基于Pytorch实现猫狗分类 

三、猫狗分类的实例

导入相应的库

# 导入库
import torch.nn.functional as F
import torch.optim as optim
import torch
import torch.nn as nn
import torch.nn.parallel
 
import torch.optim
import torch.utils.data
import torch.utils.data.distributed
import torchvision.transforms as transforms
import torchvision.datasets as datasets

设置超参数

# 设置超参数
#每次的个数
BATCH_SIZE = 20
#迭代次数
EPOCHS = 10
#采用cpu还是gpu进行计算
DEVICE = torch.device('cuda' if torch.cuda.is_available() else 'cpu')

图像处理与图像增强

# 数据预处理
 
transform = transforms.Compose([
    transforms.Resize(100),
    transforms.RandomVerticalFlip(),
    transforms.RandomCrop(50),
    transforms.RandomResizedCrop(150),
    transforms.ColorJitter(brightness=0.5, contrast=0.5, hue=0.5),
    transforms.ToTensor(),
    transforms.Normalize([0.5, 0.5, 0.5], [0.5, 0.5, 0.5])
])

读取数据集和导入数据

# 读取数据
 
dataset_train = datasets.ImageFolder('E:\\Cat_And_Dog\\kaggle\\cats_and_dogs_small\\train', transform)
 
print(dataset_train.imgs)
 
# 对应文件夹的label
 
print(dataset_train.class_to_idx)
 
dataset_test = datasets.ImageFolder('E:\\Cat_And_Dog\\kaggle\\cats_and_dogs_small\\validation', transform)
 
# 对应文件夹的label
 
print(dataset_test.class_to_idx)
 
# 导入数据
 
train_loader = torch.utils.data.DataLoader(dataset_train, batch_size=BATCH_SIZE, shuffle=True)
 
test_loader = torch.utils.data.DataLoader(dataset_test, batch_size=BATCH_SIZE, shuffle=True)

定义网络模型

# 定义网络
class ConvNet(nn.Module):
    def __init__(self):
        super(ConvNet, self).__init__()
        self.conv1 = nn.Conv2d(3, 32, 3)
        self.max_pool1 = nn.MaxPool2d(2)
        self.conv2 = nn.Conv2d(32, 64, 3) 
        self.max_pool2 = nn.MaxPool2d(2) 
        self.conv3 = nn.Conv2d(64, 64, 3) 
        self.conv4 = nn.Conv2d(64, 64, 3) 
        self.max_pool3 = nn.MaxPool2d(2) 
        self.conv5 = nn.Conv2d(64, 128, 3) 
        self.conv6 = nn.Conv2d(128, 128, 3) 
        self.max_pool4 = nn.MaxPool2d(2) 
        self.fc1 = nn.Linear(4608, 512) 
        self.fc2 = nn.Linear(512, 1)
  
    def forward(self, x): 
        in_size = x.size(0) 
        x = self.conv1(x) 
        x = F.relu(x) 
        x = self.max_pool1(x) 
        x = self.conv2(x) 
        x = F.relu(x) 
        x = self.max_pool2(x) 
        x = self.conv3(x) 
        x = F.relu(x) 
        x = self.conv4(x) 
        x = F.relu(x) 
        x = self.max_pool3(x) 
        x = self.conv5(x) 
        x = F.relu(x) 
        x = self.conv6(x) 
        x = F.relu(x)
        x = self.max_pool4(x) 
        # 展开
        x = x.view(in_size, -1)
        x = self.fc1(x)
        x = F.relu(x) 
        x = self.fc2(x) 
        x = torch.sigmoid(x) 
        return x
 
modellr = 1e-4
 
# 实例化模型并且移动到GPU
 
model = ConvNet().to(DEVICE)
 
# 选择简单暴力的Adam优化器,学习率调低
 
optimizer = optim.Adam(model.parameters(), lr=modellr)

调整学习率

def adjust_learning_rate(optimizer, epoch):
 
    """Sets the learning rate to the initial LR decayed by 10 every 30 epochs"""
    modellrnew = modellr * (0.1 ** (epoch // 5)) 
    print("lr:",modellrnew) 
    for param_group in optimizer.param_groups: 
        param_group['lr'] = modellrnew

定义训练过程

# 定义训练过程
def train(model, device, train_loader, optimizer, epoch):
 
    model.train() 
    for batch_idx, (data, target) in enumerate(train_loader):
 
        data, target = data.to(device), target.to(device).float().unsqueeze(1)
 
        optimizer.zero_grad()
 
        output = model(data)
 
        # print(output)
 
        loss = F.binary_cross_entropy(output, target)
 
        loss.backward()
 
        optimizer.step()
 
        if (batch_idx + 1) % 10 == 0:
 
            print('Train Epoch: {} [{}/{} ({:.0f}%)]\tLoss: {:.6f}'.format(
 
                epoch, (batch_idx + 1) * len(data), len(train_loader.dataset),
 
                    100. * (batch_idx + 1) / len(train_loader), loss.item()))
# 定义测试过程
 
def val(model, device, test_loader):
 
    model.eval()
 
    test_loss = 0
 
    correct = 0
 
    with torch.no_grad():
 
        for data, target in test_loader:
 
            data, target = data.to(device), target.to(device).float().unsqueeze(1)
 
            output = model(data)
            # print(output)
            test_loss += F.binary_cross_entropy(output, target, reduction='mean').item()
            pred = torch.tensor([[1] if num[0] >= 0.5 else [0] for num in output]).to(device)
            correct += pred.eq(target.long()).sum().item()
 
        print('\nTest set: Average loss: {:.4f}, Accuracy: {}/{} ({:.0f}%)\n'.format(
            test_loss, correct, len(test_loader.dataset),
            100. * correct / len(test_loader.dataset)))

定义保存模型和训练

# 训练
for epoch in range(1, EPOCHS + 1):
 
    adjust_learning_rate(optimizer, epoch)
    train(model, DEVICE, train_loader, optimizer, epoch) 
    val(model, DEVICE, test_loader)
 
torch.save(model, 'E:\\Cat_And_Dog\\kaggle\\model.pth')

训练结果

Python机器学习之基于Pytorch实现猫狗分类 

四、实现分类预测测试

准备预测的图片进行测试

from __future__ import print_function, division
from PIL import Image
 
from torchvision import transforms
import torch.nn.functional as F
 
import torch
import torch.nn as nn
import torch.nn.parallel
# 定义网络
class ConvNet(nn.Module):
    def __init__(self):
        super(ConvNet, self).__init__()
        self.conv1 = nn.Conv2d(3, 32, 3)
        self.max_pool1 = nn.MaxPool2d(2)
        self.conv2 = nn.Conv2d(32, 64, 3)
        self.max_pool2 = nn.MaxPool2d(2)
        self.conv3 = nn.Conv2d(64, 64, 3)
        self.conv4 = nn.Conv2d(64, 64, 3)
        self.max_pool3 = nn.MaxPool2d(2)
        self.conv5 = nn.Conv2d(64, 128, 3)
        self.conv6 = nn.Conv2d(128, 128, 3)
        self.max_pool4 = nn.MaxPool2d(2)
        self.fc1 = nn.Linear(4608, 512)
        self.fc2 = nn.Linear(512, 1)
 
    def forward(self, x):
        in_size = x.size(0)
        x = self.conv1(x)
        x = F.relu(x)
        x = self.max_pool1(x)
        x = self.conv2(x)
        x = F.relu(x)
        x = self.max_pool2(x)
        x = self.conv3(x)
        x = F.relu(x)
        x = self.conv4(x)
        x = F.relu(x)
        x = self.max_pool3(x)
        x = self.conv5(x)
        x = F.relu(x)
        x = self.conv6(x)
        x = F.relu(x)
        x = self.max_pool4(x)
        # 展开
        x = x.view(in_size, -1)
        x = self.fc1(x)
        x = F.relu(x)
        x = self.fc2(x)
        x = torch.sigmoid(x)
        return x
# 模型存储路径
model_save_path = 'E:\\Cat_And_Dog\\kaggle\\model.pth'
 
# ------------------------ 加载数据 --------------------------- #
# Data augmentation and normalization for training
# Just normalization for validation
# 定义预训练变换
# 数据预处理
transform_test = transforms.Compose([
    transforms.Resize(100),
    transforms.RandomVerticalFlip(),
    transforms.RandomCrop(50),
    transforms.RandomResizedCrop(150),
    transforms.ColorJitter(brightness=0.5, contrast=0.5, hue=0.5),
    transforms.ToTensor(),
    transforms.Normalize([0.5, 0.5, 0.5], [0.5, 0.5, 0.5])
])
 
 
class_names = ['cat', 'dog']  # 这个顺序很重要,要和训练时候的类名顺序一致
 
device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")
 
# ------------------------ 载入模型并且训练 --------------------------- #
model = torch.load(model_save_path)
model.eval()
# print(model)
 
image_PIL = Image.open('E:\\Cat_And_Dog\\kaggle\\cats_and_dogs_small\\test\\cats\\cat.1500.jpg')
#
image_tensor = transform_test(image_PIL)
# 以下语句等效于 image_tensor = torch.unsqueeze(image_tensor, 0)
image_tensor.unsqueeze_(0)
# 没有这句话会报错
image_tensor = image_tensor.to(device)
 
out = model(image_tensor)
pred = torch.tensor([[1] if num[0] >= 0.5 else [0] for num in out]).to(device)
print(class_names[pred])

预测结果

Python机器学习之基于Pytorch实现猫狗分类
Python机器学习之基于Pytorch实现猫狗分类

实际训练的过程来看,整体看准确度不高。而经过测试发现,该模型只能对于猫进行识别,对于狗则会误判。

到此这篇关于Python机器学习之基于Pytorch实现猫狗分类的文章就介绍到这了,更多相关Pytorch实现猫狗分类内容请搜索三水点靠木以前的文章或继续浏览下面的相关文章希望大家以后多多支持三水点靠木!

Python 相关文章推荐
python使用rsa加密算法模块模拟新浪微博登录
Jan 22 Python
python实现k均值算法示例(k均值聚类算法)
Mar 16 Python
Python复数属性和方法运算操作示例
Jul 21 Python
让Python更加充分的使用Sqlite3
Dec 11 Python
Python排序算法之选择排序定义与用法示例
Apr 29 Python
在python中将字符串转为json对象并取值的方法
Dec 31 Python
python 处理telnet返回的More,以及get想要的那个参数方法
Feb 14 Python
Python后台开发Django的教程详解(启动)
Apr 08 Python
Python爬虫使用代理IP的实现
Oct 27 Python
python对验证码降噪的实现示例代码
Nov 12 Python
浅谈在django中使用redirect重定向数据传输的问题
Mar 13 Python
Python实现从N个数中找到最大的K个数
Apr 02 Python
Python中json.load()和json.loads()有哪些区别
python 爬取哔哩哔哩up主信息和投稿视频
Jun 07 #Python
OpenCV-Python直方图均衡化实现图像去雾
OpenCV-Python实现人脸磨皮算法
Python实现拼音转换
Python实现简繁体转换
在Python中如何使用yield
Jun 07 #Python
You might like
将博客园(cnblogs.com)数据导入到wordpress的代码
2013/01/06 PHP
php从数组中随机选择若干不重复元素的方法
2015/03/14 PHP
php写入数据到CSV文件的方法
2015/03/14 PHP
thinkPHP中session()方法用法详解
2016/12/08 PHP
解决Laravel 使用insert插入数据,字段created_at为0000的问题
2019/10/11 PHP
?牟┛途W扣了一??效果出?? target=
2007/05/27 Javascript
jQuery 源码分析笔记(4) Ready函数
2011/06/02 Javascript
详解Vue-cli 创建的项目如何跨域请求
2017/05/18 Javascript
JS实现简单拖拽效果
2017/06/21 Javascript
React Native 使用Fetch发送网络请求的示例代码
2017/12/02 Javascript
vue 项目打包通过命令修改 vue-router 模式 修改 API 接口前缀
2018/06/13 Javascript
Vue中"This dependency was not found"问题的解决方法
2018/06/19 Javascript
vue.js通过路由实现经典的三栏布局实例代码
2018/07/08 Javascript
VUE-Table上绑定Input通过render实现双向绑定数据的示例
2018/08/27 Javascript
Node.js API详解之 dgram模块用法实例分析
2020/06/05 Javascript
Vue.js桌面端自定义滚动条组件之美化滚动条VScroll
2020/12/01 Vue.js
Python实现线程池代码分享
2015/06/21 Python
全面了解python字符串和字典
2016/07/07 Python
python中numpy的矩阵、多维数组的用法
2018/02/05 Python
解决pycharm运行时interpreter为空的问题
2018/10/29 Python
opencv3/C++实现视频背景去除建模(BSM)
2019/12/11 Python
python如何删除文件、目录
2020/06/23 Python
Python使用pyenv实现多环境管理
2021/02/05 Python
澳大利亚巧克力花束和礼品网站:Tastebuds
2019/03/15 全球购物
拉飞逸官网:Lafayette 148 New York
2020/07/15 全球购物
什么是SQL Server的确定性函数和不确定性函数
2016/08/04 面试题
linux比较文件内容的命令是什么
2015/09/23 面试题
一名毕业生的自我鉴定
2013/12/04 职场文书
教师网络培训感言
2014/03/09 职场文书
党的群众路线教育实践活动批评与自我批评范文
2014/10/16 职场文书
违反交通安全法检讨书
2014/10/24 职场文书
2014年小学教导处工作总结
2014/12/19 职场文书
大学生学年个人总结
2015/02/15 职场文书
讲座开场白台词和结束语
2015/05/29 职场文书
关爱留守儿童主题班会
2015/08/13 职场文书
Win11安装受阻怎么办? Windows11安装问题与解决方案汇总
2021/11/21 数码科技