TensorFlow2.X结合OpenCV 实现手势识别功能


Posted in Python onApril 08, 2020

使用Tensorflow 构建卷积神经网络,训练手势识别模型,使用opencv DNN 模块加载模型实时手势识别
效果如下:

TensorFlow2.X结合OpenCV 实现手势识别功能

先显示下部分数据集图片(0到9的表示,感觉很怪)

TensorFlow2.X结合OpenCV 实现手势识别功能

构建模型进行训练

import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import datasets,layers,optimizers,Sequential,metrics
from tensorflow.python.framework.convert_to_constants import convert_variables_to_constants_v2
import os 
import pathlib
import random
import matplotlib.pyplot as plt
os.environ['TF_CPP_MIN_LOG_LEVEL']='2'
def read_data(path):
 path_root = pathlib.Path(path)
 # print(path_root)
 # for item in path_root.iterdir():
 #  print(item)
 image_paths = list(path_root.glob('*/*'))
 image_paths = [str(path) for path in image_paths]
 random.shuffle(image_paths)
 image_count = len(image_paths)
 # print(image_count)
 # print(image_paths[:10])
 label_names = sorted(item.name for item in path_root.glob('*/') if item.is_dir())
 # print(label_names)
 label_name_index = dict((name, index) for index, name in enumerate(label_names))
 # print(label_name_index)
 image_labels = [label_name_index[pathlib.Path(path).parent.name] for path in image_paths]
 # print("First 10 labels indices: ", image_labels[:10])
 return image_paths,image_labels,image_count
def preprocess_image(image):
 image = tf.image.decode_jpeg(image, channels=3)
 image = tf.image.resize(image, [100, 100])
 image /= 255.0 # normalize to [0,1] range
 # image = tf.reshape(image,[100*100*3])
 return image
def load_and_preprocess_image(path,label):
 image = tf.io.read_file(path)
 return preprocess_image(image),label
def creat_dataset(image_paths,image_labels,bitch_size):
 db = tf.data.Dataset.from_tensor_slices((image_paths, image_labels))
 dataset = db.map(load_and_preprocess_image).batch(bitch_size) 
 return dataset
def train_model(train_data,test_data):
 #构建模型
 network = keras.Sequential([
   keras.layers.Conv2D(32,kernel_size=[5,5],padding="same",activation=tf.nn.relu),
   keras.layers.MaxPool2D(pool_size=[2, 2], strides=2, padding='same'),
   keras.layers.Conv2D(64,kernel_size=[3,3],padding="same",activation=tf.nn.relu),
   keras.layers.MaxPool2D(pool_size=[2, 2], strides=2, padding='same'),
   keras.layers.Conv2D(64,kernel_size=[3,3],padding="same",activation=tf.nn.relu),
   keras.layers.Flatten(),
   keras.layers.Dense(512,activation='relu'),
   keras.layers.Dropout(0.5),
   keras.layers.Dense(128,activation='relu'),
   keras.layers.Dense(10)])
 network.build(input_shape=(None,100,100,3))
 network.summary()
 network.compile(optimizer=optimizers.SGD(lr=0.001),
   loss=tf.losses.SparseCategoricalCrossentropy(from_logits=True),
   metrics=['accuracy']
 )
 #模型训练
 network.fit(train_data, epochs = 100,validation_data=test_data,validation_freq=2) 
 network.evaluate(test_data)
 tf.saved_model.save(network,'D:\\code\\PYTHON\\gesture_recognition\\model\\')
 print("保存模型成功")
 # Convert Keras model to ConcreteFunction
 full_model = tf.function(lambda x: network(x))
 full_model = full_model.get_concrete_function(
 tf.TensorSpec(network.inputs[0].shape, network.inputs[0].dtype))
 # Get frozen ConcreteFunction
 frozen_func = convert_variables_to_constants_v2(full_model)
 frozen_func.graph.as_graph_def()

 layers = [op.name for op in frozen_func.graph.get_operations()]
 print("-" * 50)
 print("Frozen model layers: ")
 for layer in layers:
  print(layer)

 print("-" * 50)
 print("Frozen model inputs: ")
 print(frozen_func.inputs)
 print("Frozen model outputs: ")
 print(frozen_func.outputs)

 # Save frozen graph from frozen ConcreteFunction to hard drive
 tf.io.write_graph(graph_or_graph_def=frozen_func.graph,
   logdir="D:\\code\\PYTHON\\gesture_recognition\\model\\frozen_model\\",
   name="frozen_graph.pb",
   as_text=False)
 print("模型转换完成,训练结束")


if __name__ == "__main__":
 print(tf.__version__)
 train_path = 'D:\\code\\PYTHON\\gesture_recognition\\Dataset'
 test_path = 'D:\\code\\PYTHON\\gesture_recognition\\testdata' 
 image_paths,image_labels,_ = read_data(train_path)
 train_data = creat_dataset(image_paths,image_labels,16)
 image_paths,image_labels,_ = read_data(test_path)
 test_data = creat_dataset(image_paths,image_labels,16)
 train_model(train_data,test_data)

OpenCV加载模型,实时检测

这里为了简化检测使用了ROI。

import cv2
from cv2 import dnn
import numpy as np
print(cv2.__version__)
class_name = ['0', '1', '2', '3', '4', '5', '6', '7', '8', '9']
net = dnn.readNetFromTensorflow('D:\\code\\PYTHON\\gesture_recognition\\model\\frozen_model\\frozen_graph.pb')
cap = cv2.VideoCapture(0)
i = 0
while True:
 _,frame= cap.read() 
 src_image = frame
 cv2.rectangle(src_image, (300, 100),(600, 400), (0, 255, 0), 1, 4)
 frame = cv2.cvtColor(frame,cv2.COLOR_BGR2RGB)
 pic = frame[100:400,300:600]
 cv2.imshow("pic1", pic)
 # print(pic.shape)
 pic = cv2.resize(pic,(100,100))
 blob = cv2.dnn.blobFromImage(pic,  
        scalefactor=1.0/225.,
        size=(100, 100),
        mean=(0, 0, 0),
        swapRB=False,
        crop=False)
 # blob = np.transpose(blob, (0,2,3,1))       
 net.setInput(blob)
 out = net.forward()
 out = out.flatten()

 classId = np.argmax(out)
 # print("classId",classId)
 print("预测结果为:",class_name[classId])
 src_image = cv2.putText(src_image,str(classId),(300,100), cv2.FONT_HERSHEY_SIMPLEX, 2,(0,0,255),2,4)
 # cv.putText(img, text, org, fontFace, fontScale, fontcolor, thickness, lineType)
 cv2.imshow("pic",src_image)
 if cv2.waitKey(10) == ord('0'):
  break

小结

这里本质上还是一个图像分类任务。而且,样本数量较少。优化的时候需要做数据增强,还需要防止过拟合。

到此这篇关于TensorFlow2.X结合OpenCV 实现手势识别功能的文章就介绍到这了,更多相关TensorFlow OpenCV 手势识别内容请搜索三水点靠木以前的文章或继续浏览下面的相关文章希望大家以后多多支持三水点靠木!

Python 相关文章推荐
tensorflow中next_batch的具体使用
Feb 02 Python
Python+PIL实现支付宝AR红包
Feb 09 Python
selenium+python 对输入框的输入处理方法
Oct 11 Python
PyQt5基本控件使用详解:单选按钮、复选框、下拉框
Aug 05 Python
Python面向对象之Web静态服务器
Sep 03 Python
Python tensorflow实现mnist手写数字识别示例【非卷积与卷积实现】
Dec 19 Python
pytorch::Dataloader中的迭代器和生成器应用详解
Jan 03 Python
torchxrayvision包安装过程(附pytorch1.6cpu版安装)
Aug 26 Python
pycharm 快速解决python代码冲突的问题
Jan 15 Python
如何用tempfile库创建python进程中的临时文件
Jan 28 Python
Python基础详解之描述符
Apr 28 Python
Python爬虫基础讲解之请求
May 13 Python
python 安装库几种方法之cmd,anaconda,pycharm详解
Apr 08 #Python
TensorFlow2.1.0最新版本安装详细教程
Apr 08 #Python
解决python多线程报错:AttributeError: Can't pickle local object问题
Apr 08 #Python
解决Python 异常TypeError: cannot concatenate 'str' and 'int' objects
Apr 08 #Python
TensorFlow2.1.0安装过程中setuptools、wrapt等相关错误指南
Apr 08 #Python
解决windows下python3使用multiprocessing.Pool出现的问题
Apr 08 #Python
python操作yaml说明
Apr 08 #Python
You might like
我常用的几个类
2006/10/09 PHP
PHP处理excel cvs表格的方法实例介绍
2013/05/13 PHP
PHP函数实现分页含文本分页和数字分页
2014/10/23 PHP
php商品对比功能代码分享
2015/09/24 PHP
PHP导出带样式的Excel示例代码
2016/08/28 PHP
jquery+thinkphp实现跨域抓取数据的方法
2016/10/15 PHP
JavaScript游戏之是男人就下100层代码打包
2010/11/08 Javascript
json数据处理技巧(字段带空格、增加字段、排序等等)
2013/06/14 Javascript
JavaScript使用indexOf获得子字符串在字符串中位置的方法
2015/04/06 Javascript
js实现超酷的照片墙展示效果图附源码下载
2015/10/08 Javascript
js实现打地鼠小游戏
2017/02/13 Javascript
JavaScript实现两个select下拉框选项左移右移
2017/03/09 Javascript
基于JavaScript实现焦点图轮播效果
2017/03/27 Javascript
Node.js对MongoDB数据库实现模糊查询的方法
2017/05/03 Javascript
Vue报错:Uncaught TypeError: Cannot assign to read only property’exports‘ of object’#‘的解决方法
2017/06/17 Javascript
详解vue项目打包后通过百度的BAE发布到网上的流程
2018/03/05 Javascript
React BootStrap用户体验框架快速上手
2018/03/06 Javascript
详解性能更优越的小程序图片懒加载方式
2018/07/18 Javascript
JavaScript中的惰性载入函数及优势
2020/02/18 Javascript
[37:37]DAC2018 4.4 淘汰赛 Optic vs Mineski 第二场
2018/04/05 DOTA
[03:17]史诗级大片应援2018DOTA2国际邀请赛 致敬每一位坚守遗迹的勇士
2018/07/20 DOTA
Python对字符串实现去重操作的方法示例
2017/08/11 Python
Python+Selenium+phantomjs实现网页模拟登录和截图功能(windows环境)
2019/12/11 Python
python词云库wordcloud的使用方法与实例详解
2020/02/17 Python
Python安装Bs4的多种方法
2020/11/28 Python
CSS3利用text-shadow属性实现多种效果的文字样式展现方法
2016/08/25 HTML / CSS
使用CSS3实现input多选框自定义样式的方法示例
2019/07/19 HTML / CSS
canvas实现飞机打怪兽射击小游戏的示例代码
2018/07/09 HTML / CSS
德国鞋子网上商店:Omoda.de
2017/03/31 全球购物
新闻学专业应届生求职信
2013/11/08 职场文书
甜美蛋糕店创业计划书
2014/01/30 职场文书
工地例会施工汇报材料
2014/08/22 职场文书
会议主持词结束语
2015/07/03 职场文书
手把手教你用SpringBoot将文件打包成zip存放或导出
2021/06/11 Java/Android
浅谈Web Storage API的使用
2021/06/23 Javascript
理解python中装饰器的作用
2021/07/21 Python