TensorFlow2.X结合OpenCV 实现手势识别功能


Posted in Python onApril 08, 2020

使用Tensorflow 构建卷积神经网络,训练手势识别模型,使用opencv DNN 模块加载模型实时手势识别
效果如下:

TensorFlow2.X结合OpenCV 实现手势识别功能

先显示下部分数据集图片(0到9的表示,感觉很怪)

TensorFlow2.X结合OpenCV 实现手势识别功能

构建模型进行训练

import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import datasets,layers,optimizers,Sequential,metrics
from tensorflow.python.framework.convert_to_constants import convert_variables_to_constants_v2
import os 
import pathlib
import random
import matplotlib.pyplot as plt
os.environ['TF_CPP_MIN_LOG_LEVEL']='2'
def read_data(path):
 path_root = pathlib.Path(path)
 # print(path_root)
 # for item in path_root.iterdir():
 #  print(item)
 image_paths = list(path_root.glob('*/*'))
 image_paths = [str(path) for path in image_paths]
 random.shuffle(image_paths)
 image_count = len(image_paths)
 # print(image_count)
 # print(image_paths[:10])
 label_names = sorted(item.name for item in path_root.glob('*/') if item.is_dir())
 # print(label_names)
 label_name_index = dict((name, index) for index, name in enumerate(label_names))
 # print(label_name_index)
 image_labels = [label_name_index[pathlib.Path(path).parent.name] for path in image_paths]
 # print("First 10 labels indices: ", image_labels[:10])
 return image_paths,image_labels,image_count
def preprocess_image(image):
 image = tf.image.decode_jpeg(image, channels=3)
 image = tf.image.resize(image, [100, 100])
 image /= 255.0 # normalize to [0,1] range
 # image = tf.reshape(image,[100*100*3])
 return image
def load_and_preprocess_image(path,label):
 image = tf.io.read_file(path)
 return preprocess_image(image),label
def creat_dataset(image_paths,image_labels,bitch_size):
 db = tf.data.Dataset.from_tensor_slices((image_paths, image_labels))
 dataset = db.map(load_and_preprocess_image).batch(bitch_size) 
 return dataset
def train_model(train_data,test_data):
 #构建模型
 network = keras.Sequential([
   keras.layers.Conv2D(32,kernel_size=[5,5],padding="same",activation=tf.nn.relu),
   keras.layers.MaxPool2D(pool_size=[2, 2], strides=2, padding='same'),
   keras.layers.Conv2D(64,kernel_size=[3,3],padding="same",activation=tf.nn.relu),
   keras.layers.MaxPool2D(pool_size=[2, 2], strides=2, padding='same'),
   keras.layers.Conv2D(64,kernel_size=[3,3],padding="same",activation=tf.nn.relu),
   keras.layers.Flatten(),
   keras.layers.Dense(512,activation='relu'),
   keras.layers.Dropout(0.5),
   keras.layers.Dense(128,activation='relu'),
   keras.layers.Dense(10)])
 network.build(input_shape=(None,100,100,3))
 network.summary()
 network.compile(optimizer=optimizers.SGD(lr=0.001),
   loss=tf.losses.SparseCategoricalCrossentropy(from_logits=True),
   metrics=['accuracy']
 )
 #模型训练
 network.fit(train_data, epochs = 100,validation_data=test_data,validation_freq=2) 
 network.evaluate(test_data)
 tf.saved_model.save(network,'D:\\code\\PYTHON\\gesture_recognition\\model\\')
 print("保存模型成功")
 # Convert Keras model to ConcreteFunction
 full_model = tf.function(lambda x: network(x))
 full_model = full_model.get_concrete_function(
 tf.TensorSpec(network.inputs[0].shape, network.inputs[0].dtype))
 # Get frozen ConcreteFunction
 frozen_func = convert_variables_to_constants_v2(full_model)
 frozen_func.graph.as_graph_def()

 layers = [op.name for op in frozen_func.graph.get_operations()]
 print("-" * 50)
 print("Frozen model layers: ")
 for layer in layers:
  print(layer)

 print("-" * 50)
 print("Frozen model inputs: ")
 print(frozen_func.inputs)
 print("Frozen model outputs: ")
 print(frozen_func.outputs)

 # Save frozen graph from frozen ConcreteFunction to hard drive
 tf.io.write_graph(graph_or_graph_def=frozen_func.graph,
   logdir="D:\\code\\PYTHON\\gesture_recognition\\model\\frozen_model\\",
   name="frozen_graph.pb",
   as_text=False)
 print("模型转换完成,训练结束")


if __name__ == "__main__":
 print(tf.__version__)
 train_path = 'D:\\code\\PYTHON\\gesture_recognition\\Dataset'
 test_path = 'D:\\code\\PYTHON\\gesture_recognition\\testdata' 
 image_paths,image_labels,_ = read_data(train_path)
 train_data = creat_dataset(image_paths,image_labels,16)
 image_paths,image_labels,_ = read_data(test_path)
 test_data = creat_dataset(image_paths,image_labels,16)
 train_model(train_data,test_data)

OpenCV加载模型,实时检测

这里为了简化检测使用了ROI。

import cv2
from cv2 import dnn
import numpy as np
print(cv2.__version__)
class_name = ['0', '1', '2', '3', '4', '5', '6', '7', '8', '9']
net = dnn.readNetFromTensorflow('D:\\code\\PYTHON\\gesture_recognition\\model\\frozen_model\\frozen_graph.pb')
cap = cv2.VideoCapture(0)
i = 0
while True:
 _,frame= cap.read() 
 src_image = frame
 cv2.rectangle(src_image, (300, 100),(600, 400), (0, 255, 0), 1, 4)
 frame = cv2.cvtColor(frame,cv2.COLOR_BGR2RGB)
 pic = frame[100:400,300:600]
 cv2.imshow("pic1", pic)
 # print(pic.shape)
 pic = cv2.resize(pic,(100,100))
 blob = cv2.dnn.blobFromImage(pic,  
        scalefactor=1.0/225.,
        size=(100, 100),
        mean=(0, 0, 0),
        swapRB=False,
        crop=False)
 # blob = np.transpose(blob, (0,2,3,1))       
 net.setInput(blob)
 out = net.forward()
 out = out.flatten()

 classId = np.argmax(out)
 # print("classId",classId)
 print("预测结果为:",class_name[classId])
 src_image = cv2.putText(src_image,str(classId),(300,100), cv2.FONT_HERSHEY_SIMPLEX, 2,(0,0,255),2,4)
 # cv.putText(img, text, org, fontFace, fontScale, fontcolor, thickness, lineType)
 cv2.imshow("pic",src_image)
 if cv2.waitKey(10) == ord('0'):
  break

小结

这里本质上还是一个图像分类任务。而且,样本数量较少。优化的时候需要做数据增强,还需要防止过拟合。

到此这篇关于TensorFlow2.X结合OpenCV 实现手势识别功能的文章就介绍到这了,更多相关TensorFlow OpenCV 手势识别内容请搜索三水点靠木以前的文章或继续浏览下面的相关文章希望大家以后多多支持三水点靠木!

Python 相关文章推荐
python解决汉字编码问题:Unicode Decode Error
Jan 19 Python
开源Web应用框架Django图文教程
Mar 09 Python
Python基于PyGraphics包实现图片截取功能的方法
Dec 21 Python
Windows下Anaconda的安装和简单使用方法
Jan 04 Python
TensorFlow实现Batch Normalization
Mar 08 Python
详解Python做一个名片管理系统
Mar 14 Python
pandas.DataFrame的pivot()和unstack()实现行转列
Jul 06 Python
Python浮点数四舍五入问题的分析与解决方法
Nov 19 Python
Python实现bilibili时间长度查询的示例代码
Jan 14 Python
Python 写了个新型冠状病毒疫情传播模拟程序
Feb 14 Python
python中tkinter复选框使用操作
Nov 11 Python
Python可视化神器pyecharts绘制地理图表
Jul 07 Python
python 安装库几种方法之cmd,anaconda,pycharm详解
Apr 08 #Python
TensorFlow2.1.0最新版本安装详细教程
Apr 08 #Python
解决python多线程报错:AttributeError: Can't pickle local object问题
Apr 08 #Python
解决Python 异常TypeError: cannot concatenate 'str' and 'int' objects
Apr 08 #Python
TensorFlow2.1.0安装过程中setuptools、wrapt等相关错误指南
Apr 08 #Python
解决windows下python3使用multiprocessing.Pool出现的问题
Apr 08 #Python
python操作yaml说明
Apr 08 #Python
You might like
用PHPdig打造属于你自己的Google[图文教程]
2007/02/14 PHP
用mysql内存表来代替php session的类
2009/02/01 PHP
php对csv文件的读取,写入,输出下载操作详解
2013/08/10 PHP
php获取文件类型和文件信息的方法
2015/07/10 PHP
thinkphp5.1框架模板布局与模板继承用法分析
2019/07/19 PHP
javascript实现TreeView 无刷新展开的实例代码
2013/07/13 Javascript
createTextRange()的使用示例含文本框选中部分文字内容
2014/02/24 Javascript
Javascript学习笔记之 函数篇(二) : this 的工作机制
2014/06/24 Javascript
Javscript调用iframe框架页面中函数的方法
2014/11/01 Javascript
node.js中使用socket.io的方法
2014/12/15 Javascript
Javascript实现计算个人所得税
2015/05/10 Javascript
jQuery实现图片加载完成后改变图片大小的方法
2016/03/29 Javascript
Bootstrap框架结合jQuery仿百度换肤功能实例解析
2016/09/17 Javascript
JavaScript利用正则表达式替换字符串中的内容
2016/12/12 Javascript
js中开关变量使用实例
2017/02/24 Javascript
JS实现数组删除指定元素功能示例
2019/06/05 Javascript
解决Vue watch里调用方法的坑
2020/11/07 Javascript
python实现linux服务器批量修改密码并生成execl
2014/04/22 Python
pygame学习笔记(5):游戏精灵
2015/04/15 Python
Python删除空文件和空文件夹的方法
2015/07/14 Python
用TensorFlow实现戴明回归算法的示例
2018/05/02 Python
python random从集合中随机选择元素的方法
2019/01/23 Python
python 堆和优先队列的使用详解
2019/03/05 Python
python3 pygame实现接小球游戏
2019/05/14 Python
Python通用函数实现数组计算的方法
2019/06/13 Python
Python pygame绘制文字制作滚动文字过程解析
2019/12/12 Python
PyTorch实现更新部分网络,其他不更新
2019/12/31 Python
Python原始套接字编程实例解析
2020/01/29 Python
Ubuntu权限不足无法创建文件夹解决方案
2020/11/14 Python
Jupyter Notebook 安装配置与使用详解
2021/01/06 Python
金牌葡萄酒俱乐部:Gold Medal Wine Club
2017/11/02 全球购物
内容编辑个人求职信
2013/12/10 职场文书
自我评价200字分享
2013/12/17 职场文书
2014医学院领导干部四风对照检查材料思想汇报
2014/09/16 职场文书
整改落实自查报告
2014/11/05 职场文书
Django中session进行权限管理的使用
2021/07/09 Python