TensorFlow2.X结合OpenCV 实现手势识别功能


Posted in Python onApril 08, 2020

使用Tensorflow 构建卷积神经网络,训练手势识别模型,使用opencv DNN 模块加载模型实时手势识别
效果如下:

TensorFlow2.X结合OpenCV 实现手势识别功能

先显示下部分数据集图片(0到9的表示,感觉很怪)

TensorFlow2.X结合OpenCV 实现手势识别功能

构建模型进行训练

import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import datasets,layers,optimizers,Sequential,metrics
from tensorflow.python.framework.convert_to_constants import convert_variables_to_constants_v2
import os 
import pathlib
import random
import matplotlib.pyplot as plt
os.environ['TF_CPP_MIN_LOG_LEVEL']='2'
def read_data(path):
 path_root = pathlib.Path(path)
 # print(path_root)
 # for item in path_root.iterdir():
 #  print(item)
 image_paths = list(path_root.glob('*/*'))
 image_paths = [str(path) for path in image_paths]
 random.shuffle(image_paths)
 image_count = len(image_paths)
 # print(image_count)
 # print(image_paths[:10])
 label_names = sorted(item.name for item in path_root.glob('*/') if item.is_dir())
 # print(label_names)
 label_name_index = dict((name, index) for index, name in enumerate(label_names))
 # print(label_name_index)
 image_labels = [label_name_index[pathlib.Path(path).parent.name] for path in image_paths]
 # print("First 10 labels indices: ", image_labels[:10])
 return image_paths,image_labels,image_count
def preprocess_image(image):
 image = tf.image.decode_jpeg(image, channels=3)
 image = tf.image.resize(image, [100, 100])
 image /= 255.0 # normalize to [0,1] range
 # image = tf.reshape(image,[100*100*3])
 return image
def load_and_preprocess_image(path,label):
 image = tf.io.read_file(path)
 return preprocess_image(image),label
def creat_dataset(image_paths,image_labels,bitch_size):
 db = tf.data.Dataset.from_tensor_slices((image_paths, image_labels))
 dataset = db.map(load_and_preprocess_image).batch(bitch_size) 
 return dataset
def train_model(train_data,test_data):
 #构建模型
 network = keras.Sequential([
   keras.layers.Conv2D(32,kernel_size=[5,5],padding="same",activation=tf.nn.relu),
   keras.layers.MaxPool2D(pool_size=[2, 2], strides=2, padding='same'),
   keras.layers.Conv2D(64,kernel_size=[3,3],padding="same",activation=tf.nn.relu),
   keras.layers.MaxPool2D(pool_size=[2, 2], strides=2, padding='same'),
   keras.layers.Conv2D(64,kernel_size=[3,3],padding="same",activation=tf.nn.relu),
   keras.layers.Flatten(),
   keras.layers.Dense(512,activation='relu'),
   keras.layers.Dropout(0.5),
   keras.layers.Dense(128,activation='relu'),
   keras.layers.Dense(10)])
 network.build(input_shape=(None,100,100,3))
 network.summary()
 network.compile(optimizer=optimizers.SGD(lr=0.001),
   loss=tf.losses.SparseCategoricalCrossentropy(from_logits=True),
   metrics=['accuracy']
 )
 #模型训练
 network.fit(train_data, epochs = 100,validation_data=test_data,validation_freq=2) 
 network.evaluate(test_data)
 tf.saved_model.save(network,'D:\\code\\PYTHON\\gesture_recognition\\model\\')
 print("保存模型成功")
 # Convert Keras model to ConcreteFunction
 full_model = tf.function(lambda x: network(x))
 full_model = full_model.get_concrete_function(
 tf.TensorSpec(network.inputs[0].shape, network.inputs[0].dtype))
 # Get frozen ConcreteFunction
 frozen_func = convert_variables_to_constants_v2(full_model)
 frozen_func.graph.as_graph_def()

 layers = [op.name for op in frozen_func.graph.get_operations()]
 print("-" * 50)
 print("Frozen model layers: ")
 for layer in layers:
  print(layer)

 print("-" * 50)
 print("Frozen model inputs: ")
 print(frozen_func.inputs)
 print("Frozen model outputs: ")
 print(frozen_func.outputs)

 # Save frozen graph from frozen ConcreteFunction to hard drive
 tf.io.write_graph(graph_or_graph_def=frozen_func.graph,
   logdir="D:\\code\\PYTHON\\gesture_recognition\\model\\frozen_model\\",
   name="frozen_graph.pb",
   as_text=False)
 print("模型转换完成,训练结束")


if __name__ == "__main__":
 print(tf.__version__)
 train_path = 'D:\\code\\PYTHON\\gesture_recognition\\Dataset'
 test_path = 'D:\\code\\PYTHON\\gesture_recognition\\testdata' 
 image_paths,image_labels,_ = read_data(train_path)
 train_data = creat_dataset(image_paths,image_labels,16)
 image_paths,image_labels,_ = read_data(test_path)
 test_data = creat_dataset(image_paths,image_labels,16)
 train_model(train_data,test_data)

OpenCV加载模型,实时检测

这里为了简化检测使用了ROI。

import cv2
from cv2 import dnn
import numpy as np
print(cv2.__version__)
class_name = ['0', '1', '2', '3', '4', '5', '6', '7', '8', '9']
net = dnn.readNetFromTensorflow('D:\\code\\PYTHON\\gesture_recognition\\model\\frozen_model\\frozen_graph.pb')
cap = cv2.VideoCapture(0)
i = 0
while True:
 _,frame= cap.read() 
 src_image = frame
 cv2.rectangle(src_image, (300, 100),(600, 400), (0, 255, 0), 1, 4)
 frame = cv2.cvtColor(frame,cv2.COLOR_BGR2RGB)
 pic = frame[100:400,300:600]
 cv2.imshow("pic1", pic)
 # print(pic.shape)
 pic = cv2.resize(pic,(100,100))
 blob = cv2.dnn.blobFromImage(pic,  
        scalefactor=1.0/225.,
        size=(100, 100),
        mean=(0, 0, 0),
        swapRB=False,
        crop=False)
 # blob = np.transpose(blob, (0,2,3,1))       
 net.setInput(blob)
 out = net.forward()
 out = out.flatten()

 classId = np.argmax(out)
 # print("classId",classId)
 print("预测结果为:",class_name[classId])
 src_image = cv2.putText(src_image,str(classId),(300,100), cv2.FONT_HERSHEY_SIMPLEX, 2,(0,0,255),2,4)
 # cv.putText(img, text, org, fontFace, fontScale, fontcolor, thickness, lineType)
 cv2.imshow("pic",src_image)
 if cv2.waitKey(10) == ord('0'):
  break

小结

这里本质上还是一个图像分类任务。而且,样本数量较少。优化的时候需要做数据增强,还需要防止过拟合。

到此这篇关于TensorFlow2.X结合OpenCV 实现手势识别功能的文章就介绍到这了,更多相关TensorFlow OpenCV 手势识别内容请搜索三水点靠木以前的文章或继续浏览下面的相关文章希望大家以后多多支持三水点靠木!

Python 相关文章推荐
python 图片验证码代码
Dec 07 Python
使用PyCharm配合部署Python的Django框架的配置纪实
Nov 19 Python
Python内置模块logging用法实例分析
Feb 12 Python
75条笑死人的知乎神回复,用60行代码就爬完了
May 06 Python
django 通过URL访问上传的文件方法
Jul 28 Python
python将字母转化为数字实例方法
Oct 04 Python
树莓派安装OpenCV3完整过程的实现
Oct 10 Python
Pandas+Matplotlib 箱式图异常值分析示例
Dec 09 Python
pygame实现飞机大战
Mar 11 Python
python实现猜单词游戏
May 22 Python
python:HDF和CSV存储优劣对比分析
Jun 08 Python
Pycharm自带Git实现版本管理的方法步骤
Sep 18 Python
python 安装库几种方法之cmd,anaconda,pycharm详解
Apr 08 #Python
TensorFlow2.1.0最新版本安装详细教程
Apr 08 #Python
解决python多线程报错:AttributeError: Can't pickle local object问题
Apr 08 #Python
解决Python 异常TypeError: cannot concatenate 'str' and 'int' objects
Apr 08 #Python
TensorFlow2.1.0安装过程中setuptools、wrapt等相关错误指南
Apr 08 #Python
解决windows下python3使用multiprocessing.Pool出现的问题
Apr 08 #Python
python操作yaml说明
Apr 08 #Python
You might like
PHP URL地址获取函数代码(端口等) 推荐
2010/05/15 PHP
php url地址栏传中文乱码解决方法集合
2010/06/25 PHP
解析MySql与Java的时间类型
2013/06/22 PHP
php strnatcmp()函数的用法总结
2013/11/27 PHP
JavaScript 变量命名规则
2009/09/23 Javascript
Javascript new关键字的玄机 以及其它
2010/08/25 Javascript
javascript ajax 仿百度分页函数
2013/10/29 Javascript
js实现网页倒计时、网站已运行时间功能的代码3例
2014/04/14 Javascript
node.js中的console.error方法使用说明
2014/12/10 Javascript
深入理解JavaScript的React框架的原理
2015/07/02 Javascript
yarn的使用与升级Node.js的方法详解
2017/06/04 Javascript
React Native实现进度条弹框的示例代码
2017/07/17 Javascript
Angular实现svg和png图片下载实现
2019/05/05 Javascript
在SSM框架下用laypage和ajax实现分页和数据交互的方法
2019/09/27 Javascript
JavaScript 实现自己的安卓手机自动化工具脚本(推荐)
2020/05/13 Javascript
[48:23]DOTA2上海特级锦标赛主赛事日 - 4 败者组第四轮#1COL VS EG第一局
2016/03/05 DOTA
Python实现识别手写数字大纲
2018/01/29 Python
修改python plot折线图的坐标轴刻度方法
2018/12/13 Python
python关于调用函数外的变量实例
2019/12/26 Python
Python开发企业微信机器人每天定时发消息实例
2020/03/17 Python
英国女士家居服网站:hush
2017/08/09 全球购物
师范大学音乐表演专业求职信
2013/10/23 职场文书
夜大毕业生自我鉴定
2013/10/31 职场文书
配件采购员岗位职责
2013/12/03 职场文书
初一学生评语大全
2014/04/24 职场文书
珍爱生命演讲稿
2014/05/10 职场文书
中华魂演讲稿
2014/05/13 职场文书
爱护公物标语
2014/06/24 职场文书
党员志愿者活动总结
2014/06/26 职场文书
我爱幼儿园演讲稿
2014/09/11 职场文书
信息合作协议书
2014/10/09 职场文书
干部作风建设工作总结
2014/10/29 职场文书
不同意离婚答辩状
2015/05/22 职场文书
毕业论文指导老师意见
2015/06/04 职场文书
门卫管理制度范本
2015/08/05 职场文书
导游词之西安大清真寺
2019/12/17 职场文书