TensorFLow 不同大小图片的TFrecords存取实例


Posted in Python onJanuary 20, 2020

全部存入一个TFrecords文件,然后读取并显示第一张。

不多写了,直接贴代码。

from PIL import Image
import numpy as np
import matplotlib.pyplot as plt
import tensorflow as tf


IMAGE_PATH = 'test/'
tfrecord_file = IMAGE_PATH + 'test.tfrecord'
writer = tf.python_io.TFRecordWriter(tfrecord_file)


def _int64_feature(value):
 return tf.train.Feature(int64_list=tf.train.Int64List(value=[value]))

def _bytes_feature(value):
 return tf.train.Feature(bytes_list=tf.train.BytesList(value=[value]))

def get_image_binary(filename):
  """ You can read in the image using tensorflow too, but it's a drag
    since you have to create graphs. It's much easier using Pillow and NumPy
  """
  image = Image.open(filename)
  image = np.asarray(image, np.uint8)
  shape = np.array(image.shape, np.int32)
  return shape, image.tobytes() # convert image to raw data bytes in the array.

def write_to_tfrecord(label, shape, binary_image, tfrecord_file):
  """ This example is to write a sample to TFRecord file. If you want to write
  more samples, just use a loop.
  """
  # write label, shape, and image content to the TFRecord file
  example = tf.train.Example(features=tf.train.Features(feature={
        'label': _int64_feature(label),
        'h': _int64_feature(shape[0]),
        'w': _int64_feature(shape[1]),
        'c': _int64_feature(shape[2]),
        'image': _bytes_feature(binary_image)
        }))
  writer.write(example.SerializeToString())


def write_tfrecord(label, image_file, tfrecord_file):
  shape, binary_image = get_image_binary(image_file)
  write_to_tfrecord(label, shape, binary_image, tfrecord_file)
  # print(shape)



def main():
  # assume the image has the label Chihuahua, which corresponds to class number 1
  label = [1,2]
  image_files = [IMAGE_PATH + 'a.jpg', IMAGE_PATH + 'b.jpg']

  for i in range(2):
    write_tfrecord(label[i], image_files[i], tfrecord_file)
  writer.close()

  batch_size = 2

  filename_queue = tf.train.string_input_producer([tfrecord_file]) 
  reader = tf.TFRecordReader() 
  _, serialized_example = reader.read(filename_queue) 

  img_features = tf.parse_single_example( 
                    serialized_example, 
                    features={ 
                        'label': tf.FixedLenFeature([], tf.int64), 
                        'h': tf.FixedLenFeature([], tf.int64),
                        'w': tf.FixedLenFeature([], tf.int64),
                        'c': tf.FixedLenFeature([], tf.int64),
                        'image': tf.FixedLenFeature([], tf.string), 
                        }) 

  h = tf.cast(img_features['h'], tf.int32)
  w = tf.cast(img_features['w'], tf.int32)
  c = tf.cast(img_features['c'], tf.int32)

  image = tf.decode_raw(img_features['image'], tf.uint8) 
  image = tf.reshape(image, [h, w, c])

  label = tf.cast(img_features['label'],tf.int32) 
  label = tf.reshape(label, [1])

 # image = tf.image.resize_images(image, (500,500))
  #image, label = tf.train.batch([image, label], batch_size= batch_size) 


  with tf.Session() as sess:
    coord = tf.train.Coordinator()
    threads = tf.train.start_queue_runners(coord=coord)
    image, label=sess.run([image, label])
    coord.request_stop()
    coord.join(threads)

    print(label)

    plt.figure()
    plt.imshow(image)
    plt.show()


if __name__ == '__main__':
  main()

全部存入一个TFrecords文件,然后按照batch_size读取,注意需要将图片变成一样大才能按照batch_size读取。

from PIL import Image
import numpy as np
import matplotlib.pyplot as plt
import tensorflow as tf


IMAGE_PATH = 'test/'
tfrecord_file = IMAGE_PATH + 'test.tfrecord'
writer = tf.python_io.TFRecordWriter(tfrecord_file)


def _int64_feature(value):
 return tf.train.Feature(int64_list=tf.train.Int64List(value=[value]))

def _bytes_feature(value):
 return tf.train.Feature(bytes_list=tf.train.BytesList(value=[value]))

def get_image_binary(filename):
  """ You can read in the image using tensorflow too, but it's a drag
    since you have to create graphs. It's much easier using Pillow and NumPy
  """
  image = Image.open(filename)
  image = np.asarray(image, np.uint8)
  shape = np.array(image.shape, np.int32)
  return shape, image.tobytes() # convert image to raw data bytes in the array.

def write_to_tfrecord(label, shape, binary_image, tfrecord_file):
  """ This example is to write a sample to TFRecord file. If you want to write
  more samples, just use a loop.
  """
  # write label, shape, and image content to the TFRecord file
  example = tf.train.Example(features=tf.train.Features(feature={
        'label': _int64_feature(label),
        'h': _int64_feature(shape[0]),
        'w': _int64_feature(shape[1]),
        'c': _int64_feature(shape[2]),
        'image': _bytes_feature(binary_image)
        }))
  writer.write(example.SerializeToString())


def write_tfrecord(label, image_file, tfrecord_file):
  shape, binary_image = get_image_binary(image_file)
  write_to_tfrecord(label, shape, binary_image, tfrecord_file)
  # print(shape)



def main():
  # assume the image has the label Chihuahua, which corresponds to class number 1
  label = [1,2]
  image_files = [IMAGE_PATH + 'a.jpg', IMAGE_PATH + 'b.jpg']

  for i in range(2):
    write_tfrecord(label[i], image_files[i], tfrecord_file)
  writer.close()

  batch_size = 2

  filename_queue = tf.train.string_input_producer([tfrecord_file]) 
  reader = tf.TFRecordReader() 
  _, serialized_example = reader.read(filename_queue) 

  img_features = tf.parse_single_example( 
                    serialized_example, 
                    features={ 
                        'label': tf.FixedLenFeature([], tf.int64), 
                        'h': tf.FixedLenFeature([], tf.int64),
                        'w': tf.FixedLenFeature([], tf.int64),
                        'c': tf.FixedLenFeature([], tf.int64),
                        'image': tf.FixedLenFeature([], tf.string), 
                        }) 

  h = tf.cast(img_features['h'], tf.int32)
  w = tf.cast(img_features['w'], tf.int32)
  c = tf.cast(img_features['c'], tf.int32)

  image = tf.decode_raw(img_features['image'], tf.uint8) 
  image = tf.reshape(image, [h, w, c])

  label = tf.cast(img_features['label'],tf.int32) 
  label = tf.reshape(label, [1])

  image = tf.image.resize_images(image, (224,224))
  image = tf.reshape(image, [224, 224, 3])
  image, label = tf.train.batch([image, label], batch_size= batch_size) 


  with tf.Session() as sess:
    coord = tf.train.Coordinator()
    threads = tf.train.start_queue_runners(coord=coord)
    image, label=sess.run([image, label])
    coord.request_stop()
    coord.join(threads)

    print(image.shape)
    print(label)

    plt.figure()
    plt.imshow(image[0,:,:,0])
    plt.show()

    plt.figure()
    plt.imshow(image[0,:,:,1])
    plt.show()

    image1 = image[0,:,:,:]
    print(image1.shape)
    print(image1.dtype)
    im = Image.fromarray(np.uint8(image1)) #参考numpy和图片的互转:http://blog.csdn.net/zywvvd/article/details/72810360
    im.show()

if __name__ == '__main__':
  main()

输出是

(2, 224, 224, 3)
[[1]
 [2]]

第一张图片的三种显示(略)

封装成函数:

# -*- coding: utf-8 -*-
"""
Created on Fri Sep 8 14:38:15 2017

@author: wayne


"""


'''
本文参考了以下代码,在多个不同大小图片存取方面做了重新开发:
https://github.com/chiphuyen/stanford-tensorflow-tutorials/blob/master/examples/09_tfrecord_example.py
http://blog.csdn.net/hjxu2016/article/details/76165559
https://stackoverflow.com/questions/41921746/tensorflow-varlenfeature-vs-fixedlenfeature
https://github.com/tensorflow/tensorflow/issues/10492

后续:
-存入多个TFrecords文件的例子见
http://blog.csdn.net/xierhacker/article/details/72357651
-如何作shuffle和数据增强
string_input_producer (需要理解tf的数据流,标签队列的工作方式等等)
http://blog.csdn.net/liuchonge/article/details/73649251
'''

from PIL import Image
import numpy as np
import matplotlib.pyplot as plt
import tensorflow as tf


IMAGE_PATH = 'test/'
tfrecord_file = IMAGE_PATH + 'test.tfrecord'
writer = tf.python_io.TFRecordWriter(tfrecord_file)


def _int64_feature(value):
 return tf.train.Feature(int64_list=tf.train.Int64List(value=[value]))

def _bytes_feature(value):
 return tf.train.Feature(bytes_list=tf.train.BytesList(value=[value]))

def get_image_binary(filename):
  """ You can read in the image using tensorflow too, but it's a drag
    since you have to create graphs. It's much easier using Pillow and NumPy
  """
  image = Image.open(filename)
  image = np.asarray(image, np.uint8)
  shape = np.array(image.shape, np.int32)
  return shape, image.tobytes() # convert image to raw data bytes in the array.

def write_to_tfrecord(label, shape, binary_image, tfrecord_file):
  """ This example is to write a sample to TFRecord file. If you want to write
  more samples, just use a loop.
  """
  # write label, shape, and image content to the TFRecord file
  example = tf.train.Example(features=tf.train.Features(feature={
        'label': _int64_feature(label),
        'h': _int64_feature(shape[0]),
        'w': _int64_feature(shape[1]),
        'c': _int64_feature(shape[2]),
        'image': _bytes_feature(binary_image)
        }))
  writer.write(example.SerializeToString())


def write_tfrecord(label, image_file, tfrecord_file):
  shape, binary_image = get_image_binary(image_file)
  write_to_tfrecord(label, shape, binary_image, tfrecord_file)


def read_and_decode(tfrecords_file, batch_size): 
  '''''read and decode tfrecord file, generate (image, label) batches 
  Args: 
    tfrecords_file: the directory of tfrecord file 
    batch_size: number of images in each batch 
  Returns: 
    image: 4D tensor - [batch_size, width, height, channel] 
    label: 1D tensor - [batch_size] 
  ''' 
  # make an input queue from the tfrecord file 

  filename_queue = tf.train.string_input_producer([tfrecord_file]) 
  reader = tf.TFRecordReader() 
  _, serialized_example = reader.read(filename_queue) 

  img_features = tf.parse_single_example( 
                    serialized_example, 
                    features={ 
                        'label': tf.FixedLenFeature([], tf.int64), 
                        'h': tf.FixedLenFeature([], tf.int64),
                        'w': tf.FixedLenFeature([], tf.int64),
                        'c': tf.FixedLenFeature([], tf.int64),
                        'image': tf.FixedLenFeature([], tf.string), 
                        }) 

  h = tf.cast(img_features['h'], tf.int32)
  w = tf.cast(img_features['w'], tf.int32)
  c = tf.cast(img_features['c'], tf.int32)

  image = tf.decode_raw(img_features['image'], tf.uint8) 
  image = tf.reshape(image, [h, w, c])

  label = tf.cast(img_features['label'],tf.int32) 
  label = tf.reshape(label, [1])

  ########################################################## 
  # you can put data augmentation here  
#  distorted_image = tf.random_crop(images, [530, 530, img_channel])
#  distorted_image = tf.image.random_flip_left_right(distorted_image)
#  distorted_image = tf.image.random_brightness(distorted_image, max_delta=63)
#  distorted_image = tf.image.random_contrast(distorted_image, lower=0.2, upper=1.8)
#  distorted_image = tf.image.resize_images(distorted_image, (imagesize,imagesize))
#  float_image = tf.image.per_image_standardization(distorted_image)

  image = tf.image.resize_images(image, (224,224))
  image = tf.reshape(image, [224, 224, 3])
  #image, label = tf.train.batch([image, label], batch_size= batch_size) 

  image_batch, label_batch = tf.train.batch([image, label], 
                        batch_size= batch_size, 
                        num_threads= 64,  
                        capacity = 2000) 
  return image_batch, tf.reshape(label_batch, [batch_size]) 

def read_tfrecord2(tfrecord_file, batch_size):
  train_batch, train_label_batch = read_and_decode(tfrecord_file, batch_size)

  with tf.Session() as sess:
    coord = tf.train.Coordinator()
    threads = tf.train.start_queue_runners(coord=coord)
    train_batch, train_label_batch = sess.run([train_batch, train_label_batch])
    coord.request_stop()
    coord.join(threads)
  return train_batch, train_label_batch


def main():
  # assume the image has the label Chihuahua, which corresponds to class number 1
  label = [1,2]
  image_files = [IMAGE_PATH + 'a.jpg', IMAGE_PATH + 'b.jpg']

  for i in range(2):
    write_tfrecord(label[i], image_files[i], tfrecord_file)
  writer.close()

  batch_size = 2
  # read_tfrecord(tfrecord_file) # 读取一个图
  train_batch, train_label_batch = read_tfrecord2(tfrecord_file, batch_size)

  print(train_batch.shape)
  print(train_label_batch)

  plt.figure()
  plt.imshow(train_batch[0,:,:,0])
  plt.show()

  plt.figure()
  plt.imshow(train_batch[0,:,:,1])
  plt.show()

  train_batch1 = train_batch[0,:,:,:]
  print(train_batch.shape)
  print(train_batch1.dtype)
  im = Image.fromarray(np.uint8(train_batch1)) #参考numpy和图片的互转:http://blog.csdn.net/zywvvd/article/details/72810360
  im.show()

if __name__ == '__main__':
  main()

以上这篇TensorFLow 不同大小图片的TFrecords存取实例就是小编分享给大家的全部内容了,希望能给大家一个参考,也希望大家多多支持三水点靠木。

Python 相关文章推荐
Python多线程编程(二):启动线程的两种方法
Apr 05 Python
在Python中使用__slots__方法的详细教程
Apr 28 Python
Python访问纯真IP数据库脚本分享
Jun 29 Python
python实现字典(dict)和字符串(string)的相互转换方法
Mar 01 Python
Python编程实现的简单Web服务器示例
Jun 22 Python
python sorted函数的小练习及解答
Sep 18 Python
关于Pytorch MaxUnpool2d中size操作方式
Jan 03 Python
django 前端页面如何实现显示前N条数据
Mar 16 Python
基于Python第三方插件实现西游记章节标注汉语拼音的方法
May 22 Python
python一些性能分析的技巧
Aug 30 Python
python如何获得list或numpy数组中最大元素对应的索引
Nov 16 Python
上帝为你开了一扇窗之Tkinter常用函数详解
Jun 02 Python
python各层级目录下import方法代码实例
Jan 20 #Python
Python 识别12306图片验证码物品的实现示例
Jan 20 #Python
如何基于python实现归一化处理
Jan 20 #Python
tensorflow入门:tfrecord 和tf.data.TFRecordDataset的使用
Jan 20 #Python
tensorflow入门:TFRecordDataset变长数据的batch读取详解
Jan 20 #Python
python如何通过pyqt5实现进度条
Jan 20 #Python
python super用法及原理详解
Jan 20 #Python
You might like
php curl模拟post请求小实例
2013/11/13 PHP
php in_array() 检查数组中是否存在某个值详解
2016/11/23 PHP
PHP中常用的三种设计模式详解【单例模式、工厂模式、观察者模式】
2019/06/14 PHP
JavaScript 异步调用框架 (Part 2 - 用例设计)
2009/08/03 Javascript
Javascript查询DBpedia小应用实例学习
2013/03/07 Javascript
jQuery控制TR显示隐藏的几种方法
2014/06/18 Javascript
jquery动态加载js/css文件方法(自写小函数)
2014/10/11 Javascript
BootStrap table删除指定行的注意事项(笔记整理)
2017/02/05 Javascript
node.js自动上传ftp的脚本分享
2018/06/16 Javascript
浅谈针对Vue相同路由不同参数的刷新问题
2018/09/29 Javascript
JavaScript寄生组合式继承原理与用法分析
2019/01/11 Javascript
微信小程序学习笔记之函数定义、页面渲染图文详解
2019/03/28 Javascript
js对象简介与基本用法示例
2020/03/13 Javascript
微信小程序实现选择地址省市区三级联动
2020/06/21 Javascript
Vue 数据响应式相关总结
2021/01/28 Vue.js
python sys模块sys.path使用方法示例
2013/12/04 Python
跟老齐学Python之玩转字符串(1)
2014/09/14 Python
使用Pyinstaller的最新踩坑实战记录
2017/11/08 Python
python中set()函数简介及实例解析
2018/01/09 Python
Django中的forms组件实例详解
2018/11/08 Python
python3实现斐波那契数列(4种方法)
2019/07/15 Python
解决jupyter notebook 前面书写后面内容消失的问题
2020/04/13 Python
python logging模块的使用
2020/09/07 Python
利用Python优雅的登录校园网
2020/10/21 Python
html5中svg canvas和图片之间相互转化思路代码
2014/01/24 HTML / CSS
canvas裁剪clip()函数的具体使用
2018/03/01 HTML / CSS
荷兰最大的多品牌男装连锁店:Adam Brandstore
2019/12/31 全球购物
什么是Smart Navigation?
2016/07/03 面试题
交通事故委托书范本
2014/09/28 职场文书
党的群众路线教育实践活动个人批评与自我批评
2014/10/16 职场文书
入党函调证明材料
2014/12/24 职场文书
2015年加油站工作总结
2015/05/13 职场文书
关于感恩的歌曲整理(8首)
2019/08/14 职场文书
深入理解以DEBUG方式线程的底层运行原理
2021/06/21 Java/Android
Python进程池与进程锁之语法学习
2022/04/11 Python
Shell中的单中括号和双中括号的用法详解
2022/12/24 Servers