如何迭代数据集中的图像?

数据挖掘 喀拉斯 张量流 数据 数据清理 自动编码器
2022-03-05 16:27:11

我正在这个站点的帮助下构建一个自动编码器。在那里,我试图为我自己的自定义数据构建一个自动编码器。我的图像存储在 IMG 文件夹中,名称如下0.jpg, 1.jpg, 2.jpg.....

我试图开发一个迭代器来迭代我的所有图像,但问题出现了,当我将所有 124 个图像转换为单个training_data数组时,模型会响应它期望单个数组但给了它 124 个数组。谁能告诉我应该如何编写迭代器?我尝试使用flow_from_directory“机器学习掌握”网站上的 keras 功能,但它显示0 images from 0 classes.

这是我的代码:-->

import tensorflow as tf
from keras.layers import Input, Dense, Conv2D, MaxPooling2D, UpSampling2D
from keras.models import Model
from keras import backend as K
from keras.callbacks import TensorBoard
import numpy as np
from PIL import Image

i = int(0)
images_dir = "/home/awesome_ruler/Documents/Atom projects/Compression_enc/Images/IMG/{}.jpg".format(i)
training_data = []

while i < 125:
    print("working on ", i, 'file')
    image = Image.open(images_dir)
    pic_array = np.asarray(images_dir)
    training_data.append([pic_array])
    i += 1

input_img = Input(shape=(600, 400, 3))  # adapt this if using `channels_first` image data format

x = Conv2D(48, (3, 3), activation='relu', padding='same')(input_img)
x = MaxPooling2D((2, 2), padding='same')(x)
x = Conv2D(24, (3, 3), activation='relu', padding='same')(x)
x = MaxPooling2D((2, 2), padding='same')(x)
x = Conv2D(24, (3, 3), activation='relu', padding='same')(x)
encoded = MaxPooling2D((2, 2), padding='same')(x)

# at this point the representation is (4, 4, 8) i.e. 128-dimensional

x = Conv2D(24, (3, 3), activation='relu', padding='same')(encoded)
x = UpSampling2D((2, 2))(x)
x = Conv2D(24, (3, 3), activation='relu', padding='same')(x)
x = UpSampling2D((2, 2))(x)
x = Conv2D(48, (3, 3), activation='relu')(x)
x = UpSampling2D((2, 2))(x)
decoded = Conv2D(1, (3, 3), activation='sigmoid', padding='same')(x)

autoencoder = Model(input_img, decoded)
autoencoder.compile(optimizer='adadelta', loss='binary_crossentropy')

autoencoder.fit(training_data,
                epochs=50,
                batch_size=128,
                shuffle=True,
                callbacks=[TensorBoard(log_dir='/tmp/autoencoder')])

此外,我希望图像保留“颜色”功能,因此我使用输入形状,(600,400,3)因为 RGB 位于 3 个通道上。这是正确的吗?我会简单地使用我的迭代器,但我的理解是我需要一个不同的函数来与模型通信并一个接一个地为它提供图像,而我只是将它们全部加载到一个变量中。那么有人可以帮我解决这个问题吗?

这是完整的追溯:-

Traceback (most recent call last):
  File "autoencoder.py", line 46, in <module>
    callbacks=[TensorBoard(log_dir='/tmp/autoencoder')])
  File "/home/awesome_ruler/.local/lib/python3.7/site-packages/keras/engine/training.py", line 1154, in fit
    batch_size=batch_size)
  File "/home/awesome_ruler/.local/lib/python3.7/site-packages/keras/engine/training.py", line 579, in _standardize_user_data
    exception_prefix='input')
  File "/home/awesome_ruler/.local/lib/python3.7/site-packages/keras/engine/training_utils.py", line 109, in standardize_input_data
    str(len(data)) + ' arrays: ' + str(data)[:200] + '...')
ValueError: Error when checking model input: the list of Numpy arrays that you are passing to your model is not the size the model expected. Expected to see 1 array(s), but instead got the following list of 125 arrays: [array([['/home/awesome_ruler/Documents/Atom projects/Compression_enc/Images/IMG/0.jpg']],
      dtype='<U76'), array([['/home/awesome_ruler/Documents/Atom projects/Compression_enc/Images/IMG/0.jpg']]...
1个回答

images_dir实际上似乎是通向单个图像的路径...但是,

我会简单地创建一个带有形状的数字数组:(num_images, height, width, channels)通过执行以下操作:

import os
import numpy as np

# Root directory holding all images (I recommend removing the space in "Atom projects")
images_dir = "/home/awesome_ruler/Documents/Atom projects/Compression_enc/Images/IMG/"

# Number of images you want to load
N= 125

# Get all paths and take the first N
n_image_paths = sorted([f.path for f in os.scandir(data_root)])[:N]

# Load the images using on of the variants of loading images
images = np.array([Image.open(f) for f in n_image_paths])

# Be careful with variants: the order of channels is different for different methods!
# images = np.array([plt.imread(f) for f in n_image_paths])    # Matplotlib
# images = np.array([cv2.imread(f) for f in n_image_paths])    # OpenCV

现在图像可以直接传递给您的模型:

autoencoder.fit(x=images,
                epochs=50, ...)