将处理灰度图像的 keras 模型应用于 RGB 图像

数据挖掘 Python 喀拉斯 张量流
2021-10-14 09:55:18

使用Fashion MNIST数据集遵循了这个基本分类 TensorFlow 教程训练集包含 60,000 张 28x28 像素的灰度图像,分为 10 个类别(裤子、套头衫、鞋子等)。本教程使用一个简单的模型:

model = keras.Sequential([
    keras.layers.Flatten(input_shape=(28, 28)),
    keras.layers.Dense(128, activation='relu'),
    keras.layers.Dense(10)
])

该模型在 10 个 epoch 后达到 91% 的准确率。

我现在正在使用另一个名为CIFAR-10 的数据集进行练习,该数据集由 50,000 个 32*32 像素的 RGB 图像组成,也分为 10 个类别(青蛙、马、船等)。

考虑到 Fashion MNIST 和 CIFAR-10 数据集在图像数量和图像大小方面非常相似,并且它们具有相同数量的类,我天真地尝试训练类似的模型,只需调整输入形状:

  model = keras.Sequential([
     keras.layers.Flatten(input_shape=(32, 32, 3)),
     keras.layers.Dense(128, activation='relu'),
     keras.layers.Dense(10)
  ])

唉,经过 10 个 epoch,模型的准确率达到了 45%。我究竟做错了什么?

我知道我在 RGB 图像中的样本数量是灰度图像中的三倍,所以我尝试增加 epoch 的数量以及中间密集层的大小,但无济于事。


以下是我的完整代码:

import tensorflow as tf
import IPython.display as display
from PIL import Image
from tensorflow import keras
import numpy as np
import matplotlib.pyplot as plt
import pdb
import pathlib
import os
from tensorflow.keras import layers #Needed to make the model
from tensorflow.keras import datasets, layers, models

(train_images, train_labels), (test_images, test_labels) = datasets.cifar10.load_data()

IMG_HEIGHT = 32
IMG_WIDTH = 32

class_names = ['airplane', 'automobile', 'bird', 'cat', 'deer',
               'dog', 'frog', 'horse', 'ship', 'truck']


train_images = train_images / 255.0
test_images = test_images / 255.0

def make_model():
      model = keras.Sequential([
         keras.layers.Flatten(input_shape=(IMG_HEIGHT, IMG_WIDTH, 3)),
         keras.layers.Dense(512, activation='relu'),
         keras.layers.Dense(10)
      ])
      model.compile(optimizer='adam',
                   loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),
                   metrics=['accuracy'])
      return model

model=make_model()
history = model.fit(train_images, train_labels, epochs=10)
4个回答

您的模型不够复杂,无法对 CIFAR 10 数据集进行充分分类。CIFAR-10 比 Fashion-MNIST 数据集复杂得多,因此您需要一个更复杂的模型。您可以在模型中添加更多隐藏层来实现这一点。您还应该添加 DROPOUT 层以防止过度拟合。也许最简单的解决方案是使用迁移学习。如果您想尝试迁移学习,我建议您使用 MobileNet CNN 模型。可以在此处找到相关文档。由于 CIFAR-10 有 50,000 张样本图像,我认为您不需要数据增强。首先尝试一个没有增强的更复杂的模型,看看你能达到什么准确度。如果还不够,请使用 keras ImageData Generator 来提供数据增强。文档在这里.

我正在使用这个模型(基本上建立在Chollet的工作之上)。它使用预训练模型 (VGG16) 来解决多类图像识别问题。

from keras.applications import VGG16
import os, datetime
import numpy as np
from keras.preprocessing.image import ImageDataGenerator
from keras.utils import to_categorical
from keras import models, layers, optimizers, regularizers
from keras.callbacks import EarlyStopping
from keras.callbacks import ReduceLROnPlateau
from keras.layers.core import Dense, Dropout, Activation
from keras.layers.normalization import BatchNormalization
from PIL import ImageFile
import statistics
ImageFile.LOAD_TRUNCATED_IMAGES = True

###############################################
# DIR with training images
base_dir = 'C:/pathtoimages'
# Number training images
ntrain = 2000
# Number validation images
nval  = 500
# Batch size
batch_size = 20 #20
# Epochs (fine tuning [100])
ep = 400 #400
# Epochs (first step [30])
ep_first = 30 
# Number of classes (for training, output layer)
nclasses = 30
###############################################
start = datetime.datetime.now()

conv_base = VGG16(weights='imagenet', include_top=False, input_shape=(150, 150, 3))
train_dir = os.path.join(base_dir, 'train')
validation_dir = os.path.join(base_dir, 'val')
#test_dir = os.path.join(base_dir, 'test')

datagen = ImageDataGenerator(rescale=1./255)

def extract_features(directory, sample_count):
    features = np.zeros(shape=(sample_count, 4, 4, 512))
    labels = np.zeros(shape=(sample_count))
    generator = datagen.flow_from_directory(
        directory,
        target_size=(150, 150),
        batch_size=batch_size,
        class_mode='binary')
    i = 0
    for inputs_batch, labels_batch in generator:
        features_batch = conv_base.predict(inputs_batch)
        features[i * batch_size : (i + 1) * batch_size] = features_batch
        labels[i * batch_size : (i + 1) * batch_size] = labels_batch
        i += 1
        if i * batch_size >= sample_count:
            break
    return features, labels

train_features, train_labels = extract_features(train_dir, ntrain)
validation_features, validation_labels = extract_features(validation_dir, nval)
#test_features, test_labels = extract_features(test_dir, 1000)

# Labels and features
train_labels = to_categorical(train_labels)
validation_labels = to_categorical(validation_labels)
#test_labels = to_categorical(test_labels)
train_features = np.reshape(train_features, (ntrain, 4 * 4 * 512))
validation_features = np.reshape(validation_features, (nval, 4 * 4 * 512))
#test_features = np.reshape(test_features, (1000, 4 * 4 * 512))

#######################################
# Model
model = models.Sequential()
model.add(conv_base)
model.add(layers.Flatten())
model.add(layers.Dense(4096, activation='relu',kernel_regularizer=regularizers.l2(0.003)))#0.002
model.add(BatchNormalization())

model.add(layers.Dense(2048, activation='relu',kernel_regularizer=regularizers.l2(0.003)))#0.002
model.add(layers.Dense(2048, activation='relu',kernel_regularizer=regularizers.l2(0.003)))#0.002
model.add(BatchNormalization())

model.add(layers.Dense(1024, activation='relu',kernel_regularizer=regularizers.l2(0.003)))#0.002
model.add(layers.Dense(1024, activation='relu',kernel_regularizer=regularizers.l2(0.003)))#0.002
model.add(BatchNormalization())

model.add(layers.Dense(512, activation='relu',kernel_regularizer=regularizers.l2(0.003)))#0.002
model.add(layers.Dense(512, activation='relu',kernel_regularizer=regularizers.l2(0.003)))#0.002
model.add(BatchNormalization())

model.add(layers.Dense(256, activation='relu',kernel_regularizer=regularizers.l2(0.003)))#0.002
model.add(layers.Dense(256, activation='relu',kernel_regularizer=regularizers.l2(0.003)))#0.002
model.add(BatchNormalization())

model.add(layers.Dense(128, activation='relu',kernel_regularizer=regularizers.l2(0.003)))#0.002
model.add(layers.Dense(128, activation='relu',kernel_regularizer=regularizers.l2(0.003)))#0.002
model.add(layers.Dense(128, activation='relu',kernel_regularizer=regularizers.l2(0.003)))#0.002
model.add(layers.Dense(128, activation='relu',kernel_regularizer=regularizers.l2(0.003)))#0.002

model.add(layers.Dense(nclasses, activation='softmax'))
conv_base.trainable = False

#######################################
# Data generators
train_datagen = ImageDataGenerator(
      rescale=1./255,
      rotation_range=40,
      width_shift_range=0.2,
      height_shift_range=0.2,
      shear_range=0.2,
      zoom_range=0.2,
      horizontal_flip=True,
      fill_mode='nearest')

# Note that the validation data should not be augmented!
test_datagen = ImageDataGenerator(rescale=1./255)

train_generator = train_datagen.flow_from_directory(
        # This is the target directory
        train_dir,
        # All images will be resized to 150x150
        target_size=(150, 150),
        batch_size=batch_size,
        # Since we use categorical_crossentropy loss, we need binary labels
        class_mode='categorical')

validation_generator = test_datagen.flow_from_directory(
        validation_dir,
        target_size=(150, 150),
        batch_size=batch_size,
        class_mode='categorical')

# Model compile / fit
model.compile(loss='categorical_crossentropy',
              optimizer=optimizers.RMSprop(lr=2e-5),
              metrics=['acc'])

# early stopping: https://keras.io/callbacks/#earlystopping
es = EarlyStopping(monitor='val_loss', mode='min', min_delta=0.001, verbose=1, patience=40, restore_best_weights=True)
reduce_lr = ReduceLROnPlateau(monitor='val_loss', mode='min', factor=0.9, patience=15, min_lr=1e-20, verbose=1, cooldown=3)

history = model.fit_generator(
      train_generator,
      steps_per_epoch=round((ntrain+nval)/batch_size,0),
      epochs=ep_first,
      validation_data=validation_generator,
      validation_steps=20, #50
      verbose=2,
      callbacks=[es, reduce_lr])

#######################################
# Fine tuning
conv_base.trainable = True

set_trainable = False
for layer in conv_base.layers:
    if layer.name == 'block5_conv1':
        set_trainable = True
    if set_trainable:
        layer.trainable = True
    else:
        layer.trainable = False

model.compile(loss='categorical_crossentropy',
              optimizer=optimizers.RMSprop(lr=0.00001), #1e-5
              metrics=['acc'])

history = model.fit_generator(
      train_generator,
      steps_per_epoch=round((ntrain+nval)/batch_size,0),
      epochs=ep,
      validation_data=validation_generator,
      validation_steps=20,
      callbacks=[es, reduce_lr])

#######################################
# Save model
model.save('C:/yourpath/yourmodel.hdf5')
end = datetime.datetime.now()
delta = str(end-start)

# Metrics
acc = history.history['acc']
acc = acc[-5:]
val_acc = history.history['val_acc']
val_acc = val_acc[-5:]
loss = history.history['loss']
loss = loss[-5:]
val_loss = history.history['val_loss']
val_loss = val_loss[-5:]

# End statement
print("============================================")
print("Time taken (h/m/s): %s" %delta[:7])
print("============================================")
print("Metrics (average last five steps)")
print("--------------------------------------------")
print("Loss       %.3f" %statistics.mean(loss))
print("Val. Loss  %.3f" %statistics.mean(val_loss))
print("--------------------------------------------")
print("Acc.       %.3f" %statistics.mean(acc))
print("Val. Acc.  %.3f" %statistics.mean(val_acc))
print("============================================")
print("Epochs:    %s / %s" %(ep,ep_first))

想到两件事:

您可以添加数据生成器。这将通过引入一系列小的变化(即随机旋转、缩放、剪切、水平/垂直移动......)从您当前的图像中生成新图像,从而迫使模型学习不同类别图像之间的重要区别特征。

您还可以添加 dropout 层来对抗过度拟合。

这是一个很好的例子:https ://keras.io/examples/cifar10_cnn/

我认为您的模型不够复杂,无法从 CIFAR-10 数据集中学习。

您可以在此处找到使用不同模型和激活函数的 CIFAR-10 分类数据集结果

从结果来看,我可以看到您需要使用具有指数线性单元 (ELU) 的密集 CNN 模型来获得更好的准确度。