Using InceptionV3 for greyscale images

Question

I am trying to solve an image classification problem using transfer training. I have a dataset of spectrograms of EEG signals. I want to train the last few layers of InceptionV3 on this dataset. However, InceptionV3 only takes images with three layers but I want to train it on greyscale images as the color of the image doesn't have anything to do with the classification in this particular problem and is increasing computational complexity. I have attached my code below

ROWS,COLS = 669,1026


input_shape = (ROWS, COLS, 3)

base_model = applications.Xception(weights='imagenet', 
                                include_top=False, 
                                input_shape=(ROWS, COLS,3))

l = 0
for layer in base_model.layers:
    layer.trainable = False
    l += 1

c = 0
for layer in base_model.layers:
    c += 1
    if c > l-3:
        layer.trainable = True 

# for layer in base_model.layers:
#     print(layer,layer.trainable)

# base_model.summary()

add_model = Sequential()
add_model.add(base_model)
add_model.add(GlobalAveragePooling2D())
# add_model.add(Dense(16, activation='tanh'))
# add_model.add(Dense(16, activation='tanh'))
add_model.add(Dense(8, activation='tanh'))
add_model.add(Dense(4, activation='tanh'))
add_model.add(Dense(1, activation='sigmoid'))

model = add_model
# model.compile(loss='binary_crossentropy', 
#               optimizer=optimizers.SGD(lr=1e-4, 
#                                        momentum=0.8),
#               metrics=['accuracy'])
model.compile(loss='binary_crossentropy', 
              optimizer='adam',
              metrics=['accuracy'])
model.summary()

import numpy as np
import os
from random import shuffle
from tqdm import tqdm
from PIL import Image, ImageOps
import pickle

IMG_WIDTH = 669
IMG_HEIGHT = 1026

CLASS_1_TRAIN = '/home/spectrograms/train/c1'
CLASS_3_TRAIN = '/home/spectrograms/train/c3'
CLASS_1_TEST = '/home/spectrograms/test/c1'
CLASS_3_TEST = '/home/spectrograms/test/c3'


def label_img(img):
    word_label = img[:5]
    if img[1] == '1':
      return [1,0]
    elif img[1] == '3':
      return [0,1]

def create_data(data,loc): #loads data into a list
    for img in tqdm(os.listdir(loc)):
        label = label_img(img)
        path = os.path.join(loc,img)
        img = Image.open(path) 
        img = ImageOps.grayscale(img) 
        # w,h = img.size
        # img = img.resize((w//3,h//3))
        data.append([np.array(img),np.array(label)])
    return data

def make_X_and_Y(set): #split data into numpy arrays of inputs and outputs
  set_X,set_Y = [],[]
  n = len(set)
  for i in range(n):
    set_X.append(set[i][0])
    set_Y.append(set[i][1])
  return np.array(set_X),np.array(set_Y)

data = []
data = create_data(data,CLASS_1_TRAIN)
data = create_data(data,CLASS_1_TEST)
data = create_data(data,CLASS_3_TRAIN)
data = create_data(data,CLASS_3_TEST)
data = np.array(data)

np.random.shuffle(data)

X_data,Y_data = make_X_and_Y(data)
print('X-Y splittling completed\n')
print('X_Data.shape = ', X_data.shape)
print('Y_Data.shape = ', Y_data.shape)

from sklearn.model_selection import train_test_split
X_train, X_validation, Y_train, Y_validation = train_test_split(X_data, Y_data, test_size=0.3, random_state=42)

print('Test-train split completed\n')

X_train = X_train.reshape(X_train.shape[0], IMG_WIDTH, IMG_HEIGHT, 1)
X_validation = X_validation.reshape(X_validation.shape[0], IMG_WIDTH, IMG_HEIGHT, 1)
input_shape = (IMG_WIDTH, IMG_HEIGHT, 1)

    
X_train = X_train.astype('float32')
X_validation = X_validation.astype('float32')

X_train /= 255         
X_validation /= 255

#!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

history = model.fit(X_train, Y_train,
                    batch_size=3,
                    epochs=100,
                    verbose = 1,
                    validation_data=(X_validation, Y_validation))

model.save_weights('/home/CNN/saved_model/Inception_new.h5')

How can I change my code such that I can train InceptionV3 on images with single layers? If it is not possible, are there any other pre-trained models that can do this task?

Answer 1

Implementing Inceptionv3 from scratch is easy. So you can follow https://gist.github.com/neggert/f8b86d001a367aa7dde1ab6b587246b5 or any other inceptionv3 model which had been put as open source and the number of channels from 3 to 1. But note these models will not be pre-trained models. So if you have available HPC systems you can give this a try.

Or

Visit https://github.com/keras-team/keras-applications/blob/master/keras_applications/inception_v3.py where they have used weights of the pre-trained model and you can change the number of channels from 3 to 1 in the code. I am not sure about this one because the model was trained on 3 channel images so the weights might not be the ones that we want.