My residual neural network is giving a very strange depth map as output .I dont know how to improve my model?

Question

The output i am getting from my residual model is an image with small little squares on it ( a very low resolution image), but it is supposed to give me a depth map. The objects in the image are lost and only those small squares are visible. I don't how to improvise it ?

def mini_model(input_shape) :

# Define the input as a tensor with shape input_shape
X_input = Input(input_shape)

# Zero_Padding
X = ZeroPadding2D((3,3))(X_input)
#stage_1
X = Conv2D(64,(7,7),strides = (2,2),name = 'conv1')(X)
X = BatchNormalization(axis = 3,name = 'bn_conv1')(X)
X = Activation('relu')(X)
X = MaxPooling2D((3,3),strides = (2,2))(X)
# Stage 2
X = convolutional_block(X, f = 3, filters = [64, 64, 256], stage = 2, block='a', s = 1)
X = identity_block(X, 3, [64, 64, 256], stage=2, block='b')
X = identity_block(X, 3, [64, 64, 256], stage=2, block='c')

#stage3
X = convolutional_block(X,f = 3 , filters = [128,128,512],stage = 3,block = 'a', s = 2)
X = identity_block(X,3,[128,128,512],stage = 3,block='b')
X = identity_block(X,3,[128,128,512],stage = 3 , block = 'c')
X = identity_block(X,3,[128,128,512],stage = 3 , block = 'd')

#stage 4
X = convolutional_block(X,f = 3 , filters = [256,256,1024],stage = 4,block = 'a', s = 2)
X = identity_block(X,3,[256,256,1024],stage = 4,block='b')
X = identity_block(X,3,[256,256,1024],stage = 4,block='c')
X = identity_block(X,3,[256,256,1024],stage = 4,block='d')
X = identity_block(X,3,[256,256,1024],stage = 4,block='e')
X = identity_block(X,3,[256,256,1024],stage = 4,block='f')
X = identity_block(X,3,[256,256,1024],stage = 4,block='g')
X = identity_block(X,3,[256,256,1024],stage = 4,block='h')
X = identity_block(X,3,[256,256,1024],stage = 4,block='i')
X = identity_block(X,3,[256,256,1024],stage = 4,block='j')
X = identity_block(X,3,[256,256,1024],stage = 4,block='k')
X = identity_block(X,3,[256,256,1024],stage = 4,block='l')

#stage 5
X = convolutional_block(X,f = 3 , filters = [512,512,2048],stage = 5,block = 'a', s = 2)
X = identity_block(X,3,[512,512,2048],stage = 5,block='b')
X = identity_block(X,3,[512,512,2048],stage = 5,block='c')

# AVGPOOL

    X = Conv2D(3,kernel_size=(3,3), padding = 'same',use_bias = False)(X)
    X = UpSampling2D(size=2)(X)
    X = UpSampling2D(size=2)(X)
    X = UpSampling2D(size=2)(X)
    X = UpSampling2D(size=2)(X)
X = UpSampling2D(size=2)(X)

# Create model
model = Model(inputs = X_input, outputs = X)
return(model)

my residual model!! input image shape = (480,640,3)

Actual results : The image is made of small squares , with different levels of gray. Expected results : The image should be an depth map of the same size as input (480,640,3)

Answer 1

You have five upsampling layers in sequence. That's exactly what is expected from that. Big squares of 32 pixels. (2^5 = 32)

You should probably read about U-nets, create more convolutions between the upsamplings and connections from the resnet to the upsampling results.