keras custom layer to load data

Question

I am following this tutorial to use custom layers for pre-processing.

def pre_process(file_path):
    # loading file from disk and transforming into [90,13,1] 

class PreProcessBlock(layers.Layer):
    def __init__(self):
        super(PreProcessBlock,self).__init__()
    
    def call(self, inputs):
        return pre_process(inputs.numpy())
    
    def compute_output_shape(self, input_shape):
        return input_shape

preprocess = tf.keras.Sequential([
            PreProcessBlock()
])

model = keras.Sequential(
     [
      preprocess,
      
      layers.Dense(256, activation = "relu"),
      layers.Dropout(.5), 
      layers.Dense(len(LABELS))]

I am creating my dataset as

files = ['file1,'file2`]
labels = [0,1]

def get_data_set(files, labels, is_training=False):
    dataset = tf.data.Dataset.from_tensor_slices((files, labels))

    if is_training:
        dataset = dataset.shuffle(SHUFFLE_BUFFER_SIZE, reshuffle_each_iteration = True)
    dataset = dataset.batch(BATCH_SIZE)
    dataset = dataset.prefetch(AUTOTUNE)
    return dataset

train_dataset = get_data_set(files, labels, is_training=True)
val_dataset = get_data_set(files, labels)

Model fitting fails with error

model.fit(train_dataset, epochs=1, verbose=1,validation_data=val_dataset)

Error

AttributeError: in user code:

    /opt/conda/lib/python3.7/site-packages/tensorflow/python/keras/engine/training.py:806 train_function  *
        return step_function(self, iterator)
    <ipython-input-158-3f6d9dd39f2f>:6 call  *
        return pre_process(inputs.numpy())

    AttributeError: 'Tensor' object has no attribute 'numpy'

My question

Is this a valid way of implementing the model pipeline?

Answer 1

The structure of your layers and organizing all preprocessing layers into one sequential layer is great. You should not load any learning examples in layers (it is not part of model and it makes it less portable).

Two issues:

You don't have numpy() method because of this. I recommend sticking to static graphs and do not try to convert anything to numpy in your keras graph, unless it is absolutely necessary - performance issue. Most operations on tensors can be done using tf.

Your custom preprocessing layer should inherit from tensorflow.keras.layers.experimental.preprocessing.PreprocessingLayer (all layers from tf.keras.layers.experimental.preprocessing inherit from it directly or via CombinerPreprocessingLayer from the same package). There isn't much going on in the source code of PreprocessingLayer class, but all of it is important:

1. PreprocessingLayer provides interface for adapt method:
   adapt(self, data, reset_state=True). Please see "pure" keras docs why and when we need this.

2. PreprocessingLayer class have the flag _must_restore_from_config = True which from the Layer documentation we read:

When loading from a SavedModel, Layers typically can be revived into a generic Layer wrapper. Sometimes, however, layers may implement methods that go beyond this wrapper, as in the case of PreprocessingLayers' adapt method. When this is the case, layer implementers can override must_restore_from_config to return True; layers with this property must be restored into their actual objects (and will fail if the object is not available to the restoration code).

Let's take for example Resizing layer code (comment were ommited for readability):

class Resizing(PreprocessingLayer):
  def __init__(self,
               height,
               width,
               interpolation='bilinear',
               name=None,
               **kwargs):
    self.target_height = height
    self.target_width = width
    self.interpolation = interpolation
    self._interpolation_method = get_interpolation(interpolation)
    self.input_spec = InputSpec(ndim=4)
    super(Resizing, self).__init__(name=name, **kwargs)
    base_preprocessing_layer._kpl_gauge.get_cell('V2').set('Resizing')

  def call(self, inputs):
    outputs = image_ops.resize_images_v2(
        images=inputs,
        size=[self.target_height, self.target_width],
        method=self._interpolation_method)
    return outputs

  def compute_output_shape(self, input_shape):
    input_shape = tensor_shape.TensorShape(input_shape).as_list()
    return tensor_shape.TensorShape(
        [input_shape[0], self.target_height, self.target_width, input_shape[3]])

  def get_config(self):
    config = {
        'height': self.target_height,
        'width': self.target_width,
        'interpolation': self.interpolation,
    }
    base_config = super(Resizing, self).get_config()
    return dict(list(base_config.items()) + list(config.items()))

This is a pretty generic layer. It can resize images to some width and height. However, the point of preprocessing layers is to have entire end-to-end pipeline saved in one model. So in your pipline you would have specific width and height and you do not want to be bothered with instantiating the layer with proper arguments when doing inference- it should be the same as in training (applies to any preprocessing method, really). So in the get_config() method, apart from basic config , both height and width is saved and it can be easily read when resoring model later on. Please note that this layer does not override adapt method as it is invariant to data.