CNTK has no attribute LeakyReLU

Question

While making a CNN model like this:

# function to build model

def create_model(features):
    with C.layers.default_options(init=C.glorot_uniform(), activation=C.LeakyReLU):
            h = features
            h = C.layers.Convolution2D(filter_shape=(5,5), 
                                       num_filters=8, 
                                       strides=(2,2), 
                                       pad=True, name='first_conv')(h)
            h = C.layers.Convolution2D(filter_shape=(5,5), 
                                       num_filters=16, 
                                       strides=(2,2), 
                                       pad=True, name='second_conv')(h)
            r = C.layers.Dense(num_output_classes, activation=None, name='classify')(h)
            return r

# Create the model
z = create_model(x)

# Print the output shapes / parameters of different components
print("Output Shape of the first convolution layer:", z.first_conv.shape)
print("Bias value of the last dense layer:", z.classify.b.value)

I am getting the error:

AttributeError Traceback (most recent call last) in () 1 # Create the model ----> 2 z = create_model(x) 3 4 # Print the output shapes / parameters of different components 5 print("Output Shape of the first convolution layer:", z.first_conv.shape)

in create_model(features) 2 3 def create_model(features): ----> 4 with C.layers.default_options(init=C.glorot_uniform(), activation=C.LeakyReLU): 5 h = features 6 h = C.layers.Convolution2D(filter_shape=(5,5),

AttributeError: module 'cntk' has no attribute 'LeakyReLU'

I am new to deep learning so I might be missing something simple. Any help is appreciated. Thanks!

Answer 1

Try C.leaky_relu:

>>> C.leaky_relu([[-1, -0.5, 0, 1, 2]]).eval()
array([[-0.01 , -0.005,  0.   ,  1.   ,  2.   ]], dtype=float32)