I am trying to use pybrain to output rgb values. The input layer takes an array of rgb values, and all hidden layers are linear models. I would have expected the network to output rgb values. However, the output of this network turns out to be an array of values that are no where near within the range of 0:255. The images are about 25 different .jpg images of a bull. Each image is a flattened array of length 575280. I was hoping the network would converge on an image that ends up resembling a bull.
import numpy as np
from pybrain.structure import FeedForwardNetwork, LinearLayer, SigmoidLayer, GaussianLayer, TanhLayer
from pybrain.structure import FullConnection, BiasUnit
import testabull
bull_x = 510
bull_y = 398
bull_flat = 575280
n = FeedForwardNetwork()
bias_unit = BiasUnit()
in_layer = LinearLayer(bull_flat)
hidden_A = LinearLayer(5)
hidden_B = LinearLayer(10)
out_layer = LinearLayer(bull_flat)
n.addInputModule(in_layer)
n.addModule(hidden_A)
n.addModule(hidden_B)
n.addOutputModule(out_layer)
n.addModule(bias_unit)
in_to_hidden = FullConnection(in_layer, hidden_A)
hidden_to_hidden = FullConnection(hidden_A, hidden_B)
hidden_to_out = FullConnection(hidden_B, out_layer)
bias_to_hidden = FullConnection(hidden_B, out_layer)
n.addConnection(in_to_hidden)
n.addConnection(hidden_to_hidden)
n.addConnection(bias_to_hidden)
n.addConnection(hidden_to_out)
n.sortModules()
bull_img_array = testabull.crop_the_bull_images('../../imgs/thebull/')
trainable_array = [] ## an array of flattened images
for im in bull_img_array:
flat_im = np.array(im).flatten()
trainable_array.append(flat_im)
print n
print n.activate(trainable_array[0])
output = None
for a in trainable_array:
output = n.activate(a)
print output, len(output)
If anyone has any tips I would be very greatful.
First off there are two issues here, one you need to scale your outputs between 0 and 255. You can do this with some transformation afterwards. By taking the max and min value, then transposing between 0 and 255.
On the other hand this network will likely not learn what you'd like it to, your hidden layers are using Linear Layers. This is not very useful, as the weights themselves form a linear transformation. You'll essentially end up with a linear function. ftp://ftp.sas.com/pub/neural/FAQ2.html#A_act
I would recommend using a SigmoidLayer for your hidden layers, this of course squashes the values between 0 and 1. You can correct this in the output layer by multiplying by 255. Either via a fixed layer or just transforming the values afterwards.