Neural network works with the cross entropy and does not with an other loss function

Question

I'm using tensorflow with julia to create a neural network.

I can create a network with the cross_entropy loss function and it works:

ENV["CUDA_VISIBLE_DEVICES"] = "0" # It is to use the gpu
using TensorFlow
using Distributions


function weight_variable(shape)
   initial = map(Float32, rand(Normal(0, .001), shape...))
   return Variable(initial)
end

function bias_variable(shape)
   initial = fill(Float32(.1), shape...)
   return Variable(initial)
end


sess = Session(Graph())

num_pixels = 12

num_classes = 10

x = placeholder((Float32), shape=[nothing, num_pixels])
y = placeholder(Float32, shape=[nothing, num_classes])

poids = weight_variable([num_pixels,num_classes]) # Weight

biases = bias_variable([num_classes])


cross_entropy = reduce_mean(-reduce_sum(y.*log(nn.softmax(x*poids + biases)))) # Cross entropy Loss function

optimizer = train.AdamOptimizer(0.0001)


train_op = train.minimize(optimizer,cross_entropy)

correct_prediction = equal(indmax(nn.softmax(x*poids + biases), 2), indmax(y,2))

accuracy = reduce_mean(cast(correct_prediction, Float32))


y1 = [0 0 1 0 0 0 0 0 0 0] # correct label

x1 = [0 0 0 5 6 3 2 0 0 0 0 0] # Input 

run(sess, global_variables_initializer())

for i in 1:10

x_ = run(sess,train_op,Dict(x => x1, y => y1))

acc = run(sess,accuracy,Dict(x => x1, y => y1))

info("train $i , accuracy = $acc")

end

close(sess)

Now, If I just change my loss function with the exponential cost, like here:

ENV["CUDA_VISIBLE_DEVICES"] = "0" # It is to use the gpu
using TensorFlow
using Distributions


function weight_variable(shape)
   initial = map(Float32, rand(Normal(0, .001), shape...))
   return Variable(initial)
end

function bias_variable(shape)
   initial = fill(Float32(.1), shape...)
   return Variable(initial)
end


sess = Session(Graph())

num_pixels = 12

num_classes = 10

x = placeholder((Float32), shape=[nothing, num_pixels])
y = placeholder(Float32, shape=[nothing, num_classes])

poids = weight_variable([num_pixels,num_classes]) # Weight

biases = bias_variable([num_classes])

expo = reduce_mean((0.5*exp((1/0.5).*reduce_sum((nn.softmax(x*poids + biases)- y)^2)))) # Exponential loss function

optimizer = train.AdamOptimizer(0.0001)


train_op = train.minimize(optimizer,expo)

correct_prediction = equal(indmax(nn.softmax(x*poids + biases), 2), indmax(y,2))

accuracy = reduce_mean(cast(correct_prediction, Float32))


y1 = [0 0 1 0 0 0 0 0 0 0] # correct label

x1 = [0 0 0 5 6 3 2 0 0 0 0 0] # Input 

run(sess, global_variables_initializer())

for i in 1:10

x_ = run(sess,train_op,Dict(x => x1, y => y1))

acc = run(sess,accuracy,Dict(x => x1, y => y1))

info("train $i , accuracy = $acc")

end

close(sess)

It does not work and I have the following error:

ERROR: LoadError: Tensorflow error: Status: Node name 'gradients/Softmax_grad/Sum' already exists in the Graph

Stacktrace:
 [1] (::Atom.##110#114{String,String})() at /home/jabou/.julia/v0.6/Atom/src/eval.jl:104
 [2] withpath(::Atom.##110#114{String,String}, ::String) at /home/jabou/.julia/v0.6/CodeTools/src/utils.jl:30
 [3] withpath(::Function, ::String) at /home/jabou/.julia/v0.6/Atom/src/eval.jl:38
 [4] hideprompt(::Atom.##109#113{String,String}) at /home/jabou/.julia/v0.6/Atom/src/repl.jl:66
 [5] macro expansion at /home/jabou/.julia/v0.6/Atom/src/eval.jl:99 [inlined]
 [6] (::Atom.##108#112{Dict{String,Any}})() at ./task.jl:80
while loading /home/jabou/Bureau/Minimum nouveau.jl, in expression starting on line 37

I don't understand why... Can you help me ?

Thank you

Answer 1

A solution for the problem is here: TensorFlow, Julia // Node name already exists in the Graph

The version 1.4.0 of TensorFlow have to be used.