theano - how to have many of the same function

Question

The input is a variable sized array. I can only process a given example one sample at a time in train_model. I want to accumulate the sum of objectives for the elements in the batch then apply regularization and gradient descent.

Currently, this is the training stage where updates are done for each element xi.

for epoch in range(n_epochs):
minibatch_avg_cost = 0
for xi in dataset.get_next_xi(batch_size):
  minibatch_avg_cost += train_model(xi)

  print(minibatch_avg_cost)

How can I get results from train_model(xi) for the number of elements in the batch and then do the updates?

Answer 1

just use all the elements in dataset.get_next_xi(batch_size) as input and create a theano function to calculate the average cost (instead of only one cost) and do the updates using the average cost. You can see the example code from here

they use theano function from train model like this:

train_model = theano.function(
    inputs=[index],
    outputs=cost,
    updates=updates,
    givens={
        x: train_set_x[index * batch_size: (index + 1) * batch_size],
        y: train_set_y[index * batch_size: (index + 1) * batch_size]
    }
)

with cost is average cost of datasets batch