I am a bit confused about what value I should be expecting from Keras's evaluate function.
Here is the evaluate function definition from Keras documentation:
evaluate(self, x=None, y=None, batch_size=None,
verbose=1, sample_weight=None, steps=None)
And here is the short description from the same page:
Returns the loss value & metrics values for the model in test mode.
If I have a large cross validation dataset which kind of requires me to call the evaluate function several times, does the evaluate function remember previous calls? Or does it only return, say, the loss value for the given mini-batch each time?
The evaluate() method evaluates the model on the whole data you pass to it and therefore the given loss value and the metric(s) value(s) are based on the performance of the model on the whole data.
Further, there is another method called test_on_batch() which tests the model on a single batch of data and returns the corresponding loss value and metric(s) value(s) of the model on the given data batch.
However, I am not sure what you mean by saying "I have a large cross validation data set which kind of requires me to call the evaluate function several times..." (emphasize mine). Do you mean the whole validation data does not fit in memory? If that's the case, and you have stored the validation data in a file on the disk (say a h5py file), then you can define a generator and use evaluate_generator() method to perform evaluation using the generator you have defined.