I'm looking a code from here for Python 3.5 + TensorFlow + TFLearn:
# -*- coding: utf-8 -*-
""" Convolutional Neural Network for MNIST dataset classification task.
References:
Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner. "Gradient-based
learning applied to document recognition." Proceedings of the IEEE,
86(11):2278-2324, November 1998.
Links:
[MNIST Dataset] http://yann.lecun.com/exdb/mnist/
"""
from __future__ import division, print_function, absolute_import
import tflearn
from tflearn.layers.core import input_data, dropout, fully_connected
from tflearn.layers.conv import conv_2d, max_pool_2d
from tflearn.layers.normalization import local_response_normalization
from tflearn.layers.estimator import regression
# Data loading and preprocessing
import tflearn.datasets.mnist as mnist
X, Y, testX, testY = mnist.load_data(one_hot=True)
X = X.reshape([-1, 28, 28, 1])
testX = testX.reshape([-1, 28, 28, 1])
# Building convolutional network
network = input_data(shape=[None, 28, 28, 1], name='input')
network = conv_2d(network, 32, 3, activation='relu', regularizer="L2")
network = max_pool_2d(network, 2)
network = local_response_normalization(network)
network = conv_2d(network, 64, 3, activation='relu', regularizer="L2")
network = max_pool_2d(network, 2)
network = local_response_normalization(network)
network = fully_connected(network, 128, activation='tanh')
network = dropout(network, 0.8)
network = fully_connected(network, 256, activation='tanh')
network = dropout(network, 0.8)
network = fully_connected(network, 10, activation='softmax')
network = regression(network, optimizer='adam', learning_rate=0.01,
loss='categorical_crossentropy', name='target')
# Training
model = tflearn.DNN(network, tensorboard_verbose=0)
model.fit({'input': X}, {'target': Y}, n_epoch=20,
validation_set=({'input': testX}, {'target': testY}),
snapshot_step=100, show_metric=True, run_id='convnet_mnist')
Ok, it works. But it uses only every 64th image from set while learning. Why does it happen? What should I do if I have small set and want network to use every first image?
Example of training message
Training Step: 1 | time: 2.416s
| Adam | epoch: 001 | loss: 0.00000 -- iter: 064/55000
Training Step: 2 | total loss: 0.24470 | time: 4.621s
| Adam | epoch: 001 | loss: 0.24470 -- iter: 128/55000
Training Step: 3 | total loss: 0.10852 | time: 6.876s
| Adam | epoch: 001 | loss: 0.10852 -- iter: 192/55000
Training Step: 4 | total loss: 0.20421 | time: 9.129s
| Adam | epoch: 001 | loss: 0.20421 -- iter: 256/55000
It is not only using every 64th image, it is loading batches of 64 image. That is why you see the iter increase by 64 each time, because it has processed 64 images per training step. Take a look at the documentation for the regression layer http://tflearn.org/layers/estimator/, here you can set the the batch size.