I am trying to understand how tf.nn.sparse_softmax_cross_entropy_with_logits works.
Description says:
A common use case is to have logits of shape [batch_size, num_classes]
and labels of shape [batch_size]. But higher dimensions are supported.
So it suggests that we can feed labels in raw form for example [1,2,3].
Now since all computations are done per batch I believe the following is possible:
In all cases we assume size of batch equal two.
Case 1 (with one batch): logit:
0.4 0.2 0.4
0.3 0.3 0.4
correspoding labels:
2
3
I am guessing labels might be coded as
[1 0 0]
[0 1 0]
Case 2 (with another batch): logit:
0.4 0.2 0.4
0.3 0.3 0.4
correspoding labels:
1
2
I am guessing labels might be coded as (I do not see what prevents us from this coding, unless tensorflow keeps track how it coded before)
[1 0 0]
[0 1 0]
So we have two different codings. Is it safe to assume that tensorflow keeps coding consistent from batch to batch?
There is no real coding happening. The labels are just the position of the 1 in the according one-hot vector:
0 -> [1, 0, 0]
1 -> [0, 1, 0]
2 -> [0, 0, 1]
This "coding" will be used in every batch.