here is part of my theano code for dl4mt (neural machine translation) with line number. The src_positions is a vector of int64, which I printed the result, with the value no more than 16. but when I use src_positions to index attention_mask_, of which the shape is (100, 100). it got index out of bound error.
Here comes the weird part:
I'm not sure whether the problem is ... It's really really weird. hope someone can give me some advice.
1] p_t_s = p_t * sntlens # n_samples * 1, pt in equation
2] src_positions = tensor.cast(tensor.floor(p_t_s), 'int64') # (n_samples, 1)
3] src_positions = src_positions.reshape([src_positions.shape[0], ])
4] # batch_mask = attention_mask_[tensor.cast(src_positions * 0.1, 'int64')] # n_sample * maxlen
5] batch_mask = attention_mask_[src_positions] # n_sample * maxlen
6] attn_mask = batch_mask[:, :msk_.shape[0]] * msk_.T # n_sample * n_timestep
7] # batch_gauss_mask = gaussian_mask_[tensor.cast(src_positions * 0.1, 'int64')] # n_sample * maxlen
8] batch_gauss_mask = gaussian_mask_[src_positions] # n_sample * maxlen
9] gauss_mask = batch_gauss_mask[:, :msk_.shape[0]] * msk_.T # n_sample * n_timestep
It seems that the problem occurs based on the src_positions. There won't be any problem sccording to your description. Maybe the src_positions is changed by your codes other than your posted part