mxnet ndarray indexing in python

Question

I am new to mxnet. I just installed mxnet 1.0.0 and python 3.5 on a Ubuntu 14.04 machine with CUDA 8.0 and cudnn 7.0.5.

My code is given below. I am trying to store image data in an ndarray. (see https://github.com/ypwhs/DogBreed_gluon/blob/master/get_features_v3.ipynb for the original code) -

X_224 = nd.zeros((n, 3, 224, 224))
X_299 = nd.zeros((n, 3, 299, 299))

mean = np.array([0.485, 0.456, 0.406])
std = np.array([0.229, 0.224, 0.225])

for i, (fname, breed) in tqdm(df.iterrows(), total=n):
    img = cv2.imread('data/train/%s.jpg' % fname)
    img_224 = ((cv2.resize(img, (224, 224))[:, :, ::-1] / 255.0 - mean) / std).transpose((2, 0, 1))
    img_299 = ((cv2.resize(img, (299, 299))[:, :, ::-1] / 255.0 - mean) / std).transpose((2, 0, 1))

    X_224[i] = nd.array(img_224) <-- I get error in this line
    X_299[i] = nd.array(img_299)

Here is the error I get:

ValueError: Indexing NDArray with index=0 and type=class 'numpy.int64' is not supported.

I am assuming it has to with indexing a multi dimensional nd array. So I tried slicing - X_224[i:i+1] = .... but that gave me another error.

Answer 1

You could convert the type of the index from numpy.int64 to int; e.g. i = int(i) before trying to set the slice.

df.iterrows() returns tuples, where the type of the first element depends on the type of the dataframe index. df.iterrows() returned tuples of types (int,pandas.core.series.Series) when running the Github example, so no conversion was necessary for me (using Pandas 0.22).

Aside from this specific issue, you might want to take a look at Gluon Datasets and DataLoaders for this task. mxnet.gluon.data.vision.datasets.ImageFolderDataset can be used for loading images, and it accepts an image transformation function through the transform argument.

def transform_fn(data, label):
    mean = np.array([0.485, 0.456, 0.406])
    std = np.array([0.229, 0.224, 0.225])
    data = ((cv2.resize(data, (224, 224))[:, :, ::-1] / 255.0 - mean) / std).transpose((2, 0, 1))
    return data, label

image_directory = os.path.join(data_dir, "train")
dataset = mx.gluon.data.vision.ImageFolderDataset(image_directory, transform=transform_fn)
data_loader = mx.gluon.data.DataLoader(dataset, batch_size=10, shuffle=True)

for data, label in data_loader:
    ...