Sampling data batch wise from tensor Pytorch

Question

I have train_x and valid_x splited from trainX ,train_y and valid_y splited from trainY and they are having shapes as per below. i want to classify images of labels LABELS = set(["Faces", "Leopards", "Motorbikes", "airplanes"]).

print(train_x.shape, len(train_y))
torch.Size([1339, 96, 96, 3]) 1339

print(valid_x.shape, len(valid_y))
torch.Size([335, 96, 96, 3]) 335

print(testX.shape, len(testY))
torch.Size([559, 96, 96, 3]) 559 

so i want to use regular train/valid on data batch-wise code as per below :

#train the network

n_epochs = 20

valid_loss = []
train_loss = []

for epoch in range(1,n_epochs+1):
    
    cur_train_loss = 0.0
    cur_valid_loss = 0.0
    
    #####################
    #### Train model ####
    #####################
    cnn_model.train()
    
    for data, target in trainLoader:
        if train_on_gpu:
            data, target = data.cuda(), target.cuda()
        optimizer.zero_grad()        
        output = cnn_model(data)
        loss = criterion(output, target)
        loss.backward()
        optimizer.step()
        cur_train_loss += loss.item() * data.size(0)
        
    ########################
    #### Validate model ####
    ########################
    
    cnn_model.eval()
    
    for data, target in validLoader:
        if train_on_gpu:
            data, target = data.cuda(), target.cuda()
        output = cnn_model(data)
        loss = criterion(output, target)
        cur_valid_loss += loss.item() * data.size(0)
    
    # calculate avg loss
    avg_train_loss = cur_train_loss / len(trainLoader.sampler)
    avg_valid_loss = cur_valid_loss / len(validLoader.sampler)
    
    train_loss.append(avg_train_loss)
    valid_loss.append(avg_valid_loss)
    
    print('Epoch: {} \t train_loss: {:.6f} \t valid_loss: {:.6f}'.format(epoch, avg_train_loss, avg_valid_loss))

so what i have to do for that ? i have search for that but nothing specific i found out. i want to use pytorch for this. i have built model for another problem same like this but in that i have used DataLoader for loading one batch of data at a time for training and validation.

Answer 1

You can create a dataset with torch.utils.data.TensorDataset, where each sample of train_x is associated with its corresponding label in train_y, such that the DataLoader can create batches as you are used to.

from torch.utils.data import DataLoader, TensorDataset

train_dataset = TensorDataset(train_x, train_y)
train_dataloader = DataLoader(train_dataset, batch_size=BATCH_SIZE, shuffle=True)

valid_dataset = TensorDataset(valid_x, valid_y)
valid_dataloader = DataLoader(valid_dataset, batch_size=BATCH_SIZE, shuffle=False)

test_dataset = TensorDataset(testX, testY)
test_dataloader = DataLoader(test_dataset, batch_size=BATCH_SIZE, shuffle=False)