why the `test process` is inside the epoch loop in deep learning?

Question

I am new to deep learning, here is the code I saw

The code is fine, but I don't understand the follow:

for epoch in range(1, args.epochs + 1):
    train(epoch)
    test(epoch)
    with torch.no_grad():
        sample = torch.randn(64, 20).to(device)
        sample = model.decode(sample).cpu()

In machine learning, when we finish training, we fix the model parameters for the test dataset. here are my two questions:

(1) In deep learning, we have the training, validation, test dataset. Is the code test(epoch) actually for validation set? There we fix model parameters and predict for test dataset outside the epoch loop?

(2) what is with torch.no_grad(): mean? and why it also inside the epoch loop?

Thanks a lot.

Answer 1

1. Yes, the test(epoch) is actually for validation here (Update: not exactly validation, check below answer).
2. with torch.no_grad() means that you're switching off the gradients (required for backpropagation during training). In validation/testing you don't need them, and it'll save memory and computations. Read more here.

Also, check the tutorial here.