Control the logging frequency and contents when using wandb with HuggingFace

Question

I am using the wandb with my HuggingFace code. I would like to log the loss and other metrics. Now I have two questions

• How does wandb decide when to log the loss? Is this decided by logging_steps in TrainingArguments(...)？

training_args = TrainingArguments(output_dir="test", 
                                  learning_rate=lr,
                                  num_train_epochs=n_epoch,
                                  seed=seed,
                                  per_device_train_batch_size=2,
                                  per_device_eval_batch_size=2,
                                  logging_strategy="steps",
                                  logging_steps=5,
                                  report_to="wandb")

• How do I make sure wandb log other metrics (for example, adding validation metrics after each epoch)? Does this happen automatically?

Answer 1

Correct, it is dictated by the on_log event from the Trainer, you can see it here in WandbCallback

Your validation metrics should be logged to W&B automatically every time you validate. How often Trainer does evaluation depends on what setting is used for evaluation_strategy (and potentially eval_steps if evaluation_strategy == "steps")