how to train a model in sagemaker via transformers library?

Question

Based on this example here, https://aws.amazon.com/blogs/machine-learning/fine-tune-and-host-hugging-face-bert-models-on-amazon-sagemaker/, for training a HuggingFace estimator is used and i am assuming , once the command to start training is issued ( see below) , the hugging face library , downloads the model , distilbert-base-cased in this case. and runs the train.py file. instead of using hugging face, can i set up the training in sagemaker with just transformers library?

huggingface_estimator = HuggingFace(entry_point='train.py',
                            source_dir='./scripts',
                            instance_type='ml.p3.2xlarge',
                            instance_count=1,
                            role=role,
                            transformers_version='4.6.1',
                            pytorch_version='1.7.1',
                            py_version='py36',
                            hyperparameters = hyperparameters)

start training command

/opt/conda/bin/python train.py --epochs 10 --model_name distilbert-base-cased --token_name distilbert-base-cased--train_batch_size 1024

I have run the example , provided in the link. but this uses hugging face library , how to set up the training without using hugging face library.

Answer 1

You can also run the training using Pytorch Estimator and can have full control on which library to use. Please refer the below example to use pytorch estimator and install any additional libraries using requirements.txt

https://github.com/aws/amazon-sagemaker-examples/blob/main/training/smart_sifting/Text_Classification_BERT/Train_text_classification.ipynb