Saving the custom model

Question

I have a built a custom model and planned to store the model in \ S3 which suits my project.

We have 2 different ways to save a model

nlp.to_bytes()
nlp.to_disk()

nlp.to_disk() needs a path in the argument. So I choose to go with nlp.to_bytes(),

Code:

to_bytes_model = custom_nlp.to_bytes()
In: type(to_bytes_model) 
Out: bytes

s3 = boto3.resource('s3')    
s3.Bucket('customregex').upload_file('to_bytes_model','new_folder')

The above code for boto3 gives me error saying there is no file to_bytes_model

Do need help on saving the nlp model directly to S3. Thanks.

Answer 1

to_bytes_model is an in-memory object. It is not written to a file on disk. Using upload_file() which looks for a file on disk based on the name of the file you pass in. See the documentation here.

One way to do this is to use nlp.to_disk() followed by using some library to create zip/tar and then use upload_file() to upload the zip/tar. If you wish to skip the zip/tar you will need iterate over all the files in the folder to upload them one by one. I personally prefer option 1.