Search code examples
pythonamazon-s3xgboost

Load XGBoost from an s3 bucket

I have an XGBoost model sitting in an AWS s3 bucket which I want to load. currently, I'm attempting to use s3fs to load the data, but I keep getting type errors:

from s3fs.core import S3FileSystem
import xgboost as xgb

fs = S3FileSystem()
bst = xgb.XGBClassifier()

with fs.open(f'{bucket}/{modelPath}', mode='rb') as f:
    bst.load_model(f.read())

I would include output, this I'm pretty sure this approach is completely wrong. How do I get this working?

(also, I think I would rather use boto)

Solution

  • Converting the content of the buffer to bytearray should do the trick. My inspiration come from the source code found here. It worked for me, without downloading locally the json file

    with fs.open(f'{bucket}/{modelPath}', mode='rb') as f:
    bst.load_model(bytearray(f.read()))