Search code examples
pythongoogle-cloud-platformkaggle

Push and run python script on Google Cloud for ML


I recently started to learn Python and ML using old Kaggle Airbus Ship Detection competition. I have write a code, which now contains two files: Decode.py and Train.py and some external modules, which are used for training like resnet.py, data_generator.py etc.

The first one is working fine on my computer, but for training I don't have enough resources (no good graphic card). I thought I can use Kaggle script cloud, but I can't use more than one file in the same time (I'd need to copy a lot of code from modules to main python file, which will start to be really unreadable in that editor).

I decided to take a look for a Google Cloud platform, but the amount of possibilities what can I do there is really overwhelming, I'm digging there for last few hours today and I can't find a place where I can just run my code.

Do you know which tutorials/steps do I need to take to:

  1. Push my repo to Google Cloud
  2. Copy dataset from Kaggle (or where's the point I need to upload it)
  3. Run python3 train.py -path "foo"
  4. Check the output

The amount of possibilities what can I do on Google Cloud is overwhelming me and I don't know where to start all at all.

From steps, I've already taken is create a payment profile, create a repo and push code to Google Source Repositories (I don't know if it's a good place - I think I accidentally created a new website) and uploaded zip file with train and test data (I still don't know how to unpack it, but it's in progress).

Any help would be appreciated


Solution

  • I found nice functionality described there: https://towardsdatascience.com/how-to-use-jupyter-on-a-google-cloud-vm-5ba1b473f4c2 It's a bit different comparing to Google Colab, but this quick tutorial shows where you can put notebook file from Google Colab.

    You need to change only the path from /content/ in Colab to /home/jupyter in Google Cloud