Search code examples
nginxcloudgpufastapi

Hosting fastAPI on vast.ai GPU instance

How to allow http traffic on vast.ai instance? I'd like to host GPU related code using fastAPI+nginx, but I am not seeing NGINX homepage after configuration. I am not seeing bad gateway error. What I am getting is "This site can’t be reached".

After configuration, I wanted to see Nginx homepage. It works on AWS, but when setting the instance on AWS, you get the option of choosing "Allow http/https traffic". On vast.ai, I do not see that.

Solution

  • As False Genius mentioned, we can use a combination of FastAPI and ngrok.

    Solution:

    1. In the terminal, run your FastAPI app with the command: uvicorn main:app --host 0.0.0.0 --port 8000
    2. Next, create or open a Jupyter Notebook. Enter the following: !ngrok authtoken YOUR_AUTH_TOKEN and then !ngrok http 8000

    Then it should works :)