Search code examples
javacachingherokucloudcloud-hosting

Heroku, Cedar stack: What requests take up dyno time

I have given some thought on how to calculate how many users I can handle with Heroku and one dyno. But to figure it out I need some input.

And I must say that the official documentation isn't nice to navigate and interpreter so I haven't read it all. My complaints about it are that it doesn't describe things very well. Sometimes it describes old stacks, sometimes it's ruby specific, sometime it isn't described at all and so on.

So I need some input on how Heroku, Cedar stack, works regarding requests to make my calculations.

You are more than welcome to correct me on my assumptions as I am relatively new to dyno theory.

Lets say I have a controller that takes a request and calculate a JSON response in 10ms locally will I be able to serve 100request a second?

As I understand the cedar stack doesn't have a fronting caching solution, many questions arises.

  1. Does static content requests take up dyno time?
  2. Does transfer time count to request time.
  3. Can one dyno solution transfer many response to a request at the same time if the request requires small CPU utilization.

Some of the question is intertwined so a combined answer or other thought is valued.

An example:

Static HTML page.

<HTML>...<img><css><script>...
AjaxCall //dyno processing time 10ms
AjaxCall //dyno processing time 10ms
AjaxCall //dyno processing time 10ms
AjaxCall //dyno processing time 10ms
...</HTML>

Can I serve (1000ms / (10ms x 4)) = 25HTML pages a second?

  • This assumes that static content isn't provided by a dyno.
  • This assumes that transfer time isn't blamed on the dyno.

If this isn't the case I would be a catastrophe. Lets say a mobile phone in Africa makes 10 request and have a 10sec transfer time then my App will be unavailable for over 1½ minute.

Solution

  • I can only really answer the first question: Static assets most certainly do take up dyno time. In fact, I think it's best to keep all static assets, including stylesheets and JS on an asset server when using heroku's free package. (If everyone did that, heroku would benefit and so would you). I recommend using the asset_sync gem to handle that. The Readme does explain that there are one or two, easily resolved, current issues.

    Regarding your last point, sorry if I'm misinterpreting here, but a user in south africa might take 10 seconds to have their request routed to Heroku, but most of that time is probably spent trafficking around the maze of telephone exchanges between SA and USA. Your dyno is only tied up for the portion of the request that takes place inside Heroku's servers, not the 9.9 seconds your request spent getting there. So effectively Heroku is oblivious to whether your request is coming from South Africa or Sweden.

    There are all sorts of things you can do to speed your app up: Caching, more dynos, Unicorn with several workers