In trying to understand the HSTS mechanism, I could not wrap my head around the max-age directive. Couldn't the presence/absence of the HSTS header be enough to tell the browser to switch to HTTP or HTTPS ?
Browsers could remember "forever" a site should be contacted through HTTPS upon first contact, until the header dissapears in a later response. Plus the preload directive is there to support browsers too.
I could not find anything in the specs explaining this. https://datatracker.ietf.org/doc/html/rfc6797
I feel like I'm missing something like a specific scenario. This is not a critic, I'd like to understand why this directive is necessary.
It allows it to be rolled out gradually. It is recommended to set it with a very small max-age first and then grow it if there are no issues. This avoids a real risk of DoS-ing yourself for any non-HTTPS sites. While that is becoming rarer, when this first came out that was a real risk as HTTP was still very much the norm.
Say for example you deployed it on https://www.example.com and that web server also responds to (and sets the HSTS header on) https://example.com. Now let’s say you haven’t set up HTTPS on http://blog.example.com (it’s an unimportant static only domain) or on http://intranet.example.com (it’s not Internet-facing). Without a max-age you potentially just blocked those sites forever until you can deploy HTTPS to them (which can be trickier than just adding a bit of server config).
And without being able to visit the site the browser also couldn’t see the header had since been removed for the reset you suggest. Plus there’s also the fact that not every resource needs to set the HSTS header - just one is sufficient (though best practice is to set it on every HTTPS resource and including redirects), so the absence of the header is not sufficient to reset it. You explicitly need to set max-age=0.
Of course, nowadays, the recommended approach is HTTPS on all subdomains (and this is pretty much becoming the norm as the public Internet is much more HTTPS now than it was - though still not yet the default) and also on intranet sites (though difficult to be sure if that latter really is the norm across companies large and small).
You are right and this could have been implemented by having max-age as optional (instead of mandatory as it is now) and site owners could remove it from the HSTS header, once ready to roll it out fully, but having a default max-age of infinity is pretty dangerous - for the same reasons as given above. Having no defaults and making the implementor think about it, hopefully, makes them consider the appropriate one, or at very least makes them realise the commitment they are making it.
Preloading is the way to make it permanent, at which point the max-age attribute is redundant for those user agent’s implementing preload lists (primarily the most popular browsers).
There is the argument that nothing is permanent in this world - domains come and go and are taken on by new parties who may or may not want to use HTTPS (at least initially) - though as I say with HTTPS becoming the norm that’s less of an issue.
Also clogging up browser cache with an infinite policy just cause they visited your website once, years ago, seems kinda rude. Though browsers could cap it (which they do - more on which later) but better to be explicit for the value you want.
Both of the above reasons btw, are reasons I don’t particularly like preloading HSTS either.
It’s also worth noting that browsers often implement a cap on the max-age (usually because it is stored as a 32-bit integer for example) so sticking an arbitrarily large value in there is not going to do what you think it will. In fact I recall discussion that one browser (Firefox?) didn’t do bounds checking so setting a value larger than that actually possible overflowed and so prevented the policy being set at all! As I said previously, preload is the way to go to make it permanent.
A max-age of six months or one year is the recommended best practice and after this there are diminishing returns in terms of security of a larger policy anyway. If a visitor is not visiting your site every 6-12 months (at which point they will refresh the HSTS policy for another max-age seconds) then chances are they’re on a new browser or device anyway.