I am using basic asp.net web application project template, because I want to move away from MVC into SPA.
Most of my pages will just be basic html files that will interact with the server through ajax calls. That said I want to hide the .html extension, but I don't want to create controllers just to hide this, which is too much of an overkill.
Of another note, I am using Azure as well, so setting this up on IIS directly is not going to work, as I would not be able to scale the administration nicely.
So how can I hide the html extensions without such a heavy layer as an MVC controller?
This sounds like a job for Url Routing
Url Routing allows you to intercept a request and then determine how to service it. It is how MVC does it and has many other useful benefits. If the router isn't able to service it, it falls back to the default ASP.NET pipeline processing, and then IIS.