I decided to start learning web development and for some reason I decided to start at the start🙄 and so I spend the whole day reading about network architecture, different models, layers of TCP/IP and stuff. Came across something called REST APIs. Now, upon reading I understand what REST APIs are on a fundamental level. But one question that I can't seem to answer is so then are all, essentially, web services, RESTful, or are there other styles of architecture too to build a web app?
Does everything on the World Wide Web use REST?
Sort of.
REST is an architectural style, the "coordinated set of architectural constraints" used in the development of the HTTP 1.1 standard in the 1990s. Notice that Roy Fielding, who describes RESTin the dissertation he published in 2000, is also the first author listed in RFC 2616 (and more recently, RFC 7230).
In chapter six of his dissertation, Fielding observes:
Like most real-world systems, not all components of the deployed Web architecture obey every constraint present in its architectural design.
The tracking of client application state is a common example; the web, as it is today, doesn't offer us a common, general purpose mechanism for tracking state in our clients. So instead, web designers resort to tracking that state on the server, either by using resources specific to a user (see REST mismatches in URI) or by using cookies as a hidden mechanism for tracking history on the server (see REST mismatches in HTTP).
In other words, the web we use every day isn't a perfect expression of the REST architectural style.
Further more, you have ideas like SOAP or GraphQL, where important semantic information about a message is not expressed in a standardized form, which means that our general purpose components cannot assess the semantics of the messages and take intelligent actions.
are all, essentially, web services, RESTful
No - not unless you limit yourself to a definition of "web services" that excludes implementations that violate REST.
Finding web implementations that are architecturally aligned with remote procedure calls, rather than REST, is common. See Fielding 2008. Special note: read the comments, which contain a lot of useful Q and A.