I know you can get all titles from Wikipedia's Database dump and that you can query for specific titles, but how can you get them all through the MediaWiki API?
For that you would use the Allpages API:
https://en.wikipedia.org/w/api.php?action=query&format=json&list=allpages&aplimit=max
You will not get all at once but up to 500 at a time and then a continuation marker which you can use for next batch of titles. Continue this until the JSON does not have query-continue
in the beginning of the answer.