Many programming language runtime environments use stacks as their primary storage structure (e.g. see JVM bytecode to runtime example).
Quickly recalling I see following advantages:
Is the list complete or did I miss something? Are there programming language runtime environments which are not using stacks for storage at all?
Just decided to include a link to one of the most insightful developers alive (and active), the architect of hotspot JVM:
When you compare the speed & power of a CPU executing bytecodes, you'll see lots of hardware complexity around the basic execution issues (I'm skipping on lots of obvious examples, but here's one: the stack layout sucks for wide-issue because of direct stack dependencies).
http://www.azulsystems.com/blog/cliff-click/2010-04-21-un-bear-able