It seems that on 64-bit platforms it would be reasonable to have a 8-byte length prefix. If we can address more than 4Gb of mem why not allow, say, 5Gb strings? Is the answer just "by specification" or there is some interoperability/backwards compatibility reasons that I'm not aware of? Thanks.
The BSTR data type is the standard COM string data type. Changing the length prefix would make it impossible to safely move strings between processes of different bitness (or at least make it significantly more complex). Since COM is the only relevant cross-bitness interop infrastructure it is necessary to have BSTRs behave the same way for 32-bit processes and 64-bit processes.
It is a tradeoff, imposing a 'limit' of 2GB in exchange for hassle-free marshaling of strings between processes of different bitness.