I am trying to implement functionality similar to MarkLogic REST API's /transactions endpoint in a custom API. However, it looks like only some of the functions needed achieve this are available in XQuery.
xdmp:transaction-commit() and xdmp:transaction-rollback() are provided to remotely manage existing open transactions, but there are no corresponding XQuery functions to open a new transaction or to attach new statements to it.
I snooped around in the REST API implementation to see how it's done there, and they use the undocumented function xdmp:transaction-create() with the deprecated option <transaction-mode>update</transaction-mode> (I tested with the <update> and <commit> options, which segfaults the server!).
Statements are then attached to a transaction in the XML rewriter via <set-transaction>, which doesn't correspond to an XQuery function, but I assume it's the same thing as using eval or invoke with the<transaction-id> option (also deprecated).
A simple proof of concept using these methods seems to work fine, but I'm not sure if I'm missing a way to do this without using undocumented and deprecated features. Maybe it's okay because that's how REST API does it? It just seems odd to provide functions to commit and roll back remote transactions for which there is no approved way to create or update.
Yes, unfortunately the only way to achieve that yourself and create your own API is by using undocumented functions.
Take a look at Multi-statement Transactions in XQRS with transaction.xq, this may give you a better idea still.
As for setting whether individual statements are read-only or updating, you can use the %xdmp:update annotation.