I've used asdf dozens of times in similar environments, and I'm not expecting any problems. I want the installs of ruby, python, node, etc. to happen faster by not running the built-in tests. I've tried this manually (without asdf) and there is an order of magnitude difference in the speed.
If I have any issues I promise to run it again with tests before asking for support!
This is up to each individual asdf plugin you are using. A plugin may or may not run tests for the software it compiles or installs. If you have a particular plugin that is slow because it runs automated tests, please open an issue on that plugin's issue tracker.
I know the asdf-nodejs plugin is fairly fast and does not run tests on install. The asdf-ruby plugin is slow because it compiles from source, but I don't believe it runs tests either.