How can I let my users to log into a phpBB forum by entering their user credentials while these credentials are stored in a separate MS SQL database?
You need to create a custom authentication plugin for phpbb. Since your requirement is authentication against a database, it should be pretty straight forward to re-use the existing plugin phpbb uses for authentication against it's own internal database.
Authentication plugins are located in <install_dir>/includes/auth
An authentication plugin must be named auth_<your custom name>.php
So in your case, create a copy of auth_db.php and call it e.g. auth_mssql.php
Authentication plugins are documented here: http://wiki.phpbb.com/Authentication_plugins