The full error message is:
The Project 'my.project.name' is under source control. An error occurred registering this project with source control. It is recommended that you do not make any changes to this project.
It seems that I can make the error message go away, and load the project correctly by opening the csproj file, and remove all nodes with names starting with "scc" - however, then I have to confirm wanting to use the solution Source control settings every time I open the solution.
Any suggestions?
In case anyone else has the same problem: We ended up solving it taking a backup of the project, removing the project from the solution, checking the solution into source control without it, and then adding the project again.
Luckily it was a new small project, with very little history - I would hate having to do this with a large old project...
I would probably try tbergstedt's suggestion first, if I where you.