Ok, firstly I have set up SSH keys, I am not asked for a password, this is not another authentication question!
But I am still asked the question: 'login as' in a dialog box each time I wish to pull or push my code. How can I make this go away?
I suppose your SSH-style URL for pushing/pulling does not include the/your username.
Just add it to the URL: username@servername:reponame
For this go to TortoiseGit settings -> Git -> Remote, click on the remote, change the URL and click on "Add New/Save".