Multiple SSH keys from Windows client to multiple servers

A lot of people here seem to have the same issue, but with Github. I don't think that is my issue.

I am fairly new to this so I apologize if this is a duplicate question or if this is a stupid question.

I manage multiple ubuntu servers and I access them from my Windows computer. I am currently attempting to have ssh keys for all of the servers I have, but it seems like I cannot change the name of the RSA keys or directories. If I do that, the terminal asks me for a password.

Here's what I did:

In my server:

mkdir ~/.ssh && chmod 700 ~/.ssh

On my windows machine:

ssh-keygen -b 4096

Prompts me:

Enter file in which to save the key (C:\Users\Me/.ssh/id_rsa):

Which I enter:

C:\Users\Me\.ssh\server1\server1_rsa

scp C:\Users\me\.ssh\server1_rsa.pub server@serverip:~/.ssh/authorized_keys

(I want to have different directories or something to seperate the SSH keys)

Maybe something like this?

/.ssh
  /server1
    server1 pub
    -erver1 priv
  /server2
    server2 pub
    server1 priv

or maybe even

/.ssh
  server1 pub
  server1 priv
  server2 pub
  server2 priv

But this doesnt work. Any suggestions?

Solution

I am sorry if this answer is not making you happy, but I think it will solve your problem :). There is a post on stackoverflow, where you have to add the keys to your ssh-agent.

Kind regards

.ssh/config file for windows (git)