I have about 6 different Azure websites all on the same subscription. When I look at the FTP deployment address they all have the same address (ftp://waws-prod-ml1-001.ftp.azurewebsites.windows.net) but different usernames/passwords. The problem is that at least using Windows explorer there appears to be NO way to access a second deployment once it's already remembered that I've connected to a previous one. I've tried editing the address to include the username (ftp://username:password@wasa....) but no luck, I've tried shutting down all explorer windows, clearing the credential cache etc. etc, but no dice - it still keeps connecting to whichever one I first authenticated with at least since I last logged into Windows. Does this basically mean I'll have to use a different FTP client that doesn't cache credentials like this? Bit surprised I can't find any forum posts suggesting other people haven't had this issue.
You can always right click in Windows Explorer and choose Login As option which will allow you to use a different set of credentials.
To understand how FTP credentials work for Azure App Service, check out Azure same FTP url for all azure websites sharing same appservice plan