Do I need a new API key for each of my projects or do you just need one that you can use multiple times in any project?
Each Google Maps API key is associated with the keystore you use to sign your app. So, if you use the same keystore for signing your all apps, you can use the same Maps API key.
But I not recommend you use the same Maps API key. Because now Google Maps API is no longer total free. Each Maps API key is limited query per day. Creating different Maps API key for each app is a good way to prevent limit Google Maps query.