As I understand, API key's are typically used for thirdrd party access, but does it make sense to use one in order to verify it's my frontend talking to the API? I want to secure my API as best as it can be, so any other ideas for improving its security would be great.
An api key , with internal use have sense, out of the security scope.
Exemple : you have a web front end and an app (ios) , you could identify different type of REST client. You could add rate limitation by type of client.
First check to secure your API is probably "Do you use HTTPS" ?
see also : djangorestframework-api-key and : When and why api key by google