Super annoying issue going on, and I hope it is something simple that I am missing that someone can point out.
I deployed an app using Cloud Run. I then created a job in Cloud Scheduler using a service account with Owner permissions on my project to generate the OIDC for the Auth header.
The app successfully processes the request, but then within the logic of my app, I am getting the Secret Manager permissions denied error.
My only guess as to what is going on is that the service account I am using in the cloud scheduler is not the one that cloud run is using to authenticate to Secret Manager, but I have granted the Secret Manager Secret Accessor role to every service account I can think of.
Any ideas?
After going through all the comments here. I restarted the process using a service account with owner permissions, which worked fine. I then created a new service account and gave it one permission at a time until it fixed the issue.