Is it possible to create a Firebase token using the CLI that only works for a specific project inside the account? The current way using firebase login:ci looks like it can work for all projects inside an account. I am going to use this token for specifically Github action CI work and might pass around other team members.
It's not possible.
When you get a login token, that token represents the user, unrelated to any projects that user might have access to.
If you want to control the per-user access to various projects, you will have to configure that in the console using IAM for each product that you are trying to control. If you give the user permission to make changes in the console, then they implicitly have permission to make the same changes from the CLI.