I am currently using Memory optimized DB class in AWS (8 CPUs) because some push notifications in our app cause the CPU utilization to skyrocket, but 99% of the time CPU utilization is at about 10% so 8 CPUs aren't really needed most of the time.
Would I be able to deploy less CPUs on a Burstable Instance and have CPUs adjusted when there are those heavy traffic push notifications?
How do Burstable Instances work?
I wouldn't choose a burstable instance for any production traffic.
The burstable instance accumulates a number of "performance credits" per hour. These credits can be consumed when traffic increases and you needs a significant amount of resources. When the credits are spent, the instance still runs, but has only "baseline performance" which is, frankly, insufficient to handle production traffic.
I've seen many users try to economize by using the T family of instance types. They are usually quite disappointed, because they underestimate their need for resources. They end up consuming their burst credits too quickly, and then operate at the baseline performance level too often.
I'd use a burstable instance only for CI testing servers, or development. These instances typically run idle most of the time, and accumulate a good level of performance credits. They use these credits for brief periods, and then return to an idle level of activity.
Note: There is also the "Unlimited" option, which T3 instances use by default. This changes the nature of the burst performance. You can get greater CPU performance over prolonged periods, but you end up paying more for it. To understand the details, read https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/burstable-credits-baseline-concepts.html carefully.
You can also look into Aurora Serverless. This is supposed to auto-scale more replica instances in response to traffic increases, which should give you more CPU capacity. You only pay for the instances you use. That's the theory, but I can't speak from experience because I haven't used or tested Aurora Serverless. How well it works and how economical it is for you depends on your application's workload. All I can suggest is to give it a try.
Which instance type and options are best for you is greatly dependent on your specific application's traffic pattern. Does it have consistent load, or cyclical load (for example, high load during business hours but less at night)? Is it mostly CPU-bound, memory-bound, I/O-bound, or network-bound? You need to understand your application's resource needs before you can pick the best instance type.