I found the below limit AWS describes for Parameter Store.
Default throughput: 40 (Shared by the following API actions: GetParameter, GetParameters, GetParametersByPath)
Higher throughput: 100 (GetParametersByPath)
Higher throughput: 1000 (Shared by the following API actions: GetParameter and GetParameters)
For more information about Parameter Store throughput, see Increasing Parameter Store Throughput in the AWS Systems Manager User Guide.
I am wondering if the limit is for an individual key or the whole Parameter Store service(per account or region)
I believe throughput is the maximum number of transactions per second (TPS) that Parameter Store can process. So these limits are for the Parameter Store and not for the individual key.