If I created a multi-region CMK in account A, would I be able to create replica keys in another account in a different region, assuming the right permissions are granted? Or must replica keys be created in the same AWS account?
Sadly not. Replica keys are account specific. If you want other accounts to access these keys, you have to setup such permissions through KMS key policies in account A.