RDS is running on postgres version of 9.6 while my installed version is 9.3.2 and an yum update changes the postgres version to 9.6.8 while my RDS is running on 9.6.6 . Does this version mismatch is bad or should I Upgrade my server RDS from 9.6.6 to 9.6.8
I've done exactly this and it works fine. As you're updating just the client and the client is backwards compatible (especially at such a minor revision level), it should not hurt. A good rule of thumb is to always do regression testing to make sure, and have a way to roll back. There is no rule that says client and server versions have to be identical, however.