With PostgreSQL it looks like all you do is set rds.force_ssl to true in the parameter groups, but I don't see this as an option with Aurora.
The same way you would do this in MySQL:
Run this command once for each MySQL user account you have, replacing mysqluser
with the username:
GRANT USAGE ON *.* TO 'mysqluser'@'%' REQUIRE SSL;
Then run this command to flush any cached privleges:
FLUSH PRIVILEGES;