AFAIK, one key difference between the two is that Appsync is implemented using MQTT and has an emphasis on real-time notification, while SNS is more like a general pub/sub service.
The real-time notification part of Appsync seems to be the right tool for a chat application. However with Appsync, I will not be able to push the notification to users if a connection is not established(?)
I would like to have real-time notification for users regardless the application is opened or not. What is the right tool for this purpose?
Generally, you'll probably have a more "ready-to-use" solution for chat using AppSync, e.g. https://aws.amazon.com/blogs/mobile/building-a-serverless-real-time-chat-application-with-aws-appsync/
Using SNS will probably provide more flexibility at the cost of more of your time wiring up custom code.
You are correct that AppSync doesn't natively support push notifications for background applications; for that, you'd need to wire in something like SNS. You could probably couple AppSync with SNS using an AppSync Lambda resolver.
Depending on your requirements, i.e. if it's not essential that a system notification appears for new messages, you can also query AppSync at app startup time for chat messages that were received while the client was disconnected.