I am trying to make a script that returns the amount of unread messages in messages app on a mac, I know the code for mail but can't figure this problem out.
Check some of the Apple Scripts On this thread
Specifically, this one looked promising although it didn't work for me:
tell application "Messages" to "clear unread messages"
You should be able to: