The FourSquare API seems to be very user-centric: After logging in via OAuth I can see my friends check in, and which of my friends are currently checked into a given venue.
I'm looking for something more venue-centric, where I can get a list of everybody currently checked into a venue (friends and non-friends), or ideally receive a notification each time there is a new check-in at a specific venue. Is this possible with the existing V2 API?
I know you asked this 2 months ago and surely found an answer already but to answer your question directly.
where I can get a list of everybody currently checked into a venue (friends and non-friends)
The FourSquare version 2 API has a venues endpoint
https://foursquare.com/dev/merchant/index_docs.html
A sample call will give you a stats field in the response
https://api.foursquare.com/v2/venues/5104
Stats: Contains checkinsCount (total checkins ever here) and usersCount (total users who have ever checked in here).
for more venue information, FourSuare has a subset to the version 2 API called the venues project with more venue driven data you can update and mine through.