Tester able to install Sandboxed Mac App without UDID

I have created a sandboxed version of my Mac App. Now I've sent it out to testers. One tester though didn't have his UDID registered in the provisioning file that belonged to the app. Still, he was able to run the app.

Shouldn't that be impossible?

My signing is set to "Mac App Store" in Xcode 5.

My Code Sign for Release is set to "Mac Developer". I've haven't done anything in the dev portal, I've let Xcode handle all the certificates and prov. files.

When looking in the dev portal, I see that there isn't any provisioning profiles available under "Distribution". Do I need to make these manually?

For iOS I would've created a distribution provisioning profile for ad hoc distribution.

Update

I also tried to create my own provisioning profiles and App ID's to no avail.

Should the GateKeeper icon be visible under Entitlements?

enter image description here

I don't want any other computer than the ones I've added to the devices in the provisioning file, being able to run the app.

Solution

It seems that it is not possible having only devices in the device list being able to run the app.