We're creating a video calling app for the iPhone and it's working really well and on iOS17 we can even use gestures to make the video overlay animated reactions on the video which is really nice. But that came for free without any code from us.
I was wondering if anyone knows if it's possible to access information about what reactions were sent by the user?
i.e. if a user does a two thumbs up 👍🏻👍🏻 then it will show fireworks in the video.
Can I intercept that in code in my app and send some data somewhere in response to the fireworks shown?
For reference... these are the reactions I am referring to... https://www.macrumors.com/how-to/use-reaction-effects-facetime/
You can detect which gesture effects are enabled, and whether or not they are enabled, and can trigger them programmatically, using these APIs:
Specifically, you may be able to detect changes in this property to see when gesture effects have been triggered:
Your app can opt-in to gesture effects using the NSCameraReactionEffectsEnabled Info.plist key.
Update for iOS 17.4:
From the documentation headers for AVCaptureDevice.reactionEffectGesturesEnabled which was updated for iOS 17.4/macOS 14.4:
By default, gesture detection is enabled. As of iOS 17.4 and macOS 14.4, applications can control the default value of this property by adding the following key to their Info.plist:
<key>NSCameraReactionEffectGesturesEnabledDefault</key>A value of true enables gesture detection and a value of false disables it, until such time that the user makes their own selection in Control Center.
Unfortunately, Apple's online API docs for reactionEffectGesturesEnabled have not yet been updated to reflect this change, nor is there any documentation of NSCameraReactionEffectGesturesEnabledDefault yet.