Search code examples
javaandroidgoogle-glassandroid-logcatgoogle-gdk

EyeGesture and EyeGestureManager clarity needed

Google review team requires glassewares to:

Dim the screen if there isn't an expectation that a user is looking at it.

This is consistent with the "in the here and now" experience of Glass. Glassware should always dim the screen if there isn't an expectation that a user is looking at it. Ideally it behaves like a timeline and dims after 15s. A user can 'rewake' the screen by looking up.

Update to be made: If a user is not looking at the results set in the card scroller, dim the screen.

This hints at using the EyeGesture, which doesn't seem to be mentioned anywhere on the Glass Develop Page.

After some searching I found this EyeGesture library (github) that from this stackoverflow post (Google Glass Eye Gesture Crashing (EyeGestureLib)) doesn't seem to work anymore (and hasn't been updated in 4 months+).

The accepted answer (from the stackoverflow post) proposed using this revised EyeGesture library (github)

It was also mentioned (in the stackoverflow post - as a comment ) that:

Basically, you're trying to expose classes that exist in the Glass environment itself, but not through the official APIs. By declaring these stub classes (none of the methods are implemented) and by putting them into the com.google.android.glass.eye package, we're allowing our code to compile with these unimplemented classes. At runtime, the system has implementations of those classes and the application will instead use the system's implementations.

Here are my following questions:

  1. Will there be (and when) an offcial API for EyeGesture's any time soon?
  2. I tried Implementing the revised EyeGesture library into my activity by following the guide proposed without any luck. What could I be doing wrong?
  3. Is there something I'm missing for it to be detected? I know that with the GestureDetector I'm required to Override the onGenericMotionEvent(MotionEvent event), is there something similar for the EyeGesture?

Here is what I'm currently doing:

I have a package named com.google.android.glass and in this package I have the following:

  • EyeGesture enum that implements Parcelable
  • EyeGestureManager class

I have in the main package:

  • GestureIds class (This one is different the github in that it's a public class and not private)

In my activity I have:

private void createEyeGestureDetector(ResultActivity resultActivity) {
    final GestureIds gestureIds = new GestureIds();
    //The github guide didn't mention any class names for 
    //mEyeGestureManager and mEyeGestureListener .. so I added some..
    EyeGestureManager mEyeGestureManager = EyeGestureManager.from(resultActivity);
    EyeGestureManager.Listener mEyeGestureListener = new EyeGestureManager.Listener() {
        @Override
        public void onDetected(EyeGesture gesture) {
            Log.i("EyeGestureListener", "Gesture: " + gesture.getId());
            int id = gesture.getId();
            if(id == gestureIds.WINK_ID || id == gestureIds.DOUBLE_WINK_ID) {
                Log.d("EyeGesture", "Wink");
            } else if (id == gestureIds.BLINK_ID || id == gestureIds.DOUBLE_BLINK_ID){
                Log.d("EyeGesture", "Blink");
            } else if (id == gestureIds.LOOK_AT_SCREEN_ID || id == gestureIds.LOOK_AWAY_FROM_SCREEN_ID) {
                Log.d("EyeGesture", "Screen");
            }

        }
    };
}

In my onCreate I have:

//..
super.onCreate(bundle);
createEyeGestureDetector(this);
//..

Update Logcat:

When I do:

for (EyeGesture eg : EyeGesture.values()) {
    boolean supported = mEyeGestureManager.isSupported(eg);
    Log.w("yupyup", eg.name() + ":" + supported);
}

I get:

12-10 18:40:51.252    2405-2405/com.google.android.glass.websurg.websurg W/yupyup﹕ WINK:true
12-10 18:40:51.252    2405-2405/com.google.android.glass.websurg.websurg W/yupyup﹕ DOUBLE_WINK:false
12-10 18:40:51.252    2405-2405/com.google.android.glass.websurg.websurg W/yupyup﹕ BLINK:false
12-10 18:40:51.252    2405-2405/com.google.android.glass.websurg.websurg W/yupyup﹕ DOUBLE_BLINK:true
12-10 18:40:51.260    2405-2405/com.google.android.glass.websurg.websurg W/yupyup﹕ DON:true
12-10 18:40:51.268    2405-2405/com.google.android.glass.websurg.websurg W/yupyup﹕ DOFF:true
12-10 18:40:51.268    2405-2405/com.google.android.glass.websurg.websurg W/yupyup﹕ LOOK_AT_SCREEN:true
12-10 18:40:51.268    2405-2405/com.google.android.glass.websurg.websurg W/yupyup﹕ LOOK_AWAY_FROM_SCREEN:false

I also added (from the first github link):

@Override
protected void onStart(){
    super.onStart();
    createEyeGestureDetector(this);
    for (EyeGesture eg : EyeGesture.values()) {
        boolean supported = mEyeGestureManager.isSupported(eg);
        Log.w("yupyup", eg.name() + ":" + supported);
    }
    mEyeGestureManager.register(EyeGesture.LOOK_AT_SCREEN, mEyeGestureListener);
    mEyeGestureManager.register(EyeGesture.LOOK_AWAY_FROM_SCREEN, mEyeGestureListener);
    mEyeGestureManager.register(EyeGesture.WINK, mEyeGestureListener);

}

and 

@Override
protected void onStop(){
    mEyeGestureManager.unregister(EyeGesture.LOOK_AT_SCREEN, mEyeGestureListener);
    mEyeGestureManager.unregister(EyeGesture.LOOK_AWAY_FROM_SCREEN, mEyeGestureListener);
    mEyeGestureManager.unregister(EyeGesture.WINK, mEyeGestureListener);
    super.onStop();
}

This gives me:

12-10 18:46:11.314    2553-2553/com.google.android.glass.websurg.websurg I/EyeGestureManager﹕ Removing listener: com.google.android.glass.websurg.websurg.ResultActivity$1@41b8b908 for eye gesture: LOOK_AT_SCREEN
12-10 18:46:11.314    2553-2553/com.google.android.glass.websurg.websurg I/EyeGestureManager﹕ Removing listener: com.google.android.glass.websurg.websurg.ResultActivity$1@41b8b908 for eye gesture: LOOK_AWAY_FROM_SCREEN
12-10 18:46:11.314    2553-2553/com.google.android.glass.websurg.websurg I/EyeGestureManager﹕ Removing listener: com.google.android.glass.websurg.websurg.ResultActivity$1@41b8b908 for eye gesture: WINK

However they do not get detected.. even the WINK since it seems to be supported.

Solution

  • Google team already answered some of these but I will go ahead and provide more details about their answer and also provide an alternate way of doing these stuff you requested.

    Dim the screen if there isn't an expectation that a user is looking at it.

    This is consistent with the "in the here and now" experience of Glass. Glassware should always dim the screen if there isn't an expectation that a user is looking at it. Ideally it behaves like a timeline and dims after 15s. A user can 're-wake' the screen by looking up.

    Update to be made: If a user is not looking at the results set in the card scroller, dim the screen.

    Glass handles that itself but the problem is that if the user doesn't touch the Glass pad for about 10 seconds or more, Glass will go to sleep and your App will stop running. Great way of fixing this is to make Glass screen always on and check when the user looks at the screen or when they remove the Glass.

    If the user looks at the screen, increase the brightness of the screen, if they look away, decrease the brightness of the screen.

    If they remove the Glass from their face, decrease the brightness to zero, turn off the screen and stop running all the big CPU intensive code you have.

    If they put back the Glass on their face, increase the brightness of the screen,turn on the Screen and then enable all your CPU intensive code.

    You could just have a boolean variable to determine when to start or stop running. This method is recommended if you don't want your app to stop running after no touch event for seconds. It also saves battery when running your app.

    Code Examples for the things I said above are below:

    To Get Screen Brightness:

    //Get Screen Brightness
    public float getScreenBrightness() {
       WindowManager.LayoutParams wMLayout = getWindow().getAttributes();
       return  wMLayout.screenBrightness;
    }

    To Set Screen Brightness(0 to 1):

     //Set Screen Brightness
    public boolean setScreenBrightness(float sBrightness){
        if(sBrightness>=0){
            WindowManager.LayoutParams wMLayout = getWindow().getAttributes();
            wMLayout.screenBrightness = sBrightness; //Modify Brightness
            getWindow().setAttributes(wMLayout); //Apply changes
            return true;
        }else
        {
            return false;
        }
    }

    To Keep the Screen On or Off:

        //Turn Screen On/Off
public void keepScreenOn(boolean screenOn){
    if(screenOn) {
        getWindow().addFlags(WindowManager.LayoutParams.FLAG_KEEP_SCREEN_ON);
    }else{
        getWindow().clearFlags(WindowManager.LayoutParams.FLAG_KEEP_SCREEN_ON);
    }
}

    Don't forgeet to add permision in the Manifest:

    <uses-permission android:name="android.permission.WAKE_LOCK" />

    If you are just doing developing and don't want to worry about permision right now, you can use:

    <uses-permission android:name="com.google.android.glass.permission.DEVELOPMENT" />

    and avoid having to look up what permission to use. I suggest you use that for now as it will save you time during development and you don't have to worry about permission when coding.

    [EYE GESTURE]

    No Google official API available for detecting those. Anything available now is a little hack to access hidden Glass API for that. Glass team is working on it and they said the API will only be release when it is reliable. Right now, it is NOT perfect according to them.

    NOTE

    The answer I am about to post below SHOULD work but may NOT work on the next Glass update. When they do update, something magically changes and one function will STOP working. Glass API and Glass itself is on Beta Mode and therefore expect things to keep changing until official EYE Gesture API gets released.

    There are two ways to detect Eye Gesture. One way is to use IntentFilter and wait for "gesture" message.Another way is to use Stub Library to Access the hidden Glass API. I will talk about both here as there are prons and cons for each method.

    Method 1 (Stub Lib):

    This is the way you are currently trying to do it.

    Pros:

    Can detect more gestures

    Cons:

    Wink CANNOT be stooped from taking pictures.

    Yo are using different library than the one I used that is still working. I will try to fix your problem if that doesn't work, You should then do it the way I did mine with the library I used too.

    You got Step 1 wrong.

    Step 1: Create the stubs:

    Create a package called com.google.android.glass. In this package create two classes: EyeGesture and EyeGestureManager

    It should be

    com.google.android.glass.eye

    NOT

    com.google.android.glass

    com.google.android.glass may have worked in the past but there were too many updates.

    So, EyeGesture and EyeGestureManager must be placed in your package called com.google.android.glass.eye

    If Eye gesture is still not detected, forget about that library and use the one I am currently using. Close your project and create a new one.

    Steps:

    1) Download the library from here. (Last update was 4 months ago). The one you are currently using was probably last updated 8 months ago or even a year ago.

    https://github.com/prt2121/EyeGestureLib

    The zip file will have a long name like "EyeGestureLib-fwenindioniwenubwudew".

    Rename the Zip file to "EyeGestureLib".

    Extract the folder inside with a long name like "EyeGestureLib-f8a9fef3bde4396f947106e78cd0be7c7ecdd5a6"

    Rename that folder to "EyeGestureLib"

    The "EyeGestureLib" folder should have two folders inside it called "EyeGestureStub" and "EyeGestureDemoApp" plus other useless files.

    2) Open Eclipse and create a new project. create a simple MainActivty class activity.

    3) Inside your MainActivity class:

    private EyeGestureManager mEyeGestureManager;
private EyeGestureListener mEyeGestureListener;

private EyeGesture target1 = EyeGesture.WINK;
private EyeGesture target2 = EyeGesture.DOUBLE_BLINK;
private EyeGesture target3 = EyeGesture.LOOK_AT_SCREEN;

    Inside onCreate:

    mEyeGestureManager = EyeGestureManager.from(this);
mEyeGestureListener = new EyeGestureListener();

    Inside onStart:

    mEyeGestureManager.register(target1, mEyeGestureListener);
mEyeGestureManager.register(target2, mEyeGestureListener);
mEyeGestureManager.register(target3, mEyeGestureListener);

    Inside onStop:

    mEyeGestureManager.unregister(target1, mEyeGestureListener);
mEyeGestureManager.unregister(target2, mEyeGestureListener);
mEyeGestureManager.unregister(target3, mEyeGestureListener);

    Inside MainActivity (Not inside any function but just anywhere inside you MainActivity class):

    private class EyeGestureListener implements Listener {

    @Override
    public void onEnableStateChange(EyeGesture eyeGesture, boolean paramBoolean) {

    }

    @Override
    public void onDetected(final EyeGesture eyeGesture) {
       //Show what we just detected
       Log.i(eyeGesture.toString() , " is detected");

              //Check which eye event occured
        if (eyeGesture.name() == target1.name()) {
            // Wink
            Log.i("EyeGesture: ", " you just winked");
        } else if (eyeGesture.name() == target2.name()) {
            // Double blink
            Log.i("EyeGesture: ", " you just double winked");
        } else if (eyeGesture.name() == target3.name()) {
            // Look at Screen
            Log.i("EyeGesture: ", " you Looked at Screen");
        }

    }
}

    4) You will get error. Import the EyeGestureStub that is inside the EyeGestureLib folder to fix it.

    To fix the error:

    a) Go to File -> Import -> Android -> Existing Android Code into Workspace

    Click Next, Browse and Browse the EyeGestureStub folder inside EyeGestureLib folder.

    Make sure to exclude the "EyeGestureDemoApp" if it is there. You ONLY need EyeGestureLib folder which contains EyeGesture and EyeGestureManager.

    b) Right click on "EyeGestureStub" -> Properties -> Android -> On the right side,under Project Build Target make sure that "Glass Development Kit Preview" check-box is checked.

    Under Library, make sure that the "Is Library" check-box is checked.

    Click Apply and Ok to exit the window.

    c) Open Android SDK Manger. check for the version of Android SDK Build-tools installed. I have 21.1.1.

    d) Open the project.properties of EyeGestureStub and change sdk.buildtools=18.1.1 to sdk.buildtools=21.1.1 Finish.

    Done. It should work if you followed the instruction.

    Run it and choose MainActivity as the the Launch Activity.

    <-------------------------------------------------------------------------------------------------------------------------------->

    [STILL NOT WORKING? IMPORT EVERYTHING && work from there]

    If you can't get it to Work, delete the current project and import the whole project downloaded then work from there up. This is the easiest way. You may need to fix some errors before you can compile.**

    To import the project,

    1) Go to File -> Other -> Android -> Android Project from Existing Code.

    Next -> Browse

    then choose the EyeGestureLib folder which contains both the EyeGestureStub and EyeGestureDemoApp.

    Make sure under Project to Import that both EyeGestureStub and EyeGestureDemoApp are check-box are checked then click Finish.

    2) Right click on "EyeGestureStub" -> Properties -> Android -> On the right side,under Project Build Target make sure that "Glass Development Kit Preview" check-box is checked.

    Under Library, make sure that the "Is Library" check-box is checked.

    Click Apply and Ok to exit the window.

    3) Right click on "MainActivity" -> Properties -> Android -> On the right side,under Project Build Target make sure that "Glass Development Kit Preview" check-box is checked.

    4) You will get invisible error that will not be showing.

    To see it Go to Windows -> Show View -> Problems There, you will see all the problems.

    Next step to fix it, we have to match the Android SDK Build-tools version with the ones listed in the project.properties of both EyeGestureStub and MainActivity

    a) Open Android SDK Manger. check for the version of Android SDK Build-tools installed. I have 21.1.1.

    b) Open the project.properties of EyeGestureStub and change sdk.buildtools=18.1.1 to sdk.buildtools=21.1.1

    c) Open the project.properties of MainActivity and change sdk.buildtools=18.1.1 to sdk.buildtools=21.1.1

    Note: Changing the first project.properties may automatically change the second one.

    Done. It should work if you followed the instruction.

    Run it and choose MainActivity as the the Launch Activity.

    <-------------------------------------------------------------------------------------------------------------------------------->

    Method 2 (IntentFilter)

    Pros:

    Wink CAN be stopped from taking pictures.

    Cons:

    Detects WINK ONLY

    The first method can receive four events (WINK,DOUBLE_WINK,DOUBLE_BLINK,LOOK_AT_SCREEN,) but this method can ONLY receive one event (WINK).

    This method is useful if you just want to detect ONLY WINK without Glass taking a picture.

    To listen to Intent, you have to extend BroadcastReceiver.

    public class EyeGesture extends BroadcastReceiver {
    @Override
    public void onReceive(Context context, Intent intent) {
        if (intent.getStringExtra("gesture").equals("WINK")) {

             //Disable Camera Snapshot
            abortBroadcast();

            Log.e("WINKED ","");
        }else {
            Log.e("SOMETHING", "is detected " + intent.getStringExtra("gesture"));
        }
    }
}

    You must register the intent in the Manifest as below:

     <receiver android:name="com.inno.inno.glassplugin.EyeGesture">
                <intent-filter>
                    <action android:name="com.google.android.glass.action.EYE_GESTURE" />
                </intent-filter>
 </receiver>

    The name specified in the Manifest must match the name of the class listening to the intent which is EyeGesture.

    Simple as that. No library required but only WINK can be detected. It also stops Glass from taking picture when wink is detected. You can comment abortBroadcast(); if you want Glass to take picture when event is detected.

    This is for any one looking to detect Eye Gesture from Glass at this moment. These are the only current solutions around until Google releases their official Eye Gesture API.

    You should file for a new Glass API feature here. File it as Glass Eye Gesture API Request. If the Glass team receives too much of this feature request, they will make it their top priority and release it. I already filed for one.