WTSEnumerateSessions from JNA

Question

I'm trying to start a UI application from a java based Windows Service. If figured out so far, that the only approach to make this work is to get a list of sessions, find the one thats currently active, get the user handle for that session and finally create a new process for the given user.

I'm starting off by implementing the session enumeration using WTSEnumerateSessions, yet I'm struggling to get this working. The problem seems to be my mapping of the "_Out_ PWTS_SESSION_INFO *ppSessionInfo" parameter. I wrote the following code:

public interface Wtsapi32 extends StdCallLibrary {
   Wtsapi32 INSTANCE = (Wtsapi32) Native.loadLibrary("Wtsapi32", Wtsapi32.class, W32APIOptions.DEFAULT_OPTIONS);
   boolean WTSEnumerateSessions(IntByReference hServer, int Reserved, int Version, WTS_SESSION_INFO.ByReference[] ppSessionInfo, IntByReference pCount) throws LastErrorException;

   class WTS_SESSION_INFO extends Structure {
       public static class ByReference extends WTS_SESSION_INFO implements Structure.ByReference {}
       public int sessionId;
       public String pWinStationName;
       public int state;

       @Override
       protected List getFieldOrder() {
           return Arrays.asList("sessionId", "pWinStationName", "state");
       }
   }

}

On trying invoking the code with something like this:

public static void main(String[] argv) {
        Wtsapi32.WTS_SESSION_INFO.ByReference[] sessionInfo = null;
        IntByReference sessionCount = new IntByReference();
        try {
            if (Wtsapi32.INSTANCE.WTSEnumerateSessions(new IntByReference(0), 0, 1, sessionInfo, sessionCount)) {
                System.out.println("success :-)");
            }
        } catch (LastErrorException ex) {
            ex.printStackTrace();
        }

    }

I get a error code 1784 - ERROR_INVALID_USER_BUFFER. What would be the correct mapping for said API call from JNA?

Update: I have tried a version suggested Remy Lebeau, but this gives me an Invalid memory access exception:

public interface Wtsapi32 extends StdCallLibrary {
    Wtsapi32 INSTANCE = (Wtsapi32) Native.loadLibrary("Wtsapi32", Wtsapi32.class, W32APIOptions.DEFAULT_OPTIONS);
    boolean WTSEnumerateSessions(IntByReference hServer, int Reserved, int Version, PointerByReference ppSessionInfo, IntByReference pCount) throws LastErrorException;

    class WTS_SESSION_INFO extends Structure {
        public static class ByReference extends WTS_SESSION_INFO implements Structure.ByReference {}

        public int sessionId;
        public String pWinStationName;
        public int state;

        @Override
        protected List getFieldOrder() {
            return Arrays.asList("sessionId", "pWinStationName", "state");
        }

        public WTS_SESSION_INFO() {}
        public WTS_SESSION_INFO(Pointer p) {
            super(p);
        }
    }
}

Main:

    PointerByReference sessionInfoPtr = new PointerByReference();
    IntByReference sessionCount = new IntByReference();
    try {
        if (Wtsapi32.INSTANCE.WTSEnumerateSessions(new IntByReference(0), 0, 1, sessionInfoPtr, sessionCount)) {
            System.out.println("success :-)");
        }
    } catch (LastErrorException ex) {
        ex.printStackTrace();
    }

Answer 1

WTSEnumerateSessions() returns:

• a pointer to an array of WTS_SESSION_INFO structures
• a pointer to a DWORD of the number of elements in the the array.

So you need to pass a PointerByReference for the ppSessionInfo parameter, and a IntByReference for the pCount parameters. You can then use the values being pointed at by those pointers to access the array elements as needed. There is an example of this documented here:

JNA Example #7: Retrieve an Array of Structs from C

Also, your code is using an IntByReference for the hServer parameter. It needs to be a com.sun.jna.platform.win32.WinNT.HANDLE instead, or at least a Pointer. In C, a Win32 HANDLE is just a void* pointer. You need to set the first parameter to Pointer.NULL (which is what WTS_CURRENT_SERVER_HANDLE is defined as in C) to enumerate the sessions of the local server. IntByReference(0) is not the same thing as Pointer.NULL.

And don't forget to call WTSFreeMemory() to free the array data when you are done using it.

Try something like this:

public interface Wtsapi32 extends StdCallLibrary {
   Wtsapi32 INSTANCE = (Wtsapi32) Native.loadLibrary("Wtsapi32", Wtsapi32.class, W32APIOptions.DEFAULT_OPTIONS);

   boolean WTSEnumerateSessions(Pointer hServer, int Reserved, int Version, PointerByReference ppSessionInfo, IntByReference pCount) throws LastErrorException;
   void WTSFreeMemory(Pointer pMemory);

   class WTS_SESSION_INFO extends Structure {
       public static class ByReference extends WTS_SESSION_INFO implements Structure.ByReference {}

       public int sessionId;
       public String pWinStationName;
       public int state;

       public WTS_SESSION_INFO() {}
       public WTS_SESSION_INFO(Pointer p) {
           super(p);
       }

       @Override
       protected List getFieldOrder() {
           return Arrays.asList("sessionId", "pWinStationName", "state");
       }
    }
}

public static void main(String[] argv) {
    PointerByReference sessionInfoPtr = new PointerByReference();
    IntByReference sessionCount = new IntByReference();
    try {
        if (Wtsapi32.INSTANCE.WTSEnumerateSessions(Pointer.NULL, 0, 1, sessionInfoPtr, sessionCount)) {
            Pointer sessionInfo = sessionInfoPtr.getValue();
            int count = sessionCount.getValue();
            Wtsapi32.INSTANCE.WTS_SESSION_INFO arrRef = new Wtsapi32.INSTANCE.WTS_SESSION_INFO(sessionInfo);
            arrRef.read(); // <-- not sure why this is here
            Wtsapi32.INSTANCE.WTS_SESSION_INFO[] sessions = (Wtsapi32.INSTANCE.WTS_SESSION_INFO[])arrRef.toArray(count);
            for (Wtsapi32.INSTANCE.WTS_SESSION_INFO session : sessions) {
                // use session as needed...
            }
            WTSFreeMemory(sessionInfo);
        }
    } catch (LastErrorException ex) {
        ex.printStackTrace();
    }    
}