I am working on HoloLens (Unity-UWP) and trying to make a connection with PC (UWP) or Android phone work (Xamarin). So far I tried client and host with both Bluetooth and TCP (even two versions with different libraries) on Android and UWP. I kept the code entirely separated from user interface, so that it is easier to use, to understand and modular. An Action<string> is used to output results (error logs and sent messages).
Everything that is not on the HoloLens works fine (even though it's exactly the same code). It worked from PC (UWP) to Android with client and host switched. But it doesn't even work between HoloLens and PC (UWP). The behavior ranged from crashes (mostly for Bluetooth) to instant disconnection. The last tests resulted in disconnection once bytes are about to be received. It could even read the first 4 bytes (uint for the length of the following UTF-8 message), but then it was disconnected. The other devices seemed to work fine.
What I know: Capabilities are set, the code works, the issue is likely something that is common for everything that has to do with networking and HoloLens.
So the question is, is Unity or HoloLens incompatible with something I am using? What I used which is worth mentioning: StreamSocket, BinaryWriter, BinaryReader, Task (async, await). Or is HoloLens actively blocking communication with applications on other devices? I know it can connect to devices with Bluetooth and that it can connect via TCP, and it looks like people succeed to get it to work. Are there known issues? Or is there something with Unity that causes this - a bad setting maybe? Do I have to use async methods or only sync? Are there incompatibility issues with Tasks/Threads and Unity? Is this possibly the issue (inability to consent to permissions)?
Another thing to note is that I cannot ping HoloLens via its IP by using the cmd, even though the IP is correct.
I'd appreciate any advice, answer or guess. I can provide more information if requested (see also the comments below). I would suggest to focus on the TCP connection as it seemed to be working better and appears to be more "basic." Here is the code:
using System;
using System.Text;
using System.Threading.Tasks;
using System.IO;
using Windows.Networking;
using Windows.Networking.Sockets;
#region Common
public abstract class TcpCore
{
protected StreamSocket Socket;
protected BinaryWriter BWriter;
protected BinaryReader BReader;
protected Task ReadingTask;
public bool DetailedInfos { get; set; } = false;
public bool Listening { get; protected set; }
public ActionSingle<string> MessageOutput { get; protected set; } = new ActionSingle<string> (); // Used for message and debug output. They wrap an Action and allow safer use.
public ActionSingle<string> LogOutput { get; protected set; } = new ActionSingle<string> ();
protected const string USED_PORT = "1337";
protected readonly Encoding USED_ENCODING = Encoding.UTF8;
public abstract void Disconnect ();
protected void StartCommunication ()
{
Stream streamOut = Socket.OutputStream.AsStreamForWrite ();
Stream streamIn = Socket.InputStream.AsStreamForRead ();
BWriter = new BinaryWriter (streamOut); //{ AutoFlush = true };
BReader = new BinaryReader (streamIn);
LogOutput.Trigger ("Connection established.");
ReadingTask = new Task (() => StartReading ());
ReadingTask.Start ();
}
public void SendMessage (string message)
{
// There's no need to send a zero length message.
if (string.IsNullOrEmpty (message)) return;
// Make sure that the connection is still up and there is a message to send.
if (Socket == null || BWriter == null) { LogOutput.Trigger ("Cannot send message: No clients connected."); return; }
uint length = (uint) message.Length;
byte[] countBuffer = BitConverter.GetBytes (length);
byte[] buffer = USED_ENCODING.GetBytes (message);
if (DetailedInfos) LogOutput.Trigger ("Sending: " + message);
BWriter.Write (countBuffer);
BWriter.Write (buffer);
BWriter.Flush ();
}
protected void StartReading ()
{
if (DetailedInfos) LogOutput.Trigger ("Starting to listen for input.");
Listening = true;
while (Listening)
{
try
{
if (DetailedInfos) LogOutput.Trigger ("Starting a listen iteration.");
// Based on the protocol we've defined, the first uint is the size of the message. [UInt (4)] + [Message (1*n)] - The UInt describes the length of the message (=n).
uint length = BReader.ReadUInt32 ();
if (DetailedInfos) LogOutput.Trigger ("ReadLength: " + length.ToString ());
MessageOutput.Trigger ("A");
byte[] messageBuffer = BReader.ReadBytes ((int) length);
MessageOutput.Trigger ("B");
string message = USED_ENCODING.GetString (messageBuffer);
MessageOutput.Trigger ("Received Message: " + message);
}
catch (Exception e)
{
// If this is an unknown status it means that the error is fatal and retry will likely fail.
if (SocketError.GetStatus (e.HResult) == SocketErrorStatus.Unknown)
{
// Seems to occur on disconnects. Let's not throw().
Listening = false;
Disconnect ();
LogOutput.Trigger ("Unknown error occurred: " + e.Message);
break;
}
else
{
Listening = false;
Disconnect ();
break;
}
}
}
LogOutput.Trigger ("Stopped to listen for input.");
}
}
#endregion
#region Client
public class GTcpClient : TcpCore
{
public async void Connect (string target, string port = USED_PORT) // Target is IP address.
{
try
{
Socket = new StreamSocket ();
HostName serverHost = new HostName (target);
await Socket.ConnectAsync (serverHost, port);
LogOutput.Trigger ("Connection successful to: " + target + ":" + port);
StartCommunication ();
}
catch (Exception e)
{
LogOutput.Trigger ("Connection error: " + e.Message);
}
}
public override void Disconnect ()
{
Listening = false;
if (BWriter != null) { BWriter.Dispose (); BWriter.Dispose (); BWriter = null; }
if (BReader != null) { BReader.Dispose (); BReader.Dispose (); BReader = null; }
if (Socket != null) { Socket.Dispose (); Socket = null; }
if (ReadingTask != null) { ReadingTask = null; }
}
}
#endregion
#region Server
public class GTcpServer : TcpCore
{
private StreamSocketListener socketListener;
public bool AutoResponse { get; set; } = false;
public async void StartServer ()
{
try
{
//Create a StreamSocketListener to start listening for TCP connections.
socketListener = new StreamSocketListener ();
//Hook up an event handler to call when connections are received.
socketListener.ConnectionReceived += ConnectionReceived;
//Start listening for incoming TCP connections on the specified port. You can specify any port that's not currently in use.
await socketListener.BindServiceNameAsync (USED_PORT);
}
catch (Exception e)
{
LogOutput.Trigger ("Connection error: " + e.Message);
}
}
private void ConnectionReceived (StreamSocketListener listener, StreamSocketListenerConnectionReceivedEventArgs args)
{
try
{
listener.Dispose ();
Socket = args.Socket;
if (DetailedInfos) LogOutput.Trigger ("Connection received from: " + Socket.Information.RemoteAddress + ":" + Socket.Information.RemotePort);
StartCommunication ();
}
catch (Exception e)
{
LogOutput.Trigger ("Connection Received error: " + e.Message);
}
}
public override void Disconnect ()
{
Listening = false;
if (socketListener != null) { socketListener.Dispose (); socketListener = null; }
if (BWriter != null) { BWriter.Dispose (); BWriter.Dispose (); BWriter = null; }
if (BReader != null) { BReader.Dispose (); BReader.Dispose (); BReader = null; }
if (Socket != null) { Socket.Dispose (); Socket = null; }
if (ReadingTask != null) { ReadingTask = null; }
}
}
#endregion
The answer is Threading.
For whoever may have similar issues, I found the solution. It was due to Unity itself, not HoloLens specifically. My issue was that I wrote my code separately in an own class instead of commingle it with UI code, which would have made it 1. unreadable and 2. not modular to use. So I tried a better coding approach (in my opinion). Everybody could download it and easily integrate it and have basic code for text messaging. While this was no issue for Xamarin and UWP, it was an issue for Unity itself (and there the Unity-UWP solution as well).
The receiving end of Bluetooth and TCP seemed to create an own thread (maybe even something else, but I didn't do it actively), which was unable to write on the main thread in Unity, which solely handles GameObjects (like the output log). Thus I got weird log outputs when I tested it on HoloLens.
I created a new TCP code which works for Unity but not the Unity-UWP solution, using TcpClient/TcpListener, in order to try another version with TCP connection. Luckily when I ran that in the editor it finally pointed on the issue itself: The main thread could not be accessed, which would have written into the UI - the text box for the log output. In order to solve that, I just had to use Unity's Update() method in order to set the text to the output. The variables themselves still could be accessed, but not the GameObjects.