I'm using TaskCompletionSource<> quite often. I have a network protocol design, where I receive many streams in one tcp/ip connection. I demultiplex those streams and then inform the corresponding "SubConnections" of new content.
Those "SubConnections" (which are waiting via await) then should continue in a new thread.
Usually I solve such issues by putting the TaskComplectionSource<>.Set call in an anonymous ThreadPool.QueueUserWorkItem method, like this:
ThreadPool.QueueUserWorkItem(delegate { tcs.SetResult(null); });
If I don't do this the corresponding await tcs.Task call will continue in the thread which called tcs.SetResult.
However, I'm aware of that this isn't the right way to do things. It's also possible to self-write a SynchronizationContext (or something) which will instruct the await call to continue in another thread.
My primary question here is: How would I do this in the "best practice" way?
My hope here is also to avoid the ThreadPool overhead, because it's quite high on Linux compared to just blocking a thread and waiting for a ManualResetEvent - even though the SynchronizationContext (or whatever) may also utilize the ThreadPool.
Please refrain from telling me that it's generally a bad idea to multiplex something in one tcp/ip connection or that I should just use System.IO.Pipelines, REST or whatever. This is my scenario. Thank you.
You can create the TaskCompletionSource using TaskCreationOptions.RunContinuationsAsynchronously (in .NET 4.6+):
var tcs = new TaskCompletionSource<Result>(TaskCreationOptions.RunContinuationsAsynchronously);
...
tcs.SetResult(...);
See e.g. this thread for more details.