TaskFactory.StartNew() creates a new Task, starts it and then returns it. I suppose that it is safe to assume that the following code will always work (since it was taken from MSDN):
Task.Factory.StartNew(() => Console.WriteLine("first"))
.ContinueWith(antecendent => Console.WriteLine("second"));
How does this work? How can I be assured that the task hasn't been started (or even completed) before .ContinueWith() is called?
The TPL is intended to abstract the asynchronous nature of the tasks from the consumer, so if you call ContinueWith on a completed task then the antecedent handler will be invoked immediately.
This means that you can create a Task with TaskFactory.StartNew (which will schedule the task to run asynchronously) or new Task(() => { /*...*/}) followed by task.RunSynchronously() and you can always call ContinueWith on the Task. It basically means "schedule this to run once the task is completed, or now if it is already finished".