From my understanding the delegate in a Parallel.ForeachAsync call is executed multiple times concurrently.
What if that delegate operates on a variable that is not local to the delegate?
Say I increment a static counter in the delegate. Would I need to protect the counter by a Mutex or something?
Parallel is not magic. You need to use all the usual mechanisms to prevent accessing shared state and worst case, synchronize it if you can't avoid it.
Of course, if you just synchronize in a Parallel, you'll quickly find it's completely worthless, giving you barely better results (or even worse) at the cost of occupying a lot more of your CPU. The idea is to break the work apart into jobs that do not influence one another - for example, by ensuring that the collection you are operating on already has all the inputs it needs, and only producing outputs into the output collection. Then Parallel can do its best to make that efficient. But of course, it only works if the batches are large enough to pay for the synchronization costs.
Keep in mind that Parallel has a lot of its little quirks, so profile, read up, and be very careful about what's going on. It isn't magic.