We're using robocopy in monitor mode to copy files between web servers in Azure in a daisy-chain fashion. The command is invoked as an Administrator at startup.
The process works beautifully, but every now and then it will stop without error. We recently enabled history in Task Scheduler to get more insights into what was happening. But it simply reveals two steps: Action Completed and Task Completed.
The output of Action Completed:
Task Scheduler successfully completed task "\Robocopy Content From VM1" , instance "{some-vm-guid}" , action "C:\Windows\system32\robocopy.EXE" with return code 2147942408.
The output of Task Completed:
Task Scheduler successfully finished "{some-vm-guid}" instance of the "\Robocopy Content From VM1" task for user "HCVMHOST-2\SomeAdminUser".
The robocopy command:
robocopy "\\vm-1\c$\Content\SomeFolder" "c:\Content\SomeFolder" /mon:1 /FFT /MIR /s /zb /copyall /mt:4 /ns /nc /nfl /xd OutputCache Temp /LOG:"d:\RobocopyLogs.txt" /b
I suspect it may have something to do with either latency or a VM rebooting. But I'm not convinced this is the reason, and I want to make sure that my command isn't missing a param or has a param it shouldn't. As far as I can tell, it works fantastically, until it doesn't.
Anyone experience this before?
After a bunch of digging, and some random luck (my past attempts to Google the return code were fruitless because of a simple formatting different, 2,147,942,408 versus 2147942408). It turns out that the return code actually has significance, indicating that:
Not enough storage is available to process this command.
It is a curious response though, given each of the VMs has plenty of remaining disk space on all drives. I will raise a support ticket with Azure, and post back if any relevant info comes up.