I'm working on a project that needs to look at large amounts of data (~1TB) and copy it from drive A to drive B. It will be constantly run in the background (or tray) and run a check every XX hours/mins. At that time, it will check if there are any NEW files in drive A and copy them to drive B. If there are any files that were updated and newer then it will also copy and replace the files from A to B.
I'm not really sure where to start. Should I write this in Python or C# (maybe visual?)? If someone could give me some advice I would greatly appreciate it. Thanks!
EDIT:
Just wanted to give an update! I ended up using Robocopy, which is built into Windows. I moved away from Python and just created a small batch file that would check all of the files in drive A and compare to drive B. If anything was new or didn't exist, it copies it over. I then set up a task through Task Scheduler, also built into Windows. Works PERFECTLY in literally just 1 line of code in a batch file!
I was starting to look into building something like this myself. I was going to write it in c#, probably as a system service and then have it periodically scan for new files. It would then build checksums with either sha1 or md5. You can look here about how to generate an MD5 in c#. Here is some additional information talking about byte-for-byte vs checksum comparisons.
After it has its hash list, it can do a transfer of the files then do another hash on the destination to ensure it was written properly. I was going to just hang on to all the hashes and then when it rescans the directory it has something to compare to in order to see if a file was updated. Then it would just repeat the above.