I have a requirement to pull a file from a source server. This file is usually around 500mb+ and does not have a specific timing when it is generated.
My pull script runs every 5mins however I want to make sure that the file is 100% completely in the source server before I pull it. Is there anyway to know if the file is partially or completely transferred?
There are multiple ways to do this.One sure way is the source system which is writing this file either creates a marker file after the copy is done and you can check if the marker file exists to ensure the copy is complete. Also there are other ways like renaming the file once the copy is done to indicate copy completion.
If changing the source system is outside your control, you can use the "lsof" command. The Linux lsof command lists information about files that are open by processes running on the system. (The lsof command itself stands for “list of open files.”)
lsof <filename> | wc -l
if the count is 0, there are no processes using this file which would indicate the file copy operation is complete. However, this will only work if you are sure there would be no other process opening/using this file when you are checking its status using lsof Hope this helps