Search code examples
bashscpdirectory-structure

scp, inconsistency for file structure preservation

  • My task: collect log files from several servers.
  • Server file structure: "/remote/path/dir/sub-dirs/files.log", which is the same on all servers. (All servers have the same set of "sub-dirs", absence could happen, and of course "files.log" names differ)
  • Local file structure: "/local/path/logs"
  • After copy I would like to have "/local/path/logs/dir/sub-dirs/files.log"
  • Method (in a whlile loop for servers): scp -r $SERVERS:/remote/path/dir /local/path/logs
  • Problem: For reasons I don't understand, the first scp command ignores the "dir" folder, I get "/local/path/logs/sub-dirs/files.log" But following scp commands gives me what I intended "/local/path/logs/dir/sub-dirs/files.log"
  • Why is this happening and how should I fix/get around it?

Thanks!

Solution

  • Why is this happening [...]

    In the command scp -r path/to/source dest:

    • If dest doesn't exist, the dest directory will be created, and path/to/source/* will be copied into it. For example if you have path/to/source/X then dest/X will be created.
    • If dest is a directory, then dest/source will be created, and the path/to/source/* will be copied into it. For example if you have path/to/source/X then dest/source/X will be created.

    [...] and how should I fix/get around it?

    Create dest in advance, for example:

    mkdir -p /local/path/logs
scp -r $SERVERS:/remote/path/dir /local/path/logs