Search code examples
powershellcopy-item

Copy-item duplicating some directories in sub directories. (Real head-scratcher)

When copying a Windows profile, I am successfully copying and excluding the voluminous "AppData" folder. But the side effect is that in the backup folder, sub-folder "Documents" duplicates are showing up.

$UserProfPath = "C:\Users\" + $UserID
$UserProfPathAllItems = $UserProfPath + "\*"
$UserBkpPath = $BackupLocation + "\" + $UserID

Copy-Item -Path (Get-Item -Path $UserProfPathAllItems -Exclude ('Appdata')).FullName -Destination            $UserBkpPath -Force -Recurse

This works, but it makes redundant "My Music", "My Pictures", and "My Videos" folders in the "Documents" sub-folder (but does not copy files); and that is the only redundant 3 folders it makes! Where in this logic would this line make these 3 redundant folders?

Solution

  • Those are hidden, system-defined junctions (junctions are akin to symbolic links to other directories) that exist solely for backward compatibility with pre-Vista versions of Windows.
    E.g., the legacy path $HOME\Documents\My Music simply redirects to $HOME\Music

    Your Copy-Item call includes -Force, which includes hidden items, so these system junctions are included in the copy operation and become regular, non-hidden, empty directories - this behavior is problematic, for the reasons discussed in the next section.

    Solution options:

    • Either: Simply delete the unwanted directories after the fact:

      Remove-Item -Recurse -Force $UserBkpPath\Documents\* -Include 'My Music', 'My Pictures', 'My Videos'

    • Or, assuming that the directory trees that you want to copy do not contain directory junctions / directory symlinks that you DO want to copy (as such), you can use Robocopy.exe with its /XJD option (add /SL if you want to copy file symlinks as such):

      robocopy.exe $UserProfPath $UserBkpPath /E /XD AppData /XJD
      • Using robocopy.exe has the added advantage of being faster than Copy-Item.
    Background information:

    You can discover these system junctions in a given directory tree as follows:

    Get-ChildItem -Attributes Hidden+System+ReparsePoint -Recurse -ErrorAction Ignore $UserProfPath

    • -ErrorAction Ignore is needed, because in Windows PowerShell Get-ChildItem tries to recurse into these junctions, which fails due to lack of permissions (even when running elevated). In PowerShell (Core) 7+, Get-ChildItem no longer does that (but you may still encounter permission-denied errors when running without elevation).

    • Target C:\ to find them on the entire C: drive, though you'll need to run with elevation (as administrator) to also see other users' system junctions.

    The fact that when such system junctions are copied they turn into regular directories in principle is somewhat justifiable:

    • As demonstrated in GitHub issue #5240, up to at least PowerShell (Core) 7.3.x (current as of this writing), Copy-Item currently invariably copies a symlink's / NTFS reparse point's target rather than the link itself.

      • The fact that you currently cannot even choose the latter by opt-in is a problem in itself.

    • However, there are two problematic aspects (in addition to currently not being able to request that links be copied as links):

      • With -Recurse, a directory link's target is normally also copied recursively.

        • In the case at hand, it is the lack of permissions to recurse into system junctions that (somewhat fortunately, in this particular case) prevents this recursive copying.
        • However, arguably this shouldn't fail silently, as it currently does.

      • Irrespective of whether the target is a link or not, Copy-Item currently doesn't copy file-system attributes such as Hidden, ReadOnly, or System for directories: