Search code examples
svnreverse-proxysvnsync

svnsync cannot recover from source idle disconnect

If I try to mirror a 3rd-party repo to a local one then while svnsync is uploading a 300M+ file to the slow destination server the source server disconnects and svnsync does not try to reconnect to it and enters infinite loop. How to workaround this?

C:\progs\svnsync-dir>svnsync --version
svnsync, version 1.14.0 (r1876290)
   compiled May 24 2020, 17:07:49 on x86-microsoft-windows

C:\progs\svnsync-dir>svnsync --steal-lock sync https://targetreposito.acme.com/s
vn/Reponame
Stole lock previously held by 'my-host-name:ab204a7a-756f-2947-af9c-caf904a8945d
'
Transmitting file data .........................................................
................................................................................
................................................................................
................................................................................
................................................................................
................................................................................
................................................................................
................................................................................
................................................................................
................................................................................
................................................................................
................................................................................
................................................................................
................................................................................
................................................................................
................................................................................
................................................................................
................................................................................
................................................................................
................................................................................
............< here it hangs

enter image description here

enter image description here

Solution

  • After compiling the svn communication library serf from trunk svnsync instead of hanging is now crashing with An error occurred during decompression or (if

    [global]
http-compression = no

    ) in ~/.subversion/servers then with Malformed XML: no element found.

    After dumping the HTTP conversation between svnsync and the source server I figured out that the source server closed connection while svnsync was flushing partially received delta to the destination server.

    The solution is to use the pv tool with slice disabled and a huge buffer to hold the full response: pv -q -C -B4G

    Add to /etc/hosts:

    127.0.0.3 svn.acme.com
1.2.3.4   svn.acme.com.real

    Then run:

    sudo socat TCP-LISTEN:443,reuseaddr,fork EXEC:"./hugebuf.sh svn.acme.com.real 443",nofork

    Run:

    socat TCP-LISTEN:3128,reuseaddr,fork EXEC:"./hugebuf.sh -p",nofork

    hugebuf.sh (simplified):

    #!/bin/bash
host=${1:?}
port=${2:?}
exec 4>"/dev/tcp/${host:?}/${port}"
cat <&0 >&4 &
pv -q -C -B4G <&4

    Lastly, run the svnsync command with extra args:

    svnsync --config-option servers:global:http-proxy-host=localhost --config-option servers:global:http-proxy-port=3128

    Now it's able to commit this very revision.

    The above works with msys / git-for-windows too.