Search code examples
httpftpsynchronizationrsync

How can I use rsync to retrieve files from an HTTP URL?

There is a bunch of files here: http://data.pudo.org/flexicadastre/

I want to have a local folder that contains the same files there are in that URL.

I've tried:

rsync -av http://data.pudo.org/flexicadastre/ .

But I get:

ssh: Could not resolve hostname http: nodename nor servname provided, or not known
rsync: connection unexpectedly closed (0 bytes received so far) [receiver]
rsync error: unexplained error (code 255) at /SourceCache/rsync/rsync-45/rsync/io.c(453) [receiver=2.6.9]

I normally use rsync to synchronise local folders or folder via FTP or SSH, but this is different and I'm not sure how to handle it. This site looks like an FTP to me, but it's exposed via HTTP. I can't really understand how it works. May rsync is not possible in this scenario and I should use something else?

Solution

  • It is not possible to use rsync on http URLs.

    To use the rsync algorithm the client-side rsync needs to interact with a server-side rsync process. This is done either directly, through ssh or (less common) rsh. HTTP is not an option.

    May rsync is not possible in this scenario and I should use something else?

    It looks like that you only want to mirror data from HTTP to the local file system. There are several programs out there which does this like wget or lwp-mirror and probably lots of others.