Solving rsync hangs with Cygwin

Posted on by

I was using Cygwin rsync on our two Windows laptops to copy our files over from the old laptop to the new laptop. Invariably, the rsync would copy a handful of files and then hang. Restarting the rsync would copy a few more files and then hang again. Searching around on the Web, I found a lot of people complaining about Cygwin rsync hanging and not a ton of helpful information, but a few bits here and there that suggested that the problem was limited to using rsync over ssh and that rsync to a rsync daemon works better. Probably the most helpful page I came across was this page (which I actually found after I had fixed the problem, while looking for a good link for this post).

Well, I tried it and yes, running an rsync in daemon mode on the “server” completely solved the problem. You can set up an rsync daemon as a Windows service using a command like this:

cygrunsrv --install "rsyncd" --path /usr/bin/rsync --args "--daemon --no-detach"
  --desc "Starts a rsync daemon for accepting incoming rsync connections"
  --disp "Rsync Daemon" --type auto

You also need an rsyncd.conf file. I actually already had one because I had previously installed a modified Cygwin rsyncd package from the backuppc project. I did run into a problem with that though because the rsync.exe in that package conflicted with my installed Cygwin (cygwin1.dll versioning problems I suspect). So I ended up modifying the rsyncd.conf to use cygwin paths instead of Windows paths and then used the regular cygwin rsync.exe.

Interestingly, neither the rsync FAQ nor the Cygwin FAQ mention this problem.

11 thoughts on “Solving rsync hangs with Cygwin”

  1. i was pulling my hair out trying to set this rsync over ssh on win sbs 2k3. thanks to you it now works! i wish that chaps from cygwin project would include this in their bloody FAQ!

    🙂

  2. Thanks – I was stuck on this.

    Since this is the top hit on Google for “rsync hangs windows”, I thought I’d outline how the paranoid amongst us can still use SSH:

    1. On the server, set “hosts allow = 127.0.0.1” in rsyncd.conf. This will only allow local connections to the rsync server
    2. On the client, SSH in to the server, forwarding local port 873 to localhost:873. In PuTTY, this is configured through Connection->SSH->Tunnels in the PuTTY configuration.
    3. On the client, change rsync://server/whatever to rsync://localhost/whatever and run rsync in the normal way

    Basically, instead of getting the rsync client to talk to a remote rsync server, this uses a feature of SSH that fools the client and server into thinking they’re on the same computer. I have no idea why this should be any better, but it works fine for me.

    – Andrew Sayers

  3. You do not need to run rsync as a daemon.
    Download the rsync source from cygwin.
    (using setup.exe)

    …
    In short:
    Compile rsync from source and use pipes instead of socketpair.

    # To do this:
    cd /usr/src/rsync-3.0.6/
    ./prepare-source
    ./configure

    # Now undefine HAVE_SOCKETPAIR in config.h
    # Search for /HAVE_SOCKETPAIR
    # Comment out
    # #define HAVE_SOCKETPAIR 1
    # //#define HAVE_SOCKETPAIR 1
    vi config.h
    // #define HAVE_SOCKETPAIR 1

    make
    make install

    # Instead of overwriting the original executable
    # at /bin/rsync use the compiled executable
    # (/usr/local/bin/rsync.exe) in your backup scripts.
    # This way, even if /bin/rsync gets overwritten
    # by cygwin setup.exe, we are still using the good
    # rsync version compiled from source.

    See
    http://www.mail-archive.com/cygwin@cygwin.com/msg82514.html

    The hang is occuring when rsync is attempting to exchange protocol version numbers, it writes its version and then hangs waiting endlessly for a reply. The ssh process is detached, and apparently not diretly connected to rsync, as it is an orphan, owned by process 1. The ssh process never even gets as far as attempting network access and rsync
    never does anything of value.

    Not defining HAVE_SOCKETPAIR in the make configuration is enough to use pipe() vs. socketpair(), which is enough for rsync to run using ssh, just as it does on UNIX. While I’ve not done extensive testing, I’ve used it enough to believe it is working as designed/intended.

    The source file that is affected by the above change (primarily) is
    util.c in function: fd_pair():

    #ifdef HAVE_SOCKETPAIR
    ret = socketpair(AF_UNIX, SOCK_STREAM, 0, fd);
    #else
    ret = pipe(fd);
    #endif

    While use of socketpair() may be a better method, use of
    pipe() does work.
    …

  5. Interesting. Thanks for the info! I don’t use cygwin regularly these days so I probably won’t get a chance to try this, but perhaps some others who stumble upon this page will find your comments very useful.

    So why doesn’t it work with socketpair()? Is socketpair()? broken on Cygwin?

  7. Dragos Toader’s fix also worked here.
    In my case rsync didn’t even work for a few small files.
    It stopped working around summer 2010.
    My system is still Windows XP SP3.
    On a notebook with Windows XP SP3 installed it works perfectly.
    Thanks for all efforts and the solution!
    Alois

  8. I have Novell NetWare v6.5 SP8 and Rsync 2.6.0, during backup rsync hangs.
    Is version of Rsync correct?
    I can’t make any conclusion what is going on with hang?
    Any help?

  9. Pingback: Unison sync between Windows/Linux hangs randomly during transfer

  10. Hi,
    I’m having the same issue.
    I tried Dragos Toader’s fix, but couldn’t get to work.
    i get as far as:
    # To do this:
    cd /usr/src/rsync-3.0.9
    ./prepare-source
    Then i get an error: unable to mak

    Can someone pls give me some step by step instructions to resolve pls.

  11. Thanks!!! It worked for me too!
    Since I’m an automation fun (and have a lot of win servers to backup), I include my shell script:

    #!/bin/bash

    echo “MYUSER:MYPASSWORD” > /etc/rsyncd.secrets
    chown 18.544 /etc/rsyncd.secrets
    chmod 660 /etc/rsyncd.secrets

    cat > /etc/rsyncd.conf <<EOT
    gid = 544
    uid = 18
    hosts allow = IP_OF_SERVERS_TO_BE_ALLOWED_TO_CONNECT_HERE
    auth users = MYUSER
    secrets file = /etc/rsyncd.secrets
    strict modes = true
    read only = true
    use chroot = no
    transfer logging = true
    log format = %h %o %f %l %b
    log file = /var/log/rsyncd.log

    [ROOT]
    path = /cygdrive
    comment = cygdrive root dir
    EOT

    cygrunsrv –install "rsyncd" –path /usr/bin/rsync –args "–daemon –no-detach" \
    –desc "Starts a rsync daemon for accepting incoming rsync connections" \
    –disp "Rsync Daemon" –type auto

    net stop rsyncd
    net start rsyncd

    # end-of-script

    To rsync from remote, for example:
    $ export RSYNC_PASSWORD="MYPASSWORD"; rsync -av "MYUSER@WINSERVERNAMEORIP::ROOT/C"

    N.B.: USER does *not* have to be a system user.

Leave a Reply

Your email address will not be published. Required fields are marked *