Running Dropbox on DreamHost

As promised and after a nudge from Bruce, here’s how I was able to get the Dropbox Linux CLI software running on a DreamHost VPS server. This isn’t super hard but it wasn’t a trivial install, because the Dropbox software is closed-source and it contains binaries compiled against glibc 2.4, whereas my VPS server is using glibc 2.3.

The first step is to get the Dropbox CLI software, unpack it, and move it somewhere of your choosing:

msabramo@wettoad:~/sw$ wget -O dropbox.tar.gz "http://www.dropbox.com/download/?plat=lnx.x86_64"

msabramo@wettoad:~/sw$ tar xzf dropbox.tar.gz

msabramo@wettoad:~/sw$ mv .dropbox-dist dropbox

At this point, if you try running the Dropbox daemon, you will likely run smack into the glibc problem:

msabramo@wettoad:~/sw$ cd dropbox

msabramo@wettoad:~/sw/dropbox$ ./dropbox
./dropbox: /lib/libc.so.6: version `GLIBC_2.4' not found (required by ./dropbox)

The dropbox executable is dynamically linked against /lib/libc.so.6 and expects it to be glibc 2.4. On my server, it is glibc 2.3, which can be ascertained by using the strings command and grepping for GLIBC (not shown). So we need to install glibc 2.4 and get dropbox to use it.

We start by downloading and building glibc 2.4. We specify a prefix so that the libraries get installed in a special directory dedicated to glibc 2.4 stuff. This is critical — if you install in a standard location, you will make a royal mess of the system, as tons of programs are expecting glibc 2.3:

msabramo@wettoad:~/sw$ wget http://ftp.gnu.org/gnu/glibc/glibc-2.4.tar.gz
...
msabramo@wettoad:~/sw$ tzx glibc-2.4.tar.gz

msabramo@wettoad:~/sw$ cd glibc-2.4

msabramo@wettoad:~/sw/glibc-2.4$ mkdir -p build/linux_2.6.33.7_x86_64

msabramo@wettoad:~/sw/glibc-2.4$ cd build/linux_2.6.33.7_x86_64

msabramo@wettoad:~/sw/glibc-2.4/build/linux_2.6.33.7_x86_64$ ../../configure \
> --prefix=/home/msabramo/run/glibc-2.4 --enable-kernel=2.6 --enable-addons \
> --enable-nptl --enable-tls && make && make install
...
/home/msabramo/sw/glibc-2.4/build/linux_2.6.33.7_x86_64/elf/ldconfig: Can't open configuration file 
/home/msabramo/run/glibc-2.4/etc/ld.so.conf: No such file or directory
make[1]: Leaving directory `/home/msabramo/sw/glibc-2.4'

Now here’s the part that took me a little while to figure out — getting dropbox to use the new glibc. My first attempts involved using LD_LIBRARY_PATH and LD_PRELOAD, but these cause the process to immediately segfault. Then I realized that when you build glibc 2.4, you also build a new dynamic linker (ld.so) — i.e.: ld-2.4.so. It turns out that you have to use that dynamic linker.

msabramo@wettoad:~/sw/glibc-2.4/build/linux_2.6.33.7_x86_64$ cd ~/sw/dropbox

msabramo@wettoad:~/sw/dropbox$ cp ~/run/glibc-2.4/lib/ld-2.4.so .

msabramo@wettoad:~/sw/dropbox$ ./ld-2.4.so ./dropbox
This client is not linked to any account...
Please visit https://www.dropbox.com/cli_link?host_id=a76023a26676cea22072d0a159d30b75 to link this machine.

Bingo. No segfault. At this point, you visit the URL and authorize the machine. Then some finishing touches (which you might do very differently, depending on your preferences):

msabramo@wettoad:~/sw/dropbox$  cd

msabramo@wettoad:~$ ln -s ~/sw/dropbox/dropboxd ~/bin/dropboxd

msabramo@wettoad:~$ which dropboxd
/home/msabramo/bin/dropboxd

msabramo@wettoad:~$ ls -l $(which dropboxd)
lrwxrwxrwx 1 msabramo pg156676 34 Apr 14 14:37 /home/msabramo/bin/dropboxd -> /home/msabramo/sw/dropbox/dropboxd*

msabramo@wettoad:~$ diff -u ~/sw/dropbox/dropboxd.2011-04-14-143625 ~/sw/dropbox/dropboxd
--- /home/msabramo/sw/dropbox/dropboxd.2011-04-14-143625        2011-04-14 14:36:16.000000000 -0700
+++ /home/msabramo/sw/dropbox/dropboxd  2011-04-14 14:36:30.000000000 -0700
@@ -7,4 +7,4 @@
 PAR=$(dirname $(readlink -f $0))
 OLD_LD_LIBRARY_PATH=$LD_LIBRARY_PATH
 LD_LIBRARY_PATH=$PAR:$LD_LIBRARY_PATH
-exec $PAR/dropbox $@
+exec $PAR/ld-2.4.so $PAR/dropbox $@

msabramo@wettoad:~$ dropboxd

The last part above is tweaking the dropboxd wraper shell script so that it invokes the new dynamic linker. Now we can start the dropbox process from anywhere by invoking dropboxd.

Then I wanted to manage the dropbox process with supervisor, because I recently discovered supervisor and it is a nice way to start, stop, and monitor long-running processes. Here’s how I did it (this assumes supervisor is already installed, which typically can be done with pip install supervisor or easy_install supervisor):

msabramo@wettoad:~$ grep dropbox ~/run/etc/supervisord.conf
[program:dropboxd]
command=/home/msabramo/sw/dropbox/dropboxd

msabramo@wettoad:~$ supervisorctl update
dropboxd: added process group

msabramo@wettoad:~$ supervisorctl status
celeryd                          RUNNING    pid 8715, uptime 0:02:03
dropboxd                         RUNNING    pid 8442, uptime 0:02:21

msabramo@wettoad:$ pgrep -fl dropbox
8442 /home/msabramo/sw/dropbox/ld-2.4.so /home/msabramo/sw/dropbox/dropbox

Note that you don’t have to create the ~/Dropbox directory. The dropbox process will do that for you.

At this point, hopefully things are running, and you can see dropbox populating your directory with files.

msabramo@wettoad:~$ du -sh Dropbox
9.2M    Dropbox

msabramo@wettoad:~$ du -sh Dropbox
16M     Dropbox

msabramo@wettoad:~$ du -sh Dropbox
30M     Dropbox

The one problem I have noticed is that the Dropbox process is using more than 260 MB of memory on my VPS, a pretty good chunk of the memory that I am paying for:

msabramo@wettoad:~$ ps uww $(pgrep -f dropbox)
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
msabramo 19073  0.0  6.5 270016 30016 ?        Sl   16:49   0:08 
/home/msabramo/sw/dropbox/ld-2.4.so /home/msabramo/sw/dropbox/dropbox

msabramo@wettoad:~$ free -m
             total       used       free     shared    buffers     cached
Mem:           448        223        224          0          0         82
-/+ buffers/cache:        140        307
Swap:            0          0          0

This makes me nervous. If the memory usage gets too high, things like my web site and Subversion will start dying, unless I allocate more memory to the VPS, which costs me money. So it would be nice to get the memory usage down. If you have ideas on how to do this, please drop me a line.

4 thoughts on “Running Dropbox on DreamHost

  1. Pingback: Dropbox + glibc 2.4 buildout « Marc Abramowitz

  2. I doubt it will survive too much… the famous process killer would keep stopping the process, and it’s possible that DH already tracks that kind of activity/process.

    By the way… great instructions… just what I needed!

Leave a Reply

Your email address will not be published.

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>