Using rsync to Synchronise empegs

7 Mar 2002 12:21 empeg

Introduction

A while ago, I published my Lazy Bastard’s Guide to Cloning your empeg. This has worked successfully for me and other people. However, it has one major limitation – it’s an all-or-nothing operation – it clones the entire empeg. If you add a bunch of tracks to one of your empegs, you end up having to copy all of your tracks again.

In this document, I’ll present an alternative: using rsync to keep a pair of empegs in sync.

This procedure is not without its limitations – it only works one way, so you have to upload your tracks to one of your empegs (call it the master), and then replicate it to the other (the slave). For me, this isn’t a problem.

Caveat

This is not for the faint-hearted. If you’re not reasonably competent with Linux (or another Unix), then turn back now.

The Plan

I originally attempted to get rsync working using rsh. However, I couldn’t get a rsh server working on the empeg, so I gave up. I didn’t try very hard, though, so I may try again later.

Instead, we’ll use rsync as a daemon on one of the empegs, and connect to it from the other.

Installing rsync

As with my cloning guide, we’ll grab the .deb files from a Debian mirror, remove the stuff we don’t need, and then send them to the empeg.

For this, you’ll need the rsync_2.3.2-1.2.deb file. Extract it, remove the /usr/share parts, and then pack it back up. Send this to the empeg, and then extract it. I generally store the .tar.gz files in /drive0, so that the files don’t get overwritten by upgrades, and I can reinstate the software by unpacking them again.

Anyway, binaries and source are here.

Install rsync on both empegs.

rsyncd.conf

It doesn’t particularly matter whether you run rsync as a daemon on the source empeg or on the destination empeg. In this case, we’ll run it on the source empeg.

The first thing we need is a configuration file. This should be named /etc/rsyncd.conf:

log file = /drive0/rsyncd.log
pid file = /drive0/rsyncd.pid
uid = root
gid = root

[drive0]
    path = /drive0
    use chroot = no
    read only = no

[drive1]
    path = /drive1
    use chroot = no
    read only = no</pre>

If you’re planning on running this on the source empeg, then you can remove the “read only” lines.

hostname

Because rsync calls uname(2) to find out the hostname of the empeg, you’ll need to set it first. If you don’t, you’ll see gethostbyname: "Unknown host" (none) in the log file. The easy way to fix it is to put some stuff in /etc/hosts, like this:

127.0.0.1 localhost
my-empeg-ip my-empeg-name (none)

The right way to fix it is to install hostname, from hostname_2.07.deb, and then to use this to set the hostname of the empeg that will be running rsync as a daemon. You can grab a copy from here.

Even if you fix it the right way, you’ll still need to put some stuff in /etc/hosts, but you can forget the “(none)” hack:

127.0.0.1 localhost
192.168.0.1 my-empeg my-empeg.whatever.tld

Note that the hostname you set with hostname must be listed in /etc/hosts, or resolvable via DNS.

uid/gid

Because we’ve specified a uid and gid in rsyncd.conf, we need to make sure that these IDs resolve properly. If we don’t do this, we’ll see @ERROR: invalid uid at the client, with a corresponding Invalid uid root message in the log file; or the corresponding errors, but with “gid” instead of “uid”.

To fix it:

$ echo "root::0:0:root:/:/bin/sh" >> /etc/passwd
$ echo "root:x:0:" >> /etc/group

Start the rsync daemon

source-empeg:/# rsync --daemon

Note: check that it’s running (using ps), because the rsync daemon isn’t particularly talkative when it bails. If it’s not running, check the log file, and then have a look in the troubleshooting section, below.

Start the rsync client

dest-empeg:/drive0# rsync -auv --delete rsync://source-empeg/drive0/fids .

Do the same, but for drive1.

Note that the --delete removes files from the destination if they’re not present on the source. You might want to throw a -n (dry run – display actions without doing them) in there first.

Rebuild the database

Remove the database, playlists, tags files from /empeg/var on the destination empeg. Restart the player. It’ll rebuild the database.

Troubleshooting

neighbour table overflow

If you get

# ping toothgnip
neighbour table overflow
ping: sendto: No buffer space available
ping: wrote toothgnip 64 chars, ret=-1</pre>

…then do this:

# ifconfig lo 127.0.0.1