Tricky problem when using rsync to mirror and archive Linux to MacOS X | Dafacto

The personal website of Matt Henderson.

Tricky problem when using rsync to mirror and archive Linux to MacOS X

30 May 2007

I use rsync running on MacOS X to mirror some remote Linux file systems to an externally connected Firewire drive (“Mirror”), and to archive changed and deleted files to a second externally connected Firewire drive (“Archives”). In general, it works fine, but there has been a long-standing tricky problem, that my system administrator (Niall O Broin) and I finally solved today. Since I couldn’t find anything about this in Google, I figured a blog post was in order.

Here’s the basic problem: Every single time I run the rsync command, a certain set of files are always viewed as changed, and subsequently downloaded (and archived.) And I’m talking about static files like JPG images, not constantly changing stuff like log files.

Well, we finally nailed it. Have a look at this directory listing from the Linux server:

-rwxr-xr-x    1 webrun   www         42648 Nov  7  2005 Uirapuru.jpg
-rwxr-xr-x    1 webrun   www         67248 Nov  7  2005 uirapuru.jpg

The problem is that the Linux filesystem is case sensitive – you can have two files with the same name, if the case of their names is somehow different (as above). However, on most MacOS X formatted drives, the file system is case insensitive.

So, every time rsync would run, it would compare the two Uirapuru files with the single local file, and, of course, one of them would be different, and get downloaded (and subsequently sending the alternate file to my archives.)

What’s the solution? The solution for me is to reformat my Mirror and Archive drives to case-senstive HFS+. (The problem, though, is I don’t have the additional space to copy my files temporarily, in order to reformat the volumes. So, I guess I’ll just live with this for a while…)

Enjoy this article? — You can find similar content via the category and tag links below.

Questions or comments? — Feel free to email me using the contact form below, or reach out on Twitter.