What it is
rsync is a file copying tool for Unix systems that only sends what has changed. It compares the source and destination, then transfers just the differences, which makes repeat copies of a large directory fast. It runs locally or over SSH, so it is a common way to push files to a remote server.
Its options cover the details a plain copy misses: preserving permissions and timestamps, deleting files that no longer exist at the source, excluding paths, and a dry-run mode that shows what would change before anything does. The algorithm behind it was designed by Andrew Tridgell and Paul Mackerras. Most deploy scripts that use it pair the copy with checks before and after.
How I use it
- rsync and git are what I deploy with, from scripts with their own safety checks.
- Production deploy scripts refuse to run from an unpushed branch or a dirty tree, so what rsync sends is always committed code.
- The deploy script for the stucco company's job app is locked so two deploys never interleave, and it md5-verifies what landed.
- Lend Pilot's deploy script syncs, rebuilds, migrates and smoke-tests in one run.
rsync is the right tool when a directory on one machine should end up matching a directory on another, quickly and repeatably. I reach for it in deploy scripts because it is fast, it runs over SSH, and a dry run lets a script show what it is about to change. The script around it does the checking.
Questions
How do you make sure a deploy did not half-finish?
My deploy scripts lock so two runs cannot overlap, and the stucco job app's deploy checks file hashes with md5 afterwards. Lend Pilot's deploy ends with a smoke test against production.
Why rsync instead of copying files with scp?
rsync only sends what changed, can delete stale files on the far side, and has a dry-run mode, which makes it safer and faster for repeated deploys.