VirtSync can update an older copy of a file to the latest content. Ideal for backups - either to a single backup drive, or to a set of rotated backup media.
VirtSync's efficiency allows multi-gigabyte files to be replicated to remote backup and hot standby locations. It is currently synchronizing 50GB+ disk images across the internet in under 3 minutes each
Remote synchronisation requires that the machine's public certificate has been installed in the target machine's authorized_keys file. If login to the remote machine is not possible without a password prompt, virtsync will fail. See "man ssh-copy-id".
chris@server:~$ time virtsync -v /var/lib/libvirt/images/vsws.img backup.barricane.com:/home/chris/ syncing /var/lib/libvirt/images/vsws.img to backup.barricane.com:/home/chris/vsws.img (dot = 1 GiB) [..................................................] done - 53687091200 bytes compared, 0 bytes transferred. real 2m44.662s user 0m48.730s sys 0m44.000s
(or under 20 minutes when there are 800MB of changes to synchronize).
chris@server:~$ time virtsync -v /var/lib/libvirt/images/vsws.img backup.barricane.com:/home/chris/ syncing /var/lib/libvirt/images/vsws.img to backup.barricane.com:/home/chris/vsws.img (dot = 1 GiB) [>.......>.>.>.>.>.>.>.>.>.....>.......>.......>...] done - 53687091200 bytes compared, 810745856 bytes transferred. real 17m42.061s user 0m47.590s sys 0m46.870s
(The above tests were done using 5 year old servers, and FTTC broadband.)
VIRTSYNC(1) VIRTSYNC(1) NAME virtsync - synchronise huge sparse files efficiently SYNOPSIS local: virtsync [OPTIONS] source_file dest_file virtsync [OPTIONS] source_files dest_dir virtsync [OPTIONS] source_dir dest_dir remote: (via ssh) virtsync [OPTIONS] source_file [user@]host:dest_file virtsync [OPTIONS] source_files [user@]host:dest_dir virtsync [OPTIONS] source_dir [user@]host:dest_dir DESCRIPTION Synchronise source_file to dest_file or dest_dir, multiple source_files to dest_dir, or source_dir to dest_dir. Remote synchronisation requires that the machine's public certificate has been installed in the target machine's authorized_keys file. If login to the remote machine is not possible without a password prompt, virtsync will fail. See "man ssh-copy-id". Destination files are updated in-place, with only changed blocks being written. Destination files and directories are created, if they do not already exist. OPTIONS -V, --version Output program version. -h, --help Output help information. -v, --verbose Enable verbose output. -j, --json-stats Enable JSON stats. (from version 1.1.7 onwards) -C, --compression Pass -C parameter to ssh transport. -i, --identity-file Pass -i parameter to ssh transport. REPORTING BUGS Report virtsync bugs to firstname.lastname@example.org VirtSync home page: http://www.virtsync.com AUTHOR virtsync was written by <email@example.com>. COPYRIGHT Copyright (C) 2011-2015, Barricane Technology Ltd. All Rights Reserved. September 18, 2011 VIRTSYNC(1)
Virtsync reads the whole source file on the source machine and the whole destination file on the destination machine. This is neccessary to determine which parts are different. Only the changed parts are sent over the wire from the source to the destination machine.
Use of remote filesystems is therefore unsupported. In addition, there may be issues with permissions and ioctls on non-standard filesystems.
loading libcrypto.so... loading libcrypto.so.0.9.8... loading libcrypto.so.0.9.8e... loading libcrypto.so.6... libcrypto.so.6: cannot open shared object file: No such file or directory
This is a bug in v0.9.5 - it will be fixed in the next release. You can work around this issue by installing the openssl-devel package (for .rpm users) or libssl-dev (for .deb users).
Yes, if the VM is not writing to it's disk for the duration of the synchronization. Otherwise, look at LVM snapshots.
When there is no old version of the file as a target, the entire contents of the file must be sent. For a remote sync, this will be limited by the speed of your network connection. For a local sync, this will be limited by the sequential IO rate of your drives.
Evaluation copies are licensed for 30 days from the date of download.
The majority of modern servers run 64 bit operating systems. If you need
a 32 bit package then
Virtsync is typically ten times faster than rsync when updating an existing file (tested across 100Mbit ethernet), and twice as fast for the initial copy.
If you would like to use virtsync beyond the expiry of your evaluation copy, you must buy licenses to use the software on one or more machines.
Delivery: Fully licensed copies of virtsync (in both .rpm and .deb formats) will be delivered by email within three working days of receipt of payment.
Note: The one machine license has network functionality disabled, it is intended for virtsync'ing huge files between an internal HDD and an external HDD for backup purposes.