The “rsync” is a powerful command under the Linux environment.
The rsync stands for Remote Sync. Normally rsync is used to transfer file from one server(source) to another server(destination).
rsync has a lot of switches or option for managing the command much usefully. Here I am explaining the switches of rsync command with examples.
Rsync will actually look and see what in the file has changed and upload only the part of the file that has changed. Unlike ftp and other transfer solutions rsync doesn’t simply re-upload the entire file.
Common Syntax for Rsync:
# rsync [options] Source Destinations.
Switches of rsync:
1, -v, –verbose
Increase verbosity.
Example:
I’ve created a file ‘rsync’ under ‘/root/Rsync/’ for testing purpose.
root@server [~]# rsync -v /root/Rsync/rsync .
rsync
sent 65 bytes received 31 bytes 192.00 bytes/sec
total size is 0 speedup is 0.00
2, -r, –recursive
Recurse into directories.
3, -l, –links
Copy symlinks as symlinks.
4, -p, –perms
Preserve permissions.
5, -t, –times
Preserve modification times.
6, -g, –group
Preserve group.
7, -o, –owner
preserve owner (super-user only)
8, -D
Same as –devices –specials.
–devices : preserve device files (super-user only).
–specials : preserve special files.
9, -H, –hard-links
Preserve hard links
10, -A, –acls
Preserve ACLs (implies –perms)
11, -X, –xattrs
Preserve extended attributes
12, -a, –archive
This is very important rsync switch, because it can be done the functions of some other switches combinations.
Archive mode; equals -rlptgoD (no -H,-A,-X)
13, -q, –quiet
Suppress non-error messages.
14, To specify the file size for sync:
–max-size=SIZE
Don’t transfer any file larger than SIZE
–min-size=SIZE
Don’t transfer any file smaller than SIZE
15, –delete
Delete extraneous files from destination dirs.
16, W, –whole-file
Copy files whole (without delta-xfer algorithm)
17, -u, –update
Skip files that are newer on the receiver. Means, Do Not Overwrite the Modified Files at the Destination.
18, –progress
View the rsync Progress during Transfer.
19, Include and Exclude Pattern.
–include
–exclude
Patterns are expressed in single quote.
Examples:
[root@ser crybit]# rsync -avz --include 'c*' --exclude '*' /root/crybit/ 109.200.11.67:/root/Rsinc101
[email protected]'s password:
stdin: is not a tty
sending incremental file list
cry1.doc
cry2.doc
cry3.doc
cry4.doc
cry5.doc
sent 267 bytes received 107 bytes 68.00 bytes/sec
total size is 0 speedup is 0.00
In the above example files starting with ‘c’ are included for Rsync and all other files are excluded on Rsync operation.
20 -e : Rsync over Shell(SSH)
Syntax:
# rsync -avz -e ssh Source Destination
Some useful switches combinations and Useful “Rsync” examples:
If the destination server has a different SSH port, you need to know this command to connect and Rsync files from that server. More details
1, How to Sync two directories in the same machine ?
I have created two directories(also some files inside each dir) under /root location to check the rsync usages.
/root/Rsync1/
/root/Rsync2/
Example:
root@server [~]# rsync -zvr /root/Rsync1/ /root/Rsync2/
sending incremental file list
rsync1.txt
rsync10.txt
rsync2.txt
rsync3.txt
......
......
sent 502 bytes received 202 bytes 1408.00 bytes/sec
total size is 0 speedup is 0.00
Where,
-z is to enable compression
-v verbose
-r indicates recursive
2, Preserve the following option while Sync files using -a switch.
r, l, p, t, g, o, D : Switch details are mentioned in the above section.
Recursive mode, symbolic links, permissions, timestamp, owner and group
See the example below:
root@server [~]# ll /root/Rsync1/rsync10.txt
-rw-r--r-- 1 root root 0 Feb 17 07:40 /root/Rsync1/rsync10.txt
root@server [~]# ll /root/Rsync2/rsync10.txt
-rw-r--r-- 1 root root 0 Feb 17 07:40 /root/Rsync2/rsync10.txt
3, How to Rsync files from remote server to local ?
It is quit similar to SCP, the syntax for doing the same is pasted below:
# rsync -azv Remote-IP:/path/for/Sync /Path/to/Sync
Some examples:
Sync the directory from remote to local
[root@ser crybit]# rsync -azv MY.SERVER.IP:/root/Rsync1 /root/crybit/
[email protected]'s password:
stdin: is not a tty
receiving incremental file list
Rsync1/
Rsync1/rsync1.txt
Rsync1/rsync10.txt
Rsync1/rsync2.txt
Rsync1/rsync3.txt
.....
.....
sent 205 bytes received 527 bytes 77.05 bytes/sec
total size is 0 speedup is 0.00
Sync all files from remote to local
[root@ser crybit]# rsync -azv 109.200.11.67:/root/Rsync1/* /root/crybit/
[email protected]'s password:
stdin: is not a tty
receiving incremental file list
rsync1.txt
rsync10.txt
rsync2.txt
rsync3.txt
...
...
sent 201 bytes received 499 bytes 93.33 bytes/sec
total size is 0 speedup is 0.00
4, How to sync files from local to remote server ?
Syntax:
# rsync -azv /local-path/for/Sync/ Remote-IP:/path/to/Sync
Examples:
[root@support crybit]# rsync -avz /root/crybit/ 109.200.11.67:/root/Rsinc101
[email protected]'s password:
stdin: is not a tty
sending incremental file list
created directory /root/Rsinc101
./
rsync1.txt
rsync10.txt
rsync2.txt
rsync3.txt
...
...
sent 1028 bytes received 399 bytes 190.27 bytes/sec
total size is 0 speedup is 0.00
Note 1 : In example 3 and 4 you can use user@IP:/path format, if you have only the user privilege on server
Note 2 : The rsync will automatically create the destination directory/location if it isn't there.
That’s it !! 🙂
Related:
groupdel, groupmems, groupmod, useradd , usermod , chgrp, chown, ls, head, tail, top, ps, find, crontab, ftp commands, tar, rpm
Dump Mongo DB collections and move it to an S3 bucket.
Requirement: Need to write a script to create Mongo DB collections dump and move the dump to an AWS s3 bucket.
Prerequisites: SSH access to Mongo DB server, IAM user with AWS s3 full [or write] access, aws-cli on server, knowledge in Mongo commands for dump creation.
As we need to move the dump to an S3 bucket, first we need to configure IAM user. Then only we can move the dump to S3 bucket. To configure IAM, you need to install aws-cli tool on the machine.
I notice you left out the -c,–checksum option as well as mention of –dry-run and –delete. Our typical usage starts with rsync -azvc ./ remotehost:foo/ –delete –dry-run so that one can be sure one is in the correct place, and look over the changes and deletions before actually comitting them by removing –dry-run.
later after the initial sync, when more rapid fire updates are happening, we’ll leave off -c in favor of speed to save time on checksumming all the files every time you sync.
Thanks Scott!
Hi Arunlal,
Very good post on rsync command and uses.
It is very important command for all system administrator for sync data and backup data who is working as Linux System Admin.
Thanks.