Ceci est une ancienne révision du document !
Back in issue #37, I wrote about configuring an SSH server on your computer, in order to use it as a SOCKS proxy. Since I imagine not all that many people want to use it as such, I decided to focus on my second-most used command (my first-most used command is “pacman” - ArchLinux's solution to package management). Before I get into what that command is, I'll briefly explain why you might be interested in this solution. Specifically, it lets you sync directories and files between two computers over the LAN (and, if properly configured, over the Internet as well). I use it in order to keep my music synced between my laptop and my PC, keep my configuration files up-to-date, and to copy anything I need from one device to the other. There are a few choices of commands you could use, two of which would be scp (secure copy), and rsync. I'll be focusing on rsync in this article, because it offers progress information, update features, and useful switches like –ignore-existing.
Dans le numéro 37, j'ai écrit un article sur la configuration d'un serveur SSH sur votre ordinateur, pour l'utiliser comme un proxy SOCKS. Mais comme j'imagine que tous les lecteurs ne veulent pas l'utiliser comme tel, j'ai décidé de me concentrer sur la commande que j'utilise le plus en deuxième (la première commande que j'utilise est « pacman », la solution ArchLinux pour gérer les paquets). Avant d'aller plus loin sur ce qu'est cette commande, je vous expliquerai brièvement pourquoi vous pourriez intéressé par cette solution. En particulier, elle vous permet de synchroniser les répertoires et les fichiers entre deux ordinateurs à travers le LAN (et, s'il est configuré correctement, aussi à travers Internet). Je l'utilise pour pour garder ma musique synchronisée entre mon portable et mon PC, pour garder mes fichiers de configuration à jour, et pour copier tout ce dont j'ai besoin d'un appareil à l'autre. Il existe quelques commandes qui pourraient être utilisées, deux d'entre-elles sont scp (secure copy) et rsync. Je m'attacherai à décrire rsync dans cet article, car il offre l'information de progression, une capacité de mise à jour, et des options très utiles comme –ignore-existing.
to do
A few of you may be asking why I don't just use Dropbox, an external hard drive, or a USB stick (for smaller files). The answer is quite simple: Dropbox offers a limited amount of space, and the other options require me to remember to do this regularly. If you have SSH configured on your “sender” (in this case, my PC from which I transfer the files), and an SSH client (no server required) on your “receiver” (my laptop, in this case), then you can easily write a small script to run a cron (in other words, regularly, and without any input). If you want to automate this, you will need to configure SSH to use keys instead of passwords, so that you can access your server without having to input anything. This is fairly simple (using ssh-keygen to create the keys, and then copying the public key to the server), and is explained in plenty of places (see the Links section below for a link to a Wiki).
Once you have SSH configured, it's time to think about how the script should appear. I won't supply an example script, simply because I haven't implemented a decent one yet. There are some things you should take into consideration when designing your script, such as: The script should only do something if you're on your home network (this can be done by checking the ESSID of your wireless, or, if you connect your laptop to the LAN by cable when at home, checking if eth0 is active, or simply deciding on a specific time the script should run). The reason for this is because otherwise you'll have lots of failed SSH connections when doing this in a location besides your home network. I recommend thinking about your habits, and finding a solution that works best for you. Then write it into an if-statement in the script.
How many files/directories do you want to sync, and which ones exactly? You can either hard-code each file or directory into the script, or create a text-based list of locations on your machines, and then use a while statement and readline to handle each line separately. A few files I would recommend: .bashrc (or your rc file for the shell you use), .Xdefaults (for terminal colors), Music, Pictures, any configuration files for window managers (XMonad, DWM, etc.) Do you want to update (meaning newest copies of the files are the ones to keep), or ignore files if they already exist on the receiver (useful for music and pictures), do you need to be recursive (that means following a directory tree). There are some other useful options to consider that rsync offers (see the second section of this article). Is the destination folder and the source folder in the same location? If not, you'll need to keep track of where each file is supposed to go (similar to #2).
Space – do you have enough space on your receiver for all the files from your sender, and, if not, what are you going to do about it? You can either reduce your list of sync files, or build a check into the script using df -h to set a limit (i.e. if there are only 9GB left, stop syncing entirely, and email you/prompt you). Once you've taken these points into consideration, it's time to write the script. I recommend you have at least 2 checks in the script (if you're connected to the right network, and if the sender is currently online). The rest of the script is entirely up to you, including when and how to run it. Back in issue #24, I wrote an article on cron, and since then have used plenty of examples, so I will only briefly discuss your options. When configuring the cron job, you can either dump the script in /etc/cron.hourly, or /etc/cron.daily. The other option is to edit your crontab (crontab -e <username>). In the crontab you can then create a line for the script that either runs every set number of hours/days, or set it to run at a specific time (or a specific date), and so on. I think a script that runs once a day is going to be quite enough for this.
rsync As you can see from point 3 above, rsync offers a lot of checks to avoid copying more files than necessary. Some useful ones are: -u (–update): Skips files that are newer on the receiver –inplace: updates files in-place –append: adds data on to the ends of shorter files -x: Avoid crossing filesystem boundaries (i.e. stick to one partition) –existing: Do not create new files on the receiver, only update existing files –ignore-existing: Ignore files that already exist on receiver –max-size=SIZE: Don't copy any files larger than this (–min-size also exists, though less useful in this case) –exclude=PATTERN: Excludes any file matching the pattern –exclude-from=FILE: Reads the pattern(s) from the file –partial: Keep partially copied files
Some other useful switches for rsync: –delay-updates: Puts updated files into place at the very end. -r (–recursive): Follows directory trees. -d: Copy directories without recursing (by default rsync doesn't enter any directory at all) -l (–links): Copy symlinks as symlinks -E (–executability): Keep files executable (useful for scripts) -h: Human-readable sizes and output –progress: Display a progress bar for each file For the full list, check rsync's manpage. The basic format for rsync commands is: rsync <switches> <source> <destination>
So, if I wanted to update all files from ~/scripts on my PC with ~/.bin on my laptop, I'd write: rsync -ru lswest@127.0.0.1:/home/lswest/scripts ~/.bin This will then copy it over. Logically, you'll want to use the actual IP of your PC instead of the localhost IP, but this is only an example. As we round off this article, I'd like to make a few notes on off-site syncing: Syncing over the Internet, while useful, should be kept to a minimum, simply because the traffic, while encrypted, will be rather large, and might cause issues with an admin, or any kind of data limit you might have. Also, ssh keys are (generally) more secure than passwords, so I highly recommend using them wherever possible.
If there is a large influx of requests for an actual example script, I will happily deliver it next month. I do, however, recommend you try writing your own, or customize any example scripts you find to suit your needs. If you're of the opinion you'd like one, please let me know in an email (address is below). If you have some concrete questions about a script you're writing yourself, you're also welcome to email me about it. If anyone has questions, concerns, or simply wants to share a script they've implemented, feel free to email me at lswest34@gmail.com. If you do email me, remember to include C&C or FCM in the title, so that I don't overlook it. Links: https://wiki.archlinux.org/index.php/SSH_Keys#Generating_an_SSH_key_pair