Full Circle Magazine FR

Ceci est une ancienne révision du document !

Before I begin with the actual article, I thought it might be useful to address a question I got via email. A reader was asking where best to place my wallpapers.sh script, and, in doing so, reminded me that I haven't ever explained how best to go about this. My approach is to store all my scripts in a single folder (either a folder called “scripts,” or, if I want to save some visual space in my home folder, I'll call it .bin or .scripts so the folder is hidden). Make sure any script you place here is executable (chmod +x), and that they specify in the first line which interpreter is required (#!/bin/bash, #!/usr/env python, etc.) - otherwise calling it will not work. Once that is done, you could create a symbolic link to the script in /usr/bin with

sudo ln -s /path/to/script /usr/bin/script

However, this can be complicated if you start removing scripts (it will leave broken links in your /usr/bin directory). Another method is to add the folder into your $PATH variable. This can be done with

export PATH=$PATH:<folder>

(replace folder with the actual path to the folder you created). To make it permanent, you can either place it in your .bashrc, your .xinitrc (if you start your interface with startx), or in any other script that gets called when you log in. I've had some hit-and-miss experiences with this, but generally /etc/environment works well. Since, however, the PATH is created in /etc/profile, you can also simply add the path to the scripts folder at the end of this path. I recommend avoiding any files in /etc/, and instead organize all your scripts within $HOME. This is due to the problems that can arise when creating system-wide settings using files that not all users can access. The problem is that anything in $HOME is restricted access wise to your user account. And now, on to the actual article.

The rest of my family has recently moved over to MacBooks for mobile workstations, and, for most of them, keeping files organized is fairly easy. For one of them, the MacBook is the only computer, and, for the other, it's a work machine where only company-related files need to be synchronized. However, for the latter person, whose PC runs both Arch Linux and Windows, it's a little more complicated. At first, I thought I'd use rsync in a custom script to synchronize the folders one after the other, but, since changes may be made on either machine, I needed a robust method to synchronize them (including deleted files). Rsync may be able to do this, but I have yet to get it working at 100%. As such, I began looking at unison again, which has resulted in me adopting it for my backup needs as well. Below, I describe my method for configuring Unison (which is available from the universe repository).

From the Unison manual “Unison is a file-synchronization tool for Unix and Windows. It allows two replicas of a collection of files and directories to be stored on different hosts (or different disks on the same host), modified separately, and then brought up to date by propagating the changes in each replica to the other.”

To synchronize across multiple machines, the first thing you need to do is configure SSH on one of the machines (I did this on my mother’s PC, to minimize any possible security holes which would arise from a laptop with SSH enabled in public networks). It is also extremely important that you enable key-based logins on SSH. This is fairly easy with the following two commands:

ssh-keygen -b 521 -t ecdsa -C“$(id -un)@$(hostname)-$(date –rfc-3339=date)”

This creates the key. Follow the on-screen instructions. Keep in mind that entering a passphrase requires you to type this in when you use it – which defeats the point of a key-based login.

scp ~/.ssh/id_ecdsa.pub username@remote-server.org:~/.ssh/authorized_keys

This will copy the file into the authorized keys of the server, meaning that if you try ssh <IP>, without a username, it will automatically log you in using the key, without prompting for a password. Note: If your version of ssh doesn't support ecdsa, rsa keys are fine too.

Now then, on to Unison. The easiest way to manage multiple synchronization folders is to create multiple profiles (.prf files). Since the options are all the same, I will cover only a single example. My Music sync profile looks like this:

# Unison preferences label = Music sync root = /home/lswest/Music root = ssh:user@IPhome/lswest/Music/Hyperion/ fastcheck = true dontchmod = true ignore = Name *.ini ignore = Name *.jpg ignore = Name *.jpeg sshargs = -C

From top to bottom, the settings do the following:

label – assigns the name of the profile, for the GUI's list of profiles.

root – These are the sender/receiver (in that order). It shouldn't be possible to use more than 2 roots, though I haven't tested this.

fastcheck – I find that this reduces the search time immensely. It is supposed to be the default for Unix systems, but it doesn't hurt to be certain. The way this works is that it reads a file's modification times during the first pass, so it can ignore any files that weren't changed (meaning the modification times are the same as what was stored in the database). On the second pass. it generates a fingerprint. and compares it to the last contents to be synchronized.

dontchmod – Unison tries to keep permissions the same, which is fine for syncs with Linux/Unix systems on either end, but for anything stored on NTFS (or with a Windows server), you will need to turn this off, to prevent dozens of permission warnings. Leaving this option isn't bad, as the permission settings simply fail, but it does slow down the transfers.

ignore – This option tells Unison what files to ignore. In my case, I ignore all the .ini files that Windows is so fond of creating (since my media is on an NTFS hard drive shared between Windows and Arch on my PC), and any of the cover art, since I use a different method in Linux.

sshargs – passes arguments to the ssh session. The option “-C” tells SSH to compress any of the information being sent, theoretically reducing transfer time per file (I haven't tested to see if this slows the entire process down, since the files need to be compressed, but for my rather large Music folder, it doesn't take too long to synchronize the changes these days).

Keep in mind that the first run-through will take a while, since Unison has to assume every file is new.

The first few times you run Unison, I highly recommend manually checking the sync settings for each file that pops up (options are: right to left, left to right, skip), to make sure you're not losing/gaining any files you don't want. Once you're satisfied with how Unison is handling it, you can configure it to automatically accept non-conflicting changes. Do this by adding “auto = true” to the .prf file. You can also have Unison accept the non-conflicting changes, while skipping any conflicts with “batch = true”. Whether or not you can make this a completely automatic process, I'm not sure. However, you can minimize the input that is required using the above steps. To find out more about Unison, check the Further Reading section for a link to the user manual. If you want to simply mirror a folder in one direction (i.e. copy it from one machine to the next, without synchronizing), then I would still highly recommend rsync instead, due to the fact that it doesn't require any input when run properly. It's also extremely well documented in the manpage, making it quite easy to use. As a final note, I want to point out that both Unison and Rsync work for local folders too, meaning you can use it to create backups on a USB drive as well. If you decide to run these commands automatically on a schedule, I would recommend creating a script to first check if the USB drive is connected, or to check that you have the correct remote IP address (or can ping the router's IP). This is to avoid using resources to run a command that will simply fail. It also avoids opening SSH connections with computers that have the same IP but aren't your intended target, which could lead to problems down the line.

I hope at least some of you have found this interesting or helpful. I'm going to continue expanding my preferences files as I get more comfortable with Unison, and will mention any further tips I discover as time goes on. As always, if you have any questions, suggestions, or comments, you can reach me via email at lswest34@gmail.com. If you do decide to email me, please put “C&C”or “FCM” in the subject line, so that I don't overlook it.