Laptops are the nightmare of any system-manager. Not only they are always a special case, therefore taking the same amount of work as 50 identical workstations, but also (and more importantly), their backup is under the only responsability of the user. The files on a laptop are much more in danger than those on a workstation or on a file server: the laptop HardDisk is smaller and more fragile, the laptop can get stolen or lost, laptops get into many different uncontrolled networks…. Nevertheless, we know that very few users backup their data on a regular basis. As Laptops become more and more convenient to use, the important work data also becomes more and more in danger. It is useless to have a sophisticated centralized backup system if users keep their important data on their laptop. So please backup your work data as often as possible. Afterall, your work data belongs to EPFL.

All Employee of EPFL are required to do regular backups of the laptop computer they use
For this reason EPFL provide a backup service (Druva InSync) free of charge that every user can request to use, if he/she do not have other solutions. to get your Druva InSync account, please contact your system administrator; once installed and configured, the program will backup your files every 4 hours even if you're not connected to EPFL networks.

The lab doesn't provide direct support to personal laptops, but you can use different services of our filer structure to your gain. The use of these services as backup system is under your only responsibility, we don't control what you do and how you do it.

1. Homedir: the home directory is to store your work related documents/programs/stuff, but you can use it also to store some personal related data (protos/images, videos, documents) if the footprint is relatively small (under 100 GB).
2. Scratch server: The scratch server provide space for everyone and there's no limit on the use of it. plesae use it as you like, but adopt a fair play policy, as everyone in the lab will need to store data.
3. google disk/dropbox/microsoft onedrive: these and others internet services are offering storage space for your personal data, but thir storage/use policy is against the rules of epfl and/or the European GDPR. We do not recommend using them.

Unison is a very fast, efficient and easy to use file-synchronization tool for Unix (Linux and Apple OsX) and Windows.

Note that, although very convenient, Unison is not a real backup tool: it does not keep an history of files, and it tends to replicate mistakes instead of repearing them (a file deleted by mistake on one machine will be deleted also on the other machine). On the other hand, we keep various backup of your home directory on the server. Therefore keeping your laptop in sync with your directory on our file-server is almost like backing it up.

Here we show how it can be used to keep the work stuff on your laptop in sync with your home directory on the file server. Here is a short check list of the things to do:

1. install unison
2. cleanup your directories
3. chose and/or prioritize what to synchronize
4. decide what to exclude from synchronization
5. setup unison configuration files and startup scripts

You can either download it directly from the official web site, or use a precanned package from your favorite distribution. Unison is included in most linux distribution and on both fink and mac ports for Apple OS X. For a nice feedback from the backup scripts, we also suggest to install the growl notifier on OS X and libnotify on Linux.

Let's classify our files as static or dynamic files, and as personal or professional. Static files are created once and modified at most very rarely as, e.g. pictures, mp3 files, or pdf of downloaded papers. Dynamic files are those that you edit often as, e.g. the latex source files of your latest revolutionary scientific paper, your program sources as well as your contact list. Off-course dynamic files are those that need particular attention and a more frequent backup. Luckly, dynamic files are typically much smaller than static ones. In fact, very often most of the space on the disk is taken by personal static data (pictures, movies and music collection).

<note> Please do not use our server for storing your personal static data. Buy yourself an external Firewire or USB hard-disk instead. The backup of our home directories on tapes takes a long time and the tapes have limited capacity. It is impossible to backup tens of GigaBytes of non-work-related data for every user. The largest the stuff to backup the less frequent and safe the backups are. Please, respect the other users by keeping our disk usage on the servers as limited as possible. </note>

We suggest to use two different policies (two separate configuration files) for synchronizing your data.

1. a frequent (hourly) and automatic synchronization for all your dynamic data (we accept here also your personal files if they are not too large);
2. an on-demand or less frequent for professional static files.

Clearly any file that is derived from another one (e.g. compiled files like object .o files, and latex output .dvi files) should be excluded from any backup. Strictly speaking, synchronization is not a backup but excluding non source files will speedup your synchronization process.

Version control systems can be seen also as synchronization systems. Therefore I (Giovanni) personally exclude also all the files that are under centralized version control (SVN) by adding the *_svn pattern to my exlude list and appending _svn to the name of any directory containing svn work directories. Note that this is a potentially dangerous thing (see below) and I will probably stop this practice as soon as I've migrated all my svn repositories to git.

Unison is easy to use, once the configuration file is set.

$ unison -batch myPolicy

where myPolicy is the name of the policy to use and is described in a file with the same name under the $HOME/.unison directory.

Here is how my main configuration file looks like:

# Roots of the synchronization
    root = /Users/cangiani
    root = ssh://cangiani@algosrv5.epfl.ch/

# Here is what I want to keep in sync
    path = Archives
    path = Documents
    path = Learn
    path = Notes
    path = Projects

    ignore = Path Documents/Local

# Some regexps specifying names and paths to ignore
    ignore = Name temp.*
    ignore = Name *~
    ignore = Name .*~
    ignore = Name .o~$
    ignore = Name *.{o,x}
    ignore = Name *.{tmp,aux,log,dvi}
    ignore = Name *.sparseimage
    ignore = Name .DS_Store

# always use rsync for sending files
    rsync = true

# first treat smaller files
    sortbysize = true

# On Mac the default FS is case insensitive   :(   
#    ignorecase = true

# Log actions to the terminal
    log = true

# Specific settings
    key = 1
    label = Learn directory
    batch = false

Please refer to the official documentation for all the details.

<note warning> Be careful with ignore files!!!

Here is an extract from the official documentation:

It is important to be a little careful when renaming directories containing “ignore”d files. For example, suppose Unison is synchronizing directory A between the two machines called the “local” and the “remote” machine; suppose directory A contains a subdirectory D; and suppose D on the local machine contains a file or subdirectory P that matches an ignore directive in the profile used to synchronize. Thus path A/D/P exists on the local machine but not on the remote machine.

If D is renamed to D' on the remote machine, and this change is propagated to the local machine, all such files or subdirectories P will be deleted. This is because Unison sees the rename as a delete and a separate create: it deletes the old directory (including the ignored files) and creates a new one (notincluding the ignored files, since they are completely invisible to it). </note>

Now that you've prepared and tested your perfect configuration file, it is time to make sure that unison is executed periodically. On unix (linux and mac) you can symply call unison from a script like the following:

#!/bin/sh

# make sure to use always the same hostname even if the actual address of the laptop is variable
export UNISONLOCALHOSTNAME=giovanniMBP

# make sure to be at EPFL connected on a wired connection
ifconfig | grep 128.178.70 > /dev/null 2>&1
if [ $? eq 0 ] ; then
  unison -batch main
fi

and launch the script periodically as a cron job. Edit your cron table (crontab -e) and add something like

38 8-20/2 * * * /Users/cangiani/Desktop/unison.command 2>&1 > /dev/null

which runs the script every 2 hours between 8:38 am and 8:38 pm.

Note that the line

export UNISONLOCALHOSTNAME=giovanniMBP

of the above script is quite important because the database used by unison for keeping track of which files are changed since last sync depends on the host-name (address) of your machine. Since the laptop gets a different ip address and, therefore, a different host-name each time it is connected to the network it happens often that you have false conflicts: a file is supposed to be changed on both machines only because it was synced on a previous run with a previous host-name.

Here is a nicer version of the launch script that keeps you notified about the result of your automatic synchronization:

#!/bin/sh

export UNISONLOCALHOSTNAME=giovanniMBP

ifconfig | grep 128.178.70 > /dev/null 2>&1
if [ $? -eq 0 ] ; then
  ofile=$(mktemp /tmp/unison.XXXX)
  unison -batch main 2>$ofile 1>&2
  if [ $? -eq 0 ] ; then
    msg=$(grep "Synchronization complete" $ofile; grep "Nothing to do:" $ofile)
#    DISPLAY=:0.0 notify-send -u low "Unison Sync OK" "$msg"                  # linux version
    growlnotify --title "Unison Sync OK" -p Low -m "$msg"       # mac version
  else
    msg1=$(grep "Synchronization complete" $ofile)
    msg2=$(grep '<-?->' $ofile | grep "new file" | head -n 1)
    msg3=$(grep '<-?->' $ofile | grep "changed" | head -n 1)
#    DISPLAY=:0.0 notify-send -u critical -t 0 "Unison Sync Err" "$msg1 $msg2 $msg3"        # linux version
    growlnotify --title "Unison Sync Err" -s -p High -m "$msg1 $msg2 $msg3"   # mac version
  fi
  rm $ofile
fi

Here is the file I actually use.