Canonical Voices

Posts tagged with 'configuration'

mandel

It is not a secret that I love Spring.Net, it just makes the development of big application a pleasure. During the port of Ubuntu One to Windows I have been using the framework to initialise the WCF service that we use to provide other .Net applications the ability of communicating with Ubuntu One. Yes, this is our DBus alternative!

The idea behind using WCF is to allow other applications to use the different features that Ubuntu One provides, the very first application that we would like to use this would be Banshee on Windows (I have to start looking into that, but I have too much to do right now). In order to provide this functionality we use named pipes to allow the communication, there are two reasons for this:

  • For an application to host a WCF service that uses a binding besides the named pipe binding requires special permissions. This is clearly a no no for a user application like Ubuntu One.
  • Named pipes are dammed efficient!!! Named pipes on Windows are at the kernel level, cool :)

Initially I though of hosting the WCF services as a Windows services, why not?!?! Once I had this feature implemented, I realized the following. It turns out that while impersonation does get spawn within different threads, this is not the case for processes. This is a major pain in the ass. The main reason for this being a problem is the fact that if an application is executed in a different user space, the different env variables that are used are those of the user executing the code. This means that things like your user roming app dir will not be able to use, plus other security issues.

After realizing that the WCF services could not be hosted on a Windows service, I moved to write a work a round that would do the following:

  1. Configure the WCF services to use named pipes only for the current user.
  2. Start a console application that will host the WCF services.
  3. Start the different WCF clients for Ubuntu One (currently is our clietn app, but should it could be your own!

Although the definition of the solution is simple, we have to work around the issue that up ’til now all our WCF services were defined through configuration and were injected by the spring.net IoC. Usually you can change the location of you app domain configuration by using the following code:

AppDomain.CurrentDomain.SetData(“APP_CONFIG_FILE”,”c:\\ohad.config);

In theory wth the above code you can redirect the configuration to a new file, and if you use for example:

System.Configuration.ConfigurationSettings.AppSettings["my_setting"]

you will be able to get the value of your new configuration. Unfortunatly, the Spring.Net IoC uses the ConfigurationManager class which ignores that setting… Now what?

Well, re-writting all the code to not use Spring.Net IoC was not an option because it means changing a lot of work and does mean to move from an application where dependencies are injected to one were we have to manually init all the different objects. After some careful though, I move to use a small CLR detail that I knew to make the AppDomain that executed our code to use the users configuration. The trick is the following, use one AppDomain to start the application. This would be a dummy AppDomain that does not execute any code at all but launches a second AppDomain whose configuration is the correct one and which will execute the actual code.

In case I did not make any sense, here is an example code:

using System;
using Canonical.UbuntuOne.Common.Container;
using Canonical.UbuntuOne.Common.Utils;
using log4net;
 
namespace Canonical.UbuntuOne.ProcessDispatcher
{
 
    static class Program
    {
        private static readonly ILog _logger = LogManager.GetLogger(typeof(Program));
        private static readonly ConfigurationLocator _configLocator = new ConfigurationLocator();
 
        /// <summary>
        /// This method starts the service.
        /// </summary>
        static void Main()
        {
 
            _logger.Debug("Redirecting configuration");
 
            // Setup information for the new appdomain.
            var setup = new AppDomainSetup
            {
                ConfigurationFile = _configLocator.GetCurrentUserDaemonConfiguration()
            };
 
            // Create the new appdomain with the new config.
            var executionAppDomain = AppDomain.CreateDomain("ServicesAppDomain",
                AppDomain.CurrentDomain.Evidence, setup);
 
            // Call the write config method in that appdomain.
            executionAppDomain.DoCallBack(() =>
            {
                _logger.Debug("Starting services.");
                // use the IoC to get the implementation of the SyncDaemon service, the IoC will take care of 
                // setting the object correctly.
                ObjectsContainer.Initialize(new SpringContainer());
                var syncDaemonWindowsService = 
                     ObjectsContainer.GetImplementationOf<SyncDaemonWindowsService>();
                // To run more than one service you have to add them here
                syncDaemonWindowsService.Start();
                while (true) ;
            });
 
        }
    }
}

Well I hope this helps someone else :D

Read more
pitti

Requirements

Through the last years I have used various own hacks for backing up my desktop(s). There are dozens of packaged backup solutions in Debian/Ubuntu already, but none of them did quite fit my requirements:

  • KISS! no fancy web UI, storage formats, or millions of plugins and configuration files; backups should just be a normally accessible directory
  • Supports standard backup strategy: daily backups for last week, weekly backups for last month, permanent monthly backups. This must not require my computer to be switched on all the time.
  • Runs as my own user, so that I don’t need to set it up each time I reinstall my box
  • No interactivity; any backup solution that requires me to do anything regularly is doomed to fail.
  • Push-style backup to my server through ssh (or derived, like scp or rsync)
  • Supports per-directory filtering to avoid backing up unnecessary stuff; my upload bandwidth is very small. (e. g. I don’t want to include ~/.cache and in ~/evolution I want to ignore the cache subdir).

rsnapshot

I have used rsnapshot as a basis for about a year now. It was originally intended to be used with pull-style, but that does not work for home setups behind a NAT. But it’s easy to use push-style with it (details later). It is by and large a fancy wrapper around good old trusted rsync, which is why I liked it from the start: It by and large just creates a full tree copy of your data for each snapshot, and uses hardlinks to avoid duplicate files. So restoring is easy and robust, you can use any file browser to get to your data.

File selection

If in doubt, backup should include a file rather than exclude it. I value completeness over small storage size, and I just check the volume of a snapshot from time to time to ensure that it doesn’t grow too big (it’s currently in the magnitude of 200 MB, which is small enough for daily deltas to be pushed through a slow DSL uplink without much pain). So my approach is to backup everything in /home/martin except explicitly configured files. For configuring the blacklist I use per-directory .rsync-filter files (which have builtin support by rsync).

Excerpt of my ~/.rsync-filter:

# global ignores
- *.log
- *.cache
- .*.swp
- .swp
- .*.lock
# only direct subdirectories/files
- /.ICEauth*
- /.Trash
/.aptitude
/.ccache
[...]
/download
/ubuntu
[...]

(In case you wonder, everything in ubuntu/ is either in the Ubuntu archive or in bzr, so no need to include this.)

Another example is ~/.Private/mozilla/firefox/t3znsw4q.default/.rsync-filter:

- /url*.sqlite*
- /*.bak
- /Cache
- /adblockplus
- /OfflineCache

Please see man rsync, section “FILTER RULES” for the details of the syntax.

Having and maintaining a sensible arrangement of your home directory is by far the most difficult aspect of backup, if you need to be stingy with bandwidth.

rsnapshot configuration

You need a central configuration file ~/.rsnapshotrc. The most important settings are the paths to back up, the destination directory, and the modes (daily/weekly/monthly). In addition I include my crontab into the backup, and add a post-backup action to rsync the backup tree to my server. Backups go to /var/backups/$USER on my systems, which is a different partition than /home (can’t stress that enough; today’s file systems are good, but not infallible).

config_version 1.2
snapshot_root /var/backups/martin
cmd_rsync /usr/bin/rsync

link_dest 1
one_fs 1
lockfile /home/martin/.rsnapshot.lock
rsync_long_args -F –delete –numeric-ids –delete-excluded
cmd_preexec /bin/sh -c ‘crontab -l > ~/.crontab’
cmd_postexec /bin/rm ~/.crontab
cmd_postexec /usr/bin/rsync -e ’ssh -i /home/martin/.ssh/id-backup_rsa’ -aHzvPy –delete /var/backups/martin/ piware.de:backup/tick-home

interval daily 7
interval weekly 4
interval monthly 6

backup /home/martin martin-home

cronnery

The last piece of the puzzle is a script which calls rsnapshot regularly with the desired mode. I wrote a small shell script (which lives in ~/bin/backup) which determines the age of the last daily/weekly/monthly backup, and calls rsnapshot with the correct mode argument. It doesn’t do anything if the last backup was done less than a day ago, so it’s designed to be called very often.

The actual cron job just needs to call it every hour:

$ crontab -l
# m h dom mon dow command
05 * * * * $HOME/bin/backup >/dev/null

And voila, from now on I have e. g. yesterday’s backup on piware.de:backup/tick-home/daily.1/.

Read more