Using Amanda backup with StorReduce

In this article, I’ll show you how to use Amanda backup with StorReduce.

Amanda, the Advanced Maryland Automatic Network Disk Archiver, is a highly configurable, open source backup solution that can be centrally administered from a single server. It supports Linux, multiple versions of Unix (incluing Mac OS X) and has a native Windows client.

Amanda Enterprise is a commercial version of AMANDA distributed by Zmanda in addition to the features of the open source version, it adds commercial support, a GUI management console, support for NDMP and many other features.

Amanda can backup to a variety of targets including tape, disk and systems that implement the Amazon S3 API (or other object storage APIs) like StorReduce.

Configuring Amanda to backup to StorReduce is very easy. If you are comfortable with the Unix command line, it will only take a few minutes.

Getting started with Amanda

If you have already a working Amanda system you can skip to the next section.

To get started with Amanda, I recommend you follow this 15 minute quick start guide, it will guide you though setting up an Amanda server and some clients to backup to disk. Once working you have a working Amanda system you can easily change the backup target to StorReduce.

Working through the guide took me a little longer than 15 minutes, but I fell into a few pitfalls. Here are a few suggestions to help you avoid the same pitfalls:

  • Start with a supported Linux distribution (or perhaps build from soure). I started by trying to install the RPMs for Red Hat 6 from Zmanda on Amazon Linux, this usually works (the distros are quite close) but I ran into a wierd incompatibility between Amanda and system supplied Perl shared libraries. I switched to a RedHat 7 system and the RedHat 7 RPMs from Zmanda and it just worked.

  • Don’t install the client RPMs software on the same server as the server software.

  • If you’re OS uses systemd then use:

    sudo systemctl start xinetd.service
    sudo systemctl status xinetd.service
    

    To start the service.

  • The ammt command no longer exists, use:

    amtape DailySet1 show
    

    to check that the virtual “tapes” are configured instead.

Configuring a StorReduce Backup Target

Important: Don’t compress or encrypt your backups. If you are using Amanda (or any other backup software) with StorReduce it is very important that you turn off any compression or encryption features that the backup solution offers. The reason is that both compression and encryption algorithms produce unique streams of data which can’t be deduplicated. StorReduce will compress and (optionally) encrypt the data after deduplication anyway.

Once you have a functioning Amanda system, configuring it to use StorReduce as a backup target is easy. I reccomend following the instructions for connecting to S3 here in the Zmanda Wiki. I’ll summarize the changes below, but if in doubt you should refer back to the S3 howto.

The main changes can go anywhere in your amanda.conf file:

  1. Turn off the “old” tapedev directive for naming the backup target:

    ## tapedev "chg-disk:/var/amanda/vtapes"
    

    and use the new style tpchanger directive:

    tpchanger "chg-multi:s3:srset1/slot-{01,02,03,04,05,06,07,08,09,10}"
    
  2. Set the device properties like this:

    # StorReduce
    
    # Your StorReduce Access Key ID 
    device_property "S3_ACCESS_KEY" "YOUR_STORREDUCE_ACCESS_KEY_ID"
    
    # Your StorReduce Secret Key
    device_property "S3_SECRET_KEY" "YOUR_STORREDUCE_SECRET_ACCESS_KEY"
    
    # Curl needs to have S3 Certification Authority (Verisign today)
    # in its CA list. If connection fails, try setting this no NO 
    device_property "S3_SSL" "NO"
    
    # Change this to YES if you want amanda to create the bucket in
    # StorReduce
    device_property "CREATE_BUCKET" "NO"
    
    # The hostname of the StorReduce Server
    device_property "S3_HOST" "YOUR_STORREDUCE_SERVER.YOUR_DOMAIN.COM"
    
    # Amanda will create this file
    changerfile  "s3-statefile"
    

    The S3_ACCESS_KEY and S3_SECRET_KEY are the keys issued on the Manage Keys screen in the StorReduce Dashboard.

    S3_HOST is the domain name of your StorReduce server, it must be identical to one of the hostnames in the Hostname field of the Settings tab in the StorReduce Dashboard.

    You can learn about the various S3 API specific parameters here.

  3. Define and set the tape type:

    tapetype SR
    
    define tapetype SR {
        comment "StorReduce Bucket"
        length 10240 gigabytes # Bucket size 10TB
    }
    

Follow the instructions to label the “tapes” and run amdevcheck and then you should be ready to start your first backup.

Performance

In general a StorReduce server will give you the best deduplication performance when it is receiving or sending at least as many streams of data as there are CPU cores available to it. E.g., if you are running StorReduce on a 16 core machine, try to ensure that you are sending at least 16 simultaneous transfers.

For Amanda you tune the NB_THREADS_BACKUP and NB_THREADS_RECOVERY parameters.