Automatic backup script in Perl

An automatic backup script is a very handy tool so I can "forget" the backup task everyday. It's also an exercise after I read Perl tutorial. Now, it's scheduled and run everyday via crontab on my Linux server at home.

Basically it's a lightweight backup tool. First it archive files to backup. Then it compares to last backup archive. If it's different, the archive is ftped to a remote server. A brief log and a detailed log are generated meanwhile.

It works pretty well and stable. The source code is shown as following.

#!/usr/bin/perl

use strict;
use warnings;

my ($logdir, $log, $detailedlogdir, $detailedlog, $lastsuccessfilename);
my ($targetdir, $target, $targetprefix, $targetnameonly);
my $cbserver;
my @source;
my ($timeforfilename, $timestring);

sub get_timestring {
my ($sec, $min, $hour, $mday, $mon, $year, $wday, $yday, $isdat);

($sec, $min, $hour, $mday, $mon, $year, $wday, $yday, $isdat) = localtime;

$year+=1900;
$mon = sprintf("%02d", $mon+1);
$mday = sprintf("%02d", $mday);

$timeforfilename = $year.$mon.$mday;
return $year."/".$mon."/".$mday."-".$hour.":".$min.":".$sec;

}

sub print_usage {

print "usage: squarrel daily|weekly|monthly\n";

}

sub get_argv {

if ($#ARGV!=0) {
&print_usage;
exit;
}

if ($ARGV[0] eq "daily") {
@source="..."; # file or directories to backup on daily base.
} elsif ($ARGV[0] eq "weekly") {
@source="..."; # file or directories to backup on weekly base.
} elsif ($ARGV[0] eq "monthly") {
@source="..."; # file or directories to backup on monthly base.
} else {
&print_usage;
exit;
}

$targetprefix=$ARGV[0];

}

sub init {

$logdir = "/var/www/html"; # log is written in httpd directory so I can check it via browser
$log = "$logdir"."/emc.log";
$detailedlogdir = "/home/emc";
$detailedlog = "$detailedlogdir"."/emcdetailed.log";

if (-e $log) {
open LOGFILE, ">> $log" or die $!;
} else {
open LOGFILE, "> $log" or die $!;
}

if (-e $detailedlog) {
open DETAILEDLOG, ">> $detailedlog" or die $!;
} else {
open DETAILEDLOG, "> $detailedlog" or die $!;
}

print LOGFILE "----------------------------------------------------\n";
print DETAILEDLOG "----------------------------------------------------\n";

$timestring = &get_timestring;
print LOGFILE "$timestring backup($targetprefix) starts\n";
print DETAILEDLOG "$timestring backup($targetprefix) starts\n";

$targetdir = "/home/bt/crossbackup";
# $targetnameonly = "backup".$timeforfilename.".tar.bz2";
$targetnameonly=$targetprefix.$timeforfilename.".tar.bz2";
$target = "$targetdir/".$targetnameonly;

$lastsuccessfilename = "$detailedlogdir"."/lastsuccess_".$targetprefix;
$cbserver = "..."; # remote server address

}

sub archive {
my ($archivecmd, $result);

$archivecmd = "tar cvfj $target @source";

$timestring = &get_timestring;
print LOGFILE "$timestring $archivecmd\n";
print DETAILEDLOG "$timestring $archivecmd\n";

system("$archivecmd >>$detailedlog 2>> $detailedlog");
$result = $?;
$timestring = &get_timestring;

if ($result == 0) {
print LOGFILE "$timestring archiving finish successfully\n";
print DETAILEDLOG "$timestring archiving finish successfully\n";
} else {
print LOGFILE "$timestring archiving fail $result\n";
print DETAILEDLOG "$timestring archiving fail $result\n";
}

return $result;
}

sub check_identical {
my $lastsuccesstarget;
my $different;

if (-e $lastsuccessfilename) {
open LASTSUCCESS, "$lastsuccessfilename" or die "Write $lastsuccessfilename";
$lastsuccesstarget=;
chop($lastsuccesstarget);
close LASTSUCCESS;

$timestring = &get_timestring;
print LOGFILE "$timestring compare $target $lastsuccesstarget\n";
print DETAILEDLOG "$timestring compare $target $lastsuccesstarget\n";

system("diff -q $lastsuccesstarget $target >> $detailedlog 2>> $detailedlog");
if ($?) {
$timestring = &get_timestring;
print LOGFILE "$timestring $target is different to $lastsuccesstarget\n";
print DETAILEDLOG "$timestring $target is different $lastsuccesstarget\n";
return 1;
} else {
$timestring = &get_timestring;
print LOGFILE "$timestring $target is identical to $lastsuccesstarget\n";
print DETAILEDLOG "$timestring $target is identical to $lastsuccesstarget\n";
system("rm -f $lastsuccesstarget\n");
$timestring = &get_timestring;
print LOGFILE "$timestring remove $lastsuccesstarget\n";
print DETAILEDLOG "$timestring remove $lastsuccesstarget\n";
return 0;
}
} else {
return 1;
}

}

sub ftp_upload {
my $ftpcmd;
my $result;

$ftpcmd = '"user username password\nbinary\nput '.$target.' '.$targetnameonly.'\nquit\n"';

$timestring = &get_timestring;
print LOGFILE "$timestring ftp -v -d $cbserver <<$ftpcmd\n"; print DETAILEDLOG "$timestring ftp -v -d $cbserver <<$ftpcmd\n"; system("echo -e $ftpcmd | ftp -v -d $cbserver>> $log 2>> $log");
$result=$?;

$timestring = &get_timestring;
if ($result == 0) {
print LOGFILE "$timestring uploading finish successfully\n";
print DETAILEDLOG "$timestring uploading finish successfully\n";
} else {
print LOGFILE "$timestring uploading fail $result\n";
print DETAILEDLOG "$timestring uploading fail $result\n";
}
return $result;
}

sub done {
my ($result, $stage) = @_;

$timestring = &get_timestring;
if ($result == 0) {
open LASTSUCCESS, ">$lastsuccessfilename" or die "Write $lastsuccessfilename";
print LOGFILE "$timestring backup($targetnameonly) succeed\n";
print DETAILEDLOG "$timestring backup($targetnameonly) succeed\n";
print LASTSUCCESS "$target\n";
close LASTSUCCESS;
} else {
print LOGFILE "$timestring backup($targetnameonly) failed in stage $stage\n";
print DETAILEDLOG "$timestring backup($targetnameonly) failed in stage $stage\n";
}
close LOGFILE;
close DETAILEDLOG;

}

my $result;

&get_argv;
&init;

$result = &archive;
if ($result) {
&done(1, "archive");
exit (1);
}

$result = &check_identical;
if ($result==0) {
&done(0, "");
exit(0);
}

$result = &ftp_upload;
if ($result) {
&done(2, "upload");
exit (2);
}

&done(0, "");
exit(0);

Try Emacs

Emacs is the favorite editor of my leader, as well as my leader before. It looks handy and extensible. From Google Code, there is an emacs backend to enable navigation between source files. That's a good example that Emacs can be extended and customized for my need. I would like to have a try.

I read emacs build-in tutorial. It's basic but enough to start. There is a comprehensive manual from GNU emacs official website.

For starting, it's just different commands to vi or other editors. Just remember it.

^g reset emacs commands in case of hanging

^a / ^e move to the beginning or end of the line
ESC-< / ESC-> move to the beginning or end of the file
^k kill from cursor position to end of line
^ [SPACEBAR] or ^@ mark block
^w cut
ESC w copy
^y yank it back or paste

^s search or search again
^r reverse search or reverse search again
ESC % replace (type 'y' to confirm replacement)
No [RET] after entering search string

^x ^f open a file
^x ^s save current buffer
^x ^c exit
^x ^b list all buffers

^x u or ^_ undo
^u 8 repeat command for 8 times
^x 1 / 2 / o get rid of all other windows / split current window into two /switch to another window

All these commands work for both windows and Linux.

PickupPerl & Screen

I read through PickupPerl in a few days. It's a simple but good starting point. As I am familiar with C programming, I only list what is important to me. As an exercise to study, I started to write a script for my data backup. Meanwhile I found some other good tutorials with more details @www.perl.com, an official perl website.

I also found a pretty handy tool, screen, a terminal manager. Just remember a few often used commands. There is a good introduction @ http://jmcpherson.org/screen.html

Perl

1. The first example. Just remember it.

#!/usr/bin/perl

use strict;
use warnings;

print "What is your username? ";
my $username;
$username = ;
chomp($username);
print "Hello, $username.\n";

2. Scalar variable can be either a string or a number. And a scalar variable is not defined as a string or a number. Perl does automatic conversion between string and number.

3. Single-quoted string. no special character except ' and \. Note \' and \\ is replaced but backslash followed by something else keeps both. For instance \o means \o, two characters.

4. New line can be embedded into a single-quoted string.

5. Double-quoted string is similar to that in C except that $ and @ are special characters and need backslash.

6. For number, you can use underscore for readability. It's just for readability.

7. Scalar variable support interpolation.

8. Undefined variable is allowed and assigned a special value 'undef'.

9. 0, undef, "" means false and others means true.

10. <=> cmp, . string concatenation, x string duplication.

11. Names are case sensitive.

12. Array(@) can grow and shrink dynamically as needed by setting $#array. When an array is growing very large very quickly, it can be a bit inefficient. You can pre-build an array of certain size instead. $#array: subscript of the last element in the array.

13. Everything in the array must be a scalar. No 2-D array. Array in another array simply concatenate each other.

14. Array quoted by () or qw//. Use .. with slice.

16. Push, pop at the end of array, shift/enshift at the beginning of array.

17. blocks are quoted by {}. Variables declared in the block is valid only in the block.

18. if/elsif/else, foreach. foreach my $item (@collection) { print "$item\n"; }

20. hashes(%) are not subscripted by numbers but an arbitrary scalar value. A hash is a list of value pairs, key and value. keys/values %hash return array of key or value.

22. Regular expression(quoted by //). * zero or more of the previous character; . any; | or ; () grouping; ^ start of string; $ end of string; =~ for pattern matching. \d match [0-9]. check 'perlre' for more.

24. Subrouting. sub subname { xxx; }, return; parameter. special array variable @_.


Screen

C-A c create
C-A p/n page forward/backward
C-A 0/1/2.. select a window
C-A d/r detach/retach

Write Perl Script

I did some Shell and Tcl script designing before but apparently it's not enough to write a good backup script. Before starting to play with scripts, I need to decide which script language to use, Tcl or Perl. I searched for 'comparison tcl perl' and got some interesting comments.

Tom Christiansen has a nice comparison. From his view, Perl is a good system programming language while Tcl is a metalanguage. (Metalanguage means that you can create your own language based on it.) Perl is also much more efficient because the intepreter first compile the code and run the intermediate code. Also because of compiling, it's more robust.

I would say that perl is a language designed for high-level, portable, systems programming on tasks of small to medium scope, but with support for enough underlying general-purpose programming constructs to make it suitable for many users who would not characterize themselves as systems programmers. This includes not just systems administrators but also people doing software installations, rapid prototyping, generic data munging, simple client/server programming, customer support, and test suite design and analysis. Another application area for which perl provides an elegant and convenient solution is one which is growing so fast that I suspect even its second-derivative is still positive: World Wide Web script support (a.k.a. CGI programming). I suspect than in a few years, perl may well be better known as a CGI language than it is as a sysadmin language.
(From http://www.perl.com/doc/FMTEYEWTK/versus/tcl-discussion.html)

For me there are two other advantages: similarity to C and that my colleagues know Perl. Richard Stallman is negative to Tcl either from experience of Emacs development. So conclusion is clear now. I choose Perl.

There are quite some free Perl readings on Internet. I downloaded a tutorial called PickupPerl. It's a good reading to start although it can be more compact. Start to read it and backup application is a good exercise.