I want to show how certain tasks can be automated easily step by step using just a few tools. Today’s example will be about web automation. will be used, but any other language would do.

To get things done, the important part is to choose the right ready-made components. For Perl, these can be found on the CPAN. When looking for modules, one has to be aware that there is both good and crap modules out there. Check the reverse dependencies, recent releases and maybe the ratings. A quick check on the search engine might also tell if the module is proven tech.

For web automation, the module needed is WWW::Mechanize. Often it will already be packaged by your distribution so on Debians, install libwww-mechanize-perl and on openSUSE get the perl-WWW-Mechanize package.

More to learn for SuSE users: For acquiring SuSE packages, the web interface on opensuse.org will find packages in the build service and provide one click install. Pro-tip for text mode: If you install the osc package, finding Perl packages gets yet easier: The command osc se 'perl(WWW::Mechanize)' will find the package containing this Perl module.

The work can begin. First you should know which site to automate. Often, it requires filling out of forms. Then, open the site in your web browser and open a terminal. WWW::Mechanize comes with a little tool to help us get started. Just type mech-dump http://yourwebsite to get a list of the form fields:

GET http://www.google.de/search [f]
  ie=ISO-8859-1                  (hidden readonly)
  hl=de                          (hidden readonly)
  source=hp                      (hidden readonly)
  q=                             (text)
  btnG=Google-Suche              (submit)
  btnI=Auf gut Glück!            (submit)
  gbv=1                          (hidden readonly)

there we have a possible form to fill out and can also peek at the hidden values. Nothing that the “View Source” button cannot do, but more convenient.

By the way, if you are seeing garbled output for national characters and are using UTF-8 in your terminal, then you need to fiddle with the Perl Unicode settings. This can be done by calling PERL_UNICODE=S mech-dump http://www.google.de instead. S will turn on Unicode on standard input and output, latter being your terminal. See man perlrun for more details.
For an amazing write-up about the state of Unicode and its caveats, see this post by Tom Christiansen. You will need to install the Symbola font to see this camel:

Welcome to the second edition of automation and having fun in text-mode only. We will build a small monitoring system and integrate it with a shell script. Because POSIX sh is boring and the script doesn’t need to work on any limited platform, we will use shell

So, the first line of the script is:

#!/bin/zsh

next, we call the program whose output should be monitored. The idea is that we want to get informed whenever there is a change in the output. Of course, you can fill in whatever criterion suits your needs here. The output is then stored to a temporary file:

perl googl.pl > googl-result.$$

the resulting output is compared to the output from last time the program was run, and if there is a change, notification should happen. the cmp program is used to check if two files are equal. additionally, we figure notification should only get sent when there actually is an output. a copy of the message should be logged to the terminal. so, the following code is crafted

if { ! cmp -s googl.output googl-result.$$ } {
	if [[ -s googl-result.$$ ]] {
		date
		cat googl-result.$$
		perl send-notification.pl "$(<googl-result.$$)"
	}
}

note that so far this is a place-holder, nothing has been thought up as how to deliver the notification. That’s the next topic

to finish off the script, we should store the newly recorded output as last output. Then, we wait a while before restarting the script for the next iteration

mv googl-result.$$ googl.output
sleep 60
exec $0

(first part: plugwork in textmode)