Yes, even a Linux machine could suffer from a disaster: Whether it’s because of a corrupt video configuration, a kernel update gone wrong, or a misconfigured init script, it’s inevitable. I’ve seen it happen on a number of occasions — even on my own machines, mostly from corrupt X configurations — and it’s frustrating.

The best rescue plan, in my opinion, doesn’t have to involve reinstalling. Sometimes the best rescue plan doesn’t even involve booting up a rescue disk. This article is going to offer up some tips and tricks on how to avoid failure and help you create the tools you need to recover a dead Linux machine.

Start with the right runlevel

After installing a new Linux system, I immediately take steps to ensure disaster won’t strike easily. One of the first steps is to edit the system’s runlevel. The runlevel tells the system how far to take the boot process. The runlevel is broken down into six levels:

|> 0: halt (do not set initdefault to this)
|> 1: Single user mode
|> 2: Multiuser, without NFS (the same as 3, if you do not have networking)
|> 3: Full multiuser mode
|> 4: unused
|> 5: X11
|> 6: reboot (do not set initdefault to this)

Newer Linux distributions almost always default to runlevel 5 (X11), which means that your system will stop at the graphical log-in screen when boot is complete. This is fine until something (or someone) hoses your X configuration; you will then have to find a means to log in. You could press [Ctrl][Alt][F7] to get a text-based virtual screen, but why go through that hassle? Instead, I always change my runlevel to 3 in the file /etc/inittab. The line you change is:
id:5:initdefault:

which will change to:
id:3:initdefault:

This is a very simple method of saving yourself when X doesn’t work properly.

Multiple kernels

The next obvious rescue aid is to always have a working kernel installed. I usually work from a kernel updated via yum. Kernels have occasionally been released with flaws that have caused one or more of my machines to not boot. To this end, I always make sure I have at least one perfectly running kernel on a machine. A great way to handle this is to first add plugins=1 in your /etc/yum.conf file. The next step is to take this script (written by Jeremy Katz from RedHat) and save it as n-installonly.py in /usr/lib/yum-plugins. You can change the number of kernels to retain on the system by changing the tookeepvariable (default = 2).

With a known working kernel on your system, you can upgrade safely. If the new kernel is hosed, simply boot the old kernel to solve the issue with the new kernel (be it to remove it, recompile it, or update it).

Rescue mode

If you are using Red Hat and the LILO boot loader, you can boot into rescue mode by inserting Disk 1 of your installation and entering linux rescue at the boot prompt. Once the machine has booted, you will land on the bash# prompt. From this mode, you have a number of tools to use.

As you can see, there are tools to check the integrity of a hard disk, repair hard disks, check kernel modules, mount devices, and create file systems, etc. This is a very good place to start with your rescue attempt (if you’re using a Red Hat, or Red Hat-based, system).

The next rescue method is booting into single-user mode, where your computer boots to runlevel 1. Your local file systems will be mounted, but your network will not be activated. You get a usable system maintenance shell. To boot into single-user mode, enter either:
linux single

or
linux emergency

at the LILO prompt.

Creating a rescue CD

If you’re using the LILO boot loader, there’s a great tool called mkrescue. This tool is typically used to create boot floppies, but can be used to create ISOs as well. Here’s how.

If you’re using Mandriva — as root:
mkrescue –iso –initrd /boot/initrd-KERNEL-NUMBER.img –kernel /boot/vmlinuz-KERNEL-NUMBER

Note: Where KERNEL-NUMBER is the actual release number of the kernel.

If you’re unsure what kernel release you’re using, the numbers for intrd and vmlinuz can be found with the following command:
uname -r

After running the command, you will find a rescue.iso file in the directory where you ran the mkrescue command. You can now burn the image with the following commands:

First, check for the number of the CD burn device with:
cdrecord -scanbus

Now burn the images with:
cdrecord dev=0,0,0 rescue.iso

Note: Where dev=0,0,0 is number discovered with the scanbus command above.

If you’re using Slackware, use these steps to make a boot CD:
mkrescue -iso

Note: Slackware automatically knows what kernel to put in the ISO.

You will then burn the image with the same means you did with the Mandriva image.

Because fewer people are using text-mode clients, one might think there is less use for programs like Procmail and Fetchmail. However, the use of these programs is as valid today as it was five years ago, especially on Linux systems that have a full-blown MTA like Postfix or Exim installed by default that can be used easily and with the proliferation of home servers. Imagine for a moment that you have five e-mail accounts, which isn’t uncommon for many people, and a home file/print server running Linux. With a modest amount of work, that server could also be configured to retrieve e-mail from the five accounts on a regular schedule, filter the mail with Procmail, and deliver it to mailboxes then accessed via IMAP on your personal computer.

With some extra effort, a Web-based mail program like Squirrelmail could be installed on the same server providing you with Web — or IMAP-enabled e-mail, from anywhere, containing all the e-mail from your various accounts.

One part of the solution to such a setup would be Fetchmail, a program that has existed for years to do one simple task: download e-mail. The configuration of Fetchmail is quite simple as well.

Fetchmail uses a configuration file in your home directory, namely ~/.fetchmailrc. Because this file contains passwords, it must be mode 0600 so that only the owner can read and write to the file.

The content of the file is very straightforward:

poll mail.server.com protocol pop3 user “joe@joe.com” password “secret” mda “/usr/bin/procmail -d %T” ssl

This is one line of a ~/.fetchmailrc; multiple lines can be configured to have Fetchmail poll multiple mailboxes, including IMAP. This essentially indicates that Fetchmail connects to mail.server.com using POP3 over SSL, with the username joe@joe.com and the password secret. Finally, it tells Fetchmail to deliver to Procmail instead of to the local MTA; depending on whether you wish to filter the mail via Procmail first or let your MTA handle it; you can omit the entire mda argument.

A similar setup for IMAP would be:

poll imap.server.com protocol imap user “joe” password “secret” mda “/usr/bin/procmail -d %T” ssl

The only difference here is that the protocol is IMAP, rather than POP3, although again using SSL. The Fetchmail manpage provides a wealth of information on various configuration options, including examples of how to poll mail over SSH.

Fetchmail can also operate in daemon mode to self-schedule polling. For instance:

$ fetchmail -d 300

This tells Fetchmail to daemonise and to poll for mail every 300 seconds. However, Fetchmail also runs well out of cron, with a script similar to the following:

#!/bin/sh
lockfile=”/tmp/.fetchmail.lock”
nomail=”/nomail”
if [ -f -o -f ]; then
exit 0
fi
touch
fetchmail -a -K >/dev/null 2>&1
rm -f

This simple script could be executed by cron every five minutes or so. It also allows you to simply touch the file ~/nomail to have Fetchmail not attempt to retrieve mail at all, which is useful when you’re debugging something.

With Fetchmail, you can interact with as many mail accounts as you have and can direct the mail wherever you like.

It can be used to encrypt regular files or e-mail, in either binary or ASCII format, and can also verify the integrity of files or e-mail via cryptographic signatures. GPG is a command-line tool and is available with every Linux distribution.

To begin using GPG, you must generate a public/private keypair. This keypair is generated with the -gen-key command:

$ gpg –gen-key

It will create the ~/.gnupg/ directory if it doesn’t already exist, where it will store its configuration file, gpg.conf, and the private and public keyrings where keys are stored, secring.gpg and pubring.gpg respectively, as well as the trust database.

When you generate the initial keypair, you will have to choose the key type. The default is “DSA and Elgamal,” which will allow you to sign and encrypt. You will then have to select a keysize for the key — anywhere between 1024 and 4096 bits. The default is 2048 bits and is sufficient. Next, you will need to determine whether or not the key will expire, and if so, when. A non-expiring key is most convenient, as neither you nor anyone using your public key will have to worry about new keys, however if the key is stolen or compromised, it can then be used indefinitely. Many individuals have keys that expire after one year and generate new keys at that time.

Finally, you will need to provide a user ID for the key which consists of your real name, e-mail address, and an optional comment. The user ID will then end up being “Real Name (Comment)

When the key generation is complete — which may be immediate or may take some time depending on the amount of entropy your system has collected in order to generate random bytes — you can list the keys by executing:

$ gpg –list-keys; gpg –list-secret-keys

You can also view the key’s fingerprint, a unique identifier to the key, with the command:

$ gpg –fingerprint user@domain.org
pub 1024D/9B1386E2 2007-12-01 Real Name (Comment)
Key fingerprint = 88A9 166B 13E6 516A 87C8 F127 5CA9 2D9E 9B13
86E2
sub 2048g/7F72A50F 2007-12-01

Be sure to keep your fingerprint handy. When people are attempting to use or import your key, they can ensure they have the right key if you provide them with the fingerprint.

At this point, you can start using GPG to encrypt and decrypt files. For instance, if you have a text document, and you want to ensure that no one tampers with it, you can sign it with the — clearsign command. To keep the file readable, specify the ASCII armor format with -a. After providing your passphrase, the contents of the file will be wrapped in a digital signature and a new file will be created with the new contents. If even one space is added to the file, the signature verification will fail. For instance:

$ echo “Test file” >test.txt
$ gpg –clearsign -a test.txt
You need a passphrase to unlock the secret key for
user: “Real Name (Comment) “
1024-bit DSA key, ID 9B1386E2, created 2007-12-01
Enter passphrase:
$ cat test.txt.asc
—–BEGIN PGP SIGNED MESSAGE—–
Hash: SHA1
Test file
—–BEGIN PGP SIGNATURE—–
Version: GnuPG v1.4.7 (GNU/Linux)
iD8DBQFHUh3VJnj1HnfyJpYRAjn7AKCI5DYTvvQ2J6pALyMYp26oGuZKaQCcCSZ7
O6dBveVjOgzC4HL5k8rFFHM=
=SxSW
—–END PGP SIGNATURE—–
$ gpg –verify test.txt.asc
gpg: Signature made Sat Dec 1 19:52:05 2007 MST using DSA key ID
9B1386E2
gpg: Good signature from “Real Name (Comment) “
gpg: WARNING: This key is not certified with a trusted signature!
gpg: There is no indication that the signature belongs to the
owner.
Primary key fingerprint: 88A9 166B 13E6 516A 87C8 F127 5CA9 2D9E 9B13
86E2
$ perl -pi -e ‘s|file|files|’ test.txt.asc
$ gpg –verify test.txt.asc
gpg: Signature made Sat Dec 1 19:52:05 2007 MST using DSA key ID
9B1386E2
gpg: BAD signature from “Real Name (Comment) “

As you can see from the above, changing the word “file” to “files” causes the verification of the ASCII-armored text file to fail. You can also see that GPG created a new file called test.txt.asc; GPG will attach either an .asc extension to the original file name for an ASCII-armored text file, or a .gpg extension in the case of a GPG-encrypted file.

Many IRC networks, particularly the Freenode network, cater to a lot of open source and Linux-based groups that can offer support or provide a meeting place for development effort.

There are many IRC clients available, but one of the most extensible and powerful is the command-line based Irssi. In particular, using a CLI-based client, of any kind, offers many unique possibilities — from using it remotely via SSH to using screen to manage and share “connections” to the client.

Irssi, like most CLI programs, uses a flat-text configuration file: ~/.irssi/config. This configuration file can be heavily customised and can connect to multiple servers at start and issue various commands based on channel or server. For instance, to connect to Freenode at start, use:

servers = (
{
address = “irc.freenode.net”;
chatnet = “freenode”;
port = “6667″;
autoconnect = “yes”;
}
);

To configure Irssi to automatically authenticate your nickname on Freenode, you would use:

chatnets = {
freenode = {
type = “IRC”;
autosendcmd = “/^msg nickserv identify secret;wait 2000″;

max_kicks = “4″;
max_msgs = “1″;
max_modes = “4″;
max_whois = “1″;
};

This will send the command /msg nickserv identify secret (where “secret” is your password), and then wait for nickserv to respond before executing further commands on that server.

Channel definitions are designed thus:

channels = (
{
name = “#annvix”;
chatnet = “freenode”;
autojoin = “yes”;
autosendcmd = “/^msg chanserv op #annvix”;
},
{
name = “#oss-security”;
chatnet = “freenode”;
autojoin = “yes”;
autosendcmd = “/^msg chanserv op #oss-security”;
}
);

This defines two channels that are automatically joined, and the pause after messaging nickserv is made apparent as new commands are being sent to chanserv to obtain ops on the two channels — a command that would fail if the used nickname had not been authenticated yet.

Obviously, there is a lot more to Irssi than just auto-connecting to channels and servers. Visit the Web site to view the variety of themes that can be used to customise its look and feel, and look at the Scripts page to see the variety of scripts that can be used with Irssi. Perl is the language that can be used with Irssi, which makes scripting it very powerful. If you’re looking for a good IRC client, give Irssi a try. It has a little bit of a learning curve, but it’s well worth the effort.

To begin with, on the MySQL server, the database must be created. In this scenario, the Snort server is “snort.host” and the MySQL server is “mysql.host”. Connect to the database as root:

# mysql -u root -p

mysql> create database snort;

mysql> grant INSERT,SELECT,UPDATE,CREATE,DELETE,EXECUTE on snort.* to snort@snort.host;

mysql> set password for snort@snort.host=PASSWORD(\’snortpass\’);

mysql> flush privileges;

mysql> q

With the Snort documentation comes a file called create_mysql, which has the schema for the database. On a typical Linux install, this file would be found in /usr/share/doc/snort-[version]/create_mysql. Load this file as root:

# mysql -u root -p snort </usr/share/doc/snort-doc/create_mysql

Next, on the system where Snort will be running, edit the /etc/snort/snort.conf configuration file and tell it to log to the database:

output database: log, mysql, user=snort password=snortpass dbname=snort host=mysql.host

Finally, make sure that /etc/snort/snort.conf is mode 0640 and owned root:snort:

# chown root:snort /etc/snort/snort.conf

# chmod 0640 /etc/snort/snort.conf

The next step is to start Snort; a supplied initscript will start Snort monitoring or you can launch it to the background:

# /usr/sbin/snort -c /etc/snort/snort.conf &

Starting Snort once without sending it to the background is a good idea to ensure the connection takes. You can also look on the MySQL server to ensure that logging is active:

# echo “SELECT hostname FROM sensor;” | mysql -u root -p snort

The IP address that Snort is listening on should be displayed. Now that Snort is logging data to MySQL, using BASE Basic Analysis and Security Engine is a great way to view the data via a Web interface. BASE requires a Web server and PHP. Once you have unarchived it where it needs to be, copy the base_conf.php.dist file to base_conf.php and edit it, in particular, setting the $alert_dbname and related variables to point to the Snort log database.

You will also want to add a snort@localhost user with privileges to the MySQL database if you did not do so earlier (i.e., if your Snort and MySQL servers are physically separate).

Once that is done, navigate to the BASE install that you just set up and follow the instructions presented to set up the caching table for BASE. When that is complete, BASE is now available to view and graph the logged Snort data.

The Fedora Project is in the final throes of testing version 10 of its Linux distribution, and is regularly releasing “snapshot” updates to Fedora 10 beta. We take you on a graphical tour of the installation and desktop of Fedora 10 beta, snapshot 2.

We booted the live CD version of the Linux distribution, in a VMware virtualised session. We then proceeded to install the software … but it crashed on boot-up (see the error at the end of the gallery). Such is beta living.

They all have the same feature list — indeed, they capitalise on the many aspects of the iPhone that are well below par — but they all feel cruder and more frustrating to use. You can’t just bolt this stuff on.

Usability is extremely important. It’s also very hard to do well and requires extreme corporate self-discipline. If you ever find yourself asking which of Apple and Microsoft has the more effective management, compare their products. Unless an engineer knows with total clarity that their part of the project will be thrown back for failing to meet usability standards, then the urge to cut a corner or half-bake a feature can be irresistible. Functionality is easy to specify and test: usability far less so.

That should make usability supremely challenging for open source projects, especially ones of the scale and complexity of Ubuntu. But then, open source is not known for following corporate theory: witness the fact that the Linux desktop has evolved to the point where it’s beyond merely usable but a fully-featured and viable alternative in daily work to paid-for software. That should have been impossible: it turned out to be anything but.

So when Mark Shuttleworth calls for usability to be a primary concern in future Ubuntu developments, it’s fair to say that he expects it to be achievable. That he’s employing a team to add usability to X, OpenGL, Gtk, Qt, Gnome and KDE shows that he’s prepared to back that expectation.

But it needs more. Open source needs its Steve Jobs — which is admittedly like saying America needs an emperor. It needs someone with an extremely clear and uncompromising vision of usability, and the charisma and energy to make people want to follow. As many in the movement have a visceral reaction to autocrats, the primary role of such a figurehead won’t be to tell, but to teach — to show people how to want to write usable software more than anything else.

Any glance at open source discussion groups will show just how far out of mind such considerations stand. Unless this starts to change, open source will remain limited in its power and reach — and its competitors will carry on getting a free ride.

For myself, I have a single bottom GNOME panel with the “Start” menu on the left, then a series of “Quick Launch” application launcher icons, then a “window list” (the buttons representing open windows), a notification area (system tray showing certain statuses, such as battery life, network connections, etc…) and finally a clock/weather combo. This is very much like most Windows setups except the panel menu choices didn’t quite match the “Start” menu I was so used to. Here’s how to remedy that:

1. Make an image that you want to use as a start menu (download mine at the bottom of this article). It can be anything you want but should be somewhat horizontally or vertically oriented depending on the type of panel you’ll be placing it on. Don’t worry about the original’s size as it will scale down nicely (mine’s 350 pixels, 17KB). Lastly, make the background transparent to allow the underlying panel color/background to show through.
2. Save the image as a transparent PNG file somewhere in your home directory. I personally have a Pictures/GUI subdirectory just in case I ever reinstall my Linux, I have all my GUI tweaks, wallpapers, etc… right there.
3. If you haven’t done so already, add a “Main Menu” to your panel by right-clicking on an empty area on the panel and selecting “Add to Panel”. There are two types of menus there – you want to pick the one represented by a single icon of the Ubuntu logo (or whatever your Linux distro uses as a default image). Also remove the Applications/Places/System menu from your panel if you have one of those set.
4. To set the custom menu graphic, you need to open the Configuration Editor under the “System Tools” menu – or for you hardcore types, “gconf-editor” at the command line. Go to apps->panel->objects and you’ll see a series of object_x subitems. Click through them until you find the one where the value for “object_type” is “menu-object”. Check the “use_custom_icon” option and then set the “custom_icon” value to the path to your custom button graphic, i.e. “/home/username/Pictures/ubuntumenu.png”. The graphic should update immediately.
5. For an added touch, add a “Separator” to your GNOME panel to partition it off from the rest of the panel, as seen in the screenshot above.

Considering this is Linux, there are many other file management tools to choose from, some of which you may find preferable to the “defaults”.

Gentoo file manager

One such file management tool is called Gentoo, not to be confused with the Linux distribution of the same name. Gentoo hasn’t been updated in a few years, but it is still a solid file manager. Some distributions include it, so it is just an apt-get or yum command away; for others that do not include it, all you need to build it from source are the GTK+ development files and libraries. The latter will be installed with any GNOME system. To build from source, it’s a ./configure; make; make install away from usability.

Gentoo is nice in that it is a GTK+-based application with a dual-paned file view, reminiscent of old tools like Total Commander or Norton Commander. It is extremely configurable with a powerful file recognition system that allows you to map what happens when you double-click a file type, select which icons belong to what types of files, etc.

Krusader

For those who prefer using KDE, Krusader is another stellar file management tool. Like Gentoo, it is a dual-paned file management utility that is extremely customisable. The latest beta of Krusader is compatible with KDE4, whereas previous stable versions are written for KDE3. Some of Krusader’s notable features include impressive archive handling, advanced searching, an internal viewer/editor, file content comparisons and directory synchronisation. It can also handle remote file systems by using KIO slaves, which means it can mount SMB or FTP file systems. Many distributions provide Krusader.

Midnight Commander

Finally, for those who want a file management tool very similar to Norton Commander and that operates on the CLI, Midnight Commander is the best choice. Midnight Commander is available for nearly all Linux distributions; the package may be named mc, which is also the name of the program. MC gives a dual-paned file view, each representing a different directory. When executed in a terminal under X, mouse clicks work, so if MC is your preference but you want to use it in a GUI, it’s still an option. Another nice feature of MC is that it still keeps a command-line open so you can use a combination of directory navigation and CLI commands, made even easier by using the tab key to switch active panes.

These are just three examples of many other file management, or “file commander” type programs available for Linux. Others include Tux Commander, XFE, and Gnome Commander. If the default file management tools don’t quite cut it, or if you prefer the dual-paned file system view, give one of these programs a try.