OS Archives

August 5, 2002

Choosing Debian

Recently, I chose Debian GNU/Linux. I started Linux in 1994 with Slackware and it's floppy disk sets. Before switching to Debian, I've used RedHat for the last 5 years. However, over the last few months, I've become very fond of Debian. I've replaced all of my RedHat systems with Debian.

There are a number of good reasons why I abandoned RedHat and migrated to Debian. The primary reason is Debian's package system that saves me time and effort. I maintain a large number of Linux systems. With RedHat, I had to download and compile a large number of packages on each system to achieve my standard configuration. With Debian, nearly every package that I use was already in Debian. And, if that package was not in Debian, I've since added it to the main development distribution.

As an example, I wrote a computed tomography simulator CTSim. I started this in 1980 on my IBM-PC using PC-DOS 2.0. In 1999 after I ported this to Linux using RedHat. I had to download and compile a number of libraries to build and use this with RedHat. Specifically, CTSim required FFTW, wxWindows, and CTN which were not in RedHat. As new versions of these libraries were released, I needed to re-download and re-install them.

With Debian, this is much easier. Both FFTW and wxWindows are already in Debian. As new versions of these libraries are released, I give a single command to Debian to upgrade all of my packages. CTN, being a speciality DICOM library, was not in Debian a few months ago. However, I've since added it to Debian distribution and now Debian users can easily add it to their system. Finally the best part: CTSim is now in Debian. When a user chooses to install CTSim, all of the required libraries are automatically installed along with CTSim. This is far better than me having to instruct RedHat users how to download and install all of the required libraries. (To be fair, I did have a statically-linked RPM available for RedHat, but it was 10MB in size vs. the current 1MB Debian package size.)

My Debian web page has a list of the Debian packages that I've added or adopted.

August 12, 2002

Officially Debian

Today I was approved by the Debian Application Manager as an official Debian Developer. Currently I'm supporting 16 packages in Debian. Before getting approved my sponsor had to approve and upload each of these packages into the Debian distribution. Gladly for both him and I, I can now upload my packages directly into the Debian distribution.

September 26, 2002

Debian Application Manager

I've recently become a Debian Application Manager. As such, I help test and guide people who want to become Debian Developers. You can learn more about being a Debian developer here.

October 28, 2002

Fink for OSX

Fink is a package manager for Apple's OSX operating system. I'm interested in Fink because it is based on some Debian concepts and it brings a number of ported UNIX applications to OSX.

Since I maintain the Debian OpenMCL package, I was asked to make a Fink package for OpenMCL. There are some interesting differences between making Debian and Fink packages. But, in general, the process was easy. I've forwarded by Fink OpenMCL package to Gary Byers to include on his OpenMCL web site.

November 27, 2003

Subversion Conversion

I've become a big fan of subversion source control system as a replacement for the venerable CVS. Last year I setup my home directory in CVS as explained by Joey Hess with very good results. About 6 months ago I converted my CVS repository to subversion which retained all my CVS history.

Before switching to subversion, I turned off anonymous CVS access because of security concerns about CVS pserver mode. After a long absense, I have restored anonymous access to my repository -- this time using subversion. First, I setup a public mirror of the subdirectories in my master repository that I publish as open-source projects. This is about 1/10 of my repository in terms of archive size. I have a script setup that updates that mirror on each commit to my master repository. The script also uses rsync to publish the public repository on my high-bandwidth server.

The public repository is available as browsable XML, browsable ViewCVS, and subversion repositories that can be "checked-out". You can browse the projects on my public subversion web site

I employ a hacked version of enscript with adds color to lisp files.

Mirroring Subversion

The subversion package does not currently support mirroring a repository. I tested two mirror tools for subversion: svn-push and SVN::Mirror (svm). I wasn't able to get svn-push to work. SVN::Mirror, though, has worked very well. It has an additional advantage that it allows mirroring a subdirectory of the master repository to a new location in the mirror. With that ability, I choose various directories in my master repository to and mirror them as top-level directories in my public repository.

July 12, 2004

Death of the backup MX host

A backup MX host is a backup mail server. It will handle mail for a domain if the primary server is unavailable. It's been a useful feature for email reliablility.

It appears now that backup MX hosting in no longer an option due to actions of spammers. I've recently had to delete my backup MX host because a constant and rapid flow of spam messages has overloaded the server twice. I've had over 500,000 spam messages in /var/spool/mqueue waiting to be bounced back as invalid email addresses.

The ISP where I co-locate my server told me that this attack on backup MX hosts is common and they actively recommend not using backup MX hosts. In fact, for their many thousands of customers, they no longer use a backup MX host for their domain.

It's too bad that spammers have made this useful tool for email reliability now an impractical option. For now, such barrages have been focused on backup MX hosts. I wonder if it's just a matter of time before they move such attacks to primary MX hosts.

There's a similarity to the biologic host/parasite relationship. A parasite uses enough of the hosts resources to reproduce. But, if the parasite is so virulent that the host dies, the parasite is some what less effective because it no longer has the carrier. Perhaps the spammers realize if they take down primary MX hosts like they have taken down backup MX hosts, then there won't be any email at all for them to take parasitically abuse.

Fondly remembered, RIP our backup MX hosts.

February 28, 2005

100 Packages to Orphan

I'm orphaning the following 100 Debian packages. Hopefully, one or more people will want to adopt some of these packages so they may stay in the Debian distribution.


August 24, 2005

Useful layers of indirection

Recently, I've been building systems using Linux's Logical Volume Manager (LVM) and XFS filesystems. While LVM adds a layer of indirection to disk access, it has offers significant benefits. Primarily, I've been taking advantage of this combination to back up filesystems on production servers. Using xfs_freeze to suspend disk activity, then quickly creating a snapshot of the filesystem with LVM, I can then resume activity on the production server while I copy the snapshot filesystem to a remote backup location. LVM has a large number of other features, but its snapshot abilitity has been the largest win for me.

I'm using the same mapping code in the 2.6 Linux kernels to encrypt all the partitions on my laptop. The Debian cryptsetup package uses dmsetup to provide encrypted partitions. Compared to using cryptoloop (deprecated) or loop-aes (requires modified losetup and mount), dmsetup is nearly trivial to use. In a very short period of time, I've converted my Debian laptop to use an encrypted root partition, encrypted swap partition, and encrypted LVM physical volume. I created my local data partitions as logical volumes on the encrypted LVM physicial volume. All this was quite painless to setup. Operation is similarly simple: the cryptdisk init script asks for the passphrases of the encrypted partitions upon booting.

While it's been 14 years since my last laptop was stolen, I'm reassured knowing my system is strongly encrypted if such misfortune recurs.

Encrypted root filesystem question

François-René ÐVB Rideau asked:

Interesting. Can you publish your configuration files and the list of packages used? Do you carry the key on a floppy? On a USB key? Or do you type a long key at boot up? Is all of the disk encrypted, or only the user partition? etc.

The configuration files are minimal. The important Debian package to install is cryptsetup. In the /usr/share/doc/cryptsetup directory are HOWTOs for setting up encryption on root and swap partitions. Those files are short and the instructions worked perfectly for me. I compiled my own kernel using make-kpkg from the kernel-package package. You'll need a 2.6.4 or later kernel with cryptographic routines and LVM enabled. While I use a monolithic kernel, using modules and mkinitrd works fine as well.

My key is a fairly long passphrase. It can be whatever length you want, but you need to type it into the prompt at boot time. Reading from USB keys is not supported by the startup script, but I imagine you can hack it without much difficulty if you wanted to read from a USB mount at boot time.

You setup encryption by partition. My laptop partition structure is:

/dev/hda2/19GBxfsAES encrypted
/dev/hda3swap1GBswapAES encrypted
/dev/hda4LVM60GBLVM PVAES encrypted

I created several logical volumes on the encrypted /dev/hda4 while leaving 5GB available for temporary snapshots volumes.

My /etc/crypttab looks like this:
root /dev/hda2
cswap /dev/hda3 /dev/random swap
mainpv /dev/hda4

Besides the very helpful HOWTOs in the Debian package, I also referred to this useful guide.

I hope that helps, let me know if you have more questions.

August 28, 2005

Return of the backup MX host

Last year I wrote about the death of the backup mx host for my domains. In the arms race against spam, I've found a very helpful tool allowing the return of my backup MX hosts.

Recapping the history, one of my domains has been under a persistant SMTP dictionary attack for over a year. I get 3-10 emails a second with guessed destination email address. Sendmail on the primary MX host is rather efficient at rejecting these so the only real effect is my 200MB of email logs a day. However, spammers often target the backup MX host since they usually have less spam filtering. Last year, the attack took down my backup MX host. The backup host will kindly accept a message for any username in the appropriate domain. Then, it will try to deliver them to the primary MX host. If the primary MX host rejects the username as invalid, the backup MX host tries to send a reject message back to the sender. The breaks down when the sender is a spam zombie that is not running a SMTP server. The backup MX host queues the reject message while it attemps redelivery for a number of days. My backup MX host has 500,000 reject messages in its queue before the load average become too great.

A great piece of software fixes this problem. SnertSoft's no-cost milter-ahead sendmail filter handles this sort of attack very well. The author is very responsive. I found only one issue with the software: it wouldn't compile or run on AMD64 platforms [typical issue of the C long type varying between 32 and 64-bit platforms]. In just a few hours, the author updated his code and milter-ahead is now running very well on my Debian AMD64 servers.

milter-ahead runs on a gateway or backup mail server and checks the RCPT during the SMTP connection with the a designated MX host. If that "look-ahead" host is down, milter-ahead will accept any email, which is the right thing to due. If the designated host is up, then milter-ahead can do one of several things. But, it's primary use is to query the designated host and verify that the receipient address is valid. If it is not, then the milter rejects the RCPT and avoids queueing the message. If the receipient is valid, then milter-ahead allows the backup MX host to accept the message for forwarding.

Now that my backup MX host won't be queueing email for invalid email addresses, I've brought that system back online. If you're running a gateway or backup mail server, you'll likely find milter-ahead a useful tool.

February 2, 2006

Encrypted Root Filesystem Revisited

The hard disk on my notebook computer developed a number of bad sectors so I replaced it with a new 7200rpm 100GB drive. I've discussed my experience and gave details about using an encrypted root and LVM filesystem using the Linux 2.6 device mapper.

While I found the disk speed acceptable using the encrypted filesystem, I did note that it was subjectively slower than using an unencrypted filesystem. This time, I used the bonnie++ disk benchmark to test the new disk.

Raw disk benchmark

       ------Sequential Output------ --Sequential Input- --Random-
        -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
  Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
    4G 30299  98 47348  13 20552   6 29947  91 44758   5 137.3   0
       ------Sequential Create------ --------Random Create--------
        -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
  files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
     16  3545  21 +++++ +++  3863  42  3350  20 +++++ +++   867   6

Encrypted disk benchmark

       ------Sequential Output------ --Sequential Input- --Random-
        -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
   Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
     4G 18861  60 40812   9 10534   2 18530  58 30641   4 143.6   0
       ------Sequential Create------ --------Random Create--------
        -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
  files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
     16  3150  87 +++++ +++  3510  92  2919  86 +++++ +++   900  35

While there is an overhead using the encrypted file system (which is likely magnified by using a 4-year old CPU, a 2GHz Pentium 4-M), the overall speed with the 7200rpm drive is acceptable and will be a boost compared to the old 5400rpm drive.

While my notebook previously ran Debian Sarge, I thought I'd try something slightly different, the Debian offshoot Ubuntu Breezy. I found a few issues that had to be resolved to use the encrypted filesystems.

  • Patching /usr/sbin/mkinitrd
    mkinitrd had the wrong devmapper library version. I had to change line 385 from /lib/ to /lib/
  • Add modules to /etc/mkinitrd/modules
    I added aes_i586, dm_crypt, and dm_mod to /etc/mkinitrd/modules.
  • Modifying /etc/lvm/lvm.conf
    I use an encrypted physicial LVM partition to host my logical volumes. That way, I only have to encrypt the LVM physical partition. Subsequently when booting and decrypting the filesystem, I only have to give once the key to the physical volume and all of the logical volumes created in that partition will be decrypted. This requires some editing of lvm.conf to recognize the encrypted physical volume. I named the LVM physicial volume pv and then added /dev/mapper/pv to the filter specification in the devices section. Also, LVM has to be notified of the device mapper type, so the line types=["device-mapper",16] must be added to the devices section.
  • Change the startup script order
    The script order in /etc/rcS.d/ needs to be changed so that cryptdisks is executed before lvm. This is required so that the lvm script will have access to the unencrypted physicial volume. This is simply done by renaming S28cryptdisks to S25cryptdisks which places it before the lvm script S26lvm.

After those bits of additional configuration, the encrypted root filesystem and encrypted LVM physical volume is working very well. Additionally, I'm using an encrypted swap partition, but there was no additional configuration required beyond what I previously described.

February 4, 2006

Console Password Manager

I'd been looking for a fairly secure way to store an increasing number of passwords. There's a large number of methods one can use. While the most secure might be an encrypted file on a system that has no network access, I do want to have access to the information over a network while I'm traveling. Remote access makes using a graphical client a poor choice.

After a moderate search and trying a number of candidates, I settled on Console Password Manger (CPM) written by Harry Brueckner. It's close to an ideal match to my needs: it's console-based, had an emphasis on security and uses my existing GPG key for encryption. The program is maintained and I've had no issues while using the last three beta versions of the program. Thanks, Harry for the very good tool. Recommended.

About OS

This page contains an archive of all entries posted to Kevin Rosenberg in the OS category. They are listed from oldest to newest.

Meta is the previous category.

Photography is the next category.

Many more can be found on the main index page or by looking through the archives.

Creative Commons License
This weblog is licensed under a Creative Commons License.