Note to self: Missing lvm2 and cryptsetup packages lead to non-working initrd very, very soon

I recently almost died from a heart attack because after a really horrible crash (don't ask), Debian unstable on my laptop wouldn't boot anymore. The system hung at "Waiting for root filesystem...", and I was in panic mode as I feared I lost all my data (and as usual my backups were waaay too old).

At first I was suspecting that something actually got erased or mangled due to the crash, either at the dm-crypt layer, or the LVM layer, or the ext3 filesystem on top of those. After various hours of messing with live CDs, cryptsetup, lvm commands (such as pvscan, pvs, vgchange, vgs, vgck) and finally fsck I still had not managed to successfully boot my laptop.

I finally was able to boot by changing the initrd from initrd.img-2.6.30-2-686 to initrd.img-2.6.30-2-686.bak in the GRUB2 menu (at boot-time), at which point it was clear that something was wrong with my current initrd. A bit of debugging and some initrd comparisons revealed the cause:

Both, the cryptsetup and lvm2 packages were no longer installed on my laptop, which made all update-initramfs invokations (e.g. upon kernel package updates) create initrds which did not contain the proper dm-crypt and lvm functionality support. Hence, no booting for me. I only noticed because of the crash, as I usually do not reboot the laptop very often (two or three times per year maybe).

Now, as to why those packages were removed I have absolutely no idea. I did not remove them knowingly, so I suspect some dist-upgrade did it and I didn't notice (but I do carefully check which packages dist-upgrade tries to remove, usually)...

coreboot on the cover of the Linux Journal

coreboot on Linux Journal

Nice coreboot news — the Free Software x86 firmware ("BIOS") is featured on the cover of issue 186 of the Linux Journal.

Anton Borisov's article Coreboot at Your Service! explains the basic ideas behind coreboot, how to build an image for your board, which payloads are available and how they are used, e.g. GRUB2, SeaBIOS if you need legacy BIOS callbacks (e.g. for booting Windows), Etherboot/GPXE, or more fun stuff such as space invaders or tint (a tetris clone) in your flash ROM chip...

If you read the article and think the build process is a bit complicated and ugly, do not despair! We're currently in the process of converting the whole coreboot code base to use kconfig (the widely-known configuration tool used by the Linux kernel, busybox, and other projects), so in the very near future the whole process for building a coreboot image will work like this:

  $ make menuconfig
  $ make

coreboot menuconfig

Flashing the image can then be done using an EEPROM programmer and/or via the user-space utility flashrom (available for Linux, Mac OS X, FreeBSD, etc.)...

It's nice to see that coreboot is getting more and more coverage in "mainstream" media and is growing both in number of deployments and in number of supported chipsets and boards.

We are desperately in need of more developers though, there are just way too many chipsets, boards, and datasheets out there; we're happy about every patch and every new tester or developer who likes to mess with code that runs in the very first few (micro)seconds after power-on.

If you think kernel hacking and related low-level development is nice, you might also be interested in writing code where there's no RAM yet (as coreboot has to initialize it), there's no serial port for debugging (coreboot has to initialize it), no PCI devices have been set up, most of your auxiliary hardware is not yet up (ethernet NIC, parallel port, audio, IDE, SATA, USB, you name it). It's a fun environment to work in and you'll learn a lot about PC hardware, even if you (so far) thought you knew everything there is to know.

Feel free to join us on the mailing list or on IRC in #coreboot on Freenode.

Using CRM114 for spam filtering on Debian GNU/Linux

I've been using CRM114 as spam filter for a while now, and I'm quite happy with it. Due to bug #529720 though (incompatible upstream file format changes) I decided to start my setup from scratch with a recent CRM114 version from unstable. Here's a short HOWTO, hope it's useful for others.

First you need to install crm114 and set up a few files in your $HOME directory.

  $ sudo apt-get install crm114
  $ mkdir ~/.crm114
  $ cd ~/.crm114
  $ cp /usr/share/doc/crm114/examples/mailfilter.cf.gz .
  $ gunzip mailfilter.cf.gz
  $ cp /usr/share/crm114/mailtrainer.crm .
  $ touch rewrites.mfp priolist.mfp

Edit ~/.crm114/mailfilter.cf and set the following variables (some are optional, but that's what I currently use):

  :spw: /mypassword/
  :add_verbose_stats: /no/
  :add_extra_stuff: /no/
  :rewrites_enabled: /no/
  :spam_flag_subject_string: //
  :unsure_flag_subject_string: //
  :log_to_allmail.txt: /no/

The :log_to_allmail.txt: /no/ option should probably stay at "yes" for the first few days until you have tested your setup and everything works OK. The ~/.crm114/allmail.txt file will contain all your mails, in case something goes wrong.

Now set up empty spam and nonspam files like this:

  $ cssutil -b -r spam.css
  $ cssutil -b -r nonspam.css

Test the setup by invoking mailreaver.crm as follows, typing some test text and then pressing CTRL+d:

  $ /usr/share/crm114/mailreaver.crm -u ~/.crm114
  test
  [CTRL-d]
  ** ACCEPT: CRM114 PASS osb unique microgroom Matcher **
  CLASSIFY fails; success probability: 0.5000  pR: 0.0000
  Best match to file #0 (nonspam.css) prob: 0.5000  pR: 0.0000
  Total features in input file: 8
  #0 (nonspam.css): features: 1, hits: 0, prob: 5.00e-01, pR:   0.00
  #1 (spam.css): features: 1, hits: 0, prob: 5.00e-01, pR:   0.00
  X-CRM114-Version: 200904023-BlameSteveJobs ( TRE 0.7.6 (BSD) ) MF-35EB8B9A [pR: 0.0000]
  X-CRM114-CacheID: sfid-20090920_151224_574131_D290E589
  X-CRM114-Status: UNSURE (0.0000) This message is 'unsure'; please train it!

The output should look similar to the above. If there are errors instead, you should check your settings in ~/.crm114/mailfilter.cf.

Now you have to setup a procmail rule for crm114:

  :0fw: crm114.lock
  | /usr/share/crm114/mailreaver.crm -u /home/uwe/.crm114

  :0:
  * ^X-CRM114-Status: SPAM.*
  IN.spam-crm114

In my case this rule is also followed by a spamassassin rule, so all my mail goes through two different spam filters (will look into dspam and bogofilter also I guess, the more the better).

Finally, in .muttrc I have the following configs so I can press SHIFT+x to mark a mail as spam, and SHIFT+h to mark it as non-spam (ham).

macro index X '| formail -I X-CRM114-Status -I X-CRM114-Action -I X-CRM114-Version | /usr/share/crm114/mailreaver.crm -u /home/uwe/.crm114/ --spam'
macro index H '| formail -I X-CRM114-Status -I X-CRM114-Action -I X-CRM114-Version | /usr/share/crm114/mailreaver.crm -u /home/uwe/.crm114/ --good'
macro pager X '| formail -I X-CRM114-Status -I X-CRM114-Action -I X-CRM114-Version | /usr/share/crm114/mailreaver.crm -u /home/uwe/.crm114/ --spam'
macro pager H '| formail -I X-CRM114-Status -I X-CRM114-Action -I X-CRM114-Version | /usr/share/crm114/mailreaver.crm -u /home/uwe/.crm114/ --good'

Important: crm114 is most effective if you start with empty CSS files (as shown above) and only train it by marking mails as spam/ham when it gets them wrong. The process will take a few hours or maybe a day (depending on how many mails per day you get), then the misclassification rate gets very low...

Update 2009-09-23: Changed --spam/--nonspam to the correct options for mailreaver/mailtrainer, --spam/--good.

Help add subtitle support for Miro

Miro 2.0 feed list

If you ever wanted to support an open-source project but you are not a programmer, here's one (of many possible) ways to help:

The Miro project (Internet TV / Video and Audio Podcast application for Linux, Windows, and Mac OS X) is seeking for pledges/donations that will be used to add subtitles support in Miro.

To quote from the announcement:

We’re hoping to build real subtitle support into Miro in the next couple months, but we need your help! So we’ve started a Kickstarter project to raise $1,000 to develop this feature for Miro on all three platforms: Windows, Mac, and Linux. Can you pledge to help make it happen? One of the great things about the Kickstarter model is that unless we can reach $1,000, your pledge won’t be charged.
[...]
(if you live in the United States, donations are tax deductible — we are a 501c3 non-profit)

There are 11 days left to make a pledge.

A simple DLP-USB1232H based JTAG programmer with OpenOCD support

DLP-USB1232H and OpenOCD based JTAG adapter

Here's a quick introduction to using a cheap FTDI FT2232H based module (left-hand side on the photo) as a JTAG programmer together with the OpenOCD JTAG software for ARM and MIPS devices. The module I am using for thіs purpose is a DLP Design DLP-USB1232H, which is available from various sources (Digikey, Mouser, Saelig, and probably others) for 20-30 bucks plus shipping, depending on where you live.

By properly connecting the correct pins of the DLP-USB1232H to the target JTAG
device (I used an Olimex STM32-H103 eval board for testing) you can easily abuse the DLP-USB1232H as JTAG programmer. As I chose the proper DLP-USB1232H GPIOs for the TRST and (S)RST pins, OpenOCD even worked out of the box, without having to change a single line of code.

The only thing that's required is to provide OpenOCD with an interface config file that uses the usbjtag "layout". I have already submitted that config file upstream, I guess it should be merged soonish.

The usage is then pretty simple:

  $ openocd -f interface/dlp-usb1232h.cfg -f board/olimex_stm32_h103.cfg

And in another xterm:

  $ telnet localhost 4444
  > init
  > reset halt
  > flash write_image erase fancyblink.bin 0x08000000
  > reset

This flashes the given fancyblink.bin image onto the STM32-H103 eval board via the DLP-USB1232H JTAG programmer, where fancyblink.bin is an example program from my libopenstm32 project (that aims to create a full-blown firmware library for ST STM32 microcontrollers, similar to what avr-libc does for AVRs). Contributions for libopenstm32 (license is GPLv3 or later) are highly welcome btw., hint hint...

  $ git clone git://libopenstm32.git.sourceforge.net/gitroot/libopenstm32/libopenstm32

Full schematics, datasheets, and detailed instructions for the JTAG programmer are available from a small page I created in my Random Projects wiki, which is intended for the various smaller projects I'm working on that don't warrant getting their own domain, wiki, etc:

The Random Projects wiki is open-for-all btw, feel free to use it for any freeish, software or hardware projects of your own if you want.

Anyway, the DLP-USB1232H is a really nice device as it can also be used for many other purposes, such as USB-to-Serial or SPI BIOS chip programming, but more on that in another blog post...

Syndicate content