Yes, it's that time of the year again. A disk in my desktop-replacement laptop with 2 disks and a RAID-1 has died. Time for recovery.
This laptop has been running 24/7 for the last 3 years or such, so it's not too surprising that a disk dies. Surprisingly though, for the first time in a long series of dead disks, smartctl -a does indeed show errors for this disk. Here's a short snippet of those:
$ smartctl -a /dev/sda [...] Error 1341 occurred at disk power-on lifetime: 17614 hours (733 days + 22 hours) When the command that caused the error occurred, the device was active or idle. After command completion occurred, registers were: ER ST SC SN CL CH DH -- -- -- -- -- -- -- 40 41 02 1f c0 9c 40 Error: UNC at LBA = 0x009cc01f = 10272799 Commands leading to the command that caused the error were: CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name -- -- -- -- -- -- -- -- ---------------- -------------------- 60 f8 08 20 c0 9c 40 00 41d+01:51:50.974 READ FPDMA QUEUED 60 08 00 18 c0 9c 40 00 41d+01:51:50.972 READ FPDMA QUEUED ef 10 02 00 00 00 a0 00 41d+01:51:50.972 SET FEATURES [Reserved for Serial ATA] ec 00 00 00 00 00 a0 00 41d+01:51:50.971 IDENTIFY DEVICE ef 03 45 00 00 00 a0 00 41d+01:51:50.971 SET FEATURES [Set transfer mode] SMART Self-test log structure revision number 1 Num Test_Description Status Remaining LifeTime(hours) LBA_of_first_error # 1 Short offline Completed: read failure 90% 20511 156170102 [...]
The status of the degraded RAID array looks like this:
$ cat /proc/mdstat Personalities : [raid1] md1 : active raid1 sdb7 409845696 blocks [2/1] [_U] md0 : active raid1 sda6 sdb6 291776 blocks [2/2] [UU]
The [_U] means that one of two disks has failed, it should normally be [UU]. There are two RAID-1s actually, a small md0 (sda6 + sdb6) for /boot and the main md1 (sda7 + sdb7) which holds the OS and my data. Apparently (at first at least), only sda7 was faulty and got kicked out of the array:
$ dmesg | grep kick md: kicking non-fresh sda7 from array!
Anyway, so I ordered a replacement disk, removed the dead disk (I checked the serial number and brand before, so I don't accidentally remove the wrong one), inserted the new disk and rebooted.
Note: In order for this to work you have to have (previously) installed the bootloader (usually GRUB) onto both disks, otherwise you won't be able to boot from either of them (which you'll want to do if one of them dies, of course). In my case, sda was now dead, so I put sdb into its place (physically, by using the other SATA connector/port) and the new replacement disk would become the new sdb.
After the reboot, the new disk needs to be partitioned like the other RAID disk. This can be done easily by copying the partition layout of the "good" disk (now sda after the reboot) onto the empty disk (sdb):
$ sfdisk -d /dev/sda | sfdisk /dev/sdb
Specifically, the RAID disks/partitions need to have the type/ID "fd" ("Linux raid autodetect"), check if that is the case. Then, you can add the new disk to the RAIDs:
$ mdadm /dev/md0 --add /dev/sdb6 $ mdadm /dev/md1 --add /dev/sdb7
After a few hours the RAID will be re-synced properly and all is good again. You can check the progress via:
$ watch -n 1 cat /proc/mdstat
You should probably not reboot during the resync (though I'm not 100% sure if that would be an issue in practice; please leave a comment if you know).
Also, don't forget to install GRUB on the new disk so you can still boot when the next disk dies:
$ grub-mkdevicemap $ grub-install /dev/sdb
And it might be a good idea to use S.M.A.R.T. to check the new disk, just in case. I did a quick run for the new disk via:
$ smartctl -t short /dev/sdb # Wait a few minutes after this. $ smartctl -a /dev/sdb [...] SMART Self-test log structure revision number 1 Num Test_Description Status Remaining LifeTime(hours) LBA_of_first_error # 1 Short offline Completed without error 00% 22 - [...]
Looks good. So far.
I recently wanted to buy some MP3 files from Amazon (a whole album in my case, but you can also just buy single MP3 files if you want). Digital music downloads from Amazon are often much cheaper than buying the physical CD (from Amazon), and you can also instantly get the stuff within seconds, without having to wait for the physical CD to be shipped to your place.
The good thing about Amazon's MP3 downloads is that the files are not infested with any DRM-crap (if that were the case I wouldn't spend a single penny on such useless junk, of course). This allows you to burn the MP3 files on CDs and/or play them on any device you like (MP3 player of choice, laptop, hifi-system, car, e-book reader with MP3 playback support, etc. etc).
Granted, you can not re-sell the digital files on eBay later, this is the one little drawback you have when compared to physical CDs, but I guess most people can usually live with that. Also, it would be great if Amazon would provide Ogg Vorbis files instead (or in addition to) MP3 files, of course.
Anyway, in order to download the MP3 files you buy from Amazon, they suggest to install the Amazon MP3 Downloader, which (surprisingly) is even available in a Mac and Linux version (only 32-bit though), but is (unsurprisingly) closed-source. This is no-go, of course, but luckily there is an alternative.
The clamz tool (GPL, version 3 or later) allows you to easily download single Amazon MP3 files, or whole albums. First, you need to login to your Amazon account and then visit a certain Amazon page (which sets a special "congratulations, the Amazon MP3 Downloader has been successfully installed" cookie in your browser). See the clamz website for the respective URL for your country. For Germany, use this URL.
The clamz installation is easy enough on Debian:
$ apt-get install clamz
IMPORTANT: It seems you need at least version 0.5 for recent Amazon files as they apparently changed something, see #647043. Current Debian unstable as of today already has 0.5, though.
After that is done, the rest is easy: In Amazon, click on "Buy MP3" or "Buy MP3 album", which will download a special AmazonMP3-1234567890.amz file. You can then let clamz download all the MP3s by typing:
$ clamz AmazonMP3-1234567890.amz
Wait a few minutes, and you'll have a bunch of non-DRM MP3 files in your current directory. Easy.
See the manpage for a bunch of options which let you configure clamz to your preferences.
Yep, so I bought a new laptop recently, my IBM/Lenovo Thinkpad T40p was slowly getting really unbearably sloooow (Celeron 1.5 GHz, 2 GB RAM max). After comparing some models I set out to buy a certain laptop in a local store, which they didn't have in stock, so I spontaneously got another model, the HP Pavilion dv7-3127eg (HP product number VY554EA).
Why this one? Well, the killer feature for me was that it has two SATA disks, hence allows me to run a RAID-1 in my laptop. This allows me to sleep better at night, knowing that the next dying disk will not necessarily lead to data loss (yes, I do still perform regular backups, of course).
Other pros: Much faster than the old notebook, this one is an AMD Turion II Dual-Core Mobile M520 at 2.3 GHz per core, it has 4 GB RAM (8 GB max), and uses an AMD RS780 / SB700 chipset which is supported by the Free-Software / Open-Source BIOS / firmware project coreboot, so this might make the laptop a good coreboot-target on the long run. I'll probably start working on that when I'm willing to open / dissect it or when the warranty expires, whichever happens first.
Anyway, I set up a page at randomprojects.org which contains lots more details about using Linux on this laptop:
Most of the hardware is supported out of the box, though I haven't yet tested everything. There may be issues with suspend-to-disk / suspend-to-RAM, sometimes it seems to hang (may be just a simple config change is needed in /etc/hibernate/disk.cfg).
Cons: Pretty big and heavy (but that's OK, I use it mostly as "semi-mobile desktop replacement"), glossy screen, loud fans (probably due to the two disks).
For reference, here's an lspci of the box:
$ lspci -tvnn -[0000:00]-+-00.0 Advanced Micro Devices [AMD] RS780 Host Bridge Alternate [1022:9601] +-02.0---+-00.0 ATI Technologies Inc M96 [Mobility Radeon HD 4650] [1002:9480] | \-00.1 ATI Technologies Inc RV710/730 [1002:aa38] +-04.0-[02-07]-- +-05.0-----00.0 Atheros Communications Inc. AR9285 Wireless Network Adapter (PCI-Express) [168c:002b] +-06.0-----00.0 Realtek Semiconductor Co., Ltd. RTL8111/8168B PCI Express Gigabit Ethernet controller [10ec:8168] +-0a.0-[0a]-- +-11.0 ATI Technologies Inc SB700/SB800 SATA Controller [AHCI mode] [1002:4391] +-12.0 ATI Technologies Inc SB700/SB800 USB OHCI0 Controller [1002:4397] +-12.1 ATI Technologies Inc SB700 USB OHCI1 Controller [1002:4398] +-12.2 ATI Technologies Inc SB700/SB800 USB EHCI Controller [1002:4396] +-13.0 ATI Technologies Inc SB700/SB800 USB OHCI0 Controller [1002:4397] +-13.1 ATI Technologies Inc SB700 USB OHCI1 Controller [1002:4398] +-13.2 ATI Technologies Inc SB700/SB800 USB EHCI Controller [1002:4396] +-14.0 ATI Technologies Inc SBx00 SMBus Controller [1002:4385] +-14.2 ATI Technologies Inc SBx00 Azalia (Intel HDA) [1002:4383] +-14.3 ATI Technologies Inc SB700/SB800 LPC host controller [1002:439d] +-14.4-[0b]-- +-18.0 Advanced Micro Devices [AMD] K10 [Opteron, Athlon64, Sempron] HyperTransport Configuration [1022:1200] +-18.1 Advanced Micro Devices [AMD] K10 [Opteron, Athlon64, Sempron] Address Map [1022:1201] +-18.2 Advanced Micro Devices [AMD] K10 [Opteron, Athlon64, Sempron] DRAM Controller [1022:1202] +-18.3 Advanced Micro Devices [AMD] K10 [Opteron, Athlon64, Sempron] Miscellaneous Control [1022:1203] \-18.4 Advanced Micro Devices [AMD] K10 [Opteron, Athlon64, Sempron] Link Control [1022:1204]
Full lspci -vvvxxxxnnn, lsusb -vvv, and a much more detailed list of tested hardware components is available in the wiki.
I've been buying quite a lot of (usually cheapo) gadgets recently, which I'll probably introduce / review in various blog posts sooner or later. Let me start with a fun little gadget, a digital USB-based microscope. I found out about it via this thread over at lostscrews.com.
You can get this (or a very similar device) e.g. on eBay for roughly 50 Euros. Mine seems to be from a company called Oasis (though they're probably just the reseller, not sure). The device doesn't seem to have a nice name, but I can see UMO19 MCU003 on the microscope, so I guess that's the name or model number.
It can focus on magnifications of 20x or 400x. The image resolution is said to be a max. of 1600x1200, but in practice most of my images are 640x480, maybe I have to change some settings and/or the resolution depends on the magnification factor and lighting conditions.
The device acts as a simple UVC webcam when attached to USB, so you can view the images easily via any compatible webcam software, e.g. luvcview and also save screenshots of the magnified areas (see images).
First three from left to right: SMD LED (400x), clothes/jacket (400x), random PCB (20x). The other two below: A via on a PCB (400x), and the "pixels" of a TFT screen (400x).
It worked out of the box on Linux for me, the uvcvideo kernel driver was loaded automatically.
$ lsusb Bus 001 Device 013: ID 0ac8:3610 Z-Star Microelectronics Corp.
I set up a wiki page for more details (including full lsusb -vvv) and sample images at:
I will also post some more images there over the next few days.
This is a really fun device for having a look at stuff you'd normally not see (or not well enough), and also useful for e.g. checking PCB solder joints, checking all kinds of electronics for errors or missing/misaligned parts, finding the chip name / model number of very tiny chips etc. etc. I can also imagine it's quite nice for biological use-cases, e.g. for studying insects, tissue, plants, and so on.
Anyway, definately a nice toy for relatively low price, I can highly recommend a device like this. Check eBay (search for e.g. "usb mikroskop 400") and various online shops for similar devices, there seem to be a large number of them with different names and from different vendors. Just make sure it has at least 400x magnification, there are also some with only 80x or 200x which is not as useful as 400x, of course.
This is what I set up for backups recently using a cheap USB-enclosure which can house 2 SATA disks and shows them as 2 USB mass-storage devices to my system (using only one USB cable). Without any further introduction, here goes the HOWTO:
First, create one big partition on each of the two disks (/dev/sdc and /dev/sdd in my case) of the exact same size. The cfdisk details are omitted here.
$ cfdisk /dev/sdc $ cfdisk /dev/sdd
Then, create a new RAID array using the mdadm utility:
$ mdadm --create /dev/md0 --level=1 --raid-devices=2 /dev/sdc1 /dev/sdd1
The array is named md0, consists of the two devices (--raid-devices=2) /dev/sdc1 and /dev/sdd1, and it's a RAID-1 array, i.e. data is simply mirrored on both disks so if one of them fails you don't lose data (--level=1). After this has been done the array will be synchronized so that both disks contain the same data (this process will take a long time). You can watch the current status via:
$ cat /proc/mdstat Personalities : [raid1] md0 : active raid1 sdd1 sdc1 1465135869 blocks super 1.1 [2/2] [UU] [>....................] resync = 0.0% (70016/1465135869) finish=2440.6min speed=10002K/sec unused devices:
Some more info is also available from mdadm:
$ mdadm --detail --scan ARRAY /dev/md0 metadata=1.01 name=foobar:0 UUID=1234578:1234578:1234578:1234578 $ mdadm --detail /dev/md0 /dev/md0: Version : 1.01 Creation Time : Sat Feb 6 23:58:51 2010 Raid Level : raid1 Array Size : 1465135869 (1397.26 GiB 1500.30 GB) Used Dev Size : 1465135869 (1397.26 GiB 1500.30 GB) Raid Devices : 2 Total Devices : 2 Persistence : Superblock is persistent Update Time : Sun Feb 7 00:03:21 2010 State : active, resyncing Active Devices : 2 Working Devices : 2 Failed Devices : 0 Spare Devices : 0 Rebuild Status : 0% complete Name : foobar:0 (local to host foobar) UUID : 1234578:1234578:1234578:1234578 Events : 1 Number Major Minor RaidDevice State 0 8 33 0 active sync /dev/sdc1 1 8 49 1 active sync /dev/sdd1
Next, you'll want to create a big partition on the RAID device (cfdisk details omitted)...
$ cfdisk /dev/md0
...and then encrypt all the (future) data on the device using dm-crypt+LUKS and cryptsetup:
$ cryptsetup --verbose --verify-passphrase luksFormat /dev/md0p1 Enter your desired pasphrase here (twice) $ cryptsetup luksOpen /dev/md0p1 myraid
After opening the encrypted container with cryptsetup luksOpen you can create a filesystem on it (ext3 in my case):
$ mkfs.ext3 -j -m 0 /dev/mapper/myraid
That's about it. In future you can access the RAID data by using the steps below.
Starting the RAID and mouting the drive:
$ mdadm --assemble /dev/md0 /dev/sdc1 /dev/sdd1 $ cryptsetup luksOpen /dev/md0p1 myraid $ mount -t ext3 /dev/mapper/myraid /mnt
Shutting down the RAID:
$ umount /mnt $ cryptsetup luksClose myraid $ mdadm --stop /dev/md0
That's all. Performance is shitty due to all the data being shoved out over one USB cable (and USB itself being too slow for these amounts of data), but I don't care too much about that as this setup is meant for backups, not performance-critical stuff.
Update 04/2011: Thanks to Bohdan Zograf there's a Belorussian translation of this article now!