Zugschlusbeobachtungen

Since the mid-eighties, we have been using basically the same PC-style partition table which has been extended a few times: The CHS (cylinder, head, sector) addressing has been changed to LBA (logical block addressing), and the "extended partition" has allowed us to use more than four partitions inside the four partition slots that the classical partition table allows. But all these changes have been backwards compatible, so we were never forced to adapt to something completely new. Big hard disks have recently started to hit a hard 2 TB limit, which cannot be overcome without incompatible changes. The classical PC-style partition table has only 32 bit sector numbers which precludes us from having partitions that reach beyond the 2 TB barrier.

So, an "incompatible" partitioning scheme, the GUID partition table (GPT) has been invented. Unfortunately, if one uses a pure GPT, one cannot boot from this disk with a normal PC BIOS as most vendors have not yet implemented GPT booting yet. Pure doctrine says that a new kind of PC firmware, called EFI firmware, can be used to boot from a GPT disk. However, EFI has not yet found significant market share beyond Apple hardware, so there is a backwards compatibility hack that allows a legacy PC (nearly every box sold today) to boot from a GPT disk. This backwards compatibility hack was what I originally sought to explore, but I had to spend significant time way earlier.

To try booting from a disk that is larger than 2 TB, one first needs a disk that is actually larger than 2 TB. Since the largest single disk available on the market today is just 2 TB big (and very expensive, measured by the Gigabyte per Euro benchmark), I would need to buy at least two disks (preferably the 1.5 TB class which currently delivers the most Gigabytes per Euro) and a method to RAID them together to a RAID 0 which is presented to the host as a single device even at boot time. I would have to shell out about 300 Euros for this, so I reckoned that I might get away with a virtual disk that grows as it is used and thus takes less disk space when it only holds a partition, and a nearly empty file system.

Unfortunately, my favorite virtualization solution, virtualbox, refuses to create a virtual hard disk larger than 2 TB via its GUI which I misinterpreted as a hard limit itself, so I had to resort to qemu which actually can handle a 4 TB virtual disk, which does take very little space if the dynamically-growing qcow2 format is being used.

Unfortunately, I wasn't able to boot qemu from a virtual disk this big. grub 0 refused to install in the first place, and grub 2 could be installed, but it looks like the qemu BIOS refused to even look at the MBR since it just complained that it couldn't read the boot medium. To rule out cylinder number issues, I tried first with a 1000 cylinder partition and later with a 1 cylinder partition, to no avail. Reducing the disk size to less than 2 TB solved all issues, grub 0 and grub 2 worked as advertised.

A day later, I found out that one can create Virtualbox .vdi disks with > 2 TB via the VBoxManage command line tool. After creating a 4 TB .vdi disk, creating a single 1000 cylinder partition and installing grub 2, it agreed to show me its menu just fine, but misbehaved in two interesting ways when I actually tried to boot (see #532202)

Then I tried grub 0 just to be sure. Surprisingly, grub 0 didn't stumble and booted just fine. Next try was using a 2 TB Partition on the 4 TB disk, which is the maximum one can have with a traditional partition table. Strangely, virtualbox' .vdi files grow much more than qemu's qcow2 format, and a 4 TB disk with a 2 TB ext3fs on it needs like 59 GB space on disk. grub 0 boots from this disk just fine, and grml runs fine as well.

So, after like three days of preparation, I am now ready to delve into the depths of a GUID partition table, which I will blog about in a different article.

Revisiting some four months (October 2009) later, I found out that the success I had with grub 0 is not reproducible any more. With current Virtualbox, the .vdi file holding the 4 TB disk with a 2 TB ext3fs is only like 30 GB large, but no grub variant actually boots from it via a normal partition table.