Booting from a large hard disk
I recently had to install Openfiler on a HP server with ten 750 GB hard disks on a cciss RAID controller, which proved to be a major nuisance. Since the customer wanted the box in service fast, I finally settled on wasting two of the disks as a 750 GB RAID 1 for the actual system (with like 10 GB actually used) while RAIDing the remaining disks together to a RAID 6 with spare disk for productive data.
During this task, I noticed a severe lack of current knowledge about modern PC architecture and how to boot from a big hard disk and decided to do some research into this direction. This article shows the first "results" that I have achived in the last few days.
Since the mid-eighties, we have been using basically the same PC-style partition table which has been extended a few times: The CHS (cylinder, head, sector) addressing has been changed to LBA (logical block addressing), and the "extended partition" has allowed us to use more than four partitions inside the four partition slots that the classical partition table allows. But all these changes have been backwards compatible, so we were never forced to adapt to something completely new. Big hard disks have recently started to hit a hard 2 TB limit, which cannot be overcome without incompatible changes. The classical PC-style partition table has only 32 bit sector numbers which precludes us from having partitions that reach beyond the 2 TB barrier.
So, an "incompatible" partitioning scheme, the GUID partition table (GPT) has been invented. Unfortunately, if one uses a pure GPT, one cannot boot from this disk with a normal PC BIOS as most vendors have not yet implemented GPT booting yet. Pure doctrine says that a new kind of PC firmware, called EFI firmware, can be used to boot from a GPT disk. However, EFI has not yet found significant market share beyond Apple hardware, so there is a backwards compatibility hack that allows a legacy PC (nearly every box sold today) to boot from a GPT disk. This backwards compatibility hack was what I originally sought to explore, but I had to spend significant time way earlier.
To try booting from a disk that is larger than 2 TB, one first needs a disk that is actually larger than 2 TB. Since the largest single disk available on the market today is just 2 TB big (and very expensive, measured by the Gigabyte per Euro benchmark), I would need to buy at least two disks (preferably the 1.5 TB class which currently delivers the most Gigabytes per Euro) and a method to RAID them together to a RAID 0 which is presented to the host as a single device even at boot time. I would have to shell out about 300 Euros for this, so I reckoned that I might get away with a virtual disk that grows as it is used and thus takes less disk space when it only holds a partition, and a nearly empty file system.
Unfortunately, my favorite virtualization solution, virtualbox, refuses to create a virtual hard disk larger than 2 TB via its GUI which I misinterpreted as a hard limit itself, so I had to resort to qemu which actually can handle a 4 TB virtual disk, which does take very little space if the dynamically-growing qcow2 format is being used.
Unfortunately, I wasn't able to boot qemu from a virtual disk this big. grub 0 refused to install in the first place, and grub 2 could be installed, but it looks like the qemu BIOS refused to even look at the MBR since it just complained that it couldn't read the boot medium. To rule out cylinder number issues, I tried first with a 1000 cylinder partition and later with a 1 cylinder partition, to no avail. Reducing the disk size to less than 2 TB solved all issues, grub 0 and grub 2 worked as advertised.
A day later, I found out that one can create Virtualbox .vdi disks with > 2 TB via the VBoxManage command line tool. After creating a 4 TB .vdi disk, creating a single 1000 cylinder partition and installing grub 2, it agreed to show me its menu just fine, but misbehaved in two interesting ways when I actually tried to boot (see #532202)
Then I tried grub 0 just to be sure. Surprisingly, grub 0 didn't stumble and booted just fine. Next try was using a 2 TB Partition on the 4 TB disk, which is the maximum one can have with a traditional partition table. Strangely, virtualbox' .vdi files grow much more than qemu's qcow2 format, and a 4 TB disk with a 2 TB ext3fs on it needs like 59 GB space on disk. grub 0 boots from this disk just fine, and grml runs fine as well.
So, after like three days of preparation, I am now ready to delve into the depths of a GUID partition table, which I will blog about in a different article.
Revisiting some four months (October 2009) later, I found out that the success I had with grub 0 is not reproducible any more. With current Virtualbox, the .vdi file holding the 4 TB disk with a 2 TB ext3fs is only like 30 GB large, but no grub variant actually boots from it via a normal partition table.
Display comments as Linear | Threaded
ssam on :
i have an old x31 thinkpad (Pentium M 1.4GHz era) into which i recently put a 160GB hard disk. just to see what happened I partitioned the disk as GPT.
everything works fine. fedora 10 and 11, and ubuntu 8.10 and 9.04 install and boot fine on it without any extra effort. the only thing i have found that has an issue is fdisk:
$sudo fdisk -l WARNING: GPT (GUID Partition Table) detected on '/dev/sda'! The util fdisk doesn't support GPT. Use GNU Parted. Disk /dev/sda: 160.0 GB, 160041885696 bytes 255 heads, 63 sectors/track, 19457 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Disk identifier: 0x00000000 Device Boot Start End Blocks Id System /dev/sda1 1 19458 156290903+ ee GPT
maybe the issues only crop up when you have big disks.
ch on :
Booting from a >2TB disk should "just work fine", provided that you use grub2 from lenny, make sure that the GPT partition in the MBR is there and active, and that the GPT partition entries are valid. Also note that you need to reinstall grub2 everytime you used parted (or anything which uses libparted, like gnu-fdisk).
I don't currently have such a >2TB setup with cciss, but with other raid controllers though.
Hans Bonfigt on :
Hi Marc, you've been in lack of WHAT ?
Knowledge of "modern PC Architecture" ?
Since when can one call the obsolete CHS Interface with its messy 'LBA' - Flavour a modern technology ? Together with the sick VTOC, which DOS Lamers often falsely call 'partition table', the PC BIOS stinks since its invention more than 20 years ago.
In addition: Do you really think that "architecture" is a word really describing the real PC junk ?
Why don't you put large disks not ino a PC, but in a server, for example an ibm pSeries ? In the meanwhile, the new POWER6 models are cheaper than the ordinary Dell, HP or Lenovo junk.
As an alternative, why didn't you put an IDE Flash into the PC, containing the Kernel with compiled-in DASD drivers or an appropriate initrd ?