Freie Software

Migrating a Debian installation between architectures has always been difficult. The recommended way to “crossgrade” an i386 Debian to amd64 Debian was to reinstall the system, move over data and configuration. For the more brave, in-place crossgrades usually involved chroots, rescue CDs, a lot of ar p | tar xf - data.tar.gz and luck.

I have never been brave when it comes to system administration, have done a lot of architecture migrations with reinstallation, and have always taken the opportunity to clear out the contamination that accumulates itself when a system is running for a long time. I would even recommend doing this to most people even now. However, I have a few very ugly systems in place that are still on i386 because I didn’t dare going the reinstallation path.

Doing in-place crossgrades has become a lot easier since wheezy’s release, since once now can have both i386 and amd64 libraries installed in parallel, which allows to replace foo:i386 with foo:amd64 without influencing the other parts of the system. The process is still full of pitfalls:

• apt removes a package before it reinstalls its new counterpart. This results in apt calling dpkg to remove dpkg, and then calling dpkg again to install dpkg. Guess which operation fails and the state of the system after this failure. Same applies to coreutils, which leaves the system without rm, which in turn dpkg of either architecture doesn’t like. Using apt-get --download-only install to resolve dependencies and downloading the debs, followed by a traditional dpkg --install solves this issue since multiarch dpkg will replace a package with another one without deinstalling the first one first.
• At least for the process, you need a kernel that can run both 32bit and 64bit binaries for the i386 architecture. AFAIR, setting CONFIG_64BIT, CONFIG_X86_64 and CONFIG_IA32_EMULATION in the kernel configuration takes care of this.
• During the process, apt wants to remove the better part of your system. It is important to not let it do this, as it wants to deinstall essential packages as well.
• It looks like the process is not always exactly reproducible. During the first tries, I found myself without an initrd at all, with an initrd that lacked the ext[234].ko kernel modules, without working e2fstools and in a number of other undesireable states of the system.

I have only tried this yet with a freshly installed minimal wheezy server system. I do not, however, expect surprises when it comes to using this process with “real life” systems. I will document other pitfalls I have fallen into here at a later time. My minimal wheezy system was running in a KVM VM with its virtual disk as a LVM LV in the host system. I took a snapshot before beginning and used lvconvert --merge numerous time to return my LV to the original state. Be aware that lvconvert --merge removes the snapshot after merging it, so you’ll need to re-create the snapshot before trying again.

During the process, I discussed things with Paul Tagliamonte, who has done this before, but on a live system and with a slightly more invasive approach. He has blogged about this. Thank you very much, your hints were very helpful.

In den letzten 24 Stunden habe ich endlich mal wieder was für Debian gemacht: dnstop und sipcalc vom bisherigen Maintainer übernommen, auf Vordermann gebracht und uploaded, und immerhin einen Alibi-Upload von pdns-recursor, damit auch der recursor mit der neuen Maintainer-Mailingliste im Maintainerfeld und dem korrekten Alioth-Vcs-Link im debian/control nach Wheezy kommt.

I have published PowerDNS version 3.1-1.0 on https://ivanova.notwork.de/~mh/debian/pdns/

This is a preliminary package and a release candidate to be 3.1-2 in Debian. If you’re interested in PowerDNS on Debian, please test this package.

I plan to upload next week. This package will vanish from the web server once the package is visible in Debian.

Eight days ago, I uploaded atop 1.26-1 to DELAYED/8, listing me as new maintainer. This means that the package has in the mean time appeared in unstable, and I hope that it’ll swiftly migrate to testing.

Alle Leute sagen, btrfs sei die Zukunft. Es gibt Leute, die einen schon mitleidig angucken, wenn man ihnen sagt, dass man immer noch ext4 einsetzt, wie ich das tue. Aber ich hatte neulich einen Grund, btrfs auszuprobieren. Mit btrfs kann man nämlich Snapshots innerhalb einer verschlüsselten LV einsetzen. Mit ext4 muss man vom Cryptodevice einen Snapshot machen und dann den Snapshot gesondert aufschließen. Damit ist schroot derzeit noch überfordert (#639105).

Also habe ich mal btrfs ausprobiert und musste feststellen, dass es mindestens beim Anlegen eines chroot massiv langsamer ist als ext4. Hier meine Messergebnisse für das Anlegen eines sid-chroot mit debootstrap mit und ohne eatmydata:

Ich muss sagen, ich bin entsetzt. Sowohl darüber, dass btrfs so viel langsamer ist, als auch darüber, dass eatmydata so gut wie nix bringt. Habe ich etwas falsch gemacht? Braucht btrfs beim Erstellen des Dateisystems bzw. beim Einhängen desselben irgend eine magische Option, um in die gleiche Performanceregion wie ext4 zu kommen?

Testumgebung war Debian GNU/Linux sid auf einer KVM VM.

Schon im neunten Eintrag in diesem Blog im Jahr 2005 ging es um munin. Es ist jetzt schon sieben Jahre her, dass ich dieses Tool einsetze. An vielen Stellen nervt es, aber die schlimmsten Macken sind mit der hoffentlich bald erscheinenden (aber auch als beta schon stabil laufenden) 2.0 abgestellt.

Munin 2.0 rechnet die Grafiken nur noch auf Anforderung neu, und mit einer noch mehr im Betastadium befindlichen weiteren Konfigurationsoption gehört auch munin-html der Vergangenheit an.

Bleibt nur noch das Problem, dass munin bei mehr als einer Handvoll Rechnern die Platte foltert. rrdtool rödelt auf den Datenfiles herum wie nichts gutes, und die Platte ist die ganze Zeit über beschäftigt. Auf die Dauer macht das keinen Spaß.

Mit rrdcached kann man die Datensicherheit gegen Geschwindigkeit oder geringere Systembelastung tauschen. munin 2.0 unterstützt rrdcached direkt, und nach wenigen Minuten Konfiguration und ein wenig Gefrickel mit den Permissions landen die fünfminütigen Updates nicht direkt im rrd-File, sondern erstmal im RAM des Munin-Masters. Der rrdcached schreibt die Daten dann auf Anforderung oder nach Ablauf einer bestimmten Zeit. Die Auswirkung des rrdcached sieht man hier:

Die Bilder sprechen für sich.

Im Jahr 1999 habe ich im Rahmen meiner Diplomarbeit ein Framework entwickelt, das flexibel und leistungsfähig die Erstellung von - damals noch - ipfwadm-basierten Firewalls erlaubte. Irgendwann wurde es dann auf iptables aktualisiert und war insgesamt zwölf Jahre lang in zahlreichen Installationen im produktiven Betrieb.

Eben habe ich die letzten zwei Instanzen abgeschaltet. Und ich bin froh darüber.

From the perl DBI manual page:

If you’d like the cache to managed intelligently, you can tie the hashref returned by “CachedKids” to an appropriate caching module, such as Tie::Cache::LRU

And what happens when I don’t do this? Will my cache be unintelligently managed then, with the consequence of my machine exploding when the cache is filled with more than a handful entries?

The main showstopper for IPv6 in my private network environment was the non-availability of IPv6 payload support on OpenVPN’s multi-client server mode. I am using the OpenVPN multi-client server mode exensively with a number of clients, and adding IPv6 to my OpenVPN network would have meant re-building most of it without multi-client server mode. This would mean having a rather dirty construction with one process per client or even **gasp** bridging. I did not have the heart to actually do this and stayed with IPv4.

Thankfully, these times are over: Gert Döring, Thomas Glanzmann, Bernhard Schmidt and Jan Dirnberger spent the better part of the christmas holidays implementing IPv6 payload support in OpenVPN multi-client server mode. They have published a patch against OpenVPN 2.1 and a number of binary packages implementing this feature that I’ve been waiting for.

Unfortunately, the IPv6-over-OpenVPN-multi-client-mode patch clashes with the well-known OpenVPN-over-IPv6 patch, so I had to disable it in my locally patched version of Debian’s OpenVPN package. Bernhard’s binary packages contain both patches.

Enabling IPv6 multi-client server mode is really a breeze. Add server-ipv6 and route-ipv6 statements to your server configuration, and you’re done. Client-config-dir works for IPv6 as well, so I can assign static IPv6 addresses to the clients and tell them to point their IPv6 default route into the tunnel from the server by virtue of a ifconfig-ipv6-push and a push route-ipv6 statement inside the client-config-dir file.

That’s it. Clients with unpatched client software can still connect (and will only get IPv4, just as before), and clients with patched client software will transparently get IPv6 additionally to the IPv4 tunnel. Now, I only have to pay attention again what services are running on my laptop - it’s publicly visible on the intarwebs again.

Guys, your work rocks. I really really appreciate that. Good Job. I owe you more than a beer. Now we only need to convince OpenVPN upstream to accept your patch.

It is well known that apt has an issue when it comes to resolving circular dependencies. Therefore, Debian bug reporters have set out to eradicate circular dependencies from the archive. This does, however, add significant bloat to the actual packages, and I am questioning why this is really necessary.

In the last few days, I found the time to spend some with KVM and libvirt. Unfortunately, there is a subject that I haven’t yet found a satisfying solution: Naming of block devices in guest instances.

This is surely a common issue, but solutions are rare. Neither an article on Usenet (in German) nor the German version of this blog article has found solutions for the main question. I should have written this in English in the first place and am thus translating from German to english, hoping that there will be some answers and suggestions.

KVM is quite inflexible when it coms to configure block devices. It is possible to define on the host, which files or whole devices from the host should be visible in the guest. The documentation suggests that devices should be brought into the guest with the virtio model, which needs suppport in the guest kernel. Importing a device as emulated ATA or SCSI device brings a performance penalty.

The devices brought into the guest via virtio appear in the guest’s dev as /dev/vd<x> and do also have their corresponding entries in /dev/disk/by-uuid and /dev/disk/by-path. The vd<x> node is simply numbered in consecutive order as hd<x> and sd<x>. /dev/disk/by-uuid is the correct UUID of the file system found on the device, at least if it’s a block device partitioned inside the guest and formatted with ext3 (I didn’t try anything else yet). The terminology of the /dev/disk/by-path node is not yet understood, and I am somewhat reluctant to assume the PCI paths of emulated hardware as stable.

Diesen Artikel habe ich schon im Mai 2009 geblogged. Allerdings hat sich grml seitdem so dramatisch weiterentwickelt, dass ich ihn aktualisiert und mit neuem Datum versehen habe.

Update: Das hier beschriebene Verfahren funktioniert seit grml 2011.12 nicht mehr. Der Way to Go is nun grml-rescueboot.

In Mietserver-Recovery mit veraltetem Rescuesystem habe ich beschrieben, wie man grml aus dem komprimierten Image von einer Festplatte booten kann, was zum Beispiel für Rescue-Zwecke an einem Server ohne direkt zugängliche Konsole sehr praktisch sein kann. Zu dieser Zeit (der Artikel stammt aus dem März 2007) war das noch eine größere Operation mit “CD-Image loopback mounten, die einzelnen Dateien rauskopieren und an die richtige Stelle im Dateisystem werfen”, mit neueren grml-Versionen ist es aber noch viel einfacher geworden.

Dear Lazyweb, can anybody with some advanced socat-foo tell me the command line needed to have socat create a socket in the local file system and to listen on it, so that I can have Virtualbox connect a virtual serial console to it?

The material available on socat on the web is sparse, and virtualbox-related docs usually contain “tick the create pipe option”, which is not helpful here since I would like to see the first output the virtual machine prints to its serial port. It would be vastly more useful to have the socket already created with socat listening so that I can immediately see what is being printed to the socket.

Traditionally, the Linux kernel is software that I compile myself from pristine upstream sources for various reasons. I have three major kernel flavours that get built (server, desktop and notebook), and I am pretty current in running a bleeding edge kernel. This is not really necessary any more nowadays, but it’s a tradition that works pretty well.

My kernels get built on sid and are packaged up with kernel-package, and equivs builds a dependency helper package which pulls in the kernel’s dependencies such as initramfs-tools and takes care of cross-version updates like going from 2.6.29 to 2.6.30. Up to now, I was always able to run a kernel built this way on all my systems which can range from oldstable to unstable.

I have been using current KDE since most of my Linux time (having converted over from WindowMaker to KDE 2 back in 2002). But currently, I am seriously pondering to ditch KDE since KDE upstream seems to be wildly decided to kill KDE.

I have accidentally upgraded my desktop box to KDE4 because I missed putting KDE on hold before doing a major sid update after a couple of months. KDE4’s first regression immediately showed itself - the right display doesn’t get any attention from KDE. It just shows up in a grey checkerboard background, it doesn’t have a panel, it doesn’t have a menu, right click doesn’t work. It looks like the only thing one can do with it is dragging windows onto it.

With help of #debian-kde, I quickly found out about this bug in Upstream Bugzilla, which is referred from #529487 and which was marked as Duplicate of this bug in upstream bugzilla, which is one and a half years old and was marked as “severity wishlist”.

Despite the splendid job that the Debian KDE team has done to sort out the KDE4 mess, it looks like KDE upstream has managed to break Dual Head Setups for one and a half years and doesn’t seem to be too interested in providing KDE4 in a way that it can be compared with past versions. This is very sad and will have me shopping for a new desktop environment soon, I am afraid.

Maybe it was not a so good idea to take away KDE 3 so soon and it might have been better to keep KDE 3 in Debian. Maybe it’s time to re-introduce KDE 3 as co-installable packages? I would be willing to participate in this effort as a team member.

Which other Desktop Environments and/or Window Managers should I be shopping for? I’d like to have:

• Dual-Head support (preferably with the possibility to switch desktops only on one display, but that’s something that even KDE 3 cannot do yet)
• Shortcuts like “gg:search words” or “wp:search words” to immediately open google, wikipedia, the BTS or the PTS
• Overlapping windows that are not automatically resized
• A terminal like konsole which allows me to have different session in tabs and to send my input to all tabs
• A clipboard handler that will automatically pop up a window asking me whether I want to open the URL that I just marked in a browser
• Integration with the Debian menu system
I will try adding to this list over the next days when I notice a feature that I have accustomed to so badly that I don’t even notice any more when I’m using it.