Zugschlusbeobachtungen (Entries tagged as kernel)

Unified Kernel for etch, lenny and sid

mh+blog-zugschlus-de@zugschlus.de (Marc 'Zugschlus' Haber) — Thu, 06 Aug 2009 10:44:00 +0200

Traditionally, the Linux kernel is software that I compile myself from pristine upstream sources for various reasons. I have three major kernel flavours that get built (server, desktop and notebook), and I am pretty current in running a bleeding edge kernel. This is not really necessary any more nowadays, but it’s a tradition that works pretty well.

My kernels get built on sid and are packaged up with kernel-package, and equivs builds a dependency helper package which pulls in the kernel’s dependencies such as initramfs-tools and takes care of cross-version updates like going from 2.6.29 to 2.6.30. Up to now, I was always able to run a kernel built this way on all my systems which can range from oldstable to unstable.

This has recently become a bit harder when unstable’s udev started to complain that the sysfs layout was deprecated and that I should unset CONFIG_DEPRECATED_SYSFS in my kernel configuration or lose some of udev’s functionality, since a kernel compiled without CONFIG_DEPRECATED_SYSFS doesn’t properly boot on etch.

Since I wanted to avoid building even more kernel flavours, I decided to investigate which package I need to backport to etch to get kernels without CONFIG_DEPRECATED_SYSFS to work on etch. To my surprise, the cause for not booting on etch was not udev, but lvm2. With lvm2 2.02.39 (and the corresponding libdevmapper, 1.02.27-4) ported back from lenny to etch, my etch systems boot fine with CONFIG_DEPRECATED_SYSFS unset. udev can happily stay at etch’s version, 0.105-4etch1.

From etch to lenny to squeeze, the lvm2 package has evolved quite a bit. Since there is only one LVM flavour left, the lvm-common package vanished in the transition from etch to lenny, and libdevmapper still has its own source package in lenny, but is built from lvm2’s source package itself in squeeze. So, as a heads-up, there needs to be special handling for lvm-common when installing the backport on etch.

I decided to backport lvm2 from lenny to etch to allow this particular derivation from the distributions’s versions to vanish in reasonably short time when my last etch boxes get finally updated to lenny. And my dependency helper has been extended to now pull in the backported version of lvm2 to avoid nasty reboot surprises.

Wenn ein Alarmzeichen nicht reicht...

mh+blog-zugschlus-de@zugschlus.de (Marc 'Zugschlus' Haber) — Tue, 25 Mar 2008 19:54:11 +0100

Kernel 2.6.24.4 wurde gebaut und die fertigen Packages auf die Distributionen verteilt. torres, die als “testetch”-System neue Kernels als erstes stable-System bekommt, will sich die Package nicht automatisch ziehen.

Als ob das nicht schon genug Alarmsignal ist, sich das mal genauer anzugucken, prügle ich die Kernel-Package manuell rein und wunder mich, warum ein Haufen anderer Packages mitkommt und das System danach beginnt, eine initrd zu bauen.

Als ob das nicht schon genug Alarmsignal ist, boote ich die Kiste durch und wundere mich, warum sie nicht ins Netz kommt. Glücklicherweise ist torres nicht ohne Grund Testsystem der ersten Schicht: Sie hat eine serielle Konsole. Dort zeigt’s mir, dass die Kiste sehr wohl läuft, aber kein Netz hat.

Da torres einen E100 hat, kann ich mir das nicht so wirklich vorstellen. Aber nein, da ist wirklich kein Interface in der Ausgabe von “ip addr”. Aber in der Ausgabe von lspci ist es sehr wohl sichtbar.

Dann gucke ich mir zum ersten Mal /boot etwas genauer an.

Und sehe, dass ich einen Kernel, der eigentlich für mein notebook gedacht war, auf torres installiert habe. **tischkante beiss**

Als ob eins der oben genannten Alarmzeichen nicht gereicht hätte. An manchen Tagen sollte man mich echt nicht an Computer lassen.

Beware of 2.6.23.x kernel on systems that were installed a long time ago

mh+blog-zugschlus-de@zugschlus.de (Marc 'Zugschlus' Haber) — Sat, 17 Nov 2007 22:07:34 +0100

In a nutshell: If your system is kind of older than sarge (as in installation date, updates done in the mean time don’t matter), beware of 2.6.23.x or update your grub boot sector, which Debian doesn’t do automatically on package installation.

After 2.6.23.1 lasting for more than a month, I decided on wednesday to finally roll 2.6.23.1 out to most systems. Only to be overrun by seven releases in a row (which even kernel.org infrastructure not properly handled). Ok, so it’s 2.6.23.8.

Installation of kernel on my test and semi-productive systems went just fine, so I took the opportunity of being in the datacenter for some unrelated matter on a saturday to update two productive boxes. Just to find them not coming up again. The first box, not connected to a KVM switch, but only to a serial console, looked like it simply died at the place where grub usually says “Loading Linux...”, while the second box, on a “real” KVM switch, was a little more helpful, saying “No setup signature found” (or something along this wording).

Since there was no internet on site with the boxes down (one being the main router, the second one being the full service DNS resolver), I quickly chose the fall-back kernel and got back in business. Googling revealed that this is an issue connected with old grub and kernel 2.6.23, and indeed, running grub-setup brought both systems back online. Had I done this - as usual - from remote, this would have been an unexpected downtime of at least half an hour.

Since all systems in question have the same version of grub (the one from etch) installed, I proceeded to investigate what has been going wrong here. Finding that grub does not have any maintainer scripts at all, it would have been my responsiblity to update the grub code in the boot sectors to the new version after updating to etch. The package didn’t bother doing this, and this is also not documented in README.Debian or NEWS.Debian. As this is a major pitfall for systems that have been installed pre-sarge, and since updating the Distribution does not solve the issue, I filed Bug #451701. I hope that this blog entry saves other people from unexpected downtimes.

Ah, yes, and barfing the error message on an unused console is a bad foul of grub, see #451710

Kernelupdate bei $KUNDE

mh+blog-zugschlus-de@zugschlus.de (Marc 'Zugschlus' Haber) — Tue, 13 Dec 2005 21:45:24 +0100

$KUNDE ist mein ältester Begleiter: Er lässt sich inzwischen seit über fünf Jahren von mir betreuen und ist mir über drei Arbeitgeber treu gefolgt. Die Zusammenarbeit ist prima, und natürlich ist es mir extrapeinlich, wenn mal etwas nicht funktioniert. Auch wenn ich gar nicht schuld bin.

Das vorletzte Kernelupdate hat auf vier der fünf baugleichen Maschinen problemlos funktioniert. Nur die wichtigste, die äußere Firewall, kommt nicht wieder ans Netz. Ok, dann ist da wohl doch etwas nicht baugleich. Vorher hat es funktioniert, also rollen wir zurück: $KUNDE, bitte einmal auf $MASCHINE den alten Kernel wieder booten. Geht auch nicht. Mist. grml booten. Geht auch nicht. Was? Wir beginnen, das Netz zu debuggen. Nach dem Tausch des zum Intenet führenden Switches tut es wieder.

Beim letzten Kernelupdate der Zirkus wieder. Diesmal lasse ich den Fehler gleich im äußeren Netz suchen; nach Tausch der beiden On-Board-Interfaces untereinander und Einsatz eines neuen Patchkabels geht es wieder.

Heute waren wir vorsichtiger. Wir setzen ein extralanges Servicefenster an und natürlich kommt $MASCHINE nach dem Reboot nicht wieder ans Netz. Aber diesmal hilft kein wildes Hin- und Hertauschen irgendwelcher Komponenten. Irgendwann platzt mir der Kragen, und ich bitte den Kunden, den Rechner zu tauschen. Die neue Hardware kommt mit dem grml-System direkt und sofort ans Netz, und nach Eintragung der neuen MAC-Adressen ins Firewall-Setup war auch das
“richtige” System schnell wieder einsatzbereit.

Dann musste nur noch kurz am bind gedreht werden, weil neuerdings die Antworten an die von der IP-Adresse des “inneren” Interfaces nach drauße verschickten Queries zwar beim Host, nicht aber beim Prozess ankommen. Das ist wohl neue Kernel-Paranoia, der ich demnächst im Testlab mal zusammen mit den neuen Absonderlichkeiten beim Policy-Routing auf den Grund gehen muss.

Thoughts about the Debian kernel

mh+blog-zugschlus-de@zugschlus.de (Marc 'Zugschlus' Haber) — Sun, 23 Oct 2005 16:07:39 +0200

I am one of the guys who builds Linux kernels locally, from vanilla sources. What I don’t like in this approach is that I do not get the distribution patches and might miss one of the kernel security patches, since I am way too busy to keep track of LKML any more. otoh, I am kind of a version number junkie when it comes to the kernel, so the Debian kernel sources even in sid frequently are not current enough. So, what I want to have is a compromise between a vanilla kernel and the Debian distribution kernels, built in a way that the images integrate well with Debian.

This article contains a few questions and wishes directed towards the Debian kernel team.

I have tried to get some of them answered on #debian-kernel, but the people there were quite busy with other things and didn’t answer. And while I was thinking and thinking more, and playing with the code, there were more and more questions until they became inappropriate for IRC. This is also my first try to get some discussion about technical Debian matters in the blogosphere, and I might use the insight gained here for a posting on the debian-kernel mailing list by the end of next week.

The build process is not very transparent
Documentation in the README files seems quite incomplete
In my opinion, answers to these questions are missing:
- Which steps happen in which order (prose)?
- Are there any hooks to interfere with the build process?
- How to keep patches from being applied?
- How to add local patches?
- Is there anything like dpatch-edit-patch for the (home-grown?) patch system in the Debian kernel source package?
- How do I control generation of the kernel-image-2.x-<flavour><u>2.x.y-z</u><arch>.deb helper packages? They do not seem to be controlled by debian/arch/<arch>/defines as the real kernel debs do.
- Can I have patches from a kernel-patch-foo Package automatically applied for certain flavours?
- Are there hooks for building external modules?
- Are there debian/rules parameters or environment variables to select only a certain kernel to be built (like for debugging problems)?
- Can build of helper packages (-headers, -doc, -patch, -source, -tree) be disabled?
- For local kernel builds, should one use the Debian kernel build system, or continue to use make-kpkg as it was usual previously?
there is nothing like a kernel HOWTO
The Kernel Handbook needs to be fleshed out in these regards. I might want to contribute once I have accumulated the knowledge needed to write the passages.

Patches like amd64-int3-fix need to be better commented
I think it is necessary that a patch file contains information
- What does the patch do?
- why is it applied?
- is it necessary only on certain archs?
- is it necessary only if certain drivers are in use?
- what does happen when it is omitted?
- is it security relevant?
- CAN number, if applicable.
Which role does module-assistant play here?
If one builds kernels with make-kpkg, should make-kpkg also build the modules, or should one use module-assistant instead?

Debian Linux Kernel Handboook

mh+blog-zugschlus-de@zugschlus.de (Marc 'Zugschlus' Haber) — Sat, 01 Oct 2005 22:00:14 +0200

Jurij Smakov et al have published the Debian Linux Kernel Handbook, which will help in documenting the internals of the Debian Linux Kernel build process. The document is still work in progress with a lot of sections missing, but it’s a giant step in the right direction.

Outside of the already finished document structure, I’d like to see the following things documented:

How to work with the Debian kernel svn

Documentation of the Debian kernel patches themselves, besides the succinct “fix off-by-one-error in foo.c”. I’d like to see at least CAN numbers, and/or links to a document which allows one to choose whether to take a certain patch into a local kernel or not. The Debian kernel team has repeatedly rejected that request on grounds that they don’t like to support cherry picking, which I find pretty disappointing.

I am also mildly disturbed that kernel-package, Manojs helper to build Linux kernel binaries seems to have fallen out of the kernel team’s process completely, which makes me think why we have such a great and helpful external tool in the first place when we’re not even using it ourselves.

At least, the document shows clearly where the Debian Linux Kernels are going to, and that I’ll have to adapt my local kernel building processes to get nearer to Debian’s official way of doing things.