By tags:

buildd d-i debian emacs epiphany git hacks ikiwiki kfreebsd pkg-games rcbw release-team sd sys-admin upstream xorg

By date:

Hosted on:

mraw.org

KiBi’s blog/ tags/ d-i

Alternative view: by date.

First release candidate

After a release cycle longer than between each beta (which is slightly sad, but I can’t spend all my time on Debian), here comes the first release candidate of the Debian Installer for Wheezy.

The Debian Installer 7.0 RC1 announce on the website has links to relevant bug reports in the Debian BTS, and the errata page lists things you probably want to know about (including gotchas with grub-install we’ve been having for a while, now diagnosed, and hopefully fixed for the next release candidate).

According to Arwen’s sleepy attitude, it’s officially time for a nap!

Sleepy Arwen

Posted @ 17/02/2013 Tags: d-i
Funny bug numbers

#696699 was already a funny bug number, but #696969 is even better. d-i FTW!

Posted @ 30/12/2012 Tags: d-i
X-mas gift

I’ve been working on some autotesting features for d-i over the past few weeks, and I’ve started lagging behind on my debian-boot@ reading accordingly, by several hundreds of mails. Today’s gift to myself: no more unread/unanswered mails in that mailbox!

Even if I’ve prepared things for a possible d-i wheezy rc1 release around Christmas, several things weren’t ready, most notably: grub-installer. Hopefully #696615 will be fixed soonish, probably by asking where to write when /target(/boot) isn’t on (hd0).

Wanted!

Speaking of grub-installer, if anyone can reproduce #681227, I’m all ears! Wouter has a workaround for it (only in sid for now), but I’d very much like to track down the root cause for that one. It is supposedly reproducible with wheezy beta 4 images and with weekly builds (they use components from testing), all of those are available from the debian-installer website.

Posted @ 25/12/2012 Tags: d-i
d-i hacking recipe #3: Debugging debconf

Problem #3: How to find out what’s going on with debconf?

Slightly longer version: As reported in #679327, the grub prompt during the installation process is entitled “Configuring man-db”, which isn’t exactly great. Given man-db is a known dpkg trigger, I suspected this would be the reason for the broken title, and here’s how I debugged it.

For starters, using d-i basically means asking a lot of questions, which is implemented using the debconf mechanism. One should note that two implementations are available:

  1. the historic implementation, in Perl: debconf
  2. the C reimplementation: cdebconf

d-i uses cdebconf udebs, but joy begins when packages get installed in /target: debconf is used there. We’ll see why that matters.

Recipe #3: Set DEBCONF_DEBUG=developer and read logs!

  1. Both implementations support the DEBCONF_DEBUG environment variable. The installer supports passing that as a kernel parameter; example from the syslinux prompt: select the “Graphical install” entry, press “Tab”, remove -- quiet and put DEBCONF_DEBUG=developer there instead.

  2. Let’s look at /var/log/syslog; there’s a tail -f on it in VT4, but one can run grep and friends on it in VT2 or VT3. There, one can see what happens at the debconf level (the debconf protocol is documented in debconf-devel(7)). Tiny excerpt (timestamps removed for clarity):

    frontend: --> GET preseed/early_command
frontend: <-- 0
…
frontend: --> GET debian-installer/framebuffer
frontend: <-- 0 true
…
debconf: --> GET debconf/language
debconf: <-- 0 en
debconf: --> SET debconf/language en
debconf: Setting debconf/language to en
debconf: <-- 0 value set
debconf: --> GET debconf/priority
debconf: <-- 0 high
debconf: --> GET rescue/enable
debconf: <-- 0 false

  3. The interesting point was titles, right? So let’s look at the last TITLE occurrences:

    debconf: --> SETTITLE debian-installer/main-menu-title
debconf: --> SETTITLE debian-installer/grub-installer/title
in-target: debconf (developer): ----> TITLE Configuration de man-db
debconf: --> TITLE Configuration de man-db
in-target: debconf (developer): <-- TITLE Configuration de man-db
in-target: debconf (developer): ----> TITLE Configuration de man-db
in-target: debconf (developer): ----> TITLE Configuration de grub-pc
debconf: --> TITLE Configuration de grub-pc
in-target: debconf (developer): <-- TITLE Configuration de grub-pc
in-target: debconf (developer): ----> TITLE Configuration de grub-pc
in-target: debconf (developer): ----> TITLE Configuration de man-db
debconf: --> TITLE Configuration de man-db
in-target: debconf (developer): <-- TITLE Configuration de man-db
in-target: debconf (developer): ----> TITLE Configuration de man-db
in-target: debconf (developer): ----> TITLE Configuration de grub-pc
debconf: --> TITLE Configuration de grub-pc
in-target: debconf (developer): <-- TITLE Configuration de grub-pc
in-target: debconf (developer): ----> TITLE Configuration de grub-pc
debconf: --> SETTITLE debian-installer/grub-installer/title

    Bad news: the last one is about grub, so the title should be correct, right?

  4. At this point, it might be the Gtk frontend behaving strangely. It’s shipped in cdebconf-gtk-udeb, built from the cdebconf source. Let’s look:

    • src/modules/frontend/gtk/ui.c has cdebconf_gtk_update_frontend_title(), which does what the function name suggests.
    • src/modules/frontend/gtk/di.c has cdebconf_gtk_di_run_dialog(), which calls the first function.

    • src/modules/frontend/gtk/progress.c has cdebconf_gtk_show_progress() (also calls the first function), which comes with the following comments:

      /** Show the progress widgets.
  *
  * This will actually add the widgets to the corresponding containers.
  * The main title saved when starting the PROGRESS operation will be restored
  * from the value saved when START was called.  This is needed when GO is
  * called during a PROGRESS operation.

    Now that’s something! Even if the last entry is about grub, some previous value could be restored and could be the reason for the bad behaviour (that is easily confirmed by adding some debug_printf() calls on fe->title in cdebconf, thanks to an extra "debug.h" include). Maybe that’s what needs fixing!

  5. Now it’s time to take a step back. A d-i environment is nice, but testing things can easily become a burden. Taking a random wheezy or sid chroot, and installing/removing a tiny package shipping a manpage should be enough to run into the man-db trigger. Exporting DEBCONF_DEBUG=developer there too, let’s either install or remove x11-apps (the trigger does the same job, and the debconf trace is the same), here’s the output:

    Processing triggers for man-db ...
debconf (developer): frontend started
debconf (developer): frontend running, package name is man-db
debconf (developer): starting /var/lib/dpkg/info/man-db.config configure /usr/share/man
debconf (developer): <-- VERSION 2.0
debconf (developer): --> 0 2.0
debconf (developer): <-- INPUT medium man-db/install-setuid
debconf (developer): --> 30 question skipped
debconf (developer): <-- GO
debconf (developer): --> 0 ok
debconf (developer): starting /var/lib/dpkg/info/man-db.postinst triggered /usr/share/man
debconf (developer): <-- VERSION 2.0
debconf (developer): --> 0 2.0
debconf (developer): <-- GET man-db/auto-update
debconf (developer): --> 0 true

    Even if one didn’t know about the cdebconf/debconf thing, the frontend started bit in the d-i syslog would have been the key: it’s only found in debconf, in the frontend script. That one has several ways to determine the package being acted on, and it finally calls:

    debug developer => "frontend running, package name is $package";
$frontend->default_title($package) if length $package;

    Tada, that’s likely to be the issue.

  6. Tweaking that frontend script, it’s easy to make sure that the code path for this use case is:

    elsif ($ARGV[0]=~m!^.*/(.*?)\.(?:postinst|postrm|prerm)$!) {
        $package=$1;
}

    Since $ARGV[1] is the action being passed to the maintainer script, one can intercept the triggered action, and avoid emitting a title update in this case; I’ve therefore reassigned #679327 to the debconf package, and proposed a patch implementing that, which Joey Hess seems to find reasonable.

In my message to the bug report I wrote I actually tested it in a d-i environment, and made sure the issue had gone away. How I did that will likely be described in the next d-i hacking recipe.

Posted @ 23/12/2012 Tags: d-i
Another month, another beta

Today’s recipe:

  • a day off
  • a long night of sleep
  • acid jazz music
  • some 24 episodes
  • coffee
  • trout and pasta
  • sleepy cats

Result: Debian Installer 7.0 Beta4 release!

Speaking of which, shameless plug: I'm giving a Debian Installer talk at mini-DebConf Paris this sunday.

(Update: slides & LaTeX source for this talk.)

Posted @ 22/11/2012 Tags: d-i
d-i hacking recipe #2: Adding a wireless module

Problem #2: How to add a wireless module to a d-i image?

Slightly longer version: As reported in #686605, a whole wireless module family is missing from the linux kernel udebs, meaning they’re missing from the d-i images. Please note the hypothesis here is that the regular linux kernel images contain the relevant modules, only the udebs don’t.

Recipe #2: Craft a local udeb!

  1. The following assumes one is interested in adding a wireless module, but that can be adapted to other module families. The relevant udeb base name is nic-wireless-modules.
  2. When it comes to linux kernel modules, picking the right ones is important, so one has to consider the ABI. Right now, the current ABI in testing is 3.2.0-3-amd64. The complete udeb name is therefore nic-wireless-modules-3.2.0-3-amd64-di.
  3. If you’re running a kernel with the same ABI, all good, relevant modules should be available under /lib/modules/$(uname -r), which is what will be assumed below. Otherwise, one would have to download a proper linux-image-$ABI package, and extract it into a temporary directory to grab the wanted module(s).

  4. Determine the relevant module and its dependencies. As an example, I’ll pick a Realtek module: rtl8192cu.ko. If it’s loaded, one can use the following to detect its dependencies:

    $ lsmod|grep rtl8192cu
rtl8192cu              78863  0
rtlwifi                81350  1 rtl8192cu
rtl8192c_common        52602  1 rtl8192cu
mac80211              192768  4 iwlwifi,rtl8192c_common,rtlwifi,rtl8192cu
cfg80211              137140  3 mac80211,iwlwifi,rtlwifi
usbcore               128498  7 ehci_hcd,usbhid,uvcvideo,rtlwifi,rtl8192cu

    But it isn’t even needed to load the module to get those dependencies:

    $ /sbin/modinfo /lib/modules/3.2.0-3-amd64/kernel/drivers/net/wireless/rtlwifi/rtl8192cu/rtl8192cu.ko|grep ^depends
depends:        rtlwifi,mac80211,rtl8192c-common,usbcore

    Compared to what the d-i image contains, only rtlwifi and rtl8192c-common are missing.

  5. Fetch the udeb using http://packages.debian.org/ or browsing your favourite mirror if you know the paths by heart.

  6. Extract the udeb into a temporary data/ directory, along with its meta data:

    dpkg -x nic-wireless-modules-3.2.0-3-amd64-di_3.2.23-1_amd64.udeb data
dpkg --control nic-wireless-modules-3.2.0-3-amd64-di_3.2.23-1_amd64.udeb data/DEBIAN

  7. Copy the 3 modules under the relevant subdirectory under data/.

  8. Update the version in data/DEBIAN/control, for example by adding the +local1 suffix.

  9. Build a new binary package, using fakeroot to avoid owner/group noise in the diff:

    fakeroot dpkg -b data nic-wireless-modules-3.2.0-3-amd64-di_3.2.23-1+local1_amd64.udeb

  10. Use debdiff to check the results:

    $ debdiff nic-wireless-modules-3.2.0-3-amd64-di_3.2.23-1*_amd64.udeb
Files in second .deb but not in first
-------------------------------------
-rw-r--r--  root/root   /lib/modules/3.2.0-3-amd64/kernel/drivers/net/wireless/rtlwifi/rtl8192cu/rtl8192c-common.ko
-rw-r--r--  root/root   /lib/modules/3.2.0-3-amd64/kernel/drivers/net/wireless/rtlwifi/rtl8192cu/rtl8192cu.ko
-rw-r--r--  root/root   /lib/modules/3.2.0-3-amd64/kernel/drivers/net/wireless/rtlwifi/rtlwifi.ko


Control files: lines which differ (wdiff format)
------------------------------------------------
Version: [-3.2.23-1-] {+3.2.23-1+local1+}

  11. Put the resulting udeb under localudebs/ in your d-i build tree, and rebuild your image as usual. Almost-tada!

Why “almost”? Because a firmware is needed, so one can go back to the 7th step, extract the proper file (lib/firmware/rtlwifi/rtl8192cufw.bin) from the non-free firmware-realtek package, and add it under data/lib/firmware/realtek/. Rebuild the udeb, rebuild the d-i image. Then: tada!

Posted @ 01/10/2012 Tags: d-i
d-i hacking recipe #1: Extracting logs without network

Foreword: This is the first blog post of a mini-series, trying to document some recipes about Debian Installer hacking. VirtualBox will probably be mentioned on a regular fashion, but most of those recipes should be easily adaptable to qemu/kvm.

Problem #1: How to share some d-i logs without using the network?

Slightly longer version: Investigating some issues on a wired-less virtual machine (Network → Adapter 1 → Disable Network Adapter), it’s hard to get log files out of the virtual machine.

Recipe #1: Use serial ports!

Serial Ports → Port 1 → Enable Serial Port, then pick Raw File as Port Mode, and specify a file on the host system to gather logs.

There’s nothing to do on d-i side, the serial console appears automatically as /dev/ttyS0. From a console, one can extract the needed bits:

cp /var/log/syslog /dev/ttyS0
dmesg > /dev/ttyS0
Posted @ 01/10/2012 Tags: d-i
Bus factor

Stuff/team I’m currently involved with:

  • Debian System Administration: I’m just in the porter-* groups, so that I can fulfill installation requests in various chroots on Debian porterboxes; this means real DSA members can concentrate on more important things while still keeping the turnaround quick for developers who want to debug stuff on exotic architectures. Reminder for that procedure: Chroot Install Request Guidelines.

  • After having maintained the kfreebsd-* buildds for a while, I’m still in the wanna-build team, meaning I can take care of give backs, one of the most common requests on debian-wb-team@. It’s been a while since it’s been split up from debian-release@, and instructions are still available in wanna-build.txt. Basically: anything buildd-related goes to $arch@buildd.debian.org or debian-wb-team@lists.debian.org, with the sole exception of binNMUs, which are handled by the release team.

  • Speaking of which, I joined the release team a few months ago; plenty of (not-so-)funny things to do, like coordinating transitions to testing, and more recently, dealing with unblock requests during the freeze. While the Debian world is usually driven by the “just do” motto, there’s one area where one has to learn to say “no, sorry”. Hopefully people won’t hate us for that, and we’ll find alternative ways (like backporting specific fixes, or postponing new features until wheezy-backports is open).

  • Given nobody on debian-boot@ stepped up to prepare new debian-installer releases, I figured I would give a hand there until somebody more involved with d-i would manage that on the long run. That’s a lot of heavy work, trying to get all involved packages into shape at a given moment, and getting people to fix their stuff when anything breaks. Hopefully that’ll work out and once CD images are ready, we should be able to run some more tests on those before officially publishing d-i beta 1.

  • And since the CD team is also understaffed, I joined Steve McIntyre a while ago during a point release to see how that was done, and how I could help. Things drove me to the other teams above, so I couldn’t help much so far, but at least I managed to build a few test images on pettersson (the very nice machine on which images are built), using the debian-cd account. The intent was to make sure debian-installer could safely migrate from unstable to testing. That has happened lately, so images are being prepared.

  • And of course I’m still maintaining the whole X stack, with the help of a few other guys on specific areas.

I don’t feel like getting hit by a bus, but I would hate burning out and going away from all this.

People have called repeatedly for help in core packaging team. And that has been true for the X Strike Force for a while (hint: Julien only managed to pull me into this because I ported the Graphical Installer from DirectFB to X11).

The same holds with non-packaging teams. I’m pretty sure Steve would love to see new members in debian-cd. I’m also pretty sure having more people involved in the d-i team would be very nice, especially if one could find out how to make the whole release process less insane than it is currently. I guess we could try out new ideas right after the release; it feels to me that right now is just not the time to fight the current release process.

Posted @ 15/07/2012 Tags: d-i
D-I Wheezy Alpha1

Get it while it’s hot: Debian Installer 7.0 Alpha1 release.

See how yummy it is:

Posted @ 13/05/2012 Tags: d-i
Debian XSF News #10

This is the tenth “Debian XSF News” issue. It is basically meant to be a follow-up to DXN#9.

  1. I completed the addition of the Sinhala language, and also dealt with big changes on the xkb-data (a.k.a. xkeyboard-config) side, as explained by Sergey Udaltsov.

    Protocol:

    • [KiBi] x11proto-core: the upload to experimental was a bit overcautious, so upload again → unstable.

    Data:

    Library:

    • [KiBi] libx11: new upstream release, adding support for Sinhala → unstable.

  2. The Graphical Installer started receiving improved portability. There were a few patches floating around adding udebs for generic drivers for non-Linux architectures. I cleaned them up a bit (especially on the vesa side, getting rid of libdrm entirely), got them test-built on a kfreebsd-amd64 box, while Samuel Thibault was doing the same on a hurd-i386 box. I raised the topic on debian-boot@, debian-bsd@, and debian-hurd@, mentioning the remaining steps.

    Drivers (all accepted by ftpmasters after a tiny trip to the NEW queue):

  3. Here are some other packages which got updated.

    Applications:

    Library:

    • [Chris,Julien,KiBi] mesa 7.10.1: long-awaited upload, finally checked, built, and uploaded → experimental.

  4. Finally, let’s mention migration to testing (from now on I’ll use testing instead of wheezy so that it’s clear it’s about week after week migrations to the testing distribution, rather than our final plans for the next stable release). After a while, and thanks to some hand-holding from our nice release managers (hi, Adam!), linux-2.6, libdrm, mesa, xorg-server, and several dozen drivers migrated to testing. Yay!

  5. As a consequence, some other packages got updated shortly after the successful britney run (because they were prepared right before, just in case).

    Libraries:

    • [KiBi] libdrm: replace 2.4.23 with 2.4.24 (which was in experimental previously) → unstable.

    • [KiBi] mesa: new stable release, 7.10.2 supersedes both 7.10 in unstable and 7.10.1 in experimental (see above) → unstable.

    • [KiBi] pixman: new development snapshot, 0.21.6 comes from experimental and supersedes 0.21.4 → unstable.

    Drivers:

Posted @ 11/04/2011 Tags: d-i