KiBi’s blog

Executive summary

It’s perfectly possible! Jump to the HTML demo!

Longer version

This started with a very simple need: wanting to improve the notifications I’m receiving from various sources. Those include:

changes or failures reported during Puppet runs on my own infrastructure, and on at a customer’s;
build failures for the Debian Installer;
changes in banking amounts;
and lately: build status for jobs in a customer’s Jenkins instance.

I’ve been using plaintext notifications for a number of years but I decided to try and pimp them a little by adding some colors.

While the XMPP-sending details are usually hidden in a local module, here’s a small self-contained example: connecting to a server, sending credentials, and then sending a message to someone else. Of course, one might want to tweak the Configuration section before trying to run this script…

#!/usr/bin/perl
use strict;
use warnings;

use Net::XMPP;

# Configuration:
my $hostname = 'example.org';
my $username = 'bot';
my $password = 'call-me-alan';
my $resource = 'demo';
my $recipient = 'human@example.org';

# Open connection:
my $con = Net::XMPP::Client->new();
my $status = $con->Connect(
    hostname       => $hostname,
    connectiontype => 'tcpip',
    tls            => 1,
    ssl_ca_path    => '/etc/ssl/certs',
);
die 'XMPP connection failed'
    if ! defined($status);

# Log in:
my @result = $con->AuthSend(
    hostname => $hostname,
    username => $username,
    password => $password,
    resource => $resource,
);
die 'XMPP authentication failed'
    if $result[0] ne 'ok';

# Send plaintext message:
my $msg = 'Hello, World!';
my $res = $con->MessageSend(
    to   => $recipient,
    body => $msg,
    type => 'chat',
);
die('ERROR: XMPP message failed')
    if $res != 0;

For reference, here’s what the XML message looks like in Gajim’s XML console (on the receiving end):

<message type='chat' to='human@example.org' from='bot@example.org/demo'>
  <body>Hello, World!</body>
</message>

Issues start when one tries to send some HTML message, e.g. with the last part changed to:

# Send plaintext message:
my $msg = 'This is a <b>failing</b> test';
my $res = $con->MessageSend(
    to   => $recipient,
    body => $msg,
    type => 'chat',
);

as that leads to the following message:

<message type='chat' to='human@example.org' from='bot@example.org/demo'>
  <body>This is a &lt;b&gt;failing&lt;/b&gt; test</body>
</message>

So tags are getting encoded and one gets to see the uninterpreted “HTML code”.

Trying various things to embed that inside <body> and <html> tags, with or without namespaces, led nowhere.

Looking at a message sent from Gajim to Gajim (so that I could craft an HTML message myself and inspect it), I’ve noticed it goes this way (edited to concentrate on important parts):

<message xmlns="jabber:client" to="human@example.org/Gajim" type="chat">
  <body>Hello, World!</body>
  <html xmlns="http://jabber.org/protocol/xhtml-im">
    <body xmlns="http://www.w3.org/1999/xhtml">
      <p>Hello, <strong>World</strong>!</p>
    </body>
  </html>
</message>

Two takeaways here:

The message is send both in plaintext and in HTML. It seems Gajim archives the plaintext version, as opening the history/logs only shows the textual version.
The fact that the HTML message is under a different path (/message/html as opposed to /message/body) means that one cannot use the MessageSend method to send HTML messages…

This was verified by checking the documentation and code of the Net::XMPP::Message module. It comes with various getters and setters for attributes. Those are then automatically collected when the message is serialized into XML (through the GetXML() method). Trying to add handling for a new HTML attribute would mean being extra careful as that would need to be treated with $type = 'raw'…

Oh, wait a minute! While using git grep in the sources, looking for that raw type thing, I’ve discovered what sounded promising: an InsertRawXML() method, that doesn’t appear anywhere in either the code or the documentation of the Net::XMPP::Message module.

It’s available, though! Because Net::XMPP::Message is derived from Net::XMPP::Stanza:

use Net::XMPP::Stanza;
use base qw( Net::XMPP::Stanza );

which then in turn comes with this function:

##############################################################################
#
# InsertRawXML - puts the specified string onto the list for raw XML to be
#                included in the packet.
#
##############################################################################

Let’s put that aside for a moment and get back to the MessageSend() method. It wants parameters that can be passed to the Net::XMPP::Message SetMessage() method, and here is its entire code:

###############################################################################
#
# MessageSend - Takes the same hash that Net::XMPP::Message->SetMessage
#               takes and sends the message to the server.
#
###############################################################################
sub MessageSend
{
    my $self = shift;

    my $mess = $self->_message();
    $mess->SetMessage(@_);
    $self->Send($mess);
}

The first assignment is basically equivalent to my $mess = Net::XMPP::Message->new();, so what this function does is: creating a Net::XMPP::Message for us, passing all parameters there, and handing the resulting object over to the Send() method. All in all, that’s merely a proxy.

HTML demo

The question becomes: what if we were to create that object ourselves, then tweaking it a little, and then passing it directly to Send(), instead of using the slightly limited MessageSend()? Let’s see what the rewritten sending part would look like:

# Send HTML message:
my $text = 'This is a working test';
my $html = 'This is a <b>working</b> test';

my $message = Net::XMPP::Message->new();
$message->SetMessage(
    to   => $recipient,
    body => $text,
    type => 'chat',
);
$message->InsertRawXML("<html><body>$html</body></html>");
my $res = $con->Send($message);

And tada!

<message type='chat' to='human@example.org' from='bot@example.org/demo'>
  <body>This is a working test</body>
  <html>
    <body>This is a <b>working</b> test</body>
  </html>
</message>

I’m absolutely no expert when it comes to XMPP standards, and one might need/want to set some more metadata like xmlns but I’m happy enough with this solution that I thought I’d share it as is. ;)

Posted @ 17/08/2019 Tags: debian hacks

I haven’t been posting anything on my personal blog in a long while, let’s fix that!

Partial reason for this is that I’ve been busy documenting progress on the Debian Installer on my company’s blog. So far, the following posts were published there:

Debian Installer: Stretch Alpha 8 released with details on the release process, and on the debootstrap attempt regarding merged-/usr (granted, that one was from late 2016).
Debian Installer: Stretch RC 2 released: wrapping up both RC 1 and RC 2 for Stretch, mentioning major changes instead of all the tiny details one would usually find in the release announcements published on the debian-devel-announce@ mailing list.
Debian Installer: Stretch released: aggregating RC 3 to RC 5 this time, since the last few weeks before the Stretch release date were quite busy!

After the Stretch release, it was time to attend DebConf’17 in Montreal, Canada. I’ve presented the latest news on the Debian Installer front there as well. This included a quick demo of my little framework which lets me run automatic installation tests. Many attendees mentioned openQA as the current state of the art technology for OS installation testing, and Philip Hands started looking into it. Right now, my little thing is still useful as it is, helping me reproduce regressions quickly, and testing bug fixes… so I haven’t been trying to port that to another tool yet.

I also gave another presentation in two different contexts: once at a local FLOSS meeting in Nantes, France and once during the mini-DebConf in Toulouse, France. Nothing related to Debian Installer this time, as the topic was how I helped a company upgrade thousands of machines from Debian 6 to Debian 8 (and to Debian 9 since then). It was nice to have Evolix people around, since we shared our respective experience around automation tools like Ansible and Puppet.

After the mini-DebConf in Toulouse, another event: the mini-DebConf in Cambridge, UK. I tried to give a lightning talk about “how snapshot.debian.org helped saved the release(s)” but clearly speed was lacking, and/or I had too many things to present, so that didn’t work out as well as I hoped. Fortunately, no time constraints when I presented that during a Debian meet-up in Nantes, France. :)

Since Reproducible Tails builds were announced, it seemed like a nice opportunity to document how my company got involved into early work on reproducibility for the Tails project.

On an administrative level, I’m already done with all the paperwork related to the second financial year. \o/

Next things I’ll likely write about: the first two D-I Buster Alpha releases (many blockers kept popping up, it was really hard to release), and a few more recent release critical bug reports.

Posted @ 15/01/2018 Tags: debamax debian

Executive summary

Since October 2015, I've been running a FLOSS consulting company, specialized on Debian, called DEBAMAX.

Longer version

Everything started two years ago. Back then I blogged about one of the biggest changes in my life: trying to find the right balance between volunteer work as a Debian Developer, and entrepreneurship as a Freelance Debian consultant. Big change because it meant giving up the comfort of the salaried world, and figuring out whether working this way would be sufficient to earn a living…

I experimented for a while under a simplified status. It comes with a number of limitations but that’s a huge win compared to France’s heavy company-related administrativia. Here’s what it looked like, everything being done online:

1 registration form to begin with: wait a few days, get an identifier from INSEE, mention it in your invoices, there you go!
4 tax forms a year: taxes can be declared monthly or quarterly, I went for the latter.

A number of things became quite clear after a few months:

I love this new job! Sharing my Debian knowledge with customers, and using it to help them build/improve/stabilise their products and their internal services feels great!
Even if I wasn't aware of that initially, it seems like I've got a decent network already: Debian Developers, former coworkers, and friends thought about me for their Debian-related tasks. It was nice to hear about their needs, say yes, sign paperwork, and start working right away!
While I'm trying really hard not to get too optimistic (achieving a given turnover on the first year doesn't mean you're guaranteed to do so again the following year), it seemed to go well enough for me to consider switching from this simplified status to a full-blown company.

Thankfully I was eligible to being accompanied by the local Chamber of Commerce and Industry (CCI Rennes), which provides teaching sessions for new entrepreneurs, coaching, and meeting opportunities (accountants, lawyers, insurance companies, …). Summer in France is traditionally rather quiet (read: almost everybody is on vacation), so DEBAMAX officially started operating in October 2015. Besides different administrative and accounting duties, running this company doesn't change the way I've been working since July 2014, so everything is fine!

As before, I won't be writing much about it through my personal blog, except for an occasional update every other year; if you want to follow what's happening with DEBAMAX:

Website: debamax.com — in addition to the usual company, services, and references sections, it features a blog (with RSS) where some missions are going to be detailed (when it makes sense to share and when customers are fine with it). Spoiler alert: Tails is likely to be the first success story there. ;)
Twitter: @debamax — which is going to be retweeted for a while from my personal account, @CyrilBrulebois.

Posted @ 22/08/2016 Tags: debamax debian

Time for a quick recap of the beginning of the Stretch release cycle as far as the Debian Installer is concerned:

It took nearly 3 months after the Jessie release, but linux finally managed to get into shape and fit for migration to testing, which unblocked the way for an debian-installer upload.
Trying to avoid last-minute fun, I’ve updated the britney freeze hints file to put into place a block-udeb on all packages.
Unfortunately, a recent change in systemd (implementation of Proposal v2: enable stateless persistant network interface names) found its way into testing a bit before that, so I’ve had my share of last-minute fun anyway! Indeed, that resulted in installer system and installed system having different views on interface naming. Thankfully I was approached by Michael Biebl right before my final tests (and debian-installer upload) so there was little head scratching involved. Commits were already in the master branch so a little plan was proposed in Fixing udev-udeb vs. net.ifnames for Stretch Alpha 1. This was implemented in two shots, given the extra round trip due to having dropped a binary package in the meanwhile and due to dak’s complaining about it.
After the usual round of build (see logs), and dak copy-installer to get installer files from unstable to testing, and urgent to get the source into testing as well (see request), I’ve asked Steve McIntyre to start building images through debian-cd. As expected, some troubles were run into, but they were swiftly fixed!
While Didier Raboud and Steve were performing some tests with the built images, I’ve prepared the announcement for dda@, and updated the usual pages in the debian-installer corner of the website: news entry, errata, and homepage.
Once the website was rebuilt to include these changes, I’ve sent the announce, and lifted all block-udeb.

(On a related note, I’ve started tweeting rather regularly about my actions, wins & fails, using the #DebianInstaller hashtag. I might try and aggregate my tweets as @CyrilBrulebois into more regular blog posts, time permitting.)

Executive summary: D-I Stretch Alpha 1 is released, time to stretch a bit!

(Credit: rferran on openclipart)

Posted @ 22/07/2015 Tags: d-i debian

I’m not used to talking about my day job but here’s an exception.

Over the past few years I worked in two startups (3 years each). It was nice to spend time in different areas: one job was mostly about research and development in a Linux cluster environment; the other one was about maintaining a highly-customized, Linux-based operating system, managing a small support team, and performing technological surveillance in IT security.

In the meanwhile I’ve reached a milestone: 10 years with Debian. I had been wondering for a few months whether I could try my luck going freelance, becoming a Debian consultant. I finally decided to go ahead and started in August!

The idea is to lend a hand for various Debian-related things like systems administration, development/debugging, packaging/repository maintenance, or Debian Installer support, be it one-shot or on a regular basis. I didn’t think about trainings/workshops at first but sharing knowledge is something I’ve always liked, even if I didn’t become a teacher.

For those interested, details can be found on my website: https://mraw.org/.

Update: It moved to https://debamax.com/ since then.

Of course this doesn’t mean I’m going to put an end to my volunteer activities within Debian, especially as a Debian Installer release manager. Quite the contrary in fact! See the August and September debian-boot@ archives, which have been busy months. :)

Posted @ 15/09/2014 Tags: debamax debian

Problem

Discussions are sometimes started by mailing a few different mailing lists so that all relevant parties have a chance to be aware of a new topic. It’s all nice when people can agree on a single venue to send their replies to, but that doesn’t happen every time.

Case in point, I’m getting 5 copies of a bunch of mails, through the following debian-* lists: accessibility, boot, cd, devel, project.

Needless to say: Reading, or marking a given mail as read once per maildir rapidly becomes a burden.

Solution

I know some people use a duplicate killer at procmail time (hello gregor) but I’d rather keep all mails in their relevant maildirs.

So here’s mark-read-everywhere.pl which seems to do the job just fine for my particular setup: all maildirs below ~/mails/* with the usual cur, new, tmp subdirectories.

Basically, given a mail piped from mutt, compute a hash on various headers, look at all new mails (new subdirectories), and mark the matching ones as read (move to the nearby cur subdirectories, and change suffix from , to ,S).

Mutt key binding (where X is short for cross post):

macro index X "<pipe-message>~/bin/mark-as-read-everywhere.pl<enter>"

This isn’t pretty or bulletproof but it already started saving time!

Now to wonder: was it worth the time to automate that?

Posted @ 11/08/2014 Tags: debian hacks

Browse by tag:

Browse by date:

Executive summary

Longer version

HTML demo

Executive summary

Longer version

Problem

Solution