Michael's Daemonic Doodles

...blogging bits of BSD

On FreeBSD Network Virtualization

Recently, I got the great opportunity to write an article on network virtualization for the FreeBSD Journal. The article was published in its November/December 2019 issue, which can be accessed free of charge on the FreeBSD Journal website.

Reading the FreeBSD Journal is a great way to ingest information; its magazine style layout is more geared towards reading when away from the keyboard though. To ease copy and pasting from the article, I also published it here on my blog. It's also available in PDF format.


Running FreeBSD on a Raspberry Pi3 using a custom image created with crochet and poudriere

I used to download images from raspbsd.org and install binary packages using pkg on the device. This has some disadvantages:

  • The image there is updated infrequently
  • Installing using pkg is relatively slow
  • Binary packages aren't always in sync when running CURRENT
  • Building things yourself gives you more control

Using crochet and poudriere it's quite easy to create an image with pre-installed packages, which helps a lot when provisioning a large number of devices. Packages are cross-compiled using qemu, which is slow, but still much faster than building on the device itself.


Shrinking a ZFS root pool (HDD to SSD migration)

I recently started migrating servers with relatively low storage space requirements to SSDs. In many cases the HDDs that get replaced are much bigger than required and unfortunately the zpools have been configured to use all the available space. Since shrinking a pool is not supported by ZFS directly, the procedure is a little more elaborate.


Trusted Package Distribution with pkgng

pkgng is FreeBSD's new approach to replace its aging pkg_* utility infrastructure. In this post I'll show how to set up a private package repository and - after applying a patch to libfetch - use SSL certificates to establish bidirectional trust. This is on top of pkgng's existing repository signing mechanism, which I will cover at the end of this post.


Making ZeroC Ice work with Clang, C++11 and libc++

ZeroC Ice (devel/ice) is an important part of the infrastructure of our internal services, therefore naturally it's the first major C++ port I'm trying to compile and run using Clang/C++11/libc++ on FreeBSD 9.1 RC1 as part of my project of migrating to FreeBSD 9.1, Clang, C++11 and libc++.

Earlier this year I made an effort to make Ice compile and run using Clang (and gcc47), but this was still using C++03 and gcc's libstdc++. I encountered a couple of obstacles while making this compile and run correctly using clang++ -std=c++11 -stdlib=libc++.


Testing FreeBSD 9.1 RC1 and migrating to Clang + libc++

Last year C++0x turned into C++11 and finally it's becoming a viable option for C++ programmers. At the same time, FreeBSD had been stuck with gcc 4.2.1 - the last version of gcc under the more permissive GPLv2 - for about half a decade, so the announcement that Clang will be in the base system was pretty amazing. Until recently the new libc++ (which replaces GNU libstdc++) was not part of the base system, so compiling C++11 code with Clang wasn't possible on a stable FreeBSD system. Finally 9.1 RC1 adds the bits necessary, so I decided to upgrade one of our 9.0 systems to 9.1 RC1 and build a Clang only system, using C++11 and Clangs libc++ to compile all C++ code in the required ports, so we'll end up with a complete C++11 tool chain and finally compile our own code base using Clang.

I already identified problematic ports, so this endeavor will definitely require some time, effort and patching. I'll keep posting about milestones and obstacles I encountered in the process.

Creating backup signature keys using GnuPG batch mode

I'm currently working on a new backup setup using duplicity (sysutils/duplicity). For this purpose I'm using separate keys for encrypting and signing the backup. Each host has its own signature key which enables the backup server to verify the integrity and authenticity of the backup. Since the backup is performed automatically, these keys should have no pass-phrase set. Using batch mode (and a little bit of python - duplicity is written in python anyway) creating signature keys is easy to do and doesn't require the installation of any of the pin-entry ports for GnuPG.


service(8) - a useful little gem

Most services in FreeBSD are - or at least should be - controlled through rc scripts. So if you want to reload your pf rules, you would usually use /etc/rc.d/pf reload instead of using pfctl(8) directly. For restarting the apache web server /usr/local/etc/rc.d/apache22 restart instead of using apachectl directly. To figure out which services are enabled, you would check /etc/rc.conf.

Fortunately there is a new command in FreeBSD called service(8), which helps you to manage services in a convenient and more sophisticated way.


Configure sendmail to forward root's email without DNS

For most of my professional life I tried to avoid sendmail whenever possible. Ever since qmail (and the myriad patches to it) got released I ran a couple of large scale setups for various ISPs successfully.

However, up to this day sendmail is FreeBSD's default MTA and I use it on all machines as the local mail delivery mechanism for system email (like the ones created by cron and periodic). So instead of trying to replace sendmail on all hosts I just tweak the configuration, so I can receive admin email in a centralized account without too much pain. For a couple of machines changing /etc/aliases might be enough, but there are some machines without DNS or public Internet access, for those the setup gets a little bit more complicated. This blog post obviously isn't for the sendmail savvy system administrator, but for people like me, who want to spend as little time as possible with sendmail and have it just in the background and do its job.


Setting up FreeBSD on ZFS raidz2 using mfsBSD

As I explained in an earlier post, we had issues with one of our backup servers. I decided to do a complete reinstall and ended up using an LSI SAS 9211 controller running in initiator-target mode (IT firmware), so all eight disks show up as plain HBAs and can be used to form a zpool. After considering my options I decided to go with raidz2, which will provide a good balance of storage space and safety.


Preparing an LSI SAS 2008 based controller for ZFS

In my last post I wrote about how the Adaptec 5805 RAID controller was the wrong choice for our backup server. So I started some research, based on the following criteria:

  • Production quality driver (so no more crashes, hangs and other weird behavior)
  • Pass-through support (/dev/passX), including S.M.A.R.T.
  • JBOD support (disks should show up as devices /dev/daX)
  • Decent throughput
  • Existing success stories with ZFS
  • Reasonable price and availability


Adaptec Madness (or why I started this blog)

One of the less maintained machines in our data center is a small backup server. It's basically sitting there, storing all those backups nobody really uses and slowly fills up its disk array. Once in a while somebody checks it manually to see if everything's in order (it has never been really connected to anything sitting on its own isolated VLAN).

The server uses an Adaptec 5805 controller, a product line I tried debugging and fixing a long time ago (see also http://lists.freebsd.org/pipermail/freebsd-bugs/2009-June/035612.html). Unfortunately back then our supplier stated that this is the only controller certified for this machine - so I gave in, hoping that a pretty much bored backup server won't have too much trouble keeping up with the little load it would get.

I was wrong.