Michael's Daemonic Doodles

Updating FreeBSD to 9.1-RC1

This has been relatively easy. First I did a source upgrade of a 9.0 RELEASE machine, doing the old-school make buildworld, make kernel, mergemaster -o, make installworld, mergemaster -i, make delete-old, reboot cycle. I then modified /etc/make.conf, so it contains the following (relatively conservative) entries:

.if !defined(CC) || ${CC} == "cc"
CC=clang
.endif

.if !defined(CXX) || ${CXX} == "c++"
CXX=clang++
CXXFLAGS+= -std=c++11
.endif

.if !defined(CPP) || ${CPP} == "cpp"
CPP=clang-cpp
.endif

NO_FSCHG=
WITH_CLANG_EXTRAS=1
WITH_LIBCPLUSPLUS=YES

(see also http://wiki.freebsd.org/BuildingFreeBSDWithClang for details on this and possible improvements to those settings).

Using these I recompiled and reinstalled world and kernel (make buildworld, make kernel, make installworld, reboot), which now also gave me libc++. So the result was a system, that had everything compiled using Clang and C++11, but still links to libstdc++ for some C++ tools (including Clang). Fixing this last step (by adding -stdlib=libc++) turned out to be quite some effort, and since it only affects these tools, but no the compiled result I decided to give up on it for the moment.

The last step was to create compilation jail using ezjail. It's been initialized using ezjail-admin update -ip.

The /etc/make.conf used within the compilation jail uses the following settings (the first four lines are specific to ezjail):

WRKDIRPREFIX=           /var/ports
DISTDIR=                /var/ports/distfiles
PACKAGES=               /var/ports/packages
INDEXDIR=               /var/ports

CC=clang
CXX=clang++
CXXFLAGS+= -std=c++11 -stdlib=libc++
CPP=clang-cpp

NO_FSCHG=

Updating software packages

Almost all C based ports we're using worked directly out of ports. Many of the C++ based ports required some tuning. I fixed everything I could, sent patches upstream and opened PRs in FreeBSD ports. You can find a list of links to these patches at the end of this post.

Common problems

Most of the problems I encountered during the process were compile time and connected to certain implicit type conversions being illegal in C++11. Other compile time issues were missing spaces and the use of insecure constructs (those were mostly warnings though). Some boost magic broke compile and runtime (some of the things that broke won't be needed in C++11 anymore though). Thanks to Clang's nice warnings I was even able to spot a bug in the XML library we're using. Also the implicit deletion of copy constructors in case of const data members caused some minor headaches.

The more subtle problems were runtime and caused by the (in my opinion correct) implementation of iostream in libc++. A general problem are designs that require throwing destructors (since C++11 defines those as noexcept by default), which was quite hard to debug - see also Making ZeroC Ice work with Clang, C++11 and libc++ for a more detailed discussion.

Finally I faced some nasty overloading problems involving a specialization of vector<bool>.

In general all those problems were manageable, yet anything runtime can turn out nasty, so I would suggest to actively search for those.

Problems in our own codebase

To my delight our own codebase had very little compile time problems, most of the changes done were cosmetic to silence compiler warnings caused by third party libraries (e.g. cryptopp). I also removed unused variables, added parenthesis here and there, but overall this was a pretty smooth process.

Adapting the build system

Our build system had to be adapted to allow the use of gcc as well as Clang and all the necessary flags. We're using boost build 2.0 m12, which has no support for Clang, so I had to backport clang*.jam from the current boost libs distribution.

Modifying continuous integration system

We're using buildbot for continuous integration. It's relatively easy do add additional buildslaves to it (though adding SSL is kind of hacky and I might write a blog post about this at some point). Now the Clang C++11 buildslave runs in parallel to the existing ones, so all development will be tested on both toolchains.

Giving developers access to the compile host

All developer get individual compile jails on the host, so they can start using it and play with the new features. It's almost transparent to them, since an almost identical compile host is used for the old toolchain already.

So far so good...

Even though this was a major effort and we'll have to do a lot of additional testing before we can start rolling out services in production, it wasn't as hard as I suspected it to be. The dev team can't wait to actually make use of all the new and exciting features in C++11.