The Four Words of Distress

The title of this post is taken directly from The Codeless Code, "The Four Words of Distress"

I read it again yesterday, and today I have been bitten by it.

I accidentally removed the content of a comment on this blog. I don't get many comments, so I feel that all of them are important contributions (except the incessant 50 Shades of Grey spam, urgh, do I regret writing about that.) As such, removing a comment is "serious" to me.

Upon realising my error, I immediately searched the page for an undo or a "restore" link of some sort, and eventually went to search for an answer to my problems. I only found unanswered questions on Google's Groups, and a Product Help page claiming that I should've been sent to a confirmation page (no such thing happened)

To have fixed this problem, there would have been any number of ways to deal with it:

1. A confirmation dialogue, alongside a "don't show this again" check box.
2. A trash can, which comments can go to for a fixed period of time.
3. An undo button
4. A help guide.

As it stands, I accidentally clicked the "Remove Content" link, and now, without warning, the comment has gone. I worry that this is a black mark against the commenter's account, when it is a simple mistake.

Chrooting a Mumble Server on OpenBSD

One of my colleagues is starting remote working shortly. As such, we needed a VoIP solution that worked for everyone, Mac, Linux and FreeBSD. It was discovered that Mumble provided ample quality and worked everywhere. Top it off with the fact that we could host it ourselves, and we looked to be set.

However, being security conscious, I like to sandbox any service I have. Further, since this solution is slightly ad-hoc at the moment, it's being run off a personal BigV server, running OpenBSD. So I set out to chroot the package-managed mumble server, umurmur, which does not include sandboxing as default.

Fortunately, if no logfile is specified, umurmurd will log to syslog, and it does not support config reloading, so I don't need to worry about that.

Chrooting is not entirely simple, and simple versions can be improved by "refreshing" the chroot every time the service starts. This means that if an attacker infects the binary in some way, it'll get cleared out and replaced after a restart. As another bonus, if a shared library is updated, it simply won't get found, which tells you what to update! If the binary gets updated, it'll be copied in fresh when the service is restarted.

To do this, we modify /etc/rc.d/umurmur:

#!/bin/sh
#
# $OpenBSD: umurmurd.rc,v 1.2 2011/07/08 09:09:43 dcoppa Exp $
#
# 2015-06-06, Turner:
#
# Jails the umurmurd deamon on boot, copies the daemon binary and
# libraries in each time.
#
# An adversary can still tamper with the logfiles, but everything
# else is transient.
#

chroot="/var/jails/umurmurd"
group="_umurmur"
original_daemon="/usr/local/bin/umurmurd"
daemon="chroot $chroot $original_daemon"

build_chroot() {
  # Locations of binaries and libraries.
  mkdir -p "$chroot/usr/local/bin"
  mkdir -p "$chroot/usr/lib"
  mkdir -p "$chroot/usr/local/lib"
  mkdir -p "$chroot/usr/libexec"

  # Copy in the binary.
  cp "$original_daemon" "$chroot/usr/local/bin/"

  # Copy in shared libraries
  cp "/usr/lib/libssl.so.27.0" "$chroot/usr/lib/"
  cp "/usr/lib/libcrypto.so.30.0" "$chroot/usr/lib/"
  cp "/usr/local/lib/libconfig.so.9.2" "$chroot/usr/local/lib/"
  cp "/usr/local/lib/libprotobuf-c.so.0.0" "$chroot/usr/local/lib/"
  cp "/usr/lib/libc.so.77.0" "$chroot/usr/lib/"
  cp "/usr/libexec/ld.so" "$chroot/usr/libexec/ld.so"

  # Setup /etc and copy in config.
  mkdir -p "$chroot/etc/umurmur"
  cp "/etc/umurmur/umurmur.conf" "$chroot/etc/umurmur/umurmur.conf"
  cp "/etc/umurmur/certificate.crt" "$chroot/etc/umurmur/certificate.conf"
  cp "/etc/umurmur/private_key.key" "$chroot/etc/umurmur/private_key.key"

  # Setup the linker hints.
  mkdir -p "$chroot/var/run/"
  cp "/var/run/ld.so.hints" "$chroot/var/run/ld.so.hints"
  cp "/usr/libexec/ld.so" "$chroot/usr/libexec/ld.so"

  # Copy the pwd.db password database in. This is less-than-ideal.
  cp "/etc/pwd.db" "$chroot/etc/"
  grep "$group" "/etc/group" > "$chroot/etc/group"

  # Setup /dev
  mkdir "$chroot/dev"
  mknod -m 644 "$chroot/dev/urandom" c 1 9
  mknod -m 644 "$chroot/dev/null" c 1 3
}

destroy_chroot() {
  if [ "$chroot" ]
  then
    rm -rf "$chroot"
  fi
}

case "$1" in
  start)
    build_chroot
    ;;
  stop)
    destroy_chroot
    ;;
esac

# Standard rc.d "stuff" here.
. /etc/rc.d/rc.subr

rc_reload=NO

rc_cmd $1

So there we go! When /etc/rc.d/umurmurd start is called, the chroot is setup, and umurmurd started in there. When you kill it, the chroot jail is emptied.

There are some limitations. For one, any private key (in the default, it's private_key.key) can be compromised by an attacker who can compromise umurmurd, and this can be used to impersonate the server long after the compromise. Secondly, if you do specify a log file in umurmur.conf, and you setup the relevant directory for logging to, it will be trashed when you stop the daemon. This is a real problem if you're trying to workout what happened during a compromise.

Finally, if umurmur is updated, and the required libraries do change, "ldd /usr/local/bin/umurmurd" will list the new shared objects.

Known Issues

This does not currently stop the umurmur daemon on stop. I'm not entirely sure why, but the work around is to stop the service using /etc/rc.d/umurmurd stop, then find it using ps A | grep umurmur and kill -15 it.