I’m enjoying using FreeBSD 13.1 on my old ThinkPad X230.
One thing I haven’t tried yet is using WireGuard for VPN connections on FreeBSD.
There are two WireGuard ports:
wireguard-kmod, the experimental in-kernel version, and
wireguard-go, the stable, cross-platform implementation.
I tried both to see which I should use.
Setting Up Experimental In-Kernel Wireguard
Let’s start with the fun version. This port is the in-kernel version from zx2c4, written by Jason Donenfeld and Kyle Evans—not to be confused with the infamous Netgate-sponsored port that was yanked from FreeBSD 13. It’s still under development and comes with disclaimers about using it in production.
To use it, install the package and the userland tools to control it.
# pkg install wireguard-kmod wireguard-tools
The package notes (use
pkg info -D wireguard-kmod to see them again) show a warning that the code is unvetted and might contain security issues.
They don’t include information about how to use the kernel module.
I assumed I would have to load it by name with
To find the name, I listed all the files installed by the package.
$ pkg info -l wireguard-kmod wireguard-kmod-0.0.20211105_1: /boot/modules/if_wg.ko /usr/local/share/licenses/wireguard-kmod-0.0.20211105_1/LICENSE /usr/local/share/licenses/wireguard-kmod-0.0.20211105_1/MIT /usr/local/share/licenses/wireguard-kmod-0.0.20211105_1/catalog.mk
However, I got an error when I loaded it by name.
# kldload if_wg kldload: can't load if_wg: module already loaded or in kernel
Checking whether the module is loaded gave me exit code 0 (success). I guess it’s loaded by default!
# kldstat -q -n if_wg; echo $? 0
Configuring, starting, and stopping tunnels works the same as on other platforms with
I defined the
wg0 tunnel by creating
# # wg0.conf - wireguard configuration for a client machine (e.g. a laptop) # [Interface] Address = # 10.2.0.<Your Address Here>/24 PrivateKey = # <Your Private Key Here> DNS = 126.96.36.199 # Gateway server [Peer] Endpoint = url.of.gateway:51820 PublicKey = # <Gateway Server's Public Key Here> AllowedIPs = 10.2.0.0/24,192.168.1.0/24 PersistentKeepAlive = 25
Starting the tunnel revealed another difference between FreeBSD and Linux.
# wg-quick up wg0 [#] ifconfig wg create name wg0 [#] wg setconf wg0 /dev/stdin [#] ifconfig wg0 inet 10.2.0.10/24 alias [#] ifconfig wg0 mtu 1420 [#] ifconfig wg0 up [#] resolvconf -a wg0 -x [#] route -q -n add -inet 192.168.1.0/24 -interface wg0 [#] resolvconf -d wg0 [#] ifconfig wg0 destroy
ifconfig destroyed my wg0 interface immediately after creating it!
Apparently, FreeBSD won’t let me add a route that conflicts with one on another active interface.
Linux lets me override the existing route.
I already had this route because I was setting WireGuard up while connected to my home network. Adding a route through the tunnel for 192.168.1.0/24 requires that I not already be on a network with that subnet.
This configuration would work if I were on a different network—provided that it didn’t also use that subnet—so, for now, I removed the home subnet from AllowedIPs.
AllowedIPs = 10.2.0.0/24
That change fixed the issue.
# wg-quick up wg0 [#] ifconfig wg create name wg0 [#] wg setconf wg0 /dev/stdin [#] ifconfig wg0 inet 10.2.0.10/24 alias [#] ifconfig wg0 mtu 1420 [#] ifconfig wg0 up [#] resolvconf -a wg0 -x [+] Backgrounding route monitor # wg show interface: wg0 public key: <My client's public key> private key: (hidden) listening port: 40839 peer: <My gateway server's public key> endpoint: <IP Address>:51820 allowed ips: 10.2.0.0/24 latest handshake: 30 seconds ago transfer: 92 B received, 212 B sent persistent keepalive: every 25 seconds
The last step is to enable the tunnel on startup.
I suspected that wireguard-tools came with an
rc script to do so.
$ pkg info -l wireguard-tools wireguard-tools-1.0.20210914_1: /usr/local/bin/wg /usr/local/bin/wg-quick /usr/local/etc/rc.d/wireguard [...]
Yep, there it is. The comment at the top of
/usr/local/etc/rc.c/wireguard explains how to configure it in
With those lines, the tunnel can be started using
service wireguard start.
Setting Up Stable Userland WireGuard
All we have to do to try out
wireguard-go is to swap out the package that’s installed.
Configuring, starting, and stopping tunnels is handled by
wireguard-tools and works exactly the same way with both versions.
# pkg remove wireguard-kmod [...] # pkg install wireguard-go
service wireguard start shows this:
# service wireguard start [#] ifconfig wg create name wg0 [!] Missing WireGuard kernel support (ifconfig: SIOCIFCREATE2: Invalid argument). Falling back to slow userspace implementation. [#] wireguard-go wg0 ┌──────────────────────────────────────────────────────┐ │ │ │ Running wireguard-go is not required because this │ │ kernel has first class support for WireGuard. For │ │ information on installing the kernel module, │ │ please visit: │ │ https://www.wireguard.com/install/ │ │ │ └──────────────────────────────────────────────────────┘ [...]
This is a bit of a mixed message.
wireguard-kmod, I got a scary warning about it being experimental, but
wireguard-go says it’s OBE.
I want to use the in-kernel version for better performance, but I also want security and stability.
Let’s see if the slow userspace implementation is really all that slow.
FreeBSD Wireguard Shootout
As an unscientific test, I set up a tunnel between my laptop with FreeBSD 13.1 and my Raspberry Pi with both connected to the same switch over gigabit ethernet.
iperf3 to measure the throughput and
vmstat to see CPU and memory usage.
Here are the network throughput results.
Baseline (No Tunnel)
[ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 1.09 GBytes 935 Mbits/sec 0 sender [ 5] 0.00-10.04 sec 1.09 GBytes 932 Mbits/sec receiver
[ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 661 MBytes 555 Mbits/sec 423 sender [ 5] 0.00-10.02 sec 661 MBytes 553 Mbits/sec receiver
[ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 787 MBytes 660 Mbits/sec 138 sender [ 5] 0.00-10.03 sec 786 MBytes 658 Mbits/sec receiver
Compare the CPU usage stats from
Each line is a half-second interval from the middle of the iperf test.
Baseline (No tunnel)
cpu sy cs us sy id 97596 40992 1 12 87 96434 40968 3 11 87 98020 40760 4 12 84
cpu sy cs us sy id 210358 85010 17 16 67 221720 89424 21 15 63 215976 87518 12 20 68
cpu sy cs us sy id 14908 41894 1 27 72 13274 35634 2 27 71 14110 37270 1 29 70
The number of system calls (the first
sy column) is why we want WireGuard in the kernel instead of in userland; the userland version made more system calls to transfer less data.
The user and system CPU time numbers (
us and the second
sy) reflect that difference.
That said, I’d love to know why there were fewer system calls with in-kernel wireguard than with no tunnel at all.
The idle number (
id) shows a small difference in unused CPU capacity. The in-kernel version eats up less CPU time overall.
Just for fun, I booted a live USB stick of Fedora Linux, which has WireGuard in the kernel, and tried the same test.
Baseline (No Tunnel) on Linux
[ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 1.09 GBytes 936 Mbits/sec 0 sender [ 5] 0.00-10.00 sec 1.09 GBytes 934 Mbits/sec receiver
wireguard on Linux
[ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 717 MBytes 601 Mbits/sec 78 sender [ 5] 0.00-10.00 sec 715 MBytes 600 Mbits/sec receiver
The goal of using WireGuard is to have a secure VPN connection with very little overhead.
In this test,
wireguard-go on FreeBSD doesn’t really deliver that, even for casual use.
If I were happy cutting network performance almost in half, I might as well use a different protocol.
wireguard-kmod performs much better.
It even performs as well or better than the real competition, WireGuard on Linux.
Of course, this port isn’t done yet, but considering that it’s been over a year since the in-kernel WireGuard kerfuffle, I’ll wager that it’s secure enough to use.