Very low throughput in *BSDs (but only as a router)

Lee Yates rainmakerraw at
Fri Jul 20 22:54:48 CEST 2018

Hash: SHA256

Hi all,

This is my first time posting to this list, but I've followed along for
a while now. I've been happily using wg at home for months, and it's
been a revelation in terms of speed (practically no performance hit at
all on my 350/20 ISP line).

I recently decided to stop running wg on all my (capable)LAN devices,
and to 'just' run wg on my home-made x86_64 router instead. Since
pfSense and IPFire don't have wg packages (or the ability to add them),
I decided to roll my own environment using Linux or one of the BSDs. I
did very well with a quick virtualised Arch install (masquerade for LAN
to the wg interface) and throughput was perfect - 350/20! Not being a
huge fan of systemd or iptables, I really wanted to use BSD so I tried
out an OpenBSD install. Despite reading how performant it was (capable
of >10Gbps out of the box on appropriate hardware), I noticed throughput
on the virtual router crashed to 130Mbps (30% of full speed) when wg was
connected. I confirmed that my virtual LAN clients were also limited to
around 130Mbps if wg was connected on the OpenBSD 'router'.

Not being satisfied with this and wondering what I'd done wrong (or
whether OpenBSD was indeed capable), I span up a much more familiar (to
me) FreeBSD 11.2 install and set it up the same way. Gateway=yes, pf set
to NAT the virtual LAN traffic through wg, and away we go. Again, the
virtual router could run 350/20 easily on its own, but as soon as wg was
connected (AzireVPN 10Gb node, btw) the performance dropped to the same

That just didn't seem right. I checked htop while connected to wg and
running iperf3 to a 10Gbps speedtest node in NL. Htop confirmed that the
wireguard process was only using a max of 7% CPU throughout the speed
test (the VMs have four cores from my i7 8700k at 5GHz each). So, it's
not a CPU bottleneck.

Weirdly, if I disconnect wg on the virtual router and run it from any of
the virtual LAN client machines instead, then throughput jumps back up
to 350/20 every single time. So, the virtual router seems capable of
routing 350/20 easily - provided the wg process is running on a client
machine and not itself. As soon as wg is connected on the router itself,
I'm down to 30% of my expected throughput no matter what.

To present it visually, in case it makes more sense for the visual
learners among us:

# Full speed
Virtual client OS [wg] > virtual router > real home router > WAN > [wg]
VPN server

# Crippled speed
Virtual client OS > virtual router [wg] > real router > WAN > [wg] VPN

I just can't make sense of it. I could literally run the iperf3 test on
the router+wg and get 130Mbps, but then fire up the exact same iperf3
test on any other machine on the network (connected via wg to the same
real external VPN server) and get full speed every single time.
Something seems to be hobbling wg when run on the router itself, but I'm
all out of ideas. I've tried tuning sysctl.conf etc on the virtual
routers (Open/Free BSD) but it made no difference at all.

Can anyone please offer any advice/help/tips or point out any glaring
omissions I may have made? I can upload my
rc.conf/sysctl.conf/pf.conf/dhcpd.conf/unbound.conf or other to pastebin
if anyone wishes to see them. Sorry if this would have been more
appropriate being sent to a BSD list, but unfortunately not many people
seem to be experienced with wg on BSDs yet so I'm finding help a little
thin on the ground.  Hence, posting to ask here where someone is more
likely to be experienced in the matter.

Many thanks in advance,

Lee Yates
Version: BCPG C# v1.8.1.0

-------------- next part --------------
A non-text attachment was scrubbed...
Name: rainmakerraw at
Type: application/pgp-keys
Size: 1677 bytes
Desc: not available
URL: <>

More information about the WireGuard mailing list