[WireGuard] [Cake] WireGuard Queuing, Bufferbloat, Performance, Latency, and related issues

[Dave Taht]

On Sat, Oct 1, 2016 at 8:51 AM, Toke Høiland-Jørgensen <toke at toke.dk> wrote:
> Dave Taht <dave.taht at gmail.com> writes:
>
>> My thought - given that at least on some platforms - encrypting 1000
>> packets at a time is a bad idea - would be something regulating the
>> amount of data being crypted at a time, an equivalent to byte queue
>> limits - BQL - BCL? byte crypto limits - to keep no more than, say,
>> 1ms of data in that part of the subsystem.
>
> Well, the dynamic queue limit stuff is reusable (in
> include/linux/dynamic_queue_limits.h). The netdev BQL stuff just uses
> these functions with the packet byte sizes; so adapting it to use in
> wireguard should be fairly straight forward :)

Having one global queue for all of wireguard makes a lot of sense,
one that gets divvied up as per the amount of traffic for each destination,
and regulated "fairly".

The present model - of one fixed size one per endpoint can run you out
of memory right quick.

>> ... also pulling stuff out of order from an already encrypted thing
>> leads to the same IV problems we had in mac80211.
>
> Yeah, but who needs IVs, really? ;)


Well, in wireguard's case, it does not  (yay!) have
a-must-deliver-packet-in-IV-order mechanism like 802.11 requires. It
will merely throw away things that are outside the replay window (2k
packets). So you could *encrypt* first, deliver later, if you wanted.

Still, what I'd wanted to do was push back (or pace?) from the
interface itself, as well as from the crypto subsystem, and throw away
packets before being crypted when things are starting to get out of
hand - as well as do better fq-ish mixing of what's going out.

Consider a path like this:

testbox, flooding -> 1Gbit in -> router-encrypting -> 20Mbit out to
the internet.

If you flood the thing - you get a big queue in two places - one at
the interface qdisc where we struggle to throw already encrypted
things away (eventually), and another
inside the vpn code where we are encrypting as fast as we can get stuff in.


...

As a side note, another crazy daydream of mine is to use up an entire
/64 for the vpn endpoint identifier, and pre-negotiate a "spread
spectrum" style of sending packets there. This would eliminate needing
to bundle the IV in the packet, it would be obvious (to each side)
which IV matched what IP address, we'd save 8 bytes on every packet.

and completely break every fq+congestion control qdisc we know of, as
well as most stateful firewalls, and so on. :)

> -Toke



-- 
Dave Täht
Let's go make home routers and wifi faster! With better software!
http://blog.cerowrt.org