passing-through TOS/DSCP marking

[Toke Høiland-Jørgensen]

Toke Høiland-Jørgensen <toke at toke.dk> writes:

> Daniel Golle <daniel at makrotopia.org> writes:
>
>> Hi Florent,
>>
>> On Thu, Jun 17, 2021 at 07:55:09AM +0000, Florent Daigniere wrote:
>>> On Thu, 2021-06-17 at 01:33 +0200, Toke Høiland-Jørgensen wrote:
>>> > Daniel Golle <daniel at makrotopia.org> writes:
>>> > 
>>> > > Hi Jason,
>>> > > 
>>> > > On Wed, Jun 16, 2021 at 06:28:12PM +0200, Jason A. Donenfeld wrote:
>>> > > > WireGuard does not copy the inner DSCP mark to the outside, aside
>>> > > > from
>>> > > > the ECN bits, in order to avoid a data leak.
>>> > > 
>>> > > That's a very valid argument.
>>> > > 
>>> > > However, from my experience now, Wireguard is not suitable for
>>> > > VoIP/RTP
>>> > > data (minimize-delay) being sent through the same tunnel as TCP bulk
>>> > > (maximize-throughput) traffic in bandwidth constraint and/or high-
>>> > > latency
>>> > > environments, as that ruins the VoIP calls to the degree of not
>>> > > being
>>> > > understandable. ECN helps quite a bit when it comes to avoid packet
>>> > > drops
>>> > > for TCP traffic, but that's not enough to avoid high jitter and
>>> > > drops for
>>> > > RTP/UDP traffic at the same time.
>>> > > 
>>> > > I thought about ways to improve that and wonder what you would
>>> > > suggest.
>>> > > My ideas are:
>>> > >  * have different tunnels depending on inner DSCP bits and mark them
>>> > >    accordingly on the outside.
>>> > >    => we already got multiple tunnels and that would double the
>>> > > number.
>>> > > 
>>> > >  * mark outer packets with DSCP bits based on their size.
>>> > >    VoIP RTP/UDP packets are typically "medium sized" while TCP
>>> > > packets
>>> > >    typically max out the MTU.
>>> > >    => we would not leak information, but that assumption may not
>>> > > always
>>> > >       be true
>>> > > 
>>> > >  * patch wireguard kernel code to allow preserving inner DSCP bits.
>>> > >    => even only having 2 differentl classes of traffic (critical vs.
>>> > >       bulk) would already help a lot...
>>> > > 
>>> > > 
>>> > > What do you think? Any other ideas?
>>> > 
>>> > Can you share a few more details about the network setup? I.e., where
>>> > is
>>> > the bottleneck link that requires this special treatment?
>>> 
>>> I can tell you about mine. WiFi in a congested environment: "voip on
>>> mobile phones". WMM/802.11e uses the diffserv markings; most commercial
>>> APs will do the right thing provided packets are marked appropriately.
>>> 
>>> At the time I have sent patches (back in 2019) for both the golang and
>>> linux implementation that turned it on by default. I believe that
>>> Russell Strong further improved upon them by adding a knob (20190318 on
>>> this mailing list).
>>
>> Thank you very much for the hint!
>> This patch is exactly what I was looking for:
>> https://lists.zx2c4.com/pipermail/wireguard/2019-March/004026.html
>>
>> Unfortunately it has not received a great amount of feedback back then.
>> I'll try forward-porting and deploying it now, because to me it looks
>> like the best solution money can buy :)
>
> I think you can achieve something similar using BPF filters, by relying
> on wireguard passing through the skb->hash value when encrypting.
>
> Simply attach a TC-BPF filter to the wireguard netdev, pull out the DSCP
> value and store it in a map keyed on skb->hash. Then, run a second BPF
> filter on the physical interface that shares that same map, lookup the
> DSCP value based on the skb->hash value, and rewrite the outer IP
> header.
>
> The read-side filter will need to use bpf_get_hash_recalc() to make sure
> the hash is calculated before the packet gets handed to wireguard, and
> it'll be subject to hash collisions, but I think it should generally
> work fairly well (for anything that's flow-based of course). And it can
> be done without patching wireguard itself :)

Just for fun I implemented such a pair of eBPF filters, and tested that
it does indeed work for preserving DSCP marks on a Wireguard tunnel. The
PoC is here:

https://github.com/xdp-project/bpf-examples/tree/master/preserve-dscp

To try it out (you'll need a recent-ish kernel and clang version) run:

git clone --recurse-submodules https://github.com/xdp-project/bpf-examples
cd bpf-examples/preserve-dscp
make
./preserve-dscp wg0 eth0

(assuming wg0 and eth0 are the wireguard and physical interfaces in
question, respectively).

To actually deploy this it would probably need a few tweaks; in
particular the second filter that rewrites packets should probably check
that the packets are actually part of the Wireguard tunnel in question
(by parsing the UDP header and checking the source port) before writing
anything to the packet.

-Toke