Connection hangs over CGNAT (Starlink)

[Lonnie Abelbeck]

Have you tried reducing the MTU of the WG tunnel?

I have a similar use case with a WG tunnel over a T-Mobile Home Internet (TMHI) CGNAT network.

After some testing determining the reduced MTU of the TMHI network, I set the WG endpoints' MTU to be 1340.

The WG tunnel has been rock solid.

Lonnie


> On Dec 15, 2022, at 8:12 PM, Nikolay Martynov <mar.kolya at gmail.com> wrote:
> 
> Hi!
> 
> I'm experiencing strange behaviour with wireguard: from time to time
> connection 'freezes'.
> Most often I'm observing this on an Android phone when connected from
> my home over Starlink.
> Server: latest Openwrt, Client: latest Android app.
> The connection establishes and works fine for some time. After some
> time the client still shows connection is established, but no incoming
> data is coming.
> On a server side 'latest handshake' goes into hours/days.
> The freeze happens randomly, for no apparent reason and I think only
> over starlink. I do not think I have ever observed this problem on
> cell networks.
> 
> Reconnection solves the problem immediately.
> I did some tcpdumping when the problem was present and found the following:
> * Server side sees incoming traffic from the client and sends responses.
> * On my own router connected to Starlink (i.e. interface between my
> router and Starlink router) I see data going from the client to the
> server - but no packets coming back.
> 
> So my 'hypothesis' is that somehow Starlink's CGNAT 'forgets' one side
> of the connection - and so data continues to go in one direction, but
> it doesn't come back. The thing with the wireguard is that it looks
> like it doesn't change the outgoing port when it attempts to do
> another handshake. This means that it continues using the same 'half
> broken' connection forever.
> 
> I think the same happens to me at least once on a Linux client - but
> the difference with the phone is that the phone is always on and
> therefore the duration of the connection is much longer.
> 
> I tried experimenting with keepalive messages - but it looks like they
> make no difference. Once connection freezes I see keepalived arriving
> onto the server, server sending reply - but that reply never arrives
> to the client.
> 
> It looks like the solution to this problem would be for the client to
> use a different outgoing port when sending a handshake but I was not
> able to find an option for that.
> 
> Is this something that is possible to do?
> Thanks!
> 
> 
> -- 
> Martynov Nikolay.
> Email: mar.kolya at gmail.com
>