Windows Log Output to Event Viewer or Text File

[Frank Wayne]

Jason,

> Why not? It would make it possible to tail the log more easily and pipe it into whatever log collection daemon you want. And if that's indeed still not an improvement, what is the relevance of your previous mention, "It is only possible to get the last 2048 events and no way to get just the events since the last update"? I find the tone of your messages quite abrasive rather than informative. Can you slow down a bit and try to describe the constraints and requirements of your system, and then we can try to figure out what would be a good path toward realizing a good design there? Let's start with: what's missing in a tail mode that you can pipe to whatever, other than, "it's not ms event logging"?

The requirements are providing a simple, timely, and complete feed of WireGuard events to a log aggregator.

If I could get "the events since the last update", then a scheduled process could run to copy the events to some text log. That data could be aggregated. Even so, it assumes that the number of events that occur between iterations of the scheduled job does not exceed 2048. It makes things more complicated and the chance of losing events remains. As such, this does not fit the requirements well, as it is not simple (it requires the scheduling of a job, the choice of intervals based on the expected rate of events, and may be different for each host), nor timely (since events are delayed by the duration between iterations of the scheduled job), nor complete (since it is prone to event overflow). That's why I don't consider tailing the log *much* of an improvement. It would be an improvement to be sure, just not like what is available in Linux.

Since I have mainly worked on collecting logs from Linux WireGuard hosts where the process of getting the logs into a file is simple, timely, and complete with little effort, my reaction to the Windows version was that, relative to Linux, the logging is "awful" to deal with. I apologize for using that term blithely and that it came across as abrasive. That was not my intention. I did not mean to characterize any component of WireGuard as awful, and certainly the logging for the Windows version works well (as far as I can tell), it just doesn't fit the logging paradigms that Linux or Windows applications use that lend themselves to aggregation.

I have no particular love for Windows Event Log, but it is the de facto logging facility in Windows. If the logs went to a text file, it would be even better, frankly, but there's this thing called Windows Event Log that many (most?) applications use. If WireGuard used it, the requirements would be met. My questions might imply a preference toward Windows Event Log, but it is so owing only to that feature's ubiquity.

> I'm pretty sure systemd-journald epolls on /dev/kmsg. In other words, it aggregates logs from different sources. That's not a whole lot different from a follow mode, right? But why are you talking about Linux, or about kernel logs for that matter?

Not too different from follow mode, but journald (Linux's binary log file solution) *can* forward to rsyslog. Windows WireGuard's binary log file solution can't. I'm talking about Linux because its WireGuard logging meets the requirements. Linux WireGuard is easy to deal with because it uses standard Linux logging components. I'm talking about kernel logs because Linux WireGuard logs are kernel messages; i.e., they have a facility number of 0.

> On Linux, wireguard.ko simply uses printk, like other kernel drivers, and wg-quick(8) uses stdio, like many userspace programs. But why are you talking about Linux? What's the relevance?

Again, I'm talking about Linux because Linux WireGuard uses Linux logging facilities that are easy to work with and Windows WireGuard does not use Windows logging facilities that would be easy to work with.

> You've used the word "proprietary" but I think "bespoke" might be more clear. There are open source implementations in C, C#, and Go in the git repos, and it should be rather trivial to parse in any other language too.

I agree, but it is harder than it could be, and still prone to overflow.

> Event Logging appears to be rather slow and clunky, and I'm not sure I'd be too happy about blocking on that during packet events. It's also very cumbersome to use -- especially for things like crash dumps, which require a separate process or dll -- and the boilerplate involved doesn't look very appealing. In contrast, memory mapping a file, memcpy'ing buffers into it, and getting timestamps by reading 0x7ffe0014, means no calls to libraries or any interactions with other moving components that might be in an undefined state.

> Event Logging might yet be possible to use, though. But it seems to me that'd be some significant research and development work, to figuring out how it could work in a lightweight way, and also revisiting the way wgnt logs things.

I don't pretend to understand the performance or convenience implications of using Windows Event Log instead. If using it would impact WireGuard performance, avoiding it is understandable. Thanks for describing how WireGuard writes the logs, which is both elegant and efficient. I just wish there were an easier way to reliably get the events into a sequential log (file?) of indefinite length.

Thank you for taking the time to respond to my questions. My objective was first to know that I wasn't missing something, and then more about understanding why things are the way they are than about influencing change. I imagine that other people have or will have a need to monitor WireGuard on Windows and would benefit from a simpler, external interface to the log events. I hope you keep this in mind, but for now I will try to make do with what I have.

Frank Wayne


-----Original Message-----
From: Jason A. Donenfeld <Jason at zx2c4.com> 
Sent: Wednesday, 13 October, 2021 13:17
To: Frank Wayne <frank.wayne at northwestern.edu>
Cc: WireGuard mailing list <wireguard at lists.zx2c4.com>
Subject: Re: Windows Log Output to Event Viewer or Text File

Hi Frank,

On Wed, Oct 13, 2021 at 7:30 AM Frank Wayne <frank.wayne at northwestern.edu> wrote:
> >> On Tue, Oct 12, 2021 at 3:39 PM Frank Wayne <frank.wayne at northwestern.edu> wrote:
> >> That's pretty awful. It is only possible to get the last 2048 events and no way to get just the events since the last update. There is no way for an aggregator to simply collect WireGuard logs on Windows.
>
> > Your "that's pretty awful" aside, is what you're asking for some kind of CLI "follow" mode that doesn't terminate and spits out logs to stdout perpetually?
>
> > Jason
>
> No. I'm not sure that would be much of an improvement.

Why not? It would make it possible to tail the log more easily and pipe it into whatever log collection daemon you want. And if that's indeed still not an improvement, what is the relevance of your previous mention, "It is only possible to get the last 2048 events and no way to get just the events since the last update"? I find the tone of your messages quite abrasive rather than informative. Can you slow down a bit and try to describe the constraints and requirements of your system, and then we can try to figure out what would be a good path toward realizing a good design there? Let's start with: what's missing in a tail mode that you can pipe to whatever, other than, "it's not ms event logging"?

> In Linux (under systemd), kernel logs are accessible in journald, can be forwarded to (r)syslog, and from there to a text file or external syslog or wherever.

I'm pretty sure systemd-journald epolls on /dev/kmsg. In other words, it aggregates logs from different sources. That's not a whole lot different from a follow mode, right? But why are you talking about Linux, or about kernel logs for that matter?

> I can't imagine that a proprietary log format would fly in Linux, or even be contemplated.

On Linux, wireguard.ko simply uses printk, like other kernel drivers, and wg-quick(8) uses stdio, like many userspace programs. But why are you talking about Linux? What's the relevance?

You've used the word "proprietary" but I think "bespoke" might be more clear. There are open source implementations in C, C#, and Go in the git repos, and it should be rather trivial to parse in any other language too.

> In Windows, logs would ideally get sent to Event Logging into a WireGuard log. That way, the user or administrator can use Event Viewer to view the log, forward the log,  or use a collector (like Splunk) to retrieve and aggregate the events.
> I'm not sure why WireGuard doesn't use Windows Event Logging. Is there something that precludes the use of Event Logging by WireGuard?

Event Logging appears to be rather slow and clunky, and I'm not sure I'd be too happy about blocking on that during packet events. It's also very cumbersome to use -- especially for things like crash dumps, which require a separate process or dll -- and the boilerplate involved doesn't look very appealing. In contrast, memory mapping a file, memcpy'ing buffers into it, and getting timestamps by reading 0x7ffe0014, means no calls to libraries or any interactions with other moving components that might be in an undefined state.

Event Logging might yet be possible to use, though. But it seems to me that'd be some significant research and development work, to figuring out how it could work in a lightweight way, and also revisiting the way wgnt logs things.

If this is something you'd like to work on, I'd be happy to review patches and read descriptions of a simplified event logging implementation. You're probably not the only user with this concern, and in theory I'd be open to considering it, provided there's a way to make it less clunky than initially meets the eye.

Jason