Always On VPN Troubleshooting with Windows Packet Monitor PktMon.exe

When troubleshooting Always On VPN, taking a network packet capture or network trace is sometimes required to identify the root cause of a problem. After all, Packets Don’t Lie™. There are numerous ways to capture packets. Many administrators will install Wireshark for this purpose. However, Windows has a native packet capture tool called PktMon.exe that offers many advantages over Wireshark.

Wireshark

Many Always On VPN administrators will be familiar with Wireshark. Wireshark is a popular open-source network protocol analyzer that enables the capture and analysis of network traffic for troubleshooting. A packet capture driver must first be installed to capture network traffic with Wireshark. Typically, administrators will install Npcap, which is part of the default installation of Wireshark. Installing a capture driver poses a potential problem, as the administrator must install software on the target device before capturing traffic. Installing software may not always be feasible or possible. Fortunately, there’s an alternative.

PktMon.exe

The Windows Packet Monitor (PktMon.exe) is a built-in command-line tool first introduced in Windows 10 1809 and Windows Server 2019. It is designed to capture network traffic on Windows servers and client systems. This native lightweight tool is ideal for collecting network traces for offline analysis.

Capture All Interfaces

The most common scenario for PktMon.exe is to capture data for offline analysis. Use the following command to capture all network traffic on all active network interfaces.

PktMon.exe start –capture –file c:\capture.etl –pkt-size 0 –comp nics –flags 0x10

The command breaks down as follows:

–capture – captures network traffic

–file – the path of the file to save the data to

–pkt-size 0 – captures the full packet (not truncated)

–comp nics – captures traffic on all active network interfaces

–flags 0x10 – captures the raw packet

After reproducing the issue, you can stop the trace by running the following command.

PktMon.exe stop

Capture Specific Interface

Administrators may wish to capture traffic on a specific network interface instead of all active network interfaces. In this example, I have a multi-homed VPN server and want to capture traffic on only the DMZ interface. To do this, use PktMon.exe to enumerate all interfaces using the following command.

PktMon.exe list

Note: The output of PktMon.exe filter list does not include information that easily maps to existing network interfaces. I suggest also running the Get-NetAdapter PowerShell command to view detailed information about network interfaces. You can use this information to select the correct Network ID for PktMon.exe filtering.

Next, change the value of –comp nics in the command referenced above to –comp <Network ID>. Here’s an example.

PktMon.exe start –capture –file c:\capture.etl –pkt-size 0 –comp 62 –flags 0x10

Filtering

It’s also possible to use PktMon.exe to capture network traffic selectively. Filtering allows you to narrow the capture to relevant traffic, making analysis easier and faster. Add a filter, then start a trace to restrict data capture to traffic that matches the defined filters. You can add one or more filters to apply to the capture. Here are a few examples.

Protocols and Ports

Let’s say you are troubleshooting a device tunnel connection and want to see only IKEv2 traffic. The following filter will restrict the network capture to only the IKEv2-related protocols and ports.

PktMon.exe filter add IKEv2 -t UDP -p 500
PktMon.exe filter add IKEv2 -t UDP -p 4500

IP Address

The following filter will capture data that includes the specified IP address in the source or destination address field.

PktMon.exe filter add VPN1 -i 172.21.12.50

You can also specify IP address subnets using their CIDR notation.

PktMon.exe filter add Subnet1 -i 172.16.0.0/16

View and Clear Filters

You can view configured filters using the following command.

PktMon.exe filter list

You can remove configured filters using the following command. Use with caution, as this removes ALL filters!

PktMon.exe filter remove

Reference

You’ll find a complete list of PktMon.exe filters here.

Analysis

PktMon.exe outputs captured data in ETL format. Administrators can convert captured data to the standard PCAP format by running the following command.

PktMon.exe etl2pcap <path of trace file>

This command converts the file from ETL to PCAPNG format. Administrators can then open the capture in Wireshark for further detailed analysis.

Display Only

PktMon.exe can be configured to display network traffic in the console for quick troubleshooting. Console traffic display can be helpful for those scenarios where a quick check to validate traffic is reaching a particular destination is required. Here’s an example.

PktMon.exe start –capture –pkt-size 0 –comp nics –flags 0x10 -m real-time

Note: In the example above, I applied a traffic filter to limit the capture to only SSTP traffic (TCP 443).

Limitations

One crucial limitation of PktMon.exe is that it doesn’t support persistent network captures that survive a reboot. Persistent captures can be helpful when troubleshooting a device tunnel connection or slow logons. In this scenario, you must use netsh.exe.

netsh.exe trace start capture=yes tracefile=c:\tracefile.etl persistent=yes

<reboot>

netsh.exe trace stop

Although PktMon.exe supports the ‘etl2pcap’ switch, it does NOT work for converting .etl files generated with netsh.exe. To convert captures created with netsh.exe, use the open-source etl2pcapng tool.

Learn More

PktMon.exe has many different uses. This post barely scratches the surface of what PktMon.exe can do. PktMon.exe comes with robust help, accessible by adding the ‘help’ switch to commands. Here are some examples.

PktMon.exe start help
PktMon.exe filter add help

Be sure to view the online help to explore various options for capturing and logging to meet your specific needs.

Summary

PktMon.exe is a native command-line utility in Windows that provides a lightweight solution for capturing network traffic, making it particularly useful for Always On VPN troubleshooting. Key functionalities include full-packet captures, selective filtering by protocol, port, or IP address, and conversion of ETL files to PCAPNG format for analysis in tools like Wireshark. Real-time traffic displays are also supported for quick diagnostics. While effective for many scenarios, PktMon.exe lacks support for persistent captures across reboots, for which netsh.exe is recommended. The techniques outlined above offer administrators a practical, software-free approach to deep packet inspection for troubleshooting Always On VPN issues.

Have you used PktMon.exe for network troubleshooting? Feel free to share tips and tricks in the comments section below!

Additional Information

Getting Started with Windows Packet Monitor (PktMon.exe)

PktMon.exe Filter Reference

Open-source Etl2pcap for netsh.exe captures

Always On VPN and IPv6

Internet Protocol version 6 (IPv6) has been with us for nearly 30 years. IPv6 adoption on the public Internet has steadily increased over the last decade, and today is approaching 50%. However, enterprise adoption of IPv6 has been surprisingly sluggish despite its numerous benefits. IPv6 includes an expanded address space that removes complex subnetting requirements and globally unique addressing that eliminates the need to perform Network Address Translation (NAT), among others. Organizations should consider deploying IPv6 internally to take advantage of these capabilities.

IPv6 and RRAS

I’ve deployed Microsoft Always On VPN for customers using IPv6 numerous times. The following describes configuration settings required to support IPv6 in a Microsoft environment using a Windows Server Routing and Remote Access (RRAS) server.

To begin, open the Routing and Remote Access management console (rrasmgmt.msc) on the RRAS VPN server, then follow the steps below to enable IPv6 support for Always On VPN connections.

Note: The configuration below assumes that IPv6 is already deployed on the internal network, either natively or dual-stacked with IPv4.

IPv6 Remote Access

Perform the following steps to enable IPv6 remote access on the RRAS VPN server.

  1. Right-click the RRAS VPN server in the navigation tree and choose Properties.
  2. Check the box next to the IPv6 Remote access server on the General tab.

Prefix Assignment

Next, an IPv6 prefix must be assigned to each RRAS VPN server. This IPv6 prefix must be unique for each server and not in use anywhere else on the internal network. Unlike IPv4, IPv6 addresses cannot be assigned from the same prefix (subnet) as the VPN server’s internal network interface. With that, ensure that internal network IPv6 routing returns traffic for the assigned IPv6 prefixes to the corresponding VPN server.

Perform the following steps to assign an IPv6 prefix for VPN client use.

  1. Right-click the RRAS VPN server in the navigation tree and choose Properties.
  2. Select the IPv6 tab.
  3. Check the box next to Enable IPv6 Forwarding.
  4. If force tunneling is required (not recommended), check the box next to Enable Default Route Advertisement.
  5. Enter an IPv6 prefix in the IPv6 prefix assignment field. Again, ensure the IPv6 prefix is globally unique, and that internal network routing is configured to return traffic to the VPN server that owns the prefix.
  6. If your RRAS server is multi-homed, select the internal network interface from the Adapter drop-down list.

DHCP

Organizations with IPv6 deployed internally may use Microsoft Windows DHCPv6 or a dedicated DNS/DHCP/IP Address Management (IPAM) (DDI) solution like Infoblox. However, Windows Server RRAS does not support DHCPv6 for VPN client IP address assignment. Administrators must manually assign an IPv6 prefix per server. However, administrators can use DHCP alongside IPv6 prefix assignment for VPN client IPv4 addressing.

Limitations

While IPv6 may solve some problems for Always On VPN administrators, it has some limitations. Here are some crucial considerations for IPv6 and Always On VPN at the time of this writing.

Traffic Filters

You cannot use IPv6 when configuring traffic filters for Always On VPN. Specifying IPv6 elements in a traffic filter rule will prevent Always On VPN from working at all. More details here.

Intune and Routing

When split tunneling is enabled, Microsoft Intune will not accept IPv6 routes using the standard IPv6 subnet prefix of /64. The UI complains that “the value must be between 1 and 32”.

You can use the custom XML deployment option to configure Always On VPN to support split tunneling correctly as a workaround.

Additional Information

Overview of IPv6

Everything You Never Knew about NAT

Disabling IPv6 Breaks Windows Server RRAS

Microsoft Always On VPN Traffic Filters and IPv6

Discussing Microsoft and IPv6 on the IPv6 Buzz Podcast (Packet Pushers)

Always On VPN RRAS Internal Interface Non-Operational

Windows 10 Always On VPN Routing Configuration

Always On VPN administrators troubleshooting connectivity issues may find the Internal network interface in the Routing and Remote Access management console (rrasmgmt.msc) administrative status indicates ‘Unknown’. They will also notice the Operational Status shows Non-operational.

Internal Interface

For clarification, the ‘Internal’ network interface in the Routing and Remote Access management console, as shown above, is not a physical network adapter on the server. Instead, it is a virtual network interface used only for incoming VPN connections.

Non-Operational

The Internal virtual network interface will not be created until the VPN server accepts its first VPN connection. Because of this, the Internal interface will have an operational status of non-operational until the first client attempts to connect. When this occurs, RRAS creates the interface, then assigns it the first IP address from the static IPv4 address pool. Alternatively, if DHCP is configured, it will assign the first IP address returned by the DHCP server.

Interface Names

While discussing network interfaces, I typically recommend renaming them in Windows to identify their function, especially when using two NIC configurations. However, be careful not to name the server’s internal network adapter ‘Internal’, as this can be confusing in the future. In my example above, I use the name ‘LAN’ to identify the internal adapter to distinguish it from the server’s ‘Internal’ virtual interface.

Additional Information

Windows Server RRAS Service Does Not Start

Windows Server RRAS Monitoring and Reporting

Microsoft Always On VPN and RRAS in Azure

Microsoft Always On VPN and RRAS with Signle NIC