Always On VPN Troubleshooting with Windows Packet Monitor PktMon.exe

When troubleshooting Always On VPN, taking a network packet capture or network trace is sometimes required to identify the root cause of a problem. After all, Packets Don’t Lie™. There are numerous ways to capture packets. Many administrators will install Wireshark for this purpose. However, Windows has a native packet capture tool called PktMon.exe that offers many advantages over Wireshark.

Wireshark

Many Always On VPN administrators will be familiar with Wireshark. Wireshark is a popular open-source network protocol analyzer that enables the capture and analysis of network traffic for troubleshooting. A packet capture driver must first be installed to capture network traffic with Wireshark. Typically, administrators will install Npcap, which is part of the default installation of Wireshark. Installing a capture driver poses a potential problem, as the administrator must install software on the target device before capturing traffic. Installing software may not always be feasible or possible. Fortunately, there’s an alternative.

PktMon.exe

The Windows Packet Monitor (PktMon.exe) is a built-in command-line tool first introduced in Windows 10 1809 and Windows Server 2019. It is designed to capture network traffic on Windows servers and client systems. This native lightweight tool is ideal for collecting network traces for offline analysis.

Capture All Interfaces

The most common scenario for PktMon.exe is to capture data for offline analysis. Use the following command to capture all network traffic on all active network interfaces.

PktMon.exe start –capture –file c:\capture.etl –pkt-size 0 –comp nics –flags 0x10

The command breaks down as follows:

–capture – captures network traffic

–file – the path of the file to save the data to

–pkt-size 0 – captures the full packet (not truncated)

–comp nics – captures traffic on all active network interfaces

–flags 0x10 – captures the raw packet

After reproducing the issue, you can stop the trace by running the following command.

PktMon.exe stop

Capture Specific Interface

Administrators may wish to capture traffic on a specific network interface instead of all active network interfaces. In this example, I have a multi-homed VPN server and want to capture traffic on only the DMZ interface. To do this, use PktMon.exe to enumerate all interfaces using the following command.

PktMon.exe list

Note: The output of PktMon.exe filter list does not include information that easily maps to existing network interfaces. I suggest also running the Get-NetAdapter PowerShell command to view detailed information about network interfaces. You can use this information to select the correct Network ID for PktMon.exe filtering.

Next, change the value of –comp nics in the command referenced above to –comp <Network ID>. Here’s an example.

PktMon.exe start –capture –file c:\capture.etl –pkt-size 0 –comp 62 –flags 0x10

Filtering

It’s also possible to use PktMon.exe to capture network traffic selectively. Filtering allows you to narrow the capture to relevant traffic, making analysis easier and faster. Add a filter, then start a trace to restrict data capture to traffic that matches the defined filters. You can add one or more filters to apply to the capture. Here are a few examples.

Protocols and Ports

Let’s say you are troubleshooting a device tunnel connection and want to see only IKEv2 traffic. The following filter will restrict the network capture to only the IKEv2-related protocols and ports.

PktMon.exe filter add IKEv2 -t UDP -p 500
PktMon.exe filter add IKEv2 -t UDP -p 4500

IP Address

The following filter will capture data that includes the specified IP address in the source or destination address field.

PktMon.exe filter add VPN1 -i 172.21.12.50

You can also specify IP address subnets using their CIDR notation.

PktMon.exe filter add Subnet1 -i 172.16.0.0/16

View and Clear Filters

You can view configured filters using the following command.

PktMon.exe filter list

You can remove configured filters using the following command. Use with caution, as this removes ALL filters!

PktMon.exe filter remove

Reference

You’ll find a complete list of PktMon.exe filters here.

Analysis

PktMon.exe outputs captured data in ETL format. Administrators can convert captured data to the standard PCAP format by running the following command.

PktMon.exe etl2pcap <path of trace file>

This command converts the file from ETL to PCAPNG format. Administrators can then open the capture in Wireshark for further detailed analysis.

Display Only

PktMon.exe can be configured to display network traffic in the console for quick troubleshooting. Console traffic display can be helpful for those scenarios where a quick check to validate traffic is reaching a particular destination is required. Here’s an example.

PktMon.exe start –capture –pkt-size 0 –comp nics –flags 0x10 -m real-time

Note: In the example above, I applied a traffic filter to limit the capture to only SSTP traffic (TCP 443).

Limitations

One crucial limitation of PktMon.exe is that it doesn’t support persistent network captures that survive a reboot. Persistent captures can be helpful when troubleshooting a device tunnel connection or slow logons. In this scenario, you must use netsh.exe.

netsh.exe trace start capture=yes tracefile=c:\tracefile.etl persistent=yes

<reboot>

netsh.exe trace stop

Although PktMon.exe supports the ‘etl2pcap’ switch, it does NOT work for converting .etl files generated with netsh.exe. To convert captures created with netsh.exe, use the open-source etl2pcapng tool.

Learn More

PktMon.exe has many different uses. This post barely scratches the surface of what PktMon.exe can do. PktMon.exe comes with robust help, accessible by adding the ‘help’ switch to commands. Here are some examples.

PktMon.exe start help
PktMon.exe filter add help

Be sure to view the online help to explore various options for capturing and logging to meet your specific needs.

Summary

PktMon.exe is a native command-line utility in Windows that provides a lightweight solution for capturing network traffic, making it particularly useful for Always On VPN troubleshooting. Key functionalities include full-packet captures, selective filtering by protocol, port, or IP address, and conversion of ETL files to PCAPNG format for analysis in tools like Wireshark. Real-time traffic displays are also supported for quick diagnostics. While effective for many scenarios, PktMon.exe lacks support for persistent captures across reboots, for which netsh.exe is recommended. The techniques outlined above offer administrators a practical, software-free approach to deep packet inspection for troubleshooting Always On VPN issues.

Have you used PktMon.exe for network troubleshooting? Feel free to share tips and tricks in the comments section below!

Additional Information

Getting Started with Windows Packet Monitor (PktMon.exe)

PktMon.exe Filter Reference

Open-source Etl2pcap for netsh.exe captures

Always On VPN Load Balancing with Loadbalancer.org

Recently, I had the opportunity to deploy the Loadbalancer.org load balancer as part of an enterprise Always On VPN deployment. In the past, I’ve published guidance for using F5 BIG-IP, Citrix ADC (formerly NetScaler), and Kemp LoadMaster, so in this post, I’ll provide guidance for configuring Loadbalancer.org for Always On VPN.

IKEv2

Open the Loadbalancer.org management console and follow the steps below to configure Always On VPN load balancing on the appliance.

Create Virtual Service

Create a layer 4 virtual service for IKEv2.

  1. Click Cluster Configuration.
  2. Click Layer 4 – Virtual Services.
  3. Click Add a new Virtual Service.
  4. Enter a descriptive name for the virtual service in the Label field.
  5. Enter the virtual IP address (VIP) for the service in the IP Address field.
  6. Enter 500,4500 in the Ports field.
  7. Select UDP from the Protocol drop-down list.
  8. Select NAT from the Forwarding Method drop-down list.
  9. Click Update.

Add Real Servers

Add real servers to the virtual service.

  1. Click Layer 4 – Real Servers.
  2. Click Add a new Real Server next to the IKEv2 virtual service.
  3. Enter a descriptive name for the real server in the Label field.
  4. Enter the IP address of the real server in the Real Server IP Address field.
  5. Click Update.
  6. Repeat these steps for each additional VPN server in the cluster.

SSTP

Follow the steps below to configure SSTP load balancing on the appliance.

Create Virtual Service

Create a layer 4 virtual service for SSTP.

  1. Click Cluster Configuration.
  2. Click Layer 4 – Virtual Services.
  3. Click Add a new Virtual Service.
  4. Enter a descriptive name for the virtual service in the Label field.
  5. Enter the virtual IP address (VIP) for the service in the IP Address field.
  6. Enter 443 in the Ports field.
  7. Select TCP from the Protocol drop-down list.
  8. Select NAT from the Forwarding Method drop-down list.
  9. Click Update.

Configure Virtual Service Health Check

Update the health check method for the SSTP virtual service.

  1. Click Layer 4 – Virtual Services.
  2. Click Modify on the SSTP virtual service.
  3. Select Negotiate from the Check Type drop-down list in the Health Checks section.
  4. Enter 443 in the Check Port field.
  5. Select HTTPS from the Protocol drop-down list.
  6. Enter /sra_{BA195980-CD49-458b-9E23-C84EE0ADCD75}/ in the Request to send field.
  7. Enter 401 in the Response expected field.
  8. Click Update.

Note: Using the Negotiate health check type for the SSTP monitor on Loadbalancer.org appliances requires version 8.13.0 or later. Administrators can use the External script option when using earlier releases of Loadbalancer.org appliances. An SSTP health check script for Loadbalancer.org can be found here.

Add Real Servers

Add real servers to the virtual service.

  1. Click Layer 4 – Real Servers.
  2. Click Add a new Real Server next to the SSTP virtual service.
  3. Enter a descriptive name for the real server in the Label field.
  4. Enter the IP address of the real server in the Real Server IP Address field.
  5. Click Update.
  6. Repeat these steps for each additional VPN server in the cluster.

Review

Once complete, click System Overview to view the overall health of your VPN servers.

Summary

The Loadbalancer.org appliance is an efficient, cost-effective, and easy-to-configure load-balancing solution that works well with Always On VPN implementations. It’s available as a physical or virtual appliance. There’s also a cloud-based version. It also includes advanced features such as TLS offload, web application firewall (WAF), global server load balancing (GSLB), and more. If you are looking for a layer 4-7 load balancer for Always On VPN and other workloads, be sure to check them out.

Additional Information

Loadbalancer.org Virtual Appliance

SSTP Health Check Script for Loadbalancer.org

Enable TLS in Microsoft SQL Server 2022

In a recent post, I described some of the security benefits of using Transport Layer Security (TLS) with Microsoft SQL Server. Configuration changes are required to take full advantage of these capabilities. By default, SQL Server uses an unmanaged, self-signed certificate, which provides little security value. The best practice is to use a certificate issued by the organization’s enterprise PKI. In this guide, I’ll demonstrate how to prepare and deploy a certificate template for SQL server using Active Directory Certificate Services (AD CS), enroll for the certificate, and configure SQL server to use the new certificate for TLS connections.

Note: I have recorded a video demonstration for enabling TLS in Microsoft SQL Server 2022 on my YouTube channel here. Enjoy!

Certificate Requirements

The minimum recommended requirements for a TLS certificate for SQL Server 2022 are:

  • Subject Name = Server’s fully qualified domain name or the alias name of the cluster
  • 2048-bit RSA key with SHA256
  • Server Authentication EKU (1.3.6.1.5.5.7.3.1)

Certificate Template

Administrators must prepare a certificate template in Active Directory (AD) adhering to the requirements listed above. On an issuing certification authority (CA) or an administrative workstation with the Remote Server Administration Tools (RSAT) installed, open the Certificate Templates management console (certtmpl.msc) and perform the following steps.

  1. Right-click the default Web Server template and choose Duplicate Template.
  2. Select the Compatibility tab.
    1. In the Compatibility Settings section, select the latest version of Windows Server supported by your issuing CA servers from the Certification Authority drop-down list.
    1. Select Windows 10/Windows Server 2016 from the Certificate recipient drop-down list.
  3. Select the General tab.
    1. Enter a descriptive name in the Template display name field.
    1. Select a validity period of 1 year with a renewal period of 6 weeks.
  4. Select the Cryptography tab.
    1. Select Key Storage Provider from the Provider Category drop-down list.
    1. Select RSA from the Algorithm name drop-down list.
    1. Enter 2048 in the Minimum key size field.
    1. Select SHA256 from the Request hash drop-down list.
  5. Select the Issuance Requirements tab.
    1. Check the box next to CA certificate manager approval.
  6. Select the Subject Name tab.
    1. Select Supply in the request.
  7. Select the Extensions tab.
    1. Select Application Policies.
    1. Ensure that Server Authentication is the only application policy listed.
  8. Select the Security tab.
    1. Click Add.
    1. Grant Read and Enroll permissions to the SQL Server security group or the SQL server’s computer account.
    1. Ensure no other users/groups have enroll permission.

Once complete, publish the certificate template on all issuing CA servers in the organization.

Enroll Certificate

The certificate enrollment process involves several steps.

Request Certificate

To enroll for a new TLS certificate, open the computer certificate management console (certlm.msc) on the SQL server and perform the following steps.

  1. Right-click on the Personal folder and choose All Tasks > Request New Certificate.
  2. Click Next.
  3. Click Next.
  4. Check the box next to the SQL server certificate template.
  5. Click the More information is required to enroll for this certificate. Click here to configure settings link.
  6. Select the Subject tab.
  7. In the Subject Name section, select Common Name from the Type drop-down list.
  8. Enter the SQL server’s fully qualified domain name (FQDN) or the alias name of the SQL cluster in the Value field.
  9. Click Add.
  10. In the Alternative name section, select DNS from the Type drop-down list.
  11. Enter the SQL server’s fully qualified domain name (FQDN) or the alias name of the SQL cluster in the Value field.
  12. Click Add.
  13. [OPTIONAL] Enter the SQL server’s single-label hostname in the Value field.

Note: Adding the single-label hostname to the Subject Alternative Name list allows administrators or applications to connect to the SQL server using its short name (NetBIOS name) without resulting in a subject name mismatch error.

  1. Click Add.
  2. Click Ok.
  3. Click Enroll. The status should indicate that enrollment is pending.
  4. Click Finish.

Approve Certificate

Once the certificate request is made, the request must now be approved. On an issuing certification authority (CA), or an administrative workstation with the Remote Server Administration Tools (RSAT) installed, open the Certification Authority management console (certsrv.msc) and perform the following steps.

  1. Expand the CA.
  2. Select Pending Requests.
  3. Note the request ID for the pending request. After approval, the request ID will be required later to retrieve the certificate.
  4. Right-click the pending request and choose All Tasks > Issue.

Important Note: I am performing the above tasks in a test lab environment. On a properly configured CA in a production environment, the requestor should not be able to approve their own request. In your environment, you may need to request that a CA administrator review and approve your request.

Install Certificate

Once the certificate has been approved and issued, open an elevated PowerShell or command window on the SQL server and perform the following steps.

  1. Enter certreq.exe -retrieve <request ID>.
  2. Select the CA where the certificate was issued.
  3. Click Ok.
  4. Select a location and enter a name for the file in the File name field.
  5. Click Save.
  6. Enter certreq.exe -accept <path to certificate file>.

Configure Certificate

Once the certificate has been enrolled on the SQL server, expand Personal > Certificates and refresh the view to confirm certificate enrollment. Next, perform the following steps.

  1. Right-click the SQL server certificate and choose All Tasks > Manage Private Keys.
  2. Click Add.
  3. Enter the name of the SQL server domain service account and click Check Names.
  4. If using the default SQL server service account, perform the following steps.
    1. Click on Locations.
      1. Select the local server.
      1. Click Ok.
      1. Enter NT Service\MSSQLSERVER and click Check Names.
  5. Click Ok.
  6. Uncheck Full control. The only permission required is Read.
  7. Click Ok.

SQL Configuration

Next, the new certificate must be assigned to the SQL Server service. Open the SQL Server Configuration Manager (sqlservermanager16.msc) and perform the following steps.

  1. Expand SQL Server Network Configuration.
  2. Right-click Protocols for MSSQLSERVER and choose Properties.
  3. Select the Certificate tab.
    1. Select the new certificate from the Certificate drop-down list.
  4. Select the Flags tab.
    1. Select Yes next to Force Strict Encryption.
  5. Click Ok.

Restart the SQL Server service for the changes to take effect.

Important Note: Selecting Force Strict Encryption will force encryption and certificate validation for all clients connecting to the SQL server. It will override any settings to bypass encryption or certificate checks. Force Strict Encryption may not be compatible with older applications or drivers. Please test thoroughly before enabling this setting.

Video

I’ve published a demonstration video for configuring TLS on Microsoft SQL Server 2022 on YouTube. You can find the video here.

Summary

After completing the configuration steps above, administrators can be assured that all communication between clients and the SQL server is fully protected with TLS using modern cryptography and their enterprise-managed certificate. With TLS enabled for SQL server communication, security is enhanced by encrypting data in transit, ensuring authentication, and protecting sensitive information from interception. In addition, this configuration helps meet compliance requirements.

Additional Information

TLS and Microsoft SQL Server 2022

Always On VPN and SQL Target Principal Name Incorrect