Always On VPN July 2023 Security Updates

Hello, Always On VPN administrators! It’s the second Tuesday of the month, so you know what that means. Yes, it’s Patch Tuesday! This month’s security updates include several fixes for vulnerabilities potentially affecting Microsoft Always On VPN deployments.

RRAS Vulnerabilities

Microsoft’s July 2023 security updates include fixes affecting Windows Servers with the Routing and Remote Access Service (RRAS) role installed. Security vulnerabilities CVE-2023-35365, CVE-2023-35366, and CVE-2023-35367 are all Remote Code Execution (RCE) vulnerabilities with a Critical security rating and a CVSS score of 9.8. These security vulnerabilities in Windows Server RRAS are particularly troublesome due to the nature of the workload. RRAS servers are, by design, exposed to the public Internet. Although there are no known exploits in the wild at this time, this attack requires no special privileges other than network access. Administrators running Windows Server with RRAS installed are encouraged to update as soon as possible.

AD CS Vulnerabilities

Most Always On VPN implementations leverage enterprise PKI certificates for user and device authentication. Administrators commonly deploy Microsoft Active Directory Certificate Services (AD CS) to support this. This month there are two security vulnerabilities in AD CS marked as Important. CVE-2023-35350 and CVE-2023-35351 address RCE vulnerabilities that exploit a race condition on the server. However, AD CS servers are not exposed to untrusted networks. In addition, attackers would require administrative rights on the server to exploit these vulnerabilities.

Network Load Balancing

Finally, of importance to Always On VPN administrators using Windows Network Load Balancing (NLB) to provide load balancing for their RRAS servers, there is a vulnerability in the NLB service. CVE-2023-33163 addresses an RCE vulnerability in NLB identified as Important.

Additional Information

Microsoft July 2023 Security Updates

Windows Server 2022 KB5028171 (Build 20348.1850)

Windows Server 2019 KB5028168 (Build 17763.4645)

Windows Server 2016 KB 5028169 (Build 14393.6085)

Windows 11 22H2 KB8028185 (Build 22621.1992)

Windows 11 21H2 KB5028182 (Build 22000.2176)

Always On VPN Book Available for Pre-Order

Great news! My new book, Implementing Always On VPN, is now available for pre-order on Amazon.com. This new book, scheduled for release in late 2021, is a comprehensive implementation guide for Windows 10 Always On VPN. Drawing on many years of experience deploying Always On VPN for organizations worldwide, it covers all aspects of an Always On VPN deployment, including planning and design, prerequisite gathering, infrastructure preparation, and client deployment.

In addition, it contains detailed, prescriptive guidance for advanced configuration options such as application and traffic filtering and proxy server configuration. Cloud deployments using Azure VPN gateway and Virtual WAN are covered, and it includes guidance for configuring Azure MFA and Conditional Access.

Also, the book includes thorough guidance for provisioning certificates using Microsoft Endpoint Manager/Intune using both PKCS and SCEP. It outlines options for high availability for VPN and authentication infrastructure and provides details for ongoing system maintenance and operational support.

Finally, the book has an entire chapter dedicated to troubleshooting and resolving common (and not so common!) issues encountered with Windows 10 Always On VPN.

Reserve your copy today. Pre-order Implementing Always On VPN now!

Chapter List

  1. Always On VPN Overview
  2. Plan an Always On VPN Deployment
  3. Prepare the Infrastructure
  4. Configure Windows Server for Always On VPN
  5. Provision Always On VPN clients
  6. Advanced Configuration
  7. Cloud Deployments
  8. Deploy Certificates with Intune
  9. Integrating Azure MFA
  10. High Availability
  11. Monitor and Report
  12. Troubleshooting

Always On VPN IKEv2 Load Balancing and NAT

Always On VPN IKEv2 Load Balancing and NATOver the last few weeks, I’ve worked with numerous organizations and individuals troubleshooting connectivity and performance issues associated with Windows 10 Always On VPN, and specifically connections using the Internet Key Exchange version 2 (IKEv2) VPN protocol. An issue that appears with some regularity is when Windows 10 clients fail to connect with error 809. In this scenario, the server will accept connections without issue for a period of time and then suddenly stop accepting requests. When this happens, existing connections continue to work without issue in most cases. Frequently this occurs with Windows Server Routing and Remote Access Service (RRAS) servers configured in a clustered array behind an External Load Balancer (ELB).

Network Address Translation

It is not uncommon to use Network Address Translation (NAT) when configuring Always On VPN. In fact, for most deployments the public IP address for the VPN server resides not on the VPN server, but on an edge firewall or load balancer connected directly to the Internet. The firewall/load balancer is then configured to translate the destination address to the private IP address assigned to the VPN server in the perimeter/DMZ or the internal network. This is known a Destination NAT (DNAT). Using this configuration, the client’s original source IP address is left intact. This configuration presents no issues for Always On VPN.

Source Address Translation

When troubleshooting these issues, the common denominator seems to be the use of Full NAT, which includes translating the source address in addition to the destination. This results in VPN client requests arriving at the VPN server as appearing not to come from the client’s original IP address, but the IP address of the network device (firewall or load balancer) that is translating the request. Full NAT may be explicitly configured by an administrator, or in the case of many load balancers, configured implicitly because the load balancer is effectively proxying the connection.

Known Issues

IKEv2 VPN connections use IPsec for encryption, and by default, Windows limits the number of IPsec Security Associations (SAs) coming from a single IP address. When a NAT device is performing destination/full NAT, the VPN server sees all inbound IKEv2 VPN requests as coming from the same IP address. When this happens, clients connecting using IKEv2 may fail to connect, most commonly when the server is under moderate to heavy load.

Resolution

The way to resolve this issue is to ensure that any load balancers or NAT devices are not translating the source address but are performing destination NAT only. The following is configuration guidance for F5, Citrix ADC (formerly NetScaler), and Kemp load balancers.

F5

On the F5 BIG-IP load balancer, navigate to the Properties > Configuration page of the IKEv2 UDP 500 virtual server and choose None from the Source Address Translation drop-down list. Repeat this step for the IKEv2 UDP 4500 virtual server.

Always On VPN IKEv2 Load Balancing and NAT

Citrix ADC

On the Citrix ADC load balancer, navigate to System > Settings > Configure Modes and check the option to Use Subnet IP.

Always On VPN IKEv2 Load Balancing and NAT

Next, navigate to Traffic Management > Load Balancing > Service Groups and select the IKEv2 UDP 500 service group. In the Settings section click edit and select Use Client IP. Repeat these steps for the IKEv2 UDP 4500 service group.

Always On VPN IKEv2 Load Balancing and NAT

Kemp

On the Kemp LoadMaster load balancer, navigate to Virtual Services > View/Modify Services and click Modify on the IKEv2 UDP 500 virtual service. Expand Standard Options and select Transparency. Repeat this step for the IKEv2 UDP 4500 virtual service.

Always On VPN IKEv2 Load Balancing and NAT

Caveat

Making the changes above may introduce routing issues in your environment. When configuring these settings, it may be necessary to configure the VPN server’s default gateway to use the load balancer to ensure proper routing. If this is not possible, consider implementing the workaround below.

Workaround

To fully resolve this issue the above changes should be made to ensure the VPN server can see the client’s original source IP address. If that’s not possible for any reason, the following registry key can be configured to increase the number of established SAs from a single IP address. Be advised this is only a partial workaround and may not fully eliminate failed IKEv2 connections. There are other settings in Windows that can prevent multiple connections from a single IP address which are not adjustable at this time.

To implement this registry change, open an elevated PowerShell command window on the RRAS server and run the following commands. Repeat these commands on all RRAS servers in the organization.

New-ItemProperty -Path ‘HKLM:SYSTEM\CurrentControlSet\Services\IKEEXT\Parameters\’ -Name IkeNumEstablishedForInitialQuery -PropertyType DWORD -Value 50000 -Force

Restart-Service IKEEXT -Force -PassThru

Additional Information

IPsec Traffic May Be Blocked When A Computer is Behind a Load Balancer

Windows 10 Always On VPN IKEv2 Load Balancing with Citrix NetScaler ADC

Windows 10 Always On VPN IKEv2 Load Balancing with F5 BIG-IP

Windows 10 Always On VPN IKEv2 Load Balancing with Kemp LoadMaster