Always On VPN July 2023 Security Updates

Hello, Always On VPN administrators! It’s the second Tuesday of the month, so you know what that means. Yes, it’s Patch Tuesday! This month’s security updates include several fixes for vulnerabilities potentially affecting Microsoft Always On VPN deployments.

RRAS Vulnerabilities

Microsoft’s July 2023 security updates include fixes affecting Windows Servers with the Routing and Remote Access Service (RRAS) role installed. Security vulnerabilities CVE-2023-35365, CVE-2023-35366, and CVE-2023-35367 are all Remote Code Execution (RCE) vulnerabilities with a Critical security rating and a CVSS score of 9.8. These security vulnerabilities in Windows Server RRAS are particularly troublesome due to the nature of the workload. RRAS servers are, by design, exposed to the public Internet. Although there are no known exploits in the wild at this time, this attack requires no special privileges other than network access. Administrators running Windows Server with RRAS installed are encouraged to update as soon as possible.

AD CS Vulnerabilities

Most Always On VPN implementations leverage enterprise PKI certificates for user and device authentication. Administrators commonly deploy Microsoft Active Directory Certificate Services (AD CS) to support this. This month there are two security vulnerabilities in AD CS marked as Important. CVE-2023-35350 and CVE-2023-35351 address RCE vulnerabilities that exploit a race condition on the server. However, AD CS servers are not exposed to untrusted networks. In addition, attackers would require administrative rights on the server to exploit these vulnerabilities.

Network Load Balancing

Finally, of importance to Always On VPN administrators using Windows Network Load Balancing (NLB) to provide load balancing for their RRAS servers, there is a vulnerability in the NLB service. CVE-2023-33163 addresses an RCE vulnerability in NLB identified as Important.

Additional Information

Microsoft July 2023 Security Updates

Windows Server 2022 KB5028171 (Build 20348.1850)

Windows Server 2019 KB5028168 (Build 17763.4645)

Windows Server 2016 KB 5028169 (Build 14393.6085)

Windows 11 22H2 KB8028185 (Build 22621.1992)

Windows 11 21H2 KB5028182 (Build 22000.2176)

Always On VPN RRAS and Stale Connections

Always On VPN Updates for RRAS and IKEv2

Always On VPN administrators may be familiar with an issue that affects Windows Server Routing and Remote Access Service (RRAS) servers, where many stale VPN connections appear in the list of active connections. The issue is most prevalent when using IKEv2, either for the Always On VPN device tunnel or the user tunnel. Typically, this does not cause problems, but some administrators have reported issues related to port exhaustion or failed IKEv2 connections when many stale connections are present. Stale connections happen so frequently that I created a PowerShell script to clean them up on the RRAS server. Restarting the RemoteAccess service or rebooting the server also clears stale connections.

Microsoft Fix

Thankfully, Microsoft has addressed these issues in Windows Server 2019 and Windows Server 2022 this month. An update is now available in the March 2023 security update that resolves this problem.

You can find more information about the updates here.

The update was not made available for Windows Server 2016, however. Organizations are encouraged to upgrade to Windows Server 2019 or later to address this problem.

Additional Information

Always On VPN Updates for RRAS and IKEv2

Always On VPN IKEv2 Load Balancing and NAT

Always On VPN and IKEv2 Fragmentation

Always On VPN and Split DNS

Cloudflare Public DNS Resolver Now Available

Split DNS, sometimes called ‘split brain’ DNS, is when an organization uses the same DNS namespace internally and externally. For example, the internal Active Directory domain name is example.com, so internal resources are accessed using a fully qualified domain name (FQDN) like dc1.example.com. Additionally, external properties such as mail and web services use the same namespace so that a public web server might have a name like www.example.com. Internal resources will resolve to internal, private IP addresses, whereas public services resolve to external, public IP addresses.

Complications

Things get complicated when the same resource (FQDN) is available internally and externally, especially for Always On VPN clients. For example, accessing app.example.com on the internal network resolves to a private address, but accessing the same resource on the Internet resolves to a public IP address. Often there are different authentication requirements for internal and external resources, which can yield unexpected results.

Name Resolution

Always On VPN administrators might prefer app.example.com to be accessed via the Internet when connected with Always On VPN. However, VPN clients will attempt to connect via the internal network using their default configuration. Solving this challenge requires internal DNS server changes.

NRPT?

It might be tempting for administrators to use the Name Resolution Policy Table (NRPT) to solve name resolution issues for Always On VPN. However, the NRPT has some limitations and may not always produce the desired results. For example, the NRPT only directs DNS queries. It does not define which resource records are returned by DNS. Also, some applications ignore the NRPT, which limits its usefulness. A better solution is to use DNS Policies in Windows Server.

DNS Policies

Microsoft introduced DNS policies with Windows Server 2016. DNS policies are a powerful tool administrators can use to fine-tune name resolution based on many factors. In the case of split DNS, administrators can configure internal DNS to return an IP address for a resource based on the source IP address of the name resolution query. VPN clients receive one IP address for a given DNS query, while all other clients receive a different IP address. DNS policies ensure that remote clients connected to the VPN will receive the proper IP address for the resource requested, as defined by the administrator.

Caveats

DNS policies are powerful and flexible, but there are some potential drawbacks. All enterprise DNS servers used by Always On VPN clients must be running Windows Server 2016 or later. Also, administrators must use PowerShell to configure DNS policies exclusively. There is no GUI interface to configure DNS policies. DNS policies do not appear in the DNS management interface, which could confuse an administrator unaware that DNS policies are in place. In addition, DNS client subnets and query resolution policies do not replicate across DNS servers. Administrators must manually configure these on each DNS server used by Always On VPN clients. However, zone scopes and resource records in those scopes do replicate automatically.

Scenario

For demonstration purposes, let’s assume that an Always On VPN client needs to access foo.example.com. It resolves to a private IP address on the internal network and a public IP address on the Internet. By default, foo.example.com will resolve to the internal private IP address of the server when connected with Always On VPN. However, the desire is to have foo.example.com resolve to the public IP address when connected with Always On VPN. To accomplish this, we’ll create a DNS policy to ensure that connected Always On VPN clients can resolve foo.example.com to the public IP address when resolving this name over the VPN tunnel.

DNS Policy Configuration

Open an elevated PowerShell command on a DNS server and perform the following steps to create a DNS policy for VPN clients.

Client Subnet

Run the Add-DnsServerClientSubnet PowerShell command to create a client subnet in DNS that includes all IP networks assigned to VPN clients. Summarize IP prefixes if there are multiple VPN servers in the organization.

Add-DnsServerClientSubnet -Name VPN -IPv4Subnet ‘172.16.100.0/22’ -IPv6Subnet ‘2001:db8:fcd2:1000::/60’

If summarizing IP prefixes for multiple servers isn’t possible, multiple subnets can be added to a DNS client subnet using the following command.

Add-DnsServerClientSubnet -Name VPN -IPv4Subnet @(‘172.16.100.0/24’, ‘172.16.101.0/24’, ‘172.16.102.0/24’, ‘172.16.103.0/24’) -IPv6Subnet @(‘2001:db8:fcd2:1001::/64’, ‘2001:db8:fcd2:1002::/64’, ‘2001:db8:fcd2:1003::/64’)

To make changes to an existing DNS client subnet, use the Set-DnsServerClientSubnet PowerShell command.

Note: Client Subnets do not replicate across domain controllers. Run the command above on all DNS servers or each DNS server used by Always On VPN clients.

Zone Scope

Create a Zone Scope that includes the DNS records to be returned to VPN clients. The default zone scope is the DNS zone itself. Configure an additional zone scope for the DNS zone by using the Add-DnsServerZoneScope PowerShell command.

Add-DnsServerZoneScope -ZoneName example.com -Name VPN

Resource Records

Next, add DNS records to the new zone scope using the Add-DnsServerResourceRecord PowerShell command.

Add-DnsServerResourceRecord -ZoneName example.com -A -Name foo -IPv4Address 203.0.113.12 -ZoneScope VPN

Add-DnsServerResourceRecord -ZoneName example.com -AAAA -Name foo -IPv6Address 2001:db8:21::12 -ZoneScope VPN

DNS Policy

Finally, create a DNS query resolution policy that ties everything together. Run the Add-DnsServerQueryResolutionPolicy command to create the DNS query resolution policy. Once configured, when the DNS server receives a DNS query, the policy will recognize that the query originates from a VPN client subnet and will return the resource record from the VPN zone scope with the public IP address defined previously.

Add-DnsServerQueryResolutionPolicy -Name VPN -Action ALLOW -ClientSubnet ‘EQ,VPN’ -FQDN ‘EQ,foo.example.com’ -ZoneScope ‘VPN,1’ -ZoneName example.com

Note: DNS query resolution policies do not replicate across domain controllers. Run the command above on all DNS servers or each DNS server used by Always On VPN clients.

Results

Once complete, the hostname ‘foo’ in our example above resolves to different IP addresses based on the client’s IP address.

DNS query for ‘foo’ from internal client.

DNS query for ‘foo’ from VPN client.

Summary

There are many scenarios where Windows Server DNS policies can be used to fine-tune name resolution for Always On VPN clients. Hopefully, this example gives you an idea of how DNS policies work, and you can use them to solve your unique name resolution challenges with Always On VPN.

Additional Information

Windows Server DNS Policies Overview

Always On VPN Short Name Access Failure

Always On VPN Client DNS Server Configuration