The January 2022 security updates for Microsoft Windows include several important updates that will affect Always On VPN deployments. Specifically, CVE-2022-21849 addresses a Remote Code Execution (RCE) vulnerability that should be addressed immediately. The January 2022 security update also includes updates for several IKE Denial-of-Service (DoS) vulnerabilities, in addition to privilege escalation vulnerabilities in the Remote Access Connection Manager.
Vulnerable Systems
These vulnerabilities are present on both Windows Server and Client operating systems. Essentially, any Windows server or client using IPsec is vulnerable and potentially exploitable.
Vulnerabilities
The following is a list of security updates related to Always On VPN deployments.
Windows IKE Extension Remote Code Execution (RCE) Vulnerability
Secure Socket Tunneling Protocol (SSTP) is a Microsoft-proprietary VPN protocol with several advantages over Internet Key Exchange version 2 (IKEv2) for Always On VPN user tunnel connections. SSTP uses HTTP with Transport Layer Security (TLS) to encrypt communication between the Always On VPN client and the VPN gateway. SSTP is very firewall-friendly, with VPN connections operating on the commonly open TCP port 443, resulting in more consistent VPN availability. SSTP throughput is better compared to IKEv2 as well.
Learn more about TLS with Practical TLS, a comprehensive online video training course.
TLS and Windows Server
For versions of Windows Server before Windows Server 2022, the out-of-the-box security for TLS is not ideal. TLS is notoriously complex to configure, with myriad options for administrators to choose from. However, with the release of Windows Server 2022 and Windows 11, Microsoft has introduced support for the latest TLS specification, TLS 1.3, which eases much of this configuration pain.
TLS 1.3
TLS 1.3 provides significant advantages for Always On VPN SSTP user tunnel connections in security and performance.
Security
TLS 1.3 is greatly simplified and offers only five cipher suites, all considered secure by today’s standards. In addition, all TLS 1.3 ciphers support forward secrecy, ensuring the privacy of communication even in the event of a server private key compromise.
Performance
The TLS handshake in TLS 1.3 is streamlined and requires less back-and-forth (round trips) to establish a connection. TLS 1.3 speeds connection establishment for new Always On VPN user tunnel connections.
Caveat
Adding support for TLS 1.3 on the server-side is a compelling reason to consider upgrading existing Windows Server Routing and Remote Access Service (RRAS) servers to Windows Server 2022. However, TLS 1.3 support for SSTP also requires Windows 11 on the client-side. TLS 1.3 is not currently supported in Windows 10.
Summary
Realizing the performance benefits provided by TLS 1.3 will likely only occur in large environments supporting many thousands of concurrent connections per server. However, the security benefits apply to all deployments, regardless of size. Administrators should consider upgrading to Windows Server 2022 before proceeding with Windows 11 adoption.
As a follow-up to my last post regarding Always On VPN error 13801, this post will cover a similar and related error administrators may encounter, the 13806 error. As mentioned previously, certificate configuration is crucial for Always On VPN deployments. I described some specific certificates requirements for IKEv2 in this earlier post. Following this guidance, administrators should have no issues with IKEv2 Always On VPN connections. However, it is always possible to encounter an error if any of these certificates are missing or misconfigured.
Error 13806
Much like the error 13801 described previously, 13806 is also common. When an Always On VPN connection using IKEv2 fails, the Windows Application event log will record an event ID 20227 from the RasClient source. The error message states the following:
“The user [username] dialed a connection named [connection name] which has failed. The error code returned on failure is 13806”.
IKE Failed To Find Valid Machine Certificate
Error 13806 translates to ERROR_IPSEC_IKE_NO_CERT, indicating IKE failed to find a valid machine certificate. The problem can be on the device, the VPN server, or an issue with the VPN server configuration.
Device Certificate
For the device tunnel, the most obvious cause of this error is a missing device authentication certificate on the client itself. Ensure the endpoint has a valid certificate issued by the organization’s internal PKI that includes Client Authentication EKU (OID 1.3.6.1.5.5.7.3.2). The certificate must have a subject name matching the device’s FQDN. It must also be valid (not expired), trusted, and not revoked.
Certificate Chain
A 13806 error will occur if the device certificate installed on the client is not trusted or if the client does not trust the certificate installed on the VPN server. Ensure the client has all the necessary root and intermediate certification authority (CA) certificates installed in their respective certificate stores.
VPN Server Certificate
A 13806 error can also occur if the VPN server does not have a properly configured server certificate. Ensure the VPN server has a valid certificate issued by the organization’s internal PKI that includes both the Server Authentication (OID 1.3.6.1.5.5.7.3.1) and IP security IKE intermediate (OID 1.3.6.1.5.5.8.2.2) EKUs. The subject name must match the public fully qualified domain name (FQDN) used by VPN clients to connect to the VPN server (not the server’s NetBIOS name). Again, ensure the certificate is valid (not expired), trusted, not revoked, and all necessary root and intermediate CA certificates are installed in their respective certificate stores.
RRAS Configuration
Another cause of the 13806 error for the user tunnel is a misconfigured Routing and Remote Access Service (RRAS) VPN server. An error 13806 can happen if the administrator incorrectly defines a trusted root CA using Set-VpnAuthProtocol. Ensure that the root certificate thumbprint matches exactly the root CA server’s thumbprint used to issue certificates to VPN devices and the VPN server.
Get-VpnAuthProtocol
Root CA Certificate Thumbprint
Resolution
Ensure that devices and VPN servers have correctly configured certificates installed. If the root CA certificate is assigned incorrectly on the VPN server, follow the guidelines detailed here to update the configuration.
Certificate configuration is crucial for Always On VPN deployments. I described some specific certificates requirements for IKEv2 in this previous post. Following this guidance, administrators should have no issues with IKEv2 Always On VPN connections. However, it is always possible to encounter an error if any of these certificates are missing or misconfigured.
Error 13801
One of the most common errors related to IKEv2 and certificates is 13801. When an Always On VPN connection using IKEv2 fails, the Windows Application event log will record an event ID 20227 from the RasClient source. The error message states the following:
“The user [username] dialed a connection named [connection name] which has failed. The error code returned on failure is 13801”.
IKE Authentication Credentials are Unacceptable
Error 13801 translates to ERROR_IPSEC_IKE_AUTH_FAIL, indicating an authentication failure related to IPsec. The problem can be on the device, the VPN server, or an issue with the VPN server configuration.
Certificate Chain
A 13801 error will occur if the client does not trust the certificate installed on the VPN server. Ensure the client has all the necessary root and intermediate certification authority (CA) certificates installed in their respective certificate stores.
VPN Server Certificate
A 13801 error can also occur if the VPN server does not have a properly configured server certificate. Ensure the VPN server has a valid certificate issued by the organization’s internal PKI that includes both the Server Authentication (OID 1.3.6.1.5.5.7.3.1) and IP security IKE intermediate (OID 1.3.6.1.5.5.8.2.2) EKUs. The subject name must match the public fully qualified domain name (FQDN) used by VPN clients to connect to the VPN server (not the server’s NetBIOS name). Again, ensure the certificate is valid (not expired), trusted, not revoked, and all necessary root and intermediate CA certificates are installed in their respective certificate stores.
RRAS Configuration
Another cause of the 13801 error for the device tunnel is a misconfigured Routing and Remote Access Service (RRAS) VPN server. An error 13801 can happen if the administrator incorrectly defines a trusted root CA using Set-VpnAuthProtocol. Ensure that the root certificate thumbprint matches exactly the root CA server’s thumbprint used to issue certificates to VPN devices and the VPN server.
Get-VpnAuthProtocol
Root CA Certificate Thumbprint
Resolution
Ensure that devices and VPN servers have correctly configured certificates installed. If the root CA certificate is assigned incorrectly on the VPN server, follow the guidelines detailed here to update the configuration.
Additional Information
Microsoft Windows Always On VPN Error 13806 (Coming soon)
When configuring and deploying Windows Always On VPN using Microsoft Endpoint Manager (MEM)/Intune, administrators may find that some settings are not exposed in the MEM UI. In some cases, deploying the configuration profile using custom XML is the workaround. However, many crucial Always On VPN settings are not exposed using either method. Here, administrators must resort to editing settings in the VPN configuration file on the client after provisioning the VPN profile.
Phonebook
A file called rasphone.pbk stores all Windows VPN settings on the endpoint. It includes name/value pairs that correspond to many settings administrators change manually in the GUI. Other settings can be changed using PowerShell. Depending on the connection type, the file can be found in one of two locations.
User Tunnel: $env:AppData\Microsoft\Network\Connections\Pbk\rasphone.pbk
Documentation for Windows VPN client phonebook entry settings can be found here.
Limitations
Unfortunately, editing the rasphone.pbk file isn’t always convenient. Making the changes is technically easy. Administrators can write a simple PowerShell script to update the text file as required. However, automating this at scale is challenging. Thankfully, Intune Proactive Remediations can help.
Proactive Remediations
With Intune Proactive Remediations, administrators can create and deploy script packages to monitor and optionally update specific configuration settings. The package includes two scripts, a detection script, and a remediation script. The detection script looks at the current value of a particular setting and reports on its compliance. The remediation script is triggered to update the setting if the value is incorrect.
Requirements
Intune Proactive Remediations has some specific licensing requirements. Administrators must also enroll devices into Endpoint analytics and provision a Windows Health Monitoring configuration profile. There are also limitations on the size and type of scripts that administrators can use. More information on prerequisites can be found here.
Script Packages
Administrators can create detection and remediation PowerShell scripts to update settings in rasphone.pbk, or optionally, they can download sample scripts from my GitHub repository here. This repository contains user and device tunnel detection and remediation scripts for many popular settings in rasphone.pbk. Examples include updating the VPN Strategy, changing VPN interface metrics, disabling class-based default routes, and many more.
Note: The scripts in my GitHub repository are examples only. While they can be used in production environments, they are basic and may not work as expected in all scenarios. For example, the scripts as written today assume only a single VPN profile provisioned. Unexpected results may occur if more than one VPN profile exists. Please use them at your own risk.
Deployment
In this example, we’ll deploy a Proactive Remediation to disable IKE mobility for user tunnel VPN connections. To configure an Intune Proactive Remediation, open the Microsoft Endpoint Manager portal (https://endpoint.microsoft.com/) and navigate to Reports > Endpoint analytics > Proactive remediations. After creating or downloading the detection and remediation scripts, perform the following steps to create and deploy a Proactive Remediation script package.
Click Create script package.
Enter a name for the package in the Name field.
Enter a description for the package in the Description field (optional).
Click Next.
Click the blue folder icon next to the Detection script file field and upload the detection script.
Click the blue folder icon next to the Remediation script file field and upload the associated remediation script.
For user tunnel connections, click Yes next to Run this script using the logged-on credentials. For device tunnel connections, click No.
Click Next.
Assign scope tags and group assignments as necessary, then click Create. Click Refresh to update the UI to display the newly created script package.
Caveats
Be advised that Proactive Remediation script packages run immediately after the first device sync and then every 24 hours after that. Timing issues could lead to delays in functionality. For example, if an Always On VPN profile is provisioned after a Proactive Remediation script runs, the changes made by the remediation script won’t be available until much later. Also, changes made while the VPN is active will not take effect until after restarting the connection.
Special Thanks
Special thanks to Tom Klaver at Inspark for turning me on to this feature. It has been an absolute lifesaver for sure!
Deploying user or device authentication certificates to support Always On VPN requires installing the Certificate Connector for Microsoft Intune. The same connector can link Intune to on-premises public key infrastructure (PKI) using PKCS or SCEP certificates. The connector can be configured to run in the SYSTEM context or a domain service account.
Configuration Failure
Administrators may encounter the following error message when installing the certificate connector and selecting the option to use a domain service account.
“Configuration failed. Configuring Microsoft Intune Certificate Connector failed. No changes were made to Feature or Proxy settings. Please try again.”
Root Cause
This error occurs because the service account does not have the correct permissions assigned on the server where the connector is being installed. Specifically, the service account must have the Logon as a service right assigned. To do this, open the local group policy editor (gpedit.msc) and perform the following steps.
Expand Computer Configuration > Windows Settings > Security Settings > Local Policies > User Rights Assignment.
Double-click Log on as a service.
Click Add User or Group.
Add the service account.
Click OK.
Once complete, remove the Certificate Connector for Intune and re-run the installation again.
When configuring Always On VPN for Windows 10 and Windows 11 clients, administrators may encounter a scenario where an IPv4 route defined in Microsoft Endpoint Manager/Intune or custom XML is not reachable over an established Always On VPN connection. Further investigation indicates the route is added to the configuration on the endpoint but does not appear in the routing table when the connection is active.
Routing Configuration
When split tunneling is enabled, administrators must define routes to IP networks that are reachable over the Always On VPN connection. The method of defining these routes depends on the client configuration deployment method.
Endpoint Manager
Using Microsoft Endpoint Manager, administrators define IP routes in the Split Tunneling section of the configuration settings for the Always On VPN device configuration profile. Routes are defined by entering the destination prefix and prefix size. In this example, the 10.0.0.0/8 and 172.21.12.0/21 IPv4 networks are defined for routing over the Always On VPN tunnel.
Custom XML
Using custom XML deployed using Microsoft Endpoint Manager, System Center Configuration Manager (SCCM), or PowerShell, routes are defined in the XML file using the following syntax.
Client Configuration
Validate the routing configuration has been implemented on the endpoint successfully by running the following PowerShell command.
As you can see here, the IPv4 routes 10.0.0.0/8 and 172.21.12.0/21 are included in the client’s Always On VPN configuration, as shown below.
Missing Route
However, after establishing an Always On VPN connection, the 172.21.12.0/21 network is not reachable. To continue troubleshooting, run the following PowerShell command to view the active routing table.
Get-NetRoute -AddressFamily IPv4
As you can see above, the only IPv4 route in the VPN configuration added to the routing table is the 10.0.0.0/8 network. The 172.21.12.0/21 IPv4 route is missing.
Network Prefix Definition
IPv4 routes missing from the Always On VPN client’s routing table result from incorrect network prefix definition. Specifically, the IPv4 route 172.21.12.0/21 used in the example here is not a valid network address. Rather, it is a host address in the 172.21.8.0/21 network, as shown below.
The Get-Subnet PowerShell cmdlet is part of the Subnet PowerShell module. To install this module, run the following PowerShell command.
Install-Module Subnet
Resolution
Using the example above, enabling access to the 172.21.12.0/21 subnet would require defining the IPv4 prefix in the routing configuration as 172.21.8.0/21. The moral of this story is always validate routing prefixes to ensure they are, in fact, network addresses and not host addresses.
Since the introduction of Windows 11, there have been numerous reports of issues with Always On VPN when deployed using Microsoft Endpoint Manager/Intune. Specifically, administrators have been reporting that Always On VPN profiles are being deleted, then later reappearing. Obviously, this is highly disruptive to users in the field.
Causes
According to Microsoft, there are several causes for deleted VPN profiles.
Changes to an Existing Profile
Missing Always On VPN profiles commonly occurs when updating settings for an existing VPN profile applied to Windows 11 endpoints. In this scenario, the VPN profile is deleted but not immediately replaced. Synchronize the device with Microsoft Endpoint Manager/Intune once more to return the VPN profile.
Multiple Profiles
Issues with Always On VPN profiles may also occur if two new VPN profiles are applied to the endpoint simultaneously.
Remove and Replace
Removing and replacing an Always On VPN profile at the same time will also result in connectivity issues.
There is no known workaround for these issues at this time. Microsoft is aware of the problem and is working on a fix, and until then, rolling out Windows 11 with Always On VPN should be avoided.
Additional Issues
There have been reports of other known issues with Windows 11 and Always On VPN. For instance, my PowerShell script that removes an Always On VPN connection doesn’t work with Windows 11. I’m working to resolve that issue as we speak.
Are you experiencing any issues with Always On VPN on Windows 11? Please share them in the comments below!
When configuring the Windows Server Routing and Remote Access Service (RRAS) to support Secure Socket Tunneling Protocol (SSTP) for Always On VPN user tunnel connections, administrators must install a Transport Layer Security (TLS) certificate on the VPN server. The best practice is to use a certificate issued by a public Certification Authority (CA). In addition, administrators should use a TLS certificate using Elliptic Curve Digital Signature Algorithm (ECDSA) for optimal security and performance.
Let’s Encrypt
Obtaining a public TLS certificate is not inherently difficult, nor is it expensive. However, Let’s Encrypt is a nonprofit public CA issues TLS certificates entirely for free. Always On VPN supports Let’s Encrypt TLS certificates, and installing a Let’s Encrypt certificate on the Always On VPN RRAS server is quite simple.
Pros and Cons
Using Let’s Encrypt certificates for Always On VPN has several significant advantages over traditional public CAs.
Cost – Let’s Encrypt certificates are free! No cost whatsoever.
Speed – Enrolling for a Let’s Encrypt certificate takes just a few minutes.
Trusted – Let’s Encrypt certificates are trusted by default in Windows 10 and Windows 11.
Let’s Encrypt is not without some drawbacks, however.
Lifetime – Let’s Encrypt certificates are only valid for 90 days.
Administration – Certificates must be redeployed frequently (every 90 days).
Security – PFX files (which include private keys) are left on disk by default.
It is possible to mitigate some of these drawbacks, though. For example, deleting PFX files after import can improve security. Alternatively, using a Certificate Signing Request (CSR) eliminates PFX files completely.
Also, it is possible to fully automate the Let’s Encrypt certificate enrollment and RRAS configuration process, which eases the administrative burden. And rotating certificates every 90 days could be considered an advantage from a security perspective! Enrolling new certificates (and specifically certificates with unique keys) is advantageous in that respect.
Certificate Enrollment
There are several different ways to enroll for Let’s Encrypt certificates. The preferred method is using PowerShell, as it works on both Windows Server with Desktop Experience (GUI) and Windows Server Core. Using PowerShell, administrators can also fully automate the enrollment and assignment of the certificate in RRAS.
PowerShell Module
To enroll for Let’s Encrypt TLS certificates on the VPN server, install the Posh-ACME PowerShell module. On the RRAS server, open an elevated PowerShell window and run the following command.
Install-Module Posh-ACME
Certificate Request
After installing the Posh-ACME PowerShell module, select a Let’s Encrypt environment by running the following command. Use LE_PROD for the production Let’s Encrypt server or LE_STAGE for the staging environment (used for testing).
Set-PAServer LE_PROD
Next, request a new certificate using the following command.
The administrator is prompted to create a TXT record in public DNS to prove ownership of the domain. Using the example above, create a DNS record called _acme-challenge.vpn in the example.net DNS zone.
Once complete, the TLS certificate is automatically installed in the local computer certificate store on the VPN server and can be assigned in the RRAS management console, as shown here.
Note: R3 is a Let’s Encrypt issuing certification authority.
DNS Plugin
The Posh-ACME PowerShell module supports DNS plugins that allow administrators to automate the creation of the DNS TXT record used to authorize certificate enrollment. DNS plugins for many public DNS providers are available. Some of the more popular DNS providers are listed here.
Microsoft Azure
Amazon Route53
Cloudflare
Akamai
GoDaddy
Infoblox
Windows Server
A list of all supported DNS plugins for Posh-ACME can be found here.
Certificate Binding
Administrators can use the following PowerShell example code to automate the process of binding the new TLS certificate to the SSTP listener in RRAS.
Great news! My new book, Implementing Always On VPN, is now available for pre-order on Amazon.com. This new book, scheduled for release in late 2021, is a comprehensive implementation guide for Windows 10 Always On VPN. Drawing on many years of experience deploying Always On VPN for organizations worldwide, it covers all aspects of an Always On VPN deployment, including planning and design, prerequisite gathering, infrastructure preparation, and client deployment.
In addition, it contains detailed, prescriptive guidance for advanced configuration options such as application and traffic filtering and proxy server configuration. Cloud deployments using Azure VPN gateway and Virtual WAN are covered, and it includes guidance for configuring Azure MFA and Conditional Access.
Also, the book includes thorough guidance for provisioning certificates using Microsoft Endpoint Manager/Intune using both PKCS and SCEP. It outlines options for high availability for VPN and authentication infrastructure and provides details for ongoing system maintenance and operational support.
Finally, the book has an entire chapter dedicated to troubleshooting and resolving common (and not so common!) issues encountered with Windows 10 Always On VPN.
