Always On VPN Updates for Windows 10 2004

Always On VPN Updates for Windows 10 2004Microsoft recently made available an update for Windows 10 2004 that includes many important fixes for outstanding issues with Windows 10 Always On VPN. KB4571744 (build 19041.488) addresses many challenges faced by Always On VPN administrators today, including the following.

TPM

This update addresses an issue that prevents hash signing from working correctly using the Microsoft Platform Crypto Provider for Trusted Platform Module (TPM). This issue can occur when administrators configure Always On VPN to use Protected Extensible Authentication Protocol (PEAP) with client certificate authentication using a FortiGate security device.

Sleep/Hibernate

This update also addresses issues with Windows 10 Always On VPN failing to automatically reconnect when resuming from sleep or hibernate. I’ve written about issues with Always On VPN and sleep/hibernate in the past. This is an issue that has plagued Always On VPN since its introduction, so let’s hope this finally provides some meaningful relief from this persistent problem.

Certificate Authentication

When both the Always On VPN device tunnel and user tunnel are provisioned to a Windows 10 clients, user tunnel connections may be authenticated using the machine certificate and not EAP/PEAP. This can result in connections that are not validated as intended, and allowing a user to bypass configured NPS policies, MFA requirements, or conditional access rules. This update includes a fix for this issue, restoring proper authentication for the user tunnel when the device tunnel is also provisioned.

Device and User Tunnel Coexistence

A bug that first appeared when Windows 10 2004 was introduced prevented a device tunnel and user tunnel Always On VPN connection from being established to the same VPN server if the user tunnel used Internet Key Exchange Version 2 (IKEv2). This update restores full functionality under those conditions.

Update KB4571744

To resolve these issues with Windows 10 Always On VPN as well as others, download and install update KB4571744 today. If you are experiencing any of these issues with releases of Windows 10 prior to 2004, look for updates for those build to come later this year.

Additional Information

September 3, 2020 – KB4571744 (OS Build 19041.488) Preview

Windows 10 Always On VPN Connection Issues after Sleep or Hibernate

Windows 10 Always On VPN Bug in Windows 10 2004

Removing Always On VPN Connections

Removing Always On VPN ConnectionsMuch has been written about provisioning Windows 10 Always On VPN client connections over the past few years. While the preferred method for deploying Always On VPN is Microsoft Intune, using PowerShell is often helpful for initial testing, and required for production deployment with System Center Configuration Manager (SCCM) or Microsoft Endpoint Manager (MEM). That said, there will invariably come a time when an administrator has to remove an Always On VPN connection. It is not as simple as you might think.

PowerShell

There are a variety of ways to remove an existing Always On VPN connection, with the quickest and simplest being PowerShell and the Remove-VpnConnection cmdlet.

Get-VpnConnection -Name ‘Always On VPN’ | Remove-VpnConnection -Force

There are several limitations to this method, however.

Active Connections

Administrators will quickly realize that PowerShell fails to remove a VPN connection that is currently connected. As shown here, attempting to remove an active VPN connection will return the following error message.

“The VPN connection [connection name] cannot be removed from the local user connections. Cannot delete a connection while it is connected.”

Removing Always On VPN Connections

Registry Artifacts

Removing Always On VPN connections using PowerShell commonly leaves behind registry artifacts that can potentially cause problems. For example, there are several Always On VPN-related registry entries in several locations including the HKLM\SOFTWARE\Microsoft\EnterpriseResourceManager\Tracked hive that may not be deleted when removing an Always On VPN connection. When provisioning a new Always On VPN connection after deleting one with the same name previously, the administrator may encounter the following error message.

“Unable to create [connection name] profile: A general error occurred that is not covered by a more specific error code.”

Removing Always On VPN Connections

Note: This error can also be caused by improperly formatted XML configuration files. More details here.

Remove-AovpnConnection Script

Veteran Always On VPN administrators are likely familiar with PowerShell scripts I’ve created called New-AovpnConneciton.ps1 and New-AovpnDeviceConnection.ps1, which are hosted on my GitHub. These scripts are adapted from code samples published by Microsoft to which I have included additional functionality. To address the limitations highlighted in this article I have published a new PowerShell script called Remove-AovpnConnection.ps1. It will remove any Always On VPN connection, even those that are currently active. It also includes logic to remove known registry artifacts common to Always On VPN. Download the script from GitHub and use the following syntax to remove an Always On VPN connection, established or not.

.\Remove-AovpnConnection.ps1 -ProfileName [connection name]

Running this PowerShell command will forcibly remove an Always On VPN connection. Use the -DeviceTunnel switch when removing a device tunnel connection (requires running in the system context). I have also included a -CleanUpOnly switch to remove registry artifacts when the VPN connection was previously removed using another method.

Updated Installation Scripts

I have also updated New-AovpnConnection.ps1 to include these registry clean up steps. This will prevent future errors when provisioning an Always On VPN client where a connection of the same name was removed previously.

Note: New-AovpnConnection.ps1 has also been updated to support device tunnel deployments. As such, I have deprecated New-AovpnDeviceConnection.ps1. Simply use New-AovpnConnection.ps1 with the -DeviceTunnel switch to deploy an Always On VPN device tunnel.

Additional Information

Windows 10 Always On VPN Device Tunnel Configuration using PowerShell

Troubleshooting Always On VPN Unable to Create Profile General Error

 

Always On VPN Connection Issues After Sleep or Hibernate

Always On VPN Connection Issues After Sleep or HibernateLikely the single most common complaint about Windows 10 Always On VPN is that device tunnel or user tunnel VPN connections fail to reconnect automatically after a laptop computer wakes from sleep or hibernate. You will find many complaining about this issue and discussing various attempts at resolution on the Microsoft forums. And while Microsoft has released many fixes the last few years to improve connection reliability for Always On VPN, this one seems to continue to plague them. This issue is also prevalent with DirectAccess deployments.

Fix or Workaround?

Unfortunately, I do not have a specific fix or workaround to share that will magically resolve this ongoing issue. However, there are a few group policy settings that may prove effective in some cases.

Connected Standby Settings

To help address issues with Always On VPN connections failing after sleep or hibernate, open the group policy management console and navigate to Computer Configuration > Administrative Templates > System > Power Management > Sleep Settings and enable the following settings.

  • Allow network connectivity during connected-standby (plugged in)
  • Allow network connectivity during connected-standby (on battery)

Always On VPN Connection Issues After Sleep or Hibernate

Always On VPN Connection Issues After Sleep or Hibernate

Additional Information

Are you experiencing issues with Always On VPN reconnecting automatically after sleep or hibernate? Have you found an effective workaround? Share your experience in the comments below!

Troubleshooting Always On VPN Error 691 and 812 – Part 3

Troubleshooting Always On VPN Error 691 and 812 – Part 2When implementing Windows 10 Always On VPN, administrators may encounter errors 691 or 812 when establishing a VPN connection. There are several different configuration issues that will result in these errors. For example they may occur when TLS 1.0 has been disabled on the RRAS server when installed on servers prior to Windows Server 2016. It can also happen if a user’s Active Directory account is configured to deny dial-in access and the NPS server is not configured to ignore user account dial-in properties. Another scenario that can result in 691/812 errors is when the Active Directory security groups are configured as conditions on the Network Policy Server (NPS) Network Policy. See below for more details.

SSTP and Error 691

When attempting to establish an Always On VPN connection using the Secure Socket Tunneling Protocol (SSTP), administrators may encounter the following error message.

“The remote connection was denied because the user name and password combination you provided is not recognized, or the selected authentication protocol is not permitted on the remote access server.”

Troubleshooting Always On VPN Error 691 and 812 – Part 2

In addition, an error 691 with event ID 20227 from the RasClient source can be found in the Application event log on the client.

“The user <domain\user> dialed a connection named which has failed. The error code returned on failure is 691.”

Troubleshooting Always On VPN Error 691 and 812 – Part 2

IKEv2 and Error 812

When attempting to establish an Always On VPN connection using Internet Key Exchange version 2 (IKEv2), administrators may encounter the following error message.

“The connection as prevented because of a policy configured on your RAS/VPN server. Specifically, the authentication method used by the server to verify your username and password may not match the authentication method configured in your connection profile. Please contact the Administrator of the RAS server and notify them of this error.”

Troubleshooting Always On VPN Error 691 and 812 – Part 2

In addition, an error 812 with event ID 20227 from the RasClient source can be found in the Application event log on the client.

Troubleshooting Always On VPN Error 691 and 812 – Part 2

NPS Event Log

On the NPS server the administrator will find an entry in the application event log with event ID 6273 from the Microsoft Windows security auditing source and the Network Policy Server task category indicating the network policy server denied access to the user. Looking closely at this event log message shows Reason Code 48 and the following reason.

“The connection request did not match any configured network policy.”

Troubleshooting Always On VPN Error 691 and 812 – Part 2Group Membership

As stated earlier, another scenario in which administrators will encounter errors 691 and/or 812 is when the Network Policy on the NPS server is configured incorrectly. Specifically, and administrator may wish to grant access to more than one group but intend for access to be granted to users who are a member of any of them. Conversely, they may wish to require access in all specified groups to gain access to the VPN. Configuring each of these conditions is subtly different, however.

Open the NPS management console on the NPS server and follow the steps below to configure user group conditions correctly for the following scenarios.

Any Group

1. Right-click the Always On VPN network policy and choose Properties.
2. Click on the Conditions tab.
3. Click the Add button.
4. Click User Groups.
5. Click Add.
6. Click Add Groups.
7. Enter the name of the group you want to grant access to.
8. Click Ok.
9. Repeat the steps 6-8 above to specify additional groups.

Troubleshooting Always On VPN Errors 691 and 812

All Groups

1. Right-click the Always On VPN network policy and choose Properties.
2. Click on the Conditions tab.
3. Click the Add button.
4. Click User Groups.
5. Click Add.
6. Click Add Groups.
7. Enter the name of the group you want to grant access to.
8. Click Ok.
9. Repeat steps 3-8 above to specify additional groups (you must go back to the Add button on step 3!).

Troubleshooting Always On VPN Errors 691 and 812

Additional Information

Troubleshooting Always On VPN Error 691 and 812 – Part 1

Troubleshooting Always On VPN Error 691 and 812 – Part 2

Always On VPN Bug in Windows 10 2004

Always On VPN Bug in Windows 10 2004While performing Always On VPN evaluation testing with the latest release of Windows 10 (2004), a bug was discovered that may result in failed VPN connections, but only under certain conditions. Specifically, the failure occurs when both the device tunnel and user tunnel are configured on the same client, and the user tunnel is configured to use IKEv2 exclusively.

Error 829

After upgrading to Windows 10 2004, and when the device tunnel and user tunnel are both deployed and the user tunnel is configured to use IKEv2, the administrator will notice that if the device tunnel connection is established, the user tunnel connects successfully but is then terminated abruptly with error code 829.

Always On VPN Bug in Windows 10 2004

Note: This can happen in reverse if the user tunnel is established before the device tunnel for some reason. In this scenario the user tunnel would be connected but attempts to establish the device tunnel would result in failure.

Error 619

If the user tunnel connection is initiated using rasdial.exe or rasphone.exe, the error code returned is 619.

Always On VPN Bug in Windows 10 2004

Always On VPN Bug in Windows 10 2004

Workaround

The workaround for this issue is to either use a single tunnel, or if both user tunnel and device tunnel are required, configure the user tunnel to use the SSTP VPN protocol instead of IKEv2.

Additional Information

Windows 10 Always On VPN Device Tunnel Only Deployment Considerations

Always On VPN IKEv2 Load Balancing and NAT

Always On VPN IKEv2 Load Balancing and NATOver the last few weeks, I’ve worked with numerous organizations and individuals troubleshooting connectivity and performance issues associated with Windows 10 Always On VPN, and specifically connections using the Internet Key Exchange version 2 (IKEv2) VPN protocol. An issue that appears with some regularity is when Windows 10 clients fail to connect with error 809. In this scenario, the server will accept connections without issue for a period of time and then suddenly stop accepting requests. When this happens, existing connections continue to work without issue in most cases. Frequently this occurs with Windows Server Routing and Remote Access Service (RRAS) servers configured in a clustered array behind an External Load Balancer (ELB).

Network Address Translation

It is not uncommon to use Network Address Translation (NAT) when configuring Always On VPN. In fact, for most deployments the public IP address for the VPN server resides not on the VPN server, but on an edge firewall or load balancer connected directly to the Internet. The firewall/load balancer is then configured to translate the destination address to the private IP address assigned to the VPN server in the perimeter/DMZ or the internal network. This is known a Destination NAT (DNAT). Using this configuration, the client’s original source IP address is left intact. This configuration presents no issues for Always On VPN.

Source Address Translation

When troubleshooting these issues, the common denominator seems to be the use of Full NAT, which includes translating the source address in addition to the destination. This results in VPN client requests arriving at the VPN server as appearing not to come from the client’s original IP address, but the IP address of the network device (firewall or load balancer) that is translating the request. Full NAT may be explicitly configured by an administrator, or in the case of many load balancers, configured implicitly because the load balancer is effectively proxying the connection.

Known Issues

IKEv2 VPN connections use IPsec for encryption, and by default, Windows limits the number of IPsec Security Associations (SAs) coming from a single IP address. When a NAT device is performing destination/full NAT, the VPN server sees all inbound IKEv2 VPN requests as coming from the same IP address. When this happens, clients connecting using IKEv2 may fail to connect, most commonly when the server is under moderate to heavy load.

Resolution

The way to resolve this issue is to ensure that any load balancers or NAT devices are not translating the source address but are performing destination NAT only. The following is configuration guidance for F5, Citrix ADC (formerly NetScaler), and Kemp load balancers.

F5

On the F5 BIG-IP load balancer, navigate to the Properties > Configuration page of the IKEv2 UDP 500 virtual server and choose None from the Source Address Translation drop-down list. Repeat this step for the IKEv2 UDP 4500 virtual server.

Always On VPN IKEv2 Load Balancing and NAT

Citrix ADC

On the Citrix ADC load balancer, navigate to System > Settings > Configure Modes and check the option to Use Subnet IP.

Always On VPN IKEv2 Load Balancing and NAT

Next, navigate to Traffic Management > Load Balancing > Service Groups and select the IKEv2 UDP 500 service group. In the Settings section click edit and select Use Client IP. Repeat these steps for the IKEv2 UDP 4500 service group.

Always On VPN IKEv2 Load Balancing and NAT

Kemp

On the Kemp LoadMaster load balancer, navigate to Virtual Services > View/Modify Services and click Modify on the IKEv2 UDP 500 virtual service. Expand Standard Options and select Transparency. Repeat this step for the IKEv2 UDP 4500 virtual service.

Always On VPN IKEv2 Load Balancing and NAT

Caveat

Making the changes above may introduce routing issues in your environment. When configuring these settings, it may be necessary to configure the VPN server’s default gateway to use the load balancer to ensure proper routing. If this is not possible, consider implementing the workaround below.

Workaround

To fully resolve this issue the above changes should be made to ensure the VPN server can see the client’s original source IP address. If that’s not possible for any reason, the following registry key can be configured to increase the number of established SAs from a single IP address. Be advised this is only a partial workaround and may not fully eliminate failed IKEv2 connections. There are other settings in Windows that can prevent multiple connections from a single IP address which are not adjustable at this time.

To implement this registry change, open an elevated PowerShell command window on the RRAS server and run the following commands. Repeat these commands on all RRAS servers in the organization.

New-ItemProperty -Path ‘HKLM:SYSTEM\CurrentControlSet\Services\IKEEXT\Parameters\’ -Name IkeNumEstablishedForInitialQuery -PropertyType DWORD -Value 50000 -Force

Restart-Service IKEEXT -Force -PassThru

Additional Information

IPsec Traffic May Be Blocked When A Computer is Behind a Load Balancer

Windows 10 Always On VPN IKEv2 Load Balancing with Citrix NetScaler ADC

Windows 10 Always On VPN IKEv2 Load Balancing with F5 BIG-IP

Windows 10 Always On VPN IKEv2 Load Balancing with Kemp LoadMaster

Always On VPN Force Tunneling with Office 365 Exclusions

Always On VPN Force Tunneling with Office 365 ExclusionsWith the COVID-19 global pandemic forcing nearly everyone to work from home these days, organizations that implemented force tunneling for their VPN clients are likely encountering unexpected problems. When force tunneling is enabled, all client traffic, including Internet traffic, is routed over the VPN tunnel. This often overloads the VPN infrastructure and causes serious slowdowns, which degrades the user experience and negatively impacts productivity. This is especially challenging because so many productivity applications like Microsoft Office 365 are optimized for Internet accessibility. It is one of the main reasons that force tunneling is not generally recommended.

Force Tunneling with Exceptions

When enabling split tunneling is not an option, administrators frequently ask about enabling force tunneling with some exceptions. The most common configuration is enabling force tunneling while still allowing Office 365 traffic to go outside of the tunnel. While this is something that third-party solutions do easily, it has been a challenge for Always On VPN. Specifically, Always On VPN has no way to route traffic by hostname or Fully-Qualified Domain Name (FQDN).

Exclusion Routes

To address this challenge, the administrator can configure Exclusion Routes. Exclusion Routes are supported in Windows 10 1803 with update KB4493437, Windows 10 1809 with update KB4490481, and Windows 10 1903/1909.

Exclusion routes are defined in the client routing table that are excluded from the VPN tunnel. The real challenge here is determining all the required IP addresses required for Office 365.

Microsoft Published Guidance

Given current events and the heavy demands placed on enterprises supporting exclusively remote workforces, Microsoft has recently published guidance for configuring Always On VPN force tunneling while excluding Office 365 traffic. Their documentation includes all the required IP addresses to configure exclusions for. This will make it much simpler for administrators to configure Always On VPN to support this unique scenario. The following links provide detailed configuration guidance for enabling force tunneling for Always On VPN with exceptions.

Additional Information

Windows 10 Always On VPN Split vs. Force Tunneling

Windows 10 Always On VPN Routing Configuration

Windows 10 Always On VPN Lockdown Mode

Remote Access Questions and Answers Webinar Hosted by Kemp

Join me this Thursday, April 9 at 10:00AM EDT for a Remote Access Q&A session hosted by Kemp Technologies. During this free live webinar, I’ll be answering all your questions as they relate to enterprise mobility, remote access, scalability and performance, security, and much more. Topics are not limited to Kemp products at all, so feel free to join and ask me anything you like! Register now and submit your questions!

Remote Access Q&A Webinar Hosted by Kemp

Always On VPN Device Tunnel Only Deployment Considerations

Always On VPN Device Tunnel Only Deployment ConsiderationsRecently I wrote about Windows 10 Always On VPN device tunnel operation and best practices, explaining its common uses cases and requirements, as well as sharing some detailed information about authentication, deployment recommendations, and best practices. I’m commonly asked if deploying Always On VPN using the device tunnel exclusively, as opposed to using it to supplement the user tunnel, is supported or recommended. I’ll address those topics in detail here.

Device Tunnel Only?

To start, yes, it is possible to deploy Windows 10 Always On VPN using only the device tunnel. In this scenario the administrator will configure full access to the network instead of limited access to domain infrastructure services and management servers.

Is It Recommended?

Generally, no. Remember, the device tunnel was designed with a specific purpose in mind, that being to provide pre-logon network connectivity to support scenarios such as logging on without cached credentials. Typically, the device tunnel is best used for its intended purpose, which is providing supplemental functionality to the user tunnel.

Deployment Considerations

The choice to implement Always On VPN using only the device tunnel is an interesting one. There are some potential advantages to this deployment model, but it is not without some serious limitations. Below I’ve listed some of the advantages and disadvantages to deploying the device tunnel alone for Windows 10 Always On VPN.

Advantages

Using the device tunnel alone does have some compelling advantages over the standard two tunnel (device tunnel/user tunnel) deployment model. Consider the following.

  • Single VPN Connection – Deploying the device tunnel alone means a single VPN connection to configure, deploy, and manage on the client. This also results in less concurrent connections and, importantly, less IP addresses to allocate and provision.
  • Reduced Infrastructure – The device tunnel is authenticated using only the device certificate. This certificate check is performed directly on the Windows Server Routing and Remote Access Service (RRAS) VPN server, eliminating the requirement to deploy Network Policy Server (NPS) servers for authentication.
  • User Transparency – The device tunnel does not appear in the modern Windows UI. The user will not see this connection if they click on the network icon in the notification area. In addition, they will not see the device tunnel connection in the settings app under Network & Internet > VPN. This prevents casual users from playing with the connection settings, and potentially deleting the connection entirely. It’s not that they can’t delete the device tunnel however, it’s just not as obvious.
  • Simplified Deployment – Deploying the device tunnel is less complicated than deploying the user tunnel. The device tunnel is provisioned once to the device and available to all users. This eliminates the complexity of having to deploy the user tunnel in each individual user’s profile.

Disadvantages

While there are some advantages to using the device tunnel by itself, this configuration is not without some serious limitations. Consider the following.

  • IKEv2 Only – The device tunnel uses the IKEv2 VPN protocol exclusively. It does not support SSTP. While IKEv2 is an excellent protocol in terms of security, it is commonly blocked by firewalls. This will prevent some users from accessing the network remotely depending on their location.
  • Limited OS Support – The device tunnel is only supported on Windows 10 Enterprise edition clients, and those clients must be joined to a domain. Arguably the device tunnel wouldn’t be necessary if the client isn’t domain joined, but some organizations have widely deployed Windows 10 Professional, which would then preclude them from being able to use the device tunnel.
  • Machine Certificate Authentication Only – The device tunnel is authenticated using only the certificate issued to the device. This means anyone who logs on to the device will have full access to the internal network. This may or may not be desirable, depending on individual requirements.
  • No Mutual Authentication – When the device tunnel is authenticated, the server performs authentication of the client, but the client does not authenticate the server. The lack of mutual authentication increases the risk of a man-in-the-middle attack.
  • CRL Checks Not Enforced – By default, RRAS does not perform certificate revocation checking for device tunnel connections. This means simply revoking a certificate won’t prevent the device from connecting. You’ll have to import the client’s device certificate into the Untrusted Certificates certificate store on each VPN server. Fortunately, there is a fix available to address this limitation, but it involves some additional configuration. See Always On VPN Device Tunnel and Certificate Revocation for more details.
  • No Support for Azure Conditional Access – Azure Conditional Access requires EAP authentication. However, the device tunnel does not use EAP but instead uses a simple device certificate check to authenticate the device.
  • No Support for Multifactor Authentication – As the device tunnel is authenticated by the RRAS VPN server directly and authentication requests are not sent to the NPS server, it is not possible to integrate MFA with the device tunnel.
  • Limited Connection Visibility – Since the device tunnel is designed for the device and not the user it does not appear in the list of active network connections in the Windows UI. There is no user-friendly connection status indicator, although the connection can be viewed using the classic network control panel applet (ncpa.cpl).

Summary

The choice to deploy Windows 10 Always On VPN using the device tunnel alone, or in conjunction with the user tunnel, is a design choice that administrators must make based on their individual requirements. Using the device tunnel alone is supported and works but has some serious drawbacks and limitations. The best experience will be found using the device tunnel as it was intended, as an optional component to provide pre-logon connectivity for an existing Always On VPN user tunnel.

Additional Information

Windows 10 Always On VPN Device Tunnel with Azure VPN Gateway

Windows 10 Always On VPN Device Tunnel and Certificate Revocation

Windows 10 Always On VPN Device Tunnel Configuration with Microsoft Intune

Windows 10 Always On VPN Device Tunnel Does Not Connect Automatically

Windows 10 Always On VPN Device Tunnel Missing in Windows 10 UI

Deleting a Windows 10 Always On VPN Device Tunnel

Windows 10 Always On VPN Device Tunnel Configuration using PowerShell

Windows 10 Always On VPN IKEv2 Features and Limitations

Always On VPN Client Connections Fail with Status Connecting

Administrators who have deployed Windows 10 Always On VPN may encounter a scenario in which an Always On VPN connection fails, yet the connectivity status indicator perpetually reports a “Connecting” status.

Always On VPN Client Connections Fail with Status Connecting

Affected Clients

This is a known issue for which Microsoft has recently released updates to address. Affected clients include Windows 10 1909, 1903, and 1809.

Updates Available

The following Windows updates include a fix to resolve this problem.

KB4541335 – Windows 10 1909 and 1903

KB4541331 – Windows 10 1809

Additional Information

Always On VPN Hands-On Training

%d bloggers like this: