Always On VPN Client Connections Fail with Status Connecting

Administrators who have deployed Windows 10 Always On VPN may encounter a scenario in which an Always On VPN connection fails, yet the connectivity status indicator perpetually reports a “Connecting” status.

Always On VPN Client Connections Fail with Status Connecting

Affected Clients

This is a known issue for which Microsoft has recently released updates to address. Affected clients include Windows 10 1909, 1903, and 1809.

Updates Available

The following Windows updates include a fix to resolve this problem.

KB4541335 – Windows 10 1909 and 1903

KB4541331 – Windows 10 1809

Additional Information

Always On VPN Hands-On Training

Always On VPN Trusted Network Detection

Always On VPN Trusted Network DetectionWhen deploying Windows 10 Always On VPN, administrators can configure Trusted Network Detection (TND) which enables clients to detect when they are on the internal network. With this option set, the client will only automatically establish a VPN connection when it is outside the trusted network. Trusted network detection can be configured on both device tunnel and user tunnel connections.

TND Operation

When trusted network detection is configured, the VPN client will evaluate the DNS suffix assigned to all physical (non-virtual or tunnel) adapters that are active. If any of them match the administrator-defined trusted network setting, the client is determined to be on the internal network and the VPN connection will not connect. If the DNS suffix is not present on any of these adapters, the client is determined to be outside the internal network and the VPN connection will establish automatically.

TND Configuration

Trusted network detection is defined in the Intune UI or in ProfileXML as a string that matches the DNS suffix assigned to clients on the internal network. In this example, the DNS suffix on the internal network is lab.richardhicks.net.

Always On VPN Trusted Network Detection

Note: Your organization might have more than one DNS suffix. Ensure that the trusted network detection configuration includes all DNS suffixes in use in the environment to ensure reliable operation.

Intune

Follow the steps below to configured trusted network detection in Microsoft Intune.

  1. Open the Intune management portal (https://devicemanagement.microsoft.com/).
  2. Navigate to Devices > Configuration Profiles > [Profile Name] > Properties > Settings.
  3. Click on Trusted Network Detection.
  4. Enter the DNS suffix(es) used on the internal network.

Always On VPN Trusted Network Detection

ProfileXML

To define Trusted Network Detection in ProfileXML, add the TrustedNetworkDetection element as follows.

Always On VPN Trusted Network Detection

Caveats

In some instances, an Always On VPN client connection may persist, even if the client is connected to the internal network. A common scenario is when a client device connects to a Wi-Fi network that is not connected to the corporate network (for example guest Wi-Fi), then connects to the internal network with Ethernet via a docking station. If the Wi-Fi connection is still available, the Always On VPN connection will persist, even though the machine is connected to the internal network. This is expected and by design.

Workaround

To address this specific scenario, administrators can implement changes via group policy to the way Windows handles multiple connections to the same network. For example, beginning with Windows 10 1709, group policy can be configured to ensure that Windows 10 clients prefer wired Ethernet network connections over Wi-Fi, and to ensure that Wi-Fi connections disconnect when an Ethernet connection is detected.

GPO Configuration

Open the Group Policy management console (gpmc.msc) and perform the following steps to create the required group policy objects.

  1. Create a new Group Policy Object (GPO).
  2. Right-click the new GPO and choose Edit.
  3. Expand Computer Configuration > Administrative Templates > Network > Windows Connection Manager.
  4. Double-click the policy Minimize the number of simultaneous connections to the Internet or a Windows Domain.
  5. Select Enabled.
  6. From the Minimize Policy Options drop-down list choose 1 = Minimize simultaneous connections. Optionally you can choose to disable Wi-Fi whenever connected to Ethernet by choosing 3 = Prevent Wi-Fi when on Ethernet.
  7. Click Ok.Always On VPN Trusted Network Detection
  8. Double-click the policy Enable Windows to soft-disconnect a computer from a network.
  9. Select Disabled.
  10. Click Ok.Always On VPN Trusted Network Detection

Additional Information

Understanding and Configuring Windows Connection Manager

Always On VPN IKEv2 Load Balancing with Citrix NetScaler ADC

Always On VPN SSTP Load Balancing with Citrix NetScaler ADCThe Internet Key Exchange version 2 (IKEv2) VPN protocol is the protocol of choice when the highest level of security is required for Always On VPN connections. It uses IPsec and features configurable security parameters that allow administrators to adjust policies to meet their specific security requirements. IKEv2 is not without some important limitations, but organizations may insist on the use of IKEv2 to provide the greatest protection possible for remote connected clients. Due to complexities of the IKEv2 transport, special configuration on the Citrix ADC is required when load balancing this workload.

Special Note: In December 2019 a serious security vulnerability was discovered on the Citrix ADC that gives an unauthenticated attacker the ability to arbitrarily execute code on the appliance. As of this writing a fix is not available (due end of January 2020) but a temporary workaround can be found here.

Load Balancing IKEv2

When an Always On VPN client establishes a connection using IKEv2, communication begins on UDP port 500, but switches to UDP port 4500 if Network Address Translation (NAT) is detected in the communication path between the client and the server. Because UDP is connectionless, custom configuration is required to ensure that VPN clients maintain connectivity to the same backend VPN server during this transition.

Initial Configuration

Load balancing IKEv2 using the Citrix ADC is similar to other workloads. Below are specific settings and parameters required to load balance IKEv2 using the Citrix ADC.

Note: This article is not a comprehensive configuration guide for the Citrix ADC. It assumes the administrator is familiar with basic load balancing concepts and has experience configuring the Citrix ADC.

Service Settings

The load balancing services for IKEv2 VPN will use UDP ports 500 and 4500. Create the service group and assign group members for UDP 500 as follows.

Always On VPN IKEv2 Load Balancing with Citrix NetScaler ADC

Always On VPN IKEv2 Load Balancing with Citrix NetScaler ADC

Repeat the steps above to create the service group for UDP port 4500.

Virtual Server Settings

Two virtual servers are required, one for UDP port 500 and one for UDP port 4500. Ensure that the service group using UDP port 500 is bound to the virtual server using the same port.

Always On VPN IKEv2 Load Balancing with Citrix NetScaler ADC

Always On VPN IKEv2 Load Balancing with Citrix NetScaler ADC

Repeat the steps above to create the virtual service for UDP port 4500.

Service Monitoring

Since IKEv2 uses the UDP protocol, the only option for service monitoring is to use PING, which is configured by default. Ensure that the firewall on the VPN server allows inbound ICMPv4 and ICMPv6 Echo Request. The default PING monitor on the Citrix ADC will ping the resource every 5 seconds. If a different interval is required, the administrator can edit the PING monitor and bind that to the service or service group as necessary.

Persistency Group

A Persistency Group on the Citrix ADC will be configured to ensure that IKEv2 VPN client requests from the same client are always routed to the same backend server. Follow the steps below to create a Persistency Group and assign it to both IKEv2 virtual servers created previously.

1. In the Citrix ADC management console expand Traffic Management > Load Balancing > Persistency Groups.
2. Click Add.
3. Enter a descriptive name for the Persistency Group.
4. Select SOURCEIP from the Persistence drop-down list.
5. Next to the Virtual Server Name section click the Add button.
6. Add both previously configured IKEv2 virtual servers for UDP 500 and 4500.
7. Click Create.

Always On VPN IKEv2 Load Balancing with Citrix NetScaler ADC

Additional Information

Windows 10 Always On VPN SSTP Load Balancing with Citrix NetScaler ADC

Windows 10 Always On VPN IKEv2 Features and Limitations

Windows 10 AlWAYS On VPN and IKEv2 Fragmentation

Windows 10 Always On VPN IKEv2 Security Configuration

Windows 10 Always On VPN Certificate Requirements for IKEv2

Always On VPN Device Tunnel with Azure VPN Gateway

Always On VPN Device Tunnel with Azure VPN GatewayAlways On VPN is infrastructure independent, which allows for many different deployment scenarios including on-premises and cloud-based. In Microsoft Azure, the Azure VPN gateway can be configured to support Windows 10 Always On VPN client connections in some scenarios. Recently I wrote about using the Azure VPN gateway for Always On VPN user tunnels. In this post I’ll describe how to configure the Azure VPN gateway to support an Always On VPN device tunnel.

Limitations

There are a few crucial limitations that come with using the Azure VPN gateway for Always On VPN. Importantly, the Azure VPN gateway can support either user tunnels or device tunnels, not both at the same time. In addition, Azure supports only a single VPN gateway per VNet, so deploying an additional VPN gateway in the same VNet to support Always On VPN user tunnels is not an option.

Root CA Certificate

The Always On VPN device tunnel is authenticated using a machine certificate issued to domain-joined Windows 10 Enterprise edition clients by the organization’s internal Certification Authority (CA). The CA’s root certificate must be uploaded to Azure for the VPN gateway to authorize device tunnel connections. The root CA certificate can be exported using the Certification Authority management console (certsrv.msc) or via the command line.

Export Certificate – GUI

Follow the steps below to export a root CA certificate using the Certification Authority management console.

1. On the root CA server, open the Certification Authority management console.
2. Right-click the CA and choose Properties.
3. Select the CA server’s certificate and choose View Certificate.
4. Select the Details tab and click Copy to File.
5. Click Next.
6. Choose Base-64 encoded X.509 (.CER).

Always On VPN Device Tunnel with Azure VPN Gateway

7. Click Next.
8. Enter a location to save the file to.
9. Click Next, Finish, and Ok.

Export Certificate – Command Line

Follow the steps below to export a root CA certificate using the command line.

1. On the root CA server, open an elevated command window (not a PowerShell window).
2. Enter certutil.exe -ca.cert root_certificate.cer.
3. Enter certutil.exe -encode root.cer root_certificate_base64.cer.

Copy Public Key

1. Open the saved root certificate file using Notepad.
2. Copy the file contents between the BEGIN CERTIFICATE and END CERTIFICATE tags, as shown here. Use caution and don’t copy the carriage return at the end of the string.

Always On VPN Device Tunnel with Azure VPN Gateway

Point-to-Site Configuration

The Azure VPN gateway must be deployed as a Route-Based gateway to support point-to-site VPN connections. Detailed requirements for the gateway can be found here. Once the VPN gateway has been provisioned, follow the steps below to enable point-to-site configuration for Always On VPN device tunnels.

1. In the navigation pane of the Azure VPN gateway settings click Point-to-site configuration.
2. Click the Configure now link and specify an IPv4 address pool to be assigned to VPN clients. This IP address pool must be unique in the organization and must not overlap with an IP address ranges defined in the Azure virtual network.
3. From the Tunnel type drop-down list select IKEv2.
4. In the Root certificates section enter a descriptive name for the certificate in the Name field.
5. Copy and paste the Base64 encoded public key copied previously into the Public certificate data field.
6. Click Save to save the configuration.

Always On VPN Device Tunnel with Azure VPN Gateway

VPN Client Configuration

To support the Always On VPN device tunnel, the client must have a certificate issued by the internal CA with the Client Authentication Enhanced Key Usage (EKU). Detailed guidance for deploying a Windows 10 Always On VPN device tunnel can be found here.

Download VPN Configuration

1. Click Point-to-site configuration.
2. Click Download VPN client.
3. Click Save.
4. Open the downloaded zip file and extract the VpnSettings.xml file from the Generic folder.
5. Copy the FQDN in the VpnServer element in VpnSettings.xml. This is the FQDN that will be used in the template VPN connection and later in ProfileXML.

Create a Test VPN Connection

It is recommended to create a test VPN connection to perform validation testing of the Azure VPN gateway before provisioning an Always On VPN device tunnel broadly. On a domain-joined Windows 10 enterprise client, create a new VPN connection using IKEv2 with machine certificate authentication. Use the VPN server FQDN copied from the VpnSettings.xml file previously.

Always On VPN Device Tunnel with Azure VPN Gateway

Create an Always On VPN Connection

Once the VPN has been validated using the test profile created previously, an Always On VPN profile can be created and deployed using Intune, SCCM, or PowerShell. The following articles can be used for reference.

Deploy Always On VPN device tunnel using PowerShell

Deploy Always On VPN device tunnel using Intune

IKEv2 Security Configuration

The default IKEv2 security parameters used by the Azure VPN gateway are better than Windows Server, but the administrator will notice that a weak Diffie-Hellman (DH) key (Group 2 – 1024 bit) is used during IPsec phase 1 negotiation.

Always On VPN Device Tunnel with Azure VPN Gateway

Use the following PowerShell commands to update the default IKEv2 security parameters to recommended baseline defaults, including 2048-bit keys (DH group 14) and AES-128 for improved performance.

Connect-AzAccount
Select-AzSubscription -SubscriptionName [Azure Subscription Name]

$Gateway = [Gateway Name]
$ResourceGroup = [Resource Group Name]

$IPsecPolicy = New-AzVpnClientIpsecParameter -IpsecEncryption AES128 -IpsecIntegrity SHA256 -SALifeTime 28800 -SADataSize 102400000 -IkeEncryption AES128 -IkeIntegrity SHA256 -DhGroup DHGroup14 -PfsGroup PFS14

Set-AzVpnClientIpsecParameter -VirtualNetworkGatewayName $Gateway -ResourceGroupName $ResourceGroup -VpnClientIPsecParameter $IPsecPolicy

Note: Be sure to update the cryptography settings on the test VPN connection and in ProfileXML for Always On VPN connections to match the new VPN gateway settings. Failing to do so will result in an IPsec policy mismatch error.

Additional Information

Windows 10 Always On VPN User Tunnel with Azure VPN Gateway

Windows 10 Always On VPN IKEv2 Security Configuration

Windows 10 Always On VPN Device Tunnel Configuration using Microsoft Intune

Windows 10 Always On VPN Device Tunnel Configuration using PowerShell

Windows 10 Always On VPN Options for Azure Deployments

Windows 10 Always On VPN IKEv2 Features and Limitations

Always On VPN and RRAS in Azure

Always On VPN and RRAS in AzureWhen deploying Windows 10 Always On VPN, it may be desirable to host the VPN server in Microsoft’s Azure public cloud. Recently I wrote about Always On VPN deployment options in Azure, and in that post I indicated that deploying Windows Server and the Routing and Remote Access Service (RRAS) was one of those options. Although not formally supported by Microsoft, RRAS is often deployed in Azure because it is cost-effective, easy to manage, and provides flexible scalability.

Supportability

It’s important to state once again that although it is possible to successfully deploy Windows Server with RRAS in Azure to support Always On VPN, as of this writing it is not a formally supported workload. If the administrator makes the decision to deploy RRAS in Azure, they must also accept that Microsoft may refuse to assist with troubleshooting in this specific deployment scenario.

Always On VPN and RRAS in Azure

Reference: https://support.microsoft.com/en-us/help/2721672/microsoft-server-software-support-for-microsoft-azure-virtual-machines

Azure Prerequisites

The configuration of RRAS is identical to on-premises, with a few additional steps required by Azure infrastructure.

Windows Server

RRAS can be configured on any Windows Server virtual machine supported in Microsoft Azure. As with on-premises deployments, Server GUI and Core are supported. Domain-join is optional. The server can be deployed with one network interface or two.

Public IP

A public IP address must be assigned to the VPN server’s external network interface, or the internal interface if the VPN server is configured with a single network adapter. The IP address can be static or dynamic. When using a dynamic IP address, configure a CNAME record in DNS that points to the name configured for the IP address in Azure. If using a static IP address, an A host record can be configured pointing directly to the IP address.

Network Security Group

A Network Security Group (NSG) must be configured and assigned to the VPN server’s external or public-facing network interface that allows the following protocols and ports inbound.

  • TCP port 443 (SSTP)
  • UDP port 500 (IKEv2)
  • UDP port 4500 (IKEv2 NAT traversal)

RRAS in Azure

Below are the infrastructure requirements for supporting Windows Server RRAS VPN in Azure.

Client IP Subnet

Static IP address pool assignment must be used with RRAS. Using DHCP for VPN client IP address assignment in Azure is not supported and will not work. The IP subnet assigned to VPN clients by RRAS must be unique and not overlap with any existing Azure VNet subnets. If more than one VPN server is deployed, each server should be configured to assign a unique subnet for its clients.

IP Forwarding

IP forwarding must be enabled on the VPN server’s internal network interface. Follow the steps below to enable IP forwarding.

1. In the Azure portal, open the properties page for the internal network interface for the VPN server.
2. Click IP configurations in the navigation pane.
3. Click Enabled next to IP forwarding.
4. Click Save.

Always On VPN and RRAS in Azure

Routing

Azure must be configured to route IP traffic from VPN clients back to the VPN server. Follow the steps below to create and assign a routing table in Azure.

1. Click Create Resource.
2. Enter “Route Table” in the search field and press Enter.
3. Click Route Table.
4. Click Create.
5. Enter a descriptive name for the route table in the Name field.
6. Choose an appropriate subscription from the Subscription drop-down list.
7. Select the resource group where the VPN server(s) reside.
8. Select the best location to deploy the route table resource from the Location drop-down list.
9. If the administrator wants to have the VPN client IP subnet route information published automatically, select Enabled for Virtual network gateway route propagation.
10. Click Create.

Always On VPN and RRAS in Azure

Once complete, follow the steps below to define the route for VPN clients.

1. Open the properties page for the route table.
2. Click Routes in the navigation pane.
3. Click Add.
4. Enter a descriptive name in the Route name filed.
5. Enter the IP subnet assigned to VPN clients in the Address prefix field.
6. Select Virtual appliance from the Next hop type drop-down list.
7. Enter the IPv4 address assigned to the VPN server’s internal network interface in the Next hop address field.
8. Click Ok.
9. Repeat the steps above for each VPN server configured in Azure.

Always On VPN and RRAS in Azure

Finally, follow the steps below to assign the route table to an Azure VNet subnet.

1. Open the properties page for the route table.
2. Click Subnets in the navigation pane.
3. Click Associate.
4. Click Virtual network.
5. Choose the appropriate Azure VNet.
6. Click Subnet.
7. Choose an Azure VNet subnet to assign the route table to.
8. Click Ok.
9. Repeat the steps above to assign the route table to any Azure VNet subnet that must be accessible by VPN clients. If VPN clients need access to on-premises resources via Azure site-to-site gateway, assign the route table to the Azure VPN gateway subnet.

Always On VPN and RRAS in Azure

Note: Azure only supports the assignment of one route table per subnet. If a route table is currently assigned, the VPN client subnet route can be added to an existing route table, if necessary.

Summary

Administrators have many choices when it comes to support Always On VPN connections hosted in Azure. RRAS on Windows Server can be an effective solution, assuming you can live without formal support. If having a formally supported solution is a hard requirement, consider deploying Always On VPN using the native Azure VPN gateway or another third-part Network Virtual Appliance (NVA).

Additional Information

Windows 10 Always On VPN with Azure Gateway

Windows 10 Always On VPN Options for Azure Deployments

Windows 10 Always On VPN Multisite with Azure Traffic Manager

Always On VPN IKEv2 Policy Mismatch Error

Always On VPN IKEv2 Policy Mismatch ErrorThe Internet Key Exchange version 2 (IKEv2) VPN protocol is the protocol of choice for Windows 10 Always On VPN deployments where the highest levels of security and assurance are required. However, as I’ve written about in the past, often the default IKEv2 security settings are less than desirable. Before using IKEv2 VPN in a production environment the administrator will need to update these security settings accordingly.

Connection Failure

When configuring Windows Server Routing and Remote Access Service (RRAS) or a third-party VPN appliance to support IKEv2 using custom security policies, the administrator may encounter a scenario in which a connection cannot be established due to a policy mismatch error. When the connection attempt fails, an error will be recorded in the Windows Application event log from the RasClient source with Event ID 20227. The error message states the following:

“The user [username] dialed a connection named [connection name] which has failed. The error code returned on failure is 13868.”

Always On VPN IKEv2 Policy Mismatch Error

Error Code 13868

Error code 13868 translates to ERROR_IPSEC_IKE_POLICY_MATCH. Essentially this error indicates that the IKEv2 security policy on the client did not match the configuration on the server.

Server Configuration

To view the current IKEv2 IPsec policy configuration, open an elevated PowerShell command window and run the following command.

Get-VpnServerIPsecConfiguration

Always On VPN IKEv2 Policy Mismatch Error

Client Configuration

To ensure interoperability, the VPN client must be configured to use the same IKEv2 security policy as defined on the sever. To view a VPN client’s currently configured IKEv2 security policy, open an elevated PowerShell command window and run the following command.

Get-VpnConnection -Name [connection name] | Select-Object -ExpandProperty IPsecCustomPolicy

Always On VPN IKEv2 Policy Mismatch Error

Note: If this PowerShell command returns no output, the VPN connection is not using a custom IKEv2 IPsec security policy.

Updating Settings

Guidance for configuring IKEv2 security policies on Windows Server RRAS and Windows 10 can be found here.

Summary

IKEv2 policy mismatch errors can be resolved easily by ensuring both the VPN server and client are configured to use the same IPsec security policies. Use the PowerShell commands in the above referenced above to validate settings and make changes when necessary.

Additional Information

Windows 10 Always On VPN IKEv2 Security Configuration

Windows 10 Always On VPN IKEv2 Features and Limitations

Show-VpnConnectionIPsecConfiguration PowerShell script on Github

Set-IKEv2SecurityBaseline PowerShell script on Github

Troubleshooting Always On VPN Errors 691 and 812

Troubleshooting Always On VPN Errors 691 and 812When configuring Windows 10 Always On VPN using the Routing and Remote Access Service (RRAS) on Windows Server 2012 R2 and Extensible Authentication Protocol (EAP) authentication using client certificates, clients attempting to establish a VPN connection using Internet Key Exchange version 2 (IKEv2) may receive the following error.

“The connection was prevented because of a policy configured on your RAS/VPN server. Specifically, the authentication method used by the server to verify your username and password may not match the authentication method configured in your connection profile.”

Troubleshooting Always On VPN Errors 691 and 812

The event log on the client also records RasClient event ID 20227 stating “the error code returned on failure is 812”.

Troubleshooting Always On VPN Errors 691 and 812

Always On VPN clients using the Secure Socket Tunneling Protocol (SSTP) may receive the following error.

“The remote connection was denied because the user name and password combination you provided is not recognized, or the selected authentication protocol is not permitted on the remote access server.”

Troubleshooting Always On VPN Errors 691 and 812

The event log on the client also records RasClient event ID 20227 stating “the error code returned on failure is 691”.

Troubleshooting Always On VPN Errors 691 and 812

Resolution

These errors can occur when Transport Layer Security (TLS) 1.0 has been disabled on the RRAS server. To restore functionality, enable TLS 1.0 protocol support on the RRAS server. If disabling TLS 1.0 is required for compliance reasons, consider deploying RRAS on Windows Server 2016. TLS 1.0 can be safely disabled on Windows Server 2016 without breaking EAP client certificate authentication for Windows 10 Always On VPN clients.

Additional Information

Windows 10 Always On VPN Hands-On Training

What’s the Difference Between DirectAccess and Windows 10 Always On VPN?

5 Important Things DirectAccess Administrators Should Know About Windows 10 Always On VPN

3 Important Advantages of Windows 10 Always On VPN over DirectAccess 

Windows 10 Always On VPN and the Future of DirectAccess

%d bloggers like this: