Troubleshooting Always On VPN Error 691 and 812 – Part 2

Troubleshooting Always On VPN Error 691 and 812 – Part 2A while back I wrote about troubleshooting and resolving Windows 10 Always On VPN errors 691 and 812. There are numerous issues that can result in these errors, and in that post I pointed out they can be caused by disabling TLS 1.0 on Windows Servers prior to Windows Server 2016. However, administrators may encounter a another scenario in which they receive errors 691 or 812 which is related to Active Directory user account configuration.

SSTP and Error 691

When attempting to establish an Always On VPN connection using the Secure Socket Tunneling Protocol (SSTP), administrators may encounter the following error message.

“The remote connection was denied because the user name and password combination you provided is not recognized, or the selected authentication protocol is not permitted on the remote access server.”

Troubleshooting Always On VPN Error 691 and 812 – Part 2

In addition, an error 691 with event ID 20227 from the RasClient source can be found in the Application event log on the client.

“The user <domain\user> dialed a connection named which has failed. The error code returned on failure is 691.”

Troubleshooting Always On VPN Error 691 and 812 – Part 2

IKEv2 and Error 812

When attempting to establish an Always On VPN connection using Internet Key Exchange version 2 (IKEv2), administrators may encounter the following error message.

“The connection as prevented because of a policy configured on your RAS/VPN server. Specifically, the authentication method used by the server to verify your username and password may not match the authentication method configured in your connection profile. Please contact the Administrator of the RAS server and notify them of this error.”

Troubleshooting Always On VPN Error 691 and 812 – Part 2

In addition, an error 812 with event ID 20227 from the RasClient source can be found in the Application event log on the client.

Troubleshooting Always On VPN Error 691 and 812 – Part 2

NPS Event Log

On the NPS server the administrator will find an entry in the application event log with event ID 6273 from the Microsoft Windows security auditing source and the Network Policy Server task category indicating the network policy server denied access to the user. Looking closely at this event log message shows Reason Code 65 and the following reason.

“The Network Access Permission setting in the dial-in properties of the user account in Active Directory is set to Deny access to the user. To change the Network Access permission setting to either Allow access or Control access through NPS Network Policy, obtain the properties of the user account in Active Directory Users and Computers, click the Dial-in tab, and change Network Access Permission.”

Troubleshooting Always On VPN Error 691 and 812 – Part 2

Resolution

There are two options available to address this issue. The user account in Active Directory can be configured to grant access or allow access to be controlled via NPS network policy, or the NPS network policy can be configured to ignore user account dial-in properties.

User Account

Follow the steps below to change Network Access Permission on an individual user’s Active Directory account.

  1. Open the Active Directory User and Computers (ADUC) management console (dsa.msc) and double-click the user’s account.
  2. Select the Dial-in tab.
  3. In the Network Access Permission section select the option to Allow access or Control access through NPS Network Policy.

Troubleshooting Always On VPN Error 691 and 812 – Part 2

Note: If you do not see the dial-in tab, open the ADUC console on a domain controller. The dial-in tab is not displayed when using the Remote Server Administration Tools (RSAT) for Windows clients.

Network Policy

Follow the steps below to configure NPS network policy to ignore user account dial-in properties.

  1. Open the NPS management console (nps.msc) and double-click the Always On VPN network policy.
  2. In the Access Permission section select Ignore user account dial-in properties.
  3. Click Ok to save the changes.

Troubleshooting Always On VPN Error 691 and 812 – Part 2

Additional Information

Windows 10 Always On VPN Troubleshooting Error 691 and 812

Always On VPN IKEv2 Load Balancing with Citrix NetScaler ADC

Always On VPN SSTP Load Balancing with Citrix NetScaler ADCThe Internet Key Exchange version 2 (IKEv2) VPN protocol is the protocol of choice when the highest level of security is required for Always On VPN connections. It uses IPsec and features configurable security parameters that allow administrators to adjust policies to meet their specific security requirements. IKEv2 is not without some important limitations, but organizations may insist on the use of IKEv2 to provide the greatest protection possible for remote connected clients. Due to complexities of the IKEv2 transport, special configuration on the Citrix ADC (formerly NetScaler) is required when load balancing this workload.

Special Note: In December 2019 a serious security vulnerability was discovered on the Citrix ADC that gives an unauthenticated attacker the ability to arbitrarily execute code on the appliance. As of this writing a fix is not available (due end of January 2020) but a temporary workaround can be found here.

Load Balancing IKEv2

When an Always On VPN client establishes a connection using IKEv2, communication begins on UDP port 500, but switches to UDP port 4500 if Network Address Translation (NAT) is detected in the communication path between the client and the server. Because UDP is connectionless, custom configuration is required to ensure that VPN clients maintain connectivity to the same backend VPN server during this transition.

Initial Configuration

Load balancing IKEv2 using the Citrix ADC is similar to other workloads. Below are specific settings and parameters required to load balance IKEv2 using the Citrix ADC.

Note: This article is not a comprehensive configuration guide for the Citrix ADC. It assumes the administrator is familiar with basic load balancing concepts and has experience configuring the Citrix ADC.

Service Settings

The load balancing services for IKEv2 VPN will use UDP ports 500 and 4500. Create the service group and assign group members for UDP 500 as follows.

Always On VPN IKEv2 Load Balancing with Citrix NetScaler ADC

Always On VPN IKEv2 Load Balancing with Citrix NetScaler ADC

Repeat the steps above to create the service group for UDP port 4500.

Virtual Server Settings

Two virtual servers are required, one for UDP port 500 and one for UDP port 4500. Ensure that the service group using UDP port 500 is bound to the virtual server using the same port.

Always On VPN IKEv2 Load Balancing with Citrix NetScaler ADC

Always On VPN IKEv2 Load Balancing with Citrix NetScaler ADC

Repeat the steps above to create the virtual service for UDP port 4500.

Service Monitoring

Since IKEv2 uses the UDP protocol, the only option for service monitoring is to use PING, which is configured by default. Ensure that the firewall on the VPN server allows inbound ICMPv4 and ICMPv6 Echo Request. The default PING monitor on the Citrix ADC will ping the resource every 5 seconds. If a different interval is required, the administrator can edit the PING monitor and bind that to the service or service group as necessary.

Persistency Group

A Persistency Group on the Citrix ADC will be configured to ensure that IKEv2 VPN client requests from the same client are always routed to the same backend server. Follow the steps below to create a Persistency Group and assign it to both IKEv2 virtual servers created previously.

  1. In the Citrix ADC management console expand Traffic Management > Load Balancing > Persistency Groups.
  2. Click Add.
  3. Enter a descriptive name for the Persistency Group.
  4. Select SOURCEIP from the Persistence drop-down list.
  5. Next to the Virtual Server Name section click the Add button.
  6. Add both previously configured IKEv2 virtual servers for UDP 500 and 4500.
  7. Click Create.

Always On VPN IKEv2 Load Balancing with Citrix NetScaler ADC

Use Client IP

To ensure reliable connectivity for IKEv2 VPN connections it is necessary for the VPN server to see the client’s original source IP address. Follow the steps below to configure the Service Group to forward the client’s IP address to the VPN server.

  1. In the Citrix ADC management console expand System, click Settings, and then click Configure Modes.
  2. Select Use Subnet IP.
  3. Click Ok.Always On VPN IKEv2 Load Balancing and NAT
  4. Expand Traffic Management, click Load Balancing, and then click Service Groups.
  5. Select the IKEv2 UDP 500 Service Group.
  6. Click Edit in the Settings section.
  7. Select Use Client IP.
  8. Repeat these steps on the IKEv2 UDP 4500 Service Group.Always On VPN IKEv2 Load Balancing and NAT

Note: Making the above changes will require configuring the VPN server to use the Citrix ADC as its default gateway.

Additional Information

Windows 10 Always On VPN IKEv2 Load Balancing and NAT

Windows 10 Always On VPN SSTP Load Balancing with Citrix NetScaler ADC

Windows 10 Always On VPN IKEv2 Features and Limitations

Windows 10 AlWAYS On VPN and IKEv2 Fragmentation

Windows 10 Always On VPN IKEv2 Security Configuration

Windows 10 Always On VPN Certificate Requirements for IKEv2

Always On VPN Device Tunnel with Azure VPN Gateway

Always On VPN Device Tunnel with Azure VPN GatewayAlways On VPN is infrastructure independent, which allows for many different deployment scenarios including on-premises and cloud-based. In Microsoft Azure, the Azure VPN gateway can be configured to support Windows 10 Always On VPN client connections in some scenarios. Recently I wrote about using the Azure VPN gateway for Always On VPN user tunnels. In this post I’ll describe how to configure the Azure VPN gateway to support an Always On VPN device tunnel.

Limitations

There are a few crucial limitations that come with using the Azure VPN gateway for Always On VPN. Importantly, the Azure VPN gateway can support either user tunnels or device tunnels, not both at the same time. In addition, Azure supports only a single VPN gateway per VNet, so deploying an additional VPN gateway in the same VNet to support Always On VPN user tunnels is not an option.

Root CA Certificate

The Always On VPN device tunnel is authenticated using a machine certificate issued to domain-joined Windows 10 Enterprise edition clients by the organization’s internal Certification Authority (CA). The CA’s root certificate must be uploaded to Azure for the VPN gateway to authorize device tunnel connections. The root CA certificate can be exported using the Certification Authority management console (certsrv.msc) or via the command line.

Export Certificate – GUI

Follow the steps below to export a root CA certificate using the Certification Authority management console.

1. On the root CA server, open the Certification Authority management console.
2. Right-click the CA and choose Properties.
3. Select the CA server’s certificate and choose View Certificate.
4. Select the Details tab and click Copy to File.
5. Click Next.
6. Choose Base-64 encoded X.509 (.CER).

Always On VPN Device Tunnel with Azure VPN Gateway

7. Click Next.
8. Enter a location to save the file to.
9. Click Next, Finish, and Ok.

Export Certificate – Command Line

Follow the steps below to export a root CA certificate using the command line.

1. On the root CA server, open an elevated command window (not a PowerShell window).
2. Enter certutil.exe -ca.cert root_certificate.cer.
3. Enter certutil.exe -encode root.cer root_certificate_base64.cer.

Copy Public Key

1. Open the saved root certificate file using Notepad.
2. Copy the file contents between the BEGIN CERTIFICATE and END CERTIFICATE tags, as shown here. Use caution and don’t copy the carriage return at the end of the string.

Always On VPN Device Tunnel with Azure VPN Gateway

Point-to-Site Configuration

The Azure VPN gateway must be deployed as a Route-Based gateway to support point-to-site VPN connections. Detailed requirements for the gateway can be found here. Once the VPN gateway has been provisioned, follow the steps below to enable point-to-site configuration for Always On VPN device tunnels.

1. In the navigation pane of the Azure VPN gateway settings click Point-to-site configuration.
2. Click the Configure now link and specify an IPv4 address pool to be assigned to VPN clients. This IP address pool must be unique in the organization and must not overlap with an IP address ranges defined in the Azure virtual network.
3. From the Tunnel type drop-down list select IKEv2.
4. In the Root certificates section enter a descriptive name for the certificate in the Name field.
5. Copy and paste the Base64 encoded public key copied previously into the Public certificate data field.
6. Click Save to save the configuration.

Always On VPN Device Tunnel with Azure VPN Gateway

VPN Client Configuration

To support the Always On VPN device tunnel, the client must have a certificate issued by the internal CA with the Client Authentication Enhanced Key Usage (EKU). Detailed guidance for deploying a Windows 10 Always On VPN device tunnel can be found here.

Download VPN Configuration

1. Click Point-to-site configuration.
2. Click Download VPN client.
3. Click Save.
4. Open the downloaded zip file and extract the VpnSettings.xml file from the Generic folder.
5. Copy the FQDN in the VpnServer element in VpnSettings.xml. This is the FQDN that will be used in the template VPN connection and later in ProfileXML.

Create a Test VPN Connection

It is recommended to create a test VPN connection to perform validation testing of the Azure VPN gateway before provisioning an Always On VPN device tunnel broadly. On a domain-joined Windows 10 enterprise client, create a new VPN connection using IKEv2 with machine certificate authentication. Use the VPN server FQDN copied from the VpnSettings.xml file previously.

Always On VPN Device Tunnel with Azure VPN Gateway

Create an Always On VPN Connection

Once the VPN has been validated using the test profile created previously, an Always On VPN profile can be created and deployed using Intune, SCCM, or PowerShell. The following articles can be used for reference.

Deploy Always On VPN device tunnel using PowerShell

Deploy Always On VPN device tunnel using Intune

IKEv2 Security Configuration

The default IKEv2 security parameters used by the Azure VPN gateway are better than Windows Server, but the administrator will notice that a weak Diffie-Hellman (DH) key (Group 2 – 1024 bit) is used during IPsec phase 1 negotiation.

Always On VPN Device Tunnel with Azure VPN Gateway

Use the following PowerShell commands to update the default IKEv2 security parameters to recommended baseline defaults, including 2048-bit keys (DH group 14) and AES-128 for improved performance.

Connect-AzAccount
Select-AzSubscription -SubscriptionName [Azure Subscription Name]

$Gateway = [Gateway Name]
$ResourceGroup = [Resource Group Name]

$IPsecPolicy = New-AzVpnClientIpsecParameter -IpsecEncryption AES128 -IpsecIntegrity SHA256 -SALifeTime 28800 -SADataSize 102400000 -IkeEncryption AES128 -IkeIntegrity SHA256 -DhGroup DHGroup14 -PfsGroup PFS14

Set-AzVpnClientIpsecParameter -VirtualNetworkGatewayName $Gateway -ResourceGroupName $ResourceGroup -VpnClientIPsecParameter $IPsecPolicy

Note: Be sure to update the cryptography settings on the test VPN connection and in ProfileXML for Always On VPN connections to match the new VPN gateway settings. Failing to do so will result in an IPsec policy mismatch error.

Additional Information

Windows 10 Always On VPN User Tunnel with Azure VPN Gateway

Windows 10 Always On VPN IKEv2 Security Configuration

Windows 10 Always On VPN Device Tunnel Configuration using Microsoft Intune

Windows 10 Always On VPN Device Tunnel Configuration using PowerShell

Windows 10 Always On VPN Options for Azure Deployments

Windows 10 Always On VPN IKEv2 Features and Limitations