Always On VPN Authentication Failed Reason Code 16

Strong authentication is essential for remote access to on-premises resources over the public Internet. Using the Protected Extensible Authentication Protocol (PEAP) in combination with user certificates issued by the organization’s internal certification authority (CA) provides high assurance for remote user authentication. It includes the added benefit of making the Always On VPN connection completely seamless for the user, as their certificate is presented to the authentication server transparently during VPN connection establishment. Using PEAP with user certificates is the recommended authentication method for Always On VPN deployments.

Reason Code 16

When configuring Always On VPN to use PEAP with client authentication certificates, administrators may encounter a scenario in which a user has a valid certificate. Yet, their authentication request is rejected by the Network Policy Server (NPS) server when attempting to connect remotely. Looking at the Security event log on the NPS server, administrators will find a corresponding event ID 6273 in the Network Policy Server task category from the Microsoft Windows security auditing event source. In the Authentication Details section, you’ll find that the reason code for the failed request is Reason Code 16, with the following reason specified.

“Authentication failed due to a user credentials mismatch. Either the user name provided does not map to an existing user account or the password was incorrect”.

Password Incorrect?

The reason code indicates the user may have entered an incorrect password. However, the user does not enter their password when using PEAP with client authentication certificates, so there’s no chance the password was entered incorrectly.

TPM

I have increasingly encountered this scenario with many customers deploying Always On VPN over the last year or so. This error is often caused by a known issue with older TPM models. Specifically, those with a TPM specification sub-version of 1.16 and earlier. You can view these TPM details by opening the Windows Settings app and entering ‘security processor’ in the search field.

Workaround

These older TPM models seem to have an issue with RSA-PSS signature algorithms, as described here. If possible, administrators should upgrade devices with older TPM versions to ensure the highest level of security and assurance for their remote users. However, in cases where that is not feasible, administrators can remove RSA-PSS signature algorithms from the registry, which forces the use of a different signature algorithm and seems to restore functionality.

To do this, open the registry editor (regedit.exe) and navigate to the following registry key.

HKLM\SYSTEM\CurrentControlSet\Control\Cryptography\
Configuration\Local\SSL\00010003\

Double-click the Functions entry and remove the following algorithms from the Value data section.

  • RSAE-PSS/SHA256
  • RSAE-PSS/SHA384
  • RSAE-PSS/SHA512

Once complete, reboot the device and test authentication once again.

Intune Proactive Remediation

Administrators using Intune Proactive Remediation will find detection and remediation scripts to make these changes published on GitHub.

Detect-RsaePss.ps1

Remediate-RsaePss.ps1

Additional Information

Windows TPM 2.0 Client Authentication in TLS 1.2 with RSA PSS

Always On VPN NPS Auditing and Logging

Always On VPN NPS RADIUS Configuration Missing

Always On VPN NPS Load Balancing

Always On VPN and Split DNS

Cloudflare Public DNS Resolver Now Available

Split DNS, sometimes called ‘split brain’ DNS, is when an organization uses the same DNS namespace internally and externally. For example, the internal Active Directory domain name is example.com, so internal resources are accessed using a fully qualified domain name (FQDN) like dc1.example.com. Additionally, external properties such as mail and web services use the same namespace so that a public web server might have a name like www.example.com. Internal resources will resolve to internal, private IP addresses, whereas public services resolve to external, public IP addresses.

Complications

Things get complicated when the same resource (FQDN) is available internally and externally, especially for Always On VPN clients. For example, accessing app.example.com on the internal network resolves to a private address, but accessing the same resource on the Internet resolves to a public IP address. Often there are different authentication requirements for internal and external resources, which can yield unexpected results.

Name Resolution

Always On VPN administrators might prefer app.example.com to be accessed via the Internet when connected with Always On VPN. However, VPN clients will attempt to connect via the internal network using their default configuration. Solving this challenge requires internal DNS server changes.

NRPT?

It might be tempting for administrators to use the Name Resolution Policy Table (NRPT) to solve name resolution issues for Always On VPN. However, the NRPT has some limitations and may not always produce the desired results. For example, the NRPT only directs DNS queries. It does not define which resource records are returned by DNS. Also, some applications ignore the NRPT, which limits its usefulness. A better solution is to use DNS Policies in Windows Server.

DNS Policies

Microsoft introduced DNS policies with Windows Server 2016. DNS policies are a powerful tool administrators can use to fine-tune name resolution based on many factors. In the case of split DNS, administrators can configure internal DNS to return an IP address for a resource based on the source IP address of the name resolution query. VPN clients receive one IP address for a given DNS query, while all other clients receive a different IP address. DNS policies ensure that remote clients connected to the VPN will receive the proper IP address for the resource requested, as defined by the administrator.

Caveats

DNS policies are powerful and flexible, but there are some potential drawbacks. All enterprise DNS servers used by Always On VPN clients must be running Windows Server 2016 or later. Also, administrators must use PowerShell to configure DNS policies exclusively. There is no GUI interface to configure DNS policies. DNS policies do not appear in the DNS management interface, which could confuse an administrator unaware that DNS policies are in place. In addition, DNS client subnets and query resolution policies do not replicate across DNS servers. Administrators must manually configure these on each DNS server used by Always On VPN clients. However, zone scopes and resource records in those scopes do replicate automatically.

Scenario

For demonstration purposes, let’s assume that an Always On VPN client needs to access foo.example.com. It resolves to a private IP address on the internal network and a public IP address on the Internet. By default, foo.example.com will resolve to the internal private IP address of the server when connected with Always On VPN. However, the desire is to have foo.example.com resolve to the public IP address when connected with Always On VPN. To accomplish this, we’ll create a DNS policy to ensure that connected Always On VPN clients can resolve foo.example.com to the public IP address when resolving this name over the VPN tunnel.

DNS Policy Configuration

Open an elevated PowerShell command on a DNS server and perform the following steps to create a DNS policy for VPN clients.

Client Subnet

Run the Add-DnsServerClientSubnet PowerShell command to create a client subnet in DNS that includes all IP networks assigned to VPN clients. Summarize IP prefixes if there are multiple VPN servers in the organization.

Add-DnsServerClientSubnet -Name VPN -IPv4Subnet ‘172.16.100.0/22’ -IPv6Subnet ‘2001:db8:fcd2:1000::/60’

If summarizing IP prefixes for multiple servers isn’t possible, multiple subnets can be added to a DNS client subnet using the following command.

Add-DnsServerClientSubnet -Name VPN -IPv4Subnet @(‘172.16.100.0/24’, ‘172.16.101.0/24’, ‘172.16.102.0/24’, ‘172.16.103.0/24’) -IPv6Subnet @(‘2001:db8:fcd2:1001::/64’, ‘2001:db8:fcd2:1002::/64’, ‘2001:db8:fcd2:1003::/64’)

To make changes to an existing DNS client subnet, use the Set-DnsServerClientSubnet PowerShell command.

Note: Client Subnets do not replicate across domain controllers. Run the command above on all DNS servers or each DNS server used by Always On VPN clients.

Zone Scope

Create a Zone Scope that includes the DNS records to be returned to VPN clients. The default zone scope is the DNS zone itself. Configure an additional zone scope for the DNS zone by using the Add-DnsServerZoneScope PowerShell command.

Add-DnsServerZoneScope -ZoneName example.com -Name VPN

Resource Records

Next, add DNS records to the new zone scope using the Add-DnsServerResourceRecord PowerShell command.

Add-DnsServerResourceRecord -ZoneName example.com -A -Name foo -IPv4Address 203.0.113.12 -ZoneScope VPN

Add-DnsServerResourceRecord -ZoneName example.com -AAAA -Name foo -IPv6Address 2001:db8:21::12 -ZoneScope VPN

DNS Policy

Finally, create a DNS query resolution policy that ties everything together. Run the Add-DnsServerQueryResolutionPolicy command to create the DNS query resolution policy. Once configured, when the DNS server receives a DNS query, the policy will recognize that the query originates from a VPN client subnet and will return the resource record from the VPN zone scope with the public IP address defined previously.

Add-DnsServerQueryResolutionPolicy -Name VPN -Action ALLOW -ClientSubnet ‘EQ,VPN’ -FQDN ‘EQ,foo.example.com’ -ZoneScope ‘VPN,1’ -ZoneName example.com

Note: DNS query resolution policies do not replicate across domain controllers. Run the command above on all DNS servers or each DNS server used by Always On VPN clients.

Results

Once complete, the hostname ‘foo’ in our example above resolves to different IP addresses based on the client’s IP address.

DNS query for ‘foo’ from internal client.

DNS query for ‘foo’ from VPN client.

Summary

There are many scenarios where Windows Server DNS policies can be used to fine-tune name resolution for Always On VPN clients. Hopefully, this example gives you an idea of how DNS policies work, and you can use them to solve your unique name resolution challenges with Always On VPN.

Additional Information

Windows Server DNS Policies Overview

Always On VPN Short Name Access Failure

Always On VPN Client DNS Server Configuration