All posts in category troubleshooting

Always On VPN Authentication Failed Reason Code 16

Strong authentication is essential for remote access to on-premises resources over the public Internet. Using the Protected Extensible Authentication Protocol (PEAP) in combination with user certificates issued by the organization’s internal certification authority (CA) provides high assurance for remote user authentication. It includes the added benefit of making the Always On VPN connection completely seamless for the user, as their certificate is presented to the authentication server transparently during VPN connection establishment. Using PEAP with user certificates is the recommended authentication method for Always On VPN deployments.

Reason Code 16

When configuring Always On VPN to use PEAP with client authentication certificates, administrators may encounter a scenario in which a user has a valid certificate. Yet, their authentication request is rejected by the Network Policy Server (NPS) server when attempting to connect remotely. Looking at the Security event log on the NPS server, administrators will find a corresponding event ID 6273 in the Network Policy Server task category from the Microsoft Windows security auditing event source. In the Authentication Details section, you’ll find that the reason code for the failed request is Reason Code 16, with the following reason specified.

“Authentication failed due to a user credentials mismatch. Either the user name provided does not map to an existing user account or the password was incorrect”.

Password Incorrect?

The reason code indicates the user may have entered an incorrect password. However, the user does not enter their password when using PEAP with client authentication certificates, so there’s no chance the password was entered incorrectly.

TPM

I have increasingly encountered this scenario with many customers deploying Always On VPN over the last year or so. This error is often caused by a known issue with older TPM models. Specifically, those with a TPM specification sub-version of 1.16 and earlier. You can view these TPM details by opening the Windows Settings app and entering ‘security processor’ in the search field.

Workaround

These older TPM models seem to have an issue with RSA-PSS signature algorithms, as described here. If possible, administrators should upgrade devices with older TPM versions to ensure the highest level of security and assurance for their remote users. However, in cases where that is not feasible, administrators can remove RSA-PSS signature algorithms from the registry, which forces the use of a different signature algorithm and seems to restore functionality.

To do this, open the registry editor (regedit.exe) and navigate to the following registry key.

HKLM\SYSTEM\CurrentControlSet\Control\Cryptography\
Configuration\Local\SSL\00010003\

Double-click the Functions entry and remove the following algorithms from the Value data section.

  • RSAE-PSS/SHA256
  • RSAE-PSS/SHA384
  • RSAE-PSS/SHA512

Once complete, reboot the device and test authentication once again.

Intune Proactive Remediation

Administrators using Intune Proactive Remediation will find detection and remediation scripts to make these changes published on GitHub.

Detect-RsaePss.ps1

Remediate-RsaePss.ps1

Additional Information

Windows TPM 2.0 Client Authentication in TLS 1.2 with RSA PSS

Always On VPN NPS Auditing and Logging

Always On VPN NPS RADIUS Configuration Missing

Always On VPN NPS Load Balancing

11 Comments
by Richard M. Hicks on February 13, 2023  •  Permalink
Posted in Active Directory, administration, Always On VPN, AOVPN, authentication, certificates, Cryptography, EAP, Encryption, Enterprise, enterprise mobility, Microsoft, Mobility, network policy server, NPS, Operational Support, PEAP, PKI, public key infrastructure, Remote Access, Security, TPM, troubleshooting, Trusted Platform Module, VPN, Windows 10, Windows 11
Tagged , , , , , , , , , , , , , , , , , , , , , ,

Posted by Richard M. Hicks on February 13, 2023

https://directaccess.richardhicks.com/2023/02/13/always-on-vpn-authentication-failed-reason-code-16/

Always On VPN Error 13868

Troubleshooting Always On VPN Error 691 and 812 – Part 2

The Internet Key Exchange version 2 (IKEv2) VPN protocol is the protocol of choice for Microsoft Always On VPN deployments where the highest levels of security and assurance are required. However, as I’ve written about in the past, often the default IKEv2 security settings are less than desirable. Before using IKEv2 VPN in a production environment the administrator will need to update these security settings accordingly.

Connection Failure

When configuring Windows Server Routing and Remote Access Service (RRAS) or a third-party VPN appliance to support IKEv2 using custom security policies, the administrator may encounter a scenario in which a connection cannot be established due to a policy mismatch error. When the connection attempt fails, an error will be recorded in the Windows Application event log from the RasClient source with Event ID 20227. The error message states the following:

“The user [username] dialed a connection named [connection name] which has failed. The error code returned on failure is 13868.”

Always On VPN IKEv2 Policy Mismatch Error

Error Code 13868

Error code 13868 translates to ERROR_IPSEC_IKE_POLICY_MATCH. Essentially this error indicates that the IKEv2 security policy on the client did not match the configuration on the server.

Server Configuration

To view the current IKEv2 IPsec policy configuration, open an elevated PowerShell command window and run the following command.

Get-VpnServerIPsecConfiguration

Always On VPN IKEv2 Policy Mismatch Error

Client Configuration

To ensure interoperability, the VPN client must be configured to use the same IKEv2 security policy as defined on the sever. To view a VPN client’s currently configured IKEv2 security policy, open an elevated PowerShell command window and run the following command.

Get-VpnConnection -Name [connection name] | Select-Object -ExpandProperty IPsecCustomPolicy

Always On VPN IKEv2 Policy Mismatch Error

Note: If this PowerShell command returns no output, the VPN connection is not using a custom IKEv2 IPsec security policy.

Updating Settings

Guidance for configuring IKEv2 security policies on Windows Server RRAS and Windows 10 can be found here.

NPS Policy

Another common cause of IKEv2 policy mismatch errors is a misconfigured Network Policy Server (NPS) network policy. Specifically, administrators may disable Basic and Strong encryption for MPPE in an attempt to improve security.

Always On VPN IKEv2 Policy Mismatch Error

The NPS policy for Always On VPN must include Strong encryption at a minimum. Basic and No encryption can be safely disabled.

Always On VPN IKEv2 Policy Mismatch Error

Summary

IKEv2 policy mismatch errors can be resolved easily by ensuring both the VPN server and client are configured to use the same IPsec security policies. Use the PowerShell commands in the above referenced above to validate settings and make changes when necessary.

Additional Information

Microsoft Always On VPN Error 13801

Microsoft Windows Always On VPN Error 13806

Microsoft Windows Always On VPN Certificate Requirements for IKEv2

Microsoft Windows Always On VPN IPsec Root Certificate Configuration Issue

Microsoft Windows Always On VPN IKEv2 Policy Mismatch Error

Microsoft Windows Always On VPN IKEv2 Security Configuration

Microsoft Windows Always On VPN IKEv2 Fragmentation

Microsoft Windows Always On VPN IKEv2 Load Balancing and NAT

Microsoft Windows Always On VPN IKEv2 Features and Limitations

1 Comment
by Richard M. Hicks on February 12, 2023  •  Permalink
Posted in Always On VPN, AOVPN, authentication, Azure VPN, Azure VPN Gateway, certificates, Cryptography, device tunnel, Encryption, Enterprise, enterprise mobility, IKEv2, Infrastructure, Microsoft, Mobility, network policy server, NPS, Operational Support, PKI, PowerShell, Remote Access, routing and remote access service, Security, troubleshooting, VPN, Windows 10, Windows 11, Windows Server 2012 R2, Windows Server 2016, Windows Server 2019, Windows Server 2022
Tagged , , , , , , , , , , , , , , , , , , , , , , , ,

Posted by Richard M. Hicks on February 12, 2023

https://directaccess.richardhicks.com/2023/02/12/always-on-vpn-error-13868/

Always On VPN and Split DNS

Cloudflare Public DNS Resolver Now Available

Split DNS, sometimes called ‘split brain’ DNS, is when an organization uses the same DNS namespace internally and externally. For example, the internal Active Directory domain name is example.com, so internal resources are accessed using a fully qualified domain name (FQDN) like dc1.example.com. Additionally, external properties such as mail and web services use the same namespace so that a public web server might have a name like www.example.com. Internal resources will resolve to internal, private IP addresses, whereas public services resolve to external, public IP addresses.

Complications

Things get complicated when the same resource (FQDN) is available internally and externally, especially for Always On VPN clients. For example, accessing app.example.com on the internal network resolves to a private address, but accessing the same resource on the Internet resolves to a public IP address. Often there are different authentication requirements for internal and external resources, which can yield unexpected results.

Name Resolution

Always On VPN administrators might prefer app.example.com to be accessed via the Internet when connected with Always On VPN. However, VPN clients will attempt to connect via the internal network using their default configuration. Solving this challenge requires internal DNS server changes.

NRPT?

It might be tempting for administrators to use the Name Resolution Policy Table (NRPT) to solve name resolution issues for Always On VPN. However, the NRPT has some limitations and may not always produce the desired results. For example, the NRPT only directs DNS queries. It does not define which resource records are returned by DNS. Also, some applications ignore the NRPT, which limits its usefulness. A better solution is to use DNS Policies in Windows Server.

DNS Policies

Microsoft introduced DNS policies with Windows Server 2016. DNS policies are a powerful tool administrators can use to fine-tune name resolution based on many factors. In the case of split DNS, administrators can configure internal DNS to return an IP address for a resource based on the source IP address of the name resolution query. VPN clients receive one IP address for a given DNS query, while all other clients receive a different IP address. DNS policies ensure that remote clients connected to the VPN will receive the proper IP address for the resource requested, as defined by the administrator.

Caveats

DNS policies are powerful and flexible, but there are some potential drawbacks. All enterprise DNS servers used by Always On VPN clients must be running Windows Server 2016 or later. Also, administrators must use PowerShell to configure DNS policies exclusively. There is no GUI interface to configure DNS policies. DNS policies do not appear in the DNS management interface, which could confuse an administrator unaware that DNS policies are in place. In addition, DNS client subnets and query resolution policies do not replicate across DNS servers. Administrators must manually configure these on each DNS server used by Always On VPN clients. However, zone scopes and resource records in those scopes do replicate automatically.

Scenario

For demonstration purposes, let’s assume that an Always On VPN client needs to access foo.example.com. It resolves to a private IP address on the internal network and a public IP address on the Internet. By default, foo.example.com will resolve to the internal private IP address of the server when connected with Always On VPN. However, the desire is to have foo.example.com resolve to the public IP address when connected with Always On VPN. To accomplish this, we’ll create a DNS policy to ensure that connected Always On VPN clients can resolve foo.example.com to the public IP address when resolving this name over the VPN tunnel.

DNS Policy Configuration

Open an elevated PowerShell command on a DNS server and perform the following steps to create a DNS policy for VPN clients.

Client Subnet

Run the Add-DnsServerClientSubnet PowerShell command to create a client subnet in DNS that includes all IP networks assigned to VPN clients. Summarize IP prefixes if there are multiple VPN servers in the organization.

Add-DnsServerClientSubnet -Name VPN -IPv4Subnet ‘172.16.100.0/22’ -IPv6Subnet ‘2001:db8:fcd2:1000::/60’

If summarizing IP prefixes for multiple servers isn’t possible, multiple subnets can be added to a DNS client subnet using the following command.

Add-DnsServerClientSubnet -Name VPN -IPv4Subnet @(‘172.16.100.0/24’, ‘172.16.101.0/24’, ‘172.16.102.0/24’, ‘172.16.103.0/24’) -IPv6Subnet @(‘2001:db8:fcd2:1001::/64’, ‘2001:db8:fcd2:1002::/64’, ‘2001:db8:fcd2:1003::/64’)

To make changes to an existing DNS client subnet, use the Set-DnsServerClientSubnet PowerShell command.

Note: Client Subnets do not replicate across domain controllers. Run the command above on all DNS servers or each DNS server used by Always On VPN clients.

Zone Scope

Create a Zone Scope that includes the DNS records to be returned to VPN clients. The default zone scope is the DNS zone itself. Configure an additional zone scope for the DNS zone by using the Add-DnsServerZoneScope PowerShell command.

Add-DnsServerZoneScope -ZoneName example.com -Name VPN

Resource Records

Next, add DNS records to the new zone scope using the Add-DnsServerResourceRecord PowerShell command.

Add-DnsServerResourceRecord -ZoneName example.com -A -Name foo -IPv4Address 203.0.113.12 -ZoneScope VPN

Add-DnsServerResourceRecord -ZoneName example.com -AAAA -Name foo -IPv6Address 2001:db8:21::12 -ZoneScope VPN

DNS Policy

Finally, create a DNS query resolution policy that ties everything together. Run the Add-DnsServerQueryResolutionPolicy command to create the DNS query resolution policy. Once configured, when the DNS server receives a DNS query, the policy will recognize that the query originates from a VPN client subnet and will return the resource record from the VPN zone scope with the public IP address defined previously.

Add-DnsServerQueryResolutionPolicy -Name VPN -Action ALLOW -ClientSubnet ‘EQ,VPN’ -FQDN ‘EQ,foo.example.com’ -ZoneScope ‘VPN,1’ -ZoneName example.com

Note: DNS query resolution policies do not replicate across domain controllers. Run the command above on all DNS servers or each DNS server used by Always On VPN clients.

Results

Once complete, the hostname ‘foo’ in our example above resolves to different IP addresses based on the client’s IP address.

DNS query for ‘foo’ from internal client.

DNS query for ‘foo’ from VPN client.

Summary

There are many scenarios where Windows Server DNS policies can be used to fine-tune name resolution for Always On VPN clients. Hopefully, this example gives you an idea of how DNS policies work, and you can use them to solve your unique name resolution challenges with Always On VPN.

Additional Information

Windows Server DNS Policies Overview

Always On VPN Short Name Access Failure

Always On VPN Client DNS Server Configuration

Leave a comment
by Richard M. Hicks on January 9, 2023  •  Permalink
Posted in Active Directory, administration, Always On VPN, AOVPN, DNS, DNS Policies, Enterprise, enterprise mobility, Infrastructure, Microsoft, Mobility, Name Resolution, name resolution policy table, NRPT, Operational Support, PowerShell, Remote Access, Split DNS, troubleshooting, user tunnel, VPN, Windows 10, Windows 11, Windows Server 2016, Windows Server 2019, Windows Server 2022
Tagged , , , , , , , , , , , , , , , , , ,

Posted by Richard M. Hicks on January 9, 2023

https://directaccess.richardhicks.com/2023/01/09/always-on-vpn-and-split-dns/

« Older Posts
Newer Posts »