Introduction
DirectAccess in Windows Server 2012 R2 includes support for load balancing using either Windows Network Load Balancing (NLB) or an external physical or virtual load balancer. There are advantages and disadvantages to each, but NLB is commonly deployed due to its cost (free!) and relative ease of configuration. NLB has three operation modes – Unicast, Multicast, and IGMP Multicast. It may become necessary to change the NLB operation mode depending on the environment where DirectAccess is deployed. This article describes when and how to make those changes.
Default Configuration
When NLB is first configured, the default cluster operation mode is set to Unicast. In this configuration, all nodes in the NLB cluster share the same MAC address. The NLB kernel mode driver prevents the switch from learning the MAC address for any node in the cluster by masking it on the wire. When a frame is delivered to the switch where the NLB cluster resides, without a MAC address to switch port mapping the frame is delivered to all ports on the switch. This induces switch flooding and is by design. It is required for all nodes in the cluster to “see” all traffic. The NLB driver then determines which node will handle the request.
NLB on Hyper-V
Unicast NLB typically works without issue in most physical environments. However, enabling NLB when the DirectAccess server is running on a virtual machine requires some additional configuration. For Hyper-V, the only thing that is required is to enable MAC Address Spoofing on the virtual network adapter as I discussed here. No other changes are required.
NLB on VMWare
For VMware environments, it will be necessary to change the cluster operation mode from unicast to multicast. This is because the VMware hypervisor proactively informs the virtual switch of the virtual machine’s MAC address on startup and during other virtual networking events. When this occurs, all traffic for the NLB Virtual IP Address (VIP) will be delivered to a single node in the cluster. In multicast operation mode, all nodes in the NLB cluster retain their original MAC address and a unique MAC address is assigned to the cluster VIP. As such, there’s no need to prevent the switch from learning the virtual machine’s MAC address.
Configuring Multicast NLB
To enable Multicast NLB, first enable load balancing for DirectAccess using the Remote Access Management console as usual. DO NOT perform the initial configuration of NLB outside of the Remote Access Management console! Before adding another member to the array, open the Network Load Balancing Manager, right-click the cluster and choose Cluster Properties. Select the Cluster Parameters tab and change the Cluster operation mode to Multicast.
When opening the Network Load Balancing Manager locally on the DirectAccess server, you may receive the following error message:
“Running NLB Manager on a system with all networks bound to NLB might not work as expected. If all interfaces are set to run NLB in “unicast” mode, NLB manager will fail to connect to hosts.”
If you encounter this error message it will be necessary to run the NLB Manager on another host. You can install the NLB Manager on a Windows Server 2012 R2 system by using the following PowerShell command.
Install-WindowsFeature RSAT-NLB
Optionally you can download and install the Windows Remote Server Administration Tools (RSAT) on a Windows desktop client and manage NLB remotely.
Once this change has been made you can add additional DirectAccess servers to the array using the Remote Access Management console.
Additional Configuration
If you cannot communicate with the cluster VIP from a remote subnet, but can connect to it while on the same subnet, it might be necessary to configure static ARP entries on any routers for the subnet where the NLB cluster resides. Often this is required because routers will reject responses to ARP requests that are from a host with a unicast IP address but have a multicast MAC address.
Richmond
/ November 8, 2015Richard, once again thank you for another informative piece. We are currently running two DA environments each with a pair of virtual servers using NLB in the default configuration. We are running the VMs on VMware and to resolve the issue with ‘One node only’, we implemented the change described in the link below:
http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1556
No doubt you are well aware of this option, but you were informing readers of your blog of the alternative operational mode of NLB.
Could I please ask you to write a piece on the steps to configure DA to take advantage of the use of ECC based certificates.
Richard M. Hicks
/ November 9, 2015Yes, I’m aware of that option. I don’t recommend using it though as it tends to break other things in VMware. 🙂 Regarding DirectAccess and ECC certificates, I’m assuming you are trying to meet CESG requirements for PSN compliance?
Richmond Howie
/ November 9, 2015Yes & no. We have both an interim and endstate (cesg terms I think) pki infrastructure, but when configuring it to use ECC certificates, endpoints didn’t appear to authenticate. It would be preferable to alter our setup to utilise the endstate certificates, but thankfully not enforced…..yet.
Richard M. Hicks
/ November 10, 2015Ok, thanks for the clarification. In its default configuration, DirectAccess does not work with ECC certificates. However, there is a bespoke configuration that I believe enables the functionality. If you send me an email I can provide you with contact information for resources in your region that can provide more detail.
mfekry86
/ April 17, 2016Hi Richard
How can i load balance the DA servers when its deployed in Edge mode ?
Richard M. Hicks
/ April 23, 2016The VIP on the load balancer would reside in the same subnet as the real IP address. Never tried it myself, but I think it should work.
Jeroen Burgerhout (@BurgerhoutJ)
/ April 22, 2016To enable multicast on VMware, should I set multicast on the internal and the external cluster?
Richard M. Hicks
/ April 23, 2016Yes, both must be in multicast operating mode.
Rob
/ July 12, 2016Richard,
I have 2 DA servers with single NIC’s and the multicast NLB is working great in our VMware environment. The only thing I have noticed is that the NLB reverts back to unicast after a server reboot. I have to reapply the multicast changes after each reboot or change to the Remote Access server policy.
It appears that the DA server policy is updating the NLB settings. Here is the message found in “HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\WLBS\ConfigurationHistory\{GUID}\Completions”
Processing update 2 from “DA NLB powershell cmdlets”
Starting update…
Going to bind NLB…
Bind operation succeeded.
Cluster configuration stabilized.
Going to modify cluster configuration…
Modification succeeded.
Going to modify the IP address list…
The IP addresses list modified successfully.
Update completed successfully.
Is there any way to configure the remote access server GPO to use multicast instead of manually configuring multicast?
Thanks for the great article,
-Rob
Richard M. Hicks
/ July 12, 2016That’s unusual, and not something I’ve ever encountered. Be sure to perform the initial configuration of NLB using the Remote Access Management console, not the NLB management console. Once you’ve enabled NLB initially, you can then change the cluster operating mode to multicast using the NLB management tool and the settings should stick.
Rob
/ July 12, 2016The NLB was created using the Remote Access Management Console. For now a manual startup procedure is working. If anyone else comes across this issue and finds a fix, please share the solution in the comments.
Richard M. Hicks
/ July 13, 2016Very strange. This is certainly the first time I’ve ever heard of this. Hopefully you’ll be able to find a root cause at some point.
Jos Rossiau
/ July 20, 2016Hello Richard,
I am running a test in vmware workstation with a 2012 R2 DA with 2 nics behind an edge. nic1 is in lan, nic2 is in DMZ. I have installed a second DA and want to enable Microsoft NLB.
Because this is a test and later on I want to run this in our vmware vsphere environment I needed to change nlb from unicast to multicast.
I enabled load balancing via remote access management console and everything worked fine in unicast. However, when I change to multicast (from a third computer because both nics are running unicast nlb) DA stops working because of DNS problems. It cannot access the ipv6 of the DA dns.
I noticed that before the multicast change the ipv6 address of the internal network was next to the ipv4 address configured as the cluster ip. After the change to multicast it was gone, only the ipv4 was still there.
DA still expects that ipv6 address to be available so I added it manually and after a reboot it took some time to get all the checks green but it seemed to work. Not sure if anyone else has seen this behaviour before but just so you know, adding the (old from before the change to multicast) ipv6 address of the internal adapter will solve the DNS error.
Kind regards,
Jos Rossiau
Richard M. Hicks
/ July 20, 2016I have seen some strange things happy with NLB on VMWare workstation. It seems to be more stable on ESXi/VSphere though. The workaround you discovered should work though. Hopefully you won’t have these issues when you get to production.
Jos Rossiau
/ July 20, 2016Thanks, I was hoping it would be workstation related. I will start testing soon on vpshere and see if it occurs there too.
On another note, looks like enabling load balancing broke my manage out via isatap. I followed your article on using isatap with GPO and I added the new DIP as a second isatap record in DNS but it does not work. I checked the ipv6 routes via netsh and there were no default routes for the global ipv6 to my da client. Before the change those routes did exist.
I tried adding them manually but that did not work either.
I am thinking about enabling native ipv6 instead and stop using isatap, as it is no longer supported by MS in 2012 R2.
Anyway, thank you very much for your helpful articles about DA, they helped me a lot making sense of this seemingly easy (atleast most managers think so) subject.
Kind regards,
Jos Rossiau
Richard M. Hicks
/ July 22, 2016ISATAP is not formally supported with DirectAccess in Windows Server 2012 R2 except for single server deployments. It is possible to get manage-out to work using ISATAP with load-balanced or multisite deployments, it just requires some additional infrastructure. Send me an email if you’d like more details.
Lee Humphreys
/ August 26, 2016Hi Richard,
When ISATAP is enabled using Windows NLB, is there any way to enable connections to remote clients that are not connected to the same DA Server as the Manage Out machine? For example, DA1 has a DA Client remotely connected across the Internet. My Manage Out machine is connected to DA2 for ISATAP – I can only connect to the remote machine if my ISATAP Tunnel is connected to the same DA Server.
Is there any way to workaround/fix this?
Lee
Richard M. Hicks
/ August 26, 2016Absolutely. To do this, the ISATAP DNS record must resolve to the VIP, but also the DIPs of each DirectAccess server as well. So, for two DirectAccess servers configured with NLB, there will be three DNS records for ISATAP in total. Once configured, you can connect to the remote DirectAccess client regardless which DirectAccess server it connected to.
Lee Humphreys
/ August 29, 2016Thanks Richard. That’s what I have as per Jason Jones’ article. I have a DNS Record for both DIPs and the VIP, however I can only seem to connect to the DA Client that is also connected to the same DA Server as my ISATAP Manage Out Computer?
Richard M. Hicks
/ September 3, 2016Something must be wrong then. I’d suggest checking your ISATAP client’s routing table to ensure that you are getting the necessary routes for both DirectAccess client IPv6 prefixes. You should be able to connect to DirectAccess clients regardless which server they establish a connection with.
Jakub
/ March 28, 2017Hi Richard. Quick question about creating static arp for multicast NLB. We have deployment with native IPv6, static ARP is created only for MAC address assigned to the IPv4 VIP of the NLB. Connection works for IPv4 VIP but not for IPv6 VIP. This is understable for me because we are missing static ARP for IPv6 VIP. Looking in network flow in Network Monitor tool I see that IPv6 Solicitation Messages (used by DA server to obtain ARP of default gateway) are sending with IPv6 DIP and Multicast MAC address assigned to the IPv4 VIP, not the MAC associated with IPv6 VIP. So in this case what static ARP should we create, I mean IPv6 VIP with which MAC address ?
I am checking the MAC addresses from drop down list for each VIP address in NLB Manager
Richard M. Hicks
/ March 28, 2017I have never configured multicast NLB using IPv6 native addressing, so I’m not really sure how to do this for IPv6. I’m assuming you’d use the New-NetNeighbor or Set-NetNeighbor PowerShell commands?
Jakub
/ March 29, 2017Thanks for reply. Yes I have configured MAC addresses of our routers on DA servers manually but this allowed only communication from DA servers to the IPv6 addresses of routers. Traffic from other vlans to the IPv6 VIP of NLB is not possible still. Looking in ntw packets it looks that router is dropping packets with IPv6 VIP and the same multicast MAC as for IPv4 VIP. I have already asked my network team to create a change for addding another static ARP entry for IPv6 addr. Will keep you posted!
Louis Videc
/ January 2, 2020Hello Richard, When Configuring NLB Do you load balance the internal adapter or External adapter or Both? I would think you would just need to do the external but I defer to your judgment. My external adapter in in the DMZ and I have a NAT going to the external VIP
Richard M. Hicks
/ January 4, 2020Correct. Just the external interface. 🙂
Louis Videc
/ January 16, 2020Hi Again, when using the wizard to setup the load balance cluster. It asks you to select the inbound and external NIC. When it completes it sets up a NLB cluster for both the Internal and external NIC. Should I break the internal NIC cluster? Not sure why it does both if you only want the external clustered.
Richard M. Hicks
/ January 16, 2020Microsoft load balances both the internal and external interfaces because they want to provide high availability for the NLS and web probe host URLs if they are running on the DirectAccess server. This isn’t strictly required, but they have to do this to support all deployment scenarios. If the wizard enables NLB for both sides, I’d say leave it as is. It shouldn’t hurt anything.