Local Sitecore containers unable to access Docker DNS

Question

We are working on setting up Sitecore 10 in containers in Docker. I have been able to set up a simple XP0 configuration which works when I run docker-compose up -d on a Windows Server 2019 machine.

The issue occurs, however, when I run it on a Windows 10 machine: the containers cannot access each other or themselves by name. When I run docker-compose up -d on the Windows 10 machine, it creates each of the containers successfully, but the several containers are unhealthy because the mssql container was not able to create the shard databases for xDB. It can't create those databases because the script that makes the call to create them uses the container's service name ("mssql") in the database connection string and the container is unable to resolve its own service name.

It turns out, after significant investigation, that Docker's DNS server seems to be failing to respond to requests from inside the container. When I run the below commands from inside the container, this is the result:

PS C:\> ipconfig /all

Windows IP Configuration

Host Name . . . . . . . . . . . . : 0b4ae6f46a90
Primary Dns Suffix . . . . . . . :
Node Type . . . . . . . . . . . . : Hybrid
IP Routing Enabled. . . . . . . . : No
WINS Proxy Enabled. . . . . . . . : No
DNS Suffix Search List. . . . . . : custom-domain.com


Connection-specific DNS Suffix . : custom-domain.com
Description . . . . . . . . . . . : Hyper-V Virtual Ethernet Container Adapter
Physical Address. . . . . . . . . : 00-15-5D-74-CA-B8
DHCP Enabled. . . . . . . . . . . : No
Autoconfiguration Enabled . . . . : Yes
Link-local IPv6 Address . . . . . : fe80::e566:a72a:ab7c:6108%27(Preferred)
IPv4 Address. . . . . . . . . . . : 172.31.51.68(Preferred)
Subnet Mask . . . . . . . . . . . : 255.255.240.0
Default Gateway . . . . . . . . . : 172.31.48.1
DNS Servers . . . . . . . . . . . : 172.31.48.1
                                    10.1.2.3
NetBIOS over Tcpip. . . . . . . . : Disabled
Connection-specific DNS Suffix Search List :
                                             custom-domain.com
PS C:\> ping 172.31.48.1

Pinging 172.31.48.1 with 32 bytes of data:
Reply from 172.31.48.1: bytes=32 time=1ms TTL=128
Reply from 172.31.48.1: bytes=32 time=1ms TTL=128
Reply from 172.31.48.1: bytes=32 time<1ms TTL=128

Ping statistics for 172.31.48.1:
Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 0ms, Maximum = 1ms, Average = 0ms
PS C:\> ping mssql
Ping request could not find host mssql. Please check the name and try again.
PS C:\> nslookup mssql
DNS request timed out.
timeout was 2 seconds.
Server: UnKnown
Address: 172.31.48.1

DNS request timed out.
timeout was 2 seconds.
DNS request timed out.
timeout was 2 seconds.
*** Request to UnKnown timed-out

Note: we're not doing anything fancy in our network configurations in Docker Compose or elsewhere; we're using all the defaults.

What I have verified:

1. All installed software and features are the same between the two host machines (Docker Desktop, WSL2, Hyper-V, etc.)
2. Both host machines can access the internet and are not having other connectivity issues
3. Both machines are running the containers in process isolation mode
4. The issue is not caused by a lack of hostnames. I've tried the solution in this answer.

Has anyone seen this before?

Answer 1

After a lot of research and troubleshooting, we were able to resolve this issue. It is caused by a group policy setting which prevents Docker from setting up Windows firewall rules necessary to pass the DNS requests around. I found the root cause buried in the comments of an issue raised for Docker Desktop's open source repo.

Resolution

1. Update the Apply local firewall rules group policy setting to Yes for the computer running Docker (your host machine)
2. Restart your host or execute gpupdate /force to apply the changes

Further info and troubleshooting (aka "Is this what's causing my issue?")

To identify whether this is what's causing your issue, you can do a few relatively simple checks.

1. Are DNS requests failing inside containers?

1. Open PowerShell as administrator
2. Execute docker run -it --rm --name miniwin mcr.microsoft.com/windows/servercore:20H2 powershell
   • Note: if your host machine is not compatible with the 20H2 version, you can also use one of Microsoft's other tags (ltsc2019 is a good backup option)
   • When the command completes, you will be at a PowerShell prompt inside the servercore container you just ran
3. Execute nslookup miniwin - This is the friendly name we gave the container and will be used to create a DNS lookup to Docker's embedded DNS server

If you receive a response (even a non-authoritative response), DNS is working as expected and this is not your issue.

If you receive a timeout response, then DNS requests are not working and this may be your issue.

2. Is the "Apply local firewall rules" group policy setting set to "No" on my computer?

1. Open a CMD or PowerShell window as administrator
2. Execute gpresult /h c:\temp\gp.html - this will create an HTML file that shows the contents of your computer's applied group policy
3. Open the file you just created (c:\temp\gp.html) in a web browser
4. Click the show all link at the top of the page
5. Search the page for "Apply local firewall rules"
6. You should see three entries, one each for Domain Profile Settings, Private Profile Settings, and Public Profile Settings. Check the value for each one. If it is No for the network profile you're currently using, this may be the root cause of your DNS issue.

3. Extra credit - review Windows Firewall logs

If you want to be extra sure, you can enable Windows Firewall logging, issue another nslookup command from inside your container, and watch for DROP UDP DNS entries on port 53.

1. Open the Advanced Firewall Management snap-in (WF.msc)
2. Open the Action menu and click Properties
3. On the Domain Profile tab, click Customize under the Logging section
   • Note: you should do this for whichever network profile your current network connection is using, or do it for all three if you're not sure
4. Turn on logging for dropped packets
5. Turn on logging for successful connections
   • This should begin logging future firewall traffic to %systemroot%\system32\LogFiles\Firewall\pfirewall.log by default, but you can change this setting in this properties window if you want
6. Return to your PowerShell prompt executing within the container
7. Execute nslookup miniwin again
8. Open the log file (%systemroot%\system32\LogFiles\Firewall\pfirewall.log by default) and look for your requests. They will be using the UDP protocol and be on port 53. They will include the IP address of your container as well as the IP address of the DNS server referenced in your container. (You can find both of these by running ipconfig /all from inside your container.)

Answer 2

Had the same issue, related to the DNS issues caused by company's firewall policies.

In our case adding .local token to the hostnames solved it.

For example, if the ${SQL_SERVER} variable equals mssql, than try to change it to mssql.local (same for solr -> solr.local).