Things to watch out for when creating a swarm-wide network

Relying on Docker’s network documentation we know that there are the following types of networks offered by Docker.

bridgeFor standalone containers to talk to each other.
hostOnly for swarm services. No isolation between host and container, and the container uses host’s network directly.
overlayAllows swarm services on different nodes to talk with each other.
Allows swarm services to talk to standalone containers.
Allows standalone containers on different nodes to talk with each other.
macvlanAllows containers to have their own MAC address, and the docker daemon routes traffic using MAC addresses.
Used to provide compatibility to legacy applications that expects direct connection to a physical network.
noneContainers attached to it have no networking.
Docker network types.

The Exercise

To put theory into practice, let’s suppose we have a legitimate situation at hand that requires the use of an overlay network for our swarm cluster.

Inside the cluster, we have two nodes, A and B, where A is the swarm manager. Let’s say we want all ingress network traffic to go to A, on which a Traefik instance runs to distribute the traffic to B, on which our application runs.

To make this happen, we have to solve the following problems when it comes to networking:

Make sure incoming traffic from clients goes to A.We can configure our DNS records to point our domain to A.

This can be done with a simple A record. And in the case of IPV6, an AAAA record. (I know, the name is silly.)
Make sure traffic arriving at A is correctly forwarded to B.We can make use of Traefik.

To make it work, we have to make sure B is discoverable by Traefik. This means that we need to put A and B on the same network so they have the possibility to talk to each other. We also need to make sure B comes with the correct docker flags to tell Traefik that it’s providing a service and requires Traefik to provide routing support.
Make sure B correctly advertise it to Traefik as providing services.We can add the following lines to B’s docker-compose.yml:

- "traefik.enable=true" tells Traefik this is a service.
- "traefik.http.routers.api.rule=Host(``)" tells Traefik how to route the incoming requests. In this case, we instruct Traefik to forward all requests with a host value to B.
Make sure A and B are on the same network.This is the point at which I’m stuck. ๐Ÿ˜•

The Theory

With Traefik running on A, and applications running on B, it seems natural that we need to have an overlay network for Traefik and applications to attach to, such that they can talk to each other.

We can easily create a new network on the swarm manager node (you cannot create a new network on a regular swarm node).

$ docker network create --driver overlay --attachable swarm-net

Here we used the --driver option to specify that we want an overlay network, otherwise it will defaults to a bridged network. And the --attachable option informs Docker that we intend to attach containers to this network. Without the explicit --attachable option, you cannot attach containers. Finally we name the network swarm-net.

Then to make use of the newly created network, we modify the docker-compose.yml files for our services to point network to swarm-net.

All seem good. We created an overlay network, which in theory should allow services on different swarm nodes to talk to each other.

And launching the services indeed went smoothly. But there is only one minor problem: our Traefik instance is not discovering any of the applications on B.


Notes on Traefik via Docker

It works!

Here are some profound knowledge I’ve gathered so far:

  1. Using Traefik to route its own traffic is absolutely workable, you just need to:
    • make sure Traefik itself is discoverable by Traefik, and
    • setup certificate using tls.certresolver, and
    • setup basic auth if insecure mode is not enabled, it seems to be required.
  2. Understand that a 404 error most likely means the docker-compose file is somehow incorrect, or is missing tags required by Traefik.
  3. Make sure spellings are correct. A misspelled file name or middleware name will not cause docker-compose to die.
  4. Watch for indentation errors in your .yml files. The way docker-compose complain about a malformed .yml file is not very straight-forward. But at least it fails immediately when this does happen.
  5. You must use htpasswd to generate your basic auth passwords. To install it, it is under the apache2-utils package for apt.
  6. You have to either docker-compose up or docker-compose create before some changes to the docker-compose file will take effect.
  7. For each subdomain needing a certificate, you must have a corresponding A record for that subdomain, if you are using --certificatesresolvers.mytlschallenge.acme.tlschallenge=true.
    • there’s gotta be a way to put a wildcard cert in the certificate store and make Traefik use that