Friday, April 1, 2016

Multihost Docker Networking

One of the major issues that people have with running docker is the fact that docker and containers natively only support localhost networking capabilities. So this means that by default, only the localhost and its services know about the containers. Host A will not be able to communicate with Host B by default. Here is a quick demo of how to use CoreOS's networking project Flannel, so that you can have a multi-host docker environment where all the hosts and their containers can commuincate.
More info on Flannel: https://github.com/coreos/flannel
More info on ETCD: https://github.com/coreos/etcd

You will need to have 1 or more etcd server(s). We will be using a single node for this demo.

On etcd Server(s)
1. Install etcd
2. Configure etcd - /etc/etcd/etcd.conf
    
    # cat /etc/etcd/etcd.conf | grep -v '^#'
    ETCD_NAME=default
    ETCD_DATA_DIR="/var/lib/etcd/default.etcd"
    ETCD_LISTEN_PEER_URLS="http://0.0.0.0:7001,http://localhost:2380"
    ETCD_LISTEN_CLIENT_URLS="http://0.0.0.0:2379"
    ETCD_ADVERTISE_CLIENT_URLS="http://localhost:2379"

3. Enable and start etcd 
    # systemctl enable etcd && systemctl start etcd

4. Define etcd network:
etcdctl mk /blah.com/network/config '{"Network":"172.17.0.0/16"}'

You should be able to get the json for that key.
    # curl -s -L http://ETCD_SERVER:2379/v2/keys/blah.com/network/config | python -m json.tool



On worker/slave/client(s)… any machine that is going to have flannel running
1.  Install flannel
2. Configure flannel - /etc/sysconfig/flanneld
    # cat /etc/sysconfig/flanneld | grep -v '^#'
    FLANNEL_ETCD="http://ETCD_SERVER:2379"
    FLANNEL_ETCD_KEY="/blah.com/network"

3. If Docker is already installed, stop and remove docker interface.
        systemctl stop docker
    ip link delete docker0
    systemctl start flanneld && systemctl enable flanneld
    systemctl start docker

Rinse and repeat for all other desired docker hosts: 

You should be able to see the networking configs and subnets being created.

# curl -s -L http://master:2379/v2/keys/blah.com/network/config | python -m json.tool

# curl -s -L http://master:2379/v2/keys/blah.com/network/subnets | python -m json.tool


You should now be able to ping containers between different hosts!


After starting a few containers on different hosts, try it out. Each host gets its own subnet if you notice below.

Host A:
# docker inspect f41cd57b4ef5 | grep -i ipaddress
        "IPAddress": "172.17.24.3",
        "SecondaryIPAddresses": null,

Host B:
# docker inspect 1b5b48c6be47 | grep -i ipaddress
        "IPAddress": "172.17.80.16",
        "SecondaryIPAddresses": null,


From Host A container, ping container 1b5b48c6be47 on Host B:
# docker exec -it f41cd57b4ef5 ping 172.17.80.16
PING 172.17.80.16 (172.17.80.16): 56 data bytes
64 bytes from 172.17.80.16: seq=0 ttl=62 time=2.336 ms
64 bytes from 172.17.80.16: seq=1 ttl=62 time=0.438 ms
64 bytes from 172.17.80.16: seq=2 ttl=62 time=0.506 ms
^C

Host A is even able to ping container on Host B:
# ping 172.17.80.16
PING 172.17.80.16 (172.17.80.16) 56(84) bytes of data.
64 bytes from 172.17.80.16: icmp_seq=1 ttl=63 time=0.386 ms
64 bytes from 172.17.80.16: icmp_seq=2 ttl=63 time=0.438 ms
^C


Host B is running a nginx container at port 80. Lets curl port 80 on that container from a container on Host A. (will only work if curl installed on container):
# docker exec -it f41cd57b4ef5 curl 172.17.80.16:80
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
    body {
        width: 35em;
        margin: 0 auto;
        font-family: Tahoma, Verdana, Arial, sans-serif;
    }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>
100   612  100   612    0     0   321k      0 --:--:-- --:--:-- --:--:--  597k

And from Host A:
# curl 172.17.80.16:80
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
    body {
        width: 35em;
        margin: 0 auto;
        font-family: Tahoma, Verdana, Arial, sans-serif;
    }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>

No comments:

Post a Comment