So following on from my post about the kube-exploit, I thought it would be interesting to look more at the attack surface of my sample Kubernetes cluster from the perspective of a Rogue container. The setup follows the same path as the last post and I’m running from a kali linux container running on my cluster, to simulate an attacker who has compromised a single container on a cluster.

So first obvious thing to look at is the network attack surface. Open ports are a first option for an attacker who gets unauthorised access to a system.

This cluster has three nodes on 192.168.41.233 , 192.168.41.201 and 192.168.41.232 so we can start with

nmap -sT -n -p- 192.168.41.201,232,233

From that, we can see that there are some interesting ports to look at. The first one I noticed is 4194/TCP. On the cluster this is used by cAdvisor which provides metrics about your containers and is, by default, available without authentication.

This provides quite a bit of information about the configuration of the cluster like a process list

cadvisor process

and some details on the configuration of the docker daemon on the host

cadvisor docker

There’s also a set of handy API endpoints if you want to dump the information in JSON format. For example, to get the spec for all the containers running on a host you can just go to http://192.168.41.233:4194/api/v2.0/spec?recursive=true and get output like

    "/docker/0598e0682955545ef27486ce3c04d62b6e1dc15496fb8072c297f2b548e7e10f": {
        "creation_time": "2016-10-09T03:55:54.113949226+01:00",
        "aliases": [
            "k8s_weave-npc.27539310_weave-net-xf53p_kube-system_af83b2df-8683-11e6-849b-000c29d33879_f938e1de",
            "0598e0682955545ef27486ce3c04d62b6e1dc15496fb8072c297f2b548e7e10f"
        ],
        "namespace": "docker",
        "labels": {
            "io.kubernetes.container.hash": "27539310",
            "io.kubernetes.container.name": "weave-npc",
            "io.kubernetes.container.restartCount": "0",
            "io.kubernetes.container.terminationMessagePath": "/dev/termination-log",
            "io.kubernetes.pod.name": "weave-net-xf53p",
            "io.kubernetes.pod.namespace": "kube-system",
            "io.kubernetes.pod.terminationGracePeriod": "30",
            "io.kubernetes.pod.uid": "af83b2df-8683-11e6-849b-000c29d33879"
        },
        "has_cpu": true,
        "cpu": {
            "limit": 10,
            "max_limit": 0,
            "mask": "0-1"
        },
        "has_memory": true,
        "memory": {
            "limit": 9223372036854771712,
            "reservation": 9223372036854771712
        },
        "has_custom_metrics": false,
        "has_network": false,
        "has_filesystem": true,
        "has_diskio": true,
        "image": "weaveworks/weave-npc:1.7.0"
    },

This isn’t as serious an issue as the kubelet exploit of course, but still something you’d want to change in your deployment of Kubernetes to harden it.

After noting this I had a look through the Kubernetes issue list and it looks like this is a known issue but unfortunately not one with a clear fix for now, so it’d need something like an iptables rule applied to restrict access to it.


raesene

Security Geek, Penetration Testing, Docker, Ruby, Hillwalking