- Print
- DarkLight
Kubernetes
You can also run LimaCharlie at the host level in a container cluster system like Kubernetes in order to monitor all running containers on the host with a single sensor. In fact, this is the prefered method as it reduces the overhead of running LC within every single container.
This privileged container has visibility into everything happening in the cluster and sends telemetry directly back to the LimaCharlie, in real-time, for processing.
This is accomplished by a combination of a few techniques:
- A privilged container running LC.
- LC runs with
HOST_FS
environment variable pointing to the host's root filesystem mounted within the container. - LC runs with the
NET_NS
environment variable pointing to the host's directory listing network namespaces. - Running the container with the required flags to make sure it can have proper access.
The first step is straight forward. For example, set the environment like ENV HOST_FS=/rootfs
and ENV NET_NS=/netns
as part of your Dockerfile
. This will let the LC sensor know where it can expect host-level information.
The second step is to run the container like: docker run --privileged --net=host -v /:/rootfs:ro --env HOST_FS=/rootfs -v /var/run/docker/netns:/netns:ro --env NET_NS=/netns --env LC_INSTALLATION_KEY=your_key your-lc-container-name
.
Remember to pick the approriate LC sensor architecture installer for the container that will be running LC (not the host).
So if your privileged container runs Alpine Linux, use the alpine64
version of LC.
A public version of the container described below is available from dockerhub as: refractionpoint/limacharlie_sensor:latest
.
##### Sample Configurations
This is a sample Dockerfile
you may use to run LC within a privileged container as described above:
# Requires an LC_INSTALLATION_KEY environment variable
# specifying the installation key value.
# Requires a HOST_FS environment variable that specifies where
# the host's root filesystem is mounted within the container
# like "/rootfs".
# Requires a NET_NS environment variable that specific where
# the host's namespaces directory is mounted within the container
# like "/netns".
# Example:
# export ENV HOST_FS=/rootfs
# docker run --privileged --net=host -v /:/rootfs:ro -v /var/run/docker/netns:/netns:ro --env HOST_FS=/rootfs --env NET_NS=/netns --env LC_INSTALLATION_KEY=your_key refractionpoint/limacharlie_sensor
FROM alpine
RUN mkdir lc
WORKDIR /lc
RUN wget https://downloads.limacharlie.io/sensor/linux/alpine64 -O lc_sensor
RUN chmod 500 ./lc_sensor
CMD ./lc_sensor -d -
And this is a sample Kubernetes deployment
on
a cluster supporting eBPF (kernel > 5.7):
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: lc-sensor
namespace: lc-monitoring
labels:
app: lc-monitoring
spec:
minReadySeconds: 30
updateStrategy:
type: RollingUpdate
rollingUpdate:
maxUnavailable: 1
selector:
matchLabels:
app: lc-monitoring
template:
metadata:
namespace: lc-monitoring
labels:
app: lc-monitoring
spec:
hostNetwork: true
hostPID: true
containers:
- name: lc-sensor
image: refractionpoint/limacharlie_sensor:latest
imagePullPolicy: IfNotPresent
securityContext:
allowPrivilegeEscalation: true
privileged: true
capabilities:
add: ['CAP_SYS_ADMIN']
resources:
requests:
memory: 128M
cpu: 0.01
limits:
memory: 256M
cpu: 0.9
volumeMounts:
- mountPath: /rootfs
name: all-host
- mountPath: /netns
name: all-host-ns
- mountPath: /sys/kernel/debug
name: all-host-krnl
- mountPath: /sys/kernel/btf
name: btf
- mountPath: /lib/modules
name: libmodules
env:
- name: HOST_FS
value: /rootfs
- name: NET_NS
value: /netns
- name: LC_INSTALLATION_KEY
value: <<<< YOUR INSTALLATION KEY GOES HERE >>>>
volumes:
- name: all-host
hostPath:
path: /
- name: all-host-ns
hostPath:
path: /var/run/docker/netns
- name: all-host-krnl
hostPath:
path: /sys/kernel/debug
- name: btf
hostPath:
path: /sys/kernel/btf
- name: libmodules
hostPath:
path: /lib/modules
a cluster not supporting eBPF (kernel < 5.7):
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: lc-sensor
namespace: lc-monitoring
labels:
app: lc-monitoring
spec:
minReadySeconds: 30
updateStrategy:
type: RollingUpdate
rollingUpdate:
maxUnavailable: 1
selector:
matchLabels:
app: lc-monitoring
template:
metadata:
namespace: lc-monitoring
labels:
app: lc-monitoring
spec:
containers:
- name: lc-sensor
image: refractionpoint/limacharlie_sensor:latest
imagePullPolicy: IfNotPresent
securityContext:
allowPrivilegeEscalation: true
privileged: true
resources:
requests:
memory: 128M
cpu: 0.01
limits:
memory: 256M
cpu: 0.9
volumeMounts:
- mountPath: /rootfs
name: all-host-fs
- mountPath: /netns
name: all-host-ns
env:
- name: HOST_FS
value: /rootfs
- name: NET_NS
value: /netns
- name: LC_INSTALLATION_KEY
value: <<<< YOUR INSTALLATION KEY GOES HERE >>>>
volumes:
- name: all-host-fs
hostPath:
path: /
- name: all-host-ns
hostPath:
path: /var/run/docker/netns
hostNetwork: true