Skip to content

Linux Root-Level Changes

This document summarizes the system-level changes made to a Linux host when installing a NetBox Enterprise Embedded Cluster, particularly in relation to directories, files, and runtime configurations affected under /.

Key Directories Created

During installation, the Embedded Cluster creates several directories for container runtimes, network configuration, logging, and Kubernetes internals. These directories may be used for binary placement, socket communication, runtime data, or logging.

Path Description
/etc/cni CNI (Container Network Interface) configuration
/etc/k0s Configuration for the k0s Kubernetes distribution
/opt/cni Optional location for CNI plugin binaries
/opt/containerd Data and binaries for containerd runtime
/run/calico Calico's runtime data and state
/run/containerd Runtime socket and process management for containerd
/run/k0s Runtime files and PID info for k0s
/sys/fs/cgroup/kubepods cgroup hierarchy for Kubernetes pods
/sys/fs/cgroup/system.slice/containerd.service cgroup slice for the containerd system service
/sys/fs/cgroup/system.slice/k0scontroller.service cgroup slice for the k0scontroller service
/usr/libexec/k0s Executables used by k0s, potentially for internal service components
/usr/local/bin/k0s The k0s binary, typically symlinked or copied here
/var/lib/calico Persistent data for the Calico network plugin
/var/lib/cni CNI state and cache
/var/lib/containers OCI container storage (may contain container images and layers)
/var/lib/kubelet State for the kubelet, pod manifests, volumes, etc.
/var/log/calico Logs generated by Calico agents
/var/log/containers Logs for individual containers, typically symlinked from journald
/var/log/embedded-cluster Logs specific to Embedded Cluster installation and runtime
/var/log/pods Logs organized by pod name/namespace

Runtime & Kernel Modifications

cgroups are used extensively to isolate resources:

  • kubepods cgroup under /sys/fs/cgroup
  • Systemd slices for services like containerd and k0scontroller
  • These modifications may impact systems with strict lockdowns or non-standard cgroup hierarchies.

Binaries and Services

  • The k0s binary is located at /usr/local/bin/k0s.
  • Services for k0scontroller, containerd, and possibly calico will be active post-install.
  • Services will install unit files in /etc/systemd/system/.

Firewall Requirements

To allow proper operation of the Embedded Cluster, the following network ports must be open:

Port Protocol Purpose
6443 TCP Kubernetes API server (used by kubelets, kubectl, etc.)
2379-2380 TCP etcd (Kubernetes backing store – internal traffic)
10250 TCP Kubelet API for health checks and logs
4789 UDP VXLAN overlay (used by Calico)
8472 UDP VXLAN (alternative port for Calico)
51820 UDP WireGuard (if enabled for Calico networking)
179 TCP BGP (used if Calico is in BGP mode)
443 TCP NetBox Enterprise UI and API access
8800 TCP Admin Console (legacy or diagnostic use)

Note: Ingress/egress rules must permit internal pod-to-pod communication. For systems in restricted environments, please refer to the Special Cases for Restricted Environments section in the requirements documentation. This provides detailed guidance on configuring firewalls and network policies to ensure proper operation of the Embedded Cluster.