Revision 95c1e72d7640528ee7013d414168e98b155424bc authored by Martynas Pumputis on 21 July 2023, 12:45:19 UTC, committed by Martynas Pumputis on 21 July 2023, 12:45:19 UTC
Signed-off-by: Martynas Pumputis <m@lambda.lt>
1 parent 8053488
per-node-config.rst
.. only:: not (epub or latex or html)
WARNING: You are looking at unreleased Cilium documentation.
Please use the official rendered version released here:
https://docs.cilium.io
.. _per-node-configuration:
**********************
Per-node configuration
**********************
.. include:: ../beta.rst
The Cilium agent process (a.k.a. DaemonSet) supports setting configuration
on a per-node basis. This allows overriding :ref:`cilium-config-configmap`
for a node or set of nodes. It is managed by CiliumNodeConfig objects.
This feature is useful for:
- Gradually rolling out changes.
- Selectively enabling features that require specific hardware:
* :ref:`XDP acceleration`
* :ref:`ipv6_big_tcp`
CiliumNodeConfig objects
------------------------
A CiliumNodeConfig object allows for overriding ConfigMap / Agent arguments.
It consists of a set of fields and a label selector. The label selector
defines to which nodes the configuration applies. As is the standard with
Kubernetes, an empty LabelSelector (e.g. ``{}``) selects all nodes.
.. note::
Creating or modifying a CiliumNodeConfig will not cause changes to take effect
until pods are deleted and re-created (or their node is restarted).
Example: selective XDP enablement
---------------------------------
To enable :ref:`XDP acceleration` only on nodes with necessary
hardware, one would label the relevant nodes and override their configuration.
.. code-block:: yaml
apiVersion: cilium.io/v2alpha1
kind: CiliumNodeConfig
metadata:
namespace: kube-system
name: enable-xdp
spec:
nodeSelector:
matchLabels:
io.cilium.xdp-offload: "true"
defaults:
bpf-lb-acceleration: native
Example: KubeProxyReplacement Rollout
-------------------------------------
To roll out :ref:`kube-proxy replacement <kubeproxy-free>` in a gradual manner,
you may also wish to use the CiliumNodeConfig feature. This will label all migrated
nodes with ``io.cilium.migration/kube-proxy-replacement: strict``
.. warning::
You must have installed Cilium with the Helm values ``k8sServiceHost`` and
``k8sServicePort``. Otherwise Cilium will not be able to reach the Kubernetes
APIServer after kube-proxy is uninstalled.
You can apply these two values to a running cluster via ``helm upgrade``.
#. Patch kube-proxy to only run on unmigrated nodes.
.. code-block:: shell-session
kubectl -n kube-system patch daemonset kube-proxy --patch '{"spec": {"template": {"spec": {"affinity": {"nodeAffinity": {"requiredDuringSchedulingIgnoredDuringExecution": {"nodeSelectorTerms": [{"matchExpressions": [{"key": "io.cilium.migration/kube-proxy-replacement", "operator": "NotIn", "values": ["strict"]}]}]}}}}}}}'
#. Configure Cilium to use strict kube-proxy on migrated nodes
.. code-block:: shell-session
cat <<EOF | kubectl apply --server-side -f -
apiVersion: cilium.io/v2alpha1
kind: CiliumNodeConfig
metadata:
namespace: kube-system
name: kube-proxy-replacement-strict
spec:
nodeSelector:
matchLabels:
io.cilium.migration/kube-proxy-replacement: strict
defaults:
kube-proxy-replacement: strict
kube-proxy-replacement-healthz-bind-address: "0.0.0.0:10256"
EOF
#. Select a node to migrate. Optionally, cordon and drain that node:
.. code-block:: shell-session
export NODE=kind-worker
kubectl label node $NODE --overwrite 'io.cilium.migration/kube-proxy-replacement=strict'
kubectl cordon $NODE
#. Delete Cilium DaemonSet to reload configuration:
.. code-block:: shell-session
kubectl -n kube-system delete pod -l k8s-app=cilium --field-selector spec.nodeName=$NODE
#. Ensure Cilium has the correct configuration:
.. code-block:: shell-session
kubectl -n kube-system exec $(kubectl -n kube-system get pod -l k8s-app=cilium --field-selector spec.nodeName=$NODE -o name) -c cilium-agent -- \
cilium config get kube-proxy-replacement
strict
#. Uncordon node
.. code-block:: shell-session
kubectl uncordon $NODE
#. Cleanup: set default to kube-proxy-replacement:
.. code-block:: shell-session
cilium config set --restart=false kube-proxy-replacement strict
cilium config set --restart=false kube-proxy-replacement-healthz-bind-address "0.0.0.0:10256"
kubectl -n kube-system delete ciliumnodeconfig kube-proxy-replacement-strict
#. Cleanup: delete kube-proxy daemonset, unlabel nodes
.. code-block:: shell-session
kubectl -n kube-system delete daemonset kube-proxy
kubectl label node --all --overwrite 'io.cilium.migration/kube-proxy-replacement-'
![swh spinner](/static/img/swh-spinner.gif)
Computing file changes ...