sort by:
Revision Author Date Message Commit Date
5375256 nodediscovery: explicit dependency to k8sNodeWatcher Currently, the k8sWatcher is set as dependency on the nodeDiscovery during agent initialization by using the method `RegisterK8sSetters`. Trying to add an explicit dependency from the NodeDiscovery to the `K8sWatcher` results in a cyclic dependency via datapath. With the modularization of the k8sWatcher into smaller cells, it's possible to define the explicit dependency only to the `k8sCiliumNodeWatcher`, as this is the only part the NodeDiscovery is intersted in. This way, there's no cyclic dependency. Signed-off-by: Marco Hofstetter <marco.hofstetter@isovalent.com> 12 June 2024, 11:47:59 UTC
0f586d0 k8s: move init test to new watcher_test.go This commit extracts the k8sWatcher related unit test into it's own file `watcher_test.go`. (Separate commit to keep the git history). Signed-off-by: Marco Hofstetter <marco.hofstetter@isovalent.com> 12 June 2024, 11:47:59 UTC
e44d411 k8s: rename watcher_test.go to service_test.go Currently, the file `watcher_test.go` mostly contains service related unit tests. Therefore, the file gets renamed to `service_test.go`. An upcoming commit will extract the only K8sWatcher related test into `watcher_test.go`. Signed-off-by: Marco Hofstetter <marco.hofstetter@isovalent.com> 12 June 2024, 11:47:59 UTC
4780989 k8s: remove k8sSvcCache from k8swatcher and use directly as daemon dep Currently, during daemon initialization, multiple components access the k8sSvcCache through the corresponding exported field in the k8sWatcher. This commit removes the field from the k8swatcher and forces the daemon to depend on the `k8sSvcCache` directly. In addition, some tests of the k8sWatcher would have been freed up from using the k8sWatcher at all, as they were only testing service logic. Signed-off-by: Marco Hofstetter <marco.hofstetter@isovalent.com> 12 June 2024, 11:47:59 UTC
da23ff4 k8s: extract k8sCiliumEndpointsWatcher Currently, all the k8s watchers of the `k8sWatcher` are defined in the same struct, have access to all the same dependency fields and are provided as one Cell. This commit extracts the k8s CiliumEndpoints watcher into it's own sub-cell that is provided privately. Signed-off-by: Marco Hofstetter <marco.hofstetter@isovalent.com> 12 June 2024, 11:47:59 UTC
62e214b k8s: extract k8sCiliumLRPWatcher Currently, all the k8s watchers of the `k8sWatcher` are defined in the same struct, have access to all the same dependency fields and are provided as one Cell. This commit extracts the k8s CiliumLRP watcher into it's own sub-cell that is provided privately. Signed-off-by: Marco Hofstetter <marco.hofstetter@isovalent.com> 12 June 2024, 11:47:59 UTC
0c8ab4f k8s: extract k8sEndpointsManager Currently, all the k8s watchers of the `k8sWatcher` are defined in the same struct, have access to all the same dependency fields and are provided as one Cell. This commit extracts the k8s Endpoints watcher into it's own sub-cell that is provided privately. Signed-off-by: Marco Hofstetter <marco.hofstetter@isovalent.com> 12 June 2024, 11:47:59 UTC
5249766 k8s: extract k8sServiceManager Currently, all the k8s watchers of the `k8sWatcher` are defined in the same struct, have access to all the same dependency fields and are provided as one Cell. This commit extracts the k8s Service watcher into it's own sub-cell that is provided privately. Signed-off-by: Marco Hofstetter <marco.hofstetter@isovalent.com> 12 June 2024, 11:47:59 UTC
35bbd26 k8s: extract k8sNamespaceWatcher Currently, all the k8s watchers of the `k8sWatcher` are defined in the same struct, have access to all the same dependency fields and are provided as one Cell. This commit extracts the k8s Namespace watcher into it's own sub-cell that is provided privately. Signed-off-by: Marco Hofstetter <marco.hofstetter@isovalent.com> 12 June 2024, 11:47:59 UTC
7b44b07 k8s: extract k8sCiliumNodeWatcher Currently, all the k8s watchers of the `k8sWatcher` are defined in the same struct, have access to all the same dependency fields and are provided as one Cell. This commit extracts the k8s CiliumNode watcher into it's own sub-cell that is provided privately. Signed-off-by: Marco Hofstetter <marco.hofstetter@isovalent.com> 12 June 2024, 11:47:59 UTC
de58c84 k8s: extract k8sPodWatcher Currently, all the k8s watchers of the `k8sWatcher` are defined in the same struct, have access to all the same dependency fields and are provided as one Cell. This commit extracts the k8s Pod watcher into it's own sub-cell that is provided privately. Signed-off-by: Marco Hofstetter <marco.hofstetter@isovalent.com> 12 June 2024, 11:47:59 UTC
914063a k8s: extract k8sEventReporter Currently, k8s event reporting is part of the k8sWatcher. It's used by sub-watchers of the k8swatcher itself, but also by external watchers (e.g. IPAM watcher). As a first step to further modularize the k8swatcher into its smaller components, the k89s event reporting is extracted into an own cell and struct `k8sEventReporter`. This way, other components can depend on it. Signed-off-by: Marco Hofstetter <marco.hofstetter@isovalent.com> 12 June 2024, 11:47:59 UTC
a7c3744 k8s: introduce k8s watcher cell Currently, the k8swatcher is initialized in the daemon bootstrap function `newDaemon`. With the modularization of all its dependencies into their own Hive Cell, it's about time to move the initialization of the k8sWatcher into its own Hive Cell too. In a first step, the cell only provides the pre-initialized struct, without moving any of the lifecycle aspects into the Cell. For the time being, these are being kept in the daemon initialization. Signed-off-by: Marco Hofstetter <marco.hofstetter@isovalent.com> 12 June 2024, 11:47:59 UTC
973d540 envoy: Remove un-necessary warning log filtering Relates: https://github.com/cilium/cilium/pull/31108 Relates: https://github.com/envoyproxy/envoy/pull/30735 Signed-off-by: Tam Mach <tam.mach@cilium.io> 12 June 2024, 09:58:44 UTC
ca81c9c bpf: host: use security identities in to-netdev's trace notifications For some types of traffic, to-netdev derives precise security identities. Consistently use these values in the trace notifications. Signed-off-by: Julian Wiedmann <jwi@isovalent.com> 12 June 2024, 08:20:41 UTC
65e93a2 ci: add tests for migration to CiliumEndpointSlice This commit adds CI to test that the migration from CiliumEndpoint to CiliumEndpointSlice does not disturb long-lived connections. A Kind cluster is set up without CiliumEndpointSlice enabled. Long-lived connections are set up. Then, CES is enabled, the operator is restarted and then the agent, after the CES CRD is created. Then, the connectivity test is run to ensure long-lived connections were not broken. Signed-off-by: jshr-w <shjayaraman@microsoft.com> 12 June 2024, 08:17:42 UTC
811cb7f make: Add include to Makefile.override within binary-specific makefiles make: Add include to Makefile.override in binary Makefiles This commit adds an include statement for Makefile.override in Makefiles specific to building Cilium's go binaries. Makefile.override is included in the top-level Makefile as a method for optionally overriding variables, however it is not included in any of these binary-specific Makefiles. This means that the ability to override variables is only available for targets in the top-level Makefile, preventing use cases where overriding variables used in these binary-specific Makefiles can be useful. As an example, this commit would allow one to override the GO variable to specify a specific go binary to use in order to build a target. Signed-off-by: Ryan Drew <ryan.drew@isovalent.com> 12 June 2024, 08:17:30 UTC
9cfa1a2 make, docker: Add ADDITIONAL_MODIFIERS environment variable This commit adds a new environment variable to the docker-specific aspects of the Cilium Makefiles named `ADDITIONAL_MODIFIERS`. This environment variable can be used to modify the `MODIFIERS` docker build arg, adding in any extra values that haven't previously been specified via a preset, such as `RACE` or `NOSTRIP`. Signed-off-by: Ryan Drew <ryan.drew@isovalent.com> 12 June 2024, 08:17:30 UTC
c4aebae docker, ci: Create generalized MODIFIERS build arg This commit replaces the NOSTRIP, NOOPT, LOCKDEBUG, RACE, V and LIBNETWORK_PLUGIN docker build args with a single, generic build arg named "MODIFIERS". This allows for arbitrary flags to be passed to make when building a docker image as well as removes the need for modifications to dockerfiles when a new build-time modifier is added. One example use case is using `Makefile.overrides` to define a new flag that can be passed to make when building docker images. The new flag could enable appending values to the MODIFIERS build argument, which would allow the propagation of configuration variables down to make invocations used to build binaries within a Dockerfile. Signed-off-by: Ryan Drew <ryan.drew@isovalent.com> 12 June 2024, 08:17:30 UTC
9334d97 l2-discovery: fix health reporting for link updater As-is, when l2 neighbor discovery is enabled, the node-neighbor-link-updater controller fails with "invalid node spec found in queue". This is due to a bug in the controller's DoFunc, where an empty list is treated the same as an invalid queue entry. When this controller fails, `cilium status` reports errors for all nodes in the cluster similar to the following: ``` cilium cilium-mgstt controller node-neighbor-link-updater is failing since 21s (49x): invalid node spec found in queue: (*manager.nodeQueueEntry)(nil) ``` To differentiate between an empty queue and a nil item, the queue's `pop` method now also returns a bool to indicate whether an element was successfully retrieved from the queue. Fixes: #8d525fe Signed-off-by: Tim Horner <timothy.horner@isovalent.com> 12 June 2024, 08:17:06 UTC
22b3e82 bgpv2: Allow empty advertisement Remove unnecessary restriction. Signed-off-by: Yutaro Hayakawa <yutaro.hayakawa@isovalent.com> 12 June 2024, 08:08:24 UTC
26325a8 docs: ipsec: mention dependency on transparent mode for DNS proxy For connections that are established by the DNS proxy, this is required to detect the original source IP and apply IPsec policy accordingly. The agent fatals if IPsec and L7 proxy are enabled, but the DNS proxy is not set to transparent mode. Signed-off-by: Julian Wiedmann <jwi@isovalent.com> 12 June 2024, 05:14:37 UTC
9c7bd8a gha: bump status wait timeouts in clustermesh upgrade/downgrade tests The blamed commit already increased the post-upgrade timeout. However, we have now started witnessing failures in the other wait operations as well, due to endpoint regeneration not completing on time. Hence. let's bump all timeouts to 10m. Related: 01c3b8376046 ("gha: bump post-upgrade timeout in clustermesh upgrade/downgrade tests") Signed-off-by: Marco Iorio <marco.iorio@isovalent.com> 12 June 2024, 01:53:58 UTC
a57393f README: Update releases Signed-off-by: Quentin Monnet <qmo@qmon.net> 11 June 2024, 20:45:44 UTC
a1d0307 images: update cilium-{runtime,builder} Signed-off-by: Cilium Imagebot <noreply@cilium.io> 11 June 2024, 20:08:20 UTC
8afe844 images: Fix copyo mistake in error message This error message was copied from the equivalent runtime script. Fix it. Signed-off-by: Joe Stringer <joe@cilium.io> 11 June 2024, 20:08:20 UTC
f639135 .github: Regenerate api/v1 when updating builder The builder image contains the 'protoc' binary which can generate different API files when it's updated, notably because protoc decides to encode its own version into the files it outputs. Add a step in the builder image update workflow to update the api/v1 files. Signed-off-by: Joe Stringer <joe@cilium.io> 11 June 2024, 20:08:20 UTC
a37eaad ci: Enable LRP connectivity tests Signed-off-by: Aditi Ghag <aditi@cilium.io> 11 June 2024, 16:34:05 UTC
478e637 bpf: Disable conflicting per packet LB Per-packet LB is disabled in certain cases like when socket-LB is enabled, and load-balancing is handled in bpf_sock. However, there are other features (e.g., L7 LB) that require per-packet LB. This can conflict with processing local-redirect services in some cases. Based on user configured local redirect policies, load-balancing can be skipped for certain local-redirect services. More specifically, LB is skipped in some cases when users deploy LRPs with skipRedirectFromBackend flag. Per packet LB should not override LB decisions made for local-redirect services in bpf_sock. Signed-off-by: Aditi Ghag <aditi@cilium.io> 11 June 2024, 16:34:05 UTC
961820e docs: Promote local redirect policy feature to stable Signed-off-by: Aditi Ghag <aditi@cilium.io> 11 June 2024, 15:05:14 UTC
4a3b6c8 bgpv2: Remove node selector check from v2 PodCIDRReconciler Remove unnecessary CiliumNode label selector check for PodCIDR advertisements. This was reflected from the BGPv1 code, but for BGPv2 we would like to avoid it, as this behavior is inconsistent with other advertisement types (other advertisement types advertise the paths for selected resources, but PodCIDR only applies to the local node). Signed-off-by: Rastislav Szabo <rastislav.szabo@isovalent.com> 11 June 2024, 13:44:33 UTC
085343b docs: add upgrade note about the slightly different dialer behavior The port specified as part of the kvstore address is now respected also when the address matches a Kubernetes service, to prevent inconsistencies if the service includes multiple ports. Additionally, mention that the etcd.operator option is no longer required, and has been removed. Signed-off-by: Marco Iorio <marco.iorio@isovalent.com> 11 June 2024, 13:43:14 UTC
27e425c k8s: remove the now unused TransformToK8sService helper Signed-off-by: Marco Iorio <marco.iorio@isovalent.com> 11 June 2024, 13:43:14 UTC
6e763ac kvstore: remove the now unused IsEtcdOperator,SplitK8sServiceURL funcs Additionally drop the etcd.operator kvstore option, which is no longer required as the service resolver logic is now always enabled. Signed-off-by: Marco Iorio <marco.iorio@isovalent.com> 11 June 2024, 13:43:14 UTC
28a7e82 service: drop the legacy and now unused custom dialer All usages have been converted over to the generic implementation in the previous commits. Signed-off-by: Marco Iorio <marco.iorio@isovalent.com> 11 June 2024, 13:43:14 UTC
0bd5200 cilium-dbg: use newly introduced custom dialer in troubleshoot commands Let's uniform the troubleshoot commands to also use the generic custom dialer implementation, and cleanup the existing hacks. We stick to the existing implementation and don't use the service resolver in this case, instead, to avoid starting an informer from a CLI tool. Signed-off-by: Marco Iorio <marco.iorio@isovalent.com> 11 June 2024, 13:43:14 UTC
5560cfb operator: use newly introduced custom dialer and resolver for etcd Similarly as for the Cilium agent, let's migrate the operator to use the newly introduced dialer and service resolver for etcd, and untangle it from the SyncK8sServices option, so that it can be turned off independently for performance reasons when not necessary (i.e., if clustermesh is not used). Signed-off-by: Marco Iorio <marco.iorio@isovalent.com> 11 June 2024, 13:43:14 UTC
d57a922 daemon: use newly introduced custom dialer and resolver for etcd Migrate the Cilium agent to use the newly introduced generic custom dialer and service resolver for etcd, to decouple the custom dialer logic from the service cache. In an effort to simplify the logic, the dialer is always registered (i.e., without performing the kvstore.IsEtcdOperator check), as the dialer is transparent if not matching a service name. Similarly, we don't explicitly wait for cache synchronization, as that's already automatically performed by the resolver to retrieve the service store. Additionally, in case the timeout expires, the etcd client would simply retry connecting again, eventually succeeding. Signed-off-by: Marco Iorio <marco.iorio@isovalent.com> 11 June 2024, 13:43:14 UTC
9db5384 clustermesh: switch to newly introduced custom dialer and resolver Migrate the clustermesh cells, both in the agent and in the operator (for endpointslice synchronization) to use the newly introduced generic custom dialer and service resolver. Signed-off-by: Marco Iorio <marco.iorio@isovalent.com> 11 June 2024, 13:43:14 UTC
0e8c5a6 agent: introduce service resolver to map svc DNS name to ClusterIP Let's introduce a new cell which provides the resolver logic to map DNS names matching Kubernetes services to the corresponding ClusterIP address. It is backed by a lazy resource.Store, which is started only upon the first translation request for a service DNS name (i.e., either matching name.namespace, or name.namespace.svc[.other]). Overall, it is a generalized version to replace the already existing approaches spread across the codebase, and in particular: * the reliance upon the ServiceCache, which in certain circumstances may not be available (e.g., in the operator); * the similar approach already leveraged in the clustermesh/epslicesync package, which is more naive, and doesn't support lazy startup. Signed-off-by: Marco Iorio <marco.iorio@isovalent.com> 11 June 2024, 13:43:14 UTC
7a46fd4 agent: introduce new generic context dialer with resolvers support It allows to register a set of resolvers to translate the target hostname into the corresponding IP address, or possibly another alias DNS name. The dialer eventually calls (&net.Dialer).DialContext with the first successfully translated address, or the original one otherwise (ports are never modified) It's main purpose is to be used as a DialOption for etcd, and resolve DNS names representing k8s services to the corresponding ClusterIP without depending on CoreDNS. Overall, it represents a generic version of and aims to replace the already existing k8s.CreateCustomDialer utility. Signed-off-by: Marco Iorio <marco.iorio@isovalent.com> 11 June 2024, 13:43:14 UTC
0a634bf CODEOWNERS: Move devcontainer to cilium/ci When updating the builder image, this file gets updated, then pulls in @cilium/contributing as a codeowner. Move it over to cilium/ci to reduce the number of touchpoints for builder update points. Signed-off-by: Joe Stringer <joe@cilium.io> 11 June 2024, 12:23:44 UTC
6a1222d helm: directly leverage cilium.ca.setup for hubble certs generation Rather than using the intermediate hubble-generate-certs.helm.setup-ca, which performs the same steps. This brings consistency with the same operations performed for clustermesh-related certificates, and prevents divergences when generating/retrieving the CA certificate. Signed-off-by: Marco Iorio <marco.iorio@isovalent.com> 11 June 2024, 09:43:11 UTC
519d391 helm/certgen: use namespaced RBAC for hubble certs generation Convert the ClusterRole/ClusterRoleBinding to Role/RoleBinding to reduce the overall permissions considering that certgen only needs to access the secrets in the local namespace, based on the current configuration. This also aligns it with the equivalent permissions used for clustermesh. Signed-off-by: Marco Iorio <marco.iorio@isovalent.com> 11 June 2024, 09:41:51 UTC
11aa5e3 fix(deps): update all go dependencies main Signed-off-by: cilium-renovate[bot] <134692979+cilium-renovate[bot]@users.noreply.github.com> 11 June 2024, 09:13:00 UTC
28f308a doc: Listed L2LB LB class to LB IPAM doc Added the L2LB LoadBalancerClass `io.cilium/l2-announcer` to the LB IPAM documentation page. Signed-off-by: Philip Schmid <phisch@cisco.com> 11 June 2024, 09:00:04 UTC
4b1aba4 Remove etcd.managed Helm setting The etcd-operator Helm templates rely on a piece of software which is no longer maintained upstream, and it relies on outdated CRDs which are no longer supported since Kubernetes 1.22. The setting has been hidden and not documented for several releases, we can remove it now. Signed-off-by: Joe Stringer <joe@cilium.io> 11 June 2024, 08:58:00 UTC
f99f10b docs: Deprecate support for podnetwork etcd Running Etcd in podnetwork to distribute state between Cilium instances introduces a range of challenges to bootstrapping and ensuring reliable connectivity within the cluster. We've deprecated in-built support in the Helm charts for this sort of configuration for several releases, and documented suggested alternatives. If we deprecate this feature then we can simplify some of the operations inside the cilium-agent. For alternative installation steps, see https://docs.cilium.io/en/stable/installation/k8s-install-external-etcd/#admin-install-daemonset . Signed-off-by: Joe Stringer <joe@cilium.io> 11 June 2024, 08:56:45 UTC
e9d8122 renovate: prevent upgrading certgen to v0.2 in stable branches certgen v0.2 is going to introduce breaking changes. Hence, let's introduce a new renovate rule to prevent it from being upgraded in stable version. Signed-off-by: Marco Iorio <marco.iorio@isovalent.com> 11 June 2024, 08:02:42 UTC
96989f4 renovate: remove unnecessary etcd-related constraint This etcd-related constraint appears to have been added in the blamed commit. However, it doesn't seem intentional, considering that the latest etcd version is currently v3.5.14. Hence, let's just drop it. Fixes: b3d7d4d1dcd2 ("renovate: try to group dependency updates on single PR") Signed-off-by: Marco Iorio <marco.iorio@isovalent.com> 11 June 2024, 08:02:42 UTC
936f928 ci-e2e: Add the coverage for Ingress + bpf.masquerade Hopefully, this will help to catch some issues with Ingress. Signed-off-by: Tam Mach <tam.mach@cilium.io> 11 June 2024, 07:01:14 UTC
6947d82 maps: nat: remove rtp.log Looks like this was accidentally checked in by https://github.com/cilium/cilium/pull/32152. Signed-off-by: Julian Wiedmann <jwi@isovalent.com> 10 June 2024, 19:40:44 UTC
719eb4f fqdn: ToFQDN policy performance improvements This commit implements `CFP-28427: ToFQDN policy performance improvements`. It is highly recommended to consult the CFP, as it contains all the high-level design decisions and mechanism found in this commit. The rest of this commit message therefore only explains the "what" and "where", and not the "why". Before this commit, there was circular interaction between the `SelectorCache` and `NameManager`: `SelectorCache` would tell `NameManager` about new `ToFQDN` selectors, and `NameManager` would in turn inform `SelectorCache` about the IPs selected by that `ToFQDN` selector. This commit simplifies this logic by removing the backlink from the `NameManager` to the `SelectorCache`. IPs are instead now labelled with the selector as an `fqdn` identity label in IPCache, thus not requiring any direct changes to the `SelectorCache` when a new IP is discovered that shares the identity with an old IP. If there is identity allocation needed for an observed IP, the `SelectorCache` is still updated, but only via `IPCache`, and no longer directly from `NameManager`. I recommend first looking at the changes to `SelectorCache` in `pkg/policy`. Note the following changes: 1. The `identityNotifier` interface (implemented by `NameManger`) is simplified: We no longer care about IPs selected by a FQDN selector, and we no longer need to care about potential deadlocks, as there are no calls back from `NameManager` to `SelectorCache` in the invoked functions (the indirect backlink from `NameManager` to `SelectorCache` via `IPCache` happens in `NameManager.UpdateGenerateDNS` - but this function is called by the DNS proxy whenever it observes a new DNS lookup and thus is called without the selector cache lock held. 2. `UpdateFQDNSelector` (previously invoked by `NameManager`) is removed - `SelectorCache` no longer directly needs to know the IPs matched by a selector. 3. The `fqdnSelector` type is simplified: Instead of containing the list of CIDR identities (one for each selected IP) and checking for the CIDR identity in `matches`, we now can simply treat the FQDN selector as a label and thus check if the requested identity has the FQDN selector label. 4. All the unit test logic around managing the selected IPs is removed, as all the responsibility for updating IPs now lies in `NameManager`. For the `NameManager` in `pkg/fqdn`, the changes are as follows: 1. Minor changes to for the query functions in `DNSCache`: Instead of just listing or checking the existence of an IP, we now want to know about `(name, IP)` pairs (needed later for updating `IPCache`). 2. Similarly, where before we only cared about the mapping between an `FQDNSelector` and the selected IPs, we now want to know what `(name, IP)` pairs are matched by a particular selector. Thus `mapSelectorsToIPsLocked` is replaced with `mapSelectorsToNamesLocked` and the unit tests are updated as well. 3. `RegisterFQDNSelector` now checks if the new selector needs to be added to any known `(name, IP)` pairs as an `fqdn` label, and `UnregisterFQDNSelector` potentially removes `fqdn` labels from IPs. 4. `UpdateGenerateDNS` (invoked for DNS lookups) determines the labels of any newly discovered IP and now directly spawns the go routine to wait for the new `(IP, identity)` pair to be injected into `IPCache`. Previously, this waiting was done as part of the call to `UpdateSelectors`, previously implemented in `daemon/cmd/fqdn.go` (and now removed). 5. `ForceGenerateDNS` is removed. It was previously called by the `NameManager` GC to remove IPs from the `SelectorCache`, but since the `SelectorCache` no longer knows about IPs, the function is obsolete (note that `IPCache` removals are still performed upon GC) 6. Changes in `CompleteBootstrap` to deal with the upgrade logic when upgrading from Cilium v1.15. See bullet point 9 below for details. 7. `updateDNSIPs` (called from `UpdateGenerateDNS`, i.e. upon new DNS lookups) now determines the labels for every newly observed IP based on the available FQDN selectors, and no longer upserts CIDR identites. Note that we only update the labels matching the looked up `dnsName`. If an IP happens to also map to a different domain name and uses a different set of selectors for the alternative name, those labels in IPCache are unaffected by the call to `updateMetadata`, as every call to IPCache uses the DNS name as the resource owner. 8. The `ipcacheResource`, `updateMetadata`, and `maybeRemoveMetadata` contain the calls to `IPCache` to update labels for a given `(name, IP)` pair. There are two main differences to before: Instead of upserting or removing CIDR prefixes, we now add labels. And instead of having one update per prefix, we now have one update per `(name, IP)` pair, meaning a single prefix (aka "IP") might have multiple IPCache resource owners in the `NameManager` (i.e. one for each `name` mapping to that IP). 9. `RestoreCache` and `CompleteBootstrap` contain the logic to initialize `IPCache` when upgrading from Cilium v1.15. This requires the previous Cilium instance to have checkpointed the known `ToFQDN` selectors, which are read in during upgrade and used to derive and inject the `IPCache` labels we expect to have once endpoint regeneration has finished. After endpoint regeneration, those restored labels are then removed, leaving the real labels in place. In contrast to all other `IPCache` updates (where each update to an IP is "owned" by the DNS name mapping to that IP, and we rely on `IPCache` to merge those labels), the resource owner here is static. This is, because they are all added at once (in `RestoreCache`) and removed at once (in `CompleteBootstrap`), and no per-name tracking is required. 10. Various changes to unit tests. The old unit tests tested the interaction between `NameManager` and `SelectorCache`, where as the new unit tests now test the interaction between `NameManager` and `IPCache`. Signed-off-by: Sebastian Wicki <sebastian@isovalent.com> 10 June 2024, 16:06:10 UTC
625e39f fqdn: Derive domain labels from FQDN selectors This commit adds logic to derive identity labels for `(name, IP)` pairs from selectors. The basic idea is that any ToFQDN selector matching the qname of the DNS lookup is added to a label to each IP returned by that DNS lookup. The functions added here will be used in a subsequent commit. Signed-off-by: Sebastian Wicki <sebastian@isovalent.com> 10 June 2024, 16:06:10 UTC
dfc11ab daemon: Wait for initial IPCache revision This introduces a wait for the initial IPCache revision after K8s caches have synced. This ensures that all prefix labels are injected and available in the new IPCache before restoration starts. Signed-off-by: Sebastian Wicki <sebastian@isovalent.com> 10 June 2024, 16:06:10 UTC
ed299a3 ipcache: Always add world label to identities with fqdn label A subsequent commit will change prefix labels upserted by the name manager to use `fqdn`-labels instead of `cidr`-labels. Because a CIDR identity currently always also have the world label, we want to mirror that logic for identities with an `fqdn` label, as such IPs allowed by a ToFQDN policy remains selectable by a `reserved:world` selector. Signed-off-by: Sebastian Wicki <sebastian@isovalent.com> 10 June 2024, 16:06:10 UTC
4035fea labels: Simplify `IsReserved` implementation This contians no functional changes and is a drive-by cleanup. Signed-off-by: Sebastian Wicki <sebastian@isovalent.com> 10 June 2024, 16:06:10 UTC
999d5f0 daemon: Also restore checkpointed FQDN identities This commit modifies the IPCache restoration to restore all local identity entries, not just CIDR identities. This is required because FQDN labels are derived from ToFQDN selectors, which are only available during endpoint regeneration. To ensure that identities of prefixes in IPCache don't change during initial regeneration, we provide the expected `fqdn` labels before regeneration. The real labels are added during regeneration, therefore the restored ones can be safely removed in `releaseRestoredIdentities`. Signed-off-by: Sebastian Wicki <sebastian@isovalent.com> 10 June 2024, 16:06:10 UTC
fda5b55 clustermesh: drain all known entries upon cluster ID change Recent changes introduced improved validation to ensure that the information retrieved from remote clusters matches the advertised cluster ID, and discard it otherwise. Let's additionally fully drain all previously known entries upon cluster ID change. Indeed, although synthetic deletion events would be generated in any case upon initial listing (as the entries with the incorrect cluster ID would not pass validation), that would leave a window of time in which there would still be stale entries for a cluster ID that has already been released, potentially leading to inconsistencies if the same ID is acquired again in the meanwhile by a different cluster. Signed-off-by: Marco Iorio <marco.iorio@isovalent.com> 10 June 2024, 16:06:02 UTC
17882e9 policy/api: add more CRD validations Copying some logic from `Sanitize()` in to CRD validations: - use the OpenAPI `cidr` format directly, remove baroque regex - add OneOf for FQDN selector pattern vs. name - add pre-existing MaxItems for port & ICMP rules - add OneOf for L7 filter types None of these add any new restrictions; they were always in the policy engine. Now these validation errors will be caught by the apiserver. Signed-off-by: Casey Callendrello <cdc@isovalent.com> 10 June 2024, 15:49:28 UTC
fdc9bf9 contrib,tool: Add tool + script to check for legacy header guards This tool checks for legacy header guards and will throw an error if it finds any. Adding this to the CI should ensure that we don't add any more legacy header guards once we switched to prama-once. Signed-off-by: Dylan Reimerink <dylan.reimerink@isovalent.com> 10 June 2024, 14:50:12 UTC
54eb9b8 bpf/tests: remove config_replacement.h The config_replacement.h was originally meant to replace static_data variables for tests. But since then the implementation has changed so static_data always has valid defaults and values can be changed with special test macros. So we no longer need config_replacement.h. Removing it now since it relied on header guards to prevent multiple inclusions of the replaced variables, which we removed. Signed-off-by: Dylan Reimerink <dylan.reimerink@isovalent.com> 10 June 2024, 14:50:12 UTC
eeb41d1 bpf/tests: Change nodeport lb4 nat tests to not use global variables This test was using global variables to store mocking settings. This is triggering CI, likely due to a change in the Go code somewhere. So switching the test over to using a map to store these settings. Signed-off-by: Dylan Reimerink <dylan.reimerink@isovalent.com> 10 June 2024, 14:50:12 UTC
fe9272d bpf: Replace old school header guards with #pragma once This commit replaces the old school header guards with #pragma once. This is a more modern way of preventing multiple inclusion of the same header file. In future we will be using scripts to remove macros, by replacing these now with the proper #pragma once we avoid having to write exceptions for these in the scripts. Signed-off-by: Dylan Reimerink <dylan.reimerink@isovalent.com> 10 June 2024, 14:50:12 UTC
5faea37 pkg/identity: Move GetCIDKeyFromK8sLabels to GlobalIdentity Moves the method from ciliumidentity package to GlobalIdentity and makes the method more generic by accepting the source to be used in mapping. Related #27752 Signed-off-by: Ovidiu Tirla <otirla@google.com> 10 June 2024, 14:47:07 UTC
f0384a8 api: bump protobuf version Signed-off-by: Julian Wiedmann <jwi@isovalent.com> 10 June 2024, 14:23:28 UTC
fd27c83 images: update cilium-{runtime,builder} Signed-off-by: Cilium Imagebot <noreply@cilium.io> 10 June 2024, 14:23:28 UTC
3a91460 chore(deps): update all-dependencies Signed-off-by: cilium-renovate[bot] <134692979+cilium-renovate[bot]@users.noreply.github.com> 10 June 2024, 14:23:28 UTC
e3de640 Fix #32587 concurrent hubble dynamic exporter stop and reload In rare cases when dynamic exporter lifecycle Stop() function is called during config reload it may cause deadlock on mutex. This change stops config watcher ticker before locking the mutex, as mutex lock is effectively needed only to terminate configured exporters, not for terminating config watcher itself. Fixes: #32587 Signed-off-by: Marek Chodor <mchodor@google.com> 10 June 2024, 13:41:24 UTC
4e2a66d conformance-ipsec-e2e: run leak check before/after key rotation This is because we saw a racing issue if leak detection covers the whole rotation + conn-disrupt-check: cilium connectivity will remove conn-disrupt pods in the end of connectivity test, leaving some linger packets recognized as leaked traffic. This commit avoids the issue by running leak checks separately for key rotation and after-rotation test. Signed-off-by: Julian Wiedmann <jwi@isovalent.com> Signed-off-by: gray <gray.liang@isovalent.com> 10 June 2024, 12:43:03 UTC
230c200 ci: check-ipsec-leaks.bt can tolerate proxy traffic not found Add an argument to tell check-ipsec-leaks.bt whether to report errors if proxy traffic not found. Signed-off-by: gray <gray.liang@isovalent.com> 10 June 2024, 12:43:03 UTC
e3fe4bc conformance-ipsec-e2e: add leaked unencrypted packets check Extend the conformance-ipsec-e2e GHA workflow to additionally check that we don't leak any unencrypted packets during the connectivity test. This aims to complement the validation already performed as part of the connectivity tests by the Cilium CLI. Specifically, we leverage bpftrace to analyze the packets forwarded by the bridge device (used by kind), and report those that are not encrypted. We flag packets with both the source and the destination belonging to the IPv4/6 PodCIDR, and we consider the inner headers if packets are encapsulated. In this case, we additionally skip packets originating or targeting CiliumInternalIP addresses (as these are used for node-to-pod traffic when running in tunnel mode, which is not encrypted by design). Extra checks are finally added to always include packets originating from the L7 and DNS proxies, as their source IP is not that of a pod. Signed-off-by: Marco Iorio <marco.iorio@isovalent.com> 10 June 2024, 12:43:03 UTC
ec1b796 ci: Delete deprecated conn-disrupt-test action Signed-off-by: gray <gray.liang@isovalent.com> 10 June 2024, 12:43:03 UTC
364ff9e ci: Use conn-disrupt-test-{setup,check} for ci-ipsec-upgrade Signed-off-by: gray <gray.liang@isovalent.com> 10 June 2024, 12:43:03 UTC
c430572 ci: Decouple ipsec-key-rotate action from conn-disrupt-test action So in future we can add encryption leak detection right after key rotation to avoid certain issues. ci-ipsec-e2e and ci-eks also has been adjusted to use conn-disrupt-test-* actions before and after ipsec-key-rotate action. Signed-off-by: gray <gray.liang@isovalent.com> 10 June 2024, 12:43:03 UTC
0f957a7 ci: Add conn-disrupt-test-{setup,check} actions They are to replace conn-disrupt-test action for better flexibility. Please note the new conn-disrupt-test-check doesn't run full tests by default. Signed-off-by: gray <gray.liang@isovalent.com> 10 June 2024, 12:43:03 UTC
6a0d178 ci: fix cluster name in CI tests In these workflows we used specific cluster name for kops. Cilium-cli fetched cluster name from context resulting in validation error. Signed-off-by: Marcel Zieba <marcel.zieba@isovalent.com> 10 June 2024, 12:40:28 UTC
1cfc5a9 chore(deps): update docker/build-push-action action to v5.4.0 Signed-off-by: cilium-renovate[bot] <134692979+cilium-renovate[bot]@users.noreply.github.com> 10 June 2024, 12:35:17 UTC
597e2b3 cilium, netkit: Add CI e2e coverage Add various netkit and netkit-l2 test coverage to CI: - netkit/netkit-l2 with recommended performance profile (https://docs.cilium.io/en/latest/operations/performance/tuning/) - netkit/netkit-l2 with vxlan/geneve under BPF host routing and legacy routing with ingress Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> 10 June 2024, 12:38:50 UTC
8b1f64a Bandwidth map: fix missing table in reconciler config The table wasn't assigned to the reconciler config for the bandwidth map this causes an error on startup when bandwidth manager is enabled. This commit should resolve the issue. Signed-off-by: Dylan Reimerink <dylan.reimerink@isovalent.com> 10 June 2024, 12:09:49 UTC
f3e65e3 chore(deps): update dependency cilium/cilium-cli to v0.16.10 Signed-off-by: cilium-renovate[bot] <134692979+cilium-renovate[bot]@users.noreply.github.com> 10 June 2024, 10:28:03 UTC
6a203d4 chore(deps): update all github action dependencies Signed-off-by: cilium-renovate[bot] <134692979+cilium-renovate[bot]@users.noreply.github.com> 10 June 2024, 10:26:01 UTC
03afbcc Add active connection tracking to eBPF Add new map - LB_ACT_MAP - behind ENABLE_ACTIVE_CONNECTION_TRACKING flag with counters of opened and closed connections. Behavior of eBPF remains completely unchanged when ENABLE_ACTIVE_CONNECTION_TRACKING flag is not set. When an entry, to conntrack table is created, an entry in LB_ACT_MAP.opened is incremented by one. When connection is closed, the related LB_ACT_MAP.closed is incremented by one. This works only for traffic originating from the local pods. LB_ACT_MAP is keyed by svc_id (also known as rev_nat_index) and zone, which is obtained from backend entry. Zone field in backend is populated only when EndpointSlice contains a reference to zone in FixedZoneMapping (so it is possible to convert between uint8 ID and string). Signed-off-by: Aleksander Mistewicz <amistewicz@google.com> 10 June 2024, 10:14:28 UTC
95886de GwAPI: externalTrafficPolicy support for GwAPI Added externalTrafficPolicy (eTP) support for Cilium GatewayAPI. eTP is globally configurable via `gatewayAPI.externalTrafficPolicy` Helm flag. Signed-off-by: Philip Schmid <phisch@cisco.com> 10 June 2024, 09:42:24 UTC
5af8e22 ingress, docs: eTP support for dedicated ingress Added externalTrafficPolicy support for dedicated Cilium Ingress instances. Configurable via new `ingress.cilium.io/service-external-traffic-policy` Ingress annotation. Signed-off-by: Philip Schmid <phisch@cisco.com> 10 June 2024, 09:42:24 UTC
1a1a048 helm: externalTrafficPolicy for shared ingress Added configuration option to explicitly configure the externalTrafficPolicy for the Cilium Ingress Kubernetes Service. Signed-off-by: Philip Schmid <phisch@cisco.com> 10 June 2024, 09:42:24 UTC
bfa6e5c fix(deps): update aws-sdk-go-v2 monorepo Signed-off-by: cilium-renovate[bot] <134692979+cilium-renovate[bot]@users.noreply.github.com> 10 June 2024, 09:03:56 UTC
2bf8ae7 envoy/xds: Await until endpoint restoration is done Wait until endpoint restoration is done before serving any xDS resources to Envoy, when agent is restoring and envoy daemonset is used. This reduces Envoy resource churn during agent restart. Endpoint restoration does not wait for Envoy ACKs during restart. This is needed to break dependency cycle as resources are not sent to Envoy during restoration, so they will not be ACKed. With this we only get one no-op policy update in Envoy: [cilium/network_policy.cc:1175] NetworkPolicyMap::onConfigUpdate(cilium.policymap.10.244.1.193.1.), 3 resources, version: 17 xternal/envoy/source/common/init/watcher_impl.cc:31] init manager NetworkPolicyMap manager for version 16 destroyed [cilium/network_policy.cc:1200] Received Network Policy for endpoint 1830 in onConfigUpdate() version 17 [cilium/network_policy.cc:1214] New policy is equal to old one, not updating. [cilium/network_policy.cc:1200] Received Network Policy for endpoint 3283 in onConfigUpdate() version 17 [cilium/network_policy.cc:1214] New policy is equal to old one, not updating. [cilium/network_policy.cc:1200] Received Network Policy for endpoint 3268 in onConfigUpdate() version 17 [cilium/network_policy.cc:1214] New policy is equal to old one, not updating. [external/envoy/source/common/init/target_impl.cc:34] target NetworkPolicyMap manager for version 16 destroyed [cilium/network_policy.cc:1266] Reopening ipcache on new stream [cilium/ipcache.cc:81] cilium.ipcache: Opened ipcache. [cilium/network_policy.cc:1273] Skipping empty or duplicate policy update. Signed-off-by: Jarno Rajahalme <jarno@isovalent.com> 10 June 2024, 06:30:44 UTC
3660e4d envoy/xds: Call the callback even if wg == nil Call the callback if given even if wg == nil. Define 'wait := wg != nil' to make code more readable. Move UseCurrent to ack_test.go and remove from the interface, as it is only used for testing. Signed-off-by: Jarno Rajahalme <jarno@isovalent.com> 10 June 2024, 06:30:44 UTC
4a00efb daemon: Remove endpointstate promise dependency on daemon Remove endpointstate resolver's dependency on the daemon, so that it can be used in daemon's dependencies. Suggested-by: Jussi Maki <jussi@isovalent.com> Signed-off-by: Jarno Rajahalme <jarno@isovalent.com> 10 June 2024, 06:30:44 UTC
02304ca daemon: Do not create l7 policies for health endpoint The special health endpoint should not be subject to L7 policies, so we can disable l7 proxy for it altogether. This helps reduce churn on Envoy policy updates. Signed-off-by: Jarno Rajahalme <jarno@isovalent.com> 10 June 2024, 06:30:44 UTC
f1b5334 Adding support for TrafficDistribution This commit adds support for Service Traffic Distribution, a Kubernetes feature that is on track to go beta/available-by-default in v1.31. TrafficDistribution represents the latest and hopefully final iteration in Kubernetes topology aware routing. The default logic is quite simple - route traffic within the same zone if there are any healthy endpoints in that zone. The kube-proxy implementation of that is still based on the EndpointSlice hints field, but does not strictly need to be. This KEP also comes with a more fundamental change to the implementation logic. As part of a broader goal of separating concerns, we've removed the check from Kube-Proxy that verified that a topology annotation was set before honoring hints on EndpointSlices. Now we simply check that all endpoints for an EndpointSlice have hints, and honor them if so. In this commit I've left some logic to determine if a Service is likely to have hints set, but that is now only used to determine if a Service should be reconciled again if the local node labels change. Signed-off-by: Rob Scott <robertjscott@google.com> 10 June 2024, 06:10:46 UTC
3bdfd9d dnsproxy: Pick up cilium/dns with ID retry logic Update to cilium/dns with request ID retry logic to reduce the likelihood of failures like this: level=error msg="Cannot forward proxied DNS lookup" error="duplicate request id 31372" subsys=fqdn/dnsproxy Signed-off-by: Jarno Rajahalme <jarno@isovalent.com> 09 June 2024, 15:50:29 UTC
e364fec envoy: Call given callback also when reusing a listener Call the given callback function of addListener(), if any, in all return cases, or by passing it to the completion. Signed-off-by: Jarno Rajahalme <jarno@isovalent.com> 09 June 2024, 12:18:35 UTC
025fc0f docs: document the cluster name format Signed-off-by: Marco Iorio <marco.iorio@isovalent.com> 07 June 2024, 22:08:26 UTC
32e7736 gha: configure extreme cluster names in conformance clustermesh To detect and prevent possible regressions causing valid names to not be correctly supported. Signed-off-by: Marco Iorio <marco.iorio@isovalent.com> 07 June 2024, 22:08:26 UTC
32416d2 helm: formalize and validate cluster name format Describe the cluster name specifications, and mimic the same checks performed by the Cilium components, to provide early feedback in case the cluster name is invalid. To enable users performing a smooth transition, helm validation can be skipped setting upgradeCompatibility to 1.15 or earlier. In that case, Cilium components will still emit error logs to warn users in case the cluster name is invalid. Signed-off-by: Marco Iorio <marco.iorio@isovalent.com> 07 June 2024, 22:08:26 UTC
e15911e clustermesh: validate remote cluster name Following the formalization of the cluster name format, let's additionally emit an error log when trying to connect to a cluster associated with an invalid name. Starting from v1.17, Cilium will reject connecting to a cluster with an invalid name. Signed-off-by: Marco Iorio <marco.iorio@isovalent.com> 07 June 2024, 22:08:26 UTC
b1f10bc options: formalize and validate cluster name format Formally define and validate that a cluster name must respect the following constraints: * It must contain at most 32 characters; * It must begin and end with a lower case alphanumeric character; * It may contain lower case alphanumerics and dashes between; * The "default" name is reserved, and forbidden with ClusterID != 0. The specification almost matches the cluster name definition from the Kubernetes multi-cluster services API [1] (except for the shorter maximum length), and derives from the already implicit requirements due to the usage of the cluster name as: * a k8s label value [2] (for CiliumIdentities), * a hostname [3] when configuring the host aliases during clustermesh interconnection; * part of TLS certificates common name [4]. The goal of the explicit validation is to ensure that Cilium components fail to start with a clear error if the cluster name is invalid, rather than failing silently at a later stage. Given the above constraints, the vast majority of existing deployments are not expected to affected by this change. Still, to enable users performing a smooth transition, we currently only emit an error log in case of invalid cluster names. The cluster name format will start being strictly enforced starting from the Cilium version. [1]: https://github.com/kubernetes/enhancements/tree/master/keps/sig-multicluster/1645-multi-cluster-services-api#proposal [2]: https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/#syntax-and-character-set [3]: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#dns-label-names [4]: https://stackoverflow.com/a/5142550 Signed-off-by: Marco Iorio <marco.iorio@isovalent.com> 07 June 2024, 22:08:26 UTC
c94af82 clustermesh, operator: slightly rework invoke function registration Directly register the target function, rather than creating a wrapper, both for simplicity, and to make a subsequent introduction of a new parameter in ClusterInfo.Validate transparent from this point of view. Signed-off-by: Marco Iorio <marco.iorio@isovalent.com> 07 June 2024, 22:08:26 UTC
back to top