https://github.com/cilium/cilium

sort by:
Revision Author Date Message Commit Date
ffb2f48 Revert "Revert "datapath: Remove 2005 route table"" This reverts commit 3ed62d51d5ac27bbe2c60372850d4ec78c17bb6e. 22 March 2023, 11:44:08 UTC
c7c50cb pkg: add rules for ip6table to exclude xfrm marks For skbs which are supposed to be ESP-encapped in kernel, we need to prevent them from masquerading in iptables, and this can be done by matching skb mark. Ip4table has done this properly but ip6table missed it. This commit adds rules for it. Fixes: #23461 Signed-off-by: Zhichuan Liang <gray.liang@isovalent.com> 20 March 2023, 11:57:41 UTC
364fce2 pkg: fix wrong xfrm policy for ipv6 The correct xfrm policy should be like: src $cidr_local dst $cidr_remote dir out priority 0 mark $mark tmpl src $ipv6_local dst $ipv6_remote proto esp spi $seq reqid 1 mode tunnel However, policy for ipv6 was wrong in the "src $cidr_local", and this commit fixed it. Signed-off-by: Zhichuan Liang <gray.liang@isovalent.com> 20 March 2023, 11:57:41 UTC
3f76088 bpf: remove encap_remap_v6_host_address The encap_remap_v6_host_address function in encap.h was used to SNAT egress packets and DNAT ingress packets, replacing HOST_IP by ROUTER_IP or vice versa. But it's no more needed since we set the cilium_host's ipv6 to ROUTER_IP. This commit also removes macro ENABLE_ENCAP_HOST_REMAP. Signed-off-by: Zhichuan Liang <gray.liang@isovalent.com> 20 March 2023, 11:57:41 UTC
2d2fc1c pkg: Set cilium_host's IPv6 address to internal router IP Originally the IPv6 of cilium_host is set to a native address. Instead, this commit uses an IPv6 allocated from IPAM for it. This is the same way we set IPv4 address for cilium_host. To make routing work, this commit also change the `nexthop` for ipv6 routeSpec. Signed-off-by: Zhichuan Liang <gray.liang@isovalent.com> 20 March 2023, 11:57:41 UTC
3fcaa50 .github: skip confirmation prompts on cosign With the upgrade of cosign to v3.0.0, a user needs to confirm the following prompt: ``` Note that there may be personally identifiable information associated with this signed artifact. This may include the email address associated with the account with which you authenticate. This information will be used for signing this artifact and will be stored in public transparency logs and cannot be removed later. By typing 'y', you attest that you grant (or have permission to grant) and agree to have this information stored permanently in transparency logs. ``` This commit adds a flag to skip any confirmation prompts that cosign will perform. Fixes: 86e44fbcbdd2 ("chore(deps): update sigstore/cosign-installer action to v3") Signed-off-by: André Martins <andre@cilium.io> 20 March 2023, 09:35:18 UTC
86e44fb chore(deps): update sigstore/cosign-installer action to v3 Signed-off-by: Renovate Bot <bot@renovateapp.com> 19 March 2023, 23:26:07 UTC
490ee3e chore(deps): update all github action dependencies Signed-off-by: Renovate Bot <bot@renovateapp.com> 19 March 2023, 23:23:35 UTC
cb6505c chore(deps): update all github action dependencies Signed-off-by: Renovate Bot <bot@renovateapp.com> 17 March 2023, 22:19:08 UTC
0dc2b73 chore(deps): update docker.io/library/golang docker tag to v1.20.2 Signed-off-by: Renovate Bot <bot@renovateapp.com> 17 March 2023, 22:15:48 UTC
5163094 modules_linux: Add support for builtin modules Previously, kernel modules names were read from the `/proc/modules` only which prevented k8s linux distribution with kernel modules builtin into the kernel to pass the validation of required kernel modules by Cilium. This patch adds support for systems with builtin modules by trying to read the kernel module names from following files additionally: - `/lib/modules/<kernel-version>/modules.builtin` - `/usr/lib/modules/<kernel-version>/modules.builtin` - `/usr/lib/debug/lib/modules/<kernel-version>/modules.builtin` Fixes: #23863 Signed-off-by: Mateusz Paluszkiewicz (TheAifam5) <theaifam5@gmail.com> 17 March 2023, 21:12:52 UTC
7dbd3a9 remove code review requirements Signed-off-by: Bill Mulligan <billmulligan516@gmail.com> 17 March 2023, 19:56:42 UTC
c115497 add tophat Co-authored-by: Aditi Ghag <aditi@cilium.io> Signed-off-by: Bill Mulligan <billmulligan516@gmail.com> 17 March 2023, 19:56:42 UTC
c90393a Move Maintainers to Committers Maintainers are now being tracked in https://github.com/cilium/community/blob/main/roles/Maintainers.md Signed-off-by: Bill Mulligan <billmulligan516@gmail.com> 17 March 2023, 19:56:42 UTC
0dad157 cocci: Fix Python path for coccilib Trying to run the coccinelle checks locally (with the coccicheck image from DockerHub, or by rebuilding it locally) fails with the current error: Python error: No module named 'coccilib' It so happens that it has also been failing silently in the CI for a while! Logging in to the container, we can see: bash-5.1$ python3 Python 3.10.4 (main, Apr 30 2022, 16:49:16) [GCC 11.2.1 20220219] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import sys >>> print(sys.path) ['', '/usr/lib/python310.zip', '/usr/lib/python3.10', '/usr/lib/python3.10/lib-dynload', '/usr/lib/python3.10/site-packages'] bash-5.1$ find /usr -name 'coccilib' /usr/local/lib/coccinelle/python/coccilib This can be trivially addressed by adding the relevant module path to the $PYTHONPATH environment variable in the Dockerfile. The coccinelle checks used to work, but now fail on multiple branches. The root cause for the issue is unknown at this time. Signed-off-by: Quentin Monnet <quentin@isovalent.com> 17 March 2023, 14:15:30 UTC
9d4c226 hive, k8s/resource: don't call workerpool.New from constructor workerpool.New spawns a goroutine [1]. Per the hive guidelines [2], spawning of goroutines must not be performed from constructors, but rather via the Start hook. Fix two occurrences of this anti-pattern in hive examples. [1] https://github.com/cilium/workerpool/blob/054fdc524a07dfe6cc4123aba9bd76186605d960/workerpool.go#L68 [2] https://docs.cilium.io/en/latest/contributing/development/hive/#guidelines Signed-off-by: Tobias Klauser <tobias@cilium.io> 17 March 2023, 13:51:09 UTC
eff3042 operator: don't call workerpool.New from constructor workerpool.New spawns a goroutine [1]. Per the hive guidelines [2], spawning of goroutines must not be performed from constructors, but rather via the Start hook. Fix an occurrence of this anti-pattern in the operator's identity GC constructor. Also add a missing goleak check to TestIdentitiesGC so spawning of goroutines in the constructor will be detected in the future. [1] https://github.com/cilium/workerpool/blob/054fdc524a07dfe6cc4123aba9bd76186605d960/workerpool.go#L68 [2] https://docs.cilium.io/en/latest/contributing/development/hive/#guidelines Fixes: b1159513dddd ("operator: Refactor cilium identities GC to a cell") Signed-off-by: Tobias Klauser <tobias@cilium.io> 17 March 2023, 13:51:09 UTC
0557a2f Fix for disabled cloud provider rate limiting Earlier versions of these flags had a prefix of ENI and were later renamed with a prefix of IPAM to be consistent across cloud providers. While removing the deprecated old flags, #12676 also removed lines needed for setting OperatorConfig struct fields from the values read in. This resulted in fields having golang defaults, which caused the rate limiter to be completely bypassed. Signed-off-by: Hemanth Malla <hemanth.malla@datadoghq.com> 17 March 2023, 13:50:54 UTC
1c01d51 ipcache: Fix incorrect CIDR labels associated with reserved identities An IP / CIDR can be associated with the reserved:world + CIDR labels in the IPcache, for example if a user applies a ToCIDR. Later, if this CIDR is then associated with an entity within the cluster, then the IPcache subsystem will not correctly handle this scenario. It will append the reserved:remote-node or reserved:host label, while keeping the reserved:world + CIDR labels. This doesn't make sense as reserved:world + CIDR labels are meant to represent entities outside of the cluster. This scenario can happen when a user wants to allow traffic to an IP that is firstly outside of the cluster, and then later joins the cluster. For example, a user has a K8s cluster and an external VM with an IP of x.y.z.w. The user wants to allow traffic to x.y.z.w so they create a ToCIDR rule allowing it. Later, the user joins the VM to the cluster and it becomes a worker node. Now x.y.z.w is an entity within the cluster and is therefore associated with either the reserved:remote-node or reserved:host label. Fix this by disassociating the reserved:world + CIDR labels from an entity that's now within the cluster. Related: https://github.com/cilium/cilium/issues/17962 Fixes: https://github.com/cilium/cilium/issues/23750 Signed-off-by: Chris Tarazi <chris@isovalent.com> 17 March 2023, 12:20:41 UTC
2d6423e gateway-api: Add Conformance/HTTPRouteResponseHeaderModifier tests This is a new tests introduced as part of v0.6.0. GHA action setting for this test is enabled. Signed-off-by: Tam Mach <tam.mach@cilium.io> 17 March 2023, 12:20:04 UTC
3838da3 gateway-api: Add Conformance/HTTPRouteRequestRedirect tests This is a new tests introduced as part of v0.6.0. Signed-off-by: Tam Mach <tam.mach@cilium.io> 17 March 2023, 12:20:04 UTC
693875c gateway-api: Add Conformance/HTTPRouteMethodMatching tests This is a new tests introduced as part of v0.6.0. Signed-off-by: Tam Mach <tam.mach@cilium.io> 17 March 2023, 12:20:04 UTC
9619fad gateway-api: Add Conformance/HTTPRouteSimpleSameNamespace tests This was overlooked in previous commits. Signed-off-by: Tam Mach <tam.mach@cilium.io> 17 March 2023, 12:20:04 UTC
c3c8eab gateway-api: Update ingestion test names This is to match with names used in upstream Conformance. The benefit is to have faster lookup: - If a particular test is failed - If a particular test is missing Signed-off-by: Tam Mach <tam.mach@cilium.io> 17 March 2023, 12:20:04 UTC
3548b78 gateway-api: Add Conformance/HTTPRouteRequestHeaderModifier test Also, taking this chance to replace depecrated Append attribute with AppendAction. Signed-off-by: Tam Mach <tam.mach@cilium.io> 17 March 2023, 12:20:04 UTC
337ba84 gateway-api: Add Conformance/HTTPRouteQueryParamMatching translation test Signed-off-by: Tam Mach <tam.mach@cilium.io> 17 March 2023, 12:20:04 UTC
730c543 gateway-api: Add Conformance/HTTPRouteMatching translation test Signed-off-by: Tam Mach <tam.mach@cilium.io> 17 March 2023, 12:20:04 UTC
3d58060 gateway-api: Add Conformance/HTTPRouteMatchingAcrossRoutes translation test Signed-off-by: Tam Mach <tam.mach@cilium.io> 17 March 2023, 12:20:04 UTC
4067c31 gateway-api: Add Conformance/HTTPRouteHostnameIntersection translation test Signed-off-by: Tam Mach <tam.mach@cilium.io> 17 March 2023, 12:20:04 UTC
7fbb647 gateway-api: Add Conformance/HTTPExactPathMatching translation test Signed-off-by: Tam Mach <tam.mach@cilium.io> 17 March 2023, 12:20:04 UTC
52649fa gateway-api: Add Conformance/HTTPRouteCrossNamespace translation test Signed-off-by: Tam Mach <tam.mach@cilium.io> 17 March 2023, 12:20:04 UTC
54ebe39 gateway-api: Add basic listener translation test Signed-off-by: Tam Mach <tam.mach@cilium.io> 17 March 2023, 12:20:04 UTC
700e2ae gateway/translation: Make virtualhosts order deterministic This commit is to make sure that the virtual hosts are created in the same order of appearance of HTTP Route spec. The changes are to split nested map into two separate maps, and handle uniqueness. Fixes: #24217 Signed-off-by: Tam Mach <tam.mach@cilium.io> 17 March 2023, 12:20:04 UTC
152b439 gateway: Make Header matching deterministic This commit is to make sure that header matching rules are sorted in a deterministic way, so that the behavior is predictable in envoy. The main changes are as per below: - Process http routes based on order from spec instead of random order from map. - Use sort.Stable to reverse the original order of equal elements. - Make sure that less(i, j) and less(j, i) return false if i-th and j-th elements are equal. Kindly note that envoy will just iteratively check rule one by one, if a match is found, subsequent rule will not be considered. Fixes: #23999 Signed-off-by: Tam Mach <tam.mach@cilium.io> 17 March 2023, 12:20:04 UTC
01312ef gateway/ingestion: Update the conformance test This is to update test fixture for HTTRRoute header matching from the upstream. The main point is to split header matching condition to diff sets (i.e. OR condition). Signed-off-by: Tam Mach <tam.mach@cilium.io> 17 March 2023, 12:20:04 UTC
2eac0e5 Add src_range IP family check for LB Before the fix, if we add IPv6 addresses to IPv4 LB's source, api-server would not catch it, and the IPv6 addresses may collide with the IPv4 entries. Fixes: #22658 Signed-off-by: Sugang Li <sugangli@google.com> 17 March 2023, 12:19:35 UTC
f7bd675 dos: remove limitation part for wireguard documentation The limitation for wireguard was originally recorded as not compatiable with host-routing, and it's no more the case once pr #23935 is merged. Signed-off-by: Zhichuan Liang <gray.liang@isovalent.com> 17 March 2023, 12:17:36 UTC
c08189c bpf: add unit test for l3 to l2 traffic This commits add unittest case for L3 skb fast redirecting to L2 device. Signed-off-by: Zhichuan Liang <gray.liang@isovalent.com> 17 March 2023, 12:17:36 UTC
d56653d bpf: add l2 header for skb on the l2-less dev This commit adds l2 header before handle_ipv4 calls ipv4_local_delivery, in order to enable fast-redirect from l2-less dev (e.g. cilium_wg0) to internal veth. Fixes: #15075 Signed-off-by: Zhichuan Liang <gray.liang@isovalent.com> 17 March 2023, 12:17:36 UTC
5c32cb7 daemon: enable host routing for wireguard and l2-less devices Originally the host routing mode would be set lagacy if wireguard is enabled or any l2-less device exists. This commit lifts the limitation. Fixes: #15075 Signed-off-by: Zhichuan Liang <gray.liang@isovalent.com> 17 March 2023, 12:17:36 UTC
bfbe5a2 bug: Fix Potential Nil Reference in GetLables Implementation The policyIdentityLabelLookup wrapper for Endpoint implements the GetLabels interface method. This is necessary for the constructing the MapState of the policy engine. This implementation incorrectly did not check if the identity returned by LookupIdentityByID was nil. This fixes this bug, which heretofore has not caused any issues. Signed-off-by: Nate Sweet <nathanjsweet@pm.me> 17 March 2023, 12:16:10 UTC
23586eb Add Palark to Cilium users Signed-off-by: Dmitry Shurupov <dmitry.shurupov@palark.com> 17 March 2023, 12:14:28 UTC
9fb6d7f Revert "docs: fix Rule spec document typos" This reverts commit a97f1f493202db44c3ae43ab15d338b2286ab32a. Signed-off-by: Aditi Ghag <aditi@cilium.io> 17 March 2023, 08:55:49 UTC
de3d4f0 endpoint: Update comments for ToMapState() usage Highlight the locking differences for this path and fix a typo. Signed-off-by: Joe Stringer <joe@cilium.io> 16 March 2023, 21:55:43 UTC
a13cb18 test: bump upgrade tests to test 1.13 We have released 1.13.0 so it's time for us to start testing 1.13.0 upgrade. Signed-off-by: André Martins <andre@cilium.io> 16 March 2023, 19:54:39 UTC
a97f1f4 docs: fix Rule spec document typos fix typos for natural sentences Signed-off-by: Noriki Takahashi <noriki6t@gmail.com> 16 March 2023, 18:39:31 UTC
52ace8e policy: Fix concurrent access of SelectorCache Marco Iorio reports that with previous code, Cilium could crash at runtime after importing a network policy, with the following error printed to the logs: fatal error: concurrent map read and map write The path for this issue is printed also in the logs, with the following call stack: pkg/policy.(*SelectorCache).GetLabels(...) pkg/policy.(*MapStateEntry).getNets(...) pkg/policy.entryIdentityIsSupersetOf(...) pkg/policy.MapState.denyPreferredInsertWithChanges(...) pkg/policy.MapState.DenyPreferredInsert(...) pkg/policy.(*EndpointPolicy).computeDirectionL4PolicyMapEntries(...) pkg/policy.(*EndpointPolicy).computeDesiredL4PolicyMapEntries(...) pkg/policy.(*selectorPolicy).DistillPolicy(...) pkg/policy.(*cachedSelectorPolicy).Consume(...) pkg/endpoint.(*Endpoint).regeneratePolicy(...) ... Upon further inspection, this call path is not grabbing the SelectorCache lock at any point. If we check all of the incoming calls to this function, we can see multiple higher level functions calling into this function. The following tree starts from the deepest level of the call stack and increasing indentation represents one level higher in the call stack. INCOMING CALLS - f GetLabels github.com/cilium/cilium/pkg/policy • selectorcache.go - f getNets github.com/cilium/cilium/pkg/policy • mapstate.go - f entryIdentityIsSupersetOf github.com/cilium/cilium/pkg/policy • mapstate.go - f denyPreferredInsertWithChanges github.com/cilium/cilium/pkg/policy • mapstate.go - f DenyPreferredInsert github.com/cilium/cilium/pkg/policy • mapstate.go - f computeDirectionL4PolicyMapEntries github.com/cilium/cilium/pkg/policy • resolve.go - f computeDesiredL4PolicyMapEntries github.com/cilium/cilium/pkg/policy • resolve.go + f DistillPolicy github.com/cilium/cilium/pkg/policy • resolve.go <--- No SelectorCache lock - f DetermineAllowLocalhostIngress github.com/cilium/cilium/pkg/policy • mapstate.go + f DistillPolicy github.com/cilium/cilium/pkg/policy • resolve.go <--- No SelectorCache lock - f consumeMapChanges github.com/cilium/cilium/pkg/policy • mapstate.go + f ConsumeMapChanges github.com/cilium/cilium/pkg/policy • resolve.go <--- Already locks the SelectorCache Read the above tree as "GetLabels() is called by getNets()", "getNets() is called by entryIdentityIsSupersetOf()", and so on. Siblings at the same level of indent represent alternate callers of the function that is one level of indentation less in the tree, ie DenyPreferredInsert() and consumeMapChanges() both call denyPreferredInsertWithChanges(). As annotated above, we see that calls through DistillPolicy() do not grab the SelectorCache lock. Given that ConsumeMapChanges() grabs the SelectorCache lock, we cannot introduce a new lock acquisition in any descendent function, otherwise it would introduce a deadlock in goroutines that follow that call path. This provides us the option to lock at some point from the sibling of consumeMapChanges() or higher in the call stack. Given that the ancestors of DenyPreferredInsert() are all from DistillPolicy(), we can amortize the cost of grabbing the SelectorCache lock by grabbing it once for the policy distillation phase rather than putting the lock into DenyPreferredInsert() where the SelectorCache could be locked and unlocked for each map state entry. Future work could investigate whether these call paths could make use of the IdentityAllocator's cache of local identities for the GetLabels() call rather than relying on the SelectorCache, but for now this patch should address the immediate locking issue that triggers agent crashes. CC: Nate Sweet <nathanjsweet@pm.me> Fixes: c9f0def587e6 ("policy: Fix Deny Precedence Bug") Reported-by: Marco Iorio <marco.iorio@isovalent.com> Co-authored-by: Chris Tarazi <chris@isovalent.com> Signed-off-by: Joe Stringer <joe@cilium.io> 16 March 2023, 18:39:00 UTC
6d482d3 daemon: move datapath cell to pkg/datapath Now that there are no more circular imports, we can move the datapath cell and associated initialization code to pkg/datapath. The sysctl stuff should most definitely not live toplevel here, but one step at a time. Signed-off-by: David Bimmler <david.bimmler@isovalent.com> 16 March 2023, 18:38:28 UTC
2c91c9a datapath: move interface definitions to /types In preparation of moving the datapath module into datapath/cells.go, we need to break import cycles by moving the interface definitions into their own package. Reuse the existing datapath/types package for this, and adjust code and tests everywhere accordingly. Signed-off-by: David Bimmler <david.bimmler@isovalent.com> 16 March 2023, 18:38:28 UTC
653720e bpf,test: Add an option to disable coverage report per file Add a new option -no-test-coverage=<regex> that disables the coverage report for the file matches to the given regex. The primary usecase for this is temporally disabling the test coverage report for the tests while we are dealing with the coverbee's bug affected to those tests. Signed-off-by: Yutaro Hayakawa <yutaro.hayakawa@isovalent.com> 16 March 2023, 18:36:51 UTC
166d480 Make ingress conformance ready timeout 60s This doubles the timeout that the ingress conformance will wait on pods becoming ready. In a kind cluster that has a lot of scheduling going on the 30s timeout caused flakes by 1 container still being created when it timed out. The default is 5 minutes in the code, this is still less to fail faster. Signed-off-by: Maartje Eyskens <maartje.eyskens@isovalent.com> 16 March 2023, 18:36:12 UTC
3eaa764 Increase logging verbosity of Kubernetes API Server. This will allow us to see what requests are made by Cilium components. Signed-off-by: Marcel Zieba <marcel.zieba@isovalent.com> 16 March 2023, 18:35:43 UTC
faa23a0 Turn off cnp node status gc when cnp status updates are disabled. CNP Node Status GC was still running in cilium-operator even though by default CNP Node Status updates were disabled. This GC periodically made unnecessary API calls (one call per 10 policies) to k8s apiserver, which were going directly to underlying Etcd. Signed-off-by: Marcel Zieba <marcel.zieba@isovalent.com> 16 March 2023, 18:35:21 UTC
8d28f16 docs: Document XDP support on bond devices We upstreamed XDP support for the bond devices in Linux v5.15. Let's document it. 1 - https://isovalent.com/blog/post/2021-12-release-111/#transparent-xdp-bonding-support Signed-off-by: Paul Chaignon <paul@cilium.io> 16 March 2023, 18:34:31 UTC
ee7572b docs: Merge two lists of drivers supporting XDP We are currently maintaining two lists of drivers supporting XDP, one in the kube-proxy-free guide and another one in the XDP documentation. Let's keep only the XDP documentation one, but with the kube-proxy-free one's format (it's cleaner). The list in the kube-proxy-free guide is replaced with a reference to the now-unique list. Reported-by: Daniel Borkmann <daniel@iogearbox.net> Signed-off-by: Paul Chaignon <paul@cilium.io> 16 March 2023, 18:34:31 UTC
93ebeb3 docs: Update the documentation for the conntrack-gc-interval flag The current documentation is incorrect as the default value is 0 and not 5 minutes. 0 implies a dynamic interval value, which this commit now documents. Signed-off-by: Paul Chaignon <paul@cilium.io> 16 March 2023, 18:33:56 UTC
477b43b doc: kubeProxyReplacement=strict / kube-proxy co-existence Signed-off-by: Philip Schmid <philip.schmid@isovalent.com> 16 March 2023, 18:33:08 UTC
4671ce0 Add User DaimlerTruck AG Signed-off-by: Philipp Gniewosz <philipp.gniewosz@posteo.de> 16 March 2023, 18:31:34 UTC
d5ac451 test/verifier: Fix compilation command As part of the verifier tests, we first compile the datapath then attempt to load it. We do that for a set of kernels and datapath configurations. We compile e.g. bpf_lxc with: make -C bpf KERNEL=[kernel] MAX_LXC_OPTIONS=[config] We however pass those two environment variable as the same exec.Command parameter, which causes it to escape them as KERNEL="[kernel] MAX_LXC_OPTIONS=[config]" and the MAX_LXC_OPTIONS is therefore undefined. The default MAX_LXC_OPTIONS value is used instead. That means the verifier test has been covering a subset of the expected datapath configurations. The 4.19 kernel is not affected because it was kept in the previous K8sVerifier ginkgo test. This commit fixes it by using cmd.Env instead to pass the environment variables. Note we also need to switch from %q to %s for the datapath config as we don't want it to be quoted. If quoted, it will be passed quoted to the Clang command and Clang will fail to recognize the macro definitions. Fixes: 9d3f3125bd ("test/verifier: make data path verifier complexity tests standalone") Reported-by: Maxim Mikityanskiy <maxim@isovalent.com> Signed-off-by: Paul Chaignon <paul@cilium.io> 16 March 2023, 13:10:22 UTC
efb5d65 bpf: Fix verifier issue in fib_redirect Paul reported that verifier complexity tests were broken on bpf-next for a while due to the GH action testing the wrong datapath configurations for some months. This slipped in via #22754. Paul says: As part of the verifier tests, we first compile the datapath then attempt to load it. We do that for a set of kernels and datapath configurations. We compile e.g. bpf_lxc with: make -C bpf KERNEL=[kernel] MAX_LXC_OPTIONS=[config] That method of passing the two environment variables however doesn't seem to work in the CI for some reason. The datapath config used for the compilation ends up empty and the default one is used instead. That means the verifier test has been covering a subset of the expected datapath configurations. The 4.19 kernel is not affected because it was kept in the previous K8sVerifier ginkgo test. Specifically, bpf-next fails with an invalid size of register fill error from the verifier: [...] 578: R0=scalar() 578: (79) r6 = *(u64 *)(r10 -288) ; R6_w=ctx(off=0,imm=0) R10=fp0 579: (b4) w9 = -141 ; R9_w=4294967155 580: (c6) if w0 s< 0x0 goto pc+2320 ; R0=scalar(smax=9223372034707292159,umax=18446744071562067967,var_off=(0x0; 0xffffffff7fffffff),s32_min=0,u32_max=2147483647) ; if (no_neigh) { 581: (61) r1 = *(u32 *)(r10 -292) ; R1_w=scalar(umax=4294967295,var_off=(0x0; 0xffffffff)) R10=fp0 582: (16) if w1 == 0x0 goto pc+2292 ; R1_w=scalar(umax=4294967295,var_off=(0x0; 0xffffffff)) ; dmac = nh_params.nh_family == AF_INET ? 583: (18) r1 = 0xffff8c030542d400 ; R1_w=map_ptr(off=0,ks=4,vs=8,imm=0) 585: (61) r2 = *(u32 *)(r10 -304) invalid size of register fill processed 7447 insns (limit 1000000) max_states_per_insn 5 total_states 426 peak_states 420 mark_read 117 Maxim built a small reproducer: 1) Setup LVH: # mkdir -v /tmp/lvh # docker run -v /tmp/lvh:/mnt/images quay.io/lvh-images/complexity-test:bpf-next-main cp /data/images/complexity-test_bpf-next.qcow2.zst /mnt/images/ # zstd -d /tmp/lvh/complexity-test_bpf-next.qcow2.zst -o /tmp/datapath-bpf-complexity.qcow2 # rm -vf /tmp/lvh/complexity-test_bpf-next.qcow2.zst # rmdir -v /tmp/lvh # ./lvh run --image /tmp/datapath-bpf-complexity.qcow2 --host-mount "$YOUR_CILIUM_REPOSITORY_DIR" 2) Inside the VM: # cd /host # make -C bpf KERNEL=netnext MAX_HOST_OPTIONS='-DSKIP_DEBUG=1 -DENABLE_IPV4=1 -DENABLE_IPV6=1 -DENABLE_ROUTING=1 -DPOLICY_VERDICT_NOTIFY=1 -DALLOW_ICMP_FRAG_NEEDED=1 -DENABLE_IDENTITY_MARK=1 -DMONITOR_AGGREGATION=3 -DCT_REPORT_FLAGS=0x0002 -DENABLE_HOST_FIREWALL=1 -DENABLE_ICMP_RULE=1 -DENABLE_CUSTOM_CALLS=1 -DENABLE_SRV6=1 -DHAVE_LPM_TRIE_MAP_TYPE=1 -DHAVE_LRU_HASH_MAP_TYPE=1 -DENABLE_MASQUERADE=1 -DENABLE_SRC_RANGE_CHECK=1 -DENABLE_NODEPORT=1 -DENABLE_NODEPORT_ACCELERATION=1 -DENABLE_SESSION_AFFINITY=1 -DENABLE_DSR_ICMP_ERRORS=1 -DENABLE_DSR=1 -DENABLE_DSR_HYBRID=1 -DENABLE_IPV4_FRAGMENTS=1 -DENABLE_TPROXY=1 -DENABLE_HOST_ROUTING=1 -DENABLE_BANDWIDTH_MANAGER=1 -DETH_HLEN=0 -DENCAP_IFINDEX=1 -DTUNNEL_MODE=1 -DENABLE_EGRESS_GATEWAY=1 -DENABLE_VTEP=1 -DENABLE_SCTP=1' bpf_host.o # env TC_PROGS="" XDP_PROGS="" CG_PROGS="" TC_PROGS="bpf_host" ./test/bpf/verifier-test.sh The above verifier log with nh_params.nh_family == AF_INET test seems to make little sense. It looks like the *(u32 *)(r10 -304) is about fetching the nh_family, but the verifier thinks that the data on stack was a spilled pointer. We can actually fix this by using fib_params->l.family directly which would also simplify the code a bit. For the next hop, the fib lookup helper updates the field depending on whether the next hop is a v4 or v6 address. Related: #24335 Reported-by: Paul Chaignon <paul@cilium.io> Co-developed-by: Maxim Mikityanskiy <maxim@isovalent.com> Signed-off-by: Maxim Mikityanskiy <maxim@isovalent.com> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> 16 March 2023, 12:15:00 UTC
8373eec add better errors for our calls to Setsockopt() Some users reported this failing, but we don't have enough logging to figure out what, exactly, is failing. Signed-off-by: Casey Callendrello <cdc@isovalent.com> 16 March 2023, 10:16:11 UTC
3bd923c Move leader election to a separate Kubernetes client Fixes: #24266 Signed-off-by: Alex Katsman <alexkats@google.com> 16 March 2023, 09:05:01 UTC
9ac5b53 checker: Fix incorrect checker for ExportedEqual() When using checker.ExportedEqual(), it was using the standard Equals checker under-the-hood, but this is incorrect. Fix it to use the correct checker. In the commit introducing the bug, there was no direct usage of checker.ExportedEqual(), but rather checker.ExportedEqual (note the plural). Subtle! Discovered while working on improving policy unit tests. Fixes: f4407e7c8f9 ("checker: Add ExportedEquals checker") Signed-off-by: Chris Tarazi <chris@isovalent.com> 15 March 2023, 22:20:07 UTC
f5c202d cmd/ciliumendpoint: guard against nil indexers. Make cleanup more resilient to conditions where indexer are not set. Signed-off-by: Tom Hadlaw <tom.hadlaw@isovalent.com> 15 March 2023, 12:01:39 UTC
ee7f12c daemon: fix panic when running with etcd with endpoint crd disabled When running etcd kvstore, if endpoint CRD is disabled then the stale CEP cleanup init procedure panics due to a nil indexer references returned from k8s watchers. This is because the cep/ces k8s watchers aren't initialized if this option is set to true. Fixes: #24366 Signed-off-by: Tom Hadlaw <tom.hadlaw@isovalent.com> 15 March 2023, 12:01:39 UTC
84901e4 bpf, ipcache: unconditionally assume LPM trie delete/dump support Delete [1] and dump [2] operations on LPM trie maps are supported since Linux kernel version 4.16. For Cilium v1.14, the minimum required kernel version is 4.19.57 [3]. Thus we can now unconditionally assume support for delete and dump operations on the ipcache map. [1] https://github.com/torvalds/linux/commit/e454cf5958538666635488030046b6a84a22d447 [2] https://github.com/torvalds/linux/commit/b471f2f1de8b816f1e799b80aa92588f3566e4bd [3] https://docs.cilium.io/en/latest/operations/system_requirements/#base-requirements For #22116 Signed-off-by: Tobias Klauser <tobias@cilium.io> 15 March 2023, 10:40:42 UTC
40d65b7 ctmap: unconditionally assume map to be LRU hash As of the previous commit we unconditionally assume support for LRU hash BPF maps. Thus the map type dependent GC interval calculation for the ctmap can be simplified to always assume LRU hash maps. Signed-off-by: Tobias Klauser <tobias@cilium.io> 15 March 2023, 10:23:47 UTC
5139c1f bpf, datapath: unconditionally assume support for LRU hash maps LRU hash maps are available since kernel version 4.10 [1], [2], [3]. For Cilium v1.14, the minimum required kernel version is 4.19.57 [4]. Thus we can now unconditionally assume support for that map type. [1] https://github.com/iovisor/bcc/blob/master/docs/kernel-versions.md#map-types [2] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=29ba732acbeece1e34c68483d1ec1f3720fa1bb3 [3] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=3a08c2fd763450a927d1130de078d6f9e74944fb [4] https://docs.cilium.io/en/latest/operations/system_requirements/#base-requirements For #22116 Signed-off-by: Tobias Klauser <tobias@cilium.io> 15 March 2023, 10:23:47 UTC
86aa873 controller: add new ControllerParam CancelDoFuncOnUpdate This commit adds a new ControllerParam CancelDoFuncOnUpdate which allows cancelling the controller context passed to DoFunc. This way we can terminate the controller (if stuck in e.g. etcd client retry loop) and still preserve the existing logic. Signed-off-by: Ondrej Blazek <ondrej.blazek@firma.seznam.cz> 15 March 2023, 09:37:02 UTC
94b0ee1 etcdclient: use parent ctx and fix logic around it Previously when etcd client context was not set and client created with client.New(*config) it would use context.TODO, without any way to cancel the ctx. In some cases (e.g. wrong endpoints configuration) it would be necessary to cancel the ctx, otherwise the client might be stucked in a retry loop and in case of remoteConnectionController there wouldn't be any way stop that. This commit passes a parent context to the client itself. This commit also fixes the closing of stopWatch channel and rest of the resources when ctx is done/cancelled. If ctx was closed and select statement choosed handling of that case (<-ctx.Done()) the code would quit the for loop and handling of <-w.stopWatch would never be possible which means calling w.Stop() from outside (e.g. releaseOldConnection()) would hang forever as the wg counter wouldn't be decremented. Now when the context gets closed, it correctly closes all channels and decrements the wg counter, just like when handled by releaseOldConnection func which releases everything. Signed-off-by: Ondrej Blazek <ondrej.blazek@firma.seznam.cz> 15 March 2023, 09:37:02 UTC
4b9f69d fix(deps): update module google.golang.org/protobuf to v1.29.1 [security] Signed-off-by: Renovate Bot <bot@renovateapp.com> 15 March 2023, 08:44:25 UTC
9583749 hubble: Test http metrics ListMetricVec http metrics has v1 and v2 plugins handled by the same httpHandler and registers different metrics depending on v1 vs v2. This resulted in a bug in ListMetricVec which didn't handle the v2 case. Signed-off-by: Chance Zibolski <chance.zibolski@gmail.com> 15 March 2023, 08:43:26 UTC
5c3a229 hubble: Fix panic when using http v2 metrics The new ListMetricVec didn't differentiate between v1 and v2 and incorrectly used a nil metric: h.responses which isn't registered in http v2 metrics. Signed-off-by: Chance Zibolski <chance.zibolski@gmail.com> 15 March 2023, 08:43:26 UTC
04fe6a3 Fixes: #24185 Empty VirtualMachineScaleSets returns 404 on ListVirtualMachineScaleSetNetworkInterfaces if there are no nodes in it. Continue with next VirtualMachineScaleSet in that case. Signed-off-by: Marcin Swiderski <marcin@arangodb.com> 15 March 2023, 08:43:07 UTC
c046f2c api: expose RuleOrigin for policy origin tracking RuleOrigin maps L3 selectors to policy rule labels for L4 filters. Exposing this in the API allows for better introspection. Signed-off-by: David Bimmler <david.bimmler@isovalent.com> 15 March 2023, 08:42:43 UTC
fb8cdbd policy: track policy rule origin per selector The current L4Filter.DerivedFromRules track a list of policy rules (identified through a LabelArray). As the policy subsystem is (roughly) organised L4->L3->L7, we coalesce all rules which match the same flows on the L4 layer. Hence, when trying to understand which rule is the origin for a flow verdict, we don't know which of the list of rules truly matches a flow. This patch changes the list to be a map of lists, keyed by cached selector. In effect, this adds L3 information to the rule origin tracking. No attempt is made to also understand L7 semantics, as those are handed off to the proxy. Still, this improves the accuracy of the verdict to rule matching, as the endpoints involved are now considered as well. Signed-off-by: David Bimmler <david.bimmler@isovalent.com> 15 March 2023, 08:42:43 UTC
fd7b5e6 labels: introduce LabelArrayList.Merge It rarely makes sense for LabelArrayLists to contain duplicate LabelArrays. Introduce Merge to merge new LabelArrays into an existing LabelArrayList without introducing duplicates. Note that _existing_ duplicates, in either LabelArrayList, will lead to duplicates in the output. Deduplicating both lists on merge would complicate the code. Signed-off-by: David Bimmler <david.bimmler@isovalent.com> 15 March 2023, 08:42:43 UTC
a6b99de Add make commands for setting up clustermesh in kind Now you can easily setup clustermesh in kind using: ``` make kind-clustermesh make kind-clustermesh-images make kind-install-cilium-clustermesh ``` Signed-off-by: Marcel Zieba <marcel.zieba@isovalent.com> 14 March 2023, 19:22:55 UTC
b20ab61 nodemanager: inject ipcache into nodemanager via hive Injecting the IPCache into the nodemanager via hive instead of setting it explicitly from within the daemon. Signed-off-by: Marco Hofstetter <marco.hofstetter@isovalent.com> 14 March 2023, 19:22:26 UTC
945cc9b ipam: add method to get IP owner per pool Avoid accessing a nil map once we use more than just the default pool. Fixes: e3f1a22ef7d3 ("ipam: track IP owner per pool") Signed-off-by: Tobias Klauser <tobias@cilium.io> 14 March 2023, 19:21:49 UTC
f870a3b Fix operator deadlock introduced in the #23605. Change the order of locking to keep it consistent in the CiliumEndpointSlice Manager. Lock identityLock always before locking backends lock. Fixes: #24206 Fixes: a2d0596d6d947a62211898d32aa6fb2135da1e37 Fix operator crash race condition for CES identity map concurrent read/write Signed-off-by: Alan Kutniewski <kutniewski@google.com> 14 March 2023, 18:14:14 UTC
bf1171c test/k8s: restart agent and check for svc interrupts before downgrade We would only test service interruptions using up/downgrades, but agent restarts should also be clean. Signed-off-by: Timo Beckers <timo@isovalent.com> Co-authored-by: Martynas Pumputis <m@lambda.lt> 14 March 2023, 17:15:08 UTC
afa2494 bpf,init.sh: make netdev bpf filter cleanup less eager Due to an oversight when updating init.sh to deal with the new tc filter names for bpf progs after the introduction of Go-based loader/netlink attach, all interfaces in the host namespace that didn't contain the word 'cilium' would have their egress and ingress filters stripped. This included lxc interfaces and many others. lxc programs in particular would only be reattached when the endpoint got regenerated, which can take a while on nodes with many Pods. This caused connectivity interruptions in the meantime. This commit changes the tc filter naming convention to converge on the changes introduced in 2e40d67ec9 ("bpf: Finish rename of BPF programs to cil_ prefix"), using the bpf program (function) name containing the 'cil_' prefix. The 'cilium_' prefix is no longer included explicitly, instead opting for the program name suffixed by the interface name, e.g. cil_from_netdev-eth0. init.sh no longer uses the term 'cilium' to trigger a removal of the interface's tc filters. Also switched over to a regex that acts on a word boundary to reduce the chance of a false positive (e.g. a filter pencil_foo installed by another tool should not trigger removal). Fixes commit 2a7cef4bb3 ("init,cleanup: remove TC filters containing 'cilium' in their names"). Signed-off-by: Timo Beckers <timo@isovalent.com> 14 March 2023, 17:15:08 UTC
2d230b3 Fix typo in config.go This fixes a typo in the config.go file: CilliumEndpoint -> CiliumEndpoint Signed-off-by: Maartje Eyskens <maartje.eyskens@isovalent.com> 14 March 2023, 16:17:41 UTC
4f423b5 auth: suppress policy verdict notification for authenticated packets Currently, the policy verdict notification gets reported for every packet which matches a policy with authentication - even in cases where the corresponding connection already got authenticated in the meantime and therefore not getting dropped. With this commit the policy verdict notification only gets reported if the packet actually gets dropped due to required and still missing authentication. Signed-off-by: Marco Hofstetter <marco.hofstetter@isovalent.com> 14 March 2023, 14:58:40 UTC
b8eb6bd auth: Add bpf map auth to cilium cli commands Introduce cilium CLI command "cilium bpf auth list" to inspect the entries of the bpf auth map. Signed-off-by: Marco Hofstetter <marco.hofstetter@isovalent.com> 14 March 2023, 14:31:54 UTC
f1cb0f9 auth, bpf: introduce bpf map auth Prior to this commit, auth information were kept as additional property in the ct map which required authentication on a per connection basis. This commit introduces the bpf map auth which keeps track of authentication state between security identities in combination with the remote node id, auth type and an expiration. Signed-off-by: Marco Hofstetter <marco.hofstetter@isovalent.com> 14 March 2023, 14:31:54 UTC
c913ff8 kvstore/etcd: don't use atomic type for version check timeout Commit 441759e44dfb ("pkg/kvstore: fix concurrent access of var in testing") changed versionCheckTimeout to be accessed atomically to fix a race in tests. Instead, pass the timeout value to the function it is used in. This allows to drop the use of an atomic type. Signed-off-by: Tobias Klauser <tobias@cilium.io> 14 March 2023, 13:33:55 UTC
685352f bpf: nodeport: fix handling of stale CT entry with CT_REPLY The nodeport code creates its own CT entry for a service connection to a non-DSR backend. While doing so, the initial CT lookup might find a stale (non-nodeport) CT entry that needs to be re-created. CT_REPLY in particular needs consideration here - regardless of the entry's rev_nat_index, we don't expect incoming SVC requests to be a *reply* to some existing connection. So we always need to re-create such entries. Typically ct_create() is only called after the lookup returned a CT_NEW ct_status. Meaning that the CT tuple was already flipped by ct_lookup(). If we also call ct_create() after CT_REPLY, then the tuple needs to be flipped first. Otherwise the new CT entry is created for a tuple in the wrong direction. Signed-off-by: Julian Wiedmann <jwi@isovalent.com> 14 March 2023, 11:13:21 UTC
edef93f gh/workflows: Split ci-dp encrypt tests into separate matrix configs This commit creates separate matrix configs for the encryption tests, so that they can be run in separately jobs. The main motivation for that is that previously, some configurations were not possible. E.g., DSR with WireGuard due to [1]. The whole job had to disable the DSR. Next, this commit undoes the encryption limitation changes for non-encryption jobs (IPsec with IPv6 and WireGuard with DSR). Afterwards, the commit adds Cilium configuration reporting via the "cilium status", as it's not clear which features are enabled (some might get auto-disabled by the cilium-agent). Finally, do not allow more than 8 parallel jobs to avoid starving the runners. [1]: https://github.com/cilium/cilium/issues/23328 Signed-off-by: Martynas Pumputis <m@lambda.lt> 14 March 2023, 10:19:24 UTC
a152bfe ipam: move podCIDRPool to separate file It will be used by the IPAM pools allocator in a subsequent commit. This commit contains no functional changes. Signed-off-by: Sebastian Wicki <sebastian@isovalent.com> Signed-off-by: Tobias Klauser <tobias@cilium.io> 14 March 2023, 09:13:25 UTC
5a82486 clusterpool/cidralloc: inline NewCIDRSet This function is only used once inside the package, so inline it Signed-off-by: Tobias Klauser <tobias@cilium.io> 14 March 2023, 09:13:25 UTC
b122abf clusterpool: Move CIDRAllocator into own package This commit contains no functional changes. Signed-off-by: Sebastian Wicki <sebastian@isovalent.com> Signed-off-by: Tobias Klauser <tobias@cilium.io> 14 March 2023, 09:13:25 UTC
1bdeded pkg/command: allow semicolon in map values in GetStringMapString This is needed to parse entries like: cluster-pool-map: \ "mars=ipv4-cidrs:172.16.0.0/16,172.17.0.0/16;ipv4-mask-size:24  jupiter=ipv4-cidrs:192.168.0.0/19;ipv4-mask-size:26" which will be used for IPAM pools in cilium-operator. Signed-off-by: Tobias Klauser <tobias@cilium.io> 14 March 2023, 09:13:25 UTC
a8dde6c Add missing check_url env var to conformance-clustermesh Add missing check_url env var to conformance-clustermesh workflow. Otherwise, "Details" link won't appear on the check result. Signed-off-by: Yutaro Hayakawa <yutaro.hayakawa@isovalent.com> 14 March 2023, 08:43:52 UTC
70c2da1 .gitattributes: Mark install/kubernetes/cilium/README.md as generated The README.md file inside of the install/kubernetes/cilium/ directory has been generated with helm-docs ever since its introduction in commit 72958796e2b6 ("build(helm): Add makefile target to generate helm README.md"), using README.md.gotmpl as a template, and other files such as values.yaml (in turn generated from values.yaml.tmpl). It should not be edited by hand. Pull Requests on GitHub usually display changes on this README.md, but instead we want to focus on the source files from which the changes are generated. Let's mark the file as generated, so that the GitHub interface hides the related diff, and reviewers don't lose time looking at it. Signed-off-by: Quentin Monnet <quentin@isovalent.com> 13 March 2023, 22:23:35 UTC
9152558 bpf: unconditionally assume map name support For Cilium v1.14, the minimum required kernel version is 4.19.57 [1]. Thus we can now unconditionally assume support for map names which are available in version >= 4.15. [1] https://docs.cilium.io/en/latest/operations/system_requirements/#base-requirements For #22116 Signed-off-by: Tobias Klauser <tobias@cilium.io> 13 March 2023, 19:11:55 UTC
7628b19 bpf, ipcache: unconditionally assume support for LPM trie maps LPM trie maps are available since kernel version 4.11 [1], [2]. For Cilium v1.14, the minimum required kernel version is 4.19.57 [3]. Thus we can now unconditionally assume support for that map type. [1] https://github.com/iovisor/bcc/blob/master/docs/kernel-versions.md#map-types [2] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=b95a5c4db09bc7c253636cb84dc9b12c577fd5a0 [3] https://docs.cilium.io/en/latest/operations/system_requirements/#base-requirements For #22116 Signed-off-by: Tobias Klauser <tobias@cilium.io> 13 March 2023, 19:11:55 UTC
6c8efa7 test: Remove "Checks service on same node" test It is covered by [1] running on ci-datapath: "echo" pod can run on the same node as either of the "client" pods./ [1]: https://github.com/cilium/cilium-cli/blob/v0.13.1/connectivity/tests/service.go#L17 Signed-off-by: Martynas Pumputis <m@lambda.lt> 13 March 2023, 18:51:02 UTC
b5e0e12 test: Remove K8sDatapathConfig some direct routing tests All of them are covered by ci-datapath [1]: * "Check connectivity with automatic direct nodes routes" by the "2", "7" configurations. * "Check direct connectivity with per endpoint routes" by the "3", "5" configurations. [1]: https://github.com/cilium/cilium/blob/841d1f37aba6ed54b3b40853bfbed9c9bceab58b/.github/workflows/conformance-dat apath.yaml#L122 Signed-off-by: Martynas Pumputis <m@lambda.lt> 13 March 2023, 18:51:02 UTC
back to top