| a336dda | Marco Hofstetter | 12 May 2023, 11:22:27 UTC | images: introduce update script update-cilium-envoy-image This commit introduces the script `update-cilium-envoy-image.sh` (and corresponding make target) which fetches the latest cilium-envoy image by fetching the relevant data from its github repo. It updates the cilium Dockerfile. Signed-off-by: Marco Hofstetter <marco.hofstetter@isovalent.com> | 15 June 2023, 16:12:00 UTC |
| ba9d077 | Quentin Monnet | 15 June 2023, 14:27:07 UTC | install: Update image digests for v1.11.18 Generated from https://github.com/cilium/cilium/actions/runs/5279434084. ## Docker Manifests ### cilium `docker.io/cilium/cilium:v1.11.18@sha256:dda94072012c328fe0d00838f2f7d8ead071019d1d1950ecf44060640bf93cae` `quay.io/cilium/cilium:v1.11.18@sha256:dda94072012c328fe0d00838f2f7d8ead071019d1d1950ecf44060640bf93cae` ### clustermesh-apiserver `docker.io/cilium/clustermesh-apiserver:v1.11.18@sha256:b3e8de4e56c5e16ab8f4482cebf3a12bb12826ba3da3e5890de1ecdc2b34a3ed` `quay.io/cilium/clustermesh-apiserver:v1.11.18@sha256:b3e8de4e56c5e16ab8f4482cebf3a12bb12826ba3da3e5890de1ecdc2b34a3ed` ### docker-plugin `docker.io/cilium/docker-plugin:v1.11.18@sha256:b086fc1ec24b9b2b0bc5f7f525ef76ff608c26dc1bdd76d46729871cbbfb4b08` `quay.io/cilium/docker-plugin:v1.11.18@sha256:b086fc1ec24b9b2b0bc5f7f525ef76ff608c26dc1bdd76d46729871cbbfb4b08` ### hubble-relay `docker.io/cilium/hubble-relay:v1.11.18@sha256:4899d8a98c05ccb7bb3d0b54e18dc72147995b2e8a18db19805d15933ec6e45d` `quay.io/cilium/hubble-relay:v1.11.18@sha256:4899d8a98c05ccb7bb3d0b54e18dc72147995b2e8a18db19805d15933ec6e45d` ### operator-alibabacloud `docker.io/cilium/operator-alibabacloud:v1.11.18@sha256:590062c3797c0d0732d848b8fa09cd5aaf5ce2cbbbc5f5fc860bde79d27c743c` `quay.io/cilium/operator-alibabacloud:v1.11.18@sha256:590062c3797c0d0732d848b8fa09cd5aaf5ce2cbbbc5f5fc860bde79d27c743c` ### operator-aws `docker.io/cilium/operator-aws:v1.11.18@sha256:4b3aeeb5d0de096d68ab249845c4c53c7c595735d529a13a81540597a6b29bb5` `quay.io/cilium/operator-aws:v1.11.18@sha256:4b3aeeb5d0de096d68ab249845c4c53c7c595735d529a13a81540597a6b29bb5` ### operator-azure `docker.io/cilium/operator-azure:v1.11.18@sha256:c833cd215dafcb9a73dc1d435d984038fc46ebd9a0b3d50ceeb8f8c4c7e9ac3d` `quay.io/cilium/operator-azure:v1.11.18@sha256:c833cd215dafcb9a73dc1d435d984038fc46ebd9a0b3d50ceeb8f8c4c7e9ac3d` ### operator-generic `docker.io/cilium/operator-generic:v1.11.18@sha256:bccdcc3036b38581fd44bf7154255956a58d7d13006aae44f419378911dec986` `quay.io/cilium/operator-generic:v1.11.18@sha256:bccdcc3036b38581fd44bf7154255956a58d7d13006aae44f419378911dec986` ### operator `docker.io/cilium/operator:v1.11.18@sha256:0c09e5188d5d8899e7b037fafcc1928a68872f1e48e5f7a128799594c99f8282` `quay.io/cilium/operator:v1.11.18@sha256:0c09e5188d5d8899e7b037fafcc1928a68872f1e48e5f7a128799594c99f8282` Signed-off-by: Quentin Monnet <quentin@isovalent.com> | 15 June 2023, 14:55:26 UTC |
| f5d7e2d | Michi Mutsuzaki | 14 June 2023, 03:54:13 UTC | Prepare for release v1.11.18 Signed-off-by: Michi Mutsuzaki <michi@isovalent.com> | 14 June 2023, 21:00:38 UTC |
| 9d72663 | Nate Sweet | 13 June 2023, 03:27:52 UTC | docs: Promote Deny Policies out of Beta Signed-off-by: Nate Sweet <nathanjsweet@pm.me> | 13 June 2023, 22:55:09 UTC |
| 1e8d21b | Anton Protopopov | 13 June 2023, 09:25:26 UTC | docs: fix wording for the upgrade guide Rephrase a recent change to Documentation/operations/upgrade.rst. Signed-off-by: Anton Protopopov <aspsk@isovalent.com> | 13 June 2023, 19:22:22 UTC |
| c3ffd99 | Paul Chaignon | 06 June 2023, 10:07:07 UTC | ipsec: Don't rely on output-marks to know if state exists On kernels before 4.19, the XFRM output mark is not fully supported. Thus, when comparing XFRM states, if we compare the output marks, the existing states will never match the new state. The new state will have an output mark, but the states installed in the kernel don't have it (because the kernel ignored it). As a result, our current logic will assume that the state we want to install doesn't already exist, it will try to install it, fail because it already exist, assume there's a conflicting state, throw an error, remove the conflicting state, and install the new (but identical) one. The end result is therefore the same: the new state is in place in the kernel. But on the way to installing it, we will emit an unnecessary error and temporarily remove the state (potentially causing packet drops). Instead, we can safely ignore the output-marks when comparing states. We don't expect any states with same IPs, SPI, and marks, but different output-marks anyway. The only way this could happen is if someone manually added such a state. Even if they did, the only impact would be that we wouldn't overwrite the manually-added state with the different output-mark. This patch is only necessary on v1.12 and earlier versions of Cilium because v1.13 dropped support for Linux <4.19. Signed-off-by: Paul Chaignon <paul.chaignon@gmail.com> | 13 June 2023, 19:22:04 UTC |
| 2c0a06d | Paul Chaignon | 09 June 2023, 22:22:16 UTC | ipsec: Don't attempt per-node route deletion when unexistant [ upstream commit 1e1e2f7e410d24e4af2d6dbd2cb2ceb016fb76b7 ] Commit 3e59b681f ("ipsec: Per-node XFRM states & policies for EKS & AKS") changed the XFRM config to have one state and policy per remote node in IPAM modes ENI and Azure. The IPsec cleanup logic was therefore also updated to call deleteIPsec() whenever a remote node is deleted. However, we missed that the cleanup logic also tries to remove the per-node IP route. In case of IPAM modes ENI and Azure, the IP route however stays as before: we have a single route for all remote nodes. We therefore don't have anything to cleanup. Because of this unnecessary IP route cleanup attempt, an error message was printed for every remote node deletion: Unable to delete the IPsec route OUT from the host routing table This commit fixes it to avoid attempting this unnecessary cleanup. Fixes: 3e59b681f ("ipsec: Per-node XFRM states & policies for EKS & AKS") Signed-off-by: Paul Chaignon <paul.chaignon@gmail.com> Signed-off-by: Quentin Monnet <quentin@isovalent.com> | 13 June 2023, 19:22:04 UTC |
| 60f5a1b | Paul Chaignon | 09 June 2023, 10:08:16 UTC | ipsec: Only match appropriate XFRM configs with node ID [ upstream commit 57eac9d8b42a19f5aeae412f38de3eaf8bfadc4a ] With commit 9cc8a89f9 ("ipsec: Fix leak of XFRM policies with ENI and Azure IPAMs") we rely on the node ID to find XFRM states and policies that belong to remote nodes, to clean them up when remote nodes are deleted. This commit makes sure that we only do this for XFRM states and policies that actually match on these node IDs. That should only be the same if the mark mask matches on node ID bits. Thus if should look like 0xffffff00 (matches on node ID, SPI, and encryption bit) or 0xffff0f00 (matches on node and encryption bit). Fixes: 9cc8a89f9 ("ipsec: Fix leak of XFRM policies with ENI and Azure IPAMs") Signed-off-by: Paul Chaignon <paul.chaignon@gmail.com> Signed-off-by: Sebastian Wicki <sebastian@isovalent.com> | 13 June 2023, 19:22:04 UTC |
| 319fa31 | Steven Johnson | 09 June 2023, 02:42:13 UTC | ipsec: Only delete ipsec endpoint when node ID is not 0 [ upstream commit 25064d1ec51895ab89e2f736fcf7c6c66dfb5551 ] After applying a backport of 9cc8a89f9 ("ipsec: Fix leak of XFRM policies with ENI and Azure IPAMs") to 1.11.16, I noticed that we were getting occasional spikes of "no inbound state" xfrm errors (XfrmInNoStates). These lead to packet loss and brief outages for applications sending traffic to the node on which the spikes occur. I noticed that the "No node ID found for node." logline would appear at the time of these spikes and from the code this is logged when the node ID cannot be resolved. Looking a bit further the call to `DeleteIPsecEndpoint` will end up deleting the xfrm state for any state that matches the node id as derived from the mark in the state. The problem seems to be that the inbound state for 0.0.0.0/0 -> node IP has a mark of `0xd00` which when shifted >> 16 in `getNodeIDFromXfrmMark` matches nodeID 0 and so the inbound state gets deleted and the kernel drops all the inbound traffic as it no longer matches a state. This commit updates that logic to skip the XFRM state and policy deletion when the node ID is zero. Fixes: 9cc8a89f9 ("ipsec: Fix leak of XFRM policies with ENI and Azure IPAMs") Signed-off-by: Steven Johnson <sjdot@protonmail.com> Signed-off-by: Paul Chaignon <paul.chaignon@gmail.com> Signed-off-by: Sebastian Wicki <sebastian@isovalent.com> | 13 June 2023, 19:22:04 UTC |
| 00bb13b | Paul Chaignon | 07 June 2023, 15:37:53 UTC | ipsec: Fix IPv6 wildcard CIDR used in some IPsec policies [ upstream commit d0ab559441311dbe0908834a86d633aa9eeb6a84 ] We use this wildcard IPv6 CIDR in the catch-all default-drop OUT policy as well as in the FWD policy. It was incorrectly set to ::/128 instead of ::/0 and would therefore not match anything instead of matching everything. This commit fixes it. Fixes: e802c2985 ("ipsec: Refactor wildcard IP variables") Signed-off-by: Paul Chaignon <paul.chaignon@gmail.com> Signed-off-by: Sebastian Wicki <sebastian@isovalent.com> | 13 June 2023, 19:22:04 UTC |
| 3a5aa36 | Paul Chaignon | 06 June 2023, 16:11:16 UTC | ipsec: Change XFRM FWD policy to simplest wildcard [ upstream commit ac54f2965908c06ff53e5a63a0f47b2448204a18 ] We recently changed our XFRM configuration to have one XFRM OUT policy per remote node, regardless of the IPAM mode being used. In doing so, we also moved the XFRM FWD policy to be installed once per remote node. With ENI and Azure IPAM modes, this wouldn't cause any issue because the XFRM FWD policy is the same regardless of the remote node. On other IPAM modes, however, the XFRM FWD policy is for some reason different depending on the remote node that triggered the installation. As a result, for those IPAM modes, one FWD policy is installed per remote node. And the deletion logic triggered on node deletions wasn't updated to take that into account. We thus have a leak of XFRM FWD policies. In the end, our FWD policy just needs to allow everything through without encrypting it. It doesn't need to be specific to any remote node. We can simply completely wildcard the match, to look like: src 0.0.0.0/0 dst 0.0.0.0/0 dir fwd priority 2975 ptype main tmpl src 0.0.0.0 dst 192.168.134.181 proto esp reqid 1 mode tunnel level use So we match all packets regardless of source and destination IPs. We don't match on the packet mark. There's a small implementation hurdle here. Because we used to install FWD policies of the form "src 0.0.0.0/0 dst 10.0.1.0/24", the kernel was able to deduce which IP family we are matching against and would adapt the 0.0.0.0/0 source CIDR to ::/0 as needed. Now that we are matching on 0/0 for both CIDRs, it cannot deduce this anymore. So instead, we must detect the IP family ourself and use the proper CIDRs. In addition to changing the XFRM FWD policy to the above, we can also stop installing it once per remote node. It's enough to install it when we receive the event for the local node, once. Fixes: 3e59b681f ("ipsec: Per-node XFRM states & policies for EKS & AKS") Signed-off-by: Paul Chaignon <paul.chaignon@gmail.com> Signed-off-by: Sebastian Wicki <sebastian@isovalent.com> | 13 June 2023, 19:22:04 UTC |
| 10a2851 | Jussi Maki | 06 June 2023, 12:32:26 UTC | loader: In IPsec reload ignore veth devices & fix settle wait [ upstream commit 592777da560bea7838b99223386e943c08d5d052 ] reloadIPSecOnLinkChanges() did not ignore veth device updates causing reload to be triggered when new endpoints were created. Ignore any updates with "veth" as device type. The draining of updates during settle wait was broken due to unintentional breaking out of the loop. Removed the break. Fixes: bf0940b4ff ("loader: Reinitialize IPsec on device changes on ENI") Signed-off-by: Jussi Maki <jussi@isovalent.com> | 13 June 2023, 19:22:04 UTC |
| f421409 | Jussi Maki | 06 June 2023, 12:31:43 UTC | loader: Do not fatal on IPsec reinitialization [ upstream commit 470465550bc446b920a62c5b7f7b521cd10b0a9b ] Now that the code is reloading the bpf_network program at runtime we should not fatal if we fail to reload the program since this may be caused by ongoing interface changes (e.g. interface was being removed). Change the log.Fatal into log.Error and keep loading to other interfaces. Fixes: bf0940b4ff ("loader: Reinitialize IPsec on device changes on ENI") Signed-off-by: Jussi Maki <jussi@isovalent.com> | 13 June 2023, 19:22:04 UTC |
| fdba480 | Paul Chaignon | 30 May 2023, 15:33:58 UTC | ipsec: Allow old and new XFRM OUT states to coexist for upgrade [ upstream commit c0d9b8c9e791b8419c63e5e80b52bc2b39f80030 ] Commit 73c36d45e0 ("ipsec: Match OUT XFRM states & policies using node IDs") changed our XFRM states to match on packet marks of the form 0xXXXXYe00/0xffffff00 where XXXX is the node ID and Y is the SPI. The previous format for the packet mark in XFRM states was 0xYe00/0xff00. According to the Linux kernel these two states conflict (because 0xXXXXYe00/0xffffff00 ∈ 0xYe00/0xff00). That means we can't add the new state while the old one is around. Thus, in commit ddd491bd8 ("ipsec: Custom check for XFRM state existence"), we removed any old conflicting XFRM state before adding the new ones. That however causes packet drops on upgrades because we may remove the old XFRM state before bpf_lxc has been updated to use the new 0xXXXXYe00/0xffffff00 mark. Instead, we would need both XFRM state formats to coexist for the duration of the upgrade. Impossible, you say! Don't despair. Things are actually a bit more complicated (it's IPsec and Linux after all). While Linux doesn't allow us to add 0xXXXXYe00/0xffffff00 when 0xYe00/0xff00 exists, it does allow adding in the reverse order. That seems to be because 0xXXXXYe00/0xffffff00 ∈ 0xYe00/0xff00 but 0xYe00/0xff00 ∉ 0xXXXXYe00/0xffffff00 [1]. Therefore, to have both XFRM states coexist, we can remove the old state, add the new one, then re-add the old state. That is allowed because we never try to add the new state when the old is present. During the short period of time when we have removed the old XFRM state, we can have a packet drops due to the missing state. These drops should be limited to the specific node pair this XFRM state is handling. This will also only happen on upgrades. Finally, this shouldn't happen with ENI and Azure IPAM modes because they don't have such old conflicting states. I tested this upgrade path on a 20-nodes GKE cluster running our drop-sensitive application, migrate-svc, scaled up to 50 clients and 30 backends. I didn't get a single packet drop despite the application consistently sending packets back and forth between nodes. Thus, I think the window for drops to happen is really small. Diff before/after the upgrade (v1.13.0 -> thi patch, GKE): src 10.24.1.77 dst 10.24.2.207 proto esp spi 0x00000003 reqid 1 mode tunnel replay-window 0 mark 0x3e00/0xff00 output-mark 0xe00/0xf00 aead rfc4106(gcm(aes)) 0xfc2d0c4e646b87ff2d0801b57997e3598eab0d6b 128 - anti-replay context: seq 0x0, oseq 0x2c, bitmap 0x00000000 + anti-replay context: seq 0x0, oseq 0x16, bitmap 0x00000000 sel src 0.0.0.0/0 dst 0.0.0.0/0 + src 10.24.1.77 dst 10.24.2.207 + proto esp spi 0x00000003 reqid 1 mode tunnel + replay-window 0 + mark 0x713f3e00/0xffffff00 output-mark 0xe00/0xf00 + aead rfc4106(gcm(aes)) 0xfc2d0c4e646b87ff2d0801b57997e3598eab0d6b 128 + anti-replay context: seq 0x0, oseq 0x0, bitmap 0x00000000 + sel src 0.0.0.0/0 dst 0.0.0.0/0 We can notice that the counters for the existing XFRM state also changed (decreased). That's expected since the state got recreated. 1 - I think this is because XFRM states don't have priorities. So when two XFRM states would match a given packet (in our case a packet with mark XXXXYe00), the oldest XFRM state is taken. Thus, by not allowing to add a more specific match after a more generic one, the kernel ensures that the more specific match is always taken when both match a packet. That likely corresponds to user expectations. That is, if both 0xXXXXYe00/0xffffff00 and 0xYe00/0xff00 match a packet, we would probably expect 0xXXXXYe00/0xffffff00 to be used. Fixes: ddd491bd8 ("ipsec: Custom check for XFRM state existence") Fixes: 73c36d45e0 ("ipsec: Match OUT XFRM states & policies using node IDs") Signed-off-by: Paul Chaignon <paul.chaignon@gmail.com> | 13 June 2023, 19:22:04 UTC |
| 5fd9b0b | Paul Chaignon | 28 May 2023, 21:44:58 UTC | daemon: Reload bpf_host first in case of IPsec upgrade [ upstream commit ca9c056deb31f6e0747c951be24b25d67ea99d6d ] As explained in the previous commit, we need to switch our IPsec logic from one implementation to another. This implementation requires some synchronized work between bpf_lxc and bpf_host. To enable this switch without causing drops, the previous commit made bpf_host support both implementations. This is quite enough though. For this to work, we need to ensure that bpf_host is always reloaded before any bpf_lxc is loaded. That is, we need to load the bpf_host program that supports both implementations before we actually start the switch from one implementation to the second. This commit makes that change in the order of BPF program reloads. Instead of regenerating the bpf_host program (i.e., the host endpoint's datapath) in a goroutine like other BPF programs, we will regenerate it first, as a blocking operation. Regenerating the host endpoint's datapath separately like this will delay the agent startup. This regeneration was measured to take around 1 second on an EKS cluster (though it can probably grow to a few seconds depending on the node type and current load). That should stay fairly small compared to the overall duration of the agent startup (around 30 seconds). Nevertheless, this separate regeneration is only performed when we actually need: for IPsec with EKS or AKS IPAM mode. Fixes: 4c7cce1bf ("bpf: Remove IP_POOLS IPsec code") Signed-off-by: Paul Chaignon <paul.chaignon@gmail.com> | 13 June 2023, 19:22:04 UTC |
| 7afecc7 | Paul Chaignon | 28 May 2023, 21:16:29 UTC | bpf: Support the old IP_POOLS logic in bpf_host [ upstream commit 0af2303f534bb155918e86f07f0f3f4686d2a927 ] This commit reverts the bpf_host changes of commit 4c7cce1bf ("bpf: Remove IP_POOLS IPsec code"). The IP_POOLS IPsec code was a hack to avoid having one XFRM OUT policy and state per remote node. Instead, we had a single XFRM OUT policy and state that would encrypt traffic as usual, but encapsulate it with placeholder IP addresses, such as 0.0.0.0 -> 192.168.0.0. Those outer IP addresses would then be rewritten to the proper IPs in bpf_host. To that end, bpf_lxc would pass the destination IP address, the tunnel endpoint, to bpf_host via a skb->cb slot. The source IP address was hardcoded in the object file. Commit 4c7cce1bf ("bpf: Remove IP_POOLS IPsec code") thus got rid of that hack to instead have per-node XFRM OUT policies and states. The kernel therefore directly writes the proper outer IP addresses. Unfortunately, the transition from one implementation to the other isn't so simple. If we simply remove the old IP_POOLS code as done in commit 4c7cce1bf, then we will have drops on upgrade. We have two cases, depending on which of bpf_lxc or bpf_host is reloaded first: 1. If bpf_host is reloaded before the new bpf_lxc is loaded, then it won't rewrite the outer IP addresses anymore. In that case, we end up with packets of the form 0.0.0.0 -> 192.168.0.0 leaving on the wire. Obviously, they don't go far and end up dropped. 2. If bpf_lxc is reloaded before the new bpf_host, then it will reuse skb->cb for something else and the XFRM layer will handle the outer IP addresses. But because bpf_host is still on the old implementation, it will try to use skb->cb to rewrite the outer IP addresses. We thus end up with gibberish outer destination IP addresses. One way to fix this is to have bpf_host support both implementations. This is what this commit does. The logic to rewrite the outer IP addresses is reintroduced in bpf_host, but it is only executed if the outer source IP address is 0.0.0.0. That way, we will only rewrite the outer IP addresses if bpf_lxc is on the old implementation and the XFRM layer didn't write the proper outer IPs. Fixes: 4c7cce1bf ("bpf: Remove IP_POOLS IPsec code") Signed-off-by: Paul Chaignon <paul.chaignon@gmail.com> | 13 June 2023, 19:22:04 UTC |
| a8ce874 | Paul Chaignon | 28 May 2023, 20:51:56 UTC | ipsec: Deprioritize old XFRM OUT policy for dropless upgrade [ upstream commit a11d088154b2d3fe50d0ce750aca87b3fabb19e5 ] This is a revert, or rather a reimplementation, of commit 688dc9ac8 ("ipsec: Remove stale XFRM states and policies"). In that commit, we would remove the old XFRM OUT policies and states because they conflict with the new ones and prevent the installation to proceed. This removal however causes a short window of packet drops on upgrade, between the time the old XFRM configs are removed and the new ones are added. These drops would show up as XfrmOutPolBlock because packets then match the catch-all default-drop XFRM policy. Instead of removing the old XFRM configs, a better, less-disruptive approach is to deprioritize them and add the new ones in front. To that end, we "lower" the priority of the old XFRM OUT policy from 0 to 50 (0 is the highest-possible priority). By doing this the XFRM OUT state is also indirectly deprioritized because it is selected by the XFRM OUT policy. As with the code from commit 688dc9ac8 ("ipsec: Remove stale XFRM states and policies"), this whole logic can be removed in v1.15, once we are sure that nobody is upgrading with the old XFRM configs in place. At that point, we will be able to completely clean up those old XFRM configs. The priority of 50 was chosen arbitrarily, to be between the priority of new XFRM OUT configs (0) and the priority of the catch-all default-drop policy (100), while leaving space if we need to add additional rules of different priorities. Diff before/after upgrade (v1.13.0 -> this patch, GKE): src 10.24.1.0/24 dst 10.24.2.0/24 - dir out priority 0 + dir out priority 50 mark 0x3e00/0xff00 tmpl src 10.24.1.77 dst 10.24.2.207 proto esp spi 0x00000003 reqid 1 mode tunnel Fixes: 688dc9ac8 ("ipsec: Remove stale XFRM states and policies") Signed-off-by: Paul Chaignon <paul.chaignon@gmail.com> | 13 June 2023, 19:22:04 UTC |
| e64caec | Paul Chaignon | 28 May 2023, 19:51:31 UTC | ipsec: Lower priority of catch-all XFRM policies [ upstream commit 3e898f26063531b9bf3883c5c79e347f15112631 ] This commit lowers the priority of the catch-all default-drop XFRM OUT policies, from 1 to 100. For context, 0 is the highest possible priority. This change will allow us to introduce several levels of priorities for XFRM OUT policies in subsequent commits. Diff before/after this patch: src 0.0.0.0/0 dst 0.0.0.0/0 - dir out action block priority 1 + dir out action block priority 100 mark 0xe00/0xf00 Signed-off-by: Paul Chaignon <paul.chaignon@gmail.com> | 13 June 2023, 19:22:04 UTC |
| ac40896 | Paul Chaignon | 31 May 2023, 08:59:12 UTC | ipsec: Fix leak of XFRM policies with ENI and Azure IPAMs [ upstream commit 9cc8a89f914195d52a8b3df021215b4051348b45 ] Our logic to clean up old XFRM configs on node deletion currently relies on the destination IP to identify the configs to remove. That doesn't work with ENI and Azure IPAMs, but until recently, it didn't need to. On ENI and Azure IPAMs we didn't have per-node XFRM configs. That changed in commit 3e59b681f ("ipsec: Per-node XFRM states & policies for EKS & AKS"). We now need to clean up per-node XFRM configs for ENI and Azure IPAM modes as well, and we can't rely on the destination IP for that because the XFRM policies don't match on that destination IP. Instead, since commit 73c36d45e0 ("ipsec: Match OUT XFRM states & policies using node IDs"), we match the per-node XFRM configs using node IDs encoded in the packet mark. The good news is that this is true for all IPAM modes (whether Azure, ENI, cluster-pool, or something else). So our cleanup logic can now rely on the node ID of the deleted node to clean up its XFRM states and policies. This commit implements that. Fixes: 3e59b681f ("ipsec: Per-node XFRM states & policies for EKS & AKS") Signed-off-by: Paul Chaignon <paul.chaignon@gmail.com> | 13 June 2023, 19:22:04 UTC |
| df1effa | Paul Chaignon | 30 May 2023, 22:43:41 UTC | node_ids: New helper function getNodeIDForNode [ upstream commit 3201a5ee689ba650df414d3417d9a9a0ad677bf7 ] This commit simply refactors some existing code into a new getNodeIDForNode function. This function will be called from elsewhere in a subsequent commit. Signed-off-by: Paul Chaignon <paul.chaignon@gmail.com> | 13 June 2023, 19:22:04 UTC |
| 2217b29 | Jussi Maki | 29 May 2023, 12:57:53 UTC | loader: Reinitialize IPsec on device changes on ENI [ upstream commit bf0940b4ff6fcc54227137c1322c2e632e7a1819 ] If IPsec is enabled along with the ENI IPAM mode we need to load the bpf_network program onto new ENI devices when they're added at runtime. To fix this, we subscribe to netlink link updates to detect when new (non-veth) devices are added and reinitialize IPsec to load the BPF program onto the devices. The compilation of the bpf_netowrk program has been moved to Reinitialize() to avoid spurious recompilation on reinitialize. Signed-off-by: Jussi Maki <jussi@isovalent.com> Signed-off-by: Paul Chaignon <paul.chaignon@gmail.com> | 13 June 2023, 19:22:04 UTC |
| 692f395 | Paul Chaignon | 19 May 2023, 19:38:45 UTC | loader: Allow reinitializeIPSec to run multiple times [ upstream commit e880002be665e96473daced96f809b3b04f81e27 ] reinitializeIPSec only runs the interface detection if EncryptInterface is empty. Since it sets it after detecting interfaces, it will only run the detection once. Let's change that to run the detection even if the EncryptInterface list isn't empty. That will allow us to rerun the detection when new ENI devices are added on EKS. One consequence of this change is that we will now attach to all interfaces even if the user configured --encrypt-interface. That is fine because --encrypt-interface shouldn't actually be used in ENI mode. In ENI mode, we want to attach to all interfaces as we don't have a guarantee on which interface the IPsec traffic will come in. Signed-off-by: Paul Chaignon <paul.chaignon@gmail.com> Signed-off-by: Jussi Maki <jussi@isovalent.com> Signed-off-by: Paul Chaignon <paul.chaignon@gmail.com> | 13 June 2023, 19:22:04 UTC |
| f1a44e9 | Paul Chaignon | 25 May 2023, 19:37:33 UTC | ipsec: Flag to switch between IP types used for IPsec encap [ upstream commit 963e45b1c9a0a0d6420cfed6b0aaabbe45cb630e ] On EKS and AKS, IPsec used NodeInternalIPs for the encapsulation. This commit introduces a new flag to allow switching from NodeInternalIPs to CiliumInternalIPs; it defaults to the former. This new flag allows for step 3 of the migration plan defined in the previous commit. Signed-off-by: Paul Chaignon <paul.chaignon@gmail.com> | 13 June 2023, 19:22:04 UTC |
| a4ce174 | Paul Chaignon | 25 May 2023, 17:35:49 UTC | ipsec: Accept both CiliumInternalIP and NodeInternalIP on decrypt [ upstream commit 6b3b50d2f568bb145b09e5947ebe55df46e5bc3b ] On EKS and AKS, we currently use NodeInternalIPs for the IPsec tunnels. A subsequent commit will allow us to change that to switch to using CiliumInternalIPs (as done on GKE). For that to be possible without breaking inter-node connectivity for the whole duration of the switch, we need an intermediate mode where both CiliumInternalIPs and NodeInternalIPs are accepted on ingress. The idea is that we will then have a two-steps migration from NodeInternalIP to CiliumInternalIP: 1. All nodes are using NodeInternalIP. 2. Upgrade to the version of Cilium that supports both NodeInternalIP and CiliumInternalIP and encapsulates IPsec traffic with NodeInternalIP. 3. Via an agent flag, tell Cilium to switch to encapsulating IPsec traffic with CiliumInternalIP. 4. All nodes are using CiliumInternalIP. This commit implements the logic for step 2 above. To that end, we will duplicate the XFRM IN states such that we have both: src 0.0.0.0 dst [NodeInternalIP] # existing src 0.0.0.0 dst [CiliumInternalIP] # new thus matching and being able to receive IPsec packets with an outer destination IP of either NodeInternalIP or CiliumInternalIP. Signed-off-by: Paul Chaignon <paul.chaignon@gmail.com> | 13 June 2023, 19:22:04 UTC |
| 2acc610 | Paul Chaignon | 25 May 2023, 10:39:20 UTC | ipsec: Reintroduce NodeInternalIPs for EKS & AKS IPsec tunnels [ upstream commit 66c45ace70f1355d44efb9c325694375751a943d ] This is a partial revert of commit 3e59b681f ("ipsec: Per-node XFRM states & policies for EKS & AKS"). One change that commit 3e59b681f ("ipsec: Per-node XFRM states & policies for EKS & AKS") make on EKS and AKS was to switch from using NodeInternalIPs to using CiliumInternalIPs for outer IPsec (ESN) IP addresses. That made the logic more consistent with the logic we use for other IPAM schemes (e.g., GKE). It however causes serious connectivity issues on upgrades and downgrades. This is mostly because typically not all nodes are updated to the new Cilium version at the same time. If we consider two pods on nodes A and B trying to communicate, then node A may be using the old NodeInternalIPs while node B is already on the new CiliumInternalIPs. When node B sends traffic to node A, node A doesn't have the XFRM state IN necessary to decrypt it. The same happens in the other direction. This commit reintroduces the NodeInternalIPs for EKS and AKS. Subsequent commits will introduce additional changes to enable a proper migration path from NodeInternalIPs to CiliumInternalIPs. Signed-off-by: Paul Chaignon <paul.chaignon@gmail.com> | 13 June 2023, 19:22:04 UTC |
| dbcd8a0 | Paul Chaignon | 26 May 2023, 11:41:25 UTC | ipsec: Inverse another set of conditions to reduce indentations [ upstream commit 64f4c23aefa1483e185b492dffeded6655da22e0 ] No functional changes. Best viewed with git show -b or the equivalent on GitHub to not show space-only changes. Same as the previous commit but on a different set of conditions. Signed-off-by: Paul Chaignon <paul.chaignon@gmail.com> | 13 June 2023, 19:22:04 UTC |
| ed3fb59 | Paul Chaignon | 25 May 2023, 13:08:45 UTC | ipsec: Inverse condition to reduce indentations [ upstream commit 2ada282ab10952177041367a94a001bf238798e9 ] No functional changes. Best viewed with git show -b or the equivalent on GitHub to not show space-only changes. Signed-off-by: Paul Chaignon <paul.chaignon@gmail.com> | 13 June 2023, 19:22:04 UTC |
| f9f49b4 | Paul Chaignon | 25 May 2023, 11:11:44 UTC | ipsec: Split enableIPsec into IPv4 and IPv6 [ upstream commit bbc50a3d1d18769c7a4c6751fd2eb40a678536a5 ] This small bit of refactoring will make later changes a bit easier. Signed-off-by: Paul Chaignon <paul.chaignon@gmail.com> | 13 June 2023, 19:22:04 UTC |
| e3e3509 | Paul Chaignon | 03 May 2023, 09:50:59 UTC | ipsec: Don't remove stale XFRM IN configs [ upstream commit 600c7d4846989fb058fbd7ec400fe1a0a499efc7 ] The XFRM IN policies and states didn't change so we should never need to remove any stale XFRM IN configs. Let's thus simplify the logic to find stale policies and states accordingly. I would expect this incorrect removal to cause a few drops on agent restart, but after multiple attempts to reproduce on small (3 nodes) and larger (20) clusters (EKS & GKE) with a drop-sensitive application (migrate-svc), I'm not able to see such drops. I'm guessing this is because we reinstall the XFRM IN configs right after we removed them so there isn't really much time for a packet to be received and dropped. Fixes: 688dc9ac80 ("ipsec: Remove stale XFRM states and policies") Signed-off-by: Paul Chaignon <paul@cilium.io> Signed-off-by: Paul Chaignon <paul.chaignon@gmail.com> | 13 June 2023, 19:22:04 UTC |
| 22e3800 | Anton Protopopov | 13 June 2023, 06:59:15 UTC | Revert "Temporarily disable part of the conformance-kind test" This reverts commit 296303b838acc4a8a580fc5e2199e448344fb742. Signed-off-by: Anton Protopopov <aspsk@isovalent.com> | 13 June 2023, 08:01:36 UTC |
| ce8b131 | Anton Protopopov | 13 June 2023, 06:58:58 UTC | Revert "Set hostServices=true for smoke test" This reverts commit fb34e019488f48ea085f03a1e3adbfe76922114f. Signed-off-by: Anton Protopopov <aspsk@isovalent.com> | 13 June 2023, 08:01:36 UTC |
| 584281a | Anton Protopopov | 12 June 2023, 15:04:39 UTC | test/upgrade: disable check for the number of long-term connections When downgrading from this version to v1.10 (and v1.11.{0,1}) some long-term connections may be reset which breaks the CI, e.g., /home/jenkins/workspace/Cilium-PR-K8s-1.16-kernel-net-next/src/github.com/cilium/cilium/test/ginkgo-ext/scopes.go:527 migrate-svc restart count values do not match Expected <int>: 0 to be identical to <int>: 5 /home/jenkins/workspace/Cilium-PR-K8s-1.16-kernel-net-next/src/github.com/cilium/cilium/test/k8sT/Updates.go:347 Disable this check and document in the Upgrade Guide. Signed-off-by: Anton Protopopov <aspsk@isovalent.com> | 12 June 2023, 23:31:37 UTC |
| d3cf7ff | Anton Protopopov | 07 June 2023, 16:23:15 UTC | datapath: Reduce from LXC complexity [ upstream commit 7575ba03ccaa5038255c8c1e5c31a8011ed9aaa1 ] [ Backport note: Due to many conflicts it was cherry-picked manually. This change also includes the following minor fix: 8dbde7237c ("test/bpf: Fix format of check-complexity.sh script"). ] Split from LXC code paths into two tail calls when per packet load balancing is needed. When per packet load balancing is not needed this should have minimal impact on datapath performance. Kernel 4.9 verifier was unhappy when service load balancing code was removed from handle_ipv6_from_lxc() and replaced with simpler state restoration call, but only if host firewall was enabled, and per-packet LB was on. Had to shuffle code around to make verifier happy again. Shuffled IPv4 code to keep them similar, but git diff looks scarier there. Finally had to conditionally apply a revalidate data call to make verifier happy also in the verifier complexity tests (test-verifier). Signed-off-by: Anton Protopopov <aspsk@isovalent.com> | 12 June 2023, 23:31:37 UTC |
| d675632 | David Bimmler | 12 May 2023, 13:12:11 UTC | wireguard, linuxnodehandler: untangle wg from lnh [ upstream commit c8598f8227665054d05cfeec3f65b15005f087c5 ] [ backporter notes: 1. conflicts due to moved files 2. had to embed NodeHandler in the WireguardAgent interface, so that the fact that the WG agent is a node handler is passed through to the daemon] The reason the WireGuard agent node event handling was contained within the linuxNodeHandler code was routing, which is no longer the case. In addition, entangling the two leads to a deadlock, as diagnosed in GitHub issue #24574. This patch thus implements NodeHandler for the WireGuard agent, and subscribes to the NodeManager itself. That way, the wait cycle of the deadlock is broken, as the linuxNodeHandler doesn't acquire the IPCache lock while holding its lock. From the perspective of the agent, the invocations of the callbacks change insofar that previously, only once the linuxNodeHandler considered itself "initialised" it would forward node events. Specifically, this excluded the initial sync of nodes performed on subscribe. However, I didn't see a reason to specifically replicate this behaviour. Suggested-by: Sebastian Wicki <sebastian@isovalent.com> Signed-off-by: David Bimmler <david.bimmler@isovalent.com> | 12 June 2023, 21:41:23 UTC |
| f7e3800 | David Bimmler | 15 May 2023, 12:12:31 UTC | dp/types: split NodeNeighbors out of NodeHandler [ upstream commit 1aa06c6456d943badfcddb8c5e3765bdd128c469 ] [ backporter notes: conflicts due to moved files] The NodeHandler interface still contains two different concepts. Remove the NodeNeighbor discovery/handling from it, and delete the different stub implementations. Signed-off-by: David Bimmler <david.bimmler@isovalent.com> | 12 June 2023, 21:41:23 UTC |
| 7ad5c9f | David Bimmler | 15 May 2023, 09:38:33 UTC | dp/types: move nodeIDs out of NodeHandler iface [ upstream commit 8294624839da166c8d6c5bbf473895607f56fd62 ] [ backporter notes: 1. conflicts due to moved files 2. had to mildly reshuffle fake datapath to hold a ptr to struct instead of interface, due to the splitting of the NodeHandler interface. ] The NodeHandler interface is too large, as can be seen in various implementations which are only implementing a subset of methods. This patch splits off the NodeID handling part, and removes the stub methods from noop implementations. Signed-off-by: David Bimmler <david.bimmler@isovalent.com> | 12 June 2023, 21:41:23 UTC |
| 9e191f4 | David Bimmler | 15 May 2023, 09:11:02 UTC | datapath/linux: return ptr to struct not interface [ upstream commit d8a0be674d655202277ddafd29729b032e9c2ee0 ] [ backporter notes: conflicts due to moved files ] Return pointer to implementing struct instead of implemented interface from the constructor, as is commonly considered idiomatic Go. The constructor is already in a linux-specific package. To ensure that we really do implement the interface we want, add a static type check in form of a variable assignment. As a side-effect, this allows us to drop a number of type assertions in tests. Signed-off-by: David Bimmler <david.bimmler@isovalent.com> | 12 June 2023, 21:41:23 UTC |
| 049e4d7 | renovate[bot] | 08 June 2023, 16:06:33 UTC | chore(deps): update dependency cilium/hubble to v0.11.6 Signed-off-by: renovate[bot] <bot@renovateapp.com> | 11 June 2023, 01:14:32 UTC |
| 3480b8d | Chance Zibolski | 28 April 2023, 20:30:29 UTC | Add github workflow to push development helm charts to quay.io [ upstream commit d90803cec8e7a1508558b183600cde7dcbdd719f ] [ upstream commit 2c367de92d1825bf2ab80de2fd4e848ee2e28c46 ] [ Backporter's notes: - Removed github workflow. Workflow operates from main branch. - Applied /usr/bin/env hunk to the script (was treewide). ] Signed-off-by: Chance Zibolski <chance.zibolski@gmail.com> Signed-off-by: Joe Stringer <joe@cilium.io> | 09 June 2023, 18:37:57 UTC |
| a8f0b88 | Paul Chaignon | 05 June 2023, 09:16:50 UTC | helm: Value for enable-ipsec-key-watcher [ upstream commit 3ee2fb7dd5f08dc1353266670646584e9dca1b47 ] [ backporter's note: Fixed minor conflict in the Helm template ] This commit adds a Helm value for the enable-ipsec-key-watcher agent flag introduced in the previous commit. Signed-off-by: Paul Chaignon <paul.chaignon@gmail.com> Signed-off-by: Yutaro Hayakawa <yutaro.hayakawa@isovalent.com> | 09 June 2023, 17:26:49 UTC |
| 48bf046 | Paul Chaignon | 05 June 2023, 08:55:45 UTC | ipsec, option: Allow users to disable the IPsec key watcher [ upstream commit a579e9b58fb1cdbbc2c6b88c8f85d50aec98c99c ] [ backporter's note: IPSec key duration option doesn't exist on this branch, so I removed them. Also, this PR contains the commit to add Helm option for ipsec key rotation duration. I talked with an original author and dropped that commit since it is accidentally introduced. ] The IPsec key watcher is used to automatically detect and apply changes in the key (typically during key rotations). Having this watcher avoids having to restart the agents to apply the key change. It can however be desired to only apply the key change when the agent is restarted. It gives control to the user on when exactly the change happens. It may also be used as a way to switch from one IPsec implementation to another (XFRM configs specifically): the user rotates the key just before the upgrade; on upgrade, the SPI is implicitly used to distinguish between the old and new implementations as well as the old and new keys. Signed-off-by: Paul Chaignon <paul.chaignon@gmail.com> Signed-off-by: Yutaro Hayakawa <yutaro.hayakawa@isovalent.com> | 09 June 2023, 17:26:49 UTC |
| c9b2c1f | Jarno Rajahalme | 05 June 2023, 14:58:38 UTC | install: Fail helm if kube-proxy-replacement is not valid [ upstream commit f64e0739d0293ed0e118df48371c4631f4573a06 ] [ backporter's note: Removed variables in the Helm templates that don't exist on this branch. Also, kubeProxyReplacement=probe is still valid in this branch, so I added it to the error condition. ] Fail helm if kube-proxy-replacement is set or defaults to an invalid value. kube-proxy-replacement can be defaulted to a deprecated (and since removed) "probe" value. User can also set it into an incorrect value explicitly. It is better to fail on helm than cilium agent failing to start. Signed-off-by: Jarno Rajahalme <jarno@isovalent.com> Signed-off-by: Yutaro Hayakawa <yutaro.hayakawa@isovalent.com> | 09 June 2023, 17:26:49 UTC |
| 71728fd | Michi Mutsuzaki | 30 May 2023, 20:13:31 UTC | Pick up the latest startup-script image [ upstream commit 5c9b66ce093520b29e03d2ed36f5a2bd5d1b6db4 ] [ backporter's note: Fixed conflict in the install/kubernetes/Makefile.values and regenerated relevant documents. ] Upgrading this image is not automated yet. Ref: #25773 Ref: https://github.com/cilium/image-tools/pull/218 Ref: https://quay.io/repository/cilium/startup-script?tab=tags Signed-off-by: Michi Mutsuzaki <michi@isovalent.com> Signed-off-by: Yutaro Hayakawa <yutaro.hayakawa@isovalent.com> | 09 June 2023, 11:16:01 UTC |
| 186cb98 | Paul Chaignon | 24 April 2023, 09:55:19 UTC | test: Collect sysdump as part of artifacts [ upstream commit e93fdd87b96328847bfe31ab9343ccfef4843b93 ] Once we have a sysdump in the test artifacts a lot of files we collect will become duplicates. This commit however doesn't remove all those duplicate files from the test artifacts. Let's wait a bit and confirm the sysdump collection always work before cleaning things up. The sysdump collection was tested by making a test fail on purpose. Signed-off-by: Paul Chaignon <paul@cilium.io> Signed-off-by: Yutaro Hayakawa <yutaro.hayakawa@isovalent.com> | 09 June 2023, 11:16:01 UTC |
| fb34e01 | Anton Protopopov | 07 June 2023, 12:35:41 UTC | Set hostServices=true for smoke test Setting hostServices=false with KPR=partial increases complexity which breaks the SmokeTest on newer kernels. Disable it temporarily. Signed-off-by: Anton Protopopov <aspsk@isovalent.com> | 08 June 2023, 14:35:32 UTC |
| 296303b | Anton Protopopov | 07 June 2023, 11:25:30 UTC | Temporarily disable part of the conformance-kind test Temporarily disable the ipsec part of the conformance-kind test, as it is currently broken on new kernels (most probably, due to the commit [1] which greatly increases complexity). A proper fix would be to split bpf_lxc.c:from-container into more tail calls, but this is work in progress. [1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=354e8f1970f821d4952458f77b1ab6c3eb24d530 Signed-off-by: Anton Protopopov <aspsk@isovalent.com> | 08 June 2023, 14:35:32 UTC |
| 176a695 | renovate[bot] | 07 June 2023, 18:22:18 UTC | chore(deps): update quay.io/cilium/hubble docker tag to v0.11.6 Signed-off-by: renovate[bot] <bot@renovateapp.com> | 07 June 2023, 22:06:19 UTC |
| 7dc319e | Nate Sweet | 16 March 2023, 17:35:38 UTC | bug: Fix Potential Nil Reference in GetLables Implementation [ upstream commit bfbe5a26a458e114a5b8b261ed719a85a8ceff35 ] The policyIdentityLabelLookup wrapper for Endpoint implements the GetLabels interface method. This is necessary for the constructing the MapState of the policy engine. This implementation incorrectly did not check if the identity returned by LookupIdentityByID was nil. This fixes this bug, which heretofore has not caused any issues. Signed-off-by: Nate Sweet <nathanjsweet@pm.me> | 02 June 2023, 13:46:30 UTC |
| 37a0453 | Joe Stringer | 12 March 2023, 19:21:14 UTC | policy: Fix concurrent access of SelectorCache [ upstream commit 52ace8e9ea318fe79e86731bddbc0abc97843311 ] Marco Iorio reports that with previous code, Cilium could crash at runtime after importing a network policy, with the following error printed to the logs: fatal error: concurrent map read and map write The path for this issue is printed also in the logs, with the following call stack: pkg/policy.(*SelectorCache).GetLabels(...) pkg/policy.(*MapStateEntry).getNets(...) pkg/policy.entryIdentityIsSupersetOf(...) pkg/policy.MapState.denyPreferredInsertWithChanges(...) pkg/policy.MapState.DenyPreferredInsert(...) pkg/policy.(*EndpointPolicy).computeDirectionL4PolicyMapEntries(...) pkg/policy.(*EndpointPolicy).computeDesiredL4PolicyMapEntries(...) pkg/policy.(*selectorPolicy).DistillPolicy(...) pkg/policy.(*cachedSelectorPolicy).Consume(...) pkg/endpoint.(*Endpoint).regeneratePolicy(...) ... Upon further inspection, this call path is not grabbing the SelectorCache lock at any point. If we check all of the incoming calls to this function, we can see multiple higher level functions calling into this function. The following tree starts from the deepest level of the call stack and increasing indentation represents one level higher in the call stack. INCOMING CALLS - f GetLabels github.com/cilium/cilium/pkg/policy • selectorcache.go - f getNets github.com/cilium/cilium/pkg/policy • mapstate.go - f entryIdentityIsSupersetOf github.com/cilium/cilium/pkg/policy • mapstate.go - f denyPreferredInsertWithChanges github.com/cilium/cilium/pkg/policy • mapstate.go - f DenyPreferredInsert github.com/cilium/cilium/pkg/policy • mapstate.go - f computeDirectionL4PolicyMapEntries github.com/cilium/cilium/pkg/policy • resolve.go - f computeDesiredL4PolicyMapEntries github.com/cilium/cilium/pkg/policy • resolve.go + f DistillPolicy github.com/cilium/cilium/pkg/policy • resolve.go <--- No SelectorCache lock - f DetermineAllowLocalhostIngress github.com/cilium/cilium/pkg/policy • mapstate.go + f DistillPolicy github.com/cilium/cilium/pkg/policy • resolve.go <--- No SelectorCache lock - f consumeMapChanges github.com/cilium/cilium/pkg/policy • mapstate.go + f ConsumeMapChanges github.com/cilium/cilium/pkg/policy • resolve.go <--- Already locks the SelectorCache Read the above tree as "GetLabels() is called by getNets()", "getNets() is called by entryIdentityIsSupersetOf()", and so on. Siblings at the same level of indent represent alternate callers of the function that is one level of indentation less in the tree, ie DenyPreferredInsert() and consumeMapChanges() both call denyPreferredInsertWithChanges(). As annotated above, we see that calls through DistillPolicy() do not grab the SelectorCache lock. Given that ConsumeMapChanges() grabs the SelectorCache lock, we cannot introduce a new lock acquisition in any descendent function, otherwise it would introduce a deadlock in goroutines that follow that call path. This provides us the option to lock at some point from the sibling of consumeMapChanges() or higher in the call stack. Given that the ancestors of DenyPreferredInsert() are all from DistillPolicy(), we can amortize the cost of grabbing the SelectorCache lock by grabbing it once for the policy distillation phase rather than putting the lock into DenyPreferredInsert() where the SelectorCache could be locked and unlocked for each map state entry. Future work could investigate whether these call paths could make use of the IdentityAllocator's cache of local identities for the GetLabels() call rather than relying on the SelectorCache, but for now this patch should address the immediate locking issue that triggers agent crashes. CC: Nate Sweet <nathanjsweet@pm.me> Fixes: c9f0def587e6 ("policy: Fix Deny Precedence Bug") Reported-by: Marco Iorio <marco.iorio@isovalent.com> Co-authored-by: Chris Tarazi <chris@isovalent.com> Signed-off-by: Joe Stringer <joe@cilium.io> Signed-off-by: Nate Sweet <nathanjsweet@pm.me> | 02 June 2023, 13:46:30 UTC |
| b6970da | Nate Sweet | 11 October 2022, 02:52:05 UTC | policy: Fix Deny Precedence Bug [ upstream commit c9f0def587e662c2b2ac4501362c6f44aa62ee71 ] - Add Tests for Deny Precedence Bug: Currently, when a broad "Deny" policy is paired with a specific "Unmanaged CIDR" policy, then the "Unmanaged CIDR" policy will still be inserted into the policy map for an endpoint. This results in "Deny" policies not always taking precedence over "Allow" policies. This test confirms the bugs existence. - Fix Deny Precedence Bug: When the policy map state is created CIDRs are now checked against one another to ensure that deny-rules that supersede allow-rules when they should. `DenyPreferredInsert` has been refactored to use utility methods that make the complex boolean logic of policy precedence more atomic. Add `NetsContainsAny` method to `pkg/ip` to compare cases where one set of networks conatins or is equal to any network in another set. - endpoint: Add policy.Identity Implementation A `policy.Identity` implementation is necessary for the incremental update to the endpoint's policy map that can occur with L7 changes. Valid deny-policy entries may prohibit these L7 changes based on CIDR rules, which are only obtainable by looking up all potentially conflicting policies' labels. Thus `l4.ToMapState` needs access to the identity allocater to lookup "random" identity labels. Signed-off-by: Nate Sweet <nathanjsweet@pm.me> | 02 June 2023, 13:46:30 UTC |
| ddeaf64 | Jarno Rajahalme | 25 May 2023, 12:39:57 UTC | envoy: Never use x-forwarded-for header [ upstream commit e8fcd6bfcd91e0eabe7d4049f84b1f706d68bc38 ] Envoy by default gets the source address from the `x-forwarded-for` header, if present. Always add an explicit `use_remote_address: true` for Envoy HTTP Connection Manager configuration to disable the default behavior. Also set the `skip_xff_append: true` option to retain the old behavior of not adding `x-forwarded-for` headers on cilium envoy proxy. Setting these options is not really needed for admin and metrics listeners, or most of the tests, but we add them there too in case anyone uses them as a source of inspiration for a real proxy configuration. This fixes incorrect hubble flow data when HTTP requests contain an `x-forwarded-for` header. This change has no effect on Cilium policy enforcement where the source security identity is always resolved before HTTP headers are parsed. Fixes: #25630 Signed-off-by: Jarno Rajahalme <jarno@isovalent.com> Signed-off-by: Tam Mach <tam.mach@cilium.io> | 01 June 2023, 07:35:43 UTC |
| 887c79b | Paul Chaignon | 21 May 2023, 14:47:09 UTC | test/fqdn: Switch from jenkins.cilium.io to cilium.io [ upstream commit f66f4b159d24e1b5c8e4d92a69932ef003cff87d ] jenkins.cilium.io is down since Thursday. We can simply switch to cilium.io. Signed-off-by: Paul Chaignon <paul.chaignon@gmail.com> Signed-off-by: Nicolas Busseneau <nicolas@isovalent.com> | 24 May 2023, 12:32:25 UTC |
| a7edc23 | Paul Chaignon | 21 May 2023, 14:46:28 UTC | test/fqdn: Avoid hardcoding the test FQDN [ upstream commit 4bcfebc11c29e94cc91a6cc5a027562f9ec0be20 ] Signed-off-by: Paul Chaignon <paul.chaignon@gmail.com> Signed-off-by: Nicolas Busseneau <nicolas@isovalent.com> | 24 May 2023, 12:32:25 UTC |
| 1216e5b | Martynas Pumputis | 18 May 2023, 14:17:32 UTC | test: Use DinD in L4LB tests The DinD for the tests was introduced in [1]. However, it never made into the v1.11 branch which made the GHA always to fail. Fix this by taking the test.sh files from the main branch. [1]: https://github.com/cilium/cilium/pull/22653 Signed-off-by: Martynas Pumputis <m@lambda.lt> | 22 May 2023, 07:04:58 UTC |
| 1689a2c | Jarno Rajahalme | 17 May 2023, 18:04:30 UTC | install: Update image digests for v0.11.17 Generated from https://github.com/cilium/cilium/actions/runs/5006290922. `docker.io/cilium/cilium:v1.11.17@sha256:6c3132e34e66734752de798eb8519dafa77b9f0da1033e9bed7f7be30ce10358` `quay.io/cilium/cilium:v1.11.17@sha256:6c3132e34e66734752de798eb8519dafa77b9f0da1033e9bed7f7be30ce10358` `docker.io/cilium/clustermesh-apiserver:v1.11.17@sha256:022f8b23f9e977a74b8da25ac98fbeed65bd9c132362797681264bd13abc0349` `quay.io/cilium/clustermesh-apiserver:v1.11.17@sha256:022f8b23f9e977a74b8da25ac98fbeed65bd9c132362797681264bd13abc0349` `docker.io/cilium/docker-plugin:v1.11.17@sha256:ed49556f92b95ff339e99938bbd5649d5dc90e8378cb67a820df6bac1979ffa2` `quay.io/cilium/docker-plugin:v1.11.17@sha256:ed49556f92b95ff339e99938bbd5649d5dc90e8378cb67a820df6bac1979ffa2` `docker.io/cilium/hubble-relay:v1.11.17@sha256:d880ee0184f1ca0fffbd73374424ae2c4d1c26af14005a58103ef695816a78ff` `quay.io/cilium/hubble-relay:v1.11.17@sha256:d880ee0184f1ca0fffbd73374424ae2c4d1c26af14005a58103ef695816a78ff` `docker.io/cilium/operator-alibabacloud:v1.11.17@sha256:36999e2fefb8f1ce3a791f60c61055b3bdde350dff5128ce3f4a5fbe31c6f341` `quay.io/cilium/operator-alibabacloud:v1.11.17@sha256:36999e2fefb8f1ce3a791f60c61055b3bdde350dff5128ce3f4a5fbe31c6f341` `docker.io/cilium/operator-aws:v1.11.17@sha256:e96a7d34ed9386a00b0c7d73946f92872280f84addcc951780c42a56dfaeae9c` `quay.io/cilium/operator-aws:v1.11.17@sha256:e96a7d34ed9386a00b0c7d73946f92872280f84addcc951780c42a56dfaeae9c` `docker.io/cilium/operator-azure:v1.11.17@sha256:20cf49d57fdccc599cfefc5a6ab0ed152dac52d45d8a2339fd3ad19415aaebba` `quay.io/cilium/operator-azure:v1.11.17@sha256:20cf49d57fdccc599cfefc5a6ab0ed152dac52d45d8a2339fd3ad19415aaebba` `docker.io/cilium/operator-generic:v1.11.17@sha256:f77cf55ebc47174fb64fd8ffd030015e55817ed9a6bfab46d0ee917a7ed198e5` `quay.io/cilium/operator-generic:v1.11.17@sha256:f77cf55ebc47174fb64fd8ffd030015e55817ed9a6bfab46d0ee917a7ed198e5` `docker.io/cilium/operator:v1.11.17@sha256:c1cad3137dfa80c1d415dff43f064b91992158ce56899b093b0294382ae57289` `quay.io/cilium/operator:v1.11.17@sha256:c1cad3137dfa80c1d415dff43f064b91992158ce56899b093b0294382ae57289` Signed-off-by: Jarno Rajahalme <jarno@isovalent.com> | 17 May 2023, 18:24:14 UTC |
| e86fde3 | Jarno Rajahalme | 15 May 2023, 11:45:51 UTC | Prepare for release v1.11.17 Signed-off-by: Jarno Rajahalme <jarno@isovalent.com> | 15 May 2023, 13:12:15 UTC |
| a96172d | Jarno Rajahalme | 15 May 2023, 06:28:00 UTC | images: update cilium-{runtime,builder} Signed-off-by: Jarno Rajahalme <jarno@isovalent.com> | 15 May 2023, 11:32:12 UTC |
| 0cbf76b | Yongkun Gui | 12 May 2023, 04:47:53 UTC | Update CNI to 1.3.0 [ upstream commit 4ebf4e7a81a8f60154d693ecf4c844f6bdcb62e6 ] Run `images/scripts/update-cni-version.sh 1.3.0` to update the CNI version. Ref: https://github.com/containernetworking/plugins/releases/tag/v1.3.0 Signed-off-by: Yongkun Gui <ygui@google.com> Signed-off-by: Jarno Rajahalme <jarno@isovalent.com> | 15 May 2023, 11:32:12 UTC |
| a4d2a1f | Joe Stringer | 16 June 2022, 20:01:43 UTC | Add helm-toolbox image for helm docs, lint [ upstream commit e4ba2aa24bd7f28291a24a67acabd8e2fdbd09e6 ] Use https://github.com/cilium/helm-toolbox as an image for managing all of our helm formatting & linting needs. This implicitly updates: * helm (version from dev environment) -> 3.9.0 * helm-docs (custom build from Bruno) -> 1.10.0 * m2r 0.2.1 -> m2r2 0.3.2 Signed-off-by: Joe Stringer <joe@cilium.io> | 12 May 2023, 19:04:57 UTC |
| 7a1ad6d | Jarno Rajahalme | 12 May 2023, 05:18:06 UTC | test/provision: Only install bpf mount if not already there Avoid failing VM start due to mount failing: Created symlink /etc/systemd/system/multi-user.target.wants/sys-fs-bpf.mount → /etc/systemd/system/sys-fs-bpf.mount. Failed to restart sys-fs-bpf.mount: Unit sys-fs-bpf.mount has a bad unit file setting. See system logs and 'systemctl status sys-fs-bpf.mount' for details. Signed-off-by: Jarno Rajahalme <jarno@isovalent.com> | 12 May 2023, 11:14:59 UTC |
| a9404a1 | Quentin Monnet | 26 August 2022, 14:42:52 UTC | vagrant: Bump 4.9 Vagrant box (Linux 4.9.326, to fix a kernel bug) [ upstream commit 07e7fb0073ab387108ac6b4c126df1a34e36d5d2 ] We have been hitting a kernel bug on 4.9 for the verifier tests. An underflow on the memlock rlimit counter, caused by the reallocation of BPF programs not updating the charged values, makes the counter go under zero and convert into a huge value, blocking all further loads of BPF objects [0]. This has been fixed in kernel 4.10 [1], and was backported at last in 4.9.326. We generated a new Ubuntu image based on that, let's update. [0] cilium/cilium#20288 [1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?h=5ccb071e97fbd9ffe623a0d3977cc6d013bee93c [ Backport note: Only update the v4.9 image, not the cilium-dev image because version 232 also contains an updated Go version. This is fine for VM images used in tests because they use CI images built by GH actions using the proper Go version for the branch. ] Signed-off-by: Quentin Monnet <quentin@isovalent.com> | 12 May 2023, 11:14:59 UTC |
| dc187e2 | Bryan Stenson | 05 May 2023, 17:34:39 UTC | helm chart: v1.11 base : hubble-ui deployment : restore nodeSelector and tolerations Signed-off-by: Bryan Stenson <bryan.stenson@okta.com> | 12 May 2023, 11:13:23 UTC |
| 5cf233e | Paul Chaignon | 04 May 2023, 08:46:19 UTC | ipsec: Install default-drop XFRM policy sooner [ upstream commit 2045d593f7685a63a25338f9eb85a6da4997ce85 ] We currently install the default-drop XFRM policy when we install the XFRM policies and states for the local node. It is however possible for us to start installing XFRM policies and states for remote nodes before we handle the local one. The default-drop XFRM policy is a safety measure for when we move XFRM policies around. Because we don't always have a way to atomically update XFRM policies, it's possible that we end up with a very short time where no encryption XFRM OUT policy is matching a subset of traffic. The default-drop policy ensures that we drop such traffic instead of letting it leave the node as plain-text. We therefore want this default-drop XFRM policy to be installed before we update any other other XFRM policy. This commit therefore moves its installation before any other XFRM update instead of before just local-node XFRM updates. Fixes: 7d44f37509 ("ipsec: Catch-default default drop policy for encryption") Signed-off-by: Paul Chaignon <paul.chaignon@gmail.com> Signed-off-by: Jarno Rajahalme <jarno@isovalent.com> | 12 May 2023, 05:22:54 UTC |
| 8dc5a04 | David Bimmler | 02 May 2023, 08:04:38 UTC | linux/node_id: do not attempt to map NoID [ upstream commit 9115e05703e92718b1c08e6f8faffa22ac806336 ] We correctly detect that we failed to allocate a new node ID (due to exhaustion of the idpool), but then still go ahead and map it. This leads to spurious errors which include "Failed to map node IP address to allocated ID". Instead, don't try to map NoID and return it directly. Fixes: af88b42bd4 (datapath: Introduce node IDs) Suggested-by: Paul Chaignon <paul@cilium.io> Signed-off-by: David Bimmler <david.bimmler@isovalent.com> Signed-off-by: Jarno Rajahalme <jarno@isovalent.com> | 12 May 2023, 05:22:54 UTC |
| 665be40 | Michi Mutsuzaki | 29 April 2023, 21:52:20 UTC | Delete Cilium monitor verbose mode test [ upstream commit 7335aa38a9c6a148b12b267ff6625a2290bb7d2a ] Another option would be to quarantine the test and find an assignee to make the test more robust, but I assert that we don't need test coverage for monitor verbose output. Fixes: #25178 Signed-off-by: Michi Mutsuzaki <michi@isovalent.com> Signed-off-by: Jarno Rajahalme <jarno@isovalent.com> | 12 May 2023, 05:22:54 UTC |
| ae0ed5a | Alan Kutniewski | 06 April 2023, 13:15:43 UTC | agent: Handle correctly state when CEP is present in multiple CES. [ upstream commit 71af0a2f4147c7706fb98fef0836b4de5eaa7e8a ] [ Backporter's node: Added '!privileged_tests' to the test ] There are condition possible in which CEP changes CES. This leads to CEP being present in multiple CESs for some time. In such cases the standard logic may not work as it always expect to have a single CEP representation. This commit changes to logic to handle multiple CEPs properly. Signed-off-by: Alan Kutniewski <kutniewski@google.com> Signed-off-by: Jarno Rajahalme <jarno@isovalent.com> | 12 May 2023, 05:22:54 UTC |
| 307cd56 | Paul Chaignon | 23 January 2023, 13:12:58 UTC | test: Cover IPsec + VXLAN + endpoint routes [ upstream commit 54fa995c56d914400b977230d0eb34516bf06606 ] [ Backporter's notes: Dropped call to 'helpers.RunsOnAKS()' which does not exist in v1.11 ] The previous commit fixed a connectivity bug affecting the above configuration. We can now extend tests to cover that configuration. Note that these tests are soon going to be removed and replaced by the new GitHub workflow. However, we may need to backport this pull request to stable branches where the GitHub workflow doesn't exist. Therefore, the corresponding extension of the workflow test will be done in a separate pull request. Signed-off-by: Paul Chaignon <paul@cilium.io> Signed-off-by: Jarno Rajahalme <jarno@isovalent.com> | 12 May 2023, 05:22:54 UTC |
| a66ebf4 | Paul Chaignon | 23 January 2023, 13:04:04 UTC | ipsec: Don't match on packet mark for FWD XFRM policy [ upstream commit d39ca10f849060ca90f4a1ddc54734d0e05ba80a ] While extending datapath coverage, Martynas found a new bug affecting IPsec + VXLAN + endpoint routes. In that configuration, cross-node pod connectivity seems to fail and we see the IPsec XfrmInNoPols error counter increasing. By tracing the connection, we can see that the packet disappears on the receiving node, after decryption, between bpf_overlay (second traversal) and the lxc device. On that path, given decryption already happened, we should match the FWD XFRM policy: src 0.0.0.0/0 dst 10.244.0.0/24 uid 0 dir fwd action allow index 106 priority 2975 share any flag (0x00000000) lifetime config: limit: soft (INF)(bytes), hard (INF)(bytes) limit: soft (INF)(packets), hard (INF)(packets) expire add: soft 0(sec), hard 0(sec) expire use: soft 0(sec), hard 0(sec) lifetime current: 0(bytes), 0(packets) add 2023-01-13 14:34:18 use 2023-01-13 14:34:22 mark 0/0xf00 Clearly, given the non-zero XfrmInNoPols, the packet doesn't match the policy. Note XfrmInNoPols is also reported for FWD; there is no XfrmFwdNoPols. Checking the source code [1], we can see that when endpoint routes are enabled, we encode the source security identity into the packet mark. We thus won't match the 0/0xf00 mark on the FWD XFRM policy. We shouldn't need to match on any packet mark for the FWD XFRM policy; we want to allow all packets through. This commit therefore removes the packet mark match for the FWD direction. 1 - https://github.com/cilium/cilium/blob/v1.13.0-rc4/bpf/lib/l3.h#L151-L154 Signed-off-by: Paul Chaignon <paul@cilium.io> Signed-off-by: Jarno Rajahalme <jarno@isovalent.com> | 12 May 2023, 05:22:54 UTC |
| 421d611 | renovate[bot] | 08 May 2023, 15:13:19 UTC | chore(deps): update hubble cli to v0.11.5 Signed-off-by: renovate[bot] <bot@renovateapp.com> | 11 May 2023, 17:16:44 UTC |
| d3bf9d7 | Casey Callendrello | 21 February 2023, 15:44:29 UTC | agent: dump stack on stale probes [ backport of d85c0939824ea3508f2a1b0ef56ddbc8d197e588 ] [ upstream commit 87f7a11ecc68b1efdc1454b520abc22470a91d01 ] Most of the time, when we see a stale probe, it's due to a deadlock. So, write a stack dump to disk (since we're probably going to be restarted soon due to a liveness probe). To prevent any sort of excessive resource consumption, only dump stack once every 5 minutes, and always write to the same file. Also, let's make the check lock-free while we're at it. Also, make sure we capture this file in bugtool. Signed-off-by: Casey Callendrello <cdc@isovalent.com> | 11 May 2023, 17:15:49 UTC |
| 0834b37 | Tom Hadlaw | 01 May 2023, 19:38:47 UTC | inctimer: fix test flake where timer does not fire within time. [ upstream commit e695e48b171be3e60ecd8ef56c99b1b71ba1316c ] Running the test in a cpu constrained environment, such as: ``` docker run -v $(pwd):$(pwd) -w $(pwd) --cpus=0.1 -it golang:bullseye ./inctimer.test -test.v ``` I can fairly consistency reproduce a flake where the inctimer.After does not fire in time. If I allow it to wait for an additional couple of ms, this seems to be sufficient to prevent failure. It appears that goroutine scheduling latency can be significantly delayed in cpu restricted environments. This seems unavoidable, so to fix the flake I'll allow the test to wait another 2ms to see if the inctimer eventually fires. This will also log an error for delayed test fires, so if there is any other issues we can more easily debug them in the future. Fixed: #25202 Signed-off-by: Tom Hadlaw <tom.hadlaw@isovalent.com> Signed-off-by: Jussi Maki <jussi@isovalent.com> Signed-off-by: Yutaro Hayakawa <yutaro.hayakawa@isovalent.com> | 11 May 2023, 09:57:54 UTC |
| 8fc726d | Joe Stringer | 27 April 2023, 20:43:37 UTC | docs: Add platform support to docs [ upstream commit 9a38aecc71b34c4b6ae95fcadd126f77ebb200ad ] We've been distributing ARM architecture images for Cilium for almost two years, but neglected to mention this up front in the system requirements or the main docs page. Add this to the docs. Signed-off-by: Joe Stringer <joe@cilium.io> Signed-off-by: Jussi Maki <jussi@isovalent.com> Signed-off-by: Yutaro Hayakawa <yutaro.hayakawa@isovalent.com> | 11 May 2023, 09:57:54 UTC |
| 9edbc6e | Alexandre Perrin | 28 April 2023, 11:12:25 UTC | Makefile: use a specific template for mktemp files [ upstream commit db3e0152c6583e0cbec1013e514526adf7229faf ] Before this patch, we would hit a controller-gen[1] bug when the temporary file would be of the form tmp.0oXXXXXX. This patch uses a custom mktemp template that will not trigger the bug. [1]: https://github.com/kubernetes-sigs/controller-tools/issues/734 Signed-off-by: Alexandre Perrin <alex@isovalent.com> Signed-off-by: Jussi Maki <jussi@isovalent.com> Signed-off-by: Yutaro Hayakawa <yutaro.hayakawa@isovalent.com> | 11 May 2023, 09:57:54 UTC |
| 6b920c6 | Tam Mach | 25 April 2023, 12:50:37 UTC | docs: Add matrix version between envoy and cilium [ upstream commit 11e1bcc40fd4ba884191f594b8c6fd7975b1caaf ] This is to add a small docs for version matrix between Cilium and Cilium envoy versions, which is useful with the upcoming work to move envoy proxy out of Cilium agent container. Co-authored-by: ZSC <zacharysarah@users.noreply.github.com> Signed-off-by: Tam Mach <sayboras@yahoo.com> Signed-off-by: Jussi Maki <jussi@isovalent.com> Signed-off-by: Yutaro Hayakawa <yutaro.hayakawa@isovalent.com> | 11 May 2023, 09:57:54 UTC |
| 7f90b0d | Marco Iorio | 21 April 2023, 10:21:49 UTC | helm: add clustermesh nodeport config warning about #24692 [ upstream commit 9e83a6f79940c86a95fff33de89d7ded225da25c ] Cilium is currently affected by a known bug (#24692) when NodePorts are handled by the KPR implementation, which occurs when the same NodePort is used both in the local and the remote cluster. This causes all traffic targeting that NodePort to be redirected to a local backend, regardless of whether the destination node belongs to the local or the remote cluster. This affects also the clustermesh-apiserver NodePort service, which is configured by default with a fixed port. Hence, let's add a warning message to the corresponding values file setting. Signed-off-by: Marco Iorio <marco.iorio@isovalent.com> Signed-off-by: Jussi Maki <jussi@isovalent.com> Signed-off-by: Yutaro Hayakawa <yutaro.hayakawa@isovalent.com> | 11 May 2023, 09:57:54 UTC |
| 931ebc1 | Tam Mach | 29 April 2023, 04:26:03 UTC | envoy: Upgrade to v1.23.9 This commit is to upgrade envoy to v1.23.9 for security fixes, please find below the details: Build: https://github.com/cilium/proxy/actions/runs/4827955904/jobs/8601231172 Upstream Docs: https://www.envoyproxy.io/docs/envoy/v1.23.9/ Release notes: https://www.envoyproxy.io/docs/envoy/v1.23.9/version_history/v1.23/v1.23.9 Signed-off-by: Tam Mach <tam.mach@cilium.io> | 29 April 2023, 12:37:42 UTC |
| e026bf6 | Nicolas Busseneau | 21 April 2023, 15:51:45 UTC | ci: remove `STATUS` commands from upstream tests' Jenkinsfile [ upstream commit 46de5bca9fc77cb2c02ba2873a30649dbc1b78b6 ] These are remnants of a past before GHPRB. At best they create uncessary noise in the logs, at worst they can interfere with the default behaviour, so let's just remove them. Signed-off-by: Nicolas Busseneau <nicolas@isovalent.com> Signed-off-by: Tam Mach <tam.mach@cilium.io> | 28 April 2023, 14:51:34 UTC |
| bbddee0 | harsimran pabla | 21 April 2023, 14:49:14 UTC | bgp: do not advertise ipv6 prefixes via metallb [ upstream commit 922aa1bcabd1c75c568bf56a2ca2a5eb7e624ba9 ] When using metallb as BGP speaker, if IPv6 advertisement is made - metallb will return error as unsupported. This error is logged and error is returned to control loop, which continues retrying causing log flooding and high CPU. This change filters out IPv6 prefixes before sending them to metallb library and logs one time error message. Signed-off-by: harsimran pabla <hpabla@isovalent.com> Signed-off-by: Tam Mach <tam.mach@cilium.io> | 28 April 2023, 14:51:34 UTC |
| be630a3 | Tam Mach | 26 April 2023, 12:29:50 UTC | contrib/backporting: Fix main branch reference The "master" branch was renamed to "main" recently. This commit is to adjust branch reference for chery-pick script. Signed-off-by: Tam Mach <tam.mach@cilium.io> | 26 April 2023, 14:57:01 UTC |
| bac3efe | Marco Iorio | 06 April 2023, 16:06:00 UTC | wireguard: fix issue caused by nodes with the same name in clustermesh [ upstream commit 7398de68ca940a839915107e71c00b9661bf423b ] Currently, the wireguard subsystem in the cilium agent caches information about the known peers by node name only. This can lead to conflicts in case of clustermesh, if nodes in different clusters have the same name, causing in turn connectivity issues. Hence, let's switch to identify peers by full name (i.e., cluster-name/node-name) to ensure uniqueness. This modification does not introduce issues during upgrades, since the node ID is not propagated to the datapath. Fixes: #24227 Reported-by: @oulinbao <oulinbao@163.com> Signed-off-by: Marco Iorio <marco.iorio@isovalent.com> Signed-off-by: Nicolas Busseneau <nicolas@isovalent.com> | 26 April 2023, 10:54:26 UTC |
| 0523a23 | Paul Chaignon | 12 April 2023, 20:45:40 UTC | daemon: Mark CES feature as beta in agent flag [ upstream commit a6d0142ec8f093bfb41a042e8d2cc38eff9e1cf3 ] This commit marks the CiliumEndpointSlice feature as beta (as per the documentation) in the agent flag description. This is necessary because users don't always read the full documentation before turning agent flags on. While at it, change the flag description to match the wording of other flags. Signed-off-by: Paul Chaignon <paul@cilium.io> Signed-off-by: Nicolas Busseneau <nicolas@isovalent.com> | 26 April 2023, 10:54:26 UTC |
| a5eb447 | Hemanth Malla | 06 April 2023, 18:07:09 UTC | pkg/kvstore: Fix for deadlock in etcd status checker [ upstream commit 9bb669b5705f3b283f3fac8f79b760987066d06d ] Etcd quorum checks are falsely reported as failing even though connection to etcd is intact. This can cause health checks to fail in both the agent and the operator. This happens due to a deadlock in pkg/kvstore/etcd after a prolonged downtime of etcd. Status check errors are being sent into a channel for the purpose of recreating kvstore connections in clustermesh. However when clustermesh is not used, messages from this channel are never read. The channel uses a buffer of size 128. After etcd has been down long enough to generate 128 errors, we enter a deadlock state. Agent / operator will continue to report etcd quorum failures and inturn health check failures until they're restarted. statusChecker() -> isConnectedAndHasQuorum() -> waitForInitLock() -> goroutine -> for -> ( initLockSucceeded <- err ) -> chan initLockSucceeded returned -> Block on receiving messages from initLockSucceeded channel -> e.statusCheckErrors <- e.latestErrorStatus [Blocked after 128 entries] Blocked goroutines captured from cilium 1.10 operator: goroutine 3309 [chan send, 13456 minutes]: github.com/cilium/cilium/pkg/kvstore.(*etcdClient).statusChecker(0xc00017db30) /go/src/github.com/cilium/cilium/pkg/kvstore/etcd.go:1171 +0x75a created by github.com/cilium/cilium/pkg/kvstore.connectEtcdClient /go/src/github.com/cilium/cilium/pkg/kvstore/etcd.go:801 +0x679 goroutine 7838665 [chan send, 13505 minutes]: g.com/c/cilium/pkg/kvstore.(*etcdClient).waitForInitLock.func1(-,-,-,-) /go/src/github.com/cilium/cilium/pkg/kvstore/etcd.go:433 +0x449 created by github.com/cilium/cilium/pkg/kvstore.(*etcdClient).waitForInitLock /go/src/github.com/cilium/cilium/pkg/kvstore/etcd.go:425 +0x7f Signed-off-by: Hemanth Malla <hemanth.malla@datadoghq.com> Signed-off-by: Nicolas Busseneau <nicolas@isovalent.com> | 26 April 2023, 10:54:26 UTC |
| c884102 | Joe Stringer | 01 June 2022, 01:38:06 UTC | .travis: Quieten docker build output [ upstream commit 7f9e0f9b7b9fe26d707217e8248b86dde21dae4d ] The travis logs are frequently polluted with >10K lines of docker pull and build output. While this helps to track the ongoing progress of docker builds that take a long time, it's mostly useless output that developers must scroll past in order to see the useful output. Quieten that output in Travis to just the trigger of building the image plus the final summary that docker outputs. Signed-off-by: Joe Stringer <joe@cilium.io> Signed-off-by: Nicolas Busseneau <nicolas@isovalent.com> | 26 April 2023, 10:54:26 UTC |
| d7eef48 | Joe Stringer | 26 May 2022, 21:58:53 UTC | .travis: Make output less verbose [ upstream commit 9f7e24fd7e5c355e943528ac3074a2485a14b2ad ] Pass the verbosity parameters --quiet V=0 to quieten Travis output. Signed-off-by: Joe Stringer <joe@cilium.io> Signed-off-by: Nicolas Busseneau <nicolas@isovalent.com> | 26 April 2023, 10:54:26 UTC |
| 3309e28 | Joe Stringer | 26 May 2022, 22:00:00 UTC | Makefile: Fix dirname errors with empty PRIV_TEST_PKGS [ upstream commit 29fe753e475fece540aac3fafd288147515bab08 ] When TESTPKGS only contains unprivileged tests, the PRIV_TEST_PKGS_EVAL evaluation previously filtered down to an empty list of packages that should be tested, and would pass this empty list to dirname, which then reports: dirname: missing operand Try 'dirname --help' for more information. This could happen multiple times during evaluation of the Makefile, and littered the output with no meaning. This could occur even if the privileged tests are not the target being run. Fix this by always adding "." to the list, which evaluates to the root directory of the repository. This causes dirname to succeed. Then, we can filter this root directory back out since there are no privileged tests at this level of the repository. This finally quietens the error. Signed-off-by: Joe Stringer <joe@cilium.io> Signed-off-by: Nicolas Busseneau <nicolas@isovalent.com> | 26 April 2023, 10:54:26 UTC |
| 44ebf04 | Joe Stringer | 26 May 2022, 22:04:24 UTC | test/bpf: Fix compilation with V=0 [ upstream commit 82d5adc7951de3d14bb2406112314d93d3d606bb ] When the quiet mode was enabled, the $(CLANG) var would previously have a '@' at the start, which caused errors while attempting to make in this directory because it would be run in the context of a shell rather than directly as a make instruction. Move the $(QUIET) to the start of individual make instructions to resolve this compilation failure. Signed-off-by: Joe Stringer <joe@cilium.io> Signed-off-by: Nicolas Busseneau <nicolas@isovalent.com> | 26 April 2023, 10:54:26 UTC |
| e257f08 | Joe Stringer | 24 April 2023, 17:33:47 UTC | contrib/backporting: Fix main branch reference The "master" branch was renamed to "main" recently. Fix this for the older branch here. Signed-off-by: Joe Stringer <joe@cilium.io> | 25 April 2023, 13:45:50 UTC |
| 9012f69 | Paul Chaignon | 18 April 2023, 22:05:41 UTC | docs: Document upgrade impact for IPsec The IPsec upgrade issue mentioned in ede154e27b ("Add IPSec remark for upgrade to v1.11.15") is fixed in v1.11.16. Nonetheless, a small impact remains, with a few packet drops happening during the upgrade. This commit documents that impact. Signed-off-by: Paul Chaignon <paul@cilium.io> | 19 April 2023, 17:04:48 UTC |
| 4da9949 | André Martins | 17 April 2023, 22:37:15 UTC | jenkins: bump timeout to 210 minutes The net-next test has been timing out at the 2h50m mark. We need to increase its timeout in order to avoid such failures. Signed-off-by: André Martins <andre@cilium.io> | 18 April 2023, 18:43:21 UTC |
| 13d1197 | Maxim Mikityanskiy | 18 April 2023, 13:31:42 UTC | install: Update image digests for v1.11.16 Generated from https://github.com/cilium/cilium/actions/runs/4731386331. ## Docker Manifests ### cilium `docker.io/cilium/cilium:v1.11.16@sha256:d2f2632c997a027ee4e540432edb4d8594e78e33315427e7ec3c06b473ec1e4e` `quay.io/cilium/cilium:v1.11.16@sha256:d2f2632c997a027ee4e540432edb4d8594e78e33315427e7ec3c06b473ec1e4e` ### clustermesh-apiserver `docker.io/cilium/clustermesh-apiserver:v1.11.16@sha256:67a051ef38ae113bcf7dc27ebb23a1137ece961ce86f087226ff5a0046099106` `quay.io/cilium/clustermesh-apiserver:v1.11.16@sha256:67a051ef38ae113bcf7dc27ebb23a1137ece961ce86f087226ff5a0046099106` ### docker-plugin `docker.io/cilium/docker-plugin:v1.11.16@sha256:1ee1bae0c2299d94ff162fc2847f9827823ff3d8e055e07da06e4ca28efe9391` `quay.io/cilium/docker-plugin:v1.11.16@sha256:1ee1bae0c2299d94ff162fc2847f9827823ff3d8e055e07da06e4ca28efe9391` ### hubble-relay `docker.io/cilium/hubble-relay:v1.11.16@sha256:c4c12759ba628e64a0f3fada99d2632627e5391ae0b49c3f35da51c3ba9eac9f` `quay.io/cilium/hubble-relay:v1.11.16@sha256:c4c12759ba628e64a0f3fada99d2632627e5391ae0b49c3f35da51c3ba9eac9f` ### operator-alibabacloud `docker.io/cilium/operator-alibabacloud:v1.11.16@sha256:d60aedfabf0957da1d975ee54779172f990366e9fb8bf55184ac31a0d77adc65` `quay.io/cilium/operator-alibabacloud:v1.11.16@sha256:d60aedfabf0957da1d975ee54779172f990366e9fb8bf55184ac31a0d77adc65` ### operator-aws `docker.io/cilium/operator-aws:v1.11.16@sha256:526dab3bee6231f71da44d14f25c17dfb53afba876bfc99374a11c0fb4278e36` `quay.io/cilium/operator-aws:v1.11.16@sha256:526dab3bee6231f71da44d14f25c17dfb53afba876bfc99374a11c0fb4278e36` ### operator-azure `docker.io/cilium/operator-azure:v1.11.16@sha256:0c2da6adf29f521f6d2ffe92794ad598fc99231eba2814b80cf608362cc14a3c` `quay.io/cilium/operator-azure:v1.11.16@sha256:0c2da6adf29f521f6d2ffe92794ad598fc99231eba2814b80cf608362cc14a3c` ### operator-generic `docker.io/cilium/operator-generic:v1.11.16@sha256:ea3fbe5ab65efc41228d716a64804b6fca9e2299835c3d39ae1cb248c1594c55` `quay.io/cilium/operator-generic:v1.11.16@sha256:ea3fbe5ab65efc41228d716a64804b6fca9e2299835c3d39ae1cb248c1594c55` ### operator `docker.io/cilium/operator:v1.11.16@sha256:44fb99adbba82605702aa9c41380c1c79ad5565bbd3c9d961f9aab55387be586` `quay.io/cilium/operator:v1.11.16@sha256:44fb99adbba82605702aa9c41380c1c79ad5565bbd3c9d961f9aab55387be586` Signed-off-by: Maxim Mikityanskiy <maxim@isovalent.com> | 18 April 2023, 13:49:46 UTC |
| 9ce6d64 | André Martins | 17 April 2023, 22:32:26 UTC | update CHANGELOG update changelog with the PRs that got merged after the last CHANGELOG but before tagging Signed-off-by: André Martins <andre@cilium.io> | 17 April 2023, 22:34:46 UTC |
| 54967c6 | André Martins | 17 April 2023, 16:44:08 UTC | Avoid clearing objects in conversion funcs [ upstream commit 0d7406af7bf05e1c0c49c098a3b4e025d927c3e7 ] This removes the behavior of mutating the objects received from the client-go library. To begin with there isn't really any benefit from doing so, given we don't store the object afterwards, and it will be ready for gc when it leaves the scope inside client-go. client-go can possibly return the same pointer twice here, to trigger eg. both an object update delta and then a DeletedFinalStateUnknown delta with the same pointer. For more info, see the issue 115658 in the kubernetes/kubernetes repo on github. Follow-up of: 74307f175ceb ("Avoid clearing objects in conversion funcs") Signed-off-by: André Martins <andre@cilium.io> | 17 April 2023, 22:28:49 UTC |
| e80ee2c | Odin Ugedal | 01 March 2023, 16:16:48 UTC | Avoid clearing objects in conversion funcs [ upstream commit 74307f175cebc3e097eaa4eb4e930bf26ebb8cb1 ] This removes the behavior of mutating the objects received from the client-go library. To begin with there isn't really any benefit from doing so, given we don't store the object afterwards, and it will be ready for gc when it leaves the scope inside client-go. client-go can possibly return the same pointer twice here, to trigger eg. both an object update delta and then a DeletedFinalStateUnknown delta with the same pointer. For more info, see the issue 115658 in the kubernetes/kubernetes repo on github. Signed-off-by: Odin Ugedal <ougedal@palantir.com> Signed-off-by: Odin Ugedal <odin@uged.al> Signed-off-by: André Martins <andre@cilium.io> | 17 April 2023, 22:28:49 UTC |
| fd81a6b | Tam Mach | 15 April 2023, 00:02:29 UTC | envoy: Bump envoy to v1.23.8 https://github.com/cilium/proxy/actions/runs/4698873584/jobs/8331675690 Relates: https://github.com/cilium/proxy/pull/172 Signed-off-by: Tam Mach <tam.mach@cilium.io> | 16 April 2023, 00:44:57 UTC |
| 4190a2d | Tam Mach | 15 April 2023, 00:05:03 UTC | envoy: Support more envoy image tag formats [upstream commit afcda947] This commit is to add the support for below image tags Different envoy image tag formats: ``` quay.io/cilium/cilium-envoy:f195a0a836629ceca5d7561f758c9505d9ebaebfa262647a2d4 quay.io/cilium/cilium-envoy:v1.23-f195a0a836629ceca5d7561f758c9505d9ebaebfa262647a2d4 ``` Testing was done as per below, kindly note the existing format should be working as usual. ```bash $ test=quay.io/cilium/cilium-envoy:014ceeb312a4d18dcf0ea219143f099fa91f2f28@sha256:1a3020822e8fb10b5f96bf45554690c411c2f48d8ca8fcf33da871dad1ce6b53 $ echo $test | sed -E -e 's/[^/]*\/[^:]*:(.*-)?([^:@]*).*/\2/p;d' 014ceeb312a4d18dcf0ea219143f099fa91f2f28 $ test=quay.io/cilium/cilium-envoy:v1.24-014ceeb312a4d18dcf0ea219143f099fa91f2f28@sha256:1a3020822e8fb10b5f96bf45554690c411c2f48d8ca8fcf33da871dad1ce6b53 $ echo $test | sed -E -e 's/[^/]*\/[^:]*:(.*-)?([^:@]*).*/\2/p;d' 014ceeb312a4d18dcf0ea219143f099fa91f2f28 ``` Fixes: #24749 Signed-off-by: Tam Mach <tam.mach@cilium.io> | 16 April 2023, 00:44:57 UTC |
| cf28bba | Michi Mutsuzaki | 14 April 2023, 00:35:35 UTC | Prepare for release v1.11.16 Signed-off-by: Michi Mutsuzaki <michi@isovalent.com> | 14 April 2023, 11:41:35 UTC |
| 36eaa38 | André Martins | 13 April 2023, 13:43:57 UTC | update k8s dependencies to 1.23.17 The last client-go version contains an important bug fix. See https://github.com/kubernetes/kubernetes/pull/115901 for more info. Signed-off-by: André Martins <andre@cilium.io> | 13 April 2023, 23:09:29 UTC |
| f490339 | Maartje Eyskens | 05 April 2023, 07:32:59 UTC | Remove HTTP header value from debug log Signed-off-by: Maartje Eyskens <maartje.eyskens@isovalent.com> | 13 April 2023, 18:13:22 UTC |
| 00630cc | Paul Chaignon | 03 April 2023, 13:58:35 UTC | ipsec: Remove stale XFRM states and policies [ upstream commit 688dc9ac802b11f6c16a9cbc5d60baaf77bd6ed0 ] We recently changed our XFRM states and policies (IPs and marks). We however failed to remove the stale XFRM states and policies and it turns out that they conflict (e.g., the kernel ends up picking the stale policies for encryption instead of the new one). This commit therefore cleans up those stale XFRM states and policies. We can identify them based on mark values and masks (we switched from 0xFF00 to 0XFFFFFF00). The new XFRM states and policies are added as we receive the information on remote nodes. By removing the stale states and policies before the new ones are installed for all nodes, we could cause plain-text traffic on egress and packet drops on ingress. To ensure we never let plain-text traffic out, we will clean up the stale config only once the catch-all default-drop policy is installed. In that way, if there is a brief moment where, for a connection nodeA -> nodeB, we don't have a policy, traffic will be dropped instead of sent in plain-text. For each connection nodeA -> nodeB, those packet drops on egress and ingress of nodeA will happen between the time we replace the BPF datapath and the time we've installed the new XFRM state and policy corresponding to nodeB. Waiting longer to remove the stale states and policies doesn't impact the drops as they will keep happening until the new states and policies are installed. This is all happening on agent startup, as soon as we have the necessary information from k8s. Signed-off-by: Paul Chaignon <paul@cilium.io> | 13 April 2023, 14:13:44 UTC |
| f76ce36 | Paul Chaignon | 03 April 2023, 12:58:33 UTC | ipsec: Catch-default default drop policy for encryption [ upstream commit 7d44f37509c6271f7196dcec8edc7c417c609dca ] This commit adds a catch-all XFRM policy for outgoing traffic that has the encryption bit. The goal here is to catch any traffic that may passthrough our encryption while we are replacing XFRM policies & states. Those operations cannot always be performed atomically so we may have brief moments where there is no XFRM policy to encrypt a subset of traffic. This policy ensures we drop such traffic and don't let it flow in plain text. We do need to match on the mark because there is also traffic flowing through XFRM that we don't want to encrypt (e.g., hostns traffic). Signed-off-by: Paul Chaignon <paul@cilium.io> | 13 April 2023, 14:13:44 UTC |
