https://github.com/cilium/cilium

sort by:
Revision Author Date Message Commit Date
086b27e Prepare for v1.5.9 release Signed-off by: Ian Vernon <ian@cilium.io> 08 October 2019, 16:02:29 UTC
3aeee07 envoy: Update image for Envoy CVEs 2019-10-08 [ upstream commit 6f60b4fd8fdb61c29ff1ece6126c0f1cb3eb00f9 ] Signed-off-by: Jarno Rajahalme <jarno@covalent.io> 08 October 2019, 04:45:07 UTC
4fa8979 dockerfile.runtime: always run update when building dependencies Signed-off-by: André Martins <andre@cilium.io> 26 September 2019, 15:15:47 UTC
9de7883 go: bump golang to 1.12.10 Signed-off-by: André Martins <andre@cilium.io> 26 September 2019, 15:15:47 UTC
c420232 loader: remove hash from compileQueue if build fails [ upstream commit 6a15e2b58ce722e102e2483fd6221b3f4007333b ] Otherwise, the loader erroneously thinks that the template has been built, while it actually needs to be rebuilt again. Note that if the template cannot be built, we do not have any logic to signal to other waiters on this queue that the build failed and that said waiters should try to rebuild the program; all waiters will fail as well. We also do not have retry logic if endpoints are not regenerated, so all endpoints which were waiting on a given template to be compiled will not be rebuilt. The retry logic is out of the scope of this commit. Fixes: 561100e141 Related-to: #8817 Signed-off by: Ian Vernon <ian@cilium.io> Signed-off-by: Maciej Kwiek <maciej@isovalent.com> 25 September 2019, 14:48:53 UTC
ca23ffd test: bump k8s testing versions to 1.13.11, 1.14.7 and 1.15.4 [ upstream commit 4abca2a417bd2dde44415ec07394a855b57c495f ] Signed-off-by: André Martins <andre@cilium.io> Signed-off-by: Maciej Kwiek <maciej@isovalent.com> 25 September 2019, 14:48:53 UTC
a631a22 cilium: encryption, replace Router() IP with CiliumInternal [ upstream commit e3aa68f7d32e39f957b58f2b35a799babfa43ae2 ] Apparently I thought the RouterIP _should_ be the same as the CiliumInternalIP for a local Node. And this is usually the case, but it is not required. We have had a couple reports of cases where encryption was not able to connect because of this. Fix by using the CiliumInternalIP in all cases. Fixes: 05ea001368660 ("cilium: populate wildcard src->dst policy for ipsec") Signed-off-by: John Fastabend <john.fastabend@gmail.com> Signed-off-by: Maciej Kwiek <maciej@isovalent.com> 25 September 2019, 14:48:53 UTC
6b936b6 Prepare for v1.5.8 release Signed-off by: Ian Vernon <ian@cilium.io> 09 September 2019, 22:07:02 UTC
cff47cf test: Wait for at least one Istio POD to get ready [ upstream commit 02eafbeb2637add48b1721e3f330665c9415ccf5 ] WaitforPods() succeeds if there are no PODs matching the given filter ("-l istio" in this case). Wait for at least one Istio POD to get Ready before waiting for all of them to get Ready. Signed-off-by: Jarno Rajahalme <jarno@covalent.io> 07 September 2019, 19:07:38 UTC
1d96b4d test: fix k8s upstream test [ upstream commit 900c5cb5d880701e326cb8660708782144ce20af ] Since kubetest repo switched to go modules we need to pick the last commit that still has a vendor directory so we can compile kubetest manually. Signed-off-by: André Martins <andre@cilium.io> 06 September 2019, 21:14:10 UTC
40746d5 Dockerfile: Use latest Envoy image [ upstream commit 33f020884fa8589158e995488c4fd61707cde2d8 ] Use Envoy image with access logging fixes: - Use fresh timestamp for the response messages instead of repeating the request timestamp - Correctly parse status codes from HTTP headers into access log messages Signed-off-by: Jarno Rajahalme <jarno@covalent.io> Signed-off-by: André Martins <andre@cilium.io> 06 September 2019, 21:14:10 UTC
de6ca3c bump k8s support to 1.15.3 [ upstream commit 2f1d9c24108d53793a6a1962c17c985932af87aa ] Signed-off-by: André Martins <andre@cilium.io> 06 September 2019, 21:14:10 UTC
e38e307 tofqdns: Allow "_" in DNS names to support service discovery schemes [ upstream commit f00f85772ac154fce07149935e541726a0d4dc4b ] DNS SRV records (RFC-2782 https://tools.ietf.org/html/rfc2782) and DNS-SD RFC-6736 https://tools.ietf.org/html/rfc6763 explicitly allow and expect leading "_"s in the RR names. Furthermore, DNS itself isn't restrictive about this. We add support for these types of names since they are more common than arbitrary binary names, but we may further loosen the requirements here later. Signed-off-by: Ray Bejjani <ray@isovalent.com> Signed-off-by: André Martins <andre@cilium.io> 06 September 2019, 21:14:10 UTC
1dac44e cilium: make all ct timeouts configurable [ upstream commit 201af221572a6fd07da2cab190462d461ca25919 ] Make the CT timeouts configurable for expert users. Depending on the use-case and expected traffic workloads the defaults may not be desirable e.g. shortening ct service timeouts would avoid stickyness to specific backends for the same tuples for the default time span. Accepted timing inputs are those that time.ParseDuration() will eat. Fixes: #9110 Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> 05 September 2019, 12:33:33 UTC
55e797b bpf: add separate ct_service lifetime for tcp/non-tcp [ upstream commit e9d906baeedf18fb498c1eefcb0f39521930c920 ] Goal is to make lifetimes configurable va cmdline, split off ct_service lifetime from those of normal connections. Defaults are kept the same. Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> 05 September 2019, 12:33:33 UTC
7f2c0f6 bpf: usr prandom as slave selection in lb [ upstream commit c85648b24a089b2923aedc5c022017229edaeac4 ] Given we track established connections via CT anyway, we can spare cycles and instruction complexity by just using prandom. Plus it might also result in a better distribution. Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> 05 September 2019, 12:33:33 UTC
21b014f istio: Update to 1.2.5 [ upstream commit ea07a581a06c0fe4fef2f2c9e95cf7824f718e2c ] Signed-off-by: Jarno Rajahalme <jarno@covalent.io> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> 28 August 2019, 21:46:24 UTC
3f08397 docs: Update direct routing policy limitation [ upstream commit 55a242ddb3234390ec84c40fd9e6095ca4232dc0 ] Signed-off-by: Joe Stringer <joe@cilium.io> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> 28 August 2019, 21:46:24 UTC
6373fda Prepare for v1.5.7 Signed-off by: Ian Vernon <ian@cilium.io> 23 August 2019, 17:46:49 UTC
9b49d9e cilium: update IsEtcdCluster to return true if etcd.operator="true" kv option is set [ upstream commit 5e52b9651761ed09b910476c57405627ec8f541f ] [ upstream commit: none ] Signed-off-by: Rajat Jindal <rajatjindal83@gmail.com> Signed-off-by: Michal Rostecki <mrostecki@opensuse.org> 23 August 2019, 10:19:12 UTC
44ff79f bpf: try to atomically replace filters when possible [ upstream commit cc01f5294ab4128c657518158e16423d962ed9b6 ] tc qdisc del/add combination and same with tc filters is very suboptimal as it leads to short traffic interruptions when restarting the daemon while filter replace can be done atomically. Rework the init script such that the latter can be used for BPF program management. Reported-by: Jaff Cheng and Arthur Chiao via Slack Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Signed-off-by: Michal Rostecki <mrostecki@opensuse.org> 21 August 2019, 18:39:07 UTC
8eea34a cilium: route mtu not set unless route.Spec set MTU [ upstream commit f184cadb819141fde43d4885a6b66e5c9b17de87 ] Calling route.Upsert() with an MTU parameter will only set the MTU on the route if the MTU of the route spec (first argument to Upsert) also sets the MTU to some non-zero value. However, this condition is not met by any route.Upsert() calls. It seems calling code expects the route to use the MTU param. At least Encryption code expected this and is causing routes to violate MTU. And when using vxlan I expected routes to use a lower route MTU to account for vxlan header but this is not the case. After this patch the MTU param will be used to set the route MTU. If users of the API want to explicitly set the MTU then they need to set the route spec MTU value and nil the MTU param unto route.Upsert(). Fixes: #8883 Signed-off-by: John Fastabend <john.fastabend@gmail.com> Signed-off-by: Michal Rostecki <mrostecki@opensuse.org> 21 August 2019, 18:39:07 UTC
8869262 Revert "[daemon] - Change MTU source for cilium_host (Use the Route one)" This reverts commit ad247126626174dd14876fd9128d2316cbb9cc51. This commit introduces a regression in the datapath which causes traffic coming from the outside world to have its packets fragmented since the MTU of the cilium_host interface is lower than the MTU of the external facing device. This causes 2 major problems: 1) Regression in the network speed of communications made to the outside world, here is a simple `curl` made to `www.google.com` where the first attempt was made with the commit reverted and the second attempt was made with the current master branch: ``` % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 12482 0 12482 0 0 30155 0 --:--:-- --:--:-- --:--:-- 30222 % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 12525 0 12525 0 0 5320 0 --:--:-- 0:00:02 --:--:-- 5322 ``` 2) Since the datapath does not handle IP fragmentation it creates CT entries that are wrong, since the offset of a the 5 tuple assumes the IP packet is not fragmented, here are a couple of entries that should not have been created: (One of the ports in this tuple should have port 80 and not random ports that were based in the offset of the 5 tuple of a fragmented IP packet) ``` TCP IN 172.217.168.4:26740 -> 10.16.238.103:14896 expir... TCP IN 172.217.168.4:28525 -> 10.16.238.103:28769 expir... TCP IN 172.217.168.4:28521 -> 10.16.238.103:28276 expir... TCP IN 172.217.168.4:28263 -> 10.16.238.103:24930 expir... TCP IN 172.217.168.4:26469 -> 10.16.238.103:29780 expir... TCP IN 172.217.168.4:12344 -> 10.16.238.103:11313 expir... ``` Signed-off-by: André Martins <andre@cilium.io> Signed-off-by: Michal Rostecki <mrostecki@opensuse.org> 21 August 2019, 18:39:07 UTC
8f62aa1 cilium: encryption, fix getting started guides create secrects command [ upstream commit 26e32a365a8ae7d9a5ec525f59326b0616b0ef12 ] Some users have reported a "invalid argument" from cilium-agent when it adds encryption rules after following the v1.5 getting started guide. We debugged (at least one potential cause) this to an incorrect secrets command when using the docs from http://docs.cilium.io/en/v1.5/gettingstarted/encryption/# Here the secrets command was rendered as follows, $ kubectl create -n kube-system secret generic cilium-ipsec-keys \ --from-literal=keys="3 rfc4106(gcm(aes)) $(echo dd if=/dev/urandom count=20 bs=1 2> /dev/null| xxd -p -c 64) 128" This is actually wrong! It will give you a secret that is almost correct but has a key of the wrong length and the cilium-agent will show the repoted "invalid argument" error when rules are loaded because the kernel detects the key length is wrong. The correct command is $ kubectl create -n kube-system secret generic cilium-ipsec-keys \ --from-literal=keys="3 rfc4106(gcm(aes)) $(echo $(dd if=/dev/urandom count=20 bs=1 2> /dev/null| xxd -p -c 64)) 128" Which is the command shown in the v1.6 documentation here, http://docs.cilium.io/en/v1.6/gettingstarted/encryption/# Additionally if the user (like I typically do) was following the docs from the source using a different editor they may have got this command ``` $ kubectl create -n kube-system secret generic cilium-ipsec-keys \ --from-literal=keys="3 rfc4106(gcm(aes)) $(echo `dd if=/dev/urandom count=20 bs=1 2> /dev/null| xxd -p -c 64`) 128" ``` depending on how the shell handles the ` escape around the `dd` command this may work. At least it worked for me. But at docs.cilium.io guide the markup dropped the ` and broke the command. Unfortunately when I was testing this I ran the commands from the source version with my editor that did not drop the ' and did not catch the above error. Fix this by pulling in the fix applied to v1.6 version back to the v1.5 documents to get everything consistent. We missed tagging this for backport to v1.5. To test I spun up an eks instance, # uname -a Linux ip-192-168-77-71.us-west-2.compute.internal 4.14.128-112.105.amzn2.x86_64 #1 SMP Wed Jun 19 16:53:40 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux And ran into the above fix through the html rendered version of docs. With this I see encryption rules installed correctly and no errors. This is a fix to a patch below in the Fixes tag (efd7bb47f5bld) where we lost an escape char "\" that fixed this. But Andre also fixed this upstream but we missed backporting so lets pull in Andre's fix back to v1.5. Fixes: #8484 Fixes: efd7bb47f5b1d ("cilium: docs update encryption algo example to use GCM") Signed-off-by: André Martins <andre@cilium.io> Signed-off-by: John Fastabend <john.fastabend@gmail.com> 20 August 2019, 08:09:16 UTC
2ae8ec5 datapath: Limit host->service IP SNAT to local traffic The existing rule applies to all traffic via cilium_host which also applies to traffic that is routed into cilium_host from an external interface. This breaks "externaltrafficpolicy: local". Fixes: 431bd82d39c ("cilium: fix up source address selection for cluster ip# Please enter the commit message for your changes. Lines starting") Reported-by: Ole Markus With Signed-off-by: Thomas Graf <thomas@cilium.io> 19 August 2019, 18:24:53 UTC
1fcf708 cilium: fix transient rules to use allocation cidr [ upstream commit: none, 1.5 only ] ... as otherwise they'll be dropped on cilium_net since lxc+ is not matching there. Hence use allocation cidr as in main ruleset. Fixes: 8899b0b67c1c ("cilium: install transient rules during agent restart") Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> 16 August 2019, 22:07:13 UTC
13c64ae Prepare for v1.5.6 release Signed-off by: Ian Vernon <ian@cilium.io> 16 August 2019, 15:28:16 UTC
6b91141 endpoint: Fix proxy port leak on endpoint delete [ upstream commit 0c7f066529815f4d51f64b709d6a52fba8ee8b25 ] Deep in the proxy.removeRedirect() logic, a finalize function is created to defer proxy port release to allow a port to be reused by the proxy after a grace period. The port is not actually released when the removeRedirect() function is returned, instead the finalize function is passed all the way up to the caller. In the endpoint leave case, this finalize function was being ignored, meaning that the proxy port used by this endpoint is never released. After 10,000 endpoint deletions of endpoints that have L7 policy applied, the endpoint regeneration begins to fail for new endpoints that are subject to L7 policy. As a result, the CNI times out: Failed create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "xxx" network for pod "yyy": NetworkPlugin cni failed to set up pod "yyy" network: Unable to create endpoint: Put http:///var/run/cilium/cilium.sock/v1/endpoint/cilium-local:0: context deadline exceeded Fix this by running the finalize function upon endpoint leave. Signed-off-by: Joe Stringer <joe@cilium.io> Signed-off-by: Ian Vernon <ian@cilium.io> 16 August 2019, 05:19:31 UTC
0c89b51 update cilium-docker-plugin, cilium-operator to golang 1.12.8 Signed-off by: Ian Vernon <ian@cilium.io> 16 August 2019, 05:19:31 UTC
743d1e0 dockerfiles: update golang versions to 1.12.8 [ upstream commit 3439d8895407adf4aec727778941da8ce9c1107f ] Signed-off by: Ian Vernon <ian@cilium.io> Signed-off-by: Ian Vernon <ian@cilium.io> 16 August 2019, 05:19:31 UTC
8899b0b cilium: install transient rules during agent restart [ upstream commit 80507f142c59a5d0ca6b782e6b8092e0c93a7246 ] Jaff and Arthur reported that during agent restart while ping test with low interval is running and trying to reach one of the pods on the node where the agent is restarting a few dropped packets have been seen. This is due to Cilium agent reloading its iptables rules, in particular, the CILIUM_FORWARD chain is affected here since once removed for a short window we'll be hitting the default DROP policy in FORWARD. Add a temporary transient chain that keeps basic ACCEPT rules alive during re-setup until main rules have been set in place. Reported-by: Jaff Cheng and Arthur Chiao via Slack Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Signed-off-by: Ian Vernon <ian@cilium.io> 16 August 2019, 05:19:31 UTC
e0758fe Istio: Update to 1.2.4 Istio release 1.2.4 includes the sidecar proxy with Envoy security fixes released on 8/13/2019. Signed-off-by: Jarno Rajahalme <jarno@covalent.io> 15 August 2019, 18:01:30 UTC
c8eae6d envoy: Use patched image [ upstream commit a309f9886c34a834a03bd7a4079db99119e364f8 ] Signed-off-by: Jarno Rajahalme <jarno@covalent.io> 14 August 2019, 14:52:02 UTC
06c6520 bpf: fix verifier error due to repulling of skb->data/end [ upstream commit fdca23e2b23f91f6672211961ee3b0a29449506c ] After commit 5aebc46cacba ("bpf: Attempt pulling skb->data if it is not pulled") running the agent with nodeport enabled broke with a verifier error as LLVM tried to generate code with pkt_end arithmetic which does not work. [...] level=info msg="Cilium 1.6.90 2c6863714 2019-08-07T09:06:44-07:00 go version go1.11.2 linux/amd64" subsys=daemon level=info msg="Envoy version check disabled" subsys=daemon level=info msg="clang (7.0.1) and kernel (5.3.0) versions: OK!" subsys=daemon [...] level=warning msg=" R0=inv4294967154 R1_rw=invP4294967154 R2_rw=invP4294967154 R3=inv1155182100513554433 R4=inv0 R5=inv(id=0) R6=invP0 R7=inv40 R8_rw=invP0 R9=invP0 R10=fp0 fp-56=????mmmm fp-104=??0m0000 fp-112=mmmmmmmm fp-120=mmmmmmmm fp-128=mmmmmmmm fp-136=mmmmmmmm fp-144=00000000 fp-152=mmmmmmmm fp-160=mmmmmmmm fp-168=????0000 fp-176=00000000 fp-184=00000000 fp-192=00000000 fp-200_r=ctx" subsys=daemon level=warning msg="parent didn't have regs=100 stack=0 marks" subsys=daemon level=warning msg="last_idx 306 first_idx 297" subsys=daemon level=warning msg="regs=100 stack=0 before 306: (18) r2 = 0xffffff72" subsys=daemon level=warning msg="regs=100 stack=0 before 305: (b7) r8 = 0" subsys=daemon level=warning msg="907: (61) r1 = *(u32 *)(r7 +80)" subsys=daemon level=warning msg="908: (61) r9 = *(u32 *)(r7 +76)" subsys=daemon level=warning msg="909: (18) r8 = 0xffffff7a" subsys=daemon level=warning msg="911: (b7) r6 = 0" subsys=daemon level=warning msg="912: (67) r1 <<= 32" subsys=daemon level=warning msg="R1 pointer arithmetic on pkt_end prohibited" subsys=daemon level=warning msg="processed 191 insns (limit 1000000) max_states_per_insn 0 total_states 8 peak_states 8 mark_read 5" subsys=daemon level=warning subsys=daemon level=warning msg="Error filling program arrays!" subsys=daemon level=warning msg="Unable to load program" subsys=daemon [...] Fix to force intermediate writes by using WRITE_ONCE() to avoid such code generation would work for newer kernels, but I'm getting a verifier rejection on 4.19.57: [...] level=warning msg="333: (7b) *(u64 *)(r10 -232) = r2" subsys=daemon level=warning msg="334: (65) if r1 s> 0xa goto pc+373" subsys=daemon level=warning msg=" R0=inv(id=0,umax_value=2147483647,var_off=(0x0; 0x7fffffff)) R1=inv(id=0,umax_value=10,var_off=(0x0; 0xf)) R2=inv1 R6=ctx(id=0,off=0,imm=0) R7=inv0 R8=inv8 R9=inv(id=0) R10=fp0,call_-1 fp-64=? fp-80=pkt fp-88=pkt_end fp-128=? fp-216=map_ptr fp-224=map_value" subsys=daemon level=warning msg="335: (15) if r1 == 0x0 goto pc+436" subsys=daemon level=warning msg=" R0=inv(id=0,umax_value=2147483647,var_off=(0x0; 0x7fffffff)) R1=inv(id=0,umax_value=10,var_off=(0x0; 0xf)) R2=inv1 R6=ctx(id=0,off=0,imm=0) R7=inv0 R8=inv8 R9=inv(id=0) R10=fp0,call_-1 fp-64=? fp-80=pkt fp-88=pkt_end fp-128=? fp-216=map_ptr fp-224=map_value" subsys=daemon level=warning msg="336: (15) if r1 == 0x3 goto pc+373" subsys=daemon level=warning msg=" R0=inv(id=0,umax_value=2147483647,var_off=(0x0; 0x7fffffff)) R1=inv(id=0,umax_value=10,var_off=(0x0; 0xf)) R2=inv1 R6=ctx(id=0,off=0,imm=0) R7=inv0 R8=inv8 R9=inv(id=0) R10=fp0,call_-1 fp-64=? fp-80=pkt fp-88=pkt_end fp-128=? fp-216=map_ptr fp-224=map_value" subsys=daemon level=warning msg="337: (15) if r1 == 0x8 goto pc+1" subsys=daemon level=warning msg=" R0=inv(id=0,umax_value=2147483647,var_off=(0x0; 0x7fffffff)) R1=inv(id=0,umax_value=10,var_off=(0x0; 0xf)) R2=inv1 R6=ctx(id=0,off=0,imm=0) R7=inv0 R8=inv8 R9=inv(id=0) R10=fp0,call_-1 fp-64=? fp-80=pkt fp-88=pkt_end fp-128=? fp-216=map_ptr fp-224=map_value" subsys=daemon level=warning msg="338: (05) goto pc+437" subsys=daemon level=warning msg="776: (71) r1 = *(u8 *)(r10 -127)" subsys=daemon level=warning msg="invalid size of register spill" subsys=daemon [...] After playing around half a day, the only thing I got to accept is to perform the skb_pull_data() unconditionally before the data/data_end fetch, therefore the workaround follows this approach until we find something better. Fixes: 5aebc46cacba ("bpf: Attempt pulling skb->data if it is not pulled") Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Signed-off-by: Joe Stringer <joe@cilium.io> 14 August 2019, 08:20:43 UTC
9409a2a bpf: Attempt pulling skb->data if it is not pulled [ upstream commit 5aebc46cacba2c164a6bb789f3bc185e5f0256e2 ] If the skb->data_end is too short for Cilium to perform its logic, attempt to pull it in. If it's still too short, we can legitimately drop the traffic as "invalid". Signed-off-by: Joe Stringer <joe@cilium.io> 14 August 2019, 08:20:43 UTC
ef64583 bpf: Introduce revalidate_data_first() [ upstream commit 9093a619f908647fe7f9189a5b8a84d17ed888f1 ] In some environments depending on the underlying NIC driver in use, the skb may not arrive pulled with all data contiguously. In that case, Cilium may end up dropping legitimate traffic because it is unable to access enough of the packet to perform its functions. Such cases exist today, refactor the function into a separate helper for better code reuse. Signed-off-by: Joe Stringer <joe@cilium.io> 14 August 2019, 08:20:43 UTC
4d66d3d cilium: add skb_pull_data to bpf_network to avoid revalidate error [ upstream commit a03c33a592e3c4a4933f6a951b3b0de3161af8fb ] Depending on the driver when we have a full stack of headers with tunneled IPsec its possible the headers are not part of the linear data. In this case we should pull in the data instead of immediately aborting. If the pull fails we can then abort. For this patch we only fix the immediate problem call site. We need to audit the remaining ones. However, veth devices should not have this issue in general so bpf_lxc.c should be OK. skbs on bpf_netdev either passed through bpf_network.c, bpf_lxc.c or were routed from the host networking. Fixes: 47bd875512770 ("cilium: encrypt, use ipcache to lookup IPsec destination IP") Signed-off-by: John Fastabend <john.fastabend@gmail.com> Signed-off-by: Joe Stringer <joe@cilium.io> 14 August 2019, 08:20:43 UTC
d0cd5c3 datapath/iptables: wait until acquisition xtables lock is done [ upstream commit 5f46e1abd9e92b6ff6ec4050d1e5edf3d60d117a ] This guarantees iptables interactions are not run concurretly with other programs, e.g. kube-proxy. More info: https://git.netfilter.org/iptables/commit/?id=93587a04d0f2511e108bbc4d87a8b9d28a5c5dd8 Signed-off-by: André Martins <andre@cilium.io> Signed-off-by: Ian Vernon <ian@cilium.io> 14 August 2019, 08:20:43 UTC
0f8a112 use iptables-manager to manage iptables executions [ upstream commit 7240e834b9ba492ea3554bf89c0789d5d4f0ff5c ] This will make it easier to run iptables commands based on iptables version installed in the system. Signed-off-by: André Martins <andre@cilium.io> Signed-off by: Ian Vernon <ian@cilium.io> 14 August 2019, 08:20:43 UTC
b9a8cf4 examples/kubernetes: mount xtables.lock [ upstream commit 5f46e1abd9e92b6ff6ec4050d1e5edf3d60d117a ] As Cilium access iptables it should also mount the xtables.lock file into iptables. Signed-off-by: André Martins <andre@cilium.io> Signed-off by: Ian Vernon <ian@cilium.io> 14 August 2019, 08:20:43 UTC
4fff691 eventqueue: return error if Enqueue fails [ upstream commit bb115ad8face1af7d90eb044612411e69b68c35c ] Given that an Enqueue may not occur due to the queue not being initialized correctly / the Event being nil, or the Event has already been Enqueued before, return an error which follows Golang conventions instead of encoding error state in the returned value. Signed-off by: Ian Vernon <ian@cilium.io> Signed-off-by: Ian Vernon <ian@cilium.io> 14 August 2019, 08:20:43 UTC
922dd4b eventqueue: protect against enqueueing same Event twice [ upstream commit 9e3e123872750bc40d758b5eb7860772c6d3f268 ] Previously, if an Event is enqueued twice, the following panic would occur: ``` PANIC: eventqueue_test.go:187: EventQueueSuite.TestPanic ... Panic: close of closed channel (PC=0x42EFD4) /usr/local/go/src/runtime/panic.go:522 in gopanic /usr/local/go/src/runtime/chan.go:342 in closechan eventqueue.go:188 in EventQueue.Enqueue eventqueue_test.go:197 in EventQueueSuite.TestPanic /usr/local/go/src/reflect/value.go:308 in Value.Call /usr/local/go/src/runtime/asm_amd64.s:1337 in goexit OOPS: 9 passed, 1 PANICKED Signed-off-by: Ian Vernon <ian@cilium.io> 14 August 2019, 08:20:43 UTC
19450ab eventqueue: use mutex to synchronize access to events channel [ upstream commit 2c68637145e11b89254b3c6aa431b9de5c77b6c4 ] Previously, a `sync.WaitGroup` was used to ensure the events channel for a given `EventQueue` was not closed while a call to `Enqueue` was in in-flight. However, a panic could result when stopping a queue, which called `Wait` on said `WaitGroup`, and multiple calls to `Enqueue` were called while said `Wait` was happening: ``` panic: sync: WaitGroup is reused before previous Wait has returned goroutine 55 [running]: sync.(*WaitGroup).Wait(0xc000360120) /usr/local/go/src/sync/waitgroup.go:132 +0xae github.com/cilium/cilium/pkg/eventqueue.(*EventQueue).Stop.func1() /go/src/github.com/cilium/cilium/pkg/eventqueue/eventqueue.go:279 +0x18a sync.(*Once).Do(0xc000360114, 0xc0000e57b8) /usr/local/go/src/sync/once.go:44 +0xb3 github.com/cilium/cilium/pkg/eventqueue.(*EventQueue).Stop(0xc0003600f0) /go/src/github.com/cilium/cilium/pkg/eventqueue/eventqueue.go:269 +0x81 ``` This occured because the `WaitGroup` is waiting while at the same time that `WaitGroup` has more deltas being added to its internal counter. Replace this `WaitGroup` with a mutex that is Locked when closing the events channel, blocking out in-flight calls to `Enqueue`. Calls to `Enqueue` take an `RLock` so multiple Enqueues can occur at the same time without a problem. Signed-off by: Ian Vernon <ian@cilium.io> Signed-off-by: Ian Vernon <ian@cilium.io> 14 August 2019, 08:20:43 UTC
d20ebb5 daemon: get list of frontends from ServiceCache before acquiring BPFMapMu [ upstream commit b6a1790f05c5c4dfcbe25b15db6d5ed8bd115dc6 ] The `ServiceCache` mutex should always be acquired before acquiring the `BPFMapMu` of the `LoadBalancer`. The following deadlock was observed in a cilium instance: We are trying to acquire the `BPFMapMu` in the daemon's `LoadBalancer` to add a Kubernetes service as observed by the Kubernetes watcher: ``` goroutine 150 [semacquire, 225 minutes]: sync.runtime_SemacquireMutex(0xc000401d44, 0x0) /usr/local/go/src/runtime/sema.go:71 +0x3d sync.(*Mutex).Lock(0xc000401d40) /usr/local/go/src/sync/mutex.go:134 +0x109 sync.(*RWMutex).Lock(0xc000401d40) /usr/local/go/src/sync/rwmutex.go:93 +0x2d main.(*Daemon).svcAdd(0xc0001b83c0, 0xc002d96eb0, 0x10, 0x10, 0xc001afa069, 0x3, 0x2474, 0x38f, 0x4d8bf28, 0x0, ...) /go/src/github.com/cilium/cilium/daemon/loadbalancer.go:132 +0x630 main.(*Daemon).addK8sSVCs(0xc0001b83c0, 0xc001badec0, 0x2d, 0xc001bbb000, 0xb, 0x0, 0xc000e73130, 0xc003e5e8a0, 0x0, 0x0) /go/src/github.com/cilium/cilium/daemon/k8s_watcher.go:1288 +0xf37 main.(*Daemon).k8sServiceHandler(0xc0001b83c0) /go/src/github.com/cilium/cilium/daemon/k8s_watcher.go:1064 +0xa29 created by main.(*Daemon).runK8sServiceHandler /go/src/github.com/cilium/cilium/daemon/k8s_watcher.go:1118 +0x3f ``` The controller which makes sure that the daemon's LoadbLanacer maps are in sync with Kubernetes has possession of the aforementioned `BPFMapMu` mutex, but is trying to acquire the mutex for the `ServiceCache` after having acquired the `BPFMapMu`: ``` goroutine 401 [semacquire, 225 minutes]: sync.runtime_SemacquireMutex(0xc0001b8514, 0x0) /usr/local/go/src/runtime/sema.go:71 +0x3d sync.(*RWMutex).RLock(...) /usr/local/go/src/sync/rwmutex.go:50 github.com/cilium/cilium/pkg/k8s.(*ServiceCache).UniqueServiceFrontends(0xc0001b8500, 0x0) /go/src/github.com/cilium/cilium/pkg/k8s/service_cache.go:427 +0x37d main.(*Daemon).syncLBMapsWithK8s(0xc0001b83c0, 0x0, 0x0) /go/src/github.com/cilium/cilium/daemon/loadbalancer.go:643 +0x1a4 main.(*Daemon).initRestore.func2.1(0x35013c0, 0xc00590cb80, 0x4d6d3e0, 0x0) /go/src/github.com/cilium/cilium/daemon/state.go:483 +0x2a github.com/cilium/cilium/pkg/controller.(*Controller).runController(0xc005abe000) /go/src/github.com/cilium/cilium/pkg/controller/controller.go:203 +0x9e1 created by github.com/cilium/cilium/pkg/controller.(*Manager).UpdateController /go/src/github.com/cilium/cilium/pkg/controller/manager.go:112 +0xae0 ``` The deadlock is due to the following channel send being stuck: ``` goroutine 356 [chan send, 225 minutes]: github.com/cilium/cilium/pkg/k8s.(*ServiceCache).UpdateEndpoints(0xc0001b8500, 0xc0042a87c0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) /go/src/github.com/cilium/cilium/pkg/k8s/service_cache.go:297 +0x454 main.(*Daemon).addK8sEndpointV1(...) /go/src/github.com/cilium/cilium/daemon/k8s_watcher.go:1136 main.(*Daemon).EnableK8sWatcher.func7.2(0x0, 0xc000e3ca20) /go/src/github.com/cilium/cilium/daemon/k8s_watcher.go:530 +0x46 github.com/cilium/cilium/pkg/serializer.(*FunctionQueue).run(0xc000e36930) /go/src/github.com/cilium/cilium/pkg/serializer/func_queue.go:70 +0x6e created by github.com/cilium/cilium/pkg/serializer.NewFunctionQueue /go/src/github.com/cilium/cilium/pkg/serializer/func_queue.go:50 +0xb9 ``` The function `UpdateEndpoints` is holding the mutex for the `ServiceCache`. The reason that the channel send above is stuck, is because the goroutine which receives off of the channel (the first one shown above) is trying to acquire the `BPFMapMu`. But, since the goroutine holding the `ServiceCache` mutex is blocking on sending to the channel, it will never release the mutex, which the goroutine holding the `BPFMapMu` is trying to acquire before it can release the `BPFMapMu`, which in turn is blocking the channel receive goroutine from progressing further! The simplest fix for now is to call `UniqueServiceFrontends` before acquiring the `BPFMapMu` in the controller which synchronizes state in the datapath. This ensures that the controller will eventually release the `BPFMapMu`, which will allow the goroutine receiving events in the Kubernetes watcher to acquire it, and read from the channel of service-related events. Signed-off by: Ian Vernon <ian@cilium.io> Signed-off-by: Ian Vernon <ian@cilium.io> 14 August 2019, 08:20:43 UTC
982a613 cilium: remove old probe content before restoring assets [ upstream commit a2a2c6a277378d7311dbb19d28e836d7af6c140b ] Example of compilation failure from agent when downgrading from master back to 1.5: + clang -O2 -I/tmp/tmp.WSx7wj9jyh -I/var/lib/cilium/bpf/probes -I/var/lib/cilium/bpf/include -Wall /var/lib/cilium/bpf/probes/raw_main.c -o /tmp/tmp.WSx7wj9jyh/raw_sock_cookie.t In file included from /var/lib/cilium/bpf/probes/raw_main.c:52: /tmp/tmp.WSx7wj9jyh/raw_probe.t:8:4: error: field designator 'attach_type' does not refer to any field in type 'struct bpf_test' .attach_type = BPF_CGROUP_INET4_CONNECT, ^ 1 error generated. + cleanup + '[' '!' -z /tmp/tmp.WSx7wj9jyh ']' + rm -rf /tmp/tmp.WSx7wj9jyh Reason is that run_probes.sh will just pull in every *.t test from /var/lib/cilium/bpf/probes/ and upon downgrade newer ones are still present in this directory. We probably could remove the entire option.Config.RunDir directory if option.Config.KeepTemplates is false, but given we earlier on created the os.MkdirAll(option.Config.RunDir, ...) unconditionally, it's a bit cleaner to just remove the probes dir right before the restoring of bindata assets since this is the only thing we actually need. Fixes: #7971 Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Signed-off-by: Ian Vernon <ian@cilium.io> 14 August 2019, 08:20:43 UTC
6b7d50f cilium: encryption, ensure 0x*d00 and 0x*e00 marks dont cause conflicts [ upstream commit 5aae4ab99703690c6e69a4d4dd14473d4681862a ] We mark encryption packets with 0xd00 and 0xe00 values to indicate if the packet should be encrypted (0xe00) or decrypted (0xd00). In the encryption case we also set the next byte (0x*e00) with the keyID to encrypt the packet with. This allows multiple keys to be in use at any specific time. However, it was observed that the upper bits, where the keyID is placed, may also be used by kube-proxy. To avoid collision add rules at the top of iptables to accept any packets with the encrypt/decrypt mark values. These values will be cleared again before being pushed to the stack so normal rules will still be hit. Notice, we already had these rules to exclude encrypted traffic in the masquerading case. This moves those rules installation to be done any time encryption is enabled. This issue has existed from the initial implementation, but our CI and my testing never used those key id with colliding features. Fixes: b2b901fb19163 ("cilium: ipsec, add go API to configure xfrm (IPSec)") Signed-off-by: John Fastabend <john.fastabend@gmail.com> 05 August 2019, 23:53:08 UTC
ece8603 Dockerfile: Use proxy with legacy fix [ upstream commit 57420053d387350268d49498e890523f46869b06 ] Use cilium-envoy with a fix for regression that broke support for Cilium versions without TPROXY support (< 1.6). Signed-off-by: Jarno Rajahalme <jarno@covalent.io> 31 July 2019, 21:47:35 UTC
2f12d79 envoy: Add SO_MARK option to listener config [ upstream commit a56938bc27d88ee723f70f9a1d331d5098b3352b ] Envoy is removing socket option support from Listener callbacks, so we need to set the SO_MARK option in the listener config instead. Use a cilium-envoy image that does not set the same option from the cilium filters. Signed-off-by: Jarno Rajahalme <jarno@covalent.io> 31 July 2019, 21:47:35 UTC
167c74e test: provide capability for tests to run in their own namespace [ upstream commit 2c670be9e69b98edce045a9ce5800164f5926dc0 ] Running each set of tests in a `Describe` in its own namespace instead of the `default` namespace ensures that resource leakage between tests w.r.t. policies, services, and other associated resources stored in K8s is kept to a minimum. It also allows for deletion of all resources in a namespace with one command at the end of a run of a given set of tests. To allow for this functionality, a variety of changes had to be made: * add variadic arguments for providing namespace to existing commands for backwards compatability. * change namespace parameter of `CiliumPolicyAction` to apply the policy in the namespace parameter, and hardcode the namespace used to get Cilium pods to be `kube-system`. Change callers accordingly. One given set of tests within the `K8sPolicyTest` have been changed to use a specific namespace. Misc. fixes / changes needed for said tests to pass were added as well. Other tests can be migrated over time to use their own namespaces. Signed-off by: Ian Vernon <ian@cilium.io> 31 July 2019, 21:47:35 UTC
82703c6 docs: Fix warnings [ upstream commit b2159fb0e6b69bb23f070fe1651a2379205e6bc0 ] Sphinx reported some warnings with these docs, fix them. Signed-off-by: Joe Stringer <joe@cilium.io> 31 July 2019, 21:47:35 UTC
a0bc714 test: Specify protocol during policy trace [ upstream commit aa8b483f87ae0a6a5f801878d9af21c67f2441d4 ] This avoids failures when upcoming commits make the L4 policy trace more strict, to determine exactly whether traffic is allowed depending on the protocol as well as the port. Signed-off-by: Joe Stringer <joe@cilium.io> 31 July 2019, 21:47:35 UTC
0faef63 istio: Update to 1.2.2 [ upstream commit 9e8c5dc2fda12c372da56710d0ccef48051a9a5e ] Update cilium-envoy to a version with Istio 1.2.2 filters. Update Istio GSG and CI test to use the new proxy and pilot images. Signed-off-by: Jarno Rajahalme <jarno@covalent.io> 31 July 2019, 21:47:35 UTC
ec8742d envoy: Istio 1.2.0 update [ upstream commit cf916b9f31c54d041c3da74544730a86e0300b73 ] Update to the proxy image with Istio 1.2.0 filters. Update Istio GSG and tests to 1.2.0. Signed-off-by: Jarno Rajahalme <jarno@covalent.io> 31 July 2019, 21:47:35 UTC
2863af8 istio: Update to 1.1.7 [ upstream commit 6de86b6d8f61b1d3838c79f69ad510e12a5ed2bc ] Update Istio to 1.1.7. Signed-off-by: Jarno Rajahalme <jarno@covalent.io> 31 July 2019, 21:47:35 UTC
4cb7243 test: be sure to close SSH client after a given Describe completes [ upstream commit 6181ca25f573051a2b62fd25aee84f78a2d91b0b ] We were never closing the clients we created, and the code should clean up after itself once it is finished. Do so. This adds some boilerplate code, unfortunately, but it's better than leaving open clients around. Signed-off by: Ian Vernon <ian@cilium.io>" 31 July 2019, 21:47:35 UTC
213ef4a Dockerfile: Use cilium-envoy with reduced logging. [ upstream commit 990339d2d97b343175ce12b19facc82eb9ba6528 ] Use a new envoy image with reduced logging. This should remove these from the CI output: Top 5 errors/warnings: [[cilium/network_policy.cc:129] Bad Network Policy Configuration [[cilium/host_map.cc:187] Bad NetworkPolicyHosts Configuration Signed-off-by: Jarno Rajahalme <jarno@covalent.io> 31 July 2019, 21:47:35 UTC
8c196ce Envoy: Update to the latest proxy build, use latest API [ upstream commit 2777f9b603866a413cbd5a78b7593595c46cd5fa ] Update cilium-envoy image and API reference to the latest. Signed-off-by: Jarno Rajahalme <jarno@covalent.io> 31 July 2019, 21:47:35 UTC
5aecd0d Gopkg: update cilium/proxy [ upstream commit f27130849a88ec74f1397705c659481de63c0509 ] The dependencies are now vendored in the cilium/proxy repository. Cilium will import the right dependencies for the cilium/proxy based in the Gopkg.toml of that directory. Signed-off-by: André Martins <andre@cilium.io> 31 July 2019, 21:47:35 UTC
a8fe173 envoy: Use LPM ipcache instead of xDS when available. [ upstream commit 9102ea8597cf8c3631f64820f6dc7f70497c24d4 ] Use Envoy version that can use bpf ipcache instead of NetworkPolicyHosts xDS, when available. If the bpf ipcache LPM map open fails (such as on Linux kernels before 4.11), Envoy falls back to using xDS. Signed-off-by: Jarno Rajahalme <jarno@covalent.io> 31 July 2019, 21:47:35 UTC
6d0f6d4 Envoy: Use an image with proxylib injection fix. [ upstream commit 8406aa652023a089903c0592f4e88268cd53956a ] Cilium network filter failed to call back to proxylib when the OnData call returned with a full inject buffer. Use a newer Envoy image that fixes this. Proxylib testing tooling is updated to support large injections by simulating Envoy datapath processing of the injected data by clearing the inject buffer on CheckOnData() and CheckOnDataOK() calls. blockparser (one of the test parsers) can now support larger injections in the original direction. This is used in cilium-envoy integration testing. Signed-off-by: Jarno Rajahalme <jarno@covalent.io> 31 July 2019, 21:47:35 UTC
9ac7c80 Dockerfile: Update proxy dependency [ upstream commit 813538189b888cd124950be4ddd953b6dd8d3c5d ] Use newer proxy image that has support for deriving policy name from the endpoint IP, and for deriving the proxylib parser name from the network policy. These changes allow listener configuration to be independent of an endpoint or an endpoint's network policy. Signed-off-by: Jarno Rajahalme <jarno@covalent.io> 31 July 2019, 21:47:35 UTC
face465 CI: Change Kafka runtime tests to use local conntrack maps. [ upstream commit 4c1f82042861e656d2c7457f7119c124a3265363 ] It's good to have test coverage with the local conntrack maps. Optimally would keep also the global map test cases. Signed-off-by: Jarno Rajahalme <jarno@covalent.io> 31 July 2019, 21:47:35 UTC
ad24712 [daemon] - Change MTU source for cilium_host (Use the Route one) [ upstream commit 86f926e816dc871bdd985c82f556644aae58c6d9 ] * for pods in hostNetwork the mtu was the same a the default interface. Packets were thus dropped once the stream was encrypted. Fixes: 1ce278f ("daemon: move IPSec bootstrap into separate function") Signed-off-by: Ray Bejjani <ray@isovalent.com> 31 July 2019, 21:46:04 UTC
6a35073 endpoint: fix deadlock when endpoint EventQueue is full [ upstream commit 5a18ed9d7825413fb67c1957f8350de8f6784744 ] This commit fixes a problem that affects a deadlock during Endpoint deletion: An event is being processed by the endpoint's EventQueue, but has not yet acquired the Endpoint's mutex. The endpoint's EventQueue is full (e.g., it has reached its full capacity), and there are calls to `Enqueue` for the Endpoint which are blocking on returning due to the queue being full. The endpoint is being deleted in `deleteEndpointQuiet`, and has acquired the Endpoint's mutex. The event being processed by the endpoint's EventQueue tries to acquire the Endpoint's mutex. It cannot, because `deleteEndpointQuiet` just acquired the Endpoint's mutex. The EventQueue's buffer is still full; the event currently being processed cannot be consumed due to the deadlock, so as a result, calls to Enqueue are still blocking forever. `deleteEndpointQuiet` calls `Stop` on the EventQueue. This never returns, because calls to Enqueue are still in flight, and EventQueues have a safeguard built in via a `sync.WaitGroup` which guards against writing to closed channels during `Enqueue`; that is, the EventQueue cannot be closed while Enqueues are still in progress. Enqueues in this case, are still in progress, because the channel is full. But, the channel will never drain due to `deleteEndpointQuiet` holding the lock. `deleteEndpointQuiet` will never release the lock, because it is waiting for the queue to be drained. The fix for the above is to stop the Endpoint's EventQueue before we acquire the Endpoint's mutex, and ensure that the queue is drained (e.g., no events are queued up for it) after we stop the queue as well. This ensures that if multiple calls to `deleteEndpointQuiet` occur for the same Endpoint, they all wait before acquiring the Endpoint's mutex. Add a unit test in the daemon to test this fix as well. Signed-off by: Ian Vernon <ian@cilium.io> Signed-off-by: Ray Bejjani <ray@isovalent.com> 31 July 2019, 21:46:04 UTC
751b147 daemon: register warning_error metric after parsing CLI options [ upstream commit e9e59511071f1020411f63eecf394cc1a4ee99b4 ] As the logger was being setup in an init function, and since the metrics are only enabled after parsing the configuration flag, the default metrics function being used was a NoOp which was causing the metric from not being reported. Fixes: 0fec218c33ff ("pkg/metrics: set all metrics as a no-op unless they are enabled") Signed-off-by: André Martins <andre@cilium.io> Signed-off-by: Ian Vernon <ian@cilium.io> 26 July 2019, 08:44:14 UTC
2e1b03e Fix seds in microk8s docs [ upstream commit 797c3d55c42b32e4ea162ec426c6a42881ee7db3 ] Sed commands rendered without backslashes which caused them to not be copy-able. Use semicolon as the delimiter for sed substitutions with paths. Signed-off-by: Maciej Kwiek <maciej@isovalent.com> Signed-off-by: Ian Vernon <ian@cilium.io> 26 July 2019, 08:44:14 UTC
f100934 daemon: Fix removal of non-existing SVCs in syncLBMapsWithK8s [ upstream commit a0694039b921ac5414ebeded0fd4a81958eb7e91 ] 59315dcdcf missed the fact that population of "daemon.loadBalancer" happens only via k8s_watcher's "UpdateService". So, in the case of a non-existing SVC, "daemon.loadBalancer" doesn't have an entry for the svc. Revert the change, but keep the locking at the top of the function. Fixes: 59315dcdcf ("daemon: Remove svc from cache in syncLBMapsWithK8s") Signed-off-by: Martynas Pumputis <m@lambda.lt> Signed-off-by: Ian Vernon <ian@cilium.io> 24 July 2019, 17:51:17 UTC
a8a61c3 daemon: Remove svc from cache in syncLBMapsWithK8s [ upstream commit a9d01db066d0799403b880a65a9d50d5949bcf49 ] Previously, the syncLBMapsWithK8s method did not remove a service from the d.loadBalancer, only from the BPF svc maps. This allowed for the following race condition to happen: d.delK8sSVCs(svcA) | d.syncLBMapsWithK8s() | | | k8sSvcCache.UniqueFrontends()=(no svcA) | | | d.loadBalancer.BPFMapMU.Lock() | | | lbmap.dumpServiceMapsToUserspaceV2()=(svcA) | | | delete svcA from BPF maps | | | d.loadBalancer.BPFMapMU.Unlock() | | d.loadBalancer.BPFMapMU.Lock() | svcA exists in d.loadBalancer? | delete svcA from BPF maps This race could have led to removing the same backend twice which could explain the "backend $ID not found" error observed in the wild. Also, move the locking in syncLBMapsWithK8s above the retrieval of k8s services from the cache to avoid any other not yet discovered race condition. Fixes: cc4be8e3 ("daemon: sync BPF maps with in-memory K8s service maps") Signed-off-by: Martynas Pumputis <m@lambda.lt> Signed-off-by: Ian Vernon <ian@cilium.io> 24 July 2019, 17:51:17 UTC
7f602b0 examples/kubernetes: update k8s dev VM to v1.15.1 [ upstream commit 8e4c6c42b68196915fe3c236fb6074ecf1b18015 ] Signed-off-by: André Martins <andre@cilium.io> Signed-off-by: Ian Vernon <ian@cilium.io> 24 July 2019, 17:51:17 UTC
0d2ddd8 test: update k8s test version to v1.15.1 [ upstream commit 931f688ce6169a8e19f67e21e504738680b5abaf ] Signed-off-by: André Martins <andre@cilium.io> Signed-off-by: Ian Vernon <ian@cilium.io> 24 July 2019, 17:51:17 UTC
f8fed2e Gopkg: update k8s dependencies to v1.15.1 [ upstream commit a6e0eb8ebc8b360a21b0011d516a24c13843d1d8 ] Signed-off-by: André Martins <andre@cilium.io> Signed-off-by: Ian Vernon <ian@cilium.io> 24 July 2019, 17:51:17 UTC
3b86737 Add timeout to ginkgo calls [ upstream commit 531a554b8ea139e0e23a853b46a70d671f70104d ] Add timeout to make ginkgo exit instead of being interrupted by Jenkins. Should help with builds being marked as aborted instead of failed. Signed-off-by: Maciej Kwiek <maciej@isovalent.com> Signed-off-by: Ian Vernon <ian@cilium.io> 24 July 2019, 17:51:17 UTC
c68b5f3 proxy: Do not error out if reading of open ports fails. [ upstream commit 15f5e2687d58f47f1f92847e104f2530e521560a ] Checking the random port number against known open ports is a precaution to avoid trying to re-use an open port. We should not error out if the reading of the open ports fails, however, as there is a high likelihood that the port selection will result in an usable port anyway. Signed-off-by: Jarno Rajahalme <jarno@covalent.io> Signed-off-by: Ian Vernon <ian@cilium.io> 24 July 2019, 17:51:17 UTC
276795a pkg/kvstore: wait for node delete delay in unit tests [ upstream commit 30ad6cb9fde07de42dade87c9637cd2820652f86 ] Since Close() deletes all keys from the kvstore the global kvstore client will still receive delete events from the kvstore across unit tests, causing keys from being deleted across unit tests. Fixes: 07312c6d155d ("pkg/{kvstore,node}: delay node delete event in kvstore") Signed-off-by: André Martins <andre@cilium.io> Signed-off-by: Ian Vernon <ian@cilium.io> 23 July 2019, 22:40:14 UTC
d362a6a endpoint: Create redirects before bpf map updates. [ upstream commit 7fa113fe7de6a4979396ed6f4ba40c6d12c3e60f ] Create new redirects and generate the new proxy policy before updating bpf policy maps in order to give the proxies a chance to be ready when new traffic is redirected at them. This could help reduce transient drops in some cases. Remove old redirects only if the endpoint has a security identity. To begin with, endpoints can have redirects only if they have a security identity, and the only instance where the endpoint's security identity is nilled is when it is being destroyed. Fixes: ea62a6b2eb2 ("endpoint: Fix sidecar proxy deadlock during BPF generation") Signed-off-by: Jarno Rajahalme <jarno@covalent.io> Signed-off-by: Ian Vernon <ian@cilium.io> 23 July 2019, 22:40:14 UTC
8d74c20 proxy: Perform dnsproxy Close() in the returned finalizeFunc [ upstream commit fcc7d3b4ea341f22d0c79a85c7758d74f98d0a10 ] Removing the dns redirect implementation state should be only performed when we know all updates are successful, i.e., when the returned finalizeFunc is run. Alternatively, we could revert the changes in the revertFunc, but we'd rather remove the revertFunc return value in a later commit. Signed-off-by: Jarno Rajahalme <jarno@covalent.io> Signed-off-by: Ian Vernon <ian@cilium.io> 23 July 2019, 22:40:14 UTC
ba2fecb endpoint: change transition from restore state [ upstream commit 837d920fed30eba08173d6eee7146f16825c9b37 ] Restoring state for an endpoint is quite a special case for the endpoint; it only occurs once per-endpoint per-run of cilium-agent. Whenever we restore an endpoint, we have specific logic within the restoration process which queues it for a regeneration. Allowing the transition from restoring state to waiting to regenerate means that if the endpoint is exposed to the rest of the cilium-agent via the endpointmanager, that while an endpoint is in restoring state, another thread could change its state, and the endpoint would regenerate out-of-band of the restoring goroutine for the endpoint. This has produced noise in our CI because the regeneration triggered via the restore goroutine triggered for endpoints has failed when the endpoint is restored, exposed to the endpointmanager, and another goroutine swoops in and begins to restore it, putting it into a state which restoring state cannot transition into. When the restore goroutine then tries to regenerate the endpoint, it is no longer in a state which is valid for regeneration, and the regeneration fails, putting the endpoint into a not-ready state. So, remove the transition from restoring to waiting-to-regenerate state. All other code in Cilium tries to set the endpoint into waiting-to-regenerate state, which will fail if it is in restoring state. However, the restoring goroutine will be able to transition into regenerating state directly. This semantically makes sense since restoring is a special case, and we want to guarantee exclusivity of regenerating the endpoint for the first time to just the restoring goroutine. Signed-off by: Ian Vernon <ian@cilium.io> Signed-off-by: Ian Vernon <ian@cilium.io> 23 July 2019, 22:40:14 UTC
bd2eef1 test: misc. runtime policy test fixes [ upstream commit 98f5eb2be8ebaf27922ebdc0f3f438b7d8c4b57c ] * Ensure that Cilium is started with the set of configuration options set in `SetUpCilium`, specifically of excluding of local addresses that are used to simulate external IPs for CIDR policy testing. * Fix a log error message to contain a string that was expected. Signed-off by: Ian Vernon <ian@cilium.io> Signed-off-by: Ian Vernon <ian@cilium.io> 23 July 2019, 22:40:14 UTC
fd40c3f docs: Fix up unparsed SCM_WEB literals [ upstream commit 43696add2a42ceed92d2c710036fb14db456d971 ] These weren't being parsed, and so were rendered as SCM_WEB rather than having the github link. Fix them up. Signed-off-by: Joe Stringer <joe@cilium.io> Signed-off-by: Ian Vernon <ian@cilium.io> 23 July 2019, 22:40:14 UTC
8b78780 pkg/{kvstore,node}: delay node delete event in kvstore [ upstream commit 07312c6d155d11b8a8dd5f7f621c1f13b2266066 ] Instead of checking if the node is still present in the observer, we should check if the node is still present in the store instead. Checking if the node is still available in the observer is irrelevant as the node will never be deleted since the check for its existence in the store will always be true. Instead, delay the propagation of the deletion event to the observer in the store itself when we receive shared keys notifications. Since the propagaton delay is done at the node observer the unit tests will take longer to complete. Fixes: 3137ad5eee6c ("node: Delay handling of node delete events received via kvstore") Signed-off-by: André Martins <andre@cilium.io> Signed-off-by: Ian Vernon <ian@cilium.io> 23 July 2019, 22:40:14 UTC
9525c77 operator: restart non-managed kube-dns pods before connecting to etcd [ upstream commit fd2ff2ac93e4d8527e833159b81746065556e4d6 ] In a new cluster, if [kube|core]-dns is not being managed by Cilium it can make etcd, from cilium-etcd-operator, on not being able to reach [kube|core]-dns. Restarting [kube|core]-dns to force it to be managed by Cilium will make etcd cluster to be able to be setup properly. Since cilium-operator blocks until it gets connectivity with etcd it will never restart [kube|core]-dns as the restart of such pod happens after cilium-operator gets connectivity with etcd. Signed-off-by: André Martins <andre@cilium.io> Signed-off-by: Ian Vernon <ian@cilium.io> 23 July 2019, 22:40:14 UTC
4cfe301 test: move creation of Istio resources into `It` [ upstream commit 425e6b455f8f0c663c36cba3acc3bcbdfdbb8f52 ] The creation of Istio resources in the Istio CI test contains assertions. Previously, these were ran in the `BeforeAll` block. If the assertions failed, `AfterFailed` would not be invoked (which is most likely a bug). To ensure that we are able to debug the Istio test, for now, move the creation of Istio resources / the assertions on their readiness and corectness into the `It` for the Istio test. No other `It`s are used in this test, so no other test functionality is broken here. In the future, this commit can be reverted when `AfterFailed` is invoked when assertions fail in `BeforeAll`. Signed-off by: Ian Vernon <ian@cilium.io> Signed-off-by: Ian Vernon <ian@cilium.io> 23 July 2019, 22:40:14 UTC
072ea66 test: add `ExecMiddle` function [ upstream commit 2cab00aa9abbb9d0fac307c60206765d1a1f1abc ] In certain cases, commands have failed when given a timeout of only 10 seconds, the value used for `ExecShort`. Add another small utility function which has a timeout of 30 seconds to use for said commands. Cases where this has been observed are for `Apply` for a resource in Kubernetes. Plumb accordingly. Signed-off by: Ian Vernon <ian@cilium.io> Signed-off-by: John Fastabend <john.fastabend@gmail.com> 17 July 2019, 22:58:11 UTC
35ae17e datapath: Do not fail if route contains gw equal to dst [ upstream commit 8ce9b22cd358b1a5be6db31fe09266e30fb7cf0e ] Some recent change in the kernel started to include RTA_GATEWAY field in a response to RTM_GETROUTE even if a dst was local. On Linux 5.1.6: recvmsg(3, {msg_name={sa_family=AF_NETLINK, nl_pid=0, nl_groups=00000000}, msg_namelen=12, msg_iov=[{iov_base={{len=104, type=RTM_NEWROUTE, flags=0, seq=1562755119, pid=29233}, {rtm_family=AF_INET, rtm_dst_len=32, rtm_src_len=0, rtm_tos=0, rtm_table=RT_TABLE_LOCAL, rtm_protocol=RTPROT_UNSPEC, rtm_scope=RT_SCOPE_UNIVERSE, rtm_type=RTN_LOCAL, rtm_flags=RTM_F_CLONED|0x80000000}, [{{nla_len=8, nla_type=RTA_TABLE}, RT_TABLE_LOCAL}, {{nla_len=8, nla_type=RTA_DST}, inet_addr("4.4.4.4")}, {{nla_len=8, nla_type=RTA_OIF}, if_nametoindex("lo")}, {{nla_len=8, nla_type=RTA_PREFSRC}, inet_addr("4.4.4.4")}, {{nla_len=8, nla_type=RTA_UID}, 0}, {{nla_len=36, nla_type=RTA_CACHEINFO}, {rta_clntref=1, rta_lastuse=0, rta_expires=0, rta_error=0, rta_used=0, rta_id=0, rta_ts=0, rta_tsage=0}}]}, iov_len=104}], msg_iovlen=1, msg_controllen=0, msg_flags=0}, 0) = 104 On Linux 5.2.0-rc5: recvmsg(3, {msg_name={sa_family=AF_NETLINK, nl_pid=0, nl_groups=00000000}, msg_namelen=12, msg_iov=[{iov_base={{len=112, type=RTM_NEWROUTE, flags=0, seq=1562754944, pid=4275}, {rtm_family=AF_INET, rtm_dst_len=32, rtm_src_len=0, rtm_tos=0, rtm_table=RT_TABLE_LOCAL, rtm_protocol=RTPROT_UNSPEC, rtm_scope=RT_SCOPE_UNIVERSE, rtm_type=RTN_LOCAL, rtm_flags=RTM_F_CLONED|0x80000000}, [{{nla_len=8, nla_type=RTA_TABLE}, RT_TABLE_LOCAL}, {{nla_len=8, nla_type=RTA_DST}, 4.4.4.4}, {{nla_len=8, nla_type=RTA_OIF}, if_nametoindex("lo")}, {{nla_len=8, nla_type=RTA_PREFSRC}, 4.4.4.4}, {{nla_len=8, nla_type=RTA_GATEWAY}, 4.4.4.4}, {{nla_len=8, nla_type=RTA_UID}, 0}, {{nla_len=36, nla_type=RTA_CACHEINFO}, {rta_clntref=1, rta_lastuse=0, rta_expires=0, rta_error=0, rta_used=0, rta_id=0, rta_ts=0, rta_tsage=0}}]}, iov_len=112}], msg_iovlen=1, msg_controllen=0, msg_flags=0}, 0) = 112 This commit extends the check for route to a non-directly reachable gateway by making the check to pass if a gateway IP addr is equal to a given dst IP address. Signed-off-by: Martynas Pumputis <m@lambda.lt> Signed-off-by: John Fastabend <john.fastabend@gmail.com> 17 July 2019, 22:58:11 UTC
0d7c713 update to golang 1.12.7 [ upstream commit 7266476ee5a90d13724e4cf7eeba1f888d9b2821 ] Signed-off-by: André Martins <andre@cilium.io> Signed-off-by: John Fastabend <john.fastabend@gmail.com> 17 July 2019, 22:58:11 UTC
2e9778a test: update k8s testing versions to v1.12.10, v1.13.8 and v1.14.4 [ upstream commit 17c0fabb44695a42e42d5404fc853ea1dc11d5d9 ] Signed-off-by: André Martins <andre@cilium.io> Signed-off-by: John Fastabend <john.fastabend@gmail.com> 17 July 2019, 22:58:11 UTC
a84b29b update golang to 1.12.7 for cilium-{operator,docker-plugin} [ upstream commit 0091cd685ab1927801425e92b509231133df7174 ] Fixes: 7266476ee5a9 ("update to golang 1.12.7") Signed-off-by: André Martins <andre@cilium.io> Signed-off-by: John Fastabend <john.fastabend@gmail.com> 17 July 2019, 22:58:11 UTC
0faaf26 endpoint: do not log warning for specific state transition [ upstream commit 02f05da1b036b2e774461a9fc8cf857a039a5274 ] Do not log a warning when transition from waiting-to-regenerate to waiting-to-regenerate is triggered. The correct action is to do nothing, since a regeneration is already enqueued. Several callers rely upon existing behavior (i.e., that we do not allow this as a valid state transition) so that we do not enqueue a lot of regenerations, which are costly operations. Maintain this behavior, but suppress the warning log since there will not be any degradation in performance / incorrect state realized. Signed-off by: Ian Vernon <ian@cilium.io> Signed-off-by: John Fastabend <john.fastabend@gmail.com> 17 July 2019, 22:58:11 UTC
9ba0504 Prepare for v1.5.5 Signed-off by: Ian Vernon <ian@cilium.io> 11 July 2019, 20:42:12 UTC
7eaaf08 lbmap: Get rid of bpfService cache lock [ upstream commit 48bef164ce991aab6c097079352d2de1bdf4c271 ] This commit: - Removes the bpfService cache lock, as any call to the cache happens via lbmap, and lbmap itself is protected by its own lock. - Makes sure that the lbmap lock is taken in the very beginning of each public method definition of lbmap. Signed-off-by: Martynas Pumputis <m@lambda.lt> Signed-off-by: Ian Vernon <ian@cilium.io> 10 July 2019, 16:59:40 UTC
5c623f5 retry vm provisioning, increase timeout [ upstream commit 2a50e8fd5cdd1620b23026fbdd24fa1d95ae82d5 ] Signed-off-by: Maciej Kwiek <maciej@isovalent.com> Signed-off-by: Ian Vernon <ian@cilium.io> 10 July 2019, 16:59:40 UTC
10ca060 daemon: Remove svc-v2 maps when restore is disabled [ upstream commit 302c70faef4f29ecc9a696a1d15ade95e81ad0a1 ] Previously, in the case of `--restore=false`, the svc-v2 related BPF maps were not removed. Signed-off-by: Martynas Pumputis <m@lambda.lt> Signed-off-by: Ian Vernon <ian@cilium.io> 10 July 2019, 16:59:40 UTC
2953f7d daemon: Do not remove revNAT if removing svc fails [ upstream commit 23fd0d7cd4262ee1fc647f5a1078075b1f1f4fb0 ] It has been observed that the daemon can receive the duplicate events for a k8s svc removal, e.g.: msg="Kubernetes service definition changed" action=service-deleted endpoints= k8sNamespace=default k8sSvcName=migrate-svc-6 service="frontend:10.104.17.176/ports=[]/selector=map[app:migrate-svc-server-6]" subsys=daemon <..> msg="Kubernetes service definition changed" action=service-deleted endpoints= k8sNamespace=default k8sSvcName=migrate-svc-6 service="frontend:10.104.17.176/ports=[]/selector=map[app:migrate-svc-server-6]" subsys=daemon g msg="Error deleting service by frontend" error="Service frontend not found 10.104.17.176:8000" k8sNamespace=default k8sSvcName=migrate-svc-6 obj="10.104.17.176:8000" subsys=daemon msg="# cilium lb delete-rev-nat 32" k8sNamespace=default k8sSvcName=migrate-svc-6 subsys=daemon msg="deleting L3n4Addr by ID" l3n4AddrID=33 subsys=service In such situations, the daemon tries to remove the revNAT entry twice, which can lead to a removal of a valid revNAT entry if in between the events the daemon creates a new service with the same ID. Fix this by skipping the revNAT removal if the frontend removal fails. Signed-off-by: Martynas Pumputis <m@lambda.lt> Signed-off-by: Ian Vernon <ian@cilium.io> 10 July 2019, 16:59:40 UTC
4354cd5 pkg/k8s: add conversion for DeleteFinalStateUnknown objects [ upstream commit 39b0190817642789f44f34f01fee811d330fdb3d ] As k8s watchers can return DeleteFinalStateUnknown objects, cilium needs to be able to convert those object types into DeleteFinalStateUnknown where its internal Obj is also converted into private cilium types. Fixes: 027ccdc31d11 ("pkg/k8s: add converter functions from k8s to cilium types") Signed-off-by: André Martins <andre@cilium.io> Signed-off-by: Ian Vernon <ian@cilium.io> 10 July 2019, 16:59:40 UTC
58c1314 cli: fix panic in cilium bpf sha get command [ upstream commit a9861d12a67254d2ff8ecea174c0c8a460dee4d4 ] * Fixes #8423 * Fixes panic issue in `cilium bpf sha get` command when no sha is provided Signed-off-by: Deepesh Pathak <deepshpathak@gmail.com> Signed-off-by: Ian Vernon <ian@cilium.io> 09 July 2019, 22:54:47 UTC
42232a3 Retry provisioning vagrant vms in CI [ upstream commit eed40849fe45afd6b477e4ad029b1cb8b7a793f3 ] Signed-off-by: Maciej Kwiek <maciej@covalent.io> Signed-off-by: Ian Vernon <ian@cilium.io> 09 July 2019, 22:54:47 UTC
3b22bae pkg/k8s: hold mutex while adding events to the queue [ upstream commit 7eb649efcfcc2376ecb87e3780a9aacdc55b8e48 ] As there are certain code regions where the mutex is held it might occur the events are put in to the channel out of the original order they got the mutex. Fixes: 7efa98e03b9b ("k8s: Factor out service and endpoints correlation into a new ServiceCache") Signed-off-by: André Martins <andre@cilium.io> Signed-off-by: Ian Vernon <ian@cilium.io> 09 July 2019, 22:54:47 UTC
a3fae39 Change nightly CI job label from fixed to baremetal [ upstream commit 39c279614f8ce7f6efc0e84bbb3acc3d646d33d0 ] Signed-off-by: Maciej Kwiek <maciej@covalent.io> Signed-off-by: Ian Vernon <ian@cilium.io> 09 July 2019, 22:54:47 UTC
ad2f08b test: set 1.15 by default in CI Vagrantfile [ upstream commit 81dfcb918a8d41a5ad6a3f46c770a666a3e475cc ] Fixes: d7fb1c1eec1a ("test: run k8s 1.15.0 by default in all PRs") Signed-off-by: André Martins <andre@cilium.io> Signed-off-by: Ian Vernon <ian@cilium.io> 09 July 2019, 22:54:47 UTC
fac1699 daemon: Change loglevel of "ipcache entry owned by kvstore or agent" [ upstream commit ae1cf42dd079b409ec3bd898b190563753280121 ] This commit changes the loglevel of the following log entry: level=warning msg="Unable to update ipcache entry of Kubernetes node" error="ipcache entry owned by kvstore or agent" This message is not harmful, as after cilium-agent has established connectivity to a kvstore, it's expected that IPCache entries will be owned by kvstore. Signed-off-by: Martynas Pumputis <m@lambda.lt> Signed-off-by: Ian Vernon <ian@cilium.io> 09 July 2019, 22:54:47 UTC
back to top