https://github.com/cilium/cilium

sort by:
Revision Author Date Message Commit Date
13c64ae Prepare for v1.5.6 release Signed-off by: Ian Vernon <ian@cilium.io> 16 August 2019, 15:28:16 UTC
6b91141 endpoint: Fix proxy port leak on endpoint delete [ upstream commit 0c7f066529815f4d51f64b709d6a52fba8ee8b25 ] Deep in the proxy.removeRedirect() logic, a finalize function is created to defer proxy port release to allow a port to be reused by the proxy after a grace period. The port is not actually released when the removeRedirect() function is returned, instead the finalize function is passed all the way up to the caller. In the endpoint leave case, this finalize function was being ignored, meaning that the proxy port used by this endpoint is never released. After 10,000 endpoint deletions of endpoints that have L7 policy applied, the endpoint regeneration begins to fail for new endpoints that are subject to L7 policy. As a result, the CNI times out: Failed create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "xxx" network for pod "yyy": NetworkPlugin cni failed to set up pod "yyy" network: Unable to create endpoint: Put http:///var/run/cilium/cilium.sock/v1/endpoint/cilium-local:0: context deadline exceeded Fix this by running the finalize function upon endpoint leave. Signed-off-by: Joe Stringer <joe@cilium.io> Signed-off-by: Ian Vernon <ian@cilium.io> 16 August 2019, 05:19:31 UTC
0c89b51 update cilium-docker-plugin, cilium-operator to golang 1.12.8 Signed-off by: Ian Vernon <ian@cilium.io> 16 August 2019, 05:19:31 UTC
743d1e0 dockerfiles: update golang versions to 1.12.8 [ upstream commit 3439d8895407adf4aec727778941da8ce9c1107f ] Signed-off by: Ian Vernon <ian@cilium.io> Signed-off-by: Ian Vernon <ian@cilium.io> 16 August 2019, 05:19:31 UTC
8899b0b cilium: install transient rules during agent restart [ upstream commit 80507f142c59a5d0ca6b782e6b8092e0c93a7246 ] Jaff and Arthur reported that during agent restart while ping test with low interval is running and trying to reach one of the pods on the node where the agent is restarting a few dropped packets have been seen. This is due to Cilium agent reloading its iptables rules, in particular, the CILIUM_FORWARD chain is affected here since once removed for a short window we'll be hitting the default DROP policy in FORWARD. Add a temporary transient chain that keeps basic ACCEPT rules alive during re-setup until main rules have been set in place. Reported-by: Jaff Cheng and Arthur Chiao via Slack Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Signed-off-by: Ian Vernon <ian@cilium.io> 16 August 2019, 05:19:31 UTC
e0758fe Istio: Update to 1.2.4 Istio release 1.2.4 includes the sidecar proxy with Envoy security fixes released on 8/13/2019. Signed-off-by: Jarno Rajahalme <jarno@covalent.io> 15 August 2019, 18:01:30 UTC
c8eae6d envoy: Use patched image [ upstream commit a309f9886c34a834a03bd7a4079db99119e364f8 ] Signed-off-by: Jarno Rajahalme <jarno@covalent.io> 14 August 2019, 14:52:02 UTC
06c6520 bpf: fix verifier error due to repulling of skb->data/end [ upstream commit fdca23e2b23f91f6672211961ee3b0a29449506c ] After commit 5aebc46cacba ("bpf: Attempt pulling skb->data if it is not pulled") running the agent with nodeport enabled broke with a verifier error as LLVM tried to generate code with pkt_end arithmetic which does not work. [...] level=info msg="Cilium 1.6.90 2c6863714 2019-08-07T09:06:44-07:00 go version go1.11.2 linux/amd64" subsys=daemon level=info msg="Envoy version check disabled" subsys=daemon level=info msg="clang (7.0.1) and kernel (5.3.0) versions: OK!" subsys=daemon [...] level=warning msg=" R0=inv4294967154 R1_rw=invP4294967154 R2_rw=invP4294967154 R3=inv1155182100513554433 R4=inv0 R5=inv(id=0) R6=invP0 R7=inv40 R8_rw=invP0 R9=invP0 R10=fp0 fp-56=????mmmm fp-104=??0m0000 fp-112=mmmmmmmm fp-120=mmmmmmmm fp-128=mmmmmmmm fp-136=mmmmmmmm fp-144=00000000 fp-152=mmmmmmmm fp-160=mmmmmmmm fp-168=????0000 fp-176=00000000 fp-184=00000000 fp-192=00000000 fp-200_r=ctx" subsys=daemon level=warning msg="parent didn't have regs=100 stack=0 marks" subsys=daemon level=warning msg="last_idx 306 first_idx 297" subsys=daemon level=warning msg="regs=100 stack=0 before 306: (18) r2 = 0xffffff72" subsys=daemon level=warning msg="regs=100 stack=0 before 305: (b7) r8 = 0" subsys=daemon level=warning msg="907: (61) r1 = *(u32 *)(r7 +80)" subsys=daemon level=warning msg="908: (61) r9 = *(u32 *)(r7 +76)" subsys=daemon level=warning msg="909: (18) r8 = 0xffffff7a" subsys=daemon level=warning msg="911: (b7) r6 = 0" subsys=daemon level=warning msg="912: (67) r1 <<= 32" subsys=daemon level=warning msg="R1 pointer arithmetic on pkt_end prohibited" subsys=daemon level=warning msg="processed 191 insns (limit 1000000) max_states_per_insn 0 total_states 8 peak_states 8 mark_read 5" subsys=daemon level=warning subsys=daemon level=warning msg="Error filling program arrays!" subsys=daemon level=warning msg="Unable to load program" subsys=daemon [...] Fix to force intermediate writes by using WRITE_ONCE() to avoid such code generation would work for newer kernels, but I'm getting a verifier rejection on 4.19.57: [...] level=warning msg="333: (7b) *(u64 *)(r10 -232) = r2" subsys=daemon level=warning msg="334: (65) if r1 s> 0xa goto pc+373" subsys=daemon level=warning msg=" R0=inv(id=0,umax_value=2147483647,var_off=(0x0; 0x7fffffff)) R1=inv(id=0,umax_value=10,var_off=(0x0; 0xf)) R2=inv1 R6=ctx(id=0,off=0,imm=0) R7=inv0 R8=inv8 R9=inv(id=0) R10=fp0,call_-1 fp-64=? fp-80=pkt fp-88=pkt_end fp-128=? fp-216=map_ptr fp-224=map_value" subsys=daemon level=warning msg="335: (15) if r1 == 0x0 goto pc+436" subsys=daemon level=warning msg=" R0=inv(id=0,umax_value=2147483647,var_off=(0x0; 0x7fffffff)) R1=inv(id=0,umax_value=10,var_off=(0x0; 0xf)) R2=inv1 R6=ctx(id=0,off=0,imm=0) R7=inv0 R8=inv8 R9=inv(id=0) R10=fp0,call_-1 fp-64=? fp-80=pkt fp-88=pkt_end fp-128=? fp-216=map_ptr fp-224=map_value" subsys=daemon level=warning msg="336: (15) if r1 == 0x3 goto pc+373" subsys=daemon level=warning msg=" R0=inv(id=0,umax_value=2147483647,var_off=(0x0; 0x7fffffff)) R1=inv(id=0,umax_value=10,var_off=(0x0; 0xf)) R2=inv1 R6=ctx(id=0,off=0,imm=0) R7=inv0 R8=inv8 R9=inv(id=0) R10=fp0,call_-1 fp-64=? fp-80=pkt fp-88=pkt_end fp-128=? fp-216=map_ptr fp-224=map_value" subsys=daemon level=warning msg="337: (15) if r1 == 0x8 goto pc+1" subsys=daemon level=warning msg=" R0=inv(id=0,umax_value=2147483647,var_off=(0x0; 0x7fffffff)) R1=inv(id=0,umax_value=10,var_off=(0x0; 0xf)) R2=inv1 R6=ctx(id=0,off=0,imm=0) R7=inv0 R8=inv8 R9=inv(id=0) R10=fp0,call_-1 fp-64=? fp-80=pkt fp-88=pkt_end fp-128=? fp-216=map_ptr fp-224=map_value" subsys=daemon level=warning msg="338: (05) goto pc+437" subsys=daemon level=warning msg="776: (71) r1 = *(u8 *)(r10 -127)" subsys=daemon level=warning msg="invalid size of register spill" subsys=daemon [...] After playing around half a day, the only thing I got to accept is to perform the skb_pull_data() unconditionally before the data/data_end fetch, therefore the workaround follows this approach until we find something better. Fixes: 5aebc46cacba ("bpf: Attempt pulling skb->data if it is not pulled") Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Signed-off-by: Joe Stringer <joe@cilium.io> 14 August 2019, 08:20:43 UTC
9409a2a bpf: Attempt pulling skb->data if it is not pulled [ upstream commit 5aebc46cacba2c164a6bb789f3bc185e5f0256e2 ] If the skb->data_end is too short for Cilium to perform its logic, attempt to pull it in. If it's still too short, we can legitimately drop the traffic as "invalid". Signed-off-by: Joe Stringer <joe@cilium.io> 14 August 2019, 08:20:43 UTC
ef64583 bpf: Introduce revalidate_data_first() [ upstream commit 9093a619f908647fe7f9189a5b8a84d17ed888f1 ] In some environments depending on the underlying NIC driver in use, the skb may not arrive pulled with all data contiguously. In that case, Cilium may end up dropping legitimate traffic because it is unable to access enough of the packet to perform its functions. Such cases exist today, refactor the function into a separate helper for better code reuse. Signed-off-by: Joe Stringer <joe@cilium.io> 14 August 2019, 08:20:43 UTC
4d66d3d cilium: add skb_pull_data to bpf_network to avoid revalidate error [ upstream commit a03c33a592e3c4a4933f6a951b3b0de3161af8fb ] Depending on the driver when we have a full stack of headers with tunneled IPsec its possible the headers are not part of the linear data. In this case we should pull in the data instead of immediately aborting. If the pull fails we can then abort. For this patch we only fix the immediate problem call site. We need to audit the remaining ones. However, veth devices should not have this issue in general so bpf_lxc.c should be OK. skbs on bpf_netdev either passed through bpf_network.c, bpf_lxc.c or were routed from the host networking. Fixes: 47bd875512770 ("cilium: encrypt, use ipcache to lookup IPsec destination IP") Signed-off-by: John Fastabend <john.fastabend@gmail.com> Signed-off-by: Joe Stringer <joe@cilium.io> 14 August 2019, 08:20:43 UTC
d0cd5c3 datapath/iptables: wait until acquisition xtables lock is done [ upstream commit 5f46e1abd9e92b6ff6ec4050d1e5edf3d60d117a ] This guarantees iptables interactions are not run concurretly with other programs, e.g. kube-proxy. More info: https://git.netfilter.org/iptables/commit/?id=93587a04d0f2511e108bbc4d87a8b9d28a5c5dd8 Signed-off-by: André Martins <andre@cilium.io> Signed-off-by: Ian Vernon <ian@cilium.io> 14 August 2019, 08:20:43 UTC
0f8a112 use iptables-manager to manage iptables executions [ upstream commit 7240e834b9ba492ea3554bf89c0789d5d4f0ff5c ] This will make it easier to run iptables commands based on iptables version installed in the system. Signed-off-by: André Martins <andre@cilium.io> Signed-off by: Ian Vernon <ian@cilium.io> 14 August 2019, 08:20:43 UTC
b9a8cf4 examples/kubernetes: mount xtables.lock [ upstream commit 5f46e1abd9e92b6ff6ec4050d1e5edf3d60d117a ] As Cilium access iptables it should also mount the xtables.lock file into iptables. Signed-off-by: André Martins <andre@cilium.io> Signed-off by: Ian Vernon <ian@cilium.io> 14 August 2019, 08:20:43 UTC
4fff691 eventqueue: return error if Enqueue fails [ upstream commit bb115ad8face1af7d90eb044612411e69b68c35c ] Given that an Enqueue may not occur due to the queue not being initialized correctly / the Event being nil, or the Event has already been Enqueued before, return an error which follows Golang conventions instead of encoding error state in the returned value. Signed-off by: Ian Vernon <ian@cilium.io> Signed-off-by: Ian Vernon <ian@cilium.io> 14 August 2019, 08:20:43 UTC
922dd4b eventqueue: protect against enqueueing same Event twice [ upstream commit 9e3e123872750bc40d758b5eb7860772c6d3f268 ] Previously, if an Event is enqueued twice, the following panic would occur: ``` PANIC: eventqueue_test.go:187: EventQueueSuite.TestPanic ... Panic: close of closed channel (PC=0x42EFD4) /usr/local/go/src/runtime/panic.go:522 in gopanic /usr/local/go/src/runtime/chan.go:342 in closechan eventqueue.go:188 in EventQueue.Enqueue eventqueue_test.go:197 in EventQueueSuite.TestPanic /usr/local/go/src/reflect/value.go:308 in Value.Call /usr/local/go/src/runtime/asm_amd64.s:1337 in goexit OOPS: 9 passed, 1 PANICKED Signed-off-by: Ian Vernon <ian@cilium.io> 14 August 2019, 08:20:43 UTC
19450ab eventqueue: use mutex to synchronize access to events channel [ upstream commit 2c68637145e11b89254b3c6aa431b9de5c77b6c4 ] Previously, a `sync.WaitGroup` was used to ensure the events channel for a given `EventQueue` was not closed while a call to `Enqueue` was in in-flight. However, a panic could result when stopping a queue, which called `Wait` on said `WaitGroup`, and multiple calls to `Enqueue` were called while said `Wait` was happening: ``` panic: sync: WaitGroup is reused before previous Wait has returned goroutine 55 [running]: sync.(*WaitGroup).Wait(0xc000360120) /usr/local/go/src/sync/waitgroup.go:132 +0xae github.com/cilium/cilium/pkg/eventqueue.(*EventQueue).Stop.func1() /go/src/github.com/cilium/cilium/pkg/eventqueue/eventqueue.go:279 +0x18a sync.(*Once).Do(0xc000360114, 0xc0000e57b8) /usr/local/go/src/sync/once.go:44 +0xb3 github.com/cilium/cilium/pkg/eventqueue.(*EventQueue).Stop(0xc0003600f0) /go/src/github.com/cilium/cilium/pkg/eventqueue/eventqueue.go:269 +0x81 ``` This occured because the `WaitGroup` is waiting while at the same time that `WaitGroup` has more deltas being added to its internal counter. Replace this `WaitGroup` with a mutex that is Locked when closing the events channel, blocking out in-flight calls to `Enqueue`. Calls to `Enqueue` take an `RLock` so multiple Enqueues can occur at the same time without a problem. Signed-off by: Ian Vernon <ian@cilium.io> Signed-off-by: Ian Vernon <ian@cilium.io> 14 August 2019, 08:20:43 UTC
d20ebb5 daemon: get list of frontends from ServiceCache before acquiring BPFMapMu [ upstream commit b6a1790f05c5c4dfcbe25b15db6d5ed8bd115dc6 ] The `ServiceCache` mutex should always be acquired before acquiring the `BPFMapMu` of the `LoadBalancer`. The following deadlock was observed in a cilium instance: We are trying to acquire the `BPFMapMu` in the daemon's `LoadBalancer` to add a Kubernetes service as observed by the Kubernetes watcher: ``` goroutine 150 [semacquire, 225 minutes]: sync.runtime_SemacquireMutex(0xc000401d44, 0x0) /usr/local/go/src/runtime/sema.go:71 +0x3d sync.(*Mutex).Lock(0xc000401d40) /usr/local/go/src/sync/mutex.go:134 +0x109 sync.(*RWMutex).Lock(0xc000401d40) /usr/local/go/src/sync/rwmutex.go:93 +0x2d main.(*Daemon).svcAdd(0xc0001b83c0, 0xc002d96eb0, 0x10, 0x10, 0xc001afa069, 0x3, 0x2474, 0x38f, 0x4d8bf28, 0x0, ...) /go/src/github.com/cilium/cilium/daemon/loadbalancer.go:132 +0x630 main.(*Daemon).addK8sSVCs(0xc0001b83c0, 0xc001badec0, 0x2d, 0xc001bbb000, 0xb, 0x0, 0xc000e73130, 0xc003e5e8a0, 0x0, 0x0) /go/src/github.com/cilium/cilium/daemon/k8s_watcher.go:1288 +0xf37 main.(*Daemon).k8sServiceHandler(0xc0001b83c0) /go/src/github.com/cilium/cilium/daemon/k8s_watcher.go:1064 +0xa29 created by main.(*Daemon).runK8sServiceHandler /go/src/github.com/cilium/cilium/daemon/k8s_watcher.go:1118 +0x3f ``` The controller which makes sure that the daemon's LoadbLanacer maps are in sync with Kubernetes has possession of the aforementioned `BPFMapMu` mutex, but is trying to acquire the mutex for the `ServiceCache` after having acquired the `BPFMapMu`: ``` goroutine 401 [semacquire, 225 minutes]: sync.runtime_SemacquireMutex(0xc0001b8514, 0x0) /usr/local/go/src/runtime/sema.go:71 +0x3d sync.(*RWMutex).RLock(...) /usr/local/go/src/sync/rwmutex.go:50 github.com/cilium/cilium/pkg/k8s.(*ServiceCache).UniqueServiceFrontends(0xc0001b8500, 0x0) /go/src/github.com/cilium/cilium/pkg/k8s/service_cache.go:427 +0x37d main.(*Daemon).syncLBMapsWithK8s(0xc0001b83c0, 0x0, 0x0) /go/src/github.com/cilium/cilium/daemon/loadbalancer.go:643 +0x1a4 main.(*Daemon).initRestore.func2.1(0x35013c0, 0xc00590cb80, 0x4d6d3e0, 0x0) /go/src/github.com/cilium/cilium/daemon/state.go:483 +0x2a github.com/cilium/cilium/pkg/controller.(*Controller).runController(0xc005abe000) /go/src/github.com/cilium/cilium/pkg/controller/controller.go:203 +0x9e1 created by github.com/cilium/cilium/pkg/controller.(*Manager).UpdateController /go/src/github.com/cilium/cilium/pkg/controller/manager.go:112 +0xae0 ``` The deadlock is due to the following channel send being stuck: ``` goroutine 356 [chan send, 225 minutes]: github.com/cilium/cilium/pkg/k8s.(*ServiceCache).UpdateEndpoints(0xc0001b8500, 0xc0042a87c0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) /go/src/github.com/cilium/cilium/pkg/k8s/service_cache.go:297 +0x454 main.(*Daemon).addK8sEndpointV1(...) /go/src/github.com/cilium/cilium/daemon/k8s_watcher.go:1136 main.(*Daemon).EnableK8sWatcher.func7.2(0x0, 0xc000e3ca20) /go/src/github.com/cilium/cilium/daemon/k8s_watcher.go:530 +0x46 github.com/cilium/cilium/pkg/serializer.(*FunctionQueue).run(0xc000e36930) /go/src/github.com/cilium/cilium/pkg/serializer/func_queue.go:70 +0x6e created by github.com/cilium/cilium/pkg/serializer.NewFunctionQueue /go/src/github.com/cilium/cilium/pkg/serializer/func_queue.go:50 +0xb9 ``` The function `UpdateEndpoints` is holding the mutex for the `ServiceCache`. The reason that the channel send above is stuck, is because the goroutine which receives off of the channel (the first one shown above) is trying to acquire the `BPFMapMu`. But, since the goroutine holding the `ServiceCache` mutex is blocking on sending to the channel, it will never release the mutex, which the goroutine holding the `BPFMapMu` is trying to acquire before it can release the `BPFMapMu`, which in turn is blocking the channel receive goroutine from progressing further! The simplest fix for now is to call `UniqueServiceFrontends` before acquiring the `BPFMapMu` in the controller which synchronizes state in the datapath. This ensures that the controller will eventually release the `BPFMapMu`, which will allow the goroutine receiving events in the Kubernetes watcher to acquire it, and read from the channel of service-related events. Signed-off by: Ian Vernon <ian@cilium.io> Signed-off-by: Ian Vernon <ian@cilium.io> 14 August 2019, 08:20:43 UTC
982a613 cilium: remove old probe content before restoring assets [ upstream commit a2a2c6a277378d7311dbb19d28e836d7af6c140b ] Example of compilation failure from agent when downgrading from master back to 1.5: + clang -O2 -I/tmp/tmp.WSx7wj9jyh -I/var/lib/cilium/bpf/probes -I/var/lib/cilium/bpf/include -Wall /var/lib/cilium/bpf/probes/raw_main.c -o /tmp/tmp.WSx7wj9jyh/raw_sock_cookie.t In file included from /var/lib/cilium/bpf/probes/raw_main.c:52: /tmp/tmp.WSx7wj9jyh/raw_probe.t:8:4: error: field designator 'attach_type' does not refer to any field in type 'struct bpf_test' .attach_type = BPF_CGROUP_INET4_CONNECT, ^ 1 error generated. + cleanup + '[' '!' -z /tmp/tmp.WSx7wj9jyh ']' + rm -rf /tmp/tmp.WSx7wj9jyh Reason is that run_probes.sh will just pull in every *.t test from /var/lib/cilium/bpf/probes/ and upon downgrade newer ones are still present in this directory. We probably could remove the entire option.Config.RunDir directory if option.Config.KeepTemplates is false, but given we earlier on created the os.MkdirAll(option.Config.RunDir, ...) unconditionally, it's a bit cleaner to just remove the probes dir right before the restoring of bindata assets since this is the only thing we actually need. Fixes: #7971 Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Signed-off-by: Ian Vernon <ian@cilium.io> 14 August 2019, 08:20:43 UTC
6b7d50f cilium: encryption, ensure 0x*d00 and 0x*e00 marks dont cause conflicts [ upstream commit 5aae4ab99703690c6e69a4d4dd14473d4681862a ] We mark encryption packets with 0xd00 and 0xe00 values to indicate if the packet should be encrypted (0xe00) or decrypted (0xd00). In the encryption case we also set the next byte (0x*e00) with the keyID to encrypt the packet with. This allows multiple keys to be in use at any specific time. However, it was observed that the upper bits, where the keyID is placed, may also be used by kube-proxy. To avoid collision add rules at the top of iptables to accept any packets with the encrypt/decrypt mark values. These values will be cleared again before being pushed to the stack so normal rules will still be hit. Notice, we already had these rules to exclude encrypted traffic in the masquerading case. This moves those rules installation to be done any time encryption is enabled. This issue has existed from the initial implementation, but our CI and my testing never used those key id with colliding features. Fixes: b2b901fb19163 ("cilium: ipsec, add go API to configure xfrm (IPSec)") Signed-off-by: John Fastabend <john.fastabend@gmail.com> 05 August 2019, 23:53:08 UTC
ece8603 Dockerfile: Use proxy with legacy fix [ upstream commit 57420053d387350268d49498e890523f46869b06 ] Use cilium-envoy with a fix for regression that broke support for Cilium versions without TPROXY support (< 1.6). Signed-off-by: Jarno Rajahalme <jarno@covalent.io> 31 July 2019, 21:47:35 UTC
2f12d79 envoy: Add SO_MARK option to listener config [ upstream commit a56938bc27d88ee723f70f9a1d331d5098b3352b ] Envoy is removing socket option support from Listener callbacks, so we need to set the SO_MARK option in the listener config instead. Use a cilium-envoy image that does not set the same option from the cilium filters. Signed-off-by: Jarno Rajahalme <jarno@covalent.io> 31 July 2019, 21:47:35 UTC
167c74e test: provide capability for tests to run in their own namespace [ upstream commit 2c670be9e69b98edce045a9ce5800164f5926dc0 ] Running each set of tests in a `Describe` in its own namespace instead of the `default` namespace ensures that resource leakage between tests w.r.t. policies, services, and other associated resources stored in K8s is kept to a minimum. It also allows for deletion of all resources in a namespace with one command at the end of a run of a given set of tests. To allow for this functionality, a variety of changes had to be made: * add variadic arguments for providing namespace to existing commands for backwards compatability. * change namespace parameter of `CiliumPolicyAction` to apply the policy in the namespace parameter, and hardcode the namespace used to get Cilium pods to be `kube-system`. Change callers accordingly. One given set of tests within the `K8sPolicyTest` have been changed to use a specific namespace. Misc. fixes / changes needed for said tests to pass were added as well. Other tests can be migrated over time to use their own namespaces. Signed-off by: Ian Vernon <ian@cilium.io> 31 July 2019, 21:47:35 UTC
82703c6 docs: Fix warnings [ upstream commit b2159fb0e6b69bb23f070fe1651a2379205e6bc0 ] Sphinx reported some warnings with these docs, fix them. Signed-off-by: Joe Stringer <joe@cilium.io> 31 July 2019, 21:47:35 UTC
a0bc714 test: Specify protocol during policy trace [ upstream commit aa8b483f87ae0a6a5f801878d9af21c67f2441d4 ] This avoids failures when upcoming commits make the L4 policy trace more strict, to determine exactly whether traffic is allowed depending on the protocol as well as the port. Signed-off-by: Joe Stringer <joe@cilium.io> 31 July 2019, 21:47:35 UTC
0faef63 istio: Update to 1.2.2 [ upstream commit 9e8c5dc2fda12c372da56710d0ccef48051a9a5e ] Update cilium-envoy to a version with Istio 1.2.2 filters. Update Istio GSG and CI test to use the new proxy and pilot images. Signed-off-by: Jarno Rajahalme <jarno@covalent.io> 31 July 2019, 21:47:35 UTC
ec8742d envoy: Istio 1.2.0 update [ upstream commit cf916b9f31c54d041c3da74544730a86e0300b73 ] Update to the proxy image with Istio 1.2.0 filters. Update Istio GSG and tests to 1.2.0. Signed-off-by: Jarno Rajahalme <jarno@covalent.io> 31 July 2019, 21:47:35 UTC
2863af8 istio: Update to 1.1.7 [ upstream commit 6de86b6d8f61b1d3838c79f69ad510e12a5ed2bc ] Update Istio to 1.1.7. Signed-off-by: Jarno Rajahalme <jarno@covalent.io> 31 July 2019, 21:47:35 UTC
4cb7243 test: be sure to close SSH client after a given Describe completes [ upstream commit 6181ca25f573051a2b62fd25aee84f78a2d91b0b ] We were never closing the clients we created, and the code should clean up after itself once it is finished. Do so. This adds some boilerplate code, unfortunately, but it's better than leaving open clients around. Signed-off by: Ian Vernon <ian@cilium.io>" 31 July 2019, 21:47:35 UTC
213ef4a Dockerfile: Use cilium-envoy with reduced logging. [ upstream commit 990339d2d97b343175ce12b19facc82eb9ba6528 ] Use a new envoy image with reduced logging. This should remove these from the CI output: Top 5 errors/warnings: [[cilium/network_policy.cc:129] Bad Network Policy Configuration [[cilium/host_map.cc:187] Bad NetworkPolicyHosts Configuration Signed-off-by: Jarno Rajahalme <jarno@covalent.io> 31 July 2019, 21:47:35 UTC
8c196ce Envoy: Update to the latest proxy build, use latest API [ upstream commit 2777f9b603866a413cbd5a78b7593595c46cd5fa ] Update cilium-envoy image and API reference to the latest. Signed-off-by: Jarno Rajahalme <jarno@covalent.io> 31 July 2019, 21:47:35 UTC
5aecd0d Gopkg: update cilium/proxy [ upstream commit f27130849a88ec74f1397705c659481de63c0509 ] The dependencies are now vendored in the cilium/proxy repository. Cilium will import the right dependencies for the cilium/proxy based in the Gopkg.toml of that directory. Signed-off-by: André Martins <andre@cilium.io> 31 July 2019, 21:47:35 UTC
a8fe173 envoy: Use LPM ipcache instead of xDS when available. [ upstream commit 9102ea8597cf8c3631f64820f6dc7f70497c24d4 ] Use Envoy version that can use bpf ipcache instead of NetworkPolicyHosts xDS, when available. If the bpf ipcache LPM map open fails (such as on Linux kernels before 4.11), Envoy falls back to using xDS. Signed-off-by: Jarno Rajahalme <jarno@covalent.io> 31 July 2019, 21:47:35 UTC
6d0f6d4 Envoy: Use an image with proxylib injection fix. [ upstream commit 8406aa652023a089903c0592f4e88268cd53956a ] Cilium network filter failed to call back to proxylib when the OnData call returned with a full inject buffer. Use a newer Envoy image that fixes this. Proxylib testing tooling is updated to support large injections by simulating Envoy datapath processing of the injected data by clearing the inject buffer on CheckOnData() and CheckOnDataOK() calls. blockparser (one of the test parsers) can now support larger injections in the original direction. This is used in cilium-envoy integration testing. Signed-off-by: Jarno Rajahalme <jarno@covalent.io> 31 July 2019, 21:47:35 UTC
9ac7c80 Dockerfile: Update proxy dependency [ upstream commit 813538189b888cd124950be4ddd953b6dd8d3c5d ] Use newer proxy image that has support for deriving policy name from the endpoint IP, and for deriving the proxylib parser name from the network policy. These changes allow listener configuration to be independent of an endpoint or an endpoint's network policy. Signed-off-by: Jarno Rajahalme <jarno@covalent.io> 31 July 2019, 21:47:35 UTC
face465 CI: Change Kafka runtime tests to use local conntrack maps. [ upstream commit 4c1f82042861e656d2c7457f7119c124a3265363 ] It's good to have test coverage with the local conntrack maps. Optimally would keep also the global map test cases. Signed-off-by: Jarno Rajahalme <jarno@covalent.io> 31 July 2019, 21:47:35 UTC
ad24712 [daemon] - Change MTU source for cilium_host (Use the Route one) [ upstream commit 86f926e816dc871bdd985c82f556644aae58c6d9 ] * for pods in hostNetwork the mtu was the same a the default interface. Packets were thus dropped once the stream was encrypted. Fixes: 1ce278f ("daemon: move IPSec bootstrap into separate function") Signed-off-by: Ray Bejjani <ray@isovalent.com> 31 July 2019, 21:46:04 UTC
6a35073 endpoint: fix deadlock when endpoint EventQueue is full [ upstream commit 5a18ed9d7825413fb67c1957f8350de8f6784744 ] This commit fixes a problem that affects a deadlock during Endpoint deletion: An event is being processed by the endpoint's EventQueue, but has not yet acquired the Endpoint's mutex. The endpoint's EventQueue is full (e.g., it has reached its full capacity), and there are calls to `Enqueue` for the Endpoint which are blocking on returning due to the queue being full. The endpoint is being deleted in `deleteEndpointQuiet`, and has acquired the Endpoint's mutex. The event being processed by the endpoint's EventQueue tries to acquire the Endpoint's mutex. It cannot, because `deleteEndpointQuiet` just acquired the Endpoint's mutex. The EventQueue's buffer is still full; the event currently being processed cannot be consumed due to the deadlock, so as a result, calls to Enqueue are still blocking forever. `deleteEndpointQuiet` calls `Stop` on the EventQueue. This never returns, because calls to Enqueue are still in flight, and EventQueues have a safeguard built in via a `sync.WaitGroup` which guards against writing to closed channels during `Enqueue`; that is, the EventQueue cannot be closed while Enqueues are still in progress. Enqueues in this case, are still in progress, because the channel is full. But, the channel will never drain due to `deleteEndpointQuiet` holding the lock. `deleteEndpointQuiet` will never release the lock, because it is waiting for the queue to be drained. The fix for the above is to stop the Endpoint's EventQueue before we acquire the Endpoint's mutex, and ensure that the queue is drained (e.g., no events are queued up for it) after we stop the queue as well. This ensures that if multiple calls to `deleteEndpointQuiet` occur for the same Endpoint, they all wait before acquiring the Endpoint's mutex. Add a unit test in the daemon to test this fix as well. Signed-off by: Ian Vernon <ian@cilium.io> Signed-off-by: Ray Bejjani <ray@isovalent.com> 31 July 2019, 21:46:04 UTC
751b147 daemon: register warning_error metric after parsing CLI options [ upstream commit e9e59511071f1020411f63eecf394cc1a4ee99b4 ] As the logger was being setup in an init function, and since the metrics are only enabled after parsing the configuration flag, the default metrics function being used was a NoOp which was causing the metric from not being reported. Fixes: 0fec218c33ff ("pkg/metrics: set all metrics as a no-op unless they are enabled") Signed-off-by: André Martins <andre@cilium.io> Signed-off-by: Ian Vernon <ian@cilium.io> 26 July 2019, 08:44:14 UTC
2e1b03e Fix seds in microk8s docs [ upstream commit 797c3d55c42b32e4ea162ec426c6a42881ee7db3 ] Sed commands rendered without backslashes which caused them to not be copy-able. Use semicolon as the delimiter for sed substitutions with paths. Signed-off-by: Maciej Kwiek <maciej@isovalent.com> Signed-off-by: Ian Vernon <ian@cilium.io> 26 July 2019, 08:44:14 UTC
f100934 daemon: Fix removal of non-existing SVCs in syncLBMapsWithK8s [ upstream commit a0694039b921ac5414ebeded0fd4a81958eb7e91 ] 59315dcdcf missed the fact that population of "daemon.loadBalancer" happens only via k8s_watcher's "UpdateService". So, in the case of a non-existing SVC, "daemon.loadBalancer" doesn't have an entry for the svc. Revert the change, but keep the locking at the top of the function. Fixes: 59315dcdcf ("daemon: Remove svc from cache in syncLBMapsWithK8s") Signed-off-by: Martynas Pumputis <m@lambda.lt> Signed-off-by: Ian Vernon <ian@cilium.io> 24 July 2019, 17:51:17 UTC
a8a61c3 daemon: Remove svc from cache in syncLBMapsWithK8s [ upstream commit a9d01db066d0799403b880a65a9d50d5949bcf49 ] Previously, the syncLBMapsWithK8s method did not remove a service from the d.loadBalancer, only from the BPF svc maps. This allowed for the following race condition to happen: d.delK8sSVCs(svcA) | d.syncLBMapsWithK8s() | | | k8sSvcCache.UniqueFrontends()=(no svcA) | | | d.loadBalancer.BPFMapMU.Lock() | | | lbmap.dumpServiceMapsToUserspaceV2()=(svcA) | | | delete svcA from BPF maps | | | d.loadBalancer.BPFMapMU.Unlock() | | d.loadBalancer.BPFMapMU.Lock() | svcA exists in d.loadBalancer? | delete svcA from BPF maps This race could have led to removing the same backend twice which could explain the "backend $ID not found" error observed in the wild. Also, move the locking in syncLBMapsWithK8s above the retrieval of k8s services from the cache to avoid any other not yet discovered race condition. Fixes: cc4be8e3 ("daemon: sync BPF maps with in-memory K8s service maps") Signed-off-by: Martynas Pumputis <m@lambda.lt> Signed-off-by: Ian Vernon <ian@cilium.io> 24 July 2019, 17:51:17 UTC
7f602b0 examples/kubernetes: update k8s dev VM to v1.15.1 [ upstream commit 8e4c6c42b68196915fe3c236fb6074ecf1b18015 ] Signed-off-by: André Martins <andre@cilium.io> Signed-off-by: Ian Vernon <ian@cilium.io> 24 July 2019, 17:51:17 UTC
0d2ddd8 test: update k8s test version to v1.15.1 [ upstream commit 931f688ce6169a8e19f67e21e504738680b5abaf ] Signed-off-by: André Martins <andre@cilium.io> Signed-off-by: Ian Vernon <ian@cilium.io> 24 July 2019, 17:51:17 UTC
f8fed2e Gopkg: update k8s dependencies to v1.15.1 [ upstream commit a6e0eb8ebc8b360a21b0011d516a24c13843d1d8 ] Signed-off-by: André Martins <andre@cilium.io> Signed-off-by: Ian Vernon <ian@cilium.io> 24 July 2019, 17:51:17 UTC
3b86737 Add timeout to ginkgo calls [ upstream commit 531a554b8ea139e0e23a853b46a70d671f70104d ] Add timeout to make ginkgo exit instead of being interrupted by Jenkins. Should help with builds being marked as aborted instead of failed. Signed-off-by: Maciej Kwiek <maciej@isovalent.com> Signed-off-by: Ian Vernon <ian@cilium.io> 24 July 2019, 17:51:17 UTC
c68b5f3 proxy: Do not error out if reading of open ports fails. [ upstream commit 15f5e2687d58f47f1f92847e104f2530e521560a ] Checking the random port number against known open ports is a precaution to avoid trying to re-use an open port. We should not error out if the reading of the open ports fails, however, as there is a high likelihood that the port selection will result in an usable port anyway. Signed-off-by: Jarno Rajahalme <jarno@covalent.io> Signed-off-by: Ian Vernon <ian@cilium.io> 24 July 2019, 17:51:17 UTC
276795a pkg/kvstore: wait for node delete delay in unit tests [ upstream commit 30ad6cb9fde07de42dade87c9637cd2820652f86 ] Since Close() deletes all keys from the kvstore the global kvstore client will still receive delete events from the kvstore across unit tests, causing keys from being deleted across unit tests. Fixes: 07312c6d155d ("pkg/{kvstore,node}: delay node delete event in kvstore") Signed-off-by: André Martins <andre@cilium.io> Signed-off-by: Ian Vernon <ian@cilium.io> 23 July 2019, 22:40:14 UTC
d362a6a endpoint: Create redirects before bpf map updates. [ upstream commit 7fa113fe7de6a4979396ed6f4ba40c6d12c3e60f ] Create new redirects and generate the new proxy policy before updating bpf policy maps in order to give the proxies a chance to be ready when new traffic is redirected at them. This could help reduce transient drops in some cases. Remove old redirects only if the endpoint has a security identity. To begin with, endpoints can have redirects only if they have a security identity, and the only instance where the endpoint's security identity is nilled is when it is being destroyed. Fixes: ea62a6b2eb2 ("endpoint: Fix sidecar proxy deadlock during BPF generation") Signed-off-by: Jarno Rajahalme <jarno@covalent.io> Signed-off-by: Ian Vernon <ian@cilium.io> 23 July 2019, 22:40:14 UTC
8d74c20 proxy: Perform dnsproxy Close() in the returned finalizeFunc [ upstream commit fcc7d3b4ea341f22d0c79a85c7758d74f98d0a10 ] Removing the dns redirect implementation state should be only performed when we know all updates are successful, i.e., when the returned finalizeFunc is run. Alternatively, we could revert the changes in the revertFunc, but we'd rather remove the revertFunc return value in a later commit. Signed-off-by: Jarno Rajahalme <jarno@covalent.io> Signed-off-by: Ian Vernon <ian@cilium.io> 23 July 2019, 22:40:14 UTC
ba2fecb endpoint: change transition from restore state [ upstream commit 837d920fed30eba08173d6eee7146f16825c9b37 ] Restoring state for an endpoint is quite a special case for the endpoint; it only occurs once per-endpoint per-run of cilium-agent. Whenever we restore an endpoint, we have specific logic within the restoration process which queues it for a regeneration. Allowing the transition from restoring state to waiting to regenerate means that if the endpoint is exposed to the rest of the cilium-agent via the endpointmanager, that while an endpoint is in restoring state, another thread could change its state, and the endpoint would regenerate out-of-band of the restoring goroutine for the endpoint. This has produced noise in our CI because the regeneration triggered via the restore goroutine triggered for endpoints has failed when the endpoint is restored, exposed to the endpointmanager, and another goroutine swoops in and begins to restore it, putting it into a state which restoring state cannot transition into. When the restore goroutine then tries to regenerate the endpoint, it is no longer in a state which is valid for regeneration, and the regeneration fails, putting the endpoint into a not-ready state. So, remove the transition from restoring to waiting-to-regenerate state. All other code in Cilium tries to set the endpoint into waiting-to-regenerate state, which will fail if it is in restoring state. However, the restoring goroutine will be able to transition into regenerating state directly. This semantically makes sense since restoring is a special case, and we want to guarantee exclusivity of regenerating the endpoint for the first time to just the restoring goroutine. Signed-off by: Ian Vernon <ian@cilium.io> Signed-off-by: Ian Vernon <ian@cilium.io> 23 July 2019, 22:40:14 UTC
bd2eef1 test: misc. runtime policy test fixes [ upstream commit 98f5eb2be8ebaf27922ebdc0f3f438b7d8c4b57c ] * Ensure that Cilium is started with the set of configuration options set in `SetUpCilium`, specifically of excluding of local addresses that are used to simulate external IPs for CIDR policy testing. * Fix a log error message to contain a string that was expected. Signed-off by: Ian Vernon <ian@cilium.io> Signed-off-by: Ian Vernon <ian@cilium.io> 23 July 2019, 22:40:14 UTC
fd40c3f docs: Fix up unparsed SCM_WEB literals [ upstream commit 43696add2a42ceed92d2c710036fb14db456d971 ] These weren't being parsed, and so were rendered as SCM_WEB rather than having the github link. Fix them up. Signed-off-by: Joe Stringer <joe@cilium.io> Signed-off-by: Ian Vernon <ian@cilium.io> 23 July 2019, 22:40:14 UTC
8b78780 pkg/{kvstore,node}: delay node delete event in kvstore [ upstream commit 07312c6d155d11b8a8dd5f7f621c1f13b2266066 ] Instead of checking if the node is still present in the observer, we should check if the node is still present in the store instead. Checking if the node is still available in the observer is irrelevant as the node will never be deleted since the check for its existence in the store will always be true. Instead, delay the propagation of the deletion event to the observer in the store itself when we receive shared keys notifications. Since the propagaton delay is done at the node observer the unit tests will take longer to complete. Fixes: 3137ad5eee6c ("node: Delay handling of node delete events received via kvstore") Signed-off-by: André Martins <andre@cilium.io> Signed-off-by: Ian Vernon <ian@cilium.io> 23 July 2019, 22:40:14 UTC
9525c77 operator: restart non-managed kube-dns pods before connecting to etcd [ upstream commit fd2ff2ac93e4d8527e833159b81746065556e4d6 ] In a new cluster, if [kube|core]-dns is not being managed by Cilium it can make etcd, from cilium-etcd-operator, on not being able to reach [kube|core]-dns. Restarting [kube|core]-dns to force it to be managed by Cilium will make etcd cluster to be able to be setup properly. Since cilium-operator blocks until it gets connectivity with etcd it will never restart [kube|core]-dns as the restart of such pod happens after cilium-operator gets connectivity with etcd. Signed-off-by: André Martins <andre@cilium.io> Signed-off-by: Ian Vernon <ian@cilium.io> 23 July 2019, 22:40:14 UTC
4cfe301 test: move creation of Istio resources into `It` [ upstream commit 425e6b455f8f0c663c36cba3acc3bcbdfdbb8f52 ] The creation of Istio resources in the Istio CI test contains assertions. Previously, these were ran in the `BeforeAll` block. If the assertions failed, `AfterFailed` would not be invoked (which is most likely a bug). To ensure that we are able to debug the Istio test, for now, move the creation of Istio resources / the assertions on their readiness and corectness into the `It` for the Istio test. No other `It`s are used in this test, so no other test functionality is broken here. In the future, this commit can be reverted when `AfterFailed` is invoked when assertions fail in `BeforeAll`. Signed-off by: Ian Vernon <ian@cilium.io> Signed-off-by: Ian Vernon <ian@cilium.io> 23 July 2019, 22:40:14 UTC
072ea66 test: add `ExecMiddle` function [ upstream commit 2cab00aa9abbb9d0fac307c60206765d1a1f1abc ] In certain cases, commands have failed when given a timeout of only 10 seconds, the value used for `ExecShort`. Add another small utility function which has a timeout of 30 seconds to use for said commands. Cases where this has been observed are for `Apply` for a resource in Kubernetes. Plumb accordingly. Signed-off by: Ian Vernon <ian@cilium.io> Signed-off-by: John Fastabend <john.fastabend@gmail.com> 17 July 2019, 22:58:11 UTC
35ae17e datapath: Do not fail if route contains gw equal to dst [ upstream commit 8ce9b22cd358b1a5be6db31fe09266e30fb7cf0e ] Some recent change in the kernel started to include RTA_GATEWAY field in a response to RTM_GETROUTE even if a dst was local. On Linux 5.1.6: recvmsg(3, {msg_name={sa_family=AF_NETLINK, nl_pid=0, nl_groups=00000000}, msg_namelen=12, msg_iov=[{iov_base={{len=104, type=RTM_NEWROUTE, flags=0, seq=1562755119, pid=29233}, {rtm_family=AF_INET, rtm_dst_len=32, rtm_src_len=0, rtm_tos=0, rtm_table=RT_TABLE_LOCAL, rtm_protocol=RTPROT_UNSPEC, rtm_scope=RT_SCOPE_UNIVERSE, rtm_type=RTN_LOCAL, rtm_flags=RTM_F_CLONED|0x80000000}, [{{nla_len=8, nla_type=RTA_TABLE}, RT_TABLE_LOCAL}, {{nla_len=8, nla_type=RTA_DST}, inet_addr("4.4.4.4")}, {{nla_len=8, nla_type=RTA_OIF}, if_nametoindex("lo")}, {{nla_len=8, nla_type=RTA_PREFSRC}, inet_addr("4.4.4.4")}, {{nla_len=8, nla_type=RTA_UID}, 0}, {{nla_len=36, nla_type=RTA_CACHEINFO}, {rta_clntref=1, rta_lastuse=0, rta_expires=0, rta_error=0, rta_used=0, rta_id=0, rta_ts=0, rta_tsage=0}}]}, iov_len=104}], msg_iovlen=1, msg_controllen=0, msg_flags=0}, 0) = 104 On Linux 5.2.0-rc5: recvmsg(3, {msg_name={sa_family=AF_NETLINK, nl_pid=0, nl_groups=00000000}, msg_namelen=12, msg_iov=[{iov_base={{len=112, type=RTM_NEWROUTE, flags=0, seq=1562754944, pid=4275}, {rtm_family=AF_INET, rtm_dst_len=32, rtm_src_len=0, rtm_tos=0, rtm_table=RT_TABLE_LOCAL, rtm_protocol=RTPROT_UNSPEC, rtm_scope=RT_SCOPE_UNIVERSE, rtm_type=RTN_LOCAL, rtm_flags=RTM_F_CLONED|0x80000000}, [{{nla_len=8, nla_type=RTA_TABLE}, RT_TABLE_LOCAL}, {{nla_len=8, nla_type=RTA_DST}, 4.4.4.4}, {{nla_len=8, nla_type=RTA_OIF}, if_nametoindex("lo")}, {{nla_len=8, nla_type=RTA_PREFSRC}, 4.4.4.4}, {{nla_len=8, nla_type=RTA_GATEWAY}, 4.4.4.4}, {{nla_len=8, nla_type=RTA_UID}, 0}, {{nla_len=36, nla_type=RTA_CACHEINFO}, {rta_clntref=1, rta_lastuse=0, rta_expires=0, rta_error=0, rta_used=0, rta_id=0, rta_ts=0, rta_tsage=0}}]}, iov_len=112}], msg_iovlen=1, msg_controllen=0, msg_flags=0}, 0) = 112 This commit extends the check for route to a non-directly reachable gateway by making the check to pass if a gateway IP addr is equal to a given dst IP address. Signed-off-by: Martynas Pumputis <m@lambda.lt> Signed-off-by: John Fastabend <john.fastabend@gmail.com> 17 July 2019, 22:58:11 UTC
0d7c713 update to golang 1.12.7 [ upstream commit 7266476ee5a90d13724e4cf7eeba1f888d9b2821 ] Signed-off-by: André Martins <andre@cilium.io> Signed-off-by: John Fastabend <john.fastabend@gmail.com> 17 July 2019, 22:58:11 UTC
2e9778a test: update k8s testing versions to v1.12.10, v1.13.8 and v1.14.4 [ upstream commit 17c0fabb44695a42e42d5404fc853ea1dc11d5d9 ] Signed-off-by: André Martins <andre@cilium.io> Signed-off-by: John Fastabend <john.fastabend@gmail.com> 17 July 2019, 22:58:11 UTC
a84b29b update golang to 1.12.7 for cilium-{operator,docker-plugin} [ upstream commit 0091cd685ab1927801425e92b509231133df7174 ] Fixes: 7266476ee5a9 ("update to golang 1.12.7") Signed-off-by: André Martins <andre@cilium.io> Signed-off-by: John Fastabend <john.fastabend@gmail.com> 17 July 2019, 22:58:11 UTC
0faaf26 endpoint: do not log warning for specific state transition [ upstream commit 02f05da1b036b2e774461a9fc8cf857a039a5274 ] Do not log a warning when transition from waiting-to-regenerate to waiting-to-regenerate is triggered. The correct action is to do nothing, since a regeneration is already enqueued. Several callers rely upon existing behavior (i.e., that we do not allow this as a valid state transition) so that we do not enqueue a lot of regenerations, which are costly operations. Maintain this behavior, but suppress the warning log since there will not be any degradation in performance / incorrect state realized. Signed-off by: Ian Vernon <ian@cilium.io> Signed-off-by: John Fastabend <john.fastabend@gmail.com> 17 July 2019, 22:58:11 UTC
9ba0504 Prepare for v1.5.5 Signed-off by: Ian Vernon <ian@cilium.io> 11 July 2019, 20:42:12 UTC
7eaaf08 lbmap: Get rid of bpfService cache lock [ upstream commit 48bef164ce991aab6c097079352d2de1bdf4c271 ] This commit: - Removes the bpfService cache lock, as any call to the cache happens via lbmap, and lbmap itself is protected by its own lock. - Makes sure that the lbmap lock is taken in the very beginning of each public method definition of lbmap. Signed-off-by: Martynas Pumputis <m@lambda.lt> Signed-off-by: Ian Vernon <ian@cilium.io> 10 July 2019, 16:59:40 UTC
5c623f5 retry vm provisioning, increase timeout [ upstream commit 2a50e8fd5cdd1620b23026fbdd24fa1d95ae82d5 ] Signed-off-by: Maciej Kwiek <maciej@isovalent.com> Signed-off-by: Ian Vernon <ian@cilium.io> 10 July 2019, 16:59:40 UTC
10ca060 daemon: Remove svc-v2 maps when restore is disabled [ upstream commit 302c70faef4f29ecc9a696a1d15ade95e81ad0a1 ] Previously, in the case of `--restore=false`, the svc-v2 related BPF maps were not removed. Signed-off-by: Martynas Pumputis <m@lambda.lt> Signed-off-by: Ian Vernon <ian@cilium.io> 10 July 2019, 16:59:40 UTC
2953f7d daemon: Do not remove revNAT if removing svc fails [ upstream commit 23fd0d7cd4262ee1fc647f5a1078075b1f1f4fb0 ] It has been observed that the daemon can receive the duplicate events for a k8s svc removal, e.g.: msg="Kubernetes service definition changed" action=service-deleted endpoints= k8sNamespace=default k8sSvcName=migrate-svc-6 service="frontend:10.104.17.176/ports=[]/selector=map[app:migrate-svc-server-6]" subsys=daemon <..> msg="Kubernetes service definition changed" action=service-deleted endpoints= k8sNamespace=default k8sSvcName=migrate-svc-6 service="frontend:10.104.17.176/ports=[]/selector=map[app:migrate-svc-server-6]" subsys=daemon g msg="Error deleting service by frontend" error="Service frontend not found 10.104.17.176:8000" k8sNamespace=default k8sSvcName=migrate-svc-6 obj="10.104.17.176:8000" subsys=daemon msg="# cilium lb delete-rev-nat 32" k8sNamespace=default k8sSvcName=migrate-svc-6 subsys=daemon msg="deleting L3n4Addr by ID" l3n4AddrID=33 subsys=service In such situations, the daemon tries to remove the revNAT entry twice, which can lead to a removal of a valid revNAT entry if in between the events the daemon creates a new service with the same ID. Fix this by skipping the revNAT removal if the frontend removal fails. Signed-off-by: Martynas Pumputis <m@lambda.lt> Signed-off-by: Ian Vernon <ian@cilium.io> 10 July 2019, 16:59:40 UTC
4354cd5 pkg/k8s: add conversion for DeleteFinalStateUnknown objects [ upstream commit 39b0190817642789f44f34f01fee811d330fdb3d ] As k8s watchers can return DeleteFinalStateUnknown objects, cilium needs to be able to convert those object types into DeleteFinalStateUnknown where its internal Obj is also converted into private cilium types. Fixes: 027ccdc31d11 ("pkg/k8s: add converter functions from k8s to cilium types") Signed-off-by: André Martins <andre@cilium.io> Signed-off-by: Ian Vernon <ian@cilium.io> 10 July 2019, 16:59:40 UTC
58c1314 cli: fix panic in cilium bpf sha get command [ upstream commit a9861d12a67254d2ff8ecea174c0c8a460dee4d4 ] * Fixes #8423 * Fixes panic issue in `cilium bpf sha get` command when no sha is provided Signed-off-by: Deepesh Pathak <deepshpathak@gmail.com> Signed-off-by: Ian Vernon <ian@cilium.io> 09 July 2019, 22:54:47 UTC
42232a3 Retry provisioning vagrant vms in CI [ upstream commit eed40849fe45afd6b477e4ad029b1cb8b7a793f3 ] Signed-off-by: Maciej Kwiek <maciej@covalent.io> Signed-off-by: Ian Vernon <ian@cilium.io> 09 July 2019, 22:54:47 UTC
3b22bae pkg/k8s: hold mutex while adding events to the queue [ upstream commit 7eb649efcfcc2376ecb87e3780a9aacdc55b8e48 ] As there are certain code regions where the mutex is held it might occur the events are put in to the channel out of the original order they got the mutex. Fixes: 7efa98e03b9b ("k8s: Factor out service and endpoints correlation into a new ServiceCache") Signed-off-by: André Martins <andre@cilium.io> Signed-off-by: Ian Vernon <ian@cilium.io> 09 July 2019, 22:54:47 UTC
a3fae39 Change nightly CI job label from fixed to baremetal [ upstream commit 39c279614f8ce7f6efc0e84bbb3acc3d646d33d0 ] Signed-off-by: Maciej Kwiek <maciej@covalent.io> Signed-off-by: Ian Vernon <ian@cilium.io> 09 July 2019, 22:54:47 UTC
ad2f08b test: set 1.15 by default in CI Vagrantfile [ upstream commit 81dfcb918a8d41a5ad6a3f46c770a666a3e475cc ] Fixes: d7fb1c1eec1a ("test: run k8s 1.15.0 by default in all PRs") Signed-off-by: André Martins <andre@cilium.io> Signed-off-by: Ian Vernon <ian@cilium.io> 09 July 2019, 22:54:47 UTC
fac1699 daemon: Change loglevel of "ipcache entry owned by kvstore or agent" [ upstream commit ae1cf42dd079b409ec3bd898b190563753280121 ] This commit changes the loglevel of the following log entry: level=warning msg="Unable to update ipcache entry of Kubernetes node" error="ipcache entry owned by kvstore or agent" This message is not harmful, as after cilium-agent has established connectivity to a kvstore, it's expected that IPCache entries will be owned by kvstore. Signed-off-by: Martynas Pumputis <m@lambda.lt> Signed-off-by: Ian Vernon <ian@cilium.io> 09 July 2019, 22:54:47 UTC
36437ec pkg/kvstore: add etcd lease information into cilium status [ upstream commit 8c70c3728c56d171aad0e1e5b5383fdc9fe9595c ] Signed-off-by: André Martins <andre@cilium.io> Signed-off-by: Ian Vernon <ian@cilium.io> 09 July 2019, 22:54:47 UTC
77f8997 pkg/k8s: do not parse empty annotations [ upstream commit 185a86b8f203f936b22d7ad8f04c07f947fcdae2 ] It seems annotations for a particular field can exist but they may be empty which causes the parsing of that empty string into an IP to fail. Signed-off-by: André Martins <andre@cilium.io> Signed-off-by: Ian Vernon <ian@cilium.io> 09 July 2019, 22:54:47 UTC
efe0057 maps/lbmap: protect service cache refcount with concurrent access [ upstream commit 17da3aebd48a77f9d4cb37c32f24b4aedbe52043 ] As updating services can modify the lbmapCache the mutex to access the same bpf map should be held before modifying the cache. Without it we can risk to have the bpf map out of sync with the lbmap cache. Fixes: 2766bc127368 ("daemon,lbmap: Do not update legacy svc if they are disabled") Signed-off-by: André Martins <andre@cilium.io> Signed-off-by: Ian Vernon <ian@cilium.io> 09 July 2019, 22:54:47 UTC
e6f53b2 operator: add warning message if status returns an error [ upstream commit d0db460d55f89f33f740b725c8f1a696c2347e51 ] It might be helpful to track down why kubernetes killed cilium-operator due the readiness probe failures. Signed-off-by: André Martins <andre@cilium.io> Signed-off-by: Ian Vernon <ian@cilium.io> 09 July 2019, 22:54:47 UTC
9fec2a5 pkg/kvstore: fix nil pointer in error while doing a transaction in etcd [ upstream commit 7e16b286b598980e1ee97dbf3d9798dd742d9953 ] txnresp can obviously be nil if err is != nil. We should check if txnresp.Succeeded after checking if err == nil first. Fixes: 84291c30ece5 ("pkg/kvstore: implement new *IfLocked methods for etcd") Signed-off-by: André Martins <andre@cilium.io> Signed-off-by: Ian Vernon <ian@cilium.io> 09 July 2019, 22:54:47 UTC
9d72bdf examples/kubernetes: bump cilium to v1.5.4 Signed-off-by: André Martins <andre@cilium.io> 02 July 2019, 14:32:25 UTC
73a052b bpf: Remove unneeded debug instructions to stay below instruction limit Signed-off-by: Thomas Graf <thomas@cilium.io> 02 July 2019, 01:52:05 UTC
2e29c31 bpf: Prohibit encapsulation traffic from pod when running in encapsulation mode An endpoint can emit encapsulation traffic to a worker node and given certain conditions are met, the worker node will accept the encapsulated traffic and route it. This can bypass egress security policies. If the endpoint is able to guess security identities correctly, the endpoint can also impersonate other security identities. Conditions that must be bet: * Cilium must be running in encapsulation mode. * Endpoint must be aware of at least one worker node IP or be aware of an external LB that redirects to a worker node. * The egress policy of the endpoint must allow UDP on the configured encapsulation port. Alternatively, if the endpoint has access to an external LB such as a NodePort which redirects to the configured UDP encapsulation port on a worker, the policy must allow for this. Redirection to a UDP encapsulation port using a CluserIP or headless service is not affected as the load-balancing decision happens before egress policy. * Masquerading of egress traffic to worker node IP must be enabled, if masquerading is disabled, the remote tunnel will reject the traffic. This means that known or guessed worker node IP must be a node IP which is considered outside of the cluster so masquerading is performed. * The endpoint must guess a security identity or be aware of well-known security identities. * The targeted destination endpoint as defined in the inner IP header must allow the guessed or known source security identity as per ingress policy. In order to mitigate this, any UDP traffic to one of the supported encapsulation ports is now dropped with a distinct drop reason: ``` xx drop (Encapsulation traffic is prohibited) flow 0xcab0685a to endpoint 0, identity 3087->2: 10.16.209.20:39258 -> 192.168.122.124:8472 udp ``` Signed-off-by: Thomas Graf <thomas@cilium.io> 02 July 2019, 01:52:05 UTC
d4cbbfd pkg/endpointmanager: protecting endpoints against concurrent access [ upstream commit fbc9ba004c022f8abcdb040e2689cf6dfbe9566f ] Fixes: f71d87a71c99 ("endpointmanager: signal when work is done") Signed-off-by: André Martins <andre@cilium.io> Signed-off-by: Jarno Rajahalme <jarno@covalent.io> 28 June 2019, 03:17:46 UTC
72b7a3a test: set k8s 1.15 as default k8s version [ upstream commit 8c2a33d5fcc05d567d08cc5c8094ac474fd4717a ] Fixes: 4794445b0e02 ("test: test against 1.15.0") Signed-off-by: André Martins <andre@cilium.io> Signed-off-by: Jarno Rajahalme <jarno@covalent.io> 28 June 2019, 03:17:46 UTC
3041379 CI: Clean VMs and reclaim disk in nightly test [ upstream commit d7f5931634ad239505725fa5edd70c38d5adc66c ] We didn't cleanup VM images in the nighty test. This isn't a problem overall but nodes that may see more nightly builds (e.g. jenkins-fixed-node-0) may run out of space if only these jobs execute. We now run the cleanup script in a similar way to our other jobs. fixes 0af448da73fb9d8ebb64de6a6e1052f52b9a2a43 Signed-off-by: Ray Bejjani <ray@covalent.io> Signed-off-by: Jarno Rajahalme <jarno@covalent.io> 28 June 2019, 03:17:46 UTC
a55b83f allocator: fix race condition when allocating local identities upon bootstrap [ upstream commit d146a2efd28ca6073c10a74836abf1b93011de5f ] When identities are attempted to be allocated by `AllocateIdentity`, both the labels are checked for if the identity is local, *and* whether the `localIdentities` structure is non-nil: ``` ... if !identity.RequiresGlobalIdentity(lbls) && localIdentities != nil { return localIdentities.lookupOrCreate(lbls) } // else allocate global identity ... ``` Upon bootstrap, the creation of the localIdentities structure was done asynchronously along with the creation of the global identity allocator. This meant that if a local identity for a CIDR was attempted to be allocated upon bootstrap soon after `InitIdentityAllocator` was invoked, there was no guarantee as to whether `localIdentities` was initialized or not. Consequently, even if a set of labels did correspond to a local identity (e.g., for a CIDR), there would be no local identity allocated for the CIDR, and instead, a global identity would be allocated. This is incorrect behavior. Fix this incorrect behavior by doing the following: * Create `localIdentities` synchronously with calls to `InitIdentityAllocator`. The initialization of the global allocator is still done asynchronously. * Do not asynchronously call `InitIdentityAllocator` upon bootstrap any more, since `InitIdentityAllocator` synchronously creates the local identity allocator now, but now creates the global identity allocator asynchronously. * Wait to allocate local identities until the local identity allocator is initialized instead of having the aforementioned nil-check. If an identity does not require a global identity, it should always be allocated a local identity. Instead of checking whether `localIdentities` is nil, wait for it to be initialized so that we can allocate a local identity for all identities which are not global. Additionally, return the channel which is closed upon the allocator being initialized so that callers to `InitIdentityAllocator` can wait upon it being initialized to preserve existing behavior in unit tests. Also remove oudated comments about relying on the kvstore for allocating CIDR identities upon daemon bootstrap. Fixes: f3bbcd8e88 ("identity: Use local identities to represent CIDR") Signed-off by: Ian Vernon <ian@cilium.io> Signed-off-by: Jarno Rajahalme <jarno@covalent.io> 28 June 2019, 03:17:46 UTC
2cb7d70 identity: Initialize well-known identities before the policy repository. [ upstream commit cdf3b6ec11a3437b558cdf671ebbc4f2b8f3d117 ] NewPolicyRepository() now grabs the IdentityCache, so all well-known identities must be initialized before then. This commit removes InitWellKnownIdentities() from InitIdentityAllocator() so that the well-known identities can be initialized earlier. Unfortunately the well-known identities depend on the runtime configuration and thus can't be statically initialized. Before this commit there was a potential for race in initializing and using the well-known identities map. Now that the well-known identities map is initialized earlier, this race is less likely to be a problem. Signed-off-by: Jarno Rajahalme <jarno@covalent.io> 28 June 2019, 03:17:46 UTC
b21e708 cilium: docker.go ineffectual assignment [ upstream commit c67655cf8eb198ed1b0c02dea76ad5842b467a3a ] Remove unnecessary runtime assignment. ineffassign . /home/john/go/src/github.com/cilium/cilium/pkg/workloads/docker.go:244:4: ineffectual assignment to runtimeRunning make: *** [Makefile:395: ineffassign] Error 1 Fixes: 910d8d7f6a16a ("pkg/workloads: add containerd integration") Signed-off-by: John Fastabend <john.fastabend@gmail.com> Signed-off-by: Jarno Rajahalme <jarno@covalent.io> 28 June 2019, 03:17:46 UTC
d47f399 Disable automatic direct node routes test This test is consistently failing on the v1.5 branch, disable it. Related: https://github.com/cilium/cilium/issues/8378 Signed-off-by: Joe Stringer <joe@cilium.io> 27 June 2019, 00:14:05 UTC
bff1730 kubernetes-upstream: add seperate stage to run tests [ upstream commit 46f9b503465c45a4e3f065424711bff31108821b ] Signed-off-by: André Martins <andre@cilium.io> 27 June 2019, 00:14:05 UTC
fa53405 docs: update documentation with k8s 1.15 support [ upstream commit f1aff1c681c27502169fde0a37dc415aec4daa9d ] Signed-off-by: André Martins <andre@cilium.io> 27 June 2019, 00:14:05 UTC
1816521 test: run k8s 1.15.0 by default in all PRs [ upstream commit d7fb1c1eec1a34ea9ead95d9e9c1d52283a08f32 ] Signed-off-by: André Martins <andre@cilium.io> 27 June 2019, 00:14:05 UTC
f654adf test: test against 1.15.0 [ upstream commit 4794445b0e022320924798a5c9a457fe57be7b66 ] Signed-off-by: André Martins <andre@cilium.io> 27 June 2019, 00:14:05 UTC
ce6ea6b vendor: update k8s to v1.15.0 [ upstream commit 5d8a18795370d94d3dfe9ebaf0f5878f0f19b6c6 ] As a consequence `k8s.io/kubernetes/pkg/kubelet/apis/cri/runtime/v1alpha2` is now `k8s.io/cri-api/pkg/apis/runtime/v1alpha2` Signed-off-by: André Martins <andre@cilium.io> Signed-off-by: André Martins <andre@cilium.io> 27 June 2019, 00:14:05 UTC
7ec5b44 bpf: Set random MAC addrs for cilium interfaces [ upstream commit acced7e5e0836fd6e79b6b17e2473876f1f75d2f ] To work around the systemd 242+ feature which tries to assign a persistent MAC address for any device by default (see commit message of the previous commit for more details). Signed-off-by: Martynas Pumputis <m@lambda.lt> Signed-off-by: André Martins <andre@cilium.io> 27 June 2019, 00:14:05 UTC
02477fe endpoint: Set random MAC addrs for veth when creating it [ upstream commit 719c2e5bfbf7c931da8535d491aa15a19421c072 ] systemd 242+ tries to set a "persistent" MAC addr for any virtual device by default (controlled by MACAddressPolicy). As setting happens asynchronously after a device has been created, ep.Mac and ep.HostMac can become stale which has a serious consequence - the kernel will drop any packet sent to/from the endpoint. However, we can trick systemd by explicitly setting MAC addrs for both veth ends. This sets addr_assign_type for NET_ADDR_SET which prevents systemd from changing the addrs. Signed-off-by: Martynas Pumputis <m@lambda.lt> Signed-off-by: André Martins <andre@cilium.io> 27 June 2019, 00:14:05 UTC
0e7feab vendor: Update vishvananda/netlink [ upstream commit a8bb290d45c06e80ed39299fdcbc99deecc9a065 ] To include the change which allows to specify veth peer mac address (https://github.com/vishvananda/netlink/pull/460). Signed-off-by: Martynas Pumputis <m@lambda.lt> Signed-off-by: André Martins <andre@cilium.io> 27 June 2019, 00:14:05 UTC
629fa7a mac: Add function to generate a random MAC addr [ upstream commit 143d5d36227cec18a9d5476250438ef6f3d3221f ] A generated MAC addr is unicast and locally administered. Signed-off-by: Martynas Pumputis <m@lambda.lt> Signed-off-by: André Martins <andre@cilium.io> 27 June 2019, 00:14:05 UTC
6334912 test: remove unused function [ upstream commit 4257665ab0c83e21f8ee2d821eed1d740608beda ] Signed-off by: Ian Vernon <ian@cilium.io> Signed-off-by: Joe Stringer <joe@cilium.io> 27 June 2019, 00:14:05 UTC
610b83e test: introduce `ExecShort` function [ upstream commit 47bdde0d447183e2f7f2ce5d4be4b852ffaaecdf ] This runs commands that do not require a lot of time to complete with a timeout of 10 seconds. Before, the default was up to 5 minutes. Now, we have separate timeouts depending on the nature of the command being ran. Also use context more consistently across other various helper functions. Signed-off by: Ian Vernon <ian@cilium.io> Signed-off-by: Joe Stringer <joe@cilium.io> 27 June 2019, 00:14:05 UTC
back to top