Skip to main content

Browse the archive

https://github.com/cilium/cilium
12 July 2024, 03:14:21 UTC
sort by:
Revision Author Date Message Commit Date
1de682e Ian Vernon 06 June 2019, 18:13:39 UTC Prepare for v1.5.3 Signed-off by: Ian Vernon <ian@cilium.io> 06 June 2019, 19:44:00 UTC
911eb3f Ian Vernon 04 June 2019, 20:50:35 UTC test: do not spawn goroutines to wait for canceled context in `RunCommandContext` [ upstream commit cf14b6a19283bd994f3dc6cd429c29457c40adef ] It was observed while running `gops stack` for the Ginkgo test suite locally, that we were leaking goroutines that were getting stuck while waiting for SSH sessions to finish. We accrued over 1000 of these per K8s CI run: ``` 16 unique stack traces 960 occurences. Sample stack trace: github.com/cilium/cilium/vendor/golang.org/x/crypto/ssh.(*Session).Wait(0xc00098e000, 0x230f79c, 0x1a) /Users/ianvernon/go/src/github.com/cilium/cilium/vendor/golang.org/x/crypto/ssh/session.go:403 +0x57 github.com/cilium/cilium/test/helpers.(*SSHClient).RunCommandContext.func1(0x2502600, 0xc000542280, 0xc00098e000, 0xc0001aa780, 0xc000268220) /Users/ianvernon/go/src/github.com/cilium/cilium/test/helpers/ssh_command.go:262 +0x1cc created by github.com/cilium/cilium/test/helpers.(*SSHClient).RunCommandContext /Users/ianvernon/go/src/github.com/cilium/cilium/test/helpers/ssh_command.go:253 +0x13c ``` This example shows that there were over 960 goroutines stuck on `session.Wait()`. Whenever we run a command via SSH, we call `runCommand`. When we call `runCommand`, it calls `session.Run`, which calls `session.Start()` and `session.Wait()`. I observed that that calling `Wait()` on a session which already has had `Run` invoked will never return, even if we try to call `session.Close()` before invoking `session.Wait()`. This indicates that our logic for trying to kill the session if the context which is provided to `RunCommandContext` is canceled is flawed, as waiting for the session to finish before closing it will block infinitely. I enabled debug mode for the SSH library we use (`golang.org/x/crypto/ssh`), and I see that the session receives an EOF message *before* we even try to close the session: ``` >------- session started, and session.Run() invoked 2019/06/05 08:16:59 send global(2): ssh.channelOpenMsg{ChanType:"session", PeersID:0x2, PeersWindow:0x200000, MaxPacketSize:0x8000, TypeSpecificData:[]uint8(nil)} 2019/06/05 08:16:59 decoding(2): 91 &ssh.channelOpenConfirmMsg{PeersID:0x2, MyID:0x0, MyWindow:0x0, MaxPacketSize:0x8000, TypeSpecificData:[]uint8{}} - 17 bytes 2019/06/05 08:16:59 send(2): ssh.channelRequestMsg{PeersID:0x0, Request:"exec", WantReply:true, RequestSpecificData:[]uint8{0x0, 0x0, 0x0, 0x6d, 0x6b, 0x75, 0x62, 0x65, 0x63, 0x74, 0x6c, 0x20, 0x65, 0x78, 0x65, 0x63, 0x20, 0x2d, 0x6e, 0x20, 0x6b, 0x75, 0x62, 0x65, 0x2d, 0x73, 0x79, 0x73, 0x74, 0x65, 0x6d, 0x20, 0x63, 0x69, 0x6c, 0x69, 0x75, 0x6d, 0x2d, 0x62, 0x63, 0x63, 0x6d, 0x34, 0x20, 0x2d, 0x2d, 0x20, 0x63, 0x69, 0x6c, 0x69, 0x75, 0x6d, 0x20, 0x73, 0x74, 0x61, 0x74, 0x75, 0x73, 0x20, 0x2d, 0x6f, 0x20, 0x6a, 0x73, 0x6f, 0x6e, 0x70, 0x61, 0x74, 0x68, 0x3d, 0x27, 0x7b, 0x2e, 0x63, 0x6c, 0x75, 0x73, 0x74, 0x65, 0x72, 0x2e, 0x6e, 0x6f, 0x64, 0x65, 0x73, 0x5b, 0x2a, 0x5d, 0x2e, 0x70, 0x72, 0x69, 0x6d, 0x61, 0x72, 0x79, 0x2d, 0x61, 0x64, 0x64, 0x72, 0x65, 0x73, 0x73, 0x2e, 0x2a, 0x7d, 0x27}} 2019/06/05 08:16:59 decoding(2): 93 &ssh.windowAdjustMsg{PeersID:0x2, AdditionalBytes:0x200000} - 9 bytes 2019/06/05 08:16:59 decoding(2): 99 &ssh.channelRequestSuccessMsg{PeersID:0x2} - 5 bytes >------- EOF sent on channel (not by us; we have not closed the sesion yet! 2019/06/05 08:16:59 send(2): ssh.channelEOFMsg{PeersID:0x0} 2019/06/05 08:16:59 decoding(2): data packet - 181 bytes 2019/06/05 08:16:59 send(2): ssh.windowAdjustMsg{PeersID:0x0, AdditionalBytes:0xac} 2019/06/05 08:16:59 decoding(2): 96 &ssh.channelEOFMsg{PeersID:0x2} - 5 bytes 2019/06/05 08:16:59 decoding(2): 98 &ssh.channelRequestMsg{PeersID:0x2, Request:"exit-status", WantReply:false, RequestSpecificData:[]uint8{0x0, 0x0, 0x0, 0x0}} - 25 bytes 2019/06/05 08:16:59 decoding(2): 97 &ssh.channelCloseMsg{PeersID:0x2} - 5 bytes 2019/06/05 08:16:59 send(2): ssh.channelCloseMsg{PeersID:0x0} >------- we try to close the session, and receive the following error failed to close session: EOF ``` It appears that we cannot close the session, since an EOF has already been sent for it. I am not exactly sure where this comes from. I've posted an issue / question in the GitHub repository for golang: https://github.com/golang/go/issues/32453 . Our attempts to send signals (SIGHUP and SIGINT) are met by this same EOF error as well; there is no point on waiting for the session to finish in this case, so just try to close it and move on, and not leak goroutines that will be stuck forever. When running `gops` now against the Gingko test suite, we no longer accrue a ton of these goroutines blocked on `session.Wait()` - the biggest # of occcurrences for "similar" goroutines is at most 2 in a stack trace captured below, for example: ``` $ ../contrib/scripts/consolidate_go_stacktrace.py stack9.out | head -n 15 14 unique stack traces 2 occurences. Sample stack trace: internal/poll.runtime_pollWait(0x3f14d50, 0x72, 0xc000a6cad0) /usr/local/go/src/runtime/netpoll.go:173 +0x66 internal/poll.(*pollDesc).wait(0xc00043c218, 0x72, 0xffffffffffffff00, 0x24e7ac0, 0x3225738) /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0x9a ... ``` Signed-off by: Ian Vernon <ian@cilium.io> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> 06 June 2019, 07:41:16 UTC
a049140 Thomas Graf 05 June 2019, 11:41:03 UTC node/store: Do not delete node key in kvstore on node registration failure [ upstream commit e8cb4205e5027cec21dc726691f7e5347a34438f ] When registration fails, do not attempt to remove an eventually existing key in the kvstore. This may in fact remove a key that is already present from a previous agent run. If the agent fails to update the key it will eventually expire via the lease. Removing it on the first failed attempt to udpate will trigger an unwanted delete notification. Signed-off-by: Thomas Graf <thomas@cilium.io> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> 06 June 2019, 07:41:16 UTC
15fdc43 Thomas Graf 05 June 2019, 10:28:02 UTC kvstore/store: Do not remove local key on sync failure [ upstream commit 960da244c42d1d9d806a9c76bbfc2620ccd30bb4 ] The only failure scenario is if the key could not be updated/inserted. Removing the local key may remove an already existing key in the kvstore. To keep the behavior of UpdateLocalKeySync() consistent in all cases, the store is locked and the local key is only updated after successful insertion into the kvstore. Removing the key erroneously can lead to a delete notification being triggered for the own key. Signed-off-by: Thomas Graf <thomas@cilium.io> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> 06 June 2019, 07:41:16 UTC
cd1cbc7 Thomas Graf 05 June 2019, 00:39:30 UTC node: Delay handling of node delete events received via kvstore [ upstream commit 3137ad5eee6c617e1acc8d5b9fcae7c899a77c97 ] As for other kvstore keys, each node will protect its own node key and re-create it as needed on loss of kvstore state. However, when immediately acting upon receiving a node delete event via the kvstore, the removal of routes and similar can cause etcd itself to become unreachable when etcd is hosted as a pod. A subsequent readdition of the key can then not be received and the node is lost forever. There are several more complex long-term options including relying on k8s node state. For a short-term resolution, introduce a 30 seconds delay when handling node delete events from the kvstore and require the node to not be re-created in that timeframe. This workaround works regardless of how node discovery is performed. A potential side effect of this is that when a node re-appears with different IP addressing. In that scenario, the k8s node delete event will forcefully remove all state. Signed-off-by: Thomas Graf <thomas@cilium.io> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> 06 June 2019, 07:41:16 UTC
5aeccef André Martins 28 May 2019, 07:52:58 UTC test/provision: bump k8s 1.12 to 1.12.9 [ upstream commit fcb7a5a3171e8bf5d11bcf868b7e3f6c2bbf6b76 ] Signed-off-by: André Martins <andre@cilium.io> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> 06 June 2019, 07:41:16 UTC
e282209 André Martins 03 June 2019, 14:01:35 UTC pkg/kvstore: do not always UpdateIfDifferent with and without lease [ upstream commit 361c99dd1082d63384748b44e78098281a9c5808 ] We should only update the value of the given key if the lease is different only if we are trying to update it lease as well. Fixes: 0259ad0568fd ("pkg/kvstore: perform update if value or lease are different") Signed-off-by: André Martins <andre@cilium.io> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> 06 June 2019, 07:41:16 UTC
5941dd8 Maciej Kwiek 14 May 2019, 09:08:22 UTC Don't overwrite minRequired in WaitforNPods [ upstream commit 1105fb428d2f38c3ce65215a519d3d9738d31807 ] If `minRequired` was set to 0, it was overwritten on the first pass of `body` function (since it's in this function's closure) and if some pods were being deleted during the test startup time, it could cause `minRequired` to be higher than possible with test pods number. Signed-off-by: Maciej Kwiek <maciej@covalent.io> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> 06 June 2019, 07:41:16 UTC
cab47ed Joe Stringer 11 April 2019, 18:55:43 UTC daemon: Don't log endpoint restore if IP alloc fails [ upstream commit f063e7e4b22a075f49c0f19477942cfbb9902dbe ] In the case where the IP for a restoring endpoint cannot be re-allocated, log this and error out earlier, before reporting that the endpoint is being restored. Signed-off-by: Joe Stringer <joe@cilium.io> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> 06 June 2019, 07:41:16 UTC
1fe3479 Joe Stringer 11 April 2019, 18:53:41 UTC daemon: Refactor individual endpoint restore [ upstream commit 6f01080db685816bf0d69f0f92623b86acd29378 ] Factor out aspects of restoring an endpoint which require directly interacting with other daemon-managed components. Signed-off-by: Joe Stringer <joe@cilium.io> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> 06 June 2019, 07:41:16 UTC
a7688a0 Ian Vernon 03 June 2019, 21:42:14 UTC test: provide context which will be cancled to `CiliumExecContext` [ upstream commit 42a899c30348dd77d5314d1478b45cb0fc94049f ] Providing just `context.Background()` means that the context provided to `CiliumExecContext` is never canceled. Provide a context that uses the existing 4 minute timeout, and cancel the context after the call to `CiliumExecContext` is finished so that goroutines don't leak. Signed-off by: Ian Vernon <ian@cilium.io> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> 06 June 2019, 07:41:16 UTC
3907fc9 André Martins 04 June 2019, 12:34:06 UTC Jenkinsfile: backport all Jenkinsfile from master We have made lots of changes in jenkins that were not and should have been backported into this branch. For this reason this commit is a copy of all Jenkinsfiles present in master with the k8s versions being tested changed to be v1.5 specific. Signed-off-by: André Martins <andre@cilium.io> 05 June 2019, 15:32:54 UTC
2686534 Thomas Graf 04 June 2019, 11:10:06 UTC doc: Document regressions in 1.5.0 and 1.5.1 Signed-off-by: Thomas Graf <thomas@cilium.io> 04 June 2019, 13:59:46 UTC
fce6b8f Ian Vernon 30 May 2019, 00:35:59 UTC Prepare for release v1.5.2 Signed-off by: Ian Vernon <ian@cilium.io> Signed-off-by: Thomas Graf <thomas@cilium.io> 04 June 2019, 13:59:46 UTC
e39374b Thomas Graf 04 June 2019, 07:14:04 UTC test: Disable unstable K8sDatapathConfig Encapsulation Check connectivity with transparent encryption and VXLAN encapsulation Signed-off-by: Thomas Graf <thomas@cilium.io> 04 June 2019, 10:27:50 UTC
9e010c5 Maciej Kwiek 24 May 2019, 11:03:01 UTC Add kvstore quorum check to Cilium precheck [ upstream commit 10a0e337f0e845d5fa772db8a9f7d162d6d05500 ] Signed-off-by: Maciej Kwiek <maciej@covalent.io> Signed-off-by: Thomas Graf <thomas@cilium.io> 04 June 2019, 10:27:50 UTC
9f756cb André Martins 17 May 2019, 17:15:40 UTC pkg/kvstore: acquire a random initlock [ upstream commit 394c93ec0fa69c9c6731944fe5adcb27d75265c6 ] If all cilium agents are trying to acquire the same lock they can return a wrong perception that etcd does not have quorum but in reality it can be the fact that one client got killed while it was holding the lock and its lease hasn't expire yet. Making all agents to acquire a random path will make it more unlikely for 2 agents to acquire the same path. Fixes: 680b2ee4b96c ("kvstore: Wait for kvstore to reach quorum") Signed-off-by: André Martins <andre@cilium.io> Signed-off-by: Thomas Graf <thomas@cilium.io> 04 June 2019, 10:27:50 UTC
46777c9 Thomas Graf 07 May 2019, 10:09:46 UTC kvstore: Wait for kvstore to reach quorum [ upstream commit 680b2ee4b96c335c8873301b869100a0614fc00c ] So far, the channel returned by the Connected() function of the kvstore client was closed when the kvstore could be connected to. This did not guarantee that the kvstore actually had quorum and operations could still block and timeout afterwards. Wait for a distributed lock to be acquired to identify the moment quorum has been reached. Also indicate the quorum status in the kvstore status of `cilium status` for etcd. Signed-off-by: Thomas Graf <thomas@cilium.io> 04 June 2019, 10:27:50 UTC
fe5e8f5 Thomas Graf 03 June 2019, 09:46:05 UTC ipcache: Fix automatic recovery of deleted ipcache entries [ upstream commit f7064bf5242e4e85b2b512c2e0cb8cbfceb22379 ] The existing delete recovery logic was relying on upsert() to be called on kvReferenceCounter. This was never done though so when the delete handler verified for local existance, no entry was ever found. Remove the obsolete upsert() handler of kvReferenceCounter as no reference counting is needed and insert an entry into marshaledIPIDPairs from UpsertIPToKVStore() directly. Fixes: b76bb7f504a ("ipcache: Fix refcounting of endpoint IPCache entries") Signed-off-by: Thomas Graf <thomas@cilium.io> 04 June 2019, 10:27:50 UTC
0b10a34 Daniel Borkmann 22 May 2019, 21:23:02 UTC tests, k8s: add monitor dump helper for debugging [ upstream commit c7f32d6b12a69c298ecaded5a141a6f01561dc49 ] So far MonitorStart() command was only a helper for SSHMeta type. As a result, we couldn't use it for the Kubernetes tests under k8sT/. Thus, add support for MonitorStart() for Kubectl type along with an example usage. Having the helper avoids that others need to reimplement it. I added the option that one can specify a file name where the contents should be saved for the dump in case one wants to trace multiple exec calls to curl and whatnot without appending to the same log. Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Signed-off-by: Thomas Graf <thomas@cilium.io> 04 June 2019, 10:27:50 UTC
2d5baaa Daniel Borkmann 17 May 2019, 14:45:15 UTC bugtool: add raw dumps of all lb and lb-related maps [ upstream commit e9d0989fa270256576934ad6b0e13629a0bef7ab ] It turned out to be tremendously helpful to have these at hand when debugging some of the recent issues from lb handling. For newer LLVMs we can do pretty-printing for them in future. Also add a few other misc commands from iproute2 that could be helpful in future. Fixes: #8081 Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Signed-off-by: Martynas Pumputis <m@lambda.lt> Signed-off-by: Thomas Graf <thomas@cilium.io> 04 June 2019, 10:27:50 UTC
8a1c4ba André Martins 29 May 2019, 11:50:21 UTC pkg/labels: ignore all labels that match the regex "annotation.*" [ upstream commit 53c2e816bceb415f5a18bb15b578565403a3b834 ] This will help to decrease the cardinality of labels used to create a security identity since pod annotations can be used for enumerous reasons and aren't used for network policy enforcement. Signed-off-by: André Martins <andre@cilium.io> Signed-off-by: Thomas Graf <thomas@cilium.io> 04 June 2019, 10:27:50 UTC
f6c8e38 Martynas Pumputis 29 May 2019, 14:33:58 UTC docs: Add note about keeping enable-legacy-services [ upstream commit 86ddc72e61534e940938f49403a9c5e97bf8ebe4 ] When `--enable-legacy-services` is set to true, we consider the legacy maps as a source of the truth. Thus, if a user disables and afterwards enables the option, the svc-v2 maps will be lost. Signed-off-by: Martynas Pumputis <m@lambda.lt> Signed-off-by: Thomas Graf <thomas@cilium.io> 04 June 2019, 10:27:50 UTC
2d86de3 Martynas Pumputis 29 May 2019, 14:23:31 UTC docs: Add note about running preflight-with-rm-svc-v2.yaml [ upstream commit a115fceb992ed5fa1d084650c9acbb151594f335 ] This is needed for users who want to do the v1.5 -> v1.4 -> v1.5 upgrade. Main reason for requiring this is that in v1.4 we do not maintain the svc-v2 BPF maps, so they become stale. When doing the service restoration from legacy to v2, we do not compare endpoint entries of the maps (neither does `syncLBMapsWithK8s`), so it's better to remove the v2 maps to force full restoration from the legacy maps. Signed-off-by: Martynas Pumputis <m@lambda.lt> Signed-off-by: Thomas Graf <thomas@cilium.io> 04 June 2019, 10:27:50 UTC
a4c2e6a Martynas Pumputis 29 May 2019, 14:06:24 UTC examples: Add preflight DaemonSet for svc-v2 removal [ upstream commit 04d11b5e57fb2ffd2ddeb7dcdfc1ad2dbaa2f881 ] This DaemonSet is going to be used by users who upgraded to Cilium v1.5, then downgraded to <v1.5 and want to upgrade to v1.5 again. Signed-off-by: Martynas Pumputis <m@lambda.lt> Signed-off-by: Thomas Graf <thomas@cilium.io> 04 June 2019, 10:27:50 UTC
921b816 Thomas Graf 30 May 2019, 13:18:01 UTC operator: Fix health check API [ upstream commit 064c57b9b18eaa420e061255b79161ad31f62d6d ] * Error out for real when the health check API cannot be served. The most common problem is that the port is still in use by a previous operator instance running on the same node. * Do not fail the health check if the kvstore is unavailable if the kvstore is not needed. Signed-off-by: Thomas Graf <thomas@cilium.io> 04 June 2019, 10:27:50 UTC
fdcba92 Thomas Graf 30 May 2019, 16:12:17 UTC doc: Add EKS node-init DaemonSet to mount BPF filesystem [ upstream commit 7b47c85f30b1747aa17e1f8f7ca0b25901c55361 ] Fixes: #8163 Signed-off-by: Thomas Graf <thomas@cilium.io> 04 June 2019, 10:27:50 UTC
c0d3292 André Martins 03 June 2019, 08:17:32 UTC pkg/kvstore: perform update if value or lease are different [ upstream commit 0259ad0568fdda09895ec3c89715550bc6dddecd ] When UpdateIfDifferent was introduced, the lease information of the key was not taken into account. When a cilium-agent was restarted, its lease would also expire as well, making its keys being deleted after the lease TTL (roughly 15 minutes) expired. As UpdateIfDifferent was not build accounting for the fact that leases would expire, those leased keys would never be updated with the new lease as the only attribute being compared if it was equal or not was the value of the key and it should have been the value *and* the lease information of that same key. Fixes: ccd02fb04898 ("kvstore: Add UpdateIfDifferent()") Signed-off-by: André Martins <andre@cilium.io> Signed-off-by: Thomas Graf <thomas@cilium.io> 04 June 2019, 10:27:50 UTC
9e37a66 André Martins 30 May 2019, 12:06:27 UTC kvstore/allocator: do not immediately delete master keys if unused [ upstream commit 499993ab13a7b68b5a7ea4ee991523611694f247 ] To avoid the master keys GC being a little bit agressive in the deletion of the master keys, their GC will only be performed if their revision number hasn't change in 2 consequent runs of the GC function. Signed-off-by: André Martins <andre@cilium.io> Signed-off-by: Ian Vernon <ian@cilium.io> 03 June 2019, 14:14:46 UTC
27a55bd André Martins 30 May 2019, 12:05:41 UTC pkg/kvstore: store Modified Revision number KeyValuePairs map [ upstream commit 1e4372814948dfdba67d0751a3268db2dd16b9bd ] Signed-off-by: André Martins <andre@cilium.io> Signed-off-by: Ian Vernon <ian@cilium.io> 03 June 2019, 14:14:46 UTC
dc06d0b André Martins 30 May 2019, 10:04:19 UTC kvstore/allocator: do not re-allocate localKeys [ upstream commit 33a4ba0e1ddcca71705aafcfd97733545fa98f41 ] If the master key does not exist in the KVStore does not necessary means the agent does not hold a reference for that key. For this reason we should perform a lookup of this master key in the local key references and re-use that ID to update master key in the KVStore as well as use that ID for the value of the slave key. Signed-off-by: André Martins <andre@cilium.io> Signed-off-by: Ian Vernon <ian@cilium.io> 03 June 2019, 14:14:46 UTC
16416aa André Martins 30 May 2019, 10:01:29 UTC kvstore/allocator: move invalidKey to cache.go [ upstream commit dbbfa42ee147eb3ce15eec4b1de3cd9f64570d0d ] This function is only used in cache.go so it does not make sense for it to live in allocator.go Signed-off-by: André Martins <andre@cilium.io> Signed-off-by: Ian Vernon <ian@cilium.io> 03 June 2019, 14:14:46 UTC
8b5206e André Martins 30 May 2019, 10:00:21 UTC kvstore/allocator: add lookupKey method [ upstream commit 2f77f7bbe329e271ae781e7da50bc1c798b41771 ] Signed-off-by: André Martins <andre@cilium.io> Signed-off-by: Ian Vernon <ian@cilium.io> 03 June 2019, 14:14:46 UTC
67ca1d1 Thomas Graf 30 May 2019, 09:55:23 UTC allocator: Provide additional info message on key allocation and deletion [ upstream commit 9a5669d9b00296eb5c96701457bc0488e9055857 ] Tracking down identity leaks has been difficult due to lack of info level messages. These events are rare and tied to endpoint lifecycle so providing additional info level messages is safe and will help correlate identity management across multiple nodes. Signed-off-by: Thomas Graf <thomas@cilium.io> Signed-off-by: Ian Vernon <ian@cilium.io> 03 June 2019, 14:14:46 UTC
53ea56a Thomas Graf 30 May 2019, 14:48:47 UTC allocator: Fix garbage collector to compare prefix [ upstream commit ed2206f440c8179788e3ddb2ebee9a3fd7106651 ] When a master key is unused but a slave key exists which overlaps with the master key prefix, the master key is never released. Example: key1 := label1;label2; key2 := label1;label2;label3; Assuming that key1 is unused but key2 is used, the ListPrefix(key1) will return all users of key2 and prevent key1 from being released. Signed-off-by: Thomas Graf <thomas@cilium.io> Signed-off-by: Ian Vernon <ian@cilium.io> 03 June 2019, 14:14:46 UTC
1468105 Thomas Graf 30 May 2019, 14:44:24 UTC allocator: Make GetNoCache() deterministic [ upstream commit 65d5549c08b4cd4060cc265ae98c18119588d775 ] ListPrefix() will return all keys matching the prefix, the prefix can cover multiple different keys, example: key1 := label1;label2; key2 := label1;label2;label3; In order to retrieve the correct key, the position of the last '/' is signficant, e.g. prefix := cilium/state/identities/v1/value/label;foo; key1 := cilium/state/identities/v1/value/label;foo;/172.0.124.60 key2 := cilium/state/identities/v1/value/label;foo;bar;/172.0.124.60 Only key1 should match Signed-off-by: Thomas Graf <thomas@cilium.io> Signed-off-by: Ian Vernon <ian@cilium.io> 03 June 2019, 14:14:46 UTC
ac1023d André Martins 29 May 2019, 16:18:36 UTC kvstore/allocator: protect concurrent access of slave keys [ upstream commit 9c6f19a759652fd84c1ff8f63c6c14fc05a751dc ] If a key is released from localKeys and marked as last used, it will be released from the KVStore. Unfortunately, this has a concurrent issue if at the same time this localKey allocated which means cilium-agent will update its value into the KVStore. If this update happens before the delete call, the key is forever deleted from the KVStore. Signed-off-by: André Martins <andre@cilium.io> Signed-off-by: Ian Vernon <ian@cilium.io> 03 June 2019, 14:14:46 UTC
9a2724a André Martins 29 May 2019, 14:34:25 UTC kvstore/allocator: release ID from idpool on error [ upstream commit e4cff1081ea9f0283aaba39f1e435efe5f100fce ] If the agent is unable to create the slave key in the kvstore, it will leak ids from the local idpool. In case of error we should release those IDs as well to avoid this leakage. Signed-off-by: André Martins <andre@cilium.io> Signed-off-by: Ian Vernon <ian@cilium.io> 03 June 2019, 14:14:46 UTC
5c62bc3 André Martins 29 May 2019, 14:31:48 UTC kvstore/allocator: do not re-get slave key on allocation [ upstream commit c11c9cc0e4300ec6185298403a5a8bc834b326f0 ] As the slave key is protected by the kvstore global lock it is not really necessary to do another Get of the slave key as the key will not be created by any other agent due the lock is still being held. Signed-off-by: André Martins <andre@cilium.io> Signed-off-by: Ian Vernon <ian@cilium.io> 03 June 2019, 14:14:46 UTC
5bf3f66 André Martins 29 May 2019, 14:27:23 UTC pkg/kvstore: Run GetPrefix with limit of 1 [ upstream commit 829e28c34791a8e91308071a646a86a0a8596111 ] As the function documentation states GetPrefix only returns the first key that matches the given prefix there's no reason to perform a Get WithPrefix and not limiting the number of results to 1. This will improve the speed of the Get operation as it doesn't need to return all keys that match the given prefix but a single one is enough. Signed-off-by: André Martins <andre@cilium.io> Signed-off-by: Ian Vernon <ian@cilium.io> 03 June 2019, 14:14:46 UTC
523607a Thomas Graf 30 May 2019, 09:27:40 UTC allocator: Verify locally allocated key [ upstream commit 47d49e35c178a4a70d983b0198159592431d5f8f ] The logic to re-create slave keys requires keys to be verified, i.e. successfully created in the kvstore once. So far, only keys for which a master key already existed have been marked as verified. This prevented the node that originally created a slave key to ever re-create the key on accidental deletion. Mark all keys that have been successfully created in the kvstore as verified. Reported-by: André Martins <andre@cilium.io> Signed-off-by: Thomas Graf <thomas@cilium.io> Signed-off-by: Ian Vernon <ian@cilium.io> 03 June 2019, 14:14:46 UTC
04dda38 Jarno Rajahalme 29 May 2019, 09:01:38 UTC envoy: Prevent resending NACKed resources also when there are no ACK observers. [ upstream commit ec3910f1f33e866885a08aede919f270b2912975 ] Adjust the version to prevent resending a NACKed resource also when there are no ACK observers. Currently this only appliest the NPHDS as all other resource types do have ACK observers. This fixes an issue where we would continuously keep resending a failing IP/ID mapping resource that Envoy already has NACKed. There is no indication this ever been triggered. Signed-off-by: Jarno Rajahalme <jarno@covalent.io> Signed-off-by: André Martins <andre@cilium.io> 29 May 2019, 16:15:21 UTC
6dbc319 Ray Bejjani 28 May 2019, 10:08:40 UTC endpoint: Guard against deleted endpoints in regenerate [ upstream commit 07ba2dcffe8dbef8d085d960d7ef4da033c3fb18 ] We may enqueue endpoints for regenerations that are not managed by endpoint manager when it handles the regeneration request. This change trivially guards against this to avoid crashing, at the risk of suppressing a needed regenerate. This may occur when an endpoint is still being created and has not been added to the endpoint manager, as opposed to being deleted (i.e. it is eventually incorrect because it missed the regenerate). Signed-off-by: Ray Bejjani <ray@covalent.io> Signed-off-by: André Martins <andre@cilium.io> 29 May 2019, 16:15:21 UTC
bb5442d Martynas Pumputis 29 May 2019, 07:23:45 UTC service: Reduce backend ID allocation space [ upstream commit 1c9cf6bd425952eef1983fe02b8e83f59013e94f ] Currently, we store service backend ID as uint16 in the BPF maps, which is going to be changed to uint32 in upcoming releases. Meanwhile, the backend ID allocator allocates backend IDs as uint32 and then casts the allocated ID to uint16. To avoid having the same backend IDs for two different backends in the BPF maps until the proper fix will land, make sure that the allocator won't allocate an ID > max(uint16). Signed-off-by: Martynas Pumputis <m@lambda.lt> Signed-off-by: André Martins <andre@cilium.io> 29 May 2019, 16:15:21 UTC
431bd82 Daniel Borkmann 28 May 2019, 12:27:31 UTC cilium: fix up source address selection for cluster ip [ upstream commit 2bdca067a90833a2d03d0d8a986f821a469121e0 ] The CI test 'K8sServicesTest Checks ClusterIP Connectivity Checks service on same node' failed recently due to a buggy L7 policy removal from prior test; fixed via d8ff01841646 ("test: fix incorrect deletion statement for policy"). This incorrect deletion uncovered another bug however triggered by curl failing to connect to the endpoint backing the service: # curl --path-as-is -s -D /dev/stderr --fail --connect-timeout 5 --max-time 8 http://10.110.178.169/ -w "time-> DNS: '%{time_namelookup}(%{remote_ip})', Connect: '%{time_connect}',Transfer '%{time_starttransfer}', total '%{time_total}'" time-> DNS: '0.000038()', Connect: '0.000000',Transfer '0.000000', total '3.203689' (Note, DNS has nothing to do with the failure.) # docker exec -ti 6d5465170bf4 bash # cilium bpf lb list SERVICE ADDRESS BACKEND ADDRESS 10.110.178.169:80 0.0.0.0:0 (6) <--- service from above curl 10.10.0.183:80 (6) 10.10.0.62:80 (6) 10.101.55.250:2379 10.10.1.67:2379 (3) 0.0.0.0:0 (3) 10.10.1.162:2379 (3) 10.10.0.24:2379 (3) 10.96.0.10:53 0.0.0.0:0 (2) 10.10.1.185:53 (2) 10.96.0.1:443 192.168.36.11:6443 (1) 0.0.0.0:0 (1) # exit # ip route get 10.110.178.169 10.110.178.169 via 10.0.2.2 dev enp0s3 src 10.0.2.15 uid 0 cache # ip route get 10.10.0.183 10.10.0.183 via 10.10.0.160 dev cilium_host src 10.10.0.160 uid 0 cache # ip route get 10.10.0.62 10.10.0.62 via 10.10.0.160 dev cilium_host src 10.10.0.160 uid 0 cache # ip r default via 10.0.2.2 dev enp0s3 proto dhcp src 10.0.2.15 metric 100 10.0.2.0/24 dev enp0s3 proto kernel scope link src 10.0.2.15 10.0.2.2 dev enp0s3 proto dhcp scope link src 10.0.2.15 metric 100 10.10.0.0/24 via 10.10.0.160 dev cilium_host src 10.10.0.160 10.10.0.160 dev cilium_host scope link 10.10.1.0/24 via 10.10.0.160 dev cilium_host src 10.10.0.160 172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1 172.28.128.0/24 dev enp0s9 proto kernel scope link src 172.28.128.3 192.168.9.0/24 dev br-e6fcdf93b490 proto kernel scope link src 192.168.9.1 192.168.36.0/24 dev enp0s8 proto kernel scope link src 192.168.36.11 The curl is executed out of hostns and the service IP is xlated via Kubernetes cluster IP which hooks into iptables nat table in post-routing. What could be seen in cilium monitor was that at the cilium_host device (bpf_netdev on egress), the following mapping was found in the ipcache ... Successfully mapped daddr=10.0.2.15 to identity=2 ... and the packet dropped later on due to policy. The id mapping of 2 (world) turns out to be the 0/0 catch-all: # cilium bpf ipcache get 10.0.2.15 10.0.2.15 maps to identity 2 0 0.0.0.0 Aside form the mapping, the other part which is unexpected is the source IP selection. Assumption would have been that given the backend is local, we would have used 10.10.0.160 based on this route: 10.10.0.0/24 via 10.10.0.160 dev cilium_host src 10.10.0.160 As can be seen from above `ip route get` however the source IP selection is based upon the service IP. In retrospect this makes sense because the Kubernetes iptables backend selection is at post-routing, so nothing in terms of source IP will change at that point. Thus, add a SNAT rule that comes after the KUBE-POSTROUTING chain such that we can fix the source address to cilium_host's address instead where it will later also find host identity in the ipcache. Fixes: #7902 Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Signed-off-by: André Martins <andre@cilium.io> 29 May 2019, 16:15:21 UTC
28ddd96 ifeanyi 28 April 2019, 20:34:55 UTC CI: Log at INFO and above for all unit tests [ upstream commit 538f350134fafbe942f29ff0e347204457d1070a ] By default all unit test suites already log at INFO level. However, there are currently a few test suites that explicitly enable DEBUG log levels. This commit turns DEBUG back off for these tests. Fixes #7867 Signed-off-by: ifeanyi <ify1992@yahoo.com> Signed-off-by: Martynas Pumputis <m@lambda.lt> 29 May 2019, 13:11:11 UTC
75305b0 Martynas Pumputis 27 May 2019, 18:37:51 UTC bpf: Fix dump parsers of encrypt and sockmap maps This commit fixes the compilation errors of the v1.5 backport (2019-05-27). Signed-off-by: Martynas Pumputis <m@lambda.lt> 29 May 2019, 13:11:11 UTC
f3d3974 André Martins 17 May 2019, 14:47:05 UTC pkg/maps: use pointer in receivers for GetKeyPtr and GetValuePtr [ upstream commit 28e7a82956ed424cb9c0902952def62bb2d69ebf ] Not using a pointer in the receivers causes Get{Key,Value}Ptr to return a pointer of the copy of the receiver structure being called. This can have consequences if we use Get{Key,Value}Ptr to store data and expect the data to still be present in the original structure. Signed-off-by: André Martins <andre@cilium.io> Signed-off-by: Martynas Pumputis <m@lambda.lt> 29 May 2019, 13:11:11 UTC
24ec528 Ian Vernon 25 May 2019, 03:10:44 UTC test: fix incorrect deletion statement for policy [ upstream commit d8ff018416468c0203a32abb8888269b1c11219c ] The deletion of policy should be done via the `Delete` function call instead of the `Namespace` delete function call; fix an incorrect usage of this in the Policies.go file accordingly. Signed-off by: Ian Vernon <ian@cilium.io> Signed-off-by: Martynas Pumputis <m@lambda.lt> 29 May 2019, 13:11:11 UTC
72f7991 Jarno Rajahalme 22 May 2019, 04:46:35 UTC proxylib: Fix egress enforcement [ upstream commit c74053bf3cd464f224b0194a91cb677efa4eb18a ] Pass the destination ID of egress connections for policy enforcement. Signed-off-by: Jarno Rajahalme <jarno@covalent.io> Signed-off-by: Martynas Pumputis <m@lambda.lt> 29 May 2019, 13:11:11 UTC
a51c884 Maciej Kwiek 22 May 2019, 12:07:10 UTC Recover from ginkgo fail in WithTimeout helper [ upstream commit 761854d23e2c427a64cec7166afc8406a242e13a ] If WithTimeout is called with func which is able to panic, ginkgo is not able to get information about this panic without GinkgoRecover. Signed-off-by: Maciej Kwiek <maciej@covalent.io> Signed-off-by: Martynas Pumputis <m@lambda.lt> 29 May 2019, 13:11:11 UTC
91d000e André Martins 20 May 2019, 10:27:50 UTC docs: move well known identities to the concepts section [ upstream commit 97e9825d8833d598e306663c82b44b55b1182563 ] Signed-off-by: André Martins <andre@cilium.io> Signed-off-by: Martynas Pumputis <m@lambda.lt> 29 May 2019, 13:11:11 UTC
e454885 André Martins 20 May 2019, 10:25:40 UTC docs: update well-known-identities documentation [ upstream commit 69c5b29743bacd0b0d3fd08a96c318d64b9becc8 ] Signed-off-by: André Martins <andre@cilium.io> Signed-off-by: Martynas Pumputis <m@lambda.lt> 29 May 2019, 13:11:11 UTC
fe82fee André Martins 17 May 2019, 18:12:50 UTC add support for k8s 1.14.2 [ upstream commit 8027005aa855e6c7c86c9eaa26eaa7950f9e3d8a ] Signed-off-by: André Martins <andre@cilium.io> Signed-off-by: Martynas Pumputis <m@lambda.lt> 29 May 2019, 13:11:11 UTC
c53d1a9 André Martins 16 May 2019, 09:43:13 UTC test: add v1.15.0-beta.0 to the CI [ upstream commit d84a7965650bec0291f552b6346588a5c4ff3b4d ] Signed-off-by: André Martins <andre@cilium.io> Signed-off-by: Martynas Pumputis <m@lambda.lt> 29 May 2019, 13:11:11 UTC
d76fa34 Joe Stringer 16 May 2019, 21:35:11 UTC cni: Fix incorrect logging in failure case [ upstream commit 44d61631f5266794a1a0b489edea0f6e1b1d5e21 ] This failure case should only trigger logging when the deletion of the veth fails during an endpoint add request failure. Fixes: b859b6d83c7d ("cni: Fix unexpected end of JSON input on errors") Signed-off-by: Joe Stringer <joe@cilium.io> Signed-off-by: Martynas Pumputis <m@lambda.lt> 29 May 2019, 13:11:11 UTC
cbae038 Joe Stringer 16 May 2019, 19:54:32 UTC daemon: Make policymap size configurable [ upstream commit ec07900a49bf96b2f47b6ecbc0b4c22c7ca4de46 ] Signed-off-by: Joe Stringer <joe@cilium.io> Signed-off-by: Thomas Graf <thomas@cilium.io> 28 May 2019, 10:58:31 UTC
705f6e5 Maciej Kwiek 20 May 2019, 12:45:31 UTC Add jenkins stage for loading vagrant boxes [ upstream commit d083f789c96450cc91762f767cfbb7d4c5ada34a ] Signed-off-by: Maciej Kwiek <maciej@covalent.io> 22 May 2019, 15:31:59 UTC
975f5bd Thomas Graf 17 May 2019, 00:31:49 UTC bpf: Remove several debug messages Remove several unneeded debug messages to reduce the number of instructions with debugging enabled. This serves as a workaround until instruction size can be reduced otherwise. Signed-off-by: Thomas Graf <thomas@cilium.io> 17 May 2019, 13:07:30 UTC
e0ead51 Thomas Graf 16 May 2019, 23:25:19 UTC Revert "pkg/bpf: add DeepCopyMapKey and DeepCopyMapValue" This reverts commit 043f5dd9049faded22c8dca1fda37c9e0a8aa2d3. Signed-off-by: Thomas Graf <thomas@cilium.io> 17 May 2019, 13:07:30 UTC
2a2adfa Thomas Graf 16 May 2019, 23:24:56 UTC Revert "pkg/{bpf,datapath,maps}: use same MapKey and MapValue in map iterations" This reverts commit 389cd5a788716912137ee2ffbd437b9c10f9b3e7. Signed-off-by: Thomas Graf <thomas@cilium.io> 17 May 2019, 13:07:30 UTC
3a291ca Thomas Graf 16 May 2019, 23:24:37 UTC Revert "pkg/bpf: add newer LookupElement, GetNextKey and UpdateElement functions" This reverts commit b2f4fe53ed19bb3c95515627d1360d5df40c1f5e. Signed-off-by: Thomas Graf <thomas@cilium.io> 17 May 2019, 13:07:30 UTC
8bfdcc3 Thomas Graf 16 May 2019, 23:24:08 UTC Revert "pkg/bpf: use own binary which does not require to create buffers" This reverts commit 3995a84a0516091d81538803a966b7948b06afad. Signed-off-by: Thomas Graf <thomas@cilium.io> 17 May 2019, 13:07:30 UTC
f959d1b Thomas Graf 16 May 2019, 23:23:13 UTC Revert "maps/ctmap: add ctmap benchmark" This reverts commit 67d1ffd22de790b9548cb8b89f07c70fdc216df5. Signed-off-by: Thomas Graf <tgraf@cilium.io> 17 May 2019, 13:07:30 UTC
bfacecc Daniel Borkmann 16 May 2019, 16:21:20 UTC bpf: force recreation of regular ct entry upon service collision [ upstream commit ce43de13e2e093cede0839d263e84f32b0c890ed ] If there was a service reuse in place, meaning, e.g. upon upgrade, the CT map was reused, but different services created for the same IP/port pair, then we will first detect it in the services lookup via lb{4,6}_local(). The ct_lookup{4,6}() from there will return a connection in CT_{ESTABLISHED, REPLY,RELATED} where we find that the rev-nat is mismatching via state->rev_nat_index != svc_v2->rev_nat_index. This forces to select a new slave of the actual (current) service. We then update state with the new backend_id and rev_nat_index, and propagate this info back into the CT_SERVICE entry, so it becomes persistent for the next service lookup. After that, we lookup the newly selected backend via lb{4,6}_lookup_backend() and fix up tuple address and skb with the new backend IP/port. Thus in the CT, the CT_SERVICE entry is updated/fixed for subsequent lookups. Back in bpf_lxc (e.g. handle_ipv4_from_lxc()), this ct_state info is stored in ct_state_new. After lb4_local() did its work, we do the CT lookup via ct_lookup4() for the tuple with the backend IP/port included. If it's a new connection (CT_NEW), then ct_create4() will propagate the info from ct_state_new into the ct entry/value (entry.rev_nat_index). If there is a case where there is a stale entry in the CT due to similar stale, prior used service IP, then we may find a CT_{ESTABLISHED,REPLY, RELATED case. In the latter two, we use the ct_state from the normal connection that has been looked up to then do the lb4_rev_nat() iff there was a rev-nat index in the ct_state. Similarly, for CT_ESTABLISHED, we'll do the lb4_rev_nat() through a differnt code path. CT_ESTABLISHED we'll get for outgoing connections from the endpoint itself, the CT_REPLY for packets coming back in. While we updated the services, the regular CT entries would still reuse the stale rev-nat index, which may be wrong. Thus, detect for outgoing CT_ESTABLISHED packets whether we have a mismatch in rev_nat_index, and if so, purge the whole entry from the CT table and force recreation of a new one. Following cases can be valid: the rev-nat index change as explained above, and the backend_id change for the case where in lb4_local() we hit the one where the backend was removed from underneath us and we were forced to select a completely new one, and thus rev-nat stays the same but backend_id changes. In the latter, this means, the tuple got updated in lb4_local() for the later lookup in ct_lookup4(). If that was a stale entry, then we also would have a rev-nat index mismatch, and force to purge the entry from the CT table, so this is also covered. In the case where there is no service for the given tuple, then ct_state_new will have rev-nat of 0, same for the ct_state from the lookup, so there won't be a mismatch. Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Signed-off-by: Thomas Graf <thomas@cilium.io> 17 May 2019, 13:07:30 UTC
6fb5a63 André Martins 16 May 2019, 21:22:34 UTC pkg/endpoint: fix assignment in nil map on restore [ upstream commit 9988700144e1aba706dc4a5249a8250fc4db8d45 ] If an endpoint is restored it can have its internal options map uninitialized. We need to check if the map is nil and created it if necessary. Signed-off-by: André Martins <andre@cilium.io> Signed-off-by: Thomas Graf <thomas@cilium.io> 17 May 2019, 13:07:30 UTC
d2d22bd André Martins 16 May 2019, 15:17:12 UTC pkg/ipcache: initialize globalmap at import time [ upstream commit 482ee8361fa5d970a9be70216bc46d27a79cdaa6 ] globalMap can be initialized at import time so there's no really need to call InitIPIdentityWatcher to initialize this map. Signed-off-by: André Martins <andre@cilium.io> Signed-off-by: Thomas Graf <thomas@cilium.io> 17 May 2019, 13:07:30 UTC
1ef49a1 André Martins 14 May 2019, 11:42:11 UTC test/provision: bump k8s testing to v1.13.6 [ upstream commit 9499c3d70f144d4e555ce7b282dec48613bfc743 ] Signed-off-by: André Martins <andre@cilium.io> Signed-off-by: Thomas Graf <thomas@cilium.io> 17 May 2019, 13:07:30 UTC
605b1a2 Daniel Borkmann 15 May 2019, 20:00:03 UTC bpf: do propagate backend, and rev nat to new entry [ upstream commit 641fde96523eb10d9016ffb4eae32f261a7a98ce ] If we don't do it, we'd always hit this path, and skb hash could be slightly different which then could select a different backend again. And also fix a tiny race window where we re-lookup a backend upon entry creation. Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Signed-off-by: Thomas Graf <thomas@cilium.io> 17 May 2019, 13:07:30 UTC
1be76cb Martynas Pumputis 15 May 2019, 13:53:32 UTC datapath: Redo backend selection if stale CT_SERVICE entry is found [ upstream commit 872b7f032ea30983c0c7f482b60b1508f106b35d ] Previously, the LB datapath in `lb{4,6}_local` functions was doing the CT lookup for a `CT_SERVICE` entry. If the entry was found, it was extracting backend ID and using it as the connection destination without checking whether the backend actually belonged to the service. The absence of the check caused interesting problems when the CT entry of no longer existing endpoint was selected (this can happen due to the bug in bpf map walking with `DumpReliablyWithCallback` which can prevent the CT GC from removing CT entries of the removed endpoint; and judging from a user BPF conntrack dump it seems that a source port selection in the kernel is quite deterministic per netns, but I haven't checked why). This led to the request being sent to an endpoint of a totally unrelated service. And because `rev_nat_index` was correct, the reverse NAT translation was xlating the src IP address of a reply to the one of the expected service, so the packet was not dropped. Another (less harmful) thing was that `lb{4,6}_lookup_slave{,_v2}` functions were modifying the given `key->slave` field without restoring upon a return, so the `lb{4,6}_lookup_service` calls for the legacy service were picking invalid backends for backward-compatibility. Signed-off-by: Martynas Pumputis <m@lambda.lt> Signed-off-by: Thomas Graf <thomas@cilium.io> 17 May 2019, 13:07:30 UTC
996f1c3 André Martins 14 May 2019, 10:07:32 UTC daemon/Makefile: rm -f on make clean for links [ upstream commit 5097dc1be71916d19064a16773f38de44f070634 ] This fixes error messages such as [0] while cleaning up Cilium. [0] ``` rm: cannot remove 'cilium-node-monitor': No such file or directory ``` Fixes: 8ac4ed848811 ("Single cilium binary") Signed-off-by: André Martins <andre@cilium.io> Signed-off-by: Thomas Graf <thomas@cilium.io> 17 May 2019, 13:07:30 UTC
811881b Ray Bejjani 15 May 2019, 11:36:07 UTC CI: Consolidate Vagrant box information into 1 file [ upstream commit a3430a3093ac672f7dedec6e38e2f05cb54b46de ] Setting the vagrant box name and version is needed for opertions not directly done inside the test runs. Placing these versions in 1 place, where they can be more easily parsed, allows us to build scripts that can consume this data during CI provisioning. This also reduced a possible mismatch between the dev vm and the CI VM versions. Signed-off-by: Ray Bejjani <ray@covalent.io> Signed-off-by: Maciej Kwiek <maciej@covalent.io> 16 May 2019, 12:47:50 UTC
4c8da0f John Fastabend 09 May 2019, 17:42:10 UTC cilium: encode table attribute in Route delete [ upstream commit 6667938d9377c06fcaace360b0ea7e9a64f217fd ] Deleting routes from encryption side was failing because we use a rule+table for encryption/decryption. This patch includes the table id in the route delete to fix this. Signed-off-by: John Fastabend <john.fastabend@gmail.com> Signed-off-by: Maciej Kwiek <maciej@covalent.io> 16 May 2019, 12:21:08 UTC
96375c8 Martynas Pumputis 10 May 2019, 15:43:28 UTC daemon: Remove stale maps only after restoring all endpoints [ upstream commit 6da36f5aecacf495388586ccc0bd1f8d7baacbe3 ] Previously, stale and disabled BPF maps could have been removed while not all endpoints had been restored. The b75ec9b4 ("datapath: Simplify map name handling") commit changed the format of a local BPF map name which made endpoint maps considered as stale, and thus to be removed during the bootstrap of cilium daemon. Having the race and the change combined, this led to an interesting bug when an endpoint tail call map (`cilium_calls_${ENDPOINT_ID}`) was removed due to the local name change before the new endpoint program has been compiled and loaded. So, the datapath BPF code of the endpoint was failing with `DROP_MISSED_TAIL_CALL` which led to some packets being dropped during an upgrade from <v1.5 to 1.5 (reported by a user). An example of such bug: ``` 2019-05-10T14:28:39.178985386Z level=info msg="Removed stale bpf map" file-path=/sys/fs/bpf/tc/globals/cilium_calls_1201 subsys=datapath-maps 2019-05-10T14:28:39.179951876Z level=info msg="Removed stale bpf map" file-path=/sys/fs/bpf/tc/globals/cilium_ep_config_1201 subsys=datapath-maps 2019-05-10T14:28:39.180742219Z level=info msg="Removed stale bpf map" file-path=/sys/fs/bpf/tc/globals/cilium_policy_1201 subsys=datapath-maps <..> 2019-05-10T14:28:39.681887394Z level=debug msg="Finished writing ELF" error="<nil>" new-elf-path=1201_next/bpf_lxc.o subsys=elf template-path=/var/run/cilium/state/templates/4bfc47bec0776fc129e1ac0dcbca88dc51850e87/bpf_lxc.o ``` This commit fixes the race by making the removal of the stale maps to wait until all endpoints have been restored. A related note from Daniel Borkmann: While other maps like hash tables etc are still retained in the kernel and can be accessed normally by the prog, tail call maps cannot. This is due to breaking dependency chain to avoid cyclic dependencies where they otherwise could never be released. So once the last user space reference is gone (pinned file or fd), they flush all the slots upon removal which happened here with the missed tail call. Signed-off-by: Martynas Pumputis <m@lambda.lt> Signed-off-by: John Fastabend <john.fastabend@gmail.com> 16 May 2019, 12:21:08 UTC
5cb2b31 Jarno Rajahalme 14 May 2019, 13:18:47 UTC envoy: Do not use deprecated configuration options. [ upstream commit ffcb9d40d189e81a8db880e562122621ae927c4b ] Cluster.Hosts has been deprecated in favor of more general LoadAssignment option. Use minimal LoadAssignment to keep the current behavior. This change removes logs like this: May 14 11:20:03 runtime1 cilium-agent[14157]: level=warning msg="[[external/envoy/source/common/protobuf/utility.cc:174] Using deprecated option 'envoy.api.v2.Cluster.hosts' from file cds.proto. This configuration will be removed from Envoy soon. Please see https://www.envoyproxy.io/docs/envoy/latest/intro/deprecated for details." subsys=envoy-misc threadID=15632 Signed-off-by: Jarno Rajahalme <jarno@covalent.io> Signed-off-by: John Fastabend <john.fastabend@gmail.com> 16 May 2019, 12:21:08 UTC
73af75f John Fastabend 14 May 2019, 15:39:41 UTC cilium: IsLocal() needs to compare both Name and Cluster [ upstream commit 2e97212664eca6452d913b40c89780a9600a4263 ] The cluster name should be used to determine if a node is local to avoid an error if two nodes in different clusters have the same name. A unique string identifier is Cluster + Name. Fixes: 076b0188b98dc ("Inter cluster connectivity (ClusterMesh)") Signed-off-by: John Fastabend <john.fastabend@gmail.com> 16 May 2019, 12:21:08 UTC
a6bff1e Martynas Pumputis 13 May 2019, 20:17:21 UTC daemon: Do not restore service if adding to cache fails [ upstream commit 1596b000041dcda60da2bd874733f7b6990dfd31 ] The successful cache population from a restored service is essential to keep service structures in sync. Signed-off-by: Martynas Pumputis <m@lambda.lt> Signed-off-by: John Fastabend <john.fastabend@gmail.com> 16 May 2019, 12:21:08 UTC
36844d3 Martynas Pumputis 13 May 2019, 20:15:59 UTC daemon: Improve logging of service restoration [ upstream commit f2922fa8edc66f63a169e1335e1d2c5257ce7b38 ] - Include service info into failure log messages. - Increase `failed` count if restore of the svc cache fails. Signed-off-by: Martynas Pumputis <m@lambda.lt> Signed-off-by: John Fastabend <john.fastabend@gmail.com> 16 May 2019, 12:21:08 UTC
411f7cd Thomas Graf 14 May 2019, 12:07:44 UTC doc: Adjust documentation with new dynamic gc interval [ upstream commit 6cf4f9b46ddef1afb1b00b944bf834bc9009b6c3 ] Signed-off-by: Thomas Graf <thomas@cilium.io> Signed-off-by: John Fastabend <john.fastabend@gmail.com> 16 May 2019, 12:21:08 UTC
39cb0ed Thomas Graf 14 May 2019, 10:04:56 UTC ctmap: Introduce variable conntrack gc interval [ upstream commit eafc47cee5c97fb3a0519befe47fed1660dec01d ] Commit df8582d16e5 ("datapath: Optimize connection-tracking GC interval") has introduced a regression by changing the default interval for LRU map based conntrack to 12h on the basis that the tables will automatically expire the oldest entries when filling up. Unfortunately, walking the table to clean the conntrack stale podIPs on pod removal taints all entries. This causes to random conntrack entries being evicted while the table is full. Resolve this by adjusting the conntrack gc interval automatically based on the number of entries deleted in the table with the most deletions. Fixes: #7997 Signed-off-by: Thomas Graf <thomas@cilium.io> Signed-off-by: John Fastabend <john.fastabend@gmail.com> 16 May 2019, 12:21:08 UTC
a4e0969 André Martins 07 May 2019, 20:43:10 UTC pkg/envoy: use proto.Equal instead comparing strings [ upstream commit 7bf007a8b35558d536dd340caa42617cc0cfbeb0 ] On medium-large clusters with lots of endpoints, the conversion to Strings in order to perform a comparison can be expensive. Instead, we should make use of the protobuf equal function that should have the same result. Signed-off-by: André Martins <andre@cilium.io> Signed-off-by: Maciej Kwiek <maciej@covalent.io> 14 May 2019, 19:18:32 UTC
9a44172 André Martins 08 May 2019, 13:22:58 UTC test: replace guestbook test docker image [ upstream commit ae9e4be0823d8ae2afd761b5a9138539a9ac159c ] The image docker.io/kubernetes/guestbook:v2 no longer exists so we need to update the guestbook test to make usage of the new guestbook test version. Signed-off-by: André Martins <andre@cilium.io> Signed-off-by: Maciej Kwiek <maciej@covalent.io> 14 May 2019, 19:18:32 UTC
9d7a4a7 André Martins 08 May 2019, 16:54:49 UTC docs: give better troubleshooting for conntrack-gc-interval [ upstream commit e0baae7a8458c2d06261211f1d88702ae26d1120 ] In case the default values of `conntrack-gc-interval` are not good enough for the end users we should prepare them how to solve the issue. Signed-off-by: André Martins <andre@cilium.io> Signed-off-by: Maciej Kwiek <maciej@covalent.io> 14 May 2019, 19:18:32 UTC
b0743b9 André Martins 09 May 2019, 22:47:12 UTC operator: fix concurrent access of variable in cnp garbage collection [ upstream commit 0f36e6e77cc4b4b0fa9e1bcb0c535e3383cbe060 ] As the cnp variable will be used in a different go routine we need to deepcopy it. Fixes: 0cd1be0e64dc ("operator: GC nodes from existing CNPs") Signed-off-by: André Martins <andre@cilium.io> Signed-off-by: Maciej Kwiek <maciej@covalent.io> 14 May 2019, 19:18:32 UTC
232012c Maciej Kwiek 10 May 2019, 11:34:50 UTC Bump vagrant box version for tests to 151 [ upstream commit 741a31b7fe29ec9ded1390247c6cfd28c2fe0e9e ] Signed-off-by: Maciej Kwiek <maciej@covalent.io> 14 May 2019, 19:18:32 UTC
e9e943e Thomas Graf 10 May 2019, 14:14:00 UTC cni: Fix unexpected end of JSON input on errors [ upstream commit b859b6d83c7dc861e0daf4227b929520285160f4 ] Due to a defer which overwrite the `err` variable, the cmdAdd signaled success for various error scenarios. Due to not providing a valid JSON result though, the CNI layer would complain with `unexpected end of JSON input` Also provide clearer error messages to allow pinpointing errors immediately. This will make error messages in `kubectl describe pod` a lot clearer. Signed-off-by: Thomas Graf <thomas@cilium.io> Signed-off-by: Maciej Kwiek <maciej@covalent.io> 14 May 2019, 19:18:32 UTC
994d3c4 André Martins 10 May 2019, 19:51:46 UTC docs: add missing cilium-operator-sa.yaml for k8s 1.14 upgrade guide [ upstream commit 902e19cf7b9af044277040f2bfe44bd5ac6cf59d ] Fixes: 9ab5e64f041e ("Documentation: Add Kubernetes 1.14 support.") Signed-off-by: André Martins <andre@cilium.io> Signed-off-by: Maciej Kwiek <maciej@covalent.io> 14 May 2019, 19:18:32 UTC
4df43c4 Martynas Pumputis 12 May 2019, 13:30:36 UTC maps: Remove disabled svc v2 maps [ upstream commit 062e0d789f46dc3a8a4ca2d740cd4948533c877b ] Previously, the svc v2 related maps were not removed when ipv4 or ipv6 services were disabled. Signed-off-by: Martynas Pumputis <m@lambda.lt> Signed-off-by: Maciej Kwiek <maciej@covalent.io> 14 May 2019, 19:18:32 UTC
61910a4 Ray Bejjani 10 May 2019, 14:10:13 UTC fqdn: DNSProxy does not fold similar DNS requests [ upstream commit 898022caa9e3b98d8693c101d7b03c7efcee3ceb ] We previously used a feature in the DNS package named SingeInFlight. This operated by "Suppressing multiple outstanding queries (with the same question, type and class)" when repeated. This would avoid multiple pods making multiple requests that would all be passed through the proxy, making it seem like the proxy's IP was misbehaving. Unfortunately, it also stopped DNS retries from working. This probably caused more disruption than any rate limiting on the proxy IP. We now do not use this feature so that retries operate correctly. Except for situations where the proxy would have been rate-limited, but was not due to the collation, this should not cause any change perceived behaviour of the fqdn subsystem. Signed-off-by: Ray Bejjani <ray@covalent.io> Signed-off-by: Maciej Kwiek <maciej@covalent.io> 14 May 2019, 19:18:32 UTC
76b7099 André Martins 14 May 2019, 09:12:37 UTC docs: fix architecture images' URL [ upstream commit 495c180d247f60cc2fc01e8c29fd2d88b0d8ea13 ] Fixes: 1695a801aad9 ("cilium: documentation, add arch page and describe bpf datapath") Signed-off-by: André Martins <andre@cilium.io> Signed-off-by: Maciej Kwiek <maciej@covalent.io> 14 May 2019, 19:18:32 UTC
ab048ba Ray Bejjani 07 May 2019, 15:30:49 UTC CI: Consolidate WaitforNPods and WaitForPodsRunning [ upstream commit 3c8a4c3791b2113343464010241da30cb59145d1 ] These two functions were almost similar, and both slightly buggy. WaitForPodsRunning, used by ExpectCiliumRunning, accidentally ignored the namespace and filter passed in, hardcoding them to "kube-system" and "-l k8s-app=cilium". It also used pod scheduling status as a proxy for readiness and did not account for pods that were being terminated. By consolidating with WaitforNPods we gain: - Use the per-container Ready flag. The per-container flag requires the container to pass its readiness check, a stricter test. - Pod lifecycle status is accounted for, avoiding counting pods stuck terminating or creating. - Consistent bugs with other Waitfor*Pods functions! Signed-off-by: Ray Bejjani <ray@covalent.io> 09 May 2019, 16:37:51 UTC
0e34c03 Ray Bejjani 07 May 2019, 15:18:19 UTC CI: WaitForNPods uses count of pods [ upstream commit 091efdda638066ed08fc0f80462b528e99461d74 ] WaitForNPods receives the count of pods that should be "running". These are pods where all containerStatus.Ready fields are true for at least minRequired pods. The old code would count the number of containers that were ready, possibly meeting the requirement prematurely. In cases where minRequired was 0 the count was determined consistently and the behavior was correct. This was a common situation. The new code requires Pods to be scheduled (i.e. Status.Phase == Running) and all containers to pass the liveness check (i.e. Status.ContainerStatus[*].Ready == true). When minRequired is 0 it is set to the number of pods and should function as before. Similarly, the older code treated pods that were being deleted as invalid as does the new. Signed-off-by: Ray Bejjani <ray@covalent.io> 09 May 2019, 16:37:51 UTC
65e5a20 André Martins 07 May 2019, 09:47:25 UTC Dockerfile: update golang to 1.12.5 [ upstream commit e1101d07ef40c1caec3ed0e4dd50d3960efdd99f ] Signed-off-by: André Martins <andre@cilium.io> Signed-off-by: Ray Bejjani <ray@covalent.io> 09 May 2019, 16:37:51 UTC
825de6e André Martins 08 May 2019, 13:33:12 UTC metrics: add map_ops_total by default [ upstream commit dec0e4914077235e1fecff714b5ceda62826a2de ] Fixes: 603df325f21b ("pkg/bpf: only account for bpf syscalls if syscall metric is enabled") Signed-off-by: André Martins <andre@cilium.io> Signed-off-by: Ray Bejjani <ray@covalent.io> 09 May 2019, 16:37:51 UTC
1f86fb6 Maciej Kwiek 06 May 2019, 11:36:20 UTC Bump vagrant box versions for tests [ upstream commit 1783ce88e46769d74e0b67c39513d4ee7eec339b ] Signed-off-by: Maciej Kwiek <maciej@covalent.io> Signed-off-by: Ray Bejjani <ray@covalent.io> 09 May 2019, 16:37:51 UTC
a3ad514 Maciej Kwiek 23 April 2019, 14:26:50 UTC Jenkins separate directories for parallel builds [ upstream commit b2cf046cd35901de61e3c3f80f5a8893ac7c5170 ] Signed-off-by: Maciej Kwiek <maciej@covalent.io> Signed-off-by: Ray Bejjani <ray@covalent.io> 09 May 2019, 16:37:51 UTC
78e2cf4 Ian Vernon 09 May 2019, 01:14:37 UTC Prepare for v1.5.1 Signed-off by: Ian Vernon <ian@cilium.io> 09 May 2019, 16:31:47 UTC
85d5fa6 Joe Stringer 07 May 2019, 17:23:15 UTC docs: Improve configmap documentation [ upstream commit 2a79e2059383ddcf0e227f3bb92565b048f40e3c ] The existing documentation makes it super easy to accidentally flush out all cilium state and break your cluster. Document that better. While we're at it, adapt some of the configmap descriptions for other options to provide more detail. Signed-off-by: Joe Stringer <joe@cilium.io> Signed-off-by: Ray Bejjani <ray@covalent.io> 09 May 2019, 01:02:35 UTC
6fb026b André Martins 07 May 2019, 20:29:18 UTC cilium/cmd: dump bpf lb list if map exists [ upstream commit 60862e752712ed403d88d8eb4a204d80c917f275 ] Signed-off-by: André Martins <andre@cilium.io> Signed-off-by: Ray Bejjani <ray@covalent.io> 09 May 2019, 01:02:35 UTC
24d0dd9 André Martins 07 May 2019, 09:39:29 UTC test/provision: update k8s testing versions to v1.11.10 and v1.12.8 [ upstream commit fdf5191739015e63fd8378d75246268ed46f7090 ] Signed-off-by: André Martins <andre@cilium.io> Signed-off-by: Ray Bejjani <ray@covalent.io> 09 May 2019, 01:02:35 UTC
back to top