| 8a41771 | Tim Jiang | 02 April 2020, 22:14:21 UTC | atlas-eval: fix eval flow failure on lwc instance removal (#1165) Currently the whole eval flow may fail when a lwc instance is removed. There's a subflow for lwc api call of each lwc instance, and that subflow is wrapped around by "RestartFlow" to enable auto-retry. The subflow is finished from upstream when a lwc instance is removed, and upstream of RestartFlow is finished, RestartFlow finishes and propagate the failure from the current ongoing attempt if any, to downstream, and eventually propagate to main flow. The fix is to have downstream of RestartFlow suppress the failure, since the failure is expected and can be ignored on lwc instance removal. | 02 April 2020, 22:14:21 UTC |
| e4f30ec | brharrington | 02 April 2020, 21:31:07 UTC | update dependencies (#1164) | 02 April 2020, 21:31:07 UTC |
| 58c34b4 | Brian Harrington | 02 April 2020, 20:04:05 UTC | sbt 1.3.9 | 02 April 2020, 20:04:05 UTC |
| 9ca8d51 | Brian Harrington | 02 April 2020, 19:55:43 UTC | log4j 2.13.1 | 02 April 2020, 19:55:43 UTC |
| 7bf1b92 | Brian Harrington | 02 April 2020, 19:55:09 UTC | frigga 0.24.0 | 02 April 2020, 19:55:09 UTC |
| 99cfe84 | Brian Harrington | 02 April 2020, 19:54:19 UTC | jackson 2.10.3 | 02 April 2020, 19:54:19 UTC |
| 695f6f2 | Brian Harrington | 02 April 2020, 19:47:55 UTC | roaringbitmap 0.8.13 | 02 April 2020, 19:47:55 UTC |
| 8cd9a02 | Brian Harrington | 02 April 2020, 19:47:16 UTC | equalsverifier 3.1.13 | 02 April 2020, 19:47:16 UTC |
| 3b56517 | Brian Harrington | 02 April 2020, 19:39:37 UTC | scalatest 3.1.1 | 02 April 2020, 19:39:37 UTC |
| 17e2fcd | Brian Harrington | 02 April 2020, 19:31:41 UTC | spectator 0.104.0 | 02 April 2020, 19:31:41 UTC |
| 4befafc | Brian Harrington | 02 April 2020, 19:30:14 UTC | iep 2.2.2 | 02 April 2020, 19:30:14 UTC |
| 2ee71a5 | Brian Harrington | 02 April 2020, 19:29:35 UTC | aws-java-sdk 1.11.752 | 02 April 2020, 19:29:35 UTC |
| b22cade | Brian Harrington | 02 April 2020, 19:27:18 UTC | akka 2.5.31 | 02 April 2020, 19:27:18 UTC |
| 70a2ad5 | brharrington | 02 April 2020, 18:17:55 UTC | remove -Xexperimental flag (#1163) Not needed now that we are on 2.13 only. Clears up warnings during the build. | 02 April 2020, 18:17:55 UTC |
| 3e1aa52 | brharrington | 27 March 2020, 22:51:53 UTC | fix logic for skipNext (#1162) The `skipNext` helper did not work correctly in the case of some complex objects with nested arrays. This change fixes the logic and adds more extensive testing with randomly generated JSON objects. | 27 March 2020, 22:51:53 UTC |
| 047ff39 | brharrington | 27 March 2020, 01:02:25 UTC | add helper for getting intValue from ItemId (#1161) This method is compatible with what you would get by calling `toBigInteger.intValue` but avoids the overhead of creating an intermediate BigInteger object if that is the only use. | 27 March 2020, 01:02:25 UTC |
| 1fd9473 | brharrington | 22 March 2020, 14:55:30 UTC | cache hash code for Query (#1160) This get computed a lot when leveraging a query index. In some cases it is over 5% of the time. Current JMH benchmark shows a similar improvement after caching. | Benchmark | Before | After | % Delta | |----------------------|---------------|---------------|---------| | index_100 | 2,950,658.3 | 3,032,810.1 | 2.8 | | index_1000 | 2,072,328.3 | 2,122,367.3 | 2.4 | | index_10000 | 2,527,147.8 | 2,714,280.1 | 7.4 | | index_100000 | 2,028,644.4 | 2,152,331.2 | 6.1 | | 22 March 2020, 14:55:30 UTC |
| 264dbb1 | brharrington | 22 March 2020, 13:29:54 UTC | switch to foreachEntry iterating maps (#1159) This method is more efficient in many cases and avoids temporary tuple objects. Now that we do not need to worry about 2.12 compatibility, switch to using it where possible. | 22 March 2020, 13:29:54 UTC |
| cff5442 | brharrington | 21 March 2020, 22:51:08 UTC | avoid intermediate tuple allocations (#1156) Update QueryIndex matching methods to use foreachEntry to avoid temporary tuples. | 21 March 2020, 22:51:08 UTC |
| 91712c0 | brharrington | 21 March 2020, 21:28:34 UTC | stop cross building for scala 2.12 (#1158) Internally all uses of these have moved to scala 2.13. To reduce maintenance effort, drop 2.12 builds. | 21 March 2020, 21:28:34 UTC |
| 2167785 | brharrington | 20 March 2020, 23:01:37 UTC | switch to foreachEntry for SmallHashMap (#1157) This method is available for maps in scala 2.13.0. This makes the custom SmallHashMap more inline with the scala maps. | 20 March 2020, 23:01:37 UTC |
| 9731b72 | brharrington | 19 March 2020, 21:56:04 UTC | SmallHashMap: avoid lambda overhead for add/remove (#1155) Loop over the array directly to avoid the lambda overhead for adding or removing an item. This can be a hot path for some use-cases. | 19 March 2020, 21:56:04 UTC |
| 832e827 | Tim Jiang | 18 March 2020, 12:12:09 UTC | atlas-eval: expose flow api to maximize throughput (#1154) Converting to Processor disables operator fusion at the boundary, and reduces overall throughput. This change exposes the Flow directly and enables operator fusion optimization, and downstream delay of the boundary reduced from hundreds of microseconds to tens of microseconds. | 18 March 2020, 12:12:09 UTC |
| 3aa9acc | Tim Jiang | 12 March 2020, 15:06:02 UTC | atlas-eval: lower eval input queue-size (#1153) It is for the incoming set of data sources which should not change at a high rate. The most recent can be used. | 12 March 2020, 15:06:02 UTC |
| be8c60d | brharrington | 11 March 2020, 17:00:35 UTC | disable trusted advisor collection by default (#1152) The trusted advisor data in cloudwatch is only updated when something refreshes the checks. Unless something is doing that, there isn't much point in collecting the cloudwatch data. | 11 March 2020, 17:00:35 UTC |
| 2bc772b | Tim Jiang | 11 March 2020, 12:32:08 UTC | atlas-eval: avoid back pressure of datasources (#1151) Currently in eval stream, "Datasources broadcast" is getting back-pressured by FinalExprEval under heavy load and new DataSources can not flow to LWC API calls and causes out of date subscriptions. This change avoids back pressure at sub streams of Datasources broadcast, so that they will not block each other. Only keeping the latest DataSources is fine for both sub streams. | 11 March 2020, 12:32:08 UTC |
| 1421394 | Tim Jiang | 09 March 2020, 12:31:33 UTC | atlas-eval: make queue size configurable for lwc call (#1150) Allows the queue size for members of the cluster to be configured. | 09 March 2020, 12:31:33 UTC |
| 2ce55f0 | brharrington | 04 March 2020, 23:10:12 UTC | cache the hash code for DataExprs (#1149) For streaming evaluation the DataExpr is the key in the time grouping stage. The largest expense in the time grouping stage was computing the hash code of the DataExpr to aggregate the datapoints as they were flowing through. | 04 March 2020, 23:10:12 UTC |
| 55cc0c1 | brharrington | 04 March 2020, 21:11:31 UTC | avoid unnecessary string conversion (#1148) When parsing the LWC messages converting to a string first is unnecessary. Avoiding this results in about 30% improvement in the throughput. | 04 March 2020, 21:11:31 UTC |
| 2fd4343 | brharrington | 03 March 2020, 22:01:33 UTC | optimize parsing of LWC messages (#1147) For some recent tests around 9% of the CPU time was spent parsing these messages when under heavy load. Using the JMH benchmark this provides a little over 2x faster parsing and similar reduction in allocations. | 03 March 2020, 22:01:33 UTC |
| 22292db | brharrington | 19 February 2020, 13:15:58 UTC | update dependencies (#1145) | 19 February 2020, 13:15:58 UTC |
| 7516ea6 | Brian Harrington | 19 February 2020, 13:05:15 UTC | sbt-scalafmt 2.3.1 | 19 February 2020, 13:05:15 UTC |
| d87f77a | Brian Harrington | 19 February 2020, 13:04:31 UTC | sbt-release 1.0.13 | 19 February 2020, 13:04:31 UTC |
| 250ed36 | Brian Harrington | 19 February 2020, 12:56:53 UTC | sbt 1.3.8 | 19 February 2020, 12:56:53 UTC |
| e4d1a51 | Brian Harrington | 19 February 2020, 12:53:34 UTC | jol-core 0.10 | 19 February 2020, 12:53:34 UTC |
| 7d1cc3d | Brian Harrington | 19 February 2020, 12:45:22 UTC | equalsverifier 3.1.12 | 19 February 2020, 12:45:22 UTC |
| b2088f0 | Brian Harrington | 19 February 2020, 12:43:48 UTC | aws-java-sdk 1.11.723 | 19 February 2020, 12:43:48 UTC |
| 2660ab2 | Brian Harrington | 19 February 2020, 12:43:19 UTC | iep 2.2.1 | 19 February 2020, 12:43:19 UTC |
| e28ef69 | Brian Harrington | 19 February 2020, 12:32:40 UTC | akka 2.5.29 | 19 February 2020, 12:32:40 UTC |
| 3175e29 | Brian Harrington | 19 February 2020, 12:31:44 UTC | scala-collection-compat 2.1.4 | 19 February 2020, 12:31:44 UTC |
| e096652 | Brian Harrington | 19 February 2020, 12:29:28 UTC | spectator 0.103.0 | 19 February 2020, 12:29:28 UTC |
| 1ee176f | brharrington | 11 February 2020, 17:22:15 UTC | adjust default serialization for java.time (#1144) Set the default serialization to use millisecond timestamps. This keeps it more consistent with other usage in Atlas. | 11 February 2020, 17:22:15 UTC |
| 0d5876d | brharrington | 06 February 2020, 13:41:36 UTC | update launcher url to use HTTPS (#1141) Bintray now requires HTTPS for downloading the artifacts. Also bump to the latest version. | 06 February 2020, 13:41:36 UTC |
| 45ffa4b | brharrington | 06 February 2020, 12:58:12 UTC | update settings in memory.conf (#1139) Update the memory.conf example to reflect move of graphing logic to eval library. Fixes #1135 | 06 February 2020, 12:58:12 UTC |
| 0df4e41 | Tim Jiang | 03 February 2020, 20:21:19 UTC | emit empty DataSources for a host if no more DataSource (#1137) Emit empty DataSources for a host if no more DataSource so that downstream can stop stream from LWC quickly for this host. This is in preparation for a single evaluator instance on the atlas-stream service that is part of iep-apps. | 03 February 2020, 20:21:19 UTC |
| 1563cfe | Tim Jiang | 29 January 2020, 13:44:57 UTC | fix streaming issue for multiple websocket clients (#1136) Currently WebSocket handler is singleton with the same streamID shared by different client connection, so only the first connection works. The fix is to create 1 instance of handler per request, also getting streamId from request allows client to have same streamId for all LWC API calls. | 29 January 2020, 13:44:57 UTC |
| 754eeae | brharrington | 28 January 2020, 00:35:40 UTC | move metadata counter to separate category (#1134) During the transition period it is desirable to have this metric for ASGs even though we do not need other EC2 CW data. | 28 January 2020, 00:35:40 UTC |
| b867a4f | brharrington | 27 January 2020, 23:53:06 UTC | exclude ec2 cloudwatch category by default (#1133) A user can opt-in if desired. This makes the default config line up with internal usage. | 27 January 2020, 23:53:06 UTC |
| 1fac884 | brharrington | 23 January 2020, 16:30:00 UTC | debug log for dropped messages (#1132) Adds a debug level log to see specific messages that are dropped because they cannot be enqueued. | 23 January 2020, 16:30:00 UTC |
| 9b74531 | brharrington | 21 January 2020, 17:32:40 UTC | add mapping for MetadataNoToken (#1131) Tracks the number of requests to IMDSv1. Used to help with tracking migration to IMDSv2. | 21 January 2020, 17:32:40 UTC |
| 50cbe08 | brharrington | 20 January 2020, 22:37:07 UTC | update dependencies (#1130) | 20 January 2020, 22:37:07 UTC |
| e1f3c98 | Brian Harrington | 20 January 2020, 22:15:28 UTC | scalafmt 2.3.2 Also fixed minor formatting changes that happen with this newer version. | 20 January 2020, 22:15:28 UTC |
| fb86bed | Brian Harrington | 20 January 2020, 22:01:51 UTC | sbt-scalafmt 2.3.0 | 20 January 2020, 22:01:51 UTC |
| fe338b6 | Brian Harrington | 20 January 2020, 21:58:31 UTC | sbt-bintray 0.5.6 | 20 January 2020, 21:58:31 UTC |
| 8cda737 | Brian Harrington | 20 January 2020, 21:57:42 UTC | sbt 1.3.7 | 20 January 2020, 21:57:42 UTC |
| 5dbdcb4 | Brian Harrington | 20 January 2020, 21:56:24 UTC | log4j 2.13.0 | 20 January 2020, 21:56:24 UTC |
| e2d9cef | Brian Harrington | 20 January 2020, 21:38:46 UTC | roaringbitmap 0.8.12 | 20 January 2020, 21:38:46 UTC |
| d235dd5 | Brian Harrington | 20 January 2020, 21:38:18 UTC | equalsverifier 3.1.11 | 20 January 2020, 21:38:18 UTC |
| 59f01bc | Brian Harrington | 20 January 2020, 21:29:11 UTC | caffeine 2.8.1 | 20 January 2020, 21:29:11 UTC |
| ddb2904 | Brian Harrington | 20 January 2020, 20:11:09 UTC | iep 2.1.8 | 20 January 2020, 20:11:09 UTC |
| 685c6db | Brian Harrington | 20 January 2020, 20:08:48 UTC | akka 2.5.27 | 20 January 2020, 20:08:48 UTC |
| 2ca176d | Brian Harrington | 20 January 2020, 18:53:50 UTC | slf4j 1.7.30 | 20 January 2020, 18:53:50 UTC |
| b3a10d8 | Brian Harrington | 20 January 2020, 18:53:18 UTC | aws-java-sdk 1.11.702 | 20 January 2020, 18:53:18 UTC |
| b37ab8c | Brian Harrington | 20 January 2020, 18:31:40 UTC | scala-collection-compat 2.1.3 | 20 January 2020, 18:31:40 UTC |
| 3db7529 | Brian Harrington | 20 January 2020, 18:30:31 UTC | spectator 0.101.0 | 20 January 2020, 18:30:31 UTC |
| 4a1d9e3 | Brian Harrington | 20 January 2020, 18:28:13 UTC | jackson 2.10.2 | 20 January 2020, 18:28:13 UTC |
| 0843692 | Tim Jiang | 16 January 2020, 18:15:23 UTC | fix delay of stopping host lookup (#1129) There's a delay on stopping host lookup caused by EvaluationFlows.repeatWhile, use EvaluationFlows.stoppableSource instead. The delay can cause temporarily incorrect LWC subscriptions when subscription changes. | 16 January 2020, 18:15:23 UTC |
| 84114ae | brharrington | 14 January 2020, 21:28:26 UTC | remove unused methods from TagKey (#1128) Cleanup some of the unused methods and fields for TagKey object. The validation is handled based on the config now. | 14 January 2020, 21:28:26 UTC |
| 8b65138 | Daniel Muino | 14 January 2020, 19:08:14 UTC | Allow nf.shard tags (#1127) In order to support a new method of sharding internally at Netflix, allow `nf.shard1` and `nf.shard2` tags | 14 January 2020, 19:08:14 UTC |
| 418f909 | brharrington | 10 January 2020, 00:46:07 UTC | fix unused import warnings (#1126) | 10 January 2020, 00:46:07 UTC |
| 733b7ef | brharrington | 10 January 2020, 00:28:25 UTC | update lwcapi to accept `step` (#1125) The goal is to try and make the naming more consistent and standardize on `step` everywhere. Parts of lwcapi were the only places that referred to this as `frequency`. This updates the `ExpressionMetadata` class so it will treat `step` as an alias. This will allow the eval library to be updated. For now the output of the expressions api needs to keep it as `frequency` to ensure backwards compatibility with clients. | 10 January 2020, 00:28:25 UTC |
| 6b79023 | Tim Jiang | 10 January 2020, 00:26:17 UTC | update eval to use lwcapi websocket (#1123) This allows us to simplify the flow and avoid the separate branch that manages the subscription updates. | 10 January 2020, 00:26:17 UTC |
| 56243b6 | brharrington | 09 January 2020, 23:20:23 UTC | fix style for chart title with dark theme (#1124) The text was not being rendered with the image style and was defaulting to black. So it could not be seen with a dark background. | 09 January 2020, 23:20:23 UTC |
| 1124e4f | brharrington | 02 January 2020, 15:29:38 UTC | update license headers for 2020 (#1121) | 02 January 2020, 15:29:38 UTC |
| 56457e9 | brharrington | 24 December 2019, 16:15:12 UTC | add cluster group by operator (#1120) Maintains a sub-flow for each member of a cluster. The set of flows can change as the membership of the cluster changes. This is a more generic version of the connection manager logic from the eval library. Primary use-case is for operations that require communication with all members of a cluster. | 24 December 2019, 16:15:12 UTC |
| 214374b | brharrington | 23 December 2019, 18:10:58 UTC | stream helper to filter out duplicates (#1119) Works similar to the `uniq` command on unix and filters out consecutive values that are repeated. | 23 December 2019, 18:10:58 UTC |
| 7c6eb7f | Tim Jiang | 23 December 2019, 14:30:37 UTC | allow updating subscriptions via web socket API (#1118) Before it would work for the initial subscription, but didn't properly handle updates. This change adds a session manager that will handle the updated subscriptions. | 23 December 2019, 14:30:37 UTC |
| 06b23a2 | brharrington | 17 December 2019, 01:03:04 UTC | fix Jackson deprecation warnings (#1117) Use builder for creating the JsonFactory instance. | 17 December 2019, 01:03:04 UTC |
| af59566 | brharrington | 17 December 2019, 00:52:24 UTC | fix RequestEntityTooLarge deprecation (#1116) Renamed to PayloadTooLarge in latest version of akka-http. | 17 December 2019, 00:52:24 UTC |
| b768a0c | brharrington | 16 December 2019, 20:26:45 UTC | fix scalatest warnings (#1115) Fixes the deprecation warnings after updating to scalatest 3.1.0. | 16 December 2019, 20:26:45 UTC |
| 16c5535 | brharrington | 16 December 2019, 19:14:33 UTC | add support for a dark theme (#1114) Adds a theme parameter that can be used with graph images. There are two themes right now: `light` and `dark`. The `light` theme is the default and renders the same way as before. The default armytage palette is intended for a white background. This change also adds two new palettes from the [R Polychrome] package. The `light24`palette is the default when using the `dark` theme. [R Polychrome]: https://rdrr.io/rforge/Polychrome/man/palettes.html This change is still experimental and may change prior to 1.7.0 release. | 16 December 2019, 19:14:33 UTC |
| 6a41c21 | brharrington | 10 December 2019, 17:04:51 UTC | handle some shard groups being empty (#1113) In intermediate deployment stages, some of the shard groups may be empty. Before this would result in a divide by zero error. Now it will return null. Null is used instead of `Option[T]` to maintain backwards compatibility and avoid the extra allocation if used in a hot path. | 10 December 2019, 17:04:51 UTC |
| f1d110e | brharrington | 02 December 2019, 19:49:35 UTC | simplify condition for isRestricted (#1112) The prefixes list is not used anywhere and it is faster as a simple condition. | 02 December 2019, 19:49:35 UTC |
| f4880f6 | brharrington | 02 December 2019, 17:13:25 UTC | remove empty test suite (#1111) | 02 December 2019, 17:13:25 UTC |
| 55995c2 | brharrington | 02 December 2019, 17:02:02 UTC | update dependencies (#1110) | 02 December 2019, 17:02:02 UTC |
| 7c677c6 | Brian Harrington | 02 December 2019, 16:48:40 UTC | sbt 1.3.4 | 02 December 2019, 16:48:40 UTC |
| 52d08bf | Brian Harrington | 02 December 2019, 16:46:50 UTC | jackson 2.10.1 | 02 December 2019, 16:46:50 UTC |
| 4d767b8 | Brian Harrington | 02 December 2019, 16:38:34 UTC | scalatest 3.1.0 | 02 December 2019, 16:38:34 UTC |
| aab0781 | Brian Harrington | 02 December 2019, 16:37:40 UTC | slf4j 1.7.29 | 02 December 2019, 16:37:40 UTC |
| fd15b60 | Brian Harrington | 02 December 2019, 16:36:55 UTC | spectator 0.99.2 | 02 December 2019, 16:36:55 UTC |
| 6ab22b2 | Brian Harrington | 02 December 2019, 16:11:14 UTC | iep 2.1.6 | 02 December 2019, 16:11:14 UTC |
| 0bd72f4 | Brian Harrington | 02 December 2019, 16:10:19 UTC | aws-java-sdk 1.11.681 | 02 December 2019, 16:10:19 UTC |
| d4811a1 | Brian Harrington | 02 December 2019, 16:08:36 UTC | akka-http 10.1.11 | 02 December 2019, 16:08:36 UTC |
| 5d72e0a | brharrington | 22 November 2019, 17:59:08 UTC | add mapping for transit gateway metrics (#1109) CloudWatch mappings for [TGW metrics]. Note, in CW there are drop counts for bytes as well as packets, though the docs only shows packets. [TGW metrics]: https://docs.aws.amazon.com/vpc/latest/tgw/transit-gateway-cloudwatch-metrics.html | 22 November 2019, 17:59:08 UTC |
| 8484f6e | brharrington | 20 November 2019, 16:09:21 UTC | remove old redis conf settings (#1108) Redis hasn't been used in a long time. See #561. | 20 November 2019, 16:09:21 UTC |
| 4b7e57d | Tim J | 14 November 2019, 19:00:45 UTC | fix stream aggregation with same expr for different hosts (#1107) Fix stream result aggregation issue when multiple URLs with same query in one session. When multiple urls have the same query but different host, the result is aggregation of all data from urls with the same query ignoring different host. Fixed by grouping by host, processing them in sub-streams with the existing flow and then merging results back to one stream. | 14 November 2019, 19:00:45 UTC |
| 1b93714 | brharrington | 14 November 2019, 18:21:06 UTC | avoid delay on finish for local eval processor (#1106) When using the eval processor for local data, it should stop quickly after the upstream is complete. Before the throttling on the Eureka lookup would cause about a 30s delay. This changes the Eureka lookup to use an empty source if there are no remote data sources. Also fixes a bug with TimeGrouped where it could potentially push without a downstream pull when the upstream finishes. | 14 November 2019, 18:21:06 UTC |
| db806d6 | brharrington | 13 November 2019, 02:58:11 UTC | optimize ByteStringInputStream for large payloads (#1105) Uses the `asByteBuffer` conversion to allow for more efficient reads. The `copyToArray` was really slow and was worse than just always using `apply` and reading a single byte. The byte buffer approach avoids the intermediate array allocation and supports a fast copy to another array. For the sample benchmark it is about 10x faster. | 13 November 2019, 02:58:11 UTC |
| 2d83102 | brharrington | 11 November 2019, 02:59:32 UTC | optimization for tag validation (#1103) A lot of time is spent iterating the maps. It is faster to iterate over the set of rules for each key/value pair. | 11 November 2019, 02:59:32 UTC |
| 1c2ef40 | Tim J | 09 November 2019, 04:52:22 UTC | improve tick label selection for log scales (#1093) For large ranges it will now use powers of 10 instead of using the normal linear selection. For smaller ranges the linear mode will still be used. If using binary labels, then the it will still have the old behavior. This will be fixed in a subsequent PR. Fixes #1007 | 09 November 2019, 04:52:22 UTC |
