| 832e827 | Tim Jiang | 18 March 2020, 12:12:09 UTC | atlas-eval: expose flow api to maximize throughput (#1154) Converting to Processor disables operator fusion at the boundary, and reduces overall throughput. This change exposes the Flow directly and enables operator fusion optimization, and downstream delay of the boundary reduced from hundreds of microseconds to tens of microseconds. | 18 March 2020, 12:12:09 UTC |
| 3aa9acc | Tim Jiang | 12 March 2020, 15:06:02 UTC | atlas-eval: lower eval input queue-size (#1153) It is for the incoming set of data sources which should not change at a high rate. The most recent can be used. | 12 March 2020, 15:06:02 UTC |
| be8c60d | brharrington | 11 March 2020, 17:00:35 UTC | disable trusted advisor collection by default (#1152) The trusted advisor data in cloudwatch is only updated when something refreshes the checks. Unless something is doing that, there isn't much point in collecting the cloudwatch data. | 11 March 2020, 17:00:35 UTC |
| 2bc772b | Tim Jiang | 11 March 2020, 12:32:08 UTC | atlas-eval: avoid back pressure of datasources (#1151) Currently in eval stream, "Datasources broadcast" is getting back-pressured by FinalExprEval under heavy load and new DataSources can not flow to LWC API calls and causes out of date subscriptions. This change avoids back pressure at sub streams of Datasources broadcast, so that they will not block each other. Only keeping the latest DataSources is fine for both sub streams. | 11 March 2020, 12:32:08 UTC |
| 1421394 | Tim Jiang | 09 March 2020, 12:31:33 UTC | atlas-eval: make queue size configurable for lwc call (#1150) Allows the queue size for members of the cluster to be configured. | 09 March 2020, 12:31:33 UTC |
| 2ce55f0 | brharrington | 04 March 2020, 23:10:12 UTC | cache the hash code for DataExprs (#1149) For streaming evaluation the DataExpr is the key in the time grouping stage. The largest expense in the time grouping stage was computing the hash code of the DataExpr to aggregate the datapoints as they were flowing through. | 04 March 2020, 23:10:12 UTC |
| 55cc0c1 | brharrington | 04 March 2020, 21:11:31 UTC | avoid unnecessary string conversion (#1148) When parsing the LWC messages converting to a string first is unnecessary. Avoiding this results in about 30% improvement in the throughput. | 04 March 2020, 21:11:31 UTC |
| 2fd4343 | brharrington | 03 March 2020, 22:01:33 UTC | optimize parsing of LWC messages (#1147) For some recent tests around 9% of the CPU time was spent parsing these messages when under heavy load. Using the JMH benchmark this provides a little over 2x faster parsing and similar reduction in allocations. | 03 March 2020, 22:01:33 UTC |
| 22292db | brharrington | 19 February 2020, 13:15:58 UTC | update dependencies (#1145) | 19 February 2020, 13:15:58 UTC |
| 7516ea6 | Brian Harrington | 19 February 2020, 13:05:15 UTC | sbt-scalafmt 2.3.1 | 19 February 2020, 13:05:15 UTC |
| d87f77a | Brian Harrington | 19 February 2020, 13:04:31 UTC | sbt-release 1.0.13 | 19 February 2020, 13:04:31 UTC |
| 250ed36 | Brian Harrington | 19 February 2020, 12:56:53 UTC | sbt 1.3.8 | 19 February 2020, 12:56:53 UTC |
| e4d1a51 | Brian Harrington | 19 February 2020, 12:53:34 UTC | jol-core 0.10 | 19 February 2020, 12:53:34 UTC |
| 7d1cc3d | Brian Harrington | 19 February 2020, 12:45:22 UTC | equalsverifier 3.1.12 | 19 February 2020, 12:45:22 UTC |
| b2088f0 | Brian Harrington | 19 February 2020, 12:43:48 UTC | aws-java-sdk 1.11.723 | 19 February 2020, 12:43:48 UTC |
| 2660ab2 | Brian Harrington | 19 February 2020, 12:43:19 UTC | iep 2.2.1 | 19 February 2020, 12:43:19 UTC |
| e28ef69 | Brian Harrington | 19 February 2020, 12:32:40 UTC | akka 2.5.29 | 19 February 2020, 12:32:40 UTC |
| 3175e29 | Brian Harrington | 19 February 2020, 12:31:44 UTC | scala-collection-compat 2.1.4 | 19 February 2020, 12:31:44 UTC |
| e096652 | Brian Harrington | 19 February 2020, 12:29:28 UTC | spectator 0.103.0 | 19 February 2020, 12:29:28 UTC |
| 1ee176f | brharrington | 11 February 2020, 17:22:15 UTC | adjust default serialization for java.time (#1144) Set the default serialization to use millisecond timestamps. This keeps it more consistent with other usage in Atlas. | 11 February 2020, 17:22:15 UTC |
| 0d5876d | brharrington | 06 February 2020, 13:41:36 UTC | update launcher url to use HTTPS (#1141) Bintray now requires HTTPS for downloading the artifacts. Also bump to the latest version. | 06 February 2020, 13:41:36 UTC |
| 45ffa4b | brharrington | 06 February 2020, 12:58:12 UTC | update settings in memory.conf (#1139) Update the memory.conf example to reflect move of graphing logic to eval library. Fixes #1135 | 06 February 2020, 12:58:12 UTC |
| 0df4e41 | Tim Jiang | 03 February 2020, 20:21:19 UTC | emit empty DataSources for a host if no more DataSource (#1137) Emit empty DataSources for a host if no more DataSource so that downstream can stop stream from LWC quickly for this host. This is in preparation for a single evaluator instance on the atlas-stream service that is part of iep-apps. | 03 February 2020, 20:21:19 UTC |
| 1563cfe | Tim Jiang | 29 January 2020, 13:44:57 UTC | fix streaming issue for multiple websocket clients (#1136) Currently WebSocket handler is singleton with the same streamID shared by different client connection, so only the first connection works. The fix is to create 1 instance of handler per request, also getting streamId from request allows client to have same streamId for all LWC API calls. | 29 January 2020, 13:44:57 UTC |
| 754eeae | brharrington | 28 January 2020, 00:35:40 UTC | move metadata counter to separate category (#1134) During the transition period it is desirable to have this metric for ASGs even though we do not need other EC2 CW data. | 28 January 2020, 00:35:40 UTC |
| b867a4f | brharrington | 27 January 2020, 23:53:06 UTC | exclude ec2 cloudwatch category by default (#1133) A user can opt-in if desired. This makes the default config line up with internal usage. | 27 January 2020, 23:53:06 UTC |
| 1fac884 | brharrington | 23 January 2020, 16:30:00 UTC | debug log for dropped messages (#1132) Adds a debug level log to see specific messages that are dropped because they cannot be enqueued. | 23 January 2020, 16:30:00 UTC |
| 9b74531 | brharrington | 21 January 2020, 17:32:40 UTC | add mapping for MetadataNoToken (#1131) Tracks the number of requests to IMDSv1. Used to help with tracking migration to IMDSv2. | 21 January 2020, 17:32:40 UTC |
| 50cbe08 | brharrington | 20 January 2020, 22:37:07 UTC | update dependencies (#1130) | 20 January 2020, 22:37:07 UTC |
| e1f3c98 | Brian Harrington | 20 January 2020, 22:15:28 UTC | scalafmt 2.3.2 Also fixed minor formatting changes that happen with this newer version. | 20 January 2020, 22:15:28 UTC |
| fb86bed | Brian Harrington | 20 January 2020, 22:01:51 UTC | sbt-scalafmt 2.3.0 | 20 January 2020, 22:01:51 UTC |
| fe338b6 | Brian Harrington | 20 January 2020, 21:58:31 UTC | sbt-bintray 0.5.6 | 20 January 2020, 21:58:31 UTC |
| 8cda737 | Brian Harrington | 20 January 2020, 21:57:42 UTC | sbt 1.3.7 | 20 January 2020, 21:57:42 UTC |
| 5dbdcb4 | Brian Harrington | 20 January 2020, 21:56:24 UTC | log4j 2.13.0 | 20 January 2020, 21:56:24 UTC |
| e2d9cef | Brian Harrington | 20 January 2020, 21:38:46 UTC | roaringbitmap 0.8.12 | 20 January 2020, 21:38:46 UTC |
| d235dd5 | Brian Harrington | 20 January 2020, 21:38:18 UTC | equalsverifier 3.1.11 | 20 January 2020, 21:38:18 UTC |
| 59f01bc | Brian Harrington | 20 January 2020, 21:29:11 UTC | caffeine 2.8.1 | 20 January 2020, 21:29:11 UTC |
| ddb2904 | Brian Harrington | 20 January 2020, 20:11:09 UTC | iep 2.1.8 | 20 January 2020, 20:11:09 UTC |
| 685c6db | Brian Harrington | 20 January 2020, 20:08:48 UTC | akka 2.5.27 | 20 January 2020, 20:08:48 UTC |
| 2ca176d | Brian Harrington | 20 January 2020, 18:53:50 UTC | slf4j 1.7.30 | 20 January 2020, 18:53:50 UTC |
| b3a10d8 | Brian Harrington | 20 January 2020, 18:53:18 UTC | aws-java-sdk 1.11.702 | 20 January 2020, 18:53:18 UTC |
| b37ab8c | Brian Harrington | 20 January 2020, 18:31:40 UTC | scala-collection-compat 2.1.3 | 20 January 2020, 18:31:40 UTC |
| 3db7529 | Brian Harrington | 20 January 2020, 18:30:31 UTC | spectator 0.101.0 | 20 January 2020, 18:30:31 UTC |
| 4a1d9e3 | Brian Harrington | 20 January 2020, 18:28:13 UTC | jackson 2.10.2 | 20 January 2020, 18:28:13 UTC |
| 0843692 | Tim Jiang | 16 January 2020, 18:15:23 UTC | fix delay of stopping host lookup (#1129) There's a delay on stopping host lookup caused by EvaluationFlows.repeatWhile, use EvaluationFlows.stoppableSource instead. The delay can cause temporarily incorrect LWC subscriptions when subscription changes. | 16 January 2020, 18:15:23 UTC |
| 84114ae | brharrington | 14 January 2020, 21:28:26 UTC | remove unused methods from TagKey (#1128) Cleanup some of the unused methods and fields for TagKey object. The validation is handled based on the config now. | 14 January 2020, 21:28:26 UTC |
| 8b65138 | Daniel Muino | 14 January 2020, 19:08:14 UTC | Allow nf.shard tags (#1127) In order to support a new method of sharding internally at Netflix, allow `nf.shard1` and `nf.shard2` tags | 14 January 2020, 19:08:14 UTC |
| 418f909 | brharrington | 10 January 2020, 00:46:07 UTC | fix unused import warnings (#1126) | 10 January 2020, 00:46:07 UTC |
| 733b7ef | brharrington | 10 January 2020, 00:28:25 UTC | update lwcapi to accept `step` (#1125) The goal is to try and make the naming more consistent and standardize on `step` everywhere. Parts of lwcapi were the only places that referred to this as `frequency`. This updates the `ExpressionMetadata` class so it will treat `step` as an alias. This will allow the eval library to be updated. For now the output of the expressions api needs to keep it as `frequency` to ensure backwards compatibility with clients. | 10 January 2020, 00:28:25 UTC |
| 6b79023 | Tim Jiang | 10 January 2020, 00:26:17 UTC | update eval to use lwcapi websocket (#1123) This allows us to simplify the flow and avoid the separate branch that manages the subscription updates. | 10 January 2020, 00:26:17 UTC |
| 56243b6 | brharrington | 09 January 2020, 23:20:23 UTC | fix style for chart title with dark theme (#1124) The text was not being rendered with the image style and was defaulting to black. So it could not be seen with a dark background. | 09 January 2020, 23:20:23 UTC |
| 1124e4f | brharrington | 02 January 2020, 15:29:38 UTC | update license headers for 2020 (#1121) | 02 January 2020, 15:29:38 UTC |
| 56457e9 | brharrington | 24 December 2019, 16:15:12 UTC | add cluster group by operator (#1120) Maintains a sub-flow for each member of a cluster. The set of flows can change as the membership of the cluster changes. This is a more generic version of the connection manager logic from the eval library. Primary use-case is for operations that require communication with all members of a cluster. | 24 December 2019, 16:15:12 UTC |
| 214374b | brharrington | 23 December 2019, 18:10:58 UTC | stream helper to filter out duplicates (#1119) Works similar to the `uniq` command on unix and filters out consecutive values that are repeated. | 23 December 2019, 18:10:58 UTC |
| 7c6eb7f | Tim Jiang | 23 December 2019, 14:30:37 UTC | allow updating subscriptions via web socket API (#1118) Before it would work for the initial subscription, but didn't properly handle updates. This change adds a session manager that will handle the updated subscriptions. | 23 December 2019, 14:30:37 UTC |
| 06b23a2 | brharrington | 17 December 2019, 01:03:04 UTC | fix Jackson deprecation warnings (#1117) Use builder for creating the JsonFactory instance. | 17 December 2019, 01:03:04 UTC |
| af59566 | brharrington | 17 December 2019, 00:52:24 UTC | fix RequestEntityTooLarge deprecation (#1116) Renamed to PayloadTooLarge in latest version of akka-http. | 17 December 2019, 00:52:24 UTC |
| b768a0c | brharrington | 16 December 2019, 20:26:45 UTC | fix scalatest warnings (#1115) Fixes the deprecation warnings after updating to scalatest 3.1.0. | 16 December 2019, 20:26:45 UTC |
| 16c5535 | brharrington | 16 December 2019, 19:14:33 UTC | add support for a dark theme (#1114) Adds a theme parameter that can be used with graph images. There are two themes right now: `light` and `dark`. The `light` theme is the default and renders the same way as before. The default armytage palette is intended for a white background. This change also adds two new palettes from the [R Polychrome] package. The `light24`palette is the default when using the `dark` theme. [R Polychrome]: https://rdrr.io/rforge/Polychrome/man/palettes.html This change is still experimental and may change prior to 1.7.0 release. | 16 December 2019, 19:14:33 UTC |
| 6a41c21 | brharrington | 10 December 2019, 17:04:51 UTC | handle some shard groups being empty (#1113) In intermediate deployment stages, some of the shard groups may be empty. Before this would result in a divide by zero error. Now it will return null. Null is used instead of `Option[T]` to maintain backwards compatibility and avoid the extra allocation if used in a hot path. | 10 December 2019, 17:04:51 UTC |
| f1d110e | brharrington | 02 December 2019, 19:49:35 UTC | simplify condition for isRestricted (#1112) The prefixes list is not used anywhere and it is faster as a simple condition. | 02 December 2019, 19:49:35 UTC |
| f4880f6 | brharrington | 02 December 2019, 17:13:25 UTC | remove empty test suite (#1111) | 02 December 2019, 17:13:25 UTC |
| 55995c2 | brharrington | 02 December 2019, 17:02:02 UTC | update dependencies (#1110) | 02 December 2019, 17:02:02 UTC |
| 7c677c6 | Brian Harrington | 02 December 2019, 16:48:40 UTC | sbt 1.3.4 | 02 December 2019, 16:48:40 UTC |
| 52d08bf | Brian Harrington | 02 December 2019, 16:46:50 UTC | jackson 2.10.1 | 02 December 2019, 16:46:50 UTC |
| 4d767b8 | Brian Harrington | 02 December 2019, 16:38:34 UTC | scalatest 3.1.0 | 02 December 2019, 16:38:34 UTC |
| aab0781 | Brian Harrington | 02 December 2019, 16:37:40 UTC | slf4j 1.7.29 | 02 December 2019, 16:37:40 UTC |
| fd15b60 | Brian Harrington | 02 December 2019, 16:36:55 UTC | spectator 0.99.2 | 02 December 2019, 16:36:55 UTC |
| 6ab22b2 | Brian Harrington | 02 December 2019, 16:11:14 UTC | iep 2.1.6 | 02 December 2019, 16:11:14 UTC |
| 0bd72f4 | Brian Harrington | 02 December 2019, 16:10:19 UTC | aws-java-sdk 1.11.681 | 02 December 2019, 16:10:19 UTC |
| d4811a1 | Brian Harrington | 02 December 2019, 16:08:36 UTC | akka-http 10.1.11 | 02 December 2019, 16:08:36 UTC |
| 5d72e0a | brharrington | 22 November 2019, 17:59:08 UTC | add mapping for transit gateway metrics (#1109) CloudWatch mappings for [TGW metrics]. Note, in CW there are drop counts for bytes as well as packets, though the docs only shows packets. [TGW metrics]: https://docs.aws.amazon.com/vpc/latest/tgw/transit-gateway-cloudwatch-metrics.html | 22 November 2019, 17:59:08 UTC |
| 8484f6e | brharrington | 20 November 2019, 16:09:21 UTC | remove old redis conf settings (#1108) Redis hasn't been used in a long time. See #561. | 20 November 2019, 16:09:21 UTC |
| 4b7e57d | Tim J | 14 November 2019, 19:00:45 UTC | fix stream aggregation with same expr for different hosts (#1107) Fix stream result aggregation issue when multiple URLs with same query in one session. When multiple urls have the same query but different host, the result is aggregation of all data from urls with the same query ignoring different host. Fixed by grouping by host, processing them in sub-streams with the existing flow and then merging results back to one stream. | 14 November 2019, 19:00:45 UTC |
| 1b93714 | brharrington | 14 November 2019, 18:21:06 UTC | avoid delay on finish for local eval processor (#1106) When using the eval processor for local data, it should stop quickly after the upstream is complete. Before the throttling on the Eureka lookup would cause about a 30s delay. This changes the Eureka lookup to use an empty source if there are no remote data sources. Also fixes a bug with TimeGrouped where it could potentially push without a downstream pull when the upstream finishes. | 14 November 2019, 18:21:06 UTC |
| db806d6 | brharrington | 13 November 2019, 02:58:11 UTC | optimize ByteStringInputStream for large payloads (#1105) Uses the `asByteBuffer` conversion to allow for more efficient reads. The `copyToArray` was really slow and was worse than just always using `apply` and reading a single byte. The byte buffer approach avoids the intermediate array allocation and supports a fast copy to another array. For the sample benchmark it is about 10x faster. | 13 November 2019, 02:58:11 UTC |
| 2d83102 | brharrington | 11 November 2019, 02:59:32 UTC | optimization for tag validation (#1103) A lot of time is spent iterating the maps. It is faster to iterate over the set of rules for each key/value pair. | 11 November 2019, 02:59:32 UTC |
| 1c2ef40 | Tim J | 09 November 2019, 04:52:22 UTC | improve tick label selection for log scales (#1093) For large ranges it will now use powers of 10 instead of using the normal linear selection. For smaller ranges the linear mode will still be used. If using binary labels, then the it will still have the old behavior. This will be fixed in a subsequent PR. Fixes #1007 | 09 November 2019, 04:52:22 UTC |
| b79a0bd | brharrington | 06 November 2019, 23:27:18 UTC | remove duplicate ByteStringInputStream (#1102) This helper class was duplicated in eval and akka packages. Now the usage has been consolidated. Eval library wasn't actually using the util, so a few places have been updated to avoid the intermediate array allocation. | 06 November 2019, 23:27:18 UTC |
| 22d8b39 | brharrington | 30 October 2019, 18:30:11 UTC | extract step when creating DataSource from JSON (#1101) If the step size is not in the JSON payload, then extract from the URI as expected rather than leave it as null. | 30 October 2019, 18:30:11 UTC |
| dfc8597 | brharrington | 23 October 2019, 21:52:41 UTC | add options to tune request handler (#1100) There are options now to disable automatic handling of compression and whether or not to setup access logging. For high volume services the automatic compression results in a lot of allocations for intermediate ByteStrings. The json directives will not automatically wrap with a GZIPInputStream if the data is GZIP compressed. For some services this drastically reduces the allocation rate. | 23 October 2019, 21:52:41 UTC |
| 7795f1e | brharrington | 23 October 2019, 17:25:00 UTC | reduce allocations for parsing payloads (#1099) Adds a ByteStringInputStream so we can read the data directly from the ByteString without copying to an intermediate array. | 23 October 2019, 17:25:00 UTC |
| c7c61d5 | Tim J | 23 October 2019, 13:49:50 UTC | support lwcapi instance lookup via edda (#1098) Adds support for looking up the instances of the lwcapi clusters via Edda instead of Eureka. For now it is done with minimal changes and still supports the Eureka URIs. This can be refactored to be more generic in a later version. | 23 October 2019, 13:49:50 UTC |
| b5a0d80 | brharrington | 22 October 2019, 23:16:56 UTC | use PatternMatcher helper for validation (#1096) Improves performance in many cases. For more details, see Netflix/spectator#651. | 22 October 2019, 23:16:56 UTC |
| a8b6c21 | brharrington | 22 October 2019, 21:38:36 UTC | update dependencies (#1097) | 22 October 2019, 21:38:36 UTC |
| 5f7fd94 | Brian Harrington | 22 October 2019, 20:36:51 UTC | sbt-updates 0.5.0 | 22 October 2019, 20:36:51 UTC |
| dd51b8d | Brian Harrington | 22 October 2019, 20:36:09 UTC | sbt-release 1.0.12 | 22 October 2019, 20:36:09 UTC |
| 0e729f1 | Brian Harrington | 22 October 2019, 20:32:42 UTC | sbt 1.3.3 | 22 October 2019, 20:32:42 UTC |
| 8e45214 | Brian Harrington | 22 October 2019, 19:45:12 UTC | typesafe config 1.4.0 | 22 October 2019, 19:45:12 UTC |
| db1ae24 | Brian Harrington | 22 October 2019, 19:44:33 UTC | frigga 0.20.0 | 22 October 2019, 19:44:33 UTC |
| 81a079f | Brian Harrington | 22 October 2019, 19:43:37 UTC | equalsverifier 3.1.10 | 22 October 2019, 19:43:37 UTC |
| acac947 | Brian Harrington | 22 October 2019, 18:04:36 UTC | iep 2.1.4 | 22 October 2019, 18:04:36 UTC |
| 7e88be5 | Brian Harrington | 22 October 2019, 18:04:08 UTC | spectator 0.98.0 | 22 October 2019, 18:04:08 UTC |
| d3317f4 | Brian Harrington | 22 October 2019, 17:10:47 UTC | aws-java-sdk 1.11.656 | 22 October 2019, 17:10:47 UTC |
| 4c137bc | Brian Harrington | 22 October 2019, 17:09:43 UTC | akka-http 10.1.10 | 22 October 2019, 17:09:43 UTC |
| 88d99fe | brharrington | 22 October 2019, 16:34:33 UTC | add mapping for EC2 API call metrics (#1095) For more details see: https://docs.aws.amazon.com/AWSEC2/latest/APIReference/monitor.html | 22 October 2019, 16:34:33 UTC |
| a966472 | Brian Harrington | 22 October 2019, 16:30:03 UTC | akka 2.5.26 | 22 October 2019, 16:30:03 UTC |
| 9ef97e0 | brharrington | 02 October 2019, 20:35:39 UTC | jackson 2.10.0 (#1091) Update to latest jackson release. Should help resolve the frequent stream of CVEs related to default typing: https://medium.com/@cowtowncoder/jackson-2-10-features-cd880674d8a2 | 02 October 2019, 20:35:39 UTC |
| 67f6733 | brharrington | 02 October 2019, 13:53:46 UTC | update default set of `atlas.` reserved keys (#1090) Include `atlas.aggr` used for inline aggregation. Remove the `atlas.rollup` which was for an experimental rollup support that we ended up not doing. | 02 October 2019, 13:53:46 UTC |
| 4be162e | brharrington | 23 September 2019, 14:05:13 UTC | jackson 2.9.10 (#1089) Fixes serialization issue with case objects on scala 2.13. | 23 September 2019, 14:05:13 UTC |
