| 2887b4b | brharrington | 07 August 2020, 23:52:44 UTC | reduce allocations for asJavaMap.entrySet (#1197) Updates the `SmallHashMap.asJavaMap` implementation to override `entrySet()`. The new implementation avoids allocations for each item in the map. **Throughput** | Benchmark | Before | After | % Delta | |----------------------|---------------|---------------|---------| | customEntrySet | 8,400,934.3 | 13,480,190.3 | 60.5 | **Allocations** | Benchmark | Before | After | % Delta | |----------------------|---------------|---------------|---------| | customEntrySet | 296.0 | 24.0 | -91.9 | | 07 August 2020, 23:52:44 UTC |
| e5482ab | brharrington | 07 August 2020, 23:14:39 UTC | freeze should reset to initial variables (#1196) The freeze operation is meant to reset the state of the interpreter to isolate expressions. Before it was clearing the variables meaning that after a freeze operation any variables passed in at the start of the execution would not be available. This change fixes it to reset to the initial set of variables used when starting the execution. | 07 August 2020, 23:14:39 UTC |
| 32337f4 | brharrington | 06 August 2020, 18:56:31 UTC | add support for rewrite based on host name (#1195) In some cases we have multiple DNS names for the same backend, but the behavior varies a bit depending on the host name. Internally this has been done for quite a while on the main backends, but was not done for the streaming execution. This change adds support for host based rewrite that will be consistent between streaming and the main backends. | 06 August 2020, 18:56:31 UTC |
| 1c875e0 | brharrington | 06 August 2020, 16:29:19 UTC | remove unused SubscriptionManager (#1194) Changes in #1123 made this class unnecessary and removed most of the usage. | 06 August 2020, 16:29:19 UTC |
| 324434a | Tim Jiang | 05 August 2020, 14:49:32 UTC | eval: expose data rate per datasource (#1192) Adds diagnostic messages that can be used to determine which expressions are the most expensive in terms of data volume. This can be useful for balancing load and later will be used to help inform limits that can be enforced for the expressions. Co-authored-by: Brian Harrington <brharrington@gmail.com> | 05 August 2020, 14:49:32 UTC |
| 133bb59 | brharrington | 31 July 2020, 02:00:15 UTC | fix deprecation warnings for 2.13.3 (#1193) Updates the travis build to use 2.13.3 and fixes the new deprecation warnings that were introduced. | 31 July 2020, 02:00:15 UTC |
| 7b7e7ab | brharrington | 30 July 2020, 18:47:17 UTC | update dependencies (#1191) | 30 July 2020, 18:47:17 UTC |
| c29b07c | Brian Harrington | 30 July 2020, 18:31:13 UTC | sbt 1.3.13 | 30 July 2020, 18:31:13 UTC |
| 0afeaf4 | Brian Harrington | 30 July 2020, 18:29:48 UTC | aws-java-sdk 1.11.820 | 30 July 2020, 18:29:48 UTC |
| dc6c359 | Brian Harrington | 30 July 2020, 18:17:13 UTC | jackson 2.11.1 | 30 July 2020, 18:17:13 UTC |
| d77b163 | Brian Harrington | 30 July 2020, 18:12:21 UTC | roaringbitmap 0.9.1 | 30 July 2020, 18:12:21 UTC |
| 19a4630 | Brian Harrington | 30 July 2020, 18:11:52 UTC | jol-core 0.11 | 30 July 2020, 18:11:52 UTC |
| 553ef40 | Brian Harrington | 30 July 2020, 17:55:46 UTC | equalsverifier 3.4.1 | 30 July 2020, 17:55:46 UTC |
| dac2560 | Brian Harrington | 30 July 2020, 17:52:20 UTC | caffeine 2.8.5 | 30 July 2020, 17:52:20 UTC |
| be26197 | Brian Harrington | 30 July 2020, 17:50:34 UTC | scala 2.13.3 | 30 July 2020, 17:50:34 UTC |
| 78064d9 | Brian Harrington | 30 July 2020, 17:46:49 UTC | iep 2.5.5 | 30 July 2020, 17:46:49 UTC |
| 05ec5e6 | Brian Harrington | 30 July 2020, 17:44:27 UTC | spectator 0.111.0 | 30 July 2020, 17:44:27 UTC |
| 498a204 | Brian Harrington | 30 July 2020, 17:37:48 UTC | scalatest 3.2.0 | 30 July 2020, 17:37:48 UTC |
| 1a95b60 | brharrington | 29 July 2020, 16:35:18 UTC | make datetime zone parsing more lenient (#1190) In particular, it will now support zone offsets that do not have the colons to make integration easier for some JS use case. | 29 July 2020, 16:35:18 UTC |
| 94edf54 | Tim Jiang | 16 July 2020, 23:36:07 UTC | rename root project to atlas to be more IDE friendly (#1189) Makes it easier to spot in the recently used menu. | 16 July 2020, 23:36:07 UTC |
| b9d1044 | brharrington | 29 June 2020, 17:25:15 UTC | remove duplicate mapping for GetRecords.Bytes (#1188) This causes it to get double published which can cause minor differences depending on the order when the values are normalized. | 29 June 2020, 17:25:15 UTC |
| 659e4ac | brharrington | 25 June 2020, 16:38:25 UTC | allow arbitrary number of days for larger step (#1185) It doesn't matter as much for these larger ranges and this can sometimes be useful for longer term graphs. Before it would only allow 1d and 7d. Removes the validation since it is no longer a fixed set. | 25 June 2020, 16:38:25 UTC |
| 8b2931c | brharrington | 16 June 2020, 19:24:39 UTC | add support for dstype of sum (#1181) This should be used carefully as it could cause overcounting. The primary use-case is for inline aggregation of values that are already normalized to the step size. For example, the aggregator cluster could use this to avoid a multiple on counters that are flowing through. | 16 June 2020, 19:24:39 UTC |
| b8154a4 | brharrington | 15 June 2020, 22:08:39 UTC | add mapping for CloudWatch IoT metrics (#1180) Adds initial mapping. Omits a few of the metrics where we do not have any samples to test with current usage. | 15 June 2020, 22:08:39 UTC |
| 80d40d1 | brharrington | 05 June 2020, 18:28:34 UTC | SmallHashMap: avoid BoxesRunTime.equals (#1179) It can be quite a bit slower if not inlined. | 05 June 2020, 18:28:34 UTC |
| ffd7f74 | brharrington | 02 June 2020, 17:36:05 UTC | update dependencies (#1178) | 02 June 2020, 17:36:05 UTC |
| dc31c99 | Brian Harrington | 02 June 2020, 17:27:19 UTC | sbt 1.3.12 | 02 June 2020, 17:27:19 UTC |
| 09afda1 | Brian Harrington | 02 June 2020, 17:09:50 UTC | roaringbitmap 0.8.19 | 02 June 2020, 17:09:50 UTC |
| 8920d40 | Brian Harrington | 02 June 2020, 17:08:53 UTC | caffeine 2.8.4 | 02 June 2020, 17:08:53 UTC |
| 391d03b | Brian Harrington | 02 June 2020, 17:00:42 UTC | equalsverifier 3.3 | 02 June 2020, 17:00:42 UTC |
| a850fe4 | Brian Harrington | 02 June 2020, 16:59:56 UTC | akka-http 10.1.12 | 02 June 2020, 16:59:56 UTC |
| 01f8f13 | Brian Harrington | 02 June 2020, 16:49:55 UTC | log4j 2.13.3 | 02 June 2020, 16:49:55 UTC |
| eff49b8 | Brian Harrington | 02 June 2020, 16:43:28 UTC | iep 2.5.3 | 02 June 2020, 16:43:28 UTC |
| b10f2ca | Brian Harrington | 02 June 2020, 16:29:07 UTC | aws-java-sdk 1.11.790 | 02 June 2020, 16:29:07 UTC |
| 300f38e | Brian Harrington | 02 June 2020, 16:19:10 UTC | scalatest 3.1.2 | 02 June 2020, 16:19:10 UTC |
| 6393a78 | Brian Harrington | 02 June 2020, 16:17:52 UTC | spectator 0.108.0 | 02 June 2020, 16:17:52 UTC |
| db11f21 | brharrington | 30 May 2020, 01:54:18 UTC | switch to scala.util.Using (#1177) Now that we are fully on scala 2.13, deprecate the `Streams.scope` utility function in favor of the `Using` helper in the stdlib. All uses in this code base have been migrated. | 30 May 2020, 01:54:18 UTC |
| 4d251e8 | brharrington | 29 May 2020, 23:02:03 UTC | update Streams.scope to work with AutoCloseable (#1176) This is the base interface for Closeable so it is more generic than it was before. Allows this helper to be used with Stream's that need to be closed such as those returned by `Files.list`. | 29 May 2020, 23:02:03 UTC |
| c4eeb2a | brharrington | 21 May 2020, 01:31:03 UTC | tune ByteStringInputStream for composites (#1175) Adjusts the ByteStringInputStream to work better for composite ByteString objects. Before it was using `asByteBuffer` which forces a compaction on composite types. Now it uses the `asByteBuffers` method. This results in a large reduction in allocations and increase in throughput. **Throughput** | Benchmark | Before | After | % Delta | |----------------------|---------------|---------------|---------| | compositeInputStream | 31.8 | 137.6 | 332.9 | | compositeToArray | 44.6 | 46.2 | 3.5 | | inputStream | 115.1 | 126.5 | 9.9 | | toArray | 42.8 | 45.0 | 5.2 | **Allocations** | Benchmark | Before | After | % Delta | |----------------------|---------------|---------------|---------| | compositeInputStream | 104,887,113.5 | 69,769.8 | -99.9 | | compositeToArray | 104,875,451.8 | 104,875,038.6 | -0.0 | | inputStream | 8,418.3 | 7,987.6 | -5.1 | | toArray | 104,874,398.1 | 104,874,682.8 | 0.0 | | 21 May 2020, 01:31:03 UTC |
| f096a18 | brharrington | 05 May 2020, 16:15:14 UTC | use jdk14 for travis build (#1174) OpenJDK14 has changes to font rendering that make it compatible with the lastest Zulu JDK11 we use internally. | 05 May 2020, 16:15:14 UTC |
| 6b426a2 | brharrington | 04 May 2020, 15:39:02 UTC | update version in wiki to 1.6.5 (#1173) Bump the version in the wiki to the latest. | 04 May 2020, 15:39:02 UTC |
| 6316c0c | brharrington | 04 May 2020, 14:33:16 UTC | fix compiler warnings for 2.13.2 (#1170) Most of the new warnings were for widening conversions from long to double. Since cross-building has been disabled, all warnings have been cleared up with no issues. Enabled fatal warnings to help keep the build clean. | 04 May 2020, 14:33:16 UTC |
| 7f10ff2 | brharrington | 01 May 2020, 22:48:06 UTC | enable WRITE_NAN_AS_STRINGS by default (#1169) Historically some endpoints wrote the special double values without quotes to make it clear they were numeric. However, that creates problems with tools using standard JSON parsing libraries. New uses should always emit standard JSON so this change updates the default so these values are quoted. The encoding for the legacy `json` format on the graph API still emits data without the quotes for backwards compatibility. | 01 May 2020, 22:48:06 UTC |
| 404168d | brharrington | 01 May 2020, 22:44:15 UTC | remove atlas-config sub project (#1168) Switch the code to consistently use the IEP library that is used for other projects as well. Helps to simplify the overall config story. | 01 May 2020, 22:44:15 UTC |
| d831f19 | brharrington | 01 May 2020, 22:34:54 UTC | fix memory.conf to include from classpath (#1167) Before depending on how the path for the `memory.conf` file was specified, it would try to load from the file system instead of the classpath and the base config would be missing. Now it is explicit about including from the classpath. Fixes #1146. | 01 May 2020, 22:34:54 UTC |
| ab10e03 | brharrington | 01 May 2020, 16:42:36 UTC | update dependencies (#1166) | 01 May 2020, 16:42:36 UTC |
| e5c2bcc | Brian Harrington | 01 May 2020, 16:23:29 UTC | sbt 1.3.10 | 01 May 2020, 16:23:29 UTC |
| 15d73d3 | Brian Harrington | 01 May 2020, 16:15:31 UTC | roaringbitmap 0.8.18 | 01 May 2020, 16:15:31 UTC |
| 57b65a0 | Brian Harrington | 01 May 2020, 16:13:54 UTC | caffeine 2.8.2 | 01 May 2020, 16:13:54 UTC |
| 15e4119 | Brian Harrington | 01 May 2020, 16:12:34 UTC | spectator 0.107.0 | 01 May 2020, 16:12:34 UTC |
| 38baf3d | Brian Harrington | 01 May 2020, 16:11:22 UTC | scala-collection-compat 2.1.6 | 01 May 2020, 16:11:22 UTC |
| 7a4acd8 | Brian Harrington | 01 May 2020, 16:10:37 UTC | scala 2.13.2 | 01 May 2020, 16:10:37 UTC |
| cff9822 | Brian Harrington | 01 May 2020, 15:56:23 UTC | jackson 2.11.0 | 01 May 2020, 15:56:23 UTC |
| 7956d22 | Brian Harrington | 01 May 2020, 15:54:55 UTC | iep 2.5.0 | 01 May 2020, 15:54:55 UTC |
| 4805064 | Brian Harrington | 01 May 2020, 15:54:06 UTC | aws-java-sdk 1.11.764 | 01 May 2020, 15:54:06 UTC |
| 8a41771 | Tim Jiang | 02 April 2020, 22:14:21 UTC | atlas-eval: fix eval flow failure on lwc instance removal (#1165) Currently the whole eval flow may fail when a lwc instance is removed. There's a subflow for lwc api call of each lwc instance, and that subflow is wrapped around by "RestartFlow" to enable auto-retry. The subflow is finished from upstream when a lwc instance is removed, and upstream of RestartFlow is finished, RestartFlow finishes and propagate the failure from the current ongoing attempt if any, to downstream, and eventually propagate to main flow. The fix is to have downstream of RestartFlow suppress the failure, since the failure is expected and can be ignored on lwc instance removal. | 02 April 2020, 22:14:21 UTC |
| e4f30ec | brharrington | 02 April 2020, 21:31:07 UTC | update dependencies (#1164) | 02 April 2020, 21:31:07 UTC |
| 58c34b4 | Brian Harrington | 02 April 2020, 20:04:05 UTC | sbt 1.3.9 | 02 April 2020, 20:04:05 UTC |
| 9ca8d51 | Brian Harrington | 02 April 2020, 19:55:43 UTC | log4j 2.13.1 | 02 April 2020, 19:55:43 UTC |
| 7bf1b92 | Brian Harrington | 02 April 2020, 19:55:09 UTC | frigga 0.24.0 | 02 April 2020, 19:55:09 UTC |
| 99cfe84 | Brian Harrington | 02 April 2020, 19:54:19 UTC | jackson 2.10.3 | 02 April 2020, 19:54:19 UTC |
| 695f6f2 | Brian Harrington | 02 April 2020, 19:47:55 UTC | roaringbitmap 0.8.13 | 02 April 2020, 19:47:55 UTC |
| 8cd9a02 | Brian Harrington | 02 April 2020, 19:47:16 UTC | equalsverifier 3.1.13 | 02 April 2020, 19:47:16 UTC |
| 3b56517 | Brian Harrington | 02 April 2020, 19:39:37 UTC | scalatest 3.1.1 | 02 April 2020, 19:39:37 UTC |
| 17e2fcd | Brian Harrington | 02 April 2020, 19:31:41 UTC | spectator 0.104.0 | 02 April 2020, 19:31:41 UTC |
| 4befafc | Brian Harrington | 02 April 2020, 19:30:14 UTC | iep 2.2.2 | 02 April 2020, 19:30:14 UTC |
| 2ee71a5 | Brian Harrington | 02 April 2020, 19:29:35 UTC | aws-java-sdk 1.11.752 | 02 April 2020, 19:29:35 UTC |
| b22cade | Brian Harrington | 02 April 2020, 19:27:18 UTC | akka 2.5.31 | 02 April 2020, 19:27:18 UTC |
| 70a2ad5 | brharrington | 02 April 2020, 18:17:55 UTC | remove -Xexperimental flag (#1163) Not needed now that we are on 2.13 only. Clears up warnings during the build. | 02 April 2020, 18:17:55 UTC |
| 3e1aa52 | brharrington | 27 March 2020, 22:51:53 UTC | fix logic for skipNext (#1162) The `skipNext` helper did not work correctly in the case of some complex objects with nested arrays. This change fixes the logic and adds more extensive testing with randomly generated JSON objects. | 27 March 2020, 22:51:53 UTC |
| 047ff39 | brharrington | 27 March 2020, 01:02:25 UTC | add helper for getting intValue from ItemId (#1161) This method is compatible with what you would get by calling `toBigInteger.intValue` but avoids the overhead of creating an intermediate BigInteger object if that is the only use. | 27 March 2020, 01:02:25 UTC |
| 1fd9473 | brharrington | 22 March 2020, 14:55:30 UTC | cache hash code for Query (#1160) This get computed a lot when leveraging a query index. In some cases it is over 5% of the time. Current JMH benchmark shows a similar improvement after caching. | Benchmark | Before | After | % Delta | |----------------------|---------------|---------------|---------| | index_100 | 2,950,658.3 | 3,032,810.1 | 2.8 | | index_1000 | 2,072,328.3 | 2,122,367.3 | 2.4 | | index_10000 | 2,527,147.8 | 2,714,280.1 | 7.4 | | index_100000 | 2,028,644.4 | 2,152,331.2 | 6.1 | | 22 March 2020, 14:55:30 UTC |
| 264dbb1 | brharrington | 22 March 2020, 13:29:54 UTC | switch to foreachEntry iterating maps (#1159) This method is more efficient in many cases and avoids temporary tuple objects. Now that we do not need to worry about 2.12 compatibility, switch to using it where possible. | 22 March 2020, 13:29:54 UTC |
| cff5442 | brharrington | 21 March 2020, 22:51:08 UTC | avoid intermediate tuple allocations (#1156) Update QueryIndex matching methods to use foreachEntry to avoid temporary tuples. | 21 March 2020, 22:51:08 UTC |
| 91712c0 | brharrington | 21 March 2020, 21:28:34 UTC | stop cross building for scala 2.12 (#1158) Internally all uses of these have moved to scala 2.13. To reduce maintenance effort, drop 2.12 builds. | 21 March 2020, 21:28:34 UTC |
| 2167785 | brharrington | 20 March 2020, 23:01:37 UTC | switch to foreachEntry for SmallHashMap (#1157) This method is available for maps in scala 2.13.0. This makes the custom SmallHashMap more inline with the scala maps. | 20 March 2020, 23:01:37 UTC |
| 9731b72 | brharrington | 19 March 2020, 21:56:04 UTC | SmallHashMap: avoid lambda overhead for add/remove (#1155) Loop over the array directly to avoid the lambda overhead for adding or removing an item. This can be a hot path for some use-cases. | 19 March 2020, 21:56:04 UTC |
| 832e827 | Tim Jiang | 18 March 2020, 12:12:09 UTC | atlas-eval: expose flow api to maximize throughput (#1154) Converting to Processor disables operator fusion at the boundary, and reduces overall throughput. This change exposes the Flow directly and enables operator fusion optimization, and downstream delay of the boundary reduced from hundreds of microseconds to tens of microseconds. | 18 March 2020, 12:12:09 UTC |
| 3aa9acc | Tim Jiang | 12 March 2020, 15:06:02 UTC | atlas-eval: lower eval input queue-size (#1153) It is for the incoming set of data sources which should not change at a high rate. The most recent can be used. | 12 March 2020, 15:06:02 UTC |
| be8c60d | brharrington | 11 March 2020, 17:00:35 UTC | disable trusted advisor collection by default (#1152) The trusted advisor data in cloudwatch is only updated when something refreshes the checks. Unless something is doing that, there isn't much point in collecting the cloudwatch data. | 11 March 2020, 17:00:35 UTC |
| 2bc772b | Tim Jiang | 11 March 2020, 12:32:08 UTC | atlas-eval: avoid back pressure of datasources (#1151) Currently in eval stream, "Datasources broadcast" is getting back-pressured by FinalExprEval under heavy load and new DataSources can not flow to LWC API calls and causes out of date subscriptions. This change avoids back pressure at sub streams of Datasources broadcast, so that they will not block each other. Only keeping the latest DataSources is fine for both sub streams. | 11 March 2020, 12:32:08 UTC |
| 1421394 | Tim Jiang | 09 March 2020, 12:31:33 UTC | atlas-eval: make queue size configurable for lwc call (#1150) Allows the queue size for members of the cluster to be configured. | 09 March 2020, 12:31:33 UTC |
| 2ce55f0 | brharrington | 04 March 2020, 23:10:12 UTC | cache the hash code for DataExprs (#1149) For streaming evaluation the DataExpr is the key in the time grouping stage. The largest expense in the time grouping stage was computing the hash code of the DataExpr to aggregate the datapoints as they were flowing through. | 04 March 2020, 23:10:12 UTC |
| 55cc0c1 | brharrington | 04 March 2020, 21:11:31 UTC | avoid unnecessary string conversion (#1148) When parsing the LWC messages converting to a string first is unnecessary. Avoiding this results in about 30% improvement in the throughput. | 04 March 2020, 21:11:31 UTC |
| 2fd4343 | brharrington | 03 March 2020, 22:01:33 UTC | optimize parsing of LWC messages (#1147) For some recent tests around 9% of the CPU time was spent parsing these messages when under heavy load. Using the JMH benchmark this provides a little over 2x faster parsing and similar reduction in allocations. | 03 March 2020, 22:01:33 UTC |
| 22292db | brharrington | 19 February 2020, 13:15:58 UTC | update dependencies (#1145) | 19 February 2020, 13:15:58 UTC |
| 7516ea6 | Brian Harrington | 19 February 2020, 13:05:15 UTC | sbt-scalafmt 2.3.1 | 19 February 2020, 13:05:15 UTC |
| d87f77a | Brian Harrington | 19 February 2020, 13:04:31 UTC | sbt-release 1.0.13 | 19 February 2020, 13:04:31 UTC |
| 250ed36 | Brian Harrington | 19 February 2020, 12:56:53 UTC | sbt 1.3.8 | 19 February 2020, 12:56:53 UTC |
| e4d1a51 | Brian Harrington | 19 February 2020, 12:53:34 UTC | jol-core 0.10 | 19 February 2020, 12:53:34 UTC |
| 7d1cc3d | Brian Harrington | 19 February 2020, 12:45:22 UTC | equalsverifier 3.1.12 | 19 February 2020, 12:45:22 UTC |
| b2088f0 | Brian Harrington | 19 February 2020, 12:43:48 UTC | aws-java-sdk 1.11.723 | 19 February 2020, 12:43:48 UTC |
| 2660ab2 | Brian Harrington | 19 February 2020, 12:43:19 UTC | iep 2.2.1 | 19 February 2020, 12:43:19 UTC |
| e28ef69 | Brian Harrington | 19 February 2020, 12:32:40 UTC | akka 2.5.29 | 19 February 2020, 12:32:40 UTC |
| 3175e29 | Brian Harrington | 19 February 2020, 12:31:44 UTC | scala-collection-compat 2.1.4 | 19 February 2020, 12:31:44 UTC |
| e096652 | Brian Harrington | 19 February 2020, 12:29:28 UTC | spectator 0.103.0 | 19 February 2020, 12:29:28 UTC |
| 1ee176f | brharrington | 11 February 2020, 17:22:15 UTC | adjust default serialization for java.time (#1144) Set the default serialization to use millisecond timestamps. This keeps it more consistent with other usage in Atlas. | 11 February 2020, 17:22:15 UTC |
| 0d5876d | brharrington | 06 February 2020, 13:41:36 UTC | update launcher url to use HTTPS (#1141) Bintray now requires HTTPS for downloading the artifacts. Also bump to the latest version. | 06 February 2020, 13:41:36 UTC |
| 45ffa4b | brharrington | 06 February 2020, 12:58:12 UTC | update settings in memory.conf (#1139) Update the memory.conf example to reflect move of graphing logic to eval library. Fixes #1135 | 06 February 2020, 12:58:12 UTC |
| 0df4e41 | Tim Jiang | 03 February 2020, 20:21:19 UTC | emit empty DataSources for a host if no more DataSource (#1137) Emit empty DataSources for a host if no more DataSource so that downstream can stop stream from LWC quickly for this host. This is in preparation for a single evaluator instance on the atlas-stream service that is part of iep-apps. | 03 February 2020, 20:21:19 UTC |
