| 82b412f | brharrington | 14 November 2022, 16:58:34 UTC | lwcapi: only update subscription list on changes (#1491) Creating the full list of of all subscriptions can be expensive when there are a lot of expressions. Update it to only be done when there are changes the same as the query index update. | 14 November 2022, 16:58:34 UTC |
| e8b9b49 | brharrington | 13 November 2022, 15:28:35 UTC | lwc: keep input to TimeGrouped batched (#1490) Refactors the stream from parsing the messages from lwcapi to the TimeGrouped stage to keep the datapoints batched. This significantly reduces the number of messages passed between the stream stages and achieve a higher throughput. Also creates a dedicated thread pool to use for parsing the messages as that is the most time consuming step. | 13 November 2022, 15:28:35 UTC |
| 358aef3 | brharrington | 12 November 2022, 20:37:44 UTC | lwc: remove some monitorFlow steps (#1489) When processing data with a high input volume the `ConnectionSources`, `InputLines`, and `LwcDatapoints` monitors would have a high throughput and the same number of elements flowing through. This change removes two of them so there is now just `InputLines`. This helps reduce overhead in the hot path. | 12 November 2022, 20:37:44 UTC |
| d68272f | brharrington | 12 November 2022, 00:32:23 UTC | lwc: use SortedTagMap for data points (#1488) The main advantage is that for eval it can just quickly copy into an array without all the hashing and extra space in the hash array to avoid collisions. Note, this change is safe to roll out as long as the lwcapi service is pushed first. | 12 November 2022, 00:32:23 UTC |
| 23bc96d | brharrington | 11 November 2022, 20:04:19 UTC | switch to spectator batch updaters (#1487) Use the spectator batch updaters in some of the key parts of the eval stream that were manually batched before. | 11 November 2022, 20:04:19 UTC |
| 55485bd | brharrington | 10 November 2022, 19:37:28 UTC | batch meter updates for TimeGrouped (#1486) Batch the counter updates in time grouped to avoid updating them in a tight loop for high throughput streams. | 10 November 2022, 19:37:28 UTC |
| 2374aff | brharrington | 10 November 2022, 19:33:22 UTC | batch updates to meters when monitoring flows (#1485) Updates the monitor flow stage to batch the updates to the meters. This avoids some of the overhead for calling gettimeofday frequently for high volume streams. | 10 November 2022, 19:33:22 UTC |
| 3e935c1 | brharrington | 10 November 2022, 02:15:48 UTC | only update index for new items (#1484) The updates for the existing items will get filtered out later, but we can avoid adding them in the first place to reduce the overhead a little bit. | 10 November 2022, 02:15:48 UTC |
| 70565a0 | brharrington | 10 November 2022, 00:12:58 UTC | remove usage of javax annotations (#1482) These have been moved to jakarta in later versions. With the way we are using Spring, the annotations are not necessary. | 10 November 2022, 00:12:58 UTC |
| a9d0d9b | brharrington | 09 November 2022, 19:36:18 UTC | json: enable fast double operations (#1483) Enable the fast double reading/writing that was added in jackson 2.14.0. | 09 November 2022, 19:36:18 UTC |
| 4bcee87 | brharrington | 09 November 2022, 15:49:58 UTC | update dependencies (#1481) | 09 November 2022, 15:49:58 UTC |
| 6ef6a5e | Brian Harrington | 09 November 2022, 15:28:55 UTC | RoaringBitmap 0.9.35 | 09 November 2022, 15:28:55 UTC |
| 6711fb4 | Brian Harrington | 09 November 2022, 15:28:03 UTC | equalsverifier 3.11 | 09 November 2022, 15:28:03 UTC |
| 5023a1d | Brian Harrington | 09 November 2022, 15:27:35 UTC | jackson 2.14.0 | 09 November 2022, 15:27:35 UTC |
| 9b1ec8f | Brian Harrington | 09 November 2022, 15:26:54 UTC | iep 4.0.2 | 09 November 2022, 15:26:54 UTC |
| 679aa21 | Brian Harrington | 09 November 2022, 15:26:29 UTC | spectator 1.3.10 | 09 November 2022, 15:26:29 UTC |
| e2dc326 | Brian Harrington | 09 November 2022, 15:24:40 UTC | sbt 1.7.3 | 09 November 2022, 15:24:40 UTC |
| 6b11ea0 | brharrington | 08 November 2022, 19:19:48 UTC | improve performance of batch index update (#1479) A lot of the time and allocations were being spent sorting the items passed to the index. Java uses TimSort internally when sorting object arrays and that uses a merge sort. It allocates a bunch of temporary arrays when performing the merges. This change leverages the fact that the items that are already indexed will be sorted, so we can just sort the new items and merge the two sorted sets. | 08 November 2022, 19:19:48 UTC |
| 397cdcf | brharrington | 08 November 2022, 19:15:09 UTC | add expression to v2.json output (#1480) Update the `v2.json` format to include the query expression for the lines. This makes it more consistent with the time series messages used with fetch and LWC. | 08 November 2022, 19:15:09 UTC |
| 71098c9 | brharrington | 07 November 2022, 22:06:11 UTC | json: fix double parsing for large integers (#1478) Before when using `JsonParserHelper.nextDouble` if the raw encoded value looked like an integer type it would try to access it as a long, then convert the long to a double. This causes problems for larger integers that cannot fit within the bounds of a Java long. Now it will access it as a double. | 07 November 2022, 22:06:11 UTC |
| 3143e37 | brharrington | 28 October 2022, 22:44:34 UTC | use custom JSON encoding if specified (#1477) If a class extended JsonSupport and provided a custom encoding it would get used by the interited methods, but would get ignored when using `Json.encode`. This could create problems when an object is nested because the custom encode would not get used. The `Json.encode` could not directly call encode, because it would create a loop with the default implementation. This change adds a method that can indicate if it is desirable for `Json.ecnode` to use the custom encoding. | 28 October 2022, 22:44:34 UTC |
| 6096696 | brharrington | 27 October 2022, 20:22:11 UTC | allow image to be disabled for v2.json (#1476) The image rendering can be computationally expensive and increase the size of the payload quite a bit. For some cases it is desirable to use the `v2.json` format since it is similar to fetch and LWC, but the image is not needed. Alert processing is one example. This change adds a `hints` URI parameter that can be used to pass arbitrary hints back to the rendering engine that is used. The `v2.json` engine has been updated to omit the image if a `no-image` hint is passed. In the future we may want to only include the image if requested, but that would be backwards incompatible and break some current usage. | 27 October 2022, 20:22:11 UTC |
| 3205dfb | brharrington | 27 October 2022, 17:20:17 UTC | improve legend for custom avg with nested grouping (#1475) Updates the simple legends generation to handle the case of custom averages used with nested grouping. Before the logic to remove the named rewrite would also remove the first grouping and break the rewrite of any later groupings. | 27 October 2022, 17:20:17 UTC |
| b9f4d8c | brharrington | 25 October 2022, 21:45:16 UTC | update normalize behavior for UI (#1474) Updates the behavior of normalize to reflect preferences of the UI team. In particular: - It will not try to extract common query aspects into a `:cq` clause. - The helpers for filtering based on summary stats will be preserved rather than expanding. - Adjust handling of OR clauses in a query. First convert to DNF and then sort the CNF clauses to avoid awkward expansion with repeated terms. | 25 October 2022, 21:45:16 UTC |
| 72cadea | brharrington | 13 October 2022, 13:29:11 UTC | update dependencies (#1472) | 13 October 2022, 13:29:11 UTC |
| d0671b6 | Brian Harrington | 13 October 2022, 13:15:32 UTC | jackson 2.13.4 | 13 October 2022, 13:15:32 UTC |
| ead8118 | Brian Harrington | 13 October 2022, 13:12:50 UTC | log4j 2.19.0 | 13 October 2022, 13:12:50 UTC |
| ca9db8f | Brian Harrington | 13 October 2022, 13:12:11 UTC | RoaringBitmap 0.9.32 | 13 October 2022, 13:12:11 UTC |
| 6f43714 | Brian Harrington | 13 October 2022, 13:11:09 UTC | spring 5.3.23 | 13 October 2022, 13:11:09 UTC |
| 37e7b5d | Brian Harrington | 13 October 2022, 13:09:33 UTC | iep 4.0.1 | 13 October 2022, 13:09:33 UTC |
| e49670b | Brian Harrington | 13 October 2022, 13:09:12 UTC | spectator 1.3.9 | 13 October 2022, 13:09:12 UTC |
| bc13fc2 | Brian Harrington | 13 October 2022, 13:07:37 UTC | sbt-updates 0.6.4 | 13 October 2022, 13:07:37 UTC |
| 8972da7 | Brian Harrington | 13 October 2022, 12:59:05 UTC | add jdk19 to CI | 13 October 2022, 12:59:05 UTC |
| 2517c9a | Brian Harrington | 13 October 2022, 12:58:02 UTC | sbt 1.7.2 | 13 October 2022, 12:58:02 UTC |
| 9deca0c | Brian Harrington | 13 October 2022, 12:56:42 UTC | scala 2.13.10 | 13 October 2022, 12:56:42 UTC |
| c24f0ae | Brian Harrington | 13 October 2022, 11:46:10 UTC | scalafmt 3.5.9 Tries to update the config to be close to the previous and minimize unnecessary diffs. | 13 October 2022, 11:46:10 UTC |
| 8044337 | Chris Larsen | 27 September 2022, 19:48:23 UTC | logging queries.add `:starts` and `:ends` for log queries (#1471) Adds support for `:starts` and `:ends` which are frequently used with some log queries internally. | 27 September 2022, 19:48:23 UTC |
| bcf651d | Chris Larsen | 27 September 2022, 19:19:07 UTC | Add `:contains` predicate for compatibility with log queries (#1470) Adds support for `:contains` which is frequently used with some log queries internally. Bump Spectator to 1.3.8 | 27 September 2022, 19:19:07 UTC |
| 07cb339 | brharrington | 23 September 2022, 16:17:30 UTC | optimize result tags computation (#1469) A long time ago the tag set was made deterministic based on the query expression where it only includes tags with an exact match or that are listed in the group by. As a result, it isn't necessary to compute the intersection of dimensions while aggregating the matching time series. | 23 September 2022, 16:17:30 UTC |
| 7fb6b14 | brharrington | 22 September 2022, 14:55:56 UTC | make default dstype consistent (#1468) This change makes the default dstype consistently rate for both apply methods of DsType as well as matching the behavior for the publish path when feeding data into the normalization cache. | 22 September 2022, 14:55:56 UTC |
| 42c1f2f | brharrington | 22 September 2022, 11:38:42 UTC | adjust consolidation behavior for rates (#1467) Port an old internal fix to ensure we have consistent behavior. If the data source type is a rate, then treat NaN values as 0 as it means no activity measured for those intervals when consolidating. That means the avg will always be computed over the entire consolidated interval. For gauges, the NaN values are ignored because to be consistent with values reported to the gauge. The behavior of gauges is not impacted with this change. | 22 September 2022, 11:38:42 UTC |
| 89d799c | brharrington | 22 September 2022, 01:18:30 UTC | update CollectorStatsBuilder to allow chaining (#1466) Return a reference to `this` from the builder methods so that they can be chained. | 22 September 2022, 01:18:30 UTC |
| 11cae07 | brharrington | 09 September 2022, 19:16:56 UTC | improve couldMatch check for not clauses (#1465) If the inverted query has an exact match, then adding dimensions cannot change the result and it should be excluded. | 09 September 2022, 19:16:56 UTC |
| 64627ef | brharrington | 09 September 2022, 16:43:19 UTC | add note about akka versions (#1464) Update to latest ASL licensed Akka and Akka-HTTP and add comment about license change. | 09 September 2022, 16:43:19 UTC |
| de163ee | brharrington | 08 September 2022, 18:26:52 UTC | avoid calling sort for SortedTagMap (#1463) When computing the id from a tag map, avoid making the sort call when a SortedTagMap is used. That type will already be sorted so it is not necessary. | 08 September 2022, 18:26:52 UTC |
| fa16821 | brharrington | 08 September 2022, 18:16:40 UTC | reduce allocations for ItemId.toString (#1462) Initialize capacity for the builder to avoid a resize while filling the buffer. Reduces allocations for the sample benchmark from 440 B/op to 216 B/op. | 08 September 2022, 18:16:40 UTC |
| 18a2c6d | brharrington | 01 September 2022, 16:09:51 UTC | fix typo in config method name (#1460) | 01 September 2022, 16:09:51 UTC |
| df8bb16 | brharrington | 01 September 2022, 16:09:20 UTC | update dependencies (#1461) | 01 September 2022, 16:09:20 UTC |
| 0a9caff | Brian Harrington | 01 September 2022, 15:50:28 UTC | iep 4.0.0 | 01 September 2022, 15:59:38 UTC |
| 8145a1d | Brian Harrington | 01 September 2022, 15:42:53 UTC | embedded-postgres 2.0.1 | 01 September 2022, 15:42:53 UTC |
| f3b249f | Brian Harrington | 01 September 2022, 15:42:15 UTC | postgresql 42.5.0 | 01 September 2022, 15:42:15 UTC |
| 62a5cc7 | Brian Harrington | 01 September 2022, 15:41:00 UTC | spectator 1.3.7 | 01 September 2022, 15:41:00 UTC |
| 40ab7ff | brharrington | 28 August 2022, 15:44:11 UTC | switch to using Spring for DI (#1455) Since Netflix is moving from Guice to Spring, switch the DI framework to use Spring. This is mostly just migrating the Guice modules. | 28 August 2022, 15:44:11 UTC |
| 5a547c8 | brharrington | 08 August 2022, 22:30:35 UTC | update dependencies (#1453) | 08 August 2022, 22:30:35 UTC |
| 2f07860 | Brian Harrington | 08 August 2022, 22:22:28 UTC | log4j 2.18.0 | 08 August 2022, 22:22:28 UTC |
| 268bc50 | Brian Harrington | 08 August 2022, 22:21:38 UTC | RoaringBitmap 0.9.31 | 08 August 2022, 22:21:38 UTC |
| 3799059 | Brian Harrington | 08 August 2022, 22:20:45 UTC | datasketches-java 3.3.0 | 08 August 2022, 22:20:45 UTC |
| 68157dc | Brian Harrington | 08 August 2022, 22:19:54 UTC | postgresql 42.4.1 | 08 August 2022, 22:19:54 UTC |
| 3eb1be1 | Brian Harrington | 08 August 2022, 22:19:09 UTC | equalsverifier 3.10.1 | 08 August 2022, 22:19:09 UTC |
| 38a20cb | Brian Harrington | 08 August 2022, 22:18:13 UTC | scala-collection-compat 2.8.1 | 08 August 2022, 22:18:13 UTC |
| 7a5f503 | Brian Harrington | 08 August 2022, 22:17:01 UTC | iep 3.0.21 | 08 August 2022, 22:17:01 UTC |
| 4b166b3 | Brian Harrington | 08 August 2022, 22:16:05 UTC | spectator 1.3.6 | 08 August 2022, 22:16:05 UTC |
| 1239586 | brharrington | 02 August 2022, 17:03:01 UTC | fix shard lookup for Integer.MIN_VALUE (#1452) Before it was relying on `abs` to get a non-negative value for computing an index to the array. This breaks when the int value of the id is `Integer.MIN_VALUE`. Update it to explicitly clear the sign bit to ensure the value will always be non-negative. | 02 August 2022, 17:03:01 UTC |
| 483dfc5 | brharrington | 22 July 2022, 16:59:45 UTC | lwcapi: remove the /v1/subscribe endpoint (#1451) This was an earlier version that relied used unbatched text messages for passing the data and was much less efficient. It is no longer in use internally. | 22 July 2022, 16:59:45 UTC |
| 7b02e34 | sullis | 12 July 2022, 16:40:29 UTC | sbt 1.7.1 (#1450) https://github.com/sbt/sbt/releases/tag/v1.7.1 | 12 July 2022, 16:40:29 UTC |
| 20042fc | sullis | 11 July 2022, 10:10:56 UTC | sbt 1.7.0 (#1449) https://github.com/sbt/sbt/releases/tag/v1.7.0 | 11 July 2022, 10:10:56 UTC |
| a850013 | Chris Larsen | 29 June 2022, 12:36:02 UTC | lwcapi: ensure failed clients are unregistered (#1448) Move the termination watch to the root flow in order to catch improperly terminated clients. Also adds a note about dealing with clients that connect and fail to send a message wherein the queue shuts down after a timeout period and fails to notify the rest of the flow. | 29 June 2022, 12:36:02 UTC |
| 852b05e | brharrington | 17 June 2022, 16:56:31 UTC | avoid boxing when parsing special double values (#1447) For special double values, e.g. `NaN`, use `parseDouble` instead of `valueOf` to avoid allocating an instance of the primitive wrapper type. | 17 June 2022, 16:56:31 UTC |
| c0a9919 | brharrington | 02 June 2022, 17:31:55 UTC | update dependencies (#1446) | 02 June 2022, 17:31:55 UTC |
| 605b850 | Brian Harrington | 02 June 2022, 17:22:08 UTC | sbt-updates 0.6.3 | 02 June 2022, 17:22:08 UTC |
| 2f55e93 | Brian Harrington | 02 June 2022, 17:18:41 UTC | sbt-git 2.0.0 | 02 June 2022, 17:18:41 UTC |
| 00a0f74 | Brian Harrington | 02 June 2022, 17:16:03 UTC | sbt-sonatype 3.9.13 | 02 June 2022, 17:16:03 UTC |
| abf080b | Brian Harrington | 02 June 2022, 17:13:13 UTC | embedded-postgres 2.0.0 | 02 June 2022, 17:13:13 UTC |
| b6d3a9c | Brian Harrington | 02 June 2022, 17:12:18 UTC | postgresql 42.3.6 | 02 June 2022, 17:12:18 UTC |
| c9fa07e | Brian Harrington | 02 June 2022, 17:11:16 UTC | RoaringBitmap 0.9.28 | 02 June 2022, 17:11:16 UTC |
| 650f646 | Brian Harrington | 02 June 2022, 17:09:25 UTC | iep 3.0.19 | 02 June 2022, 17:09:25 UTC |
| 216c9be | Brian Harrington | 02 June 2022, 17:08:46 UTC | jackson 2.13.3 | 02 June 2022, 17:08:46 UTC |
| c6e515c | Brian Harrington | 02 June 2022, 17:07:38 UTC | scala-logging 3.9.5 | 02 June 2022, 17:07:38 UTC |
| 42140b8 | Brian Harrington | 02 June 2022, 17:05:49 UTC | spectator 1.3.5 | 02 June 2022, 17:05:49 UTC |
| 8c92e94 | brharrington | 24 May 2022, 17:15:05 UTC | fix `:cg` with non-grouped math aggregate (#1445) If there was an aggregation of a group by result that was not grouped, then it was not getting the common grouping applied. | 24 May 2022, 17:15:05 UTC |
| 3623642 | brharrington | 16 May 2022, 17:56:00 UTC | fix cg with custom averages (#1444) The subset check for nested grouping would check all data expressions. If the underlying expression was combined with binary operations with the grouping for one side being a subset of the other, then a subsequent grouping operation could fail the check. Now it checks based on the final grouping of the pre-aggregate. | 16 May 2022, 17:56:00 UTC |
| b06171b | sullis | 05 May 2022, 20:35:44 UTC | GitHub Actions cache sbt (#1442) https://github.com/actions/setup-java/blob/main/README.md | 05 May 2022, 20:35:44 UTC |
| c5681f0 | brharrington | 04 May 2022, 18:19:53 UTC | update dependencies (#1441) | 04 May 2022, 18:19:53 UTC |
| eed59d7 | Brian Harrington | 04 May 2022, 18:12:41 UTC | update CI builds to use jdk 17 and 18 | 04 May 2022, 18:12:41 UTC |
| d9f9ac8 | Brian Harrington | 04 May 2022, 18:08:50 UTC | postgresql 42.3.5 | 04 May 2022, 18:08:50 UTC |
| 68289aa | Brian Harrington | 04 May 2022, 18:07:23 UTC | datasketches-java 3.2.0 | 04 May 2022, 18:07:23 UTC |
| c364c62 | Brian Harrington | 04 May 2022, 18:06:04 UTC | iep 3.0.18 | 04 May 2022, 18:06:04 UTC |
| 85d9aab | Brian Harrington | 04 May 2022, 18:04:46 UTC | spectator 1.3.2 | 04 May 2022, 18:04:46 UTC |
| 57ac20c | brharrington | 04 May 2022, 16:20:45 UTC | fix common grouping for aggr helpers (#1440) Updates the rewrite used for math functions that behave like aggregation functions, e.g. `:avg`, so that the rewrite works correctly when the display expression is a `Query`. Before it would always apply the rewrite to just the eval expression which could lead to a different and sometimes broken expression if reinterpreted from the display expression. | 04 May 2022, 16:20:45 UTC |
| 99afe17 | brharrington | 28 April 2022, 21:41:14 UTC | fix id when common tags are after metric data (#1439) If the common tags were after the metric data in the batch json payload, then the tags in the tuple would get updated, but not the id. This change fixes it so that the id is updated based on the final tags for a metric. Also updates the behavior so the conflicts resolution is consistent for both paths. | 28 April 2022, 21:41:14 UTC |
| 6c66d89 | brharrington | 28 April 2022, 20:24:52 UTC | ensure map keys are deduped (#1438) Fixes SortedTagMap to address some update patterns that could result in duplicate keys. | 28 April 2022, 20:24:52 UTC |
| 3f92628 | brharrington | 18 April 2022, 19:56:17 UTC | use sorted map when decoding tags (#1436) This is typically more efficient overall as it avoids the need to sort multiple times later on. For example, in publish the sort is needed for computing the ids and then ends up getting performed again to encode as a compact batch. | 18 April 2022, 19:56:17 UTC |
| ce358d7 | sullis | 18 April 2022, 12:07:49 UTC | update documentation link (#1437) | 18 April 2022, 12:07:49 UTC |
| d3b45f0 | brharrington | 15 April 2022, 21:49:31 UTC | lwc: update operator validation (#1435) Exclude operators that are confusing when used in a streaming context. Specifically: - `integral`: accumulates the value over the life of the evaluation. For a streaming evaluation the user has no way to know the time window it has been accumulating the value over. - _Filters_: all filtering operators rely on statistics from the evaluation context to select the results. For streaming evaluation, that context is always a single step interval. Filtering is best left to the consumer of the data. | 15 April 2022, 21:49:31 UTC |
| 440f4d5 | brharrington | 15 April 2022, 16:51:16 UTC | allow mixed grouping for custom averages (#1434) In the past this wasn't permitted because binary operations had to have the same grouping on both sides. In #1244 that was changed so that one side could have a subset of the other. | 15 April 2022, 16:51:16 UTC |
| d6d281c | brharrington | 13 April 2022, 15:54:51 UTC | update dependencies (#1433) | 13 April 2022, 15:54:51 UTC |
| b7eb60d | Brian Harrington | 13 April 2022, 15:48:54 UTC | equalsverifier 3.10 | 13 April 2022, 15:48:54 UTC |
| 4586fa7 | Brian Harrington | 13 April 2022, 15:48:10 UTC | RoaringBitmap 0.9.26 | 13 April 2022, 15:48:10 UTC |
| 114cedf | Brian Harrington | 13 April 2022, 15:47:12 UTC | scala-collection-compat 2.7.0 | 13 April 2022, 15:47:12 UTC |
| c4a68c8 | Brian Harrington | 13 April 2022, 15:46:19 UTC | spectator 1.3.0 | 13 April 2022, 15:46:19 UTC |
