| c2dd5c3 | brharrington | 14 December 2022, 23:59:05 UTC | remove redundant clauses when normalizing (#1505) In some cases with tooling a `:cq` can get applied multiple times. If the common query condition has an OR clause, then that will get expanded with the cross product when computing the DNF. As a result, the normalized query resulting from applying a `:cq` once would differ from applying the same `:cq` multiple times. This change removes redundant clauses from the normalized OR set to avoid the discrepancy. Redundant means that the removed clause will not change the result since another branch of the OR would have matched. | 14 December 2022, 23:59:05 UTC |
| 8e7368c | brharrington | 14 December 2022, 18:34:08 UTC | dedup or clauses when normalizing (#1504) Update the expression normalization to dedup OR clauses if the conditions are the same. | 14 December 2022, 18:34:08 UTC |
| 9a59fca | brharrington | 14 December 2022, 13:09:40 UTC | use stat placeholders for normalize (#1503) When normalizing an expression that uses a filter, use one of the `:stat-$(aggr)` placeholders if possible. This avoids duplication of the input expression making them easier to read. It also helps if comparing expressions after normalization to see if they are equivalent. | 14 December 2022, 13:09:40 UTC |
| 3fe64d9 | brharrington | 14 December 2022, 01:24:48 UTC | dedup query clauses during normalization (#1502) Update the normalization logic to deduplicate the same query clause within an conjunction. Also deduplicates terms within an IN clause and maps to an equals check if there is a single term. | 14 December 2022, 01:24:48 UTC |
| 7ead8dd | brharrington | 09 December 2022, 20:46:16 UTC | update dependencies (#1501) | 09 December 2022, 20:46:16 UTC |
| d60763d | Brian Harrington | 09 December 2022, 20:30:30 UTC | equalsverifier 3.12.2 | 09 December 2022, 20:30:30 UTC |
| a6682fd | Brian Harrington | 09 December 2022, 20:29:11 UTC | spring 5.3.24 | 09 December 2022, 20:29:11 UTC |
| f7c8fb6 | Brian Harrington | 09 December 2022, 20:28:25 UTC | embedded-postgres 2.0.2 | 09 December 2022, 20:28:25 UTC |
| 44c6092 | Brian Harrington | 09 December 2022, 20:27:48 UTC | postgresql 42.5.1 | 09 December 2022, 20:27:48 UTC |
| 6223592 | Brian Harrington | 09 December 2022, 20:26:50 UTC | iep 4.0.4 | 09 December 2022, 20:26:50 UTC |
| fc5136b | Brian Harrington | 09 December 2022, 20:25:48 UTC | scala-collection-compat 2.9.0 | 09 December 2022, 20:25:48 UTC |
| 9268831 | Brian Harrington | 09 December 2022, 20:25:00 UTC | spectator 1.4.2 | 09 December 2022, 20:25:00 UTC |
| 6e60919 | Brian Harrington | 09 December 2022, 20:23:44 UTC | sbt 1.8.0 | 09 December 2022, 20:23:44 UTC |
| 8bcf47a | Chris Larsen | 01 December 2022, 13:47:07 UTC | add support for duration tick labels (#1497) Adding support for `&tick_labels=duration` that will scale ticks from picoseconds to years. Also change the legend behavior so that counts are no longer tagged with the same prefix as the other stats to differentiate in binary and duration mode. Fixes #1047 | 01 December 2022, 13:47:07 UTC |
| 2f7054a | brharrington | 29 November 2022, 12:27:33 UTC | add value function to dedup inputs (#1500) When pushing data to memory the blocks can easily overwrite when an update for a given datapoint is received. In other use-cases this can be quite expensive and undesirable. This change adds a value function that can be used to buffer the updates for a window of time and dedup the data. | 29 November 2022, 12:27:33 UTC |
| 30b4a5b | brharrington | 23 November 2022, 21:15:48 UTC | add option to disable CORS (#1499) If we are proxying to another server that already supports CORS, then having multiple layers adding the CORS headers can cause problems. | 23 November 2022, 21:15:48 UTC |
| 25d2005 | brharrington | 22 November 2022, 17:09:08 UTC | jackson 2.14.1 (#1498) Pull in some memory and performance optimizations in jackson ([2.14.1]). [2.14.1]: https://github.com/FasterXML/jackson/wiki/Jackson-Release-2.14.1 | 22 November 2022, 17:09:08 UTC |
| ae06f6b | brharrington | 21 November 2022, 19:48:43 UTC | flush meters when a time grouping is emitted (#1496) When TimeGrouped flushes a finalized time grouping also have it explicitly flush the batch meter updaters. This avoids the counts potentially showing up much later due to a low rate. | 21 November 2022, 19:48:43 UTC |
| 34ed816 | brharrington | 21 November 2022, 15:02:23 UTC | ignore heartbeat for dropped old (#1495) The heartbeat just ensures the stream keeps moving even if there is no data. It is expected to be delayed from the actual flow a bit. | 21 November 2022, 15:02:23 UTC |
| 73e7ec8 | brharrington | 21 November 2022, 13:17:47 UTC | parse messages after merge of streams (#1494) Moves the step for parsing messages after the merge of streams from each of the LWCAPI nodes. This makes it easier to keep track of the overall number of concurrent requests and ensure it backpressures rather than enqueues. Also changes to use the ordered variant. Preserving the order could be important to avoid newer messages getting pushed through first and moving the cutoff point for the time grouping. | 21 November 2022, 13:17:47 UTC |
| bf3d844 | brharrington | 19 November 2022, 15:29:20 UTC | monitor flow just after evaluation (#1493) Add a check to monitor the flow just after the final evaluation step and before flattening the messages. This helps show the overall delay for a downstream consumer which is more relevant than the delay for the indvidual messages. | 19 November 2022, 15:29:20 UTC |
| 5e60008 | brharrington | 18 November 2022, 04:10:49 UTC | add hint for controlling y-axis color (#1492) Adds a rendering hint, `ambiguous-multi-y`, for the graph image that will disable the behavior of using the first line on an axis for the axis color. Instead it will use the color selected based on the theme. As indicated by the name, in this mode there will not be any visual cue as to which axis corresponds to data on the graph image. | 18 November 2022, 04:10:49 UTC |
| 82b412f | brharrington | 14 November 2022, 16:58:34 UTC | lwcapi: only update subscription list on changes (#1491) Creating the full list of of all subscriptions can be expensive when there are a lot of expressions. Update it to only be done when there are changes the same as the query index update. | 14 November 2022, 16:58:34 UTC |
| e8b9b49 | brharrington | 13 November 2022, 15:28:35 UTC | lwc: keep input to TimeGrouped batched (#1490) Refactors the stream from parsing the messages from lwcapi to the TimeGrouped stage to keep the datapoints batched. This significantly reduces the number of messages passed between the stream stages and achieve a higher throughput. Also creates a dedicated thread pool to use for parsing the messages as that is the most time consuming step. | 13 November 2022, 15:28:35 UTC |
| 358aef3 | brharrington | 12 November 2022, 20:37:44 UTC | lwc: remove some monitorFlow steps (#1489) When processing data with a high input volume the `ConnectionSources`, `InputLines`, and `LwcDatapoints` monitors would have a high throughput and the same number of elements flowing through. This change removes two of them so there is now just `InputLines`. This helps reduce overhead in the hot path. | 12 November 2022, 20:37:44 UTC |
| d68272f | brharrington | 12 November 2022, 00:32:23 UTC | lwc: use SortedTagMap for data points (#1488) The main advantage is that for eval it can just quickly copy into an array without all the hashing and extra space in the hash array to avoid collisions. Note, this change is safe to roll out as long as the lwcapi service is pushed first. | 12 November 2022, 00:32:23 UTC |
| 23bc96d | brharrington | 11 November 2022, 20:04:19 UTC | switch to spectator batch updaters (#1487) Use the spectator batch updaters in some of the key parts of the eval stream that were manually batched before. | 11 November 2022, 20:04:19 UTC |
| 55485bd | brharrington | 10 November 2022, 19:37:28 UTC | batch meter updates for TimeGrouped (#1486) Batch the counter updates in time grouped to avoid updating them in a tight loop for high throughput streams. | 10 November 2022, 19:37:28 UTC |
| 2374aff | brharrington | 10 November 2022, 19:33:22 UTC | batch updates to meters when monitoring flows (#1485) Updates the monitor flow stage to batch the updates to the meters. This avoids some of the overhead for calling gettimeofday frequently for high volume streams. | 10 November 2022, 19:33:22 UTC |
| 3e935c1 | brharrington | 10 November 2022, 02:15:48 UTC | only update index for new items (#1484) The updates for the existing items will get filtered out later, but we can avoid adding them in the first place to reduce the overhead a little bit. | 10 November 2022, 02:15:48 UTC |
| 70565a0 | brharrington | 10 November 2022, 00:12:58 UTC | remove usage of javax annotations (#1482) These have been moved to jakarta in later versions. With the way we are using Spring, the annotations are not necessary. | 10 November 2022, 00:12:58 UTC |
| a9d0d9b | brharrington | 09 November 2022, 19:36:18 UTC | json: enable fast double operations (#1483) Enable the fast double reading/writing that was added in jackson 2.14.0. | 09 November 2022, 19:36:18 UTC |
| 4bcee87 | brharrington | 09 November 2022, 15:49:58 UTC | update dependencies (#1481) | 09 November 2022, 15:49:58 UTC |
| 6ef6a5e | Brian Harrington | 09 November 2022, 15:28:55 UTC | RoaringBitmap 0.9.35 | 09 November 2022, 15:28:55 UTC |
| 6711fb4 | Brian Harrington | 09 November 2022, 15:28:03 UTC | equalsverifier 3.11 | 09 November 2022, 15:28:03 UTC |
| 5023a1d | Brian Harrington | 09 November 2022, 15:27:35 UTC | jackson 2.14.0 | 09 November 2022, 15:27:35 UTC |
| 9b1ec8f | Brian Harrington | 09 November 2022, 15:26:54 UTC | iep 4.0.2 | 09 November 2022, 15:26:54 UTC |
| 679aa21 | Brian Harrington | 09 November 2022, 15:26:29 UTC | spectator 1.3.10 | 09 November 2022, 15:26:29 UTC |
| e2dc326 | Brian Harrington | 09 November 2022, 15:24:40 UTC | sbt 1.7.3 | 09 November 2022, 15:24:40 UTC |
| 6b11ea0 | brharrington | 08 November 2022, 19:19:48 UTC | improve performance of batch index update (#1479) A lot of the time and allocations were being spent sorting the items passed to the index. Java uses TimSort internally when sorting object arrays and that uses a merge sort. It allocates a bunch of temporary arrays when performing the merges. This change leverages the fact that the items that are already indexed will be sorted, so we can just sort the new items and merge the two sorted sets. | 08 November 2022, 19:19:48 UTC |
| 397cdcf | brharrington | 08 November 2022, 19:15:09 UTC | add expression to v2.json output (#1480) Update the `v2.json` format to include the query expression for the lines. This makes it more consistent with the time series messages used with fetch and LWC. | 08 November 2022, 19:15:09 UTC |
| 71098c9 | brharrington | 07 November 2022, 22:06:11 UTC | json: fix double parsing for large integers (#1478) Before when using `JsonParserHelper.nextDouble` if the raw encoded value looked like an integer type it would try to access it as a long, then convert the long to a double. This causes problems for larger integers that cannot fit within the bounds of a Java long. Now it will access it as a double. | 07 November 2022, 22:06:11 UTC |
| 3143e37 | brharrington | 28 October 2022, 22:44:34 UTC | use custom JSON encoding if specified (#1477) If a class extended JsonSupport and provided a custom encoding it would get used by the interited methods, but would get ignored when using `Json.encode`. This could create problems when an object is nested because the custom encode would not get used. The `Json.encode` could not directly call encode, because it would create a loop with the default implementation. This change adds a method that can indicate if it is desirable for `Json.ecnode` to use the custom encoding. | 28 October 2022, 22:44:34 UTC |
| 6096696 | brharrington | 27 October 2022, 20:22:11 UTC | allow image to be disabled for v2.json (#1476) The image rendering can be computationally expensive and increase the size of the payload quite a bit. For some cases it is desirable to use the `v2.json` format since it is similar to fetch and LWC, but the image is not needed. Alert processing is one example. This change adds a `hints` URI parameter that can be used to pass arbitrary hints back to the rendering engine that is used. The `v2.json` engine has been updated to omit the image if a `no-image` hint is passed. In the future we may want to only include the image if requested, but that would be backwards incompatible and break some current usage. | 27 October 2022, 20:22:11 UTC |
| 3205dfb | brharrington | 27 October 2022, 17:20:17 UTC | improve legend for custom avg with nested grouping (#1475) Updates the simple legends generation to handle the case of custom averages used with nested grouping. Before the logic to remove the named rewrite would also remove the first grouping and break the rewrite of any later groupings. | 27 October 2022, 17:20:17 UTC |
| b9f4d8c | brharrington | 25 October 2022, 21:45:16 UTC | update normalize behavior for UI (#1474) Updates the behavior of normalize to reflect preferences of the UI team. In particular: - It will not try to extract common query aspects into a `:cq` clause. - The helpers for filtering based on summary stats will be preserved rather than expanding. - Adjust handling of OR clauses in a query. First convert to DNF and then sort the CNF clauses to avoid awkward expansion with repeated terms. | 25 October 2022, 21:45:16 UTC |
| 72cadea | brharrington | 13 October 2022, 13:29:11 UTC | update dependencies (#1472) | 13 October 2022, 13:29:11 UTC |
| d0671b6 | Brian Harrington | 13 October 2022, 13:15:32 UTC | jackson 2.13.4 | 13 October 2022, 13:15:32 UTC |
| ead8118 | Brian Harrington | 13 October 2022, 13:12:50 UTC | log4j 2.19.0 | 13 October 2022, 13:12:50 UTC |
| ca9db8f | Brian Harrington | 13 October 2022, 13:12:11 UTC | RoaringBitmap 0.9.32 | 13 October 2022, 13:12:11 UTC |
| 6f43714 | Brian Harrington | 13 October 2022, 13:11:09 UTC | spring 5.3.23 | 13 October 2022, 13:11:09 UTC |
| 37e7b5d | Brian Harrington | 13 October 2022, 13:09:33 UTC | iep 4.0.1 | 13 October 2022, 13:09:33 UTC |
| e49670b | Brian Harrington | 13 October 2022, 13:09:12 UTC | spectator 1.3.9 | 13 October 2022, 13:09:12 UTC |
| bc13fc2 | Brian Harrington | 13 October 2022, 13:07:37 UTC | sbt-updates 0.6.4 | 13 October 2022, 13:07:37 UTC |
| 8972da7 | Brian Harrington | 13 October 2022, 12:59:05 UTC | add jdk19 to CI | 13 October 2022, 12:59:05 UTC |
| 2517c9a | Brian Harrington | 13 October 2022, 12:58:02 UTC | sbt 1.7.2 | 13 October 2022, 12:58:02 UTC |
| 9deca0c | Brian Harrington | 13 October 2022, 12:56:42 UTC | scala 2.13.10 | 13 October 2022, 12:56:42 UTC |
| c24f0ae | Brian Harrington | 13 October 2022, 11:46:10 UTC | scalafmt 3.5.9 Tries to update the config to be close to the previous and minimize unnecessary diffs. | 13 October 2022, 11:46:10 UTC |
| 8044337 | Chris Larsen | 27 September 2022, 19:48:23 UTC | logging queries.add `:starts` and `:ends` for log queries (#1471) Adds support for `:starts` and `:ends` which are frequently used with some log queries internally. | 27 September 2022, 19:48:23 UTC |
| bcf651d | Chris Larsen | 27 September 2022, 19:19:07 UTC | Add `:contains` predicate for compatibility with log queries (#1470) Adds support for `:contains` which is frequently used with some log queries internally. Bump Spectator to 1.3.8 | 27 September 2022, 19:19:07 UTC |
| 07cb339 | brharrington | 23 September 2022, 16:17:30 UTC | optimize result tags computation (#1469) A long time ago the tag set was made deterministic based on the query expression where it only includes tags with an exact match or that are listed in the group by. As a result, it isn't necessary to compute the intersection of dimensions while aggregating the matching time series. | 23 September 2022, 16:17:30 UTC |
| 7fb6b14 | brharrington | 22 September 2022, 14:55:56 UTC | make default dstype consistent (#1468) This change makes the default dstype consistently rate for both apply methods of DsType as well as matching the behavior for the publish path when feeding data into the normalization cache. | 22 September 2022, 14:55:56 UTC |
| 42c1f2f | brharrington | 22 September 2022, 11:38:42 UTC | adjust consolidation behavior for rates (#1467) Port an old internal fix to ensure we have consistent behavior. If the data source type is a rate, then treat NaN values as 0 as it means no activity measured for those intervals when consolidating. That means the avg will always be computed over the entire consolidated interval. For gauges, the NaN values are ignored because to be consistent with values reported to the gauge. The behavior of gauges is not impacted with this change. | 22 September 2022, 11:38:42 UTC |
| 89d799c | brharrington | 22 September 2022, 01:18:30 UTC | update CollectorStatsBuilder to allow chaining (#1466) Return a reference to `this` from the builder methods so that they can be chained. | 22 September 2022, 01:18:30 UTC |
| 11cae07 | brharrington | 09 September 2022, 19:16:56 UTC | improve couldMatch check for not clauses (#1465) If the inverted query has an exact match, then adding dimensions cannot change the result and it should be excluded. | 09 September 2022, 19:16:56 UTC |
| 64627ef | brharrington | 09 September 2022, 16:43:19 UTC | add note about akka versions (#1464) Update to latest ASL licensed Akka and Akka-HTTP and add comment about license change. | 09 September 2022, 16:43:19 UTC |
| de163ee | brharrington | 08 September 2022, 18:26:52 UTC | avoid calling sort for SortedTagMap (#1463) When computing the id from a tag map, avoid making the sort call when a SortedTagMap is used. That type will already be sorted so it is not necessary. | 08 September 2022, 18:26:52 UTC |
| fa16821 | brharrington | 08 September 2022, 18:16:40 UTC | reduce allocations for ItemId.toString (#1462) Initialize capacity for the builder to avoid a resize while filling the buffer. Reduces allocations for the sample benchmark from 440 B/op to 216 B/op. | 08 September 2022, 18:16:40 UTC |
| 18a2c6d | brharrington | 01 September 2022, 16:09:51 UTC | fix typo in config method name (#1460) | 01 September 2022, 16:09:51 UTC |
| df8bb16 | brharrington | 01 September 2022, 16:09:20 UTC | update dependencies (#1461) | 01 September 2022, 16:09:20 UTC |
| 0a9caff | Brian Harrington | 01 September 2022, 15:50:28 UTC | iep 4.0.0 | 01 September 2022, 15:59:38 UTC |
| 8145a1d | Brian Harrington | 01 September 2022, 15:42:53 UTC | embedded-postgres 2.0.1 | 01 September 2022, 15:42:53 UTC |
| f3b249f | Brian Harrington | 01 September 2022, 15:42:15 UTC | postgresql 42.5.0 | 01 September 2022, 15:42:15 UTC |
| 62a5cc7 | Brian Harrington | 01 September 2022, 15:41:00 UTC | spectator 1.3.7 | 01 September 2022, 15:41:00 UTC |
| 40ab7ff | brharrington | 28 August 2022, 15:44:11 UTC | switch to using Spring for DI (#1455) Since Netflix is moving from Guice to Spring, switch the DI framework to use Spring. This is mostly just migrating the Guice modules. | 28 August 2022, 15:44:11 UTC |
| 5a547c8 | brharrington | 08 August 2022, 22:30:35 UTC | update dependencies (#1453) | 08 August 2022, 22:30:35 UTC |
| 2f07860 | Brian Harrington | 08 August 2022, 22:22:28 UTC | log4j 2.18.0 | 08 August 2022, 22:22:28 UTC |
| 268bc50 | Brian Harrington | 08 August 2022, 22:21:38 UTC | RoaringBitmap 0.9.31 | 08 August 2022, 22:21:38 UTC |
| 3799059 | Brian Harrington | 08 August 2022, 22:20:45 UTC | datasketches-java 3.3.0 | 08 August 2022, 22:20:45 UTC |
| 68157dc | Brian Harrington | 08 August 2022, 22:19:54 UTC | postgresql 42.4.1 | 08 August 2022, 22:19:54 UTC |
| 3eb1be1 | Brian Harrington | 08 August 2022, 22:19:09 UTC | equalsverifier 3.10.1 | 08 August 2022, 22:19:09 UTC |
| 38a20cb | Brian Harrington | 08 August 2022, 22:18:13 UTC | scala-collection-compat 2.8.1 | 08 August 2022, 22:18:13 UTC |
| 7a5f503 | Brian Harrington | 08 August 2022, 22:17:01 UTC | iep 3.0.21 | 08 August 2022, 22:17:01 UTC |
| 4b166b3 | Brian Harrington | 08 August 2022, 22:16:05 UTC | spectator 1.3.6 | 08 August 2022, 22:16:05 UTC |
| 1239586 | brharrington | 02 August 2022, 17:03:01 UTC | fix shard lookup for Integer.MIN_VALUE (#1452) Before it was relying on `abs` to get a non-negative value for computing an index to the array. This breaks when the int value of the id is `Integer.MIN_VALUE`. Update it to explicitly clear the sign bit to ensure the value will always be non-negative. | 02 August 2022, 17:03:01 UTC |
| 483dfc5 | brharrington | 22 July 2022, 16:59:45 UTC | lwcapi: remove the /v1/subscribe endpoint (#1451) This was an earlier version that relied used unbatched text messages for passing the data and was much less efficient. It is no longer in use internally. | 22 July 2022, 16:59:45 UTC |
| 7b02e34 | sullis | 12 July 2022, 16:40:29 UTC | sbt 1.7.1 (#1450) https://github.com/sbt/sbt/releases/tag/v1.7.1 | 12 July 2022, 16:40:29 UTC |
| 20042fc | sullis | 11 July 2022, 10:10:56 UTC | sbt 1.7.0 (#1449) https://github.com/sbt/sbt/releases/tag/v1.7.0 | 11 July 2022, 10:10:56 UTC |
| a850013 | Chris Larsen | 29 June 2022, 12:36:02 UTC | lwcapi: ensure failed clients are unregistered (#1448) Move the termination watch to the root flow in order to catch improperly terminated clients. Also adds a note about dealing with clients that connect and fail to send a message wherein the queue shuts down after a timeout period and fails to notify the rest of the flow. | 29 June 2022, 12:36:02 UTC |
| 852b05e | brharrington | 17 June 2022, 16:56:31 UTC | avoid boxing when parsing special double values (#1447) For special double values, e.g. `NaN`, use `parseDouble` instead of `valueOf` to avoid allocating an instance of the primitive wrapper type. | 17 June 2022, 16:56:31 UTC |
| c0a9919 | brharrington | 02 June 2022, 17:31:55 UTC | update dependencies (#1446) | 02 June 2022, 17:31:55 UTC |
| 605b850 | Brian Harrington | 02 June 2022, 17:22:08 UTC | sbt-updates 0.6.3 | 02 June 2022, 17:22:08 UTC |
| 2f55e93 | Brian Harrington | 02 June 2022, 17:18:41 UTC | sbt-git 2.0.0 | 02 June 2022, 17:18:41 UTC |
| 00a0f74 | Brian Harrington | 02 June 2022, 17:16:03 UTC | sbt-sonatype 3.9.13 | 02 June 2022, 17:16:03 UTC |
| abf080b | Brian Harrington | 02 June 2022, 17:13:13 UTC | embedded-postgres 2.0.0 | 02 June 2022, 17:13:13 UTC |
| b6d3a9c | Brian Harrington | 02 June 2022, 17:12:18 UTC | postgresql 42.3.6 | 02 June 2022, 17:12:18 UTC |
| c9fa07e | Brian Harrington | 02 June 2022, 17:11:16 UTC | RoaringBitmap 0.9.28 | 02 June 2022, 17:11:16 UTC |
| 650f646 | Brian Harrington | 02 June 2022, 17:09:25 UTC | iep 3.0.19 | 02 June 2022, 17:09:25 UTC |
| 216c9be | Brian Harrington | 02 June 2022, 17:08:46 UTC | jackson 2.13.3 | 02 June 2022, 17:08:46 UTC |
| c6e515c | Brian Harrington | 02 June 2022, 17:07:38 UTC | scala-logging 3.9.5 | 02 June 2022, 17:07:38 UTC |
