https://github.com/Netflix/atlas

sort by:
Revision Author Date Message Commit Date
f2edd36 included rendered image in v2.json response (#568) Fixes #564. We may move this behind a flag before 1.6 final, but for now it is enabled by default. By including in the v2.json format the UI/dashboard can use the static image and switch to a dynamic rendering without making another request. The message has a type of `graph-image` with a data field that is a data uri of the image without the legend. Sample usage: ```html <html> <body> <div id="content"></div> <script> var content = document.getElementById('content'); fetch('http://localhost:7101/api/v1/graph?q=name,sps,:eq,(,nf.cluster,),:by&format=v2.json') .then(function(response) { return response.json(); }) .then(function(json) { var html = ''; json.forEach(function(msg) { if (msg.type === 'graph-image') { html += '<div><img src="' + msg.data + '"/></div>'; } else if (msg.type === 'timeseries') { html += '<div>' + msg.label + '</div>'; } }); content.innerHTML = html; }); </script> </body> </html> ``` 01 May 2017, 22:19:52 UTC
16a734c remove ExpressionDatabaseActor from sample config (#567) Updates the lwcapi.conf sample config to remove the actor since it went away in #561. Also updates to spectator 0.55.0. 01 May 2017, 21:23:55 UTC
1481015 add additional graph api tests (#566) Add some additional graph api test images that were being used for the dynamic rendering component. Trying to keep them in sync. 29 April 2017, 00:23:17 UTC
f766add fix #560, axis params now work with axis_per_line (#563) When using `axis_per_line` flag the specific url params for other y-axes would get ignored because it was just copying from the main axis. Now overrides will take effect just like they do for explicit multi-y. 28 April 2017, 04:35:15 UTC
9d0db91 make axisPerLine with explicit axis a warning (#562) Instead of giving an error to the user make it a warning and disable the axisPerLine flag. 28 April 2017, 01:39:26 UTC
cbb0c6c fix #467, remove the use of redis for lwcapi (#561) The only use-case was for ensuring the set of expressions was sync'd up across instances so that clients would not see expressions come and go very quickly due to timing issues for the subscribers. To simplify the overall setup the clients will do a merge and maintain the expression with a TTL. 27 April 2017, 23:39:59 UTC
210322e update to frigga 0.18.0 (#558) Adds support for `~` in the detail for group names (Netflix/frigga#20). 26 April 2017, 15:07:56 UTC
91049e7 remove LazyTagIndex implementation (#557) We have been using the implementation based on roaring bitmaps (#479) for a few months internally and it is working well. 26 April 2017, 14:54:06 UTC
f848278 switch to Files.new{Input,Output}Stream (#556) They avoid finalize making them a bit more GC friendly. https://www.cloudbees.com/blog/fileinputstream-fileoutputstream-considered-harmful 26 April 2017, 14:38:44 UTC
650d555 fix spacing on sidebar header (#555) Fixes the spacing so that it will render correctly on the wiki. 26 April 2017, 00:14:23 UTC
76ed570 update to sbt 0.13.15 (#554) This version seems to fix `formatLicenseHeaders` so it will work when run from the root. In #553 it was noticed that it worked when running for a particular subproject, but not when run for the root project. 25 April 2017, 23:32:36 UTC
f56a60f add eval lib to root project (#553) It wasn't getting published to jfrog/jcenter since it wasn't part of the root. Also fixes license header for SourceRef. 25 April 2017, 15:44:52 UTC
d68cf46 docs: fix link to sorting modes (#552) 25 April 2017, 15:03:51 UTC
8a1b68e add reference config for eval lib (#550) This was accidentally left out of a previous commit. 25 April 2017, 14:59:58 UTC
5af80e1 add module for configuring evaluator (#549) Minimal module that can setup an evaluator instance that can be injected. 24 April 2017, 20:08:59 UTC
bb30335 simplify eval usage for common use-cases (#548) Adds an `Evaluator` class with a simple public API for evaluating the stream. It is a java class that exposes the reactive streams types so it is easy to use with various implementations. There are two main methods: **createPublisher(uri)** Creates a publisher for a graph uri so the same data can be accessed as a stream. This makes it really easy for a user to transition from one view to another. ``` +------------+ | URI | | Eval | -------- [TimeSeriesMessage] ---> | Source | +------------+ ``` **createStreamsProcessor** Takes a set of id/URI pairs as inputs and evaluates all of the expressions. This allows for better reuse when evaluating many expressions. There is an id for each input that will be added to the output so that corresponding entries can be correlated. The complete set of current data sources must be supplied as part of each input event. ``` +------+ | | --- [DataSources] ---> | Eval | --- [MessageEnvelope] ---> Set((id,URI), ...) | | (id, TimeSeriesMessage) +------+ ``` 24 April 2017, 16:01:58 UTC
f0ea676 update to scala 2.12.2 and fix warnings (#545) Updates to scala 2.12.2 (and 2.11.11) and fixes a bunch of new compiler warnings. 19 April 2017, 15:20:02 UTC
e46a321 fix rewrite to use new display expr (#544) Fixes a regression introduced in #540. The non-query case was re-evaluating the rewrite command on the original query rather than the new rewritten display expression. 19 April 2017, 14:36:00 UTC
a37ee20 add util for wrapping ByteString as InputStream (#543) This makes it easier to quickly pass the ByteString to jackson and keep the number of additional allocations minimal. 18 April 2017, 16:47:38 UTC
924a076 update to akka 2.5.0 (#542) Switch to akka 2.5.0 to get improved materialization performance for streams. Throughput for the simple benchmark in #541 improved by 26%. 18 April 2017, 15:25:02 UTC
6ed04f0 reduce allocations for processing messages (#541) Some initial performance checks on the streaming evaluation show a lot of allocations. The biggest contributor was converting from ByteString to a form that jackson could parse. This has been reduced by reusing buffers. Some other minor changes have been made to reduce allocations for fields and to go direct to SmallHashMap for the tag data. Biggest bottlenecks right now are parsing the json messages from the SSE stream followed by the materialization speed. The materialization speed should be less of an issue in practice because the streams will be longer lived and is supposed to be improved in akka 2.5.0. 18 April 2017, 14:46:13 UTC
fed398a named rewrite expr string now preserves offset (#540) Fixes a bug with the expression string for named rewrites that contain offsets. If the rewrite is being used similar to an aggregate function and the display expression is a query, then the offset was getting lost because that is an attribute of the aggregate functions that are only present in the expanded evaluation expression. Now the named rewrite will extract the offset and group by information if the display type is a query. For others the rewrite will be re-evaluated using the same interpreter. 15 April 2017, 04:32:01 UTC
8063495 initial streaming eval library (#538) This isn't yet fully baked or hardened, but is a good enough check point. This is an initial version of a library to evaluate expressions using the LWC service as a source. Still left to do: 1. Expose simple and stable java API for accessing the stream for a corresponding graph uri. This is to make it really easy to transition between the backend graph and a streaming source. 2. Source identification in LWC so we can reliably dedup messages. 3. Improving test coverage of Eureka source and evaluator helpers. This is really sparse right now. 4. Refactor Datapoint so that the step size is provided rather than coming from a static config. 5. Performance testing and tuning. These follow up items will be addressed in later PRs. 13 April 2017, 20:45:27 UTC
19a9a1a enable self-reporting for standalone example (#537) The standalone example will not report data to itself when running with the sample `memory.conf`. A similar thing can be done with the `lwcapi.conf` by uncommenting the enabled flag. It is disabled by default since there are some ordering issues on startup that cause exceptions in the log. 11 April 2017, 16:18:45 UTC
0e77e68 disable log4j shutdown hook on default config (#536) Updates the default logging config for standalone so that it disables the log4j shutdown hook. This is to avoid missing log messages for the app shutdown: https://github.com/Netflix/iep/tree/master/iep-guice#shutdown 11 April 2017, 15:20:09 UTC
d4b8470 only run lwc when using the lwcapi.conf (#535) Fixes the standalone example to only run the lwc api and services when using the lwcapi config. The webapi portions have been restricted in a similar way. This avoids the need for having redis to run the basic standalone example. 11 April 2017, 15:16:34 UTC
cd03716 add limit operation that can be used with sort (#534) Adds a `:limit` operation that can be used with the existing `:sort` and `:order` to restrict the output set. The `:head` operator has been removed as a data operator and changed to be an alias for limit. Limit was chosen because it is more familiar to those with a SQL background. The `:head` operator was never really optimized in the data layer and a desire to use sort with a limit for top-N use-cases is more prominent than the sampling use-cases for which `:head` was originally intended, but that did not really materialize in practice. 10 April 2017, 23:07:41 UTC
d02eb51 support merging rollup blocks (#533) With inline rollups the the blocks from a single mirror can contain a rollup. For now it follows a simple strategy of just merging the individual blocks for the rollup. 09 April 2017, 15:45:54 UTC
fee4182 fix prefix for cloudwatch sqs metrics (#532) It was incorrectly specified as `s3`. 05 April 2017, 18:00:41 UTC
d34e90d store the interpreted expression in the Request (#531) For some use-cases we access `GraphApi.Request.exprs` many times, there is no reason to interpret it each time. 31 March 2017, 06:29:46 UTC
30fbbf0 add support for listing all expressions (#530) Modifies the `/lwc/api/v1/expressions` endpoint to support listing all expressions. Before it would only list out expresssions for a particular cluster. 30 March 2017, 17:47:12 UTC
a2e8d20 fix warnings for lwcapi (#529) Cleanup minor warnings such as unused references in lwcapi. Also simplifies to get rid of separate impl classes. The only api considered public for this module is the interaction with HTTP interface. The separate classes were just extra boilerplate. 30 March 2017, 14:49:50 UTC
4bb0ecf add `atlas.rollup` to allowed keys (#528) Updates the default allowed keys list for `atlas.` prefix to include `atlas.rollup`. 27 March 2017, 15:51:39 UTC
1995eb4 add additional build metadata to jar (#527) Add the build date and number to the jar manifest to aid in debugging. 26 March 2017, 03:36:22 UTC
57516f5 improve performance of SmallHashMap.{+,-} (#526) Changes the small hash map implementation of `+` and `-` to retain the type. This is primarily to avoid using a less memory efficient map type if the map is modified somewhere along the way. A side benefit is better performance and reduced allocations. **Before** ``` Benchmark Mode Cnt Score Error Units addPair thrpt 10 781344.330 ± 25173.790 ops/s removePair thrpt 10 738796.252 ± 44849.997 ops/s ``` **After** ``` Benchmark Mode Cnt Score Error Units addPair thrpt 10 1637470.772 ± 29948.315 ops/s removePair thrpt 10 2520939.662 ± 32320.759 ops/s ``` 25 March 2017, 20:16:39 UTC
c68cda0 support inline rollup of datapoints (#525) This change allows datapoints to be rolled up as they are flowing into the in memory storage. Typically this has been done later as part of offline processing using EMR or via a separate aggregator cluster that would perform the aggregations prior to being sent to the in memory storage. To be eligible for the rollup the data must have already been normalized to the step boundary and rate conversion for counters should have already been performed. The inline rollup will create a rollup block so it will have more overhead in terms of memory use unless the reduction multiple is over 4. Data will only go through the rollup path if it is tagged with `atlas.rollup=true`. The publish payload can now take an `id` field which will get used to dedup messages. If used, then it should avoid over counting if retries are present in the publishing pipeline as long as the updates for a given datapoint are always mapped to the same id for a given interval. The message id should not be used for multiple intervals or subsequent messages would incorrectly get lost. 24 March 2017, 21:32:42 UTC
51eb66f fix poller client to discard entity bytes (#524) The poller client was not properly discarding the entity bytes for success or unexpected status codes. 23 March 2017, 16:50:23 UTC
d97cf2d update dependencies (#523) 18 March 2017, 20:50:01 UTC
3c0eca7 remove unused spray dependencies 18 March 2017, 20:09:26 UTC
828db9e sbt-jmh 0.2.24 18 March 2017, 20:08:20 UTC
9965b54 roaring bitmap 0.6.39 18 March 2017, 20:06:15 UTC
17378c7 caffeine 2.4.0 18 March 2017, 20:05:41 UTC
e5df37c slf4j 1.7.25 18 March 2017, 20:05:12 UTC
4b60f16 log4j2 2.8.1 18 March 2017, 20:04:22 UTC
4ffb559 jackson 2.8.7 18 March 2017, 20:03:57 UTC
e8649b9 aws-java-sdk 1.11.105 18 March 2017, 20:03:27 UTC
bd1ee76 akka-http 10.0.5 18 March 2017, 20:02:51 UTC
65e1fd7 akka 2.4.17 18 March 2017, 20:01:29 UTC
9260f98 allow preflight check to be used separately (#522) Moves the handler for CORS preflight checks to CustomDirectives and adds a `cors` helper for wrapping a route with both the preflight handler and filter to add CORS headers to the inner route. This change allows the CORS support to be used without pulling in all of the standard options from the default request handler. 17 March 2017, 21:08:13 UTC
1947373 add efs metrics (#520) Adds mapping for cloudwatch EFS metrics and adjust some of the ids to be more consistent with casing. Some users found it annoying and hard to remember which ones had a starting capital. This change makes them more consistent. 17 March 2017, 18:00:20 UTC
3f9afdf emit subscriptions back in the sse response (#519) Cleanup a TODO in the code from when we switched to akka-http. The subscriptions objects are passed into the stream source actor. Also fixes a bug where we would only write the chunks on receive, but not when we had messages and available room based on `totalDemand`. 16 March 2017, 21:04:48 UTC
8662c2b enable caching of CORS preflight (#518) Adds the `Access-Control-Max-Age` to the CORS preflight response. Age is set to 10 minutes as that seems to be a common max allowed by various browsers. 16 March 2017, 06:12:37 UTC
e10275e add vary origin to cors responses (#517) Avoids confusing errors due to requests being cached with the wrong origin. 16 March 2017, 05:49:13 UTC
403e1e2 fix default content type in tags docs (#516) 16 March 2017, 00:23:06 UTC
582583c fix bounds exception for Integer.MIN_VALUE hash (#515) If an object hashes to `Integer.MIN_VALUE`, then the absolute value will be `Integer.MIN_VALUE` and a negative values was used for looking up data in the underlying array. This change will map those to be `0` instead. 15 March 2017, 23:25:38 UTC
cbd35c1 add method to explicitly rotate blocks (#514) Adds a method to the block store to explicitly rotate and compress the blocks. Before it would only happen when there was data written for a given time series. So if no data comes in after the time boundary it would not get automatically rotated. 14 March 2017, 20:30:57 UTC
bc9524a make ValidCharactersRule more flexible (#513) It now allows the set of valid characters to be set via a pattern in the config and for overrides to be set for given tag values. This is mostly to support an internal decision to relax group names so that they allow `^` and `~`. Others values should not be relaxed as it could result in backwards compatibility issues for names that were already being converted to `_` when published. 13 March 2017, 21:47:43 UTC
5f4bf7a add macro for stddev of group by (#512) Helper function for computing the stddev over the results of a group by operation. Sometimes useful as an offset for outlier detection. 06 March 2017, 22:26:44 UTC
a689bc5 avoid passing JsonParser across threads (#511) The `jsonParser` directive could potentially result in the created parser being created in one thread and then used in another. Jackson uses some thread local buffers for keeping track of state and this behavior means multiple threads could be updating those buffers. This change removes that directive and adds a `customJson` that allows the decoding to use the parser directly to create an object, but keeps it all local to one block that should all be run on a single thread. 28 February 2017, 15:19:06 UTC
151b062 allow `last` summary stat to be used with filters (#510) The last value for a given time series is already being tracked as part of the summary stats, but was not supported by the `:stat` operator. This change allows that stat to be used as part of a filter. 23 February 2017, 22:36:18 UTC
ac799c7 update dependencies (#509) Main change is to pick up spectator version that fixes an issue with timers for some request stats under reporting (Netflix/spectator#376). 20 February 2017, 17:22:09 UTC
a10d4c8 equalsverifier 2.2.1 20 February 2017, 16:59:12 UTC
6a78c42 sbt-jmh 0.2.21 (jmh 1.17.4) 20 February 2017, 16:52:37 UTC
c5fdad8 jol 0.7.1 20 February 2017, 16:50:35 UTC
3f4fe0d aws-java-sdk 1.11.93 20 February 2017, 16:49:41 UTC
28d3535 slf4j 1.7.23 20 February 2017, 16:48:57 UTC
7ee182c iep 0.4.16 20 February 2017, 16:48:32 UTC
0c88315 jackson 2.8.6 20 February 2017, 16:47:25 UTC
bb2edf1 spectator 0.52.0 Fixes issue with timers for some request stats under reporting (https://github.com/Netflix/spectator/pull/376). 20 February 2017, 16:45:07 UTC
2a6cf89 rewrite simple regex OR queries (#508) Changes simple regex OR queries to use a set of basic `StringMatcher`s. This significantly improves performance for the most common cases we see in practice. 20 February 2017, 02:15:21 UTC
a9fb61a short circuit AND queries in roaring index (#507) Updates the `RoaringTagIndex` to short-circuit AND queries just like `LazyTagIndex`. The second query will not only be performed if the first query has a non-empty result set. In the test environment this was the main cause of slower results when using `RoaringTagIndex` instead of `LazyTagIndex`. Also updates the version to 0.6.36. 20 February 2017, 02:15:02 UTC
95ee30d check for null token in foreachItem helper (#506) If the function `f` has a bug and consumes the `END_ARRAY` token, then before it would result in an endless loop. With this change it will fail quickly. 18 February 2017, 03:29:30 UTC
9d4b2c8 workaround for `:percentiles` on no data line (#505) In some use-cases the internal legacy system will try to eval with an explicit no data line being passed to the percentile operator. This results in an error: ``` java.util.NoSuchElementException: key not found: percentile ``` With this change we just propagate the no data line if the input does not contain the expected tags. 18 February 2017, 03:20:47 UTC
aac06cc do not apply heartbeat interval to rates (#504) If a heartbeat interval larger than the step is applied to rates, then it can artificially inflate the overall count. This can happen on occasion with the expiration pattern used by spectator and servo when reporting counters if the counter gets an update exactly one interval after it expired. Counters should be fine because they will compute the delta and average over the missed interval. The heartbeat is retained for this use-case as poller using a fixed delay may have a reporting interval that is slightly larger than the specified step. Fixes #497. 15 February 2017, 17:21:26 UTC
0540a55 split out helper method for creating the graph (#503) Adds a helper method for creating the graph definition without actually rendering it. Useful for some cases where we want to do a standalone eval and then manipulate the results rather than output to the user. 15 February 2017, 16:11:45 UTC
098274f fix internal server error due to access logging (#501) The access logger would create a URI instance and the `java.net.URI` class is strict when parsing. This has been changed to just pass in the string. It appears that the `toString` for the spray/akka-http Uri class changed in akka-http to be more consistent with what the user actually used. 11 February 2017, 03:37:12 UTC
4770896 fix #498, always honor order (#500) The order was getting ignored if there wasn't an explicit sort setting. 10 February 2017, 04:50:47 UTC
4c3a235 fix CORS headers on preflight request (#499) When switching to akka-http there was a regression in the routes. The CORS headers were no longer getting added to the OPTIONS preflight request. For the original issue see #259. This passes the test case in that commit comment using chrome. Also adds local unit test that verifies the headers are present. 10 February 2017, 03:02:41 UTC
d3caf33 update reference.conf to use akka-http settings (#496) Updates our reference.conf to use the settings from akka-http rather than spray. 08 February 2017, 20:12:46 UTC
a2340b4 cleanup usages of DiagnosticMessage.sendError (#495) The DiagnosticMessage object had some helper methods left over for sending an error response to an ActorRef. These will not work correctly using `akka-http` and the usages would not work as expected. * `TagsRequestActor`: using the handleExceptions which would not send the exception to the correct place if caught. This has been changed to remove the actor and inline into the tags api using an ask on the db actor. For now a timeout of 60s is used, but that should probably get changed. Ideally the future will get mapped directly to the data request call and then the server request timeout will be sufficient. That way there would be no nested timeouts. Will follow up on that front in later PRs. * `EvaluateActor`: was broken. It would ignore the `ImperativeRequestContext` messages being sent and the user would get a request timeout. This was also inlined into the evaluate api as there doesn't seem to be a good reason for the separate actor. Still need to create a test case for the evaluate api. 07 February 2017, 17:44:36 UTC
cc79e54 remove WebApi.getJsonParser (#494) Replaces the old `getJsonParser` helper with a directive `jsonParser`. The directive will work for all entity types rather than just strict entities that have a single `ByteString`. 07 February 2017, 15:34:49 UTC
dff38bf fix compiler warnings under 2.12 (#492) 02 February 2017, 02:56:57 UTC
b8b4dce setup cross build for 2.12 (#491) 01 February 2017, 23:04:24 UTC
fc355f7 enable cross build with 2.12.1 and 2.11.8 The 2.12.1 version is now the default version. Travis is set to build for both 2.12 and 2.11. 01 February 2017, 22:45:54 UTC
33a1a31 scala-logging 3.5.0 01 February 2017, 22:01:25 UTC
6eb2f74 sbt-scoverage 1.5.0 01 February 2017, 21:58:11 UTC
6160081 redisclient 3.3 01 February 2017, 21:56:59 UTC
c3bc608 switch from spray to akka-http (#490) Initial attempt at switching from spray to akka-http. 01 February 2017, 21:08:52 UTC
49e9a05 remove use of sleep in SSEActorSuite (#489) 31 January 2017, 19:07:24 UTC
2347719 propagate exception if thrown in constructor (#488) The json decoder will now unwrap the `InvocationTargetException` and throw the cause. This avoids a bunch of reflection non-sense the user does not care about and make it easier to see the root issue for why decoding failed. 31 January 2017, 19:00:01 UTC
0916d5f fix possible IAE if NaN values are in sorted set (#487) In some cases the sort would fail with: ``` IllegalArgumentException: Comparison method violates its general contract! ``` This was caused by having a mix of NaNs in the set of values being sorted. This change switches to use `Double.compare` which takes NaN values into account. 28 January 2017, 05:55:08 UTC
75aa46a update license year (#486) 27 January 2017, 03:08:03 UTC
6a1144d remove test sqs filter from config (#485) The filter for testing was left in the config. As it is not generally applicable it should not be in the base config files. 27 January 2017, 02:56:39 UTC
908011d move ec2 credit to separate block (#484) This makes it easier to query for just the t2 credit metrics rather than all ec2 metrics. 27 January 2017, 02:56:23 UTC
04686b5 add in mapping for t2 cpu credit metrics (#483) 26 January 2017, 17:15:16 UTC
e0972c7 fix bug with equality check on SmallHashMap (#482) For maps with the same length and hashCodes the equality check would fall back to `dataEquals`. That method had a bug where if different keys were in the same position of the underlying data array, then the values would not get checked and the entry would get skipped. This could cause some maps to incorrectly get treated as being equal even though they are not. 26 January 2017, 04:46:44 UTC
e5ccaa9 cleanup imports for lwcapi (#481) 25 January 2017, 21:35:55 UTC
3d8d58f set content type for sse and json responses (#480) 24 January 2017, 05:46:25 UTC
520493a impl of TagIndex using roaring bitmaps (#479) Adds an implementation of the `TagIndex` that uses roaring bitmaps as the underlying integer sets. Will follow up with some benchmarks and then see how it does on real world data sets. 17 January 2017, 16:47:01 UTC
22e306b update dependencies (#478) 16 January 2017, 21:19:21 UTC
back to top