https://github.com/twitter/heron

sort by:
Revision Author Date Message Commit Date
8f81fa6 add debug print before shell kills executor 08 May 2018, 08:20:04 UTC
100d671 replace log(fatal) with manual abort in piper 07 May 2018, 21:26:53 UTC
ffc2449 Fix kryo factor config (#2868) * Fix runtime config patching in stmgr. The old logic store and restore all the configs. However it doesn't work for non-string configs because they have empty value field and the real data is in the serialized_value field which is not covered. * fix unit tests 21 April 2018, 04:20:47 UTC
b6a75f0 Integrate runtime config and rate limit (#2846) * Integrate runtime config and rate limit * more clean up * Fix integration test by avoiding changing hydrated topology in stmgr 12 April 2018, 22:12:03 UTC
ad29cc3 [Streamlet Scala API] Add Scala Streamlet Integration Tests Part I (#2826) * Scala Streamlet Integration Tests Part I * Minor change is applied to TextTransformer * Fix Scala Streamlet Integration Test Packaging Problem * Fix Scala Streamlet Integration Test Result Problem * Add Fix for Travis * Add Fix for Travis 07 April 2018, 23:07:09 UTC
c936cf1 Rate limit needs to be disabled before the bufferevent_free call (#2848) 07 April 2018, 16:06:28 UTC
56bcc6d Add sha256 checksums for all http_archives (#2856) I have taken the current sha256 checksum for all of the workspace archives so we can verify their signature on download. Since some of the artifacts are downloaded over http, we want to make sure there was no man in the middle attack done to change the resulting binary. This also gives assurance that the code we are downloading has not been tampered with in any way either over the wire or at the source. 07 April 2018, 16:03:04 UTC
2a4c1c1 expose heron metrics as a service in Nomad (#2859) 07 April 2018, 15:58:54 UTC
e51093d Add two backup urls for zookeeper (#2857) 06 April 2018, 20:33:19 UTC
ffae7af Joshfischer/eco stateful topology (#2851) * saving work. * adding examples for stateful topologies with ECO 06 April 2018, 05:04:43 UTC
98aa2a2 Eco for heron topologies (#2849) * added cereal serialization/deserialization library * initial restructuring for heron eco * support for heron eco * add the pivot * fixing storm submitter checkstyles * fixing eco checkstyles * checkstyles * added heron eco tests * fixing check styles * add submitter test * fixing ecoSubmitter test * fixing eco submitter test for storm topology * clean up * adding test to verify heron topology submitter * adding test to verify heron topology submitter * fixing checkstyles? * fixed the check style * added the check for topology type to be rigorous * added heron eco examples * modify travis and intellij scripts * fix checkstyle issues 05 April 2018, 14:35:48 UTC
9f3def0 Add json format for dry-run (#2842) This adds JSON as an optional format for the dry-run output. JSON is more easily machine readable than the other output formats and can be used to make deploy decisions based on the resources required and the resources available at deploy time. 04 April 2018, 22:39:03 UTC
860a926 add config for apiserver overrides (#2845) 03 April 2018, 18:55:01 UTC
1c47ef8 While emitting in spout, adhere to the batch size limit (#2798) * While emitting in spout, adhere to the batch size limit * Added accessor * Added missing imports * Make sure that C++ instance also adheres to batch emit size constraint 03 April 2018, 18:54:44 UTC
f5ccbe1 avoid make install for libunwind (#2844) * avoid make install * add a missing file to be output * Added simple easy fix 03 April 2018, 14:13:41 UTC
976f89f Heron JavaDoc and Documentation Fixes (#2838) 02 April 2018, 16:17:12 UTC
5a7faa6 removed glog/gperftools/libevent/libunwind/zookeeper tar.gz from source (#2833) * removed glog/gperftools/libevent/libunwind/zookeeper tar.gz from source instead all of them are downloaded on the fly and compiled WORKSPACE file consists of one entry for each software item and its download URL * fix errors in compilation * added a default user since in a docker environment root user is not set * remove package_dir from libunwind srcs * fix libunwind dependency * fix ubuntu compilation * fix unit errors due to workspace name change * updated the download path for libunwind * truncate lines greater than 100 01 April 2018, 19:53:03 UTC
3822400 update README.md (#2835) * update README.md [https://github.com/twitter/heron/issues/2664] * update README.md further [https://github.com/twitter/heron/issues/2664] * update travis link; add meetup group; [https://github.com/apache/incubator-heron/issues/2834] 01 April 2018, 06:31:03 UTC
47ae988 Fix bug in dry-run argument for update command (#2831) Since list_extra_args is an array the `+=` operator deconstructs a string and add each character as an individual array element. This made it so when the update command it would not detect if it was in dry-run mode and always run the update command. By encapsulating the `--dry-run` string in an list it is properly added as an argument and runs with the expected behavior. This also fixes an issue with the way the `dry_run_format` argument was being updated in the extra args dictionary 01 April 2018, 00:27:26 UTC
17bedb7 removing abstract requirement (#2830) 01 April 2018, 00:26:15 UTC
ddbb98b [Documentation] Improve Java Streamlet API doc (#2819) * Fix Java Streamlet API Documentation * Fix filter operation description 31 March 2018, 08:05:00 UTC
c5caf72 add rate limit change log in stream manager (#2822) 30 March 2018, 06:27:07 UTC
174bbd7 Fix URL issues associated with Hugo upgrade (#2825) * change to list templates where necessary * add missing list template for the docs section 29 March 2018, 23:15:38 UTC
2692700 More website fixes (#2824) * update site layouts for upgraded hugo version * fix JS and CSS import paths * add final step to make site command 29 March 2018, 22:49:32 UTC
ca06dc9 Update Git submodule URL for GitHub pages branch (#2823) * Update GitHug pages branch URL in gitmodules * update make clean command 29 March 2018, 22:01:43 UTC
3a88261 update baseURL for site generation (#2820) 29 March 2018, 21:05:13 UTC
3acd323 Add missing command line options for runtime config (#2818) * Add missing command line options for runtime config * update cli description 29 March 2018, 16:53:54 UTC
0e69468 Allow the apiserver to set the direct memory size. (#2813) * Allow the apiserver to set the direct memory size. * Fix spelling mistake. 28 March 2018, 21:33:10 UTC
af606a7 Add a new tmaster endpoint for runtime config (#2747) * Add runtime config endpoint and handler in TMaster 28 March 2018, 19:05:15 UTC
c86e21e [Streamlet Scala API] Add Scala Example Support - Part II (#2810) * Add Scala Streamlet Examples - Part II * Review comments are addressed. 28 March 2018, 18:00:50 UTC
b0289bf Refactor HttpUploader - Part II (#2812) 28 March 2018, 17:59:46 UTC
a560e59 Fix comment in heron_internals.yaml (#2811) 27 March 2018, 23:22:47 UTC
b325db2 fix library link in WORKSPACE (#2816) 27 March 2018, 19:41:32 UTC
1fce491 modified bazel build to include samoa (#2809) 25 March 2018, 18:19:22 UTC
2141b85 Nomad docker support (#2804) * adding docker support for Nomad Scheduler * fixing formatting * modifying uploader config * cleaning up * disable httpclient logging * setting java home * changing nomad config strategy * removed unnecessary import 24 March 2018, 21:32:42 UTC
038d35a Revert "remove cpu rounding in packing algorithms (#2805)" (#2808) This reverts commit c25edf55fa23a6dc2d97184dff2400cf3827dea5. 24 March 2018, 08:33:30 UTC
8a00cc1 Add memory unit to k8s resource request. (#2802) * Add memory unit to k8s resource request. * Add link to documentation. 23 March 2018, 20:06:35 UTC
c25edf5 remove cpu rounding in packing algorithms (#2805) 23 March 2018, 20:05:53 UTC
5310516 Do not include scribe sink when compiling on non x86 platforms. (#2801) * Do not include scribe sink when compiling on non x86 platforms. This is due to scribe dependency on thrift which needs to be generate java files - the binaries used are only x86 platforms. * reverted the change tools/platform - not relevant for this PR * updated the docker rules version 23 March 2018, 20:04:53 UTC
2b0aa28 Add per component resource config for cpu and disk (#2800) * Add per component configs for cpu and disk * change all cpu related config/functions to use double instead of float 23 March 2018, 19:47:31 UTC
afb96c9 Fix a typo and comments/doc for per component rate limit config (#2796) 22 March 2018, 06:06:41 UTC
71c7143 [Streamlet Scala API] Add Scala Example Support - Part I (#2791) * Add Scala Streamlet Examples - Part I * minor refactoring is being applied. * Add precondition check for Topology Name * Kryo dependency is being added. * Review comments are addressed. 21 March 2018, 06:14:57 UTC
d3a49a7 [Streamlet Scala API] Add Scala StreamletImpl Support - Part V (#2790) * Add Scala StreamletImpl Support Part 5 * minor refactoring is being applied. 21 March 2018, 06:14:40 UTC
58bd88a Add rate limiter in stmgr (#2783) * Add rate limiter in stream manager for instance connections * Add unit tests and per-instance rate limit 20 March 2018, 23:41:08 UTC
49a806a Correct print errors (#2789) 20 March 2018, 20:46:34 UTC
40385a9 Fix error message when large instance is packed (#2784) Fixes #2777 19 March 2018, 16:02:02 UTC
b8b1490 Add a new escaping '(61)' for '=' because the original one '='… (#2785) * Add a new escaping '(61)' for '=' because the original one '=' is not friendly to bash * Update comments about why '=' is needed * remove escaping during encoding 19 March 2018, 00:21:44 UTC
a2c1c2b Add runtime config in heron update command (#2760) * Add runtime config support in cli and runtime manager 19 March 2018, 00:20:52 UTC
f446135 Fix tracker instance groups. (#2782) * Fix tracker instance groups. * Fix spelling error. 15 March 2018, 22:52:17 UTC
467bbaa add a switch for metricscache (#2744) * add switch for metricscache * fix ning comment * fix unit test * fix ning comment, add to config 15 March 2018, 20:34:50 UTC
1322440 [healthmgr] add a class to expose metrics to local metricsmgr (#2771) * add a class to expose metrics to local metricsmgr * fix ning coment 15 March 2018, 20:34:34 UTC
a553ced [Streamlet Scala API] Add Scala StreamletImpl Support - Part IV (#2767) * Add Scala StreamletImpl Support Part 4 * Review comments are addressed. 15 March 2018, 02:33:41 UTC
d0e226a Add a topology config endpoint to the tracker. (#2781) 15 March 2018, 02:31:05 UTC
6351108 Small change to Nomad doc (#2780) * change intro paragraph in nomad doc * add note on small plus large scale 14 March 2018, 17:45:20 UTC
3519605 Retain heron-core package for YARN like schedulers (#2775) Fixes #2774 13 March 2018, 22:14:27 UTC
26bee3d [Streamlet Scala API] Add Scala StreamletImpl Support - Part III (#2764) * Add Scala StreamletImpl Support Part 3 * Review comments are addressed. * Review comment is addressed. * Merge conflicts and compile warnings are fixed 13 March 2018, 01:00:00 UTC
74ddb5e Builder creation (#2769) * Builder code initial cut * more progress on Builder * new files and fixes for Builder * added BuilderImpl * more changes for Builder * added unit tests and initial implementation of Builder * Added unit tests and initial implementation of Builder Builder code initial cut more progress on Builder * resolve merge conflicts * more merge conflicts * merge conflicts * PR feedback * test failure fixes * removed extraneous file that is no longer needed * added the override method for newSource created from Source[R] object * pr feedback responses 12 March 2018, 15:19:46 UTC
e932a1a Fix heron update on k8s. (#2770) 10 March 2018, 06:34:39 UTC
69a6677 Improve Communicator tests (#2768) * Improve unit tests for Communicator * more 10 March 2018, 05:33:02 UTC
5b5cabc [Streamlet Scala API] Add Scala StreamletImpl Support - Part II (#2761) * Add Scala StreamletImpl Support Part 2 * Review comments are addressed. * Review comments are addressed. 09 March 2018, 16:11:54 UTC
909b62c fixing bug in wakeable loop causing IndexOutOfBoundException (#2753) * fixing bug in wakeable loop causing IndexOutOfBoundException 08 March 2018, 02:00:35 UTC
c4c5d5b Fix heron-explorer components command. (#2762) 04 March 2018, 22:43:13 UTC
7b7c385 Re-organize documentation nav (#2756) * re-organize TOC * streamline Makefile and JS imports * add node_modules removal to 'make clean' command * use non-jQuery function for app.js * more gulpfile re-organization * finish fixing JS/CSS pipeline * final version of new nav 01 March 2018, 22:47:40 UTC
b41a6e9 Add Scala StreamletImpl Support (#2740) 01 March 2018, 18:12:52 UTC
574b007 [API] Improve HeronSubmitter (#2746) * Improve HeronSubmitter * TestSpout2 is updated * Review comment is addressed. 01 March 2018, 07:02:18 UTC
d53a6f2 fix WindowedWordCountTopology (#2755) * fix WindowedWordCountTopology * fix checkstyle 28 February 2018, 18:40:56 UTC
d37ec86 adding additional info to standalone documentation (#2748) 28 February 2018, 18:20:29 UTC
9fb33cd fix bug in stateful windows (#2754) 28 February 2018, 18:16:43 UTC
52d0c6c Pass role and environment to hava topoology components. (#2752) 28 February 2018, 17:39:00 UTC
5a72bdc [HttpUploader] Add Documentation (#2750) * HttpUploader Documentation is being added. * Review comment is addressed. 28 February 2018, 08:40:11 UTC
fc08d52 Fixed example name in getting-started.md (#2751) I can't find any reference to `WordCountDslTopology` anywhere but here, it looks like the correct classname might be `WindowedWordCountTopology`? That is, if I've chosen the correct jar file.. 27 February 2018, 22:35:53 UTC
ada6052 fixing bugs in the API server when downloading files (#2749) * fixing bugs in the API server when downloading files * addressing comments * cleaning up * refactoring some code 27 February 2018, 03:15:35 UTC
2f26b49 Documentation for the Nomad scheduler (#2723) * add new nomad doc and add doc to TOC * add list of uploaders to nomad doc * configuration and other instructions for Nomad * finish draft of nomad docs * fix typos 26 February 2018, 18:53:20 UTC
acf1fa2 Allowing API Server to distribute heron core package (#2724) * Allowing API Server to distribute heron core package * modifying endpoint 25 February 2018, 06:32:36 UTC
08c254d Allow user to run kill command when some topology data is missing (#2730) * Allow user to re-run kill command when the previous kill or submit command was interrupted and topology data is only partially available (and heron kill command couldn't pass the validation) 24 February 2018, 05:07:45 UTC
6f43cd7 [#2735 follow up] move out of core into aurora scheduler (#2741) * move 2735 out of core into aurora scheduler * fix fdai comment 23 February 2018, 08:56:14 UTC
c1c6d4a Fix installation docs (#2742) * remove mention of multiple scripts * add homebrew docs 23 February 2018, 06:29:03 UTC
f5dc4b3 add warning when `aurora job add` command fails during `heron update` (#2735) add warning when `aurora job add` command fails during `heron update` 18 February 2018, 09:01:24 UTC
7991716 add topology version in tracker runtimestate (#2739) 17 February 2018, 04:36:10 UTC
bee6bf8 include eco into storm compatibility jar (#2736) 16 February 2018, 10:33:06 UTC
f2719e3 JavaCheckstyle fix for IntelliJ setup script (#2738) 16 February 2018, 09:48:44 UTC
c5ab95d [healthmgr] add healthmgr src exportation (#2734) * export healthmgr src * export deps as well * revert deps exportation 14 February 2018, 12:24:34 UTC
b393193 Check that current process is not session master (#2731) 14 February 2018, 05:30:14 UTC
3c2e0a9 update heron version in config.yaml (#2732) 13 February 2018, 21:06:53 UTC
68643ed Refactor hostname logic (#2718) * Refactor hostname logic * Fix checksytle issues 13 February 2018, 03:47:55 UTC
a2f0008 Add Sink Trait Support (#2726) 12 February 2018, 18:10:20 UTC
152f47e Log configs in packing algorithms (#2727) 11 February 2018, 00:18:31 UTC
1ff5d02 Joshfischer/eco docs redeux (#2703) * adding back changes to eco examples * adding back changes to eco main * adding back changes to eco test * adding back ide set up * adding back static docs * removing uneeded code * fixing checkstyles * adding jvm options to eco config * adding test coverage and confirmation of setting component level ram specs and container JVM arguments * clean up * adding property substitution, still need to cover with tests * modify bazel rule to include properties files * starting work to include environment variable substitution * adding test to validate property substitution * explicitly declaring filter option not required * adding validation that enums will work * adding more to docs * add hyperlink to other eco examples * fixing verbiage * clean up docs * verifying property substitution * adding final changes * adding disclaimer * adding link to environment variable subsitution * removing whitespace * adding logging to deployment process 10 February 2018, 18:51:06 UTC
f554b79 fix null in printout during submission (#2722) * fix null warning * revert function sig * fix unit 08 February 2018, 06:28:45 UTC
7cab3f3 use actual path (#2725) 08 February 2018, 02:46:54 UTC
81a75f4 Simplify website setup (#2721) * remove nodejs install from macos setup * update README 07 February 2018, 20:07:48 UTC
75c25a0 deprecate RRPacking in default config (#2717) * deprecate RRPacking in default config * update to the same alg * fix typo 07 February 2018, 17:41:21 UTC
6cba81f adding some confs for heron nomad and standalone (#2716) * adding some confs for heron nomad and standalone * fxing nomad client.yaml 07 February 2018, 15:26:33 UTC
ad3ecfc fix a RRrepacking bug (#2720) * fix a bug * update unit 07 February 2018, 02:46:49 UTC
a8a4a3c mitigate prompt (#2719) 07 February 2018, 00:30:34 UTC
ca7d5ca [Streamlet Scala API] Add FunSuite Support (#2695) * Add FunSuite Support to Streamlet Scala API * Remove unused spec2 rule * Review comments are addressed. 06 February 2018, 22:49:00 UTC
8f36a93 Move stmgr quitting logic from stmgr client to tmaster client (#2711) * Move stmgr quitting logic from stmgr client to tmaster client We found out that when a bad stmgr is experiencing connection issue to all other stmgrs, it takes the stmgr a lot longer to reach the per stmgr client max attempt and quite becaus bad connections cause extra delays. In our case, instead of quitting after 5 minutes (1s reconnect timeout, 300 max attempts), the bad stmgr quitted after more than 2 hours. At the same time, all other good stmgrs were quitting every 5 minutes. Here is a more detailed document about the issue: https://docs.google.com/document/d/1WHNc2NEp2gVL9ge2QVKp9t4Hpd4U9sAbzBqCu4-iDUM/edit# 06 February 2018, 05:55:34 UTC
9ffeb39 fixes the helm chart, config and api server memory restriction for k8s (#2714) 06 February 2018, 05:35:01 UTC
c62ec91 Expose the component config through the tracker. (#2709) 06 February 2018, 03:33:23 UTC
bc5236e Add Streamlet Trait Support (#2696) 05 February 2018, 19:39:14 UTC
back to top