https://github.com/twitter/heron

sort by:
Revision Author Date Message Commit Date
bf9fe93 changing watermark generator to be triggered by tick tuples for event time based windowing (#2370) * changing watermark generator to be triggered by tick tuples for event time based windowing * fixing unit test 01 October 2017, 15:32:49 UTC
f8cd589 Fix operator to support python 2 and 3. (#2369) * Fix operator to support python 2 and 3. * Allow query operators to compile on python 2 and 3. 01 October 2017, 03:33:34 UTC
72d5bfb Add component type label for prometheus metrics. (#2364) * Add component type label for prometheus metrics. * Remove unused variable. * Do not assume default for __execute-time-ns metric. * Fix code style error. 30 September 2017, 23:23:50 UTC
02312d5 Create connection buffer and metrics map every time the connection comes up (#2366) 30 September 2017, 01:21:31 UTC
5f1b8bd Add helper method to specify ram (#2365) * Add helper method to specify ram * Made naming more java friendly 29 September 2017, 16:11:03 UTC
af70b45 Seperate out the low level api examples and higher level dsl examples (#2361) * Seperate out the low level api examples and higher level dsl examples * Fix package targets 29 September 2017, 03:44:53 UTC
d51021c Changing TimeTriggerPolicy to use tick tuples as a timer (#2357) Changed Tick Tuple to frequency to milliseconds 29 September 2017, 01:41:34 UTC
6d27c3c remove note on production usage (#2363) 29 September 2017, 00:47:57 UTC
8fa07fc Make it clear that the Functional API is in beta (#2362) * merge website with master * add beta mode alerts 29 September 2017, 00:18:53 UTC
4e56512 re-address submodule issue (#2360) 28 September 2017, 23:08:33 UTC
492acf4 Heron DSL documentation (#2352) * add generated CSS files to gitignore for website * add benefits of DSL * add more info on partitioning, why the DSL, etc * add link to delivery semantics image * adjust CSS for blockquote elements * add description of KV streamlets * finish rough draft * remove unnecessary image for now * bump version and add note on type safety 28 September 2017, 23:02:22 UTC
a402049 Fix website submodule issue (#2359) * revert to older commit * revert website * remove stray references to DSL 28 September 2017, 20:58:25 UTC
35a351a Small fixes for new Java DSL docs (#2358) * adjust heron version * s/declarative/functional * s/dsl/api * remove usage of DSL * fix example repo URL * fix broken url 28 September 2017, 19:15:42 UTC
2b18743 Heron Java DSL docs (WIP) (#2342) Heron Java DSL docs 28 September 2017, 18:38:50 UTC
a471aff Fix src collection for java api (#2356) 27 September 2017, 23:10:21 UTC
eb90566 More Python3 Friendliness (#2355) * Second round of python3 * More Python3 * long to int * convert reduce to use functools * More python3 * more python3 27 September 2017, 23:02:09 UTC
cde3cf5 fix intellij setup script (#2354) 27 September 2017, 18:30:26 UTC
022d645 Move java dsl to be part of heron api (#2353) 27 September 2017, 01:31:11 UTC
1c94687 Added a generic source that has open/cleanup. Also renamed a few functions (#2351) 26 September 2017, 15:44:10 UTC
1b37539 Fix mktemp issue and revert ci-docker (#2344) 25 September 2017, 22:49:50 UTC
d8049f4 fix simulator for not initializing instance (#2347) * fix simulator * fix unit test 25 September 2017, 19:31:18 UTC
a55490d Fix website asset pipeline (#2348) * move img folder into static * update gitignore * remove static folders from git * update gitignore * fix make clean command * remove gulp clean command * add font awesome CSS 25 September 2017, 18:42:05 UTC
9e8ebfe Delivery semantics documentation (WIP) (#2324) * add new delivery semantics doc * add stateful processing doc * fix CSS setup and add delivery semantics image * remove stateful processing doc for now 25 September 2017, 17:18:55 UTC
0214a6b heronpy now needs protobuf 3.4.0 (#2345) 25 September 2017, 13:53:40 UTC
3f1c55c Checkpoint manager does not support override (#2343) 25 September 2017, 01:58:26 UTC
9ccedee Fix missing artifact for docker build (#2341) 24 September 2017, 22:39:43 UTC
d9ef89d Add heron cli config setting for service_url. (#2326) * Add heron cli config setting for service_url. * Remove commented out code. * Change cli config to yaml format. * Update help messages. * More help message changes. 23 September 2017, 20:15:30 UTC
9342b14 clean heron java instance for stateful processing (#2336) * initial commits to fix instnace with stateful processing functionality * fix instance stateful processing * fix typo and refine example * Fixed bug * clean debug code * clean comments * address comments 23 September 2017, 14:36:22 UTC
c5214b6 After building make sure that all the components have been built (#2340) 23 September 2017, 02:53:01 UTC
124511b Consider override yaml (#2339) 23 September 2017, 02:31:38 UTC
d568f71 Fix default acking (#2338) 22 September 2017, 23:53:49 UTC
5b6bb2f Make delivery semantics public (#2337) 22 September 2017, 21:41:44 UTC
72cff11 Add prometheus scrape annotations to pods in kubernetes scheduler. (#2331) * Add prometheus scrape annotations to pods in kubernetes scheduler. * Remove unused code. 22 September 2017, 17:31:18 UTC
a1705f7 Added a Config class to remove api.Config depedency (#2334) * Introduced Config into the dsl to remove dep on heron.api.Config * Updated comments * Removed unneeded interface * Made transform function more general * Fix checkstyle 22 September 2017, 03:16:42 UTC
6cdfd35 Python3 compatibility (#2335) * Switched from 2.7 to generic python * Removed 2.7 * made all print statements python2 and python3 friendly * Made it python3 friendly * Upgraded unittest to use 1.1.0 * Made print statements compliant with both python2 and python3 * Made statemgrs python2 and 3 compatible * Make raise and print python3 compativle * Made python3 compatible * More python3 compat * Move to beginning of file * python3 fying * Python3fy * Fixed compile bug * Style changes * More python3fy * Python3fy * Pythonify * More Pythonify3 * Unwind some changes for now * More python3fy * More python3fy * Python3fy 22 September 2017, 03:14:54 UTC
44586a5 pex_pytest migration (#2333) 21 September 2017, 03:03:54 UTC
c84ec53 Revert "Take override.yaml in stmgr, metricsmgr, metricsmgr-cache, tmaster, instance (#2311)" (#2332) This reverts commit 3df0a4db2cb51a6bd96df7afd0b9d9eb3b6ada96. 20 September 2017, 23:42:08 UTC
3df0a4d Take override.yaml in stmgr, metricsmgr, metricsmgr-cache, tmaster, instance (#2311) 20 September 2017, 17:14:14 UTC
839649a clean python state manager (#2327) * clean python state manager * update tracker 20 September 2017, 17:06:26 UTC
07713e9 Add prometheus sink to default kubernetes metrics config. (#2329) 20 September 2017, 07:03:25 UTC
389b3f5 Java DSL for Heron (#2285) * Placeholder for Java DSL * More changes to address comments * Compiling version * Added FlatMap * Add filter * Added repartition * Added WindowConfig * First cut of JoinStreamlet * Fixed comparison bug * Added reduce * Added ability to create KVStreamlet from Streamlet * Added java docs * Added Union and reduceByWindow functions * Introduced clone and generalized repartition function * Make Window bolt conform to the proposed window interface * Finally compiling * Fix style checks * Moved Bolts to bolts dir and Streamlets to streamlets dir * Convert Streamlet/KVStreamlet into interfaces * moved window config to base * Moved all impl classes into impl * Made the Streamlet interface generics based * Added inner/left/outer joins * Add support for atleast once semantics by doing acking * Assign WindowConfig parameters to the WindowBolt * Made non windowed bolts stateful * Introducing Builder and keeping track of all children * Calculate children for kvstreamlets as well * Moved StreamletImpl and KVStreamletImpl to more intuitive locations * Moved groupings to their own dir * Adjusted the build into the new semantics * Introduced a Context class that allows us to run dsl streamlets * Added a createStreamlet interface to have an easy way to create Streamlets * WordCount Topology using Java DSL. * Fixed logic while building * Added SerializableSupplier concept to make Supplier Spouts Serializable * Introduced SerializablePredicate * Added SerializableBinaryOperator * Added SerializableBiFunction * Introduced SerializableFunction * Made KeyValue Serializable. With this the word count topology is now running * Added a trivial sink log * Reduced Operations should return WindowInformation * Updated java docs * Added JavaDocs for groupings * Added JavaDocs to spouts * Added Javadocs for all streamlet implementations * Added a ContextImpl class and made Context an interface * Added WindowConfigImpl to hide implementation details * Renamed as per feedback * Fixed build * Took feedback * Took feedback * Took feedback * Took feedback * Added a generic to function as a sink * Renamed spouts to sources and bolts to operators * Added sinks folder * Moved Spout references to Source * Moved Bolts to operators * Renamed Bolts to Sinks * Renamed Impl to Base * Took Bill's suggestion * Renamed Context as Runner * Added a transform operator that gives users more flexible transforms * Renamed to newSource * Renamed window functions based on feedback * Renamed to toSink * added count info to windows * Added check for name * Fixdd misplaced comments * Fixed comments * Refactored common code to single place * Took feedback into account * Fixed comments * Took feedback * Moved the examples to the newly formed example * Added unittest for BaseStreamlet * Added unittest for KVBaseStreamlet 20 September 2017, 05:48:20 UTC
9a4d39e upgrade to protobuf3.4 (#2316) 19 September 2017, 20:40:44 UTC
a507947 take into account feedback from Neng (#2322) 19 September 2017, 18:06:46 UTC
3d8aed1 clean tmater client (#2321) * clean tmater client 19 September 2017, 17:15:11 UTC
a0d80a3 Issue 2219: Provide a bookkeeper based uploader and downloader (#2220) * BK Uploader * Add a BK downloader * remove local repo * Add all dependencies * Fix * Fix errors after bumping guava to 20 * Add more tests * Address checkstyle problems * fix typo * Move heron/dlog to `heron/io/dlog` * Use `shaded` distributedlog jar * Include dlog uploader in apiserver and tools * Allow configuring num-replicas for topologies stored in bookkeeper * Remove redundant dependency * Revert "Use `shaded` distributedlog jar" This reverts commit c63ffbb9034d50211a649cbdca4859b6e079de95. * Revert "Revert "Use `shaded` distributedlog jar"" This reverts commit 9595aca4e62dcc129def11309b06947e5bda0301. * use the new shaded jar * update * Provide a dlog tool for verifying downloader and uploader * Add a README instruction for how to use the dlog tool for verifying downloader and uploader * Disable write lock * Use latest 0.5.0 release * Fix the guava dependency in tests * Fix checkstyle problems * private constructor * Revert the Extractor from singleton to static methods * Make extractor final * Fix checkstyle 19 September 2017, 05:22:02 UTC
dd5f2d7 fix python state manager (#2319) * heron executor exits when statemanager raises exception 19 September 2017, 05:12:38 UTC
481064e Added a Stateful example topology. (#2318) * Added a Stateful example topology. Also fixed a bug while running this * Added the example topology 19 September 2017, 04:39:37 UTC
f00a9f1 Add prometheus metrics web sink. (#2243) * Add prometheus metrics web sink. Provides a simple web server that exposes and endpoint for prometheus to scrape metrics. * revert local metrics_sinks config and remove commented out code. * Remove instance id from stmgr metrices when exporting to prometheus. * Revert local metrics_sink.yaml config. * Rename Metrics to MetricsUtil. 19 September 2017, 03:07:40 UTC
71676ab Seperate Stmgr and Instance Serving in Stmgr (#2313) * Refactored stmgr and instance related communication to seperate classes * First compiling version * Tests compile * More compile time fixes * Pass local data port to stmgr * ensure that we preserve backwards compat with metrics 19 September 2017, 00:58:16 UTC
9ff9889 Upgrade pex (#2314) * upgrade pex in place 18 September 2017, 20:21:07 UTC
f3ea101 Add image pull policy param for kubernetes scheduler. (#2315) 18 September 2017, 05:17:22 UTC
f87d05e Always show links button (logs, job, exceptions, etc..) for instances. (#2312) 15 September 2017, 20:12:00 UTC
2f65be5 Use bufferevent to do Network I/O (#2297) 14 September 2017, 19:12:38 UTC
2d96ddc Fixed the path of the python examples (#2305) 14 September 2017, 06:07:40 UTC
1a4dd55 Bug fixes and integration test for Python DSL (#2304) * Bug fixes and integration tests for python dsl * cleaning up * addressing comments 14 September 2017, 06:07:23 UTC
b12c7e6 fix bugs in stmgr unit test (#2306) * fix stmgr unit test * fix reconnect timer 14 September 2017, 03:57:31 UTC
b06fd91 Split executor args in aurora file so easy to read diff. (#2303) 13 September 2017, 21:18:06 UTC
3843924 Refactoring storm api (#2300) * refactoring storm compatability code and api - Moved all the Storm API to storm-compatibility/ - Modified heron/examples to use storm api only and move to storm-compatibility-examples/. Also removed all dependencies of the twitter API from these examples - Modified heron/examples to use the heron api only and moved to examples/ - Made sure the correct headers are on the storm api related files * fixing storm compatibility api paths * adding back logic to use simulator * fixing examples paths 12 September 2017, 21:48:34 UTC
4dae714 fix tmaster client sends incomplete register request to tmaster (#2291) 12 September 2017, 19:14:56 UTC
150c4a4 Add verbose logging to apiserver. (#2288) 12 September 2017, 17:59:45 UTC
9799dba Add error logging for topology actions on the apiserver. (#2296) * Add error logging for topologoy actions on the apiserver. * Fix spelling. 12 September 2017, 17:54:16 UTC
eb52b81 Fix Python documentation generation issues (#2302) 11 September 2017, 22:17:19 UTC
9f4db0e remove startup scripts (#2298) * remove startup scripts * remove the copying of script file 11 September 2017, 02:50:02 UTC
12c80ac Register requests are sent by instance as request/response pairs (#2294) 08 September 2017, 20:57:27 UTC
19db90d Heron Storm Windowing Compatibility Layer (#2292) 08 September 2017, 17:58:41 UTC
a2acab8 Correcting integration tests structure (#2293) * correcting integration tests structure * correcting build file 08 September 2017, 13:08:57 UTC
2417058 Use Piper class instead of repeating functionality (#2289) 06 September 2017, 21:36:24 UTC
2e7bc17 adding additional unit and integration tests for heron windowing support (#2284) * adding additional unit and integration tests for heron windowing support * moving tests 06 September 2017, 18:52:06 UTC
1f5b57b Reduce example topologies resources. (#2276) * Redeuce example topologies resources. * Reduce examples resources more. 06 September 2017, 04:41:56 UTC
3d18239 healthmgr: aurora heron shell controller (#2283) * healthmgr add aurora heron-shell controller * fix style * wrapper class * address maosong comment * address maosong comment: throw excetion to external logic * update example yaml * fix style 06 September 2017, 03:18:17 UTC
c309ecc Add config mapping for gcs bucket in api server deploy. (#2281) * Add config mapping for gcs bucket in api server deploy. * User heron image. 03 September 2017, 08:22:03 UTC
44e2cd6 cleanup docker scripts (#2277) * add scripts for startup docker and include client install * start a few topologies when you start the docker * added a conf file to start essential services in docker in sandbox mode * docker changes * streamlined docker stuff * remove start services * add supported platforms 03 September 2017, 08:18:33 UTC
21c2378 Add kubernetes service for apiserver. (#2280) * Add kubernetes service for apiserver. * Add comment to service component. 03 September 2017, 05:45:48 UTC
bd77cdf Notify both producer and consumer when init the Communicator (#2279) Currently, when we init the Communicator, the changes will be picked up only when the producer/consumer are notified by other events. It can lead to the race condition. This pull request fixes the issue by notifying both producer/consumer immediately to run with new values. 03 September 2017, 03:07:58 UTC
5a5912b add error log for physical plan check in tmaster (#2267) 01 September 2017, 21:07:00 UTC
bda9ef0 Add kubernetes deploy files. (#2265) * Add kubernetes deploy files. * Make default zookeeper deployment have only 1 container. * Move apiserver yaml config to gcp subdirectory. 01 September 2017, 20:29:54 UTC
da081a7 consistent behavior about when to shade. Shading is done when the final artifacts are generated (#2273) 01 September 2017, 20:29:01 UTC
40a5447 Normalize Shading and fix simulator build (#2271) 01 September 2017, 15:36:50 UTC
110ebd0 Calculate the hostname during constructor rather than during register (#2270) 01 September 2017, 15:36:35 UTC
ca8a05f Make shading uniform accross instance and api (#2266) 31 August 2017, 19:03:39 UTC
23dcad4 Add topology name validation for kubernetes scheduler. (#2264) 31 August 2017, 00:55:56 UTC
aedeab4 [ISSUE-1124] - Windows Bolt support (#2241) https://github.com/twitter/heron/issues/1124 31 August 2017, 00:07:26 UTC
bdd66b3 Read http response for all http response codes. (#2263) * Read http response for all http response codes. * Remove unused import. 30 August 2017, 22:20:55 UTC
0020bd3 ensure empty function with unique names (#2262) 30 August 2017, 20:39:03 UTC
fadd863 Add zookeeper to the heron dist docker ubuntu14.04 (#2259) * Add zookeeper to the heron dist docker ubuntu14.04 * Remove zookeeper home env variable from container. * remove versioned zookeeper directory. 30 August 2017, 19:45:19 UTC
fb441ef changes to make heron compile with bazel 0.5.4 (#2207) * changes to make heron compile with bazel 0.5.1 * changes to compile with bazel 0.5.2 * added bazel 0.5.2 for travis * remove crosstool_top for travis with bazel 0.5.2 * remove crosstool_top for applatix with bazel 0.5.2 * added support for bazel 0.5.4 rc4 candidate * fixed warnings in pkg_tar and heron downloader * upgrade to new bazel 0.5.4 release * changed the path for heron-python-instance target * updated docs to use bazel 0.5.4 and updated heron version to be 0.15.1 30 August 2017, 18:30:46 UTC
c44c4f0 [ISSUE-2170] fix tracker crash when zk root topologies path does not … (#2260) * [ISSUE-2170] fix tracker crash when zk root topologies path does not exist. * Fix checksytle. 30 August 2017, 17:54:43 UTC
b094a62 fix the executor unit test (#2261) 30 August 2017, 07:31:46 UTC
d9176f0 healthmgr check null return value (#2258) * fix null exception * add verbose 29 August 2017, 23:36:14 UTC
9e74bf5 Support components name generated by SummingBird. (#2256) 29 August 2017, 17:32:10 UTC
9df0e59 Renamed Exactly Once to Effectively Once (#2254) * Renamed Exactly Once to Effectively Once * Substitute exactly once with effectively once * Replaced exactly once with effectively once 29 August 2017, 16:58:36 UTC
e931d49 add `restartcontainer` resolver to healthmgr (#2230) * add restartcontainer resolver * add physicalplanprovider * update healthmgr.yaml for local test * fix styple and compile * fix maosong comment * fix compile * GrowingWaitQueueDetector removed * fix ashvin comment * fix compile * change min to ms * add `final` * fix 1. `final` ,2. `empty list`, 3. `synchronized`. 4. `toString()->getName() * add `sync` * refactor PhysicalPlanProvider.get() 29 August 2017, 00:07:48 UTC
8b98ebd Add src id in exception message. (#2252) 28 August 2017, 20:07:09 UTC
15a4621 Move system_config to from common to instance since only instance uses it (#2251) * Move system_config to from common to instance since only instance uses it * Added system config to the init file 27 August 2017, 22:47:55 UTC
57daad4 Move EventLooper and GatewayLooper to instance since no one else uses them (#2250) * Move EventLooper and GatewayLooper to instance since no one else uses them * Fix path * Fix path. Attempt 2 27 August 2017, 21:15:39 UTC
af2a0c6 Moved system_constants from common to instance since only instance uses it (#2249) 27 August 2017, 18:54:38 UTC
4e4bbb1 Deleted unused file (#2248) 27 August 2017, 16:41:36 UTC
fed3aff Include only Topology proto in heron java api jar (#2247) 26 August 2017, 23:52:00 UTC
987975d Remove unused proto (#2246) 26 August 2017, 17:57:51 UTC
d816f44 Upgraded zookeeper to the latest stable version (#2245) 26 August 2017, 16:04:18 UTC
back to top