| f707856 | Matei Zaharia | 24 May 2011, 22:59:01 UTC | Removed java-opts.template | 24 May 2011, 22:59:01 UTC |
| cec427e | Matei Zaharia | 23 May 2011, 00:12:29 UTC | Fixed a bug with preferred locations having changed meaning in new RDDs | 23 May 2011, 00:12:29 UTC |
| 4c888b2 | Matei Zaharia | 22 May 2011, 23:42:05 UTC | Fix queue type for executor | 22 May 2011, 23:42:05 UTC |
| bea3a33 | Matei Zaharia | 22 May 2011, 23:03:41 UTC | doc tweak | 22 May 2011, 23:03:41 UTC |
| 9bde5a5 | Matei Zaharia | 22 May 2011, 23:00:41 UTC | class loader fix | 22 May 2011, 23:00:41 UTC |
| 91c07a3 | Matei Zaharia | 22 May 2011, 05:50:08 UTC | Various fixes to serialization | 22 May 2011, 05:50:08 UTC |
| f61b61c | Matei Zaharia | 22 May 2011, 04:25:58 UTC | Merge branch 'master' into new-rdds | 22 May 2011, 04:25:58 UTC |
| 24a1e7f | Matei Zaharia | 20 May 2011, 07:19:53 UTC | Scheduler can now recover from lost map outputs | 20 May 2011, 07:19:53 UTC |
| 82329b0 | Matei Zaharia | 19 May 2011, 19:47:09 UTC | Updated scheduler to support running on just some partitions of final RDD | 19 May 2011, 19:47:09 UTC |
| 328e51b | Matei Zaharia | 19 May 2011, 18:19:25 UTC | Various minor fixes | 19 May 2011, 18:19:25 UTC |
| fd1d255 | Matei Zaharia | 17 May 2011, 19:41:13 UTC | Stop objectifying various trackers, caches, etc. | 17 May 2011, 19:41:13 UTC |
| 4db50e2 | Matei Zaharia | 13 May 2011, 19:03:58 UTC | Fixed unit tests by making them clean up the SparkContext after use and thus clean up the various singletons (RDDCache, MapOutputTracker, etc). This isn't perfect yet (ideally we shouldn't use singleton objects at all) but we can fix that later. | 13 May 2011, 19:03:58 UTC |
| aca8150 | Matei Zaharia | 13 May 2011, 18:43:52 UTC | Ensure that AddedToCache messages make it home before tasks finish | 13 May 2011, 18:43:52 UTC |
| 16c886a | Matei Zaharia | 13 May 2011, 17:41:34 UTC | Optimization for count() | 13 May 2011, 17:41:34 UTC |
| 4b1f0f1 | Matei Zaharia | 13 May 2011, 04:34:38 UTC | Merge pull request #48 from ankurdave/bagel-new Bagel: Large-scale graph processing on Spark | 13 May 2011, 04:34:38 UTC |
| f40a089 | Ankur Dave | 09 May 2011, 22:23:21 UTC | Rename bagel to spark.bagel and Pregel to Bagel | 09 May 2011, 22:23:21 UTC |
| 7e20648 | Matei Zaharia | 09 May 2011, 21:48:39 UTC | Upgraded to SBT 0.7.5 | 09 May 2011, 21:48:39 UTC |
| 4bedf5b | Matei Zaharia | 09 May 2011, 21:22:56 UTC | Merge pull request #47 from ankurdave/cache-to-disk Merging in Ankur's code for a cache that spills to disk | 09 May 2011, 21:22:56 UTC |
| c110405 | Ankur Dave | 04 May 2011, 01:53:58 UTC | Move shortest path and PageRank to bagel.examples | 04 May 2011, 01:53:58 UTC |
| 563c5e7 | Ankur Dave | 23 April 2011, 20:38:37 UTC | Refactor and add aggregator support Refactored out the agg() and comp() methods from Pregel.run. Defined an implicit conversion to allow applications that don't use aggregators to avoid including a null argument for the result of the aggregator in the compute function. | 03 May 2011, 22:40:45 UTC |
| c18fa3e | Ankur Dave | 16 April 2011, 04:40:54 UTC | Package combiner functions into a trait | 03 May 2011, 22:40:41 UTC |
| 1c8ca0e | Ankur Dave | 15 April 2011, 20:43:29 UTC | Add Bagel test suite Note: This test suite currently fails for the same reason that the Spark Core test suite fails: Spark currently seems to have a bug where any test after the first one fails. | 03 May 2011, 22:40:31 UTC |
| c5b3ea7 | Ankur Dave | 14 April 2011, 01:40:44 UTC | Clean up Bagel source and interface | 03 May 2011, 22:40:01 UTC |
| 19122af | Ankur Dave | 14 April 2011, 00:19:41 UTC | Update ShortestPath to work with controllable partitioning | 03 May 2011, 22:39:39 UTC |
| 45ec9db | Ankur Dave | 13 April 2011, 21:59:19 UTC | Add Bagel classpath to run script | 03 May 2011, 22:39:21 UTC |
| 62ef620 | Ankur Dave | 13 April 2011, 06:38:58 UTC | Clean up Pregel.run, add logging | 03 May 2011, 22:38:01 UTC |
| c0736f6 | Ankur Dave | 13 April 2011, 00:57:04 UTC | Add Bagel, an implementation of Pregel on Spark | 03 May 2011, 22:37:08 UTC |
| a4c04f3 | Ankur Dave | 28 April 2011, 06:23:29 UTC | Error handling for disk I/O in DiskSpillingCache Also renamed the property spark.DiskSpillingCache.cacheDir to spark.diskSpillingCache.cacheDir in order to follow conventions. | 28 April 2011, 06:23:29 UTC |
| 12ff0d2 | Ankur Dave | 28 April 2011, 05:59:05 UTC | Bring an entry back into memory after fetching it from disk | 28 April 2011, 05:59:05 UTC |
| e30313a | Ankur Dave | 28 April 2011, 05:32:35 UTC | Added DiskSpillingCache DiskSpillingCache is a BoundedMemoryCache that spills entries to disk when it runs out of space. Currently the implementation is very simple. In particular, it's missing the following features: - Error handling for disk I/O, including checking of disk space levels - Bringing an entry back into memory after fetching it from disk In addition, here are some features that aren't critical but should be implemented soon: - Spilling based on a user-set priority in addition to LRU - Caching into a subdirectory of spark.DiskSpillingCache.cacheDir rather than the root directory | 28 April 2011, 05:32:35 UTC |
| 9d2d533 | Mosharaf Chowdhury | 22 April 2011, 00:40:26 UTC | Temporary fix for issue #42. | 22 April 2011, 00:40:26 UTC |
| 5c95352 | Timothy Hunter | 15 April 2011, 06:28:37 UTC | fixed small bug when classpath has some strange formatting | 19 April 2011, 00:12:29 UTC |
| 94ba95b | Matei Zaharia | 13 April 2011, 02:51:58 UTC | Added flatMapValues | 13 April 2011, 02:51:58 UTC |
| d840fa8 | Matei Zaharia | 09 March 2011, 08:40:07 UTC | Merge remote branch 'origin/custom-serialization' into new-rdds | 09 March 2011, 08:40:07 UTC |
| ff5b137 | root | 09 March 2011, 08:31:50 UTC | Some tweaks to make Kryo cache work better | 09 March 2011, 08:31:50 UTC |
| 7febdfb | Matei Zaharia | 08 March 2011, 20:36:36 UTC | Better reuse of buffers in Kryo serialization | 08 March 2011, 20:36:36 UTC |
| 8ee3ec2 | Matei Zaharia | 08 March 2011, 19:58:19 UTC | Merge remote branch 'origin/custom-serialization' into new-rdds | 08 March 2011, 19:58:19 UTC |
| 7408230 | Matei Zaharia | 08 March 2011, 19:58:08 UTC | Updated modified Kryo to use objenesis | 08 March 2011, 19:58:08 UTC |
| ab1216c | Matei Zaharia | 08 March 2011, 19:52:58 UTC | Register None and Nil properly | 08 March 2011, 19:52:58 UTC |
| d39f5dd | Matei Zaharia | 08 March 2011, 18:28:50 UTC | Merge remote branch 'origin/custom-serialization' into new-rdds | 08 March 2011, 18:28:50 UTC |
| 4f0d0a7 | Matei Zaharia | 08 March 2011, 18:28:26 UTC | stuff | 08 March 2011, 18:28:26 UTC |
| 8b6f3db | Matei Zaharia | 08 March 2011, 03:20:28 UTC | Merge remote branch 'origin/custom-serialization' into new-rdds | 08 March 2011, 03:20:28 UTC |
| 38f6bce | Matei Zaharia | 08 March 2011, 03:16:24 UTC | Added SerializingCache | 08 March 2011, 03:16:24 UTC |
| 6316c79 | Matei Zaharia | 08 March 2011, 02:56:36 UTC | Remove some logging | 08 March 2011, 02:56:36 UTC |
| e7b4b04 | Matei Zaharia | 08 March 2011, 02:41:53 UTC | Added pluggable serializers and Kryo serialization | 08 March 2011, 02:41:53 UTC |
| 467f056 | Matei Zaharia | 07 March 2011, 07:38:41 UTC | Remove commented code | 07 March 2011, 07:38:41 UTC |
| bce95b8 | Matei Zaharia | 07 March 2011, 07:38:16 UTC | Finished cogroup stuff | 07 March 2011, 07:38:16 UTC |
| 04c2d6a | Matei Zaharia | 07 March 2011, 03:27:03 UTC | stuff | 07 March 2011, 03:27:03 UTC |
| 0fb691d | Matei Zaharia | 07 March 2011, 00:16:38 UTC | Various fixes to get MesosScheduler working with new RDDs | 07 March 2011, 00:16:38 UTC |
| 1df5a65 | Matei Zaharia | 06 March 2011, 20:16:38 UTC | Pass cache locations correctly to DAGScheduler. | 06 March 2011, 20:16:38 UTC |
| e1436f1 | Matei Zaharia | 06 March 2011, 19:11:47 UTC | Merge remote branch 'origin/master' into new-rdds | 06 March 2011, 19:11:47 UTC |
| 370b958 | Matei Zaharia | 06 March 2011, 19:11:20 UTC | Added sampling for large arrays in SizeEstimator | 06 March 2011, 19:11:20 UTC |
| a789e9a | Matei Zaharia | 01 March 2011, 18:33:37 UTC | Merge remote branch 'origin/master' into new-rdds | 01 March 2011, 18:33:37 UTC |
| 021c50a | Matei Zaharia | 11 February 2011, 02:13:36 UTC | Remove unnecessary lock which was there to work around a bug in Configuration in Hadoop 0.20.0 | 01 March 2011, 18:28:38 UTC |
| adaba4d | Matei Zaharia | 11 February 2011, 02:07:03 UTC | Removed old slf4j jars that came with Hadoop | 01 March 2011, 18:28:21 UTC |
| 447debb | Matei Zaharia | 11 February 2011, 00:17:44 UTC | Updated Hadoop to 0.20.2 to include some bug fixes | 01 March 2011, 18:27:48 UTC |
| 9e59afd | Matei Zaharia | 28 February 2011, 03:15:52 UTC | More work on new RDD design | 28 February 2011, 03:15:52 UTC |
| f38f86d | Matei Zaharia | 27 February 2011, 22:27:12 UTC | More stuff | 27 February 2011, 22:27:12 UTC |
| 2e6023f | Matei Zaharia | 27 February 2011, 07:41:44 UTC | stuff | 27 February 2011, 07:41:44 UTC |
| 309367c | Matei Zaharia | 27 February 2011, 07:15:33 UTC | Initial work towards new RDD design | 27 February 2011, 07:15:33 UTC |
| dc24aec | Matei Zaharia | 10 February 2011, 19:20:51 UTC | Close record readers in HadoopFile after finishing a split | 10 February 2011, 20:07:48 UTC |
| 62f1c6f | Matei Zaharia | 09 February 2011, 19:52:56 UTC | Remove build.properties from version control | 09 February 2011, 19:52:56 UTC |
| d3df963 | Matei Zaharia | 09 February 2011, 05:27:36 UTC | Brought in some reorganization of build file from Hive branch | 09 February 2011, 05:27:36 UTC |
| e8df4bb | Matei Zaharia | 09 February 2011, 01:06:07 UTC | Added more SBT stuff to gitignore | 09 February 2011, 01:06:07 UTC |
| 26b77ae | Matei Zaharia | 09 February 2011, 01:03:28 UTC | Increased SBT mem to 700 MB so that unit tests run more nicely | 09 February 2011, 01:03:28 UTC |
| 99f3f23 | Matei Zaharia | 09 February 2011, 01:03:03 UTC | Changed default shuffle to LocalFileShuffle because it's way faster for small files | 09 February 2011, 01:03:03 UTC |
| f4f7aa2 | Matei Zaharia | 09 February 2011, 00:39:17 UTC | formatting | 09 February 2011, 00:39:17 UTC |
| ee60aaa | Matei Zaharia | 09 February 2011, 00:38:10 UTC | Added a pointer to wiki in readme | 09 February 2011, 00:38:10 UTC |
| c1c766a | Matei Zaharia | 03 February 2011, 03:21:49 UTC | Updated readme | 03 February 2011, 03:21:49 UTC |
| 50df43b | Matei Zaharia | 03 February 2011, 03:08:14 UTC | Added SBT target for building a single JAR with Spark Core and its dependencies | 03 February 2011, 03:08:14 UTC |
| a11fe23 | Matei Zaharia | 03 February 2011, 00:28:57 UTC | Moved examples to spark.examples package | 03 February 2011, 00:30:27 UTC |
| 8217060 | Matei Zaharia | 02 February 2011, 08:30:29 UTC | Added IntelliJ's build directory to gitignore | 02 February 2011, 08:30:29 UTC |
| ec28b60 | Matei Zaharia | 02 February 2011, 08:25:54 UTC | Merge branch 'master' into sbt Conflicts: Makefile core/src/main/java/spark/compress/lzf/LZF.java core/src/main/java/spark/compress/lzf/LZFInputStream.java core/src/main/java/spark/compress/lzf/LZFOutputStream.java core/src/main/native/spark_compress_lzf_LZF.c run | 02 February 2011, 08:25:54 UTC |
| 7f74ee9 | Matei Zaharia | 02 February 2011, 08:08:13 UTC | Added support for IntelliJ IDEA | 02 February 2011, 08:08:13 UTC |
| e5c4cd8 | Matei Zaharia | 01 February 2011, 23:11:08 UTC | Made examples and core subprojects | 01 February 2011, 23:11:08 UTC |
| 817e722 | Matei Zaharia | 16 December 2010, 03:40:35 UTC | Merge branch 'master' of github.com:mesos/spark | 16 December 2010, 03:40:35 UTC |
| 14c29c1 | Matei Zaharia | 16 December 2010, 03:40:27 UTC | Fixed import | 16 December 2010, 03:40:27 UTC |
| 5cafdd7 | Mosharaf Chowdhury | 16 December 2010, 03:11:23 UTC | Removed some unused imports from Broadcast.scala | 16 December 2010, 03:11:23 UTC |
| be0ce57 | Mosharaf Chowdhury | 16 December 2010, 02:34:27 UTC | - Fixed an compilation error due to wrong 'import' of legacy lzf libraries in DfsBroadcast.scala - Updated to use ning libraries. - Passes all unit tests | 16 December 2010, 02:34:27 UTC |
| 5c222db | Matei Zaharia | 15 December 2010, 18:57:39 UTC | Merge branch 'master' into mos-bt Conflicts: src/scala/spark/Broadcast.scala | 15 December 2010, 18:57:39 UTC |
| 0a5c24a | Mosharaf Chowdhury | 13 December 2010, 22:36:39 UTC | - Default broadcast mechanism is set to DfsBroadcast - Configuration parameters are renamed to follow our convention - Master now automatically supplies its hostAddress instead of reading from config file - sendBroadcast has been removed from the Broadcast trait | 13 December 2010, 22:36:39 UTC |
| 3439573 | Timothy Hunter | 12 December 2010, 07:42:45 UTC | Someone forgot to pass the parameters: fixes SPARK_MEM set from main script but not passed to executor. | 12 December 2010, 21:30:49 UTC |
| 0d895ba | Matei Zaharia | 07 December 2010, 18:32:17 UTC | Added BSD license | 07 December 2010, 18:32:17 UTC |
| 06dc4a5 | Mosharaf Chowdhury | 07 December 2010, 18:17:47 UTC | - Removed config files from git's control. - Changed DfsShuffle to default in RDD.scala. | 07 December 2010, 18:17:47 UTC |
| 799c1b1 | Joshua Hartman | 07 December 2010, 16:30:29 UTC | Adding license file for compress-lzf | 07 December 2010, 16:30:29 UTC |
| 2fb8495 | Joshua Hartman | 06 December 2010, 05:20:15 UTC | Replacing the native lzf compression code with the ning open-source compress-lzf library. (Apache 2.0 liscense) | 06 December 2010, 05:20:15 UTC |
| 1013a01 | Mosharaf Chowdhury | 02 December 2010, 10:36:21 UTC | Made the root Broadcast object initilization method 'synchronized'. | 02 December 2010, 10:36:21 UTC |
| a69638d | Mosharaf Chowdhury | 01 December 2010, 02:45:04 UTC | Removed deprecated repl file ClassServer.scala | 01 December 2010, 02:45:04 UTC |
| b7dda4c | Mosharaf Chowdhury | 01 December 2010, 02:29:38 UTC | Merge branch 'multi-tracker' into mos-bt Conflicts: conf/java-opts src/scala/spark/Broadcast.scala src/scala/spark/DfsBroadcast.scala src/scala/spark/SparkContext.scala | 01 December 2010, 02:29:38 UTC |
| 815ecd3 | Mosharaf Chowdhury | 01 December 2010, 02:08:49 UTC | Made Broadcast Pluggable. Finally! | 01 December 2010, 02:08:49 UTC |
| ea28a30 | Mosharaf Chowdhury | 01 December 2010, 00:13:56 UTC | Makes transfer faster for small swarms (with size < MaxRxPeers) | 01 December 2010, 00:13:56 UTC |
| 191af65 | Mosharaf Chowdhury | 30 November 2010, 04:35:47 UTC | - Using the new Cache implementation. | 30 November 2010, 04:35:47 UTC |
| c9cad03 | Mosharaf Chowdhury | 30 November 2010, 04:30:38 UTC | - Using the new Cache implementation. - Removed unused code related to dualMode (deprecated). | 30 November 2010, 04:30:38 UTC |
| 73714da | Mosharaf Chowdhury | 30 November 2010, 03:19:21 UTC | Made LocalFileShuffle to be the default. | 30 November 2010, 03:19:21 UTC |
| c62f2fe | Mosharaf Chowdhury | 30 November 2010, 03:17:01 UTC | Made LocalFileShuffle to be the default. | 30 November 2010, 03:17:01 UTC |
| 15f1f68 | Mosharaf Chowdhury | 29 November 2010, 22:15:36 UTC | Variable renaming. | 29 November 2010, 22:15:36 UTC |
| bb3e7fb | Mosharaf Chowdhury | 29 November 2010, 22:13:35 UTC | Variable renaming. | 29 November 2010, 22:13:35 UTC |
| 1157380 | Mosharaf Chowdhury | 29 November 2010, 20:21:46 UTC | Removed unused third party libraries. | 29 November 2010, 20:21:46 UTC |
| 1f7fcb4 | Mosharaf Chowdhury | 29 November 2010, 20:10:06 UTC | Merge branch 'master' into mos-bt Conflicts: Makefile run src/scala/spark/Broadcast.scala src/scala/spark/Executor.scala src/scala/spark/HdfsFile.scala src/scala/spark/MesosScheduler.scala src/scala/spark/RDD.scala src/scala/spark/SparkContext.scala src/scala/spark/Split.scala src/scala/spark/Utils.scala src/scala/spark/repl/SparkInterpreter.scala third_party/mesos.jar | 29 November 2010, 20:10:06 UTC |
| 9e93f8d | Mosharaf Chowdhury | 29 November 2010, 19:43:20 UTC | Moved broadcast implementations to separate files. Ready to merge with 'master' | 29 November 2010, 19:43:20 UTC |
