Skip to main content

Browse the archive

https://github.com/apache/spark
06 August 2024, 02:29:59 UTC
sort by:
Revision Author Date Message Commit Date
e9eba8c Patrick Wendell 13 September 2013, 21:06:45 UTC Use different Hadoop version for YARN artifacts. This uses a seperate Hadoop version for YARN artifact. This means when people link against spark-yarn, things will resolve correctly. 13 September 2013, 22:34:57 UTC
45ec1cc Patrick Wendell 12 September 2013, 20:47:09 UTC Add git scm url for publishing 12 September 2013, 20:47:31 UTC
a310de6 Patrick Wendell 12 September 2013, 02:36:11 UTC Merge pull request #926 from kayousterhout/dynamic Changed localProperties to use ThreadLocal (not DynamicVariable). 12 September 2013, 02:36:11 UTC
58c7d8b Matei Zaharia 12 September 2013, 00:33:42 UTC Merge pull request #927 from benh/mesos-docs Updated Spark on Mesos documentation. 12 September 2013, 00:33:42 UTC
8e2602d Benjamin Hindman 11 September 2013, 23:08:54 UTC More updates to Spark on Mesos documentation. 11 September 2013, 23:08:54 UTC
a0f0c1b Benjamin Hindman 11 September 2013, 23:05:25 UTC Updated Spark on Mesos documentation. 11 September 2013, 23:05:25 UTC
93c4253 Kay Ousterhout 11 September 2013, 19:12:20 UTC Changed localProperties to use ThreadLocal (not DynamicVariable). The fact that DynamicVariable uses an InheritableThreadLocal can cause problems where the properties end up being shared across threads in certain circumstances. 11 September 2013, 20:01:39 UTC
91a59e6 Patrick Wendell 11 September 2013, 17:21:48 UTC Merge pull request #919 from mateiz/jets3t Add explicit jets3t dependency, which is excluded in hadoop-client 11 September 2013, 17:21:48 UTC
b9128d3 Patrick Wendell 11 September 2013, 17:03:06 UTC Merge pull request #922 from pwendell/port-change Change default port number from 3030 to 4030. 11 September 2013, 17:03:06 UTC
bddf135 Patrick Wendell 11 September 2013, 06:12:27 UTC Change port from 3030 to 4040 11 September 2013, 17:01:38 UTC
e07eef8 Reynold Xin 11 September 2013, 14:35:39 UTC Merge pull request #925 from davidmccauley/master SPARK-894 - Not all WebUI fields delivered VIA JSON 11 September 2013, 14:35:39 UTC
5dd875c David McCauley 11 September 2013, 09:46:37 UTC SPARK-894 - Not all WebUI fields delivered VIA JSON 11 September 2013, 09:46:37 UTC
8432f27 Matei Zaharia 11 September 2013, 06:19:53 UTC Merge pull request #923 from haoyuan/master fix run-example script 11 September 2013, 06:19:53 UTC
45d964b Haoyuan Li 11 September 2013, 06:18:22 UTC better expression 11 September 2013, 06:18:22 UTC
d40f140 Patrick Wendell 11 September 2013, 06:05:29 UTC Merge pull request #921 from pwendell/master Fix HDFS access bug with assembly build. 11 September 2013, 06:05:29 UTC
56b9407 Haoyuan Li 11 September 2013, 06:03:09 UTC fix run-example script 11 September 2013, 06:03:09 UTC
0c1985b Patrick Wendell 11 September 2013, 04:38:22 UTC Fix HDFS access bug with assembly build. Due to this change in HDFS: https://issues.apache.org/jira/browse/HADOOP-7549 there is a bug when using the new assembly builds. The symptom is that any HDFS access results in an exception saying "No filesystem for scheme 'hdfs'". This adds a merge strategy in the assembly build which fixes the problem. 11 September 2013, 05:05:13 UTC
2425eb8 Matei Zaharia 10 September 2013, 18:12:59 UTC Update Python API features 10 September 2013, 18:12:59 UTC
f117dc6 Matei Zaharia 10 September 2013, 06:39:25 UTC Add explicit jets3t dependency, which is excluded in hadoop-client 10 September 2013, 06:39:25 UTC
8c14f4b shivaram 10 September 2013, 05:07:58 UTC Merge pull request #917 from pwendell/master Document libgfortran dependency for MLBase 10 September 2013, 05:07:58 UTC
cefee1e Patrick Wendell 10 September 2013, 04:45:04 UTC Document fortran dependency for MLBase 10 September 2013, 04:45:04 UTC
c81377b Matei Zaharia 10 September 2013, 03:16:19 UTC Merge pull request #915 from ooyala/master Get rid of / improve ugly NPE when Utils.deleteRecursively() fails 10 September 2013, 03:16:19 UTC
61d2a01 Matei Zaharia 10 September 2013, 01:21:01 UTC Merge pull request #916 from mateiz/mkdist-fix Fix copy issue in https://github.com/mesos/spark/pull/899 10 September 2013, 01:21:01 UTC
fdb8b0e Evan Chan 09 September 2013, 21:29:32 UTC Style fix: put body of if within curly braces 09 September 2013, 21:29:32 UTC
f5a8afa Matei Zaharia 09 September 2013, 20:47:56 UTC Fix copy issue in https://github.com/mesos/spark/pull/899 09 September 2013, 20:47:56 UTC
a85758c Matei Zaharia 09 September 2013, 20:45:40 UTC Merge pull request #907 from stephenh/document_coalesce_shuffle Add better docs for coalesce. 09 September 2013, 20:45:40 UTC
2772607 Evan Chan 09 September 2013, 19:58:12 UTC Print out more friendly error if listFiles() fails listFiles() could return null if the I/O fails, and this currently results in an ugly NPE which is hard to diagnose. 09 September 2013, 19:58:12 UTC
084fc36 Matei Zaharia 09 September 2013, 19:01:35 UTC Merge pull request #912 from tgravescs/ganglia-pom Add metrics-ganglia to core pom file 09 September 2013, 19:01:35 UTC
2186d93 Y.CORP.YAHOO.COM\tgraves 09 September 2013, 17:37:33 UTC Add metrics-ganglia to core pom file 09 September 2013, 17:37:33 UTC
0456384 Matei Zaharia 09 September 2013, 16:57:54 UTC Merge pull request #911 from pwendell/ganglia-sink Adding Manen dependency for Ganglia 09 September 2013, 16:57:54 UTC
59003d3 Stephen Haberman 09 September 2013, 16:45:03 UTC Use a set since shuffle could change order. 09 September 2013, 16:45:03 UTC
6471bfe Stephen Haberman 09 September 2013, 16:44:15 UTC Reword 'evenly distributed' to 'distributed with a hash partitioner. 09 September 2013, 16:44:15 UTC
528fdba Patrick Wendell 09 September 2013, 16:32:18 UTC Adding Manen dependency 09 September 2013, 16:32:18 UTC
bf984e2 Matei Zaharia 09 September 2013, 06:50:24 UTC Merge pull request #890 from mridulm/master Fix hash bug 09 September 2013, 06:50:24 UTC
e9d4f44 Reynold Xin 09 September 2013, 06:36:48 UTC Merge pull request #909 from mateiz/exec-id-fix Fix an instance where full standalone mode executor IDs were passed to 09 September 2013, 06:36:48 UTC
2447b1c Matei Zaharia 09 September 2013, 05:27:49 UTC Merge pull request #910 from mateiz/ml-doc-tweaks Small tweaks to MLlib docs 09 September 2013, 05:27:49 UTC
7a5c4b6 Matei Zaharia 09 September 2013, 04:47:24 UTC Small tweaks to MLlib docs 09 September 2013, 04:47:24 UTC
7d3204b Matei Zaharia 09 September 2013, 04:39:12 UTC Merge pull request #905 from mateiz/docs2 Job scheduling and cluster mode docs 09 September 2013, 04:39:12 UTC
f1f8371 Matei Zaharia 09 September 2013, 04:26:11 UTC Merge pull request #896 from atalwalkar/master updated content 09 September 2013, 04:26:11 UTC
b458854 Matei Zaharia 09 September 2013, 04:25:49 UTC Fix some review comments 09 September 2013, 04:25:49 UTC
81a8bd4 Ameet Talwalkar 09 September 2013, 02:21:30 UTC respose to PR comments 09 September 2013, 02:21:30 UTC
bf280c8 Ameet Talwalkar 09 September 2013, 01:41:38 UTC Merge remote-tracking branch 'upstream/master' 09 September 2013, 01:41:38 UTC
f68848d Patrick Wendell 09 September 2013, 01:32:16 UTC Merge pull request #906 from pwendell/ganglia-sink Clean-up of Metrics Code/Docs and Add Ganglia Sink 09 September 2013, 01:32:16 UTC
f9b7f58 Matei Zaharia 09 September 2013, 01:27:50 UTC Fix an instance where full standalone mode executor IDs were passed to StandaloneSchedulerBackend instead of the smaller IDs used within Spark (that lack the application name). This was reported by ClearStory in https://github.com/clearstorydata/spark/pull/9. Also fixed some messages that said slave instead of executor. 09 September 2013, 01:27:50 UTC
170b386 Matei Zaharia 09 September 2013, 00:51:27 UTC Fix unit test failure due to changed default 09 September 2013, 00:51:27 UTC
5ac62db Ameet Talwalkar 09 September 2013, 00:39:08 UTC updates based on comments to PR 09 September 2013, 00:39:08 UTC
b4e382c Patrick Wendell 08 September 2013, 23:06:49 UTC Adding sc name in metrics source 08 September 2013, 23:06:49 UTC
8026537 Patrick Wendell 08 September 2013, 23:06:32 UTC Fixing package name in template conf 08 September 2013, 23:06:32 UTC
0b95799 Matei Zaharia 08 September 2013, 22:30:16 UTC Merge pull request #908 from pwendell/master Fix target JVM version in scala build 08 September 2013, 22:30:16 UTC
27bd74c Patrick Wendell 08 September 2013, 21:37:45 UTC Fix target JVM version in scala build 08 September 2013, 21:37:45 UTC
5a587fb Matei Zaharia 08 September 2013, 20:51:57 UTC Updated cluster diagram to show caches 08 September 2013, 20:51:57 UTC
c190b48 Patrick Wendell 08 September 2013, 17:47:45 UTC Adding more docs and some code cleanup 08 September 2013, 20:46:28 UTC
df5fd35 Stephen Haberman 08 September 2013, 20:39:03 UTC Add better docs for coalesce. Include the useful tip that if shuffle=true, coalesce can actually increase the number of partitions. This makes coalesce more like a generic `RDD.repartition` operation. (Ideally this `RDD.repartition` could automatically choose either a coalesce or a shuffle if numPartitions was either less than or greater than, respectively, the current number of partitions.) 08 September 2013, 20:39:04 UTC
af8ffdb Matei Zaharia 08 September 2013, 20:36:50 UTC Review comments 08 September 2013, 20:36:50 UTC
04cfb3a Matei Zaharia 08 September 2013, 17:33:20 UTC Merge pull request #898 from ilikerps/660 SPARK-660: Add StorageLevel support in Python 08 September 2013, 17:33:20 UTC
8de8ee5 Patrick Wendell 08 September 2013, 17:07:29 UTC Ganglia sink 08 September 2013, 17:08:18 UTC
c0d3751 Matei Zaharia 08 September 2013, 07:44:41 UTC Some tweaks to CDH/HDP doc 08 September 2013, 07:44:41 UTC
a386854 Aaron Davidson 08 September 2013, 07:30:47 UTC Whoopsy daisy 08 September 2013, 07:30:47 UTC
f261d2a Matei Zaharia 08 September 2013, 04:41:18 UTC Added cluster overview doc, made logo higher-resolution, and added more details on monitoring 08 September 2013, 07:29:11 UTC
651a96a Matei Zaharia 07 September 2013, 04:34:12 UTC More fair scheduler docs and property names. Also changed uses of "job" terminology to "application" when they referred to an entire Spark program, to avoid confusion. 08 September 2013, 07:29:11 UTC
98fb698 Matei Zaharia 06 September 2013, 04:29:37 UTC Work in progress: - Add job scheduling docs - Rename some fair scheduler properties - Organize intro page better - Link to Apache wiki for "contributing to Spark" 08 September 2013, 07:29:11 UTC
38488ac Matei Zaharia 08 September 2013, 07:28:53 UTC Merge pull request #900 from pwendell/cdh-docs Provide docs to describe running on CDH/HDP cluster. 08 September 2013, 07:28:53 UTC
a8e376e Patrick Wendell 08 September 2013, 04:16:01 UTC Merge pull request #904 from pwendell/master Adding Apache license to two files 08 September 2013, 04:16:01 UTC
6d21986 Patrick Wendell 08 September 2013, 03:46:58 UTC Adding Apache license to two files 08 September 2013, 03:46:58 UTC
c1cc8c4 Aaron Davidson 07 September 2013, 21:41:31 UTC Export StorageLevel and refactor 07 September 2013, 21:41:31 UTC
22b982d Patrick Wendell 07 September 2013, 21:37:54 UTC File rename 07 September 2013, 21:38:54 UTC
cfde85e Matei Zaharia 07 September 2013, 20:53:08 UTC Merge pull request #901 from ooyala/2013-09/0.8-doc-changes 0.8 Doc changes for make-distribution.sh 07 September 2013, 20:53:08 UTC
4a7813a Matei Zaharia 07 September 2013, 20:52:24 UTC Merge pull request #903 from rxin/resulttask Fixed the bug that ResultTask was not properly deserializing outputId. 07 September 2013, 20:52:24 UTC
61c4762 Patrick Wendell 07 September 2013, 18:51:40 UTC Changes based on feedback 07 September 2013, 18:55:10 UTC
8001687 Aaron Davidson 07 September 2013, 16:28:39 UTC Remove reflection, hard-code StorageLevels The sc.StorageLevel -> StorageLevel pathway is a bit janky, but otherwise the shell would have to call a private method of SparkContext. Having StorageLevel available in sc also doesn't seem like the end of the world. There may be a better solution, though. As for creating the StorageLevel object itself, this seems to be the best way in Python 2 for creating singleton, enum-like objects: http://stackoverflow.com/questions/36932/how-can-i-represent-an-enum-in-python 07 September 2013, 16:34:07 UTC
be1ee28 Evan Chan 07 September 2013, 15:56:24 UTC CR feedback from Matei 07 September 2013, 15:56:24 UTC
afe46ba Matei Zaharia 07 September 2013, 14:28:51 UTC Merge pull request #892 from jey/fix-yarn-assembly YARN build fixes 07 September 2013, 14:28:51 UTC
210eae2 Reynold Xin 07 September 2013, 13:59:47 UTC Fixed the bug that ResultTask was not properly deserializing outputId. 07 September 2013, 13:59:47 UTC
b8a0b6e Aaron Davidson 06 September 2013, 22:36:04 UTC Memoize StorageLevels read from JVM 06 September 2013, 22:36:04 UTC
2eebeff Patrick Wendell 06 September 2013, 22:25:22 UTC Merge pull request #897 from pwendell/master Docs describing Spark monitoring and instrumentation 06 September 2013, 22:25:22 UTC
ff1dbf2 Evan Chan 06 September 2013, 21:20:44 UTC Add references to make-distribution.sh 06 September 2013, 21:20:44 UTC
88d53f0 Evan Chan 06 September 2013, 21:03:44 UTC "launch" scripts is more accurate terminology 06 September 2013, 21:03:44 UTC
5a18b85 Evan Chan 06 September 2013, 20:59:43 UTC Easier way to start the master 06 September 2013, 20:59:43 UTC
76d5d2d Evan Chan 06 September 2013, 20:53:00 UTC Add notes about starting spark-shell 06 September 2013, 20:53:00 UTC
a2a0cf9 Patrick Wendell 06 September 2013, 06:33:44 UTC Docs describing Spark monitoring and instrumentation 06 September 2013, 20:52:57 UTC
e653a9d Patrick Wendell 06 September 2013, 19:15:49 UTC Provide docs to describe running on CDH/HDP cluster. This doc consolidates information relevant to CDH/HDP users in a single place. 06 September 2013, 20:49:57 UTC
30a32c8 Jey Kottalam 06 September 2013, 17:35:15 UTC Minor YARN build cleanups 06 September 2013, 18:31:16 UTC
7066124 Jey Kottalam 06 September 2013, 17:35:06 UTC Fix YARN assembly generation under Maven 06 September 2013, 18:31:16 UTC
35ed09f Jey Kottalam 04 September 2013, 18:52:16 UTC Clarify YARN example 06 September 2013, 18:31:16 UTC
1e15feb Reynold Xin 06 September 2013, 14:44:05 UTC Hot fix to resolve the compilation error caused by SPARK-821. 06 September 2013, 14:44:05 UTC
ddcb9d3 Patrick Wendell 06 September 2013, 06:54:09 UTC Merge pull request #895 from ilikerps/821 SPARK-821: Don't cache results when action run locally on driver 06 September 2013, 06:54:09 UTC
a63d4c7 Aaron Davidson 06 September 2013, 06:36:27 UTC SPARK-660: Add StorageLevel support in Python It uses reflection... I am not proud of that fact, but it at least ensures compatibility (sans refactoring of the StorageLevel stuff). 06 September 2013, 06:36:27 UTC
3a04e76 Aaron Davidson 06 September 2013, 04:43:26 UTC Reynold's second round of comments 06 September 2013, 04:43:26 UTC
d52edfa Ameet Talwalkar 06 September 2013, 04:06:50 UTC updated content 06 September 2013, 04:06:50 UTC
699c331 Matei Zaharia 06 September 2013, 03:21:53 UTC Merge pull request #891 from xiajunluan/SPARK-864 [SPARK-864]DAGScheduler Exception if we delete Worker and StandaloneExecutorBackend then add Worker 06 September 2013, 03:21:53 UTC
4f2236a Aaron Davidson 06 September 2013, 01:01:43 UTC Add unit test and address comments 06 September 2013, 01:06:30 UTC
1418d18 Aaron Davidson 05 September 2013, 22:28:14 UTC SPARK-821: Don't cache results when action run locally on driver Caching the results of local actions (e.g., rdd.first()) causes the driver to store entire partitions in its own memory, which may be highly constrained. This patch simply makes the CacheManager avoid caching the result of all locally-run computations. 05 September 2013, 22:34:42 UTC
7c15e3c Andrew xia 04 September 2013, 01:19:11 UTC Fix bug SPARK-864 05 September 2013, 07:56:11 UTC
5c7494d Patrick Wendell 05 September 2013, 05:47:03 UTC Merge pull request #893 from ilikerps/master SPARK-884: Add unit test to validate Spark JSON output 05 September 2013, 05:47:03 UTC
714e7f9 Aaron Davidson 05 September 2013, 05:40:08 UTC Fix line over 100 chars 05 September 2013, 05:40:08 UTC
37db141 Aaron Davidson 05 September 2013, 04:32:17 UTC Address Patrick's comments 05 September 2013, 04:34:20 UTC
a547866 Matei Zaharia 05 September 2013, 04:11:56 UTC Merge pull request #894 from c0s/master Updating assembly README to reflect recent changes in the build. 05 September 2013, 04:11:56 UTC
7c7c7e1 Konstantin Boudnik 05 September 2013, 03:47:45 UTC Updating assembly README to reflect recent changes in the build. 05 September 2013, 03:54:35 UTC
9e6f2b6 Aaron Davidson 04 September 2013, 22:06:52 UTC SPARK-884: Add unit test to validate Spark JSON output This unit test simply validates that the outputs of the JsonProtocol methods are syntactically valid JSON. 04 September 2013, 22:26:46 UTC
1e2474b Mridul Muralidharan 04 September 2013, 02:16:46 UTC Address review comments - rename toHash to nonNegativeHash 04 September 2013, 02:16:46 UTC
back to top