https://github.com/apache/spark
Revision 083adb07fc78b702bb84fa43373969ca7c5eb29a authored by IngoSchuster on 30 June 2017, 03:16:09 UTC, committed by Wenchen Fan on 30 June 2017, 03:16:38 UTC
## What changes were proposed in this pull request? Please see also https://issues.apache.org/jira/browse/SPARK-21176 This change limits the number of selector threads that jetty creates to maximum 8 per proxy servlet (Jetty default is number of processors / 2). The newHttpClient for Jettys ProxyServlet class is overwritten to avoid the Jetty defaults (which are designed for high-performance http servers). Once https://github.com/eclipse/jetty.project/issues/1643 is available, the code could be cleaned up to avoid the method override. I really need this on v2.1.1 - what is the best way for a backport automatic merge works fine)? Shall I create another PR? ## How was this patch tested? (Please explain how this patch was tested. E.g. unit tests, integration tests, manual tests) The patch was tested manually on a Spark cluster with a head node that has 88 processors using JMX to verify that the number of selector threads is now limited to 8 per proxy. gurvindersingh zsxwing can you please review the change? Author: IngoSchuster <ingo.schuster@de.ibm.com> Author: Ingo Schuster <ingo.schuster@de.ibm.com> Closes #18437 from IngoSchuster/master. (cherry picked from commit 88a536babf119b7e331d02aac5d52b57658803bf) Signed-off-by: Wenchen Fan <wenchen@databricks.com>
1 parent 26f4f34
Tip revision: 083adb07fc78b702bb84fa43373969ca7c5eb29a authored by IngoSchuster on 30 June 2017, 03:16:09 UTC
[SPARK-21176][WEB UI] Limit number of selector threads for admin ui proxy servlets to 8
[SPARK-21176][WEB UI] Limit number of selector threads for admin ui proxy servlets to 8
Tip revision: 083adb0
File | Mode | Size |
---|---|---|
.github | ||
R | ||
assembly | ||
bin | ||
build | ||
common | ||
conf | ||
core | ||
data | ||
dev | ||
docs | ||
examples | ||
external | ||
graphx | ||
launcher | ||
licenses | ||
mesos | ||
mllib | ||
mllib-local | ||
project | ||
python | ||
repl | ||
sbin | ||
sql | ||
streaming | ||
tools | ||
yarn | ||
.gitattributes | -rw-r--r-- | 40 bytes |
.gitignore | -rw-r--r-- | 1.2 KB |
.travis.yml | -rw-r--r-- | 1.7 KB |
CONTRIBUTING.md | -rw-r--r-- | 995 bytes |
LICENSE | -rw-r--r-- | 17.4 KB |
NOTICE | -rw-r--r-- | 24.1 KB |
README.md | -rw-r--r-- | 3.7 KB |
appveyor.yml | -rw-r--r-- | 1.8 KB |
pom.xml | -rw-r--r-- | 98.5 KB |
scalastyle-config.xml | -rw-r--r-- | 16.7 KB |
![swh spinner](/static/img/swh-spinner.gif)
Computing file changes ...