Revision c6ba647935e9108d139cc8914091790917567ad7 authored by IngoSchuster on 30 June 2017, 03:16:09 UTC, committed by Wenchen Fan on 30 June 2017, 03:16:19 UTC
## What changes were proposed in this pull request? Please see also https://issues.apache.org/jira/browse/SPARK-21176 This change limits the number of selector threads that jetty creates to maximum 8 per proxy servlet (Jetty default is number of processors / 2). The newHttpClient for Jettys ProxyServlet class is overwritten to avoid the Jetty defaults (which are designed for high-performance http servers). Once https://github.com/eclipse/jetty.project/issues/1643 is available, the code could be cleaned up to avoid the method override. I really need this on v2.1.1 - what is the best way for a backport automatic merge works fine)? Shall I create another PR? ## How was this patch tested? (Please explain how this patch was tested. E.g. unit tests, integration tests, manual tests) The patch was tested manually on a Spark cluster with a head node that has 88 processors using JMX to verify that the number of selector threads is now limited to 8 per proxy. gurvindersingh zsxwing can you please review the change? Author: IngoSchuster <ingo.schuster@de.ibm.com> Author: Ingo Schuster <ingo.schuster@de.ibm.com> Closes #18437 from IngoSchuster/master. (cherry picked from commit 88a536babf119b7e331d02aac5d52b57658803bf) Signed-off-by: Wenchen Fan <wenchen@databricks.com>
1 parent 8de67e3
| File | Mode | Size |
|---|---|---|
| beeline | -rwxr-xr-x | 1.1 KB |
| beeline.cmd | -rw-r--r-- | 899 bytes |
| find-spark-home | -rwxr-xr-x | 1.9 KB |
| load-spark-env.cmd | -rw-r--r-- | 1.9 KB |
| load-spark-env.sh | -rw-r--r-- | 2.1 KB |
| pyspark | -rwxr-xr-x | 2.9 KB |
| pyspark.cmd | -rw-r--r-- | 1002 bytes |
| pyspark2.cmd | -rw-r--r-- | 1.5 KB |
| run-example | -rwxr-xr-x | 1.0 KB |
| run-example.cmd | -rw-r--r-- | 988 bytes |
| spark-class | -rwxr-xr-x | 3.1 KB |
| spark-class.cmd | -rw-r--r-- | 1012 bytes |
| spark-class2.cmd | -rw-r--r-- | 2.4 KB |
| spark-shell | -rwxr-xr-x | 2.9 KB |
| spark-shell.cmd | -rw-r--r-- | 1010 bytes |
| spark-shell2.cmd | -rw-r--r-- | 1.5 KB |
| spark-sql | -rwxr-xr-x | 1.0 KB |
| spark-submit | -rwxr-xr-x | 1.0 KB |
| spark-submit.cmd | -rw-r--r-- | 1012 bytes |
| spark-submit2.cmd | -rw-r--r-- | 1.1 KB |
| sparkR | -rwxr-xr-x | 1.0 KB |
| sparkR.cmd | -rw-r--r-- | 1000 bytes |
| sparkR2.cmd | -rw-r--r-- | 1014 bytes |
Computing file changes ...
