https://github.com/apache/spark
Revision ded902c3a90a9340e551091d554245df5982590c authored by Ryan Blue on 24 January 2019, 22:17:38 UTC, committed by Marcelo Vanzin on 24 January 2019, 22:17:38 UTC
## What changes were proposed in this pull request? Updates the attempt ID used by FileFormatWriter. Tasks in stage attempts use the same task attempt number and could conflict. Using Spark's task attempt ID guarantees that Hadoop TaskAttemptID instances are unique. This is a backport of d5a97c1 to the 2.3 branch. ## How was this patch tested? Existing tests. Also validated that we no longer detect this failure case in our logs after deployment. Closes #23640 from rdblue/SPARK-26682-backport-to-2.3. Authored-by: Ryan Blue <blue@apache.org> Signed-off-by: Marcelo Vanzin <vanzin@cloudera.com>
1 parent 23e35d4
Tip revision: ded902c3a90a9340e551091d554245df5982590c authored by Ryan Blue on 24 January 2019, 22:17:38 UTC
[SPARK-26682][SQL] Use taskAttemptID instead of attemptNumber for Had…
[SPARK-26682][SQL] Use taskAttemptID instead of attemptNumber for Had…
Tip revision: ded902c
| File | Mode | Size |
|---|---|---|
| .github | ||
| R | ||
| assembly | ||
| bin | ||
| build | ||
| common | ||
| conf | ||
| core | ||
| data | ||
| dev | ||
| docs | ||
| examples | ||
| external | ||
| graphx | ||
| hadoop-cloud | ||
| launcher | ||
| licenses | ||
| mllib | ||
| mllib-local | ||
| project | ||
| python | ||
| repl | ||
| resource-managers | ||
| sbin | ||
| sql | ||
| streaming | ||
| tools | ||
| .gitattributes | -rw-r--r-- | 40 bytes |
| .gitignore | -rw-r--r-- | 1.3 KB |
| .travis.yml | -rw-r--r-- | 1.7 KB |
| CONTRIBUTING.md | -rw-r--r-- | 995 bytes |
| LICENSE | -rw-r--r-- | 17.6 KB |
| NOTICE | -rw-r--r-- | 25.7 KB |
| README.md | -rw-r--r-- | 3.7 KB |
| appveyor.yml | -rw-r--r-- | 2.3 KB |
| pom.xml | -rw-r--r-- | 99.3 KB |
| scalastyle-config.xml | -rw-r--r-- | 17.2 KB |
Computing file changes ...
