Revision be891ad99083564a7bf7f421e00b2cc4759a679f authored by Maryann Xue on 07 July 2022, 04:35:04 UTC, committed by Dongjoon Hyun on 07 July 2022, 04:35:04 UTC
### What changes were proposed in this pull request? This is a backport of https://github.com/apache/spark/pull/36953 This PR adds a check for invalid plans in AQE replanning process. The check will throw exceptions when it detects an invalid plan, causing AQE to void the current replanning result and keep using the latest valid plan. ### Why are the changes needed? AQE logical optimization rules can lead to invalid physical plans and cause runtime exceptions as certain physical plan nodes are not compatible with others. E.g., `BroadcastExchangeExec` can only work as a direct child of broadcast join nodes, but it could appear under other incompatible physical plan nodes because of empty relation propagation. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Added UT. Closes #37108 from dongjoon-hyun/SPARK-39551. Authored-by: Maryann Xue <maryann.xue@gmail.com> Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
1 parent 1c0bd4c
| File | Mode | Size |
|---|---|---|
| .github | ||
| .idea | ||
| R | ||
| assembly | ||
| bin | ||
| binder | ||
| build | ||
| common | ||
| conf | ||
| core | ||
| data | ||
| dev | ||
| docs | ||
| examples | ||
| external | ||
| graphx | ||
| hadoop-cloud | ||
| launcher | ||
| licenses | ||
| licenses-binary | ||
| mllib | ||
| mllib-local | ||
| project | ||
| python | ||
| repl | ||
| resource-managers | ||
| sbin | ||
| sql | ||
| streaming | ||
| tools | ||
| .asf.yaml | -rw-r--r-- | 1.1 KB |
| .gitattributes | -rw-r--r-- | 130 bytes |
| .gitignore | -rw-r--r-- | 1.8 KB |
| CONTRIBUTING.md | -rw-r--r-- | 997 bytes |
| LICENSE | -rw-r--r-- | 13.1 KB |
| LICENSE-binary | -rw-r--r-- | 22.3 KB |
| NOTICE | -rw-r--r-- | 2.0 KB |
| NOTICE-binary | -rw-r--r-- | 56.3 KB |
| README.md | -rw-r--r-- | 4.4 KB |
| appveyor.yml | -rw-r--r-- | 2.6 KB |
| pom.xml | -rw-r--r-- | 128.0 KB |
| scalastyle-config.xml | -rw-r--r-- | 21.0 KB |
Computing file changes ...
