Revision 24a9bace131465bf6a177f304cf8f05b0e4fe6ed authored by Liang-Chi Hsieh on 29 July 2017, 17:02:56 UTC, committed by gatorsmile on 29 July 2017, 17:03:09 UTC
## What changes were proposed in this pull request?

When there are aliases (these aliases were added for nested fields) as parameters in `RuntimeReplaceable`, as they are not in the children expression, those aliases can't be cleaned up in analyzer rule `CleanupAliases`.

An expression `nvl(foo.foo1, "value")` can be resolved to two semantically different expressions in a group by query because they contain different aliases.

Because those aliases are not children of `RuntimeReplaceable` which is an `UnaryExpression`. So we can't trim the aliases out by simple transforming the expressions in `CleanupAliases`.

If we want to replace the non-children aliases in `RuntimeReplaceable`, we need to add more codes to `RuntimeReplaceable` and modify all expressions of `RuntimeReplaceable`. It makes the interface ugly IMO.

Consider those aliases will be replaced later at optimization and so they're no harm, this patch chooses to simply override `canonicalized` of `RuntimeReplaceable`.

One concern is about `CleanupAliases`. Because it actually cannot clean up ALL aliases inside a plan. To make caller of this rule notice that, this patch adds a comment to `CleanupAliases`.

## How was this patch tested?

Added test.

Author: Liang-Chi Hsieh <viirya@gmail.com>

Closes #18761 from viirya/SPARK-21555.

(cherry picked from commit 9c8109ef414c92553335bb1e90e9681e142128a4)
Signed-off-by: gatorsmile <gatorsmile@gmail.com>
1 parent df6cd35
History
File Mode Size
LICENSE-AnchorJS.txt -rw-r--r-- 1.1 KB
LICENSE-DPark.txt -rw-r--r-- 1.5 KB
LICENSE-Mockito.txt -rw-r--r-- 1.1 KB
LICENSE-SnapTree.txt -rw-r--r-- 1.7 KB
LICENSE-antlr.txt -rw-r--r-- 1.5 KB
LICENSE-boto.txt -rw-r--r-- 1.0 KB
LICENSE-cloudpickle.txt -rw-r--r-- 1.6 KB
LICENSE-d3.min.js.txt -rw-r--r-- 1.4 KB
LICENSE-dagre-d3.txt -rw-r--r-- 1.0 KB
LICENSE-f2j.txt -rw-r--r-- 1.5 KB
LICENSE-graphlib-dot.txt -rw-r--r-- 1.0 KB
LICENSE-heapq.txt -rw-r--r-- 14.2 KB
LICENSE-javolution.txt -rw-r--r-- 1.4 KB
LICENSE-jbcrypt.txt -rw-r--r-- 836 bytes
LICENSE-jline.txt -rw-r--r-- 1.5 KB
LICENSE-jpmml-model.txt -rw-r--r-- 1.5 KB
LICENSE-jquery.txt -rw-r--r-- 1.1 KB
LICENSE-junit-interface.txt -rw-r--r-- 1.3 KB
LICENSE-kryo.txt -rw-r--r-- 1.5 KB
LICENSE-minlog.txt -rw-r--r-- 1.5 KB
LICENSE-modernizr.txt -rw-r--r-- 1.1 KB
LICENSE-netlib.txt -rw-r--r-- 2.2 KB
LICENSE-paranamer.txt -rw-r--r-- 1.6 KB
LICENSE-postgresql.txt -rw-r--r-- 1.2 KB
LICENSE-protobuf.txt -rw-r--r-- 2.1 KB
LICENSE-py4j.txt -rw-r--r-- 1.4 KB
LICENSE-pyrolite.txt -rw-r--r-- 1.3 KB
LICENSE-reflectasm.txt -rw-r--r-- 1.5 KB
LICENSE-sbt-launch-lib.txt -rw-r--r-- 1.5 KB
LICENSE-scala.txt -rw-r--r-- 1.5 KB
LICENSE-scalacheck.txt -rw-r--r-- 1.5 KB
LICENSE-scopt.txt -rw-r--r-- 1.1 KB
LICENSE-slf4j.txt -rw-r--r-- 1.1 KB
LICENSE-sorttable.js.txt -rw-r--r-- 937 bytes
LICENSE-spire.txt -rw-r--r-- 1.0 KB
LICENSE-xmlenc.txt -rw-r--r-- 1.5 KB

back to top