https://github.com/apache/spark
Revision eb1e6ad13aab3960f1543b75bf3b75b3a7d62746 authored by Kent Yao on 13 December 2023, 10:04:38 UTC, committed by Kent Yao on 13 December 2023, 10:04:53 UTC
### What changes were proposed in this pull request?

This PR adds `query.resolved` as a pattern guard when HiveAnalysis converts InsertIntoStatement to InsertIntoHiveTable.

### Why are the changes needed?

Due to https://github.com/apache/spark/pull/41262/files#diff-ed19f376a63eba52eea59ca71f3355d4495fad4fad4db9a3324aade0d4986a47R1080, the `table` field is resolved regardless of the query field. Before, it never got a chance to be resolved as `HiveTableRelation` and then match any rule of HiveAnalysis. But now, it gets the chance always and results in a spark-kernel bug - `Invalid call to toAttribute on unresolved object.`

```
insert into t2 select cast(a as short) from t where b=1;
Invalid call to toAttribute on unresolved object
```

### Does this PR introduce _any_ user-facing change?

no, bugfix for 3.5 and later

### How was this patch tested?

added new test

### Was this patch authored or co-authored using generative AI tooling?

no

Closes #44326 from yaooqinn/SPARK-46388.

Authored-by: Kent Yao <yao@apache.org>
Signed-off-by: Kent Yao <yao@apache.org>
(cherry picked from commit ccc436d829cd0b07088e2864cb1ecc55ab97a491)
Signed-off-by: Kent Yao <yao@apache.org>
1 parent ac031d6
History
Tip revision: eb1e6ad13aab3960f1543b75bf3b75b3a7d62746 authored by Kent Yao on 13 December 2023, 10:04:38 UTC
[SPARK-46388][SQL] HiveAnalysis misses the pattern guard of `query.resolved`
Tip revision: eb1e6ad

README.md

back to top