Revision 6d32472b2a4225bd847953a679d3be2b077d212e authored by Ruifeng Zheng on 06 August 2024, 01:00:08 UTC, committed by Hyukjin Kwon on 06 August 2024, 01:00:08 UTC
### What changes were proposed in this pull request?
Make `createLocalRelationProto` support relation with `TimestampType`

### Why are the changes needed?
existing helper function `createLocalRelationProto` cannot create table with `TimestampType`:

```
  org.apache.spark.SparkException: [INTERNAL_ERROR] Missing timezoneId where it is mandatory. SQLSTATE: XX000
  at org.apache.spark.SparkException$.internalError(SparkException.scala:99)
  at org.apache.spark.SparkException$.internalError(SparkException.scala:103)
  at org.apache.spark.sql.util.ArrowUtils$.toArrowType(ArrowUtils.scala:57)
  at org.apache.spark.sql.util.ArrowUtils$.toArrowField(ArrowUtils.scala:139)
  at org.apache.spark.sql.util.ArrowUtils$.$anonfun$toArrowSchema$1(ArrowUtils.scala:181)
  at scala.collection.TraversableLike.$anonfun$map$1(TraversableLike.scala:286)
  at scala.collection.Iterator.foreach(Iterator.scala:943)
  at scala.collection.Iterator.foreach$(Iterator.scala:943)
  at scala.collection.AbstractIterator.foreach(Iterator.scala:1431)
  at scala.collection.IterableLike.foreach(IterableLike.scala:74)
```

### Does this PR introduce _any_ user-facing change?
No, test-only

### How was this patch tested?
added ut

### Was this patch authored or co-authored using generative AI tooling?
no

Closes #47608 from zhengruifeng/create_timestamp_localrel.

Authored-by: Ruifeng Zheng <ruifengz@apache.org>
Signed-off-by: Hyukjin Kwon <gurwls223@apache.org>
1 parent de8ee94
History

README.md

back to top