Revision 76bbcc5d8c190aeb0af85e29ba981cfaa02155cf authored by Anish Shrigondekar on 30 July 2024, 05:13:25 UTC, committed by Jungtaek Lim on 30 July 2024, 05:13:25 UTC
### What changes were proposed in this pull request?
Add schema path in metadata table entry, verify expected version and add operator metadata related test for operator metadata format v2

### Why are the changes needed?
Changes needed for version verification and for subsequent integration with state data source reader

### Does this PR introduce _any_ user-facing change?
No

### How was this patch tested?
Added unit tests

```
===== POSSIBLE THREAD LEAK IN SUITE o.a.s.sql.execution.streaming.state.OperatorStateMetadataSuite, threads: Idle Worker Monitor for python3 (daemon=true), rpc-boss-3-1 (daemon=true), ForkJoinPool.commonPool-worker-3 (daemon=true), ForkJoinPool.commonPool-worker-2 (daemon=true), shuffle-boss-6-1 (daemon=true), ForkJoinPool.commonPool-worker-1 (daemon=true) =====
[info] Run completed in 26 seconds, 651 milliseconds.
[info] Total number of tests run: 11
[info] Suites: completed 1, aborted 0
[info] Tests: succeeded 11, failed 0, canceled 0, ignored 0, pending 0
[info] All tests passed.
```

### Was this patch authored or co-authored using generative AI tooling?
No

Closes #47510 from anishshri-db/task/SPARK-49032.

Authored-by: Anish Shrigondekar <anish.shrigondekar@databricks.com>
Signed-off-by: Jungtaek Lim <kabhwan.opensource@gmail.com>
1 parent f9d6315
History

README.md

back to top