Improve the performance of getDistinctPartitionReadFileFormats for HiveTableScanExecTransformer #11797
Description
Description
Currently, the original code has a lot of issues.
-
Perform expensive
HiveClientImpl.fromHivePartition()operation on each partition -
Perform two traversal using exists() and map()
-
No caching mechanism, repeated calculation of the same format conversion
Gluten version
main branch