Skip to content

Conversation

Copy link

Copilot AI commented Nov 26, 2025

JDBC drivers for some databases (e.g., Apache Hive, Spark SQL) throw SQLFeatureNotSupportedException for standard ResultSetMetaData methods. This prevents reading data from those sources.

Changes

  • Fallback handling in getTableColumnsMetadata() (DbType.kt):

    • getColumnName()getColumnLabel()"column_N"
    • getTableName() → extract from qualified name → empty string
    • isNullable()DatabaseMetaData.getColumns()true (safe default)
    • getColumnTypeName()"OTHER"
    • getColumnType()Types.OTHER
    • getColumnClassName()"java.lang.Object"
  • BIGINT type consistency: Force Types.BIGINT to always map to Long::class regardless of javaClassName reported by driver (fixes MariaDB driver reporting Integer for small BIGINT values)

  • Extension point documentation: KDoc explains how to override getTableColumnsMetadata() in subclasses for databases where DatabaseMetaData.getColumns() is slow (Teradata, Oracle)

Usage for unsupported databases

// Create custom DbType for unsupported database
object HiveDb : DbType("hive") {
    override val driverClassName = "org.apache.hive.jdbc.HiveDriver"
    // ... implement required abstract methods
}

// Pass custom dbType to read operations
DataFrame.readSqlQuery(connection, "SELECT * FROM table", dbType = HiveDb)

Fixes #1220


💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

Copilot AI changed the title [WIP] Add fallback mechanism for reading metadata Added a fallback mechanism for the reading metadata Nov 26, 2025
Copilot AI requested a review from zaleslaw November 26, 2025 16:05
Copilot finished work on behalf of zaleslaw November 26, 2025 16:05
@zaleslaw zaleslaw closed this Nov 26, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants