Skip to content

[SPARK-49314][BUILD][TESTS] Upgrade h2 to 2.3.232, postgresql to 42.7.4 and mssql to 12.8.1.jre11 #47810

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 2 commits into from
Closed

[SPARK-49314][BUILD][TESTS] Upgrade h2 to 2.3.232, postgresql to 42.7.4 and mssql to 12.8.1.jre11 #47810

wants to merge 2 commits into from

Conversation

@wayneguow
Copy link
Contributor

@wayneguow wayneguow commented Aug 20, 2024 •
edited
Loading

What changes were proposed in this pull request?

This PR aims to upgrade h2 to 2.3.232, postgresql to 42.7.4 and mssql to 12.8.1.jre11.

Why are the changes needed?

  1. For h2, there are some issues fixed in version 2.3.232(full release notes: https://www.h2database.com/html/changelog.html):

    • Issue #3945: Column not found in correlated subquery, when referencing outer column from LEFT JOIN .. ON clause
    • Issue #4097: StackOverflowException when using multiple SELECT statements in one query (2.3.230)
    • Issue #3982: Potential issue when using ROUND
    • Issue #3894: Race condition causing stale data in query last result cache
    • Issue #4075: infinite loop in compact
    • Issue #4091: Wrong case with linked table to postgresql
    • Issue #4088: BadGrammarException when the same alias is used within two different CTEs

  2. For postgresql, there are some issues fixed and improvements in version 42.7.4(full release notes: https://jdbc.postgresql.org/changelogs/2024-08-22-42.7.4-release/):

    • fix: PgInterval ignores case for represented interval string PR #3344
    • perf: Avoid extra copies when receiving int4 and int2 in PGStream PR #3295
    • fix: Add support for Infinity::numeric values in ResultSet.getObject PR #3304
    • fix: Ensure order of results for getDouble PR #3301
    • perf: Replace BufferedOutputStream with unsynchronized PgBufferedOutputStream, allow configuring different Java and SO_SNDBUF buffer sizes PR #3248
    • fix: Fix SSL tests PR #3260
    • fix: Support bytea in preferQueryMode=simple PR #3243
    • fix: Fix Issue #3234 - Return -1 as update count for stored procedure calls PR #3235
    • fix: Fix Issue #3224 - conversion for TIME ‘24:00’ to LocalTime breaks in binary-mode PR #3225

  3. For mssql, there are some issues fixed in 12.8.1.jre11(full release notes: https://github.com/microsoft/mssql-jdbc/releases/tag/v12.8.1):

    • Adjusted DESTINATION_COL_METADATA_LOCK, in SQLServerBulkCopy, so that is properly released in all cases PR #2492
    • Reverted "Execute Stored Procedures Directly" feature, as well as subsequent changes related to the feature PR #2493
    • Changed driver behavior to allow prepared statement objects to be reused, preventing a "multiple queries are not allowed" error PR #2494

Does this PR introduce any user-facing change?

No.

How was this patch tested?

Pass GA.

Was this patch authored or co-authored using generative AI tooling?

No.

@wayneguow wayneguow marked this pull request as ready for review August 20, 2024 06:26
@wayneguow wayneguow changed the title [SPARK-49314][BUILD][TESTS] Upgrade h2 to 2.3.232 [SPARK-49314][BUILD][TESTS] Upgrade h2 to 2.3.232, postgresql to 42.7.4 and mssql to 12.8.1.jre11 Aug 23, 2024
@wayneguow
Copy link
Contributor Author

wayneguow commented Aug 24, 2024

cc @dongjoon-hyun @LuciferYang

@KamilKandzia KamilKandzia mentioned this pull request Aug 24, 2024
Closed
@yaooqinn
Copy link
Member

yaooqinn commented Aug 26, 2024

Could you resolve the markdown formatting issue in your PR desc?

@wayneguow
Copy link
Contributor Author

wayneguow commented Aug 26, 2024

Could you resolve the markdown formatting issue in your PR desc?

Thanks, I updated it.

@yaooqinn
Copy link
Member

yaooqinn commented Aug 26, 2024

Can you recheck it? Some links still point to wrong destinations.

@wayneguow
Copy link
Contributor Author

wayneguow commented Aug 26, 2024

Can you recheck it? Some links still point to wrong destinations.

Oh, I got it, and fixed some links about postgresql.

@yaooqinn
Copy link
Member

yaooqinn commented Aug 26, 2024

Merged to master, thank you @wayneguow

@yaooqinn yaooqinn closed this in d7f0c8f Aug 26, 2024
@vladanvasi-db vladanvasi-db mentioned this pull request Jan 20, 2025
Closed
cloud-fan pushed a commit that referenced this pull request Jan 21, 2025
@vladanvasi-db @cloud-fan
[SPARK-50894][SQL] Postgres driver version bump to 42.7.5
1cdb918 
### What changes were proposed in this pull request?

In this PR, I propose upgrading the version of `PostgreSQL` driver version to `42.7.5`. More specifically, in this [PR](#47810), the following regression was introduced:
`Postgres` driver was not compatible for connection to the `Redshift` Data Source. This happened because the `42.7.4` driver version unnecessarily sent `SET extra_float_digits = 3` queries to the destination data source when establishing the connection. However, the upper limit for `extra_float_digits` parameter in `Redshift` is 2.

Regarding the release notes of version [42.7.5](https://github.com/pgjdbc/pgjdbc/releases/tag/REL42.7.5), I propose to upgrade the version of the driver, because the behavior of sending queries which set `extra_flow_digits` when establishing connection was fixed in this version.

### Why are the changes needed?

This changes are essential when trying to properly connect to the Redshift instance from OSS.

### Does this PR introduce _any_ user-facing change?

No.

### How was this patch tested?

N/A.

### Was this patch authored or co-authored using generative AI tooling?

No.

Closes #49575 from vladanvasi-db/vladanvasi-db/postrgesql-driver-fix.

Authored-by: Vladan Vasić <vladan.vasic@databricks.com>
Signed-off-by: Wenchen Fan <wenchen@databricks.com>
cloud-fan pushed a commit that referenced this pull request Jan 21, 2025
@vladanvasi-db @cloud-fan
[SPARK-50894][SQL] Postgres driver version bump to 42.7.5
5397e12 
### What changes were proposed in this pull request?

In this PR, I propose upgrading the version of `PostgreSQL` driver version to `42.7.5`. More specifically, in this [PR](#47810), the following regression was introduced:
`Postgres` driver was not compatible for connection to the `Redshift` Data Source. This happened because the `42.7.4` driver version unnecessarily sent `SET extra_float_digits = 3` queries to the destination data source when establishing the connection. However, the upper limit for `extra_float_digits` parameter in `Redshift` is 2.

Regarding the release notes of version [42.7.5](https://github.com/pgjdbc/pgjdbc/releases/tag/REL42.7.5), I propose to upgrade the version of the driver, because the behavior of sending queries which set `extra_flow_digits` when establishing connection was fixed in this version.

### Why are the changes needed?

This changes are essential when trying to properly connect to the Redshift instance from OSS.

### Does this PR introduce _any_ user-facing change?

No.

### How was this patch tested?

N/A.

### Was this patch authored or co-authored using generative AI tooling?

No.

Closes #49575 from vladanvasi-db/vladanvasi-db/postrgesql-driver-fix.

Authored-by: Vladan Vasić <vladan.vasic@databricks.com>
Signed-off-by: Wenchen Fan <wenchen@databricks.com>
(cherry picked from commit 1cdb918)
Signed-off-by: Wenchen Fan <wenchen@databricks.com>
@wayneguow wayneguow deleted the ug_h2 branch February 11, 2025 04:27
zifeif2 pushed a commit to zifeif2/spark that referenced this pull request Nov 14, 2025
@vladanvasi-db @cloud-fan
[SPARK-50894][SQL] Postgres driver version bump to 42.7.5
79fb2c5 
### What changes were proposed in this pull request?

In this PR, I propose upgrading the version of `PostgreSQL` driver version to `42.7.5`. More specifically, in this [PR](apache#47810), the following regression was introduced:
`Postgres` driver was not compatible for connection to the `Redshift` Data Source. This happened because the `42.7.4` driver version unnecessarily sent `SET extra_float_digits = 3` queries to the destination data source when establishing the connection. However, the upper limit for `extra_float_digits` parameter in `Redshift` is 2.

Regarding the release notes of version [42.7.5](https://github.com/pgjdbc/pgjdbc/releases/tag/REL42.7.5), I propose to upgrade the version of the driver, because the behavior of sending queries which set `extra_flow_digits` when establishing connection was fixed in this version.

### Why are the changes needed?

This changes are essential when trying to properly connect to the Redshift instance from OSS.

### Does this PR introduce _any_ user-facing change?

No.

### How was this patch tested?

N/A.

### Was this patch authored or co-authored using generative AI tooling?

No.

Closes apache#49575 from vladanvasi-db/vladanvasi-db/postrgesql-driver-fix.

Authored-by: Vladan Vasić <vladan.vasic@databricks.com>
Signed-off-by: Wenchen Fan <wenchen@databricks.com>
(cherry picked from commit 81de2ab)
Signed-off-by: Wenchen Fan <wenchen@databricks.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@yaooqinn yaooqinn yaooqinn approved these changes

Assignees

No one assigned

Labels

BUILD CONNECT SQL

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

CallableStatement#getUpdateCount() seems wrong for stored procedures conversion for TIME '24:00' to LocalTime breaks in binary-mode

2 participants

@wayneguow @yaooqinn