Skip to content

Commit 722bcc0

Browse files
committed
[SPARK-54375][CONNECT][TESTS] Add assume to cases in PythonPipelineSuite to skip tests when PyConnect dependencies is not available
### What changes were proposed in this pull request? SPARK-54020 added some new test cases in `PythonPipelineSuite`. This pr incorporates `assume(PythonTestDepsChecker.isConnectDepsAvailable)` for these test cases to ensure that the tests are skipped rather than failing when PyConnect dependencies are missing. ### Why are the changes needed? Enhance the robustness of test cases. Prior to this, when executing `build/sbt "connect/testOnly org.apache.spark.sql.connect.pipelines.PythonPipelineSuite"`: ``` [info] - reading internal datasets outside query function that trigger eager analysis or execution will fail (spark.sql("SELECT * FROM src")) *** FAILED *** (4 milliseconds) [info] "org.apache.spark.sql.connect.PythonTestDepsChecker.isConnectDepsAvailable was false" did not contain "TABLE_OR_VIEW_NOT_FOUND" (PythonPipelineSuite.scala:546) [info] org.scalatest.exceptions.TestFailedException: [info] at org.scalatest.Assertions.newAssertionFailedException(Assertions.scala:472) [info] at org.scalatest.Assertions.newAssertionFailedException$(Assertions.scala:471) [info] at org.scalatest.Assertions$.newAssertionFailedException(Assertions.scala:1231) [info] at org.scalatest.Assertions$AssertionsHelper.macroAssert(Assertions.scala:1295) [info] at org.apache.spark.sql.connect.pipelines.PythonPipelineSuite.$anonfun$new$43(PythonPipelineSuite.scala:546) [info] at org.apache.spark.sql.connect.pipelines.PythonPipelineSuite.$anonfun$new$43$adapted(PythonPipelineSuite.scala:532) [info] at org.apache.spark.SparkFunSuite.$anonfun$gridTest$2(SparkFunSuite.scala:241) [info] at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18) [info] at org.scalatest.enablers.Timed$$anon$1.timeoutAfter(Timed.scala:127) [info] at org.scalatest.concurrent.TimeLimits$.failAfterImpl(TimeLimits.scala:282) [info] at org.scalatest.concurrent.TimeLimits.failAfter(TimeLimits.scala:231) [info] at org.scalatest.concurrent.TimeLimits.failAfter$(TimeLimits.scala:230) ... [info] *** 24 TESTS FAILED *** [error] Failed tests: [error] org.apache.spark.sql.connect.pipelines.PythonPipelineSuite [error] (connect / Test / testOnly) sbt.TestsFailedException: Tests unsuccessful ``` ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? - Pass Github Actions - Manually verify that the relevant tests will no longer fail when PyConnect dependencies are missing. ### Was this patch authored or co-authored using generative AI tooling? No Closes #53088 from LuciferYang/SPARK-54375. Authored-by: yangjie01 <yangjie01@baidu.com> Signed-off-by: yangjie01 <yangjie01@baidu.com>
1 parent 3757091 commit 722bcc0

File tree

1 file changed

+3
-0
lines changed

1 file changed

+3
-0
lines changed

sql/connect/server/src/test/scala/org/apache/spark/sql/connect/pipelines/PythonPipelineSuite.scala

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -530,6 +530,7 @@ class PythonPipelineSuite
530530
"eager analysis or execution will fail")(
531531
Seq("""spark.sql("SELECT * FROM src")""", """spark.read.table("src").collect()""")) {
532532
command =>
533+
assume(PythonTestDepsChecker.isConnectDepsAvailable)
533534
val ex = intercept[RuntimeException] {
534535
buildGraph(s"""
535536
|@dp.materialized_view
@@ -1047,6 +1048,7 @@ class PythonPipelineSuite
10471048

10481049
gridTest("Unsupported SQL command outside query function should result in a failure")(
10491050
unsupportedSqlCommandList) { unsupportedSqlCommand =>
1051+
assume(PythonTestDepsChecker.isConnectDepsAvailable)
10501052
val ex = intercept[RuntimeException] {
10511053
buildGraph(s"""
10521054
|spark.sql("$unsupportedSqlCommand")
@@ -1061,6 +1063,7 @@ class PythonPipelineSuite
10611063

10621064
gridTest("Unsupported SQL command inside query function should result in a failure")(
10631065
unsupportedSqlCommandList) { unsupportedSqlCommand =>
1066+
assume(PythonTestDepsChecker.isConnectDepsAvailable)
10641067
val ex = intercept[RuntimeException] {
10651068
buildGraph(s"""
10661069
|@dp.materialized_view()

0 commit comments

Comments
 (0)