Skip to content

Commit 4fe5548

Browse files
authored
Add (py)spark properties (#524)
1 parent c2de0fe commit 4fe5548

File tree

2 files changed

+2
-0
lines changed

2 files changed

+2
-0
lines changed

docs/user_guides/projects/jobs/pyspark_job.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -243,6 +243,7 @@ The following table describes the job configuration parameters for a PYSPARK job
243243
| <nobr>`conf['pyFiles']`</nobr> | string | Comma-separated string of HDFS path(s) to python modules to be made available to the application. Example: `hdfs:///Project/<project_name>/Resources/file1.py,...` | `null` |
244244
| <nobr>`conf['jars']`</nobr> | string | Comma-separated string of HDFS path(s) to jars to be included in CLASSPATH. Example: `hdfs:///Project/<project_name>/Resources/app.jar,...` | `null` |
245245
| <nobr>`conf['archives']`</nobr> | string | Comma-separated string of HDFS path(s) to archives to be made available to the application. Example: `hdfs:///Project/<project_name>/Resources/archive.zip,...` | `null` |
246+
| <nobr>`conf['properties']`</nobr> | string | A new line separated (`\n`) list of properties to pass to the Spark application. The properties should be in the format `name=value` | `null` |
246247

247248

248249
## Accessing project data

docs/user_guides/projects/jobs/spark_job.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -245,6 +245,7 @@ The following table describes the job configuration parameters for a SPARK job.
245245
| <nobr>`conf['pyFiles']`</nobr> | string | Comma-separated string of HDFS path(s) to Python modules to be made available to the application. Example: `hdfs:///Project/<project_name>/Resources/file1.py,...` | `null` |
246246
| <nobr>`conf['jars']`</nobr> | string | Comma-separated string of HDFS path(s) to jars to be included in CLASSPATH. Example: `hdfs:///Project/<project_name>/Resources/app.jar,...` | `null` |
247247
| <nobr>`conf['archives']`</nobr> | string | Comma-separated string of HDFS path(s) to archives to be made available to the application. Example: `hdfs:///Project/<project_name>/Resources/archive.zip,...` | `null` |
248+
| <nobr>`conf['properties']`</nobr> | string | A new line separated (`\n`) list of properties to pass to the Spark application. The properties should be in the format `name=value` | `null` |
248249

249250

250251
## Accessing project data

0 commit comments

Comments
 (0)