Skip to content

Commit ea50691

Browse files
update readme
1 parent 8ec81ad commit ea50691

File tree

2 files changed

+10
-17
lines changed

2 files changed

+10
-17
lines changed

README.md

Lines changed: 10 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -14,14 +14,14 @@ pip install pyspark-data-sources[all]
1414
```
1515

1616
## Usage
17-
18-
Install the pyspark 4.0 [preview version](https://pypi.org/project/pyspark/4.0.0.dev2/)
17+
Make sure you use pyspark 4.0. You can install pyspark 4.0 [preview version](https://pypi.org/project/pyspark/4.0.0.dev2/)
1918

2019
```
2120
pip install "pyspark[connect]==4.0.0.dev2"
2221
```
2322

24-
Or use Databricks Runtime 15.4 LTS or above.
23+
Or use [Databricks Runtime 15.4 LTS](https://docs.databricks.com/aws/en/release-notes/runtime/15.4lts) or above versions, or [Databricks Serverless](https://docs.databricks.com/aws/en/compute/serverless/).
24+
2525

2626
Try the data sources!
2727

@@ -52,6 +52,13 @@ poetry install
5252
poetry shell
5353
```
5454

55+
### Install PySpark from the latest Spark master
56+
- Clone the Apache Spark repo: `git clone git@github.com:apache/spark.git`
57+
- Build Spark: `build/sbt clean package`
58+
- Build PySpark: `cd python/packaging/classic && python setup.py sdist`
59+
- Install PySpark: `poetry run pip install <path-to-spark-repo>/python/dist/pyspark-4.1.0.dev0.tar.gz`
60+
61+
5562
### Build docs
5663
```
5764
mkdocs serve

pyspark_datasources/README.md

Lines changed: 0 additions & 14 deletions
This file was deleted.

0 commit comments

Comments
 (0)