Skip to content

Commit 14cb4e7

Browse files
committed
pyyaml extra and installation instructions
1 parent 80da53e commit 14cb4e7

File tree

2 files changed

+25
-0
lines changed

2 files changed

+25
-0
lines changed

python/docs/source/getting_started/install.rst

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -295,3 +295,20 @@ Package Supported version Note
295295
======= ================= ==================
296296
`numpy` >=1.21 Required for MLlib
297297
======= ================= ==================
298+
299+
Declarative Pipelines
300+
^^^^^^^^^^^^^^^^^^^^^
301+
302+
Installable with ``pip install "pyspark[pipelines]"``. Includes all dependencies for both Spark SQL and Spark Connect, because Declarative Pipelines is built on top of both.
303+
304+
========================== ================= ===================================================
305+
Package Supported version Note
306+
========================== ================= ===================================================
307+
`pandas` >=2.0.0 Required for Spark Connect and Spark SQL
308+
`pyarrow` >=11.0.0 Required for Spark Connect and Spark SQL
309+
`grpcio` >=1.67.0 Required for Spark Connect
310+
`grpcio-status` >=1.67.0 Required for Spark Connect
311+
`googleapis-common-protos` >=1.65.0 Required for Spark Connect
312+
`pyyaml` >=3.11 Required for spark-pipelines command line interface
313+
`graphviz` >=0.20 Optional for Spark Connect
314+
========================== ================= ===================================================

python/packaging/classic/setup.py

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -366,6 +366,14 @@ def run(self):
366366
"grpcio-status>=%s" % _minimum_grpc_version,
367367
"googleapis-common-protos>=%s" % _minimum_googleapis_common_protos_version,
368368
"numpy>=%s" % _minimum_numpy_version,
369+
],
370+
"pipelines": [
371+
"pandas>=%s" % _minimum_pandas_version,
372+
"pyarrow>=%s" % _minimum_pyarrow_version,
373+
"numpy>=%s" % _minimum_numpy_version,
374+
"grpcio>=%s" % _minimum_grpc_version,
375+
"grpcio-status>=%s" % _minimum_grpc_version,
376+
"googleapis-common-protos>=%s" % _minimum_googleapis_common_protos_version,
369377
"pyyaml>=%s" % _minimum_pyyaml_version,
370378
],
371379
},

0 commit comments

Comments
 (0)