Testing PySpark��

In order to run PySpark tests, you should build Spark itself first via Maven or SBT. For example,

build/mvn -DskipTests clean package

After that, the PySpark test cases can be run via using python/run-tests. For example,

python/run-tests --python-executable=python3

Note that you may set OBJC_DISABLE_INITIALIZE_FORK_SAFETY environment variable to YES if you are running tests on Mac OS.

Please see the guidance on how to build Spark, run tests for a module, or individual tests.

Running Individual PySpark Tests��

You can run a specific test via using python/run-tests, for example, as below:

python/run-tests --testnames pyspark.sql.tests.test_arrow

Please refer to Testing PySpark for more details.

Running tests using GitHub Actions��

You can run the full PySpark tests by using GitHub Actions in your own forked GitHub repository with a few clicks. Please refer to Running tests in your forked repository using GitHub Actions for more details.