Sail
0.6.4+ef2084b

Appearance

This guide is up-to-date with the main branch.

Running Ibis Tests ​

You can run Ibis unit tests with the Spark backend connected to the Sail server.

WARNING

Due to a recent issue with the mimalloc library, the scripts/spark-tests/run-server.sh script may abort with a stack overflow error whenever Python code is involved. This happens when the script runs the Sail server via cargo run on macOS.

To work around this issue, build the package first via hatch run maturin develop and then run the script with the CI environment variable set, which runs the server using the built package.

bash
env CI=1 hatch run scripts/spark-tests/run-server.sh

Running the Spark Connect Server ​

Use the following command to build and run the Sail Spark Connect server in the test-ibis environment.

bash
hatch run test-ibis:scripts/spark-tests/run-server.sh

Running the Tests ​

After running the server, start another terminal and use the following command to run the Ibis tests.

bash
hatch run test-ibis:env scripts/spark-tests/run-tests.sh

The test logs will be written to tmp/spark-tests/<name> where <name> is defined by the TEST_RUN_NAME environment variable whose default value is ibis.

Test Selection ​

You can run selected tests by passing pytest arguments to the script.

bash
hatch run test-ibis:env \
  TEST_RUN_NAME=ibis-selected \
  scripts/spark-tests/run-tests.sh \
  --pyargs ibis.backends -m pyspark -k "test_array"

Note that for the above command, the test logs are written to the directory tmp/spark-tests/ibis-selected, due to the TEST_RUN_NAME environment variable.