Testing and benchmarking¶

Testing¶

Requirements¶

Install the following dependencies into the default Python installation:

pip install pytest selenium pytest-instafail pytest-httpserver

Install geckodriver and chromedriver and check that they are in your PATH.

Running the test suite¶

To run the pytest suite of tests, type on the command line:

pytest src/ pyodide_build/ packages/*/test_*

There are 3 test locations,

src/tests/: general Pyodide tests and tests running the CPython test suite
pyodide_build/tests/: tests related to Pyodide build system (do not require selenium to run)
packages/*/test_*: package specific tests.

Manual interactive testing¶

To run manual interactive tests, a docker environment and a webserver will be used.

Bind port 8000 for testing. To automatically bind port 8000 of the docker environment and the host system, run: ./run_docker
Now, this can be used to test the Pyodide builds running within the docker environment using external browser programs on the host system. To do this, run: ./bin/pyodide serve
This serves the build directory of the Pyodide project on port 8000.
- To serve a different directory, use the --build_dir argument followed by the path of the directory.
- To serve on a different port, use the --port argument followed by the desired port number. Make sure that the port passed in --port argument is same as the one defined as DOCKER_PORT in the run_docker script.
Once the webserver is running, simple interactive testing can be run by visiting this URL: http://localhost:8000/console.html

Benchmarking¶

To run common benchmarks to understand Pyodide’s performance, begin by installing the same prerequisites as for testing. Then run:

make benchmark

Linting¶

Python is linted with flake8. C and Javascript are linted with clang-format.

To lint the code, run:

make lint

Testing framework¶

run_in_pyodide¶

Many tests simply involve running a chunk of code in Pyodide and ensuring it doesn’t error. In this case, one can use the run_in_pyodide decorate from pyodide_build/testing.py, e.g.

from pyodide_build.testing import run_in_pyodide

@run_in_pyodide
def test_add():
    assert 1 + 1 == 2

In this case, the body of the function will automatically be run in Pyodide. The decorator can also be called with arguments. It has two configuration options — standalone and packages.

Setting standalone = True starts a standalone browser session to run the test (the session is shared between tests by default). This is useful for testing things like package loading.

The packages option lists packages to load before running the test. For example,

from pyodide_build.testing import run_in_pyodide

@run_in_pyodide(standalone = True, packages = ["regex"])
def test_regex():
    import regex
    assert regex.search("o", "foo").end() == 2

Version 0.17.0