Testing and benchmarking#

Testing#

Requirements#

Install the following dependencies into the default Python installation:

pip install pytest selenium pytest-instafail pytest-httpserver

Install geckodriver and chromedriver and check that they are in your PATH.

Running the Python test suite#

To run the pytest suite of tests, type on the command line:

pytest

There are 3 test locations that are collected by pytest,

  • src/tests/: general Pyodide tests and tests running the CPython test suite

  • pyodide-build/pyodide_build/tests/: tests related to Pyodide build system (do not require selenium to run)

  • packages/*/test_*: package specific tests.

Running the JavaScript test suite#

To run tests on the JavaScript Pyodide package using Mocha, run the following commands,

cd src/js
npm test

To check TypeScript type definitions run,

npx tsd

Manual interactive testing#

To run manual interactive tests, a docker environment and a webserver will be used.

  1. Bind port 8000 for testing. To automatically bind port 8000 of the docker environment and the host system, run: ./run_docker

  2. Now, this can be used to test the Pyodide builds running within the docker environment using external browser programs on the host system. To do this, run: pyodide-build serve

  3. This serves the build directory of the Pyodide project on port 8000.

    • To serve a different directory, use the --build_dir argument followed by the path of the directory.

    • To serve on a different port, use the --port argument followed by the desired port number. Make sure that the port passed in --port argument is same as the one defined as DOCKER_PORT in the run_docker script.

  4. Once the webserver is running, simple interactive testing can be run by visiting this URL: http://localhost:8000/console.html

Benchmarking#

To run common benchmarks to understand Pyodide’s performance, begin by installing the same prerequisites as for testing. Then run:

make benchmark

Linting#

Python is linted with flake8, black and mypy. JavaScript is linted with prettier. C is linted with clang-format.

To lint the code, run:

make lint

Testing framework#

run_in_pyodide#

Many tests simply involve running a chunk of code in Pyodide and ensuring it doesn’t error. In this case, one can use the run_in_pyodide decorate from pyodide_build.testing, e.g.

from pyodide_build.testing import run_in_pyodide

@run_in_pyodide
def test_add():
    assert 1 + 1 == 2

In this case, the body of the function will automatically be run in Pyodide. The decorator can also be called with arguments. It has two configuration options — standalone and packages.

Setting standalone = True starts a standalone browser session to run the test (the session is shared between tests by default). This is useful for testing things like package loading.

The packages option lists packages to load before running the test. For example,

from pyodide_build.testing import run_in_pyodide

@run_in_pyodide(standalone = True, packages = ["regex"])
def test_regex():
    import regex
    assert regex.search("o", "foo").end() == 2