Testing and benchmarking¶
Testing¶
Requirements¶
Install the following dependencies into the default Python installation:
pip install pytest selenium pytest-instafail pytest-httpserver
Install geckodriver and
chromedriver
and check that they are in your PATH
.
Running the test suite¶
To run the pytest suite of tests, type on the command line:
pytest src/ pyodide_build/ packages/*/test_*
There are 3 test locations,
src/tests/
: general Pyodide tests and tests running the CPython test suitepyodide_build/tests/
: tests related to Pyodide build system (do not require selenium to run)packages/*/test_*
: package specific tests.
Manual interactive testing¶
To run manual interactive tests, a docker environment and a webserver will be used.
Bind port 8000 for testing. To automatically bind port 8000 of the docker environment and the host system, run:
./run_docker
Now, this can be used to test the Pyodide builds running within the docker environment using external browser programs on the host system. To do this, run:
./bin/pyodide serve
This serves the
build
directory of the Pyodide project on port 8000.To serve a different directory, use the
--build_dir
argument followed by the path of the directory.To serve on a different port, use the
--port
argument followed by the desired port number. Make sure that the port passed in--port
argument is same as the one defined asDOCKER_PORT
in therun_docker
script.
Once the webserver is running, simple interactive testing can be run by visiting this URL: http://localhost:8000/console.html
Benchmarking¶
To run common benchmarks to understand Pyodide’s performance, begin by installing the same prerequisites as for testing. Then run:
make benchmark
Linting¶
Python is linted with flake8
. C and Javascript are linted with
clang-format
.
To lint the code, run:
make lint
Testing framework¶
run_in_pyodide¶
Many tests simply involve running a chunk of code in Pyodide and ensuring it
doesn’t error. In this case, one can use the run_in_pyodide
decorate from
pyodide_build/testing.py
, e.g.
from pyodide_build.testing import run_in_pyodide
@run_in_pyodide
def test_add():
assert 1 + 1 == 2
In this case, the body of the function will automatically be run in Pyodide. The decorator can also be called with arguments. It has two configuration options — standalone and packages.
Setting standalone = True
starts a standalone browser session to run the test
(the session is shared between tests by default). This is useful for testing
things like package loading.
The packages
option lists packages to load before running the test. For
example,
from pyodide_build.testing import run_in_pyodide
@run_in_pyodide(standalone = True, packages = ["regex"])
def test_regex():
import regex
assert regex.search("o", "foo").end() == 2