Change Log#

Unreleased#

  • Enhancement Added a return type PackageData to pyodide.loadPackage, containing the loaded package’s metadata. #4306

  • BREAKING CHANGE Node.js < 18 is no longer officially supported. Older versions of Node.js might still work, but they are not tested or guaranteed to work. #4269

  • Fix Fixed default indexURL calculation in Node.js environment. #4288

  • Fix Fixed a bug that webpack messes up dynamic import of pyodide.asm.js. #4294

  • Enhancement Added experimental support for stack switching. #3957, #3964, #3987, #3990, #3210

  • Fix jsarray.pop now works correctly. It previously returned the wrong value and leaked memory. #4236

  • BREAKING CHANGE PyProxy.toString now calls str instead of repr. For now you can opt into the old behavior by passing pyproxyToStringRepr: true to loadPyodide, but this may be removed in the future. #4247

  • Fix import type { PyProxy } from "pyodide/ffi" now works with the NodeNext typescript target. #4256

  • Fix when accessing a JsProxy attribute invokes a getter and the getter throws an error, that error is propagated instead of being turned into an AttributeError. #4254

  • Fix Fixed a bug that occurs when using toJs with both dictConverter and defaultConverter arguments. #4263

Pyodide CLI#

  • Enhancement pyodide config command now show additional config variables: rustflags, cmake_toolchain_file, pyo3_config_file, rust_toolchain, cflags cxxflags, ldflags, meson_cross_file. These variables can be used in out-of-tree build to set the same variables as in-tree build. #4241

  • Enhancement pyodide build command now accepts --config-setting (-C) option to pass flags to the build backend, just like python -m build command. #4308

Packages#

Load time & size optimizations#

  • Performance Do not use importlib.metadata when identifying installed packages, which reduces the time to load Pyodide. #4147

Build system#

  • Fix Fixed Emscripten.cmake not vendored in pyodide-build since 0.24.0. #4223

  • Fix pyodide-build now does not override CMAKE_CONFIG_FILE and PYO3_CONFIG_FILE env variables if provided by user. #4223

Version 0.24.1#

September 25, 2023

  • Fix Fixed LONG_BIT definition appears wrong for platform error happened in out-of-tree build. #4136

  • Fix Fixed an Emscripten bug that broke some matplotlib functionality. #4163

  • Fix pyodide.checkInterrupt works when there is no interrupt buffer and the gil is not held. #4164

Packages#

  • Upgraded scipy to 1.11.2 #4156

  • Upgraded sourmash to 4.8.4 #4154

  • Upgraded scikit-learn to 1.3.1 #4161

  • Upgraded micropip to 0.5.0 #4167

Version 0.24.0#

September 13, 2023

General#

JavaScript API#

  • Performance Added a packages optional argument to loadPyodide. Passing packages here saves time by downloading them during the Pyodide bootstrap. #4100

  • Enhancement runPython and runPythonAsync now accept a filename optional argument which is passed as the filename argument to eval_code (resp. eval_code_async). Also, if a filename is passed to eval_code which does not start with < and end with >, Pyodide now uses the linecache module to ensure that source lines can appear in tracebacks. #3993

  • Performance For performance reasons, don’t render extra information in PyProxy destroyed message by default. By using pyodide.setDebug(true), you can opt into worse performance and better error messages. #4027

  • Enhancement It is now possible to pass environment variables to loadPyodide via the env argument. homedir is deprecated in favor of {env: {HOME: whatever_directory}}. #3870

  • Enhancement The setStdin, setStdout and setStderr APIs have been improved with extra control and better performance. #4035

Python API#

  • Enhancement Added headers property to pyodide.http.FetchResponse. #2078

  • Enhancement Added FetchResponse.text() as a synonym to FetchResponse.string() for better compatibility with other requests APIs. #4052

  • BREAKING CHANGE Changed the FetchResponse body getter methods to no longer throw an OSError exception for 400 and above response status codes. Added FetchResponse.raise_for_status to raise an OSError for error status codes. #3986 #4053

Python / JavaScript Foreign Function Interface#

  • Performance Improved performance of PyProxy creation. #4096

  • Fix Fixed adding getters/setters to a PyProxy with Object.defineProperty and improved compliance with JavaScript rules around Proxy traps. #4033

  • Enhancement The promise methods then, catch and finally_ are now present also on Tasks as well as Futures. #3748

  • Enhancement Added methods to a PyProxy of a list to make these work as drop-in replacements for JavaScript Arrays. #3853

  • Enhancement When a JsProxy of an array is passed to Python builtin functions that use the PySequence_* APIs, it now works as expected. Also jsarray * n repeats the array n times and jsarray + iterable returns a new array with the result values from the iterable appended. #3904

Deployment#

  • API Change Changed the name of the default lockfile from repodata.json to pyodide-lock.json #3824

Build System#

  • Update The docker image now has node v20 instead of node v14. #3819

  • Enhancement Added check_wasm_magic_number function to validate .so files for WebAssembly (WASM) compatibility. #4018

  • Enhancement In pyodide build, automatically skip building package dependencies that are already included in the pyodide distribution. #4058

Packages#

Pyodide CLI#

  • Enhancement pyodide build-recipes now accepts a --metadata-files option to install *.whl.metadata files as specified in PEP 658. #3981

Misc#

  • Enhancement Add an example for loadPyodide and pyodide.runPython {pr}4012, {pr}4011`

Version 0.23.4#

July 6, 2023

  • Enhancement The environment variable PYODIDE_BUILD_EXPORTS can now be used instead of the --exports argument to pyodide build to specify .so file exports of packages. #3973

  • Fix Pin pydantic to <2. #3971

  • Enhancement Allow customizing cache location for packages when running in Node #3967

  • Enhancement Re-enabled sparseqr, freesasa, lightgbm, opencv-python, and wordcloud #3783, #3970

  • Fix A JSProxy of a DOMException will now inherit from exception so it can be raised in Python. #3868

  • Fix The feature detection for JSProxy has been improved so that it should never fail even when handling strange or ill-behaved JavaScript proxy objects. #3740, #3750

  • Fix A PyProxy of a callable is now an instanceof Function. (If you are trying to feature detect whether something is callable or not in JavaScript, the correct way is to use typeof o === "function". But you may have dependencies that don’t do this correctly.) #3925

  • Fix from jsmodule import * now works. #3903

Version 0.23.3#

June 17, 2023

  • Fix getattr(jsproxy, 'python_reserved_word') works as expected again (as well as hasattr and setattr). This fixes a regression introduced in #3617. #3926

  • Fix pyodide build now replaces native .so slugs with Emscripten slugs. Usually .sos in the generated wheels are actually Emscripten .sos so this is good. If they are actually native .sos then there is a problem either way. #3903

Version 0.23.2#

May 2, 2023

  • Enhancement Changed the name of the --output-directory argument to pyodide build to --outdir to match pypa/build. --output-directory is still accepted for backwards compatibility. #3811

Version 0.23.1#

April 13, 2023

Deployment#

  • Fix Export python_stdlib.zip in package.json. #3723

CLI#

  • Enhancement pyodide build now accepts an --output-directory argument. #3746

  • Fix Fix pyodide py-compile not to ignore the --compression-level option when applied on a single file. #3727

  • Fix Fix an issue where the pyodide venv command did not work correctly in pyodide-build version 0.23.0 because of missing python_stdlib.zip. #3760

  • Fix python -m pip works correctly in the Pyodide venv now. #3761

  • Fix Executables installed in a Pyodide virtual environment now run in Pyodide not in the host Python. #3752

Build System#

  • Fix Fix PYODIDE_ROOT to point the correct directory when running out-of-tree build. #3751

Version 0.23.0#

March 30, 2023

General#

  • Update Pyodide now runs Python 3.11.2 which officially supports WebAssembly as a PEP11 Tier 3 platform. #3252, #3614

  • Update We now build libpyodide.a so the Pyodide foreign function interface can be experimentally linked into other Emscripten builds of Python. #3335

  • Enhancement Updated Emscripten to version 3.1.32 #3471, #3517, #3599

JavaScript API#

  • BREAKING CHANGE Type exports of PyProxy subtypes have been moved from pyodide to pyodide/ffi and many of them have changed names. The original exports are still available but they are deprecated. #3523

  • BREAKING CHANGE The methods for checking PyProxy capabilities (e.g., supportsHas, isCallable) are now deprecated. Use e.g., instanceof pyodide.ffi.PyCallable instead. #3523

  • Enhancement Added subclasses of PyProxy for each mixin. These can be used to check whether a PyProxy supports a given set of methods with instanceof e.g., x instanceof pyodide.ffi.PyDict. #3523

  • Enhancement Added stdLibURL parameter to loadPyodide allowing to customize the URL from which the Python standard library is loaded. #3670

  • Enhancement Checking whether an object is an instance of a PyProxy now only recognizes a PyProxy generated from the same Python interpreter. This means that creating multiple interpreters and importing a PyProxy from one into another no longer causes a fatal error. #3545

  • Enhancement as_object_map now accepts a keyword argument hereditary. If set to True and indexing the object returns a plain-old-object, then the return value will be automatically mapped in as_object_map as well. #3638

  • Enhancement A JsProxy of a JavaScript error object can be directly thrown as Python exceptions. Previously Pyodide automatically wrapped them in a JsException but that is no longer needed – now JsException inherits from both JsProxy and Exception. #3455

  • Enhancement runPython and runPythonAsync now accept a locals argument. #3618

  • Fix Calling loadPyodide repeatedly in Node no longer results in MaxListenersExceededWarning. Also, calling loadPyodide in Node v14 no longer changes unhandled rejections in promises. #3542

  • Fix If the locals argument to eval_code or eval_code_async is None it now uses locals=globals as the documentation says. #3580

Python standard library#

  • BREAKING CHANGE Unvendored _pydecimal and pydoc_data from the standard library. Now these modules need to be loaded with pyodide.loadPackage or micropip.install, or auto-loaded via imports in pyodide.runPythonAsync #3525

  • BREAKING CHANGE Test files of stdlib ctypes and unittest are now moved to test/ctypes and test/unittest respectively. This change is adapted from CPython 3.12. #3507

Deployment#

  • BREAKING CHANGE Pyodide no longer uses Emscripten preload plugin, hence pyodide.asm.data is removed, in favor of python_stdlib.zip. This change normally shouldn’t affect users, but if you were using this file in a bundler, you will need to remove it. #3584

  • BREAKING CHANGE pyodide_py.tar file is removed. This change normally shouldn’t affect users, but if you were using this file in a bundler, you will need to remove it. #3621

  • BREAKING CHANGE Python standard libraries are now vendored in a zipfile: /lib/python{version}.zip in the in-browser MEMFS file system. If you need to access the standard library source code, you need to unpack the zip file. For example: import shutil; shutil.unpack_archive('/lib/python311.zip', '/lib/python3.11', 'zip) #3584

  • Fix Improves the compression of wheel files with the JsDelivr CDN. For browsers that support the Brotli compression (most modern ones) this should result in a size reduction of 20-30%. Also most many pyodide CLI sub-commands now support --compression-level as an optional parameter. #3655

  • BREAKING CHANGE Following libraries are now not linked to the Pyodide main module: libgl, libal, libhtml5. This normally shouldn’t affect users, but if you are using these libraries in a package that are built out-of-tree, you will need to link them to the package manually. #3505

Python / JavaScript Foreign Function Interface#

  • Fix PyProxies of Async iterators are now async iterable JavaScript objects. The code:

    for await (let x of async_iterator_pyproxy) {
  // ...
}

    would previously fail with TypeError: async_iterator_pyproxy is not async iterable. (Python async iterables that were not also iterators were already async iterable, the problem was only with Python objects that are both async iterable and an async iterator.) #3708

  • Enhancement A py-compiled build which has smaller and faster-to-load packages is now deployed under https://cdn.jsdelivr.net/pyodide/v0.23.0/pyc/ (also for future versions). The exceptions obtained with this builds will not include code snippets however. #3701

  • BREAKING CHANGE Removed support for calling functions from the root of pyodide package directly. This has been deprecated since v0.21.0. Now all functions are only available under submodules. #3677

  • BREAKING CHANGE Removed support for passing the “message” argument to PyProxy.destroy in a positional argument. This has been deprecated since v0.22.0. #3677

  • Enhancement Python does not allow reserved words to be used as attributes. For instance, Array.from is a SyntaxError. (JavaScript has a more robust parser which can handle this.) To handle this, if an attribute to a JsProxy consists of a Python reserved word followed by one or more underscores, we remove a single underscore from the end of the attribute. For instance, Array.from_ would access from on the underlying JavaScript object, whereas o.from__ accesses the from_ attribute. #3617

Build System#

  • BREAKING CHANGE When building meta-packages (core and min-scipy-stack), you must prefix tag: to the meta-package name. For example, to build the core meta-package, you must run pyodide build-recipes tag:core, or PYODIDE_PACKAGES="tag:core" make. #3444

  • Enhancement Add --build-dependencies to pyodide build command to fetch and build dependencies of a package being built. Also adds --skip-dependency to ignore selected dependencies. #3310

  • Enhancement Added pyodide build support for building a list of packages from a requirements.txt file with pyodide build -r <requirements.txt>. Also can output a list of chosen dependencies in the same format when building a package and dependencies using the --output-lockfile <lockfile.txt> argument. This enables repeatable builds of packages. #3469

  • Enhancement Added package/tag key to the meta.yaml spec to group packages. #3444

  • Enhancement pyodide build-recipes now autodetects the number of CPU cores in the system and uses them for parallel builds. #3559 #3598

  • Fix Fixed pip install error when installing cross build environment. #3562

  • Enhancement Response files are now correctly handled when calculating exported symbols. #3645

  • Fix Fix occasional build failure when building rust packages. #3607

  • Enhancement Improved logging in pyodide-build with rich. #3442

  • Enhancement pyodide build-recipes now accepts --no-deps parameter, which skips building dependencies of the package. This replaces pyodide-build buildpkg. #3520

  • Enhancement pyodide build-recipes now works out-of-tree.

Pyodide CLI#

  • BREAKING CHANGE Removed deprecated CLI entrypoints pyodide-build buildall which is replaced by pyodide build-recipes, and pyodide-build mkpkg which is replaced by pyodide skeleton pypi #3668.

  • Feature Added pyodide py-compile CLI command that py compiles a wheel or a zip file, converting .py files to .pyc files. It can also be applied to a folder with wheels / zip files. If the input folder contains the repodata.json the paths and checksums it contains will also be updated #3253 #3700

  • Feature Added pyodide create-zipfile CLI command that creates a zip file of a directory. This command is hidden by default since it is not intended for use by end users. #3411 #3463

REPL#

  • Fix Non-breaking space characters are now automatically converted to regular spaces in pyodide REPL. #3558

  • Enhancement Allow changing the build type used in the REPL by passing the build argument to the REPL URL. For instance, https://pyodide.org/en/latest/console.html?build=debug will load debug dev build. #3671

Packages#

List of Contributors#

Alexey Ignatiev, Andrea Giammarchi, Arpit, Christian Clauss, Deepak Cherian, Eli Lamb, Feodor Fitsner, Gyeongjae Choi, Hood Chatham, Jeff Glass, Jo Bovy, Joe Marshall, josephrocca, Loïc Estève, martinRenou, messense, Nicholas Bollweg, Roman Yurchak, TheOnlyWayUp, Victor Blomqvist, Ye Joo Park

Version 0.22.1#

January 25, 2023

  • BREAKING CHANGE setStdin now accepts an extra autoEOF parameter. If true, it will insert an EOF automatically after each string or buffer. Defaults to true. This also affects the behavior of the stdin argument to loadPyodide. #3488

  • Fix from pyodide.ffi import * doesn’t raise an ImportError anymore. #3484

  • Enhancement Pyodide displays a better message when someone calls posix exit or os._exit. #3496

Package Loading#

  • Fix Fix incorrect error message when loading a package include in Pyodide fails. #3435

Build system#

  • Fix Emscripten is no longer required to create a Pyodide virtual environment. #3485

  • Fix Fixed a bug where pyodide build would fail on package that use CMake, when run multiple times. #3445

  • Fix pyodide build: Don’t pass the directory to the build backend args, only pass the arguments. #3490

  • Fix pyodide config won’t print extra messages anymore. #3483

  • Fix Pass the same environment variables for out of tree builds as for in tree builds. #3495

Version 0.22.0#

January 3, 2023

See the release notes for a summary.

Deployment and testing#

  • BREAKING CHANGE pyodide-cdn2.iodide.io is not available anymore. Please use https://cdn.jsdelivr.net/pyodide instead. #3150.

  • BREAKING CHANGE We don’t publish pre-built Pyodide docker images anymore. Note that ./run_docker --pre-built was not working for a while and it was actually equivalent to ./run_docker. If you need to build a single Python wheel out of tree, you can use the pyodide build command instead. See our blog post for more information. #3342.

  • Enhancement The releases are now called pyodide-{version}.tar.gz rather than pyodide-build-{version}.tar.gz #2996

  • Enhancement Added a new release file called pyodide-core-{version}.tar.gz intended for use in Node. It contains the files needed to start Pyodide and no additional packages. #2999

  • Enhancement The full test suite is now run in Safari #2578, #3095.

  • Enhancement Added Gitpod configuration to the repository. #3201

Foreign function interface#

JsProxy / JavaScript from Python#

  • Enhancement Implemented reverse, __reversed__, count, index, append, and pop for JsProxy of Javascript arrays so that they implement the collections.abc.MutableSequence API. #2970

  • Enhancement Implemented methods keys, items, values, get, pop, setdefault, popitem, update, and clear for JsProxy of map-like objects so that they implement the collections.abc.MutableMapping API. #3275

  • Enhancement It’s now possible to destructure a JavaScript array, map, or object returned by as_object_map with a match statement. #2906

  • Enhancement Added then, catch, and finally_ methods to the Futures used by Pyodide’s event loop so they can be used like Promises. #2997

  • Enhancement create_proxy now takes an optional roundtrip parameter. If this is set to True, then when the proxy is converted back to Python, it is converted back to the same double proxy. This allows the proxy to be destroyed from Python even if no reference is retained. #3163, #3369

  • Enhancement A JsProxy of a function now has a __get__ descriptor method, so it’s possible to use a JavaScript function as a Python method. When the method is called, this will be a PyProxy pointing to the Python object the method is called on. #3130

  • Enhancement A JsProxy now has an as_object_map method. This will treat the object as a mapping over its ownKeys so for instance: run_js("({a:2, b:3})").as_object_map()["a"] will return 2. These implement collections.abc.MutableMapping. #3273, #3295, #3297

  • Enhancement Split up the JsProxy documentation class into several classes, e.g., JsBuffer, JsPromise, etc. Implemented issubclass and isinstance on the various synthetic and real JsProxy classes so that they behave the way one might naively expect them to (or at least closer to that than it was before). #3277

  • Enhancement Added type parameters to many of the JsProxy subtypes. #3387

  • Enhancement Added JsGenerator and JsIterator types to pyodide.ffi. Added send method to JsIterators and throw, and close methods to JsGenerators. #3294

  • Enhancement It is now possible to use asynchronous JavaScript iterables, iterators and generators from Python. This includes support for aiter for async interables, anext and asend for async iterators, and athrow and aclose for async generators. #3285, #3299, #3339

  • Enhancement JavaScript generators and async generators that are created from Python now are wrapped so that Python objects sent to them as arguments or from .send / .asend are kept alive until the generator is exhausted or .closed. This makes generators significantly more ergonomic to use, at the cost of making memory leaks more likely if the generator is never finalized. #3317

  • Enhancement Added a mypy typeshed for some common functionality for the js module. #3298

  • Enhancement mypy understands the types of more things now. #3385

  • Fix Fixed bug in split argument of pyodide.console.repr_shorten. Added shorten function. #3178

PyProxy / Using Python from JavaScript#

  • Enhancement Added a type field to PythonError (e.g., a StopIteration error would have e.type === "StopIteration") #3289

  • Enhancement It is now possible to use asynchronous Python generators from JavaScript. #3290

  • Enhancement PyProxies of synchronous and asynchronous Python generators now support return and throw APIs that behave like the ones on JavaScript generators. #3346

  • Enhancement It is possible to make a PyProxy that takes this as the first argument using the PyProxy.captureThis method. The create_proxy method also has a capture_this argument which causes the PyProxy to receive this as the first argument if set to True #3103, #3145

JavaScript API#

  • Enhancement Users can do a static import of pyodide/pyodide.asm.js to avoid issues with dynamic imports. This allows the use of Pyodide with module-type service workers. #3070

  • Enhancement Added a new API pyodide.mountNativeFS which mounts a FileSystemDirectoryHandle into the Pyodide file system. #2987

  • Enhancement loadPyodide has a new option called args. This list will be passed as command line arguments to the Python interpreter at start up. #3021, #3282

  • Removed “Python initialization complete” message printed when loading is finished. {pr}`3247

  • BREAKING CHANGE The messageCallback and errorCallback argument to loadPackage and loadPackagesFromImports is now passed as named arguments. The old usage still works with a deprecation warning. #3149

  • Enhancement loadPackage and loadPackagesFromImports now accepts a new option checkIntegrity. If set to False, integrity check for Python Packages will be disabled.

  • Enhancement Added APIs pyodide.setStdin, pyodide.setStdout, pyodide.setStderr for changing the stream handlers after loading Pyodide. Also added more careful control over whether isatty returns true or false on stdin, stdout, and stderr. #3268

Package Loading#

  • Enhancement Pyodide now shows more helpful error messages when importing packages that are included in Pyodide fails. #3137, #3263

  • Fix Shared libraries with version suffixes are now handled correctly. #3154

  • BREAKING CHANGE Unvendored the sqlite3 module from the standard library. Before sqlite3 was included by default. Now it needs to be loaded with pyodide.loadPackage or micropip.install. #2946

  • BREAKING CHANGE The Pyodide Python package is installed into /lib/python3.10 rather than /lib/python3.10/site-packages. #3022

  • BREAKING CHANGE The matplotlib HTML5 backends are now available as part of the matplotlib-pyodide package. If you use the default backend from Pyodide, no changes are necessary. However, if you previously specified the backend with matplotlib.use, the URL is now different. See package readme for more details. #3061

  • BREAKING CHANGE The micropip package was moved to a separate repository pyodide/micropip. In addion to installing the version shipped with a given Pyodide release, you can also install a different micropip version from PyPi with,

    await pyodide.loadPackage('packaging')
await pyodide.loadPackage('<URL of the micropip wheel on PyPI>')

    from Javascript. From Python you can import the Javascript Pyodide package,

    import pyodide_js

    and call the same functions as above. #3122

  • Enhancement The parsing and validation of meta.yaml according to the specification is now done more rigorously with Pydantic. #3079

  • BREAKING CHANGE The source/md5 checksum field is not longer supported in meta.yaml files, use source/sha256 instead #3079

  • BREAKING CHANGE pyodide_build.io.parse_package_config function is removed in favor of pyodide_build.MetaConfig.from_yaml #3079

  • Fix ctypes.util.find_library will now search WASM modules from LD_LIBRARY_PATH. #3353

Build System#

  • Enhancement Updated Emscripten to version 3.1.27 #2958, #2950, #3027, #3107, #3148, #3236, #3239, #3280, #3314

  • Enhancement Added requirements/host key to the meta.yaml spec to allow host dependencies that are required for building packages. #2132

  • Enhancement Added package/top-level key to the meta.yaml spec to calculate top-level import names for the package. Previously test/imports key was used for this purpose. #3006

  • Enhancement Added build/vendor-sharedlib key to the meta.yaml spec which vendors shared libraries into the wheel after building. #3234 #3264

  • Enhancement Added build/type key to the meta.yaml spec which specifies the type of the package. #3238

  • Enhancement Added requirements/executable key to the meta.yaml spec which specifies the list of executables required for building a package. #3300

  • BREAKING CHANGE build/library and build/sharedlibrary key in the meta.yaml spec are removed. Use build/type instead. #3238

  • Fix Fixed a bug that backend-flags propagated to dependencies. #3153

  • Fix Fixed a bug that shared libraries are not copied into distribution directory when it is already built. #3212

  • Enhancement Added a system for making Pyodide virtual environments. This is for testing out of tree builds. For more information, see the documentation. #2976, #3039, #3040, #3044, #3096, #3098, #3108, #3109, #3241

  • Added a new CLI command pyodide skeleton which creates a package build recipe. pyodide-build mkpkg will be replaced by pyodide skeleton pypi. #3175

  • Added a new CLI command pyodide build-recipes which build packages from recipe folder. It replaces pyodide-build buildall. #3196 #3279

  • Added a new CLI command pyodide config which shows config variables used in Pyodide. #3376

  • Added subcommands for pyodide build which builds packages from various sources.

    command

    result

    pyodide build pypi

    build or fetch a single package from pypi

    pyodide build source

    build the current source folder (same as pyodide build)

    pyodide build url

    build or fetch a package from a url either tgz, tar.gz zip or wheel

    #3196

Packages#

  • New packages: pycryptodome #2965, coverage-py #3053, bcrypt #3125, lightgbm #3138, pyheif, pillow_heif, libheif, libde265 #3161, wordcloud #3173, gdal, fiona, geopandas #3213, the standard library _hashlib module #3206 , pyinstrument #3258, gensim #3326, smart_open #3326, pyodide-http #3355.

  • Fix Scipy CSR data is now handled correctly in XGBoost. #3194

  • Update Upgraded packages: SciPy 1.9.1 #3043, pandas 1.5.0 #3134, numpy 1.23.3 #3284, scikit-learn 1.1.3 #3324 as well as most of the other packages #3348 #3365. See Packages built in Pyodide for more details.

  • Fix Fix scipy handling of exceptions that are raised from C++ code. #3384.

List of Contributors#

Aierie, dataxerik, David Lechner, Deepak Cherian, Filipe, Gyeongjae Choi, Hood Chatham, H.Yamada, Jacques Boscq, Jeremy Tuloup, Joe Marshall, John Wason, Loïc Estève, partev, Patrick Arminio, Péter Ferenc Gyarmati, Prete, Qijia Liu, Roman Yurchak, ryanking13, skelsec, Starz0r, Will Lachance, YeonWoo, Yizhi Liu

Version 0.21.3#

September 15, 2022

  • Fix When loading sqlite3, loadPackage no longer also loads nltk and regex. #3001

  • Fix Packages are now loaded in a topologically sorted order regarding their dependencies. #3020

  • BREAKING CHANGE Loading the soupsieve package will not automatically load beautifulsoup4 together. #3020

  • Fix Fix the incorrect package name ruamel to ruamel.yaml. #3036

  • Fix loadPyodide will now raise error when the version of JavaScript and Python Pyodide package does not match. #3074

  • Enhancement Pyodide now works with a content security policy that doesn’t include unsafe-eval. It is still necessary to include wasm-unsafe-eval (and probably always will be). Since current Safari versions have no support for wasm-unsafe-eval, it is necessary to include unsafe-eval in order to work in Safari. This will likely be fixed in the next Safari release: https://bugs.webkit.org/show_bug.cgi?id=235408 #3075

  • Fix It works again to use loadPyodide with a relative URL as indexURL (this was a regression in v0.21.2). #3077

  • Fix Add url to list of pollyfilled packages for webpack compatibility. #3080

  • Fix Fixed warnings like Critical dependency: the request of a dependency is an expression. when using Pyodide with webpack. #3080

  • Enhancement Add binary files to exports in JavaScript package #3085.

  • Fix Source maps are included in the distribution again (reverting #3015 included in 0.21.2) and if there is a variable in top level scope called __dirname we use that for the indexURL. #3088

  • Fix PyProxy.apply now correctly handles the case when something unexpected is passed as the second argument. #3101

Version 0.21.2#

August 29, 2022

  • Fix The standard library packages ssl and lzma can now be installed with pyodide.loadPackage("ssl") or micropip.install("ssl") (previously they had a leading underscore and it was only possible to load them with pyodide.loadPackage). #3003

  • Fix If a wheel path is passed to pyodide.loadPackage, it will now be resolved relative to document.location (in browser) or relative to the current working directory (in Node) rather than relative to indexURL. #3013, #3011

  • Fix Fixed a bug in Emscripten that caused Pyodide to fail in Jest. #3014

  • Fix It now works to pass a relative url to indexURL. Also, the calculated index URL now works even if node is run with --enable-source-maps. #3015

Version 0.21.1#

August 22, 2022

  • New packages: the standard library lzma module #2939

  • Enhancement Pyodide now shows more helpful error messages when importing unvendored or removed stdlib modules fails. #2973

  • BREAKING CHANGE The default value of fullStdLib in loadPyodide has been changed to false. This means Pyodide now will not load some stdlib modules like distutils, ssl, and sqlite3 by default. See Pyodide Python compatibility for detail. If fullStdLib is set to true, it will load all unvendored stdlib modules. However, setting fullStdLib to true will increase the initial Pyodide load time. So it is preferable to explicitly load the required module. #2998

  • Enhancement pyodide build now checks that the correct version of the Emscripten compiler is used. #2975, #2990

  • Fix Pyodide works in Safari v14 again. It was broken in v0.21.0 #2994

Version 0.21.0#

August 9, 2022

See the release notes for a summary.

Build system#

  • Enhancement Emscripten was updated to Version 3.1.14 #2775, #2679, #2672

  • Fix Fix building on macOS #2360 #2554

  • Enhancement Update Typescript target to ES2017 to generate more modern Javascript code. #2471

  • Enhancement We now put our built files into the dist directory rather than the build directory. #2387

  • Fix The build will error out earlier if cmake or libtool are not installed. #2423

  • Enhancement The platform tags of wheels now include the Emscripten version in them. This should help ensure ABI compatibility if Emscripten wheels are distributed outside of the main Pyodide distribution. #2610

  • Enhancement The build system now uses the sysconfigdata from the target Python rather than the host Python. #2516

  • Enhancement Pyodide now builds with -sWASM_BIGINT. #2643

  • Enhancement Added cross-script key to the meta.yaml spec to allow executing custom logic in the cross build environment. #2734

Pyodide Module and type conversions#

  • API Change All functions were moved out of the root pyodide package into various submodules. For backwards compatibility, they will be available from the root package (raising a FutureWarning) until v0.23.0. #2787, #2790

  • Enhancement loadPyodide no longer uses any global state, so it can be used more than once in the same thread. This is recommended if a network request causes a loading failure, if there is a fatal error, if you damage the state of the runtime so badly that it is no longer usable, or for certain testing purposes. It is not recommended for creating multiple execution environments, for which you should use pyodide.runPython(code, { globals : some_dict}); #2391

  • Enhancement pyodide.unpackArchive now accepts any ArrayBufferView or ArrayBuffer as first argument, rather than only a Uint8Array. #2451

  • Feature Added pyodide.code.run_js API. #2426

  • Fix BigInt’s between 2^{32*n - 1} and 2^{32*n} no longer get translated to negative Python ints. #2484

  • Fix Pyodide now correctly handles JavaScript objects with null constructor. #2520

  • Fix Fix garbage collection of once_callable #2401

  • Enhancement Added the js_id attribute to JsProxy to allow using JavaScript object identity as a dictionary key. #2515

  • Fix Fixed a bug with toJs when used with recursive structures and the dictConverter argument. #2533

  • Enhancement Added Python wrappers set_timeout, clear_timeout, set_interval, clear_interval, add_event_listener and remove_event_listener for the corresponding JavaScript functions. #2456

  • Fix If a request fails due to CORS, pyfetch now raises an OSError not a JSException. #2598

  • Enhancement Pyodide now directly exposes the Emscripten PATH and ERRNO_CODES APIs. #2582

  • Fix The bool operator on a JsProxy now behaves more consistently: it returns False if JavaScript would say that !!x is false, or if x is an empty container. Otherwise it returns True. #2803

  • Fix Fix loadPyodide errors for the Windows Node environment. #2888

  • Enhancement Implemented slice subscripting, +=, and extend for JsProxy of Javascript arrays. #2907

REPL#

  • Enhancement Add a spinner while the REPL is loading #2635

  • Enhancement Cursor blinking in the REPL can be disabled by setting noblink in URL search params. #2666

  • Fix Fix a REPL error in printing high-dimensional lists. #2517 #2919

  • Fix Fix output bug with using input() on online console #2509

micropip and package loading#

  • API Change packages.json which contains the dependency graph for packages was renamed to repodata.json to avoid confusion with package.json used in JavaScript packages.

  • Enhancement Added SHA-256 hash of package to entries in repodata.json #2455

  • Enhancement Integrity of Pyodide packages is now verified before loading them. This is for now limited to browser environments. #2513

  • Enhancement micropip supports loading wheels from the Emscripten file system using the emfs: protocol now. #2767

  • Enhancement It is now possible to use an alternate repodata.json lockfile by passing the lockFileURL option to loadPyodide. This is particularly intended to be used with micropip.freeze. #2645

  • Fix micropip now correctly handles package names that include dashes #2414

  • Enhancement Allow passing credentials to micropip.install() #2458

  • Enhancement micropip.install() now accepts a deps parameter. If set to False, micropip will not install dependencies of the package. #2433

  • Fix micropip now correctly compares packages with prerelease version #2532

  • Enhancement micropip.install() now accepts a pre parameter. If set to True, micropip will include pre-release and development versions. #2542

  • Enhancement micropip was refactored to improve readability and ease of maintenance. #2561, #2563, #2564, #2565, #2568

  • Enhancement Various error messages were fine tuned and improved. #2562, #2558

  • Enhancement micropip was adjusted to keep its state in the wheel .dist-info directories which improves consistenency with the Python standard library and other tools used to install packages. #2572

  • Enhancement micropip can now be used to install Emscripten binary wheels. #2591

  • Enhancement Added micropip.freeze to record the current set of loaded packages into a repodata.json file. #2581

  • Fix micropip.list now works correctly when there are packages that are installed via pyodide.loadPackage from a custom URL. #2743

  • Fix micropip now skips package versions which do not follow PEP440. #2754

  • Fix micropip supports extra markers in packages correctly now. #2584

Packages#

  • Enhancement Update sqlite version to latest stable release #2477 and #2518

  • Enhancement Pillow now supports WEBP image format #2407.

  • Enhancement Pillow and opencv-python now support the TIFF image format. #2762

  • Pandas is now compiled with -Oz, which significantly speeds up loading the library on Chrome #2457

  • New packages: opencv-python #2305, ffmpeg #2305, libwebp #2305, h5py, pkgconfig and libhdf5 #2411, bitarray #2459, gsw #2511, cftime #2504, svgwrite, jsonschema, tskit #2506, xarray #2538, demes, libgsl, newick, ruamel, msprime #2548, gmpy2 #2665, xgboost #2537, galpy #2676, shapely, geos #2725, suitesparse, sparseqr #2685, libtiff #2762, pytest-benchmark #2799, termcolor #2809, sqlite3, libproj, pyproj, certifi #2555, rebound #2868, reboundx #2909, pyclipper #2886, brotli #2925, python-magic #2941

Miscellaneous#

  • Fix We now tell packagers (e.g., Webpack) to ignore npm-specific imports when packing files for the browser. #2468

  • Enhancement run_in_pyodide now has support for pytest assertion rewriting and decorators such as pytest.mark.parametrize and hypothesis. #2510, #2541

  • BREAKING CHANGE pyodide_build.testing is removed. run_in_pyodide decorator can now be accessed through pytest-pyodide package. #2418

List of contributors#

Alexey Ignatiev, Andrey Smelter, andrzej, Antonio Cuni, Ben Jeffery, Brian Benjamin Maranville, David Lechner, dragoncoder047, echorand (Amit Saha), Filipe, Frank, Gyeongjae Choi, Hanno Rein, haoran1062, Henry Schreiner, Hood Chatham, Jason Grout, jmdyck, Jo Bovy, John Wason, josephrocca, Kyle Cutler, Lester Fan, Liumeo, lukemarsden, Mario Gersbach, Matt Toad, Michael Droettboom, Michael Gilbert, Michael Neil, Mu-Tsun Tsai, Nicholas Bollweg, pysathq, Ricardo Prins, Rob Gries, Roman Yurchak, Ryan May, Ryan Russell, stonebig, Szymswiat, Tobias Megies, Vic Kumar, Victor, Wei Ji, Will Lachance

Version 0.20.0#

April 9th, 2022

See the release notes for a summary.

CPython and stdlib#

  • Update Pyodide now runs Python 3.10.2. #2225

  • Enhancement All ctypes tests pass now except for test_callback_too_many_args (and we have a plan to fix test_callback_too_many_args upstream). libffi-emscripten now also passes all libffi tests. #2350

Packages#

  • Fix matplotlib now loads multiple fonts correctly #2271

  • New packages: boost-histogram #2174, cryptography v3.3.2 #2263, the standard library ssl module #2263, python-solvespace v3.0.7, lazy-object-proxy #2320.

  • Many more scipy linking errors were fixed, mostly related to the Fortran f2c ABI for string arguments. There are still some fatal errors in the Scipy test suite, but none seem to be simple linker errors. #2289

  • Removed pyodide-interrupts. If you were using this for some reason, use pyodide.setInterruptBuffer instead. #2309

  • Most included packages were updated to the latest version. See Packages built in Pyodide for a full list.

Type translations#

  • Fix Python tracebacks now include Javascript frames when Python calls a Javascript function. #2123

  • Enhancement Added a default_converter argument to JsProxy.to_py and pyodide.toPy which is used to process any object that doesn’t have a built-in conversion to Python. Also added a default_converter argument to PyProxy.toJs and pyodide.ffi.to_js to convert. #2170 and #2208

  • Enhancement Async Python functions called from Javascript now have the resulting coroutine automatically scheduled. For instance, this makes it possible to use an async Python function as a Javascript event handler. #2319

Javascript package#

  • Enhancement It is no longer necessary to provide indexURL to loadPyodide. #2292

  • BREAKING CHANGE The globals argument to pyodide.runPython and pyodide.runPythonAsync is now passed as a named argument. The old usage still works with a deprecation warning. #2300

  • Enhancement The Javascript package was migrated to Typescript. #2130 and #2133

  • Fix Fix importing pyodide with ESM syntax in a module type web worker. #2220

  • Enhancement When Pyodide is loaded as an ES6 module, no global loadPyodide variable is created (instead, it should be accessed as an attribute on the module). #2249

  • Fix The type Py2JsResult has been replaced with any which is more accurate. For backwards compatibility, we still export Py2JsResult as an alias for any. #2277

  • Fix Pyodide now loads correctly even if requirejs is included. #2283

  • Enhancement Added robust handling for non-Error objects thrown by Javascript code. This mostly should never happen since well behaved Javascript code ought to throw errors. But it’s better not to completely crash if it throws something else. #2294

pyodide_build#

  • Enhancement Pyodide now uses Python wheel files to distribute packages rather than the emscripten file_packager.py format. #2027

  • Enhancement Pyodide now uses pypa/build to build packages. We (mostly) use build isolation, so we can build packages that require conflicting versions of setuptools or alternative build backends. #2272

  • Enhancement Most pure Python packages were switched to use the wheels directly from PyPI rather than rebuilding them. #2126

  • Enhancement Added support for C++ exceptions in packages. Now C++ extensions compiled and linked with -fexceptions can catch C++ exceptions. Furthermore, uncaught C++ exceptions will be formatted in a human-readable way. #2178

  • BREAKING CHANGE Removed the skip-host key from the meta.yaml format. If needed, install a host copy of the package with pip instead. #2256

Uncategorized#

  • Enhancement The interrupt buffer can be used to raise all 64 signals now, not just SIGINT. Write a number between 1<= signum <= 64 into the interrupt buffer to trigger the corresponding signal. By default everything but SIGINT will be ignored. Any value written into the interrupt buffer outside of the range from 1 to 64 will be silently discarded. #2301

  • Enhancement Updated to Emscripten 2.0.27. #2295

  • BREAKING CHANGE The extractDir argument to pyodide.unpackArchive is now passed as a named argument. The old usage still works with a deprecation warning. #2300

  • Enhancement Support ANSI escape codes in the Pyodide console. #2345

  • Fix pyodide_build can now be installed in non-editable ways. #2351

List of contributors#

Boris Feld, Christian Staudt, Gabriel Fougeron, Gyeongjae Choi, Henry Schreiner, Hood Chatham, Jo Bovy, Karthikeyan Singaravelan, Leo Psidom, Liumeo, Luka Mamukashvili, Madhur Tandon, Paul Korzhyk, Roman Yurchak, Seungmin Kim, Thorsten Beier, Tom White, and Will Lachance

Version 0.19.1#

February 19, 2022

Packages#

  • New packages: sqlalchemy #2112, pydantic #2117, wrapt #2165

  • Update Upgraded packages: pyb2d (0.7.2), #2117

  • Fix A fatal error in scipy.stats.binom.ppf has been fixed. #2109

  • Fix Type signature mismatches in some numpy comparators have been fixed. #2110

Type translations#

  • Fix The “PyProxy has already been destroyed” error message has been improved with some context information. #2121

REPL#

  • Enhancement Pressing TAB in REPL no longer triggers completion when input is whitespace. #2125

List of contributors#

Christian Staudt, Gyeongjae Choi, Hood Chatham, Liumeo, Paul Korzhyk, Roman Yurchak, Seungmin Kim, Thorsten Beier

Version 0.19.0#

January 10, 2021

See the release notes for a summary.

Python package#

  • Enhancement If find_imports is used on code that contains a syntax error, it will return an empty list instead of raising a SyntaxError. #1819

  • Enhancement Added the pyodide.http.pyfetch API which provides a convenience wrapper for the Javascript fetch API. The API returns a response object with various methods that convert the data into various types while minimizing the number of times the data is copied. #1865

  • Enhancement Added the unpack_archive API to the pyodide.http.FetchResponse object which treats the response body as an archive and uses shutil to unpack it. #1935

  • Fix The Pyodide event loop now works correctly with cancelled handles. In particular, asyncio.wait_for now functions as expected. #2022

JavaScript package#

  • Fix loadPyodide no longer fails in the presence of a user-defined global named process. #1849

  • Fix Various webpack buildtime and runtime compatibility issues were fixed. #1900

  • Enhancement Added the pyodide.pyimport API to import a Python module and return it as a PyProxy. Warning: this is different from the original pyimport API which was removed in this version. #1944

  • Enhancement Added the pyodide.unpackArchive API which unpacks an archive represented as an ArrayBuffer into the working directory. This is intended as a way to install packages from a local application. #1944

  • API Change loadPyodide now accepts a homedir parameter which sets home directory of Pyodide virtual file system. #1936

  • BREAKING CHANGE The default working directory(home directory) inside the Pyodide virtual file system has been changed from / to /home/pyodide. To get the previous behavior, you can

    • call os.chdir("/") in Python to change working directory or

    • call loadPyodide with the homedir="/" argument #1936

Python / JavaScript type conversions#

  • BREAKING CHANGE Updated the calling convention when a JavaScript function is called from Python to improve memory management of PyProxies. PyProxy arguments and return values are automatically destroyed when the function is finished. #1573

  • Enhancement Added JsProxy.to_string, JsProxy.to_bytes, and JsProxy.to_memoryview to allow for conversion of TypedArray to standard Python types without unneeded copies. #1864

  • Enhancement Added JsProxy.to_file and JsProxy.from_file to allow reading and writing Javascript buffers to files as a byte stream without unneeded copies. #1864

  • Fix It is now possible to destroy a borrowed attribute PyProxy of a PyProxy (as introduced by #1636) before destroying the root PyProxy. #1854

  • Fix If __iter__() raises an error, it is now handled correctly by the PyProxy[Symbol.iterator()] method. #1871

  • Fix Borrowed attribute PyProxys are no longer destroyed when the root PyProxy is garbage collected (because it was leaked). Doing so has no benefit to nonleaky code and turns some leaky code into broken code (see #1855 for an example). #1870

  • Fix Improved the way that pyodide.globals.get("builtin_name") works. Before we used __main__.__dict__.update(builtins.__dict__) which led to several undesirable effects such as __name__ being equal to "builtins". Now we use a proxy wrapper to replace pyodide.globals.get with a function that looks up the name on builtins if lookup on globals fails. #1905

  • Enhancement Coroutines have their memory managed in a more convenient way. In particular, now it is only necessary to either await the coroutine or call one of .then, .except or .finally to prevent a leak. It is no longer necessary to manually destroy the coroutine. Example: before:

async function runPythonAsync(code, globals) {
  let coroutine = Module.pyodide_py.eval_code_async(code, globals);
  try {
    return await coroutine;
  } finally {
    coroutine.destroy();
  }
}

After:

async function runPythonAsync(code, globals) {
  return await Module.pyodide_py.eval_code_async(code, globals);
}

#2030

pyodide-build#

  • API Change By default only a minimal set of packages is built. To build all packages set PYODIDE_PACKAGES='*' In addition, make minimal was removed, since it is now equivalent to make without extra arguments. #1801

  • Enhancement It is now possible to use pyodide-build buildall and pyodide-build buildpkg directly. #2063

  • Enhancement Added a --force-rebuild flag to buildall and buildpkg which rebuilds the package even if it looks like it doesn’t need to be rebuilt. Added a --continue flag which keeps the same source tree for the package and can continue from the middle of a build. #2069

  • Enhancement Changes to environment variables in the build script are now seen in the compile and post build scripts. #1706

  • Fix Fix usability issues with pyodide-build mkpkg CLI. #1828

  • Enhancement Better support for ccache when building Pyodide #1805

  • Fix Fix compile error wasm-ld: error: unknown argument: --sort-common and wasm-ld: error: unknown argument: --as-needed in ArchLinux. #1965

micropip#

  • Fix micropip now raises an error when installing a non-pure python wheel directly from a url. #1859

  • Enhancement micropip.install() now accepts a keep_going parameter. If set to True, micropip reports all identifiable dependencies that don’t have pure Python wheels, instead of failing after processing the first one. #1976

  • Enhancement Added a new API micropip.list() which returns the list of installed packages by micropip. #2012

Packages#

  • Enhancement Unit tests are now unvendored from Python packages and included in a separate package <package name>-tests. This results in a 20% size reduction on average for packages that vendor tests (e.g. numpy, pandas, scipy). #1832

  • Update Upgraded SciPy to 1.7.3. There are known issues with some SciPy components, the current status of the scipy test suite is here #2065

  • Fix The built-in pwd module of Python, which provides a Unix specific feature, is now unvendored. #1883

  • Fix pillow and imageio now correctly encode/decode grayscale and black-and-white JPEG images. #2028

  • Fix The numpy fft module now works correctly. #2028

  • New packages: logbook #1920, pyb2d #1968, and threadpoolctl (a dependency of scikit-learn) #2065

  • Upgraded packages: numpy (1.21.4) #1934, scikit-learn (1.0.2) #2065, scikit-image (0.19.1) #2005, msgpack (1.0.3) #2071, astropy (5.0.3) #2086, statsmodels (0.13.1) #2073, pillow (9.0.0) #2085. This list is not exhaustive, refer to packages.json for the full list.

Uncategorized#

  • Enhancement PyErr_CheckSignals now works with the keyboard interrupt system so that cooperative C extensions can be interrupted. Also, added the pyodide.checkInterrupt function so Javascript code can opt to be interrupted. #1294

  • Fix The _ variable is now set by the Pyodide repl just like it is set in the native Python repl. #1904

  • Enhancement pyodide-env and pyodide Docker images are now available from both the Docker Hub and from the Github Package registry. #1995

  • Fix The console now correctly handles it when an object’s __repr__ function raises an exception. #2021

  • Enhancement Removed the -s EMULATE_FUNCTION_POINTER_CASTS flag, yielding large benefits in speed, stack usage, and code size. #2019

List of contributors#

Alexey Ignatiev, Alex Hall, Bart Broere, Cyrille Bogaert, etienne, Grimmer, Grimmer Kang, Gyeongjae Choi, Hao Zhang, Hood Chatham, Ian Clester, Jan Max Meyer, LeoPsidom, Liumeo, Michael Christensen, Owen Ou, Roman Yurchak, Seungmin Kim, Sylvain, Thorsten Beier, Wei Ouyang, Will Lachance

Version 0.18.1#

September 16, 2021

Console#

  • Fix Ctrl+C handling in console now works correctly with multiline input. New behavior more closely approximates the behavior of the native Python console. #1790

  • Fix Fix the repr of Python objects (including lists and dicts) in console #1780

  • Fix The “long output truncated” message now appears on a separate line as intended. #1814

  • Fix The streams that are used to redirect stdin and stdout in the console now define isatty to return True. This fixes pytest. #1822

Python package#

  • Fix Avoid circular references when runsource raises SyntaxError #1758

JavaScript package#

  • Fix The pyodide.setInterruptBuffer command is now publicly exposed again, as it was in v0.17.0. #1797

Python / JavaScript type conversions#

  • Fix Conversion of very large strings from JavaScript to Python works again. #1806

  • Fix Fixed a use after free bug in the error handling code. #1816

Packages#

  • Fix pillow now correctly encodes/decodes RGB JPEG image format. #1818

Micellaneous#

  • Fix Patched emscripten to make the system calls to duplicate file descriptors closer to posix-compliant. In particular, this fixes the use of dup on pipes and temporary files, as needed by pytest. #1823

Version 0.18.0#

August 3rd, 2021

General#

  • Update Pyodide now runs Python 3.9.5. #1637

  • Enhancement Pyodide can experimentally be used in Node.js #1689

  • Enhancement Pyodide now directly exposes the Emscripten filesystem API, allowing for direct manipulation of the in-memory filesystem #1692

  • Enhancement Pyodide’s support of emscripten file systems is expanded from the default MEMFS to include IDBFS, NODEFS, PROXYFS, and WORKERFS, allowing for custom persistence strategies depending on execution environment #1596

  • API Change The packages.json schema for Pyodide was redesigned for better compatibility with conda. #1700

  • API Change run_docker no longer binds any port to the docker image by default. #1750

Standard library#

  • API Change The following standard library modules are now available as standalone packages

    • distlib

    They are loaded by default in loadPyodide, however this behavior can be disabled with the fullStdLib parameter set to false. All optional stdlib modules can then be loaded as needed with pyodide.loadPackage. #1543

  • Enhancement The standard library module audioop is now included, making the wave, sndhdr, aifc, and sunau modules usable. #1623

  • Enhancement Added support for ctypes. #1656

JavaScript package#

  • Enhancement The Pyodide JavaScript package is released to npm under npmjs.com/package/pyodide #1762

  • API Change loadPyodide no longer automatically stores the API into a global variable called pyodide. To get old behavior, say globalThis.pyodide = await loadPyodide({...}). #1597

  • Enhancement loadPyodide now accepts callback functions for stdin, stdout and stderr #1728

  • Enhancement Pyodide now ships with first party typescript types for the entire JavaScript API (though no typings are available for PyProxy fields). #1601

  • Enhancement It is now possible to import Comlink objects into Pyodide after using pyodide.registerComlink #1642

  • Enhancement If a Python error occurs in a reentrant runPython call, the error will be propagated into the outer runPython context as the original error type. This is particularly important if the error is a KeyboardInterrupt. #1447

Python package#

  • Enhancement Added a new pyodide.code.CodeRunner API for finer control than eval_code and eval_code_async. Designed with the needs of REPL implementations in mind. #1563

  • Enhancement Added pyodide.console.Console class closely based on the Python standard library code.InteractiveConsole but with support for top level await and stream redirection. Also added the subclass pyodide.console.PyodideConsole which automatically uses pyodide.loadPackagesFromImports on the code before running it. #1125, #1155, #1635

  • Fix pyodide.code.eval_code_async no longer automatically awaits a returned coroutine or attempts to await a returned generator object (which triggered an error). #1563

Python / JavaScript type conversions#

  • API Change pyodide.runPythonAsync no longer automatically calls pyodide.loadPackagesFromImports. #1538.

  • Enhancement Added the PyProxy.callKwargs method to allow using Python functions with keyword arguments from JavaScript. #1539

  • Enhancement Added the PyProxy.copy method. #1549 #1630

  • API Change Updated the method resolution order on PyProxy. Performing a lookup on a PyProxy will prefer to pick a method from the PyProxy api, if no such method is found, it will use getattr on the proxied object. Prefixing a name with $ forces getattr. For instance, PyProxy.destroy now always refers to the method that destroys the proxy, whereas PyProxy.$destroy refers to an attribute or method called destroy on the proxied object. #1604

  • API Change It is now possible to use Symbol keys with PyProxies. These Symbol keys put markers on the PyProxy that can be used by external code. They will not currently be copied by PyProxy.copy. #1696

  • Enhancement Memory management of PyProxy fields has been changed so that fields looked up on a PyProxy are “borrowed” and have their lifetime attached to the base PyProxy. This is intended to allow for more idiomatic usage. (See #1617.) #1636

  • API Change The depth argument to toJs is now passed as an option, so toJs(n) in v0.17 changed to toJs({depth : n}). Similarly, pyodide.toPy now takes depth as a named argument. Also to_js and to_py only take depth as a keyword argument. #1721

  • API Change PyProxy.toJs and pyodide.ffi.to_js now take an option pyproxies, if a JavaScript Array is passed for this, then any proxies created during conversion will be placed into this array. This allows easy cleanup later. The create_pyproxies option can be used to disable creation of pyproxies during conversion (instead a ConversionError is raised). #1726

  • API Change toJs and to_js now take an option dict_converter which will be called on a JavaScript iterable of two-element Arrays as the final step of converting dictionaries. For instance, pass Object.fromEntries to convert to an object or Array.from to convert to an array of pairs. #1742

pyodide-build#

  • API Change pyodide-build is now an installable Python package, with an identically named CLI entrypoint that replaces bin/pyodide which is removed #1566

micropip#

  • Fix micropip now correctly handles packages that have mixed case names. (See #1614). #1615

  • Enhancement micropip now resolves dependencies correctly for old versions of packages (it used to always use the dependencies from the most recent version, see #1619 and #1745). micropip also will resolve dependencies for wheels loaded from custom urls. #1753

Packages#

  • Enhancement matplotlib now comes with a new renderer based on the html5 canvas element. #1579 It is optional and the current default backend is still the agg backend compiled to wasm.

  • Enhancement Updated a number of packages included in Pyodide.

List of contributors#

Albertas Gimbutas, Andreas Klostermann, Arfy Slowy, daoxian, Devin Neal, fuyutarow, Grimmer, Guido Zuidhof, Gyeongjae Choi, Hood Chatham, Ian Clester, Itay Dafna, Jeremy Tuloup, jmsmdy, LinasNas, Madhur Tandon, Michael Christensen, Nicholas Bollweg, Ondřej Staněk, Paul m. p. P, Piet Brömmel, Roman Yurchak, stefnotch, Syrus Akbary, Teon L Brooks, Waldir

Version 0.17.0#

April 21, 2021

See the Version 0.17.0 Release Notes for more information.

Improvements to package loading and dynamic linking#

  • Enhancement Uses the emscripten preload plugin system to preload .so files in packages

  • Enhancement Support for shared library packages. This is used for CLAPACK which makes scipy a lot smaller. #1236

  • Fix Pyodide and included packages can now be used with Safari v14+. Safari v13 has also been observed to work on some (but not all) devices.

Python / JS type conversions#

  • Feature A JsProxy of a JavaScript Promise or other awaitable object is now a Python awaitable. #880

  • API Change Instead of automatically converting Python lists and dicts into JavaScript, they are now wrapped in PyProxy. Added a new PyProxy.toJs API to request the conversion behavior that used to be implicit. #1167

  • API Change Added JsProxy.to_py API to convert a JavaScript object to Python. #1244

  • Feature Flexible jsimports: it now possible to add custom Python “packages” backed by JavaScript code, like the js package. The js package is now implemented using this system. #1146

  • Feature A PyProxy of a Python coroutine or awaitable is now an awaitable JavaScript object. Awaiting a coroutine will schedule it to run on the Python event loop using asyncio.ensure_future. #1170

  • Enhancement Made PyProxy of an iterable Python object an iterable Js object: defined the [Symbol.iterator] method, can be used like for(let x of proxy). Made a PyProxy of a Python iterator an iterator: proxy.next() is translated to next(it). Made a PyProxy of a Python generator into a JavaScript generator: proxy.next(val) is translated to gen.send(val). #1180

  • API Change Updated PyProxy so that if the wrapped Python object supports __getitem__ access, then the wrapper has get, set, has, and delete methods which do obj[key], obj[key] = val, key in obj and del obj[key] respectively. #1175

  • API Change The pyodide.pyimport function is deprecated in favor of using pyodide.globals.get('key'). #1367

  • API Change Added PyProxy.getBuffer API to allow direct access to Python buffers as JavaScript TypedArrays. #1215

  • API Change The innermost level of a buffer converted to JavaScript used to be a TypedArray if the buffer was contiguous and otherwise an Array. Now the innermost level will be a TypedArray unless the buffer format code is a ‘?’ in which case it will be an Array of booleans, or if the format code is a “s” in which case the innermost level will be converted to a string. #1376

  • Enhancement JavaScript BigInts are converted into Python int and Python ints larger than 2^53 are converted into BigInt. #1407

  • API Change Added pyodide.isPyProxy to test if an object is a PyProxy. #1456

  • Enhancement PyProxy and PyBuffer objects are now garbage collected if the browser supports FinalizationRegistry. #1306

  • Enhancement Automatic conversion of JavaScript functions to CPython calling conventions. #1051, #1080

  • Enhancement Automatic detection of fatal errors. In this case Pyodide will produce both a JavaScript and a Python stack trace with explicit instruction to open a bug report. pr{1151}, pr{1390}, pr{1478}.

  • Enhancement Systematic memory leak detection in the test suite and a large number of fixed to memory leaks. pr{1340}

  • Fix getattr and dir on JsProxy now report consistent results and include all names defined on the Python dictionary backing JsProxy. #1017

  • Fix JsProxy.__bool__ now produces more consistent results: both bool(window) and bool(zero-arg-callback) were False but now are True. Conversely, bool(empty_js_set) and bool(empty_js_map) were True but now are False. #1061

  • Fix When calling a JavaScript function from Python without keyword arguments, Pyodide no longer passes a PyProxy-wrapped NULL pointer as the last argument. #1033

  • Fix JsBoundMethod is now a subclass of JsProxy, which fixes nested attribute access and various other strange bugs. #1124

  • Fix JavaScript functions imported like from js import fetch no longer trigger “invalid invocation” errors (issue #461) and js.fetch("some_url") also works now (issue #768). #1126

  • Fix JavaScript bound method calls now work correctly with keyword arguments. #1138

  • Fix JavaScript constructor calls now work correctly with keyword arguments. #1433

pyodide-py package#

  • Feature Added a Python event loop to support asyncio by scheduling coroutines to run as jobs on the browser event loop. This event loop is available by default and automatically enabled by any relevant asyncio API, so for instance asyncio.ensure_future works without any configuration. #1158

  • API Change Removed as_nested_list API in favor of JsProxy.to_py. #1345

pyodide-js#

  • API Change Removed iodide-specific code in pyodide.js. This breaks compatibility with iodide. #878, #981

  • API Change Removed the pyodide.autocomplete API, use Jedi directly instead. #1066

  • API Change Removed pyodide.repr API. #1067

  • Fix If messageCallback and errorCallback are supplied to pyodide.loadPackage, pyodide.runPythonAsync and pyodide.loadPackagesFromImport, then the messages are no longer automatically logged to the console.

  • Feature runPythonAsync now runs the code with eval_code_async. In particular, it is possible to use top-level await inside of runPythonAsync.

  • eval_code now accepts separate globals and locals parameters. #1083

  • Added the pyodide.setInterruptBuffer API. This can be used to set a SharedArrayBuffer to be the keyboard interrupt buffer. If Pyodide is running on a webworker, the main thread can signal to the webworker that it should raise a KeyboardInterrupt by writing to the interrupt buffer. #1148 and #1173

  • Changed the loading method: added an async function loadPyodide to load Pyodide to use instead of languagePluginURL and languagePluginLoader. The change is currently backwards compatible, but the old approach is deprecated. #1363

  • runPythonAsync now accepts globals parameter. #1914

micropip#

  • Feature micropip now supports installing wheels from relative URLs. #872

  • API Change micropip.install now returns a Python Future instead of a JavaScript Promise. #1324

  • Fix micropip.install now interacts correctly with pyodide.loadPackage(). #1457

  • Fix micropip.install now handles version constraints correctly even if there is a version of the package available from the Pyodide indexURL.

Build system#

  • Enhancement Updated to latest emscripten 2.0.13 with the upstream LLVM backend #1102

  • API Change Use upstream file_packager.py, and stop checking package abi versions. The PYODIDE_PACKAGE_ABI environment variable is no longer used, but is still set as some packages use it to detect whether it is being built for Pyodide. This usage is deprecated, and a new environment variable PYODIDE is introduced for this purpose.

    As part of the change, Module.checkABI is no longer present. #991

  • uglifyjs and lessc no longer need to be installed in the system during build #878.

  • Enhancement Reduce the size of the core Pyodide package #987.

  • Enhancement Optionally to disable docker port binding #1423.

  • Enhancement Run arbitrary command in docker #1424

  • Docker images for Pyodide are now accessible at pyodide/pyodide-env and pyodide/pyodide.

  • Enhancement Option to run docker in non-interactive mode #1641

REPL#

  • Fix In console.html: sync behavior, full stdout/stderr support, clean namespace, bigger font, correct result representation, clean traceback #1125 and #1141

  • Fix Switched from ̀Jedi to rlcompleter for completion in pyodide.console.InteractiveConsole and so in console.html. This fixes some completion issues (see #821 and #1160)

  • Enhancement Support top-level await in the console #1459

Packages#

  • six, jedi and parso are no longer vendored in the main Pyodide package, and need to be loaded explicitly #1010, #987.

  • Updated packages #1021, #1338, #1460.

  • Added Plotly version 4.14.3 and retrying dependency #1419

List of contributors#

(in alphabetic order)

Aditya Shankar, casatir, Dexter Chua, dmondev, Frederik Braun, Hood Chatham, Jan Max Meyer, Jeremy Tuloup, joemarshall, leafjolt, Michael Greminger, Mireille Raad, Ondřej Staněk, Paul m. p. P, rdb, Roman Yurchak, Rudolfs

Version 0.16.1#

December 25, 2020

Note: due to a CI deployment issue the 0.16.0 release was skipped and replaced by 0.16.1 with identical contents.

  • Pyodide files are distributed by JsDelivr, https://cdn.jsdelivr.net/pyodide/v0.16.1/full/pyodide.js The previous CDN pyodide-cdn2.iodide.io still works and there are no plans for deprecating it. However please use JsDelivr as a more sustainable solution, including for earlier Pyodide versions.

Python and the standard library#

  • Pyodide includes CPython 3.8.2 #712

  • ENH Patches for the threading module were removed in all packages. Importing the module, and a subset of functionality (e.g. locks) works, while starting a new thread will produce an exception, as expected. #796. See #237 for the current status of the threading support.

  • ENH The multiprocessing module is now included, and will not fail at import, thus avoiding the necessity to patch included packages. Starting a new process will produce an exception due to the limitation of the WebAssembly VM with the following message: Resource temporarily unavailable #796.

Python / JS type conversions#

  • FIX Only call Py_INCREF() once when proxied by PyProxy #708

  • JavaScript exceptions can now be raised and caught in Python. They are wrapped in pyodide.JsException. #891

pyodide-py package and micropip#

  • The pyodide.py file was transformed to a pyodide-py package. The imports remain the same so this change is transparent to the users #909.

  • FIX Get last version from PyPI when installing a module via micropip #846.

  • Suppress REPL results returned by pyodide.eval_code by adding a semicolon #876.

  • Enable monkey patching of eval_code and find_imports to customize behavior of runPython and runPythonAsync #941.

Build system#

  • Updated docker image to Debian buster, resulting in smaller images. #815

  • Pre-built docker images are now available as iodide-project/pyodide #787

  • Host Python is no longer compiled, reducing compilation time. This also implies that Python 3.8 is now required to build Pyodide. It can for instance be installed with conda. #830

  • FIX Infer package tarball directory from source URL #687

  • Updated to emscripten 1.38.44 and binaryen v86 (see related commits)

  • Updated default --ldflags argument to pyodide_build scripts to equal what Pyodide actually uses. #817

  • Replace C lz4 implementation with the (upstream) JavaScript implementation. #851

  • Pyodide deployment URL can now be specified with the PYODIDE_BASE_URL environment variable during build. The pyodide_dev.js is no longer distributed. To get an equivalent behavior with pyodide.js, set

    window.languagePluginUrl = "./";

    before loading it. #855

  • Build runtime C libraries (e.g. libxml) via package build system with correct dependency resolution #927

  • Pyodide can now be built in a conda virtual environment #835

Other improvements#

  • Modify MEMFS timestamp handling to support better caching. This in particular allows to import newly created Python modules without invalidating import caches #893

Packages#

  • New packages: freesasa, lxml, python-sat, traits, astropy, pillow, scikit-image, imageio, numcodecs, msgpack, asciitree, zarr

    Note that due to the large size and the experimental state of the scipy package, packages that depend on scipy (including scikit-image, scikit-learn) will take longer to load, use a lot of memory and may experience failures.

  • Updated packages: numpy 1.15.4, pandas 1.0.5, matplotlib 3.3.3 among others.

  • New package pyodide-interrupt, useful for handling interrupts in Pyodide (see project description for details).

Backward incompatible changes#

  • Dropped support for loading .wasm files with incorrect MIME type, following #851

List of contributors#

abolger, Aditya Shankar, Akshay Philar, Alexey Ignatiev, Aray Karjauv, casatir, chigozienri, Christian glacet, Dexter Chua, Frithjof, Hood Chatham, Jan Max Meyer, Jay Harris, jcaesar, Joseph D. Long, Matthew Turk, Michael Greminger, Michael Panchenko, mojighahar, Nicolas Ollinger, Ram Rachum, Roman Yurchak, Sergio, Seungmin Kim, Shyam Saladi, smkm, Wei Ouyang

Version 0.15.0#

May 19, 2020

  • Upgrades Pyodide to CPython 3.7.4.

  • micropip no longer uses a CORS proxy to install pure Python packages from PyPI. Packages are now installed from PyPI directly.

  • micropip can now be used from web workers.

  • Adds support for installing pure Python wheels from arbitrary URLs with micropip.

  • The CDN URL for Pyodide changed to https://pyodide-cdn2.iodide.io/v0.15.0/full/pyodide.js It now supports versioning and should provide faster downloads. The latest release can be accessed via https://pyodide-cdn2.iodide.io/latest/full/

  • Adds messageCallback and errorCallback to pyodide.loadPackage.

  • Reduces the initial memory footprint (TOTAL_MEMORY) from 1 GiB to 5 MiB. More memory will be allocated as needed.

  • When building from source, only a subset of packages can be built by setting the PYODIDE_PACKAGES environment variable. See partial builds documentation for more details.

  • New packages: future, autograd

Version 0.14.3#

Dec 11, 2019

  • Convert JavaScript numbers containing integers, e.g. 3.0, to a real Python long (e.g. 3).

  • Adds __bool__ method to for JsProxy objects.

  • Adds a JavaScript-side auto completion function for Iodide that uses jedi.

  • New packages: nltk, jeudi, statsmodels, regex, cytoolz, xlrd, uncertainties

Version 0.14.0#

Aug 14, 2019

  • The built-in sqlite and bz2 modules of Python are now enabled.

  • Adds support for auto-completion based on jedi when used in iodide

Version 0.13.0#

May 31, 2019

  • Tagged versions of Pyodide are now deployed to Netlify.

Version 0.12.0#

May 3, 2019

User improvements:

  • Packages with pure Python wheels can now be loaded directly from PyPI. See micropip for more information.

  • Thanks to PEP 562, you can now import js from Python and use it to access anything in the global JavaScript namespace.

  • Passing a Python object to JavaScript always creates the same object in JavaScript. This makes APIs like removeEventListener usable.

  • Calling dir() in Python on a JavaScript proxy now works.

  • Passing an ArrayBuffer from JavaScript to Python now correctly creates a memoryview object.

  • Pyodide now works on Safari.

Version 0.11.0#

Apr 12, 2019

User improvements:

  • Support for built-in modules:

    • sqlite, crypt

  • New packages: mne

Developer improvements:

  • The mkpkg command will now select an appropriate archive to use, rather than just using the first.

  • The included version of emscripten has been upgraded to 1.38.30 (plus a bugfix).

  • New packages: jinja2, MarkupSafe

Version 0.10.0#

Mar 21, 2019

User improvements:

  • New packages: html5lib, pygments, beautifulsoup4, soupsieve, docutils, bleach, mne

Developer improvements:

  • console.html provides a simple text-only interactive console to test local changes to Pyodide. The existing notebooks based on legacy versions of Iodide have been removed.

  • The run_docker script can now be configured with environment variables.