Change Log#
Version 0.26.4#
November 15, 2024
Version 0.26.3#
October 19, 2024
Performance Attribute lookup on a
JsProxyis now about 40% faster. #4961Performance Method calls on a
JsProxyare now much faster. If the method has no arguments and no return value, it is about 80% faster. The speedup for methods with arguments is less drastic but still quite a lot. #4963Enhancement Updated stack switching support to handle new JSPI. #4982
Fix
pyimport("a.b")won’t fail whenais removed bydel sys.modules["a"]#4993Fix It now works to convert a 0d Python buffer to JavaScript. #5092
Fix It now works to convert buffers of 64 bit signed or unsigned integers to JavaScript. #5092
Version 0.26.2#
July 26, 2024
Fix Pyodide now loads correctly when
defineanddefine.amdare defined in the global scope. #4866Fix Fixed keyboard input handling in SDL-based packages. #4865
Fix Don’t leak the values in a dictionary when applying
to_jsto it. #4853Fix Loading of dynamic libraries now works slightly better in the cli runner. Resolved #3865. #4871
Fix Restored the pre-0.26.0 behavior of calling
exit()or raisingSystemExit. In 0.26.0 and 0.26.1,exit()shuts down the Python interpreter. In all other versions of Pyodide it does not. #4867Fix Fixed a weird regression occurring in difficult to describe circumstances introduced by #4837. See #4861. #4861
Fix Recursive fortran functions now work correctly in scipy #4818. #4822
Enhancement Allow setting
dont_inheritandoptimizeforcompileinCodeRunnerandConsole. #4897Fix Fixed a bug that caused
Console’sformatted_tracebackbeing truncated unexpectedly whenfilenameis specified. #4905Fix Locked
PyodideConsole.runcodeto blockloadPackagesFromImports. #4905Enhancement Added
checkAPIVersionoption toloadPyodideto allow bootstrapping pyodide with a different version. #4907Enhancement Added
can_run_syncto test whether or notrun_syncshould work. #4913Fix Fixed a bug with the JSPI that made it interact incorrectly with JavaScript code that iterates a
PyProxy. #4919Fix Pyodide now loads correctly when
defineanddefine.amdare defined in the global scope. #4866
Packages#
Added
duckdb1.0.0 #4684
Version 0.26.1#
June 7, 2024
Package build system#
Fix Fix
pyodide configcommand printing extra output. #4814Enhancement Added implementation to read build settings from
pyproject.toml. #4831Fix In the Pyodide virtual environment, pip sees
platform.system()as “Emscripten” and not as “emscripten”. #4812Fix Resolution of JavaScript symbols in dynamic libraries doesn’t fail anymore in the command line runner. #4836
Fix Pyodide virtual environments now work correctly in Fedora and other platforms with platlibdir not equal to “lib”. #4844
Runtime / FFI#
Enhancement Added the
enableRunUntilCompleteoption toloadPyodidewhich makesrun_until_completeblock using stack switching, or crash if stack switching is disabled. #4817Fix Resolved an issue where string keys in
PyProxyJsonAdaptorwere unexpectedly cast to numbers. #4825Fix When a
Futureconnected to aPromiseis cancelled, don’t raiseInvalidStateError. #4837
Packages#
New Packages:
pytest-asyncio#4819
Version 0.26.0#
May 27, 2024
General#
Update The wheel tag for Pyodide wheels has changed to pyodide_2024_0_wasm32. #4777, #4780
Enhancement ABI Break: Updated Emscripten to version 3.1.58 #4399 #4715
BREAKING CHANGE Pyodide will not fallback to
node-fetchanymore whenfetchis not available in the Node.js < 18 environment. #4417Enhancement Improved support for stack switching. #4532, #4547, #4615, #4639
BREAKING CHANGE The experimental
callSyncifyingmethod was renamed tocallPromising. #4608Fix
dupnow works correctly in the Node filesystem. #4554Enhancement
asyncio.sleep(0)now runs the next task a lot faster. #4590
JavaScript APIs#
Enhancement
pyodide.loadPackagenow checks if the cache directory exists and callsmkdironly when it doesn’t to avoid an error on read-only file systems in Node.js environment. #4738Fix
pyodide.mountNativeFSwill no longer silently overwrite an existing nonempty directory. Also it throws much clearer error messages when it fails. #4559Enhancement Added a new API
pyodide.mountNodeFSwhich mounts a host directory into the Pyodide file system when running in node. #4561Enhancement Updated
pyimportto supportpyimport("module.attribute"). #4395
FFI#
Enhancement
str(jsproxy)has been adjusted to not raise an error ifjsproxy.toStringis undefined. Instead, it will useObject.prototype.toStringin this case. Ifjsproxy.toStringis defined and throws or is not defined butjsproxy[Symbol.toStringTag]is defined and throws, thenstrwill still raise. #4574Enhancement Fixed a memory leak when iterating over a PyProxy. #4546
Enhancement When a dictionary is converted to JavaScript with
toJsthe result is now aLiteralMap. String keys are accessible via direct property access unless they match a function on theMapprototype. #4576Fix
toJsnow works as expected on subclasses ofdict. #4637Enhancement Added
PyProxy.asJsJsonmethod to adapt between Python JSON (lists and dicts) and JavaScript JSON (Arrays and Objects). #4666Enhancement Added a new
callRelaxedto PyProxies of callables that discards extra arguments rather than raising aTypeError``. {pr}4392`Enhancement A new
callWithOptionsmethod was added to PyProxies of callables. #4608
Build#
Fix pyodide-build now use response file when passing list of exported symbols to
emcc. This fixes “Argument list too long” error. #4717`Fix Pass through
-E(command mode) arguments in CMake wrapper #4705.Fix Fix exception handling in dynamic linking of int64 functions #4698.
BREAKING CHANGE
pyodide-buildentrypoint is removed in favor ofpyodide. This entrypoint was deprecated since 0.22.0. #4368BREAKING CHANGE The
--no-depsoption topyodide build-recipeshas been replaced with a separate subcommandpyodide build-recipes-no-deps. #4443Enhancement The
build/postscript now runs under the directory where the built wheel is unpacked. #4481
Packages#
New Packages:
cysignals,ppl,pplpy#4407,flint,python-flint#4410,memory_allocator#4393,primesieve,primecount,primecountpy#4477,pyxirr#4513,ipython,asttokens,executing,prompt_toolkit,pure_eval,stack_data,traitlets,wcwidth#4452,altair#4580,cvxpy#4587,clarabel#4587,matplotlib-inline#4626,pygame-ce#4602,libcst#4665,mmh3,pyiceberg#4648,lakers-python#4763,crc32c#4789,zstandard#4792Upgraded
contourpyto 1.2.1 #4680Upgraded
sourmashto 4.8.8 #4683
Version 0.25.1#
March 31, 2024
Version 0.25.0#
January 18, 2024
General#
Enhancement ABI Break: Updated Emscripten to version 3.1.46 #4359
BREAKING CHANGE Node.js < 18 is no longer officially supported. Older versions of Node.js might still work, but they are not tested or guaranteed to work. #4269
Enhancement Added experimental support for stack switching. #3957, #3964, #3987, #3990, #3210
JavaScript API#
Fix
pyodide.setStdinnow does not consider an empty string as EOF. #4327BREAKING CHANGE
loadPyodidedoes not accepthomediroption anymore, useenv: {HOME: "/the/home/directory"}instead. This have been deprecated since Pyodide 0.24. #4342Enhancement
pyodide.loadPackagenow returns an object with metadata about the loaded packages. #4306Fix Fixed default indexURL calculation in Node.js environment. #4288
Python API#
Enhancement The
pyodide-pypackage onpypinow includespy.typedmarkers so mypy will use the types. #4321Fix Fixed a bug that micropip would fail to install packages from pyodide-lock.json if the package’s name differs from its normalized name. #4319
Enhancement Added a no-op
WebLoop.closemethod so that attempts to close the event loop will not raise an exception. #4329
Python / JavaScript Foreign Function Interface#
Fix
jsarray.popnow works correctly. It previously returned the wrong value and leaked memory. #4236BREAKING CHANGE
PyProxy.toStringnow callsstrinstead ofrepr. For now you can opt into the old behavior by passingpyproxyToStringRepr: truetoloadPyodide, but this may be removed in the future. #4247Fix when accessing a
JsProxyattribute invokes a getter and the getter throws an error, that error is propagated instead of being turned into anAttributeError. #4254Fix
import type { PyProxy } from "pyodide/ffi"now works with theNodeNexttypescript target. #4256Fix Fixed a bug that occurs when using
toJswith bothdictConverteranddefaultConverterarguments. #4263Enhancement Added
JsArray.removeandJsArray.insertmethods. #4326BREAKING CHANGE Type exports of
PyProxysubtypes have been moved frompyodidetopyodide/ffiand many of them have changed names. #4342BREAKING CHANGE The methods for checking
PyProxycapabilities (e.g.,supportsHas,isCallable) are now removed. Use e.g.,instanceof pyodide.ffi.PyCallableinstead. #4342
Pyodide CLI#
Enhancement
pyodide configcommand now show additional config variables:rustflags,cmake_toolchain_file,pyo3_config_file,rust_toolchain,cflagscxxflags,ldflags,meson_cross_file. These variables can be used in out-of-tree build to set the same variables as in-tree build. #4241Enhancement
pyodide buildcommand now accepts--config-setting(-C) option to pass flags to the build backend, just likepython -m buildcommand. #4308
Load time & size optimizations#
Performance Do not use
importlib.metadatawhen identifying installed packages, which reduces the time to load Pyodide. #4147
Build system#
Packages#
Version 0.24.1#
September 25, 2023
Fix Fixed
LONG_BIT definition appears wrong for platformerror happened in out-of-tree build. #4136Fix Fixed an Emscripten bug that broke some matplotlib functionality. #4163
Fix
pyodide.checkInterruptworks when there is no interrupt buffer and the gil is not held. #4164
Packages#
Version 0.24.0#
September 13, 2023
General#
JavaScript API#
Performance Added a
packagesoptional argument toloadPyodide. Passing packages here saves time by downloading them during the Pyodide bootstrap. #4100Enhancement
runPythonandrunPythonAsyncnow accept afilenameoptional argument which is passed as thefilenameargument toeval_code(resp.eval_code_async). Also, if afilenameis passed toeval_codewhich does not start with<and end with>, Pyodide now uses thelinecachemodule to ensure that source lines can appear in tracebacks. #3993Performance For performance reasons, don’t render extra information in PyProxy destroyed message by default. By using
pyodide.setDebug(true), you can opt into worse performance and better error messages. #4027Enhancement It is now possible to pass environment variables to
loadPyodidevia theenvargument.homediris deprecated in favor of{env: {HOME: whatever_directory}}. #3870Enhancement The
setStdin,setStdoutandsetStderrAPIs have been improved with extra control and better performance. #4035
Python API#
Enhancement Added
headersproperty topyodide.http.FetchResponse. #2078Enhancement Added
FetchResponse.text()as a synonym toFetchResponse.string()for better compatibility with other requests APIs. #4052BREAKING CHANGE Changed the
FetchResponsebody getter methods to no longer throw anOSErrorexception for 400 and above response status codes. AddedFetchResponse.raise_for_statusto raise anOSErrorfor error status codes. #3986 #4053
Python / JavaScript Foreign Function Interface#
Performance Improved performance of PyProxy creation. #4096
Fix Fixed adding getters/setters to a
PyProxywithObject.definePropertyand improved compliance with JavaScript rules around Proxy traps. #4033Enhancement The promise methods
then,catchandfinally_are now present also onTasks as well asFutures. #3748Enhancement Added methods to a
PyProxyof alistto make these work as drop-in replacements for JavaScript Arrays. #3853Enhancement When a
JsProxyof an array is passed to Python builtin functions that use thePySequence_*APIs, it now works as expected. Alsojsarray * nrepeats the arrayntimes andjsarray + iterablereturns a new array with the result values from the iterable appended. #3904
Deployment#
API Change Changed the name of the default lockfile from
repodata.jsontopyodide-lock.json#3824
Build System#
Update The docker image now has node v20 instead of node v14. #3819
Enhancement Added
check_wasm_magic_numberfunction to validate.sofiles for WebAssembly (WASM) compatibility. #4018Enhancement In pyodide build, automatically skip building package dependencies that are already included in the pyodide distribution. #4058
Packages#
New packages: sourmash #3635, screed #3635, bitstring #3635, deprecation #3635, cachetools #3635, xyzservices #3786, simplejson #3801, protobuf #3813, peewee #3897, Cartopy #3909, pyshp #3909, netCDF4 #3910, igraph #3991, CoolProp #4028, contourpy #4102, awkward-cpp #4101, orjson #4036.
Upgraded numpy to 1.25.2 #4125
OpenBLAS has been added and scipy now uses OpenBLAS rather than CLAPACK #3331.
Pyodide CLI#
Misc#
Enhancement Add an example for
loadPyodideandpyodide.runPython {pr}4012, {pr}4011`
Version 0.23.4#
July 6, 2023
Enhancement The environment variable
PYODIDE_BUILD_EXPORTScan now be used instead of the--exportsargument topyodide buildto specify.sofile exports of packages. #3973Fix Pin
pydanticto<2. #3971Enhancement Allow customizing cache location for packages when running in Node #3967
Enhancement Re-enabled sparseqr, freesasa, lightgbm, opencv-python, and wordcloud #3783, #3970
Fix A
JSProxyof aDOMExceptionwill now inherit from exception so it can be raised in Python. #3868Fix The feature detection for
JSProxyhas been improved so that it should never fail even when handling strange or ill-behaved JavaScript proxy objects. #3740, #3750Fix A
PyProxyof a callable is now aninstanceof Function. (If you are trying to feature detect whether something is callable or not in JavaScript, the correct way is to usetypeof o === "function". But you may have dependencies that don’t do this correctly.) #3925Fix
from jsmodule import *now works. #3903
Version 0.23.3#
June 17, 2023
Fix
getattr(jsproxy, 'python_reserved_word')works as expected again (as well ashasattrandsetattr). This fixes a regression introduced in #3617. #3926Fix
pyodide buildnow replaces native.soslugs with Emscripten slugs. Usually.sos in the generated wheels are actually Emscripten.sos so this is good. If they are actually native.sos then there is a problem either way. #3903
Version 0.23.2#
May 2, 2023
Enhancement Changed the name of the
--output-directoryargument topyodide buildto--outdirto match pypa/build.--output-directoryis still accepted for backwards compatibility. #3811
Version 0.23.1#
April 13, 2023
Deployment#
Fix Export
python_stdlib.zipinpackage.json. #3723
CLI#
Enhancement
pyodide buildnow accepts an--output-directoryargument. #3746Fix Fix
pyodide py-compilenot to ignore the--compression-leveloption when applied on a single file. #3727Fix Fix an issue where the
pyodide venvcommand did not work correctly in pyodide-build version 0.23.0 because of missingpython_stdlib.zip. #3760Fix
python -m pipworks correctly in the Pyodide venv now. #3761Fix Executables installed in a Pyodide virtual environment now run in Pyodide not in the host Python. #3752
Build System#
Fix Fix
PYODIDE_ROOTto point the correct directory when running out-of-tree build. #3751
Version 0.23.0#
March 30, 2023
General#
Update Pyodide now runs Python 3.11.2 which officially supports WebAssembly as a PEP11 Tier 3 platform. #3252, #3614
Update We now build libpyodide.a so the Pyodide foreign function interface can be experimentally linked into other Emscripten builds of Python. #3335
Enhancement Updated Emscripten to version 3.1.32 #3471, #3517, #3599
JavaScript API#
BREAKING CHANGE Type exports of
PyProxysubtypes have been moved frompyodidetopyodide/ffiand many of them have changed names. The original exports are still available but they are deprecated. #3523BREAKING CHANGE The methods for checking
PyProxycapabilities (e.g.,supportsHas,isCallable) are now deprecated. Use e.g.,instanceof pyodide.ffi.PyCallableinstead. #3523Enhancement Added subclasses of
PyProxyfor each mixin. These can be used to check whether aPyProxysupports a given set of methods withinstanceofe.g.,x instanceof pyodide.ffi.PyDict. #3523Enhancement Added
stdLibURLparameter toloadPyodideallowing to customize the URL from which the Python standard library is loaded. #3670Enhancement Checking whether an object is an instance of a
PyProxynow only recognizes aPyProxygenerated from the same Python interpreter. This means that creating multiple interpreters and importing aPyProxyfrom one into another no longer causes a fatal error. #3545Enhancement
as_object_mapnow accepts a keyword argumenthereditary. If set toTrueand indexing the object returns a plain-old-object, then the return value will be automatically mapped inas_object_mapas well. #3638Enhancement A
JsProxyof a JavaScript error object can be directly thrown as Python exceptions. Previously Pyodide automatically wrapped them in aJsExceptionbut that is no longer needed – nowJsExceptioninherits from bothJsProxyandException. #3455Enhancement
runPythonandrunPythonAsyncnow accept alocalsargument. #3618Fix Calling
loadPyodiderepeatedly in Node no longer results inMaxListenersExceededWarning. Also, callingloadPyodidein Node v14 no longer changes unhandled rejections in promises. #3542Fix If the
localsargument toeval_codeoreval_code_asyncisNoneit now useslocals=globalsas the documentation says. #3580
Python standard library#
BREAKING CHANGE Unvendored
_pydecimalandpydoc_datafrom the standard library. Now these modules need to be loaded withpyodide.loadPackageormicropip.install, or auto-loaded via imports inpyodide.runPythonAsync#3525BREAKING CHANGE Test files of stdlib
ctypesandunittestare now moved totest/ctypesandtest/unittestrespectively. This change is adapted from CPython 3.12. #3507
Deployment#
BREAKING CHANGE Pyodide no longer uses Emscripten preload plugin, hence
pyodide.asm.datais removed, in favor ofpython_stdlib.zip. This change normally shouldn’t affect users, but if you were using this file in a bundler, you will need to remove it. #3584BREAKING CHANGE
pyodide_py.tarfile is removed. This change normally shouldn’t affect users, but if you were using this file in a bundler, you will need to remove it. #3621BREAKING CHANGE Python standard libraries are now vendored in a zipfile:
/lib/python{version}.zipin the in-browser MEMFS file system. If you need to access the standard library source code, you need to unpack the zip file. For example:import shutil; shutil.unpack_archive('/lib/python311.zip', '/lib/python3.11', 'zip)#3584Fix Improves the compression of wheel files with the JsDelivr CDN. For browsers that support the Brotli compression (most modern ones) this should result in a size reduction of 20-30%. Also most many
pyodideCLI sub-commands now support--compression-levelas an optional parameter. #3655BREAKING CHANGE Following libraries are now not linked to the Pyodide main module:
libgl,libal,libhtml5. This normally shouldn’t affect users, but if you are using these libraries in a package that are built out-of-tree, you will need to link them to the package manually. #3505
Python / JavaScript Foreign Function Interface#
Fix PyProxies of Async iterators are now async iterable JavaScript objects. The code:
for await (let x of async_iterator_pyproxy) { // ... }
would previously fail with
TypeError: async_iterator_pyproxy is not async iterable. (Python async iterables that were not also iterators were already async iterable, the problem was only with Python objects that are both async iterable and an async iterator.) #3708Enhancement A py-compiled build which has smaller and faster-to-load packages is now deployed under
https://cdn.jsdelivr.net/pyodide/v0.23.0/pyc/(also for future versions). The exceptions obtained with this builds will not include code snippets however. #3701BREAKING CHANGE Removed support for calling functions from the root of
pyodidepackage directly. This has been deprecated since v0.21.0. Now all functions are only available under submodules. #3677BREAKING CHANGE Removed support for passing the “message” argument to
PyProxy.destroyin a positional argument. This has been deprecated since v0.22.0. #3677Enhancement Python does not allow reserved words to be used as attributes. For instance,
Array.fromis aSyntaxError. (JavaScript has a more robust parser which can handle this.) To handle this, if an attribute to aJsProxyconsists of a Python reserved word followed by one or more underscores, we remove a single underscore from the end of the attribute. For instance,Array.from_would accessfromon the underlying JavaScript object, whereaso.from__accesses thefrom_attribute. #3617
Build System#
BREAKING CHANGE When building meta-packages (
coreandmin-scipy-stack), you must prefixtag:to the meta-package name. For example, to build thecoremeta-package, you must runpyodide build-recipes tag:core, orPYODIDE_PACKAGES="tag:core" make. #3444Enhancement Add
--build-dependenciestopyodide buildcommand to fetch and build dependencies of a package being built. Also adds--skip-dependencyto ignore selected dependencies. #3310Enhancement Added
pyodide buildsupport for building a list of packages from a requirements.txt file withpyodide build -r <requirements.txt>. Also can output a list of chosen dependencies in the same format when building a package and dependencies using the--output-lockfile <lockfile.txt>argument. This enables repeatable builds of packages. #3469Enhancement Added
package/tagkey to themeta.yamlspec to group packages. #3444Enhancement
pyodide build-recipesnow autodetects the number of CPU cores in the system and uses them for parallel builds. #3559 #3598Fix Fixed pip install error when installing cross build environment. #3562
Enhancement Response files are now correctly handled when calculating exported symbols. #3645
Fix Fix occasional build failure when building rust packages. #3607
Enhancement Improved logging in
pyodide-buildwith rich. #3442Enhancement
pyodide build-recipesnow accepts--no-depsparameter, which skips building dependencies of the package. This replacespyodide-build buildpkg. #3520Enhancement
pyodide build-recipesnow works out-of-tree.
Pyodide CLI#
BREAKING CHANGE Removed deprecated CLI entrypoints
pyodide-build buildallwhich is replaced bypyodide build-recipes, andpyodide-build mkpkgwhich is replaced bypyodide skeleton pypi#3668.Feature Added
pyodide py-compileCLI command that py compiles a wheel or a zip file, converting .py files to .pyc files. It can also be applied to a folder with wheels / zip files. If the input folder contains therepodata.jsonthe paths and checksums it contains will also be updated #3253 #3700Feature Added
pyodide create-zipfileCLI command that creates a zip file of a directory. This command is hidden by default since it is not intended for use by end users. #3411 #3463
REPL#
Fix Non-breaking space characters are now automatically converted to regular spaces in pyodide REPL. #3558
Enhancement Allow changing the build type used in the REPL by passing the
buildargument to the REPL URL. For instance,https://pyodide.org/en/latest/console.html?build=debugwill load debug dev build. #3671
Packages#
List of Contributors#
Alexey Ignatiev, Andrea Giammarchi, Arpit, Christian Clauss, Deepak Cherian, Eli Lamb, Feodor Fitsner, Gyeongjae Choi, Hood Chatham, Jeff Glass, Jo Bovy, Joe Marshall, josephrocca, Loïc Estève, martinRenou, messense, Nicholas Bollweg, Roman Yurchak, TheOnlyWayUp, Victor Blomqvist, Ye Joo Park
Version 0.22.1#
January 25, 2023
BREAKING CHANGE
setStdinnow accepts an extraautoEOFparameter. Iftrue, it will insert an EOF automatically after each string or buffer. Defaults totrue. This also affects the behavior of thestdinargument toloadPyodide. #3488Fix
from pyodide.ffi import *doesn’t raise anImportErroranymore. #3484Enhancement Pyodide displays a better message when someone calls posix
exitoros._exit. #3496
Package Loading#
Fix Fix incorrect error message when loading a package include in Pyodide fails. #3435
Build system#
Fix Emscripten is no longer required to create a Pyodide virtual environment. #3485
Fix Fixed a bug where
pyodide buildwould fail on package that use CMake, when run multiple times. #3445Fix pyodide build: Don’t pass the directory to the build backend args, only pass the arguments. #3490
Fix
pyodide configwon’t print extra messages anymore. #3483Fix Pass the same environment variables for out of tree builds as for in tree builds. #3495
Version 0.22.0#
January 3, 2023
See the release notes for a summary.
Deployment and testing#
BREAKING CHANGE
pyodide-cdn2.iodide.iois not available anymore. Please usehttps://cdn.jsdelivr.net/pyodideinstead. #3150.BREAKING CHANGE We don’t publish pre-built Pyodide docker images anymore. Note that
./run_docker --pre-builtwas not working for a while and it was actually equivalent to./run_docker. If you need to build a single Python wheel out of tree, you can use thepyodide buildcommand instead. See our blog post for more information. #3342.Enhancement The releases are now called
pyodide-{version}.tar.gzrather thanpyodide-build-{version}.tar.gz#2996Enhancement Added a new release file called
pyodide-core-{version}.tar.gzintended for use in Node. It contains the files needed to start Pyodide and no additional packages. #2999Enhancement The full test suite is now run in Safari #2578, #3095.
Enhancement Added Gitpod configuration to the repository. #3201
Foreign function interface#
JsProxy / JavaScript from Python#
Enhancement Implemented
reverse,__reversed__,count,index,append, andpopforJsProxyof Javascript arrays so that they implement thecollections.abc.MutableSequenceAPI. #2970Enhancement Implemented methods
keys,items,values,get,pop,setdefault,popitem,update, andclearforJsProxyof map-like objects so that they implement thecollections.abc.MutableMappingAPI. #3275Enhancement It’s now possible to destructure a JavaScript array, map, or object returned by
as_object_mapwith amatchstatement. #2906Enhancement Added
then,catch, andfinally_methods to theFutures used by Pyodide’s event loop so they can be used likePromises. #2997Enhancement
create_proxynow takes an optionalroundtripparameter. If this is set toTrue, then when the proxy is converted back to Python, it is converted back to the same double proxy. This allows the proxy to be destroyed from Python even if no reference is retained. #3163, #3369Enhancement A
JsProxyof a function now has a__get__descriptor method, so it’s possible to use a JavaScript function as a Python method. When the method is called,thiswill be aPyProxypointing to the Python object the method is called on. #3130Enhancement A
JsProxynow has anas_object_mapmethod. This will treat the object as a mapping over itsownKeysso for instance:run_js("({a:2, b:3})").as_object_map()["a"]will return 2. These implementcollections.abc.MutableMapping. #3273, #3295, #3297Enhancement Split up the
JsProxydocumentation class into several classes, e.g.,JsBuffer,JsPromise, etc. Implementedissubclassandisinstanceon the various synthetic and realJsProxyclasses so that they behave the way one might naively expect them to (or at least closer to that than it was before). #3277Enhancement Added type parameters to many of the
JsProxysubtypes. #3387Enhancement Added
JsGeneratorandJsIteratortypes topyodide.ffi. Addedsendmethod toJsIterators andthrow, andclosemethods toJsGenerators. #3294Enhancement It is now possible to use asynchronous JavaScript iterables, iterators and generators from Python. This includes support for
aiterfor async interables,anextandasendfor async iterators, andathrowandaclosefor async generators. #3285, #3299, #3339Enhancement JavaScript generators and async generators that are created from Python now are wrapped so that Python objects sent to them as arguments or from
.send/.asendare kept alive until the generator is exhausted or.closed. This makes generators significantly more ergonomic to use, at the cost of making memory leaks more likely if the generator is never finalized. #3317Enhancement Added a mypy typeshed for some common functionality for the
jsmodule. #3298Enhancement mypy understands the types of more things now. #3385
Fix Fixed bug in
splitargument ofpyodide.console.repr_shorten. Addedshortenfunction. #3178
PyProxy / Using Python from JavaScript#
Enhancement Added a type field to
PythonError(e.g., a StopIteration error would havee.type === "StopIteration") #3289Enhancement It is now possible to use asynchronous Python generators from JavaScript. #3290
Enhancement PyProxies of synchronous and asynchronous Python generators now support
returnandthrowAPIs that behave like the ones on JavaScript generators. #3346Enhancement It is possible to make a
PyProxythat takesthisas the first argument using thePyProxy.captureThismethod. Thecreate_proxymethod also has acapture_thisargument which causes thePyProxyto receivethisas the first argument if set toTrue#3103, #3145
JavaScript API#
Enhancement Users can do a static import of
pyodide/pyodide.asm.jsto avoid issues with dynamic imports. This allows the use of Pyodide with module-type service workers. #3070Enhancement Added a new API
pyodide.mountNativeFSwhich mounts aFileSystemDirectoryHandleinto the Pyodide file system. #2987Enhancement
loadPyodidehas a new option calledargs. This list will be passed as command line arguments to the Python interpreter at start up. #3021, #3282Removed “Python initialization complete” message printed when loading is finished. {pr}`3247
BREAKING CHANGE The messageCallback and errorCallback argument to
loadPackageandloadPackagesFromImportsis now passed as named arguments. The old usage still works with a deprecation warning. #3149Enhancement
loadPackageandloadPackagesFromImportsnow accepts a new optioncheckIntegrity. If set to False, integrity check for Python Packages will be disabled.Enhancement Added APIs
pyodide.setStdin,pyodide.setStdout,pyodide.setStderrfor changing the stream handlers after loading Pyodide. Also added more careful control over whetherisattyreturns true or false on stdin, stdout, and stderr. #3268
Package Loading#
Enhancement Pyodide now shows more helpful error messages when importing packages that are included in Pyodide fails. #3137, #3263
Fix Shared libraries with version suffixes are now handled correctly. #3154
BREAKING CHANGE Unvendored the sqlite3 module from the standard library. Before
sqlite3was included by default. Now it needs to be loaded withpyodide.loadPackageormicropip.install. #2946BREAKING CHANGE The Pyodide Python package is installed into
/lib/python3.10rather than/lib/python3.10/site-packages. #3022BREAKING CHANGE The matplotlib HTML5 backends are now available as part of the
matplotlib-pyodidepackage. If you use the default backend from Pyodide, no changes are necessary. However, if you previously specified the backend withmatplotlib.use, the URL is now different. See package readme for more details. #3061BREAKING CHANGE The micropip package was moved to a separate repository pyodide/micropip. In addion to installing the version shipped with a given Pyodide release, you can also install a different micropip version from PyPi with,
await pyodide.loadPackage('packaging') await pyodide.loadPackage('<URL of the micropip wheel on PyPI>')
from Javascript. From Python you can import the Javascript Pyodide package,
import pyodide_js
and call the same functions as above. #3122
Enhancement The parsing and validation of
meta.yamlaccording to the specification is now done more rigorously with Pydantic. #3079BREAKING CHANGE The
source/md5checksum field is not longer supported inmeta.yamlfiles, usesource/sha256instead #3079BREAKING CHANGE
pyodide_build.io.parse_package_configfunction is removed in favor ofpyodide_build.MetaConfig.from_yaml#3079Fix
ctypes.util.find_librarywill now search WASM modules from LD_LIBRARY_PATH. #3353
Build System#
Enhancement Updated Emscripten to version 3.1.27 #2958, #2950, #3027, #3107, #3148, #3236, #3239, #3280, #3314
Enhancement Added
requirements/hostkey to themeta.yamlspec to allow host dependencies that are required for building packages. #2132Enhancement Added
package/top-levelkey to themeta.yamlspec to calculate top-level import names for the package. Previouslytest/importskey was used for this purpose. #3006Enhancement Added
build/vendor-sharedlibkey to themeta.yamlspec which vendors shared libraries into the wheel after building. #3234 #3264Enhancement Added
build/typekey to themeta.yamlspec which specifies the type of the package. #3238Enhancement Added
requirements/executablekey to themeta.yamlspec which specifies the list of executables required for building a package. #3300BREAKING CHANGE
build/libraryandbuild/sharedlibrarykey in themeta.yamlspec are removed. Usebuild/typeinstead. #3238Fix Fixed a bug that
backend-flagspropagated to dependencies. #3153Fix Fixed a bug that shared libraries are not copied into distribution directory when it is already built. #3212
Enhancement Added a system for making Pyodide virtual environments. This is for testing out of tree builds. For more information, see the documentation. #2976, #3039, #3040, #3044, #3096, #3098, #3108, #3109, #3241
Added a new CLI command
pyodide skeletonwhich creates a package build recipe.pyodide-build mkpkgwill be replaced bypyodide skeleton pypi. #3175Added a new CLI command
pyodide build-recipeswhich build packages from recipe folder. It replacespyodide-build buildall. #3196 #3279Added a new CLI command
pyodide configwhich shows config variables used in Pyodide. #3376Added subcommands for
pyodide buildwhich builds packages from various sources.command
result
pyodide build pypibuild or fetch a single package from pypi
pyodide build sourcebuild the current source folder (same as pyodide build)
pyodide build urlbuild or fetch a package from a url either tgz, tar.gz zip or wheel
Packages#
New packages: pycryptodome #2965, coverage-py #3053, bcrypt #3125, lightgbm #3138, pyheif, pillow_heif, libheif, libde265 #3161, wordcloud #3173, gdal, fiona, geopandas #3213, the standard library _hashlib module #3206 , pyinstrument #3258, gensim #3326, smart_open #3326, pyodide-http #3355.
Fix Scipy CSR data is now handled correctly in XGBoost. #3194
Update Upgraded packages: SciPy 1.9.1 #3043, pandas 1.5.0 #3134, numpy 1.23.3 #3284, scikit-learn 1.1.3 #3324 as well as most of the other packages #3348 #3365. See Packages built in Pyodide for more details.
Fix Fix scipy handling of exceptions that are raised from C++ code. #3384.
List of Contributors#
Aierie, dataxerik, David Lechner, Deepak Cherian, Filipe, Gyeongjae Choi, Hood Chatham, H.Yamada, Jacques Boscq, Jeremy Tuloup, Joe Marshall, John Wason, Loïc Estève, partev, Patrick Arminio, Péter Ferenc Gyarmati, Prete, Qijia Liu, Roman Yurchak, ryanking13, skelsec, Starz0r, Will Lachance, YeonWoo, Yizhi Liu
Version 0.21.3#
September 15, 2022
Fix When loading
sqlite3,loadPackageno longer also loadsnltkandregex. #3001Fix Packages are now loaded in a topologically sorted order regarding their dependencies. #3020
BREAKING CHANGE Loading the
soupsievepackage will not automatically loadbeautifulsoup4together. #3020Fix Fix the incorrect package name
ruameltoruamel.yaml. #3036Fix
loadPyodidewill now raise error when the version of JavaScript and Python Pyodide package does not match. #3074Enhancement Pyodide now works with a content security policy that doesn’t include
unsafe-eval. It is still necessary to includewasm-unsafe-eval(and probably always will be). Since current Safari versions have no support forwasm-unsafe-eval, it is necessary to includeunsafe-evalin order to work in Safari. This will likely be fixed in the next Safari release: https://bugs.webkit.org/show_bug.cgi?id=235408 #3075Fix It works again to use
loadPyodidewith a relative URL asindexURL(this was a regression in v0.21.2). #3077Fix Add
urlto list of pollyfilled packages for webpack compatibility. #3080Fix Fixed warnings like
Critical dependency: the request of a dependency is an expression.when using Pyodide with webpack. #3080Enhancement Add binary files to exports in JavaScript package #3085.
Fix Source maps are included in the distribution again (reverting #3015 included in 0.21.2) and if there is a variable in top level scope called
__dirnamewe use that for theindexURL. #3088Fix
PyProxy.applynow correctly handles the case when something unexpected is passed as the second argument. #3101
Version 0.21.2#
August 29, 2022
Fix The standard library packages
sslandlzmacan now be installed withpyodide.loadPackage("ssl")ormicropip.install("ssl")(previously they had a leading underscore and it was only possible to load them withpyodide.loadPackage). #3003Fix If a wheel path is passed to
pyodide.loadPackage, it will now be resolved relative todocument.location(in browser) or relative to the current working directory (in Node) rather than relative toindexURL. #3013, #3011Fix Fixed a bug in Emscripten that caused Pyodide to fail in Jest. #3014
Fix It now works to pass a relative url to
indexURL. Also, the calculated index URL now works even ifnodeis run with--enable-source-maps. #3015
Version 0.21.1#
August 22, 2022
New packages: the standard library lzma module #2939
Enhancement Pyodide now shows more helpful error messages when importing unvendored or removed stdlib modules fails. #2973
BREAKING CHANGE The default value of
fullStdLibinloadPyodidehas been changed tofalse. This means Pyodide now will not load some stdlib modules like distutils, ssl, and sqlite3 by default. See Pyodide Python compatibility for detail. IffullStdLibis set totrue, it will load all unvendored stdlib modules. However, settingfullStdLibto true will increase the initial Pyodide load time. So it is preferable to explicitly load the required module. #2998Enhancement
pyodide buildnow checks that the correct version of the Emscripten compiler is used. #2975, #2990Fix Pyodide works in Safari v14 again. It was broken in v0.21.0 #2994
Version 0.21.0#
August 9, 2022
See the release notes for a summary.
Build system#
Enhancement Emscripten was updated to Version 3.1.14 #2775, #2679, #2672
Enhancement Update Typescript target to ES2017 to generate more modern Javascript code. #2471
Enhancement We now put our built files into the
distdirectory rather than thebuilddirectory. #2387Fix The build will error out earlier if
cmakeorlibtoolare not installed. #2423Enhancement The platform tags of wheels now include the Emscripten version in them. This should help ensure ABI compatibility if Emscripten wheels are distributed outside of the main Pyodide distribution. #2610
Enhancement The build system now uses the sysconfigdata from the target Python rather than the host Python. #2516
Enhancement Pyodide now builds with
-sWASM_BIGINT. #2643Enhancement Added
cross-scriptkey to themeta.yamlspec to allow executing custom logic in the cross build environment. #2734
Pyodide Module and type conversions#
API Change All functions were moved out of the root
pyodidepackage into various submodules. For backwards compatibility, they will be available from the root package (raising aFutureWarning) until v0.23.0. #2787, #2790Enhancement
loadPyodideno longer uses any global state, so it can be used more than once in the same thread. This is recommended if a network request causes a loading failure, if there is a fatal error, if you damage the state of the runtime so badly that it is no longer usable, or for certain testing purposes. It is not recommended for creating multiple execution environments, for which you should usepyodide.runPython(code, { globals : some_dict}); #2391Enhancement
pyodide.unpackArchivenow accepts anyArrayBufferVieworArrayBufferas first argument, rather than only aUint8Array. #2451Feature Added
pyodide.code.run_jsAPI. #2426Fix BigInt’s between 2^{32*n - 1} and 2^{32*n} no longer get translated to negative Python ints. #2484
Fix Pyodide now correctly handles JavaScript objects with
nullconstructor. #2520Fix Fix garbage collection of
once_callable#2401Enhancement Added the
js_idattribute toJsProxyto allow using JavaScript object identity as a dictionary key. #2515Fix Fixed a bug with
toJswhen used with recursive structures and thedictConverterargument. #2533Enhancement Added Python wrappers
set_timeout,clear_timeout,set_interval,clear_interval,add_event_listenerandremove_event_listenerfor the corresponding JavaScript functions. #2456Fix If a request fails due to CORS,
pyfetchnow raises anOSErrornot aJSException. #2598Enhancement Pyodide now directly exposes the Emscripten
PATHandERRNO_CODESAPIs. #2582Fix The
booloperator on aJsProxynow behaves more consistently: it returnsFalseif JavaScript would say that!!xisfalse, or ifxis an empty container. Otherwise it returnsTrue. #2803Fix Fix
loadPyodideerrors for the Windows Node environment. #2888Enhancement Implemented slice subscripting,
+=, andextendforJsProxyof Javascript arrays. #2907
REPL#
micropip and package loading#
API Change
packages.jsonwhich contains the dependency graph for packages was renamed torepodata.jsonto avoid confusion withpackage.jsonused in JavaScript packages.Enhancement Added SHA-256 hash of package to entries in
repodata.json#2455Enhancement Integrity of Pyodide packages is now verified before loading them. This is for now limited to browser environments. #2513
Enhancement
micropipsupports loading wheels from the Emscripten file system using theemfs:protocol now. #2767Enhancement It is now possible to use an alternate
repodata.jsonlockfile by passing thelockFileURLoption toloadPyodide. This is particularly intended to be used withmicropip.freeze. #2645Fix micropip now correctly handles package names that include dashes #2414
Enhancement Allow passing
credentialstomicropip.install()#2458Enhancement
micropip.install()now accepts adepsparameter. If set toFalse, micropip will not install dependencies of the package. #2433Fix micropip now correctly compares packages with prerelease version #2532
Enhancement
micropip.install()now accepts apreparameter. If set toTrue, micropip will include pre-release and development versions. #2542Enhancement
micropipwas refactored to improve readability and ease of maintenance. #2561, #2563, #2564, #2565, #2568Enhancement Various error messages were fine tuned and improved. #2562, #2558
Enhancement
micropipwas adjusted to keep its state in the wheel.dist-infodirectories which improves consistenency with the Python standard library and other tools used to install packages. #2572Enhancement
micropipcan now be used to install Emscripten binary wheels. #2591Enhancement Added
micropip.freezeto record the current set of loaded packages into arepodata.jsonfile. #2581Fix
micropip.listnow works correctly when there are packages that are installed viapyodide.loadPackagefrom a custom URL. #2743Fix micropip now skips package versions which do not follow PEP440. #2754
Fix
micropipsupports extra markers in packages correctly now. #2584
Packages#
Enhancement Update sqlite version to latest stable release #2477 and #2518
Enhancement Pillow now supports WEBP image format #2407.
Enhancement Pillow and opencv-python now support the TIFF image format. #2762
Pandas is now compiled with
-Oz, which significantly speeds up loading the library on Chrome #2457New packages: opencv-python #2305, ffmpeg #2305, libwebp #2305, h5py, pkgconfig and libhdf5 #2411, bitarray #2459, gsw #2511, cftime #2504, svgwrite, jsonschema, tskit #2506, xarray #2538, demes, libgsl, newick, ruamel, msprime #2548, gmpy2 #2665, xgboost #2537, galpy #2676, shapely, geos #2725, suitesparse, sparseqr #2685, libtiff #2762, pytest-benchmark #2799, termcolor #2809, sqlite3, libproj, pyproj, certifi #2555, rebound #2868, reboundx #2909, pyclipper #2886, brotli #2925, python-magic #2941
Miscellaneous#
Fix We now tell packagers (e.g., Webpack) to ignore npm-specific imports when packing files for the browser. #2468
Enhancement
run_in_pyodidenow has support for pytest assertion rewriting and decorators such aspytest.mark.parametrizeand hypothesis. #2510, #2541BREAKING CHANGE
pyodide_build.testingis removed.run_in_pyodidedecorator can now be accessed throughpytest-pyodidepackage. #2418
List of contributors#
Alexey Ignatiev, Andrey Smelter, andrzej, Antonio Cuni, Ben Jeffery, Brian Benjamin Maranville, David Lechner, dragoncoder047, echorand (Amit Saha), Filipe, Frank, Gyeongjae Choi, Hanno Rein, haoran1062, Henry Schreiner, Hood Chatham, Jason Grout, jmdyck, Jo Bovy, John Wason, josephrocca, Kyle Cutler, Lester Fan, Liumeo, lukemarsden, Mario Gersbach, Matt Toad, Michael Droettboom, Michael Gilbert, Michael Neil, Mu-Tsun Tsai, Nicholas Bollweg, pysathq, Ricardo Prins, Rob Gries, Roman Yurchak, Ryan May, Ryan Russell, stonebig, Szymswiat, Tobias Megies, Vic Kumar, Victor, Wei Ji, Will Lachance
Version 0.20.0#
April 9th, 2022
See the release notes for a summary.
CPython and stdlib#
Packages#
Fix matplotlib now loads multiple fonts correctly #2271
New packages: boost-histogram #2174, cryptography v3.3.2 #2263, the standard library ssl module #2263, python-solvespace v3.0.7, lazy-object-proxy #2320.
Many more scipy linking errors were fixed, mostly related to the Fortran f2c ABI for string arguments. There are still some fatal errors in the Scipy test suite, but none seem to be simple linker errors. #2289
Removed pyodide-interrupts. If you were using this for some reason, use
pyodide.setInterruptBufferinstead. #2309Most included packages were updated to the latest version. See Packages built in Pyodide for a full list.
Type translations#
Fix Python tracebacks now include Javascript frames when Python calls a Javascript function. #2123
Enhancement Added a
default_converterargument toJsProxy.to_pyandpyodide.toPywhich is used to process any object that doesn’t have a built-in conversion to Python. Also added adefault_converterargument toPyProxy.toJsandpyodide.ffi.to_jsto convert. #2170 and #2208Enhancement Async Python functions called from Javascript now have the resulting coroutine automatically scheduled. For instance, this makes it possible to use an async Python function as a Javascript event handler. #2319
Javascript package#
Enhancement It is no longer necessary to provide
indexURLtoloadPyodide. #2292BREAKING CHANGE The
globalsargument topyodide.runPythonandpyodide.runPythonAsyncis now passed as a named argument. The old usage still works with a deprecation warning. #2300Enhancement The Javascript package was migrated to Typescript. #2130 and #2133
Fix Fix importing pyodide with ESM syntax in a module type web worker. #2220
Enhancement When Pyodide is loaded as an ES6 module, no global
loadPyodidevariable is created (instead, it should be accessed as an attribute on the module). #2249Fix The type
Py2JsResulthas been replaced withanywhich is more accurate. For backwards compatibility, we still exportPy2JsResultas an alias forany. #2277Fix Pyodide now loads correctly even if requirejs is included. #2283
Enhancement Added robust handling for non-
Errorobjects thrown by Javascript code. This mostly should never happen since well behaved Javascript code ought to throw errors. But it’s better not to completely crash if it throws something else. #2294
pyodide_build#
Enhancement Pyodide now uses Python wheel files to distribute packages rather than the emscripten
file_packager.pyformat. #2027Enhancement Pyodide now uses
pypa/buildto build packages. We (mostly) use build isolation, so we can build packages that require conflicting versions of setuptools or alternative build backends. #2272Enhancement Most pure Python packages were switched to use the wheels directly from PyPI rather than rebuilding them. #2126
Enhancement Added support for C++ exceptions in packages. Now C++ extensions compiled and linked with
-fexceptionscan catch C++ exceptions. Furthermore, uncaught C++ exceptions will be formatted in a human-readable way. #2178BREAKING CHANGE Removed the
skip-hostkey from themeta.yamlformat. If needed, install a host copy of the package with pip instead. #2256
Uncategorized#
Enhancement The interrupt buffer can be used to raise all 64 signals now, not just
SIGINT. Write a number between1<= signum <= 64into the interrupt buffer to trigger the corresponding signal. By default everything butSIGINTwill be ignored. Any value written into the interrupt buffer outside of the range from 1 to 64 will be silently discarded. #2301Enhancement Updated to Emscripten 2.0.27. #2295
BREAKING CHANGE The
extractDirargument topyodide.unpackArchiveis now passed as a named argument. The old usage still works with a deprecation warning. #2300Enhancement Support ANSI escape codes in the Pyodide console. #2345
Fix
pyodide_buildcan now be installed in non-editable ways. #2351
List of contributors#
Boris Feld, Christian Staudt, Gabriel Fougeron, Gyeongjae Choi, Henry Schreiner, Hood Chatham, Jo Bovy, Karthikeyan Singaravelan, Leo Psidom, Liumeo, Luka Mamukashvili, Madhur Tandon, Paul Korzhyk, Roman Yurchak, Seungmin Kim, Thorsten Beier, Tom White, and Will Lachance
Version 0.19.1#
February 19, 2022
Packages#
Type translations#
Fix The “PyProxy has already been destroyed” error message has been improved with some context information. #2121
REPL#
Enhancement Pressing TAB in REPL no longer triggers completion when input is whitespace. #2125
List of contributors#
Christian Staudt, Gyeongjae Choi, Hood Chatham, Liumeo, Paul Korzhyk, Roman Yurchak, Seungmin Kim, Thorsten Beier
Version 0.19.0#
January 10, 2021
See the release notes for a summary.
Python package#
Enhancement If
find_importsis used on code that contains a syntax error, it will return an empty list instead of raising aSyntaxError. #1819Enhancement Added the
pyodide.http.pyfetchAPI which provides a convenience wrapper for the JavascriptfetchAPI. The API returns a response object with various methods that convert the data into various types while minimizing the number of times the data is copied. #1865Enhancement Added the
unpack_archiveAPI to thepyodide.http.FetchResponseobject which treats the response body as an archive and usesshutilto unpack it. #1935Fix The Pyodide event loop now works correctly with cancelled handles. In particular,
asyncio.wait_fornow functions as expected. #2022
JavaScript package#
Fix
loadPyodideno longer fails in the presence of a user-defined global namedprocess. #1849Fix Various webpack buildtime and runtime compatibility issues were fixed. #1900
Enhancement Added the
pyodide.pyimportAPI to import a Python module and return it as aPyProxy. Warning: this is different from the originalpyimportAPI which was removed in this version. #1944Enhancement Added the
pyodide.unpackArchiveAPI which unpacks an archive represented as an ArrayBuffer into the working directory. This is intended as a way to install packages from a local application. #1944API Change
loadPyodidenow accepts ahomedirparameter which sets home directory of Pyodide virtual file system. #1936BREAKING CHANGE The default working directory(home directory) inside the Pyodide virtual file system has been changed from
/to/home/pyodide. To get the previous behavior, you cancall
os.chdir("/")in Python to change working directory orcall
loadPyodidewith thehomedir="/"argument #1936
Python / JavaScript type conversions#
BREAKING CHANGE Updated the calling convention when a JavaScript function is called from Python to improve memory management of PyProxies. PyProxy arguments and return values are automatically destroyed when the function is finished. #1573
Enhancement Added
JsProxy.to_string,JsProxy.to_bytes, andJsProxy.to_memoryviewto allow for conversion ofTypedArrayto standard Python types without unneeded copies. #1864Enhancement Added
JsProxy.to_fileandJsProxy.from_fileto allow reading and writing Javascript buffers to files as a byte stream without unneeded copies. #1864Fix It is now possible to destroy a borrowed attribute
PyProxyof aPyProxy(as introduced by #1636) before destroying the rootPyProxy. #1854Fix If
__iter__()raises an error, it is now handled correctly by thePyProxy[Symbol.iterator()]method. #1871Fix Borrowed attribute
PyProxys are no longer destroyed when the rootPyProxyis garbage collected (because it was leaked). Doing so has no benefit to nonleaky code and turns some leaky code into broken code (see #1855 for an example). #1870Fix Improved the way that
pyodide.globals.get("builtin_name")works. Before we used__main__.__dict__.update(builtins.__dict__)which led to several undesirable effects such as__name__being equal to"builtins". Now we use a proxy wrapper to replacepyodide.globals.getwith a function that looks up the name onbuiltinsif lookup onglobalsfails. #1905Enhancement Coroutines have their memory managed in a more convenient way. In particular, now it is only necessary to either
awaitthe coroutine or call one of.then,.exceptor.finallyto prevent a leak. It is no longer necessary to manually destroy the coroutine. Example: before:
async function runPythonAsync(code, globals) {
let coroutine = Module.pyodide_py.eval_code_async(code, globals);
try {
return await coroutine;
} finally {
coroutine.destroy();
}
}
After:
async function runPythonAsync(code, globals) {
return await Module.pyodide_py.eval_code_async(code, globals);
}
pyodide-build#
API Change By default only a minimal set of packages is built. To build all packages set
PYODIDE_PACKAGES='*'In addition,make minimalwas removed, since it is now equivalent tomakewithout extra arguments. #1801Enhancement It is now possible to use
pyodide-build buildallandpyodide-build buildpkgdirectly. #2063Enhancement Added a
--force-rebuildflag tobuildallandbuildpkgwhich rebuilds the package even if it looks like it doesn’t need to be rebuilt. Added a--continueflag which keeps the same source tree for the package and can continue from the middle of a build. #2069Enhancement Changes to environment variables in the build script are now seen in the compile and post build scripts. #1706
Fix Fix usability issues with
pyodide-build mkpkgCLI. #1828Enhancement Better support for ccache when building Pyodide #1805
Fix Fix compile error
wasm-ld: error: unknown argument: --sort-commonandwasm-ld: error: unknown argument: --as-neededin ArchLinux. #1965
micropip#
Fix micropip now raises an error when installing a non-pure python wheel directly from a url. #1859
Enhancement
micropip.install()now accepts akeep_goingparameter. If set toTrue, micropip reports all identifiable dependencies that don’t have pure Python wheels, instead of failing after processing the first one. #1976Enhancement Added a new API
micropip.list()which returns the list of installed packages by micropip. #2012
Packages#
Enhancement Unit tests are now unvendored from Python packages and included in a separate package
<package name>-tests. This results in a 20% size reduction on average for packages that vendor tests (e.g. numpy, pandas, scipy). #1832Update Upgraded SciPy to 1.7.3. There are known issues with some SciPy components, the current status of the scipy test suite is here #2065
Fix The built-in pwd module of Python, which provides a Unix specific feature, is now unvendored. #1883
Fix pillow and imageio now correctly encode/decode grayscale and black-and-white JPEG images. #2028
Fix The numpy fft module now works correctly. #2028
New packages: logbook #1920, pyb2d #1968, and threadpoolctl (a dependency of scikit-learn) #2065
Upgraded packages: numpy (1.21.4) #1934, scikit-learn (1.0.2) #2065, scikit-image (0.19.1) #2005, msgpack (1.0.3) #2071, astropy (5.0.3) #2086, statsmodels (0.13.1) #2073, pillow (9.0.0) #2085. This list is not exhaustive, refer to
packages.jsonfor the full list.
Uncategorized#
Enhancement
PyErr_CheckSignalsnow works with the keyboard interrupt system so that cooperative C extensions can be interrupted. Also, added thepyodide.checkInterruptfunction so Javascript code can opt to be interrupted. #1294Fix The
_variable is now set by the Pyodide repl just like it is set in the native Python repl. #1904Enhancement
pyodide-envandpyodideDocker images are now available from both the Docker Hub and from the Github Package registry. #1995Fix The console now correctly handles it when an object’s
__repr__function raises an exception. #2021Enhancement Removed the
-s EMULATE_FUNCTION_POINTER_CASTSflag, yielding large benefits in speed, stack usage, and code size. #2019
List of contributors#
Alexey Ignatiev, Alex Hall, Bart Broere, Cyrille Bogaert, etienne, Grimmer, Grimmer Kang, Gyeongjae Choi, Hao Zhang, Hood Chatham, Ian Clester, Jan Max Meyer, LeoPsidom, Liumeo, Michael Christensen, Owen Ou, Roman Yurchak, Seungmin Kim, Sylvain, Thorsten Beier, Wei Ouyang, Will Lachance
Version 0.18.1#
September 16, 2021
Console#
Fix Ctrl+C handling in console now works correctly with multiline input. New behavior more closely approximates the behavior of the native Python console. #1790
Fix Fix the repr of Python objects (including lists and dicts) in console #1780
Fix The “long output truncated” message now appears on a separate line as intended. #1814
Fix The streams that are used to redirect stdin and stdout in the console now define
isattyto returnTrue. This fixes pytest. #1822
Python package#
Fix Avoid circular references when runsource raises SyntaxError #1758
JavaScript package#
Fix The
pyodide.setInterruptBuffercommand is now publicly exposed again, as it was in v0.17.0. #1797
Python / JavaScript type conversions#
Packages#
Fix pillow now correctly encodes/decodes RGB JPEG image format. #1818
Micellaneous#
Fix Patched emscripten to make the system calls to duplicate file descriptors closer to posix-compliant. In particular, this fixes the use of
dupon pipes and temporary files, as needed bypytest. #1823
Version 0.18.0#
August 3rd, 2021
General#
Update Pyodide now runs Python 3.9.5. #1637
Enhancement Pyodide can experimentally be used in Node.js #1689
Enhancement Pyodide now directly exposes the Emscripten filesystem API, allowing for direct manipulation of the in-memory filesystem #1692
Enhancement Pyodide’s support of emscripten file systems is expanded from the default
MEMFSto includeIDBFS,NODEFS,PROXYFS, andWORKERFS, allowing for custom persistence strategies depending on execution environment #1596API Change The
packages.jsonschema for Pyodide was redesigned for better compatibility with conda. #1700API Change
run_dockerno longer binds any port to the docker image by default. #1750
Standard library#
API Change The following standard library modules are now available as standalone packages
distlib
They are loaded by default in
loadPyodide, however this behavior can be disabled with thefullStdLibparameter set tofalse. All optional stdlib modules can then be loaded as needed withpyodide.loadPackage. #1543Enhancement The standard library module
audioopis now included, making thewave,sndhdr,aifc, andsunaumodules usable. #1623Enhancement Added support for
ctypes. #1656
JavaScript package#
Enhancement The Pyodide JavaScript package is released to npm under npmjs.com/package/pyodide #1762
API Change
loadPyodideno longer automatically stores the API into a global variable calledpyodide. To get old behavior, sayglobalThis.pyodide = await loadPyodide({...}). #1597Enhancement
loadPyodidenow accepts callback functions forstdin,stdoutandstderr#1728Enhancement Pyodide now ships with first party typescript types for the entire JavaScript API (though no typings are available for
PyProxyfields). #1601Enhancement It is now possible to import
Comlinkobjects into Pyodide after usingpyodide.registerComlink#1642Enhancement If a Python error occurs in a reentrant
runPythoncall, the error will be propagated into the outerrunPythoncontext as the original error type. This is particularly important if the error is aKeyboardInterrupt. #1447
Python package#
Enhancement Added a new
pyodide.code.CodeRunnerAPI for finer control thaneval_codeandeval_code_async. Designed with the needs of REPL implementations in mind. #1563Enhancement Added
pyodide.console.Consoleclass closely based on the Python standard librarycode.InteractiveConsolebut with support for top level await and stream redirection. Also added the subclasspyodide.console.PyodideConsolewhich automatically usespyodide.loadPackagesFromImportson the code before running it. #1125, #1155, #1635Fix
pyodide.code.eval_code_asyncno longer automatically awaits a returned coroutine or attempts to await a returned generator object (which triggered an error). #1563
Python / JavaScript type conversions#
API Change
pyodide.runPythonAsyncno longer automatically callspyodide.loadPackagesFromImports. #1538.Enhancement Added the
PyProxy.callKwargsmethod to allow using Python functions with keyword arguments from JavaScript. #1539API Change Updated the method resolution order on
PyProxy. Performing a lookup on aPyProxywill prefer to pick a method from thePyProxyapi, if no such method is found, it will usegetattron the proxied object. Prefixing a name with$forcesgetattr. For instance,PyProxy.destroynow always refers to the method that destroys the proxy, whereasPyProxy.$destroyrefers to an attribute or method calleddestroyon the proxied object. #1604API Change It is now possible to use
Symbolkeys with PyProxies. TheseSymbolkeys put markers on the PyProxy that can be used by external code. They will not currently be copied byPyProxy.copy. #1696Enhancement Memory management of
PyProxyfields has been changed so that fields looked up on aPyProxyare “borrowed” and have their lifetime attached to the basePyProxy. This is intended to allow for more idiomatic usage. (See #1617.) #1636API Change The depth argument to
toJsis now passed as an option, sotoJs(n)in v0.17 changed totoJs({depth : n}). Similarly,pyodide.toPynow takesdepthas a named argument. Alsoto_jsandto_pyonly take depth as a keyword argument. #1721API Change
PyProxy.toJsandpyodide.ffi.to_jsnow take an optionpyproxies, if a JavaScript Array is passed for this, then any proxies created during conversion will be placed into this array. This allows easy cleanup later. Thecreate_pyproxiesoption can be used to disable creation of pyproxies during conversion (instead aConversionErroris raised). #1726API Change
toJsandto_jsnow take an optiondict_converterwhich will be called on a JavaScript iterable of two-element Arrays as the final step of converting dictionaries. For instance, passObject.fromEntriesto convert to an object orArray.fromto convert to an array of pairs. #1742
pyodide-build#
API Change pyodide-build is now an installable Python package, with an identically named CLI entrypoint that replaces
bin/pyodidewhich is removed #1566
micropip#
Fix micropip now correctly handles packages that have mixed case names. (See #1614). #1615
Enhancement micropip now resolves dependencies correctly for old versions of packages (it used to always use the dependencies from the most recent version, see #1619 and #1745). micropip also will resolve dependencies for wheels loaded from custom urls. #1753
Packages#
Enhancement matplotlib now comes with a new renderer based on the html5 canvas element. #1579 It is optional and the current default backend is still the agg backend compiled to wasm.
Enhancement Updated a number of packages included in Pyodide.
List of contributors#
Albertas Gimbutas, Andreas Klostermann, Arfy Slowy, daoxian, Devin Neal, fuyutarow, Grimmer, Guido Zuidhof, Gyeongjae Choi, Hood Chatham, Ian Clester, Itay Dafna, Jeremy Tuloup, jmsmdy, LinasNas, Madhur Tandon, Michael Christensen, Nicholas Bollweg, Ondřej Staněk, Paul m. p. P, Piet Brömmel, Roman Yurchak, stefnotch, Syrus Akbary, Teon L Brooks, Waldir
Version 0.17.0#
April 21, 2021
See the Version 0.17.0 Release Notes for more information.
Improvements to package loading and dynamic linking#
Enhancement Uses the emscripten preload plugin system to preload .so files in packages
Enhancement Support for shared library packages. This is used for CLAPACK which makes scipy a lot smaller. #1236
Fix Pyodide and included packages can now be used with Safari v14+. Safari v13 has also been observed to work on some (but not all) devices.
Python / JS type conversions#
Feature A
JsProxyof a JavaScriptPromiseor other awaitable object is now a Python awaitable. #880API Change Instead of automatically converting Python lists and dicts into JavaScript, they are now wrapped in
PyProxy. Added a newPyProxy.toJsAPI to request the conversion behavior that used to be implicit. #1167API Change Added
JsProxy.to_pyAPI to convert a JavaScript object to Python. #1244Feature Flexible jsimports: it now possible to add custom Python “packages” backed by JavaScript code, like the
jspackage. Thejspackage is now implemented using this system. #1146Feature A
PyProxyof a Python coroutine or awaitable is now an awaitable JavaScript object. Awaiting a coroutine will schedule it to run on the Python event loop usingasyncio.ensure_future. #1170Enhancement Made
PyProxyof an iterable Python object an iterable Js object: defined the[Symbol.iterator]method, can be used likefor(let x of proxy). Made aPyProxyof a Python iterator an iterator:proxy.next()is translated tonext(it). Made aPyProxyof a Python generator into a JavaScript generator:proxy.next(val)is translated togen.send(val). #1180API Change Updated
PyProxyso that if the wrapped Python object supports__getitem__access, then the wrapper hasget,set,has, anddeletemethods which doobj[key],obj[key] = val,key in objanddel obj[key]respectively. #1175API Change The
pyodide.pyimportfunction is deprecated in favor of usingpyodide.globals.get('key'). #1367API Change Added
PyProxy.getBufferAPI to allow direct access to Python buffers as JavaScript TypedArrays. #1215API Change The innermost level of a buffer converted to JavaScript used to be a TypedArray if the buffer was contiguous and otherwise an Array. Now the innermost level will be a TypedArray unless the buffer format code is a ‘?’ in which case it will be an Array of booleans, or if the format code is a “s” in which case the innermost level will be converted to a string. #1376
Enhancement JavaScript
BigInts are converted into Pythonintand Pythonints larger than 2^53 are converted intoBigInt. #1407API Change Added
pyodide.isPyProxyto test if an object is aPyProxy. #1456Enhancement
PyProxyandPyBufferobjects are now garbage collected if the browser supportsFinalizationRegistry. #1306Enhancement Automatic conversion of JavaScript functions to CPython calling conventions. #1051, #1080
Enhancement Automatic detection of fatal errors. In this case Pyodide will produce both a JavaScript and a Python stack trace with explicit instruction to open a bug report. pr
{1151}, pr{1390}, pr{1478}.Enhancement Systematic memory leak detection in the test suite and a large number of fixed to memory leaks. pr
{1340}Fix getattr and dir on JsProxy now report consistent results and include all names defined on the Python dictionary backing JsProxy. #1017
Fix
JsProxy.__bool__now produces more consistent results: bothbool(window)andbool(zero-arg-callback)wereFalsebut now areTrue. Conversely,bool(empty_js_set)andbool(empty_js_map)wereTruebut now areFalse. #1061Fix When calling a JavaScript function from Python without keyword arguments, Pyodide no longer passes a
PyProxy-wrappedNULLpointer as the last argument. #1033Fix JsBoundMethod is now a subclass of JsProxy, which fixes nested attribute access and various other strange bugs. #1124
Fix JavaScript functions imported like
from js import fetchno longer trigger “invalid invocation” errors (issue #461) andjs.fetch("some_url")also works now (issue #768). #1126Fix JavaScript bound method calls now work correctly with keyword arguments. #1138
Fix JavaScript constructor calls now work correctly with keyword arguments. #1433
pyodide-py package#
Feature Added a Python event loop to support asyncio by scheduling coroutines to run as jobs on the browser event loop. This event loop is available by default and automatically enabled by any relevant asyncio API, so for instance
asyncio.ensure_futureworks without any configuration. #1158API Change Removed
as_nested_listAPI in favor ofJsProxy.to_py. #1345
pyodide-js#
API Change Removed iodide-specific code in
pyodide.js. This breaks compatibility with iodide. #878, #981API Change Removed the
pyodide.autocompleteAPI, use Jedi directly instead. #1066API Change Removed
pyodide.reprAPI. #1067Fix If
messageCallbackanderrorCallbackare supplied topyodide.loadPackage,pyodide.runPythonAsyncandpyodide.loadPackagesFromImport, then the messages are no longer automatically logged to the console.Feature
runPythonAsyncnow runs the code witheval_code_async. In particular, it is possible to use top-level await inside ofrunPythonAsync.eval_codenow accepts separateglobalsandlocalsparameters. #1083Added the
pyodide.setInterruptBufferAPI. This can be used to set aSharedArrayBufferto be the keyboard interrupt buffer. If Pyodide is running on a webworker, the main thread can signal to the webworker that it should raise aKeyboardInterruptby writing to the interrupt buffer. #1148 and #1173Changed the loading method: added an async function
loadPyodideto load Pyodide to use instead oflanguagePluginURLandlanguagePluginLoader. The change is currently backwards compatible, but the old approach is deprecated. #1363runPythonAsyncnow acceptsglobalsparameter. #1914
micropip#
Feature
micropipnow supports installing wheels from relative URLs. #872API Change
micropip.installnow returns a PythonFutureinstead of a JavaScriptPromise. #1324Fix
micropip.installnow interacts correctly withpyodide.loadPackage(). #1457Fix
micropip.installnow handles version constraints correctly even if there is a version of the package available from the PyodideindexURL.
Build system#
Enhancement Updated to latest emscripten 2.0.13 with the upstream LLVM backend #1102
API Change Use upstream
file_packager.py, and stop checking package abi versions. ThePYODIDE_PACKAGE_ABIenvironment variable is no longer used, but is still set as some packages use it to detect whether it is being built for Pyodide. This usage is deprecated, and a new environment variablePYODIDEis introduced for this purpose.As part of the change, Module.checkABI is no longer present. #991
uglifyjs and lessc no longer need to be installed in the system during build #878.
Enhancement Reduce the size of the core Pyodide package #987.
Enhancement Optionally to disable docker port binding #1423.
Enhancement Run arbitrary command in docker #1424
Docker images for Pyodide are now accessible at pyodide/pyodide-env and pyodide/pyodide.
Enhancement Option to run docker in non-interactive mode #1641
REPL#
Fix In console.html: sync behavior, full stdout/stderr support, clean namespace, bigger font, correct result representation, clean traceback #1125 and #1141
Fix Switched from ̀Jedi to rlcompleter for completion in
pyodide.console.InteractiveConsoleand so inconsole.html. This fixes some completion issues (see #821 and #1160)Enhancement Support top-level await in the console #1459
Packages#
List of contributors#
(in alphabetic order)
Aditya Shankar, casatir, Dexter Chua, dmondev, Frederik Braun, Hood Chatham, Jan Max Meyer, Jeremy Tuloup, joemarshall, leafjolt, Michael Greminger, Mireille Raad, Ondřej Staněk, Paul m. p. P, rdb, Roman Yurchak, Rudolfs
Version 0.16.1#
December 25, 2020
Note: due to a CI deployment issue the 0.16.0 release was skipped and replaced by 0.16.1 with identical contents.
Pyodide files are distributed by JsDelivr,
https://cdn.jsdelivr.net/pyodide/v0.16.1/full/pyodide.jsThe previous CDNpyodide-cdn2.iodide.iostill works and there are no plans for deprecating it. However please use JsDelivr as a more sustainable solution, including for earlier Pyodide versions.
Python and the standard library#
Pyodide includes CPython 3.8.2 #712
ENH Patches for the threading module were removed in all packages. Importing the module, and a subset of functionality (e.g. locks) works, while starting a new thread will produce an exception, as expected. #796. See #237 for the current status of the threading support.
ENH The multiprocessing module is now included, and will not fail at import, thus avoiding the necessity to patch included packages. Starting a new process will produce an exception due to the limitation of the WebAssembly VM with the following message:
Resource temporarily unavailable#796.
Python / JS type conversions#
pyodide-py package and micropip#
The
pyodide.pyfile was transformed to a pyodide-py package. The imports remain the same so this change is transparent to the users #909.FIX Get last version from PyPI when installing a module via micropip #846.
Suppress REPL results returned by
pyodide.eval_codeby adding a semicolon #876.Enable monkey patching of
eval_codeandfind_importsto customize behavior ofrunPythonandrunPythonAsync#941.
Build system#
Updated docker image to Debian buster, resulting in smaller images. #815
Pre-built docker images are now available as
iodide-project/pyodide#787Host Python is no longer compiled, reducing compilation time. This also implies that Python 3.8 is now required to build Pyodide. It can for instance be installed with conda. #830
FIX Infer package tarball directory from source URL #687
Updated to emscripten 1.38.44 and binaryen v86 (see related commits)
Updated default
--ldflagsargument topyodide_buildscripts to equal what Pyodide actually uses. #817Replace C lz4 implementation with the (upstream) JavaScript implementation. #851
Pyodide deployment URL can now be specified with the
PYODIDE_BASE_URLenvironment variable during build. Thepyodide_dev.jsis no longer distributed. To get an equivalent behavior withpyodide.js, setwindow.languagePluginUrl = "./";
before loading it. #855
Build runtime C libraries (e.g. libxml) via package build system with correct dependency resolution #927
Pyodide can now be built in a conda virtual environment #835
Other improvements#
Modify MEMFS timestamp handling to support better caching. This in particular allows to import newly created Python modules without invalidating import caches #893
Packages#
New packages: freesasa, lxml, python-sat, traits, astropy, pillow, scikit-image, imageio, numcodecs, msgpack, asciitree, zarr
Note that due to the large size and the experimental state of the scipy package, packages that depend on scipy (including scikit-image, scikit-learn) will take longer to load, use a lot of memory and may experience failures.
Updated packages: numpy 1.15.4, pandas 1.0.5, matplotlib 3.3.3 among others.
New package pyodide-interrupt, useful for handling interrupts in Pyodide (see project description for details).
Backward incompatible changes#
Dropped support for loading .wasm files with incorrect MIME type, following #851
List of contributors#
abolger, Aditya Shankar, Akshay Philar, Alexey Ignatiev, Aray Karjauv, casatir, chigozienri, Christian glacet, Dexter Chua, Frithjof, Hood Chatham, Jan Max Meyer, Jay Harris, jcaesar, Joseph D. Long, Matthew Turk, Michael Greminger, Michael Panchenko, mojighahar, Nicolas Ollinger, Ram Rachum, Roman Yurchak, Sergio, Seungmin Kim, Shyam Saladi, smkm, Wei Ouyang
Version 0.15.0#
May 19, 2020
Upgrades Pyodide to CPython 3.7.4.
micropip no longer uses a CORS proxy to install pure Python packages from PyPI. Packages are now installed from PyPI directly.
micropip can now be used from web workers.
Adds support for installing pure Python wheels from arbitrary URLs with micropip.
The CDN URL for Pyodide changed to https://pyodide-cdn2.iodide.io/v0.15.0/full/pyodide.js It now supports versioning and should provide faster downloads. The latest release can be accessed via https://pyodide-cdn2.iodide.io/latest/full/
Adds
messageCallbackanderrorCallbacktopyodide.loadPackage.Reduces the initial memory footprint (
TOTAL_MEMORY) from 1 GiB to 5 MiB. More memory will be allocated as needed.When building from source, only a subset of packages can be built by setting the
PYODIDE_PACKAGESenvironment variable. See partial builds documentation for more details.New packages: future, autograd
Version 0.14.3#
Dec 11, 2019
Convert JavaScript numbers containing integers, e.g.
3.0, to a real Python long (e.g.3).Adds
__bool__method to forJsProxyobjects.Adds a JavaScript-side auto completion function for Iodide that uses jedi.
New packages: nltk, jeudi, statsmodels, regex, cytoolz, xlrd, uncertainties
Version 0.14.0#
Aug 14, 2019
The built-in
sqliteandbz2modules of Python are now enabled.Adds support for auto-completion based on jedi when used in iodide
Version 0.13.0#
May 31, 2019
Tagged versions of Pyodide are now deployed to Netlify.
Version 0.12.0#
May 3, 2019
User improvements:
Packages with pure Python wheels can now be loaded directly from PyPI. See
micropipfor more information.Thanks to PEP 562, you can now
import jsfrom Python and use it to access anything in the global JavaScript namespace.Passing a Python object to JavaScript always creates the same object in JavaScript. This makes APIs like
removeEventListenerusable.Calling
dir()in Python on a JavaScript proxy now works.Passing an
ArrayBufferfrom JavaScript to Python now correctly creates amemoryviewobject.Pyodide now works on Safari.
Version 0.11.0#
Apr 12, 2019
User improvements:
Support for built-in modules:
sqlite,crypt
New packages:
mne
Developer improvements:
The
mkpkgcommand will now select an appropriate archive to use, rather than just using the first.The included version of emscripten has been upgraded to 1.38.30 (plus a bugfix).
New packages:
jinja2,MarkupSafe
Version 0.10.0#
Mar 21, 2019
User improvements:
New packages:
html5lib,pygments,beautifulsoup4,soupsieve,docutils,bleach,mne
Developer improvements:
console.htmlprovides a simple text-only interactive console to test local changes to Pyodide. The existing notebooks based on legacy versions of Iodide have been removed.The
run_dockerscript can now be configured with environment variables.