Skip to content

Add the numcodecs-wasm package#455

Open
juntyr wants to merge 6 commits intopyodide:mainfrom
juntyr:numcodecs-wasm
Open

Add the numcodecs-wasm package#455
juntyr wants to merge 6 commits intopyodide:mainfrom
juntyr:numcodecs-wasm

Conversation

@juntyr
Copy link
Copy Markdown
Contributor

@juntyr juntyr commented Jan 7, 2026

  • numcodecs-wasm is the root package for my https://numcodecs-wasm.readthedocs.io/en/latest/ project, which provides several compression algorithms compiled to WASM. numcodecs-wasm provides the extension module to run the compressors, while all other packages, e.g. numcodecs-wasm-sz3, are pure-Python packages that can be installed on-demand with micropip
  • numcodecs-combinators is a helper package for writing meta-compressors with numcodecs
  • numcodecs-observers is a meta-compressor for recording performance stats during compression, numcodecs-wasm hooks into it to provide an additional performance stat (an instruction count measured inside WASM)
  • varint is required by numcodecs-combinators

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Jan 7, 2026

Package Build Results

Total packages built: 35
Total build time: 0:07:32

Package Build Times (click to expand)
Package Build Time
libopenssl 6m 48s
numpy 6m 23s
numcodecs-wasm 6m 12s
sqlite3 3m 43s
liblzma 1m 54s
numcodecs 1m 6s
test 25s
regex 11s
wrapt 9s
lzma 9s
pydoc_data 7s
MarkupSafe 7s
pydecimal 4s
atomicwrites 3s
hashlib 3s
pytz 2s
pluggy 1s
py 1s
typing-extensions 1s
numcodecs-combinators 1s
attrs 1s
micropip 1s
more-itertools 1s
pytest 1s
iniconfig 1s
deprecated 1s
tblib 1s
setuptools 1s
exceptiongroup 1s
pyparsing 1s
numcodecs-observers 1s
pytest-asyncio 1s
Jinja2 0s
leb128 0s
six 0s

Longest build: libopenssl (6m 48s)
Packages built in more than 10 minutes: 0

@hoodmane
Copy link
Copy Markdown
Member

hoodmane commented Jan 7, 2026

Could you add tests?

@juntyr
Copy link
Copy Markdown
Contributor Author

juntyr commented Jan 8, 2026

Could you add tests?

Absolutely! The easiest way to test would likely be to use micropip to install (so importantly we wouldn't need to store the WASM file in this repo since it's hosted on PyPi) one of the actual codecs from PyPi and let that encode + decode some data, which exercises all code in numcodecs-wasm. Does that sound good?

@ryanking13
Copy link
Copy Markdown
Member

Absolutely! The easiest way to test would likely be to use micropip to install (so importantly we wouldn't need to store the WASM file in this repo since it's hosted on PyPi) one of the actual codecs from PyPi and let that encode + decode some data, which exercises all code in numcodecs-wasm. Does that sound good?

I would prefer making no network requests in our unittest, but if there are no small enough codecs you can use for testing, then yes, that approach would be fine.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants