This is the github for Hyperparam, where we share open-source contributions to the AI and Data Engineering communities. AI needs lots of data, so we're building tools for working with massive text datasets in the browser.
Parquet file parser for loading datasets in the browser.
https://github.com/hyparam/hyparquet
Parquet file writer in JavaScript.
https://github.com/hyparam/hyparquet-writer
Decompress every parquet compression format.
https://github.com/hyparam/hyparquet-compressors
Snappy decompression optimized with WebAssembly for faster parquet parsing.
https://github.com/hyparam/hysnappy
Windowed table component for viewing arbitrarily large datasets.
https://github.com/hyparam/hightable
Apache Iceberg table reader in JavaScript.
https://github.com/hyparam/icebird
Squirreling Async SQL Engine for querying large datasets in the browser.
https://github.com/hyparam/squirreling
Parse metadata from llama.cpp gguf files in JavaScript.
https://github.com/hyparam/hyllama
Scalable dataset viewer for machine learning datasets.