Embedder for refinery. Manages the creation of document- and token-level embeddings using the embedders library.

Gateway for refinery. Manages incoming requests and holds the workflow logic. To interact with the gateway, the UI or Python SDK can be used.

Data model for refinery. Manages entities and their access for multiple services, e.g. the gateway.

Scripts used for Kern AI CI/CD efforts

Defines parent image for the Docker images of the refinery services that require torch (gpu).

Weak supervision for refinery. Manages the integration of heuristics such as labeling functions, active learners or zero-shot classifiers. Uses the weak-nlp library for the actual integration logic and algorithms.

Updater for refinery. Manages migration logic to new versions if required.

Tokenizer for refinery. Manages the creation and storage of spaCy tokens for text-based record attributes and supports multiple language models. It is used by the gateway.