SLM4IE¶
SLM4IE develops small language models (SLMs) for zero-shot information extraction across European languages, with emphasis on Slovenian.
Why SLM4IE¶
The project targets three limitations of current LLMs:
- Compute cost — LLMs require infrastructure beyond the reach of smaller organizations for local deployment.
- Low-resource gaps — limited training data for sensitive domains and underrepresented languages.
- Output inconsistency — unreliable structured extraction from generative models.
We build computationally efficient models optimized for commodity hardware, create multilingual benchmark datasets for sensitive domains, and evaluate against existing SLMs and LLMs. All artifacts (models, datasets, code) will be released publicly where possible.
Project status¶
The workstreams are at different stages of maturity, and this site is explicit about which is which. Pages for workflows that have not yet been run end-to-end carry a 🚧 In progress banner.
| Area | Status |
|---|---|
| Download → Extract → Pretraining corpus | Verified — run end-to-end; see Corpus Statistics |
| Task converters (NER, sentiment, SuperGLUE) | Implemented, not yet run |
| Tokenizer comparison sweep | Implemented, not yet run |
| Model training & evaluation | Scaffolded — entry points stubbed, not yet implemented |
Workstreams¶
- Data pipeline — download, extract, and curate Slovenian corpora into a pretraining corpus and into IE training shapes.
- Datasets — catalog of pretraining corpora and downstream benchmarks (NER, SA, SuperGLUE, tokenizer evaluation), plus statistics of the curated corpus.
- Tokenizers — train and score six tokenizers across a vocab sweep for Slovenian morphology.
- Training and Evaluation — pretrain or fine-tune SLMs from YAML configs, with MLflow tracking and SLURM batch scripts.
Acknowledgments¶
The project is funded by ARIS (Slovenian Research and Innovation Agency) under the project number Z2-70067.
