# Multi-scale Pretraining

OpenELM is a series of efficient language models introduced by Apple, utilizing a hierarchical scaling strategy to optimize parameter allocation, offering pretrained and instruction-tuned models ranging from 270M to 3B parameters.

FlauBERT is a French BERT model pretrained on a large-scale French corpus, developed by the French National Centre for Scientific Research (CNRS), offering different versions to accommodate various needs.