Checkout OpenELM, a new efficient language model family that optimizes parameters for accuracy with fewer tokens using layer-wise scaling! Training code is on Github, models are also on HuggingFace.