Blog

LightOn Pulse #2: Discover the New Paradigm Platform Updates

Version Glorious-Giraffe 20240828 Release Date: August 28, 2024

PyLate: Flexible Training and Retrieval for ColBERT Models

We release PyLate, a new user-friendly library for training and experimenting with ColBERT models, a family of models that exhibit strong retrieval capabilities on out-of-domain data.

ArabicWeb24: Creating a high quality Arabic Web-only pre-training dataset

LightOn Pulse #1

Innovations and Advancements (2024.07)

Training Mamba Models on AMD MI250/MI250X GPUs with Custom Kernels

In this blogpost we show how we can train a Mamba model interchangeably on both NVIDIA and AMD and we compare both training performance and convergence in both cases. This shows that our training stack is becoming more GPU-agnostic.

Transforming LLMs into Agents for Enterprise Automation

Developing Agentic Capabilities for LLMs to automate business workflows and create smart assistants.

Passing the Torch: Training a Mamba Model for Smooth Handover

We present our explorations on training language models based on the new Mamba architecture, which deviates from the traditional Transformer architecture.

LightOn AI Meetup Creating a Large Dataset for Pretraining LLMs

Partnership LightOn & Orange Business

Orange Business, LightOn, and HPE partner to offer trusted AI generative solutions