Biblio Discovery — Demo Discovery Service

2 hasil untuk "Burchell, Laurie"

Menampilkan 1–2 · 0,11 detik · RSS

  1. Preprint Teks lengkap

    HPLT 3.0: Very Large-Scale Multilingual Resources for LLMs and MT. Mono- and Bi-lingual Data, Multilingual Evaluation, and Pre-Trained Models

    Oepen, Stephan; Arefev, Nikolay; Aulamo, Mikko dkk.

    arXiv · 2025 · Inggris · arXiv

    We present an ongoing initiative to provide open, very large, high-quality, and richly annotated textual datasets for almost 200 languages. At 30 trillion tokens, this is likely the largest generally available multilingual collection of LLM pre-training data. These datasets are derived from web…

  2. Preprint Teks lengkap

    CommonLID: Re-evaluating State-of-the-Art Language Identification Performance on Web Data

    Suarez, Pedro Ortiz; Burchell, Laurie; Arnett, Catherine dkk.

    arXiv · 2026 · Inggris · arXiv

    Language identification (LID) is a fundamental step in curating multilingual corpora. However, LID models still perform poorly for many languages, especially on the noisy and heterogeneous web data often used to train multilingual language models. In this paper, we introduce CommonLID, a…