Biblio Discovery — Demo Discovery Service

18 results for "Augenstein, Isabelle"

Showing 1–18 · 0.20 seconds · RSS

  1. Preprint Full text

    Understanding the Interplay between LLMs' Utilisation of Parametric and Contextual Knowledge: A keynote at ECIR 2025

    Augenstein, Isabelle

    arXiv · 2026 · English · arXiv

    Language Models (LMs) acquire parametric knowledge from their training process, embedding it within their weights. The increasing scalability of LMs, however, poses significant challenges for understanding a model's inner workings and further for updating or correcting this embedded knowledge…

  2. Journal Article Full text

    Aggregating soft labels from crowd annotations improves uncertainty estimation under distribution shift

    Wright, Dustin; Augenstein, Isabelle

    PLoS One · 2025 · English · PubMed Central

    Selecting an effective training signal for machine learning tasks is difficult: expert annotations are expensive, and crowd-sourced annotations may not be reliable. Recent work has demonstrated that learning from a distribution over labels acquired from crowd annotations can be effective both for…

  3. Journal Article Full text

    Quantifying gender biases towards politicians on Reddit

    Marjanovic, Sara; Stańczak, Karolina; Augenstein, Isabelle

    PLoS One · 2022 · English · PubMed Central

    Despite attempts to increase gender parity in politics, global efforts have struggled to ensure equal female representation. This is likely tied to implicit gender biases against women in authority. In this work, we present a comprehensive study of gender biases that appear in online political…

  4. Journal Article Full text

    Quantifying gender bias towards politicians in cross-lingual language models

    Stańczak, Karolina; Ray Choudhury, Sagnik; Pimentel, Tiago et al.

    PLoS One · 2023 · English · PubMed Central

    Recent research has demonstrated that large pre-trained language models reflect societal biases expressed in natural language. The present paper introduces a simple method for probing language models to conduct a multilingual study of gender bias towards politicians. We quantify the usage of…

  5. Preprint Full text

    Multi-Step Knowledge Interaction Analysis via Rank-2 Subspace Disentanglement

    Islam, Sekh Mainul; Atanasova, Pepa; Augenstein, Isabelle

    arXiv · 2025 · English · arXiv

    Natural Language Explanations (NLEs) describe how Large Language Models (LLMs) make decisions by drawing on external Context Knowledge (CK) and Parametric Knowledge (PK). Understanding the interaction between these sources is key to assessing NLE grounding, yet these dynamics remain underexplored…

  6. Preprint Full text

    Stress Testing Factual Consistency Metrics for Long-Document Summarization

    Mujahid, Zain Muhammad; Wright, Dustin; Augenstein, Isabelle

    arXiv · 2025 · English · arXiv

    Evaluating the factual consistency of abstractive text summarization remains a significant challenge, particularly for long documents, where conventional metrics struggle with input length limitations and long-range dependencies. In this work, we systematically evaluate the reliability of six…

  7. Preprint Full text

    Show me the evidence: Evaluating the role of evidence and natural language explanations in AI-supported fact-checking

    Warren, Greta; Sun, Jingyi; Shklovski, Irina et al.

    arXiv · 2026 · English · arXiv

    Although much research has focused on AI explanations to support decisions in complex information-seeking tasks such as fact-checking, the role of evidence is surprisingly under-researched. In our study, we systematically varied explanation type, AI prediction certainty, and correctness of AI…

  8. Preprint Full text

    Explaining Sources of Uncertainty in Automated Fact-Checking

    Sun, Jingyi; Warren, Greta; Shklovski, Irina et al.

    arXiv · 2025 · English · arXiv

    Understanding sources of a model's uncertainty regarding its predictions is crucial for effective human-AI collaboration. Prior work proposes using numerical uncertainty or hedges ("I'm not sure, but ..."), which do not explain uncertainty that arises from conflicting evidence, leaving users unable…

  9. Preprint Full text

    Do We Still Need Humans in the Loop? Comparing Human and LLM Annotation in Active Learning for Hostility Detection

    Hakimi, Ahmad Dawar; Hirlimann, Lea; Augenstein, Isabelle et al.

    arXiv · 2026 · English · arXiv

    Instruction-tuned LLMs can annotate thousands of instances from a short prompt at negligible cost. This raises two questions for active learning (AL): can LLM labels replace human labels within the AL loop, and does AL remain necessary when entire corpora can be labelled at once? We investigate…

  10. Preprint Full text

    Analysing Differences in Persuasive Language in LLM-Generated Text: Uncovering Stereotypical Gender Patterns

    Pauli, Amalie Brogaard; Barrett, Maria; Müller-Eberstein, Max et al.

    arXiv · 2026 · English · arXiv

    Large language models (LLMs) are increasingly used for everyday communication tasks, including drafting interpersonal messages intended to influence and persuade. Prior work has shown that LLMs can successfully persuade humans and amplify persuasive language. It is therefore essential to understand…

  11. Preprint Full text

    Reliable Evaluation Protocol for Low-Precision Retrieval

    Yang, Kisu; Jang, Yoonna; Jang, Hwanseok et al.

    arXiv · 2025 · English · arXiv

    Lowering the numerical precision of model parameters and computations is widely adopted to improve the efficiency of retrieval systems. However, when computing relevance scores between the query and documents in low-precision, we observe spurious ties due to the reduced granularity. This introduces…

  12. Preprint Full text

    Evaluation Framework for Highlight Explanations of Context Utilisation in Language Models

    Sun, Jingyi; Atanasova, Pepa; Choudhury, Sagnik Ray et al.

    arXiv · 2025 · English · arXiv

    Context utilisation, the ability of Language Models (LMs) to incorporate relevant information from the provided context when generating responses, remains largely opaque to users, who cannot determine whether models draw from parametric memory or provided context, nor identify which specific…

  13. Preprint Full text

    BiasGym: A Simple and Generalizable Framework for Analyzing and Removing Biases through Elicitation

    Islam, Sekh Mainul; Borenstein, Nadav; Pawar, Siddhesh Milind et al.

    arXiv · 2025 · English · arXiv

    Understanding biases and stereotypes encoded in the weights of Large Language Models (LLMs) is crucial for developing effective mitigation strategies. However, biased behaviour is often subtle and non-trivial to isolate, even when deliberately elicited, making systematic analysis and debiasing…

  14. Preprint Full text

    CUB: Benchmarking Context Utilisation Techniques for Language Models

    Hagström, Lovisa; Kim, Youna; Yu, Haeun et al.

    arXiv · 2025 · English · arXiv

    Incorporating external knowledge is crucial for knowledge-intensive tasks, such as question answering and fact checking. However, language models (LMs) may ignore relevant information that contradicts outdated parametric memory or be distracted by irrelevant contexts. While many context utilisation…

  15. Preprint Full text

    Entangled in Representations: Mechanistic Investigation of Cultural Biases in Large Language Models

    Yu, Haeun; Jeong, Seogyeong; Pawar, Siddhesh et al.

    arXiv · 2025 · English · arXiv

    The growing deployment of large language models (LLMs) across diverse cultural contexts necessitates a deeper understanding of LLMs' representations of different cultures. Prior work has focused on evaluating the cultural awareness of LLMs by only examining the text they generate. This approach…

  16. Journal Article Full text

    The LEADING guideline: Reporting standards for expert panel, best-estimate diagnosis, and longitudinal expert all data (LEAD) methods

    Eijsbroek, Veerle C.; Kjell, Katarina; Andrew Schwartz, H. et al.

    Compr Psychiatry · 2025 · English · PubMed Central

    Accurate assessments of symptoms and illnesses are essential for health research and clinical practice but face many challenges. The absence of a single error-free measure is currently addressed by assessment methods involving experts reviewing several sources of information to achieve a…

  17. Journal Article Full text

    The LEADING Guideline

    Eijsbroek, Veerle C; Kjell, Katarina; Schwartz, H Andrew et al.

    medRxiv · 2024 · English · PubMed Central

    BACKGROUND: Accurate assessments of symptoms and illnesses are essential for health research and clinical practice but face many challenges. The absence of a single error-free measure is currently addressed by assessment methods involving experts reviewing several sources of information to achieve…

  18. Preprint Full text

    Epistemic Diversity and Knowledge Collapse in Large Language Models

    Wright, Dustin; Masud, Sarah; Moore, Jared et al.

    arXiv · 2025 · English · arXiv

    Large language models (LLMs) tend to generate homogenous texts, which may impact the diversity of knowledge generated across different outputs. Given their potential to replace existing forms of knowledge acquisition, this poses a risk of knowledge collapse, where homogenous LLMs may lead most…