FROMANNUAL REVIEWS

CogSci 2025

•

August 01, 2025

•

San Francisco, United States

keywords:

semantics of language

psychology

artificial intelligence

linguistics

natural language processing

Large language models segment words into chunks called tokens, using compression algorithms that ignore semantics. We investigated whether tokenization corrupts representations of word meanings in 17 languages. We found that GPT-4o and Llama 3 inflate the similarity of words that share tokens. However, tokens turned out to be good predictors of orthographic priming, such that people recognize a target word faster after reading a prime that ends with the same token. This boost in priming far exceeds what other overlapping strings of letters explain, which suggests that tokenization selectively identifies functional subword units. The pattern extends to the production of word associates in English: Tokens capture phonologically motivated associations, while other strings of letters do not. So, tokenization does influence semantic representations, but because tokens correspond to psychologically salient orthographic and/or phonological constituents, they may endow large language models with human-like language networks and facilitate alignment with human word processing.

Downloads

Paper

Next from CogSci 2025

Experience-driven discovery of planning strategies
poster

Experience-driven discovery of planning strategies

CogSci 2025

Ruiqi He and 1 other author

01 August 2025

Similar lecture

How much Semantic Information is Available in Large Language Model Tokens
technical paper

How much Semantic Information is Available in Large Language Model Tokens

EMNLP 2025

Zhenguang Cai and 1 other author

06 November 2025

Stay up to date with the latest Underline news!

Select topic of interest (you can select more than one)

PRESENTATIONS

  • All Presentations
  • For Librarians
  • Resource Center
  • Free Trial
Underline Science, Inc.
1216 Broadway, 2nd Floor, New York, NY 10001, USA

© 2026 Underline - All rights reserved

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.