Lecture image placeholder

Premium content

Access to this content requires a subscription. You must be a premium user to view this content.

Monthly subscription - $9.99Pay per view - $4.99Access through your institutionLogin with Underline account
Need help?
Contact us
Lecture placeholder background
VIDEO DOI: https://doi.org/10.48448/mge7-z941

poster

ACL 2024

August 12, 2024

Bangkok, Thailand

On the Representational Capacity of Neural Language Models with Chain-of-Thought Reasoning

keywords:

formal language

chain of thought

rnn

language model

transformer

The performance of modern language models (LMs) has been improved by chain-of-thought (CoT) reasoning, i.e., the process of generating intermediate results that guide the model towards a final answer. A possible explanation for this improvement is that CoT reasoning extends an LM’s computational power, as RNNs and transformers with additional scratch space are known to be Turing complete. Comparing LMs to Turing machines, however, introduces a category error—Turing machines decide language membership, whereas LMs define distributions over strings. To bridge this gap, we formalize CoT reasoning in a probabilistic setting. We present several results on the representational capacity of recurrent and transformer LMs with CoT reasoning, showing that they can represent the same family of distributions over strings as probabilistic Turing machines.

Downloads

SlidesTranscript English (automatic)

Next from ACL 2024

Cross-Modal Projection in Multimodal LLMs Doesn’t Really Project Visual Attributes to Textual Space
poster

Cross-Modal Projection in Multimodal LLMs Doesn’t Really Project Visual Attributes to Textual Space

ACL 2024

+3Srijan KumarGaurav Verma
Gaurav Verma and 5 other authors

12 August 2024

Stay up to date with the latest Underline news!

Select topic of interest (you can select more than one)

PRESENTATIONS

  • All Lectures
  • For Librarians
  • Resource Center
  • Free Trial
Underline Science, Inc.
1216 Broadway, 2nd Floor, New York, NY 10001, USA

© 2023 Underline - All rights reserved