Recently, domain-general recurrent neural networks, 
without explicit linguistic inductive biases, 
have been shown to successfully reproduce a range 
of human language behaviours, 
such as accurately predicting number agreement 
between nouns and verbs. 
We show that such networks will also learn number agreement 
within unnatural sentence structures, i.e. structures that 
are not found within any natural languages and 
which humans struggle to process. 
These results suggest that the models are learning 
from their input in a manner that is substantially 
different from human language acquisition, 
and we undertake an analysis of how the learned 
knowledge is stored in the weights of the network. 
We find that while the model has an effective understanding 
of singular versus plural for individual sentences, 
there is a lack of a unified concept of number agreement 
connecting these processes across the full range of inputs. 
Moreover, the weights handling natural and unnatural structures 
overlap substantially, in a way that underlines the 
non-human-like nature of the knowledge learned by the network.

Priorless Recurrent Networks Learn Curiously

LONG35: Machine Learning 3

technical paper

COLING, the International Conference on Computational Linguistics, is one of the premier conferences for natural language processing and computational linguistics. Often grouped within the field of artificial intelligence, but actually pre-dating the development of artificial intelligence, advances in computational linguistics and natural language processing are now some of the major drivers behind the use of artificial intelligence for commercial and social applications – for example, on-line search, machine translation and with voice-assisted conversational devices.

First established in 1965, the biennial COLING conference is held in diverse parts of the globe and attracts participants from both top-ranked research centers and emerging countries. Today, the most important developments in our field are taking place not only in universities and academic research institutes, but also in industrial research departments and in technological startups. COLING conferences provide opportunities for all these communities to showcase their exciting developments.

COLING 2020

COLING, the International Conference on Computational Linguistics, is one of the premier conferences for natural language processing and computational linguistics. Often grouped within the field of artificial intelligence, but actually pre-dating the development of artificial intelligence, advances in computational linguistics and natural language processing are now some of the major drivers behind the use of artificial intelligence for commercial and social applications – for example, on-line search, machine translation and with voice-assisted conversational devices.

Jeff Mitchell

1

2

SHORT BIO

Presentations

Priorless Recurrent Networks Learn Curiously

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES