Contextual predictability drives both word form and word choice in language use. The effects of the predictability of a word given its previous context are generally well-understood in both production and comprehension, but studies of naturalistic production have also revealed a poorly-understood backward predictability effect of a word given its future context, which may be related to planning. In this study, we revisit backward predictability effects using improved measures and more powerful language models, and introduce a principled measure of planning based on the pointwise mutual information between the word and the future context after controlling for the effects of previous context. We evaluate both measures for predicting word duration, and then extend the scope of these effects to a novel paradigm that involves predicting substitution errors in naturalistic productions. Our findings reveal that the proposed PMI-based measure of planning performs comparably to backward predictability. This analysis provides a useful test-bed for probing the link between past and future context predictability and underlying cognitive processes.

Keywords: Language production; Information-theoretic linguistics; Corpus Research;