Canada

User-generated ratings — often elicited and presented as “star ratings” — have become a ubiquitous feature of the online consumer experience. While most research agrees that these user-generated ratings influence individual consumer decisions and overall consumer demand, there is less consensus as to whether user- generated ratings help consumers make better, welfare-enhancing decisions. In this manuscript, we expound on an intrinsic problem with the use of user-generated ratings in product choice decisions. Specifically, product ratings are typically given in an isolated (non-comparative) context, but are typically used in a comparative context, where relative differences in ratings may not reflect relative differences in quality. We provide a simple empirical demonstration of how this structural misalignment can lead consumers to choose suboptimal products and, ultimately, yield reduced consumer welfare.

CogSci 2022

User-Generated Star Ratings Are Not Inherently Comparable: How Star Ratings Structure Leads to Poor Choices

poster

Join us for CogSci2022! Our 44th Annual Conference will be centered around the theme of diversity, with an accompanying range of keynotes and invited speakers who themselves highlight the diversity of our field. Our three symposia are Cognition across Diverse Populations, Cognition within Diverse Societies, and Global Challenges for Cognitive Science. The conference is being offered in a fully hybrid environment for onsite and online attendees.
 
 [**The Program at a Glance can be found here**](https://cognitivesciencesociety.org/wp-content/uploads/2022/07/Cogsci-2022-PAAG-Detailed-July-12final-1.pdf)
 [**The Detailed Program can be found here**](https://cognitivesciencesociety.org/wp-content/uploads/2022/08/DeArmond-CogSci2022-program-v113-final.pdf)

This conference requires a registration to attend. Please use the link below to secure your ticket. The website will be available to all registered attendees on July 25th. 

[Register to Attend](https://cognitivesciencesociety.org/registration/)

Registration Is Required

The 44th Annual Meeting of the Cognitive Science Society highlighted research on the theme Cognitive Diversity in addition to the full breadth and diversity of research topics offered by the society’s membership. 





Decision-making is commonly impaired in those with cognitive decline. I compared individuals with Mild Cognitive Impairment (MCI; n = 45) and cognitively healthy older adults (n = 45) in their ability to improve their decision-making in the Iowa Gambling Task over time. On each trial, participants chose from one of four decks of cards and received a net gain or loss. The four decks differed in their expected value (EV). They continued choosing among the decks for one hundred trials, trying to maximize their winnings. Results indicate that both groups learned to choose from the higher EV decks over time, but those with MCI learned more slowly. The data were fit to a reinforcement learning model that indicated those with MCI performed worse in part because of task strategy: those with MCI were less likely to perseverate on wins and updated their beliefs more slowly to losses.

Individuals with Mild Cognitive Impairment are impaired in a gambling task

Procrastination is universal but not monolithic; a second dimension, the Intention–Behavior Gap (IBG), has been suggested to be relevant. The IBG is the difference between one’s intentions and one’s behavior. So far, the IBG has not been directly measured within a behavioral experiment. We assigned subjects a week to work on a lengthy online reading task. Before the task started, subjects reported their plan by the number of reading units for each day. We found great variability in their plans, actual time course and the difference between the two (IBG). The time course of work ranged from completing earlier than planned to not finishing. We calculated correlations between the mean unit completion day, the IBG, the Causal Dimension Scale, indices of regret and satisfaction, and scales for procrastination (General, Active and Irrational). Besides the correlations expected based on definitions, we did not find any significant correlations.

Procrastination and the Intention-Behavior Gap

The propensity for people to avoid mentally demanding tasks in the absence of reward is well documented. As a result, humans are often described as cognitive misers. This characterisation, while consistent with the psychological literature, contradicts everyday instances of effort being sought: reading, board games, and brain-teasing puzzles. Such examples however are markedly different from the types of tasks typically used in the mental effort literature (e.g., working memory tasks, demand selection tasks). The current set of experiments assessed whether the type of task (i.e., N-Back, Number Sequence Problems [NSP], or Anagrams) affects people’s aversion to, or desire for, increased effort. On average, across 3 experiments, participants showed an aversion to effort regardless of whether the effort required was more attentional (N-Back) or cognitive (NSP and anagrams) in nature, and were willing to forgo financial reward in order to avoid more difficult tasks. A minority of participants, however, sought more effortful tasks for equal or lesser reward.

Does Mental Effort Avoidance Depend on the ‘Type of Effort’?

It may seem pointless to compare two products with the exact same average rating and total number of reviews without other review information. Now imagine a scenario in which the distribution of star ratings is also available to decision makers in addition to these two attributes. Will the decision still be uncertain as it is before or the distributions of stars will engender a preference towards one of the products? To answer this question, the current study used variability of star ratings as an approximation of a product’s distribution. The behavioral studies showed that participants exhibited distinctive choice patterns when the distribution of ratings was provided even when the average rating and total number of reviews were the same between two products involved in a comparison. A utility-based cognitive model was therefore developed to identify the underlying mechanism as to why people chose the way they did.

How Do People Use Star Rating Distributions?

We investigate the way people judge how helpful a review is in informing their decision as to whether to make a purchase. In particular, we are interested in how the summary statistics an individual sees influences judgments of a review’s helpfulness. We find perceived helpfulness of a given review decreases as the star rating of that review gets further from the mean rating. Additionally, participants were more likely to search for reviews close to the mean. Both of these findings are consistent with confirmation bias. We explore, but do not find support for, alternative possible explanations.

The Influence of Mean Product Ratings on Review Judgments and Search

Choice blindness is a puzzling phenomenon, showing how people often fail to detect a mismatch between their intentions and outcomes and then confabulate reasons for choices they never made. Despite extensive replications, little is known about the cognitive mechanisms underpinning it. To tackle this question, we report an online experiment showing for the first time that people still illusorily self-attribute unchosen outcomes despite being explicitly instructed to detect false feedbacks. Comparing standard implicit choice blindness with our new explicit task, we show how prior beliefs about manipulations and stimuli discriminability influence detection. Building on these results, we sketch a preliminary Bayesian model of choice blindness suggesting that choice blindness occurs when beliefs in the reliability of feedback override monitoring related cues indicative of a mismatched outcome. Finally, we discuss practical advantages of explicit task to study choice blindness and implications for the sense of agency and self-knowledge.


Choice blindness persists despite explicit instructions to detect false feedbacks: preliminary evidence for a Bayesian model of choice blindness

Expectations play a critical role in human perception, cognition, and decision-making. There has been a recent surge in modelling such expectations and the resulting behaviour when they are violated. One prominent recent proposal is the ViolEx model. To move the model & the literature forward, we identified three areas of concern and addressed two in this study - Lack of formalization and implementation. Specifically, we aim to provide the first implementation of ViolEx using the Active Inference formalism by leveraging the rich overlap between them. As a result, we successfully simulated all expectation violation coping strategies of ViolEx and analyzed their potential mediators. Through this analysis, we identified novel predictors of immunization & assimilation and documented the computational mechanism of their mediation. We see this as the first step in developing a formal research framework to study expectation violations and hope to serve as a base for future ViolEx studies.

Formalization and Implementation of ViolEx: An Active Inference perspective

We test whether people flexibly shift their sampling strategy for learning a functional relationship, based on the strategy’s perceived effectiveness. While general-purpose heuristics such as gathering information evenly across the environment may often approximate optimal sampling policies when opportunities for learning are sparse, these strategies may systematically fail when much of the environment is uninformative. Across several different classes of arbitrary smooth functions, participants (N = 89) made sampling choices that were initially consistent with a simple heuristic, but shifted their sampling strategy when this heuristic failed to be informative. People were subsequently more accurate at approximating the true function for smoother functions that can be more easily predicted by sampling evenly, t(645.3) = 6.803, p < .001; nevertheless, while individuals vary considerably, aggregate judgments across all functions are very accurate, suggesting that people show inductive biases in active learning, but effectively learn when to adjust their policies.

Dynamic Strategy Selection in Active Function Learning

Sequential sampling models have provided accurate accounts of people’s choice, response time, and preference strength in value-based decision-making tasks. Conventionally, these models are developed as Markov-type processes (such as random walks or diffusion processes) following the Kolmogorov axioms. Quantum probability theory has been proposed as an alternative framework upon which to develop models of cognition, including quantum random walk models. When modeling people's behavior during decision-making tasks, previous work has demonstrated that both the Markov and quantum models have their respective strengths. Recently, the open system model, which is a hybrid version of the Markov and quantum models, has been shown to provide a more accurate account of preference strength compared to the Markov and quantum models in isolation. In this work, we extend the open system model to make predictions on pairwise choice and response time and compare it to the Markov and quantum random walk models.


Open System Model of Choice and Response Time

The direction of our attention can influence our performance on a variety of tasks. For example, reading from left to right relates to people associating small numbers on the left and large numbers on the right. In contrast, reading from right to left relates to people associating small numbers on the right and large numbers on the left. The current study tests if this type of “attentional momentum” can be induced by storytelling based on pictures and whether it affects college students’ reaction time on an arithmetic verification task with equations in a traditional (e.g., 2+2=4) or non-traditional (e.g., 4=2+2) direction. Our results show that students were faster at verifying simple traditional math problems after telling stories based on pictures arranged from left to right, but faster at verifying simple non-traditional math problems after telling stories based on pictures arranged from right to left.


Premium content

User-Generated Star Ratings Are Not Inherently Comparable: How Star Ratings Structure Leads to Poor Choices

Next from CogSci 2022

Individuals with Mild Cognitive Impairment are impaired in a gambling task

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES