United States

The role played by YouTube&#39;s recommendation algorithm in unwittingly promoting misinformation and conspiracy theories is not entirely understood. Yet, this can have dire real-world consequences, especially when pseudoscientific content is promoted to users at critical times, such as the COVID-19 pandemic. In this paper, we set out to characterize and detect pseudoscientific misinformation on YouTube. We collect 6.6K videos related to COVID-19, the Flat Earth theory, as well as the anti-vaccination and anti-mask movements. Using crowdsourcing, we annotate them as pseudoscience, legitimate science, or irrelevant and train a deep learning classifier to detect pseudoscientific videos with an accuracy of 0.79.

We quantify user exposure to this content on various parts of the platform and how this exposure changes based on the user&#39;s watch history. We find that YouTube suggests more pseudoscientific content regarding traditional pseudoscientific topics (e.g., flat earth, anti-vaccination) than for emerging ones (like COVID-19). At the same time, these recommendations are more common on the search results page than on a user&#39;s homepage or in the recommendation section when actively watching videos. Finally, we shed light on how a user&#39;s watch history substantially affects the type of recommended videos.

ICWSM 2022

&quot;It is just a flu&quot;: Assessing the Effect of Watch History on YouTube&#39;s Pseudoscientific Video Recommendations

recommendation algorithm audits

youtube's recommendation algorithm

pseudoscience

youtube

machine learning

**ICWSM-2022 will be held from June 6th – 9th in a hybrid format, both in-person in Atlanta, Georgia and online.**

**The Workshop and Tutorial Sessions are accessible to those that registered separately for these sessions. If you registered to attend but are not able to access the these sessions from the [Schedule tab](https://underline.io/events/291/schedule?day=2022-06-06T04%3A00%3A00.000Z), please email icwsm@underline.io
**

The International AAAI Conference on Web and Social Media (ICWSM) is a forum for researchers from multiple disciplines to come together to share knowledge, discuss ideas, exchange information, and learn about cutting-edge research in diverse fields with the common theme of online social media. This overall theme includes research in new perspectives in social theories, as well as computational algorithms for analyzing social media. ICWSM is a singularly fitting venue for research that blends social science and computational approaches to answer important and challenging questions about human social behavior through social media while advancing computational tools for vast and unstructured data.


ICWSM, now in its sixteenth year, has become one of the premier venues for computational social science, and previous years of ICWSM have featured papers, posters, and demos that draw upon network science, machine learning, computational linguistics, sociology, communication, and political science. The uniqueness of the venue and the quality of submissions have contributed to a rapidly growing conference, and a competitive acceptance rate of approximately 20% for full-length research papers published in the proceedings by the Association for the Advancement of Artificial Intelligence (AAAI).


**ICWSM Code of Conduct**

All persons, organizations and entities that attend AAAI conferences and events are subject to the standards of conduct set forth on the [AAAI Code of Conduct for Events and Conferences](https://aaai.org/Conferences/code-of-conduct.php). AAAI expects all community members to formally endorse this code of conduct, and to actively prevent and discourage any undesired behaviors. Everyone should feel empowered to politely engage when they or others are disrespected, and to raise awareness and understanding of this code of conduct. AAAI event participants asked to stop their unacceptable behavior are expected to comply immediately. Sponsors are also subject to this code of conduct in their participation in AAAI events.


Additionally, participants are encouraged to be courteous when sharing screen captures and photographs of conference events. Seek permission when possible and respect requests to take down images if those featured ask. Concerns around code of conduct or inclusion may be sent to icwsm22@aaai.org. If you have any concerns or items to report, please reach out to the General Chairs: Diyi Yang and Yelena Mejova.<br><br>


A ticket is required to attend the event, please use the below URL to register for the event if you have not already done so:

https://aaaiconf.cventevents.com/event/be8c26b0-a76f-4753-babb-de2bd9d2a619/summary

Virtual-Only:
$200 AAAI Member
$125 AAAI Student Member
$225 Nonmember
$150 Nonmember Student

Virtual Silver: Includes discounted conference registration, plus a one-year online new or renewed membership in AAAI.
$299 Silver Regular
$174 Silver Student

Virtual - Workshop and Tutorials:
$45 Regular
$25 Student


Registration Is Required

The International AAAI Conference on Web and Social Media (ICWSM) is a forum for researchers from multiple disciplines to come together to share knowledge, discuss ideas, exchange information, and learn about cutting-edge research in diverse fields with the common theme of online social media. This overall theme includes research in new perspectives in social theories, as well as computational algorithms for analyzing social media. ICWSM is a singularly fitting venue for research that blends social science and computational approaches to answer important and challenging questions about human social behavior through social media while advancing computational tools for vast and unstructured data.

The role played by YouTube's recommendation algorithm in unwittingly promoting misinformation and conspiracy theories is not entirely understood. Yet, this can have dire real-world consequences, especially when pseudoscientific content is promoted to users at critical times, such as the COVID-19 pandemic. In this paper, we set out to characterize and detect pseudoscientific misinformation on YouTube. We collect 6.6K videos related to COVID-19, the Flat Earth theory, as well as the anti-vaccination and anti-mask movements. Using crowdsourcing, we annotate them as pseudoscience, legitimate science, or irrelevant and train a deep learning classifier to detect pseudoscientific videos with an accuracy of 0.79.

We quantify user exposure to this content on various parts of the platform and how this exposure changes based on the user's watch history. We find that YouTube suggests more pseudoscientific content regarding traditional pseudoscientific topics (e.g., flat earth, anti-vaccination) than for emerging ones (like COVID-19). At the same time, these recommendations are more common on the search results page than on a user's homepage or in the recommendation section when actively watching videos. Finally, we shed light on how a user's watch history substantially affects the type of recommended videos.

"It is just a flu": Assessing the Effect of Watch History on YouTube's Pseudoscientific Video Recommendations

technical paper

Facebook users interact with algorithms every day. These algorithms can perpetuate harm via incongruent targeted ads, echo chambers, or "rabbit hole" recommendations. Education around the machine learning (ML) behind Facebook (FB) can help users to point out algorithmic bias and harm, and advocate for themselves effectively when things go wrong. One algorithm that FB users interact with regularly is User-Based Collaborative Filtering (UB-CF) which provides the basis for ad recommendation. We contribute a novel research approach for teaching users about a commonly used algorithm in machine learning in real-world context -- an instructive web application using real examples built from the user's own FB data on ad interests. The instruction also prompts users to reflect on their interactions with ML systems, specifically Facebook. In a between-subjects design, we tested both Data Science Novices and Experts on the efficacy of the UB-CF instruction. Taking care to highlight the voices of marginalized users, we use the application as a prompt for surfacing potential harms perpetuated by FB ad recommendations, and qualitatively analyze themes of harm and proposed solutions provided by users themselves. The instruction increased comprehension of UB-CF for both groups, and we show that comprehension is associated with mentioning the mechanisms of the algorithm more in advocacy statements, a crucial component of a successful argument. We provide recommendations for increased algorithmic transparency on social media and for including marginalized voices in the conversation of algorithmic harm that are of interest both to social media researchers and ML educators.

Developing Self-Advocacy Skills Through Machine Learning Education: The Case of Ad Recommendation on Facebook

Link recommender algorithms may affect the exposure that users receive in social networking platforms, changing attention dynamics and potentially strengthening pre-existing inequalities that disproportionately affect certain groups (the so-called algorithmic glass-ceiling effect). In this paper we introduce a model able to simulate the feedback loop created by multiple rounds of interactions between users and a link recommender in a social network, which allows us to study the long-term consequences of link recommenders. Our model is equipped with several parameters that allow: (i) to control the level of homophily of connections, (ii) to set the size of the groups, (iii) to select among several state-of-the-art link recommenders, (iv) to chose among three different stochastic user behavior models for deciding which link recommendations to accept or reject. Our extensive experimentation with the proposed model shows that a minority group, if homophilic enough, can get a disproportionate advantage in exposure from all link recommenders, instead when it is heterophilic it gets underexposed. Moreover, while the homophily level of the minority affects the speed of the growth of disparate exposure, its size affects the magnitude of the effect. Finally, link recommenders strengthen exposure inequalities at the individual level, exacerbating the“rich-get-richer” effect: this happens for both the minority and the majority class and independently of their level of homophily.

Exposure Inequality in People Recommender Systems: The Long-Term Effects

Recommender systems are central to modern online platforms, but a popular concern is that they may be pulling society in dangerous directions (e.g., towards filter bubbles). However, a challenge with measuring the effects of recommender systems is how to compare user outcomes under these systems to outcomes under a credible counterfactual world without such systems. We take a model-based approach to this challenge, introducing a dichotomy of process models that we can compare: (1) a "recommender" model describing a generic item-matching process under a personalized recommender system and (2) an "organic" model describing a baseline counterfactual where users search for items without the mediation of any system. Our key finding is that the recommender and organic models result in dramatically different outcomes at both the individual and societal level, as supported by theorems and simulation experiments with real data. The two process models also induce different trade-offs during inference: regularization improves the mean squared error of matches in both settings, but at the cost of less diverse (less radical) items chosen in the recommender model but more diverse (more radical) items chosen in the organic model. These findings provide a formal framework for how recommender systems may be fundamentally altering how we search for and interact with content, in a world increasingly mediated by such systems.

To Recommend or Not? A Model-Based Comparison of Item-Matching Processes

We present analyses from a real-time information monitoring system of online local news in the U.S. We study relationships among online local news coverage of COVID, cases and deaths in an area, and properties of local news outlets and their audiences. Our analysis relies on a unique dataset of the online content of over 300 local news outlets, encompassing over 750,000 articles over a period of 10 months spanning April 2020 to February 2021. We find that the rate of COVID coverage over time by local news outlets was primarily associated with death rates at the national level, but that this effect dissipated over the course of the pandemic as news about COVID was steadily displaced by sociopolitical events, like the 2020 U.S. elections. We also find that both the volume and content of COVID coverage differed depending on local politics, and outlet audience size, as well as evidence that more vulnerable populations received less pandemic-related news.

Local News Online and COVID in the U.S.: Relationships among Coverage, Cases, Deaths, and Audience

Social media reduces barriers for the formation of large, self-organizing grassroots communities. For political campaigns this poses significant opportunities to address declining party membership, but also reputational risks and potential loss of campaign coherence. While balancing these factors is often done informally, this study implements implement a data-driven method for measuring the alignment between online political communities and their corresponding campaigns. We apply this technique to the 2020 U.S. Democratic presidential primaries, providing novel insights into the important tension between campaigns and third-party actors.

Adopting a behavioural approach, we apply neural embedding techniques, creating a “community embedding”, to evaluate grassroots communities along various cultural, political, and demographic dimensions. Our embedding demonstrates that these communities align with the views of a candidate’s supporters. However, this does not necessarily reflect the campaign’s policy platforms. Finally, we introduce temporal aspects to our community embedding to evaluate the stability of political communities and their interrelations. Our methods provide new insights into how politics on Reddit operates in relation to its offline analogs.

Measuring Alignment of Online Grassroots Political Communities with Political Campaigns

We draw insights from the social psychology literature to identify two facets of Twitter deliberations about migrants, i.e., perceptions about migrants and behaviors towards migrants. Our theoretical anchoring helped us in identifying two prevailing perceptions (i.e., sympathy and antipathy) and two dominant behaviors (i.e., solidarity and animosity) of social media users towards migrants. We have employed unsupervised and supervised approaches to identify these perceptions and behaviors. In the domain of applied NLP, our study offers a nuanced understanding of migrant-related Twitter de-liberations. Our proposed transformer-based model, i.e., BERT + CNN, has reported an F1-score of 0.76 and outperformed other models. Additionally, we argue that tweets conveying antipathy or animosity can be broadly considered hate speech towards migrants, but they are not the same. Thus, our approach has fine-tuned the binary hate speech detection task by highlighting the granular differences between perceptual and behavioral aspects of hate speeches.

Unraveling Social Perceptions & Behaviors towards Migrants on Twitter

In this paper, we analyze the service providers that power 440 misinformation and hate sites, including hosting platforms, domain registrars, CDN providers, DDoS protection companies, advertising networks, donation processors, and e-mail providers. We find that several providers are disproportionately responsible for serving misinformation websites, most prominently Cloudflare. We further show that misinformation sites also disproportionately rely on several popular ad networks and payment processors, including RevContent and Google DoubleClick. When misinformation websites are deplatformed by hosting providers, DDoS protection services, and registrars, sites nearly always resurface through alternative providers. However, anecdotally, we find that sites struggle to remain online when mainstream monetization channels are severed. We conclude with insights for infrastructure providers and researchers working to stem the spread of misinformation and hateful content.

On the Infrastructure Providers that Support Misinformation Websites

Social media posts that direct users to YouTube videos are one of the most effective techniques for spreading misinformation. However, it has been observed that such posts rarely get deleted or flagged.
Since multi-modal misinformation that leads to compelling videos has more impact than using just textual content, it is important to characterize and detect such textual post and video pairs to help users from becoming victims of misinformation. To address this gap, we build a taxonomy of how links to YouTube videos are used on social media platforms. We then use pairs of posts and videos annotated with this taxonomy to test several classification models built using cross-platform features. Our work reveals several characteristics of post-video pairs, in terms of how posts and videos are related to each other, the type of content they share, and their collective outcome.
In addition, we find that traditional approaches to misinformation detection that rely only on text from posts miss a significant number of post-video pairs that contain misinformation.
More importantly, we find that to reduce the spread of misinformation via post and video pairs, classifiers would be more effective if they are designed to use data and features from multiple diverse platforms.

Cross-Platform Multimodal Misinformation: Taxonomy, Characteristics and Detection for Textual Posts and Videos

Twitter is one of the most popular social networks attracting millions of users, while a considerable proportion of online discourse is captured. It provides a simple usage framework with short messages and an efficient application programming interface (API) enabling the research community to study and analyze several aspects of this social network. However, the Twitter usage simplicity can lead to malicious handling by various bots. The malicious handling phenomenon expands in online discourse, especially during the electoral periods, where except the legitimate bots used for dissemination and communication purposes, the goal is to manipulate the public opinion and the electorate towards a certain direction, specific ideology, or political party.

This paper focuses on the design of a novel system for identifying Twitter bots based on labeled Twitter data. To this end, a supervised machine learning (ML) framework is adopted using an Extreme Gradient Boosting (XGBoost) algorithm, where the hyper-parameters are tuned via cross-validation. Our study also deploys Shapley Additive Explanations (SHAP) for explaining the ML model predictions by calculating feature importance, using the game theoretic-based Shapley values. Experimental evaluation on distinct Twitter datasets demonstrate the superiority of our approach, in terms of bot detection accuracy, when compared against a recent state-of-the-art Twitter bot detection method.

Identification of Twitter Bots Based on an Explainable Machine Learning Framework: The US 2020 Elections Case Study

Malicious Twitter bots are detrimental to public discourse on social media. Past studies have looked at spammers, fake followers, and astroturfing bots, but retweet bots, which artificially inflate content, are not well understood. We present the first study focusing exclusively on retweet bots. We characterize retweet bots that have been uncovered by purchasing retweets from the black market. We detect whether they are fake or genuine accounts involved in inauthentic activities and what they do in order to appear legitimate. We also analyze their differences from human-controlled accounts. From our findings on the nature and life-cycle of retweet bots, we also point out several inconsistencies between the retweet bots used in this work and bots studied in prior works. Our findings challenge some of the fundamental assumptions related to bots and in particular how to detect them.

"It is just a flu": Assessing the Effect of Watch History on YouTube's Pseudoscientific Video Recommendations

Downloads

Next from ICWSM 2022

Developing Self-Advocacy Skills Through Machine Learning Education: The Case of Ad Recommendation on Facebook

Similar lecture

The Web We Weave: Untangling the Social Graph of the IETF

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES

"It is just a flu": Assessing the Effect of Watch History on YouTube's Pseudoscientific Video Recommendations

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from ICWSM 2022

Developing Self-Advocacy Skills Through Machine Learning Education: The Case of Ad Recommendation on Facebook

Similar lecture

The Web We Weave: Untangling the Social Graph of the IETF

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES

Downloads