ACM source attribution: Complete full-text transcription and rendered visual material from the ACM Hypertext and Social Media 2022 proceedings PDF. Original publication: ACM, Proceedings of the 33rd ACM Conference on Hypertext and Social Media (HT ’22), Barcelona, Spain, June 28–July 1, 2022. DOI: 10.1145/3511095.3536377.
Understanding Effects of Moderation and Migration on Online
Video Sharing Platforms
Tales Panoutsos Federal University of Minas Gerais
Belo Horizonte, Brazil tales.panoutsos@gmail.com
Gabriel Luis Freire Federal University of Minas Gerais
Belo Horizonte, Brazil
gabriellsf@gmail.com
Fabricio Benevenuto Federal University of Minas Gerais
Belo Horizonte, Brazil
fabricio@dcc.ufmg.br
ABSTRACT
To mitigate the propagation of potentially dangerous information (e.g., fake news), social media platforms usually rely on the deletion or censoring of content (here called moderation). In this research, we measure how content moderation on YouTube affects a channel’s popularity. To achieve our goal, we gather information on videos that were deleted from YouTube using the altCensored platform. We cross-section this data with channel popularity time series from SocialBlade. After characterizing this novel dataset, we employ Regression Discontinuity Design (RDD) to effectively measure impact. Using RDD we categorize the impact of censorship on deletion in four different patterns: (PP) channels with positive regression slopes (e.g., indicating growth) both before and after deletion; (PN) channels with positive growth before deletion and negative after, capturing a positive-negative relation, or a deletion that inverts the trend, (NP) negative-positive, those which were decreasing in growth with a change in trend after deletion (NN) as well as negative to negative. These groups represent 16% (PP), 26% (PN), 16% (NP), and 42% (NN) of our moderated videos. As a final result, we also show that videos may yet be found on other websites. The large amounts of events in (PP) and (NP), as well as the fact that videos are still available on the Web, indicate that moderation may not be as effective as it seems.
CCS CONCEPTS
• Information systems →Social networks.
KEYWORDS
misinformation, content moderation, video platform, youtube, regression discontinuity, fake news
ACM Reference Format: Gabriel Luis Freire, Tales Panoutsos, Lucas Perez, Fabricio Benevenuto, and Flavio Figueiredo. 2022. Understanding Effects of Moderation and Migration on Online Video Sharing Platforms. In Proceedings of the 33rd ACM
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the owner/author(s). HT ’22, June 28-July 1, 2022, Barcelona, Spain © 2022 Copyright held by the owner/author(s). ACM ISBN 978-1-4503-9233-4/22/06. https://doi.org/10.1145/3511095.3536377
Lucas Perez Federal University of Minas Gerais
Belo Horizonte, Brazil perezlucas2903@gmail.com
Flavio Figueiredo Federal University of Minas Gerais
Belo Horizonte, Brazil
flaviovdf@gmail.com
Conference on Hypertext and Social Media (HT ’22), June 28-July 1, 2022, Barcelona, Spain. ACM, New York, NY, USA, 5 pages. https://doi.org/10. 1145/3511095.3536377
1 INTRODUCTION
The sharing of hate speech, conspiratorial theories, and other toxic content – here defined as misinformation – currently has a direct impact society [3, 18]. To deal with misinformation, one of the most common countermeasures is content moderation, i.e., removal or account banishment. As a response to moderation, several platforms are appearing on the Web where content creators may share their information without restrictions [2, 20].
To cite some examples platforms such as altCensored1, Bitchute2, and DTube3, are now focused on curating content which is no longer available on YouTube. Current evidence suggests that such platforms are creating communities focused on extreme ideologies and radicalization [1]. Moreover, some content creators (i.e., YouTube channels) have accounts both on YouTube and such platforms.
Our research is focused on understanding the link between content moderation and content creator popularity. To gather which content was moderated, we make use of the full dataset from the altCensored platform. altCensored is a website focused on tracking and re-uploading deleted videos from YouTube. The website provides which channel the video belonged to, as well as a best estimate on when the video was deleted. The popularity of content creators was captured using SocialBlade4 that provides weekly popularity time series, in views, for channels.
[Main Hypothesis] To measure such impact, we employ Discontinuity Design (RDD) [13, 19]. Before doing so, we provide a general overview of this novel altCensored dataset. Finally, we also show through examples that several deleted videos survive not only on altCensored, but also on other platforms such as Facebook, DailyMotion and Vimeo. Our main research hypothesis is defined as: Can the moderation of content benefit (in the sense of becoming more popular) content creators? As stated, this hypothesis is tested via RDD. Some examples or results from this approach are shown in Figure 1 (channel ids are titles). Here, the figure shows four possible
1http://altcensored.com 2http://bitechute.com 3http://d.tube 4http://socialblade.com
UCcnozK25vIa0HsKPaFIVBYw
UCYQlj5ERJzCylETSZlRUECQ
6.0 (PN)
log10(views(x))
(PP)
3
5.5
2
5.0
20 10 0 10 20
20 10 0 10 20
UCoWQunH1zvFa6u3IoMXQjRw
UCiVuRSoLiet2lCTzQUOwHow
3.00
4 (NN)
log10(views(x))
(NP)
2.75
3
2.50
2.25
2
20 10 0 10 20 Weeks - x
20 10 0 10 20 Weeks - x
Figure 1: Examples of Trends Extracted by RDD
cases in which RDD is able to measures impact: Positive-to-Positive (PP) slopes, Positive-to-Negative (NP) slopes, Negative-to-Positive (NP) slopes, Negative-to-Negative (NN) slopes.
[Results and Limitations] Overall, our results show that 16% of the moderation events we track has a (PP) relationship. Also, 26% have an (PN) relationship, while 16%, and 42% have (NP) and (NN) relation on channel popularity time series. While the majority of moderation either stay negative (NN) or invert a positive trend to a negative one (PN) – the sum of both is 68% – a large fraction of moderation events indicate a possible increase (NP) or at least a continuing increase (PP) relationship – 32%. We also manually search for some of the moderated videos on Google and find that, at least for the top-200 most popular videos, a large fraction of them, 72%, are still on the Web. Overall, these results indicate that content moderation may not be as effective as it seems.
Initially, it is quite important to point out that while altCensored contains several videos that may be deemed as misinformation, the website does not limit itself for any particular type of content. Some popular videos on the websites are censored cartoons posted as piracy on YouTube (e.g., South Park). Thus, we call the videos we explore as potentially dangerous, as to avoid any direct labels.
Secondly, while RDD may have a causal interpretation [7], both in temporal [11] (i.e., our explanatory variable is time) and highly skewed [5] (i.e., popularity values are usually modeled by longtailed distributions) datasets such as ours, this causal interpretation is much more limited. For instance, a channel may become more popular after a content was deleted (NP) due to several cofounding variables: other videos, the popularity of certain topics worldwide, promotion, and so forth. Thus, we refrain from making any causal assumption. Nevertheless, RDD is able to measure how popularity time series changed both in intercept and slope around the time of content deletion. For such a reason, we argue that RDD is able to provide supporting evidence for the impact of moderation.
2 RELATED WORK
Regarding the popularity of YouTube videos, a broad number of studies have been made. Here, we point out to some of these efforts. One of the first of these efforts is the work of [5]. Here, the authors focus mostly on describing the long-tailed natured of popularity values. The dynamics of popularity over time has been discussed in [8]. Complementary, [6] focuses on the correlation of different popularity values (e.g., views, comments, likes, and so forth).
Regarding social media and Web moderation, [9] present a extensive study drawing attention to the need of a broad discussion on the subject. In particular, the authors argue that as moderation is a social-technical phenomenon, that encompasses policy making, advocacy, and public concerns. Other researchers have focused on many aspects of moderation. For instance, the work of [10] maps the regulatory dimension of moderation. In a similar manner, [4] discusses labor behind moderation while [16] discusses how to apply moderation to mainstream media.
One aspect that researchers still don’t seem to converge on is the efficiency of deplatforming i.e. moderation by removal, and best practices on how to do it. [17] find positive results analyzing the popularity of original content and their duplicates in other alternative social media and finding a loss in their reach. [14] finds a similar result but also indicates an increase in activity and toxicity levels of their supporters in the platform [14]. However [1] presents negative results by analyzing the behavior of deplatoformed users on alternative social media and also finding an increase in activity and toxicity as these users are presented to radical content [1]. Meanwhile [15, 21] studies other ways that moderation is being applied from platforms to moderate content, as the efficiency seems to vary by platform and type of content analyzed, still presents alternatives to deplataformization.
Finally, [12] also employs RDD to understand moderation and migration. However, the authors neither focus on video content (e.g., YouTube) nor a large scale dataset as we do. That is, in their work moderation and migration are studied, in depth, focusing on two Reddit communities. We view our approach as complementary to the authors, providing a broader view on the subject.
3 DATASET DESCRIPTION
As of January 14th 2022, altCensored describes itself as a “an unbiased community catalog of 187,257 limited state, removed, and self-censored YouTube videos across 9,648 monitored channels, of which 2345 have been deleted and 250 are being archived in case of deletion.”. Moreover, the website also defines itself as a “nonprofit, open-sourced Community Archive of Censored YouTube (YT) Videos, begun in February 2019”.
To perform our research, we asked the creators of altCensored for a snapshot of their database. This was made available to us on September 14th 2021. The dataset consists of a PostgreSQL dump. In our snapshot, we identified 1,416,848 videos from 9,348 channels. For deleted channels, altCensored also provides an approximate date when the channel was last available. This is the date the platform’s crawler last saw that channel on YouTube. The website also provides the information if the video was removed or not and a estimate of the number of views (popularity) of that video on YouTube on the last available date. It is important to note that neither the website nor the developers we contact provide details on how videos are chosen to be monitored.
For each of the 9,348 channels we attempted to gather popularity time series (in views) from the SocialBlade website. SocialBlade is a freemium Web platform that gathers channel statistics from YouTube. Weekly popularity time series are made available for
Table 1: Videos and Channels per Language
Language # Videos # Channels
English 1.006.087 3994 German 124.218 365 French 51.950 136 Portuguese 35.661 87 Spanish 26.885 97 Russian 16.762 57 Italian 16.168 72
Table 2: Categories of Moderated Videos
Category Quantity
News and Politics 132.788 People and Blogs 99.783 Education 43.905 Entertainment 40.705 Nonprofits and Activism 17.883 Music 12.045 Film and Animation 7.220 Comedy 6.767 Science and Technology 5.779
1.0
0.8
0.6
P[X > x]
0.4
Entertainment Nonprofits & Activism News & Politics Education People & Blogs
0.2
0.0
100 101 102 103 104 105 106 107
of views - x
Figure 2: View Count by Category
several monitored channels on their website5. Out of the 9,348 altCensored channels, we managed to gather 8,886 channel popularity time series.
Before discussing our results, we point some descriptive statistics about our dataset. Besides the numerical values described above, on the video and channel pages, altCensored also provides the title, description and category of that video/channel. Using the langdetect6 library, we extracted the language of both channels and videos. To do so, we simply combined the content of the title and the description and fed it to the library.
These numbers are shown on Table 1. From the table, we can see is predominantly of the English language. Other popular languages are German, French and Portuguese. Similarly, in Table 2, we show the top categories of our videos. From this second table, we can see that our dataset is mostly focused on News and Political content, Vlogs, and Educational videos. In order to understand the
5https://socialblade.com/youtube/channel/channelid/monthly 6https://pypi.org/project/langdetect/
relationship between categories and popularity, we show in Figure 2 the Complementary Cumulative Distributions (CCDFs) of the popularity values within the five most common categories.
From the figure, we can see that around 40% of the videos (yaxis of 0.4) from the People & Blogs categories have over 10,000 (x-axis) views. For Education, this value is close to 55%, while for the remaining categories, News & Politics, Nonprofits & Activism, and Entertainment, it is over 60%. This is a tendency that is maintained for most values in the plot, indicating that the most popular categories are News & Politics, Nonprofits & Activism and Entertainment.
4 REGRESSION DISCONTINUITY ANALYSIS
We now present our main results. More importantly, we present supporting evidence towards our main hypothesis: Can the moderation of content benefit (in the sense of becoming more popular) content creators?
In order to perform our analysis, we initially considered only the channels from SocialBlade where we have some moderation event (e.g., video removal indicator) from altCensored. Then, on SocialBlade we detect an week with negative views count in the channel’s views history, which is caused by the removal of a video and the subtraction of all it’s views count. Next, we decided to perform RDD by considering a 50 week window. That is, we consider 25 points before and after the event. The number of views on the week of moderation itself removed before regression. This second heuristic was done as to have sufficient points for the regression. Thirdly, we only considered 50-point windows were no other event happened within those 50 weeks. This aims to provide some control for confounding events, which may affect the channel. Lastly, in order to make sure no data collection errors from SocialBlade impact our analysis, we only consider events that 25 points before and after the exclusion are not completely equal to zero.
After such filters, we performed our analysis design using the following Regression Discontinuity Design (RDD):
log10(views(x)) ∼1 + 1x >0 + x · 1x >0 + x
Here, x is simply our time variable changed so that the removal event is at point zero. This is a common approach for RDD as it simplifies the interpretation of model parameters. 1x >0 is an indicator variable that takes value of one when x > 0, and zero otherwise. Thus, whenx = 0 the weight of this term is the difference in intercepts. Finally, the increase/decrease in regression slope is captured by the weight assigned to the term x · 1x >0. The log term was used due to the long-tailed nature of changes in popularity as exemplified in Figure 1 (i.e., in all examples there usually at least a 10-fold increase/decrease in popularity around the event). We analyzed RDD results only when the regression was deemed as statistically significant by three complementary approaches. Initially, we only looked at regression when the p-value, p, of the F-statistic was of: p < 0.05. With this approach, we reject the nullhypothesis that the proportion of the variance of the data explained by the regression, or R2, was explained by chance. Secondly, we only consider cases where R2 > 0.33, this leads to at least 57% (
√
R2 = 0.57) of the data’s standard deviation to be explained by the model. Finally, we only looked at cases where the errors were
35
0.100
some of the manually
30
inspected outliers
0.075
25
0.050
After Slope
videos
0.025
20
0.000
15
0.025
10
0.050
5
0.075
0.05 0.00 0.05 0.10 0.15 Before Slope
Figure 3: Difference in Slopes
1.0
(NN) (NP) (PN) (PP)
0.8
P[X > x]
0.6
0.4
0.2
0.0
3 2 1 0 1 2 3 Increase in Intercept - x
Figure 4: Intercept Difference
deemed as normally distributed following a Kolmogrov-Smirnoff test.
While this approach will filter most of our events, due to the moderate scale of our dataset we preferred to employ a conservative analysis. The filters above aim to justify the usage of a regression model, not only RDD, overall. In the end, we were left with 612 windows/regressions. We begin our analysis by looking at the change in slopes in Figure 3. The x-axis of the figure shows the slope before the event, while the y-axis shows the slope after. Vertical and horizontal lines indicate the boundaries of the groups.
From this figure, we find that, 16% of the 612 regressions have a (PP) relationship, 26% have an (PN) relationship, while 16%, and 42% have (NP) and (NN) relationships, respectively. More importantly, we find that moderaion may increase the popularity of the creator (NP) in 16% of the cases. This provides supporting evidence to our hypothesis.
From the figure we can see that most events have a small impact on channel popularity. Most points lie in the shaded region where the maximum absolute change in slope only of 0.05. Given that we are looking into the popularity of channel time series, this result indicates that the moderation of isolated content has a small impact. This points towards evidence that the discourse of the channel will remain active, either through re-uploads of the same video or others.
We also manually looked into some of the more extreme points (indicated by arrows) and found that no major change occurs around the moderation events. These extreme points happen mostly due to large increases/decreases at other dates. Thus, we consider such points as outliers.
Considering the change in intercept, Figure 4 shows the CCDFs for the four groups. Here, we can see that less than 20% of the videos have a change of intercept greater than zero for all groups. In fact, it is interesting that Positive-to-Negative relationships have
Table 3: Websites with the most reposts
Website Reposts
YouTube 109 DailyMotion 34 Facebook 31 Coub 15 SouthParkStudios 11 Vimeo 6 Aparat 5
smaller changes than Negative-to-Positive ones. Overall, again our results point that video moderation has a small impact on channel popularity.
Finally, to point towards evidence that moderated videos are still available in other websites apart from altCensored, we Google searched the top-200, in views, video titles. The authors were responsible for the search and the confirmation of the duplicity following: (1) the found video needed to have the exact same title as the original; (2) videos with generic titles which may be associated to multiple videos (e. g. music titles) were skipped; (3) videos have the same thumbnail; and, (4) the search goes up to five pages.
This processed occurred in January 10, 2021. Initially, we point out that 72% of the 200 videos were found using our criteria. Table 3 shows some of the top websites where these videos were found. YouTube itself is the most popular website. Videos were either reuploaded or were never fully deleted in the first place (e.g., the creator may have marked the video as private only). Nevertheless, we do find evidence of videos being available in social media sites as Facebook or other video sharing platforms as Daily Motion or Vimeo.
5 CONCLUSIONS
In this paper, we looked at the effect of the moderation of individual content (i.e., YouTube videos) on content creator popularity. Overall, our results point towards evidence that content moderation is mostly ineffective inefficiency when we consider channels as a whole. With the current and growing problem of misinformation, we argue that new forms of moderation need to be discussed and put into practice.
More importantly, we find supporting evidence that moderation may increase the viewership of some channels. A in-depth exploration of such cases is left as future work.
Rendered visual and table regions
REFERENCES
[1] Shiza Ali, Mohammad Hammas Saeed, Esraa Aldreabi, Jeremy Blackburn, Emil-
iano De Cristofaro, Savvas Zannettou, and Gianluca Stringhini. 2021. Understanding the Effect of Deplatforming on Social Networks. In 13th ACM Web Science Conference 2021 (Virtual Event, United Kingdom) (WebSci ’21). Association for Computing Machinery, New York, NY, USA, 187–195. https: //doi.org/10.1145/3447535.3462637 [2] Max Aliapoulios, Emmi Bevensee, Jeremy Blackburn, Barry Bradlyn, Emiliano De
Cristofaro, Gianluca Stringhini, and Savvas Zannettou. 2021. An Early Look at the Parler Online Social Network. arXiv:2101.03820 [cs.SI] [3] Hunt Allcott and Matthew Gentzkow. 2017. Social Media and Fake News in the
2016 Election. SSRN (1 2017). https://doi.org/10.3386/W23089 [4] Diego Cerna Aragon. 2020. Behind the screen: content moderation in the shadows
of social media. https://doi.org/10.1080/15295036.2020.1800895 37 (10 2020), 512– 514. Issue 5. https://doi.org/10.1080/15295036.2020.1800895
[5] Meeyoung Cha, Haewoon Kwak, Pablo Rodriguez, Yong-Yeol Ahn, and Sue
Moon. 2007. I Tube, You Tube, Everybody Tubes: Analyzing the World’s Largest User Generated Content Video System. In Proceedings of the 7th ACM SIGCOMM Conference on Internet Measurement (San Diego, California, USA) (IMC ’07). Association for Computing Machinery, New York, NY, USA, 1–14. https://doi.org/10.1145/1298306.1298309 [6] Gloria Chatzopoulou, Cheng Sheng, and Michalis Faloutsos. 2010. A First Step
Towards Understanding Popularity in YouTube. 1 – 6. https://doi.org/10.1109/ INFCOMW.2010.5466701 [7] Scott Cunningham. 2021. Causal Inference: The Mixtape. Yale University Press,
United States. [8] Flavio Figueiredo, Jussara M Almeida, Marcos André Gonçalves, and Fabricio
Benevenuto. 2014. On the dynamics of social media popularity: A YouTube case study. ACM Transactions on Internet Technology (TOIT) 14, 4 (2014), 1–23. [9] Tarleton Gillespie, Patricia Aufderheide, Elinor Carmi, Ysabel Gerrard, Robert
Gorwa, Ariadna Matamoros-Fernández, Sarah T. Roberts, Aram Sinnreich, and Sarah Myers West. 2020. Expanding the debate about content moderation: Scholarly research agendas for the coming policy debates. Internet Policy Review 9 (2020), 1–29. Issue 4. https://doi.org/10.14763/2020.4.1512 [10] Robert Gorwa. 2019. What is platform governance? https://doi.org/10.1080/1369118X.2019.1573914 22 (5 2019), 854–871. Issue 6. https://doi.org/10.1080/1369118X.2019.1573914 [11] Catherine Hausman, David S Rapson, Michael Anderson, Max Auffhammer, Sev-
erin Borenstein, Matias Cattaneo, Lucas Davis, Stephen Holland, Ryan Kellogg, Doug Miller, and Jeff Smith. 2017. Regression Discontinuity in Time: Considerations for Empirical Applications. (7 2017). https://doi.org/10.3386/W23602 [12] Manoel Horta Ribeiro, Shagun Jhaver, Savvas Zannettou, Jeremy Blackburn,
Gianluca Stringhini, Emiliano De Cristofaro, and Robert West. 2021. Do platform migrations compromise content moderation? evidence from r/thedonald and r/incels. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–24. [13] Guido W. Imbens and Thomas Lemieux. 2008. Regression discontinuity designs:
A guide to practice. Journal of Econometrics 142, 2 (2008), 615–635. https:
//doi.org/10.1016/j.jeconom.2007.05.001 The regression discontinuity design: Theory and applications. [14] Shagun Jhaver, Christian Boylston, Diyi Yang, and Amy Bruckman. 2021. Evaluat-
ing the Effectiveness of Deplatforming as a Moderation Strategy on Twitter; Evaluating the Effectiveness of Deplatforming as a Moderation Strategy on Twitter. Proc. ACM Hum.-Comput. Interact 5 (2021), 30. https://doi.org/10.1145/3479525 [15] Paul Mena. 2020. Cleaning Up Social Media: The Effect of Warning Labels on
Likelihood of Sharing False News on Facebook. Policy & Internet 12 (6 2020), 165–183. Issue 2. https://doi.org/10.1002/POI3.214 [16] Tanner Mirrlees. 2021. GAFAM and Hate Content Moderation: Deplatforming
and Deleting the Alt-right. Sociology of Crime Law and Deviance 26 (2021), 81–97. https://doi.org/10.1108/S1521-613620210000026006 [17] Adrian Rauchfleisch and Jonas Kaiser. 2021. Deplatforming the Far-right: An
Analysis of YouTube and BitChute. (6 2021). https://doi.org/10.2139/SSRN. 3867818 [18] Yasmim Mendes Rocha, Gabriel Acácio de Moura, Gabriel Alves Desidério, Car-
los Henrique de Oliveira, Francisco Dantas Lourenço, and Larissa Deadame de Figueiredo Nicolete. 2021. The impact of fake news on social media and its influence on health during the COVID-19 pandemic: a systematic review. Zeitschrift Fur Gesundheitswissenschaften (2021), 1. https://doi.org/10.1007/S10389-02101658-Z [19] Donald L. Thistlethwaite and Donald T. Campbell. 1960. Regression-discontinuity
analysis: An alternative to the ex post facto experiment. Journal of Educational Psychology 51 (1960). Issue 6. https://doi.org/10.1037/h0044319 [20] Milo Trujillo, Maurício Gruppi, Cody Buntain, and Benjamin D. Horne. 2020.
What is BitChute? Characterizing The. In Proceedings of the 31st ACM Conference on Hypertext and Social Media (Virtual Event, USA) (HT ’20). Association for Computing Machinery, New York, NY, USA, 139–140. https://doi.org/10.1145/ 3372923.3404833 [21] Savvas Zannettou. 2021. "I Won the Election!": An Empirical Analysis of Soft
Moderation Interventions on Twitter. (1 2021). https://arxiv.org/abs/2101.07183v2
Do you like what you are reading? Subscribe to receive updates.
Unsubscribe anytime