A critical barrier facing engineering is inclusiveness of women in the profession. In recent years, engineering diversity advocates have taken to social media platforms to raise awareness of the issue and redress this problem. A recurring challenge for their initiatives though is attracting and mobilizing participants efficiently. For a successful mobilization campaign, organizers need real-time information about their users and also need to understand what messaging works to attract and mobilize them. We hypothesize that participants in any given campaign related to engineering diversity will also be interested in other campaigns related to that issue. Furthermore, since the primary signal for a social media campaign is a hashtag, by using clustering patterns of various co-occurring hashtags along with relevant topics and relatable sentiments, we can better understand participation and also mobilize users for the target campaign. To empirically examine our hypothesis, we study two diversity hashtag activism campaigns on Twitter (#ILookLikeAnEngineer and #WomenInEngineering) using a real-time predictive analytics framework. We design and evaluate the framework with a set of novel features that uses retweetability as an indicator of participation. Our result analysis for topical features found that monetary gain and advertisement-oriented content were less likely to be propagated in the campaigns whereas messaging aligned directly with the issue at hand such as breaking stereotypes in engineering was deemed more retweetable and engaging. In terms of sentiments, informal tone in the messages were considered desirable whereas short-form messaging were not very popular in either movements. These analytical insights can inform activists in effective resource mobilization through message content design, in order to expand the reach of an activism campaign. Our work shows how data-driven techniques can assist in increasing the participation of women in engineering education and the workforce.

• Networks → Online social networks; • Information systems → Data mining; Clustering and classification; Content analysis and feature selection.

Hashtag Activism; Engineering Diversity

Habib Karbasian, Hemant Purohit, and Aditya Johri. 2021. Improving Diversity in Engineering: A Data-Driven Approach to Support Resource Mobilization and Participation in Hashtag Activism Campaigns. In Proceedings of the 32nd ACM Conference on Hypertext and Social Media (HT ’21), August 30–September 2, 2021, Virtual Event, Ireland. ACM, New York, NY, USA, 11 pages. https://doi.org/10.1145/3465336.3475103

Engineering is probably one of the most male-dominated professions in the world and overall lacks diversity across all categories but particularly in percentage of women in the profession. While other professions such as law and medicine have now achieved gender parity in many industrialized nations at least at the student level, the number of women entering the engineering profession remains low in advanced countries such as the U.S. [1–3]. Further, of those women who graduate with an engineering degree and then go on to stay in the engineering workforce, this number is even lower. The low retention of women in the engineering workforce has been attributed largely to masculine culture of the profession, among factors such as work-life balance [5, 39, 42, 50]. Although empirical evidence is clear that women are equally proficient and competent as their male peers, the cultural norms and expectations in engineering education and the workforce constitutes a real barrier to the advancement of women [16, 39, 42]. To bring change, one strategy is to make visible and raise awareness of the nature of problems that women encounter in the engineering workplace and to propose and popularize counter narratives that break stereotypes and fervent participation. With the introduction of social media, public discourse and communication has shifted online and many platforms have emerged as useful tools for individuals and small groups to be able to connect with like-minded others around issues that they find engaging such as health communication [33], politics [24], humanitarian crisis [55], and social movements such as the Arab Spring [15, 22] and Black Lives Matters [56]. Therefore, it is unsurprising that many efforts for increasing diversity in engineering have similarly started to leverage these platforms

for their campaigns [28]. Hashtags campaigns that exemplify diversity of women in engineering such as #ILookLikeAnEngineer, #WomenInSTEM, and #WomenInEngineering are now common. Yet, there is little understanding of how these campaigns overcome the challenge of mobilizing users. We address this gap and start with an overview of challenges for resource mobilization theory and its implications for success of hashtag campaigns. It is noteworthy that the types of activism campaigns studied in this work (i.e., engineering diversity) are different from other types of campaigns such as advertising or political ones since advertising campaigns are highly centralized and professionally managed and political campaigns, once again, are highly managed, and extremely negative at times to change opinions of individuals, which have overall different characteristics in terms of motivation for participation. The participation in the campaigns we study include the actors who participate (both individuals and organizations [31] , and the overall goal/mission of the campaign than what we find in these other types of campaigns. Moreover, our campaigns are a lot more about “expressing an identity” [27] as opposed to “spreading information or misinformation” [18, 69].

1.1 Challenge: Mobilization of Users and Participation

1.2 Potential Solution: Mobilizing by Improving User Participation

One of the purposes of hashtag activism is to create an online community where people share ideas and experiences related to a larger common goal. According to [51, 52], a key factor in creating a sense of community is the number of messages exchanged indicating

how engaged people are within the community. In social media campaigns, sharing is one form of “user participation” that is often measured by reposting of a message (retweet on Twitter) written by another person. To increase user participation in the online community through resource mobilization, reposting becomes central. In that regard, researchers have found correlation between the content characteristics of the message and its propagation [49]. Therefore, the most viable option available to every activist is to know how to create an attention-seeking message to be able to raise awareness and promote their cause. As discussed earlier, online social media, e.g. Twitter, has dynamic nature and its fast paced content generation makes it extremely time consuming and difficult to be analyzed manually. Yet, there is a paucity of analytical frameworks to efficiently help with analysis of real-time data. As existing frameworks require offline analysis or a classification task where it needs to collect extra information to build the predictive model such as historical tweets/retweets and ego-network. These models are not readily applicable in real world situations. Hence, this study attempts to answer this primary question: Given the data collection limitations and fast paced content generation on social media platforms, how can we increase the success of hashtag activism to support resource mobilization and encourage user participation in real-time by examining the elements of attractive and engaging content? We address two specific research questions related to this overarching question:

(1) Can we build a real-time analytical framework to predict if an activist’s message will be reshared in a given hashtag activism campaign to increase user participation? (2) What parts of the messages in a given hashtag activism campaign are deemed interesting and socially engaging to its participants?

These questions are investigated using Twitter data [71]. A message on Twitter is called tweet, which is comprised of different information elements, such as text, hashtags, link, and media (video, photos). Creating engaging textual part of the tweet, text and hashtags, is the focus of our approach to address the above two questions. The analysis of text, usually as unstructured data, has been done with different approaches such as topic modeling and sentiment analysis [43, 45, 81]. An alternative approach to improve the content of tweets in movements is the use of hashtags, which are essentially user-defined, topical tags. Hashtags make the tweets semi-structured and semantically related to each other [75, 76]. During the Arab Spring, hashtags like #egypt #jasminerevolution, and #jan25 became popular and created the communication leverage that increased the movement’s growth [41]. Using a shared set of hashtags, a distributed community was able to locate, self-organize, and collectively contribute to the information channels about a networked social movement [7, 41, 75]. We believe that the same approach is also applicable to hashtags related to diversity campaigns; ILookLikeAnEngineer and WomenInEngineering. Using the two key textual parts of a tweet in an activism campaign, text and hashtags, we propose a systematic approach to analyze content of messages for online hashtag activism campaigns. we hypothesize that the co-occurring hashtags along with relevant topics and relatable sentiments for a given cause are likely to create more socially

engaging and attractive content that leads to increasing online user participation.

2.1 Contributions

In Twitter, engaging and attractive messages are more likely to be retweeted through the network as the content of the message is an important component to have a positive effect in spreading in the online community as well [49]. Some researchers focused on the types or content of tweets that are more likely to be retweeted than others [14, 59, 60, 63, 67, 68, 79] and found that the more frequently retweeted tweets are the ones that tend to be sentimental and emotional, such as tweets that include news, describe negative consequences or emotions or remind personal experiences [59, 60, 65]. Event-centric hashtag analyses of tweets such as disastrous events, e.g. hurricane and terrorist attack, show that tweets describing impacts from hazards were frequently retweeted, whereas tweets expressing gratitude were less retweeted [67, 68] and also information including phone number, incident-related data, date or time regarding an update make the tweets in these events more important to be retweeted [44]. There are some works studying technical factors (such as external links, hashtags, images, videos, and the use of mentions in tweets) to find their effects on retweetability. Their findings showed that tweets with hashtags [4, 63, 66, 67] and multimedia, such as pictures, videos, and emoticons, were more likely to be retweeted [26]. There were also some studies regarding how retweeting behavior was influenced by user and network related factors [53, 74]. Such works showed that a message was more likely to be retweeted when the message is in the follower’s areas of interests [36, 74, 78, 82] and when the message was frequently retweeted by followers [4, 80]. In the unstructured part of tweet content, text understanding has been studied extensively but using coarse-grained features like topics that helped with the findings of interesting themes in tweets [43, 81]. These works approached the retweetability problem from different angles, such as the cascade size of information diffusion in the network [11, 12, 34] or the influence of user’s ego-network [53, 74].

These approaches were later formulated as a factor analysis or a prediction task. Improvements to Prior Works. The discussion above reveals one problem among the approaches taken so far: the applicability of proposed methods in real-time were not discussed and in most papers, the features cannot be constructed due to the aforementioned limitations imposed by the social media platforms. On the other hand, our framework uses the mutual relationship between topic and hashtag as a promising indicator [77] to improve the existing features by adding a set of coarse-grained features discussed earlier which only relies on just current tweet in the data stream and the user’s metadata.

We frame the solution to the first research question as the development of a predictive model of retweetability for a given tweet. The target label of this prediction task is a binary variable, whether a given tweet will be retweeted or not. In the following, we describe the comprehensive feature design for the input of this task. These features use the information available from a user’s single tweet and the associated user profile on Twitter platform in real-time.

4.1 Fine-Grained Features

4.2 Coarse-Grained Features

Contrary to fine-grained features, coarse-grained features are constructed with the help of external knowledge resources, e.g. humancurated psycholinguistics lexicon [46], pretrained model [31] , or another source such as tweet embeddings [82]. These features create a higher level of abstraction from the data that are more interpretable and understandable by humans (e.g. topics or emotional sentiment). In the following, the methodology for generating each coarse-grained feature is described.

4.2.1 Sentiments

Emotional and sentimental tone in the messages can be useful to attract more like-minded audience in the social

that may exist within the same hashtag campaign to support different objectives [21]. These examples show that identifying topics of discussion in social movements is helpful with spreading the messages online. We used Latent Dirichlet Allocation (LDA) [8] as a topic modeling algorithm, since it has been proven effective for finding discussion topics in natural language text documents [43]. The LDA model was applied to the textual part of the tweets to identify topics in a hashtag activism campaign. The number of topics, likewise other clustering algorithms, are pre-specified and can be found in "Experimental Setup" section.

network [65]. It is associated with increasing the social reward as it captures emotional and psychological themes of the message that encourage people to chime in and feel included in the cause. Studies have shown that emotional engagement are effective persuasive devices [64]. In the context of written communication, previous research has indicated that emotional stimuli in terms of emotion words or emotional framing of messages may increase the level of attention [6, 32, 62]. Hence increased attention may lead to higher likelihood of behavioral response to emotional stimuli in terms of information sharing [23, 35, 48, 57]. Emotion triggering content such as positive (awe) or negative (anger or anxiety) emotions has often shown more contributions to viral information diffusion. It can also be argued that increased level of attention triggered by emotions in written communication is determinant of sharing behavior. We used Linguistic Inquiry and Word Count (LIWC) to capture sentimental and psychological tone of the tweets in the movements. LIWC is a word-category lexicon [46, 47, 61] and quantifies psychological characteristics of tweets’ text with 93 different categories ranging from part-of-speech, i.e. articles, prepositions, past-tense verbs, numbers, etc. to topical categories, i.e. family, cognitive mechanisms, affect, occupation, body, etc. as well as a few other attributes such as total number of words [13, 46]. It uses a word count strategy and calculates the relative frequency of words and word stems with the scale ranging from 0 to 100 in a text that fall into a specific category. Therefore, the resulting feature vector constructed by LIWC is 93 numerical values representing each category.

Figure 1: The feature design of fine- and coarse-grained content-based features.

4.2.2 Topics

The collective identity of an online social movement could be strengthened by the issue-based themes embedded in the messages, which leads to greater information diffusion in the network. For example in the Egyptian revolution, the detailed instructions and suggestions based on lessons learned from Tunisian protesters were widely propagated in the social network that helped the movement [15]. Also in a qualitative analysis of ILookLikeAnEngineer movement, the main themes of the online conversations revolved around challenging stereotypes, promoting role models and campaign strategies [37]. Moreover, there are different topics

4.2.3 Hashtag Clusters

Hashtags have been used as attention mobilizer and have played a major role in information diffusion in social movements [75]. The choice of using a particular hashtag is affected by two processes: seeking attention from interested users [10], and contagion process driven by the virality of certain hashtags [58]. Leveraging the structure of a “hashtag co-occurrence network” with certain topics could help strengthen the latent relationship between hashtags and tweets to speed up the information diffusion. As each hashtag represents a circle of like-minded users, the strategic combinations of hashtags allow social movement participants to mobilize public attention from different social circles efficiently, thus achieving higher visibility and message propagation [20]. To be able to capture coarse-grained features from hashtag co-occurrence network comprehensively, we employed two different clustering algorithms to detect the clusters of the hashtags as follows.

Soft Clusters. We used LDA as a soft clustering algorithm for hashtag clusters. The hashtags of each tweet were treated as a document for topic modeling to cluster semantically related hashtags. The hyper-parameters of soft clustering will be specified in "Experimental Setup" section.

Hard Clusters. Community detection is a clustering algorithm for graph-based problems. As opposed to topic modeling, community detection is a hard clustering algorithm where each data point either belongs to a cluster community or not. Each cluster is a set of nodes in the graph that have more edges linking among its members than edges linking outside to the rest of the graph [17]. In a co-occurrence hashtag network, links between hashtags can be either binary or weighted signifying the magnitude of their relationship. Louvain community detection algorithm [9] was selected because of its efficiency for large networks. Hard cluster features extracted from the communities result into a vector of items, each representing a community membership. The specific value of each item is the presence of that community in the tweet; either binary or weighted. The number of clusters in Louvain algorithm is determined by maximizing the entire network’s modularity metric [9]. Here is the list of specific coarse-grained features studied:

• Setiments (LIWC): 93 numerical values of the tweet characteristics corresponding to LIWC psycholinguistics categories • Topics (LDA): K 1 numerical values representing topic membership weights for the text of the tweet from LDA

1It will be specified in "Experimental Setup" section.

Table 1: Statistics of ILookLikeAnEngineer and WomenInEnginnering datasets for tweets authored by all and verified user accounts.

We collected two datasets based on hashtags for engineering diversity campaigns from Twitter. The brief description is provided for each in the following sections. The datasets are available at [30] and the statistics of both datasets is provided in Table 1.

5.1 ILookLikeAnEngineer (ILLAE)

5.2 WomenInEngineering (WIE)

5.3 Exploratory Analysis

5.4 Labeled data

In this section, first we briefly explain the steps to make the data ready to be analyzed for answering the research questions in “Preprocessing” section. Then it continues with “Prediction Task Analysis for RQ1” section that analyzes whether a tweet is going to be retweeted in a given hashtag activism, where we compare the proposed predictive analytics framework for multiple baseline schemes against the proposed feature design. Next, “Feature Importance Analysis for RQ2” section explores which parts of the tweets are resonating with the people of a given hashtag activism, where it shows the characteristics of the tweets in engineering diversity hashtag activism that were found to be the most and least retweetable in terms of fine- and coarse-grained features.

6.1 Experimental Setup

We cleansed the textual content of the tweet in multiple steps. First, we removed any HTML tags, URLs, emails, user mentions and hashtags. Then we applied the Porter stemming algorithm [29]. The next step was to remove common English-language stopwords. Since 2-grams (equivalently, bi-grams) have been shown to increase the quality of text analysis [70], we included the bi-grams as well for LDA model. To get more meaningful words for each tweet, we used lemmatization technique which identifies the intended part of speech and meaning of a word in a sentence. So we just used adjective, adverb, noun, and verb as accepted parts of speech. The final step was to remove less and highly frequently used words from the tweets. We set the minimum threshold (nobelow) and the maximum threshold (noabove) for each word based on the percentage of tweets containing the word. In this study, we used the implementation of the LDA model provided by MALLET version 2.0.8 [38] 5, which is an implementation of the Gibbs sampling algorithm [19].

Once all features are generated as described before, the values of features are standardized using z-score. You can find the list of hyper-parameters for each feature in Table 2.

Note 5: http://mallet.cs.umass.edu (accessed May 7, 2020).

Table 2: Features Hyper-parameters.

6.2 Prediction Task Analysis for RQ1

Our predictive model is a binary classifier to answer the first research question (RQ1) for retweetability prediction for a given tweet. We first describe the classification algorithm, followed by the model construction steps, and different modeling schemes for baselines.

6.2.1 Support Vector Machine (SVM) Classifier

The reason for employing an SVM classifier is due to SVMs becoming one of the most popular machine learning techniques for binary classification problems and it has been proven effective in non-linear feature space due to the kernel use. SVMs have shown great success in complex classification problems and have a solid theoretical foundation in statistical learning theory [72, 73]. The hyper-parameters for SVM used in the experiments are presented in Table 3.

6.2.2 Model Selection

Hold-out cross validation (CV) was used for training and testing the prediction model using various framework schemes and the baselines. 20% of the dataset was held out for the final testing and the rest of it (80%) was used for 5-fold cross validation. For performance measurement in hyper-parameter optimization for each CV iteration, “F1 score” for positive class was used.

5http://mallet.cs.umass.edu accessed on May 7, 2020

Table 3: Model Hyper-parameters.

6.2.3 Baselines and Proposed Modeling Schemes

We created the following feature sets to create different modeling schemes of the predictive framework and understand which feature set has more impact for the retweetability prediction task in a given hashtag activism campaign: • All: contains all fine-grained features and coarse-grained features. • Content-All: contains all of the content features. • Content-Coarse: contains coarse-grained features. • Content-Hashtags: contains features for both soft and hard clusters from hashtag co-occurrence network. • Content-Topics: contains features for topics (LDA). • Content-Sentiments: contains sentiments features (LIWC). • Content-Fine: contains fine-grained features. • User: contains user’s metadata features. The baselines for this prediction task from related works that have only real-time features are [66] (Baseline-1), [49] (Baseline-2), [36] (Baseline-3), and [44] (Baseline-4).

6.3 Feature Importance Analysis for RQ2

We describe the results for the experiments corresponding to the two research questions and discuss their implications.

7.1 Results for RQ1 analysis

The results of ILookLikeAnEngineer and WomenInEngineering are shown in Figures 2 and 3 respectively in terms of accuracy and

Figure 2: Experimental results of 4 baselines with different sets of features from proposed framework for ILookLikeAnEngineer dataset.

F1 score for positive class. After applying McNemar test to the results, Framework (All) against 4 baselines in both datasets were statistically significant (p < 0.05). Our observations from experiments are:

• The proposed framework’s modeling scheme (All) outperformed all baselines with statistically significant difference. Increase of accuracy between framework(All) and the best baseline were 6.61% on ILLAE dataset and 8.25% on WIE. It shows that adding the comprehensive features capturing diverse relevant information can improve the retweetability prediction-task performance. • In ILLAE and WIE, we see 8.53% and 6.41% accuracy increase respectively between Content-All and User schemes. Contrary to the findings in [49], content-related features alone (Content-All) are more indicative of retweetablity than just user-related features (User) such as status and social network connections (i.e. followers and followees). It shows that messaging and content in these two hashtag activism played a major role in information diffusion in the network. • Among coarse-grained features, sentiments (Content-Sentiments) have the highest accuracy with statistical significance. It

Figure 3: Experimental results of 4 baselines with different sets of features from proposed framework for WomenInEngineering dataset.

shows that exploring sentiments (psycholinguistics characteristics) as a persuasive method is a good determinant of information diffusion in the hashtag activism campaign. • Although sentiments feature (Content-Sentiments) was the strongest indicator among coarse-grained features for both dataset, other coarse-grained features, topics (Content-Topics) and hashtags (Content-Hashtags), improve its performance where their collective feature set, (Content-Coarse), could result in a more accurate modelling scheme. It shows that strategic hashtag selection along with relevant topics can increase retweetability likelihood. • Comparing the fine-grained features (Content-Fine) with coarse-grained features (Content-Coarse), they are not statistically different in both datasets. Although combining them in a common scheme (Content-all) shows a significant improvement in the performance. Extracting more coarsegrained features from content (i.e features from different modality of the data, such photo, video and hyperlinks) could be further helpful in this prediction task.

7.2 Results for RQ2 analysis

Table 6: List of the most positively and negatively correlated topics, soft clusters and hard clusters for ILookLikeAnEngineer and WomenInEngineering datasets in terms of retweetability; orange color signifies the features that are positively correlated to the retweetability whereas blue color signifies the features that are negatively correlated. is for 95% significance.

Table 4 shows the weights of fine-grained features. As the sentiments feature set contains LIWC’s 93 features, we just show the weights of the first and last three features in Table 5. Table 6 are list of the highly positively and negatively correlated topics, soft clusters and hard clusters for ILookLikeAnEngineer and WomenInEngineering datasets in terms of retweetability. In the following, we present our observations from the regression analysis:

• In Table 4, status count is one of the strongest indicators of retweetablity in both datasets meaning that the more status count the author of the tweet has, the less likely the tweet will be retweeted. • In ILLAE, Followers Count has the largest weight showing that the more follower you have, the more likely your tweet will get retweeted. Although Followers Count has positive weight in WIE, but Length has the largest value. On another note, Listed Count is the second largest weight in ILLAE and Media is the second largest weight in WIE. We can infer that ILLAE movement was more driven by the more famous and well-known people who merely used their social media reputation to promote the movement message [28] whereas in WIE, the content has more importance than user’s social network. • Table 5 shows the features from LIWC. To further study each feature, please refer to their manual6. An interesting observation between these datasets is that the favorable tone of language in the tweets was informal whereas netspeak (shortened words for online conversations like btw, lol, thx, etc.) had negative effect for attracting people. • In Table 6 under topic section, the most negatively correlated topic extracted by LDA in ILookLikeAnEngineer was the Spam News Ad which was negatively correlated to the retweetability as there were many tweets with the same content and they were not retweeted by the users, even though the topic and the keywords were relevant.

6https://liwc.wpengine.com/wp-content/uploads/2015/11/LIWC2015 LanguageManual.pdf accessed on May 7, 2020

Table 5: List of weights for coarse-grained features of LIWC in the sentiments feature set (LIWC) for ILookLikeAnEngineer and WomenInEngineering; orange color signifies the features that are positively correlated to the retweetability whereas blue color signifies the features that are negatively correlated. is for 95% significance.

Table 4: List of weights for fine-grained features learned from logistic regression for ILookLikeAnEngineer and WomenInEngineering; orange color signifies the features that are positively correlated to the retweetability whereas blue color signifies the features that are negatively correlated. is for 95% significance.

• Another attempt by a company using the hashtags for the movement can be seen in Table 6 as a negatively weighted topic. A doll company, called Lotti, used the hashtag to promote its products but people in the movement didn’t find its tweets relevant and interesting to reshare. It is a good example of failed advertising campaigns to gain visibility advantage via WomenInEngineering campaign. • In Table 6 under soft cluster section, Challenge News Ad is the most negatively correlated hashtags cluster as opposed to it being the most positively correlated topic in terms of textual content. It shows that even though the relevant keywords were used to promote the site and products, the tweets were less likely to get retweeted whereas STEM related cluster of hashtags like INWED18 (International Women In Engineering Day 2018), was more likely to get retweeted. Here is another example where advertisement-oriented content didn’t attract the audience of WomenInEngineering campaign. • In Table 6 under hard clusters, the most positively and negatively correlated tweets in both datasets seem to be relevant to the engineering diversity movements. We analyzed the feature matrix constructed by Louvain algorithm and found out that the sparsity of the features was extreme (i.e. 98.5% and 99.5% were sparse for ILLAE and WIE respectively). As Louvain is a hard clustering algorithm, it is unsuitable for indepth feature analysis and thus, the features created by topic modeling were more insightful to learn relevant patterns of enticing and engaging content.

7.3 Limitations and Future Work

In this paper we present an analytical framework for retweetability prediction on Twitter that allows real-time monitoring of social activism campaigns to better mobilize participants, in particular, for engineering diversity related campaigns. With the help of our feature design (fine- and coarse-grained features) relying on realtime data from tweets in a given social hashtag movement, the proposed framework outperformed all of 4 baselines with 9.4% and 13.94% increase in accuracy for the two experimental datasets we used. We found that sentiment was important to increase people’s participation in these engineering diversity related campaigns. This framework can be applied in other social hashtag campaigns where time and resources are limited such as disaster response campaigns where people use their real identity to raise awareness and seek help. This work has implications for improving the participation of women in engineering education and the engineering workforce.

The work presented here was supported in part by U.S. National Science Foundation Awards: DUE-1707837 and DUE-1712129. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author and do not necessarily reflect the views of the funding agencies. The authors also would like to thank ARGO 7, a research computing cluster provided by the Office of Research Computing at George Mason University to facilitate the computation needed for this research.

[1] 2007. National Science Foundation (2007) Women, minorities, and persons with disabilities in science and engineering. Arlington, VA: Division of Science Resources Statistics. http://www.nsf.gov/statistics/wmpd [2] 2009. American Society of Engineering Education. http://www.asee.org/ publications/connections/index.cfm [3] 2010. UK Resource Centre for Women in Science, Engineering and Technology (2010) Students in higher education 2009. http://theukrc.org/files/useruploads/ files/finalsept15th15.42ukrcstatisticsguide2010.pdf [4] Peng Bao, Hua-Wei Shen, Wei Chen, and Xue-Qi Cheng. 2013. Cumulative effect in information diffusion: empirical study on a microblogging network. PloS one 8, 10 (2013), e76027.

7http://wiki.orc.gmu.edu/index.php/MainPage accessed on May 7, 2021

[5] Wendy Bastalich, Julie Mills, Suzanne Franzway, Judith Gill, Rhonda Sharp, et al. 2003. ’I Had This Real Feeling That It Was a Boys Club’. In Engineering Education for a Sustainable Future: Proceedings of the 14th Annual Conference for Australasian Association for Engineering Education and 9th Australasian Women in Engineering Forum. Australasian Association for Engineering Education, 499. [6] Mareike Bayer, Werner Sommer, and Annekathrin Schacht. 2012. Font size matters—emotion and attention in cortical responses to written words. PloS one 7, 5 (2012), e36042. [7] W Lance Bennett and Alexandra Segerberg. 2012. The logic of connective action: Digital media and the personalization of contentious politics. Information, communication & society 15, 5 (2012), 739–768. [8] David M Blei, Andrew Y Ng, and Michael I Jordan. 2003. Latent dirichlet allocation. Journal of machine Learning research 3, Jan (2003), 993–1022. [9] Vincent D Blondel, Jean-Lou Guillaume, Renaud Lambiotte, and E Lefebvre. 2011. The Louvain method for community detection in large networks. J of Statistical Mechanics: Theory and Experiment 10 (2011), P10008. [10] Axel Bruns and Jean E Burgess. 2011. The use of Twitter hashtags in the formation of ad hoc publics. In Proceedings of the 6th European Consortium for Political Research (ECPR) General Conference 2011. [11] Justin Cheng, Lada Adamic, P Alex Dow, Jon Michael Kleinberg, and Jure Leskovec. 2014. Can cascades be predicted?. In Proceedings of the 23rd international conference on World wide web. ACM, 925–936. [12] Justin Cheng, Jon Kleinberg, Jure Leskovec, David Liben-Nowell, Karthik Subbian, Lada Adamic, et al. 2018. Do Diffusion Protocols Govern Cascade Growth?. In Twelfth International AAAI Conference on Web and Social Media. [13] Cindy Chung and James W Pennebaker. 2007. The psychological functions of function words. Social communication (2007), 343–359. [14] Jae Eun Chung. 2017. Retweeting in health promotion: Analysis of tweets about Breast Cancer Awareness Month. Computers in Human Behavior 74 (2017), 112– 119. [15] Nahed Eltantawy and Julie B Wiest. 2011. The Arab spring| Social media in the Egyptian revolution: reconsidering resource mobilization theory. International journal of communication 5 (2011), 18. [16] Julia Evetts. 1998. Managing the technology but not the organization: Women and career in engineering. Women in management review 13, 8 (1998), 283–290. [17] Ullas Gargi, Wenjun Lu, Vahab Mirrokni, and Sangho Yoon. 2011. Large-scale community detection on youtube for topic discovery and exploration. In Fifth International AAAI Conference on Weblogs and Social Media. [18] Elizaveta Gaufman. 2018. The Trump carnival: popular appeal in the age of misinformation. International Relations 32, 4 (2018), 410–429. [19] Stuart Geman and Donald Geman. 1987. Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. In Readings in computer vision. Elsevier, 564–584. [20] Benjamin Gleason. 2013. # Occupy Wall Street: Exploring informal learning about a social movement on Twitter. American Behavioral Scientist 57, 7 (2013), 966–982. [21] Spencer P Greenhalgh, Joshua M Rosenberg, K Bret Staudt Willet, Matthew J Koehler, and Mete Akcaoglu. 2020. Identifying multiple learning spaces within a single teacher-focused Twitter hashtag. Computers & education 148 (2020), 103809. [22] Maximillian Hänska Ahy. 2016. Networked communication and the Arab Spring: Linking broadcast and social media. New Media & Society 18, 1 (2016), 99–116. [23] Chip Heath. 1996. Do people prefer to pass along good or bad news? Valence and relevance of news as predictors of transmission propensity. Organizational behavior and human decision processes 68, 2 (1996), 79–94. [24] Tim Highfield. 2017. Social media and everyday politics. John Wiley & Sons. [25] David W Hosmer Jr, Stanley Lemeshow, and Rodney X Sturdivant. 2013. Applied logistic regression. Vol. 398. John Wiley & Sons. [26] Jue Huang and Jianxiong Xue. 2015. The Predictive Power of Content and Temporal Features of Posts in Information Dissemination in Microblogging. Journal of China Tourism Research 11, 2 (2015), 150–165. [27] Aditya Johri, Cassie Heyman-Schrum, Daniel Ruiz, Aqdas Malik, Habib Karbasian, Rajat Handa, and Hemant Purohit. 2018. More Than an Engineer: Intersectional Self-Expressions in a Hashtag Activism Campaign for Engineering Diversity. In Proceedings of the 1st ACM SIGCAS Conference on Computing and Sustainable Societies. ACM, 44. [28] Aditya Johri, Habib Karbasian, Aqdas Malik, Rajat Handa, and Hemant Purohit. 2018. How Diverse Users and Activities Trigger Connective Action via Social Media: Lessons from the Twitter Hashtag Campaign# ILookLikeAnEngineer. In Proceedings of the 51st Hawaii International Conference on System Sciences. [29] Karen Sparck Jones. 1997. Readings in information retrieval. Morgan Kaufmann. [30] Habib Karbasian. 2020. ILookLikeAnEngineer and WomenInEngineering Datasets. Mendeley Data (2020). https://doi.org/10.17632/c6x5vy4xdw.1 [31] Habib Karbasian, Hemant Purohit, Rajat Handa, Aqdas Malik, and Aditya Johri. 2018. Real-Time Inference of User Types to Assist with more Inclusive and Diverse Social Media Activism Campaigns. In Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society. ACM, 171–177.

[32] Johanna Kissler, Cornelia Herbert, Peter Peyk, and Markus Junghofer. 2007. Buzzwords: early cortical responses to emotional words during reading. Psychological Science 18, 6 (2007), 475–480. [33] Amy L Kotsenas, Makala Arce, Lee Aase, Farris K Timimi, Colleen Young, and John T Wald. 2018. The strategic imperative for the use of social media in health care. Journal of the American College of Radiology 15, 1 (2018), 155–161. [34] Andrey Kupavskii, Liudmila Ostroumova, Alexey Umnov, Svyatoslav Usachev, Pavel Serdyukov, Gleb Gusev, and Andrey Kustarev. 2012. Prediction of retweet cascade size over time. In Proceedings of the 21st ACM international conference on Information and knowledge management. ACM, 2335–2338. [35] Olivier Luminet IV, Patrick Bouts, Frédérique Delie, Antony SR Manstead, and Bernard Rimé. 2000. Social sharing of emotion following exposure to a negatively valenced situation. Cognition & Emotion 14, 5 (2000), 661–688. [36] Zhunchen Luo, Miles Osborne, Jintao Tang, and Ting Wang. 2013. Who will retweet me?: finding retweeters in twitter. In Proceedings of the 36th international ACM SIGIR conference on Research and development in information retrieval. ACM, 869–872. [37] Aqdas Malik, Aditya Johri, Rajat Handa, Habib Karbasian, and Hemant Purohit. 2018. How social media supports hashtag activism through multivocality: A case study of# ILookLikeanEngineer. First Monday 23, 11 (2018). [38] Andrew Kachites McCallum. 2002. Mallet: A machine learning for language toolkit. (2002). [39] Judith S McIlwee and J Gregg Robinson. 1992. Women in engineering: Gender, power, and workplace culture. SUNY Press. [40] Quinn McNemar. 1947. Note on the sampling error of the difference between correlated proportions or percentages. Psychometrika 12, 2 (1947), 153–157. [41] Sharon Meraz and Zizi Papacharissi. 2013. Networked gatekeeping and networked framing on# Egypt. The international journal of press/politics 18, 2 (2013), 138–166. [42] Gloria E Miller. 2004. Frontier masculinity in the oil industry: The experience of women engineers. Gender, Work & Organization 11, 1 (2004), 47–73. [43] Nasir Naveed, Thomas Gottron, Jérôme Kunegis, and Arifah Che Alhadi. 2011. Bad news travel fast: A content-based analysis of interestingness on twitter. In Proceedings of the 3rd international web science conference. ACM, 8. [44] Venkata Kishore Neppalli, Murilo Cerqueira Medeiros, Cornelia Caragea, Doina Caragea, Andrea H Tapia, and Shane E Halse. 2016. Retweetability Analysis and Prediction during Hurricane Sandy.. In ISCRAM. [45] Venkata Sasank Pagolu, Kamal Nayan Reddy, Ganapati Panda, and Babita Majhi. 2016. Sentiment analysis of Twitter data for predicting stock market movements. In 2016 international conference on signal processing, communication, power and embedded system (SCOPES). IEEE, 1345–1350. [46] James W Pennebaker, Ryan L Boyd, Kayla Jordan, and Kate Blackburn. 2015. The development and psychometric properties of LIWC2015. Technical Report. [47] James W Pennebaker, Matthias R Mehl, and Kate G Niederhoffer. 2003. Psychological aspects of natural language use: Our words, our selves. Annual review of psychology 54, 1 (2003), 547–577. [48] Kim Peters, Yoshihisa Kashima, and Anna Clark. 2009. Talking about others: Emotionality and the dissemination of social information. European Journal of Social Psychology 39, 2 (2009), 207–222. [49] Sasa Petrovic, Miles Osborne, and Victor Lavrenko. 2011. Rt to win! predicting message propagation in twitter. In Fifth International AAAI Conference on Weblogs and Social Media. [50] Abigail Powell, Barbara Bagilhole, and Andrew Dainty. 2009. How women engineers do and undo gender: Consequences for gender equality. Gender, work & organization 16, 4 (2009), 411–428. [51] Jenny Preece. 2000. Online communities: designing usability, supporting sociability. Industrial Management & Data Systems 100, 9 (2000), 459–460. [52] Jenny Preece. 2001. Sociability and usability in online communities: Determining and measuring success. Behaviour & Information Technology 20, 5 (2001), 347–356. [53] Hemant Purohit, Yiye Ruan, Amruta Joshi, Srinivasan Parthasarathy, and Amit Sheth. 2011. Understanding user-community engagement by multi-faceted features: A case study on twitter. In WWW 2011 Workshop on Social Media Engagement (SoME). [54] Alan Radley and Susan E Bell. 2007. Artworks, collective experience and claims for social justice: The case of women living with breast cancer. Sociology of health & illness 29, 3 (2007), 366–390. [55] Christian Reuter and Marc-André Kaufhold. 2018. Fifteen years of social media in emergencies: a retrospective review and future directions for crisis informatics. Journal of Contingencies and Crisis Management 26, 1 (2018), 41–57. [56] Russell Rickford. 2016. Black lives matter: Toward a modern practice of mass struggle. In New Labor Forum, Vol. 25. SAGE Publications Sage CA: Los Angeles, CA, 34–42. [57] Bernard Rimé. 2009. Emotion elicits the social sharing of emotion: Theory and empirical review. Emotion review 1, 1 (2009), 60–85. [58] Daniel M Romero, Brendan Meeder, and Jon Kleinberg. 2011. Differences in the mechanics of information diffusion across topics: idioms, political hashtags, and complex contagion on twitter. In Proceedings of the 20th international conference on World wide web. 695–704.

[59] Anja Rudat and Jürgen Buder. 2015. Making retweeting social: The influence of content and context information on sharing news in Twitter. Computers in Human Behavior 46 (2015), 75–84. [60] Anja Rudat, Jürgen Buder, and Friedrich W Hesse. 2014. Audience design in Twitter: Retweeting behavior between informational value and followers’ interests. Computers in human behavior 35 (2014), 132–139. [61] H Andrew Schwartz, Johannes C Eichstaedt, Margaret L Kern, Lukasz Dziurzynski, Stephanie M Ramones, Megha Agrawal, Achal Shah, Michal Kosinski, David Stillwell, Martin EP Seligman, et al. 2013. Personality, gender, and age in the language of social media: The open-vocabulary approach. PloS one 8, 9 (2013), e73791. [62] Stephen M Smith and Richard E Petty. 1996. Message framing and persuasion: A message processing analysis. Personality and Social Psychology Bulletin 22, 3 (1996), 257–268. [63] Insoo Son, Dongwon Lee, and Youngkyu Kim. 2013. Understanding the Effect of Message Content and User Identity on Information Diffusion in Online Social Networks.. In PACIS. 8. [64] Stefan Stieglitz and Linh Dang-Xuan. 2012. Political communication and influence through microblogging–An empirical analysis of sentiment in Twitter messages and retweet behavior. In 2012 45th Hawaii International Conference on System Sciences. IEEE, 3500–3509. [65] Stefan Stieglitz and Linh Dang-Xuan. 2013. Emotions and information diffusion in social media—sentiment of microblogs and sharing behavior. Journal of management information systems 29, 4 (2013), 217–248. [66] Bongwon Suh, Lichan Hong, Peter Pirolli, and Ed H Chi. 2010. Want to be retweeted? large scale analytics on factors impacting retweet in twitter network. In 2010 IEEE Second International Conference on Social Computing. IEEE, 177–184. [67] Jeannette Sutton, C Ben Gibson, Nolan Edward Phillips, Emma S Spiro, Cedar League, Britta Johnson, Sean M Fitzhugh, and Carter T Butts. 2015. A crosshazard analysis of terse message retransmission on Twitter. Proceedings of the National Academy of Sciences 112, 48 (2015), 14793–14798. [68] Jeannette Sutton, C Ben Gibson, Emma S Spiro, Cedar League, Sean M Fitzhugh, and Carter T Butts. 2015. What it takes to get passed on: message content, style, and structure as predictors of retransmission in the Boston Marathon bombing response. PLoS one 10, 8 (2015), e0134452. [69] Briony Swire, Adam J Berinsky, Stephan Lewandowsky, and Ullrich KH Ecker. 2017. Processing political misinformation: comprehending the Trump phenomenon. Royal Society open science 4, 3 (2017), 160802. [70] Chade-Meng Tan, Yuan-Fang Wang, and Chan-Do Lee. 2002. The use of bigrams to enhance text categorization. Information processing & management 38, 4 (2002), 529–546.

[71] Ying Tang and Khe Foon Hew. 2017. Using Twitter for education: Beneficial or simply a waste of time? Computers & education 106 (2017), 97–118. [72] V Vapnik. 1995. The Nature of Statistical Learning Theory· 6·[MJ New York. Springer-Verlag 1 (1995), 995. [73] Vladimir Vapnik and Vlamimir Vapnik. 1998. Statistical learning theory. [74] Hao Wang, Yiping Li, Zhuonan Feng, and Ling Feng. 2013. ReTweeting analysis and prediction in microblogs: An epidemic inspired approach. China Communications 10, 3 (2013), 13–24. [75] Rong Wang, Wenlin Liu, and Shuyang Gao. 2016. Hashtags and information virality in networked social movement: Examining hashtag co-occurrence patterns. Online Information Review 40, 7 (2016), 850–866. [76] Yuan Wang, Jie Liu, Yalou Huang, and Xia Feng. 2016. Using hashtag graph-based topic model to connect semantically-related words without co-occurrence in microblogs. IEEE Transactions on Knowledge and Data Engineering 28, 7 (2016), 1919–1933. [77] Chen Xing, Yuan Wang, Jie Liu, Yalou Huang, and Wei-Ying Ma. 2016. Hashtagbased sub-event discovery using mutually generative lda in twitter. In Thirtieth AAAI Conference on Artificial Intelligence. [78] Zi Yang, Jingyi Guo, Keke Cai, Jie Tang, Juanzi Li, Li Zhang, and Zhong Su. 2010. Understanding retweeting behaviors in social networks. In Proceedings of the 19th ACM international conference on Information and knowledge management. ACM, 1633–1636. [79] Tauhid R Zaman, Ralf Herbrich, Jurgen Van Gael, and David Stern. 2010. Predicting information spreading in twitter. In Workshop on computational social science and the wisdom of crowds, nips, Vol. 104. Citeseer, 17599–601. [80] Jing Zhang, Jie Tang, Juanzi Li, Yang Liu, and Chunxiao Xing. 2015. Who influenced you? predicting retweet via social influence locality. ACM Transactions on Knowledge Discovery from Data (TKDD) 9, 3 (2015), 25. [81] Qi Zhang, Yeyun Gong, Ya Guo, and Xuanjing Huang. 2015. Retweet behavior prediction using hierarchical dirichlet process. In Twenty-Ninth AAAI Conference on Artificial Intelligence. [82] Qi Zhang, Yeyun Gong, Jindou Wu, Haoran Huang, and Xuanjing Huang. 2016. Retweet prediction with attention-based deep neural network. In Proceedings of the 25th ACM international on conference on information and knowledge management. ACM, 75–84.