Korina Mavrokordatou: Data Science Project

(a) Why is the movement necessary?

During 2013-2021, police have killed 9879 people, with 98.3% of killings resulting to officers being acquitted. Figure 1 gathers evidence from US largest 50 cities to prove that crime rates do not determine police violence rates. I combined data from 2 different sources; Mapping Police Violence and Federal Bureau of Investigation, Crime Data Explorer both downloaded as csv files, to performing a regression of violent crime rates on police killings rates. The regression yields an R-squared value of 0.035 and a coefficient which is not significant at 5% level, proving that there is no correlation between the 2 variables. Figure 2 then shows Black-White disparity in the same cities for 2013-2021 with values as high as 23.5. Data is again downloaded from Mapping Police Violence. The disparity is also depicted in Figure 3; where it is clear that Black people are killed at much higher rates than white people, given the population structure. The data used was easy to access, but a lot of cleaning was required, performed in Stata. The main challenge faced was the merging of the data which required exactly matching the states; a fairly difficult task when they are not identically formatted.

(b) Social Media data analysis:

This was the most challenging part of the project; since data from Social Media platforms is not as easily accessible. To fetch data from Reddit; I created a developer’s account to obtain a unique client_id and client_secret; used as API keys. I then used PRAW, to fetch the data and pandas to store the data. Further analysis was conducted using TextBlot and Vader libraries to perform a sentiment analysis on the comments of the top 10-scored Reddit posts under the subreddit ‘r/BlackLivesMatter’. I used both libraries since I wanted to test how each one performs. Through research I found that Textblot includes subjectivity parameters whereas Vader responds better to special characters and emojis. I initially signed up for all Facebook, Twitter and Reddit APIs but the FacebookAPI does not allow fetching for specific words and my Twitter developer application was suspended by mistake. Twitter-support did not respond to reactivate my key until the 5th of January. Hence, the Twitter sentiment dataset was downloaded from Kaggle, where the TextBlot library was used. However, the dataset required a lot of cleaning, performed in Stata.

Reddit’s score represents the number of upvotes minus downvotes, hence as seen in Figure 4, the movement gained a lot of positive attention, with the top-scored post at 15864. Reading through the top-20 scored posts it’s obvious that the tone varies, from positive to negative. Figure 5 supports this statement, displaying much variation in the comment section of the posts examined; with the majority of the posts having a neutral tone. Both Textblot and Vader calculate higher numbers of positive sentiment tweets than negative ones, however this is a small sample and in order to yield conclusive results, a bigger sample should have been examined. Hence, the analysis is expanded by including the Twitter dataset, consisting of 1000 tweets shown in Figure6. The Twitter dataset supports the view that the movement brought about many toxic and hateful comments, evident in Figure 6 where negative-sentiment tweets are more than positive ones.

Conclusion

To conclude, the BlackLivesMatter movement sparked a lot of controversy among US citizens and became the source of hateful comments circulating social media platforms. Although the movement is really important, it is unclear whether the social-media approach is enough to provide a consistent barrier against racism in the US. Figure 7 displays that although support for BlackLivesMatter increased during the George-Floyd Protests (June2020), a year and a half later, support declined; especially in the White group, where share of respondents opposing the movement was even higher than in January 2020. This raises a question of whether support was raised due to it being a “trend” and whether people took this movement as an opportunity to attack certain beliefs. The dataset for Figure7 was obtained from Civiqs and formatted in Excel.

Data Sources and Useful links

https://blacklivesmatter.com/?__cf_chl_jschl_tk__=p0HvahzdsfBbq9EmEr14eqMs2HuyaT6nymBMcTXL.9U-1641532848-0-gaNycGzNCJE

https://civiqs.com/results/black_lives_matter?uncertainty=true&annotations=true&zoomIn=true

https://mappingpoliceviolence.org

https://towardsdatascience.com/automate-sentiment-analysis-process-for-reddit-post-textblob-and-vader-8a79c269522f

https://www.pnas.org/content/116/32/15877

https://www.kaggle.com/yash612/black-lives-matter-twitter-dataset

https://www.harpersbazaar.com.sg/life/celebrities-gone-above-beyond-support-blm/

https://crime-data-explorer.fr.cloud.gov/pages/downloads

https://towardsdatascience.com/scraping-reddit-data-1c0af3040768

https://towardsdatascience.com/a-beginners-guide-to-sentiment-analysis-in-python-95e354ea84f6