Sentiment Analysis: Analyzing the Impact of a PR crisis — Project
Greta Hess, Martin Kvammen, Rosa Kristoffersen, Medou Odeta Barkauskaitė, Linea Weien Lippert, Stefan Figoluschka
Every year there is a series of stories about organizations that misstep, in small and large ways. A Public Relations crisis is a nightmare scenario that befalls even the most prestigious and well-meaning businesses. It can manifest itself in many different forms — it could occur as a result of a mistake in one area of the business, such as operations (e.g. failing to secure enough inventory to satisfy demand) or it might happen after the marketing arm threads into troubled waters (e.g. an intensive Twitter reply to a customer complaint or an offensive ad campaign). A PR crisis can damage the brand, kill the customer trust, and hinder — sometimes permanently — any preceding marketing or sales efforts. For some businesses, it’s just a bump in the road. For others, it’s the beginning of the end. So how does it (if at all) affect people’s perception of the company?
One of the ways to scrutinize and evaluate customers’ responses to the phenomenon is to employ the so-called Sentiment Analysis. Sentiment Analysis (aka opinion mining or emotion AI) refers to the process of computationally identifying and categorizing opinions expressed in a piece of text in order to determine whether the writer’s attitude towards a particular topic, product, etc. is positive, negative, or neutral. The Analysis is widely applied to voice-of-customer materials such as reviews and survey responses, online and social media.
In this project, we aspired to conduct such analysis to better comprehend the customers’ feedback to one of the most profound recent PR crises in later times: the Boeing 737 Max crisis. The project was further motivated by the opportunity to practically apply the skills acquired through TechLabs tracks.
The Boeing 737 Max Crisis in Brief
Boeing has been under scrutiny since the fatal crashes of two 737 Max planes, which killed 346 people in late 2018 and early 2019. The truth came out that Boeing hadn’t alerted operators about all the new features on its latest popular airplane. The autopilot system that engaged during takeoff was susceptible to error — and pilots were furious that they hadn’t been warned. The Boeing 737 Max was the fastest-selling airliner in Boeing history, selling more than 5,000 aircraft over the past few years. But in April and May 2019, it sold zero.
The crisis seems to unravel to be of a malevolent nature because of Boeing’s tone-deaf response to consumer concerns and its persistent claims that the aircraft was safe. A lack of transparency and a failure to listen doomed Boeing’s early attempts to address public concerns. It is speculated that Boeing is a case study in what not to do. A cardinal rule in a PR crisis is to guide the narrative before others guide it for you, but Boeing’s apparent passive response has allowed others to tell the story and the crisis to spur at the detriment of the brand’s reputation.
Now the airline can only hope it can regain the trust of aviators and passengers without any deceitful tactics. Can Boeing bounce back and return to its position as the world’s unquestioned #1 plane maker? Through the Sentiment Analysis, our goal was to explore how greatly Boing’s image has suffered and how the brand reputation has been damaged as a consequence of the recent incidents.
From Inspiration to Conceptualization
The project drew inspiration from Meltwater, a Norwegian founded company offering a range of data analysis tools for media traction. By signing up to Meltwater, firms can gain valuable insights into the performances of the brands’ PR activities as their technologies crawl the internet for relevant data. The conceptualization of our project relies on a similar business concept — namely, to provide firms with the opportunity to track their reputational performance in real-time by analyzing public data.
As social media is increasingly becoming the host of most, if not all, public communication, users’ activity comprise a powerful influence and may largely define a brands’ connotation. This makes social media an interesting and highly relevant source for data on brand publicity. While Meltwater offers analysis across a wide range of data, we thus limit ourselves to the social media sphere in which we analyze Twitter posts in the wake of the Boeing 737 Max crisis, as these might be well-representative of the social atmosphere.
The Team
The Team consists of 6 dedicated Digital Shapers enrolled in Data Science and Web Development tracks. We are of vastly diverse backgrounds, ranging from IT to legal, which helps us to comprehensively assess processes and tasks from different angles and come up with innovative solutions. We believe that even though the expertise acquired through engaging in our tracks is a valuable asset, engaging and motivating one’s colleagues is the real power of development. Thus, lessons are learned — both — on and off tracks.
The Project Outline: Initial Ambition and Approach
In order to pursue our inspiration and accommodate our IT learning tracks, we set out high ambitions that were twofold, consisting of 1) web development and 2) sentiment analysis. Hence, our aspiration was not only to develop a data analysis tool, but also to define the outline of a website that would host the sentiment analysis and allow the user to manipulate the data timeframe. The focus was mainly on the creation of a front-end design that would compliment the project’s business concept and leave us with the feeling of a holistic product.
Secondly, yet most importantly, our target was to create a viable sentiment analysis. By extracting data from Twitter, we aimed to code a Python script that would enable us to render posts with reference to Boeing and ascribe each word a value using a score-based method, such as Afinn. Ultimately, this would result in a sentiment score to be plotted along the y-axis with time factoring the x-axis. Our goal therefore included plotting the data in Python, enabling us to successfully visualize our dataset.
The project was structured as follows: first, we built the data foundation by scraping data from Twitter using an API. The second part concerned the actual programming of our sentiment analysis in priming the data and applying a score-based method. This also constituted the greatest and most important part of our project. Lastly, we developed a website with the main focus being the front-end interface and integration with the Python script creating the sentiment analysis.
Building the Data Foundation
As we set out to build our data foundation, the original idea was to retrieve tweets by using twitter developer API with the Tweepy package. Therefore, we went through the steps of applying for access as well as an account to be allowed to pull tweets through the API. As soon as the access was granted we could start working with the API. Initially, we wanted to scrape Twitter for the hashtags #boeing, #737 and #737max. However, it was deemed too complicated with Tweepy. Consequently, we investigated other solutions and managed to produce code that pulled all the tweets produced by Boeing themselves. As date specification was not possible, all the most recent tweets were pulled which counted more than 2500 tweets and dated back to 2012. The data were then transformed to a Pandas series which sequentially could be downloaded as a CSV file with the columns “Index”, “Date” and “Tweet”.
Constructing the Analysis
As we received more than 2500 tweets we could start working with the data analysis. Initially we installed the required packages (Numpy, Pandas, Tweepy, Matplotlib, Seaborn, Afinn among others), and imported the csv-file to Python as a DataFrame. Having a first glance at the data in Python, the order of the tweets was going from the latest date (2020–08–01) as index 0 and the first date in the dataset (2012–05–17) as index 2696. When plotting the data points, an opposite order would make more sense, thus, we both indexed and sorted values by ‘Date’ to set the first date as index 0. Furthermore, for Python to know that the ‘Date’ column is a DatetimeIndex, we passed the ‘Date’ column to ‘datetime’.Then we applied the Afinn score to each tweet which would add a new column ‘afinn_score’ to the dataset. As a step in exploring the applied Afinn method, we used the ‘describe’-function and found out that the minimum value was -9 and the maximum value was 15.
When further investigating the new ‘afinn’ column, we used ‘head’ and ‘tail’ to observe the most and the least extreme scores. Further, we grouped by date and plotted a distribution of all the 2697 sentiments (see figure 1 below).
It seemed that the distribution was rather skewed (compared to a Gaussian distribution) towards the more negative values. It came to our information that the raw Afinn score is highly affected by the number of words per text (i.e. tweet) meaning that the longer tweets, the more positive the score will be, simply because it contains more words. We discovered that the word count in the Boeing tweets vary from 2 to 28 words with a median of 17. Hence, there is a big variation in word counts with a tendency to longer tweets. Therefore, we tried to adjust for this bias by dividing the original score by the word count, and from this we got the ‘afinn_adjusted’ which we applied to a new column.
In terms of visualizing the Afinn scores as a time series, our data set was rather large, and thus, the plot was at first glance quite messy and hard to gain insights from. Therefore, we resampled the data set using both weekly, monthly and yearly mean-resampling (see plots below).
While the yearly mean plot shows a heavy downturn around year 2019 and thus, could indicate that the 737 Max has a negative effect on the Twitter sentiments, the monthly mean plot shows rather volatile fluctuations over the entire time period since 2012 with could by contrast indicate that down- and upturns will inevitably occur over time; PR crisis or not. As a result, we divided the data set into two timeframes: one before the 737 Max PR crisis and one during. It seems that the mean does not differ significantly between the two periods, and further, the standard deviation is relatively high.
Considering the messy nature of sentiments in general and the potential effect of outliers — for which there seem to be quite some of (see boxplot below) — we suggest that the sentiments of Boeing tweets are further investigated by applying deep learning methods (e.g. BERT) in order to be more significantly confident in a PR crisis’ effect on sentiments.
Although more refinement could be done, we were generally able to produce a sentiment analysis using the Afinn score-based method. By resampling the data, we were able to visualize the data in a graph that would give us an overall indication of the brand’s performance over the years. Figure 3 provides the most distinct picture, confirming a significant dive in the curve starting in 2018 when the first Boeing 737 Max crashed. According to our graph, the crisis most likely resulted in great reputational damage with a lasting negative effect, as the curve has not yet recovered. Through the sentiment analysis, we were thus briefly able to explore how greatly Boeing’s image has suffered as a consequence of the recent incidents based on social media data.
Developing the Front-end
For the front-end, we initially designed the preliminary interface to include several options for displaying and manipulating the data. Thus, aside from the primary homepage for displaying the sentiment analysis, we designed two additional pages for data visualization, one of which would visualize the geo-location of the Twitter posts on a map, and the second of which would display the most frequent hashtags relating to the Boeing crisis.
However, we quickly came to realise the magnitude of our ambitions and significantly scaled down expectations towards the web development as a result. As this was the last part of the project, we naturally came to dedicate less time to this section which we deemed a reasonable prioritization, as we did not consider this a bearing aspect of the project. Nonetheless, we were able to code the primary homepage meant for hosting our sentiment analysis, consisting of a simplified set of an HTML, CSS and javascript document. Although the back-end was missing, we successfully produced a viable front-end platform making use of Bootstrap 4, adding several web components, such as forms, dropdown menus, toggle buttons etc. The result can be seen in the picture below.
The Roadblocks
As the learning phase on the respective tracks and the project work went hand in hand, a major challenge consisted in the coordination of these two elements. Finding the right balance between developing the needed skills and progressing with the project was a constant struggle, as the ambitions for our project would constantly force us to develop new skills while the project was unfolding. The varying progress on the tracks of the individual team members also contributed to a more challenging coordination of the project work and thus hampered the progress of the project. All this led to the project going back and forth and coming to a halt at some points. This is certainly one of the reasons for the deviation of the project results from the initial ambitions of the project.
Furthermore, as we were learning along with the project — with most of us starting from zero technical knowledge — the project ended up rather fragmented with data analysis and web development comprising two separate parts. Thus, we never succeeded in the integration of the data analysis and web development elements. Our initial aim in this regard appeared to be, in fact, too ambitious and demanded a much higher skill level than what we were able to acquire within the given time constraints.
Conclusion
Confronting the end result with our initial ambitions, one might question the success of the project, which did not end up producing the holistic product we initially hoped for. As to why, several factors can be pointed to, some of which are mentioned above. Concluding on that note would, however, not serve the project justice, which did in fact live up to its raison d’etre: namely, to facilitate learning and provide an opportunity to make use of our newly acquired skills. Setting forth high ambitions turned out to be an important driving force that continually forced us to climb the steep learning curve — thus, not only were our skills put to the test, but others we acquired along the way as we stumbled towards the finishing line.
Connect with the Authors
Greta Hess : Data Science track
Martin Møinichen Kvammen : Data Science track
Medou Odeta Barkauskaitė : Data Science track
Rosa Kristoffersen : Data Science track
Stefan Figoluschka : Data Science track
Linea Weien Lippert : Web Development track
