- Open Access
Analysis of tweets on toothache during the COVID-19 pandemic using the CrystalFeel algorithm: a cross-sectional study
BMC Oral Health volume 21, Article number: 418 (2021)
Reasons such as the lack of information on the COVID-19 disease, lack of proven treatment for it, uncertainty about the duration of the pandemic, or social isolation affect people’s mental health. This study aimed to analyze the emotional type and intensity in tweets on toothache posted during the COVID-19 pandemic.
Using the Twitter Search API, we collected tweets in English associated with the keywords “Corona, Toothache” “Corona, Tooth, Pain” “Corona, Dental Pain” “Covid-19, Toothache” “Covid-19, Tooth, Pain” and “Covid-19, Dental Pain” posted between March 11, 2020 and June 30, 2020 all around the world. After the relevant inclusion and exclusion criteria were applied, 426 posts were selected and analyzed using the CrystalFeel algorithm, a sensitivity analytical technology with proven accuracy. The chi-square test (SPSS v23, IBM) was used to compare emotions and emotional intensities according to the words used.
It was determined that 80.3% of the participants experienced fear and 61.7% had a negative emotional intensity. There was no statistically significant difference between the distributions of emotions according to the words without time distinction (p = 0.136). There was a statistically significant difference between the distributions of emotional intensity according to the words without time distinction (p = 0.006). The keyword “Corona, Toothache” was used the most frequently by 30.8% of the participants.
This study is the first to analyze the emotional reactions of individuals who experienced toothaches during the COVID-19 pandemic using the CrystalFeel algorithm. Monitoring the social media posts of individuals experiencing toothache during the pandemic will help reduce fear and anger emotions and design public information messages that are compatible with the target group’s needs.
The World Health Organization (WHO) announced on March 11, 2020 that the outbreak of a new type of coronavirus (COVID-19) constituted an alarming international public health emergency and declared it a global pandemic . The outbreak, which spread to the whole world and turned into a pandemic in a short amount of time, became of particular concern to dentists and the patient group in need of dental treatment due to the characteristics of its transmission routes. Working close to the patient and the aerosols and droplets generated during the treatment constitute novel and significant risks for dentists regarding highly infectious epidemics, such as COVID-19 [2, 3]. During the pandemic, while some dental institutions and clinics completely stopped the admission of patients, some of them continued to provide only emergency treatment and care services .
Pain is a difficult experience and is generally the most important condition requiring health support . Severe toothache, dentoalveolar trauma, and swelling due to abscess are the main emergency dental interventions, and pain control should be provided . Untreated pain has physical effects, such as difficulty breathing, increased heart rate and blood pressure, myocardial oxygen requirement, and increased stress hormones, vomiting, and muscle tension . Toothache can affect social well-being and negatively impact the individual’s quality of life . The frequency and severity of dental pain negatively affect daily nutrition, sleep, and smiling and oral care [9, 10]. In addition to physical effects, pain may also increase emotions such as anxiety, fear, distress, helplessness, and hopelessness . Furthermore, the management of this condition imposes a huge financial burden on society and individuals [12, 13].
Social media sites (Facebook, Instagram, and Twitter) allow users to play active roles in reporting and disseminating news and sharing insights and apprehensions while disseminating interpretations of health events outside a public health context . Twitter is a well-known microblogging service where users post messages, denoted “tweets”. These are short messages with a maximum length of (currently) 280 characters. Since people who post those messages share preferences and opinions with other users, tweets are a valuable source of people’s opinions and sentiments . Over 645 million registered Twitter users exist globally to distribute over 58 million tweets daily . The community of Twitter users reflects a diverse and rapidly growing global population .
In the past decade, social media analytic tools have been utilized to monitor public sentiments and communication patterns of public health emergencies like the Ebola and Zika epidemics . The analysis of the emotional responses of individuals who experienced toothache during the COVID-19 pandemic through texts shared on social media may provide a quick understanding of the needs in health-related planning. In this study, the authors aimed to analyze the number and content of tweets on toothache during the peak of the COVID-19 pandemic by using the CrystalFeel algorithm, which is a sensitivity analytical artificial intelligence technology.
Considering the intensity distributions; The number of samples to be included in the study was determined as 420 with 95% confidence (1-α), 80% test power (1-β) and w = 0.1368 effect size .
Tweets in English posted between 11 and 2020 and 30 June 2020 worldwide were included in the study. Authors analyzed the data considering those periods: 11 March 2020–31 March 2020; 1 April 2020–3 April 02020; 1 May 2020–31 May 2020; and 1 June 2020–30 June 2020, because the World Health Organization (WHO) declared COVID-19 a global epidemic on March 11, 2020. Popular tweets were found using the keywords “Corona, Toothache” “Corona, Tooth, Pain” “Corona, Dental Pain” “Covid-19, Toothache” “Covid-19, Tooth, Pain” and “Covid-19, Dental Pain” in the Twitter Search API. These tweets represented the emotions of individuals who experienced toothache after the COVID-19 pandemic started.
Two researchers (A.C and H.A) collected the tweet texts regarding its inclusion and exclusion criteria, and the posts were analyzed using CrystalFeel Algorithm. The researchers checked the data obtained using the CrystalFeel Algorithm. Account of Twitter was used only to collect information (with no followers). Only the tweet texts were collected, excluding age, gender, and user type. Exclusion criteria included the following: advertising or promotional posts, posts involving statements that did not include content that could be used to make inferences about how people felt about their braces, hyperlinks (content beginning with “http://” or “https://”), posts that had been posted previously and then re-posted by another user (retweets), irrelevant, incoherent or ambiguous posts, non-English posts, and tweets sent from corporate accounts—since these were posted for informational purposes and had no emotional meaning. Metaphorical sentences containing allegory were also not included in the study. Tweets containing multiple keywords were used only once. Furthermore, tweets containing only emojis, icons, and videos were excluded from the study.
No ethics committee approval was required for this work since only public data were used and no human subjects were included.
Analysis of emotions
The emotions underlying the tweets were analyzed using the CrystalFeel algorithm, which is a proven sensitivity analytical technology. CrystalFeel Algorithm gives us five emotion dimensions and five emotion intensities. Also, this program reports the most prominent emotion. In this study, the most prominent emotion and related emotion intensity from analyzed tweets chose only by researchers.
CrystalFeel is a collection of five machine-learning-based algorithms that run simultaneously and independently to analyze the emotional characteristics of a text, providing five output dimensions: anger intensity, fear intensity, sadness intensity, joy intensity, and valence intensity. The intensities were expressed as continuous values between 0 and 1. 0—this text expresses no emotional intensity or at a very low level; 1—this text expresses extremely high emotional intensity. Besides, based on the intensity scores, CrystalFeel Algorithm also generates two converted outputs: sentiment category and emotion category .
According to Plutchik’s wheel of emotions, sadness-joy and anger-fear are the basic emotional pairs of opposite experiences . Sadness is an unpleasant emotion caused by something that didn’t happen or something that wasn’t wanted to happen, while Joy is a positive emotion that occurs when something is highly desired or pleasant . Fear is an unpleasant feeling caused by the thought of danger, while anger is caused by uncertainty caused by others . Investigating the evolution of these four basic emotions may show the changing dynamics of public experience in crisis  (Table 1).
Data were analyzed with IBM SPSS V23. Chi-square test was used to compare emotions and emotion intensities according to the words used. The comparison of the ratios of each column as a result of the chi-square test was performed with the Bonferroni test. Analysis results were presented as frequency (percentage) for categorical data. Significance level was taken as p < 0.050.
In total, 842 tweets were obtained by entering the specified keywords in the search application of Twitter. After applying the exclusion criteria, 416 tweets were removed. The remaining 426 tweets were included in the study and analyzed.
It was determined that 80.3% of the participants experienced fear and that 61.7% had a negative emotional intensity; 38% of the keywords were used mostly between 1 and 2020 and 30 June 2020. The keyword “Corona, Toothache” was used the most frequently by 30.8% of the participants (Table 2).
There was no statistically significant difference between the distributions of emotions according to the words used between the following periods: 11 March 2020–31 March 2020 (p = 0.510); 1 April 2020–3 April 02020 (p = 0.410); 1 May 2020–31 May 2020 (p = 0.328); and 1 June 2020–30 June 2020 (p = 0.074). There was also no statistically significant difference between the distributions of emotions according to the words without time distinction (p = 0.136) (Table 3).
However, there was a statistically significant difference between the distributions of emotional intensity according to the words used between 11 and 2020 and 31 March 2020 (p = 0.006). There was a statistically significant difference between the distributions of emotional intensity according to the words without time distinction (p = 0.006). Finally, there was no statistically significant difference between the distributions of emotional intensity according to the words used between the following periods: 1 April 2020–30 April 2020 (p = 0.141); 1 May 2020–31 May 2020 (p = 0.709); and 1 June 2020–30 June 2020 (p = 0.093) (Table 4).
During the COVID-19 pandemic, most dental treatments had to be delayed or canceled for the safety of patients and healthcare workers . The pandemic caused changes in people’s moods until it was brought under control, and we could quickly learn about these changes through social media. Due to easy access to social networks such as Twitter, researchers can use these tools as sources of data on many topics, such as attitudes toward health-related issues [25, 26]. Twitter analysis revealed an increase in the number of tweets mentioning the toothache during the most intense period of the COVID-19 pandemic (11 March 2020 and 30 June 2020) and the intensity of emotions of “fear” and “anger”.
Toothache disrupts the mood, sleep, eating habits, and social functioning of the individual, negatively affects quality of life, and is a condition that needs to be treated [10, 13, 27]. A study that analyzed the dental treatment needs of Brazilian Twitter users between 23 and 2020 and 4 May 2020 found that the need for treatment due to dental pain was ranked first . Moreover, in the study in which Zhuo-Ying Tao et al. analyzed the nature and spread of oral health information on COVID-19 using the Weibo media application, it was reported that the main health problem requiring treatment was toothache . In this study conducted with different keywords related to toothache, the highest number of tweets were found for the following keywords: “Corona-Toothache” (30.8%), “Covid 19-Toothache” (23.2%), and “Covid-19-Dental Pain” (15.0%), a result in agreement with the literature.
In this study, the emotions underlying the tweets on toothache during the COVID-19 pandemic were analyzed for the first time using the CrystalFeel algorithm, a sensitivity analytical technology. Since tweets in English posted not only in a single region but all over the world were analyzed, the data contributed to obtaining general information about social media users’ toothaches and emotional states.
Except for a few studies, there have not been many investigations of the psychological effects of the COVID-19 pandemic on society [23, 30,31,32]. In a study conducted by Wang et al., the psychological effects, depression, stress, and anxiety were evaluated in 1210 participants from 194 cities in China at the beginning of the pandemic . The results of the study indicated that 53.8% of these individuals experienced severe psychological effects. Furthermore, 16.5%, 28.8%, and 8.1% of the participants reported moderate to severe depression, anxiety, and stress levels, respectively. In a recent study, it was argued that the COVID-19 pandemic affected orthodontic appointments and provoked patients’ anxiety and that the biggest concern of more than half of the patients was the delay in treatment . The emotions felt about toothache during the pandemic were fear (80.3%), anger (11.7%), sadness (5.9%), and joy (2.1%), respectively. In our study, the feelings of fear and anger were found to be high in March and April, which was considered to be caused by the cancelation of elective and aerosol dental treatments, the inability to receive dental treatment opinions face to face, misinformation, and uncertainty.
In this study, the least number of tweets were related to “joy” and were about medication or dental treatment during the pandemic period. From the beginning of the pandemic until June, the emotional intensity of the tweets on pain showed a negative or very negative distribution. However, in May and June, the number of negative and very negative tweets decreased. It was considered this change to be related to partial control of pandemic and resumption of dental services.
In this study, we found that Twitter users reported high intensity of pain in 89.0% of the tweets (n = 379) by using the bilateral pain intensity variable. In the studies on toothache intensity on social media conducted before the pandemic, pain intensity was high, but the percentage rates were very low compared to our study . In the literature, people’s sharing of their pain experiences on social media is considered one coping method [17, 35]. The authors assume that there was an increase in the number of tweets expressing high pain intensity during the pandemic because people could not receive dental treatment and thus used Twitter to cope with pain and seek support.
The WHO indicated that social media can be used “to attract public attention during an emergency, to facilitate one-to-one communication, to create situational awareness, to monitor and respond to rumors, public reactions and concerns, and to facilitate local level response” . As indicated by the WHO, social media can be used as a very effective tool not only to disseminate information but also to collect feedback. The reactions, comments, and posts of those who read the post can be seen and evaluated instantly, ensuring that their needs are understood and a faster response is developed [37, 38].
The authors believe that Twitter could be used as a source of data in studies to developed community-oriented communication strategies for toothache experiences during and after the pandemic. With such content analysis studies, in cooperation with dental professionals and the government, live broadcasts, “question-answer hour,” and informative videos about oral and dental health can help to guide individuals have toothache during epidemic periods. In this way, increasing negative emotions can be helped to control.
Concerning the limitations of our study, the CrystalFeel program can only analyze emotions in English content; therefore, tweets in other languages were not included in this study. CrystalFeel is also unable to determine emotions conveyed by metaphorical sentences, which prevents the analysis of sentences containing allegory. Another shortcoming of this work consists of having collected data from only one social media platform. This study findings may not be generalizable to offline population. Therefore, caution is advised before assuming the generalizability of the results, as everyone in the population does not use Twitter .
Monitoring the social media posts of individuals experiencing toothache during the pandemic will help reduce fear and anger emotions and design public information messages that are compatible with the target group’s needs.
Availability of data and materials
The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.
World Health Organization. WHO Director-General’s opening remarks at the media briefing on COVID-19 - 11 March 2020. https://www.who.int/director-general/speeches/detail/who-director-general-s-opening-remarks-at-the-media-briefing-on-covid-19---11-march-2020. Accessed 28 Mar 2020.
Peng X, Xu X, Li Y, Cheng L, Zhou X, Ren B. Transmission routes of 2019-nCoV and controls in dental practice. Int J Oral Sci. 2020;12(1):1–6. https://doi.org/10.1038/s41368-020-0075-9.
Meng L, Hua F, Bian Z. Coronavirus disease 2019 (COVID-19): emerging and future challenges for dental and oral medicine. J Dent Res. 2020;99(5):481–7. https://doi.org/10.1177/0022034520914246.
Karaca B, Güneri P. In: Emingil G, editor. Dental procedures during SARS-CoV-2 pandemic and normalization period: triage, oral and radiographic examination. 1st ed. Ankara: Türkiye Klinikleri; 2020. p. 24–32.
Lacerda JTd, Simionato EM, Peres KG, Peres MA, Traebert J, Marcenes W. Dental pain as the reason for visiting a dentist in a Brazilian adult population. Rev Saude Publica. 2004;38:453–8.
Pau A, Croucher R, Marcenes W. Prevalence estimates and associated factors for dental pain: a review. Oral Health Prev Dent. 2003;1(3):209–20.
Lohman D, Schleifer R, Amon JJ. Access to pain treatment as a human right. BMC Med. 2010;8(1):1–9. https://doi.org/10.1186/1741-7015-8-8.
Ortiz FR, Tomazoni F, Oliveira MDM, Piovesan C, Mendes F, Ardenghi TM. Toothache, associated factors, and its impact on Oral Health-Related Quality of Life (OHRQoL) in preschool children. Braz Dent J. 2014;25:546–53. https://doi.org/10.1590/0103-6440201302439.
Pau A, Baxevanos KG, Croucher R. Family structure is associated with oral pain in 12-year‐old Greek schoolchildren. Int J Paediatr Dent. 2007;17(5):345–51. https://doi.org/10.1111/j.1365-263X.2007.00842.x.
Altan H, Çevik H, Doğru S, Coşgun A, Süren M, Okan İ. The pain colour of children with toothache in Turkish population. BMC Oral Health. 2019;19(1):1–8. https://doi.org/10.1186/s12903-019-0756-y.
Constante HM, Bastos JL, Peres KG, Peres MA. Socio-demographic and behavioural inequalities in the impact of dental pain among adults: a population‐based study. Community Dent Oral Epidemiol. 2012;40(6):498–506. https://doi.org/10.1111/j.1600-0528.2012.00701.x.
Leal S, Bronkhorst E, Fan M, Frencken J. Untreated cavitated dentine lesions: impact on children’s quality of life. Caries Res. 2012;46(2):102–6. https://doi.org/10.1159/000336387.
Altan A, Akkoç S, Erdil A, Çolak S, Demir O, Altan H. Effects of pain catastrophizing and anxiety on analgesic use after surgical removal of impacted mandibular third molars. J Dent Anesth Pain Med. 2019;19(6):379–88. https://doi.org/10.17245/jdapm.2019.19.6.379.
Khan AS, Fleischauer A, Casani J, Groseclose SL. The next public health revolution: public health information fusion and social networks. Am J Public Health. 2010;100(7):1237–42. https://doi.org/10.2105/AJPH.2009.180489
Arolfo F, Rodriguez KC, Vaisman A. Analyzing the quality of Twitter data streams. Inf Syst Front. 2020;1–21. https://doi.org/10.1007/s10796-020-10072-x.
Post H. Twitter statistics. Statistic Brain. 2014.
Heaivilin N, Gerbert B, Page J, Gibbs J. Public health surveillance of dental pain via Twitter. J Dent Res. 2011;90(9):1047–51. https://doi.org/10.1177/0022034511415273.
Lwin MO, Lu J, Sheldenkar A, Schulz PJ. Strategic uses of Facebook in Zika outbreak communication: implications for the crisis and emergency risk communication model. Int J Environ Res Public Health. 2018;15(9):1974. https://doi.org/10.3390/ijerph15091974.
Ahlwardt K, Heaivilin N, Gibbs J, Page J, Gerbert B, Tsoh JY. Tweeting about pain: comparing self-reported toothache experiences with those of backaches, earaches and headaches. JADA. 2014;145(7):737–43. https://doi.org/10.14219/jada.2014.30.
Ocak H, Akkoyun E, Çolpak H, Demetoğlu U, Yücesoy T, Kılıç E, et al. Is the jet injection effective for teeth extraction? J Stomatol Oral and Maxi. 2020;121(1):19–24. https://doi.org/10.1016/j.jormas.2019.05.001.
Plutchik R. A general psychoevolutionary theory of emotion. Theories of emotion: Elsevier; 1980. p. 3–33. https://doi.org/10.1016/B978-0-12-558701-3.50007-7.
Roseman IJ. Appraisal determinants of emotions: Constructing a more accurate and comprehensive theory. Cogn Emot. 1996;10(3):241–78. https://doi.org/10.1080/026999396380240.
Lwin MO, Lu J, Sheldenkar A, Schulz PJ, Shin W, Gupta R, et al. Global sentiments surrounding the COVID-19 pandemic on Twitter: analysis of Twitter trends. JMIR Public Health Surveill. 2020;6(2):e19447.
Arapostathis KN, Dabarakis NN, Coolidge T, Tsirlis A, Kotsanos N. Comparison of acceptance, preference, and efficacy between jet injection INJEX and local infiltration anesthesia in 6 to 11 year old dental patients. Anesthesia progress. 2010;57(1):3–12.
Sinnenberg L, Buttenheim AM, Padrez K, Mancheno C, Ungar L, Merchant RM. Twitter as a tool for health research: a systematic review. Am J Public Health. 2017;107(1):e1-e8. https://doi.org/10.2105/AJPH.2016.303512.
Guo F, Tang B, Qin D, Zhao T, Su Y, McGrath C, et al. The impact of the COVID-19 epidemic on orthodontic patients in China: an analysis of posts on Weibo. Front Med. 2020;7:1–11. https://doi.org/10.3389/fmed.2020.577468.
Stinson JN, Kavanagh T, Yamada J, Gill N, Stevens B. Systematic review of the psychometric properties, interpretability and feasibility of self-report pain intensity measures for use in clinical trials in children and adolescents. Pain. 2006;125(1–2):143–57. https://doi.org/10.1016/j.pain.2006.05.006.
Oliveira LM, Zanatta FB. Self-reported dental treatment needs during the COVID-19 outbreak in Brazil: an infodemiological study. Braz Oral Res. 2020;34:e114. https://doi.org/10.1590/1807-3107bor-2020.vol34.0114
Tao Z-Y, Chu G, McGrath C, Hua F, Leung YY, Yang W-F, et al. Nature and diffusion of COVID-19–related oral health information on Chinese social media: analysis of tweets on weibo. J Med Internet Res. 2020;22(6):e19981. https://doi.org/10.2196/19981.
Lai J, Ma S, Wang Y, Cai Z, Hu J, Wei N, et al. Factors associated with mental health outcomes among health care workers exposed to coronavirus disease 2019. JAMA network open. 2020;3(3):e203976. https://doi.org/10.1001/jamanetworkopen.2020.3976.
Li W, Yang Y, Liu Z-H, Zhao Y-J, Zhang Q, Zhang L, et al. Progression of mental health services during the COVID-19 outbreak in China. Int J Biol Sci. 2020;16(10):1732–8. https://doi.org/10.7150/ijbs.45120.
Altan A. Altan A. Emotional effect of the Covid-19 pandemic on oral surgery procedures: a social media analysis. J Dent Anesth Pain Med. 2021 Jun;21(3):237–244. https://doi.org/10.17245/jdapm.2021.21.3.237
Wang C, Horby PW, Hayden FG, Gao GF. A novel coronavirus outbreak of global health concern. The Lancet. 2020;395(10223):470–3. https://doi.org/10.1016/S0140-6736(20)30185-9.
Ahlwardt K, Heaivilin N, Gibbs J, Page J, Gerbert B, Tsoh JY. Tweeting about pain: comparing self-reported toothache experiences with those of backaches, earaches and headaches. J Am Dent Assoc. 2014;145(7):737–43. https://doi.org/10.14219/jada.2014.30.
Greene JA, Choudhry NK, Kilabuk E, Shrank WH. Online social networking by patients with diabetes: a qualitative evaluation of communication with Facebook. J Gen Intern Med. 2011;26(3):287–92. https://doi.org/10.1007/s11606-010-1526-3.
Makade CS, Shenoi PR, Gunwal MK. Comparison of acceptance, preference and efficacy between pressure anesthesia and classical needle infiltration anesthesia for dental restorative procedures in adult patients. J Conserv Dent. 2014;17(2):169–74. https://doi.org/10.4103/0972-0707.128063.
Lundgren RE, McMakin AH. Risk communication: a handbook for communicating environmental, safety, and health risks. London: Wiley; 2018.
Lazard AJ, Scheinfeld E, Bernhardt JM, Wilcox GB, Suran M. Detecting themes of public concern: a text mining analysis of the Centers for Disease Control and Prevention’s Ebola live Twitter chat. Am J Infect Control. 2015;43(10):1109–11. https://doi.org/10.1016/j.ajic.2015.05.025.
Abd-Alrazaq A, Alhuwail D, Househ M, Hamdi M, Shah Z. Top concerns of tweeters during the COVID-19 pandemic: infoveillance study. J Med Internet Res. 2020;22(4):e19016. https://doi.org/10.2196/19016.
The study was self-funded. No public or private funding source was used for this research.
Ethics approval and consent to participate
In this study, no ethics committee approval was required since public data were used and the study did not include human subjects. Not applicable.
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Altan, H., Coşgun, A. Analysis of tweets on toothache during the COVID-19 pandemic using the CrystalFeel algorithm: a cross-sectional study. BMC Oral Health 21, 418 (2021). https://doi.org/10.1186/s12903-021-01778-8
- Social media